openforce/docs/SALESFORCE_FLOW_ENGINE_SPEC.md

111 lines
7.2 KiB
Markdown

# Salesforce Flow Engine Deep Dive & OpenForce Implementation Blueprint
## 1. Executive Summary & Objective
Salesforce Flow (Lightning Flow / Flow Builder) is Salesforce's visual Turing-complete orchestration and declarative automation engine. It subsumed Workflow Rules, Process Builder, and Visual Workflow into a unified DAG-driven runtime.
To achieve Salesforce parity in **OpenForce**, we must bridge the gap between our current DAG topology checker (`Kahn's algorithm`) and Salesforce's full Flow runtime specification:
1. **Flow Types & Trigger Subsystems** (Record-Triggered, Schedule-Triggered, Autolaunched, Subflows).
2. **Core Flow Elements** (Interaction, Logic/Conditionals, Data DML operations).
3. **Execution Semantics & State Machine** (Variables, Evaluation Loops, Bulkification, Transaction Boundaries).
4. **Custom `__c` Field Support on Standard CRM Objects** (Extensible JSONB metadata overlay on `Account`, `Contact`, `Lead`, `Opportunity`).
---
## 2. Salesforce Flow Engine Architecture Deconstruction
### 2.1 The 4 Primary Flow Types & Triggers
| Flow Type | Trigger Mechanism | Equivalent Salesforce Concept | OpenForce Target Implementation |
| :--- | :--- | :--- | :--- |
| **Record-Triggered Flow** | `post_save` / `pre_save` model lifecycle | Before-Save (Fast Field Updates) & After-Save (Related Records & Actions) | Django model signals (`post_save`, `pre_delete`) routing to `FlowDispatcher` |
| **Schedule-Triggered Flow** | Batch cron / recurrent time interval | Runs on a specified frequency (Once, Daily, Weekly) for a batch of records | Celery Beat / Redis queue dispatching chunks of record IDs |
| **Autolaunched Flow (No Trigger)** | Invoked imperatively via API, Apex, or REST | Can be called by external systems, buttons, or parent flows | `POST /api/workflows/{id}/execute/` |
| **Subflow** | Invoked as an element within another flow | Reusable modular automation with defined Input/Output parameters | `SubflowElement` executing a child `FlowDefinition` within parent context |
### 2.2 Complete Flow Element Taxonomy
Salesforce divides Flow elements into three core categories:
#### A. Logic Elements
1. **Decision (`DecisionNode`)**:
- Multiple outcome branches (e.g. `High Value`, `Mid Tier`, `Default/Else`).
- Each outcome contains a set of logic conditions (e.g., `{!Record.amount} >= 50000 AND {!Record.stage} == 'Proposal'`).
- Condition operators: `equals`, `not_equals`, `greater_than`, `less_than`, `contains`, `is_null`.
- Supports custom condition logic: `AND`, `OR`, custom expressions (`(1 AND 2) OR 3`).
2. **Assignment (`AssignmentNode`)**:
- Sets, increments, decrements, or appends values to variables.
- Operators: `Equals`, `Add`, `Subtract`, `Add to Collection`, `Remove from Collection`.
3. **Loop (`LoopNode`)**:
- Iterates over a collection of records or variables (e.g., all `OpportunityLineItems` on an `Opportunity`).
- Emits `For Each` child execution branch and `After Last` exit branch.
4. **Subflow (`SubflowNode`)**:
- Calls an active Flow definition.
- Maps parent context variables $
ightarrow$ child input variables, and child output variables $
ightarrow$ parent context.
#### B. Data (DML) Elements
1. **Create Records (`CreateRecordNode`)**:
- Inserts 1 or multiple records into standard or custom objects (`Lead`, `Account`, `CustomRecord__c`).
2. **Update Records (`UpdateRecordNode`)**:
- Updates records matching filter conditions or updates the triggering `{!$Record}`.
3. **Get Records (`GetRecordsNode`)**:
- Queries database with filter conditions, sort order, and limit (`Find First` vs `Find All`).
4. **Delete Records (`DeleteRecordNode`)**:
- Removes records matching criteria.
#### C. Action Elements
1. **Send Email / Outreach**:
- Dispatches transactional email or triggers outreach campaign with merge tags.
2. **Post to Chatter / Feed**:
- Creates a polymorphic `Activity` / note on the target record.
3. **Custom Action / Webhook / Task**:
- Spawns background tasks or invokes external webhooks.
---
## 3. Dynamic Custom Fields (`__c`) on Standard Objects
### 3.1 Architectural Analysis & Pattern Selection
In Salesforce, `Account`, `Contact`, `Lead`, and `Opportunity` seamlessly accept `custom_field__c` without database downtime or schema migrations.
**Recommended Pattern for OpenForce:**
- Keep existing relational columns for core standard fields (for high-speed relational queries, foreign keys, and indexes).
- Add a `custom_fields = models.JSONField(default=dict, blank=True)` column with PostgreSQL GIN index (`gin (custom_fields jsonb_path_ops)`) to `Account`, `Contact`, `Lead`, and `Opportunity`.
- Update `CustomFieldDefinition` to support standard object attachment via an `entity_name` attribute (e.g. `entity_name="Account"`, `entity_name="Lead"`, or `custom_object=custom_obj_id`).
- When serializing or validating standard models, validate `custom_fields` against registered `CustomFieldDefinition` records for that entity using `schema_engine.validate_record_data`.
---
## 4. Phase 4 Technical Specification: The 3 Core Enhancements
### Task 1: Standard Objects Dynamic Custom Fields Engine
1. **Model Modification**:
- Add `custom_fields = models.JSONField(default=dict, blank=True)` to `Account`, `Contact`, `Lead`, `Opportunity`.
- Add GIN index on `custom_fields` across all 4 standard models.
2. **Field Registry**:
- Enable `CustomFieldDefinition` to belong either to a `CustomObjectDefinition` or a `standard_entity` (`'Account'`, `'Contact'`, `'Lead'`, `'Opportunity'`).
- Automatically enforce `__c` naming convention on standard entity custom fields.
3. **Serializer & Dynamic Deserialization**:
- Update `crm_core/serializers.py` to dynamically surface `custom_fields` as top-level fields or under `custom_fields`, validating with `Draft7Validator`.
### Task 2: Advanced Flow Builder Execution Engine
1. **Decision Branching Runtime**:
- Add `decision` node execution in `workflows/engine.py`.
- Evaluate outcomes sequentially; take the first matching branch or `default_outcome`.
2. **Subflow Execution**:
- In `workflows/engine.py`, when a `subflow` node is encountered, load the child `FlowDefinition`, map inputs, run child DAG, and pull outputs back into parent context.
3. **Loop & Dynamic Context Resolution**:
- Support nested expression evaluation (`{!Record.amount}`, `{!Context.score}`).
### Task 3: Trigger Subsystems & Scheduling
1. **Record-Change Triggers**:
- Implement Django `post_save` and `post_delete` signal handlers in `workflows/signals.py`.
- Filter active `FlowDefinition` records where `trigger_type='record_created'` or `'record_updated'`.
- Dispatch matching flows synchronously or asynchronously via Celery worker.
2. **Scheduled Flow Triggers**:
- Add `cron_expression` or interval settings to `FlowDefinition`.
- Expose management command / Celery Beat task `dispatch_scheduled_flows` that queries eligible records and launches flow batches.
---
## 5. Architectural Verification & Quality Contract
- Zero breaking changes to existing 55 passing tests.
- Maintain $\ge 85\%$ test coverage across new workflow elements and custom field models.
- Transactional atomicity: if an after-save record-triggered flow raises an unhandled error, the triggering DML transaction rolls back cleanly (Salesforce all-or-nothing semantic).