docs: add CRM benchmarks and standard schema specifications

This commit is contained in:
Bumble 2026-09-26 10:49:12 +00:00
parent f51055cea5
commit 929845fb9a
2 changed files with 250 additions and 0 deletions

View file

@ -0,0 +1,99 @@
# OpenForce Benchmark Analysis: CRM Architectures, Custom Object Engines & Visual Workflow Editors
## 1. Executive Summary
This document establishes architectural benchmarks, technical design trade-offs, and proven implementation patterns for **OpenForce**—a Python/Django + Vue 3 open-source CRM targeting Salesforce parity.
Key benchmark insights:
1. **Custom Object Storage**: Salesforce utilizes a multi-tenant universal table architecture (MT_Objects, MT_Fields, MT_Value_Data). Modern PostgreSQL allows us to achieve superior developer ergonomics and query performance via a **Hybrid JSONB + Typed Auxiliary Indexes** architecture (as proven by Twenty CRM and Supabase dynamic schema patterns).
2. **Visual Workflow Canvas**: `@vue-flow/core` provides native reactivity and customizable node sockets that match the UX paradigms of Salesforce Flow Builder, Zapier, and n8n.
3. **Execution Runtime**: A lightweight Directed Acyclic Graph (DAG) state machine running atop Celery/Redis guarantees idempotency, branch isolation, and transactional step retries.
---
## 2. Comparative Analysis of Open-Source CRM Architectures
| Dimension | Salesforce (Proprietary) | Twenty CRM (TypeScript/Nest/Postgres) | SuiteCRM 8 (PHP/Symfony) | OpenForce (Proposed: Django/Vue) |
| :--- | :--- | :--- | :--- | :--- |
| **Object Engine** | Proprietary Force.com multi-tenant UDM (EAV variant) | Dynamic PostgreSQL tables generated per workspace via TypeORM metadata | Traditional relational EAV (`*_cstm` tables per module) | **Hybrid PostgreSQL JSONB with GIN index + Metamodel Validation** |
| **Schema Flexibility** | High (Custom fields, relationships, validation rules) | High (Dynamic schema migrations per workspace) | Medium (Alters database schema on field creation) | **High (Instant zero-migration schema updates via JSONB)** |
| **Workflow Engine** | Salesforce Flow / Process Builder (Apex engine) | Action trigger workflows (Server-side TS) | Logic Hooks + WorkFlow module (Cron/Event triggers) | **DAG Execution Engine with Celery task orchestration** |
| **Frontend Framework** | Lightning Web Components (LWC) | React + Tailwind + Vite | Angular + Legacy Smarty/Vue | **Vue 3 (Composition API) + Pinia + TailwindCSS** |
| **API Architecture** | REST / SOAP / GraphQL / Bulk API | GraphQL first + REST | REST API v8 (JSON:API spec) | **Django REST Framework (DRF) + OpenAPI v3 / Dynamic ViewSets** |
---
## 3. Deep-Dive: Custom Object & Dynamic Field Engine
Salesforce's Force.com architecture avoids running `ALTER TABLE` whenever an administrator creates a custom object or field. Instead, it relies on metadata tables describing tenant fields and a massive denormalized storage layer.
### 3.1 Evaluated Architecture Patterns
#### Pattern A: Strict EAV (Entity-Attribute-Value)
- **Mechanism**: Table `CustomValues (record_id, field_id, value_string, value_int, ...)`.
- **Pros**: Pure relational modeling, strict SQL types.
- **Cons**: Massive JOIN overhead when querying 20+ fields per record; poor sorting/filtering performance across dynamic fields.
#### Pattern B: Dynamic DDL (`ALTER TABLE` on the fly)
- **Mechanism**: Django dynamically creates and executes `ALTER TABLE my_table ADD COLUMN custom_field VARCHAR`.
- **Pros**: Native DB columns, native indexes, simple SQL.
- **Cons**: DB lock contention, DDL transaction risks, migration tracking complexity across production instances.
#### Pattern C: Hybrid JSONB + Metadata Metamodel (Recommended for OpenForce)
- **Mechanism**:
- Standard relational tables for fixed objects (`Account`, `Contact`, `Lead`, `Opportunity`).
- Metamodel tables: `CustomObjectDefinition` and `CustomFieldDefinition`.
- Data table: `CustomRecord` with `(id, object_definition_id, data JSONB, created_at, updated_at, created_by)`.
- Native PostgreSQL GIN indexes on `data`: `CREATE INDEX idx_custom_record_gin ON custom_records USING gin (data jsonb_path_ops);`.
- Dynamic virtual serialization in DRF with JSONSchema validation at ingestion.
- **Pros**:
- Zero-downtime dynamic schema updates.
- Sub-millisecond JSONB query performance via GIN indexing and expression indexes for frequent filters (e.g. `CREATE INDEX idx_status ON custom_records ((data->>'status'));`).
- Seamless support for all Salesforce data types (picklist, multiselect, lookup, currency, formula).
---
## 4. Visual Workflow Automation Engine Benchmark
### 4.1 Frontend Canvas Benchmark: Vue Flow vs. Alternatives
- **Selected: `@vue-flow/core` (v1.30+)**:
- Direct Vue 3 port inspired by React Flow.
- Native Vue reactivity: Custom node components can be pure `.vue` Single File Components (SFCs).
- Out-of-the-box support for interactive controls, minimap, background grid, smooth bezier edges, and multi-socket connection validation.
- UX Pattern matches Salesforce Flow Builder:
- **Palette Drawer**: Drag triggers, logic nodes, and action blocks into viewport.
- **Step Inspector Side-Panel**: Contextual form for editing trigger conditions or email action parameters when a node is selected.
- **Live Graph Validation**: Visual feedback if branches are orphaned or cycles are detected.
### 4.2 Backend Execution Runtime (DAG Engine)
- **Workflow Representation**: Stored as a JSON DAG:
```json
{
"nodes": [
{"id": "node_1", "type": "trigger", "data": {"event": "lead.created"}},
{"id": "node_2", "type": "condition", "data": {"rules": [{"field": "lead_score", "op": "gte", "value": 50}]}},
{"id": "node_3", "type": "action", "data": {"action": "email.send_template", "template_id": 4}}
],
"edges": [
{"source": "node_1", "target": "node_2"},
{"source": "node_2", "target": "node_3", "sourceHandle": "true"}
]
}
```
- **Execution Model**:
- Synchronous / Celery worker dispatches flow runs on Django model signals (`post_save`).
- Topologically sorted step resolution with branch isolation.
- Idempotency keys generated per event trigger to prevent duplicate execution.
---
## 5. Standard CRM Life-Cycle Recommendations
1. **Lead Conversion Flow**:
- Atomic transaction converting `Lead(status='Qualified')` -> creates `Account` (or merges into existing), creates `Contact`, and optionally creates `Opportunity`.
- Copies custom fields from Lead to matching target fields on Account/Contact/Opportunity via field mapping metadata.
2. **Polymorphic Activity Timeline**:
- Django `GenericForeignKey` to associate `Tasks`, `Notes`, `Emails`, and `StageHistory` with any model (`Lead`, `Opportunity`, or `CustomRecord`).
3. **Email Outreach Engine**:
- Queue-backed (Celery) bulk sending with rate-limiting.
- Template engine using Django templating or Jinja2 syntax (`{{ contact.first_name }}`) with fallback defaults.

View file

@ -0,0 +1,151 @@
# OpenForce Standard CRM Schema Specification & Field Dictionary
**Target Entities**: `Account`, `Contact`, `Lead`, `Opportunity`, and `Activity`
This document defines the production field dictionary, constraint rules, indexing strategies, and lifecycle conversion dynamics for OpenForce's Phase 1 standard objects, benchmarking Salesforce standard fields, Twenty CRM, and SuiteCRM.
---
## 1. Account (Standard Object)
Represents a business organization, partner, or enterprise client.
### Fields & Constraints
| Field Name | Type | Constraints | Description |
| :--- | :--- | :--- | :--- |
| `id` | UUID | Primary Key, default `uuid.uuid4` | Global unique identifier |
| `name` | String(255) | Required, indexed | Company or organization legal/trading name |
| `account_type` | Enum / String(50) | Choices: `Customer`, `Prospect`, `Partner`, `Competitor`, `Vendor`, `Other` | Relationship type |
| `industry` | Enum / String(100) | Choices: `Technology`, `Finance`, `Healthcare`, `Manufacturing`, `Retail`, `Services`, `Other` | Sector categorization |
| `annual_revenue` | Decimal(18, 2) | Nullable, default `None` | Annual estimated or reported turnover |
| `employees_count`| Integer | Nullable, min 0 | Total workforce count |
| `website` | URLField | Nullable, max_length 255 | Corporate website URL |
| `phone` | String(40) | Nullable | Primary corporate telephone number |
| `billing_address`| JSONB | Schema: `{street, city, state, postal_code, country}` | Structured billing address |
| `shipping_address`| JSONB | Schema: `{street, city, state, postal_code, country}` | Structured shipping/logistics address |
| `parent_account` | ForeignKey(Account) | Nullable, `on_delete=SET_NULL`, related_name=`child_accounts` | Corporate hierarchy/subsidiary tree |
| `owner` | ForeignKey(User) | Nullable, `on_delete=SET_NULL`, related_name=`accounts` | Account executive / sales rep owner |
| `created_at` | DateTime | Auto now add, indexed | Creation timestamp |
| `updated_at` | DateTime | Auto now, indexed | Last updated timestamp |
---
## 2. Contact (Standard Object)
Represents an individual person associated with an Account.
### Fields & Constraints
| Field Name | Type | Constraints | Description |
| :--- | :--- | :--- | :--- |
| `id` | UUID | Primary Key | Global unique identifier |
| `account` | ForeignKey(Account) | Nullable, `on_delete=CASCADE`, related_name=`contacts`, indexed | Linked company/account |
| `first_name` | String(100) | Required | Individual's given name |
| `last_name` | String(100) | Required, indexed | Individual's family / surname |
| `email` | EmailField | Indexed, nullable | Primary email address |
| `phone` | String(40) | Nullable | Direct business telephone |
| `mobile_phone` | String(40) | Nullable | Mobile/cellular telephone |
| `title` | String(128) | Nullable | Job title / professional role (e.g. VP Engineering) |
| `department` | String(128) | Nullable | Department within organization |
| `is_primary` | Boolean | Default `False` | Primary contact flag for the parent Account |
| `do_not_call` | Boolean | Default `False` | Compliance flag for outbound calling |
| `email_opt_out` | Boolean | Default `False`, indexed | Unsubscribe flag for automated outreach campaigns |
| `owner` | ForeignKey(User) | Nullable, `on_delete=SET_NULL`, related_name=`contacts` | Relationship manager |
| `created_at` | DateTime | Auto now add | Creation timestamp |
| `updated_at` | DateTime | Auto now | Last updated timestamp |
---
## 3. Lead (Standard Object)
Represents an unqualified prospect or inbound marketing inquiry.
### Fields & Constraints
| Field Name | Type | Constraints | Description |
| :--- | :--- | :--- | :--- |
| `id` | UUID | Primary Key | Global unique identifier |
| `first_name` | String(100) | Required | Prospect given name |
| `last_name` | String(100) | Required, indexed | Prospect family name |
| `company` | String(255) | Required, indexed | Organization name provided by prospect |
| `email` | EmailField | Indexed, nullable | Prospect email address |
| `phone` | String(40) | Nullable | Phone number |
| `lead_source` | Enum / String(64) | Choices: `Inbound Web`, `Referral`, `Conference`, `Cold Outreach`, `Partner`, `Other` | Acquisition origin |
| `status` | Enum / String(50) | Choices: `New`, `Contacted`, `Working`, `Qualified`, `Unqualified` | Lead lifecycle status |
| `lead_score` | Integer | Default 0, indexed | Heuristic engagement score (0-100) |
| `is_converted` | Boolean | Default `False`, indexed | True if converted to Account/Contact/Opp |
| `converted_at` | DateTime | Nullable | Conversion timestamp |
| `converted_account` | ForeignKey(Account) | Nullable, `on_delete=SET_NULL` | Account generated during conversion |
| `converted_contact` | ForeignKey(Contact) | Nullable, `on_delete=SET_NULL` | Contact generated during conversion |
| `converted_opportunity` | ForeignKey(Opportunity) | Nullable, `on_delete=SET_NULL` | Deal created during conversion |
| `owner` | ForeignKey(User) | Nullable, `on_delete=SET_NULL`, related_name=`leads` | Sales development representative |
| `created_at` | DateTime | Auto now add | Creation timestamp |
| `updated_at` | DateTime | Auto now | Last updated timestamp |
---
## 4. Opportunity (Standard Object)
Represents a prospective deal or revenue-generating engagement in the sales pipeline.
### Fields & Constraints
| Field Name | Type | Constraints | Description |
| :--- | :--- | :--- | :--- |
| `id` | UUID | Primary Key | Global unique identifier |
| `name` | String(255) | Required, indexed | Deal name (e.g. "Acme Corp - 50 Seat License") |
| `account` | ForeignKey(Account) | Required, `on_delete=CASCADE`, related_name=`opportunities` | Associated client account |
| `primary_contact`| ForeignKey(Contact) | Nullable, `on_delete=SET_NULL` | Lead champion / signatory contact |
| `amount` | Decimal(18, 2) | Default `0.00`, indexed | Total deal value in base currency |
| `stage` | Enum / String(50) | Indexed | Pipeline stage (see Stage Dictionary below) |
| `probability` | Integer | Min 0, Max 100 | Percentage probability of winning (stage-derived default) |
| `expected_revenue`| Decimal(18, 2) | Computed (`amount * probability / 100`) | Weighted pipeline revenue |
| `close_date` | Date | Required, indexed | Anticipated or actual transaction close date |
| `lead_source` | Enum / String(64) | Nullable | Origination channel |
| `deal_type` | Enum / String(50) | Choices: `New Business`, `Existing Business - Expansion`, `Renewal` | Commercial categorization |
| `next_step` | String(255) | Nullable | Immediate operational action step |
| `owner` | ForeignKey(User) | Nullable, `on_delete=SET_NULL`, related_name=`opportunities` | Deal lead |
| `created_at` | DateTime | Auto now add | Creation timestamp |
| `updated_at` | DateTime | Auto now | Last updated timestamp |
### Opportunity Stage & Probability Matrix
| Stage Name | Default Probability | Forecast Category | Description |
| :--- | :--- | :--- | :--- |
| `Prospecting` | 10% | Pipeline | Initial contact made, interest identified |
| `Qualification`| 20% | Pipeline | BANT (Budget, Authority, Need, Timeline) assessed |
| `Needs Analysis`| 40% | Pipeline | Deep technical & requirements discovery |
| `Proposal / Quote`| 60% | Best Case | Formal quote or proposal delivered |
| `Negotiation / Review`| 80% | Commit | Legal, procurement, and terms discussion |
| `Closed Won` | 100% | Closed | Executed agreement, deal won |
| `Closed Lost` | 0% | Omitted | Deal abandoned or lost to competitor |
---
## 5. Unified Activity & Task Model
Tracks all activities, calls, emails, and to-dos against standard and custom entities.
### Activity / Task Schema
| Field Name | Type | Constraints | Description |
| :--- | :--- | :--- | :--- |
| `id` | UUID | Primary Key | Global unique identifier |
| `subject` | String(255) | Required | Activity headline / task title |
| `activity_type` | Enum / String(32) | Choices: `Task`, `Call`, `Meeting`, `Email`, `Note` | Classification of interaction |
| `status` | Enum / String(32) | Choices: `Not Started`, `In Progress`, `Completed`, `Deferred` | Execution status |
| `priority` | Enum / String(20) | Choices: `High`, `Normal`, `Low` | Urgency tier |
| `due_date` | DateTime | Nullable, indexed | Target completion or occurrence time |
| `description` | TextField | Nullable | Body notes or meeting transcript |
| `content_type` | ForeignKey(ContentType) | Indexed | Generic relation to any Django model |
| `object_id` | String(64) | Indexed | ID of target entity (Account, Contact, Lead, etc.) |
| `content_object`| GenericForeignKey | Resolves `(content_type, object_id)` | Polymorphic target link |
| `assigned_to` | ForeignKey(User) | Nullable, `on_delete=SET_NULL` | Action assignee |
| `completed_at` | DateTime | Nullable | Actual completion timestamp |
---
## 6. Lead Conversion Pipeline Mechanics
When a user or automated flow invokes `POST /api/leads/{id}/convert/`:
1. **Validation**: Verify `lead.is_converted == False`.
2. **Atomic Transaction (`transaction.atomic`)**:
- Locate or create `Account` matching `lead.company`.
- Create `Contact` with `first_name`, `last_name`, `email`, `phone`, linked to `Account`.
- If requested, create `Opportunity` with `name=f"{lead.company} - New Deal"`, `stage="Prospecting"`, linked to `Account` and `Contact`.
- Update `Lead`:
- `is_converted = True`
- `converted_at = timezone.now()`
- `converted_account = account`
- `converted_contact = contact`
- `converted_opportunity = opportunity`
- Re-link all related polymorphic `Tasks` and `Activities` from `(Lead, lead.id)` to `(Contact, contact.id)` or `(Account, account.id)`.