Valstorm Document & CLM (Contract Lifecycle Management) API Guide

This guide details the endpoints, data mapping mechanisms, and PDF generation processes within Valstorm's CLM and Document subsystems.

The CLM module handles multi-tenant template rendering, dynamic data-merging from CRM records, document-filling sessions, PDF stenciling/flattening via PyMuPDF, S3 uploads, and secure token-based signing flows.

---

🏗️ Architecture Overview

Valstorm CLM consists of three core layers working together:

1. API Layer (document_routes.py): Handles internal session setups, generates secure sign-off tokens, and exposes authenticated endpoints for prep and public endpoints for signing.
2. Merging & Fetching Layer (data_fetcher.py): Resolves dynamic coordinates by mapping them to MongoDB records or feeding them into developer-defined Functions and Automations for server-side preprocessing.
3. Generation & Stenciling Layer (pdf_generator.py): Stencils dynamic strings and base64 signatures (images) onto PDF files at precise coordinates, flattening annotations via .bake() using PyMuPDF before returning directly or streaming to AWS S3.

---

📦 Data Structures & Schemas

1. Coordinates and Field Metadata

Each coordinate field is defined using absolute page numbers and horizontal/vertical coordinates represented as a percentage of page dimensions.

{
  "id": "field_sig_1",
  "type": "signature",       // 'signature', 'text', or 'date'
  "page": 1,                 // 1-indexed
  "x": 10.5,                 // % from left of page
  "y": 80.2,                 // % from top of page
  "width": 25.0,             // width as % of page width
  "height": 7.5,             // height as % of page height
  "signerRole": "Signer 1",  // role authorized to sign this field
  "mapping": "contact.first_name", // dot-notation mapping paths
  "staticValue": "John",     // fallback/static content
  "dateBehavior": "today"    // 'today' or 'empty' for date fields
}

2. Signer JWT Token Context

Signing tokens encode access credentials and context so signers don't require platform user profiles:

• user: Authorized system integration user UUID.
• org: Active organization UUID.
• signer_context:
  • contract_version_id: Target contract version UUID.
  • signer_role: Designated signer role string.
  • contact_id: Signer's CRM contact UUID.
  • email: Signer's email.

---

🔑 Route References

1. Document Preview

Generates a flattened, on-the-fly preview of the PDF by combining coordinate field metadata and record data without committing files to S3.

• Method: POST
• Route: /v1/document/preview
• Content-Type: multipart/form-data
• Payload Params (Form Fields):
  • template_id (string, Optional): Associated template UUID.
  • file_id (string, Optional): File S3 ID.
  • file (UploadFile, Optional): Local PDF file attachment.
  • fields_metadata (stringified JSON array): Array of Field Metadata objects.
  • record_data (stringified JSON dict): Active state mapping values.
• Response: 200 OK (binary stream with media_type="application/pdf").

---

2. Internal Initialization

Sets up an internal session to fill out an active template. Resolves all dynamic mappings via Functions/Automations and returns a preview PDF in base64.

• Method: POST
• Route: /v1/document/internal/init
• Headers: Bearer Auth Token
• JSON Payload (GenerateContractRequest):

  {
    "template_id": "e41d4c3e-feae-4b4a-a432-75f7cbb30e47",
    "contract_name": "Test NDA Agreement",
    "inputs": {
      "extra_terms": "Mutual non-disclosure for 5 years."
    },
    "originating_record": {
      "id": "519c7145-d5c6-45db-9cf8-2838cea08a42",
      "schema_name": "contact"
    }
  }

• Response (200 OK):

  {
    "pdf_base64": "JVBERi0xLjQKJ...",
    "fields": [
      { "id": "sig_1", "type": "signature", "signerRole": "Signer 1", "page": 1, "x": 10, "y": 80, "width": 20, "height": 5 }
    ]
  }

---

3. Internal Finalize

Finalizes an internally populated document. Applies manual signatures and text over top of the resolved preview, compiles the final PDF, uploads it, and attaches it to the originating record via the related_file schema.

• Method: POST
• Route: /v1/document/internal/finalize
• Headers: Bearer Auth Token
• JSON Payload (InternalFinalizePayload):

  {
    "template_id": "e41d4c3e-feae-4b4a-a432-75f7cbb30e47",
    "originating_record": {
      "id": "519c7145-d5c6-45db-9cf8-2838cea08a42",
      "schema_name": "contact",
      "schema_api_name": "contact",
      "name": "Jane Doe"
    },
    "signatures": {
      "sig_1": "data:image/png;base64,iVBORw0KGgoAAAAN..." // PNG Base64
    }
  }

• Response (200 OK):

  {
    "status": "success",
    "file_id": "9b1deb4d-3b7d-4ba9-9141-11d24c004a43"
  }

---

4. Generate Contract

Drafts a fresh parent contract and version record, merges CRM fields, uploads the prefilled PDF to S3, and issues secure signer-specific JWT tokens.

• Method: POST
• Route: /v1/document/generate-contract
• Headers: Bearer Auth Token
• JSON Payload (GenerateContractRequest):

  {
    "template_id": "e41d4c3e-feae-4b4a-a432-75f7cbb30e47",
    "contract_name": "Test NDA Agreement",
    "originating_record": {
      "id": "519c7145-d5c6-45db-9cf8-2838cea08a42",
      "schema_name": "contact"
    }
  }

• Response (200 OK):

  {
    "contract": { "id": "mock_contract_id", "status": "Draft", "name": "Test NDA Agreement" },
    "contract_version": { "id": "mock_contract_version_id", "version_number": 1, "status": "Draft" },
    "file_id": "9b1deb4d-3b7d-4ba9-9141-11d24c004a43",
    "signing_tokens": {
      "Signer 1": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
      "Signer 2": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
    }
  }

---

5. Public Retrieve Document

Exposes public fetching of documents requiring sign-off. Uses the embedded signing token for authorization. No general login is required. Returns the base64 PDF and coordinate fields restricted only to the authorized role context.

• Method: GET
• Route: /v1/public/document/{token}
• Response (200 OK):

  {
    "pdf_base64": "JVBERi0xLjQKJ...",
    "fields": [
      { "id": "sig_1", "type": "signature", "signerRole": "Signer 1", "page": 1, "x": 10, "y": 80, "width": 20, "height": 5 }
    ],
    "contract_name": "Test NDA Agreement"
  }

---

6. Public Sign Document

Accepts base64 signature images or text inputs, stencils them onto the current state of the document, uploads the updated version to S3, and marks the contract version status as "Signed".

• Method: POST
• Route: /v1/public/document/{token}/sign
• JSON Payload (PublicSignPayload):

  {
    "signatures": {
      "sig_1": "data:image/png;base64,iVBORw0KGgoAAA...", // png base64 signature
      "text_field_2": "Acknowledged by Jane Doe" // arbitrary typed text
    }
  }

• Response (200 OK):

  {
    "status": "success",
    "file_id": "7ca64e7c-88ab-49ef-b32c-63df77cbda6e"
  }

---

🛠️ Internal Mechanics & Integration Detail

1. PyMuPDF Rendering Engine (pdf_generator.py)

Coordinates are mapped from abstract percentages to absolute page pixels before being painted onto pages:

• Coordinates mapping:
  • $x_0 = \text{page_width} \times \frac{x%}{100}$
  • $y_0 = \text{page_height} \times \frac{y%}{100}$
  • $x_1 = x_0 + (\text{page_width} \times \frac{\text{width}%}{100})$
  • $y_1 = y_0 + (\text{page_height} \times \frac{\text{height}%}{100})$
• Dynamic Sizing & Signatures: Text stenciling uses page.insert_textbox(rect, text) to wrap contents. Signatures accept base64-encoded strings, stripping prefixes like data:image/png;base64, to load raw PNG/JPG frames directly into page.insert_image(rect, stream=img_bytes).
• Flattening: Calls .update() on annotations and flattens the file hierarchy using doc.bake() prior to extracting stream bytes via doc.tobytes().

2. Merged Data Pipelines (data_fetcher.py)

CLM dynamically binds custom code or workflows to prepops. If a template has fields mapped via mapping:

• Linked Function: Fetches the code sandbox module from FunctionManager and safely executes the whitelisted module context via safe_execute_async returning merged outcomes.
• Linked Automation (Workflow): Invokes the target automation via execute_workflow_by_id, feeding the record details directly into the variables pipeline, and returns the finished data map.

---

🧪 Testing and Mocks (document_test.py)

Use pytest to test the entire CLM system. To run the tests locally, run:

pytest app/document/document_test.py

The test files demonstrate standard patterns for:

1. Mocking S3 Access: Patching system_get_s3_file and system_upload_file_from_data to simulate reading and writing files from S3 in-memory.
2. Generating Dummy PDF Streams: Constructing in-memory PyMuPDF documents with blank pages to test stenciling coordinates.
3. Validating JWT Signature Issuance: Constructing signing tokens and verifying token context parameters decode precisely.