Pipeline

The pipeline module implements a type-state pattern for the document processing workflow.

Related Chapters

• Validation - Disposition and confidence types
• Visualization - PipelineGraph rendering
• Workflow - TOML DSL compilation
• Verify - Multi-model verification

State Types

fn ingested() -> validating
fn validating() -> classifying
fn classifying() -> reconciling
fn reconciling() -> committed
if confidence < 0.5 -> needs_review
if human_approved == true -> classifying
if fatal_error == true -> error

flowchart TD
    ingested["ingested"]
    validating["validating"]
    classifying["classifying"]
    reconciling["reconciling"]
    committed["committed"]
    confidence_lt_0_5{"confidence < 0.5"}
    needs_review["needs_review"]
    human_approved____true{"human_approved == true"}
    fatal_error____true{"fatal_error == true"}
    error["error"]
    ingested --> validating
    validating --> classifying
    classifying --> reconciling
    reconciling --> committed
    confidence_lt_0_5 -->|"true"|needs_review
    human_approved____true -->|"true"|classifying
    fatal_error____true -->|"true"|error

• Ingested: Document has been parsed
• Validating: Checking data integrity
• Classifying: Categorizing transactions
• Reconciling: Matching against external data
• Committed: Finalized andauditable
• NeedsReview: Awaiting human approval

Type-State Pattern

The pipeline uses Rust’s type system to enforce state transitions at compile time:

#![allow(unused)]
fn main() {
pub struct PipelineState<S> { /* ... */ }

impl PipelineState<Ingested> {
    pub fn validate(self) -> PipelineState<Validating> { ... }
}

impl PipelineState<Validating> {
    pub fn classify(self, category: String) -> PipelineState<Classified> { ... }
}
}

This ensures invalid state transitions are caught at compile time.

Statig Integration

The pipeline uses statig for hierarchical state machine (HSM) implementation with:

• Superstates for grouping related states
• State-local storage for context
• Async-first design