Architecture

The workspace separates cognoxium-core, the PyO3 extension, and the typed Python facade.

Python application
      |
      v
typed Python API and immutable planner
      |                    |
      |                    +--> gettext diagnostics and renderers
      v
PyO3 extension (binary wheel)
      |
      v
Rust core: Arrow schema, hashing, native tokenizers, DataFusion session

Current execution boundary

Rust defines the canonical Arrow schema, payload enum, diagnostics shape, extension traits, hashing, tokenization, and an in-process DataFusion session. Binary wheels expose hashing and batched tokenization to Python.

The 0.1-series Python facade implements frame transformations with an immutable Python planner. It does not translate that plan into DataFusion. This distinction matters when evaluating current performance claims; see Performance.

Python owns record ergonomics, planning, plugin protocols, localized diagnostics, packing, manifests, and provider-shaped renderers. Python tokenizer callbacks are an explicitly slower fallback.

Dependency boundary

The core has no network client, model invocation, API-key handling, account requirement, or telemetry. Provider and framework dependencies belong in host applications or explicit plugins.

Plugin fingerprints are part of reproducibility, but Cognoxium cannot verify that a plugin’s declared fingerprint accurately represents external model or service state.

Extension direction

The payload struct reserves nullable children for Text, JSON, Binary, and Reference. Future renderer or token-profile support can therefore use the existing top-level payload shape. This is an architectural affordance, not a promise that multimodal packing exists in the 0.1 series. See the non-binding Status and non-binding roadmap.