API Reference
Typed, composable GStreamer pipelines in C++20 topics with brief descriptions are:
| Pipeline runtime | Graph, PipelineRun, options, reports, errors |
| Builder internals | Node interfaces and builder utilities used by the public Graph compiler |
| Nodes | Pipeline building blocks |
| Common nodes | Common GStreamer utility nodes (caps, convert, appsink, etc.) |
| I/O nodes | Appsrc/RTSP/file/UDP source and sink nodes |
| Graph fragments | Higher-level reusable Graph fragments composed of multiple nodes |
| Sima nodes | SimaAI-specific nodes (decode/encode/preproc/postproc) |
| RTP nodes | RTP helpers (depay/payload/caps fixups) |
| Model archive internals | Internal .tar.gz model archive loading and validation |
| Contracts | Builder-level validation rules and reports |
| GStreamer helpers | Thin wrappers and utilities around GStreamer APIs |
| Tensor | Tensor core types, constraints, conversions, adapters |
| Diagnostics | PipelineReport, debug probes, and observability helpers |
| Model | Loaded model archives, preprocess plans, and the simplified inference entry point |
| Runtime graph internals | Actor-style runtime DAG, named-port stages, and compiler/runtime substrate used by public Graph |
| Runtime graph nodes | Stage, fan-out, join, scheduler, and adapter nodes for runtime graphs |
| Runtime graph internals | Queue and mailbox primitives used by runtime graph execution |
| Policy | Decoder/encoder/RTSP/memory policy decision helpers used by the builder |
| Framework internals (SiMa reach-through) | Reach-through layer between core/ and the SiMa SoC pipeline internals |
Description
The Neat framework is the C++/Python library and runtime that runs models on the SiMa Modalix chip. It loads compiled .tar.gz model archives, assembles them into deterministic GStreamer pipelines, and executes inference at frame rate on heterogeneous compute (A65 host CPU + EV74 vector cores + MLA accelerator + hardware codec).
This site is the reference documentation for the framework's public API. For design rationale and worked examples, see the Architecture deep dive, which mirrors the architect-level documentation.
Neat has two parts
People at SiMa use the word "Neat" to mean two related but distinct things. Both exist; both are called Neat; they're complementary:
- Neat framework — the library you #include, the runtime that loads models, the GStreamer-based pipelines that execute inference. The subject of this site.
- Neat environment — a Docker container with shared filesystem mounts to the DevKit, integrated agent skills, and a single-point-of-contact workflow that lets developers (and AI agents) compile on a fast x86 host and test on real Modalix hardware. The environment is what enables the agentic workflow — but it works because the framework underneath is built for agents.
The framework runs on the chip, in production, on every Modalix everywhere. The environment is a developer convenience that exists alongside. Production ships the framework; developers also use the environment.
What the framework solves
Modern AI inference on the edge has hard constraints: frame-rate throughput on small power budgets, heterogeneous compute (CPU + vector + accelerator), zero-copy memory across processors, deterministic latency, and the ability to integrate cleanly with media pipelines (cameras, codecs, RTSP). Solving all of these by hand for every application is a year of work. The Neat framework absorbs that work into a single typed C++/Python API.
A typical Neat-framework application looks like this:
A dozen lines from "RTSP camera → real-time YOLO detection running on Modalix accelerator." Behind that simplicity is a lot of carefully-designed machinery — that's what this reference documents.
The eight main concepts
The framework's public surface is a small set of primary concepts. They form a clean progression from "the file on disk" to "the data that flows between processors at runtime":
| # | Concept | Purpose | Reference |
|---|---|---|---|
| 1 | Model archive | Sealed .tar.gz file from the compiler — kernels, weights, configs, and the MPK contract. | Model, MPK contract |
| 2 | Model | Loaded form of a model archive; the simplified entry point. | Model |
| 3 | Tensor | Typed data unit that flows between stages. | Tensor, Memory model, dtype contract |
| 4 | Nodes | Smallest building blocks; each wraps one (or a few) GStreamer elements. | Node APIs, GStreamer underneath |
| 5 | Reusable Graph fragments | Pre-made Graph fragments capturing common patterns. | Reusable Graph fragments |
| 6 | Graph | Assembly stage that turns Nodes, Models, and reusable Graph fragments into a runnable pipeline. | Graph, GraphOptions |
| 7 | Run | Live, running pipeline produced by Graph::build(). | Run, Async vs sync timing, Threading model |
Where to read what
| What you want | Where to look |
|---|---|
| Conceptual overview, architecture, design rationale | Architecture deep dive — the architect-level documentation, surfaced as navigable site pages |
| C++ public API — every class, method, field | C++ API Reference — generated from Doxygen comments |
| Python public API — pyneat module surface | Python API Reference — generated from nanobind bindings |
| Conceptual deep-dives (dtype contract, memory model, error codes, etc.) | dtype contract, memory model, graphs, timing model, threading, processor backends, GStreamer underneath, CVU kernels, MPK contract, error codes, build options, agentic workflow |
| Glossary, environment variables, scripts, error format | Glossary, env vars, scripts inventory, plugin error format |
| Onboarding, build, first inference | Installation, Build, Hello Neat |
| How to do specific things (debugging, runtime tuning, plugin failures) | Tutorial 015: tune throughput & queues, Tutorial 011: diagnose a pipeline |
| Coding standards, MPK contract, contribution policy | Coding standard, MPK contract, Architecture |
Why the framework is built for agents
A consequence of how the framework was designed: it's an exceptionally good substrate for AI code generation. Every framework error produces a structured GraphReport (machine-readable error code + reproducer command). Every public symbol has stable naming. Validation runs before any pipeline starts. Pipelines are serializable JSON. The public API is ABI-stable. Errors fail fast with actionable messages instead of silent fallbacks.
These properties weren't picked for AI agents specifically — they came from "make the framework deterministic, debuggable, and never hang the process" (see docs/contribute/architecture.md). But they happen to be exactly what an AI agent needs to write code that converges quickly. That's why the agentic workflow in the Neat environment delivers what it does — the framework is the substrate, the environment is the workshop.
The full story of why the framework is good for agents — fifteen specific design properties, each tied to architect-doc rationale — is in the Architecture deep dive.
Conventions in this reference
- All public types live under the simaai::neat namespace.
- Nodes live under simaai::neat::nodes::{common,io,rtp,sima,groups}.
- Headers are organized by responsibility: model/, pipeline/, builder/, nodes/, contracts/, policy/, gst/, plus convenience umbrellas under neat/. The source-tree include/graph/ runtime substrate is intentionally excluded from the public reference; application code should use simaai::neat::Graph / Run.
- Every public header has a file-level @file / @ingroup / @brief block. Group definitions live in docs/doxygen/groups.dox.
- Errors are returned as NeatError exceptions carrying a structured GraphReport. See the Error code catalog for the full taxonomy.
The rest of this site is the API itself. Start by browsing classes, or jump to the headline types: Model, Graph, Run, Tensor, Node APIs.
Generated via doxygen2docusaurus 2.0.0 by Doxygen 1.9.8.