Most streaming pipelines hand model scoring to a separate inference service. As a result, every event makes a network round trip to get its score. The service also runs as its own tier, so teams deploy and scale it separately. In addition, every new model version means redeploying the service or the pipeline. Even the model format depends on the serving stack behind the endpoint. In the end, teams operate two systems for what should be one step in the data path.
What Active(AI) changes
Active(AI) moves that step into the pipeline itself. It is a set of VoltSP plugins that run model inference inside the pipeline worker. In other words, scoring happens in the same process that handles each event. The sections below cover how events become model inputs and how models load and cache. They also show how new versions reach a running pipeline through MLflow. Finally, the scored event can go straight into a Volt stored procedure. There, the model’s output becomes one input to a decision, not the decision itself.
How it works
Three components do the work. First, the ONNX processor loads a standard ONNX model and runs it on each event. A tensor extractor turns the event into the model’s input tensor. You can write it in Java or as a JavaScript function in YAML. Then a compute function builds the outgoing event from the original input and the model’s outputs.
Second, the ONNX GenAI processor does the same for generative models. It supports chat templates, generation parameters and optional token-by-token streaming. Third, the MLflow model resource watches one registered model in an MLflow registry. It tracks either a named stage, such as Production, or the latest version. By default, it polls every 60 seconds. When a new version appears, the resource downloads it. The ONNX processor then reloads the model without restarting the pipeline.
Meanwhile, models load from local disk or an S3-compatible bucket and are cached locally. Inference runs on CPU by default, and the CUDA execution provider is available through configuration.
Who it’s for
Active(AI) is built for platform engineers running VoltSP pipelines and data science teams shipping models to production. For both, the shift is from maintaining an inference fleet to treating inference as one more processor. Data science keeps its tooling, because any framework that exports to ONNX can supply the model. MLflow also stays the registry of record. As a result, promoting a version to Production becomes the deployment. Likewise, moving the earlier version back becomes the rollback. Inference capacity grows as the pipeline scales out. Plus, the scored event can flow to any VoltSP sink. Read on to see how Active(AI) fits alongside Volt for Streaming Decisions in a working pipeline.