Components

A component is a reusable and composable unit of logic that performs a specific task. It encapsulates any transformation, inference, or computation and runs in a containerized environment to ensure reproducibility and platform consistency.

Components have typed inputs and outputs to ensure compatibility in pipeline workflows.
They may include logic for preprocessing, model inference, or postprocessing.
Example use cases: running an ONNX model, applying a Kalman filter, resizing an image, or parsing raw data.

Components are versioned, shareable, and can be reused across multiple pipelines and projects.

Technical Definition

In Pipelogic’s execution model, a component behaves like a stateless function that consumes input streams and emits output streams:

func(Stream a, Stream b, ...) returns (Stream x, Stream y, ...)

The author defines:

The required types and number of inputs
The conditions under which the function executes
The logic that determines how outputs are generated

Example

func(Stream a, Stream b) returns (Stream c)

This function might be defined to run when:

It receives 2 values of a and 3 values of b, or
It receives 3 values of a and 1 value of b

Execution is determined by matching one of these predefined consumption rules. The function returns zero or more outputs depending on the inputs received:

func(Stream a, Stream b) returns (Stream c) {
  if len(a) == 2 and len(b) == 3 {
    return {c1, c2, c3};
  } else if len(a) == 3 and len(b) == 1 {
    return {};
  }
}

Stateful Extension

To support more advanced functionality, Pipelogic extends this model:

func(VirtualInput, State, Stream a, Stream b, ...) returns (NewState, Stream x, ...)

This allows components to:

Maintain internal state across executions
React to system events or generate side effects
Leverage the platform’s managed state and error handling

Pipelogic manages the state lifecycle, enabling parallel execution and automatic fault recovery. This functional design philosophy supports scalable pipelines, similar to map-reduce, while letting users write logic in any language.

Component Structure

A Pipelogic component is structured as a self-contained project with the following layout:

.
├── component.yml         # Configuration metadata
├── .pipecomponent        # Unique component ID (managed by Pipelogic)
├── src/                  # Source code
│   ├── main.py or main.cpp
│   └── requirements.txt  # (Python only)
└── tests/                # Input/output test cases
    ├── test_0/
    │   ├── input_0.txt
    │   └── output_0.txt
    └── test_1/
        ├── input_0.txt
        └── output_0.txt

Key Directories and Files

src/ — Source code folder with the component’s implementation.
tests/ — Contains test cases with input and expected output files.
.pipecomponent — Internally generated metadata file with the component ID.
component.yml — Main configuration file that defines the interface and metadata.

Configuration File

The component.yml file defines the behavior and metadata for a component. It specifies:

Name
Language and platform
Input/output data types
Tags and build system version

Example:

name: "Double Value"
platform: linux
language: py
build_system: 2
tags: ["default", "one-to-one"]
worker:
  input_type: Int64
  output_type: Int64

Explanation:

The component reads a stream of Int64, doubles the values, and emits the result.
It is compatible with both linux/amd64 and linux/arm64.
Written in Python with versioned build tooling.
Tagged for categorization and discovery in the UI.

⚠️ Type matching is strict — pipelines will only connect components if input/output types are compatible.