Components

A component is a reusable and composable unit of logic that performs a specific task. It encapsulates any transformation, inference, or computation and runs in a containerized environment to ensure reproducibility and platform consistency.

• Components have typed inputs and outputs to ensure compatibility in pipeline workflows.
• They may include logic for preprocessing, model inference, or postprocessing.
• Example use cases: running an ONNX model, applying a Kalman filter, resizing an image, or parsing raw data.

Components are versioned, shareable, and can be reused across multiple pipelines and projects.

Technical Definition

In Pipelogic’s execution model, a component behaves like a stateless function that consumes input streams and emits output streams:

func(Stream a, Stream b, ...) returns (Stream x, Stream y, ...)

The author defines:

• The required types and number of inputs
• The conditions under which the function executes
• The logic that determines how outputs are generated

Example

func(Stream a, Stream b) returns (Stream c)

This function might be defined to run when:

• It receives 2 values of a and 3 values of b, or
• It receives 3 values of a and 1 value of b

Execution is determined by matching one of these predefined consumption rules. The function returns zero or more outputs depending on the inputs received:

func(Stream a, Stream b) returns (Stream c) {
  if len(a) == 2 and len(b) == 3 {
    return {c1, c2, c3};
  } else if len(a) == 3 and len(b) == 1 {
    return {};
  }
}

Stateful Extension

To support more advanced functionality, Pipelogic extends this model:

func(VirtualInput, State, Stream a, Stream b, ...) returns (NewState, Stream x, ...)

This allows components to:

• Maintain internal state across executions
• React to system events or generate side effects
• Leverage the platform’s managed state and error handling

Pipelogic manages the state lifecycle, enabling parallel execution and automatic fault recovery. This functional design philosophy supports scalable pipelines, similar to map-reduce, while letting users write logic in any language.

Component Structure

A Pipelogic component is structured as a self-contained project with the following layout:

.
├── component.yml         # Configuration metadata
├── .pipecomponent        # Unique component ID (managed by Pipelogic)
├── src/                  # Source code
│   ├── main.py or main.cpp
│   └── requirements.txt  # (Python only)
└── tests/                # Input/output test cases
    ├── test_0/
    │   ├── input_0.txt
    │   └── output_0.txt
    └── test_1/
        ├── input_0.txt
        └── output_0.txt

Key Directories and Files

• src/ — Source code folder with the component’s implementation.
• tests/ — Contains test cases with input and expected output files.
• .pipecomponent — Internally generated metadata file with the component ID.
• component.yml — Main configuration file that defines the interface and metadata.

Configuration File

The component.yml file defines the behavior and metadata for a component. It specifies:

• Name
• Language and platform
• Input/output data types
• Tags and build system version

Example:

name: "Double Value"
platform: linux
language: py
build_system: 2
tags: ["default", "one-to-one"]
worker:
  input_type: Int64
  output_type: Int64

Explanation:

• The component reads a stream of Int64, doubles the values, and emits the result.
• It is compatible with both linux/amd64 and linux/arm64.
• Written in Python with versioned build tooling.
• Tagged for categorization and discovery in the UI.

⚠️ Type matching is strict — pipelines will only connect components if input/output types are compatible.

Next Steps

Learn to Create Your Own Component

• Add parameters, logic, and tests
• Release your component and reuse it across pipelines

🎓 Pro tip: Once you get comfortable with building components, explore the PipeLogic CLI for full control via terminal.