Types
Types define how data is serialized, decoded, and communicated between components. They are the foundation for interoperability within your pipeline:
components can only communicate if their input/output types match.
Each component processes a stream of data with a specific type signature:
- It receives a stream described by a type
- Decodes, processes, and
- Emits a new typed stream
Types in Pipelogic are defined using Pipelang—our metaprogramming language based on the ANTLR g4 grammar format. This makes them expressive, extensible, and familiar to anyone with experience in grammar-based definitions.
đź’ˇ When is fine-grained type control needed?
Python: You usually don’t need to explicitly annotate types unless the type is ambiguous (e.g. Union
). If the value is clearly inferrable from context, the system will detect it automatically.
C++: You always need to define the type explicitly for the component to compile.
Atomic Types
Atomic types are the most fundamental types in Pipelogic’s type system. They represent a single, indivisible value — ideal for simple data like numbers, booleans, text, or raw binary.
âś… Supported Atomic Types
Type | Description | Example Value |
---|---|---|
Int32 | 32-bit signed integer | -42 , 1024 |
Int64 | 64-bit signed integer | 9223372036854 |
UInt32 | 32-bit unsigned integer | 300 |
UInt64 | 64-bit unsigned integer | 184467440737 |
Bool | Boolean true/false | true , false |
Float | 32-bit floating-point number | 3.14 |
Double | 64-bit floating-point number | 2.718281828459 |
String | UTF-8 encoded string | "hello world" |
Bytes | Raw binary data (e.g., encoded file) | b"\x89PNG..." |
Example: Streaming Text from a Large Language Model (LLM)
You're building a component that connects to a text generation model (like GPT, LLaMA, or Mistral) and streams output tokens as they are generated. Each token is a single string, sent one at a time. This makes it a perfect candidate for using the atomic type String
.
An example definition of the Atomic
type String
in Pipelang, Python, and C++:
Atomic Type | Definition
String
The code block below shows example values that can be assigned to the List
in Pipelang, Python, C++, and JSON:
Atomic Type | Values
"Hello"
"world"
","
"how"
"are"
"you"
"?"
Lists
List types represent an ordered collection of elements, all of the same type. They are ideal when you need to transmit a variable number of values — whether that's a stream of sensor readings, a collection of IDs, or a group of recent events.
Example: Log Aggregation from Edge Devices
You’re building a component that aggregates error codes from edge IoT devices and sends them periodically as a list. Each device accumulates a list of recent Int32 error codes, and your component emits this list every few seconds. This is useful for components that track device stability, trigger alerts if certain codes appear, or count frequency of specific errors.
An example definition of the List
type in Pipelang, Python, and C++:
List | Definition
[Int32]
The code block below shows example values that can be assigned to the List
in Pipelang, Python, C++, and JSON:
List | Values
[102, 504, 102, 301]
Tuples
Tuple types are ordered collections of values with fixed length and explicit types at each position. Unlike records (which use named fields), tuples rely on the order of elements to convey meaning. They’re ideal for compact, positionally meaningful data — like coordinates, RGB values, or sensor triplets.
Example: Geolocation Stream from GPS Sensor
You're building a component that receives raw GPS data from a sensor and emits a data stream containing latitude (Float) and longitude (Float). This data doesn’t need named fields — the order of elements carries meaning. Every reading follows the same position-based convention.
An example definition of the Tuple
type in Pipelang, Python, and C++:
Tuple | Definition
(Float, Float)
The code block below shows example values that can be assigned to the Tuple
in Pipelang, Python, C++, and JSON:
Tuple | Values
[48.1374, 11.5755]
Records (Structs)
Record types, also called structs, are collections of named fields with specific types. Unlike tuples (which are ordered and positional), records use named keys — making the data self-descriptive, easier to maintain, and ideal for structured communication between components.
🆚 What is the difference between Tuple and Record?
Feature | Tuple | Record |
---|---|---|
Structure | Ordered, unnamed | Unordered, named fields |
Example | (Float, Float) | {lat: Float, long: Float} |
Use Case | Coordinates, RGB, events | Profiles, messages, structured data |
Readability | Low | High |
Flexibility | Low (strict length/order) | High |
Example: License Plate Recognition (LPR) System
You're building a vehicle monitoring system for a parking garage. One component processes camera input and extracts structured information about each detected vehicle — including: (a) license plate number (text), (b) confidence score of detection (float), (c) time of detection (timestamp), and (d) the camera ID (int) where the vehicle was seen.
You want to package all this together into a structured output so it’s easy for downstream components to consume.
An example definition of the Record
type in Pipelang, Python, and C++:
Record | Definition
{
plate: String,
confidence: Float,
timestamp: UInt64,
camera_id: Int32
}
The code block below shows example values that can be assigned to the Record
in Pipelang, Python, C++, and JSON:
Record | Values
{
plate: "ABC-1234",
confidence: 0.93,
timestamp: 1728483923,
camera_id: 7
}
Unions
Union types represent values that can be of one of several distinct types. They are useful when input or output data can validly take on different forms, and you want to explicitly support that variability in your type system.
Example: User input for volume control in a multimedia application
In a smart TV input component called Input Google TV
, users can set the volume level either by: (a) Using a remote control, which sends whole numbers like 25, 50, or 100 (Int32), or (b) via voice command or slider, which may produce decimal values like 32.5 or 47.2 (Float). You want to allow both styles of input for maximum UX flexibility, but treat them as equivalent in logic.
In Pipelogic, union types are written using the | operator. This means a value can be either an Int32 or a Float, but not both at the same time.
An example definition of the Union
type in Pipelang, Python, and C++:
Union | Definition
Int32 | Float
The code block below shows example values that can be assigned to the Union
in Pipelang, Python, C++, and JSON:
Union | Values
# Int32
5
# or Float
5.25
Named Types
Named types are custom-defined types that give structure and meaning to complex data. They act like schemas—enabling reuse, strong typing, and cleaner definitions across components.
Example: Smart metering solution for classifying anomalies in electric current
In a project involving anomaly detection from smart meter readings in a power plant, we built a classification component called Classify Anomaly
. This component returns a named type called DetectedClass
, which contains a class ID and its associated confidence score.
The code block below shows an example definition of the DetectedClass
named type in Pipelang, Python, and C++:
DetectedClass | Definition
{
id: UInt64,
confidence: Double
}
The code block below shows example values that can be assigned to the named type DetectedClass
in Pipelang, Python, C++, and JSON:
DetectedClass | Values
{
id: 451,
confidence: 0.978
}
Nested Named Types
Just like named types allow you to define reusable schemas for structured data, nested named types let you compose these schemas into more complex hierarchies. This means you can define a named type (e.g. User
) and then use it as a field inside another named type (e.g. ChatMessage
), making your types more modular and expressive.
Example: Customer Support Chat Analytics
You’re building a Pipelogic component that analyzes customer support chats in real time. Each chat message comes with metadata like sender info, timestamp, and a sentiment analysis result. Instead of using one big record, you break the data into named types for reusability and clarity.
The code block below shows an example definition of nested named types in Pipelang, Python, and C++:
Nested Named Types | Definition
{
sender: User,
message: String,
timestamp: UInt64,
sentiment: Sentiment
}
The code block below shows example values that can be assigned to nested named types in Pipelang, Python, C++, and JSON:
Nested Named Types | Values
{
sender: {
id: 3344,
name: "Jane Doe",
is_agent: false
},
message: "I'm really frustrated with my last order!",
timestamp: 1717542001,
sentiment: {
label: "negative",
score: 0.91
}
}
Generic Types (Type Variables)
Generic types allow you to define flexible and reusable type structures that can work with any type, without sacrificing type safety. They are especially useful when a component needs to operate on data where the exact type is unknown or can vary, but must remain consistent throughout the structure.
đź§ How It Works
A type variable is any name starting with a lowercase letter, such as t
, a
, or myType
. It acts as a placeholder for a specific type, which will be substituted when the component is instantiated or connected. The same variable will always refer to the same type within a single type definition, thus, enforcing consistency. The type system is strongly typed, not weakly typed, and this is strictly enforced on named types as well—ensuring that type mismatches are caught early and reliably.
Type variables are local to each component. If a
appears multiple times within the same component, it always refers to the same underlying type.
Example: Building a Generic Geometry Component
You're developing a Pipelogic component that performs geometric calculations — such as measuring distances or resizing bounding boxes. These calculations could work with various numeric types (Int32
, Float
, Double
) depending on the source of the data. Instead of rewriting the same logic for each numeric type, you can define the component using a generic type variable, so it works with any numeric type. This approach allows you to write type-safe, reusable, and flexible logic using type variables — also known as generics.
The code block below shows an example definition of the Generic type in Pipelang, Python, and C++:
Generic Type | Definition
Point<t> := {
x: t,
y: t
}
Rectangle<t> := {
top_left: Point<t>,
bottom_right: Point<t>
}
The code block below shows example values that can be assigned to the named type Generic type in Pipelang, Python, C++, and JSON:
Generic Type | Values
{
top_left: { x: 12.5, y: 20.0 },
bottom_right: { x: 100.0, y: 220.5 }
}