Start and End nodes
The pipeline's entry and exit — the dataset the Start node reads, how its filter and item limit choose which items a run launches, which dataset fields become inputs, and which produced fields the End node returns.
Every pipeline has a fixed entry and a fixed exit: the Start node, where a run reads its data, and the End node, where it returns its results. For the shorter entries alongside the other node types, see the overview of pipeline nodes.
The Start node is where the pipeline reads its data. A pipeline has exactly one Start node, and it cannot be added or removed. It reads no fields from other nodes: it binds to a dataset and exposes that dataset’s fields as the pipeline’s input fields.
Dataset
Section titled “Dataset”The one dataset the pipeline reads its items from. You cannot deploy until a dataset is chosen. Binding a dataset exposes its fields as the pipeline’s inputs and supplies the items a run launches from.
Switching the dataset
Section titled “Switching the dataset”What a switch keeps depends on where you make it.
| Setting | Switch in the editor | Switch through the agent |
|---|---|---|
| Item filter | Cleared — a set filter never survives the switch. | Replaced by the filter the change carries. Whatever the change leaves out is cleared. A change whose filter names fields the new dataset lacks is rejected. |
| Max items to launch | Always cleared. | Replaced by the maximum the change carries. Whatever the change leaves out is cleared. |
| Input fields | Kept only when no filter is set and the new dataset has a field for every input, matched by name. Otherwise, reloaded with all of the new dataset’s fields. | Kept when the new dataset has every one of them, and emptied otherwise. |
Item filter
Section titled “Item filter”Narrows which items a run launches. You build it by hand on the Start node, or ask the agent, which sets the dataset, filter, and item cap in one change. Conditions across fields combine with “and”: an item launches only if it satisfies every one. For one field, the filter matches one of three ways:
| Mode | An item matches when | Notes |
|---|---|---|
| By value | a recorded value for the field equals (=) or does not equal (≠) the text you give, entered under JSONPath, Operator, and Value; Latest revision only narrows it to the newest value | Latest value by default, or any value the field has held. Combinable with a source restriction on the same field. |
| By source | under Created by, the field’s value was recorded by a source you select — a specific person, or an automated source such as an earlier pipeline run | |
| By absence | the item has no recorded value for the field — the No revision for this field checkbox | The only way to select such items: value matching requires a recorded value, so an item without one matches neither = nor ≠. Because each run writes its outputs back to the dataset, this is also how a run skips items an earlier run already labeled. |
The value comparison is textual: values match as text, character for character. Greater-than, less-than, range, and contains tests do not exist, so a numeric threshold (such as a score above 0.8) needs an upstream node to write a text flag the filter can match. The match can target one property inside a structured value. A property missing from the recorded value counts as empty text — it equals an empty comparison value and differs from every non-empty one. Value matching is unavailable for a field where any recorded value is larger than 32 KB as JSON text. Matching by source and by absence still work there.
The filter selects items independently of what the run then reads. An item may qualify on an older value or on a value from a selected source, yet at launch each input field still delivers the item’s latest value from any source — so the value that selects an item can differ from the value the pipeline receives.
The filter’s timing has two consequences:
- It evaluates once, at launch, against a frozen cutoff. The filter reads only values recorded before the run launches. A value recorded later never qualifies an item.
- An earlier run still in flight is invisible to it. Output values another run writes back after this run launches fall past that cutoff, so this run’s filter never sees them. Wait for an earlier run to finish before launching the next batch.
Max items to launch
Section titled “Max items to launch”Caps how many matching items the run launches: an optional whole number from 1 to 10,000. Set it by hand on the Start node, or ask the agent, which sets it with the dataset and filter in one change. Items launch in dataset order — the order they were added to the dataset.
| Maximum | The run launches |
|---|---|
| Empty | every matching item |
| At or below the matching count | the first that many matching items |
| Above the matching count | every matching item, and the field warns: Only N items match the current filter. Lower the limit or widen the filter. |
The Start node reports the selection as N matching items and, once a maximum is set, N matching items · will launch first M. While the matching count is known, the field lowers an entry above that count to the count. A maximum can still end up above it — you typed it while the count was loading, or a filter change later narrowed the selection.
No run launches more than 10,000 items. A selection above 10,000 — more than 10,000 matching items with no maximum set — turns the count red and shows the alert This run would launch N items, more than the maximum of 10,000 per run. Set “Max items to launch” to 10,000 or fewer, or narrow the filter. Two gates then refuse the run: the deploy check reports the same limits, without the “per run” wording and with plain 10000, and if a filter widens the selection after that check, the run itself is refused with This pipeline run would process N items, which exceeds the maximum of 10000 items per pipeline run. Setting a maximum of 10,000 or fewer, or narrowing the filter below 10,000, clears the block.
Pipeline input fields
Section titled “Pipeline input fields”The item filter and input optionality answer different questions: the By absence filter selects which items launch at all, while marking an input optional lets a launched item run without that value — the field model covers optionality in full.
The dataset fields exposed as the pipeline’s inputs. Selecting a dataset for the first time adds all of its fields. Internal fields the platform maintains are never exposed. What a later switch keeps or reloads follows the rules under Dataset. A field can be removed from the inputs and added back from the list of dataset fields not yet used as inputs — by hand on the Start node, or by asking the agent. Each input is required or optional, as covered in the shared field model. A launched item with no recorded value for a required input fails on entry with a message naming the missing inputs, and still occupies one of the run’s launched items. If the bound dataset later loses a field the inputs or the filter reference, the Start node flags the stale reference.
The End node is where the pipeline returns its results. A pipeline has exactly one End node, and it cannot be added or removed. It collects the fields produced by upstream nodes and returns them as the run’s output. It binds to no dataset and produces no new fields of its own.
Pipeline output fields
Section titled “Pipeline output fields”Which of the fields produced upstream are returned as the run’s output. The node lists them under Output. Choose them by hand on the End node’s Output list, or ask the agent. These fields are carried into a run’s export, one entry per item — see exporting your results.
Related
Section titled “Related”- Overview of pipeline nodes — the other node kinds and how they compare
- Label a large dataset in batches — the batch workflow that skips items an earlier run already labeled
- Export your labeled results — how the output fields you choose become the run’s export