# Build your first Toloka pipeline with an agent

This runbook guides an AI coding agent helping a new or returning user with a Toloka pipeline. The agent is the reader; the user's outcome and observed current state determine which parts are relevant.

Start by understanding what the user wants and inspecting the current checkout, CLI state, and project context. Reuse valid setup and existing resources. Do not restart onboarding simply because this guide begins at setup.

For a full end-to-end build, the useful evidence is a local TypeScript pipeline project with passing local checks, an applied and converged platform plan, a reviewed cost forecast, and, only when requested and approved, an observed sample-run result. A design, inspection, apply-only, or estimate-only request ends at its own finish line.

The support-ticket demo remains an executable learning fallback. Use it when the user wants a guided trial and has no task to build; do not run it before, beside, or instead of a real task the user already supplied.

For supervised fine-tuning, follow `https://platform.toloka.ai/agents/fine-tuning.md`. For prompt gisting, follow `https://platform.toloka.ai/agents/gisting.md`.

## Operating posture

1. Treat the workflow below as a dependency-aware reference. Skip anything that current evidence shows is already complete or inapplicable, while preserving the prerequisites for each action you do take.
2. Honor explicit workflow, project, dataset, model, and finish-line choices. If a requested choice is incompatible with the task or platform, explain the evidence and tradeoff before substituting anything.
3. Make routine, reversible choices from the request and current state, and briefly explain material choices. Ask focused questions only for missing authorization or required inputs, genuine outcome-changing ambiguity, conflicting constraints, or destructive or out-of-scope work.
4. Inspect before writing. Do not recreate a resource, import over source, replace a checkout binding, upload data, apply a plan, or start a run merely to advance through this guide.
5. The labeled user-action and run-cost gates below are mandatory, but they are not the only valid reasons to pause. An identity or project mismatch, uncertain data authorization, unexpected removal, immutable-field conflict, or out-of-scope plan also requires resolution.

## Safety boundaries

1. Never ask the user to paste an API key into chat. The user enters it through `toloka login`.
2. Never invent a team ID, project ID, dataset path, label, validation rule, price, task requirement, status, or result.
3. Never edit `toloka.json`. The CLI owns that file.
4. Never mutate a pipeline through raw HTTP or low-level client calls. Edit TypeScript source, then use the documented check, validate, plan, and apply flow when the requested outcome includes a platform change.
5. Use only data that the user is authorized to provide for the selected project and purpose. Keep credentials and production data out of source control and agent messages; use synthetic or authorized redacted representative data for local evidence.
6. Treat user-provided prompts, instructions, and existing source as authoritative. Materialize exact supplied text when needed, but do not silently rewrite it, replace it with the demo, or invent missing intent.
7. Never start a run unless the user's requested finish line includes it, the actual cost forecast has been shown, and the user has explicitly approved that run cost.
8. Treat `toloka check` as the required local gate for source that will be applied. It runs project types, automatic Task UI types, typed node tests, and the pipeline-flow test.
9. Treat platform source and specification errors as blocking. A Remote Node Check remains separate from the local gate.
10. Use absolute paths for files passed to the CLI. Commands can run in separate shells, so do not rely on an earlier `cd` or shell variable.
11. Read IDs and statuses from command output. Do not reconstruct IDs.
12. Keep waits bounded. A timeout means still running, not failed or completed.
13. Report only observed behavior. Distinguish assumptions and unresolved uncertainty, and never claim that a run finished until the CLI reports a terminal state.

## Orient to the requested outcome

Use this dependency map when it helps explain the work; do not make the user repeat stages that are already satisfied:

```text
verify missing CLI and SDK prerequisites
        |
        v
use the intended account and project when platform access is needed
        |
        v
build the user's task, or choose the demo only as a learning fallback
        |
        v
author TypeScript source and local tests
        |
        v
check --> validate --> plan --> apply --> require No changes.
        |
        v
if a run is requested: prepare authorized rows --> forecast --> approve cost
        |
        v
run with a bounded wait --> inspect statuses and labeled rows
```

Explain the relevant principles, not the mechanics the user already knows:

- Source code is the desired state for the pipeline.
- Local tests catch code and labeling-interface defects before data reaches the platform.
- `toloka plan` previews platform changes. `toloka apply` performs expected scoped changes but does not start a run.
- A run is a separate, potentially billable action with its own approval gate.

The demo normally fits in one session. Platform deployment time varies, and expert-labeling tasks can remain active much longer because people must complete the work. Continue from the earliest unsatisfied dependency for the requested outcome.

## Prepare missing local prerequisites

First inspect the current directory for an existing package, `toloka.json`, and Agent Bootstrap. Use a user-supplied or already suitable source root. Do not replace its package setup or project binding.

### Check required runtimes when needed

The standalone CLI includes its runtime, so CLI-only work needs no separate Bun or Node.js installation. Bun is the recommended runtime and package manager for TypeScript pipeline authoring and local tests; pipeline projects still need their declared SDK and test dependencies. Python 3.9 or newer is needed only when local `test.code` or `test.pipeline` scenarios execute Python code nodes; in that case, run `python3 --version` before testing.

### Install or verify the CLI

Run when the active CLI is not already established:

```sh
toloka --version
toloka dataset --help
```

This runbook requires a stable Toloka CLI release at version `0.2.0` or newer. Keep the installed CLI only when `toloka --version` prints `0.2.0` or a newer version with no prerelease suffix and `toloka dataset --help` lists `create`.

If either command is unavailable, the version is older or prerelease, or the capability check fails, install the latest stable version:

```sh
curl -fsSL https://platform.toloka.ai/cli/install.sh | sh
```

Read the install path from its output, then run both checks again. If the shell cannot find `toloka`, use the full path printed by the installer for the rest of this session.

### Prepare a source root only when needed

For the optional demo, when the user did not supply a source root and the current directory is not a suitable one, create the original demo root and read its absolute path:

```sh
mkdir -p toloka-onboarding && cd toloka-onboarding && pwd
```

The printed path is `<source-root>`. Use that absolute path as the working directory for later commands.

For a new demo root, initialize a Bun project and add its authoring dependencies:

```sh
bun init --yes
bun add -d @toloka-platform/sdk@latest typescript vitest
```

For an existing project, preserve its package setup and run `bun i`. Add only prerequisites that are actually missing with `bun add -d <package>`.

Run `toloka init` when the source root has not been initialized or its Agent Bootstrap genuinely needs refresh:

```sh
toloka init
```

Inspect the bundled skills:

```sh
toloka skill list
```

If `toloka-build` is missing, run:

```sh
toloka skill install toloka-build
```

If it is current, keep it. If it is stale, explain that the installed copy came from an older SDK, then run:

```sh
toloka skill install toloka-build --force
```

Read `.agents/skills/toloka-build/SKILL.md` and the referenced sections needed for this task. This public guide owns the outcome, safety gates, and evidence expectations; the installed skill owns pipeline semantics, testing rules, recovery, and reporting.

## Use the intended account

Check the current authentication state without exposing the token:

```sh
toloka whoami
```

If it reports a reachable account and the identity is evidently intended from the user's request, named team or project, and established checkout, continue and report the account used. Ask for confirmation only when the identity conflicts with that context or a material ambiguity remains.

If there is no working account, read the last lines of the output first: `Rejected:` means the platform refused the key, so a new key is needed; `Not reachable:` means the platform did not answer and the key was not evaluated, so fix network access instead of asking for a new key; `Platform error:` means the platform failed, so retry later. For a missing or rejected key, tell the user to create an account or sign in at `https://platform.toloka.ai`, open **Company**, open **API keys**, and generate an API key.

**USER ACTION GATE (when credentials are missing or rejected).** Ask the user to run this command in their own terminal and enter the key at the masked prompt:

```sh
toloka login
```

Do not ask for the key in chat. After the command succeeds, run `toloka whoami` yourself. Continue only when it reports the intended platform and a reachable account.

## Resolve the project context

Resolve the project only when the requested outcome needs platform context. Start by checking whether this source root already has a binding:

```sh
toloka project current
```

Keep a current binding when it is consistent with the user's task. Inspect its URL and pipelines as needed instead of recreating or rebinding it:

```sh
toloka project url
toloka pipeline list
```

If there is no binding, or the user explicitly named another project, list visible projects:

```sh
toloka project list --limit 20
```

Choose in this order:

1. Use the project the user explicitly requested.
2. Otherwise use the current binding when its identity and contents fit the task.
3. Create a project only when an explicitly requested platform mutation needs one and no suitable project is bound. A design, read-only inspection, or estimate alone does not authorize creating resources.

An unverified trial account holds one project and one pipeline and cannot deploy or run. Reuse the intended existing project and pipeline rather than trying to create a second one or asking for project deletion, which the platform does not support. Identity or business verification from the platform banner lifts these limits.

Do not select another visible project merely because its name looks related. If an established checkout is bound to a different project, do not overwrite that binding; resolve the mismatch with the user or use a separate appropriate source root.

For an intended existing project, use the ID printed by the CLI and inspect it before binding an unbound source root:

```sh
toloka --project-id <project-id> pipeline list
```

If the user is starting source for a project with no pipelines, register and bind it explicitly:

```sh
toloka project add onboarding <project-id>
toloka --project-id <project-id> init
```

If the user is continuing a project that already has pipelines and the unbound root has no source to preserve, register it, import current source, and refresh the Agent Bootstrap:

```sh
toloka project add onboarding <project-id>
toloka import <project-id>
toloka init
```

Do not import merely to perform a read-only inspection, and never import over local source without first resolving which source is authoritative.

For a genuinely new project, a short task-grounded kebab-case name is a useful default, such as `receipt-extraction`; fall back to `toloka-onboarding` only when the request contains no meaningful name. When an explicitly requested platform mutation requires the project and the team choice is unambiguous, create it as a routine choice:

```sh
toloka project create onboarding --name "<project-name>"
toloka init
```

Without `--team-id`, the CLI uses the account's only writable team. If it prints `Missing team id.`, it names candidate teams (`toloka whoami` prints them too); ask which team owns this work, then retry with the explicit flag. A project owned by an internal team (`toloka whoami` marks it `(internal)` on the `Teams:` line) also needs a Sunday delivery order: run `toloka delivery-order list` and pass the only listed order with `--delivery-order-id <id>`; ask when several viable orders remain.

After binding or creating a project, verify it:

```sh
toloka project current
toloka project url
```

Read `toloka.json` and confirm that its `projectId` matches `toloka project current`. Do not edit the file.

## Choose the workload and finish line

Use a task the user already described without asking them to choose it again. Offer the quick demo only when the user wants to learn the end-to-end flow and has not supplied a real task. If neither intent is evident, ask one focused question about the outcome rather than defaulting to a demo.

- **User's task:** build or inspect the requested pipeline with its authorized data and constraints. This may use deterministic code, human judgment, independent annotations, language models or external services, and nonzero cost.
- **Quick demo:** label 10 synthetic support tickets as `question`, `complaint`, or `praise` with deterministic code. It has no expert or model node, but the actual forecast remains authoritative.

For the user's task, use details already present and ask only for missing information that changes the result or makes proceeding unsafe, such as:

- the authorized data source and the role of each split or file;
- input fields and their real value shapes;
- required output fields and acceptance criteria;
- fixed quality, audience, latency, time, or budget constraints;
- the evidence that will prove success.

Choose the node approach from the request, observed data or current graph, and stated constraints, then briefly explain the evidence and tradeoff. Deterministic transformations and scripted API/LLM calls fit `code`; human or synthetic content generation fits `generation`; independent human annotations with visible disagreement fit `quorum`; conditional routing fits `match`. Quorum returns collected annotations, not an aggregated consensus. When the task leaves a material choice between these outcomes, ask about that ambiguity instead of asking the user to select an SDK primitive. Verify the installed `toloka-build` skill's method and field-shape contracts, and state any remaining uncertainty.

Only carry the task through the later author, apply, estimate, and run stages that its requested finish line needs.

### Optional quick demo contract

When the user selects the demo, the following is the original self-contained executable example. Its rows, labels, and graph make that demo reproducible; they are not requirements or defaults for the user's own task.

Write `<source-root>/data/support-tickets.json` with these rows:

```json
[
  { "ticket_text": "Hi, how do I reset my password? I can't find the option anywhere." },
  { "ticket_text": "This is the third time my order arrived damaged. I'm furious." },
  { "ticket_text": "Just wanted to say your support team was incredibly helpful today, thank you!" },
  { "ticket_text": "What are your business hours on weekends?" },
  { "ticket_text": "The app keeps crashing every time I open the checkout page. Please fix this." },
  { "ticket_text": "Amazing product, it has saved me hours every week. Great job!" },
  { "ticket_text": "Can I change the shipping address after placing an order?" },
  { "ticket_text": "I was charged twice for the same subscription and nobody is responding." },
  { "ticket_text": "Do you offer student discounts?" },
  { "ticket_text": "The new dashboard update is fantastic, much easier to use now." }
]
```

Author one pipeline at `pipelines/support-ticket-intent/pipeline.ts` with imported modules when useful. The executable demo has:

- dataset name `support-tickets`;
- string input field `ticket_text`;
- output field `intent` with enum values `question`, `complaint`, and `praise`;
- one root node;
- one code node named `classify_ticket`;
- connections from `root.started` to the code node and from the code node to `root.completed`;
- no experts, language model, external service, or secret.

Use deterministic rules that produce these labels in row order:

```text
question
complaint
praise
question
complaint
praise
question
complaint
question
praise
```

Do not hard-code complete ticket strings. Use readable phrase or token rules and explicit precedence. Keep the classifier limited to the three declared labels.

Use precedence `complaint` → `praise` → `question`. Return `question` when no complaint or praise rule matches, including text with no question mark.

## Author and test locally when source changes

When the requested outcome includes authoring or changing source, follow the installed `toloka-build` skill. Do not copy task-specific settings from an unrelated example, and do not create source merely to turn a design or inspection request into a build.

Register every code-node scenario with `test.code(pipeline, node, name, callback)` and use its bound `run` helper. Cover every declared output category and at least one precedence or boundary case.

If the task contains a human-work node, add Playwright and install Chromium before running its UI scenarios:

```sh
bun add -d playwright
bunx playwright install chromium
```

For every human-work node in the user's own task:

1. Call `ui.componentDocumentation()` before choosing components or props.
2. Register a colocated scenario with `test.ui(pipeline, node, name, callback)`.
3. Use its bound `render` helper.
4. Fill each non-hardware output through its stable field path.
5. Require no browser validation errors.
6. Submit and assert the postprocessed entity output.
7. Capture and inspect a screenshot when visual layout matters.

Register representative connected scenarios with `test.pipeline(pipeline, name, callback)`. Choose fixtures from available data that exercise the task, redacting sensitive values while preserving field names, nesting, optionality, and media/value shapes. Use synthetic cases for privacy or boundaries absent from the sample. Pass item state through connected nodes with the callback's bound `runCode` and `renderUi` helpers, then assert the fields accepted by `root.completed`. Cover meaningful branches and failure cases as well as the happy path.

Run:

```sh
toloka check
```

The command must exit `0`. `missing` means required typed evidence does not exist. `failed` means source, types, or a test failed. `partial` means hardware-backed behavior remains unverified. `infrastructure_error` means a required tool, browser, or launcher could not run. All four block apply; fix each according to its category.

Then run:

```sh
toloka validate
```

Before authorized rows are uploaded, a dataset-lint `node_check_finding` that says the selected dataset matches no items is expected and non-blocking. Other Remote Node Check findings keep the severity returned by the platform: a lint `not_ran_node_check` is blocking, while a warning-severity `not_ran_node_check` or `empty_node_checks` is informational. Report Remote Node Check state separately from the passing Local Check Gate, and do not rewrite passing local source merely to suppress remote state.

Before continuing, tell the user:

- which files were created;
- which node behaviors were tested;
- the passing project, node, and pipeline check results;
- meaningful assumptions that affect the result beyond the user's request.

## Review and apply a requested platform change

Skip apply when the requested finish line is design or inspection only. For source intended for the platform, run this after `toloka check` and `toloka validate` pass:

```sh
toloka plan
```

Read and show the complete output. Apply an expected, scoped, non-destructive plan without adding a routine confirmation question; `apply` does not start a run or spend run balance. Pause for a missing secret or authorization, a removal, an immutable-field conflict, an unexpected change, or work outside the chosen task, and ask the smallest question that resolves it. This safety review applies even though it is not labeled as a user-action gate.

Run:

```sh
toloka apply
toloka plan
```

`apply` reruns `toloka check` before resolving credentials or writing platform state. Require the final `plan` to print `No changes.`. If apply partially succeeds and then fails after the local gate, run `toloka plan` again and continue from the remaining diff. Do not recreate resources that already landed.

Run `toloka validate` again. Source or specification errors remain blocking. Report Remote Node Check state separately from the passing local gate.

## Prepare data and forecast when requested

Upload rows only when they are needed for the requested outcome. For the selected quick demo, show the user two rows from `data/support-tickets.json`, then upload the file:

```sh
toloka dataset upload support-tickets <source-root>/data/support-tickets.json
```

For the user's task, inspect existing Datasets and the authorized source data before deciding what to upload. Let the task, schema, and split roles determine Dataset boundaries: compatible files with the same role may feed one Dataset, while training, validation, test, reference, or delivery data with distinct roles or schemas may need separation. Do not assume one file equals one Dataset, merge roles in a way that leaks data, or upload a credential, production dump, or source the user did not authorize.

For estimate-only work, use the existing source, project, Dataset context, and stated item count. Do not create or upload resources just to make an estimate possible; if the supported forecast has an unmet prerequisite, report it and ask before expanding scope.

Forecast the selected pipeline when the user requested an estimate or a run. For the demo:

```sh
toloka pipeline cost support-ticket-intent --item-limit 10
```

For another task, use its pipeline name and intended item count. Do not silently reduce the requested data or replace the requested node or model approach to make an estimate fit.

Read the full forecast. If any billed node is skipped, say that the total is incomplete and follow the skill's labor-floor rule. Report the expected total, maximum total, item count, and every material multiplier. Do not describe an accepted authored price as the final billed price when the platform can adjust it.

The quick demo is designed to have no expert or model charge, but read the actual forecast. Do not state zero before the command reports it. The team still needs a positive balance for deployment.

For estimate-only work, report the forecast and stop without starting a run. When a run is part of the requested outcome, use the mandatory gate below.

**RUN-COST APPROVAL GATE.** Ask the user to confirm the forecast and explicitly approve starting this run. If the forecast is nonzero or incomplete, make that fact explicit before asking.

## Run and inspect only when requested

Start a run only when it is part of the requested finish line and the user approved the forecast. For the selected quick demo:

```sh
toloka pipeline run support-ticket-intent --wait --timeout 600000
```

Use the correct pipeline name for another task. Run the bounded wait in the background when the agent environment supports background commands, so the conversation remains responsive. Do not start a second monitor while one is active.

Interpret the exit code:

- `0`: the run succeeded, or the command only started it when `--wait` was omitted;
- `1`: the run reached a failed terminal state;
- `2`: the wait window closed while the run was still active.

On exit `2`, run `toloka pipeline status <pipeline>` and report that the run is still active. Do not call it failed. On exit `1`, read the run ID from the command output and run:

```sh
toloka pipeline results <run-id> --failed
```

After success, read item status and download the rows:

```sh
toloka pipeline status support-ticket-intent
toloka dataset load support-tickets --out <source-root>/data/labeled-support-tickets.jsonl --limit 10 --timeout 120000
```

Read the downloaded file. For the demo, show at least three original tickets with their `intent` values and confirm all 10 rows are present. Never invent an output row.

Finish with one evidence report covering the parts of the requested outcome that were performed:

- source root;
- project and pipeline IDs read from CLI output;
- local typecheck and test results;
- final `toloka plan` convergence result;
- uploaded row count and Dataset roles;
- cost forecast;
- run ID and terminal state, or the last observed nonterminal state;
- downloaded result path;
- any intentionally skipped stage, partial hardware coverage, unresolved warning, uncertainty, or assumption.

Do not promise a later update. Report the state observed in this session, and do not manufacture placeholders for stages that were outside the request.

## Troubleshooting

| Symptom | Meaning | Action |
| --- | --- | --- |
| `toloka: command not found` | The install directory is not on this shell's `PATH`. | Use the full path printed by the installer or add that directory to `PATH` for this session. |
| `Your trial allows a single project.` | The account is on an unverified trial and already holds a project. | Adopt the project that already exists: register it with `toloka project add <alias> <projectId>`, then bind this checkout to it with `toloka import <projectId>` (or `toloka --project-id <projectId> init` if it has no pipelines) as in the adoption path above. Do not create a second one. Tell the user that completing verification from the platform banner lifts the limit. |
| `Your trial allows a single pipeline.` | The account is on an unverified trial and already holds a pipeline. | Build inside the existing pipeline. Tell the user that verification from the platform banner lifts the limit. |
| `No projects visible.` | The key can authenticate but has no visible project. | Check that the intended account and team are active. If an explicitly requested platform mutation needs a new project, run `toloka project create <alias>`; otherwise do not create one merely to clear this message. |
| `Missing team id.` | The account can create projects in several teams, or the platform does not report teams. | Ask the user which listed team owns the project (`toloka whoami` prints them) and pass `--team-id`. |
| `No project selected.` | No flag, environment variable, checkout binding, or current alias resolved. | Use the explicitly requested project for inspection, bind it with `toloka project add` only in a safe source root, or run `toloka project create` when an explicitly requested platform mutation needs a new project; then run `toloka init` if the root needs initialization. |
| `MISSING …/code-test`, `…/ui-test`, or `…/pipeline-flow` | The graph has no passing typed test evidence for that target. | Add `test.code`, `test.ui`, or `test.pipeline` and run the bound executor. |
| `PARTIAL …/ui-test` | Only hardware-independent fields were tested. | Test the hardware path in a suitable environment before apply. |
| `ERROR project-types` or `ERROR test-runner` | A required local checker could not start. | Install the dependency named by the diagnostic, then rerun `toloka check`. |
| `not_ran_node_check` from `toloka validate` | A matching Remote Node Check has not run on the active graph. Lint checks report this as a blocking error; other checks can report it as a warning. | Preserve the passing Local Check result, report the remote state separately, and do not claim remote validation passed while the finding is an error. |
| An upload conflicts with a field schema | Dataset fields are immutable after creation. | Use a new field name or a new dataset. Do not rewrite the live field. |
| The forecast skips a billed node | The displayed total is incomplete. | Apply the skill's labor-floor calculation and do not present the total as the price. |
| `Cannot create a new checkpoint` after a run | The live pipeline is locked after a run whose deployment did not fail. | Clone it in the platform, then import or bind it only in a separate or explicitly confirmed source root. Do not overwrite an established checkout binding. |

## What comes next

Keep the installed `toloka-build` skill. Start later sessions inside the source root so the Agent Bootstrap is loaded. Inspect the current state, run the complete local gate for source changes, show and apply an expected scoped plan, and ask before destructive, conflicting, or out-of-scope changes.

Use these references when relevant:

- `https://platform.toloka.ai/docs/integration/cli/`
- `https://platform.toloka.ai/docs/integration/sdk/`
- `https://platform.toloka.ai/docs/explanation/pricing-and-billing/`
- `https://platform.toloka.ai/docs/reference/errors/`
- `https://platform.toloka.ai/agents/fine-tuning.md`
- `https://platform.toloka.ai/agents/gisting.md`