# Compress a prompt with gisting This runbook guides an AI coding agent through prompt gisting with the Toloka CLI. It applies to both new and existing workflows. ## Mission Help the user learn a compact representation of their static prompt from representative examples while preserving the exact prompt and their model, data, quality, latency, deployment, and budget constraints. The requested outcome may be to inspect existing state, prepare or update a Training draft, calculate an estimate, start an approved run, monitor a run, or retrieve a completed Model Version. Do not create later-stage resources merely to complete the whole lifecycle. For model weight adaptation from labeled conversations, explain the difference and route to `https://platform.toloka.ai/agents/fine-tuning.md`. For labeled-data creation, route to `https://platform.toloka.ai/agents/data-labeling.md`. Never silently change the method. ## How to work 1. Read the request and current CLI, project, Training, Dataset, prompt, and run state before writing. 2. Treat the sections below as a dependency-aware reference, not a script. Skip steps already satisfied or irrelevant to the requested outcome, while preserving dependencies and safety gates. 3. Honor explicit choices first. If a requested project, source, Dataset, prompt, or configuration is incompatible, explain the evidence and tradeoff before proposing a substitute. 4. Make routine reversible choices and briefly explain them. Ask focused questions only for material ambiguity, missing authorization or inputs, conflicting constraints, or destructive or out-of-scope changes. 5. An inspection or estimate-only request ends after its answer. Start a Training Run only when the user requested a run and approved its exact ready estimate. ## Safety and platform invariants - Never ask the user to paste an API key into chat. The user enters it through `toloka login`. - Never invent a project ID, Dataset ID, Base Model ID, Model ID, Model Version ID, filter, prompt, price, or task requirement. Read them from CLI output. - Never edit `toloka.json`; the CLI owns that file. - Use project-owned Datasets containing representative, authorized examples. Keep credentials and production data out of source control and agent messages. - Preserve user-provided prompt text exactly unless the user asks to edit it. The prompt file must be non-empty UTF-8 text no larger than 256 KiB. - Preserve training and validation semantics. Keep validation examples out of training and avoid duplicate or near-duplicate examples across the roles. - Without a validation selection, the provider reserves 10% of the training rows as a holdout. - Never start a Training Run before the user sees the ready estimate and explicitly approves its exact `maximumCharge`. - A Training is an editable draft. Each Training Run is an immutable, billable execution; changing the draft does not mutate an existing run. - Do not pass `--new-model`. The first run creates the output Model from the Training name. - Gisting starts from a catalog Base Model, or from a fine-tuned model's LoRA adapter Model Version via `--model-version` (to compress a prompt for that fine-tuned model). A `--model-version` source must be a LoRA adapter version; gisting is limited to the qwen3-4b and qwen3-8b tiers because it publishes a full model. - Use absolute paths for prompt and estimate files. Commands can run in separate shells. - Keep waits bounded. A timeout means the run is still running, not failed or completed. - Report only IDs, prices, states, and behavior observed in command output. A completed run does not prove deployment or inference readiness. ## Domain model and full lifecycle - **Prompt gisting** learns soft prompt tokens that stand in for a static instruction prompt. Training conversations should contain dynamic inputs and desired assistant responses, not a repeated copy of the static prompt. - A **source** is either a pinned catalog Base Model or a project-owned immutable Model Version. An explicit Model Version bypasses Base Model selection. - A **Dataset selection** is a Dataset plus an optional top-level equality filter. Training and validation may use separate Datasets or disjoint selections from one Dataset. - The **prompt file** is the exact static instruction text being compressed. It is a required Training input, distinct from the examples. - Cost estimation validates frozen Dataset selections, the prompt, and the selected source before GPU work. - A successful paid run publishes one immutable Model Version. Serving or deployment is a separate decision. A full create-and-run request has these dependencies: ```text working CLI and identity | v appropriate project binding | v exact prompt + compatible source + training/validation selections | v gisting Training draft --> ready estimate --> approve maximumCharge | v one Training Run | v observed Model Version ``` ## Prepare the environment when needed Reuse a compatible existing workspace. Do not force a fresh onboarding directory, reinstall dependencies, or rerun initialization when the current workflow is already prepared. When CLI state is unknown, run: ```sh toloka --version toloka dataset --help ``` The standalone CLI includes its runtime, so Training-only work needs no separate Bun or Node.js installation, package manifest, SDK, TypeScript, or test packages. This runbook requires a stable Toloka CLI release at version `0.2.0` or newer: `toloka --version` must print `0.2.0` or a newer version with no prerelease suffix, and `toloka dataset --help` must list `create`. If either command is unavailable, the version is older or prerelease, or the capability check fails, install the latest stable version: ```sh curl -fsSL https://platform.toloka.ai/cli/install.sh | sh ``` Use the install path printed by the command if the shell does not refresh its `PATH`. Only for a new workspace, use the directory supplied by the user or create a task-appropriate directory such as `toloka-gisting`. Run `toloka init` here only to create the Agent Bootstrap used to install the workflow skill; its missing-SDK warning applies to pipeline authoring, not Training commands: ```sh mkdir -p toloka-gisting && cd toloka-gisting && pwd toloka init ``` The printed directory is ``. Use absolute paths for later local files. In an initialized workspace, keep its source root and inspect installed skills instead: ```sh toloka skill list ``` Install `toloka-gisting` only when absent, or refresh it with `--force` when the installed copy is stale: ```sh toloka skill install toloka-gisting toloka skill install toloka-gisting --force ``` If the catalog does not contain the skill, update to the latest stable CLI with the installer above. Read `.agents/skills/toloka-gisting/SKILL.md` before acting. ## Establish identity and project context Check authentication without exposing the token: ```sh toloka whoami ``` If it reports the intended reachable account, continue without another login. Read failures precisely: `Rejected:` means the platform refused the key; `Not reachable:` means the platform did not answer and the key was not evaluated; `Platform error:` means the platform failed. For a missing or rejected credential, tell the user to generate an API key at `https://platform.toloka.ai` under **Company** → **API keys**, then ask them to run `toloka login` in their own terminal and enter it at the masked prompt. Continue only after `toloka whoami` reports the intended account. Fix connectivity instead of requesting a new key for `Not reachable:`. Inspect project context before selecting or creating anything: ```sh toloka project current toloka project list --limit 20 ``` Choose the project from evidence: 1. If the user explicitly named a project, use that exact target. Register it with `toloka project add ` when necessary, or use `toloka project switch ` when it is already registered. 2. Otherwise, keep the current binding when its identity and resources are consistent with the task. 3. If no suitable project is bound and a mutation requires one, create a new task-named project. Do not create a project for an inspection that can be answered from an explicitly addressed resource. Do not silently choose an unrelated project or repoint an existing checkout. If the current binding conflicts with the request and the intended target is genuinely ambiguous, ask one focused project question. When an existing checkout is correctly bound to another project, keep that binding intact. Use an explicit `--project-id ` for isolated inspection commands or a separate source root for a mutable workflow instead of overwriting `toloka.json`. For a new project, derive a short kebab-case alias and name from the task, such as `cv-parser-gisting`: ```sh toloka project create --name "" [--team-id ] [--delivery-order-id ] ``` Without `--team-id`, the CLI uses the account's only writable team. If it reports `Missing team id.`, ask the user to choose from the listed teams. A project owned by an internal team (`toloka whoami` marks it `(internal)` on the `Teams:` line) needs a Sunday delivery order: run `toloka delivery-order list` and pass the only listed order with `--delivery-order-id `; ask when several viable orders remain. Verify the selected context: ```sh toloka project current toloka project url ``` When using a bound source root, read `toloka.json` and confirm its `projectId` matches `toloka project current`. Do not edit it. ## Inspect the requested state and inputs For an existing workflow, start with the resource the user referenced: ```sh toloka training list --json toloka training get --json toloka training runs --json toloka training status --json ``` Run only the commands needed to answer the request. Do not manufacture a new Training, prompt file, or run for inspection. ### Choose training and validation data Inspect project Datasets and candidate fields: ```sh toloka dataset list toloka dataset fields ``` Use the data and role assignments the user supplied. A typical compatible row is: ```json {"messages":[{"role":"user","content":"Classify this request"},{"role":"assistant","content":"billing"}]} ``` The user message should contain only the dynamic input; the final assistant message is the desired output. Do not repeat the static prompt being compressed inside every example. Check that examples are representative, authorized, and consistent with the prompt. A separate validation selection gives a stable evaluation set; when none exists, omit validation flags and use the provider's 10% holdout rather than inventing validation data. When local JSONL files need uploading, organize Datasets by role, not automatically by file. `dataset upload` appends, so several compatible files for one role may share one Dataset: ```sh toloka dataset create toloka dataset upload toloka dataset create toloka dataset upload toloka dataset fields ``` Create and upload only the role Datasets that are needed. CLI uploads to an otherwise undeclared Dataset store each row whole. A `split` property nested inside that uploaded object is not a top-level Dataset field, so it cannot drive `--train-filter` or `--validation-filter`; use separate role Datasets or provider holdout. An existing Dataset with a real top-level split field may use disjoint filters such as `split=train` and `split=validation`. Never configure overlapping training and validation selections. ### Preserve the prompt The prompt file must be non-empty UTF-8 text no larger than 256 KiB. Use the exact file the user supplied. If the user supplied the prompt text in their message, materialize that exact text as a UTF-8 file. Do not summarize, rewrite, or complete it unless asked. Ask a focused question only when the required prompt or its boundaries are genuinely missing. ### Choose and verify the model source Gisting supports only `qwen3-4b` and `qwen3-8b` because it publishes full model weights. This includes the underlying Base Model of an existing Model Version. Unsupported tiers are rejected when creating or updating a Training draft; explain the constraint rather than silently substituting a model. Honor an explicit Base Model or Model Version. If neither is specified, choose a suitable source from evidence rather than defaulting to the smallest or largest model. Consider: - upstream documentation and user-provided benchmarks for prompt, language, and domain fit; - prompt complexity, quality target, data volume, and desired compression fidelity; - training budget and duration; - serving latency, memory, and deployment constraints; - relevant prior results from an existing project Model Version. A smaller model is useful as a cost and latency baseline when the quality target permits it, but size is not evidence of fitness by itself. Briefly state the choice, evidence, and tradeoffs. For a catalog Base Model, inspect the available identities, then verify the exact catalog ID and pinned upstream revision: ```sh toloka base-model list --json toloka base-model get --json ``` The catalog output identifies the upstream model and revision; consult upstream documentation or user evidence for context length, languages, and other capabilities it does not expose. To compress a prompt for a model the user already fine-tuned, pass that model's LoRA adapter Model Version with `--model-version` instead of `--base-model`: the adapter is merged into the base and the gist tokens are trained over the fine-tuned weights. The source must be a LoRA adapter version, and gisting is limited to the qwen3-4b and qwen3-8b tiers. ## Reuse, create, or update the Training draft If an existing Training already matches the requested method and inputs, reuse it. A new draft is appropriate only when the user requested one or no suitable draft exists. Use a descriptive name such as `cv-parser-gisting-1` unless the user supplied a name. Always identify gisting explicitly and pass the exact prompt file: ```sh toloka training create "" \ --method gisting \ --prompt-file \ --base-model \ --dataset \ --validation-dataset \ --json ``` Omit `--validation-dataset` when using provider holdout. For disjoint top-level selections, add `--train-filter =` and `--validation-filter =`. Preserve user-specified hyperparameters and choose unspecified values from prompt complexity, compression fidelity, validation evidence, and deployment constraints. The current CLI uses explicit parameters rather than `regime`: `--gist-tokens` (default 32), `--epochs 1..5` (default 3), `--learning-rate` within 0.001–0.3 (default 0.03), `--batch-size 8|16|32|64` (default 8, sequences per optimizer step across all GPUs), and `--max-sequence-length 64..16384` (default 3072 tokens per example). Choose sequence length from actual examples within the supported limit. These defaults are starting points, not mandatory choices. `--lora-rank` is SFT-only and rejected here. `training update ` takes the same flags except `--method`; omitted flags retain their current values. `--gist-tokens` takes either an absolute count `8|32|64` or a fraction of the prompt being compressed: `1/4`, `1/8`, `1/16`, or `1/32`. An absolute count fixes the soft-prompt size whatever the prompt; eight gist tokens favor compression for simple prompts, and 64 helps when 32 loses holdout accuracy. A fraction fixes the compression ratio instead: the run resolves it to `ceil(prompt tokens / N)` after it tokenizes the prompt, clamps the result to 8 through 256, and trains that many tokens. Prefer a fraction when the prompt is long or its length changes between drafts, and start at `1/32` for a long prompt, moving to `1/16` or `1/8` when the tighter ratio loses holdout accuracy. The resolved count is chosen inside the run, so `training get` reports the fraction you set, not the token count it became. After creating or updating a draft, verify its normalized method, exact source, prompt file, selections, filters, and configuration: ```sh toloka training get --json ``` Do not start a run yet. ## Validate inputs and estimate cost Create an absolute estimate-file path under ``, then run: ```sh toloka training cost --json > ``` The command freezes Dataset selections, waits for validation exports, and writes a ready estimate to the file. It is the concrete compatibility check for the selected source, gisting method, prompt, and data. If the estimate is blocked, the command exits 1, writes nothing to the file, and prints `Training cannot run with these inputs:` with one `- ` line per issue on stderr. Report every issue, correct its cause, and calculate a new estimate. Never reuse an estimate after the Training configuration changes. For a duration issue, use the reported row and token counts, epochs, GPU count, and wall time to compare supported options. Gisting publishes full model weights, so only `qwen3-4b` and `qwen3-8b` are supported; switching to the larger 8-GPU tiers is not a valid workaround. Reducing rows can weaken coverage and reducing epochs can weaken compression quality. Do not silently change the requested model, discard data, or alter epochs merely to clear the error. Explain the tradeoff, obtain direction when material, update the draft, and re-estimate. For a ready estimate, report: - Training ID, exact source, and prompt file; - training and validation selections and row counts; - token totals; - maximum GPU seconds; - `maximumCharge` and currency; - estimate fingerprint. If the user asked only for validation or an estimate, stop here. Otherwise present the approach and estimate together and ask the user to approve the exact `maximumCharge`. Any later source, prompt, data, filter, or hyperparameter change requires a new estimate and new approval. ## Start and monitor only the requested run After both a run request and explicit approval, start one run with the approved estimate: ```sh toloka training run \ --estimate-file \ --version-name \ --json ``` Both flags are optional at the CLI level. Pass `--estimate-file` here so a fresh unseen quote cannot replace the approved one. Without `--version-name`, the Model Version takes the Training name. Read the returned Training Run ID. Monitor only as far as useful for the user's request, using bounded commands: ```sh toloka training status --json toloka training logs toloka training metrics [--key ] [--resolution ] [--json] ``` Use `toloka training logs --follow` only with a bounded command timeout. Relevant states include `queued`, `running`, `stopping`, `blocked`, `completed`, `failed`, and `canceled`. Metrics report the loss and accuracy published so far; an unavailable explanation is not evidence that the run failed. If the run is blocked on funds or payment, report the exact message. Resume only after the user confirms the underlying billing issue is resolved: ```sh toloka training resume --json ``` Cancel only on explicit request: ```sh toloka training cancel --json ``` ## Verify and report the requested result For a completed run, read the Model Version ID and server-assigned revision from status output, then verify the Training's output Model: ```sh toloka training get --json toloka model get --json toloka model versions --json ``` Download the immutable Model Version only when requested: ```sh toloka model version download --out ``` Report the identifiers and evidence relevant to the user's request. For a completed lifecycle, include the Training ID, Training Run ID and terminal state, approved maximum charge and observed charged amount, output Model ID, Model Version ID and revision, whether logs were complete or truncated, and any unresolved block or timeout. The downloaded folder contains full model weights (`config.json` and `model-*.safetensors`) and tokenizer files, including supported text sidecars such as `merges.txt` and `chat_template.jinja`. Keep the files together. The learned prompt is represented by `<|gist_…|>` tokens that prefix requests in place of the original prompt; serving with a compatible runtime such as vLLM is a separate step to verify. Do not claim deployment or inference readiness. The durable Model Version is the training output; deployment is a separate workflow.