Compress a prompt with gisting

Learn a compact prompt representation from examples plus a prompt text file. A Training is an editable draft. A Training Run is one billed execution. The first run creates the output Model from the Training name and publishes an immutable Model Version.

Gisting is a different method from supervised fine-tuning. Always pass --method gisting and --prompt-file. For weight adaptation from labeled conversations, see Fine-tune a model.

Note

Try it with Claude:

claude -p "Let's use gisting. Follow https://platform.toloka.ai/agents/gisting.md"

Copy IDs from command output into the next command. Do not invent them.

Before you start

You need:

• A supported standalone CLI system: macOS or glibc Linux on ARM64 or x64. Native Windows and musl Linux releases are not available yet. The CLI includes its runtime; see CLI installation.
• An API key from the Business Console: Company → API keys → Generate API key. See Manage your team and balance.
• A team with a funded balance. Cost estimation is free. Starting a Training Run charges up to the printed maximum charge.
• Example conversations and a non-empty UTF-8 prompt file no larger than 256 KiB.

Step 1 — Install and sign in

curl -fsSL https://platform.toloka.ai/cli/install.sh | sh
toloka version --json
toloka login
toloka whoami

Training requires stable CLI 0.2.0 or newer. This CLI-only workflow does not import the SDK and needs no separate runtime. For version pins, updates, PATH fixes, and migration, see Updates and migration from Bun global installs. There is no toloka update command; rerun the installer for an immediate update.

login prompts for the API key without echoing it. whoami prints the platform URL, a masked key, and Reachable: N project(s) visible.

Step 2 — Create or select a project

The Training, dataset, prompt file, billing, and resulting Model belong to one project.

toloka project list
toloka project create gisting --name "Support ticket gisting"
toloka project current

project create prints Created project pp.… as gisting. If you have more than one team, pass --team-id. To reuse an existing project:

toloka project add support-gisting pp.YOUR_PROJECT_ID
toloka project switch support-gisting

Step 3 — Put example conversations in a dataset

Each row is a chat. The last turn must be assistant, and at least one turn must be user. Save this file as support-tickets.jsonl:

{"messages":[{"role":"user","content":"Hi, how do I reset my password? I can't find the option anywhere."},{"role":"assistant","content":"question"}]}
{"messages":[{"role":"user","content":"This is the third time my order arrived damaged. I'm furious."},{"role":"assistant","content":"complaint"}]}
{"messages":[{"role":"user","content":"Just wanted to say your support team was incredibly helpful today, thank you!"},{"role":"assistant","content":"praise"}]}
{"messages":[{"role":"user","content":"What are your business hours on weekends?"},{"role":"assistant","content":"question"}]}
{"messages":[{"role":"user","content":"The app keeps crashing every time I open the checkout page. Please fix this."},{"role":"assistant","content":"complaint"}]}
{"messages":[{"role":"user","content":"Amazing product, it has saved me hours every week. Great job!"},{"role":"assistant","content":"praise"}]}
{"messages":[{"role":"user","content":"Can I change the shipping address after placing an order?"},{"role":"assistant","content":"question"}]}
{"messages":[{"role":"user","content":"I was charged twice for the same subscription and nobody is responding."},{"role":"assistant","content":"complaint"}]}
{"messages":[{"role":"user","content":"Do you offer student discounts?"},{"role":"assistant","content":"question"}]}
{"messages":[{"role":"user","content":"The new dashboard update is fantastic, much easier to use now."},{"role":"assistant","content":"praise"}]}

Do not split the file yourself. With one dataset and no --validation-dataset, the run holds out 10% of the rows automatically.

Create the dataset, then append the rows:

toloka dataset create support-tickets

Then:

toloka dataset upload support-tickets ./support-tickets.jsonl
# Uploaded 10 rows to dataset support-tickets.
toloka dataset list
# support-tickets    ds.…

Copy the dataset id.

Step 4 — Write the prompt file

Save this as prompt.txt. The file must be non-empty UTF-8 and at most 256 KiB.

Classify each support ticket as question, complaint, or praise.
Reply with one of those three labels and nothing else.

Step 5 — Choose a base model

toloka base-model list

The catalog currently includes qwen3-4b, qwen3-8b, qwen3-32b, and qwen3-235b. For gisting, choose qwen3-4b or qwen3-8b: gisting publishes the full model, the larger tiers exceed the Model Version size limits, and creating a gisting Training with them is rejected. This page uses qwen3-8b. The printed ID is what --base-model accepts.

Step 6 — Create the gisting Training draft

Pass --method gisting and --prompt-file. Replace ds.… with the id from toloka dataset list:

toloka training create "Support ticket gisting" \
  --method gisting \
  --prompt-file ./prompt.txt \
  --base-model qwen3-8b \
  --dataset ds.YOUR_DATASET_ID
# Created training trg.…

Confirm the draft:

toloka training get trg.YOUR_TRAINING_ID

The table shows Method as Prompt gisting and Output Model as - until the first run publishes one.

--gist-tokens, --epochs 1..5, --learning-rate, --batch-size 8|16|32|64, and --max-sequence-length 64..16384 set the gisting hyperparameters. Omit them to take the defaults: 32 tokens, 3 epochs, learning rate 0.03, batch size 8, and 3,072 tokens.

--gist-tokens takes an absolute count — 8, 32, or 64 — or a fraction of the prompt you are compressing: 1/4, 1/8, 1/16, or 1/32. A fraction fixes the compression ratio rather than the token count: the run divides the prompt’s token length by that number, rounds up, and clamps the result to 8 through 256. Use a fraction when the prompt is long or its length changes between drafts. training get reports the fraction you set; the count it resolved to is chosen inside the run. See Base models and settings.

Step 7 — Review the cost

toloka training cost trg.YOUR_TRAINING_ID

The command prints a table with training rows, tokens, maximum GPU seconds, and Maximum charge. If the estimate is blocked, the command exits with code 1 and prints Training cannot run with these inputs: followed by one - <message> line per problem. Fix each and calculate again. With --json, a blocked estimate writes nothing to stdout.

Caution

Starting the Training Run charges up to the printed maximum charge. Do not run the next command until you accept that amount.

Step 8 — Start the Training Run

toloka training run trg.YOUR_TRAINING_ID

The command calculates a fresh ready estimate and names the Model Version after the Training (Support ticket gisting). Copy the Training Run id (trn.…).

toloka training status trn.YOUR_RUN_ID
toloka training logs trn.YOUR_RUN_ID

Status values include queued, running, stopping, blocked, completed, failed, and canceled. If the run is blocked, restore billing and run toloka training resume trn.YOUR_RUN_ID.

A run can train longer than the estimate predicted. It stays running, the charge stays capped at the maximum charge you approved, and the Model Version is published when training finishes. Do not start a second run because the first one is slower than estimated. A run that fails because of the platform is not charged: the whole hold returns to the team balance, even though training status still shows the GPU seconds it used.

Step 9 — Use the trained model

When status is completed, the table shows the Model Version. The Training now names its output Model:

toloka training get trg.YOUR_TRAINING_ID
toloka model versions mdl.YOUR_MODEL_ID

Download the files. Copy the Model Version id (mdv.…) from the table:

toloka model version download mdv.YOUR_VERSION_ID --out ./support-ticket-gisting
# Downloaded Model Version mdv.… revision 1 to /…/support-ticket-gisting

The folder holds a full model: config.json, the model-*.safetensors weight shards, and the tokenizer files, including the text files merges.txt and chat_template.jinja. The learned prompt is baked in as <|gist_…|> tokens. Serve the folder as-is, for example with vLLM, and start each request with the gist tokens in place of the original prompt.

You trained the model. To test it, deploy it. You can use our partner, Nebius.

If something fails

What you see	What it means	What to do
Missing API key. Run toloka login once to store credentials for this machine:	No key is in effect.	Run toloka login, or set TOLOKA_API_KEY.
No project selected. Run toloka project create for a new project, toloka project add for an existing project, or cd into a checkout.	No project is selected.	toloka project create, toloka project add, or pass --project-id.
Dataset "…" does not exist … no pipeline in source declares it.	The dataset has to exist before rows can be uploaded.	Run toloka dataset create <name>, then upload again.
The gisting prompt file is empty. / The gisting prompt file exceeds the 256 KiB limit. / The gisting prompt file must contain valid UTF-8 text.	--prompt-file failed a local check.	Point at a non-empty UTF-8 file no larger than 256 KiB.
--prompt-file is required for gisting.	--method gisting was set without a prompt file.	Pass --prompt-file.
--lora-rank applies to SFT; gisting takes --gist-tokens.	--lora-rank was passed on a gisting Training.	Use --gist-tokens instead.
Choose exactly one of --base-model and --model-version.	Both source flags were set, or neither was.	Pass one.
Training from a Model Version requires a LoRA adapter source	--model-version points at full weights. Gisting continues a LoRA adapter version (one produced by fine-tuning).	Pass a fine-tuned Model Version, or --base-model.
Training cannot run with these inputs:	The cost estimate is blocked. The command exits with code 1.	Read each - <message>, fix inputs, calculate again.
This Training would take up to … hours … which exceeds the 6-hour limit.	The estimated wall time is over the 6-hour limit.	Reduce rows or epochs. Gisting supports only qwen3-4b and qwen3-8b.
A Model Version with this name already exists	A previous run already used this Training name as the version name.	Pass --version-name with a new name.
status is blocked	Funds or payment stopped the run.	Restore billing, then toloka training resume.

Related

• Training — how Trainings, cost estimates, Training Runs, and Model Versions fit together
• CLI — install, sign in, and project selection
• Fine-tune a model — supervised fine-tuning on the same Training commands
• Create and manage datasets — creating a dataset on the Datasets page
• Pricing and billing — how the team balance is drawn down