Training

Training trains a model on datasets in your project. You start from a catalog Base model or from one of your own LoRA adapter Model Versions. Training has two methods: supervised fine-tuning (SFT) and gisting. Both can continue from a fine-tuned LoRA adapter Model Version; gisting is limited to the qwen3-4b and qwen3-8b tiers because it publishes a full model.

Both methods follow the same path. You create an editable Training draft. A free cost estimate validates the inputs. A billed Training Run publishes a Model Version when it completes.

The inputs are a project, a model source, and a dataset. Gisting also needs a prompt text file. The output is a Model Version that you download and deploy yourself. The platform does not host or serve it.

You work with Training from the CLI, an agent runbook, or the HTTP API.

Two methods

Method	What it reads	What it produces
Supervised fine-tuning (SFT)	Conversations in a dataset	A LoRA adapter that runs on top of the Base model
Gisting	Conversations in a dataset plus a prompt text file	A full model

In the CLI, SFT is the default method.

A Model Version contains .json and .safetensors files, plus for gisting the tokenizer text files merges.txt and chat_template.jinja. An SFT Model Version includes adapter_model.safetensors and adapter_config.json. A gisting Model Version is a full model: config.json, the model-*.safetensors weight shards, and the tokenizer files. Load and serve the downloaded folder as-is.

From draft to Model Version

A Training is a project-owned, editable draft with a name of 1–120 characters. The name must be unique among the project’s Trainings that are not archived. You archive or rename a Training through the HTTP API. The draft holds the method, source, dataset selections, and settings. Editing it does not change existing Training Runs.

A cost estimate is free. It freezes the dataset selection at a cutoff time and validates every selected row. The estimate is either blocked, with a list of problems to fix, or ready, with a maximum charge.

• A ready estimate expires 24 hours after its cutoff time.
• At most 3 estimates per Training can be open at once. An estimate is open until it starts a Training Run or expires.
• One ready estimate starts one Training Run.
• If you edit the Training, an estimate you saved earlier is no longer accepted. Calculate it again.

A Training Run is one immutable, billed execution of a Training. The run copies the Training configuration when it is created. Starting a Training Run places a hold for the maximum charge on the team balance. The maximum charge is the price of the estimate’s maximum GPU seconds: its maximum wall time multiplied by the GPUs the Base model uses. A run that trains longer than that wall time is still charged no more than the maximum charge and still publishes its Model Version. Wall time is clock time.

A completed Training Run publishes one immutable Model Version. Its name is unique within its Model. Its revision number is 1, 2, 3, and so on.

The first run creates the Model named after the Training. Every Model Version in that Model shares one Base model. In the CLI, the default version name is the Training name. If an earlier run with that name completed or is still in progress, pass --version-name.

toloka model version download <version> --out <directory> downloads the files and checks each file’s size and SHA-256.

Training Run states

toloka training status <training-run-id> shows which state a run is in. toloka training logs <training-run-id> shows its logs, and toloka training metrics <training-run-id> shows the loss and accuracy the run has reported so far.

A run normally goes queued → running → completed. The other states are stops or holds.

Metrics follow the training itself, not the run’s final state. Values keep arriving while the model trains, and stop as soon as the platform observes that training has ended — which is before the run reaches completed, since checking the output, billing, and publishing still follow. From that moment the numbers are final; the run’s own state keeps moving for a while longer.

State	What is happening	What you can do
queued	The platform prepares the selected data. For gisting, it copies the prompt file. It places a hold for the maximum charge on the team balance.	You can cancel. Nothing is charged.
running	The model trains and is evaluated. The platform then checks the output files. It charges for the GPU seconds used and publishes the Model Version.	You can cancel until the platform starts checking the output files. After that, cancel is refused and the run finishes on its own. Canceling during training charges the GPU seconds used so far.
blocked, reason funds	The team balance cannot cover the hold for the maximum charge.	Top up the team balance and resume, or cancel.
blocked, reason payment	The platform cannot collect the final charge.	Top up the team balance and resume before the payment window ends. The default payment window is 7 days. Cancel is unavailable. If the window ends, the run fails and nothing is charged.
blocked, reason billing_review	Toloka staff must resolve the billing review.	Resume and cancel are unavailable until they do.
stopping	The Training Run is stopping after a cancellation or a failure. A canceled run is charged for the GPU seconds used; a run that failed on the platform is not charged.	No control is available.
completed	The Model Version is published.	You can inspect and download the Model Version.
failed	The Training Run ended with an error.	Resume is unavailable.
canceled	The Training Run was canceled.	Resume is unavailable.

A run fails when:

• Its data cannot be exported.
• The prepared data no longer matches the estimate.
• Billing is rejected.
• Training fails.
• The trained files fail the platform’s checks.
• The prompt file was changed after upload.
• The payment window ends.

Data

A dataset row is one chat example. This row teaches the model to label a support ticket:

{
  "messages": [
    {
      "role": "user",
      "content": "Hi, how do I reset my password? I can't find the option anywhere."
    },
    { "role": "assistant", "content": "question" }
  ]
}

Each selected row must follow these rules:

• The row has messages and can also have tools.
• Allowed roles are system, user, assistant, and tool.
• At least one turn has the user role.
• The last turn has the assistant role and contains content.
• System turns appear only before the conversation starts.
• Consecutive turns cannot have the same role. Tool and system turns are exempt from this rule.
• Every system, user, and tool turn has content. An assistant turn has content or tool calls.
• Each tool turn answers a preceding assistant tool call.
• Every assistant tool call is answered by a later tool turn.

A dataset can come from an uploaded file whose rows have messages. A dataset can also keep the conversation in a messages column and the expected answer in a label column. The platform adds the label as the final assistant turn.

The CLI appends rows to an existing dataset. It cannot create a new dataset unless a pipeline in this checkout already declares it. You can create a dataset from the Datasets section of the project sidebar and rename it on the dataset’s page, or do both through the agent. The HTTP API can also create one. See Create and manage datasets.

Each method uses one training dataset selection. A validation dataset selection is optional. In the CLI, a selection is one dataset plus an optional field=value filter. The HTTP API accepts more filter shapes.

The two selections must not overlap. Use two datasets, or two filters on one dataset that select different rows.

Without a validation selection, the run holds out 10% of the training rows for evaluation.

The cost estimate is blocked when:

• A selected row breaks the rules above.
• An example is longer than the Training’s maxSequenceLength, which is at most 16,384 tokens.
• The training selection leaves fewer rows than batchSize after the 10% holdout.
• A selection is empty or over the row limit of 100,000 valid rows.
• The training and validation selections overlap.
• A dataset export fails, or the platform cannot count the tokens exactly.
• The run would exceed the 6-hour limit. The message states the hours the run would take and the GPUs it would use. On a 1-GPU Base model, switch to an 8-GPU Base model (qwen3-32b or qwen3-235b) or reduce rows or epochs. On an 8-GPU Base model, reduce rows or epochs.

Fix the inputs and calculate again. In the CLI, a blocked estimate makes toloka training cost exit with code 1 and print the problems to stderr.

Base models and settings

A Training source is either a catalog Base model or an existing LoRA adapter Model Version in the same project (one produced by fine-tuning). Both methods accept a LoRA adapter source: SFT continues training the adapter, and gisting merges it into the base and trains the gist tokens over the fine-tuned weights. A full-weights version is rejected with “Training from a Model Version requires a LoRA adapter source”. After a Training has an output Model, a replacement source must use the same Base model.

ID	Name	Provider	GPUs used per Training Run	Pinned Hugging Face snapshot
qwen3-4b	Qwen3 4B	Qwen	1	Qwen/Qwen3-4B-Instruct-2507 @ cdbee75f
qwen3-8b	Qwen3 8B	Qwen	1	Qwen/Qwen3-8B @ b968826d
qwen3-32b	Qwen3 32B	Qwen	8	Qwen/Qwen3-32B @ 9216db57
qwen3-235b	Qwen3 235B	Qwen	8	Qwen/Qwen3-235B-A22B-Instruct-2507 @ ac9c66cc

Every Training Run trains on that exact snapshot. A published Model Version records its Base model ID in content.runtimeBase; the catalog entry for that ID carries the repository, the pinned revision, and the snapshot link. It also records what it was derived from in content.parent: the catalog Base model for a from-scratch run, or the source Model Version for a warm start. Both the SFT adapter output and the gisting full-model output record this lineage. For a LoRA adapter, the downloaded adapter_config.json records the same revision, and a run whose adapter reports a different revision fails before it publishes a Model Version. To load the same base model yourself, pass the revision: AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", revision="b968826d9c46dd6066d109eabc6255188de91218"), or vllm serve Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218. toloka base-model list shows the revision and toloka base-model get <id> prints the link.

The estimate spreads the work over the GPUs. On an 8-GPU Base model about 8 times more tokens fit inside the 6-hour limit, and the maximum charge for the same tokens is about the same as on 1 GPU.

Setting	Rule
LoRA rank (loraRank)	SFT only. The rank of the LoRA adapter: 8, 16, or 32. The default is 16. Alpha is always twice the rank. A larger rank gives the adapter more capacity and needs more data to avoid overfitting. Serving the adapter needs --max-lora-rank at or above this value. The rank does not change the maximum charge.
Gist tokens (gistTokens)	Gisting only. The number of learned soft tokens that replace the prompt: an absolute 8, 32, or 64, or a fraction of the prompt — "1/4", "1/8", "1/16", or "1/32". The default is 32. Fewer tokens compress harder and serve cheaper; more tokens keep more of the prompt. A fraction fixes the compression ratio instead of the count: the run rounds the prompt’s token length divided by that number up to the next whole token and clamps it to 8 through 256, so the same setting suits prompts of different lengths. Neither form changes the maximum charge.
Epochs (epochs)	For SFT and gisting, an integer from 1 through 5. The default is 3. The maximum charge grows linearly with epochs.
Learning rate (learningRate)	Optimizer step size. SFT accepts 0.00001 through 0.001 and defaults to 0.00015. Gisting accepts 0.001 through 0.3 and defaults to 0.03; it trains only the soft tokens, which is why its rate is about 200 times higher. The learning rate does not change the maximum charge.
Batch size (batchSize)	Sequences per optimizer step across all GPUs of the base model: 8, 16, 32, or 64. The default is 8. The training selection must leave at least that many rows after the 10% holdout, or the cost estimate is blocked. The batch size does not change the maximum charge.
Max sequence length (maxSequenceLength)	Token budget per training example, prompt and target together: 64 through 16,384. The default is 12,288 for SFT and 3,072 for gisting. The cost estimate is blocked when the longest example exceeds it, so nothing is truncated silently.

The rest of the recipe is fixed by the platform: a cosine schedule with warmup, LoRA alpha at twice the rank, LoRA dropout 0.05, adapters on every linear projection (attention-only on the 235B model), and bf16 precision.

Limits

Limit	Value
Training name	1–120 characters, unique among the project’s Trainings that are not archived
Valid rows per selection	1–100,000
Tokens per example	At most the Training’s maxSequenceLength, up to 16,384, at about 4 characters per token
Gisting prompt file	Not empty (whitespace only counts as empty), UTF-8, and at most 256 KiB
Open cost estimates	3 per Training; each expires 24 hours after its cutoff time
Model Version	At most 64 files, 10 GiB per file, and 50 GiB total
Download links	Valid for 30 minutes
Training time	The maximum charge covers the estimate’s maximum wall time (6 hours or less); a longer run pays no more

Cost

You pay only for a Training Run. Maximum GPU seconds are the maximum wall time multiplied by the GPUs the Base model uses. The maximum charge is the price of the maximum GPU seconds, rounded up to the cent.

The maximum charge grows with the number of training tokens, the number of epochs, and the size of the validation selection. Gisting counts each training token twice.

Caution

Starting a Training Run places a hold for the maximum charge on the team balance. Review the exact maximum charge before starting the Training Run.

When the Training Run ends, the platform charges for the GPU seconds it used. The charge is capped at the maximum charge. The platform returns the rest of the hold to the team balance. This applies to runs that complete or that you cancel during training. A run that fails because of the platform, including the training provider, is not charged: the whole hold is returned.

Nothing is charged when:

• The run stops before training starts.
• The trained files fail the platform’s checks.
• The payment window ends.

See Manage your team and balance to top up.

Ways to work with Training

• CLI — create and update Trainings, calculate cost estimates, control Training Runs, inspect logs, and download Model Versions
• Fine-tuning agent runbook — guide a coding agent through an SFT Training Run
• Gisting agent runbook — guide a coding agent through a gisting Training Run
• HTTP API — how to authenticate and call the API; the Training, Training Run, Base model, and Model endpoints are listed in the API reference

For Training, you use the web app to create and rename datasets and to open the balance and its top-up link. It has no pages for Trainings, Training Runs, Models, or Model Versions.

Note

Your sidebar may show ML services (Preview) with Fine-tuning and Prompt gisting. Those two pages are a separate tool that belongs to your team, not to a project. They do not create Trainings, Models, or Model Versions. This page does not cover them.

Related

• Train your first model — a guided first Training Run on a 10-row demo dataset
• Fine-tune a model — the SFT commands step by step
• Compress a prompt with gisting — the gisting commands step by step
• Create and manage datasets — creating and renaming a dataset in the web app
• Pricing and billing — how pipeline runs are priced and how the team balance is drawn down
• Security and data handling — who in your team can see project data