4.5.3, on two backends. Eight methods on the Hugging Face Trainer
backend: sft, and dpo / kto / cpo / orpo / rm / grpo / gkd behind swift rlhf.
Three more on Megatron: megatron-sft, megatron-grpo, megatron-gkd. Configuration is flat
command-line flags — one level, no dotted paths — and the adapter launches the training entry
point under torchrun.
What the flags bind to
A validation file is optional here, unlike verl and TRL. Set
split_dataset_ratio instead and
ms-swift carves one out of the training set; passing --validation-data gives it an explicit set
and the ratio is ignored.
Training data is a local jsonl / json / csv / parquet file, read through datasets, so the
extension decides the loader. The columns follow ms-swift’s own schema (messages, or
query/response) — the platform does not remap them.
LoRA is a parameter, not a separate method
tuner_type picks the trainer on any of the eight methods: full trains every parameter, lora
trains adapters only. The learning rate follows it — LoRA wants roughly 10–100× the full-parameter
rate, and ms-swift’s own defaults are 1e-5 and 1e-4.
Nothing else changes, including export. The platform reads what the run actually wrote: a
full-parameter checkpoint is registered as it stands, an adapter directory is merged back into the
base model with swift export --merge_lora. There is no -lora method to pick and no format to
get wrong.
The experiment config is an escape hatch
config.yaml in the experiment carries ms-swift flags the method does not expose as a parameter,
written flat:
model, dataset,
val_dataset, output_dir, logging_dir, add_version, report_to), is refused rather than
ignored — set the parameter, so the console shows the value that actually runs.
One GRPO trainer, several published methods
importance_sampling_level=sequence is GSPO. epsilon_high above epsilon is DAPO’s
clip-higher. advantage_estimator reaches RLOO and REINFORCE++. scale_rewards=none is
Dr. GRPO’s unbiased form. None of them needs a different method.
The same holds across the preference family: ms-swift/cpo with loss_type=simpo is SimPO, and
neither CPO nor ORPO loads a reference model at all — roughly half the memory DPO needs at the same
model size.
The rollout engine is vLLM on the training cards (vllm_mode=colocate), so
vllm_gpu_memory_utilization shares each card with the training state and is the first dial to
drop on OOM — before batch size.
The Megatron backend
Dockerfile.msswift-megatron) and a separate
setting (TUNEPLANE_IMAGE_MS_SWIFT_MEGATRON), because Megatron-core and TransformerEngine are not
in the other one. If the model fits on the Hugging Face backend, use that instead: same model
coverage, simpler stack.
megatron-grpo and megatron-gkd go through ms-swift’s own Ray pipeline, which is the one place
ms-swift uses Ray. The platform compiles the placement — how many cards each worker group gets,
across how many machines, and which groups share cards — from the profile and the pool mapping.
There is nothing to write in config.yaml about cards, and an attempt to is refused. Map the
rollout (or teacher) role to its own resource pool to give it its own cards; mapped to the same
pool, the groups are declared colocated.
Megatron-core’s own flags (save_interval, eval_interval, log_interval) belong in the
experiment’s config.yaml rather than being hyperparameters — they are upstream Megatron’s
surface, not ms-swift’s, so the catalogue does not declare them as if it owned them.
Check it worked
tp job logs shows the dataset preprocessing, then the first optimizer step. The Charts tab fills
with train/loss and train/token_accuracy (for GRPO, train/reward and
train/reward_zero_std_frac). Watch that last one: every completion in a group scoring the same
gives a zero advantage and therefore no gradient, while the loss curve still looks alive.
Validation samples are not reported for ms-swift runs — it encodes its datasets before the trainer
sees them, so the shared sample observer has no text to take. The metric series are unaffected.