Training Platforms & Tools

1. Unsloth: 2x Faster Fine-Tuning

Unsloth is an open-source library that achieves roughly 2x training speedup and 50% memory reduction compared to standard Hugging Face training, with zero accuracy loss. It accomplishes this through hand-written Triton kernels for attention, RoPE, cross-entropy loss, and other operations, bypassing the overhead of PyTorch's autograd in performance-critical paths.

Unsloth integrates seamlessly with the Hugging Face ecosystem: you load models through Unsloth's optimized loader, and then use standard SFTTrainer or DPOTrainer for the actual training. The output is a standard PEFT adapter that can be loaded by any tool.

from unsloth import FastLanguageModel
from trl import SFTTrainer
from transformers import TrainingArguments
from datasets import load_dataset

# 1. Load model with Unsloth (handles quantization + LoRA setup)
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B",
    max_seq_length=2048,
    dtype=None,              # Auto-detect (BF16 on Ampere+)
    load_in_4bit=True,       # QLoRA mode
)

# 2. Add LoRA adapters (Unsloth optimized)
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    lora_alpha=16,
    lora_dropout=0,          # Unsloth recommends 0 for speed
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
    use_gradient_checkpointing="unsloth",  # 30% less VRAM
)

# 3. Standard SFTTrainer workflow
dataset = load_dataset("tatsu-lab/alpaca", split="train")

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=5,
        num_train_epochs=1,
        learning_rate=2e-4,
        bf16=True,
        logging_steps=1,
        optim="adamw_8bit",
        output_dir="outputs",
    ),
)

trainer.train()

# 4. Export to various formats
model.save_pretrained("lora_model")              # Save LoRA adapter
model.save_pretrained_merged("merged_model",      # Merged FP16
    tokenizer, save_method="merged_16bit")
model.save_pretrained_gguf("gguf_model",          # GGUF for llama.cpp
    tokenizer, quantization_method="q4_k_m")

2. Axolotl: Configuration-Driven Training

Axolotl takes a different approach: instead of writing Python code, you define your entire training run in a YAML configuration file. This makes experiments reproducible, shareable, and easy to iterate on. Axolotl supports all major model architectures, PEFT methods, dataset formats, and training features (DeepSpeed, FSDP, multi-GPU) through configuration alone.

# axolotl_config.yml
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

# Dataset configuration
datasets:
  - path: tatsu-lab/alpaca
    type: alpaca
  - path: ./my_custom_data.jsonl
    type: sharegpt

# QLoRA configuration
load_in_4bit: true
adapter: qlora
lora_r: 32
lora_alpha: 64
lora_dropout: 0.05
lora_target_modules:
  - q_proj
  - k_proj
  - v_proj
  - o_proj
  - gate_proj
  - up_proj
  - down_proj
lora_target_linear: true

# Training parameters
sequence_len: 4096
sample_packing: true          # Pack multiple samples per sequence
pad_to_sequence_len: true
micro_batch_size: 2
gradient_accumulation_steps: 4
num_epochs: 3
learning_rate: 0.0002
lr_scheduler: cosine
warmup_ratio: 0.05
optimizer: paged_adamw_8bit
bf16: auto
gradient_checkpointing: true
flash_attention: true

# Evaluation and logging
val_set_size: 0.05
eval_steps: 100
logging_steps: 10
save_strategy: steps
save_steps: 200
output_dir: ./outputs/llama3-qlora

3. LLaMA-Factory: Web UI for Fine-Tuning

LLaMA-Factory provides a graphical web interface (LLaMA Board) for configuring and launching fine-tuning runs. It is particularly valuable for teams where not everyone is comfortable writing YAML or Python configurations. The web UI lets you select models, datasets, PEFT methods, and hyperparameters through dropdown menus and sliders, then generates and executes the corresponding training code.

# Install LLaMA-Factory
# pip install llamafactory

# Launch the web UI
# llamafactory-cli webui

# Or use CLI for scriptable workflows
import json

# LLaMA-Factory uses a JSON config (similar to Axolotl YAML)
config = {
    "model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct",
    "stage": "sft",
    "finetuning_type": "lora",
    "lora_rank": 16,
    "lora_alpha": 32,
    "lora_target": "all",
    "dataset": "alpaca_en",
    "template": "llama3",
    "quantization_bit": 4,
    "per_device_train_batch_size": 4,
    "gradient_accumulation_steps": 4,
    "num_train_epochs": 3.0,
    "learning_rate": 2e-4,
    "output_dir": "./llama_factory_output",
}

# Save and run via CLI
with open("train_config.json", "w") as f:
    json.dump(config, f, indent=2)

# llamafactory-cli train train_config.json

4. torchtune: PyTorch-Native Fine-Tuning

torchtune is PyTorch's official library for fine-tuning LLMs. Its philosophy is transparency and composability: rather than hiding complexity behind abstractions, it provides well-documented, hackable recipes that you can read, understand, and modify. Each recipe is a self-contained Python script, not a framework that manages your training loop.

torchtune is the best choice when you need full control over the training process, want to implement custom training logic, or are integrating fine-tuning into an existing PyTorch codebase.

# torchtune uses YAML configs and CLI recipes
# Install: pip install torchtune

# Download a model
# tune download meta-llama/Meta-Llama-3.1-8B-Instruct \
#   --output-dir ./models/llama3-8b

# Run a built-in recipe (LoRA single GPU)
# tune run lora_finetune_single_device \
#   --config llama3_1/8B_lora_single_device

# Custom config override
# tune run lora_finetune_single_device \
#   --config llama3_1/8B_lora_single_device \
#   batch_size=4 \
#   epochs=3 \
#   lora_rank=32

# torchtune YAML config example:
# model:
#   _component_: torchtune.models.llama3_1.lora_llama3_1_8b
#   lora_attn_modules: ['q_proj', 'v_proj', 'k_proj', 'output_proj']
#   apply_lora_to_mlp: True
#   lora_rank: 16
#   lora_alpha: 32

# torchtune is also great for programmatic use:
from torchtune.models.llama3_1 import lora_llama3_1_8b
from torchtune.modules.peft import get_adapter_params

# Build LoRA model with full control
model = lora_llama3_1_8b(
    lora_attn_modules=["q_proj", "v_proj"],
    apply_lora_to_mlp=True,
    lora_rank=16,
    lora_alpha=32,
)

# Get only adapter parameters for the optimizer
adapter_params = get_adapter_params(model)
optimizer = torch.optim.AdamW(adapter_params, lr=2e-4)

5. TRL: Transformer Reinforcement Learning

TRL (Transformer Reinforcement Learning) from Hugging Face is the standard library for alignment training, including SFT, RLHF, DPO, and other preference optimization methods. While its scope extends beyond PEFT, TRL integrates deeply with the PEFT library, making it the natural choice when your fine-tuning involves alignment stages.

from trl import SFTTrainer, SFTConfig, DPOTrainer, DPOConfig
from peft import LoraConfig

# SFT with LoRA (most common PEFT + TRL pattern)
sft_config = SFTConfig(
    output_dir="./sft_output",
    max_seq_length=2048,
    per_device_train_batch_size=4,
    num_train_epochs=3,
    learning_rate=2e-4,
    packing=True,            # Sample packing for efficiency
)

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules="all-linear",
    task_type="CAUSAL_LM",
)

trainer = SFTTrainer(
    model="meta-llama/Meta-Llama-3.1-8B",
    args=sft_config,
    train_dataset=dataset,
    peft_config=peft_config,   # TRL handles PEFT setup automatically
)
trainer.train()

# DPO with LoRA (preference optimization after SFT)
dpo_config = DPOConfig(
    output_dir="./dpo_output",
    per_device_train_batch_size=2,
    num_train_epochs=1,
    learning_rate=5e-5,
    beta=0.1,               # DPO temperature
)

dpo_trainer = DPOTrainer(
    model=sft_model,          # Start from SFT checkpoint
    args=dpo_config,
    train_dataset=preference_dataset,
    peft_config=peft_config,
)
dpo_trainer.train()

6. Tool Comparison Matrix

7. Cloud Compute Options

Choosing the right GPU infrastructure depends on your budget, scale, and workflow preferences. Here is a comparison of the major options available for LLM fine-tuning.

8. Recommended Workflows

Here are recommended end-to-end workflows depending on your experience level and requirements.

Beginner: First Fine-Tune

1. Use Google Colab with a free T4 GPU
2. Install Unsloth for optimized training
3. Fine-tune a 7B model with QLoRA (r=16)
4. Export to GGUF and test with Ollama locally

Intermediate: Production Fine-Tune

1. Use Axolotl for reproducible YAML-based configuration
2. Train on RunPod or Lambda Labs with an A100
3. Run evaluation suite before and after training
4. Merge adapter and deploy via vLLM

Advanced: Multi-Stage Alignment

1. SFT with TRL + LoRA on instruction data
2. DPO with TRL + LoRA on preference pairs
3. Merge both adapters sequentially
4. Evaluate with custom benchmarks and human evaluation
5. Deploy with vLLM or serve adapters via LoRAX