Fine-Tuning Pipelines

Managed SFT · LoRA · RLHF · Distillation — reproducible on your data

job_a7c1 — cortex-med-7b SFT round 12

8 × H100 · us-east-2 · started 4h 22m ago

Step
6,800 / 10,000
Tokens/s
42,180
Loss
0.284
Learning rate
3.2e-5
Grad norm
0.42
Perplexity
1.34
ETA
1h 12m
Cost so far
$284.10

Live logs

stdout · streaming

[14:22:11] rank=0 step=6800 loss=0.284 lr=3.2e-5
[14:22:12] rank=1 step=6800 loss=0.281 lr=3.2e-5
[14:22:12] rank=2 step=6800 loss=0.286 lr=3.2e-5
[14:22:13] rank=3 step=6800 loss=0.283 lr=3.2e-5
[14:22:14] eval: perplexity=1.34 acc=0.941
[14:22:15] checkpoint saved: s3://cortex-med/ckpt-6800
[14:22:18] tokens/s=42,180 gpu_util=94%
[14:22:20] grad_norm=0.42 clip=1.0
[14:22:21] rank=0 step=6801 loss=0.279 lr=3.2e-5
[14:22:22] rank=1 step=6801 loss=0.282 lr=3.2e-5
[14:22:23] enclave attestation refreshed ok
[14:22:25] rank=0 step=6802 loss=0.277 lr=3.2e-5
[14:22:26] rank=1 step=6802 loss=0.280 lr=3.2e-5
[14:22:28] warmup complete after 500 steps
[14:22:30] tokens/s=42,240 gpu_util=95%
autoscroll: onlines: 12,482

Job queue

All fine-tuning jobs across workspace

job_a7c1cortex-med-7b — SFT round 12
James Okafor
8 × H100· us-east-2· ETA 1h 12m· $284.10
68%Running
job_b920radiology-vit — LoRA sweep
Anna Weiss
4 × A100· eu-west-1· ETA 3h 04m· $96.80
42%Running
job_c412fraud-detector — retrain
Priya Mehta
2 × A10· us-west-2· ETA -· $0.00
0%Queued
job_d001voice-triage — RLHF
Sara Kim
16 × H100· us-east-2· ETA 18m· $1,204.22
91%Running
job_e552claims-nlp — domain adapt
Marcus Lin
4 × A100· eu-west-1· ETA -· $52.40
33%Failed
job_f118notes-summarizer — SFT
James Okafor
8 × H100· us-east-2· ETA -· $612.00
100%Completed

Recipe · SFT + LoRA

recipe: sft-lora-v3
base_model: llama-3.1-8b
method: lora
lora_rank: 16
lora_alpha: 32
target_modules: [q_proj, k_proj, v_proj, o_proj]
optimizer: adamw
learning_rate: 3.2e-5
warmup_steps: 500
batch_size: 32
gradient_accumulation: 4
max_steps: 10000
eval_every: 200
save_every: 500
enclave: intel_tdx
dataset: readmit-notes-2025q1@v14

Hyperparameter sweep

lr=1e-5
0.312
lr=3e-5
0.289
lr=5e-5
0.284
best
lr=1e-4
0.302
lr=3e-4
0.358