Workbench

Notebooks with direct access to your tenant data and model registry

notebook: cortex-med-eval.ipynb

Kernel: python 3.11 · GPU: 1 × A10 · Enclave: TDX

In [1]:

from cortex import Registry, FineTune

reg = Registry(workspace="acme.health")
base = reg.pull("llama-3.1-8b")
ds = reg.dataset("readmit-notes-2025q1", version=14)

job = FineTune(
  base=base, dataset=ds, method="lora",
  lora_rank=16, lr=3.2e-5,
  enclave="intel_tdx",
).submit(gpus="8xH100")
job.tail()

Out [1]

Submitted job_a7c1 → us-east-2
[14:22:11] step=6800 loss=0.284 tokens/s=42,180
[14:22:14] eval: perplexity=1.34 acc=0.941

In [2]:

preds = reg.endpoint("ep_prod_med").predict([
  "Patient reports chest pain radiating to left arm...",
  "Follow-up MRI showed no acute findings...",
])
preds

Out [2]

[
  {'risk': 'high', 'score': 0.87, 'flags': ['cardiac']},
  {'risk': 'low',  'score': 0.12, 'flags': []}
]

Environment

cortex==0.42.1        transformers==4.44.0
torch==2.4.0+cu124    peft==0.12.0
vllm==0.5.4           datasets==2.20.0
bitsandbytes==0.43    accelerate==0.33.0

Session

RuntimePython 3.11

Idle timeout30m

IsolationNamespace acme-med

Data accessreadmit-*

EgressDeny by default