Methods · LLM / LoRA

Foundation-model training with Hugging Face + PEFT. model: is required. Devices: CUDA, MPS, ROCm, CPU. QLoRA is CUDA + bitsandbytes only.

Prerequisitefamily: llm · recipe.model set · kernel HF deps installed

method: lora | qlora | llm

LoRA / QLoRA / full LLM training. Objectives: next-token, sft, full-ft, lora, qlora, fim, mlm, span, continued-pretrain.

example

family: llmmethod: loramodel: meta-llama/Llama-3.2-1B-Instructobjective: sftrank: 16alpha: 32steps: 100lr: 2.0e-4data:  path: data.jsonl  prompt: prompt  completion: completioneval:  metric: loss

SFT masks prompt tokens (loss on completion). full-ft trains all non-pad tokens. mlm needs a MaskedLM-capable model. size: is a label only.

aq serve

Generate from the latest (or named) checkpoint after an LLM train.

Flag	Description
--ckpt <name>	Checkpoint name.
--max-tokens n	Max tokens.

example

$aq serve "hello" --max-tokens 32