Home/Courses/AI Architect/AI Architect: Compute-Constrained Training & Synthetic Data
Advanced CoursePart of AI Architect

AI Architect: Compute-Constrained Training & Synthetic Data

Train high-quality small language models on limited hardware. Learn to generate synthetic datasets, optimize GPU memory usage, apply LoRA and QLoRA fine-tuning, and align models with DPO and ORPO — from raw data to GGUF inference.

No rating yet
5 weeks

What You'll Learn

Generate and filter high-quality synthetic training data
Optimize GPU memory with gradient checkpointing and quantization
Apply LoRA, QLoRA, and DoRA for parameter-efficient fine-tuning
Align models using DPO and ORPO preference optimization
Export trained models to GGUF format for local inference

Course Content

W1
Week 1: Synthetic Data Engineering
Generate the data you need instead of waiting for it.
1
The Synthetic Data Engine
Learn how to use LLMs to programmatically generate domain-specific training examples at scale without manual labeling.
2
Semantic Document Chunking
Chunk source documents using semantic boundaries rather than fixed token counts to preserve meaning across training examples.
3
Multi-Turn Scenario Generation
Synthesize realistic multi-turn conversations that teach models complex reasoning and instruction-following behaviors.
4
Dataset Evolution Algorithms
Iteratively improve dataset quality by evolving prompts toward greater complexity, diversity, and coverage of edge cases.
5
Quality Assurance and Filtering
Apply perplexity scoring, deduplication, and LLM-as-judge filtering to remove low-quality examples before training.
Weekly Win
Filtered Synthetic Dataset
A 1,000-example domain-specific dataset generated, evolved, and filtered — ready for fine-tuning.
W2
Week 2: GPU Memory Optimization
Fit larger models into smaller budgets without sacrificing quality.
1
The Physics of GPU Memory
Understand how weights, activations, gradients, and optimizer states compete for VRAM during a training step.
2
Gradient Checkpointing
Trade compute for memory by recomputing activations during the backward pass instead of storing them all in VRAM.
3
Unsloth Optimization
Apply Unsloth's kernel-level optimizations to reduce memory footprint and accelerate training on consumer GPUs.
4
Fused Cross-Entropy Loss
Replace the standard loss computation with a fused kernel that dramatically reduces peak memory usage on large vocabularies.
5
The GaLore Algorithm
Use Gradient Low-Rank Projection to train full-parameter models with LoRA-level memory requirements.
Weekly Win
Memory-Optimized Training Run
Complete a full fine-tuning run on a 7B model using gradient checkpointing, Unsloth, and fused loss — under 16 GB VRAM.
W3
Week 3: Low-Rank Adaptation Methods
Fine-tune billion-parameter models with a fraction of the parameters.
1
The Mathematics of LoRA
Derive the low-rank decomposition at the heart of LoRA and understand why rank, alpha, and target modules matter.
2
4-Bit NF4 Quantization
Quantize model weights to NormalFloat4 format, preserving outliers while shrinking the memory footprint by 75%.
3
High-Rank QLoRA Configurations
Push QLoRA past its defaults with higher rank, more target modules, and longer schedules to match full fine-tune quality.
4
DoRA (Weight-Decomposed LoRA)
Decompose weight updates into magnitude and direction components for more stable and expressive parameter-efficient training.
5
Continuous Pre-training Strategies
Extend a model's knowledge on new domain text without catastrophic forgetting using replay buffers and learning rate schedules.
Weekly Win
QLoRA Fine-Tuned Checkpoint
A fine-tuned 7B model checkpoint trained with QLoRA on your synthetic dataset, evaluated on a domain-specific benchmark.
W4
Week 4: Alignment & Preference Optimization
Teach the model to prefer good outputs without a separate reward model.
1
The RLHF Alignment Bottleneck
Understand why classical RLHF requires a separate reward model and how newer methods eliminate this dependency.
2
Direct Preference Optimization (DPO)
Align model behavior from preference pairs using a closed-form loss that implicitly optimizes the reward function.
3
ORPO Mechanics
Combine supervised fine-tuning and preference alignment in a single training step with the Odds Ratio Preference Optimization objective.
4
Dataset Formatting for ORPO
Structure chosen-rejected pairs from your synthetic data into the ORPO format for seamless single-stage training.
5
Alignment Evaluation
Measure alignment quality with MT-Bench, AlpacaEval, and custom domain benchmarks to validate model behavior changes.
Weekly Win
ORPO-Aligned Model
An ORPO-aligned checkpoint that scores measurably better on helpfulness and refusal benchmarks than the base fine-tuned model.
W5
Week 5: Capstone — End-to-End Training Pipeline
Go from raw budget to a deployable model inference endpoint.
1
Capstone: Compute Arbitrage and Spot Pricing
Select optimal cloud spot instances and implement preemption-safe checkpointing to minimize training cost.
2
Capstone: Data Ingestion and Environment Setup
Stand up the training environment, ingest source documents, and run the full synthetic data generation pipeline.
3
Capstone: SLM Training Execution
Execute the full QLoRA fine-tuning run with memory optimizations, logging metrics to Weights & Biases.
4
Capstone: Post-Training Alignment
Apply ORPO alignment to the fine-tuned checkpoint and validate against domain-specific evaluation criteria.
5
Capstone: GGUF Export and Inference
Convert the aligned model to GGUF format, quantize for CPU inference, and serve locally with llama.cpp.
Weekly Win
Deployable Domain-Specific SLM
A fully trained, aligned, and GGUF-exported small language model running locally — built entirely on spot-instance budget.

Prerequisites

Python and PyTorch experience
Basic fine-tuning knowledge
Access to a GPU instance

Hands-on Project

Fine-tune a small language model on synthetic data, align it with ORPO, and export it for CPU inference — all within spot-instance budget constraints.

📚
Advanced Level
Course Price
₹14,999
India
$249
International · One-time payment
Next cohort starts Mar 30
Duration5 weeks
LevelAdvanced
FormatCohort-based
Modules5

What's included:

Live cohort sessions
Hands-on projects
Certificate of completion
Lifetime access
Career support

Part of Learning Track

🏗️
AI Architect
7 courses in track