Artificial Intelligence — A Complete Advanced Course Material
About This Course
Who This Is For
Practitioners who have completed intermediate AI engineering and want research-level depth: theory, large-scale training, and frontier methods.
How to Use This
Treat each chapter as a research seminar. Pair it with the original papers it points toward and expect to derive, not just read.
Estimated Time
Approximately 130–170 hours of self-paced study to work through all 13 chapters with the rigor they’re written for.
What You’ll Build
The ability to reason from first principles about why models train, scale, fail, and align — and to design new methods, not just apply existing ones.
Chapter Road Map
Ch 1–3
Theoretical Bedrock
Learning theory, optimisation theory, and the systems that make scale possible.
Ch 4–6
Frontier Modelling
Architecture research, advanced RL, and the theory behind generative models.
Ch 7–8
Reasoning & Coordination
Causal inference, robustness, and advanced multi-agent systems.
Ch 9–10
Safety & Hardware
Frontier alignment research and hardware-aware systems co-design.
Ch 11–13
Frontiers & Craft
AI for science and embodiment, multimodal research, and the practice of research itself.
01
Before designing new methods, an advanced practitioner needs to understand why learning works at all — the formal guarantees, and lack thereof, behind generalisation.
1.1PAC Learning & Sample Complexity
The formal Probably-Approximately-Correct framework for how many examples a learner needs to guarantee a bound on its error.
1.2VC Dimension & Capacity Measures
Quantifying a hypothesis class’s expressive power, and why VC theory alone fails to explain why over-parameterised networks generalise.
1.3Rademacher Complexity & Margin Theory
A tighter, data-dependent complexity measure, and how margin-based bounds explain generalisation in classifiers with huge capacity.
1.4The Double Descent Phenomenon
Why test error can rise then fall again as model size grows past the interpolation threshold, overturning classical bias-variance intuition.
1.5Neural Tangent Kernel Theory
How infinitely wide networks behave like kernel machines during training, and what this reveals about why gradient descent finds good solutions.
1.6The Lottery Ticket Hypothesis
Evidence that dense networks contain sparse, independently trainable subnetworks that explain much of the full model’s performance.
1.7Implicit Regularisation in Gradient Descent
Why unregularised gradient descent on over-parameterised models still tends toward low-complexity, generalising solutions.
1.8The Loss Landscape — Geometry of Deep Networks
Saddle points, flat versus sharp minima, and mode connectivity — what the high-dimensional surface gradient descent navigates actually looks like.
1.9Information-Theoretic Generalisation Bounds
Using mutual information between training data and learned weights to bound how much a model has memorised versus generalised.
1.10Grokking & Delayed Generalisation
The puzzling case where a model memorises training data first, then suddenly generalises long after training loss has plateaued.
02
Adam and SGD are starting points, not endpoints. This chapter goes underneath the optimiser to the theory of why certain update rules and schedules behave the way they do at scale.
2.1Convergence Proofs for First-Order Methods
The formal conditions — smoothness, convexity, step-size bounds — under which gradient-based methods are guaranteed to converge.
2.2Second-Order & Quasi-Newton Methods
Using curvature information directly, and why methods like K-FAC approximate the Hessian rather than computing it exactly.
2.3Adaptive Optimiser Theory — Why Adam Works
The bias-corrected moment estimates inside Adam, its known convergence pathologies, and why AdamW decouples weight decay correctly.
2.4Neural Scaling Laws — Deriving the Exponents
The power-law relationships between loss, model size, data, and compute, and the Chinchilla result on compute-optimal allocation.
2.5The Muon & Shampoo Family of Optimisers
Orthogonalised and matrix-aware update rules designed specifically for the geometry of large transformer weight matrices.
2.6Learning Rate Transfer Across Scale (μP)
Maximal Update Parametrisation, and how it lets hyperparameters tuned on a small model transfer reliably to a much larger one.
2.7Critical Batch Size & the Compute-Time Tradeoff
Why doubling batch size doesn’t always halve training time, and how to find the batch size where data parallelism stops paying off.
2.8Training Instabilities at Scale
Loss spikes, activation explosions, and the architectural and optimiser fixes — like QK-normalisation — used to keep large runs stable.
2.9Sharpness-Aware Minimisation
Explicitly optimising for flat minima during training as a route to better generalisation, and its computational cost in practice.
2.10Federated & Decentralised Optimisation
Optimising across devices that never share raw data, and the convergence challenges introduced by non-IID, unevenly sized local datasets.
03
Frontier models are trained on thousands of accelerators simultaneously. This chapter covers the parallelism strategies and infrastructure engineering that make that possible.
3.1Data Parallelism & Gradient Synchronisation
Splitting batches across devices, the all-reduce communication pattern, and how it bottlenecks as the number of workers grows.
3.2Tensor Parallelism
Splitting individual weight matrices across devices so a single layer’s computation spans multiple accelerators.
3.3Pipeline Parallelism
Splitting a model’s layers across devices and the micro-batching schedules used to keep every stage of the pipeline busy.
3.4Combining Parallelism — 3D & 5D Strategies
How data, tensor, pipeline, and expert parallelism are composed together for trillion-parameter training runs.
3.5ZeRO & Fully Sharded Data Parallel (FSDP)
Sharding optimiser states, gradients, and parameters across devices to train models far larger than any single device’s memory.
3.6Activation Checkpointing & Memory Engineering
Trading recomputation for memory by discarding and later recomputing activations, and where the optimal tradeoff point sits.
3.7Communication-Efficient Training
Gradient compression, overlapping communication with computation, and why interconnect bandwidth is often the true scaling bottleneck.
3.8Fault Tolerance in Multi-Week Training Runs
Checkpointing strategy, automatic node failure recovery, and elastic re-scheduling for jobs that can’t afford to restart from zero.
3.9Data Pipeline Engineering for Trillion-Token Corpora
Streaming, sharding, and deduplicating web-scale datasets fast enough that the GPUs are never left waiting for data.
3.10Cluster Scheduling & Job Orchestration
How frameworks like Slurm and Kubernetes-based schedulers allocate accelerators across competing large-scale training jobs.
04
Beyond using a Transformer — the open architectural questions researchers are currently working on to push past its known limitations.
4.1Beyond Standard MoE — Fine-Grained & Shared-Expert Routing
Routing strategies that go beyond top-k expert selection, including shared experts and load-balancing losses that prevent expert collapse.
4.2Hybrid Attention–State-Space Architectures
Interleaving attention layers with Mamba-style state-space layers to combine recall quality with linear-time inference.
4.3Multi-Token Prediction & Speculative Architectures
Training models to predict several future tokens at once, improving sample efficiency and enabling faster speculative decoding.
4.4Test-Time Compute & Inference-Time Scaling
Why letting a model “think longer” at inference — search, self-consistency, verifier-guided decoding — can rival training-time scaling.
4.5Reasoning Models & Chain-of-Thought Training
Architectures and training procedures purpose-built to produce long, verifiable intermediate reasoning traces before a final answer.
4.6Extreme Long-Context Architectures
Retrieval-augmented attention, recurrent memory compression, and architectural changes aimed at million-token-plus context.
4.7Tokenizer-Free & Byte-Level Modelling
Approaches that operate directly on raw bytes or learn dynamic patch boundaries, removing the fixed vocabulary bottleneck entirely.
4.8World Models & Predictive Architectures
Models trained to predict future states of an environment rather than just the next token, as a route toward planning and reasoning.
4.9Reproducing a Paper’s Architecture From Scratch
The discipline of implementing a published architecture without the authors’ code, and the subtle details papers routinely omit.
4.10Ablation Study Design
Isolating which architectural component actually drives a result, and the controls needed to make an ablation scientifically convincing.
05
RLHF is one shallow application of RL. This chapter covers the field of reinforcement learning itself — the theory and algorithms behind sequential decision-making.
5.1Markov Decision Processes — The Formal Framework
States, actions, transition dynamics, and reward, and the Bellman equations that define optimal value functions.
5.2Value-Based Methods — Q-Learning to DQN
Learning action-values directly, and the stabilisation tricks — target networks, experience replay — that made deep Q-learning work.
5.3Policy Gradient Methods
Optimising a policy directly via the policy gradient theorem, and why variance reduction with a baseline is essential in practice.
5.4Trust-Region & Proximal Methods — TRPO to PPO
Constraining how far a policy update can move in a single step, and why PPO’s clipped objective became the default in practice.
5.5Actor-Critic Architectures
Combining a learned value estimate with a policy network, and modern variants like Soft Actor-Critic for continuous control.
5.6Model-Based Reinforcement Learning
Learning a model of environment dynamics to plan or simulate experience, trading sample efficiency against model error.
5.7Offline & Batch Reinforcement Learning
Learning a policy purely from a fixed, previously collected dataset, and the distribution-shift problems unique to this setting.
5.8RLHF and RLAIF in Full Algorithmic Detail
The complete pipeline — reward modelling, KL-constrained PPO, and AI-feedback variants — beyond the high-level summary.
5.9Group Relative & Critic-Free Policy Optimisation
Newer methods, such as GRPO, that estimate advantage from groups of sampled completions instead of training a separate value model.
5.10Exploration Strategies & the Exploration-Exploitation Tradeoff
Intrinsic motivation, curiosity-driven exploration, and why exploration is especially hard in sparse-reward environments.
5.11Multi-Agent Reinforcement Learning
Cooperative and competitive settings, non-stationarity from co-adapting agents, and self-play as a training paradigm.
06
The mathematics underneath diffusion and generative models — not how to fine-tune them, but why the underlying probability theory makes them work.
6.1Probabilistic Graphical Models
Bayesian networks and Markov random fields as a language for expressing conditional independence structure in complex systems.
6.2Variational Inference & the ELBO
Approximating intractable posteriors by optimising a tractable lower bound, the core trick behind VAEs and much of modern Bayesian ML.
6.3Score-Based Generative Models
Modelling the gradient of the log-density directly, and how this score function connects diffusion models to stochastic differential equations.
6.4Flow-Based & Normalising Flow Models
Building exact-likelihood generative models from invertible transformations with a tractable Jacobian determinant.
6.5Flow Matching as a Diffusion Alternative
Learning a direct vector field between noise and data distributions, simplifying training and improving sampling efficiency.
6.6Energy-Based Models
Defining a distribution through an unnormalised energy function, and the sampling challenges this approach introduces.
6.7Autoregressive Models as Density Estimators
Why next-token prediction is itself a form of exact maximum-likelihood density estimation over sequences.
6.8Theoretical Limits of Generative Models
Mode collapse, the likelihood-versus-sample-quality tradeoff, and formal results on what generative models provably cannot do.
6.9Gaussian Processes & Bayesian Deep Learning
Treating model uncertainty itself as a first-class quantity, and why this matters for safety-critical decision-making.
07
Correlation-based models break under distribution shift. This chapter covers reasoning about cause and effect, and building models that hold up outside their training distribution.
7.1The Causal Hierarchy — Association, Intervention, Counterfactual
Pearl’s three-level ladder of causation, and why a model that predicts well can still answer “what if” questions incorrectly.
7.2Structural Causal Models & Causal Graphs
Representing cause-effect relationships explicitly as a graph, and reading off which variables confound an observed association.
7.3Do-Calculus & Identifying Causal Effects
The formal rules for determining when a causal effect can be estimated from observational data alone, without an experiment.
7.4Causal Discovery From Data
Algorithms that infer plausible causal graph structure directly from observational data, and the strong assumptions they rely on.
7.5Domain Generalisation & Out-of-Distribution Robustness
Training strategies aimed at performance under distribution shift rather than just on an i.i.d. held-out test set.
7.6Adversarial Robustness — Attacks & Formal Defences
Gradient-based adversarial attacks, certified defences with provable robustness guarantees, and their accuracy-robustness tradeoff.
7.7Spurious Correlations & Shortcut Learning
Why models latch onto easy, non-causal signals in training data, and methods designed to penalise reliance on shortcuts.
7.8Uncertainty Quantification in Deep Learning
Distinguishing aleatoric from epistemic uncertainty, and calibration methods like conformal prediction with formal coverage guarantees.
7.9Causal Representation Learning
Learning representations whose components correspond to disentangled causal factors rather than entangled statistical patterns.
08
Past single agents and basic orchestration — the open research problems in building agent systems that scale, coordinate, and remain reliable over long horizons.
8.1Long-Horizon Planning Under Uncertainty
Why agent reliability degrades compounding over many steps, and search and verification strategies that slow that decay.
8.2Hierarchical Task Decomposition
Recursively breaking a goal into subgoals across multiple levels of abstraction, and managing context across that hierarchy.
8.3Formal Coordination Protocols for Multi-Agent Systems
Negotiation, contract-net style task allocation, and consensus mechanisms for agents that must agree without a central controller.
8.4Emergent Behaviour in Agent Populations
How collective strategies arise from simple agent-to-agent interaction rules, and why this is hard to predict in advance.
8.5Self-Improving & Self-Verifying Agents
Agents that critique, test, and revise their own outputs autonomously, and the risk of compounding errors without external grounding.
8.6Learned Tool Use & Tool Synthesis
Agents that write and validate their own new tools at runtime rather than being limited to a pre-defined tool set.
8.7Formal Verification of Agent Actions
Checking an agent’s proposed action against explicit constraints before execution, especially where real-world side effects are irreversible.
8.8Sim-to-Real & Sandboxed Agent Training
Training and evaluating agents safely in simulated environments before allowing real-world tool access.
8.9Benchmarking Long-Horizon Agentic Reliability
Why standard one-shot benchmarks fail to capture agent reliability, and the multi-step evaluation suites built to measure it properly.
09
Beyond applying RLHF and red-teaming — the open research questions about aligning systems that may exceed human ability to directly supervise.
9.1Scalable Oversight
Techniques for supervising AI systems on tasks where humans cannot easily verify correctness directly, such as debate and recursive reward modelling.
9.2Weak-to-Strong Generalisation
Studying whether and how a stronger model can be reliably supervised using labels from a weaker one.
9.3Deceptive Alignment & Goal Misgeneralisation
The theoretical risk of a model appearing aligned during training while pursuing a different objective, and how it could arise.
9.4Interpretability for Alignment — Circuit-Level Analysis
Using sparse autoencoders and feature attribution to find and verify specific computational circuits inside trained models.
9.5Eliciting Latent Knowledge
The open problem of extracting what a model “knows” internally, even when its stated output doesn’t reflect that knowledge truthfully.
9.6Formal Safety Specifications & Verification
Specifying safety properties mathematically and attempting to prove, rather than empirically test, that a system satisfies them.
9.7Evaluating Dangerous Capabilities
Structured methodology for assessing whether a model could meaningfully uplift high-risk activities before deciding on deployment.
9.8Model Organisms of Misalignment
Deliberately training small-scale models that exhibit a specific misalignment failure mode, to study it safely and reproducibly.
9.9Governance Mechanisms for Frontier Training Runs
Responsible scaling policies, capability thresholds, and the internal evaluation gates used before training or deploying a more capable model.
10
At the frontier, the model and the chip are designed together. This chapter covers how hardware constraints shape algorithm design, and vice versa.
10.1GPU & Accelerator Microarchitecture for ML
Streaming multiprocessors, tensor cores, and memory hierarchy, and how they shape which operations are actually fast in practice.
10.2The Roofline Model — Compute vs. Memory Bound
A framework for diagnosing whether a kernel’s performance is limited by arithmetic throughput or memory bandwidth.
10.3Writing Custom GPU Kernels
When and why a hand-written CUDA or Triton kernel outperforms a framework’s default operator, and the engineering cost of writing one.
10.4Compiler-Level Optimisation for ML — XLA, TorchInductor
Graph-level fusion, kernel auto-generation, and how ML compilers close the gap between research code and hardware-efficient execution.
10.5Interconnects & Network Topology for Training Clusters
NVLink, InfiniBand, and the topology choices that determine how well a parallelism strategy actually scales across nodes.
10.6Custom Silicon for AI — TPUs & Beyond
Domain-specific architecture design choices, and how training on TPU-class hardware changes algorithmic and numerical considerations.
10.7Numerical Precision Strategy — FP8, Stochastic Rounding
Pushing precision lower than standard mixed precision, and the rounding and scaling tricks needed to keep training stable.
10.8Serving Infrastructure for Frontier-Scale Inference
Continuous batching, disaggregated prefill-decode serving, and KV-cache management at the scale of millions of concurrent requests.
10.9Power, Cooling & Datacenter-Scale Constraints
How power delivery and thermal limits, not just chip count, increasingly bound how large a single training run can practically be.
11
AI applied where the model must contend with the physical world or with scientific ground truth, not just with data — a different set of constraints entirely.
11.1AI for Protein Structure & Molecular Design
How models like AlphaFold reframed structure prediction as a learning problem, and how generative models now design novel molecules.
11.2Physics-Informed Neural Networks
Embedding known differential equations directly into the training loss so models respect physical laws rather than just fitting data.
11.3Neural Operators & Scientific Surrogate Models
Learning mappings between entire function spaces to replace expensive numerical simulations with fast, learned approximations.
11.4Graph Neural Networks for Structured Scientific Data
Message-passing architectures suited to molecules, materials, and other data that is naturally a graph rather than a grid or sequence.
11.5Robot Learning & Sensorimotor Policies
Learning control policies that map raw sensor input directly to actuator commands, and the unique data scarcity this setting faces.
11.6Vision-Language-Action Models
Architectures that unify perception, language understanding, and physical action generation in a single trained policy.
11.7Sim-to-Real Transfer for Robotics
Domain randomisation and other techniques for closing the gap between policies trained in simulation and real robot hardware.
11.8Imitation Learning & Learning From Demonstration
Training policies directly from human demonstrations, and the compounding-error problems that distinguish this from standard supervised learning.
11.9Embodied Evaluation & Real-World Benchmarking
Why benchmarking a physical agent is fundamentally harder than benchmarking a text model, and the standardised testbeds built to address it.
12
Architecture-level depth in modalities beyond text and static images — and the research challenges of fusing modalities into a single coherent model.
12.1Speech Representation Learning
Self-supervised audio encoders that learn phonetic and acoustic structure from raw waveforms without transcribed labels.
12.2End-to-End Speech Recognition Architectures
Connectionist Temporal Classification and attention-based sequence models that map audio directly to text without separate acoustic stages.
12.3Neural Audio Codecs & Discrete Audio Tokens
Compressing raw audio into discrete tokens, enabling language-model-style architectures to generate speech and music directly.
12.4Expressive & Real-Time Text-to-Speech
Architectures producing natural prosody and voice cloning capability, and the latency constraints of real-time conversational speech.
12.5Cross-Modal Alignment & Joint Embedding Spaces
Contrastive training objectives like CLIP that pull matching content from different modalities into a shared representation space.
12.6Native Multimodal Architectures
Models trained jointly across text, image, audio, and video from the start, versus bolting separate modality encoders onto a language model.
12.7Any-to-Any Generation Architectures
Single models capable of generating across modalities — text-to-video, audio-to-image — without separate task-specific pipelines.
12.8Temporal Consistency in Long-Form Video Generation
The open problem of maintaining coherent objects, identity, and physics across video far longer than a few seconds.
12.9Evaluating Multimodal Models Rigorously
Why text-based benchmarks transfer poorly to multimodal evaluation, and the human-preference and grounding-based alternatives in use.
13
Technical depth alone doesn’t make a researcher. This closing chapter covers the craft of doing and communicating original AI research.
13.1Formulating a Genuine Research Question
Distinguishing an open problem worth pursuing from an incremental variation, and scoping a question to something testable.
13.2Designing Rigorous, Reproducible Experiments
Controls, seeds, statistical significance testing, and the discipline needed to make a result trustworthy to others.
13.3Reading Research at Speed and Depth
A systematic method for triaging the literature, separating load-bearing claims from framing, and tracking a fast-moving subfield.
13.4Reproducing Published Results
Why a large fraction of papers don’t reproduce cleanly, and the systematic process for finding where and why a reimplementation diverges.
13.5Writing a Research Paper
Structuring a clear contribution statement, presenting results honestly including negative ones, and writing for a peer-review audience.
13.6The Peer Review Process
What reviewers actually evaluate, common rejection reasons, and how to respond constructively to critical feedback.
13.7Open-Source Research Contribution
Contributing to major open frameworks and model releases, and the engineering bar expected of a credible open-source contribution.
13.8Research Ethics & Responsible Disclosure
Handling dual-use findings responsibly, and the norms around disclosing capability or safety-relevant results before publication.
13.9Building an Independent Research Agenda
Moving from executing assigned problems to identifying and pursuing your own coherent research direction over time.