Iliad Intensive Curriculum

• 0Prerequisites

  The prerequisites for the Iliad Intensive program.

• A.1AI Alignment Introduction

  A conceptual introduction to AI alignment: choosing an alignment target, ensuring trained systems actually pursue it, added challenges with goal-directedness, and views on risks and their mitigations.

• A.2Alignment in Practice

  What do labs do to make their models aligned in practice? Covers the different stages in the training pipeline, deployment and methods labs use to stress-test the result to see if the alignment holds.

• B.2Mysteries of Deep Learning

  Why deep learning works so well — approximation, generalization and optimization — and the parts nobody can explain yet.

• B.3Singular Learning Theory

  Singular learning theory (SLT) places degeneracy as a core part of understanding how neural networks learn. We cover the parameter-function map, the meaning of degeneracy through the local learning coefficient, to Watanabe's free energy formula and Bayesian phase transitions.

• B.4Training Dynamics

  Exact learning dynamics of deep linear networks -- loss-landscape geometry, balanced gradient flow and the NTK, the rich (saddle-to-saddle) and lazy regimes, their mixed unification, and the implicit bias of SGD noise.

• B.5Data Attribution

  Which training examples caused a model to behave the way it does? These notes develop the main approaches to data attribution -- counterfactuals and Shapley values, classical and Bayesian influence functions, and training-dynamics unrolling -- with derivation exercises throughout.

• C.1Intro to ML Engineering

  Crash course in ML and LLM basics in PyTorch, how to setup machines for ML research and engineering, and making effective use of Claude Code.

• C.2Mechanistic Interpretability

  What can we learn about how a model works by looking at its internals? We cover the main methods for reverse engineering neural networks, from features and circuits to sparse autoencoders and auto-interp of activations, and discuss the relevance to safety.

D.1Decision Theory and Reinforcement Learning

• D.1.1Preferences to Rewards

  Building from preferences and a minimal set of axioms to a utility function expressible as a sum of discounted rewards: the familiar framing in reinforcement learning.

• D.1.2Reinforcement Learning

  The Bellman equations and what follows from them: the existence of optimal policies, the policy improvement theorem, the rate of convergence of Bellman updates, and the convergence of Q-learning.

• D.2Policy Gradients and Misgeneralization

  Recorded lecture for the day. The coding-day worksheet is not yet ported.

D.3AIXI

• D.3.1Solomonoff Induction

  How an idealized agent should predict. A Bayesian mixture over a countable class of computable hypotheses learns to predict any sequence, with total error bounded by the description length of the truth -- a formal Occam's razor.

• D.3.2AIXI

  Exploring the Bayesian optimal policy for history based reinforcement learning.

D.4Agent Foundations

• D.4.1Agent Foundations

  Superintelligence alignment may allow only one critical try, with no trial and error on a system more capable than anything yet observed. Agent foundations seeks research directions that survive extreme optimization and self-modification.

• D.4.2Optimization and Thermodynamics

  An optimizer funnels a broad range of starting conditions into a narrow target; the entropy it removes measures how much. Physics charges for every bit, so an embedded agent pays for optimization with what it knows.

D.5Decision Theory

• D.5.1Decision Theory

  Safety proofs require decision-theoretic assumptions invariant under self-modification. Covers ideal choice in exotic environments (copies, simulations) and multiagent cooperation via open-source game theory and safe Pareto improvements.

• D.5.2Open-Source Game Theory

  When agents can read each other's code, cooperation needs neither repetition nor trust — but the same transparency starts races to commit first. Conditional commitment, safe Pareto improvements, and a tournament.

• D.6Instrumental Convergence

  We discuss a simple mathematical formalization of what it means to "seek power" in a Markov decision process (MDP), and conditions under which such behavior emerges.

• E.2Steganography & Backdoors

  Steganography is the study of hiding messages in plain sight. We run a demo for intuition, show that perfect undetectability is expensive but computational undetectability is cheap, and hide a backdoor in a model by tampering with weight initialization.