Deep Learning & Modern AI
Mission. Neural networks from first principles through modern practice.
Scope
- Neural nets from scratch, bridging from math foundations
- PyTorch
- CNNs, sequence models, transformers
- Embeddings and fine-tuning
- LLM application patterns
Learning objectives
- Implement backprop by hand once, then trust the framework
- Fine-tune a pretrained model on your own data
- Build a working LLM-backed application with honest evaluation
Lessons
This track is scaffolded and waiting for content. See the curriculum dashboard for the roadmap and progress across all tracks.
Planned lessons (21)
Wave 3 — depth (provisional roadmap) — 21 lessons
foundations
- The Perceptron A linear model with a learning rule, and the boundary of what it can learn.
- Backpropagation by Hand The chain rule applied to a small network, on paper.
- Backprop in NumPy A tiny neural network with the backward pass written out.
pytorch
- PyTorch Tensors and Autograd The tensor, the graph, and the automatic gradient.
- Training Loops in PyTorch Batches, forward, backward, step — the loop you'll write a thousand times.
training
- Overfitting and Dropout The failure mode of deep nets and the regularization that fights it.
- Optimizers: SGD to Adam Momentum, adaptivity, and the defaults everyone uses.
- Learning Rates and Schedules The most important hyperparameter, tuned without superstition.
architectures
- Convolutions Locality and weight sharing: why CNNs see images.
- CNN Architectures in Practice Blocks, transfer, and not designing from scratch.
- Embeddings Meaning as geometry: entities as vectors.
- Sequence Models: RNNs Recurrent state, its limits, and why attention replaced it.
- Attention Queries, keys, values: weighted lookup as a differentiable operation.
- The Transformer The architecture behind the modern AI wave, layer by layer.
practice
- Transfer Learning Start from pretrained weights; fine-tune deliberately.
- Fine-Tuning a Pretrained Model Heads, freezing, and catastrophic forgetting on your own data.
- Evaluating Neural Nets Honest evaluation when the model has millions of parameters.
llms
- LLM Application Patterns Prompts, tools, agents: the patterns behind LLM-backed products.
- Retrieval-Augmented Generation Grounding LLM output in your documents, with evaluation.
- Serving and Monitoring Models From checkpoint to endpoint, with feedback loops.
projects
- Case Study: Image Classifier End to End Data, transfer learning, evaluation, and deployment.