|
Preface
|
Intern |
doFollow |
|
Installation
|
Intern |
doFollow |
|
Notation
|
Intern |
doFollow |
|
1. Introduction
|
Intern |
doFollow |
|
2. Preliminaries
|
Intern |
doFollow |
|
2.1. Data Manipulation
|
Intern |
doFollow |
|
2.2. Data Preprocessing
|
Intern |
doFollow |
|
2.3. Linear Algebra
|
Intern |
doFollow |
|
2.4. Calculus
|
Intern |
doFollow |
|
2.5. Automatic Differentiation
|
Intern |
doFollow |
|
2.6. Probability and Statistics
|
Intern |
doFollow |
|
2.7. Documentation
|
Intern |
doFollow |
|
3. Linear Neural Networks for Regression
|
Intern |
doFollow |
|
3.1. Linear Regression
|
Intern |
doFollow |
|
3.2. Object-Oriented Design for Implementation
|
Intern |
doFollow |
|
3.3. Synthetic Regression Data
|
Intern |
doFollow |
|
3.4. Linear Regression Implementation from Scratch
|
Intern |
doFollow |
|
3.5. Concise Implementation of Linear Regression
|
Intern |
doFollow |
|
3.6. Generalization
|
Intern |
doFollow |
|
3.7. Weight Decay
|
Intern |
doFollow |
|
4. Linear Neural Networks for Classification
|
Intern |
doFollow |
|
4.1. Softmax Regression
|
Intern |
doFollow |
|
4.2. The Image Classification Dataset
|
Intern |
doFollow |
|
4.3. The Base Classification Model
|
Intern |
doFollow |
|
4.4. Softmax Regression Implementation from Scratch
|
Intern |
doFollow |
|
4.5. Concise Implementation of Softmax Regression
|
Intern |
doFollow |
|
4.6. Generalization in Classification
|
Intern |
doFollow |
|
4.7. Environment and Distribution Shift
|
Intern |
doFollow |
|
5. Multilayer Perceptrons
|
Intern |
doFollow |
|
5.1. Multilayer Perceptrons
|
Intern |
doFollow |
|
5.2. Implementation of Multilayer Perceptrons
|
Intern |
doFollow |
|
5.3. Forward Propagation, Backward Propagation, and Computational Graphs
|
Intern |
doFollow |
|
5.4. Numerical Stability and Initialization
|
Intern |
doFollow |
|
5.5. Generalization in Deep Learning
|
Intern |
doFollow |
|
5.6. Dropout
|
Intern |
doFollow |
|
5.7. Predicting House Prices on Kaggle
|
Intern |
doFollow |
|
6. Builders’ Guide
|
Intern |
doFollow |
|
6.1. Layers and Modules
|
Intern |
doFollow |
|
6.2. Parameter Management
|
Intern |
doFollow |
|
6.3. Parameter Initialization
|
Intern |
doFollow |
|
6.4. Lazy Initialization
|
Intern |
doFollow |
|
6.5. Custom Layers
|
Intern |
doFollow |
|
6.6. File I/O
|
Intern |
doFollow |
|
6.7. GPUs
|
Intern |
doFollow |
|
7. Convolutional Neural Networks
|
Intern |
doFollow |
|
7.1. From Fully Connected Layers to Convolutions
|
Intern |
doFollow |
|
7.2. Convolutions for Images
|
Intern |
doFollow |
|
7.3. Padding and Stride
|
Intern |
doFollow |
|
7.4. Multiple Input and Multiple Output Channels
|
Intern |
doFollow |
|
7.5. Pooling
|
Intern |
doFollow |
|
7.6. Convolutional Neural Networks (LeNet)
|
Intern |
doFollow |
|
8. Modern Convolutional Neural Networks
|
Intern |
doFollow |
|
8.1. Deep Convolutional Neural Networks (AlexNet)
|
Intern |
doFollow |
|
8.2. Networks Using Blocks (VGG)
|
Intern |
doFollow |
|
8.3. Network in Network (NiN)
|
Intern |
doFollow |
|
8.4. Multi-Branch Networks (GoogLeNet)
|
Intern |
doFollow |
|
8.5. Batch Normalization
|
Intern |
doFollow |
|
8.6. Residual Networks (ResNet) and ResNeXt
|
Intern |
doFollow |
|
8.7. Densely Connected Networks (DenseNet)
|
Intern |
doFollow |
|
8.8. Designing Convolution Network Architectures
|
Intern |
doFollow |
|
9. Recurrent Neural Networks
|
Intern |
doFollow |
|
9.1. Working with Sequences
|
Intern |
doFollow |
|
9.2. Converting Raw Text into Sequence Data
|
Intern |
doFollow |
|
9.3. Language Models
|
Intern |
doFollow |
|
9.4. Recurrent Neural Networks
|
Intern |
doFollow |
|
9.5. Recurrent Neural Network Implementation from Scratch
|
Intern |
doFollow |
|
9.6. Concise Implementation of Recurrent Neural Networks
|
Intern |
doFollow |
|
9.7. Backpropagation Through Time
|
Intern |
doFollow |
|
10. Modern Recurrent Neural Networks
|
Intern |
doFollow |
|
10.1. Long Short-Term Memory (LSTM)
|
Intern |
doFollow |
|
10.2. Gated Recurrent Units (GRU)
|
Intern |
doFollow |
|
10.3. Deep Recurrent Neural Networks
|
Intern |
doFollow |
|
10.4. Bidirectional Recurrent Neural Networks
|
Intern |
doFollow |
|
10.5. Machine Translation and the Dataset
|
Intern |
doFollow |
|
10.6. The Encoder–Decoder Architecture
|
Intern |
doFollow |
|
10.7. Sequence-to-Sequence Learning for Machine Translation
|
Intern |
doFollow |
|
10.8. Beam Search
|
Intern |
doFollow |
|
11. Attention Mechanisms and Transformers
|
Intern |
doFollow |
|
11.1. Queries, Keys, and Values
|
Intern |
doFollow |
|
11.2. Attention Pooling by Similarity
|
Intern |
doFollow |
|
11.3. Attention Scoring Functions
|
Intern |
doFollow |
|
11.4. The Bahdanau Attention Mechanism
|
Intern |
doFollow |
|
11.5. Multi-Head Attention
|
Intern |
doFollow |
|
11.6. Self-Attention and Positional Encoding
|
Intern |
doFollow |
|
11.7. The Transformer Architecture
|
Intern |
doFollow |
|
11.8. Transformers for Vision
|
Intern |
doFollow |
|
11.9. Large-Scale Pretraining with Transformers
|
Intern |
doFollow |
|
12. Optimization Algorithms
|
Intern |
doFollow |
|
12.1. Optimization and Deep Learning
|
Intern |
doFollow |
|
12.2. Convexity
|
Intern |
doFollow |
|
12.3. Gradient Descent
|
Intern |
doFollow |
|
12.4. Stochastic Gradient Descent
|
Intern |
doFollow |
|
12.5. Minibatch Stochastic Gradient Descent
|
Intern |
doFollow |
|
12.6. Momentum
|
Intern |
doFollow |
|
12.7. Adagrad
|
Intern |
doFollow |
|
12.8. RMSProp
|
Intern |
doFollow |
|
12.9. Adadelta
|
Intern |
doFollow |
|
12.10. Adam
|
Intern |
doFollow |
|
12.11. Learning Rate Scheduling
|
Intern |
doFollow |
|
13. Computational Performance
|
Intern |
doFollow |
|
13.1. Compilers and Interpreters
|
Intern |
doFollow |
|
13.2. Asynchronous Computation
|
Intern |
doFollow |
|
13.3. Automatic Parallelism
|
Intern |
doFollow |
|
13.4. Hardware
|
Intern |
doFollow |
|
13.5. Training on Multiple GPUs
|
Intern |
doFollow |
|
13.6. Concise Implementation for Multiple GPUs
|
Intern |
doFollow |
|
13.7. Parameter Servers
|
Intern |
doFollow |
|
14. Computer Vision
|
Intern |
doFollow |
|
14.1. Image Augmentation
|
Intern |
doFollow |
|
14.2. Fine-Tuning
|
Intern |
doFollow |
|
14.3. Object Detection and Bounding Boxes
|
Intern |
doFollow |
|
14.4. Anchor Boxes
|
Intern |
doFollow |
|
14.5. Multiscale Object Detection
|
Intern |
doFollow |
|
14.6. The Object Detection Dataset
|
Intern |
doFollow |
|
14.7. Single Shot Multibox Detection
|
Intern |
doFollow |
|
14.8. Region-based CNNs (R-CNNs)
|
Intern |
doFollow |
|
14.9. Semantic Segmentation and the Dataset
|
Intern |
doFollow |
|
14.10. Transposed Convolution
|
Intern |
doFollow |
|
14.11. Fully Convolutional Networks
|
Intern |
doFollow |
|
14.12. Neural Style Transfer
|
Intern |
doFollow |
|
14.13. Image Classification (CIFAR-10) on Kaggle
|
Intern |
doFollow |
|
14.14. Dog Breed Identification (ImageNet Dogs) on Kaggle
|
Intern |
doFollow |
|
15. Natural Language Processing: Pretraining
|
Intern |
doFollow |
|
15.1. Word Embedding (word2vec)
|
Intern |
doFollow |
|
15.2. Approximate Training
|
Intern |
doFollow |
|
15.3. The Dataset for Pretraining Word Embeddings
|
Intern |
doFollow |
|
15.4. Pretraining word2vec
|
Intern |
doFollow |
|
15.5. Word Embedding with Global Vectors (GloVe)
|
Intern |
doFollow |
|
15.6. Subword Embedding
|
Intern |
doFollow |
|
15.7. Word Similarity and Analogy
|
Intern |
doFollow |
|
15.8. Bidirectional Encoder Representations from Transformers (BERT)
|
Intern |
doFollow |
|
15.9. The Dataset for Pretraining BERT
|
Intern |
doFollow |
|
15.10. Pretraining BERT
|
Intern |
doFollow |
|
16. Natural Language Processing: Applications
|
Intern |
doFollow |
|
16.1. Sentiment Analysis and the Dataset
|
Intern |
doFollow |
|
16.2. Sentiment Analysis: Using Recurrent Neural Networks
|
Intern |
doFollow |
|
16.3. Sentiment Analysis: Using Convolutional Neural Networks
|
Intern |
doFollow |
|
16.4. Natural Language Inference and the Dataset
|
Intern |
doFollow |
|
16.5. Natural Language Inference: Using Attention
|
Intern |
doFollow |
|
16.6. Fine-Tuning BERT for Sequence-Level and Token-Level Applications
|
Intern |
doFollow |
|
16.7. Natural Language Inference: Fine-Tuning BERT
|
Intern |
doFollow |
|
17. Reinforcement Learning
|
Intern |
doFollow |
|
17.1. Markov Decision Process (MDP)
|
Intern |
doFollow |
|
17.2. Value Iteration
|
Intern |
doFollow |
|
17.3. Q-Learning
|
Intern |
doFollow |
|
18. Gaussian Processes
|
Intern |
doFollow |
|
18.1. Introduction to Gaussian Processes
|
Intern |
doFollow |
|
18.2. Gaussian Process Priors
|
Intern |
doFollow |
|
18.3. Gaussian Process Inference
|
Intern |
doFollow |
|
19. Hyperparameter Optimization
|
Intern |
doFollow |
|
19.1. What Is Hyperparameter Optimization?
|
Intern |
doFollow |
|
19.2. Hyperparameter Optimization API
|
Intern |
doFollow |
|
19.3. Asynchronous Random Search
|
Intern |
doFollow |
|
19.4. Multi-Fidelity Hyperparameter Optimization
|
Intern |
doFollow |
|
19.5. Asynchronous Successive Halving
|
Intern |
doFollow |
|
20. Generative Adversarial Networks
|
Intern |
doFollow |
|
20.1. Generative Adversarial Networks
|
Intern |
doFollow |
|
20.2. Deep Convolutional Generative Adversarial Networks
|
Intern |
doFollow |
|
21. Recommender Systems
|
Intern |
doFollow |
|
21.1. Overview of Recommender Systems
|
Intern |
doFollow |
|
21.2. The MovieLens Dataset
|
Intern |
doFollow |
|
21.3. Matrix Factorization
|
Intern |
doFollow |
|
21.4. AutoRec: Rating Prediction with Autoencoders
|
Intern |
doFollow |
|
21.5. Personalized Ranking for Recommender Systems
|
Intern |
doFollow |
|
21.6. Neural Collaborative Filtering for Personalized Ranking
|
Intern |
doFollow |
|
21.7. Sequence-Aware Recommender Systems
|
Intern |
doFollow |
|
21.8. Feature-Rich Recommender Systems
|
Intern |
doFollow |
|
21.9. Factorization Machines
|
Intern |
doFollow |
|
21.10. Deep Factorization Machines
|
Intern |
doFollow |
|
22. Appendix: Mathematics for Deep Learning
|
Intern |
doFollow |
|
22.1. Geometry and Linear Algebraic Operations
|
Intern |
doFollow |
|
22.2. Eigendecompositions
|
Intern |
doFollow |
|
22.3. Single Variable Calculus
|
Intern |
doFollow |
|
22.4. Multivariable Calculus
|
Intern |
doFollow |
|
22.5. Integral Calculus
|
Intern |
doFollow |
|
22.6. Random Variables
|
Intern |
doFollow |
|
22.7. Maximum Likelihood
|
Intern |
doFollow |
|
22.8. Distributions
|
Intern |
doFollow |
|
22.9. Naive Bayes
|
Intern |
doFollow |
|
22.10. Statistics
|
Intern |
doFollow |
|
22.11. Information Theory
|
Intern |
doFollow |
|
23. Appendix: Tools for Deep Learning
|
Intern |
doFollow |
|
23.1. Using Jupyter Notebooks
|
Intern |
doFollow |
|
23.2. Using Amazon SageMaker
|
Intern |
doFollow |
|
23.3. Using AWS EC2 Instances
|
Intern |
doFollow |
|
23.4. Using Google Colab
|
Intern |
doFollow |
|
23.5. Selecting Servers and GPUs
|
Intern |
doFollow |
|
23.6. Contributing to This Book
|
Intern |
doFollow |
|
23.7. Utility Functions and Classes
|
Intern |
doFollow |
|
References
|
Intern |
doFollow |
|
¶
|
Intern |
doFollow |
|
Star
|
Extern |
doFollow |
|
Follow @D2L_ai
|
Extern |
doFollow |
|
order
|
Extern |
doFollow |
|
best seller
|
Extern |
doFollow |
|
us
|
Extern |
doFollow |
|
Portuguese
|
Extern |
doFollow |
|
Turkish
|
Extern |
doFollow |
|
Vietnamese
|
Extern |
doFollow |
|
Korean
|
Extern |
doFollow |
|
Japanese
|
Extern |
doFollow |
|
run this book
|
Intern |
doFollow |
|
SageMaker Studio Lab
|
Extern |
doFollow |
|
syllabus page
|
Extern |
doFollow |
|
Aston Zhang
|
Extern |
doFollow |
|
Zack C. Lipton
|
Extern |
doFollow |
|
Mu Li
|
Extern |
doFollow |
|
Alex J. Smola
|
Extern |
doFollow |
|
Pratik Chaudhari
|
Extern |
doFollow |
|
Rasool Fakoor
|
Extern |
doFollow |
|
Kavosh Asadi
|
Extern |
doFollow |
|
Andrew Gordon Wilson
|
Extern |
doFollow |
|
Aaron Klein
|
Extern |
doFollow |
|
Matthias Seeger
|
Extern |
doFollow |
|
Cedric Archambeau
|
Extern |
doFollow |
|
Shuai Zhang
|
Extern |
doFollow |
|
Yi Tay
|
Extern |
doFollow |
|
Brent Werness
|
Extern |
doFollow |
|
Rachel Hu
|
Extern |
doFollow |
|
Anirudh Dagar
|
Extern |
doFollow |
|
Yuan Tang
|
Extern |
doFollow |
|
community contributors
|
Extern |
doFollow |
|
Contribute to the book
|
Extern |
doFollow |
|
community
|
Extern |
doFollow |
|
1.1. A Motivating Example
|
Intern |
doFollow |
|
1.2. Key Components
|
Intern |
doFollow |
|
1.3. Kinds of Machine Learning Problems
|
Intern |
doFollow |
|
1.4. Roots
|
Intern |
doFollow |
|
1.5. The Road to Deep Learning
|
Intern |
doFollow |
|
1.6. Success Stories
|
Intern |
doFollow |
|
1.7. The Essence of Deep Learning
|
Intern |
doFollow |
|
1.8. Summary
|
Intern |
doFollow |
|
1.9. Exercises
|
Intern |
doFollow |