jeffheaton/t81_558_deep_learning
Washington University (in St. Louis) Course T81-558: Applications of Deep Neural Networks
| repo name | jeffheaton/t81_558_deep_learning |
| repo link | https://github.com/jeffheaton/t81_558_deep_learning |
| homepage | https://sites.wustl.edu/jeffheaton/t81-558/ |
| language | Jupyter Notebook |
| size (curr.) | 195840 kB |
| stars (curr.) | 1615 |
| created | 2016-07-31 |
| license | Apache License 2.0 |
T81 558:Applications of Deep Neural Networks
Washington University in St. Louis
Instructor: Jeff Heaton
The content of this course changes as technology evolves, to keep up to date with changes follow me on GitHub.
- Section 2. Spring 2020, Monday, 2:30 PM - 5:20 PM Online & Cupples I / 215
- Section 1. Spring 2020, Monday, 6:00 PM - 9:00 PM Online & Cupples I / 215
Course Description
Deep learning is a group of exciting new technologies for neural networks. Through a combination of advanced training techniques and neural network architectural components, it is now possible to create neural networks that can handle tabular data, images, text, and audio as both input and output. Deep learning allows a neural network to learn hierarchies of information in a way that is like the function of the human brain. This course will introduce the student to classic neural network structures, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Neural Networks (GRU), General Adversarial Networks (GAN) and reinforcement learning. Application of these architectures to computer vision, time series, security, natural language processing (NLP), and data generation will be covered. High Performance Computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Focus is primarily upon the application of deep learning to problems, with some introduction to mathematical foundations. Students will use the Python programming language to implement deep learning using Google TensorFlow and Keras. It is not necessary to know Python prior to this course; however, familiarity of at least one programming language is assumed. This course will be delivered in a hybrid format that includes both classroom and online instruction.
Objectives
- Explain how neural networks (deep and otherwise) compare to other machine learning models.
- Determine when a deep neural network would be a good choice for a particular problem.
- Demonstrate your understanding of the material through a final project uploaded to GitHub.
Syllabus
This syllabus presents the expected class schedule, due dates, and reading assignments. Download current syllabus.
| Module | Content |
|---|---|
| Module 1Meet on 01/13/2020 | Module 1: Python PreliminariesPart 1.1: Course OverviewPart 1.2: Introduction to PythonPart 1.3: Python Lists, Dictionaries, Sets & JSONPart 1.4: File HandlingPart 1.5: Functions, Lambdas, and Map/ReducePython PreliminariesWe will meet on campus this week! (first meeting) |
| Module 2Week of 01/27/2020 | Module 2: Python for Machine Learning Part 2.1: Introduction to Pandas for Deep LearningPart 2.2: Encoding Categorical Values in PandasPart 2.3: Grouping, Sorting, and ShufflingPart 2.4: Using Apply and Map in PandasPart 2.5: Feature Engineering in PadasModule 1 Assignment Due: 01/28/2020 |
| Module 3Week of 02/03/2020 | Module 3: TensorFlow and Keras for Neural NetworksPart 3.1: Deep Learning and Neural Network IntroductionPart 3.2: Introduction to Tensorflow & KerasPart 3.3: Saving and Loading a Keras Neural NetworkPart 3.4: Early Stopping in Keras to Prevent OverfittingPart 3.5: Extracting Keras Weights and Manual Neural Network CalculationModule 2: Assignment due: 02/04/2020 |
| Module 4Week of 02/10/2020 | Module 4: Training for Tabular DataPart 4.1: Encoding a Feature Vector for Keras Deep LearningPart 4.2: Keras Multiclass Classification for Deep Neural Networks with ROC and AUCPart 4.3: Keras Regression for Deep Neural Networks with RMSEPart 4.4: Backpropagation, Nesterov Momentum, and ADAM TrainingPart 4.5: Neural Network RMSE and Log Loss Error Calculation from ScratchModule 3 Assignment due: 02/11/2020 |
| Module 5Week of 02/17/2020 | Module 5: Regularization and DropoutPart 5.1: Introduction to Regularization: Ridge and LassoPart 5.2: Using K-Fold Cross Validation with KerasPart 5.3: Using L1 and L2 Regularization with Keras to Decrease OverfittingPart 5.4: Drop Out for Keras to Decrease OverfittingPart 5.5: Bootstrapping and Benchmarking HyperparametersModule 4 Assignment due: 02/18/2020 |
| Module 6Meet on 02/24/2020 | Module 6: CNN for Vision Part 6.1: Image Processing in PythonPart 6.2: Keras Neural Networks for MINST and Fashion MINSTPart 6.3: Implementing a ResNet in KerasPart 6.4: Computer Vision with OpenCVPart 6.5: Recognizing Multiple Images with DarknetModule 5 Assignment due: 02/25/2020We will meet on campus this week! (2nd Meeting) |
| Module 7Week of 03/02/2020 | Module 7: GANPart 7.1: Introduction to GANS for Image and Data GenerationPart 7.2: Implementing a GAN in KerasPart 7.3: Face Generation with StyleGAN and PythonPart 7.4: GANS for Semi-Supervised Learning in KerasPart 7.5: An Overview of GAN ResearchModule 6 Assignment due: 03/03/2020 |
| Module 8Week of 03/16/2020 | Module 8: KagglePart 8.1: Introduction to KagglePart 8.2: Building Ensembles with Scikit-Learn and KerasPart 8.3: How Should you Architect Your Keras Neural Network: HyperparametersPart 8.4: Bayesian Hyperparameter Optimization for KerasPart 8.5: Current Semester’s KaggleModule 7 Assignment due: 03/17/2020 |
| Module 9Meet on 03/23/2020 | Module 9: Transfer LearningPart 9.1: Introduction to Keras Transfer LearningPart 9.2: Popular Pretrained Neural Networks for Keras. Part 9.3: Transfer Learning for Computer Vision and KerasPart 9.4: Transfer Learning for Languages and KerasPart 9.5: Transfer Learning for Keras Feature EngineeringModule 8 Assignment due: 03/24/2020We will meet on campus this week! (3rd Meeting) |
| Module 10Week of 03/30/2020 | Module 10: Time Series in KerasPart 10.1: Time Series Data Encoding for Deep Learning, TensorFlow and KerasPart 10.2: Programming LSTM with Keras and TensorFlowPart 10.3: Image Captioning with Keras and TensorFlowPart 10.4: Temporal CNN in Keras and TensorFlowPart 10.5: Predicting the Stock Market with Keras and TensorFlowModule 9 Assignment due: 03/31/2020 |
| Module 11Week of 04/06/2020 | Module 11: Natural Language ProcessingPart 11.1: Getting Started with Spacy in PythonPart 11.2: Word2Vec and Text ClassificationPart 11.3: Natural Language Processing with Spacy and KerasPart 11.4: What are Embedding Layers in KerasPart 11.5: Learning English from Scratch with Keras and TensorFlowModule 10 Assignment due: 04/07/2020 |
| Module 12Week of 04/13/2020 | Module 12: Reinforcement LearningKaggle Assignment due: 04/13/2020 (approx 4-6PM, due to Kaggle GMT timezone)Part 12.1: Introduction to the OpenAI GymPart 12.2: Introduction to Q-Learning for KerasPart 12.3: Keras Q-Learning in the OpenAI GymPart 12.4: Atari Games with Keras Neural NetworksPart 12.5: How Alpha Zero used Reinforcement Learning to Master Chess |
| Module 13Meet on 04/20/2020 | Module 13: Deployment and MonitoringPart 13.1: Deploying a Model to AWSPart 13.2: Flask and Deep Learning Web ServicesPart 13.3: AI at the Edge: Using Keras on a Mobile DevicePart 13.4: When to Retrain Your Neural NetworkPart 13.5: Using a Keras Deep Neural Network with a Web ApplicationWe will meet on campus this week! (4th Meeting) |
| Module 14Week of 04/27/2020 | Module 14: Other Neural Network TechniquesPart 14.1: What is AutoMLPart 14.2: Using Denoising AutoEncoders in KerasPart 14.3: Training an Intrusion Detection System with KDD99Part 14.4: Anomaly Detection in KerasPart 14.5: New Technology in Deep LearningFinal Project due 05/04/2020 |
