March 18, 2021

969 words 5 mins read

ahmedbahaaeldin/From-0-to-Research-Scientist-resources-guide

ahmedbahaaeldin/From-0-to-Research-Scientist-resources-guide

Detailed and tailored guide for undergraduate students or anybody want to dig deep into the field of AI with solid foundation.

repo name ahmedbahaaeldin/From-0-to-Research-Scientist-resources-guide
repo link https://github.com/ahmedbahaaeldin/From-0-to-Research-Scientist-resources-guide
homepage
language
size (curr.) 188 kB
stars (curr.) 751
created 2021-03-16
license

From Zero to Research Scientist full resources guide.

Full Guide Version 0.0.1

Guide description

This guide is designated to anybody with basic programming knowledge or a computer science background interested in becoming a Research Scientist with :dart: on Deep Learning and NLP.

You can go Bottom-Up or Top-Down both works well and it is actually crucial to know which approach suites you the best. If you are okay with studying lots of mathematical concepts without application then use Bottom-Up. If you want to go hands-on first then use the Top-Down first.

Contents:

Mathematical Foundations:

The Mathematical Foundation part is for all Artificial Intelligence branches such as Machine Learning, Reinforcement Learning, Computer Vision and so on. AI is heavily math-theory based so a solid foundation is essential.

Linear Algebra

This branch of Math is crucial for understanding the mechanism of Neural Networks which are the norm for NLP methodologies in nowadays State-of-The-Art.

Resource Difficulty Relevance
MIT Gilbert Strang 2005 Linear Algebra πŸŽ₯ β˜…β˜…β˜†β˜†β˜† 100% 50% 75%
Linear Algebra 4th Edition by Friedberg πŸ“˜ β˜…β˜…β˜…β˜…β˜† 100%
Mathematics for Machine Learning Book: Chapter 2 πŸ“˜ β˜…β˜…β˜…β˜†β˜† 50% 75%
James Hamblin Awesome Lecture Series πŸŽ₯ β˜…β˜…β˜…β˜†β˜† 100%
3Blue1Brown Essence of Linear Algebra πŸŽ₯ β˜…β˜†β˜†β˜†β˜† 25% 100%
Mathematics For Machine Learning Specialization: Linear Algebra πŸŽ₯ β˜…β˜†β˜†β˜†β˜† 50% 100%
Matrix Methods for Linear Algebra for Gilber Strang UPDATED! πŸŽ₯ β˜…β˜…β˜…β˜†β˜† 100%

Probability

Most of Natural Language Processing and Machine Learning Algorithms are based on Probability theory. So this branch is extremely important for grasping how old methods work.

Resource Difficulty Relevance
Joe Blitzstein Harvard Probability and Statistics Course πŸŽ₯ β˜…β˜…β˜…β˜…β˜… 50% 25% 100%
MIT Probability Course 2011 Lecture videos πŸŽ₯ β˜…β˜…β˜…β˜†β˜† 50% 75%
MIT Probability Course 2018 short videos UPDATED! πŸŽ₯ β˜…β˜…β˜†β˜†β˜† 25% 25% 100%
Mathematics for Machine Learning Book: Chapter 6 πŸ“˜ β˜…β˜…β˜…β˜†β˜† 75% 25% 75%
Probalistic Graphical Models CMU Advanced πŸŽ₯ β˜…β˜…β˜…β˜…β˜… 50% 25% 100%
Probalistic Graphical Models Stanford Daphne Advanced πŸŽ₯ β˜…β˜…β˜…β˜…β˜… 50% 25% 25%
A First Course In Probability Book by Ross πŸ“˜ β˜…β˜…β˜…β˜…β˜† 50%
Joe Blitzstein Harvard Professor Probability Awesome Book πŸ“˜ β˜…β˜…β˜…β˜†β˜† 50%

Calculus

Resource Difficulty Relevance
Essence of Calculus by 3Blue1BrownπŸŽ₯ β˜…β˜…β˜†β˜†β˜† 75%
Single Variable Calculus MIT 2007πŸŽ₯ β˜…β˜…β˜…β˜…β˜† 75%
Strang’s Overview of CalculusπŸŽ₯ β˜…β˜…β˜…β˜…β˜† 100%
MultiVariable Calculus MIT 2007πŸŽ₯ β˜…β˜…β˜…β˜…β˜… 100%
Princeton University Multivariable Calculus 2013πŸŽ₯ β˜…β˜…β˜…β˜…β˜† 100%
Calculus Book by Stewart πŸ“˜ β˜…β˜…β˜…β˜…β˜† 100% 25%
Mathematics for Machine Learning Book: Chapter 5 πŸ“˜ β˜…β˜…β˜…β˜†β˜† 75% 50%

Optimization Theory

-Resource Difficulty Relevance
CMU optimization course 2018πŸŽ₯ β˜…β˜…β˜…β˜…β˜… 100% 25%
CMU Advanced optimization courseπŸŽ₯ β˜…β˜…β˜…β˜…β˜… 100%
Stanford Famous optimization course πŸŽ₯ β˜…β˜…β˜…β˜…β˜… 100%
Boyd Convex Optimization Book πŸ“• β˜…β˜…β˜…β˜…β˜… 100%

Machine Learning

Considered a fancy name for Statistical models where its main goal is to learn from data for several usages. It is considered highly recommended to master these statistical techniques before Research as most of research is inspired by most of the Algorithms.

Resource Difficulty Level
Mathematics for Machine Learning Part 2 πŸ“š Intermediate
Pattern Recognition and Machine LeanringπŸ“š Intermediate
Elements of Statistical Learning πŸ“š Advanced
Introduction to Statistical Learning πŸ“š Introductory
Machine Learning: A Probalisitic Perspective πŸ“š Advanced
Berkley CS188 Introduction to AI course πŸŽ₯ Introductory
MIT Classic AI course taught by Prof. Patrick H. Winston πŸŽ₯ Introductory
Stanford AI course 2018 πŸŽ₯ Intermediate
California Instuite of Technology Learning from Data course πŸŽ₯ Intermediate
CMU Machine Learning 2015 10-601 πŸŽ₯ Intermediate
CMU Statistical Machine Learning 10-702 πŸŽ₯ Intermediate
Information Theory, Pattern Recognition ML course 2012 πŸŽ₯ Intermediate
Large Scale Machine Learning Toronto University 2015 πŸŽ₯ Advanced
Algorithmic Aspects of Machine Learning MIT πŸŽ₯ Advanced
MIT Course 9.520 - Statistical Learning Theory and Applications, Fall 2015 πŸŽ₯ Advanced
Undergraduate Machine Learning Course University of British Columbia 2013 πŸŽ₯ Introductory

Deep Learning

One of the major breakthroughs in the field of intersection between Artificial Intelligence and Computer Science. It lead to countless advances in technology and considered the standard way to do Artificial Intelligence.

Resource Difficulty Level
Deep Learning Book by Ian Goodfellow πŸ“š Advanced
UCL Deepmind Deep Learning πŸŽ₯ Intermediate
Advanced Talks by Deep Learning Pioneers πŸŽ₯ Advanced
Stanford Autumn 2018 Deep Learning Lectures πŸŽ₯ Intermediate
FAU Deep Learning 2020 Series πŸŽ₯ Introductory
CMU Deep Learning course 2020 πŸŽ₯ Introductory
Stanford Convolutional Neural Network 2017 πŸŽ₯ Intermediate
Oxford Deep Learning Awesome Lectures 2015 πŸŽ₯ Intermediate
Stanford NLP with Deep Learning 2019 πŸŽ₯ Intermediate
Deep Learning from Probability and Statistics POV πŸŽ₯ Introductory
Advanced Deep Learning UCL 2017 course + Reinforcement Learning πŸŽ₯ Intermediate
Deep Learning UC Berkley 2020 Course πŸŽ₯ Introductory
NYU Deep Learning with Pytorch hands on πŸŽ₯ Intermediate
Classic Jeoffrey Hinton Old course OUTDATED πŸŽ₯ Intermediate
Pieter Abdeel Deep Unsupervised Learning πŸŽ₯ Advanced
Hugo Larochelle Deep Learning series πŸŽ₯ Introductory
Deep Learning Book Explanation Series πŸŽ₯ Advanced
Deep Learning Introduction by Durham University πŸŽ₯ Introductory
Fast.ai Practical Deep Learning πŸŽ₯ Introductory
Fast.ai Deep Learning From Foundations πŸŽ₯ Introductory

Reinforcement Learning

It is a sub-field of AI which focuses on learning by observation/rewards.

Resource Difficulty Level
Introduction to Reinforcement Learning πŸ“š Intermediate
David Silver Deep Mind Introductory Lectures πŸŽ₯ Introductory
Stanford 2018 cs234 Reinforcement LearningπŸŽ₯ Intermediate
Stanford 2019 cs330 Meta Learning advanced course πŸŽ₯ Advanced
Sergie Levine 2018 UC Berkley Lecture Videos πŸŽ₯ Advanced
Waterloo cs885 Reinforcement Learing πŸŽ₯ Advanced
Sergie Levine 2020 Deep Reinforcement Learning πŸŽ₯ Advanced
Reinforcement Learning Specialization Coursea GOLDEN coursesπŸŽ₯ (Though it is not free but you can apply for financial aid) Intermediate

Natural Language Processing

It is a sub-field of AI which focuses on the interpretation of Human Language.

Resource Difficulty Level
Jurafsky Speech and Language Processing πŸ“š Intermediate
Christopher Manning Foundations of Statistical NLPπŸ“š Advanced
Christopher Manning Introduction to Information RetrievalπŸ“š Advanced
cs224n Natural Language Processing with Deep Learning GOLDEN 2019πŸŽ₯ Intermediate
Oxford Natural Language Processing with Deep Learning 2017πŸŽ₯ Intermediate
Michigan Introduction to NLPπŸŽ₯ Introductory
cs224u Natural Language Understanding 2019 πŸŽ₯ Intermediate
cmu 2021 Neural Nets for NLP 2021πŸŽ₯ Intermediate
Jurafsky and Manning Introduction to Natural Language ProcessingπŸŽ₯ Introductory

Must Read NLP Papers:

In this section, I am going to list the most influential papers that help people who want to dig deeper into the research world of NLP to catch up.

Paper Comment

TODO

comments powered by Disqus