Machine Learning Foundations: Algebra, Calculus, Statistics & Computer Science

Machine Learning Foundations

This repo is home to the code that accompanies Jon Krohn’s Machine Learning Foundations course, which provides a comprehensive overview of all of the subjects – across mathematics, statistics, and computer science – that underlie contemporary machine learning approaches, including deep learning and other artificial intelligence techniques.

There are a total of eight subjects covered in the course, organized into four couplets:

• Linear Algebra
  • 1: Intro to Linear Algebra
  • 2: Linear Algebra II: Matrix Operations
• Calculus
  • 3: Calculus I: Limits & Derivatives
  • 4: Calculus II: Partial Derivatives & Integrals
• Statistics
  • 5: Probability & Information Theory
  • 6: Intro to Statistics
• Computer Science
  • 7: Algorithms & Data Structures
  • 8: Optimization

Later subjects build upon content from earlier subjects, so the recommended approach is to progress through the eight subjects in the order provided. That said, you’re welcome to pick and choose individual subjects based on your interest or existing familiarity with the material.

Where and When

The eight ML Foundations subjects were initially offered by Jon Krohn as eight 4-hour live online trainings in the O’Reilly learning platform from May 2020 through September 2020. The content is now being rolled out via several different platforms to suit your preferred mode of learning:

• YouTube: Free videos via the ML Foundations playlist (regular ongoing uploads since July 2020)
• Udemy: First two hours went live in October 2020 as a free course (Later in 2020, we’ll add all the remaining linear algebra content, which will put us over Udemy’s two-hour maximum for free courses. From that point on, the course will be paid in Udemy, but anyone who enrolled while the course was free will automatically get unlimited access to all of the paid-tier content. No strings attached!)
• Open Data Science Conference: Live and on-demand trainings in the ODSC AI+ Platform (starting Nov 2020)
• O’Reilly: Video courses and another round of live training in the online learning platform (starting early 2021)
• Book (chapter drafts to begin appearing in 2021)

To stay informed of future live training sessions and new video releases consider:

• Signing up for Jon Krohn’s email newsletter via his homepage
• Or, remembering to check for updates on his talks page

(Note that the paid video options – e.g., Udemy, ODSC, and O’Reilly – each contain exclusive exercises and comprehensive solution walk-throughs that are not available on YouTube. In the future, some of the paid options will also include interactive testing and the awarding of credentials for successful course completion.)

Notebooks

All code is provided within Jupyter notebooks in this directory.

These notebooks are intended for use within the (free) Colab cloud environment and that is the only environment currently actively supported. That said, if you’re keen to run the notebooks locally, you’re welcome to do so (for the Jupyter and Docker uninitiated, check out the installation instructions here).

Pedagogical Approach

As with other materials created by Jon Krohn (such as the book Deep Learning Illustrated and his 18-hour video series Deep Learning with TensorFlow, Keras, and PyTorch), the content in the series is brought to life through the combination of:

• Vivid full-color illustrations
• Straightforward examples of Python code within hands-on Jupyter notebooks
• Comprehension exercises with fully-worked solutions

Why Study the Foundations of Machine Learning?

The purpose of this series it to provide you with a practical, functional understanding of the content covered. Context will be given for each topic, highlighting its relevance to machine learning. You will be better-positioned to understand cutting-edge machine learning papers and you will be provided with resources for digging even deeper into topics that pique your curiosity.

The content in this series may be particularly relevant for you if:

• You use high-level software libraries (e.g., scikit-learn, Keras, TensorFlow, PyTorch) to train or deploy machine learning algorithms, and would now like to understand the fundamentals underlying the abstractions, enabling you to expand your capabilities
• You’re a software developer who would like to develop a firm foundation for the deployment of machine learning algorithms into production systems
• You’re a data scientist who would like to reinforce your understanding of the subjects at the core of your professional discipline
• You’re a data analyst or A.I. enthusiast who would like to become a data scientist or data/ML engineer, and so you’re keen to deeply understand the field you’re entering from the ground up (very wise of you!)

Prerequisities

Programming: All code demos will be in Python so experience with it or another object-oriented programming language would be helpful for following along with the code examples.

Mathematics: Familiarity with secondary school-level mathematics will make the class easier to follow along with. If you are comfortable dealing with quantitative information – such as understanding charts and rearranging simple equations – then you should be well-prepared to follow along with all of the mathematics.

Finally, here’s an illustration of Oboe, the Machine Learning Foundations mascot, created by the wonderful artist Aglaé Bassens: