Machine Learning
Tools
- Scikit-multilearn
- scikit-multiflow: for multi-output/multi-label and stream data
- scikit-plot: plot models analysis
- yellowbrick various model visualizations
- Scikit-lego: interesting prebuild pipelines and transformers for sklearn
- List of various scikits
- Information Theoretical Estimators (ITE) in Python Paper
- HYPERBOLIC DEEP LEARNING - A nascent and promising field
- Optimial Transport
- EMLP: Equivariant Representations layers
- BOLT: 10x faster matrix and vector operations
- Macarico: learning to search
- A Collection of Conference & School Notes in Machine Learning
- Python Toolkit of Statistics for Pairwise Interactions (pyspi)
- pulearn: Positive-unlabeled learning with Python
ML Engineering
- https://github.com/EthicalML/awesome-production-machine-learning
- Machine Learning Systems Design
- https://www.oreilly.com/radar/what-you-need-to-know-about-product-management-for-ai/
- https://github.com/Intellicode/ml-engineering-resources
- Top 30 ML in Production Resources
- 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com
- Rules of Machine Learning: Best Practices for ML Engineering On google blog
- Troubleshooting Deep Neural Networks: A Field Guide to Fixing Your Model
- Lecture 6: MLOps Infrastructure & Tooling
- Made with ML
- Machine Learning Systems Design
- Monitering model/data Drift
- Machine Learning in Science and Industry slides
- Machine learning in production
- How to navigate through the ML research information flood
- SRE for ML: The First 10 Years and the Next 10
- ML Crash Course from Google which included practical aspects of ML
On-Device ML
Model Speedup
- Scalable Vector Search - Includes Locally-Adaptive Quantization for Streaming Vector Search
- DeepSpeed Compression: A composable library for extreme compression and zero-cost quantization
Theory
- The Theory Behind Overfitting, Cross Validation, Regularization, Bagging, and Boosting: Tutorial
- Blog: Shape matching with time series data
- Notes on information theory - Information Theory for Intelligent People - Simon DeDeo
- Notes on Bayesian reasoning - Bayesian Reasoning for Intelligent People - Simon DeDeo
- Mathematics of Machine Learning book
- ALGORITHMS FOR DECISION MAKING
- Linear Algebra Done Right - Videos and Slides
- Old and New Matrix Algebra Useful for Statistics
- Linear Algebra by Jim Hefferon
- Intro to linear algebra in Numpy
- DATASET SHIFT IN MACHINE LEARNING
- Probability Divergences and Generative Models Video
- Generalized energy-based models Video
- Kernel Methods
- Non-Euclidean Methods in Machine Learning - “Data has Shape and Shape has Meaning” - Stanford Course
- THE SOCIETY OF MIND by Marvin Minsky MIT Course
- SVD and its applications (PDF)
Optimization
- Jupyter notebooks on various optimization topics
- Notebook on Optimal transport
- Numerical Tours - Python
- Algebraic and Topological Tools in Linear Optimization
- Algebraic, Geometric, and Topological Methods in Optimization
- MOSEC Cookbook on Convex Optimization - Handy cheatsheet
- Energy Based Machine Learning video tutorial from IBM research
- Google OR Tools for Operations Research optimization problems
- Computational Control Theory - Princton course
- Proximal Operators implementations
- CMU series of tutorials for conducting computational experiments with optimization solvers
- Computational Optimal Transport
Structured prediction
Graphical models and variational inference
- Graphical Models, Exponential Families, and Variational Inference: https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf
- Probabilistic Graphical Models stanford course: https://cs228.stanford.edu/ [Course notes]
- Differential Inference: A Criminally Underused Tool
- Probabilistic Machine Learning: Advanced Topics
Extreme classification
Online learning
- Online Learning, Interactive Machine Learning, and Learning from Human Feedback
- Lecture notes on Online Learning
- Machine Learning for Data Streams using MOA
- Introduction to Online Convex Optimization - Website
- Prediction, Learning, and Games
- Online learning, boosting and Games - Prof. Yoav Freund
- Python implementation of concept-drift algorithms for Online Learning
- ICML 2020 TUTORIAL ON PARAMETER-FREE ONLINE OPTIMIZATION
- A Modern Introduction to Online Learning
Positive Unlabelled (PU) learning
- An introductory tutorial to the "Learning from Positive and Unlabeled Data" field videos
- PU Learning - Learning from Positive and Unlabeled Examples - UIC Resources
Concept Drift
Domain Adaptation
Bandit Algorithms
- A blog and book on bandit algorithms by Tor Lattimore and Csaba Szepesvari
- I'm a bandit - Random topics in optimization, probability, and statistics. By Sébastien Bubeck
- A tutorial on Thompson Sampling
Continual learning
- Meta-Learning Representations for Continual Learning
- Distillation and Incremental Classifier Learning
- Awesome Continual Learning
- Continual Unsupervised Representation Learning
- Continual Learning Papers Library by ContinualAI
- Sequoia A Playground for research at the intersection of Continual, Reinforcement, and Self-Supervised Learning.
- Continual Learning: Towards “Broad” AI Winter 2021, A Course offered by the Université de Montréal
- A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning - Martin Mundt
- Tutorial 2: Out-of-distribution (OOD) Learning Week 3, Day 4: Continual Learning
- CVPR21 Continual Learning Challenge
- Avalanche: an End-to-End Library for Continual Learning
Multi task learning
Learning to Rank
Neural Ordinary Differential Equations
- https://github.com/msurtsukov/neural-ode
- https://jontysinai.github.io/jekyll/update/2019/01/18/understanding-neural-odes.html
- Augumented Neural ODE: https://arxiv.org/abs/1904.01681
Active learning
- Importance weighted active learning for streams
- Summary of active learning
- Good survey of potential issues with active learning
- Human-in-the-Loop Machine Learning by Robert Munro - Active learning in pytorch from book Cheatsheat on active learning
- DUALIST: Utility for Active Learning with Instances and Semantic Terms
- CleanLab - The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels
Multiple instance learning
- Good survey: https://arxiv.org/abs/1612.03365
Self-supervised learning
- Slides by Andrew Zisserman (DeepMind) with focus on Images and Videos: https://project.inria.fr/paiss/files/2018/07/zisserman-self-supervised.pdf
Reinforcement learning
- Course on Online Learning, Bandits, Reinforcement and Imitation Learning: https://sites.google.com/view/cs-159-spring-2018/home
- Python code examples for the book: https://github.com/ShangtongZhang/reinforcement-learning-an-introduction
- Approximate Dynamic Programming
- Multiagent Systems - Algorithmic, Game-Theoretic, and Logical Foundations
- Book on Distributional Reinforcement Learning
Transfer Learning
- Good overview of all types of transfer learning approaches (in Chinese): https://github.com/jindongwang/transferlearning
Multimodal learning
Interpretability
- ML Interpratation in python
- Explanatory Model Analysis
- Interesting resources related to XAI (Explainable Artificial Intelligence)
- Great blogs on visualization of ML models and topics
Control Theory
Graph Neural Networks
- A Complete Beginner's Guide to G-Invariant Neural Networks
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
- Non-Euclidean Methods in Machine Learning
- Introduction to Graph Neural Nets with JAX/jraph
- How AWS uses graph neural networks to meet customer needs
Applications
- List of applications across various industries
- Visual explanations of ML algorithms
- Javascript animated examples from Artificial Intelligence - A Modern Approach by Stuart Russell and Peter Norvig: http://aimacode.github.io/aima-javascript/
- Adaptive Language Models in JavaScript