Machine learning Notes from IIIT Hyderabad:
The following notes represent a complete, stand alone interpretation of IIIT
Hyderabad’s machine learning course presented by Shridhar Khobe. The
topics covered are shown below, although for a more detailed summary see
lecture 19. The only content not covered here is the Octave/MATLAB
programming.
All diagrams are my own or are directly taken from the lectures, full credit to
Shridhar Khobe for a truly exceptional lecture course.
What are these notes?
Originally written as a way for me personally to help solidify and document
the concepts, these notes have grown into a reasonably complete block of
reference material spanning the course in its entirety in just over 40 000
words and a lot of diagrams! The target audience was originally me, but more
broadly, can be someone familiar with programming although no assumption
regarding statistics, calculus or linear algebra is made. We go from the very
introduction of machine learning to neural networks, recommender systems
and even pipeline design. The one thing I will say is that a lot of the later
topics build on those of earlier sections, so it's generally advisable to work
through in chronological order.
The notes were written in Evernote, and then exported to HTML
automatically. As a result I take no credit/blame for the web formatting.
,Content
• 01 and 02: Introduction, Regression Analysis and Gradient
Descent
• 03: Linear Algebra - review
• 04: Linear Regression with Multiple Variables
• 05: Octave[incomplete]
• 06: Logistic Regression
• 07: Regularization
• 08: Neural Networks – Representation
• 09: Neural Networks – Learning
• 10: Advice for applying machine learning techniques
• 11: Machine Learning System Design
• 12: Support Vector Machine
• 13: Clustering
• 14: Dimensionality Reduction
• 15: Anomaly Detection
• 16: Recommender Systems
• 17: Large Scale Machine Learning
• 18: Application Example - Photo OCR
• 19: Course Summary
2
,01 and 02: Introduction, Regression Analysis, and
Gradient Descent
Introduction to the course
• We will learn about
o State of the art
o How to do the implementation
• Applications of machine learning include
o Search
o Photo tagging
o Spam filters
• The AI dream of building machines as intelligent as humans
o Many people believe best way to do that is mimic how humans
learn
• What the course covers
o Learn about state of the art algorithms
o But the algorithms and math alone are no good
o Need to know how to get these to work in problems
• Why is ML so prevalent?
o Grew out of AI
o Build intelligent machines
▪ You can program a machine how to do some simple thing
▪ For the most part hard-wiring AI is too difficult
▪ Best way to do it is to have some way for machines to learn
things themselves
▪ A mechanism for learning - if a machine can learn
from input then it does the hard work for you
Examples
• Database mining
o Machine learning has recently become so big party because of the
huge amount of data being generated
o Large datasets from growth of automation web
o Sources of data include
▪ Web data (click-stream or click through data)
▪ Mine to understand users better
▪ Huge segment of silicon valley
▪ Medical records
3
, ▪ Electronic records -> turn records in knowledges
▪ Biological data
▪ Gene sequences, ML algorithms give a
better understanding of human genome
▪ Engineering info
▪ Data from sensors, log reports, photos etc.
• Applications that we cannot program by hand
o Autonomous helicopter
o Handwriting recognition
▪ This is very inexpensive because when you write
an envelope, algorithms can automatically route envelopes
through the post
o Natural language processing (NLP)
▪ AI pertaining to language
o Computer vision
▪ AI pertaining vision
• Self customizing programs
o Netflix
o Amazon
o iTunes genius
o Take users info
▪ Learn based on your behavior
• Understand human learning and the brain
o If we can build systems that mimic (or try to mimic) how the brain
works, this may push our own understanding of the associated
neurobiology
What is machine learning?
• Here we...
o Define what it is
o When to use it
• Not a well defined definition
o Couple of examples of how people have tried to define it
• Arthur Samuel (1959)
o Machine learning: "Field of study that gives computers
the ability to learn without being explicitly programmed"
▪ Samuels wrote a checkers playing program
▪ Had the program play 10000 games against itself
▪ Work out which board positions were good and bad
depending on wins/losses
• Tom Michel (1999)
4
The following notes represent a complete, stand alone interpretation of IIIT
Hyderabad’s machine learning course presented by Shridhar Khobe. The
topics covered are shown below, although for a more detailed summary see
lecture 19. The only content not covered here is the Octave/MATLAB
programming.
All diagrams are my own or are directly taken from the lectures, full credit to
Shridhar Khobe for a truly exceptional lecture course.
What are these notes?
Originally written as a way for me personally to help solidify and document
the concepts, these notes have grown into a reasonably complete block of
reference material spanning the course in its entirety in just over 40 000
words and a lot of diagrams! The target audience was originally me, but more
broadly, can be someone familiar with programming although no assumption
regarding statistics, calculus or linear algebra is made. We go from the very
introduction of machine learning to neural networks, recommender systems
and even pipeline design. The one thing I will say is that a lot of the later
topics build on those of earlier sections, so it's generally advisable to work
through in chronological order.
The notes were written in Evernote, and then exported to HTML
automatically. As a result I take no credit/blame for the web formatting.
,Content
• 01 and 02: Introduction, Regression Analysis and Gradient
Descent
• 03: Linear Algebra - review
• 04: Linear Regression with Multiple Variables
• 05: Octave[incomplete]
• 06: Logistic Regression
• 07: Regularization
• 08: Neural Networks – Representation
• 09: Neural Networks – Learning
• 10: Advice for applying machine learning techniques
• 11: Machine Learning System Design
• 12: Support Vector Machine
• 13: Clustering
• 14: Dimensionality Reduction
• 15: Anomaly Detection
• 16: Recommender Systems
• 17: Large Scale Machine Learning
• 18: Application Example - Photo OCR
• 19: Course Summary
2
,01 and 02: Introduction, Regression Analysis, and
Gradient Descent
Introduction to the course
• We will learn about
o State of the art
o How to do the implementation
• Applications of machine learning include
o Search
o Photo tagging
o Spam filters
• The AI dream of building machines as intelligent as humans
o Many people believe best way to do that is mimic how humans
learn
• What the course covers
o Learn about state of the art algorithms
o But the algorithms and math alone are no good
o Need to know how to get these to work in problems
• Why is ML so prevalent?
o Grew out of AI
o Build intelligent machines
▪ You can program a machine how to do some simple thing
▪ For the most part hard-wiring AI is too difficult
▪ Best way to do it is to have some way for machines to learn
things themselves
▪ A mechanism for learning - if a machine can learn
from input then it does the hard work for you
Examples
• Database mining
o Machine learning has recently become so big party because of the
huge amount of data being generated
o Large datasets from growth of automation web
o Sources of data include
▪ Web data (click-stream or click through data)
▪ Mine to understand users better
▪ Huge segment of silicon valley
▪ Medical records
3
, ▪ Electronic records -> turn records in knowledges
▪ Biological data
▪ Gene sequences, ML algorithms give a
better understanding of human genome
▪ Engineering info
▪ Data from sensors, log reports, photos etc.
• Applications that we cannot program by hand
o Autonomous helicopter
o Handwriting recognition
▪ This is very inexpensive because when you write
an envelope, algorithms can automatically route envelopes
through the post
o Natural language processing (NLP)
▪ AI pertaining to language
o Computer vision
▪ AI pertaining vision
• Self customizing programs
o Netflix
o Amazon
o iTunes genius
o Take users info
▪ Learn based on your behavior
• Understand human learning and the brain
o If we can build systems that mimic (or try to mimic) how the brain
works, this may push our own understanding of the associated
neurobiology
What is machine learning?
• Here we...
o Define what it is
o When to use it
• Not a well defined definition
o Couple of examples of how people have tried to define it
• Arthur Samuel (1959)
o Machine learning: "Field of study that gives computers
the ability to learn without being explicitly programmed"
▪ Samuels wrote a checkers playing program
▪ Had the program play 10000 games against itself
▪ Work out which board positions were good and bad
depending on wins/losses
• Tom Michel (1999)
4
