Why is Math needed for Machine Learning ?
Originally written on:- 22nd September, 2019.
Math is everything..!!
“Lesson one starts simple gotta get that data
Don’t even pick out the thetas until we get that data
And if we open the file, it might look like a haze,
But if we keep it algorithmic we can set it ablaze
Hello! Algebra ops its input times weight
It’s a Matrix of nums, add a bias, activate
And when it learns from the data it’ll tell us things
Like how to cure a disease or how to spend our money
Or even, generate a game you won’t believe,
It’s like build train test done, NLP
Damn!
Bad drivers let me show you my friend
Cause you can drive it hands free and still have so much to spend
Make the Data Lit, Make the Data Lit
Make the Data, Make the Data Lit, Make the Data Lit, Learn from
it”
-Siraj Raval, 2019.
A little Rant
You have
to understand a crucial point here, there is no Artificial Intelligence without
math. Math is everything. It is all about the applications of algorithms and
logic to problems. Real world problems. I hate to break it to you, but one
should never call themselves data scientists if you are ignoring math. Now, one
of the things I need you, the reader to understand is that with the introduction
of advanced libraries, advanced tools, it can be very easy to ignore the
in-depth mathematics that is being performed right under your nose.(Auto-ML).
The ones that remain hidden under the commands, the libraries , the tools etc.
But since we are talking about becoming actual data scientists and researchers,
math will remain essential forever .
Let me give you an example. Very few understand the in-depth mathematics
behind computational graphs which by the way is one of the basic things you
need to learn during backpropagation and ANNs. Why you ask. Because you are
importing tensorflow as tf every time. A lot of things go on
behind the scenes that people seem to ignore. But tensorflow isn’t at fault
here at all and you shouldn’t stop using it.
Once
again, I am not against using tensorflow, torch, pytorch, keras etc. I go back
and forth all the time and I can understand that ALL the in-depth math is not
possible to remember while you are doing a project, but there are many that
skip everything, including the minimum effort that needs to be given to math
before even attempting anything. I am not very sympathetic towards them as you
may have sensed. Also, I am not asking you to perform backpropagation on a
piece of paper manually and then code it out on MATLAB. I am only asking you to
keep the concepts at your fingertips as much as possible. MathOverflow.net , Math.stackexchange.com and Reddit threads are
always there to help you . As a data scientist, you have to deal with math
every day. You will learn, you will fail and the cycle continues. Happens to me
daily and I don’t even consider myself to be a data scientist right now. That
will still take at least a couple decades of hard work. I am just an AI
enthusiast.
PS:- I still Google stuff about eigenvalues, eigenvectors
, eigendecomposition etc.
Math Needed for Machine Learning
The backbone of ML is formed using these
four. Linear Algebra, Probability, statistics, Calculus. I already have a full
curriculum on GitHub called “Mathematics for AI in 5 weeks”. Check here.
The
amount of math you’ll need depends on the role you are playing. First, every
data scientist needs to know statistics and probability theory. Machine
Learning is all about statistics. There’s a guide for that:
What
about other types of math? Well, here’s where the answer is more nuanced… it
depends on how much original machine learning
research you’ll be doing. For your reference. Introduction to
Statistical Machine Learning is an excellent e-book (with free PDF version),
the example is the use of R language, this book covers a wider range of topics,
when you make more progress in machine learning This is a valuable
tool. Although , I have provided the GitHub link to my own Mathematics
curriculum, the basics are really important.
Why is statistics so important for data science ?
In
a blog post titled, Why I Majored in
Statistics for a Career in Artificial Intelligence, its author, Roman Ring admitted that
there is indeed a dearth of undergraduates programme exclusively for AI and
tells his readers how he fared in the industry despite obtaining a Master’s
degree in Statistics.
Ring
says that there is a big misconception surrounding the subject and assures his
readers that “deep learning, is just applied statistics in disguise.”
“It
is still very rare to have an undergraduate degree fully dedicated to AI, so
people opt for what they perceive to be the next best thing – Computer Science
(CS),” he said. “But I believe there is a better alternative: statistics. Many
ML techniques and algorithms are either fully borrowed from or heavily rely on
the theory from statistics,” Ring adds.
To
land in an AI/ML job with a statistics degree might sound absurd at first,
according to Ring, that was precisely how his peers reacted to his choice of
majoring in the subject. He goes on to say that while maths students saw
statistics as a not being “pure” enough, the CS students thought that it was
not “engineering oriented.”
However,
Ring admits that both subjects have their own applications when it comes to AI
and ML, for instance, continuity and differentiability in maths are
widely used in AI/ML algorithms. He further goes on to explain how
Probability Theory and Statistic, which to a certain degree is taught in
CS, numerical method, Matrix Calculus, Monte Carlo Methods, Stochastic
Processes, Data Analysis and Experimental Design can enhance your knowledge
about AI/ML.
Importance
Of Statistics
To
the question of ‘Is statistics a prerequisite
for machine learning‘,
a Quora user said that it is important to learn the subject to interpret the
results of logistic regression or you will end up being baffled by how bad your
models perform due to non-normalised predictors. “If this is the case, probably
the person will be using the wrong splitting criterion for their tree-based
models”, he explains. “Absolutely anybody can take a dataset, throw a machine
learning algorithm at it, and then proclaim “I did machine learning,” he
commented.
ML
in the workplace
Application-Heavy Machine Learning Positions
In
practice, especially in entry-level data science roles, you’ll often be using
out-of-the-box ML implementations. There are robust libraries of common libraries
in many programming languages. You don’t need to reinvent the wheel.
Even so,
interviewers may still test your basic linear algebra and
multivariable calculus. Why do they do this?
Well, at
some point, your team may still need to build custom implementations of ML
algorithms. For example, you may need to adapt one to your tech stack or to
expand its base functionality. To do so, you must be able to peel back ML
algorithms and work with their innards.
R&D-Heavy Machine Learning Positions
Other roles
need much more original ML research and development. You may need to translate
algorithms from academic papers into working code. Or, you might research
enhancements based on your business’s unique challenges.
In other
words, you’ll be implementing algorithms from scratch much more often.
For these
positions, mastery of both linear algebra and multivariable
calculus is a must.
The Best Way to Learn Math for ML
The
self-starter way to learning math for data science is to learn by
“doing shit.” So we’re going to tackle linear algebra and calculus by
using them in real algorithms!
Even so,
you’ll want to learn or review the underlying theory up front. Read the
textbook , but you’ll want to learn the key concepts first.
For application-heavy roles...
For R&D-heavy roles...
MIT OpenCourseWare offers a
rigorous classes. The video lectures and course materials are all included. Do
the tutorials on Graph theory, Linear Algebra, Mathematics for AI tutorials if
possible by MIT Opencourseware.
Why
Worry About The Maths?
There
are many reasons why the mathematics of Machine Learning is important and I
will highlight some of them below:
1. Selecting
the right algorithm which includes giving considerations to accuracy, training
time, model complexity, number of parameters and number of features.
2. Choosing
parameter settings and validation strategies.
3. Identifying
underfitting and overfitting by understanding the Bias-Variance tradeoff.
4. Estimating
the right confidence interval and uncertainty.
Math you do need for ML (Your walk in the
door checklist)
What
solid mathematical-related skills do you need to walk in the door?
This
varies on what company you're working for and what kind of position you're
applying for. If you want to get into machine learning
theory, you're going to need a fair bit of mathematics (like PCA and
calculus). However, if you're wanting to snag a junior analyst job
and dip your toes into becoming a machine learning practitioner, you can get by
with a few basics. At a bare minimum, you should know:
§ Data manipulation skills: basic algebra,
like the meaning of variables (x,y,z) basic function types (e.g.
exponential, logarithmic, or linear) and function notation (e.g. f(x) = 8 +
783^x 8x).
§ Data visualization skills: Ability to create
scatter plots, histograms and x-y charts (all of which are covered in basic
high school math). You'll definitely need to know how to interpret a
scatter chart, but you'll never need to sketch a histogram by hand.
§ Data analysis skills: basic descriptive
statistics terms like mean, mode, median, standard deviation and variance.
Summation notation is extremely important, as it appears frequently in machine
learning. Sharp Sight calls data analysis
the "real prerequisite for machine
learning."
This is
an absolute basic minimum. The more you know, the more you'll be able to jump
into a project. While you'll probably have no trouble cleaning and performing
basic data analytics, you might run into difficulties with improving models and
making predictions unless you've got higher level math skills.
Math that WILL be VERY useful down the
road
As a
junior data analyst, you probably won't be involved in many machine learning
projects. But the further down your career path you go, the more math you'll
need. According to the University of Warwick, a strong mathematics background
gives you more than just math knowledge, it enhances your ability to think
clearly, follow complex reasoning, and pay attention to detail. Some areas that
are recommended as helpful include and as I have mentioned previously:
§ Calculus: functions, derivatives,
multi-variable calculus, differential equations.
§ Linear algebra: vectors, matrices,
eigenvalues/eigenvectors, principle component analysis, singular value
decomposition.
§ Statistics: statistics is really at
the heart of data science and machine learning. A course in statistics
gives you much more than just "statistics." You'll learn probability
and data analysis too.
§ Logic. Rather than memorizing all
those properties and acronyms from high school algebra, a much better use of
your valuable learning time is to stretch your brain and learn logic. If you
can think logically, you can learn anything at a later date. They don't
actually teach "logic" in high school, and the college
"logic" class is probably overkill. But you can teach yourself.
Summary
To summarize machine learning consists mainly of
statistics
calculus linear algebra and probability theory
Calculus tells us how to optimize linear algebra makes executing
algorithms feasible on massive data sets.
Probability helps predict the likelihood of a certain outcome and
statistics tells us what our goal is.
Certainly
having a strong background in mathematics will make it easier to understand
machine learning at a conceptual level.
When
someone introduces you to the inference function in logistic regression, you’ll say, “Hey, that’s
just linear algebra!”
Then
to optimize over that linear model they’ll show you gradient descent, and again you’ll note, “Well,
that’s just calculus!”
Just
more calculus. Literally. Not harder, just more.
Supposing
you did well in those undergrad classes, machine learning should come easy
enough at first. At least, until it comes time to actually build a
model in the real world.
One
more thing that I would like to add is, Math becomes more and more intuitive as
you progress in your journey of ML,DL, Computer vision, NLP etc. You have to
have an intuition of what is going on behind the scenes, even though it is not
possible to remember everything. This is necessary as a data scientist and as a
ML engineer. To develop such as intuition, you need visualization skills. You
need to be able to visualize in your head what is going on. I’ll give you an
example:- given a Fourier series equation, how many can visualize what is going
on. What do we mean when we say that step function turns out to be equal to the
infinite sum and the coefficient of all the odd frequencies are rescaled by
4/pi. This needs visualization and developing such an intuition takes time. A
long time..!!. But it can be done. After all you are going to be a scientist. I would
suggest you to watch 3blue1brown on Youtube.
 |
| 3blue1Brown on Youtube |
So,
learn and fail. This cycle will continue. Don’t worry about being good in
mathematics. Only a few great minds in our human race can claim to be good, not
all. Have an intuition, keep your basics in check, develop the curiosity and
use Google to your full advantage. That’s it. Nothing more. Best of luck for
your journey in ML.
Credits:-
1)
Siraj Raval
Comments
Post a Comment