WHAT IS MACHINE LEARNING?

What is machine learning?

Machine learning is a branch of artificial intelligence (AI) and computer science which focuses on the use of data and algorithms to imitate the way that humans learn, gradually improving its accuracy.

IBM has a rich history with machine learning. One of its own, Arthur Samuel, is credited for coining the term, “machine learning” with his research (PDF, 481 KB) (link resides outside IBM) around the game of checkers. Robert Nealey, the self-proclaimed checkers master, played the game on an IBM 7094 computer in 1962, and he lost to the computer. Compared to what can be done today, this feat almost seems trivial, but it’s considered a major milestone within the field of artificial intelligence. Over the next couple of decades, the technological developments around storage and processing power will enable some innovative products that we know and love today, such as Netflix’s recommendation engine or self-driving cars.

Machine learning is an important component of the growing field of data science. Through the use of statistical methods, algorithms are trained to make classifications or predictions, uncovering key insights within data mining projects. These insights subsequently drive decision making within applications and businesses, ideally impacting key growth metrics. As big data continues to expand and grow, the market demand for data scientists will increase, requiring them to assist in the identification of the most relevant business questions and subsequently the data to answer them.

Machine learning is the study of computer algorithms that can improve automatically through experience and by the use of data. It is seen as a part of artificial intelligence

Types of Machine Learning

There are 4 different types of machine learning algorithms.

Here they are:

1. Supervised Learning

The input data in supervised learning algorithms is labeled, and the output is known and accurate. In order to use this class of algorithms, you’d need a large amount of labeled data. And that may not always be an easy task.

Supervised algorithms fall into two categories – regression and classification. Each examines different sets of data.

Regression algorithms are the ones that make predictions and forecasts. Among others, these include weather forecasts, population growth, and life expectancy estimates, market forecasts.

Classification algorithms are used for diagnostics, identity fraud detection, customer retention, and as the name suggests – image classification.

2. Unsupervised Learning

It occurs when the input data is not labeled. They organize the data into structures of clusters. Thus any input data is immediately ready for analysis.

Since data is not labeled, there is no way of evaluating the accuracy of the outcome. That said, it is not accuracy that unsupervised algorithms are designed to pursue. The clusters that the algorithm creates are in no way familiar to the program. So the idea is to input data, analyze it, and group it into clusters.

Just like the supervised algorithms, their unsupervised cousins are divided into 2 categories – dimensionality reduction and clustering.

Clustering algorithms themselves are obviously a part of all this. It’s useful to group data into categories, so you don’t have to deal with every piece on its own. These algorithms are used above all for customer segmentation and targeted marketing.

Dimensionality reduction algorithms are used for structure discovery, big data visualization, feature elicitation, and meaningful compression. If clustering is one side of the coin, dimensionality reduction would be the other. By grouping data into clusters, the algorithms inevitably reduce the number of meaningful variables (dimensions) that describe the set of data.

Now, there is a class of machine learning algorithms that combines the previous 2 classes:

3. Semi-supervised Learning

It stands between supervised with labeled data, and unsupervised algorithms with unlabeled data.

Semi-supervised algorithms use a small amount of labeled data and a large amount of unlabeled data. This can lead to an improvement in learning accuracy.

It’s also a huge relief in terms of data gathering since it takes a good deal of resources to generate labeled data.

4. Reinforcement Learning

Unlike the 3 previous types, reinforcement algorithms choose an action based on a data set. Then they evaluate the outcome and change the strategy if needed.

In reinforcement algorithms, you create a network and a loop of actions, and that’s it. Without creating a database, you have a winner. Why?

Well, it was reinforcement algorithms that figured out the games of checkers, chess and Go.

Reinforcement learning work on the principle of trial and error. The system will be given a reward of some sort that will help it measure its success rate. In the case of games – the reward will be the scoreboard. Whenever the system wins a point, it evaluates that as a successful move and the status of this move becomes higher. It will keep repeating the loop until all its moves are successful.

And that’s how we have an algorithm that can master the game of chess in 4 hours.

How machine learning works

The learning system of machine learning algorithm are three main parts.

1. A Decision Process: In general, machine learning algorithms are used to make a prediction or classification. Based on some input data, which can be labelled or unlabeled, your algorithm will produce an estimate about a pattern in the data.
2. An Error Function: An error function serves to evaluate the prediction of the model. If there are known examples, an error function can make a comparison to assess the accuracy of the model.
3. An Model Optimization Process: If the model can fit better to the data points in the training set, then weights are adjusted to reduce the discrepancy between the known example and the model estimate. The algorithm will repeat this evaluate and optimize process, updating weights autonomously until a threshold of accuracy has been met.  

10 Machine Learning Methods that Every Data Scientist Should Know

Jump-start your data science skills

Machine learning is a hot topic in research and industry, with new methodologies developed all the time. The speed and complexity of the field makes keeping up with new techniques difficult even for experts — and potentially overwhelming for beginners.

To demystify machine learning and to offer a learning path for those who are new to the core concepts, let’s look at ten different methods, including simple descriptions, visualizations, and examples for each one.

A machine learning algorithm, also called model, is a mathematical expression that represents data in the context of a ­­­problem, often a business problem. The aim is to go from data to insight. For example, if an online retailer wants to anticipate sales for the next quarter, they might use a machine learning algorithm that predicts those sales based on past sales and other relevant data. Similarly, a windmill manufacturer might visually monitor important equipment and feed the video data through algorithms trained to identify dangerous cracks.

The ten methods described offer an overview — and a foundation you can build on as you hone your machine learning knowledge and skill:

1. Regression
2. Classification
3. Clustering
4. Dimensionality Reduction
5. Ensemble Methods
6. Neural Nets and Deep Learning
7. Transfer Learning
8. Reinforcement Learning
9. Natural Language Processing
10. Word Embeddings

One last thing before we jump in. Let’s distinguish between two general categories of machine learning: supervised and unsupervised. We apply supervised ML techniques when we have a piece of data that we want to predict or explain. We do so by using previous data of inputs and outputs to predict an output based on a new input. For example, you could use supervised ML techniques to help a service business that wants to predict the number of new users who will sign up for the service next month. By contrast, unsupervised ML looks at ways to relate and group data points without the use of a target variable to predict. In other words, it evaluates data in terms of traits and uses the traits to form clusters of items that are similar to one another. For example, you could use unsupervised learning techniques to help a retailer that wants to segment products with similar characteristics — without having to specify in advance which characteristics to use.

Regression

Regression methods fall within the category of supervised ML. They help to predict or explain a particular numerical value based on a set of prior data, for example predicting the price of a property based on previous pricing data for similar properties.

The simplest method is linear regression where we use the mathematical equation of the line (y = m * x + b) to model a data set. We train a linear regression model with many data pairs (x, y) by calculating the position and slope of a line that minimizes the total distance between all of the data points and the line. In other words, we calculate the slope (m) and the y-intercept (b) for a line that best approximates the observations in the data.

Let’s consider a more a concrete example of linear regression. I once used a linear regression to predict the energy consumption (in kWh) of certain buildings by gathering together the age of the building, number of stories, square feet and the number of plugged wall equipment. Since there were more than one input (age, square feet, etc…), I used a multi-variable linear regression. The principle was the same as a simple one-to-one linear regression, but in this case the “line” I created occurred in multi-dimensional space based on the number of variables.

The plot below shows how well the linear regression model fit the actual energy consumption of building. Now imagine that you have access to the characteristics of a building (age, square feet, etc…) but you don’t know the energy consumption. In this case, we can use the fitted line to approximate the energy consumption of the particular building.

Note that you can also use linear regression to estimate the weight of each factor that contributes to the final prediction of consumed energy. For example, once you have a formula, you can determine whether age, size, or height is most important.

Regression techniques run the gamut from simple (like linear regression) to complex (like regularized linear regression, polynomial regression, decision trees and random forest regressions, neural nets, among others). But don’t get bogged down: start by studying simple linear regression, master the techniques, and move on from there.

Classification

Another class of supervised ML, classification methods predict or explain a class value. For example, they can help predict whether or not an online customer will buy a product. The output can be yes or no: buyer or not buyer. But classification methods aren’t limited to two classes. For example, a classification method could help to assess whether a given image contains a car or a truck. In this case, the output will be 3 different values: 1) the image contains a car, 2) the image contains a truck, or 3) the image contains neither a car nor a truck.

The simplest classification algorithm is logistic regression — which makes it sounds like a regression method, but it’s not. Logistic regression estimates the probability of an occurrence of an event based on one or more inputs.

For instance, a logistic regression can take as inputs two exam scores for a student in order to estimate the probability that the student will get admitted to a particular college. Because the estimate is a probability, the output is a number between 0 and 1, where 1 represents complete certainty. For the student, if the estimated probability is greater than 0.5, then we predict that he or she will be admitted. If the estimated probabiliy is less than 0.5, we predict the he or she will be refused.

The chart below plots the scores of previous students along with whether they were admitted. Logistic regression allows us to draw a line that represents the decision boundary.

Because logistic regression is the simplest classification model, it’s a good place to start for classification. As you progress, you can dive into non-linear classifiers such as decision trees, random forests, support vector machines, and neural nets, among others.

Clustering

With clustering methods, we get into the category of unsupervised ML because their goal is to group or cluster observations that have similar characteristics. Clustering methods don’t use output information for training, but instead let the algorithm define the output. In clustering methods, we can only use visualizations to inspect the quality of the solution.

The most popular clustering method is K-Means, where “K” represents the number of clusters that the user chooses to create. (Note that there are various techniques for choosing the value of K, such as the elbow method.)

Roughly, what K-Means does with the data points:

1. Randomly chooses K centers within the data.
2. Assigns each data point to the closest of the randomly created centers.
3. Re-computes the center of each cluster.
4. If centers don’t change (or change very little), the process is finished. Otherwise, we return to step 2. (To prevent ending up in an infinite loop if the centers continue to change, set a maximum number of iterations in advance.)

The next plot applies K-Means to a data set of buildings. Each column in the plot indicates the efficiency for each building. The four measurements are related to air conditioning, plugged-in equipment (microwaves, refrigerators, etc…), domestic gas, and heating gas. We chose K=2 for clustering, which makes it easy to interpret one of the clusters as the group of efficient buildings and the other cluster as the group of inefficient buildings. To the left you see the location of the buildings and to right you see two of the four dimensions we used as inputs: plugged-in equipment and heating gas.

As you explore clustering, you’ll encounter very useful algorithms such as Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Mean Shift Clustering, Agglomerative Hierarchical Clustering, Expectation–Maximization Clustering using Gaussian Mixture Models, among others.

Dimensionality Reduction

As the name suggests, we use dimensionality reduction to remove the least important information (sometime redundant columns) from a data set. In practice, I often see data sets with hundreds or even thousands of columns (also called features), so reducing the total number is vital. For instance, images can include thousands of pixels, not all of which matter to your analysis. Or when testing microchips within the manufacturing process, you might have thousands of measurements and tests applied to every chip, many of which provide redundant information. In these cases, you need dimensionality reduction algorithms to make the data set manageable.

The most popular dimensionality reduction method is Principal Component Analysis (PCA), which reduces the dimension of the feature space by finding new vectors that maximize the linear variation of the data. PCA can reduce the dimension of the data dramatically and without losing too much information when the linear correlations of the data are strong. (And in fact you can also measure the actual extent of the information loss and adjust accordingly.)

Another popular method is t-Stochastic Neighbor Embedding (t-SNE), which does non-linear dimensionality reduction. People typically use t-SNE for data visualization, but you can also use it for machine learning tasks like reducing the feature space and clustering, to mention just a few.

The next plot shows an analysis of the MNIST database of handwritten digits. MNIST contains thousands of images of digits from 0 to 9, which researchers use to test their clustering and classification algorithms. Each row of the data set is a vectorized version of the original image (size 28 x 28 = 784) and a label for each image (zero, one, two, three, …, nine). Note that we’re therefore reducing the dimensionality from 784 (pixels) to 2 (dimensions in our visualization). Projecting to two dimensions allows us to visualize the high-dimensional original data set.

Ensemble Methods

Imagine you’ve decided to build a bicycle because you are not feeling happy with the options available in stores and online. You might begin by finding the best of each part you need. Once you assemble all these great parts, the resulting bike will outshine all the other options.

Ensemble methods use this same idea of combining several predictive models (supervised ML) to get higher quality predictions than each of the models could provide on its own. For example, the Random Forest algorithms is an ensemble method that combines many Decision Trees trained with different samples of the data sets. As a result, the quality of the predictions of a Random Forest is higher than the quality of the predictions estimated with a single Decision Tree.

Think of ensemble methods as a way to reduce the variance and bias of a single machine learning model. That’s important because any given model may be accurate under certain conditions but inaccurate under other conditions. With another model, the relative accuracy might be reversed. By combining the two models, the quality of the predictions is balanced out.

The great majority of top winners of Kaggle competitions use ensemble methods of some kind. The most popular ensemble algorithms are Random Forest, XGBoost and LightGBM.

Neural Networks and Deep Learning

In contrast to linear and logistic regressions which are considered linear models, the objective of neural networks is to capture non-linear patterns in data by adding layers of parameters to the model. In the image below, the simple neural net has three inputs, a single hidden layer with five parameters, and an output layer.

In fact, the structure of neural networks is flexible enough to build our well-known linear and logistic regression. The term Deep learning comes from a neural net with many hidden layers (see next Figure) and encapsulates a wide variety of architectures.

It’s especially difficult to keep up with developments in deep learning, in part because the research and industry communities have doubled down on their deep learning efforts, spawning whole new methodologies every day.

For the best performance, deep learning techniques require a lot of data — and a lot of compute power since the method is self-tuning many parameters within huge architectures. It quickly becomes clear why deep learning practitioners need very powerful computers enhanced with GPUs (graphical processing units).

In particular, deep learning techniques have been extremely successful in the areas of vision (image classification), text, audio and video. The most common software packages for deep learning are Tensorflow and PyTorch.

Transfer Learning

Let’s pretend that you’re a data scientist working in the retail industry. You’ve spent months training a high-quality model to classify images as shirts, t-shirts and polos. Your new task is to build a similar model to classify images of dresses as jeans, cargo, casual, and dress pants. Can you transfer the knowledge built into the first model and apply it to the second model? Yes, you can, using Transfer Learning.

Transfer Learning refers to re-using part of a previously trained neural net and adapting it to a new but similar task. Specifically, once you train a neural net using data for a task, you can transfer a fraction of the trained layers and combine them with a few new layers that you can train using the data of the new task. By adding a few layers, the new neural net can learn and adapt quickly to the new task.

The main advantage of transfer learning is that you need less data to train the neural net, which is particularly important because training for deep learning algorithms is expensive in terms of both time and money (computational resources) — and of course it’s often very difficult to find enough labeled data for the training.

Let’s return to our example and assume that for the shirt model you use a neural net with 20 hidden layers. After running a few experiments, you realize that you can transfer 18 of the shirt model layers and combine them with one new layer of parameters to train on the images of pants. The pants model would therefore have 19 hidden layers. The inputs and outputs of the two tasks are different but the re-usable layers may be summarizing information that is relevant to both, for example aspects of cloth.

Transfer learning has become more and more popular and there are now many solid pre-trained models available for common deep learning tasks like image and text classification.

Reinforcement Learning

Imagine a mouse in a maze trying to find hidden pieces of cheese. The more times we expose the mouse to the maze, the better it gets at finding the cheese. At first, the mouse might move randomly, but after some time, the mouse’s experience helps it realize which actions bring it closer to the cheese.

The process for the mouse mirrors what we do with Reinforcement Learning (RL) to train a system or a game. Generally speaking, RL is a machine learning method that helps an agent learn from experience. By recording actions and using a trial-and-error approach in a set environment, RL can maximize a cumulative reward. In our example, the mouse is the agent and the maze is the environment. The set of possible actions for the mouse are: move front, back, left or right. The reward is the cheese.

You can use RL when you have little to no historical data about a problem, because it doesn’t need information in advance (unlike traditional machine learning methods). In a RL framework, you learn from the data as you go. Not surprisingly, RL is especially successful with games, especially games of “perfect information” like chess and Go. With games, feedback from the agent and the environment comes quickly, allowing the model to learn fast. The downside of RL is that it can take a very long time to train if the problem is complex.

Just as IBM’s Deep Blue beat the best human chess player in 1997, AlphaGo, a RL-based algorithm, beat the best Go player in 2016. The current pioneers of RL are the teams at DeepMind in the UK

On April, 2019, the OpenAI Five team was the first AI to beat a world champion team of e-sport Dota 2, a very complex video game that the OpenAI Five team chose because there were no RL algorithms that were able to win it at the time. The same AI team that beat Dota 2’s champion human team also developed a robotic hand that can reorient a block.

You can tell that Reinforcement Learning is an especially powerful form of AI, and we’re sure to see more progress from these teams, but it’s also worth remembering the method’s limitations.

Natural Language Processing

A huge percentage of the world’s data and knowledge is in some form of human language. Can you imagine being able to read and comprehend thousands of books, articles and blogs in seconds? Obviously, computers can’t yet fully understand human text but we can train them to do certain tasks. For example, we can train our phones to autocomplete our text messages or to correct misspelled words. We can even teach a machine to have a simple conversation with a human.

Natural Language Processing (NLP) is not a machine learning method per se, but rather a widely used technique to prepare text for machine learning. Think of tons of text documents in a variety of formats (word, online blogs, ….). Most of these text documents will be full of typos, missing characters and other words that needed to be filtered out. At the moment, the most popular package for processing text is NLTK (Natural Language ToolKit), created by researchers at Stanford.

The simplest way to map text into a numerical representation is to compute the frequency of each word within each text document. Think of a matrix of integers where each row represents a text document and each column represents a word. This matrix representation of the word frequencies is commonly called Term Frequency Matrix (TFM). From there, we can create another popular matrix representation of a text document by dividing each entry on the matrix by a weight of how important each word is within the entire corpus of documents. We call this method Term Frequency Inverse Document Frequency (TFIDF) and it typically works better for machine learning tasks.

Word Embeddings

TFM and TFIDF are numerical representations of text documents that only consider frequency and weighted frequencies to represent text documents. By contrast, word embeddings can capture the context of a word in a document. With the word context, embeddings can quantify the similarity between words, which in turn allows us to do arithmetic with words.

Word2Vec is a method based on neural nets that maps words in a corpus to a numerical vector. We can then use these vectors to find synonyms, perform arithmetic operations with words, or to represent text documents (by taking the mean of all the word vectors in a document). For example, let’s assume that we use a sufficiently big corpus of text documents to estimate word embeddings. Let’s also assume that the words king, queen, man and woman are part of the corpus. Let say that vector(‘word’) is the numerical vector that represents the word ‘word’. To estimate vector(‘woman’), we can perform the arithmetic operation with vectors:

vector(‘king’) + vector(‘woman’) — vector(‘man’) ~ vector(‘queen’)

Word representations allow finding similarities between words by computing the cosine similarity between the vector representation of two words. The cosine similarity measures the angle between two vectors.

We compute word embeddings using machine learning methods, but that’s often a pre-step to applying a machine learning algorithm on top. For instance, suppose we have access to the tweets of several thousand Twitter users. Also suppose that we know which of these Twitter users bought a house. To predict the probability of a new Twitter user buying a house, we can combine Word2Vec with a logistic regression.

You can train word embeddings yourself or get a pre-trained (transfer learning) set of word vectors. To download pre-trained word vectors in 157 different languages, take a look at FastText.

Top 10 Machine Learning Algorithms

Now, before we start, let’s take a look at one of the core concepts in machine learning.  Regression, when it comes to machine learning regression algorithms, means the algorithm will try to establish a relationship between two variables.

There are many types of regression – linear, logistic, polynomial, ordinary least squares regression, and so on. Today we’ll just cover the first 2 types because otherwise this will be better published as a book, rather than an article.

As we’ll see in a moment, most of the top 10 algorithms are supervised learning algorithms and are best used with Python.

Here comes the top 10 machine learning algorithms list:

1. Linear Regression

It is among the most popular machine learning algorithms. It works to establish a relation between two variables by fitting a linear equation through the observed data.

In other words, this type of algorithms observes various features in order to come to a conclusion. If the number of variables is bigger than two – the algorithm will be called multiple linear regression.

Linear regression is also one of the supervised machine learning algorithms that work well in Python. It is a powerful statistical tool and can be applied for predicting consumer behavior, estimating forecasts, and evaluating trends. A company can benefit from conducting linear analysis and forecast the sales for a future period of time.

So, if we have two variables, one of them is explanatory, and the other is the dependent. The dependent variable represents the value you want to research or make a prediction about. The explanatory variable is independent. The dependent variable always counts on the explanatory.

The point of the linear machine learning is to see whether there is a significant relationship between the two variables and if there is, to see exactly what it represents.

Linear regression is considered a simple machine learning algorithm and is therefore popular among scientists.  

Now, there is linear regression, and there is logistic regression. Let’s have a look at the difference:

2. Logistic Regression

This is one of the basic machine learning algorithms. It is a binomial classifier that has only 2 states, or 2 values – to which you can assign the meanings of yes and no, true and false, on and off, or 1 and 0. This kind of algorithm classifies the input data as category or non-category. The input data is compressed and then analyzed.

Unlike linear regression, the logistic algorithms make predictions by using a nonlinear function. Logistic regression algorithms are used for classification and not for regression tasks. The “regression” in the name suggests that the algorithms use a linear model and incorporate it into the future space.

Logistic regression is a supervised machine learning algorithm, which, like linear regression, works well in Python. From a mathematical point of view, if the output data of research is expected to be in terms of sick/healthy or cancer/no cancer, then a logistic regression is the perfect algorithm to use.   

Unlike linear regression where the output data may have different values, logistic regression can have as output only 1 and 0.

There are 3 types of logistic regression, based on the categorical response. These are:

• Binary logistic regression – this is the most frequently used type if the output is some variety of “yes”/”no”.   
• Multi-nominal logistic regression – when there is the possibility of 3 or more answers with no ordering.
• Ordinal logistic regression – again 3 or more answers, but with ordering. For example, when the expected results are on a scale of 1 to 10.

Let’s see another great classifying algorithm:

3. Linear Discriminant Analysis

This method finds linear combinations of features, that separates different input data. The purpose of an LDA algorithm is to examine a dependable variable as a linear union of features. It is a great classification technique.

This algorithm examines the statistical qualities of the input data and makes calculations for each class. It measures the value of the class and then the variance among all classes.

During the process of modeling the differences among classes, the algorithm examines the input data according to independent variables.

The output data contains information about the class with the highest value. The Linear Discriminant Analysis algorithms work best for separating among known categories. When several factors need to be mathematically divided into categories, we use an LDA algorithm.

4. K-Nearest Neighbors

The kNN algorithm is one of the great machine learning algorithms for beginners. They make predictions based on old available data, in order to classify data into categories based on different characteristics.

It is on the supervised machine learning algorithm list, which is mostly used for classification. It stores available data and uses it to measure similarities in new cases.

The K in kNN is a parameter that denotes the number of nearest neighbors that will be included in the “majority voting process”. This way, each element’s neighbors “vote” to determine his class.  

One of the best ways to use the kNN algorithm is when you have a small, noise-free dataset and all data in labeled. The algorithm is not a quick one and doesn’t teach itself to recognize unclean data. When the dataset is larger, it is not a good idea to use kNN.

The kNN algorithm works like this: first, the parameter K is specified, after which the algorithm makes a list of entries, that is close to the new data sample. Then it finds the most common classification of the entries, and finally, it gives a classification to the new data input.  

In terms of real-life applications, kNN algorithms are used by search engines to establish whether search results are relevant to the query. They are the unsung hero that saves users time when they do a search.  

Next comes the Tree-Trio: Regression Trees, Random Forest, and AdaBoost.

Here we go:

5. Regression Trees (a.k.a. Decision Trees)

Yes, they are called trees, but since we’re talking about machine learning algorithms, imagine them with the roots on top and branches and leaves at the bottom.

Regression trees are a type of supervised learning algorithm, that – surprise, works well in Python. (Most ML algorithms do, by the way.)

These “trees” are also called decision trees and are used for predictive modeling. They require relatively little effort from the user in terms of the quantity of input data.

Their representation is a binary tree and they solve classification problems. As the name suggests, this type of algorithm uses a tree-like model of decisions. They perform variable screening or feature selection. The input data can be both numerical and categorical.

Translation, please!

Sure. Any time you make a decision, you transition to a new situation – with new decisions to be made. Each of the possible routes you can take is a “branch”, while the decisions themselves are the “nodes”. Your initial starting point is the primary node.

That’s how a decision tree algorithm creates a series of nodes and leaves. The important thing here is that all of them come from one node. (In contrast, random forest algorithms produce a number of trees, each with its primary node.)

In terms of real-life application, regression trees can be used to predict survival rates, insurance premiums, and the price of real estate, based on various factors.  

Regression trees “grow” branches of decisions until a stopping criterion is reached. It works better with small amounts of input data because otherwise, you might get a biased output dataset.

The algorithm decides where to split and form a new branch out of a decision, based on multiple algorithms. The data is split into regions of sub-notes, which gather around all available variables.  

6. Random Forest

The random forest algorithm is another form of supervised machine learning. It produces multiple decision trees, instead of only one like Regression Trees. The nodes are spread randomly and their order is of no significance to the output data. The larger the quantity of the trees, the more accurate the result.  

This type of algorithm can be used for both classification and regression. One of the awesome features of the random forest algorithm is that it can work when a large proportion of the data is missing. It also has the power to work with a large dataset.

In the case of regression, these algorithms are not the best choice, because it doesn’t have much control over what the model does.

Random Forest algorithms can be very useful in e-commerce. If you need to establish whether your customers will like a particular pair of shoes, you only need to collect information on their previous purchases.

You include the type of shoes, whether they had a heel or not, the gender of the buyer, and the price range of the previous pairs they ordered. This will be your input data.

The algorithm will generate enough trees to provide you with an accurate estimate.

You are welcome!

And here comes the last tree-system algorithm:

7. AdaBoost

AdaBoost is short for Adaptive Boosting. The algorithm won the Gödel Prize in 2003 for its creators.

Like the previous two, this one also uses the system of trees. Only instead of multiple nodes and leaves, the trees in AdaBoost produce only 1 node and 2 leaves, a.k.a. a stump.

AdaBoost algorithms differ substantially from decision trees and random forests.

Let’s see:

A decision tree algorithm will use many variables before it produces an output. A stump can only use 1 variable to make a decision.

In the case of random forest algorithms, all the trees are equally important for the final decision. AdaBoost algorithms set priority to some stumps over others.

And last but not least, random forest trees are more chaotic, so to speak. Meaning that the sequence of trees is irrelevant. The outcome doesn’t depend on the order in which the trees got produced. In contrast, for AdaBoost algorithms – order is essential.

The outcome of every tree is the basis for the next. So if there is a mistake along the way, every subsequent tree becomes affected.

Alright, so what can this algorithm do in real life?

AdaBoost algorithms already shine in healthcare, where researchers use them to measure the risks of disease. You have the data, but different factors have different gravity. (Imagine you fell on your arm and your doctors use an algorithm to determine whether it is broken or not. If the input data contains both the x-ray of your arm and a photo of your broken fingernail… well, it’s quite obvious which stump will be given more importance to.)

Now, we are out of the forest, so to speak, so let’s have a look at 3 other kinds of machine learning algorithms:

8. Naive Bayes

This one comes in handy when you have a text classification problem. It is the machine learning algorithm used when one has to deal with high-dimensional data sets, such as spam filtration or news articles classification.

The algorithm carries this signature name because it regards each variable as independent. In other words, it considers the different features of the input data as completely unrelated. This makes it a simple and effective probabilistic classifier.

The “Bayes” part of the name refers to the man who invented the theorem used for the algorithm, namely – Thomas Bayes. His theorem, as you might suspect, examines the conditional probability of events.

Probabilities are calculated on two levels. First, the probability of each class. And second, the conditional probability according to a given factor.

9. Learning Vector Quantization

The Learning Vector Quantization algorithm, or LVQ, is one of the more advanced machine learning algorithms.

Unlike the kNN, the LVQ algorithm represents an artificial neural network algorithm. In other words, it aims to recreate the neurology of the human brain.

The LVQ algorithm uses a collection of codebook vectors as a representation. Those are basically lists of numbers, which have the same input and output qualities as your training data.

10. Support Vector Machines

These are one of the most popular machine learning algorithms.

The Support Vector Machines algorithm is suitable for extreme cases of classifications. Meaning – when the decision boundary of the input data is unclear. The SVM serves as a frontier which best segregates the input classes.

SVMs can be used in multidimensional datasets. The algorithm transforms the non-linear space into a linear space. In 2 dimensions you can visualize the variables as a line and thus have an easier time identifying the correlations.

SVMs have already been used in a variety of fields in real life:

• In medical imaging and medical classification tasks
• To study the air quality in largely populated areas
• To help with financial analysis
• In page ranking algorithms for search engines
• For text and object recognition.

It sounds like the Swiss knife of ML algorithms, doesn’t it?

Conclusion

Humans and computers can work together successfully.

Researchers assure us that this partnership can, and will give amazing results. Machine learning algorithms are already helping humanity in a number of ways.

One of the most important functions of machine learning and AI algorithms is to classify.

Let’s see the top 10 machine learning algorithms once again in a nutshell:

• Linear Regression – used to establish the relation between 2 variables – an explanatory and a dependent variable.  
• Logistic Regression – a binomial classifier, there are only 2 possible outcomes of each query.
• Linear Discriminant Analysis – works best for classifying data among known categories.
• K-Nearest Neighbor – classifies data into categories.
• Regression Trees – used for predictive modeling.
• Random Forest – used with large datasets, and when a large proportion of the input data is missing.
• AdaBoost – binary classifications.
• Naive Bayes – predictive modeling.
• Learning Vector Quantization – an artificial neural network algorithm.
• Support Vector Machines – extreme cases of classification in a multidimensional dataset.

All these algorithms (plus the new ones that are yet to come) will lay the foundation for a new age of prosperity for humanity. It will make possible (and even necessary) a universal basic income to ensure the survival of the less capable people. (Who will otherwise revolt and mess up our society. Oh, well.)

Well, who would have thought an article about machine learning algorithms would be such a doozy. Well, that was it for today.

Drop your comment