Implement machine learning algorithms from scratch
Implement the most important algorithms for machine learning and searching from scratch.
- K-nearest neighbors (KNN)
KNN just memorizes the data and make prediction by the majority vote of its K nearest neighbors. - Naive Bayes
Naive Bayes assumes that all features in are independent, calculate the likelihood of each class from features based on Bayes Rule. - Decision tree
Decision tree make the prediction by answering a series of question for each features. - Random forest
Random forest is an tree-based ensemble algorithm. It consists of numbers of decision trees, each tree is built on random subset of samples with random subset of features. The overall output is normally the majority vote of all trees. - Adaboost
It is a sequential ensemble tree-base algorithm. It boost the overall performance by adding weak learners sequentially and put more weights on the misclassified samples by the previous learner. - Gradient boosting
Gradient boosting, an powerful sequential tree-based learning algorithm. It is aim to boost the overall performance by sequentially updating the results with adding the prediction of each learner. Each learner is built by taking the gradient of the previous learner as the target variables. - Linear regression
Find the linear relation between X and continuous target variables y. - Logistic regression
It is constructed by a linear regression and a logistic function - Neural network
Neural networks generally have multiple connected layers between their input and output, called “hidden” layers. There are numbers of neurons in each layer, each neuron receive the signals from those from the previous layer and process it to the next layer (Mathematically, the sum of weighted input followed by a activation function).
(to be continued … )