Scikit Machine Learning

Feb 19, 2019 • 10 Minute Read

Introduction

Machine Learning is one of the most sought-after disciplines in today’s Artificial Intelligence driven world. But what is Machine Learning? In simple terms, it is the field of teaching machines and computers to learn from existing data and to make predictions on the new unseen data. There are three types of Machine Learning Algorithms: Supervised, Unsupervised, and Reinforcement Learning.

In Supervised Learning, we have a target/outcome variable which is to be predicted from a given set of features/independent variables. The algorithm works by using the set of features and generating a function that maps inputs to desired outputs. The training process continues until the model achieves the desired level of accuracy on the training data, which is then used on the new unseen data. There are two types of supervised machine learning algorithms – Classification and Regression.

Classification and Regression

Classification models are models which predict a categorical label. Good examples of this are predicting whether a customer will churn or not, or whether a bank loan will default or not.

On the other hand, Regression models are models which predict a continuous label. The goal is to produce a model that represents the ‘best fit’ to some observed data, according to an evaluation criterion we choose. Good examples of this are predicting the price of the house, sales of a retail store, or life expectancy of an individual.

In this guide, we will focus on Classification.

Supervised Learning with Scikit-learn

Scikit - Learn, or sklearn, is one of the most popular libraries in Python for doing supervised machine learning. It integrates well with the SciPy stack, making it robust and powerful. Scikit-learn can be used for both classification and regression problems, however, this guide will focus on the classification problem.

Classification with Scikit-learn

The first step in any machine learning process is understanding the Problem Statement and the Data before jumping into predictive modeling.

Problem Statement

Diabetes is considered one of the serious health issues which cause an increase in blood sugar. Many complications occur if diabetes remains untreated and unidentified. The aim of this guide is to build a classification model to detect diabetes. We will be using the diabetes dataset which contains 768 observations and 9 variables, as described below:

pregnancies - Number of times pregnant
glucose - Plasma glucose concentration
diastolic - diastolic blood pressure (mm Hg)
triceps - Skinfold thickness (mm)
insulin - Hour serum insulin (mu U/ml) in
bmi - BMI (weightinkg/(heightinm)
dpf - Diabetes pedigree function
age - Age in years
diabetes - 1 represents the presence of diabetes while 0 represents the absence of it. This is the target variable.

Also, the classification algorithm selected is the Logistic Regression Model as it is one of the most widely used Classification Algorithms.

Evaluation Metrics

We will evaluate the performance of the model using four metrics

Accuracy Accuracy is the fraction of cases correctly classified. For a binary classifier, it is represented as accuracy = (TP+TN)/(TP+TN+FP+FN), where

True Positive or TP are cases with positive labels which have been correctly classified as positive. True Negative or TN are cases with negative labels which have been correctly classified as negative. False Positive or FP are cases with negative labels which have been incorrectly classified as positive. False Negative or FN are cases with positive labels which have been incorrectly classified as negative.

Precision Precision is the fraction of correctly classified label cases out of all cases classified with that label value. It is represented as Precision = P = TP / (TP+ FP)
Recall Recall is the fraction of cases of a label value correctly classified out of all cases that actually have that label value. It is represented as Recall = R = TP / (TP+FN)
F1-score The F1 statistic is a weighted average of precision and recall. It is represented as F1 = =2*(P*R) / (P+R)

Steps

Following are the steps which are commonly followed while implementing classification with Scikit-learn.

Step 1 - Loading the required libraries and modules.

Step 2 - Loading the data and performing basic data checks.

Step 3 - Creating arrays for the features and the response variable.

Step 4 - Creating the Training and Test datasets.

Step 5 - Create and fit the classifier.

Step 6 - Predict on the test data and compute evaluation metrics.

The following sections will cover these steps.

Step 1 - Loading the Required Libraries and Modules

          # Import required libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

# Import necessary modules
from sklearn.linear_model import LogisticRegression 
from sklearn.model_selection import train_test_split  
from sklearn.metrics import confusion_matrix, classification_report
    

Step 2 - Loading the Data and Performing Basic Data Checks

The first line of code reads in the data as pandas dataframe, while the second line of code prints the shape - 768 observations of 9 variables. The third line gives summary statistics of the numerical variables. We can see that all the variables have 768 as 'count' which is equal to the number of records in the dataset that means we don't have missing values.

          # Load data
df = pd.read_csv("diabetes.csv")
print(df.shape)
df.describe()
    

Output:

      (768, 9)

	pregnancies	glucose	diastolic	triceps	insulin	bmi	dpf	age	diabetes
count	768	768	768	768	768	768	768	768	768
mean	3.8	120.9	69.1	20.5	79.8	32.0	0.5	33.2	0.3
std	3.4	32.0	19.4	16.0	115.2	7.9	0.3	11.8	0.5
min	0	0	0	0	0	0	0.1	21	0
25%	1	99	62	0	0	27.3	0.2	24	0
50%	3	117	72	23	30.5	32	0.4	29	0
75%	6	140.3	80	32	127.3	36.6	0.6	41	1
max	17	199	122	99	846	67.1	2.4	81	1

Step 3 - Creating Arrays for the Features and the Response Variable.

The first line of code creates an array of the target variable, while the second line of code gives us the array of all the features after excluding the target variable 'diabetes'.

          # Create arrays for the features and the response variable
y = df['diabetes'].values
X = df.drop('diabetes', axis=1).values
    

Step 4 - Creating the Training and Test Datasets

The first line of code splits the data into training and test dataset, while the second line of code gives us the shape of the training set (460 observations of 8 variables) and test set (308 observations of 8 variables).

          # Create training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.4, random_state=42) 
X_train.shape, X_test.shape
    

      ((460, 8), (308, 8))

Step 5 - Create and Fit the Classifier

The first line of code instantiates a LogisticRegression classifier called logreg; while the second line of code fits the classifier on the training set.

          # Create the classifier: logreg
logreg = LogisticRegression()

# Fit the classifier to the training data
logreg.fit(X_train, y_train)
    

Output:

          LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
              intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
              penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
              verbose=0, warm_start=False)
    

Step 6 - Predict on the Test Data and Compute Evaluation Metrics;

The first line of code predicts the label on the test data, the second line prints the confusion matrix, while the third line prints the classification report.

          # Predict the labels of the test set: y_pred
y_pred = logreg.predict(X_test)

# Compute and print the confusion matrix and classification report
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))
    

Output:

          [174  32]
     [ 36  66]]
                 precision    recall  f1-score   support
    
              0       0.83      0.84      0.84       206
              1       0.67      0.65      0.66       102
    
    avg / total       0.78      0.78      0.78       308
    

Evaluation of the Model Performance

We will now evaluate the model performance on the basis of the confusion matrix created above. The results of the evaluation are given below:

Accuracy = (174+66)/(174+66+32+36) = 78%
Precision = 66/(66+32) = 67%
Recall = 66/ (66+36) = 65%
F1 Score = 2*(0.65*0.67)/(0.65+0.67) = 66%

Conclusion

In this guide we have given you a brief introduction to supervised machine learning and implementation of one of the most popular classification algorithm ‘Logistic Regression’ in Python using Scikit-learn. The guide used the diabetes dataset and built a classifier algorithm to predict detection of diabetes.

Our model is achieving a decent accuracy of 78%, However because of the imbalance in the data, the Precision, Recall and F1 Score values are in the 65% to 67% range. The model can be further improved by doing cross-validation, features analysis, and feature engineering and, of course, by trying out more advanced machine learning algorithms such as Tree Family of Algorithms (Decision Tree and Random Forest) or Optimization Algorithms (Support Vector Machines and Neural Networks). However, that is not in the scope of this guide which is aimed at being a good starting point for individuals aspiring to start using Python’s Scikit-learn machine learning library for building classification algorithms.