Explore R Libraries: Rpart

Jul 16, 2020 • 12 Minute Read

Introduction

Rpart is a powerful machine learning library in R that is used for building classification and regression trees. This library implements recursive partitioning and is very easy to use. In this guide, you will learn how to work with the rpart library in R.

Data

In this guide, you will use fictitious data of loan applicants containing 600 observations and eight variables, as described below:

Is_graduate: Whether the applicant is a graduate ("Yes") or not ("No")
Income: Annual Income of the applicant in USD
Loan_amount: Loan amount in USD for which the application was submitted
Credit_score: Whether the applicant's credit score is statisfactory ("Satisfactory") or not ("Not_Satisfactory")
approval_status: Whether the loan application was approved ("Yes") or not ("No")
Age: The applicant's age in years
Investment: Total investment in stocks and mutual funds in USD as declared by the applicant
Purpose: Purpose of applying for the loan

The first step is to load the required libraries and the data.

      library(plyr)
library(readr)
library(dplyr)
library(caret)
library(rpart)
library(rpart.plot)

dat <- read_csv("data.csv")

glimpse(dat)
    

Output:

      Observations: 600
Variables: 8
$ Is_graduate     <chr> "No", "Yes", "Yes", "Yes", "Yes", "Yes", "Yes", "Yes",...
$ Income          <int> 3000, 3000, 3000, 3000, 8990, 13330, 13670, 13670, 173...
$ Loan_amount     <dbl> 6000, 9000, 9000, 9000, 8091, 11997, 12303, 12303, 155...
$ Credit_score    <chr> "Satisfactory", "Satisfactory", "Satisfactory", "Not _...
$ approval_status <chr> "Yes", "Yes", "No", "No", "Yes", "No", "Yes", "Yes", "...
$ Age             <int> 27, 29, 27, 33, 29, 25, 29, 27, 33, 29, 25, 29, 27, 33...
$ Investment      <dbl> 9331, 9569, 2100, 2100, 6293, 9331, 9569, 9569, 12124,...
$ Purpose         <chr> "Education", "Travel", "Others", "Others", "Travel", "...
    

The output shows that the dataset has four numerical (labelled as int) and four character variables (labelled as chr). You will convert these into factor variables using the line of code below.

      names <- c(1,4,5,8)
dat[,names] <- lapply(dat[,names] , factor)
glimpse(dat)
    

Output:

      Observations: 600
Variables: 8
$ Is_graduate     <fct> No, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, No, Y...
$ Income          <int> 3000, 3000, 3000, 3000, 8990, 13330, 13670, 13670, 173...
$ Loan_amount     <dbl> 6000, 9000, 9000, 9000, 8091, 11997, 12303, 12303, 155...
$ Credit_score    <fct> Satisfactory, Satisfactory, Satisfactory, Not _satisfa...
$ approval_status <fct> Yes, Yes, No, No, Yes, No, Yes, Yes, Yes, No, No, No, ...
$ Age             <int> 27, 29, 27, 33, 29, 25, 29, 27, 33, 29, 25, 29, 27, 33...
$ Investment      <dbl> 9331, 9569, 2100, 2100, 6293, 9331, 9569, 9569, 12124,...
$ Purpose         <fct> Education, Travel, Others, Others, Travel, Travel, Tra...
    

Data Partition

The createDataPartition function is used to split the data into training and test data. This is called the holdout-validation method for evaluating model performance.

The first line of code below sets the random seed for reproducibility of results. The second line performs the data partition, while the third and fourth lines create the training and test set. Finally, the fifth line prints the dimension of the training and test data.

      set.seed(100)
trainRowNumbers <- createDataPartition(dat$approval_status, p=0.7, list=FALSE)
train <- dat[trainRowNumbers,]
test <- dat[-trainRowNumbers,]
dim(train); dim(test)
    

Output:

      1] 420   8

[1] 180   8

Feature Scaling

The numeric features need to be scaled because the units of the variables differ significantly and may influence the modeling process. The first line of code below creates a list that contains the names of numeric variables. The second line uses the preProcess function from the caret library to complete this task. The method employed is of centering and scaling the numeric features, and the preprocessing object is fit only to the training data.

The scaling is applied on both the train and test data partitions, which is done in the third and fourth lines of code below. The fifth line prints the summary of the preprocessed train set. The output shows that now all the numeric features have a mean value of zero.

      cols = c('Income', 'Loan_amount', 'Age', 'Investment')

pre_proc_val <- preProcess(train[,cols], method = c("center", "scale"))

train[,cols] = predict(pre_proc_val, train[,cols])
test[,cols] = predict(pre_proc_val, test[,cols])

summary(train)

Output:

      Is_graduate     Income         Loan_amount                 Credit_score
 No : 90     Min.   :-1.3309   Min.   :-1.6568   Not _satisfactory: 97  
 Yes:330     1st Qu.:-0.5840   1st Qu.:-0.3821   Satisfactory     :323  
             Median :-0.3190   Median :-0.1459                          
             Mean   : 0.0000   Mean   : 0.0000                          
             3rd Qu.: 0.2341   3rd Qu.: 0.2778                          
             Max.   : 5.2695   Max.   : 3.7541                          
 approval_status      Age               Investment            Purpose   
 No :133         Min.   :-1.7607181   Min.   :-1.09348   Education: 76  
 Yes:287         1st Qu.:-0.8807620   1st Qu.:-0.60103   Home     :100  
                 Median :-0.0008058   Median :-0.28779   Others   : 45  
                 Mean   : 0.0000000   Mean   : 0.00000   Personal :113  
                 3rd Qu.: 0.8114614   3rd Qu.: 0.02928   Travel   : 86  
                 Max.   : 1.8944843   Max.   : 4.54891
    

Model Building with rpart

The data is ready for modeling and the next step is to build the classification decision tree. Start by setting the seed in the first line of code. The second line use the rpart function to specify the parameters used to control the model training process.

The important arguments of the rpart function are given below.

formula: a formula that links the target variable to the independent features.
data: the data to be used for modeling. In this case, you are building the model on training data.
method: defines the algorithm. It can be one of anova, poisson, class or exp. In this case, the target variables is categorical, so you will use the method as class.
minsplit: the minimum number of observations that must exist in a node in order for a split to be attempted.
minbucket: the minimum number of observations in any terminal node. If only one of minbucket or minsplit is specified, the code either sets minsplit to minbucket*3 or minbucket to minsplit/3, as appropriate.

You will build the classification decision tree with the following argument:

      set.seed(100)

tree_model = rpart(approval_status ~ Is_graduate + Income + Loan_amount + Credit_score + Age + Investment + Purpose, data = train, method="class", minsplit = 10, minbucket=3)

You can examine the model with the command below.

      summary(tree_model)

Output:

      Call:
rpart(formula = approval_status ~ Is_graduate + Income + Loan_amount + 
    Credit_score + Age + Investment + Purpose, data = train, 
    method = "class", minsplit = 10, minbucket = 3)
  n= 420 

          CP nsplit rel error    xerror       xstd
1 0.60902256      0 1.0000000 1.0000000 0.07167876
2 0.06766917      1 0.3909774 0.3909774 0.05075155
3 0.01503759      3 0.2556391 0.2631579 0.04258808
4 0.01002506      6 0.2030075 0.2706767 0.04313607
5 0.01000000      9 0.1729323 0.2857143 0.04420254

Variable importance
     Purpose Credit_score          Age   Investment  Loan_amount       Income 
          46           31           17            2            2            1
    

An advantage of a decision tree is that you can actually visualize the model. This is done with the code below.

      prp(tree_model)

The above plot shows the important features used by the algorithm for classifying observations. The variables Purpose and Credit_score emerge as the most important variables for carrying out recursive partitioning.

Model Evaluation

You have built the algorithm on the training data and the next step is to evaluate its performance on the training and test dataset.

The code below predicts on training data, creates the confusion matrix, and finally computes the model accuracy.

      PredictCART_train = predict(tree_model, data = train, type = "class")

table(train$approval_status, PredictCART_train)

(131+266)/(131+266+23) #94.5%

Output:

      PredictCART_train
       No Yes
  No  131   2
  Yes  21 266
  
 (131+266)/(131+266+23) #94.5%
    

The accuracy on the training data is very good at 94.5%. The next step is to repeat the above step and check the model's accuracy on the test data.

      PredictCART = predict(tree_model, newdata = test, type = "class")

table(test$approval_status, PredictCART)

166/180 #92.2%

Output:

      PredictCART
       No Yes
  No   53   4
  Yes  10 113
  
  166/180  #92.2%
    

The output shows that the accuracy on the test data is 92%. Since the model performed well on both training and test data, it shows that the model is robust and its performance is good.

Conclusion

In this guide, you learned about the rpart library, which is one of the most powerful libraries in R to build non linear regression trees. You learned how to build and evaluate decision tree models, and also learned how to visualize the decision tree with the prp function.

To learn more about data science and machine learning with R, please refer to the following guides: