Pump it Up: Data Mining the Water Table

My analysis for the Driven Data competition

By Ville Heilala, 2017

Research question

The Driven Data competition describes the question as follows:

Can you predict which water pumps are faulty?

Using data from Taarifa and the Tanzanian Ministry of Water, can you predict which pumps are functional, which need some repairs, and which don't work at all? ... Predict one of these three classes based on a number of variables about what kind of pump is operating, when it was installed, and how it is managed. A smart understanding of which waterpoints will fail can improve maintenance operations and ensure that clean, potable water is available to communities across Tanzania.

Picture: Wikimedia Commons

The data

Data can be obtained from the competition site by registering and downloading the data sets. Data set contains training variables, labels and test set for participating the competition. Total data size is 74250 observations, 40 independent variables and 1 dependent variable. Predicted status group can be either "functional", "non functional" or "functional needs repair".

Tools

• Anaconda + Jupyter
• Numpy Stack, pandas, scikit-learn
• Amazon Web Services (AWS): some analysis was run in EC2 instance for increased speed compared to Macbook Pro
• Domino: Domino Datalab is described as a github for data science, some analysis was run under their free trial

Methods

• basic exploration methods
• Random Forest
• Density-based spatial clustering of applications with noise (DBSCAN)
  • Ester, M., Kriegel, H., Sander, J. & Xu, X. (1996). A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise
  • scikit-learn: sklearn.cluster.DBSCAN
• Principal Componen Analysis (PCA)
  • scikit-learn: sklearn.decomposition.PCA

Workflow

I'm using analysis workflow prensented in this Quora answer.

1. Ask A Question

This question is provided by the competition.

Can you predict which water pumps are faulty?

2. Get the Data

• Jupyter notebook: Get the data

3. Explore the Data

• Jupyter notebook: Explore the data
  • A basic starting point using R is presented in a tutorial by DataCamp
  • The main problem with the data is missing values: 12 of the 40 variables have missing data, which needs to be dealt with. Exploration revealed some potential ways to do imputation.
  • Categorical values need to be converted numerical. Numerical values might need to be normalized.

Plotting 2000 pumps each by status_group

4. Model the data

• Jupyter notebook: Preprocess the data I: Get missing gps_height values
  • Using geocoder library to obtain missing gps_height values
• Jupyter notebook: Preprocess the data II: Process variables
  • Combine train and test sets to do processing for the whole data, combine train set and train labels
  • Drop, round, combine to larger bins, normalize, one-hot encode
  • Make a new variable by clustering latitude and longitude with DBSCAN
• Jupyter notebook: Model the data I: Optimize metaparameters
  • Excluding few redundant variables
  • Estimating optimal max_features parameter

Optimizing max_features parameter, which is the number of features to consider when looking for the best split. Best value seems to be 0.20 (20 %) of the features.

• Jupyter notebook: Model the data II: Experimenting with feature selection
  • Removing variables with low feature importances one by one
  • Applying PCA before Random Forest

Removing variables with low feature importances one by one and how it affects Out of Bag error

• Jupyter notebook: Model the data III: Making prediction

Results

Best version of this model scored 0.8239 in the competition.