Explainable Airline Customer Sentiment Analysis

NLP · Logistic Regression · ADASYN Oversampling

---

Abstract

A multiclass sentiment classifier trained on 14,640 airline customer tweets. The project tackles severe class imbalance (63% negative) using ADASYN (Adaptive Synthetic Sampling), and provides full model explainability through logistic regression coefficient analysis — revealing exactly which words drive each sentiment prediction.

---

Dataset

Property	Value
Source	Kaggle — Twitter US Airline Sentiment
Total tweets	14,640 (14,427 after deduplication)
Airlines	American, Delta, Southwest, United, US Airways, Virgin America
Labels	negative (63%), neutral (21%), positive (16%)
Key columns	text, airline_sentiment, airline

---

Project Structure

airline-sentiment-nlp/
├── data/
│   └── Tweets.csv                   # Download from Kaggle
├── notebooks/
│   └── sentiment_analysis.ipynb     # Full walkthrough notebook
├── src/
│   ├── adasyn.py                    # Custom ADASYN implementation
│   ├── preprocess.py                # Text cleaning + TF-IDF
│   ├── train.py                     # Model training
│   ├── evaluate.py                  # Metrics + plots
│   └── main.py                      # End-to-end pipeline runner
├── outputs/
│   ├── figures/                     # All generated plots
│   └── comparison_table.csv         # Model comparison results
├── requirements.txt
└── README.md

---

Methodology

1. Text Preprocessing

• Remove @mentions, URLs, punctuation, digits
• Lowercase and strip whitespace
• TF-IDF vectorization: max_features=10000, ngram_range=(1,2), sublinear_tf=True

2. Class Imbalance

The dataset has a 3.97:1 negative-to-positive ratio. Without correction, a naive model achieves ~77% accuracy by over-predicting "negative" — but Neutral recall collapses to 0.44 and Positive recall to 0.50.

3. ADASYN Oversampling

ADASYN generates synthetic minority samples adaptively — more samples are created near the decision boundary where the classifier struggles most. Applied only to the training set to prevent data leakage.

4. Dimensionality Reduction

TF-IDF features (10,000 dimensions) are reduced to 100 via TruncatedSVD before ADASYN to keep computation tractable. LR coefficients are back-projected to the original feature space for explainability.

5. Logistic Regression

Two models trained with identical hyperparameters (C=1.0, solver=lbfgs, max_iter=1000):

• Model 1: raw imbalanced training data
• Model 2: ADASYN-balanced training data

---

Results

Model	Accuracy	Precision	Recall	F1
Without ADASYN	0.7678	0.7616	0.7678	0.7493
With ADASYN	0.7100	0.7513	0.7100	0.7209

Per-class breakdown

Class	Without ADASYN (Recall)	With ADASYN (Recall)	Change
Negative	0.94	0.72	−0.22
Neutral	0.44	0.64	+0.20
Positive	0.50	0.77	+0.27

Key insight: ADASYN trades ~6pp of overall accuracy for a substantially fairer classifier. Minority class recall improves by 20–27 percentage points — a much more useful model in practice.

---

Explainability

Logistic Regression coefficients (back-projected from SVD space) reveal interpretable word-level signals:

Class	Top supporting words
Negative	delayed, cancelled, hours, no, hold, dont, worst
Neutral	can, any, how, need, dm, flights, change
Positive	great, thanks, thank you, love, best, amazing, guys

---

Visualizations

Figure	Description
01_class_distribution.png	Sentiment class counts
02_tweet_lengths.png	Character length by sentiment
03_airline_breakdown.png	Per-airline sentiment stacked bar
04_adasyn_comparison.png	Class distribution before vs after ADASYN
05_confusion_matrices.png	Side-by-side confusion matrices
06_comparison_table.png	Grouped bar chart of all metrics
07_explainability.png	Coefficient bar plots per class

---

Setup

# 1. Clone the repo
git clone https://github.com/yourusername/airline-sentiment-nlp.git
cd airline-sentiment-nlp

# 2. Install dependencies
pip install -r requirements.txt

# 3. Download the dataset
# Place Tweets.csv in data/

# 4a. Run the full pipeline
python src/main.py

# 4b. Or open the notebook
jupyter notebook notebooks/sentiment_analysis.ipynb

---

Technical Notes

• Why ADASYN over SMOTE? ADASYN concentrates synthetic samples near the decision boundary, which is more effective for high-dimensional, sparse text features.
• Why Logistic Regression? It's interpretable (coefficient-level explainability), computationally efficient, and a strong baseline for TF-IDF text classification.
• Why TruncatedSVD before ADASYN? KNN-based oversampling methods are prohibitively slow and memory-intensive on 10,000-dimensional sparse matrices. SVD reduction to 100 components captures ~85% of variance while making ADASYN tractable.
• All random seeds are fixed (random_state=42) for full reproducibility.

---