[🇧🇷 Português] [🇺🇸 English]

7- Social Buzz AI - Black Box Models in AI and Data Science

Course: Humanistic AI & Data Science (4th Semester)
Institution: PUC-SP
Professor: ✨ Rooney Ribeiro Albuquerque Coelho

Tip

This repository 2-social-buzz-ai-GBoost-and-LowDefault-Modeling is part of the main project 1-social-buzz-ai-main. To explore all related materials, analyses, and notebooks, visit the main repository

• 1-social-buzz-ai-main Part of the Humanistic AI Research & Data Modeling Series — where data meets human insight.

• 4- Social Buss: NLP - Class 1

• Embedding Projector

Important

⚠️ Heads Up

• Projects and deliverables may be made publicly available whenever possible.

• The course prioritizes hands-on practice with real data in consulting scenarios.

• All activities comply with the academic and ethical guidelines of PUC-SP.

• Confidential information from this repository remains private in private repositories.

Table of Contents

• Introduction to the Black Box Model
• How Black Box Models Work
• Why Use Black Box Models?
• Challenges of Black Box Models
• Explainable AI (XAI)
• Interpretation Methods: LIME and SHAP
• Practical Python Examples
• Common SHAP Visualizations and How to Interpret Them
• Domain-Specific Use Cases
• References and Further Reading

Introduction to the Black Box Model

A black box model in AI or data science is a system whose internal workings are not understandable or visible to users. You can see the inputs and outputs, but not the decision-making process inside. This term is typically applied to complex models like deep neural networks and ensembles.

How Black Box Models Work

These models learn from large datasets to capture hidden patterns. When fed new inputs, they produce predictions without revealing how each feature or data point influenced the output internally.

Why Use Black Box Models ?

• They often achieve higher accuracy for complex problems.
  
• They can model nonlinear and high-dimensional relationships that simpler models cannot capture.
  
• They can adapt continuously to new data in dynamic environments.

Challenges of Black Box Models

• Their lack of transparency complicates trust and validation.
  
• Difficult to debug or identify biases inside the model.
  
• Raise ethical and legal concerns in sensitive applications like healthcare or finance.

Explainable AI (XAI)

XAI encompasses techniques designed to explain black box models, making them more interpretable and trustworthy. It aims to provide local explanations (individual predictions) as well as global insights (overall model behavior).

Interpretation Methods: LIME and SHAP

LIME (Local Interpretable Model-Agnostic Explanations)

LIME explains a single prediction by approximating the black box locally with a simple interpretable model, revealing feature influences near that specific data point.

SHAP (SHapley Additive exPlanations)

SHAP uses game theory to fairly allocate the contribution of each feature to a prediction, providing both local and global explanations that satisfy consistency and accuracy properties.

LIME Example

import numpy as np  
from sklearn.datasets import load_iris  
from sklearn.ensemble import RandomForestClassifier  
from sklearn.model_selection import train_test_split  
import lime  
import lime.lime_tabular  

data = load_iris()  
X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, random_state=42)  

model = RandomForestClassifier(random_state=42)  
model.fit(X_train, y_train)  

explainer = lime.lime_tabular.LimeTabularExplainer(  
    X_train,   
    feature_names=data.feature_names,  
    class_names=data.target_names,  
    discretize_continuous=True  
)  

exp = explainer.explain_instance(X_test[^0], model.predict_proba, num_features=4)  
exp.show_in_notebook(show_table=True)  

SHAP Example

import shap  
from sklearn.datasets import load_iris  
from sklearn.ensemble import RandomForestClassifier  
from sklearn.model_selection import train_test_split  

data = load_iris()  
X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, random_state=42)  

model = RandomForestClassifier(random_state=42)  
model.fit(X_train, y_train)  

explainer = shap.TreeExplainer(model)  
shap_values = explainer.shap_values(X_test)  

shap.summary_plot(shap_values[^0], X_test, feature_names=data.feature_names)  

Common SHAP Visualizations and How to Interpret Them

1. SHAP Summary Plot

• Displays global feature importance and the effect direction.
  
• Each dot: SHAP value for a feature and instance.
  
• X-axis: impact on prediction (right increases, left decreases).
  
• Color: feature value (red = high, blue = low).
  

Tip

Interpretation: see which features push predictions higher or lower and how feature values relate.

2. SHAP Dependence Plot

• Plots SHAP values of a single feature versus actual feature values.
  
• Color encodes interaction with another feature.
  

Tip

Reveals nonlinear effects and interactions.

3. SHAP Force Plot

• Visualizes feature contributions for a single instance.
  
• Shows how features cumulatively push from average prediction to final output.
  

Tip

Useful for explaining specific predictions.

4. SHAP Decision Plot

• Shows cumulative SHAP values as features are considered.
  
• Traces how the prediction evolves step by step.
  

Tip

Useful to understand the decision-making path.

Example code to generate SHAP plots

# assume shap_values, model, X_test from above  

shap.summary_plot(shap_values[^0], X_test, feature_names=data.feature_names)  
shap.dependence_plot(0, shap_values[^0], X_test, feature_names=data.feature_names)  
shap.force_plot(explainer.expected_value[^0], shap_values[^0][^0], X_test[^0], feature_names=data.feature_names)  
shap.decision_plot(explainer.expected_value[^0], shap_values[^0][0:10], X_test[0:10], feature_names=data.feature_names)  

Domain-Specific Use Cases

Healthcare

• Deep learning models interpret medical images with SHAP to highlight important regions.
  

Tip

LIME explains predictions on tabular clinical data to support diagnoses.

Finance

• Black box models detect fraudulent transactions.
  

Tip

XAI methods provide explanations for flagged transactions ensuring regulatory compliance.

Marketing

• Models predict customer churn.
  
• SHAP and LIME explain drivers of individual and aggregate churn predictions.
  

References and Further Reading

• Ribeiro et al., 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier (LIME).
• Lundberg & Lee, 2017. A Unified Approach to Interpreting Model Predictions (SHAP).
• Rudin, 2019. "Stop Explaining Black Box Models for High Stakes Decisions and Use Interpretable Models Instead."
• IBM article: What Is Black Box AI and How Does It Work?
• SEON and Unit21 articles on Black Box Machine Learning.

💌 Let the data flow... Ping Me !

🛸๋ My Contacts Hub

────────────── ⊹🔭๋ ──────────────

➣➢➤ Back to Top

Copyright 2025 Mindful-AI-Assistants. Code released under the MIT license.