Edge Vision Inference Monitoring Platform

Overview

This project implements an end-to-end image inference platform using FastAPI and PyTorch. A pre-trained MobileNetV3 model is deployed as a REST API service to perform real-time image classification.

In addition to model inference, the project includes batch image processing, latency benchmarking, SQLite-based prediction logging, a Streamlit monitoring dashboard, and Docker containerization.

The goal of this project is to gain hands-on experience with practical AI deployment workflows, including model serving, API development, performance monitoring, database integration, and containerized deployment.

---

System Overview

API Documentation

Monitoring Dashboard

---

Features

• Real-time image classification using MobileNetV3 Small
• FastAPI-based REST API service
• Single-image prediction endpoint
• Batch-image prediction endpoint
• Top-5 prediction results with confidence scores
• SQLite prediction logging
• Streamlit monitoring dashboard
• Inference latency benchmarking
• Dockerized deployment

---

Technology Stack

Machine Learning

• PyTorch
• TorchVision
• MobileNetV3 Small

Backend

• FastAPI
• Uvicorn

Data Storage

• SQLite

Visualization

• Streamlit
• Pandas

Deployment

• Docker

---

Project Structure

edge_vision_inference/
│
├── app.py
├── db.py
├── dashboard.py
├── test_client.py
├── requirements.txt
├── Dockerfile
├── README.md
├── .gitignore
├── images/
└── docs/
    ├── swagger.png
    └── dashboard.png

---

API Endpoints

GET /

Health check endpoint.

Response:

{
  "message": "Service is running"
}

GET /status

Returns service status information.

GET /model_info

Returns model metadata.

POST /predict

Performs image classification for a single image.

POST /predict_batch

Performs image classification for multiple images in a single request.

---

Monitoring Dashboard

The Streamlit dashboard provides:

• Prediction history
• Total prediction count
• Average inference latency
• Class distribution statistics
• Latency trend visualization

Run the dashboard:

streamlit run dashboard.py

---

Performance Benchmark

Benchmark testing was performed using a custom Python client.

Example benchmark results:

Metric	Value
Average Batch Latency	~101 ms
Images Per Request	3
Throughput	~29.5 images/sec

Example prediction results:

Image	Top Prediction
test.jpg	tiger cat
test2.jpg	toy poodle
test3.png	sports car

---

Docker Deployment

Build Docker image:

docker build -t edge-vision-service .

Run Docker container:

docker run -p 8000:8000 edge-vision-service

Open Swagger API documentation:

text http://localhost:8000/docs

---

Local Installation

Clone repository:

git clone https://github.com/wuchengshao/edge-vision-inference-platform.git

cd edge_vision_inference

Install dependencies:

pip install -r requirements.txt

Start FastAPI service:

uvicorn app:app --reload

Open API documentation:

http://127.0.0.1:8000/docs

---

Future Improvements

• Support custom-trained image classification models
• Add GPU inference acceleration
• Store prediction logs in PostgreSQL
• Enable automatic dashboard refresh
• Deploy the service to a cloud environment

---

Author

WuCheng Shao

M.Sc. Student Elektrotechnik-Elektronik-Informationstechnik (EEI), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

Interests:

• Machine Learning
• Computer Vision
• Edge AI
• Industrial AI
• Embedded Systems