Ai-Pi-Cam

Ai-Pi-Cam turns a Raspberry Pi 4 and a Raspberry Pi Camera Rev 1.3 (OV5647) into an appliance-style AI camera. It runs TFLite object detection on the live camera feed, serves an annotated MJPEG stream to a browser on your LAN at http://<pi-ip>:8080/, shows a live HUD on any attached native display, and writes event snapshots and clips locally when detections fire. Everything runs on the Pi CPU — no AI HAT, no cloud account, no paid services, no internet connection required. [implemented-here]

---

Supported Hardware

[Pi-required]

Component	Specification
Board	Raspberry Pi 4 (1GB, 2GB, 4GB, or 8GB)
Camera	Raspberry Pi Camera Module Rev 1.3 / OV5647 (CSI ribbon connector)
Accelerator	None required for v1 — CPU-only. Coral USB Accelerator is optional future work.
OS	Raspberry Pi OS Bookworm or Trixie (64-bit, aarch64)

No Coral/AI HAT required for v1.

---

Features (v1.5)

[implemented-here]

Five detection modes — switch between them live via the web control panel or native HUD:

Mode	What it detects
OBJECTS	All 80 COCO classes (person, car, bottle, etc.)
PEOPLE	Persons only, with per-session persistent labels (person-1, person-2, …)
HANDS	21-point hand skeleton via BlazePalm + hand landmark models; multi-hand; L/R numbering
FACE	Face boxes via BlazeFace; per-session persistent identities (face-1, face-2, …) with leave/return re-acquisition
ANIMALS	Animal classes from the COCO set (cat, dog, bird, horse, sheep, cow, elephant, bear, zebra, giraffe)

Identity persistence — in FACE and PEOPLE modes, distinct individuals keep stable per-session labels (face-N, person-N) that re-acquire after brief exits. Non-biometric: no identity claims, in-memory only, no enrollment database.

Visible preprocessing experiments — a set of input transforms you can toggle live and see in MODEL VIEW, to compare what helps detection in your environment. Modes: none, auto, grayscale, clahe_l (contrast-limited adaptive histogram equalization), otsu_thresh (binary threshold), canny_edge. The transform is applied to the frame the detector actually sees; toggling it visibly changes the model view. Honest note: heavy transforms usually lower accuracy on these RGB-trained models — they ship as an experimentation surface, not a default (default stays clean color).

Switchable HUD themes — three palettes: phosphor (green-on-black), amber (amber-on-black), mono (white-on-black). Switchable via config or live from the native HUD.

Camera view / model view toggle — the HUD and web stream default to a clean color camera view. A view-toggle button reveals the preprocessed 320×240 model-input frame upscaled with a "MODEL VIEW" badge, so you can see exactly what the detector sees.

LAN MJPEG web stream + native touchscreen HUD — browser view at http://<pi-ip>:8080/ with a live control panel; full-screen HUD on a locally-attached display. All on-device — no cloud, no auth.

Event recording — qualifying detections write timestamped JPEG snapshots and short video clips to a configurable local directory, with debounce, cooldown, and post-event tail to prevent SD card flooding.

systemd appliance — reboot-surviving headless service via deploy_rpi_service.sh. Restart-on-failure, network-ordered startup.

---

Quickstart (fresh Raspberry Pi OS)

[Pi-required]

Step 1 — Flash Raspberry Pi OS

Flash Raspberry Pi OS Bookworm or Trixie, 64-bit (aarch64), to your SD card using Raspberry Pi Imager. Enable SSH in the imager's advanced settings. Boot and log in.

Step 2 — Clone this repo

git clone <repo-url>
cd Ai-Pi-Cam

Step 3 — Bootstrap the stack (one-time setup, requires sudo)

[Pi-required]

bash scripts/bootstrap_pi.sh

This installs the apt camera stack (python3-picamera2, python3-libcamera, python3-opencv, rpicam-apps) and creates the project virtualenv with --system-site-packages. It also pip-installs ai-edge-litert, flask, and python-dotenv. Idempotent — safe to re-run if interrupted. See docs/SETUP_RPI4.md for a detailed walkthrough.

Step 4 — Download detection models

[implemented-here]

bash scripts/fetch_models.sh

Downloads ssd_mobilenet_v2_coco_quant_postprocess.tflite, efficientdet_lite0_320_ptq.tflite, palm_detection_lite.tflite, hand_landmark_lite.tflite, and blaze_face_short_range.tflite into models/. Skips files already present.

Step 5 — Verify camera

[Pi-required]

bash scripts/check_camera.sh

Runs three ordered checks: rpicam-hello OV5647 detection, picamera2 import inside the project venv, and a 640x480 JPEG still capture with JPEG header validation.

Step 6 — Run live detection

[implemented-here]

./ai-pi-cam-cli run

Starts the appliance: picamera2 capture → TFLite detection → annotated MJPEG web stream at port 8080 + native HUD (if a display is attached). Press Ctrl+C to stop.

Headless / SSH mode (no local display window):

./ai-pi-cam-cli run --headless

Custom config:

./ai-pi-cam-cli run --config config/config.yaml

Other subcommands:

./ai-pi-cam-cli check       # run camera hardware checks (Pi + camera required)
./ai-pi-cam-cli version     # print package version
./ai-pi-cam-cli --help      # list all subcommands and flags

The default config is config/config.yaml (copy from config/examples/config.example.yaml if it doesn't exist yet).

---

Viewing the Stream

[implemented-here]

Web browser (any device on your LAN):

Open http://<pi-ip>:8080/ in any browser. The annotated MJPEG stream loads automatically. The control panel on the same page lets you switch modes, adjust thresholds, change the preprocessing filter, switch the HUD theme, and toggle between camera view and model view.

Native display / touchscreen:

If a monitor or touchscreen is attached to the Pi, the app opens a full-screen HUD window. Controls are laid out in two rows along the bottom:

• Row 1 (mode buttons): OBJECTS · HANDS · PEOPLE · FACE · ANIMALS
• Row 2 (preprocessing + view toggle): AUTO · NONE · BRIGHT · DARK · [CAM/MODEL]

Tap to switch. The active button is highlighted per the current HUD theme.

For headless (SSH-only) operation, set runtime.headless: true in your config — the native window is suppressed but the web stream continues unaffected.

---

Security (LAN-Only Warning)

The web server binds 0.0.0.0:8080 with no authentication. Any host that can reach the Pi on port 8080 can view the live feed and change detection settings without a password. This is safe on a trusted home LAN behind a NAT router. It is not safe if the Pi is internet-facing or on an untrusted shared network.

Before connecting to any non-trusted network, restrict the port with ufw:

sudo ufw allow from 192.168.1.0/24 to any port 8080
sudo ufw deny 8080
sudo ufw enable

Replace 192.168.1.0/24 with your LAN subnet (run ip route on the Pi to confirm). See docs/DEPLOYMENT.md for the full firewall guidance and the SSH-tunnel (host: 127.0.0.1) option for maximum isolation.

---

Configuration

[implemented-here]

Copy config/examples/config.example.yaml to your working config and pass it with --config:

cp config/examples/config.example.yaml config/config.yaml
./ai-pi-cam-cli run --config config/config.yaml

Key tunables:

Key	Default	Description
detect.mode	coco	Detection mode: coco / hand / people / face / animal
model.path	models/efficientdet_lite0.tflite	Path to the .tflite COCO model file
detect.threshold	0.5	Confidence cutoff for COCO/people/animal detection
detect.classes	[] (all)	Allowlist of COCO class names; empty = all 80 classes
detect.every_n	3	Run inference on every Nth frame (~10 fps at 30 fps camera)
preprocess.mode	auto	Preprocessing: none / auto / gamma_brighten / gamma_darken / clahe_l / grayscale
hud_theme	phosphor	HUD palette: phosphor / amber / mono
runtime.headless	false	Set true to suppress the local preview window (SSH/service mode)
web.port	8080	LAN port for the MJPEG stream

See config/examples/config.example.yaml for the full list including event debounce, identity TTLs, and feature flags.

---

Run as a Service (systemd)

[implemented-here]

To run as a system service that starts at boot and restarts on failure:

bash scripts/deploy_rpi_service.sh

This substitutes the project root path into the unit file, copies it to /etc/systemd/system/ai-pi-cam.service, reloads systemd, enables, and starts the service in one pass. The service is configured Restart=on-failure with a 5-second restart delay. See docs/DEPLOYMENT.md for the full deployment guide including headless setup and feature-flag opt-in.

---

Documentation

Doc	Description
docs/SETUP_RPI4.md	Verified stack walkthrough: trixie/Python 3.13/LiteRT, bootstrap stages, --system-site-packages venv, what NOT to do
docs/CAMERA_REV_1_3.md	Camera Rev 1.3 / OV5647 hardware, boot config (/boot/firmware/config.txt), live detection evidence
docs/TFLITE_OBJECT_DETECTION.md	Model I/O, output tensor layout, fetch, run, tuning guide, fps expectations
docs/DEPLOYMENT.md	systemd service install, enable, reboot survival, headless operation, feature-flag opt-in
docs/TROUBLESHOOTING.md	Real pitfalls: picamera2 ImportError, BGR/RGB, /boot/firmware, rpicam vs libcamera, SD flooding, systemd headless, animal detection limits
docs/PI_EVIDENCE.md	Master Pi-only verification checklist with real captured evidence
docs/WHATS_NEW.md	Per-milestone release notes: what each version added

---

Known Limitations (Camera Rev 1.3 / OV5647)

• Legacy OV5647 sensor; maximum native resolution 2592x1944 (5MP). This project targets 640x480 (main stream) and 320x240 (inference lores) for performance.
• No autofocus. Fixed-focus lens shipped on most Rev 1.3 modules; manual focus ring on some. See docs/TROUBLESHOOTING.md §9.
• 10-bit GBRG Bayer native sensor output; libcamera converts to RGB888 (stored as BGR in memory) for the main stream automatically.
• Not compatible with legacy picamera v1 or raspistill/raspivid — both are removed on Bookworm/trixie. Use picamera2 and rpicam-* tools.
• CPU-only performance: ~14 fps (SSD-MobileNet v2), ~9 fps (EfficientDet-Lite0), ~19 fps for tracked hands (palm re-detect only on re-acquire). Rough estimates — actual fps depends on Pi 4 RAM, SD card, and thermal throttling.
• ANIMALS mode: small subjects (occupying < ~10% of frame area) are unreliable — the COCO model cannot recover them at any resolution. Move the camera closer. See docs/TROUBLESHOOTING.md §7 for measured evidence.
• Hand skeleton geometry is best-effort at this stage (21-pt landmarks are rendered but alignment quality varies with hand pose). [Pi-required verification ongoing]
• [upstream-reference] Coral USB Accelerator is not required for v1 CPU-only operation. It can be added in v2 for a significant fps improvement.

---

Troubleshooting

See docs/TROUBLESHOOTING.md. [implemented-here]