ADVIS_UniGra

ADVIS_UniGra is a research repository for reconstruction-based visual anomaly detection in collaborative robotic environments. It supports safety-area training, threshold calibration, and inference using VAE-GAN models.

---

Overview

This repository provides the anomaly detection pipeline developed for DistriMuSe UC3: Safe Interaction with Robots. The system monitors predefined safety areas such as the robot arm, conveyor belt, and personnel zones. A VAE-GAN model is trained on normal data, and unexpected conditions are detected using reconstruction-based anomaly scores.

Raw Frames / Video
        ↓
Safety-area preprocessing
        ↓
VAE-GAN training
        ↓
Threshold calibration
        ↓
Inference and evaluation

---

Key Features

• VAE-GAN based visual anomaly detection
• Per-safety-area model training
• Validation-based and labelled-test threshold calibration
• Config-driven experiments using configs/config.yaml
• Installable Python package named advis
• CLI commands for training, calibration, and inference
• Support for preprocessed frames, raw frames, videos, and camera streams
• Folder-level README files for easier navigation

---

Safety Areas

Area	Description
RoboArm	Robot arm safety region
ConvBelt	Conveyor belt safety region
PLeft	Left personnel safety region
PRight	Right personnel safety region
ALL	Run all available safety areas

---

Repository Structure

ADVIS_UniGra/
├── advis/
│   ├── config.py
│   ├── data.py
│   ├── models.py
│   ├── scoring.py
│   ├── utils.py
│   └── cad_utils.py
├── configs/
│   └── config.yaml
├── scripts/
│   ├── train.py
│   ├── calibrate_threshold.py
│   └── inference.py
├── docs/
├── data/
├── results/
├── pyproject.toml
├── requirements.txt
└── README.md

---

Installation

1. Clone repository

git clone https://github.com/rashidrao-pk/advis_distrimuse_unito.git
cd advis_distrimuse_unito

2. Create environment

conda create -n advis python=3.9 -y
conda activate advis

3. Install PyTorch

For CUDA 11.8:

conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia -y

For CPU only:

conda install pytorch torchvision torchaudio cpuonly -c pytorch -y

4. Install ADVIS package

pip install -e .

This installs the package name advis and provides these CLI commands:

advis-train
advis-calibrate
advis-infer

---

Configuration

All default paths and parameters are stored in:

configs/config.yaml

Important fields:

data:
  dataset_version: V6
  dataset_type: fronttop

model:
  latent_dim: 64

training:
  epochs: 1000
  batch_size: 32

threshold:
  score_method: max
  offset: 1
  sigma: 1.0
  quantile: 1.0

You can still override most values from the command line.

---

Training

Train one safety area:

advis-train --config configs/config.yaml --safety_area RoboArm

Train all safety areas:

advis-train --config configs/config.yaml --safety_area ALL

Override parameters:

advis-train \
  --config configs/config.yaml \
  --safety_area PLeft \
  --epochs 500 \
  --batch_size 32

Equivalent script command:

python scripts/train.py --config configs/config.yaml --safety_area RoboArm

---

Threshold Calibration

Validation mode

Use this when only normal validation data is available.

advis-calibrate \
  --config configs/config.yaml \
  --mode val \
  --safety_area ALL

Labelled-test mode

Use this when labelled normal/anomalous test data is available.

advis-calibrate \
  --config configs/config.yaml \
  --mode test \
  --safety_area ALL \
  --gt_csv scripts/data/annotations/anom_metadata_unexpected_person.csv

Calibration outputs are saved under:

results/thresholds/

---

Inference

Run inference on preprocessed frames:

advis-infer \
  --config configs/config.yaml \
  --data_source preprocessed \
  --input_dir data/test_processed/unexpected_person \
  --safety_area RoboArm

Run inference on all safety areas:

advis-infer \
  --config configs/config.yaml \
  --data_source preprocessed \
  --input_dir data/test_processed/unexpected_person \
  --safety_area ALL

Run inference on video:

advis-infer \
  --config configs/config.yaml \
  --data_source video \
  --input_video data/videos/test_video.mp4 \
  --safety_area ALL

Run inference and save figures:

advis-infer \
  --config configs/config.yaml \
  --data_source preprocessed \
  --input_dir data/test_processed/unexpected_person \
  --safety_area ALL \
  --save_figures

---

Pipeline Figures

Workflow

Preprocessing

VAE-GAN Architecture

Sample Results

---

Supported Dataset

Dataset	Source	Description
Synthetic Palletizing	University of Granada, Valeria Lab	Synthetic collaborative robotics dataset for DistriMuSe UC3

Dataset generation tool:

https://github.com/valerialabugr/SimIndus-Dataset

---

Outputs

results/
├── models/
├── thresholds/
├── training/
├── inference/
└── logs/

Large checkpoints and generated outputs should not be committed to Git.

---

Notes on Refactoring

The active scripts now import only from the installable advis package. The old notebook utility files are removed from the runtime path. Reusable code is organized into advis.models, advis.cad_utils, advis.scoring, advis.data, and advis.config.

---

Citation

Coming Soon

---

Acknowledgements

This work is developed in the context of the DistriMuSe project on distributed multi-sensor systems for human safety and health.

---

Keywords

Anomaly Detection · VAE-GAN · Explainable AI · Industrial Safety · Collaborative Robotics · Computer Vision · Human-Robot Interaction