SmartBiz v2 - Modern BI Platform

A complete re-architecture of SmartBiz transforming it into an enterprise-grade Business Intelligence platform.

🚀 Architecture Highlights

Tech Stack

Backend:

• Python 3.12 + FastAPI (async-first, modern)
• PostgreSQL 16 (operational database with Row-Level Security)
• ClickHouse (analytics database - sub-second queries)
• Redis 7 (caching, pub/sub, sessions)
• Arq (background tasks - replaces Celery)

Frontend:

• Next.js 14 (App Router)
• React 18 + TypeScript
• Zustand (client state)
• React Query (server state)
• Tailwind CSS + shadcn/ui
• Apache ECharts (visualizations)

Infrastructure:

• AWS (ECS Fargate, RDS, ElastiCache)
• Docker + GitHub Actions CI/CD
• Prometheus + Grafana monitoring

Key Features Implemented ✅

1. Multi-Tenancy with Row-Level Security

   • Tenant isolation at database level
   • PostgreSQL RLS policies prevent data leakage
   • Shared database architecture (cost-effective)

2. Modern Background Tasks (Arq)

   • Replaces Celery with async-native solution
   • Lighter weight, simpler setup
   • Better performance with FastAPI

3. Rate Limiting

   • Per-tenant limits (prevent noisy neighbors)
   • Per-user limits
   • Redis-backed distributed limiting

4. Structured Logging

   • structlog for JSON logs
   • Audit logging for compliance (GDPR, SOC 2)
   • Security event tracking

5. Connection Pooling (PgBouncer)

   • Transaction pooling mode
   • Critical for 10K+ concurrent users

📦 Project Structure

smartbiz-v2/
├── backend/
│   ├── app/
│   │   ├── api/v1/          # API endpoints
│   │   ├── core/            # Config, security, logging
│   │   ├── db/              # Database configuration
│   │   ├── middleware/      # Rate limiting, auth
│   │   ├── models/          # Database models
│   │   ├── services/        # Business logic
│   │   ├── tasks/           # Arq background tasks
│   │   └── utils/           # Utilities
│   ├── tests/               # Pytest tests
│   ├── alembic/             # Database migrations
│   ├── Dockerfile
│   ├── requirements.txt
│   └── pyproject.toml
├── frontend/                # Next.js app (TODO)
├── terraform/               # Infrastructure as Code (TODO)
└── docker-compose.yml       # Local development

🛠️ Development Setup

Prerequisites

• Python 3.12+
• Docker & Docker Compose
• Node.js 18+ (for frontend)

Quick Start

1. Clone and navigate to v2:

cd smartbiz-v2

2. Set up environment variables:

cp .env.example .env
# Edit .env and set secure passwords for production!

3. Start all services:

docker-compose up -d

This starts:

• PostgreSQL (port 5432)
• PgBouncer (port 6432)
• Redis (port 6379)
• ClickHouse (ports 8123, 9000)
• Backend API (port 8000)
• Arq worker

3. Access the API:

http://localhost:8000
http://localhost:8000/docs (API documentation)
http://localhost:8000/health (Health check)

Running Tests

cd backend
pytest

Running Migrations

cd backend
alembic upgrade head

🔐 Security Features

• Row-Level Security (RLS) for tenant isolation
• JWT authentication with refresh tokens
• Rate limiting (per-tenant, per-user)
• Audit logging for compliance
• Structured logging with security events
• CORS configuration
• Password hashing with bcrypt

📊 Performance Features

• PgBouncer connection pooling
• Redis caching
• ClickHouse for analytics
• Async/await throughout
• Efficient database queries
• Prometheus metrics

🚦 Next Steps

• Complete database models
• Implement API endpoints
• Set up CI/CD pipeline
• Set up Next.js frontend
• Create Terraform configurations
• Add E2E tests with Playwright

🔄 CI/CD Pipeline

Automated Workflows

The project uses GitHub Actions for continuous integration and deployment:

Workflow	Trigger	Description
CI	Push to main/develop, PRs	Linting, security scans, tests
Deploy	Push to main	Build & deploy to VPS

Required GitHub Secrets

Set these secrets in your GitHub repository settings:

VPS_HOST              # Your VPS IP or hostname
VPS_USER              # SSH user (e.g., deploy)
SSH_PRIVATE_KEY       # SSH private key for authentication
DEPLOY_PATH           # Deployment path (default: /opt/smartbiz-v2)

Deployment Process

1. Push to main triggers automatic deployment
2. CI checks run (lint, security, tests)
3. Docker image is built and pushed to GHCR
4. SSH deployment to VPS with zero-downtime
5. Health check verifies successful deployment

Manual Rollback

# Via GitHub Actions UI
# Select "rollback" as environment in workflow_dispatch

Local Testing Before Deploy

# Build and test locally
docker compose build
docker compose run --rm backend pytest

# Test production compose
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

📚 Documentation

• Architecture Decisions
• API Documentation
• Deployment Guide
• Development Guide

🤝 Contributing

This is a greenfield project. All modern best practices from 2025 are being applied.

📄 License

Proprietary - All rights reserved