LabCouncil (Global Hack Week: Agents Week)

LabCouncil is a Flask web application that convenes five persistent Backboard AI assistants to turn a research idea into a reviewed, feasible experimental design and a scored final decision. The single-page interface streams live workflow status and presents the council's work in seven Markdown-rendered report tabs.

Installation

Requirements: Python 3.10+ and a Backboard API key.

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txt

Set the API key in the process environment. It is read only at runtime and is never stored in source code or sent to the browser.

export BACKBOARD_API_KEY="your-backboard-api-key"

Optional model overrides must be supplied as a pair. If omitted, the Backboard API defaults are used.

export BACKBOARD_LLM_PROVIDER="openai"
export BACKBOARD_MODEL_NAME="gpt-4o"

Run

python3 app.py

Open http://127.0.0.1:5000. Check service readiness at http://127.0.0.1:5000/api/health.

LABCOUNCIL_HOST and LABCOUNCIL_PORT may be set to change the bind address. Backboard calls default to a 120-second request timeout; set BACKBOARD_TIMEOUT to override it.

Architecture

Browser SPA
  ├─ POST /api/design (streaming NDJSON status + final report)
  └─ GET  /api/health
             │
          Flask app
             │
   BackboardCouncil service
     ├─ assistant registry (agents.json)
     ├─ thread orchestration
     ├─ one-retry API boundary
     └─ response and decision validation
             │
        Backboard SDK/API

• app.py contains HTTP routes, input validation, the worker thread, and NDJSON streaming.
• labcouncil/backboard_service.py contains assistant definitions, assistant reuse, retry behavior, thread/message operations, workflow orchestration, and Program Chair response validation.
• templates/index.html and static/ contain the responsive single-page interface.
• Each failed assistant, thread, or message API operation is attempted once more and then surfaced as a readable error.

Agent workflow

1. Research Scientist receives the idea in a new thread and develops the question, hypotheses, variables, controls, and initial experiment.
2. Skeptical Reviewer receives the complete Scientist output in a new thread and identifies at least three concrete issues and fixes.
3. Experimentalist receives both labeled outputs in a new thread and creates the rigorous protocol.
4. Resource Manager receives the protocol in a new thread, trims unnecessary work, estimates effort, and creates a Minimum Viable Experiment.
5. The optimized plan returns to the Reviewer's same thread for PASS or REVISE. On REVISE, the feedback returns to the Experimentalist's same thread for exactly one revision.
6. Program Chair receives every contribution and the final protocol, assigns four 1–10 scores, computes the overall mean, and returns APPROVED, REVISE, or REJECT.

The report always has seven sections: Hypothesis, Peer Review, Experiment Protocol, Feasibility, Final Review, Revision, and Council Decision. When the final review passes, Revision contains No revision required.

Assistant reuse

On the first successful run, LabCouncil creates the five named Backboard assistants and atomically writes their IDs to agents.json. Every subsequent run loads those IDs and creates only new conversation threads; it does not recreate assistants.

agents.json must be a JSON object containing exactly these non-empty string keys:

{
  "chair": "...",
  "experimentalist": "...",
  "resource_manager": "...",
  "reviewer": "...",
  "scientist": "..."
}

If the file is malformed, LabCouncil stops with a clear error rather than accidentally creating duplicate assistants. Fix the file, or remove it intentionally to create a fresh set on the next run.

API

GET /api/health

Returns service status and whether BACKBOARD_API_KEY is configured. It never returns the key.

POST /api/design

Request:

{"idea": "Does data augmentation improve image classification accuracy on small datasets?"}

The response uses application/x-ndjson: status objects arrive while each agent works, followed by one complete object containing all seven results and workflow metadata. Input must be non-empty JSON text.