🧊 mc-chassis

Minecraft — reconfigured as a robot.

mc-chassis exposes a Minecraft player as a standard embodied-AI chassis — the exact camera + joint-state observation space and delta-movement action space that every VLA model (SmolVLA, OpenVLA, π₀.₅, …) knows how to drive.

One adapter. Your VLA doesn't know it's in a game.

---

✨ Concept

VLA models are trained to control robots — cameras in, motor commands out. mc-chassis is the thinnest possible adapter that makes a Minecraft player look exactly like a robot to any VLA:

VLA Model (SmolVLA / OpenVLA / π₀.₅ / …)
        │
        │  standard Obs + Action
        │  ─────────────────────►
        ▼
  ┌─────────────────────────┐
  │      mc-chassis          │  ← this repo
  │  (MinecraftChassis)      │
  └──────────┬──────────────┘
             │
             │  Mineflayer JSON-over-TCP
             ▼
  ┌─────────────────────────┐
  │      bridge.js           │
  │   (Node.js Mineflayer)   │
  └──────────┬──────────────┘
             │
             ▼
  ⛏️  Minecraft Java Edition

No changes to the VLA. No custom training. No hacky input simulation. Just a compatibility layer that says: "this game character is your robot."

---

📦 Architecture

mc-chassis/
│
├── chassis/                          ◀  Abstract interface (game-agnostic)
│   ├── base.py                         Chassis ABC, Obs & Action dataclasses
│   └── __init__.py
│
├── adapters/
│   └── minecraft/                    ◀  Minecraft implementation
│       ├── adapter.py                  MinecraftChassis ← Python → TCP
│       ├── bridge.js                   Mineflayer bridge  ← Node.js
│       └── __init__.py
│
├── pyproject.toml
├── README.md
└── LICENSE (MIT)

The abstract Chassis class is deliberately three methods — anyone can write an adapter for any game and reuse the same VLA pipeline.

---

📐 Standard Interface

Observation Space

What every VLA model sees when it calls read_obs():

Field	Shape	Description
camera	(H, W, 3) uint8	RGB screenshot from the player's eyes
joint_states	(7,) float32	[x, z, yaw, pitch, health, food, y]
gripper_state	scalar float	0.0 (empty hand) / 1.0 (holding item / tool equipped)
metadata	dict	Held item name, block at feet, nearby entity count, …

Action Space

What the VLA sends to apply_action():

Field	Shape	Description
delta	(2,) float	[dx, dz] movement vector, normalised ~[-1, 1]
head_rotation	(2,) float	[yaw, pitch] look direction in radians
grasp	scalar float	0.0 = release / 1.0 = attack / interact / grab
metadata	dict	jump, sprint, chat, item selection, …

The Minecraft adapter buffers high-frequency actions (20 Hz VLA → 20-tick/s Minecraft server) so motion stays smooth even when the VLA is slower than the game loop.

---

🚀 Quick Start

Prerequisites

Dependency	Version	Why
Minecraft Java Edition	1.21.x (recommended 1.21.6)	The game the bot plays
Node.js	v18 or v20 LTS	Mineflayer bridge runtime
Python	≥ 3.10	Chassis adapter & VLA integration

1. Install

# Python side — install the chassis library
pip install -e .

# JavaScript side — set up the Mineflayer bridge
cd adapters/minecraft
npm init -y
npm install mineflayer
cd ../..

2. Launch Minecraft

1. Open your Minecraft world
2. Press Esc → Open to LAN
3. Note the port (default: 55916)

3. Run the Mineflayer bridge

node adapters/minecraft/bridge.js

You should see:

[bridge] listening on port 55917
[bridge] connecting to Minecraft at localhost:55916
[bridge] bot spawned

4. Drive the bot from Python

import numpy as np
from chassis.base import Action
from adapters.minecraft import MinecraftChassis

chassis = MinecraftChassis()

# See what the bot sees
obs = chassis.read_obs()
print(f"📍 Position: ({obs.joint_states[0]:.1f}, {obs.joint_states[6]:.1f}, {obs.joint_states[1]:.1f})")
print(f"❤️  Health: {obs.joint_states[4]:.0f} / 20")
print(f"🍗 Food: {obs.joint_states[5]:.0f} / 20")
print(f"🤲 Holding item: {obs.metadata.get('held_item_name')}")

# Walk forward and look right
chassis.apply_action(Action(
    delta=np.array([0.0, -0.8]),
    head_rotation=np.array([0.5, 0.0]),
    grasp=0.0,
))

# Attack!
chassis.apply_action(Action(
    delta=np.array([0.0, 0.0]),
    head_rotation=np.array([0.0, 0.0]),
    grasp=1.0,
))

chassis.close()

---

🔧 Environment Variables

Variable	Default	Description
MC_HOST	localhost	Minecraft server hostname
MC_PORT	55916	Minecraft LAN port
BRIDGE_PORT	55917	Mineflayer bridge TCP port
BOT_USERNAME	ChassisBot	Bot's in-game name

---

🗺️ Roadmap

• Abstract chassis interface (game-agnostic)
• Minecraft adapter — Python side
• Mineflayer bridge — JavaScript side
• Camera stream (real screenshot → numpy array)
• Example: drive a bot with SmolVLA (450M)
• Example: drive a bot with OpenVLA-OFT
• VLASH-style asynchronous action chunking
• Action smoothing / interpolation between VLA frames
• More game adapters (Minecraft is just the first!)

---

🔗 Related Projects

Project	What it does
MindCraft	LLM-driven Minecraft bots (text state + code execution)
Mineflayer	Node.js API for Minecraft bots
VLASH	Asynchronous VLA inference for real-time control
PuppetCraft	LLM-as-director for Minecraft entities
LeRobot	HuggingFace's real-world robot learning framework

---

📄 License

MIT — do what you want, just don't blame us when your VLA thinks creepers are cute.

---

_{Built by Louis Yang · inspired by a conversation about why games shouldn't be robot simulators, but are.}