Malkuth

Composable service-supervision toolkit for Rust

English · 简体中文 · 繁體中文 · 日本語 · 한국어 · Français · Español · Русский · العربية

Malkuth helps automated, long-running programs handle supervision — graceful shutdown, health probes, coordination locks, and rolling updates:

1. Pluggable transport — JSON-RPC over TCP, WebSocket, or IPC (Unix sockets / named pipes). One Transport trait, dispatched by URL scheme.
2. Supervised workers — spawn a process, monitor its health, restart it on failure, drain connections before shutdown.
3. Optional facilities — exit source, probes, heartbeat and drain hooks are traits. Use the defaults or supply your own.
4. A watchdog CLI — malkuth -- <cmd> wraps any program with file watching, a pod pool, and an L4 sticky reverse proxy.

As a CLI

malkuth [--watch PATH]... [--proxy PUBLIC:LO-HI] [--pod-count N] -- <cmd> [args...]

Run 5 parallel copies of your server (each listening on the PORT env var → they self-assign 3001–3005), fronted by a sticky proxy on 3000:

malkuth --watch ./src --watch ./res \
        --proxy 3000:3000-3999 --pod-count 5 \
        -- cargo run

The proxy routes each client IP to a fixed backend via consistent hashing, so a client keeps hitting the same pod until that pod restarts or scales down — the basis for gray release / rolling restart. On a file change it drains and restarts one pod at a time.

npx (no Rust toolchain required)

Prebuilt binaries are published to npm, so you can run malkuth with a single command — no cargo build:

npx @celestia-island/malkuth --watch ./src -- cargo run
npx @celestia-island/malkuth mcp        # the MCP server (needs the mcp build)

The @celestia-island/malkuth root package pulls the right platform subpackage (-linux-x64 / -darwin-arm64 / -win32-x64) automatically. To pin a version:

npx @celestia-island/malkuth@0.1.0 -- cargo run

As a library

[dependencies]
malkuth = "0.1"
# features: tcp (default) | ws | ipc | signals (default) | worker | probes |
#           file-lock | lease | pg-lock | replica | leader-follower | schema | cli

use std::sync::Arc;
use malkuth::{Client, Router, Server, Supervised, Transport};
use malkuth::transport::TcpTransport;
use serde_json::json;

#[tokio::main]
async fn main() -> std::io::Result<()> {
    // Bind once; build a router with the standard lifecycle RPC + your methods.
    let lis = TcpTransport.listen("tcp://127.0.0.1:0").await?;
    let supervised = Supervised::new().signals();           // OS-signal exit source
    let ctrl = supervised.drain_controller();
    let handler = Arc::new(
        Router::new()
            .lifecycle(ctrl, None)                          // Lifecycle.Drain/Status/...
            .route("ping", |_| Box::pin(async { Ok(json!("pong")) })),
    );
    // Race the server against the exit source, then run drain hooks.
    supervised.serve_rpc_listener(lis, handler).await
}

Need drain triggered by your own logic instead of signals? Implement malkuth::ExitSource and pass it via .exit(...). Want Postgres-backed coordination? The pg-lock feature provides a CoordinationLock backend.

Feature flags

Feature	Enables
tcp (default)	JSON-RPC over local/remote TCP (tokio::net)
ws	JSON-RPC over WebSocket (tokio-tungstenite)
ipc	JSON-RPC over local IPC (interprocess)
signals (default)	Default OS-signal ExitSource (tokio::signal)
worker	Supervised child-process workers (tokio::process)
probes	axum /healthz + /readyz router
file-lock	POSIX flock CoordinationLock backend (unix)
lease	File-lease CoordinationLock with TTL auto-expiry (crash-safe)
pg-lock	PostgreSQL pg_advisory_lock backend (tokio-postgres)
replica	In-memory InstanceRegistry
leader-follower	LeaseLeaderElector (over the lease backend)
schema	schemars::JsonSchema derives for wire types
cli	The malkuth watchdog binary (pod pool + sticky proxy)

Status

Layers 1–3 (lifecycle/drain, probes, listener handoff) and the JSON-RPC core (codec + server/client + tcp/ws/ipc transports) are implemented and tested end-to-end. The CLI pod pool + sticky proxy is working (e2e-verified). All three CoordinationLock backends (file-lock, lease, pg-lock) and the leader-follower LeaseLeaderElector are implemented. See docs/design/ for the design.

MCP server

Build malkuth with the mcp feature and run the stdio server — it exposes the supervision toolkit to AI coding assistants over the Model Context Protocol:

malkuth mcp

The server advertises two tools: malkuth_supervise (launch a set of workers under the supervisor with restart policies + a sliding-window rate limit; blocks until they exit or the timeout fires, then returns the final status snapshot) and malkuth_probe (HTTP healthz / readyz check against a service URL). Wire it into an MCP client:

{
  "mcpServers": {
    "malkuth": { "command": "malkuth", "args": ["mcp"] }
  }
}

The mcp feature implies worker + schema; it adds rmcp and a reqwest client for the probe tool.

License

SySL-1.0 (Synthetic Source License). See LICENSE or the SySL website.

MCP Server Deployment

For production MCP deployments, use an auto-restart wrapper to keep the server alive across updates without interrupting the client session.

Recommended launcher

#!/bin/bash while true; do /path/to/malkuth mcp sleep 0.2 done

How it works

1. The wrapper runs malkuth mcp in a while true loop.
2. If the process exits, it restarts within 0.2 seconds.
3. To update: kill $(pgrep -f "malkuth mcp" | head -1)
4. malkuth can also supervise other MCP tools — use it as a watcher for the entire MCP toolchain.