Development

This guide covers building AxonOps Schema Registry from source, running the test suite, and contributing changes.

Prerequisites
Building from Source
- Cross-Compilation
Project Structure
Running Tests
Code Quality
Running the Server Locally
API Documentation
Docker
- Build the Image
- Run with Docker
CI Pipeline
Code Conventions
Contributing
Related Documentation

Prerequisites

Go 1.26+ (download)
Docker (for integration, BDD, auth, KMS, and concurrency tests)
golangci-lint (install) for linting
Node.js (optional, for generating static API documentation with Redocly)

Optional tools for the full compatibility test suite:

Maven and Java 17+ (for Java Confluent serializer and data contract tests)
Python 3 (for Python Confluent serializer tests)

Building from Source

git clone https://github.com/axonops/axonops-schema-registry.git
cd axonops-schema-registry
make build

The binary is placed at ./build/schema-registry. The Makefile injects version, commit hash, and build date via linker flags automatically:

./build/schema-registry --version

To build manually without Make:

go build \
  -ldflags "-X main.version=$(git describe --tags --always) \
            -X main.commit=$(git rev-parse --short HEAD) \
            -X main.buildDate=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  -o schema-registry ./cmd/schema-registry

Cross-Compilation

Build binaries for all supported platforms:

make build-all

This produces:

build/schema-registry-linux-amd64
build/schema-registry-linux-arm64
build/schema-registry-darwin-amd64
build/schema-registry-darwin-arm64

Project Structure

cmd/schema-registry/              Entry point (cobra CLI)
cmd/generate-mcp-docs/            MCP reference doc generator
internal/
  analysis/                       Shared analysis (field extraction, quality scoring, fuzzy matching)
  api/                            HTTP server, handlers, types, OpenAPI
    handlers/                     REST endpoint handlers (schema, admin, account, analysis, DEK, exporter)
    types/                        Request/response structs + Confluent error codes
  auth/                           Authentication, RBAC, JWT, rate limiting, audit, TLS
  compatibility/                  Compatibility checkers (Avro, JSON Schema, Protobuf)
  config/                         YAML config loading with env var overrides
  context/                        Multi-tenant context management
  exporter/                       Schema exporter (Schema Linking) logic
  kms/                            KMS provider interface + implementations
    vault/                        HashiCorp Vault Transit
    aws/                          AWS KMS
    azure/                        Azure Key Vault
    gcp/                          GCP Cloud KMS
  mcp/                            MCP (Model Context Protocol) server
    content/                      Embedded markdown (16 glossary + 35 prompt files)
  metrics/                        Prometheus metrics (REST + MCP)
  registry/                       Core business logic (shared by REST + MCP)
  rules/                          Data contract rules engine (CEL, JSONata)
  schema/                         Schema parsers (Avro, JSON Schema, Protobuf)
  storage/                        Storage interface (71 methods) + backends
    memory/                       In-memory backend
    postgres/                     PostgreSQL backend
    mysql/                        MySQL backend
    cassandra/                    Cassandra backend
    vault/                        HashiCorp Vault auth storage
tests/
  api/                            External black-box API tests
  bdd/                            BDD/Cucumber tests (feature files + scenarios)
    features/                     Top-level feature files
    features/mcp/                 MCP-specific feature files
    steps/                        Step definition files
    configs/                      Per-backend config (memory, postgres, mysql, cassandra)
    docker-compose*.yml           Docker Compose files for test backends
  integration/                    Integration tests
  concurrency/                    Concurrency tests
  storage/conformance/            Storage conformance suite (shared across all backends)
  migration/                      Migration tests
  compatibility/
    go/                           Go serializer tests
    go-serde/                     Go SerDe data contract + CSFLE tests
    java/                         Java Confluent SerDe tests (wire compat + data contracts + CSFLE)
    python/                       Python serializer tests
api/
  openapi.yaml                    OpenAPI spec (embedded into binary)

Running Tests

The Makefile provides targets for every test category. Most targets accept a BACKEND variable to select the storage backend. Database containers are started and stopped automatically.

make help    # Show all available targets

Unit Tests

make test-unit

Runs go test -race -v -timeout 5m ./.... No Docker or external services required. This is the fastest feedback loop and SHOULD be run before every commit.

BDD Tests

BDD tests use Gherkin feature files with godog and cover all API endpoints, compatibility rules, error codes, MCP tools/resources/prompts, permissions, encryption, exporters, data contracts, and operational resilience.

make test-bdd                        # In-process, memory backend (no Docker)
make test-bdd-functional             # Same as above (explicit alias)
make test-bdd BACKEND=postgres       # Docker Compose with PostgreSQL
make test-bdd BACKEND=mysql          # Docker Compose with MySQL
make test-bdd BACKEND=cassandra      # Docker Compose with Cassandra
make test-bdd BACKEND=confluent      # Against Confluent Schema Registry 8.1.1
make test-bdd BACKEND=all            # All backends sequentially

Additional BDD targets for specific scenarios:

make test-bdd-db BACKEND=postgres    # In-process server with real DB
make test-bdd-db BACKEND=all         # In-process server with all DBs
make test-bdd-auth BACKEND=postgres  # BDD auth tests with real DB
make test-bdd-auth BACKEND=all       # BDD auth tests with all DBs
make test-bdd-kms BACKEND=memory     # BDD KMS encryption tests (Vault + OpenBao)
make test-bdd-kms BACKEND=all        # BDD KMS tests with all backends

# Audit BDD tests
make test-bdd-rest-audit             # REST audit event assertions
make test-bdd-mcp-audit              # MCP audit event assertions
make test-bdd-mcp-metrics            # MCP Prometheus metrics tests
make test-bdd-audit-outputs          # Multi-output audit (file + syslog TLS + webhook)

The confluent backend starts a real Confluent Schema Registry via Docker Compose to verify wire compatibility.

The test-bdd-audit-outputs target uses a Docker Compose overlay (docker-compose.audit-outputs.yml) that adds syslog-ng (TCP+TLS on port 6514) and a webhook-receiver container alongside the schema registry.

Integration Tests

Integration tests exercise the full stack against real database backends. The Makefile starts the required database container, runs the tests, and tears down the container.

make test-integration BACKEND=postgres
make test-integration BACKEND=mysql
make test-integration BACKEND=cassandra
make test-integration BACKEND=all

There is no memory backend for integration tests -- they require a database.

Concurrency Tests

Tests concurrent schema registration, version updates, mixed operations, hot subject contention, schema idempotency, and ID uniqueness under parallel load.

make test-concurrency BACKEND=postgres
make test-concurrency BACKEND=mysql
make test-concurrency BACKEND=cassandra
make test-concurrency BACKEND=all

Storage Conformance Tests

A shared test suite that verifies identical behavior across all storage backends. This catches cases where a backend silently differs from the expected contract.

make test-conformance                     # Memory (no Docker)
make test-conformance BACKEND=postgres    # PostgreSQL
make test-conformance BACKEND=mysql       # MySQL
make test-conformance BACKEND=cassandra   # Cassandra
make test-conformance BACKEND=all         # All backends

Auth Tests

Each auth test suite starts its own service container (OpenLDAP, HashiCorp Vault, or Keycloak).

make test-ldap       # LDAP integration tests
make test-vault      # Vault integration tests
make test-oidc       # OIDC integration tests
make test-auth       # All three

Migration Tests

Tests schema import and migration workflows, including Go unit tests and shell-based integration tests that exercise the binary directly.

make test-migration

API Endpoint Tests

Starts a schema-registry binary with in-memory storage and runs external HTTP tests against it.

make test-api

Compatibility Tests

Verifies wire compatibility with real Confluent serializer clients in Go, Java (4 Confluent versions), and Python (3 confluent-kafka versions). Requires Maven and Python 3 for the full suite; missing runtimes are skipped with a warning.

make test-compatibility

Data Contract & CSFLE Tests

Java integration tests using real Confluent SerDe clients to verify end-to-end data contract rule execution and CSFLE field-level encryption against the running schema registry.

# Start schema registry
make run &

# Data contract tests (no KMS required)
cd tests/compatibility/java
mvn test -P confluent-8.1 -Dschema.registry.url=http://localhost:8081 -Dgroups=data-contracts

# CSFLE tests (requires Vault)
cd tests/bdd && docker compose -f docker-compose.kms.yml up -d vault && docker compose -f docker-compose.kms.yml run --rm setup-kms
cd ../../tests/compatibility/java
mvn test -P confluent-8.1 \
  -Dschema.registry.url=http://localhost:8081 \
  -Dgroups=csfle \
  -Dvault.url=http://localhost:18200 \
  -Dvault.token=test-root-token

Requires Maven and Java 17+. The tests use JUnit 5 tags (data-contracts, csfle) to separate data contract tests from CSFLE tests that require KMS infrastructure.

Go SerDe Data Contract & CSFLE Tests

Go integration tests using confluent-kafka-go/v2 (v2.8.0) schema registry client to verify data contract rule execution (CEL conditions, CEL_FIELD transforms, JSONata migration rules, global policies) and CSFLE field-level encryption via Vault Transit KMS. No CGO is required since only the schemaregistry/* sub-packages are imported.

30 tests across 7 files in tests/compatibility/go-serde/.

# Start schema registry
make run &

# Data contract tests only (no KMS required)
cd tests/compatibility/go-serde
SCHEMA_REGISTRY_URL=http://localhost:8081 go test -v -run "TestCel|TestMigration|TestGlobalPolicies" ./...

# CSFLE tests (requires Vault)
cd tests/bdd && docker compose -f docker-compose.kms.yml up -d vault && docker compose -f docker-compose.kms.yml run --rm setup-kms
cd ../../tests/compatibility/go-serde
SCHEMA_REGISTRY_URL=http://localhost:8081 \
  VAULT_URL=http://localhost:18200 \
  VAULT_TOKEN=test-root-token \
  go test -v -run "TestCsfle" -timeout 10m ./...

# All Go SerDe tests
SCHEMA_REGISTRY_URL=http://localhost:8081 \
  VAULT_URL=http://localhost:18200 \
  VAULT_TOKEN=test-root-token \
  go test -v -timeout 10m ./...

A convenience script is also available: tests/compatibility/go-serde/run_tests.sh.

Full Test Suite

Run everything:

make test                    # All tests with default backend (memory where applicable)
make test BACKEND=all        # All tests against all backends

Coverage

make test-coverage

Generates a coverage report at ./coverage/coverage.html.

Code Quality

make lint           # Run golangci-lint (configuration in .golangci.yaml)
make fmt            # Format code with gofmt

The linter configuration is in .golangci.yaml at the repository root. CI runs both golangci-lint, go vet, gosec, and Trivy on every push.

Running the Server Locally

Start with the default in-memory backend:

make run

Or with hot reload during development (requires air):

make dev

To run with a custom configuration file:

./build/schema-registry --config config.yaml

To enable the MCP server alongside the REST API, add to your config:

mcp:
  enabled: true
  host: "127.0.0.1"
  port: 9081

Or via environment variable:

SCHEMA_REGISTRY_MCP_ENABLED=true ./build/schema-registry --config config.yaml

API Documentation

OpenAPI Spec

The OpenAPI specification lives at api/openapi.yaml and is embedded into the binary at compile time. A test (TestOpenAPISpecMatchesRoutes) enforces bidirectional sync between the spec and the router -- every route in the router must appear in the spec and vice versa.

Swagger UI

Set server.docs_enabled: true in the configuration file to serve Swagger UI at /docs.

Static Documentation

Generate a static HTML version of the API documentation using Redocly:

make docs-api

Output is written to docs/api/index.html.

MCP Reference

Generate the MCP API reference (all tools, resources, and prompts) from live server introspection:

make docs-mcp

Output is written to docs/mcp-reference.md.

Docker

Build the Image

make docker-build

This builds a multi-stage image (Go build stage, then Alpine runtime) and tags it as axonops/schema-registry:latest and axonops/schema-registry:<version>.

Run with Docker

make docker-run

Or manually:

docker run -p 8081:8081 axonops/schema-registry:latest

To expose the MCP server:

docker run -p 8081:8081 -p 9081:9081 \
  -e SCHEMA_REGISTRY_MCP_ENABLED=true \
  -e SCHEMA_REGISTRY_MCP_HOST=0.0.0.0 \
  axonops/schema-registry:latest

CI Pipeline

GitHub Actions runs 43 jobs on every push to main or feature/** branches:

Build & Unit Tests: Compile binary + all test binaries, run unit tests with coverage
Static Analysis: golangci-lint, go vet, gosec, Trivy vulnerability scanner
Docker: Multi-stage Docker image build verification
Database Tests: Integration + concurrency against PostgreSQL 15, MySQL 8, Cassandra 5.0
Conformance: Storage conformance against memory, PostgreSQL, MySQL, Cassandra (4 jobs)
API: Black-box API endpoint tests
Auth: LDAP (OpenLDAP), Vault (HashiCorp Vault 1.15), OIDC (Keycloak 24.0)
Migration: Import/migration tests
BDD (Docker Compose): memory, PostgreSQL, MySQL, Cassandra, Confluent 8.1.1 (5 jobs)
BDD Functional: In-process functional tests
BDD DB: In-process with real DB: PostgreSQL, MySQL, Cassandra (3 jobs)
BDD Auth: Auth BDD with real DB: PostgreSQL, MySQL, Cassandra (3 jobs)
BDD KMS: KMS encryption: memory, PostgreSQL, MySQL, Cassandra (4 jobs)
BDD Audit: REST audit, MCP audit, MCP metrics, audit outputs with syslog TLS + webhook (4 jobs)
Compatibility: Go + Java (4 Confluent versions) + Python (3 versions)
Data Contracts: Java SerDe, Go SerDe, Python SerDe — data contract + CSFLE with Vault (3 jobs)

The build job compiles all test binaries and uploads them as artifacts. Downstream jobs download pre-compiled binaries to avoid redundant compilation.

Code Conventions

Standard Go formatting (gofmt), enforced by golangci-lint
Structured logging via log/slog
Context-first function signatures: func (s *Store) Method(ctx context.Context, ...) error
Multi-tenant context: all storage/registry methods take registryCtx string (default ".")
Sentinel errors for storage operations, compared with errors.Is()
Build tags to separate test categories: //go:build integration, //go:build bdd, //go:build api, //go:build concurrency, //go:build conformance, //go:build ldap, //go:build vault, //go:build oidc, //go:build migration
Schema fingerprints use SHA-256 for content-addressed deduplication
Soft-delete with a boolean flag, not physical deletion
Global configuration uses an empty string ("") as the subject key
The import API preserves original schema IDs for Confluent migration
Both REST handlers and MCP tools call registry.Registry for all business logic (shared layer)
BDD-first testing: every feature MUST have BDD coverage; Makefile targets are the canonical way to run tests

Contributing

Fork the repository
Create a feature branch from main
Write tests for new functionality (BDD feature files + unit tests)
Ensure unit tests and linting pass: make test-unit && make lint
Run BDD tests: make test-bdd
Submit a pull request

For larger changes, open an issue first to discuss the approach.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Development

Contents

Prerequisites

Building from Source

Cross-Compilation

Project Structure

Running Tests

Unit Tests

BDD Tests

Integration Tests

Concurrency Tests

Storage Conformance Tests

Auth Tests

Migration Tests

API Endpoint Tests

Compatibility Tests

Data Contract & CSFLE Tests

Go SerDe Data Contract & CSFLE Tests

Full Test Suite

Coverage

Code Quality

Running the Server Locally

API Documentation

OpenAPI Spec

Swagger UI

Static Documentation

MCP Reference

Docker

Build the Image

Run with Docker

CI Pipeline

Code Conventions

Contributing

Related Documentation

Uh oh!

FilesExpand file tree

development.md

Latest commit

History

development.md

File metadata and controls

Development

Contents

Prerequisites

Building from Source

Cross-Compilation

Project Structure

Running Tests

Unit Tests

BDD Tests

Integration Tests

Concurrency Tests

Storage Conformance Tests

Auth Tests

Migration Tests

API Endpoint Tests

Compatibility Tests

Data Contract & CSFLE Tests

Go SerDe Data Contract & CSFLE Tests

Full Test Suite

Coverage

Code Quality

Running the Server Locally

API Documentation

OpenAPI Spec

Swagger UI

Static Documentation

MCP Reference

Docker

Build the Image

Run with Docker

CI Pipeline

Code Conventions

Contributing

Related Documentation