Skip to content

Latest commit

 

History

History
405 lines (324 loc) · 17.2 KB

File metadata and controls

405 lines (324 loc) · 17.2 KB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

This is the Mesh API repository containing protobuf definitions and multi-language SDK generation for Mesh's trading platform APIs. The repository follows a schema-first approach where protobuf definitions are the source of truth, with generated client libraries in Go, Python, and TypeScript.

Workspace Structure: This is a yarn workspace with the following members:

  • ts/ - TypeScript SDK package (@meshtrade/api)
  • docs/ - Docusaurus documentation site (mesh-api-docs)
  • tool/protoc-gen-meshts/cmd/ - TypeScript protobuf generator tool

Workspace Commands: All commands should be run from the repository root:

  • yarn install - Install all workspace dependencies
  • yarn build - Build TypeScript SDK
  • yarn build:docs - Build documentation site
  • yarn start:docs - Start documentation dev server at http://localhost:3000/api/
  • yarn serve:docs - Serve built documentation
  • yarn test - Run TypeScript tests
  • yarn lint - Lint TypeScript code
  • yarn generate - Run code generation script

Key Commands

Documentation Site

Code Generation & Testing

  • ./dev/tool.sh all - Main development tool that cleans, generates, and builds all client libraries
  • ./dev/tool.sh generate - Generate code from protobuf definitions
  • ./dev/tool.sh build - Build SDK packages
  • ./dev/tool.sh test - Run comprehensive test suites for all languages
  • ./dev/tool.sh doctor - Check development environment health
  • ./dev/tool.sh clean - Clean generated files
  • buf generate - Direct buf generation (used internally by dev scripts)

Playwright Testing Screenshots

Playwright screenshots for testing purposes should be stored in the docs/testing_screenshots directory. CRITICAL: Playwright should always be run in headless mode.

Language-Specific Commands

Go

  • ./dev/test/go.sh - Run comprehensive Go tests with coverage and linting
  • go test ./... - Run basic Go tests
  • go mod tidy - Clean up Go module dependencies

Python

  • ./dev/test/python.sh - Run comprehensive Python tests with coverage and linting
  • pip install -e ".[dev]" - Install Python package in development mode
  • pytest - Run Python tests
  • ruff check python/ --fix - Lint Python code (CRITICAL: Always run after Python changes)
  • ruff format python/ - Format Python code
  • tox - Run full test suite with tox

TypeScript

  • ./dev/test/typescript.sh - Run comprehensive TypeScript tests with Jest and linting
  • yarn test - Run TypeScript tests
  • yarn build - Build TypeScript library
  • yarn lint - Lint TypeScript code

Java

  • ./dev/test/java.sh - Run comprehensive Java tests with Maven and coverage analysis

Python Environment Setup

CRITICAL: Python code must run in a virtual environment:

# Setup virtual environment
python -m venv .venv
source .venv/bin/activate

# Install dependencies in development mode
pip install -e ".[dev]"

# Run tests
pytest

# Run linting (CRITICAL: Always run after Python changes)
ruff check . --fix
ruff format .

Python Linting Standards

Configuration: Uses ruff with 150-character line limit (see pyproject.toml)

Key Linting Rules:

  • E501: Line length (150 chars max)
  • E711: Use is/is not for None comparisons (never == None)
  • F401: Remove unused imports OR add proper __all__ lists to modules
  • SIM112: Environment variables must use UPPER_CASE naming

Critical Python Module Organization Rule:

  • NEVER put code other than imports in __init__.py files - use dedicated modules instead. This ensures clean module structure and prevents import issues.

Best Practices for Line Length:

  1. Use parentheses for implicit line continuation on function calls
  2. Break long docstrings across multiple lines
  3. Use # noqa: E501 ONLY for extreme cases (300+ chars) like malformed test data
  4. Break long f-strings using multiple f-string concatenation

Example Good Line Breaking:

# Good - function call with parentheses
result = (
    some_long_function_name_that_exceeds_limit(
        parameter_one=value,
        parameter_two=other_value,
    )
)

# Good - docstring breaking
def function():
    """
    This is a long docstring that needs to be broken across multiple lines
    to respect the line length limit while maintaining readability.
    """

# Good - f-string breaking
message = (
    f"This is a long message with {variable_one} and "
    f"another {variable_two} that spans multiple lines"
)

Java Linting Standards

Configuration: Uses comprehensive linting stack with multiple tools (see java/pom.xml)

Linting Tools:

  • Checkstyle: Code style enforcement (Google Java Style Guide)
  • PMD: Code quality analysis and complexity checking
  • Error Prone: Compile-time bug detection (Google)

Key Style Rules:

  • Line Length: 120 characters max (adjusted for Java verbosity)
  • JavaDoc: Required for all public classes, methods, and constructors
  • Indentation: 4 spaces (no tabs)
  • Naming: camelCase for variables/methods, PascalCase for classes, UPPER_SNAKE_CASE for constants
  • Imports: No star imports, organized by groups (java., javax., , co.meshtrade.)

Running Linters:

cd java

# Run all linters (part of test suite)
mvn verify

# Run individual linters
mvn checkstyle:check     # Code style
mvn pmd:check            # Code quality

# View HTML reports
open target/site/checkstyle.html
open target/site/pmd.html

Configuration Files:

  • java/checkstyle.xml - Checkstyle rules (Google Style)
  • java/pmd-ruleset.xml - PMD custom rules
  • java/.editorconfig - Editor consistency settings

Best Practices:

  1. Run mvn verify before committing to catch all violations
  2. Fix violations immediately - don't accumulate technical debt
  3. Use @SuppressWarnings sparingly and only with justification comments
  4. Generated protobuf code is automatically excluded from all checks

Architecture

Directory Structure

  • /proto/ - Protobuf API definitions (source of truth)
    • /meshtrade/ - All API services organized by domain
    • Each service follows pattern: domain/resource/v1/
  • /go/ - Generated Go client libraries and utilities
  • /python/ - Generated Python packages with additional utilities
  • /ts/ - Generated TypeScript modules
  • /docs/ - Docusaurus-based documentation site
    • 📖 IMPORTANT FOR AGENTS: Always read /docs/CLAUDE.md for detailed documentation site guidance
    • docs/ - Main documentation content (MDX files)
    • blog/ - News/blog posts
    • src/ - React components and pages
    • static/ - Static assets (images, logos, etc.)
    • docusaurus.config.ts - Docusaurus configuration
    • sidebars.ts - Navigation sidebar configuration
    • package.json - Documentation site dependencies
  • /dev/ - Development tools (generation, build, clean, test, deploy)
    • tool.sh - Main orchestration script with comprehensive help
    • generate/ - Code generation scripts for each language
    • build/ - Build scripts for SDK packages
    • clean/ - Cleanup scripts for generated files
    • test/ - Test execution scripts for all languages
    • env/ - Environment validation scripts and doctor tool
  • /tool/protoc-gen-meshgo/ - Custom protobuf generator for Go

API Services Structure

Services are organized by business domain:

  • compliance/client/v1 - Client compliance and KYC
  • iam/role/v1 & iam/group/v1 - Identity and access management
  • issuance_hub/instrument/v1 - Financial instrument management
  • trading/direct_order/v1, trading/limit_order/v1, trading/spot/v1 - Trading services
  • wallet/account/v1 - Account and wallet management
  • type/v1 - Shared types used across services (Amount, Decimal, Token, etc.)

Code Generation Flow

  1. Protobuf definitions in /proto/ define the API contracts
  2. ./dev/tool.sh all or ./dev/tool.sh generate runs comprehensive code generation:
    • Validates environment prerequisites for each language
    • Cleans generated files using language-specific cleanup scripts
    • Runs buf generate with individual language configurations
    • Applies post-processing (formatting, index generation)
  3. Multiple protobuf plugins generate language-specific code:
    • Go: Standard protobuf + gRPC + custom meshgo generator
    • Python: Standard protobuf generators + custom meshpy utilities
    • TypeScript: protobuf-js + grpc-web + custom meshts enhancements
    • Java: Standard protobuf + gRPC + custom meshjava utilities
    • Docs: Custom meshdoc generator for MDX documentation
  4. Language-specific build processes create final packages

Shared Types

The /proto/meshtrade/type/v1/ directory contains foundational types used across multiple services:

  • Decimal - High-precision decimal arithmetic
  • Amount - Monetary amounts with currency
  • Token - Blockchain token representations
  • Ledger - Ledger-related types
  • Custom Go and TypeScript utilities extend these with helper functions

Development Workflow

  1. Making API Changes: Always modify protobuf files in /proto/ first
  2. Code Generation: Run ./dev/tool.sh all to clean, generate, and build all client libraries
    • For selective generation: ./dev/tool.sh generate --targets=go,python
    • For individual languages: ./dev/tool.sh generate --targets=typescript
  3. Testing: Use the comprehensive testing infrastructure after generation
    • Run all tests: ./dev/tool.sh test
    • Run specific language tests: ./dev/tool.sh test --targets=python,java
    • Environment validation: ./dev/tool.sh doctor
  4. Version Management: API versions are managed through protobuf package paths (v1, v2, etc.)
  5. Environment Requirements: The dev tool validates all prerequisites automatically:
    • Go 1.21+, Python 3.12+ with active venv, Node.js 18+, Java 21, Maven, Yarn, buf

Testing Infrastructure

Comprehensive Test Execution

The testing system provides robust validation across all SDK languages:

# Test all languages with environment validation
./dev/tool.sh test

# Test specific languages  
./dev/tool.sh test --targets=python,java,typescript

# Verbose output for debugging
./dev/tool.sh test --targets=go --verbose

# Individual language tests
./dev/test/python.sh      # Python with pytest, coverage, ruff linting
./dev/test/java.sh        # Java with Maven, JaCoCo coverage
./dev/test/go.sh          # Go with race detection, coverage, golangci-lint
./dev/test/typescript.sh  # TypeScript with Jest, type checking, ESLint

Environment Health Validation

Before testing, validate your development environment:

# Comprehensive environment check
./dev/tool.sh doctor

# Individual environment validation
./dev/env/python.sh       # Python venv, dependencies
./dev/env/java.sh         # Java 21, Maven setup
./dev/env/go.sh           # Go version, modules
./dev/env/typescript.sh   # Node.js, Yarn, dependencies
./dev/env/general.sh      # buf, git, general tools

Test Features

Python Tests: pytest with coverage, ruff linting, virtual environment validation Java Tests: Maven surefire/failsafe, JaCoCo coverage Go Tests: Standard testing, race detection, coverage analysis, security linting TypeScript Tests: Jest framework, type checking, ESLint validation, build verification

CI/CD Integration

# Fail-fast for CI pipelines
./dev/test/all.sh --fail-fast

# Environment + testing workflow
./dev/tool.sh doctor && ./dev/tool.sh test

Documentation Work

🤖 For AI Agents Working with Documentation

CRITICAL: When working with documentation, ALWAYS read /docs/CLAUDE.md first for comprehensive documentation site guidance, including:

  • Playwright MCP setup and testing workflows
  • Background server management
  • Documentation site architecture
  • Testing and screenshot procedures

Documentation Site Workflow

Structure and Organization

  • Root README.md: Simplified overview with link to full documentation site
  • docs/docs/: Main documentation content in MDX format
    • introduction.mdx - Getting started guide
    • api-reference/ - Generated API documentation from protobuf
    • architecture/ - Architecture and design documentation
  • docs/blog/: News and updates
  • docs/src/: Custom React components and pages

Docusaurus Configuration

  • Uses @docusaurus/preset-classic with custom theme
  • Mermaid diagrams supported via @docusaurus/theme-mermaid
  • Configured for multi-language code examples (Go, Python, TypeScript, Protobuf)
  • Custom CSS styling in src/css/custom.css

Adding Documentation Pages

  1. Create MDX files in docs/docs/ with front matter:
    ---
    sidebar_position: 1
    title: Page Title
    ---
  2. Update sidebars.ts to include new pages in navigation
  3. Use MDX features like React components and code blocks

Documentation Maintenance

  • docs/docs/api-reference/: Auto-generated from protobuf using protoc-gen-meshdoc
  • Manual pages: Architecture, introduction, and other hand-written content
  • Navigation: Managed through sidebars.ts configuration

Important Notes

Code Generation

  • All generated files (*.pb.go, *_pb2.py, *pb.js, etc.) should not be manually edited
  • The repository uses buf for protobuf management and linting
  • Each language SDK is independently packaged and versioned
  • Breaking changes require new API versions (e.g., v1 -> v2)
  • Custom protobuf generator protoc-gen-meshgo creates additional Go utilities

Documentation Site

  • Docusaurus site is self-contained in /docs/ directory as a yarn workspace member
  • Use yarn start:docs from repository root to start development server
  • API documentation auto-generated from protobuf using protoc-gen-meshdoc tool
  • Site builds successfully and serves on http://localhost:3000/api/
  • Generated documentation includes interactive code examples and type definitions

Protobuf Service Patterns

  • Resource Service Naming: All resource services follow consistent pattern:
    • Method names include resource name (e.g., GetAccount, CreateClient, MintInstrument)
    • Request/Response messages include resource name (e.g., GetAccountRequest, ListClientsResponse)
    • Get/Create methods return the resource directly, not a response wrapper
  • Authorization Model: Uses Role enum from meshtrade/option/v1/role.proto:
    • File-level standard_roles option declares all roles used by service
    • Method-level roles option specifies which roles can access each method
    • Extension tags: standard_roles = 50003, roles = 50005, method_type = 50004
    • Role definitions follow pattern: ROLE_{DOMAIN}_{ADMIN|VIEWER} (e.g., ROLE_COMPLIANCE_ADMIN, ROLE_IAM_VIEWER)
    • Each service domain has both admin and viewer roles with appropriate permissions
  • Method Type Classification: Uses MethodType enum from meshtrade/option/v1/method_type.proto:
    • All RPC methods must specify method_type option as either METHOD_TYPE_READ or METHOD_TYPE_WRITE
    • Read operations (Get, List, Search) use METHOD_TYPE_READ
    • Write operations (Create, Update, Delete, Mint, Burn) use METHOD_TYPE_WRITE
  • Extension Tag Management: Be careful with protobuf extension tag conflicts across option files
  • Response Message Cleanup: Remove unused response messages when methods return resources directly

Protobuf Refactoring Best Practices

Role and Method Type Management

  • Always run ./dev/tool.sh all after protobuf changes to regenerate all language bindings
  • Use buf lint to validate protobuf syntax and style before generation
  • Role definitions must be added to meshtrade/option/v1/role.proto following the ROLE_{DOMAIN}_{ADMIN|VIEWER} pattern
  • Method type annotations are mandatory for all RPC methods using METHOD_TYPE_READ or METHOD_TYPE_WRITE
  • File-level role declarations ensure service authorization model is self-documenting
  • Method-level role assignments control granular access permissions per operation

Code Generation Workflow

  1. Modify protobuf files in /proto/ directory
  2. Run buf lint to validate changes
  3. Run ./dev/tool.sh all to regenerate all client libraries
  4. Run language-specific tests and linting to verify correctness

Protobuf Syntax for Options

When adding options to protobuf files, keep the following in mind:

  • Option Syntax: Custom options for services or methods should be declared directly. The correct syntax is option (custom.option) = VALUE; and not option (custom.option) = { key: VALUE };.
  • Enum Scopes: When referencing enums from an imported .proto file (like method_type.proto or role.proto), you should use the enum value directly (e.g., METHOD_TYPE_READ) without the fully qualified package path, as long as the necessary import statement is present.
  • Option Scopes (FileOptions vs. ServiceOptions): It's critical to apply options at the correct level. For example, standard_roles is a FileOption and must be declared at the top level of the file, not within a service definition, which uses ServiceOptions.