Skip to content

feat: extract KMS abstraction into standalone Go library (axonops/go-kms) #383

Description

@millerjp

Summary

The internal/kms package in axonops-schema-registry provides a pluggable KMS (Key Management Service) abstraction for key wrapping, unwrapping, and data key generation across multiple cloud and on-premises providers. This package is entirely generic — it has no schema-registry-specific logic — and should be extracted into a standalone, community-available Go library at github.com/axonops/go-kms.

The library will provide a minimal, composable interface for KMS key wrapping operations (not envelope encryption — that is a higher-level concern for the consumer). It targets the same quality bar as Go standard library packages (encoding/json, net/http): small interface surface, zero-dependency core, pluggable providers via separate modules, and comprehensive observability hooks. See Market Gap for positioning against existing solutions.

This extraction achieves three goals: (1) reducing the schema-registry test matrix by moving provider-specific unit and integration tests into the library's own CI, (2) enabling reuse in other AxonOps projects and the broader Go community, and (3) establishing a clean separation between KMS primitives and application-level concerns like DEK/KEK management, algorithm mapping, and audit event taxonomy.

Goals

  • Reduce schema-registry CI scope — all provider-specific tests (mock HTTP servers for Vault/AWS, SoftHSM for PKCS#11, PyKMIP, etc.) move to go-kms CI
  • Enable reuse — any Go application needing multi-provider KMS key wrapping can import go-kms without pulling in schema-registry
  • Community contribution — Apache-2.0 licensed, idiomatic Go, production-grade library for the open-source ecosystem

Naming & Publishing

Field Value
Repository github.com/axonops/go-kms
Core module github.com/axonops/go-kms
Provider modules github.com/axonops/go-kms/providers/{name} (separate go.mod per provider)
Core package kms
Go version 1.22+ (for log/slog stability, generics maturity)
License Apache-2.0
Versioning Semver (v0.1.0 initial, v1.0.0 after schema-registry migration validates the API)
Publishing Tag-based releases, pkg.go.dev auto-indexing

Multi-module rationale: The core module (github.com/axonops/go-kms) has zero external dependencies — only Go stdlib. Each provider lives in its own Go module with its own go.mod, so consumers only pull in the SDK dependencies they actually use. This follows the pattern established by OpenTelemetry Go and avoids forcing all consumers to transitively depend on every cloud SDK.

Naming convention: Provider module names use lowercase without hyphens: awskms, azurekms, gcpkms, ocikms, ovhkms, pkcs11kms, kmipkms. The vault and openbao providers use their product names directly.

Features Required

Core Provider Interface

The Provider interface is the central abstraction. It defines three key wrapping operations plus lifecycle methods:

package kms

import "context"

// Provider wraps and unwraps key material using an external KMS.
//
// Implementations MUST be safe for concurrent use by multiple goroutines.
// All operations accept a context for cancellation and deadline propagation.
type Provider interface {
    // Wrap encrypts plaintext key material using the KMS key identified by keyID.
    // The keyID format is provider-specific (e.g., ARN for AWS, key name for Vault,
    // OCID for OCI, PKCS#11 key label, etc.).
    //
    // Returns the encrypted (wrapped) key material.
    Wrap(ctx context.Context, keyID string, plaintext []byte, opts ...Option) ([]byte, error)

    // Unwrap decrypts previously wrapped key material using the KMS key identified by keyID.
    //
    // Returns the original plaintext key material.
    Unwrap(ctx context.Context, keyID string, ciphertext []byte, opts ...Option) ([]byte, error)

    // GenerateKey generates size bytes of cryptographically random key material,
    // wraps it using the KMS key identified by keyID, and returns both the plaintext
    // and wrapped forms.
    //
    // The plaintext MUST be generated using crypto/rand. The size parameter is in bytes
    // (e.g., 16 for AES-128, 32 for AES-256, 64 for AES-256-SIV).
    //
    // Providers that support native data key generation (e.g., AWS KMS GenerateDataKey)
    // SHOULD use the native API. Providers without native support MUST generate locally
    // and call Wrap.
    GenerateKey(ctx context.Context, keyID string, size int, opts ...Option) (plaintext, wrapped []byte, err error)

    // Type returns the provider type identifier (e.g., "aws-kms", "hcvault", "pkcs11").
    // This value is used as the registry key and MUST be unique across providers.
    Type() string

    // Close releases any resources held by the provider (connections, sessions, etc.).
    // After Close returns, all other methods MUST return ErrProviderClosed.
    Close() error
}

Design decisions:

  • GenerateKey takes size int (bytes), not algorithm names. Algorithm-to-key-size mapping (e.g., AES256_GCM → 32) is an application concern. The library generates size random bytes and wraps them.
  • keyID is an opaque string. Each provider interprets it according to its own conventions (AWS ARN, Vault key name, PKCS#11 label, OCI OCID, etc.).
  • All operations accept ...Option for per-operation overrides. Simple consumers ignore options entirely; advanced consumers pass provider-specific properties.
  • Thread-safety is mandatory. The Provider interface contract requires all implementations to be safe for concurrent use.

Options

// Option configures a single KMS operation.
type Option func(*OperationConfig)

// OperationConfig holds per-operation settings. Providers extract what they need.
type OperationConfig struct {
    Properties map[string]string // Provider-specific key-value configuration
}

// WithProperties sets provider-specific properties for this operation.
// Properties override the provider's base configuration for this call only.
//
// Example: kms.WithProperties(map[string]string{"vault.namespace": "prod"})
func WithProperties(props map[string]string) Option

// ApplyOptions processes options into an OperationConfig.
// Exported for use by provider implementations.
func ApplyOptions(opts ...Option) OperationConfig

Properties convention: Each provider defines its property keys with a dotted prefix matching its type identifier. For example, Vault properties use vault. prefix, AWS uses aws. prefix. Properties are documented per provider.

Provider Registry

The Registry manages multiple providers by type identifier with thread-safe access:

// Registry is a thread-safe container for KMS providers.
type Registry struct {
    // unexported: mu sync.RWMutex, providers map[string]Provider,
    //             logger *slog.Logger, auditSink AuditSink, metrics MetricsCollector
}

// NewRegistry creates a Registry with the given options.
func NewRegistry(opts ...RegistryOption) *Registry

// RegistryOption configures a Registry.
type RegistryOption func(*Registry)

func WithLogger(l *slog.Logger) RegistryOption
func WithAuditSink(s AuditSink) RegistryOption
func WithMetrics(m MetricsCollector) RegistryOption

// Register adds a provider to the registry. Returns ErrDuplicateProvider
// if a provider with the same Type() is already registered.
// Emits OnProviderRegistered audit event on success.
func (r *Registry) Register(p Provider) error

// Get returns the provider registered for the given type, or nil if not found.
func (r *Registry) Get(kmsType string) Provider

// Has returns true if a provider is registered for the given type.
func (r *Registry) Has(kmsType string) bool

// Types returns a sorted list of all registered provider type identifiers.
func (r *Registry) Types() []string

// Close closes all registered providers and clears the registry.
// Returns the first error encountered; logs all others.
// Emits OnProviderClosed audit event for each provider.
func (r *Registry) Close() error

Instrumented wrapper: The Registry optionally wraps each registered Provider in an instrumented decorator that automatically:

  • Records audit events (OnKeyWrapped, OnKeyUnwrapped, OnKeyGenerated, OnOperationFailed)
  • Records metrics (duration, error counts)
  • Logs operations via slog

This instrumentation is transparent to the provider implementation — providers do not need to be aware of audit/metrics/logging. The registry handles it at the boundary.

Error Handling

package kms

import "errors"

var (
    // ErrProviderClosed is returned when an operation is attempted on a closed provider.
    ErrProviderClosed = errors.New("kms: provider is closed")

    // ErrDuplicateProvider is returned when registering a provider with a type
    // that is already registered.
    ErrDuplicateProvider = errors.New("kms: duplicate provider type")

    // ErrKeyNotFound is returned when the specified key does not exist in the KMS.
    ErrKeyNotFound = errors.New("kms: key not found")

    // ErrAccessDenied is returned when the caller lacks permission to use the key.
    ErrAccessDenied = errors.New("kms: access denied")

    // ErrInvalidKeySize is returned when GenerateKey receives an invalid size.
    ErrInvalidKeySize = errors.New("kms: invalid key size")

    // ErrDecryptionFailed is returned when unwrapping fails due to wrong key,
    // corrupted ciphertext, or tampered AAD.
    ErrDecryptionFailed = errors.New("kms: decryption failed")
)

// ProviderError wraps a provider-specific error with the provider type and operation.
type ProviderError struct {
    ProviderType string // e.g., "aws-kms"
    Operation    string // "wrap", "unwrap", "generate_key", "close"
    KeyID        string // key identifier (may be empty for lifecycle ops)
    Err          error  // underlying provider error
}

func (e *ProviderError) Error() string
func (e *ProviderError) Unwrap() error

Provider implementations SHOULD return sentinel errors where possible and MUST wrap provider-specific errors in ProviderError for consistent error handling by consumers.

Observability: Audit Sink

The AuditSink interface provides typed, per-event audit hooks. The library calls these methods at the registry instrumentation layer — provider implementations do not call them directly.

// AuditSink receives typed audit events from KMS operations.
// Implement this interface to integrate with your audit framework (e.g., go-audit).
//
// All methods MUST be safe for concurrent use.
// All methods MUST NOT block — audit is fire-and-forget at the KMS layer.
type AuditSink interface {
    OnKeyWrapped(ctx context.Context, event KeyWrappedEvent)
    OnKeyUnwrapped(ctx context.Context, event KeyUnwrappedEvent)
    OnKeyGenerated(ctx context.Context, event KeyGeneratedEvent)
    OnProviderRegistered(ctx context.Context, event ProviderRegisteredEvent)
    OnProviderClosed(ctx context.Context, event ProviderClosedEvent)
    OnOperationFailed(ctx context.Context, event OperationFailedEvent)
}

Event structs — each event carries exactly the fields relevant to that operation:

type KeyWrappedEvent struct {
    Timestamp    time.Time
    ProviderType string        // e.g., "aws-kms"
    KeyID        string        // KMS key identifier used
    InputSize    int           // plaintext size in bytes (NOT the plaintext itself)
    OutputSize   int           // ciphertext size in bytes
    Duration     time.Duration // operation wall-clock time
}

type KeyUnwrappedEvent struct {
    Timestamp    time.Time
    ProviderType string
    KeyID        string
    InputSize    int           // ciphertext size in bytes
    OutputSize   int           // plaintext size in bytes
    Duration     time.Duration
}

type KeyGeneratedEvent struct {
    Timestamp    time.Time
    ProviderType string
    KeyID        string
    KeySize      int           // requested key size in bytes
    Duration     time.Duration
    NativeAPI    bool          // true if provider used native GenerateDataKey API
}

type ProviderRegisteredEvent struct {
    Timestamp    time.Time
    ProviderType string
}

type ProviderClosedEvent struct {
    Timestamp    time.Time
    ProviderType string
    Error        error         // nil on clean close
}

type OperationFailedEvent struct {
    Timestamp    time.Time
    ProviderType string
    KeyID        string
    Operation    string        // "wrap", "unwrap", "generate_key"
    Error        error
    Duration     time.Duration
}

BaseAuditSink — embed for selective auditing:

// BaseAuditSink provides no-op implementations for all AuditSink methods.
// Embed this in your audit implementation to only override events you care about.
// When new events are added in future versions, your code won't break.
type BaseAuditSink struct{}

func (BaseAuditSink) OnKeyWrapped(context.Context, KeyWrappedEvent)               {}
func (BaseAuditSink) OnKeyUnwrapped(context.Context, KeyUnwrappedEvent)           {}
func (BaseAuditSink) OnKeyGenerated(context.Context, KeyGeneratedEvent)           {}
func (BaseAuditSink) OnProviderRegistered(context.Context, ProviderRegisteredEvent) {}
func (BaseAuditSink) OnProviderClosed(context.Context, ProviderClosedEvent)       {}
func (BaseAuditSink) OnOperationFailed(context.Context, OperationFailedEvent)     {}

Consumer usage (schema-registry example):

type KMSAuditBridge struct {
    kms.BaseAuditSink          // embed — new events get no-op default
    logger *audit.Logger       // go-audit logger
}

// Override only the events you want to audit
func (b *KMSAuditBridge) OnKeyWrapped(ctx context.Context, e kms.KeyWrappedEvent) {
    b.logger.Log(ctx, audit.Event{
        Type:   "kek.key_wrapped",           // YOUR taxonomy, defined in go-audit
        Target: e.KeyID,
        Details: map[string]any{
            "provider":    e.ProviderType,
            "duration_ms": e.Duration.Milliseconds(),
            "input_size":  e.InputSize,
        },
    })
}

Three levels of audit control:

Level How
Global no-op Don't pass WithAuditSink() — no audit events emitted
Selective Embed BaseAuditSink, override only events you care about
Full Implement every method directly (don't embed base)

Provider-Specific Audit Sinks

Each provider package defines its own audit sink interface for events unique to that provider. These are independent of the core AuditSink:

// In package vault
type AuditSink interface {
    OnTransitEncrypt(ctx context.Context, event TransitEncryptEvent)
    OnTransitDecrypt(ctx context.Context, event TransitDecryptEvent)
}

type BaseAuditSink struct{}
func (BaseAuditSink) OnTransitEncrypt(context.Context, TransitEncryptEvent) {}
func (BaseAuditSink) OnTransitDecrypt(context.Context, TransitDecryptEvent) {}
// In package awskms
type AuditSink interface {
    OnNativeGenerateDataKey(ctx context.Context, event NativeGenerateDataKeyEvent)
    OnLocalFallbackGenerate(ctx context.Context, event LocalFallbackGenerateEvent)
    OnRegionResolved(ctx context.Context, event RegionResolvedEvent)
}

type BaseAuditSink struct{}
// ... no-op implementations
// In package pkcs11kms
type AuditSink interface {
    OnSessionOpened(ctx context.Context, event SessionOpenedEvent)
    OnSessionClosed(ctx context.Context, event SessionClosedEvent)
    OnSessionPoolExhausted(ctx context.Context, event SessionPoolExhaustedEvent)
    OnMechanismSelected(ctx context.Context, event MechanismSelectedEvent)
}

type BaseAuditSink struct{}
// ... no-op implementations

Provider-specific sinks are passed to the provider at creation time:

p, err := vault.NewProvider(
    vault.WithConfig(cfg),
    vault.WithAuditSink(myVaultAuditBridge),  // vault-specific events
)

p2, err := pkcs11kms.NewProvider(
    pkcs11kms.WithConfig(cfg),
    pkcs11kms.WithAuditSink(myPKCS11AuditBridge),  // PKCS#11-specific events
)

Key principle: Core events (wrap/unwrap/generate) are emitted by the registry's instrumentation layer. Provider-specific events are emitted by the provider itself. No circular dependencies.

Observability: Structured Logging

Logging uses Go's standard log/slog package. The consumer passes a configured logger; the library uses a no-op logger by default (not slog.Default() — libraries MUST be silent unless the consumer opts in). The consumer controls output format, level filtering, and destination by configuring their slog.Handler.

Message prefix convention: All log messages MUST use the "kms: " prefix for grep-ability (e.g., "kms: wrap complete", "kms: provider registered").

Structured fields MUST NOT be embedded in message strings. Use separate slog attributes:

// WRONG — fields embedded in message
l.logger.Info(fmt.Sprintf("kms: wrapped key %s in %dms", keyID, dur.Milliseconds()))

// CORRECT — structured attributes
l.logger.Info("kms: wrap complete", "kms.key_id", keyID, "kms.duration_ms", dur.Milliseconds())

Logging Level Strategy

Level When Examples
ERROR Data loss or operation failure: provider operation returned error, provider close failure, resource cleanup failure Wrap/Unwrap/GenerateKey error, Close error
WARN Degraded operation: slow operations exceeding threshold, fallback paths triggered, session pool near exhaustion Slow wrap (>threshold), AWS native GenerateDataKey fallback to local
INFO Significant state transitions: provider lifecycle, registry lifecycle Provider registered, provider closed, registry created, registry shutdown start/complete
DEBUG Detailed troubleshooting: operation start/end with timing, key IDs, per-operation detail Wrap start, wrap complete with duration, session pool borrow/return

Slow Operation Threshold

// SlowOperationThreshold configures the duration above which operations
// are logged at WARN level instead of DEBUG. Default: 5 * time.Second.
func WithSlowOperationThreshold(d time.Duration) RegistryOption

Complete Call Site Table

Every slog call in the library is enumerated below with exact level, message, and structured fields:

Registry lifecycle (registry.go):

Call Site Level Message Structured Fields
NewRegistry() success INFO "kms: registry created"
Register() success INFO "kms: provider registered" kms.provider
Register() duplicate error WARN "kms: duplicate provider registration attempted" kms.provider
Close() start INFO "kms: registry shutdown started" kms.provider_count
Close() per-provider success DEBUG "kms: provider closed" kms.provider
Close() per-provider error ERROR "kms: provider close failed" kms.provider, kms.error
Close() complete INFO "kms: registry shutdown complete" kms.duration_ms, kms.provider_count, kms.errors

Instrumented operations (instrumented.go):

Call Site Level Message Structured Fields
Wrap() start DEBUG "kms: wrap started" kms.provider, kms.key_id, kms.input_size
Wrap() success DEBUG "kms: wrap complete" kms.provider, kms.key_id, kms.input_size, kms.output_size, kms.duration_ms
Wrap() success but slow WARN "kms: wrap slow" kms.provider, kms.key_id, kms.duration_ms, kms.slow_threshold_ms
Wrap() error ERROR "kms: wrap failed" kms.provider, kms.key_id, kms.duration_ms, kms.error
Unwrap() start DEBUG "kms: unwrap started" kms.provider, kms.key_id, kms.input_size
Unwrap() success DEBUG "kms: unwrap complete" kms.provider, kms.key_id, kms.input_size, kms.output_size, kms.duration_ms
Unwrap() success but slow WARN "kms: unwrap slow" kms.provider, kms.key_id, kms.duration_ms, kms.slow_threshold_ms
Unwrap() error ERROR "kms: unwrap failed" kms.provider, kms.key_id, kms.duration_ms, kms.error
GenerateKey() start DEBUG "kms: generate_key started" kms.provider, kms.key_id, kms.key_size
GenerateKey() success DEBUG "kms: generate_key complete" kms.provider, kms.key_id, kms.key_size, kms.native_api, kms.duration_ms
GenerateKey() success but slow WARN "kms: generate_key slow" kms.provider, kms.key_id, kms.duration_ms, kms.slow_threshold_ms
GenerateKey() error ERROR "kms: generate_key failed" kms.provider, kms.key_id, kms.duration_ms, kms.error

Provider-specific logging (inside each provider):

Provider Call Site Level Message Fields
AWS Native GenerateDataKey fallback WARN "kms: aws native GenerateDataKey unsupported, using local fallback" kms.provider, kms.key_id
PKCS#11 Session opened DEBUG "kms: pkcs11 session opened" kms.provider, kms.slot, kms.token_label
PKCS#11 Session pool exhausted (waiting) WARN "kms: pkcs11 session pool exhausted, waiting" kms.provider, kms.pool_size, kms.max_sessions
PKCS#11 Session error ERROR "kms: pkcs11 session error" kms.provider, kms.error
KMIP TLS handshake complete DEBUG "kms: kmip TLS handshake complete" kms.provider, kms.endpoint, kms.cert_subject
KMIP Connection pool exhausted WARN "kms: kmip connection pool exhausted, waiting" kms.provider, kms.pool_size, kms.max_connections
OVH mTLS authenticated DEBUG "kms: ovh mTLS authenticated" kms.provider, kms.endpoint, kms.cert_subject
OCI Crypto endpoint resolved DEBUG "kms: oci crypto endpoint resolved" kms.provider, kms.endpoint

Structured Field Reference

Field Name Type Description
kms.provider string Provider type identifier (e.g., "hcvault", "aws-kms")
kms.operation string Operation name: "wrap", "unwrap", "generate_key", "close"
kms.key_id string KMS key identifier used for the operation
kms.input_size int Input byte slice size
kms.output_size int Output byte slice size
kms.key_size int Requested key size in bytes (GenerateKey only)
kms.duration_ms int64 Operation wall-clock duration in milliseconds
kms.slow_threshold_ms int64 Configured slow operation threshold (on WARN logs)
kms.native_api bool Whether native KMS API was used (GenerateKey only)
kms.error string Error message (on ERROR/WARN logs)
kms.provider_count int Number of providers (registry lifecycle)
kms.errors int Number of errors during Close (registry shutdown)
kms.pool_size int Current session/connection pool utilization
kms.max_sessions int Max configured sessions (PKCS#11)
kms.max_connections int Max configured connections (KMIP)
kms.slot int PKCS#11 slot number
kms.token_label string PKCS#11 token label
kms.endpoint string Provider endpoint URL (OCI, OVH, KMIP)
kms.cert_subject string TLS client certificate subject (OVH, KMIP)

Observability: Metrics

// MetricsCollector receives operational metrics from KMS operations.
// Implement this interface to integrate with your metrics framework
// (Prometheus, OpenTelemetry, Dropwizard, etc.).
//
// All methods MUST be safe for concurrent use by multiple goroutines.
// All methods MUST NOT block — metrics recording is fire-and-forget.
type MetricsCollector interface {
    // --- Counters (monotonically increasing) ---

    // IncrementWrapTotal increments the total number of wrap operations attempted.
    // Called once per Wrap() invocation regardless of outcome.
    IncrementWrapTotal(providerType string)

    // IncrementUnwrapTotal increments the total number of unwrap operations attempted.
    IncrementUnwrapTotal(providerType string)

    // IncrementGenerateKeyTotal increments the total number of GenerateKey operations attempted.
    IncrementGenerateKeyTotal(providerType string)

    // IncrementOperationError increments the error counter for a specific operation.
    // Called when Wrap/Unwrap/GenerateKey returns a non-nil error.
    IncrementOperationError(providerType string, operation string)

    // --- Histograms (duration distributions) ---

    // RecordWrapDuration records the duration of a wrap operation.
    // Called on both success and failure. Check err to distinguish.
    RecordWrapDuration(providerType string, duration time.Duration, err error)

    // RecordUnwrapDuration records the duration of an unwrap operation.
    RecordUnwrapDuration(providerType string, duration time.Duration, err error)

    // RecordGenerateKeyDuration records the duration of a GenerateKey operation.
    RecordGenerateKeyDuration(providerType string, duration time.Duration, err error)

    // --- Gauges (point-in-time values) ---

    // SetActiveProviders sets the current number of registered providers.
    // Called after Register() and Close().
    SetActiveProviders(count int)

    // SetInFlightOperations sets the number of currently executing operations.
    // Incremented before an operation starts, decremented when it completes.
    SetInFlightOperations(providerType string, count int64)
}

BaseMetricsCollector — no-op base for selective implementation:

type BaseMetricsCollector struct{}

func (BaseMetricsCollector) IncrementWrapTotal(string)                                  {}
func (BaseMetricsCollector) IncrementUnwrapTotal(string)                                {}
func (BaseMetricsCollector) IncrementGenerateKeyTotal(string)                           {}
func (BaseMetricsCollector) IncrementOperationError(string, string)                     {}
func (BaseMetricsCollector) RecordWrapDuration(string, time.Duration, error)            {}
func (BaseMetricsCollector) RecordUnwrapDuration(string, time.Duration, error)          {}
func (BaseMetricsCollector) RecordGenerateKeyDuration(string, time.Duration, error)     {}
func (BaseMetricsCollector) SetActiveProviders(int)                                     {}
func (BaseMetricsCollector) SetInFlightOperations(string, int64)                        {}

All metrics calls MUST be guarded by if r.metrics != nil (metrics are optional).

Core Metrics Call Site Table

Every metrics call in the instrumented provider is enumerated below:

Code Path Metrics Call When
Wrap() entry IncrementWrapTotal(providerType) Always, before calling provider
Wrap() entry SetInFlightOperations(providerType, +1) Before calling provider
Wrap() exit SetInFlightOperations(providerType, -1) After provider returns (success or error)
Wrap() exit RecordWrapDuration(providerType, duration, err) Always, after provider returns
Wrap() error IncrementOperationError(providerType, "wrap") Only when err != nil
Unwrap() entry IncrementUnwrapTotal(providerType) Always
Unwrap() entry SetInFlightOperations(providerType, +1) Before calling provider
Unwrap() exit SetInFlightOperations(providerType, -1) After provider returns
Unwrap() exit RecordUnwrapDuration(providerType, duration, err) Always
Unwrap() error IncrementOperationError(providerType, "unwrap") Only when err != nil
GenerateKey() entry IncrementGenerateKeyTotal(providerType) Always
GenerateKey() entry SetInFlightOperations(providerType, +1) Before calling provider
GenerateKey() exit SetInFlightOperations(providerType, -1) After provider returns
GenerateKey() exit RecordGenerateKeyDuration(providerType, duration, err) Always
GenerateKey() error IncrementOperationError(providerType, "generate_key") Only when err != nil
Register() success SetActiveProviders(len(providers)) After successful registration
Close() complete SetActiveProviders(0) After all providers closed

Provider-Specific Metrics

Each provider with internal resource pools or unique operational characteristics defines its own metrics interface. Provider-specific metrics are passed to the provider at creation time via WithMetrics().

// In package pkcs11kms
type MetricsCollector interface {
    RecordSessionPoolSize(size int)
    RecordSessionWaitDuration(duration time.Duration)
    RecordSessionError(err error)
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordSessionPoolSize(int)                   {}
func (BaseMetricsCollector) RecordSessionWaitDuration(time.Duration)     {}
func (BaseMetricsCollector) RecordSessionError(error)                    {}
// In package kmipkms
type MetricsCollector interface {
    RecordConnectionPoolSize(size int)
    RecordTLSHandshakeDuration(duration time.Duration)
    RecordConnectionError(err error)
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordConnectionPoolSize(int)                {}
func (BaseMetricsCollector) RecordTLSHandshakeDuration(time.Duration)    {}
func (BaseMetricsCollector) RecordConnectionError(error)                 {}
// In package vault
type MetricsCollector interface {
    RecordTransitRequestDuration(operation string, duration time.Duration, err error)
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordTransitRequestDuration(string, time.Duration, error) {}
// In package openbao
type MetricsCollector interface {
    RecordTransitRequestDuration(operation string, duration time.Duration, err error)
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordTransitRequestDuration(string, time.Duration, error) {}
// In package awskms
type MetricsCollector interface {
    RecordAWSAPICallDuration(operation string, region string, duration time.Duration, err error)
    IncrementLocalFallback()  // native GenerateDataKey unavailable, fell back to local
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordAWSAPICallDuration(string, string, time.Duration, error) {}
func (BaseMetricsCollector) IncrementLocalFallback()                                       {}
// In package azurekms
type MetricsCollector interface {
    RecordAzureAPICallDuration(operation string, duration time.Duration, err error)
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordAzureAPICallDuration(string, time.Duration, error) {}
// In package gcpkms
type MetricsCollector interface {
    RecordGCPAPICallDuration(operation string, duration time.Duration, err error)
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordGCPAPICallDuration(string, time.Duration, error) {}
// In package ocikms
type MetricsCollector interface {
    RecordOCIAPICallDuration(operation string, duration time.Duration, err error)
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordOCIAPICallDuration(string, time.Duration, error) {}
// In package ovhkms
type MetricsCollector interface {
    RecordOVHAPICallDuration(operation string, duration time.Duration, err error)
    RecordTLSHandshakeDuration(duration time.Duration)
}

type BaseMetricsCollector struct{}
func (BaseMetricsCollector) RecordOVHAPICallDuration(string, time.Duration, error) {}
func (BaseMetricsCollector) RecordTLSHandshakeDuration(time.Duration)              {}

Metrics Provider Agnosticism

CRITICAL: The MetricsCollector interface is a pure Go interface with ZERO dependencies on any metrics framework. There is no dependency on Prometheus, OpenTelemetry, Dropwizard, or any other metrics library. The MetricsCollector interface uses only Go stdlib types (string, int, int64, time.Duration, error).

The consumer implements the interface with whatever metrics framework they use. For example:

// Consumer's Prometheus adapter (lives in consumer's codebase, NOT in go-kms)
type PrometheusMetrics struct {
    kms.BaseMetricsCollector
    wrapTotal   *prometheus.CounterVec
    wrapLatency *prometheus.HistogramVec
    // ...
}

func (m *PrometheusMetrics) IncrementWrapTotal(providerType string) {
    m.wrapTotal.WithLabelValues(providerType).Inc()
}

func (m *PrometheusMetrics) RecordWrapDuration(providerType string, d time.Duration, err error) {
    m.wrapLatency.WithLabelValues(providerType).Observe(d.Seconds())
}

This is the same pattern used by go-audit's Metrics interface — the library defines what to measure, the consumer decides how to record it. The interface methods use semantic names (IncrementWrapTotal, RecordWrapDuration) that map naturally to counters, histograms, and gauges in any framework, but the interface itself has no opinion on the implementation.

The go-kms go.mod MUST NOT contain any metrics framework dependency.

Mock Provider (kmstest Package)

The core module includes a kmstest package with a mock provider for consumer testing:

package kmstest

// MockProvider implements kms.Provider with configurable behavior.
// It performs real AES-GCM encryption using an in-memory key, so wrap/unwrap
// round-trips produce correct results without any external dependencies.
type MockProvider struct {
    TypeID    string            // default: "mock"
    WrapFunc  func(ctx context.Context, keyID string, plaintext []byte) ([]byte, error)
    // ... override hooks for custom behavior
}

func NewMockProvider(opts ...MockOption) *MockProvider

// NewMockRegistry creates a Registry with a single MockProvider registered.
// Useful for one-line test setup.
func NewMockRegistry(opts ...MockOption) *kms.Registry

The mock provider uses real crypto/aes + GCM with a deterministic (or random) key, so consumers can test actual wrap/unwrap round-trips without mocking the entire interface.

Local AEAD Provider

The core module also includes a local AEAD provider for development, testing, and offline use:

package aead

// Provider implements kms.Provider using local AES-GCM encryption.
// The key material is held in-process memory — there is no external KMS.
//
// This provider is suitable for:
//   - Development and testing without cloud credentials
//   - Offline/air-gapped environments
//   - Bootstrapping before a cloud KMS is configured
//
// This provider is NOT suitable for production use where key material
// must be protected by an HSM or external KMS.
type Provider struct { /* unexported */ }

// NewProvider creates a local AEAD provider.
// The key must be exactly 16, 24, or 32 bytes (AES-128, AES-192, AES-256).
// The keyID parameter in Wrap/Unwrap/GenerateKey is ignored — all operations
// use the provided key.
func NewProvider(key []byte) (*Provider, error)

// NewProviderFromHex creates a provider from a hex-encoded key string.
func NewProviderFromHex(hexKey string) (*Provider, error)

// NewProviderRandom creates a provider with a randomly generated 256-bit key.
// Useful for testing. The key is lost when the provider is garbage collected.
func NewProviderRandom() (*Provider, error)

The AEAD provider lives in the core module (github.com/axonops/go-kms/aead) because it has zero external dependencies — only Go stdlib crypto/aes, crypto/cipher, crypto/rand.

Provider Specifications

1. HashiCorp Vault Transit Provider

Field Value
Package github.com/axonops/go-kms/providers/vault
Type ID hcvault
SDK github.com/hashicorp/vault/api v1.22.0+
SDK License MPL-2.0 (API client remains MPL-2.0; Vault core moved to BUSL-1.1 in Aug 2023)

Config:

type Config struct {
    Address      string // Vault server URL. Default: VAULT_ADDR env → "http://127.0.0.1:8200"
    Token        string // Auth token. Default: VAULT_TOKEN env
    Namespace    string // Enterprise namespace. Default: VAULT_NAMESPACE env
    TransitMount string // Transit engine mount path. Default: "transit"
}

Per-operation properties:

  • vault.address — override server URL
  • vault.token — override auth token
  • vault.namespace — override namespace
  • vault.transit.mount — override mount path

Implementation notes:

  • Wrap → POST /v1/{mount}/encrypt/{keyName} with base64-encoded plaintext
  • Unwrap → POST /v1/{mount}/decrypt/{keyName} returning base64-decoded plaintext
  • GenerateKey → generate locally with crypto/rand, then Wrap
  • Close → no-op (HTTP client is stateless)

Provider-specific audit events:

  • OnTransitEncrypt — Vault Transit encrypt call with mount, namespace
  • OnTransitDecrypt — Vault Transit decrypt call with mount, namespace

2. OpenBao Transit Provider

Field Value
Package github.com/axonops/go-kms/providers/openbao
Type ID openbao
SDK github.com/openbao/openbao/api (or github.com/hashicorp/vault/api if OpenBao API client not yet independently published)
SDK License MPL-2.0

IMPORTANT: OpenBao is a fully independent implementation. No code is shared with the Vault provider. Although the Transit APIs are currently compatible, the implementations MUST be separate to support future divergence (different auth methods, new Transit features, different defaults).

Config:

type Config struct {
    Address      string // Server URL. Default: BAO_ADDR env → "http://127.0.0.1:8200"
    Token        string // Auth token. Default: BAO_TOKEN env
    Namespace    string // Namespace. Default: BAO_NAMESPACE env
    TransitMount string // Transit mount path. Default: "transit"
}

Per-operation properties:

  • openbao.address — override server URL
  • openbao.token — override auth token
  • openbao.namespace — override namespace
  • openbao.transit.mount — override mount path

Implementation notes:

  • Same Transit API semantics as Vault (encrypt/decrypt endpoints) but with independent HTTP client, environment variable handling, and error wrapping
  • Environment variable precedence: BAO_*VAULT_* → defaults
  • Close → no-op

Provider-specific audit events:

  • OnTransitEncrypt — OpenBao Transit encrypt call
  • OnTransitDecrypt — OpenBao Transit decrypt call

3. AWS KMS Provider

Field Value
Package github.com/axonops/go-kms/providers/awskms
Type ID aws-kms
SDK github.com/aws/aws-sdk-go-v2/service/kms + config + credentials
SDK License Apache-2.0

Config:

type Config struct {
    Region          string // Default: AWS_REGION env → AWS_DEFAULT_REGION env → "us-east-1"
    AccessKeyID     string // Default: AWS_ACCESS_KEY_ID env or IAM role chain
    SecretAccessKey string // Default: AWS_SECRET_ACCESS_KEY env or IAM role chain
    Endpoint        string // Custom endpoint (e.g., LocalStack). Default: empty (use AWS)
}

Per-operation properties:

  • aws.region — override region
  • aws.access.key.id — override access key
  • aws.secret.access.key — override secret key
  • aws.endpoint — override endpoint

Implementation notes:

  • Wrap → AWS KMS Encrypt API with SYMMETRIC_DEFAULT algorithm
  • Unwrap → AWS KMS Decrypt API
  • GenerateKey → attempt AWS KMS GenerateDataKey API first (native); fall back to local generation + Wrap if unsupported by key type
  • Close → no-op (SDK manages connections)

Provider-specific audit events:

  • OnNativeGenerateDataKey — AWS GenerateDataKey API used (with key spec)
  • OnLocalFallbackGenerate — native GenerateDataKey unsupported, fell back to local generation
  • OnRegionResolved — final region used for operation

4. Azure Key Vault Provider

Field Value
Package github.com/axonops/go-kms/providers/azurekms
Type ID azure-kms
SDK github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys + azidentity
SDK License MIT

Config:

type Config struct {
    TenantID     string // AZURE_TENANT_ID env. Required.
    ClientID     string // AZURE_CLIENT_ID env. Required.
    ClientSecret string // AZURE_CLIENT_SECRET env. Required.
    VaultURL     string // Key Vault URL (e.g., "https://myvault.vault.azure.net"). Required.
    KeyVersion   string // Optional; defaults to latest key version.
}

Per-operation properties:

  • azure.tenant.id — override tenant
  • azure.client.id — override client ID
  • azure.client.secret — override client secret
  • azure.keyvault.url — override vault URL
  • azure.key.version — override key version
  • azure.key.name — override keyID

Implementation notes:

  • Wrap → Azure WrapKey API with RSA-OAEP-256
  • Unwrap → Azure UnwrapKey API with RSA-OAEP-256
  • GenerateKey → generate locally with crypto/rand, then Wrap (Azure has no native GenerateDataKey)
  • VaultURL is mandatory — return error if empty
  • Close → no-op

Provider-specific audit events:

  • OnKeyVersionResolved — resolved key version (latest or specific)

5. GCP Cloud KMS Provider

Field Value
Package github.com/axonops/go-kms/providers/gcpkms
Type ID gcp-kms
SDK cloud.google.com/go/kms
SDK License Apache-2.0

Config:

type Config struct {
    ProjectID       string // GOOGLE_CLOUD_PROJECT or GCLOUD_PROJECT env. Required.
    Location        string // KMS location. Default: "global"
    KeyRing         string // Key ring name. Required.
    CredentialsPath string // GOOGLE_APPLICATION_CREDENTIALS env. Optional (uses default chain).
    Endpoint        string // Custom endpoint (testing). Default: empty (use GCP).
}

Per-operation properties:

  • gcp.project.id — override project ID
  • gcp.location — override location
  • gcp.key.ring — override key ring
  • gcp.credentials.path — override credentials
  • gcp.endpoint — override endpoint

Key resource name format:
projects/{projectID}/locations/{location}/keyRings/{keyRing}/cryptoKeys/{keyID}

Implementation notes:

  • Wrap → GCP Cloud KMS Encrypt API
  • Unwrap → GCP Cloud KMS Decrypt API
  • GenerateKey → generate locally, then Wrap (GCP has no native GenerateDataKey)
  • Close → closes gRPC client connection

Provider-specific audit events:

  • OnKeyRingResolved — resolved full crypto key resource name
  • OnCredentialsLoaded — credentials source (file, default chain, etc.)

6. Oracle OCI KMS Provider

Field Value
Package github.com/axonops/go-kms/providers/ocikms
Type ID oci-kms
SDK github.com/oracle/oci-go-sdk/v65/keymanagement
SDK License Apache-2.0 / UPL-1.0 (dual licensed; we use under Apache-2.0)

Config:

type Config struct {
    // CryptoEndpoint is the vault-specific crypto endpoint URL.
    // Required. Obtained from the Vault object's CryptoEndpoint field.
    // Example: "https://<prefix>.kms.<region>.oraclecloud.com"
    CryptoEndpoint string

    // ConfigProvider supplies OCI authentication credentials.
    // Default: common.DefaultConfigProvider() (~/.oci/config).
    // Supports: API key, instance principal, resource principal, session token.
    ConfigProvider common.ConfigurationProvider
}

Per-operation properties:

  • oci.crypto.endpoint — override crypto endpoint
  • oci.key.version.id — pin to specific key version (OCID)

Architectural note: OCI KMS uses vault-specific endpoints (not a single regional endpoint). The consumer must resolve a vault's crypto endpoint before creating the provider. This is different from AWS/Azure/GCP which use regional endpoints. The provider accepts the crypto endpoint directly — vault resolution is the consumer's responsibility.

Implementation notes:

  • Wrap → OCI KMS Encrypt API (base64-encoded plaintext, up to 4KB)
  • Unwrap → OCI KMS Decrypt API
  • GenerateKey → OCI KMS GenerateDataEncryptionKey API (native support with IncludePlaintextKey: true)
  • Supports AAD via AssociatedData (passed as properties)
  • Close → no-op (HTTP client managed by SDK)

Provider-specific audit events:

  • OnCryptoEndpointUsed — which vault crypto endpoint was called
  • OnNativeGenerateDataKey — OCI GenerateDataEncryptionKey API used

7. OVHcloud OKMS Provider

Field Value
Package github.com/axonops/go-kms/providers/ovhkms
Type ID ovh-kms
SDK github.com/ovh/okms-sdk-go v0.5.2+
SDK License Apache-2.0

Config:

type Config struct {
    // Endpoint is the OKMS instance URL (e.g., "https://eu-west-rbx.okms.ovh.net").
    // Required.
    Endpoint string

    // CertFile is the path to the client certificate for mTLS authentication.
    // Required (unless using OAuth2).
    CertFile string

    // KeyFile is the path to the client private key for mTLS authentication.
    // Required (unless using OAuth2).
    KeyFile string

    // CAFile is the path to the CA certificate for server verification.
    // Optional; defaults to system CA pool.
    CAFile string

    // ServiceAccountID is the OAuth2 service account ID (alternative to mTLS).
    // Optional.
    ServiceAccountID string

    // ServiceAccountSecret is the OAuth2 service account secret.
    // Optional.
    ServiceAccountSecret string
}

Per-operation properties:

  • ovh.endpoint — override endpoint
  • ovh.cert.file — override client certificate
  • ovh.key.file — override client key

Implementation notes:

  • Wrap → OKMS Encrypt API (returns JWE token)
  • Unwrap → OKMS Decrypt API (accepts JWE token)
  • GenerateKey → OKMS DataKey API (native support for envelope encryption)
  • Primary auth is mTLS (client certificates); OAuth2 as alternative
  • Ciphertext format is JWE (JSON Web Encryption) — this is opaque to the consumer
  • Close → closes HTTP/TLS client

Provider-specific audit events:

  • OnMTLSAuthenticated — mTLS handshake completed with certificate subject
  • OnJWEEncrypted — JWE token produced (algorithm, key ID)

EU Sovereignty note: OVHcloud holds SecNumCloud certification (ANSSI, France) and is not subject to US Cloud Act. This provider enables EU data sovereignty requirements.

8. PKCS#11 Provider

Field Value
Package github.com/axonops/go-kms/providers/pkcs11kms
Type ID pkcs11
Library github.com/miekg/pkcs11 v1.1.2+
Library License BSD-3-Clause

Config:

type Config struct {
    // LibraryPath is the path to the PKCS#11 shared library.
    // Examples:
    //   SoftHSM:     "/usr/lib/softhsm/libsofthsm2.so"
    //   AWS CloudHSM: "/opt/cloudhsm/lib/libcloudhsm_pkcs11.so"
    //   Thales Luna:  "/usr/safenet/lunaclient/lib/libCryptoki2_64.so"
    // Required.
    LibraryPath string

    // TokenLabel identifies the HSM token/slot by label.
    // Either TokenLabel or SlotNumber must be set.
    TokenLabel string

    // SlotNumber identifies the HSM slot by number.
    // Either TokenLabel or SlotNumber must be set.
    SlotNumber *int

    // PIN is the user PIN for the token. Required.
    PIN string

    // MaxSessions is the maximum number of concurrent PKCS#11 sessions.
    // Default: 10.
    MaxSessions int

    // WrapMechanism is the PKCS#11 mechanism to use for wrap/unwrap.
    // Default: CKM_AES_KEY_WRAP (0x2109).
    // Other options: CKM_AES_GCM, CKM_RSA_PKCS_OAEP, CKM_AES_KEY_WRAP_PAD.
    WrapMechanism uint
}

CGo requirement: This provider requires CGO_ENABLED=1 because PKCS#11 is a C API. The multi-module structure ensures consumers who do not import pkcs11kms are not affected.

Session pool: The provider manages a pool of PKCS#11 sessions. Sessions are borrowed for each operation and returned when complete. The pool handles login state, thread safety, and automatic session recovery.

Implementation notes:

  • Wrap → EncryptInit + Encrypt using the configured mechanism and the wrapping key (identified by keyID which maps to a PKCS#11 key label or handle)
  • Unwrap → DecryptInit + Decrypt
  • GenerateKey → crypto/rand for random bytes, then Encrypt to wrap
  • Key lookup → FindObjectsInit + FindObjects by CKA_LABEL matching keyID
  • Close → closes all sessions, finalizes PKCS#11 context

Testing: SoftHSM 2 (BSD-2-Clause) in Docker for CI. Initialize with:

softhsm2-util --init-token --slot 0 --label "test" --pin 1234 --so-pin 5678

Provider-specific audit events:

  • OnSessionOpened — PKCS#11 session opened (slot, token label)
  • OnSessionClosed — PKCS#11 session closed
  • OnSessionPoolExhausted — all sessions in use, operation waiting
  • OnMechanismSelected — wrap mechanism used for operation

Provider-specific metrics:

  • RecordSessionPoolSize(size int) — current pool utilization
  • RecordSessionWaitDuration(duration time.Duration) — time waiting for available session
  • RecordSessionError(err error) — session-level errors

9. KMIP Provider

Field Value
Package github.com/axonops/go-kms/providers/kmipkms
Type ID kmip
Library github.com/ovh/kmip-go v0.7.2+
Library License Apache-2.0

Config:

type Config struct {
    // Endpoint is the KMIP server address (host:port).
    // Required.
    Endpoint string

    // CertFile is the path to the client certificate for mTLS.
    // Required (KMIP universally requires mTLS).
    CertFile string

    // KeyFile is the path to the client private key for mTLS.
    // Required.
    KeyFile string

    // CAFile is the path to the CA certificate.
    // Required (KMIP servers typically use private CAs).
    CAFile string

    // MaxConnections is the maximum number of persistent TCP/TLS connections.
    // Default: 5.
    MaxConnections int
}

Implementation notes:

  • Wrap → KMIP Encrypt operation using server-side encryption key
  • Unwrap → KMIP Decrypt operation
  • GenerateKey → generate locally, then Encrypt to wrap (or Create + Get with KeyWrappingSpecification)
  • Connection management: persistent TLS connections pooled for reuse
  • Close → closes all connections, cleans up TLS state

KMIP wrapping model: KMIP does not define explicit Wrap/Unwrap operations. Instead, key wrapping is achieved through Encrypt/Decrypt (for wrapping arbitrary byte slices) or Get/Export with KeyWrappingSpecification (for exporting key objects in wrapped form). Our provider uses Encrypt/Decrypt as it maps cleanly to the Provider interface semantics.

Server compatibility: Thales CipherTrust, Entrust KeyControl, Fortanix SDKMS, IBM HPCS, OVHcloud OKMS, PyKMIP (for testing).

Testing: PyKMIP (Apache-2.0) in Docker for CI. Requires TLS certificate generation for mTLS.

Provider-specific audit events:

  • OnTLSHandshakeCompleted — mTLS connection established (server, cert subject)
  • OnConnectionPoolExhausted — all connections in use

Provider-specific metrics:

  • RecordConnectionPoolSize(size int) — current pool utilization
  • RecordTLSHandshakeDuration(duration time.Duration) — TLS setup time

10. Local AEAD Provider

See Local AEAD Provider section above. This provider lives in the core module, not a separate provider module.

Market Gap

Existing Solutions

Library License Level Providers Key Difference
HashiCorp go-kms-wrapping MPL-2.0 Envelope encryption 10 cloud + Vault Higher-level abstraction (DEK+payload encryption, not raw key wrapping)
OpenBao go-kms-wrapping MPL-2.0 Envelope encryption 12 (adds KMIP, PKCS#11) Fork of HashiCorp; same abstraction level
Smallstep step-kms-plugin Apache-2.0 Signing/PKI AWS, PKCS#11, YubiKey Focused on crypto.Signer, not key wrapping
Sigstore sigstore Apache-2.0 Code signing 4 cloud + plugins Signing/verification only
Google Tink Apache-2.0 Keyset encryption AWS, GCP, Vault Tink-specific keyset model

What go-kms Provides That Others Don't

  1. Pure key wrapping primitives — not envelope encryption. Consumers who need envelope encryption build it on top. This is one abstraction level below HashiCorp's go-kms-wrapping.
  2. Apache-2.0 license — go-kms-wrapping is MPL-2.0, which some organizations avoid.
  3. Zero-dependency core — the core module has no external dependencies. Only imported provider modules add SDK dependencies.
  4. PKCS#11 + KMIP at launch — hardware HSM support from day one, not as a plugin afterthought.
  5. OVHcloud OKMS support — no existing library supports OVH KMS.
  6. Oracle OCI KMS support — go-kms-wrapping supports OCI but as one of 10 providers in a single module.
  7. Pluggable observability — typed audit sink, structured logging, metrics interface. None of the existing libraries offer this level of instrumentation.
  8. OpenBao as first-class citizen — fully independent implementation, not a Vault wrapper.

Positioning

go-kms sits at the KMS primitive layer:

Application (schema-registry, etc.)
    ↓ uses
Envelope Encryption (application concern: DEK/KEK model, algorithm mapping)
    ↓ uses
go-kms (key wrapping primitives: Wrap, Unwrap, GenerateKey)
    ↓ calls
Cloud KMS APIs / HSMs / Transit engines

HashiCorp's go-kms-wrapping conflates the middle two layers. go-kms separates them.

Testing Requirements

Unit Tests

Coverage target: 90%+

Core package (kms):

  • Registry Register — success, duplicate error, nil provider error
  • Registry Get — found, not found (nil)
  • Registry Has — exists, not exists
  • Registry Types — empty, sorted, after register/close
  • Registry Close — all providers closed, empty after close, close error propagation
  • Registry Close — returns first error, logs subsequent errors
  • ApplyOptions — empty options, single property, multiple properties, property override
  • ProviderError — Error() format, Unwrap() returns inner error
  • Sentinel errors — errors.Is checks for all sentinel errors

Core audit events:

  • BaseAuditSink — all methods are no-op (no panic, no error)
  • Instrumented provider — OnKeyWrapped emitted with correct fields after successful Wrap
  • Instrumented provider — OnKeyUnwrapped emitted after successful Unwrap
  • Instrumented provider — OnKeyGenerated emitted after successful GenerateKey, NativeAPI field correct
  • Instrumented provider — OnOperationFailed emitted on Wrap/Unwrap/GenerateKey error
  • Instrumented provider — OnProviderRegistered emitted on Register
  • Instrumented provider — OnProviderClosed emitted on Close (with error if any)
  • Instrumented provider — Duration field is wall-clock accurate (not zero)
  • Custom sink with embedded BaseAuditSink — only overridden methods called

Core metrics:

  • BaseMetricsCollector — all methods are no-op
  • Instrumented provider — RecordWrapDuration called with correct provider type and duration
  • Instrumented provider — RecordWrapDuration called with non-nil error on failure
  • Instrumented provider — SetActiveProviders called on Register/Close

Mock provider (kmstest):

  • Wrap/Unwrap round-trip — plaintext survives encryption cycle
  • GenerateKey — returns correct size plaintext, wrapped is different from plaintext
  • Custom WrapFunc — override hook is called
  • Type — returns configured TypeID
  • Close — idempotent, subsequent operations return ErrProviderClosed

Local AEAD provider:

  • NewProvider — 16, 24, 32 byte keys accepted; other sizes rejected
  • NewProviderFromHex — valid hex accepted, invalid hex rejected
  • NewProviderRandom — creates working provider
  • Wrap/Unwrap round-trip — plaintext survives
  • Wrap — same plaintext produces different ciphertext (random nonce)
  • Unwrap — wrong key returns ErrDecryptionFailed
  • Unwrap — corrupted ciphertext returns error
  • GenerateKey — returns correct size, wrapped round-trips
  • Close — subsequent operations return ErrProviderClosed

Per-provider unit tests (each provider package):

Each provider must have unit tests using mock HTTP servers or mock PKCS#11/KMIP:

  • Type() — returns correct type identifier
  • NewProvider — valid config creates provider
  • NewProvider — missing required config fields return error
  • Wrap/Unwrap round-trip via mock server — plaintext survives
  • GenerateKey — correct size generated, native vs local fallback tested
  • Properties override — per-operation properties take precedence over config
  • Environment variable defaults — env vars used when config empty
  • Close — idempotent, no error on clean close
  • Provider-specific audit events — emitted with correct fields
  • Error wrapping — provider errors wrapped in ProviderError

Integration Tests (Docker Compose, BDD)

Per-Provider Integration Testing Strategy

Every provider MUST have a concrete integration testing path. Not all providers have Docker-based emulators; those that don't use in-process mock servers or SDK-level mocks.

Provider Integration Test Path Docker Image CI Job Crypto Fidelity
Vault Docker dev server hashicorp/vault:1.18 integration Real Transit encryption
OpenBao Docker dev server quay.io/openbao/openbao:2.5 integration Real Transit encryption
AWS KMS LocalStack in Docker nsmithuk/local-kms:3.11.4 integration Real symmetric crypto
Azure Key Vault Docker emulator jamesgoulddev/azure-keyvault-emulator:latest integration Real RSA crypto
GCP Cloud KMS In-process fake gRPC server N/A (Go test) integration Canned responses
OCI KMS In-process mock HTTP server (httptest) N/A (Go test) integration Canned responses
OVHcloud OKMS SDK-provided mockery mocks (mocks.APIMock) N/A (Go test) integration Canned responses
PKCS#11 SoftHSM 2 in Docker vegardit/docker-softhsm2-pkcs11-proxy:latest integration Real HSM crypto
KMIP PyKMIP in Docker python:3.12 + pip install pykmip integration Real KMIP protocol
Local AEAD In-process (stdlib only) N/A unit-tests Real AES-GCM

Cloud-credential tests: AWS (real), Azure (real), GCP (real), OCI (real), OVH (real) additionally run against real cloud KMS in a separate integration-cloud CI job, gated behind environment variables / CI secrets. These verify SDK version compatibility and real-world auth flows.

Azure Key Vault Emulator Details

jamesgoulddev/azure-keyvault-emulator (v2.8.4+) supports WrapKey/UnwrapKey, Encrypt/Decrypt, Sign/Verify. Requires:

  • HTTPS with self-signed certificate (generate in CI setup step)
  • DisableChallengeResourceVerification: true on azkeys.ClientOptions
  • Accepts DefaultAzureCredential without real Azure account
GCP Cloud KMS Fake gRPC Server

No Docker emulator exists for GCP KMS. Integration tests use an in-process fake gRPC server implementing kmspb.KeyManagementServiceServer:

type fakeKMSServer struct {
    kmspb.UnimplementedKeyManagementServiceServer
    key []byte // AES key for real encrypt/decrypt
}

func (s *fakeKMSServer) Encrypt(ctx context.Context, req *kmspb.EncryptRequest) (*kmspb.EncryptResponse, error) {
    // Real AES-GCM encryption using s.key
}

func (s *fakeKMSServer) Decrypt(ctx context.Context, req *kmspb.DecryptRequest) (*kmspb.DecryptResponse, error) {
    // Real AES-GCM decryption using s.key
}

Provider created with gcpkms.Config{Endpoint: fakeServer.Addr} and option.WithoutAuthentication().

OCI KMS Mock HTTP Server

No Docker emulator exists for OCI KMS. Integration tests use net/http/httptest.Server mimicking the OCI REST API:

handler := http.NewServeMux()
handler.HandleFunc("POST /20180608/encrypt", func(w http.ResponseWriter, r *http.Request) { /* ... */ })
handler.HandleFunc("POST /20180608/decrypt", func(w http.ResponseWriter, r *http.Request) { /* ... */ })
handler.HandleFunc("POST /20180608/generateDataEncryptionKey", func(w http.ResponseWriter, r *http.Request) { /* ... */ })

The mock server ignores OCI request signing (Authorization header) and performs real AES-GCM encryption for crypto fidelity.

OVHcloud OKMS SDK Mocks

No Docker emulator exists for OVH OKMS. The ovh/okms-sdk-go ships pre-generated mockery mocks:

import "github.com/ovh/okms-sdk-go/mocks"

mockAPI := mocks.NewMockAPI(t)
mockAPI.EXPECT().Encrypt(ctx, keyID, data).Return(jweToken, nil)

Integration tests use these mocks for API-level verification. Real OVH OKMS testing runs in integration-cloud with mTLS certificates.

Docker Compose Setup

services:
  vault:
    image: hashicorp/vault:1.18
    ports:
      - "8200:8200"
    environment:
      VAULT_DEV_ROOT_TOKEN_ID: test-token
    command: ["server", "-dev"]
    healthcheck:
      test: ["CMD", "vault", "status"]
      interval: 5s
      retries: 5

  openbao:
    image: quay.io/openbao/openbao:2.5
    ports:
      - "8210:8200"
    environment:
      BAO_DEV_ROOT_TOKEN_ID: test-token
    command: ["server", "-dev"]
    healthcheck:
      test: ["CMD", "bao", "status"]
      interval: 5s
      retries: 5

  local-kms:
    image: nsmithuk/local-kms:3.11.4
    ports:
      - "8080:8080"
    volumes:
      - ./testdata/aws-seed.yaml:/init/seed.yaml

  azure-kv:
    image: jamesgoulddev/azure-keyvault-emulator:latest
    ports:
      - "4997:4997"

  softhsm:
    image: vegardit/docker-softhsm2-pkcs11-proxy:latest
    ports:
      - "5657:5657"
    volumes:
      - softhsm-tokens:/var/lib/softhsm/tokens

  pykmip:
    image: python:3.12
    ports:
      - "5696:5696"
    command: >
      bash -c "pip install pykmip && python -c \"
      from kmip.services.server import KmipProxyKmipServer
      import kmip.services.server as server
      server.main()
      \""
    volumes:
      - ./testdata/kmip-certs:/certs
      - ./testdata/pykmip-server.conf:/etc/pykmip/server.conf

volumes:
  softhsm-tokens:

BDD Scenarios (Godog/Cucumber)

Feature: Key wrapping with Vault Transit

  Background:
    Given a Vault server is running with dev token "test-token"
    And the Transit engine is enabled at mount "transit"
    And a Transit key "test-key" exists

  Scenario: Wrap and unwrap round-trip
    Given a Vault KMS provider configured with address from environment
    When I wrap 32 bytes of random data with key "test-key"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "test-key"
    Then the unwrapped output should equal the original input

  Scenario: Wrap emits audit event
    Given a Vault KMS provider with a test audit sink
    When I wrap 32 bytes with key "test-key"
    Then the audit sink should have received 1 OnKeyWrapped event
    And the event provider type should be "hcvault"
    And the event key ID should be "test-key"
    And the event input size should be 32
    And the event duration should be greater than 0

  Scenario: Unwrap with wrong key fails
    Given a Transit key "wrong-key" exists
    When I wrap 32 bytes with key "test-key"
    And I unwrap the result with key "wrong-key"
    Then the operation should fail with an error
    And the audit sink should have received 1 OnOperationFailed event
    And the error operation should be "unwrap"

  Scenario: GenerateKey produces correct size
    When I generate a key of size 32 with key "test-key"
    Then the plaintext should be exactly 32 bytes
    And the wrapped output should be non-empty
    When I unwrap the wrapped output with key "test-key"
    Then the unwrapped output should equal the plaintext

  Scenario: Provider close prevents further operations
    Given a Vault KMS provider
    When I close the provider
    And I attempt to wrap 32 bytes with key "test-key"
    Then the operation should fail with "provider is closed"
Feature: Key wrapping with OpenBao Transit

  Background:
    Given an OpenBao server is running with dev token "test-token"
    And the Transit engine is enabled at mount "transit"
    And a Transit key "test-key" exists

  Scenario: Wrap and unwrap round-trip
    Given an OpenBao KMS provider configured with address from environment
    When I wrap 32 bytes of random data with key "test-key"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "test-key"
    Then the unwrapped output should equal the original input

  Scenario: OpenBao environment variables take precedence
    Given BAO_ADDR is set to the OpenBao server address
    And VAULT_ADDR is set to "http://should-not-be-used:8200"
    When I create an OpenBao provider with empty config
    Then the provider should connect to the BAO_ADDR server
    And wrapping should succeed
Feature: Key wrapping with AWS KMS (local-kms)

  Background:
    Given local-kms is running
    And an AWS KMS key "alias/test-key" exists

  Scenario: Wrap and unwrap round-trip
    Given an AWS KMS provider configured with local-kms endpoint
    When I wrap 32 bytes with key "alias/test-key"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "alias/test-key"
    Then the unwrapped output should equal the original input

  Scenario: GenerateKey uses native API
    Given an AWS KMS provider with a test audit sink
    When I generate a key of size 32 with key "alias/test-key"
    Then the audit sink should have received 1 OnKeyGenerated event
    And the event NativeAPI field should be true

  Scenario: Unwrap with wrong key fails
    Given an AWS KMS key "alias/wrong-key" exists
    When I wrap 32 bytes with key "alias/test-key"
    And I unwrap the result with key "alias/wrong-key"
    Then the operation should fail with an error
    And the audit sink should have received 1 OnOperationFailed event

  Scenario: Metrics recorded for successful wrap
    Given an AWS KMS provider with a test metrics collector
    When I wrap 32 bytes with key "alias/test-key"
    Then IncrementWrapTotal should have been called with "aws-kms"
    And RecordWrapDuration should have been called with "aws-kms" and nil error
    And SetInFlightOperations should have been incremented and decremented

  Scenario: Provider-specific audit event for native GenerateDataKey
    Given an AWS KMS provider with a provider-specific test audit sink
    When I generate a key of size 32 with key "alias/test-key"
    Then the AWS audit sink should have received 1 OnNativeGenerateDataKey event
Feature: Key wrapping with Azure Key Vault (emulator)

  Background:
    Given the Azure Key Vault emulator is running
    And an RSA key "test-key" exists in the emulator vault

  Scenario: Wrap and unwrap round-trip
    Given an Azure KMS provider configured with the emulator endpoint
    When I wrap 32 bytes with key "test-key"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "test-key"
    Then the unwrapped output should equal the original input

  Scenario: GenerateKey wraps locally
    Given an Azure KMS provider with a test audit sink
    When I generate a key of size 32 with key "test-key"
    Then the audit sink should have received 1 OnKeyGenerated event
    And the event NativeAPI field should be false

  Scenario: Missing vault URL returns error
    When I create an Azure KMS provider with empty VaultURL
    Then the provider creation should fail with an error containing "vault URL"
Feature: Key wrapping with GCP Cloud KMS (fake gRPC server)

  Background:
    Given a fake GCP KMS gRPC server is running
    And a symmetric key "test-key" exists in key ring "test-ring"

  Scenario: Wrap and unwrap round-trip
    Given a GCP KMS provider configured with the fake server endpoint
    When I wrap 32 bytes with key "test-key"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "test-key"
    Then the unwrapped output should equal the original input

  Scenario: Key ring resolved audit event
    Given a GCP KMS provider with a provider-specific test audit sink
    When I wrap 32 bytes with key "test-key"
    Then the GCP audit sink should have received 1 OnKeyRingResolved event
    And the event should contain the full crypto key resource name
Feature: Key wrapping with Oracle OCI KMS (mock HTTP server)

  Background:
    Given a mock OCI KMS HTTP server is running
    And a master encryption key with OCID "ocid1.key.test" exists

  Scenario: Wrap and unwrap round-trip
    Given an OCI KMS provider configured with the mock server crypto endpoint
    When I wrap 32 bytes with key "ocid1.key.test"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "ocid1.key.test"
    Then the unwrapped output should equal the original input

  Scenario: GenerateKey uses native OCI API
    Given an OCI KMS provider with a test audit sink
    When I generate a key of size 32 with key "ocid1.key.test"
    Then the audit sink should have received 1 OnKeyGenerated event
    And the event NativeAPI field should be true

  Scenario: Crypto endpoint audit event
    Given an OCI KMS provider with a provider-specific test audit sink
    When I wrap 32 bytes with key "ocid1.key.test"
    Then the OCI audit sink should have received 1 OnCryptoEndpointUsed event
Feature: Key wrapping with OVHcloud OKMS (SDK mocks)

  Background:
    Given a mock OVH OKMS API is configured
    And a symmetric key "test-key-id" exists in the mock

  Scenario: Wrap and unwrap round-trip
    Given an OVH KMS provider configured with the mock API
    When I wrap 32 bytes with key "test-key-id"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "test-key-id"
    Then the unwrapped output should equal the original input

  Scenario: GenerateKey uses native DataKey API
    Given an OVH KMS provider with a test audit sink
    When I generate a key of size 32 with key "test-key-id"
    Then the audit sink should have received 1 OnKeyGenerated event
Feature: Key wrapping with PKCS#11 (SoftHSM)

  Background:
    Given SoftHSM is initialized with token "test" and PIN "1234"
    And an AES-256 wrapping key labeled "wrap-key" exists on the token

  Scenario: Wrap and unwrap round-trip
    Given a PKCS#11 KMS provider configured for SoftHSM
    When I wrap 32 bytes with key "wrap-key"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "wrap-key"
    Then the unwrapped output should equal the original input

  Scenario: Session pool metrics
    Given a PKCS#11 provider with max 2 sessions and a test metrics collector
    When I perform 10 concurrent wrap operations
    Then the metrics collector should have recorded session pool sizes
    And all operations should succeed

  Scenario: Session pool exhaustion emits audit event
    Given a PKCS#11 provider with max 1 session and a test audit sink
    When I perform 5 concurrent wrap operations
    Then the audit sink should have received OnSessionPoolExhausted events
Feature: Key wrapping with KMIP (PyKMIP)

  Background:
    Given a PyKMIP server is running with TLS
    And a symmetric key "test-key" exists on the KMIP server

  Scenario: Wrap and unwrap round-trip
    Given a KMIP KMS provider configured with mTLS certificates
    When I wrap 32 bytes with key "test-key"
    Then the wrapped output should be different from the input
    When I unwrap the wrapped output with key "test-key"
    Then the unwrapped output should equal the original input

  Scenario: Connection pool exhaustion emits audit event
    Given a KMIP provider with max 1 connection and a test audit sink
    When I perform 5 concurrent wrap operations
    Then the audit sink should have received OnConnectionPoolExhausted events
    And all operations should eventually succeed

  Scenario: TLS handshake audit event
    Given a KMIP provider with a provider-specific test audit sink
    When I wrap 32 bytes with key "test-key"
    Then the KMIP audit sink should have received 1 OnTLSHandshakeCompleted event
    And the event should contain the server endpoint
Feature: Provider Registry

  Scenario: Register and retrieve providers
    Given an empty KMS registry
    When I register a mock provider with type "mock-1"
    And I register a mock provider with type "mock-2"
    Then the registry should have 2 providers
    And Get("mock-1") should return the first provider
    And Get("mock-2") should return the second provider
    And Get("nonexistent") should return nil
    And Types() should return ["mock-1", "mock-2"]

  Scenario: Duplicate registration fails
    Given a registry with a mock provider type "mock-1"
    When I register another provider with type "mock-1"
    Then the operation should fail with "duplicate provider type"

  Scenario: Close cleans up all providers
    Given a registry with 3 mock providers
    And a test audit sink is attached
    When I close the registry
    Then all 3 providers should be closed
    And the audit sink should have received 3 OnProviderClosed events
    And the registry should have 0 providers
Feature: Observability — Metrics

  Scenario: Counters and durations recorded for successful wrap
    Given a registry with a mock provider and a test metrics collector
    When I wrap 32 bytes
    Then IncrementWrapTotal should have been called with provider type "mock"
    And RecordWrapDuration should have been called with provider type "mock" and nil error
    And SetInFlightOperations should have been set to 1 then back to 0

  Scenario: Counters and durations recorded for successful unwrap
    Given a registry with a mock provider and a test metrics collector
    When I wrap 32 bytes and then unwrap the result
    Then IncrementUnwrapTotal should have been called with provider type "mock"
    And RecordUnwrapDuration should have been called with provider type "mock" and nil error

  Scenario: Counters and durations recorded for successful generate_key
    Given a registry with a mock provider and a test metrics collector
    When I generate a key of size 16
    Then IncrementGenerateKeyTotal should have been called with provider type "mock"
    And RecordGenerateKeyDuration should have been called with provider type "mock" and nil error

  Scenario: Error counter incremented on failure
    Given a registry with a mock provider that always fails and a test metrics collector
    When I attempt to wrap 32 bytes
    Then IncrementWrapTotal should have been called with provider type "mock"
    And IncrementOperationError should have been called with provider type "mock" and operation "wrap"
    And RecordWrapDuration should have been called with a non-nil error

  Scenario: Active providers gauge updated on register and close
    Given an empty registry with a test metrics collector
    When I register a mock provider with type "mock-1"
    Then SetActiveProviders should have been called with 1
    When I register a mock provider with type "mock-2"
    Then SetActiveProviders should have been called with 2
    When I close the registry
    Then SetActiveProviders should have been called with 0

  Scenario: Nil metrics collector does not panic
    Given a registry with a mock provider and NO metrics collector
    When I wrap 32 bytes
    And I unwrap the result
    And I generate a key of size 16
    Then all operations should succeed without panic

  Scenario: Concurrent operations record correct in-flight count
    Given a registry with a mock provider (100ms latency) and a test metrics collector
    When I perform 5 concurrent wrap operations
    Then SetInFlightOperations should have peaked at 5
    And after all complete SetInFlightOperations should be 0
Feature: Observability — Structured Logging

  Scenario: Successful wrap logs at DEBUG level
    Given a registry with a mock provider and a test slog handler at DEBUG level
    When I wrap 32 bytes with key "test-key"
    Then a DEBUG log "kms: wrap started" should have been emitted with field "kms.key_id"="test-key"
    And a DEBUG log "kms: wrap complete" should have been emitted with fields "kms.duration_ms" and "kms.output_size"

  Scenario: Failed unwrap logs at ERROR level
    Given a registry with a mock provider and a test slog handler
    When I attempt to unwrap corrupted data with key "test-key"
    Then an ERROR log "kms: unwrap failed" should have been emitted
    And the log should have field "kms.provider"="mock"
    And the log should have field "kms.key_id"="test-key"
    And the log should have field "kms.error" containing an error message

  Scenario: Slow operation logs at WARN level
    Given a registry with a mock provider (200ms latency) and slow threshold 100ms
    And a test slog handler at WARN level
    When I wrap 32 bytes with key "test-key"
    Then a WARN log "kms: wrap slow" should have been emitted
    And the log should have field "kms.slow_threshold_ms"=100

  Scenario: Provider registration logs at INFO level
    Given an empty registry with a test slog handler
    When I register a mock provider with type "mock-1"
    Then an INFO log "kms: provider registered" should have been emitted with field "kms.provider"="mock-1"

  Scenario: Registry shutdown logs start and complete
    Given a registry with 2 mock providers and a test slog handler
    When I close the registry
    Then an INFO log "kms: registry shutdown started" should have been emitted with field "kms.provider_count"=2
    And an INFO log "kms: registry shutdown complete" should have been emitted with field "kms.duration_ms"

  Scenario: Provider close error logs at ERROR level
    Given a registry with a mock provider that fails on Close and a test slog handler
    When I close the registry
    Then an ERROR log "kms: provider close failed" should have been emitted with field "kms.error"

  Scenario: No-op logger produces no output
    Given a registry with a mock provider and NO logger configured
    When I wrap 32 bytes
    Then no log output should have been produced
Feature: Concurrent multi-provider operations

  Scenario: Simultaneous operations across different providers
    Given a registry with mock providers "vault" and "aws" and a test metrics collector
    When I perform 10 concurrent wraps: 5 with provider "vault" and 5 with provider "aws"
    Then all 10 operations should succeed
    And IncrementWrapTotal should have been called 5 times with "vault"
    And IncrementWrapTotal should have been called 5 times with "aws"
    And no data race should be detected
Feature: Local AEAD provider

  Scenario: Round-trip with 256-bit key
    Given a local AEAD provider with a random 256-bit key
    When I wrap the plaintext "hello world"
    And I unwrap the result
    Then the output should equal "hello world"

  Scenario: Different ciphertext each time
    Given a local AEAD provider with a fixed key
    When I wrap the same plaintext twice
    Then the two ciphertexts should be different

  Scenario: Wrong key fails to unwrap
    Given two local AEAD providers with different random keys
    When I wrap with provider A and unwrap with provider B
    Then the unwrap should fail with a decryption error

Project Setup Requirements

Repository Structure

go-kms/
├── go.mod                          # Core module: github.com/axonops/go-kms (zero deps)
├── go.sum
├── LICENSE                         # Apache-2.0
├── README.md
├── CONTRIBUTING.md
├── CHANGELOG.md
├── Makefile
│
├── kms.go                          # Provider interface, Option, OperationConfig
├── kms_test.go
├── registry.go                     # Registry implementation
├── registry_test.go
├── instrumented.go                 # Instrumented provider wrapper (audit/metrics/logging)
├── instrumented_test.go
├── errors.go                       # Sentinel errors, ProviderError
├── errors_test.go
├── audit.go                        # AuditSink interface, event types, BaseAuditSink
├── audit_test.go
├── metrics.go                      # MetricsCollector interface, BaseMetricsCollector
├── metrics_test.go
│
├── aead/                           # Local AEAD provider (in core module)
│   ├── provider.go
│   └── provider_test.go
│
├── kmstest/                        # Mock provider for consumer testing (in core module)
│   ├── mock.go
│   └── mock_test.go
│
├── providers/
│   ├── vault/                      # HashiCorp Vault Transit
│   │   ├── go.mod                  # Depends on: github.com/hashicorp/vault/api
│   │   ├── go.sum
│   │   ├── provider.go
│   │   ├── provider_test.go
│   │   ├── audit.go               # Vault-specific AuditSink + BaseAuditSink
│   │   └── config.go
│   │
│   ├── openbao/                    # OpenBao Transit (fully independent)
│   │   ├── go.mod
│   │   ├── go.sum
│   │   ├── provider.go
│   │   ├── provider_test.go
│   │   ├── audit.go
│   │   └── config.go
│   │
│   ├── awskms/                     # AWS KMS
│   │   ├── go.mod                  # Depends on: aws-sdk-go-v2/service/kms, config, credentials
│   │   ├── go.sum
│   │   ├── provider.go
│   │   ├── provider_test.go
│   │   ├── audit.go
│   │   └── config.go
│   │
│   ├── azurekms/                   # Azure Key Vault
│   │   ├── go.mod                  # Depends on: azure-sdk-for-go/sdk/security/keyvault/azkeys, azidentity
│   │   ├── go.sum
│   │   ├── provider.go
│   │   ├── provider_test.go
│   │   ├── audit.go
│   │   └── config.go
│   │
│   ├── gcpkms/                     # GCP Cloud KMS
│   │   ├── go.mod                  # Depends on: cloud.google.com/go/kms
│   │   ├── go.sum
│   │   ├── provider.go
│   │   ├── provider_test.go
│   │   ├── audit.go
│   │   └── config.go
│   │
│   ├── ocikms/                     # Oracle OCI KMS
│   │   ├── go.mod                  # Depends on: oracle/oci-go-sdk/v65
│   │   ├── go.sum
│   │   ├── provider.go
│   │   ├── provider_test.go
│   │   ├── audit.go
│   │   └── config.go
│   │
│   ├── ovhkms/                     # OVHcloud OKMS
│   │   ├── go.mod                  # Depends on: ovh/okms-sdk-go
│   │   ├── go.sum
│   │   ├── provider.go
│   │   ├── provider_test.go
│   │   ├── audit.go
│   │   └── config.go
│   │
│   ├── pkcs11kms/                  # PKCS#11 (CGo required)
│   │   ├── go.mod                  # Depends on: miekg/pkcs11
│   │   ├── go.sum
│   │   ├── provider.go
│   │   ├── provider_test.go
│   │   ├── audit.go
│   │   ├── metrics.go             # PKCS#11-specific MetricsCollector
│   │   ├── config.go
│   │   └── session_pool.go        # Session pool implementation
│   │
│   └── kmipkms/                    # KMIP
│       ├── go.mod                  # Depends on: ovh/kmip-go
│       ├── go.sum
│       ├── provider.go
│       ├── provider_test.go
│       ├── audit.go
│       ├── metrics.go             # KMIP-specific MetricsCollector
│       └── config.go
│
├── tests/
│   ├── bdd/
│   │   ├── features/              # Gherkin feature files
│   │   │   ├── vault.feature
│   │   │   ├── openbao.feature
│   │   │   ├── awskms.feature
│   │   │   ├── pkcs11.feature
│   │   │   ├── kmip.feature
│   │   │   ├── registry.feature
│   │   │   ├── observability.feature
│   │   │   └── aead.feature
│   │   ├── steps/                 # Step definitions
│   │   └── bdd_test.go            # Godog entry point
│   ├── docker/
│   │   ├── docker-compose.yml     # Vault, OpenBao, LocalStack, SoftHSM, PyKMIP
│   │   └── testdata/              # TLS certs, PyKMIP config, SoftHSM setup
│   └── integration/               # Go integration tests (non-BDD)
│
├── .github/
│   └── workflows/
│       ├── ci.yml                 # Unit tests, lint, race detection (all platforms)
│       ├── integration.yml        # Docker-based integration tests (Vault, OpenBao, LocalStack, SoftHSM, PyKMIP)
│       └── integration-cloud.yml  # Real cloud KMS tests (gated by secrets)
│
└── docs/
    └── providers.md               # Provider configuration reference

Community Files

  • README.md — project overview, quick start, provider list, badges
  • CONTRIBUTING.md — development setup, testing guide, provider implementation guide
  • CHANGELOG.md — semver changelog
  • LICENSE — Apache-2.0
  • SECURITY.md — vulnerability reporting process
  • CODE_OF_CONDUCT.md — Contributor Covenant

Makefile Targets

.PHONY: test test-race lint integration bdd clean

test:              ## Run unit tests for all modules
	go test ./...
	cd providers/vault && go test ./...
	cd providers/openbao && go test ./...
	cd providers/awskms && go test ./...
	cd providers/azurekms && go test ./...
	cd providers/gcpkms && go test ./...
	cd providers/ocikms && go test ./...
	cd providers/ovhkms && go test ./...
	cd providers/pkcs11kms && go test ./...
	cd providers/kmipkms && go test ./...

test-race:         ## Run unit tests with race detector
	go test -race ./...
	# ... same per-module pattern

lint:              ## Run golangci-lint on all modules
	golangci-lint run ./...
	# ... per-module

integration:       ## Run integration tests (requires Docker)
	docker compose -f tests/docker/docker-compose.yml up -d
	go test -tags=integration ./tests/integration/...
	docker compose -f tests/docker/docker-compose.yml down

bdd:               ## Run BDD tests (requires Docker)
	docker compose -f tests/docker/docker-compose.yml up -d
	go test -tags=bdd ./tests/bdd/...
	docker compose -f tests/docker/docker-compose.yml down

clean:             ## Clean test artifacts
	docker compose -f tests/docker/docker-compose.yml down -v

CI Jobs

Job Trigger What
unit-tests Every push/PR Unit tests, race detector, lint across Go 1.22/1.23, Linux/macOS
integration Every push/PR Docker-based tests: Vault, OpenBao, LocalStack, SoftHSM, PyKMIP
integration-cloud Manual / nightly Real cloud KMS: AWS, Azure, GCP, OCI, OVH (requires secrets)
release Tag push GoReleaser, pkg.go.dev indexing

Migration Path for Schema Registry

  1. Create github.com/axonops/go-kms repository with the structure above
  2. Port core interface from internal/kms/provider.gokms.go, adding Options, errors, audit/metrics interfaces
  3. Port each provider from internal/kms/{provider}/provider.goproviders/{provider}/provider.go
    • Vault: Port as-is, add slog logging and audit sink hooks
    • OpenBao: Rewrite independently (no longer wraps Vault provider)
    • AWS, Azure, GCP: Port as-is with minor API adjustments
    • OCI, OVH, PKCS#11, KMIP: New implementations
  4. Add GenerateKey size parameter — change from algorithm string to byte size; move algorithm→size mapping to schema-registry
  5. Port tests from internal/kms/*/provider_test.go → provider packages
  6. Add BDD scenarios using Docker Compose infrastructure
  7. Tag v0.1.0 once core + Vault + OpenBao + AWS + local AEAD pass all tests
  8. In schema-registry: Replace internal/kms imports with github.com/axonops/go-kms
    • import kms "github.com/axonops/go-kms" (core)
    • import "github.com/axonops/go-kms/providers/vault" (per provider)
  9. In schema-registry: Add algorithm→keySize mapping in internal/registry/registry_dek.go:
    func keySize(algorithm string) (int, error) {
        switch algorithm {
        case "AES128_GCM": return 16, nil
        case "AES256_GCM": return 32, nil
        case "AES256_SIV": return 64, nil
        default: return 0, fmt.Errorf("unsupported algorithm: %s", algorithm)
        }
    }
  10. In schema-registry: Wire audit/metrics bridges
    kmsReg := kms.NewRegistry(
        kms.WithLogger(slogLogger),
        kms.WithAuditSink(&KMSAuditBridge{logger: auditLogger}),
        kms.WithMetrics(prometheusAdapter),
    )
  11. Delete internal/kms/ from schema-registry
  12. Remove KMS provider tests from schema-registry CI (they now live in go-kms)

Dependencies

Core Module (github.com/axonops/go-kms)

Dependency Purpose Required
Go stdlib only crypto/aes, crypto/cipher, crypto/rand, log/slog, sync, context Yes
No external dependencies Core module is zero-dep

Provider Modules

Provider SDK Dependency License Compatible with Apache-2.0
vault github.com/hashicorp/vault/api v1.22.0+ MPL-2.0 Yes
openbao github.com/hashicorp/vault/api v1.22.0+ (or openbao/openbao/api when published) MPL-2.0 Yes
awskms github.com/aws/aws-sdk-go-v2/service/kms, config, credentials Apache-2.0 Yes
azurekms github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys, azidentity MIT Yes
gcpkms cloud.google.com/go/kms Apache-2.0 Yes
ocikms github.com/oracle/oci-go-sdk/v65 Apache-2.0 / UPL-1.0 (dual) Yes
ovhkms github.com/ovh/okms-sdk-go Apache-2.0 Yes
pkcs11kms github.com/miekg/pkcs11 BSD-3-Clause Yes
kmipkms github.com/ovh/kmip-go Apache-2.0 Yes

Test Dependencies

Dependency Purpose Required
github.com/stretchr/testify Test assertions Yes
github.com/cucumber/godog BDD testing Yes
SoftHSM 2 (system package) PKCS#11 integration tests Integration only
PyKMIP (Python package) KMIP integration tests Integration only
Docker Compose Integration test infrastructure Integration only

Licensing Summary

All provider SDK dependencies are compatible with Apache-2.0:

  • Apache-2.0: AWS, GCP, OCI (dual), OVH, KMIP — perfect match
  • MIT: Azure — more permissive, no issues
  • MPL-2.0: Vault, OpenBao — compatible per Mozilla FAQ; copyleft applies only to modifications of MPL files themselves
  • BSD-3-Clause: PKCS#11 — permissive, no issues

Multi-module structure mitigates license concerns: consumers who do not import the Vault or OpenBao providers never pull in MPL-2.0 dependencies. The core module is pure Apache-2.0 with zero external dependencies.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrefactorCode refactoring

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions