feat(instrumentation): add Mistral AI auto-instrumentation - #92
Open
h30s wants to merge 2 commits into
Open
Conversation
There was a problem hiding this comment.
1 issue found across 6 files
Confidence score: 3/5
- There is a concrete regression risk in
packages/traceroot/src/mistral.ts:unpatch()may fail to restore the originalchatgetter if lazy patching has not completed, which can leave instrumentation active unexpectedly. - Given the medium severity (5/10) and high confidence (9/10), this is more than a cosmetic issue and could affect runtime behavior for users relying on patch/unpatch correctness.
- Pay close attention to
packages/traceroot/src/mistral.ts- ensureunpatch()reliably restores the original getter in both pre- and post-lazy-patch states.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/traceroot/src/mistral.ts">
<violation number="1" location="packages/traceroot/src/mistral.ts:242">
P2: `unpatch()` does not restore the original `chat` getter when lazy patching has not yet completed, so instrumentation can remain active after unpatch.</violation>
</file>
Architecture diagram
sequenceDiagram
participant User as User Code
participant Initialization as Initialize()
participant Instrumentation as wireInstrumentations()
participant MistralInstr as MistralInstrumentation
participant Tracer as OpenTelemetry Tracer
participant MistralSDK as @mistralai/mistralai
participant MistralAPI as Mistral API
Note over User,MistralAPI: NEW: Mistral AI Auto-Instrumentation Flow
User->>Initialization: Initialize({ instrumentModules: { mistral: ... } })
Initialization->>Instrumentation: wireInstrumentations(opts)
alt Auto-instrumentation path (CommonJS)
Instrumentation->>Instrumentation: registerPlugin(MistralInstrumentation)
Instrumentation->>MistralInstr: init() returns module definition
MistralInstr->>MistralInstr: patch(Mistral.prototype)
else Manual instrument path (ESM)
Instrumentation->>MistralInstr: new MistralInstrumentation()
Instrumentation->>MistralInstr: manuallyInstrument(mistralModule)
MistralInstr->>MistralInstr: patch(mistralModule)
end
Note over MistralInstr: Replaces Mistral.prototype.chat getter<br/>to intercept first access
User->>MistralSDK: new Mistral()
User->>MistralSDK: mistral.chat.complete({ model, messages, ... })
MistralSDK->>MistralInstr: CHANGED: Patched chat getter fires
alt First access (not yet patched)
MistralInstr->>MistralInstr: _wrap(Chat.prototype, 'complete')
MistralInstr->>MistralInstr: Restore original getter
MistralInstr->>MistralSDK: Return Chat with patched prototype
else Subsequent access
MistralInstr->>MistralSDK: Return cached Chat with patched complete
end
Note over MistralInstr,Tracer: Span creation
Tracer->>Tracer: startActiveSpan("mistral.chat.complete {model}")
Tracer->>Tracer: Set attributes:
Note over Tracer: openinference.span.kind = "LLM"<br/>llm.system = "mistralai"<br/>llm.provider = "mistralai"<br/>llm.model_name<br/>input.value / input.mime_type<br/>llm.invocation_parameters<br/>(strips model & messages)
MistralSDK->>MistralAPI: POST /chat/completions
MistralAPI-->>MistralSDK: Response with usage, choices
alt Success
MistralInstr->>MistralInstr: extractOutputValue(response)
Tracer->>Tracer: Set remaining attributes:
Note over Tracer: output.value / output.mime_type<br/>llm.token_count.prompt<br/>llm.token_count.completion<br/>llm.token_count.total<br/>llm.response.finish_reasons
MistralSDK-->>User: Response
Tracer-->>Tracer: span.end()
else Error
MistralSDK->>MistralInstr: Exception thrown
Tracer->>Tracer: Set SpanStatusCode.ERROR
Tracer->>Tracer: Add exception event
MistralSDK-->>User: Exception
Tracer-->>Tracer: span.end()
end
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Author
|
@XinweiHe PTAL 👀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes traceroot-ai/traceroot#739 (TypeScript SDK side)
Summary
Adds
MistralInstrumentationfor@mistralai/mistralaiv2, sending OpenInference-compatible spans. Works with both auto-instrumentation (RITM) and manual (instrumentModules) paths.Since there’s no official npm package yet (
@arizeai/openinference-instrumentation-mistralai), this is the Option B approach requested in the issue.Type of Change
Details
Patches
Mistral.prototype.chatlazily to wrapcomplete()and emit spans.Prevents double-wrapping with a
Symbolmarker.Captures all OpenInference chat-completion attributes (model, token counts, inputs/outputs, finish reasons, errors).
ESM-only support: users should pass the module via
instrumentModules.mistralif auto-hook won’t fire.Files added/updated:
packages/traceroot/src/mistral.ts—MistralInstrumentation+ helperspackages/traceroot/src/instrumentation.ts— registers instrumentorpackages/traceroot/src/types.ts— addsmistraloptionpackages/traceroot/tests/mistral.test.ts— 5 unit tests with fake Mistral moduleValidation
Companion PR
Example usage in
traceroot-ai/traceroot#…. UsestracedComplete()for now to match the future instrumentation shape.Checklist
instrumentModules.mistral)Summary by cubic
Adds Mistral instrumentation for
@mistralai/mistralaiv2 to emit OpenInference-compatible spans for chat completions. Traces model inputs/outputs, token usage, finish reasons, and errors; supports ESM manual wiring viainstrumentModules.mistraland clean uninstrumenting.New Features
MistralInstrumentationthat lazily patchesMistral.prototype.chatto wrapchat.complete(); prevents double-wrapping and is idempotent.llm.model_name, token counts, finish reasons,input.value/output.value, andllm.invocation_parameters.instrumentModules.mistral(ESM-safe); also registered inwireInstrumentations.manuallyUninstrument()to restore the original getter and unwrapcomplete(), even if lazy patching hasn’t fired; tests cover success, tool calls, errors, malformed inputs, idempotency, and uninstrument behavior.Dependencies
@mistralai/mistralai(>=2.0.0); lockfile updated to 2.2.1.Written for commit 58acb1b. Summary will update on new commits.