優化掃描穩定性與吞吐（SSE / FileStore / SARIF / DAST） - #19
Conversation
There was a problem hiding this comment.
Pull request overview
This PR enhances scan stability and throughput across multiple dimensions: SSE-based streaming with keepalive for the extension, FileStore fast-path for scan tasks, SARIF 2.1.0 export, persistent file analysis cache, JS/TS AST worker pool, CLI DAST verification command, and CI workflow optimization with path filtering and Turborepo remote cache.
Changes:
- SSE streaming with event IDs, keepalive, and
Last-Event-IDresume; extension client uses SSE primary with polling fallback and exponential backoff - FileStore
scanTaskfast read/write path,stableFingerprint/sourcefields on vulnerabilities, persistent file analysis cache, and SARIF 2.1.0 export format - CLI
verify webcommand integrating ZAP baseline and Nuclei with conservative defaults, plus CI path filtering and Turborepo remote cache environment variables
Reviewed changes
Copilot reviewed 30 out of 32 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
web/src/server/routes/scan.ts |
SSE keepalive, event IDs, Last-Event-ID support, actionable LLM failure messages |
web/src/server/routes/scan.test.ts |
Test for missing API key actionable error propagation |
web/src/server/routes/export.ts |
SARIF 2.1.0 renderer, stableFingerprint/source in serialization and CSV columns |
web/src/server/routes/export.test.ts |
SARIF export test, updated CSV header assertion |
web/src/server/file-analysis-cache-store.ts |
New: persistent file analysis cache with version-gated hydration |
web/src/server/file-analysis-cache-store.test.ts |
Tests for hydration, version mismatch, and persistence |
web/src/server/db.ts |
Fast-path scan task read/write, stableFingerprint/source fields, meta normalization |
web/src/server/cache.ts |
24h TTL for file analysis cache, shared key builder |
web/src/server/agents/orchestrator.ts |
Persistent cache integration, Map for language lookup |
web/src/server/agents/agentic-beta/orchestrator.ts |
Same persistent cache integration, rememberFirstFailureMessage |
web/src/server/agents/analysis-agent.ts |
lastErrorMessage in LlmUsageStats, rememberFirstFailureMessage |
web/src/server/agents/jsts-agent.ts |
Worker pool for JS/TS AST analysis with sequential fallback |
web/src/server/agents/messages.pbt.test.ts |
Added stableFingerprint/source to PBT arbitraries |
web/src/common/libs/types.ts |
stableFingerprint, source, sarif in shared types |
web/src/common/libs/api-client.ts |
resolveBaseUrl uses window.location.origin for webview |
web/src/common/components/dashboard/main.tsx |
SARIF option in export dialog |
extension/src/scan-client.ts |
SSE primary with polling fallback, SSE message parser, retry logic |
extension/src/types.ts |
stableFingerprint/source fields |
extension/src/webview.test.ts |
Updated test fixtures with new fields |
confession-cli/bin/confession.js |
verify web command with ZAP/Nuclei integration |
confession-cli/bin/confession.test.js |
Tests for verify web success and tool-missing paths |
web/benchmark-scan-workspace.mjs |
New: benchmark script for scan throughput measurement |
web/package.json |
Added benchmark:scan script |
.github/workflows/ci.yml |
Path filtering, Turborepo remote cache env vars |
AGENTS.md |
Updated documentation for all new features |
.kiro/steering/*.md |
Updated steering docs to reflect new capabilities |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| region: { | ||
| startLine: item.line, | ||
| startColumn: item.column, | ||
| endLine: item.endLine, | ||
| endColumn: item.endColumn, | ||
| snippet: { | ||
| text: item.codeSnippet, | ||
| }, | ||
| }, |
There was a problem hiding this comment.
SARIF 2.1.0 requires startLine and startColumn to be 1-based (Section 3.30.5). The vulnerability data model allows line and column to be 0 (default in normalizeVulnerabilityCreate), which would produce invalid SARIF region values. Consider clamping to Math.max(1, item.line) and Math.max(1, item.column) (and similarly for endLine/endColumn) when building the SARIF region to ensure spec compliance.
| function rememberFirstFailureMessage(stats: LlmUsageStats, err: unknown): void { | ||
| if (typeof stats.lastErrorMessage === 'string' && stats.lastErrorMessage.length > 0) { | ||
| return | ||
| } | ||
|
|
||
| const raw = err instanceof Error ? err.message : String(err) | ||
| const message = raw.trim() | ||
| if (message.length === 0) return | ||
| stats.lastErrorMessage = message.slice(0, 280) | ||
| } |
There was a problem hiding this comment.
The rememberFirstFailureMessage function is duplicated identically here and in analysis-agent.ts (line 427). Since accumulateFailureKind and isConcurrencyThrottleError are also already duplicated between these files, this follows the existing pattern. However, as the number of duplicated utility functions grows, consider extracting these shared LLM stats helpers (including accumulateFailureKind, isConcurrencyThrottleError, and rememberFirstFailureMessage) into a shared module to reduce future maintenance burden.
變更摘要
/api/export新增 SARIF 2.1.0，CSV 補 stableFingerprint/source，Dashboard 新增 SARIF 選項confession verify web，整合 ZAP baseline + Nuclei（保守 timeout/rate/concurrency）pnpm --filter web benchmark:scan.kiro/steering/*與AGENTS.md驗證
pnpm check:ci全部通過（lint/build/test）風險與回滾