Fix sentiment 502 and cut per-request latency across slow endpoints - #2
Open
nabindev3 wants to merge 1 commit into
Open
Fix sentiment 502 and cut per-request latency across slow endpoints#2nabindev3 wants to merge 1 commit into
nabindev3 wants to merge 1 commit into
Conversation
The /api/v1/sentiment/analyse endpoint was returning 502 Bad Gateway from Render's proxy because the engine could spend up to ~400s per request in the worst case. Several other endpoints had silent slowness from re-reading CSVs and rewriting cache files on every request. Sentiment engine (src/hf_sentiment_engine.py, src/sentiment_engine.py): - Replace per-request disk cache (full JSON read + full JSON rewrite on every call) with a thread-safe in-memory LRU. Disk cache was O(N) I/O per request and burned Render's ephemeral disk. - Cap HFInferenceClient TIMEOUT 25s->8s, RETRY_ATTEMPTS 3->1, COLD_START_WAIT 20s->3s, and disable wait_for_model so HF returns 503 fast on a cold model instead of blocking up to 20s. - Fan out the 3 HF model calls (sentiment, irony, zero-shot aspect) concurrently with a ThreadPoolExecutor. Latency is now bounded by the slowest of the three, not their sum. - Drop the 0.3-0.5s sleep between batch items; replace sequential batch with bounded concurrent execution. - Don't cache failure results so the next request can retry the warm engine. - Include engine identifier in cache key so a TextBlob fallback doesn't poison later warm-engine lookups. Sentiment router (backend/routers/sentiment.py): - Hard endpoint deadline (12s single, scaled+capped 45s batch). If the underlying engine doesn't return in time we serve a TextBlob result instead of letting the request hang until Render's proxy emits 502. Hot-path I/O caching: - backend/routers/forecast.py: add @lru_cache to _load_daily and _load_ext (were re-parsing 2192-row + 6-year CSVs on every /forecast call). - backend/routers/pricing.py: same fix for daily_kpis and external_regs. - backend/routers/xai.py: cache the 180k-row bookings DataFrame used by /global-importance instead of re-reading it per request. All 25 existing tests pass; suite runtime dropped from 5.0s to 3.9s.
nabindev3
added a commit
that referenced
this pull request
Jun 13, 2026
- api.cached_get(): short-lived (API_CACHE_TTL, default 60s) cache keyed on path+params, so reruns from slider moves / tab switches don't re-fire the same GETs. Sidebar KPIs/health/data-quality and every tab's data GET now use it; get_schema uses it too (front-end review #1, #3). - The sidebar no longer pings http://localhost:5001 on every render (#2): the MLflow status row is gated behind MLFLOW_UI_URL (off by default) and its health check is cached 30s, so cloud deploys don't eat a 2s blocking timeout per load. - Dropped the hardcoded localhost:8501 dashboard link (#18); /health 'degraded' now counts as up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The /api/v1/sentiment/analyse endpoint was returning 502 Bad Gateway from Render's proxy because the engine could spend up to ~400s per request in the worst case. Several other endpoints had silent slowness from re-reading CSVs and rewriting cache files on every request.
Sentiment engine (src/hf_sentiment_engine.py, src/sentiment_engine.py):
Sentiment router (backend/routers/sentiment.py):
Hot-path I/O caching:
All 25 existing tests pass; suite runtime dropped from 5.0s to 3.9s.