4676a8c5 - Add bank-processing backlog observer (open transactions monitoring) - #4635
4676a8c5 - Add bank-processing backlog observer (open transactions monitoring)#4635TaprootFreak wants to merge 16 commits into
Conversation
214a625 to
2c81905
Compare
|
4 review passes until zero findings. Fixed along the way: cron-job inventory completeness (including two pre-existing missing declarations), CHF rounding via the shared Util helper, type-level enforcement of the tolerance field, fail-loud guards for non-finite aggregation values, lock-timeout parity with the sibling observers, and adoption of the new mandatory cron scope parameter after rebasing onto the current develop. |
41d4f8d to
5e7ee83
Compare
|
Reworked after the handbook landed on develop: rebased, adopted the mandatory cron scope parameter, and added a handbook-discoverable docs page for the feature (rule model, tolerance semantics, log-line contract) with its metadata entry — the handbook build passes with the new page. Two further review passes on the rework brought the total to 6 passes until zero findings. |
Pure, unit-tested modules for a new backlog observer over open bank transactions: ~36 invariant rules (bank_tx, buy_crypto, buy_fiat, fiat_output, bank_tx_return) with per-rule counts, CHF sums and age-based overdue tracking, plus a time-of-day/weekend tolerance helper (Europe/Zurich). Observer wiring follows in the next commit.
Runs the rule catalog every 5 minutes (one bundled aggregation query per base table), emits results into the monitoring system state and writes the snapshot log lines consumed by downstream monitoring: a summary heartbeat plus one JSON line per rule. Own process gate (BankProcessingMonitoring) with an explicit timeout; a failing block aborts the whole run so a heartbeat never covers partial data.
- add the new cron job to the docs/cron-jobs.md inventory - use Util.round for CHF sums instead of a local helper; assert the repo-wide half-cent semantics in the tests - enforce toleranceField at the type level (discriminated union) - fail loud on non-finite aggregation values before emitting - type getRawOne and guard against an empty aggregation row - align the lock timeout with the sibling observers (1800s) and document why a finite timeout is mandatory - document the accepted cross-block snapshot skew - add missing return types; translate a leftover German comment
…s without strictNullChecks)
…were missing) The self-check line claimed consistency while two source declarations (ProcessService::resyncStaffKycClearance, StaffKycClearanceService:: syncStaffKycClearance) were absent from the table. Add both rows and re-derive every dependent number (134 declarations / 94 files, flag counts, interval and area tables). Also align two rule labels to pure German for dashboard consistency.
…tory after rebase The cron/HTTP process split made scope mandatory on @DfxCron; the observer registers as a worker job like its siblings. The inventory was rewritten on develop meanwhile — re-add the new job row against the new format (142 declarations / 99 files, flag, interval, area and self-check numbers).
…dbook New docs page covering the rule model, the tolerance semantics, the log-line contract (heartbeat plus per-rule JSON, null as the display-only marker) and the accepted limitations, with a German handbook metadata entry. The page participates in the handbook's recursive markdown discovery.
…se-filter exclusions, tolerance offsets)
0105121 to
05f2c59
Compare
What
Adds a
BankProcessingObserver(monitoring subdomain) that watches open bank-processing transactions across five tables —bank_tx,buy_crypto(fiat- and crypto-funded),buy_fiat,fiat_output,bank_tx_return— as 45 declarative invariant rules. Per rule it reports:count/chfSum: current backlog and CHF valueoverdueCount/overdueChf: entries older than the rule's tolerance (fixed minutes or a time-of-day/weekend-dependent tolerance in Europe/Zurich)Every 5 minutes the observer emits the results into the monitoring
SystemStateand writes structured log lines (a summary heartbeat plus one JSON line per rule). The log lines are a monitoring interface: downstream dashboards parse them, and a missing heartbeat means "monitoring is down" — missing data is never treated as "all clear".Why
This replaces a legacy spreadsheet-based ops monitor that polled
/gs/dband mailed alerts through a retired host (silently broken). The rules were re-derived as invariants on the current schema; per-ID exceptions, manual offsets and stale status values from the legacy rules were deliberately not ported.Notes for review
bank-processing.rules.ts); the SQL fragments quote camelCase identifiers for Postgres and were verified against a real Postgres 16 instance.worker-scope cron with its own process gate (BankProcessingMonitoring) and an explicit finite lock timeout (1800s, sibling parity). The monitoring state reaches the API process through the persisted snapshot.bank_tx/fiat_outputare approximations viafiat.approxPriceChf; non-convertible currencies drop out of the sum (counts stay complete). CHF rounding follows the repo-wideUtil.roundsemantics.docs/bank-processing-monitoring.md(rule model, tolerance semantics, log-line contract, accepted limitations) plus a German metadata entry; the handbook build passes with the new page (21 docs).docs/cron-jobs.mdgains the new job row; two pre-existing missing declarations were added and all inventory counts re-derived (142 = 142 self-check).aml/paymentobservers keep their coarser counts; this observer does not touch them.