Antalya 26.6: Add commit info to partition exports table - #2210
Conversation
…next commit) --- Original cherry-pick message follows: Merge pull request #1832 from Altinity/expand-replicated-partition-exports-columns Add commit info to partition exports table # Conflicts: # docs/en/antalya/partition_export.md # src/Storages/ObjectStorage/DataLakes/IDataLakeMetadata.h # src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp # src/Storages/ObjectStorage/StorageObjectStorage.cpp
Kept antalya-26.6's metadata_snapshot / configuration arguments while adopting the PR's ExportPartitionCommitInfo return values.
CI triage for #2210Verdict — 5 distinct failures: 2 PR-related, 3 not.
The top-level PR-related (need attention)
Decisive evidence it's this port, not a pre-existing 26.6 issue:
Suggested next steps for the author:
Not PR-related
I did not push anything — this was a triage request. If you'd like, I can (a) dig into the (Analysis from run 31787415505 @ |
RelEasy
|
CI triageVerdict: 4 distinct failure groups. 2 are not caused by this PR (Swarms + Stateless — unrelated tests, flaky/timeout, also red on Reduced "releasy" workflow, so most jobs show 1. Stateless tests — NOT PR‑caused (flaky / timeout)
Neither test touches export code (the diff only changes 2. Swarms (aarch64 + release) — NOT PR‑caused (pre‑existing / flaky infra)Only 2–3 scenarios fail, all unrelated to exports: 3.
|
CI triage for #2210Verdict: 9 failing checks. None is a clean, provably PR-introduced hard failure I can pin from the logs, but two regression suites land squarely on this PR's feature area and deserve the author's eyes. Breakdown: 2 stateless = flaky/unrelated, swarms = flaky (branch baseline), iceberg_2 + s3_export_part = pre-existing "known-red" on
Not caused by this PR1. Stateless 2. Stateless 3. Regression Swarms (aarch64 + release) — flaky Pre-existing "known-red" on the base branch, but touching this PR's code — please verify4. Regression Iceberg (2) The other ~88 are bare result-content 5. Regression S3Export (part) Why "pre-existing" but still flagged: these exact suites ( But don't dismiss them outright: #2210's Suggested next steps
If you'd like, I can go deeper on any single suite, or draft a fix once we know whether the break is code vs. regression-suite expectations — just point me at it. |
CI triage for #2210 — correctionVerdict: none of the CI failures are caused by this PR. This supersedes my earlier triage comment, which classified MethodByte-level diff of the per-scenario result trees for every regression suite, against two independent baselines on the same branch:
All at The 17 failing checks = 5 distinct failures
The top-level
|
| Scenario | #2208 | #2209 | #2210 x86 | #2210 aarch64 |
|---|---|---|---|---|
node failure/check restart swarm node |
Fail | OK | Fail | Fail |
feature/object storage cluster profile events with overloaded node |
OK | OK | Fail | OK |
task rescheduling/rescheduling with bucket granularity |
OK | Fail | Fail | Fail |
object storage cluster profile events with overloaded node fails on x86 and passes on aarch64 at the same SHA. check restart swarm node fails with Code: 394 ... QUERY_WAS_CANCELLED during a deliberate node kill.
One of these is not flaky and deserves its own issue: task rescheduling/rescheduling with bucket granularity fails with the identical count — Expected 200000 total rows, but got 168783 — on #2209 and on both architectures of #2210. Deterministic, and reproducible on a settings-only pull request, so it is a base-branch issue. This PR touches no swarm scheduling code; its only StorageObjectStorageCluster change is a void → ExportPartitionCommitInfo return-type plumb.
Stateless tests
03599_lightweight_delete_vertical_merge—Code: 159. DB::Exception: Timeout exceeded (180 s) while flushing system log 'DB::SystemLogQueue<DB::PartLogElement>'onSYSTEM FLUSH LOGS part_log, under debug + s3 + distributed plan. Lightweight-delete / vertical-merge subsystem, untouched by this PR.00024_random_counters—[ FAIL ] 600.03 sec, the harness test timeout under amd_debug sequential load. CI's own randomized-settings diagnosis re-ran it with the same settings and it passed: "All reruns passed. The failure is not reproducible (likely a transient issue)."
Positive evidence that the port is correct
My earlier comment claimed the integration tests were skipped. They were not — they ran and passed:
Integration tests (arm_binary, distributed plan), all 4 shards, 0 failures out of 6002 results. Those shards include the files this PR modifies:
test_export_replicated_mt_partition_to_object_storage/test.py(shard 2)test_export_replicated_mt_partition_to_iceberg/test.py(shard 3)
which are exactly the tests asserting the new system.replicated_partition_exports commit-info columns and the UUIDv4 transaction_id. Regression release/aarch64 s3_export_partition, iceberg_1 and all parquet* suites passed on both architectures as well.
Conflict-resolution audit
Re-checked c1efccca against 0329de17 (the original #1832 merge):
- No leftover conflict markers in
src/,docs/ortests/. IcebergMetadata.cpp—storage_manifest_entry_path→storage_manifest_entry_nameis a required rename: 26.6 renamed the local, and both holdresolver.resolve(manifest_entry_path), so the value is unchanged.IcebergMetadata::commitExportPartitionTransaction— the retry loop preserves Add commit info to partition exports table #1832's semantics:if (commit_info) return *commit_info;otherwise retry, throwingUNFINISHEDafterMAX_TRANSACTION_RETRIES.StorageObjectStorage::commitExportPartitionTransaction— keeps 26.6'smetadata_snapshot/configurationarguments while returning Add commit info to partition exports table #1832'sExportPartitionCommitInfo.docs/en/antalya/partition_export.md— retains the full new content (destination_file_pathsand thecommitted_*sections).
Recommendation
Nothing to fix in this PR. Re-running swarms is the only action that could turn a check green; iceberg_2 and s3_export_part are branch-level noise that no change here can clear.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add Iceberg metadata file paths and data file paths to the
system.replicated_partition_exportsto improve observability and debugging. Partially tackles #1824Also replaces clickhouse snowflakeid implementation with uuidv4 (#1832 by @arthurpassos).
CI/CD Options
Exclude tests:
Regression jobs to run:
Combined port of 1 PR(s) (group
pr-1832). Cherry-picked from #1832.