[Feature] Add an offline tool to backfill historical SectionBloom data

[317787106]

Summary

Add an offline Toolkit command to rebuild historical SectionBloom data whose db name is section-bloom from the existing records in db transactionRetStore. This would allow node operators to restore historical log filtering without resyncing the node or replaying the blockchain.

Problem

SectionBloom data is used by eth_getLogs to identify blocks that may contain logs matching a contract address or topic by bloom filter quickly. If node.jsonrpc.httpFullNodeEnable was disabled while historical blocks were being processed in version v4.8.0 and earlier, SectionBloom data was not generated for those blocks. In v4.8.1 and later, this switch has no effect, and the data is always written. Enabling the option later only creates SectionBloom data for newly processed blocks. It does not repair the missing historical data. As a result, eth_getLogs queries that filter historical ranges by address or topics may fail to find all matching logs.

There is currently no offline tool for rebuilding this data from the local database. Operators may therefore need to resync the node or replay historical blocks, even when the required transaction results are already available in transactionRetStore.

Proposed Solution

Add the following command to Toolkit:

java -jar Toolkit.jar db backfill-bloom

The command should read historical transaction results from transactionRetStore, calculate block Bloom filters using the same logic as SectionBloomStore, and create or update the corresponding records in the section-bloom database.

It should support an inclusive block range through --start-block and --end-block. If no start block is provided, the command should use the earliest non-zero block available in transactionRetStore. If no end block is provided, it should use the latest solidified block recorded in the properties database. Values outside the available range should be adjusted to the actual database boundaries.

The command should process data by Section, with each Section containing 2048 blocks, and support configurable concurrency through --max-concurrency. The expected usage is:

java -jar Toolkit.jar db backfill-bloom \
  [-d <databaseDirectory>] \
  [-s <startBlock>] \
  [-e <endBlock>] \
  [-c <maxConcurrency>]

The default database directory should be output-directory/database, and the default maximum concurrency should be 8. The actual concurrency should not exceed the number of Sections being processed.

For long-running backfills, the command should display terminal progress, periodically write progress information to toolkit.log, and print a final summary containing the number of scanned and successfully processed blocks, blocks containing logs, errors, Bloom writes, elapsed time, processing rate, and concurrency used.

The operation is idempotent so that the same block range can be safely processed again after an interruption. Existing SectionBloom bits should be preserved when records are updated.

The implementation should also validate the database directory, required databases, block range, and concurrency value. Unit tests should cover parameter validation, automatic range detection and adjustment, missing or invalid databases, processing failures, progress and summary output, help output, and safe reprocessing.

Operational Requirements and Risks

The node and any other process accessing the database must be stopped before running the command because the database requires exclusive access. Multiple backfill processes must not operate on the same database concurrently.

The target blocks must have been processed while storage.transHistory.switch was enabled. Otherwise, transactionRetStore will not contain the historical transaction results required to rebuild SectionBloom data.

The backfill may generate significant disk I/O and CPU load when processing large block ranges. Operators should adjust concurrency according to their storage hardware and monitor disk latency and CPU usage during execution.

Impact

After the missing SectionBloom data is rebuilt, eth_getLogs can correctly filter the affected historical blocks by contract address and topics. This feature introduces only an offline maintenance command. It does not add a new network interface or change the normal block-processing flow. Runtime behavior is unaffected unless an operator explicitly runs the command.

Compatibility

• Breaking change: No.
• Default behavior change: No.
• Migration required: No.

Nodes without missing historical SectionBloom data do not need to run this command. Existing configurations and JSON-RPC APIs remain unchanged.

References

Use PR tronprotocol#6390 as a reference, but the implementation may be reworked or redesigned from there.

Additional Notes

• Do you have ideas regarding implementation? Yes.
• Are you willing to implement this feature? Yes.