Skip to content

[Tech Debt] Python e2e tests set a non-existent checkpointing key, so they never checkpoint #976

Description

@weiqingy

Search before asking

  • I searched in the issues and found nothing similar.

Description

checkpointing.interval is not a Flink configuration key, so the Python e2e tests that set it are running with checkpointing disabled.

The tests set this three-line block in 11 places across 5 files, for example python/flink_agents/e2e_tests/e2e_tests_integration/flink_intergration_test.py:53-55:

config.set_string("state.backend.type", "rocksdb")
config.set_string("checkpointing.interval", "1s")
config.set_string("restart-strategy.type", "disable")

The real option is execution.checkpointing.interval, defined at CheckpointingOptions.java:512. The bare checkpointing. form is not registered anywhere, and it is not wired up as a deprecated or fallback key either, so Flink ignores it silently. execution.checkpointing.interval has .noDefaultValue() (CheckpointingOptions.java:514), so when the key is ignored no periodic checkpointing is scheduled at all.

The net effect is that these tests configure a RocksDB state backend that is never snapshotted, while reading as though they exercise checkpointing.

Verified by grepping the Flink source at every version this repo builds against (dist/ has flink-1.20 through flink-2.3). On release-1.20, release-2.2.0, release-2.3 and master, key("execution.checkpointing.interval") is present and key("checkpointing.interval") returns nothing. Also checked that PyFlink does not remap the short form on the Python side: no hits for "checkpointing.interval" anywhere under pyflink/.

Affected files:

  • python/flink_agents/e2e_tests/e2e_tests_integration/execute_test.py
  • python/flink_agents/e2e_tests/e2e_tests_integration/flink_intergration_test.py
  • python/flink_agents/e2e_tests/e2e_tests_integration/workflow_memory_remote_test.py
  • python/flink_agents/e2e_tests/e2e_tests_integration/e2e_tests_mcp/mcp_test.py
  • python/flink_agents/e2e_tests/e2e_tests_integration/mock_chat_model_test.py

Suggested fix: rename the key to execution.checkpointing.interval in all 11 places. Worth noting that the fix may not be purely mechanical. These tests have never actually checkpointed, so turning checkpointing on for the first time could surface real failures, and those failures would be the more interesting half of this issue.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority/majorDefault priority of the PR or issue.tech debt[Issue Type] User-unaware issues, such as code refactor and infrastructure maintenance.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions