Gas station aborts (SIGABRT) on every Redis reconnection — the redis crate is built with async-std-comp while the app runs on Tokio

[philou-078]

Summary

Any event that drops the Redis connection (a Redis restart, a rolling update, a Sentinel failover)
makes the gas station abort the whole process instead of reconnecting.

The panic does not come from a missing reconnection: it comes from inside the reconnection path.
RedisStorage uses redis::aio::ConnectionManager, which is designed to reconnect transparently,
but the redis crate is enabled with the async-std-comp feature while the application runs on
a Tokio runtime. ConnectionManager's retry logic (tokio-retry) is therefore driven from an
async-std thread, where no Tokio reactor exists, and it panics.

Impact

Every Redis connection drop costs a full process restart. In a Kubernetes deployment this is worse
than it sounds:

• all gas station replicas hold connections to the same Redis, so they abort simultaneously —
  running two replicas does not protect the service;
• the container restart is subject to Kubernetes' exponential backoff, so the more often it
  happens, the longer the recovery.

Measured on our staging cluster (Redis 8.8, 3 nodes with Sentinel, single master failover):

Redis write unavailability (the actual failover)	1.4 s
gas station unavailability, replica A	8 s (abort → ready)
gas station unavailability, replica B	19 s (abort → ready; delayed by restart backoff)
gas station unavailability with no replica serving	~9 s

So a 1.4 s Redis failover becomes a ~9 s total outage of the gas station.

Evidence

Panic, captured at the moment of a Sentinel failover:

thread 'async-std/runtime' panicked at /usr/local/cargo/registry/src/index.crates.io-.../tokio-retry-0.3.0/src/future.rs:136:30:
there is no reactor running, must be called from the context of a Tokio 1.x runtime

Relevant backtrace frame:

tokio::runtime::scheduler::Handle::current::panic_cold_display

Note the thread name: async-std/runtime. Process exit code is 134 (SIGABRT).

Shortly before the abort, the connection loss is visible and correctly handled on another code
path, which is what makes the crash look sporadic:

ERROR iota_gas_station::gas_station::gas_station_core: Failed to call expire_coins to the storage: broken pipe

Root cause

Cargo.toml (still the case on main at the time of writing):

redis = { version = "0.24.0", features = [
  "default",
  "async-std-comp",     # <-- async-std support
  "connection-manager",
] }
tokio = { version = "1.46", features = ["full"] }

src/storage/redis/mod.rs:

use redis::aio::ConnectionManager;
// ...
let client = redis::Client::open(redis_url).unwrap();
let mut conn_manager = ConnectionManager::new(client).await.unwrap();

The application is Tokio-based, so the redis crate should be enabled with tokio-comp, not
async-std-comp. With async-std-comp, the reconnection task runs outside the Tokio runtime and
tokio-retry panics on its first attempt.

Suggested fix

Swap the feature:

redis = { version = "0.24.0", features = [
  "default",
  "tokio-comp",
  "connection-manager",
] }

If nothing in the codebase genuinely needs async-std, async-std-comp can simply be removed.

We have not been able to verify the fix ourselves: we consume the published image
iotaledger/gas-station:0.5.2 rather than building from source. The diagnosis, however, is
unambiguous — the panicking thread is named async-std/runtime, and async-std-comp is the only
reason an async-std runtime exists in this binary.

Steps to reproduce

1. Run the gas station against a Redis instance.
2. Restart Redis, or trigger a Sentinel failover, or simply kill the Redis server — anything that
   closes the established connection.
3. The gas station aborts with exit code 134 and the panic above, instead of reconnecting.

Environment

• gas station 0.5.2 (iota-gas-station 0.5.2-8544e55, image iotaledger/gas-station:0.5.2)
• Redis 8.8.0, 3 nodes with Redis Sentinel (single master, 2 replicas)
• Kubernetes (AKS), 2 gas station replicas

Side note

Neither the README nor https://docs.iota.org/operator/gas-station/deployment/ says anything about
Redis high availability, Sentinel/Cluster support, or the expected behaviour when Redis becomes
unavailable — the deployment guide only recommends "a centralized Redis instance". Documenting that
the gas station is expected to survive a Redis restart (once this is fixed) would help operators size
their Redis setup.