Skip to content

APPLYF-VS-AKKA-RESULTS.md

Latest commit

 

History

History
282 lines (219 loc) · 7.31 KB

APPLYF-VS-AKKA-RESULTS.md

File metadata and controls

282 lines (219 loc) · 7.31 KB
Raw

SequentiallyCats.applyF vs Akka Benchmark Results

Benchmark Results (Batch Size: 1000)

Implementation Throughput Performance
Cats applyF (F[T]) 2,058.68 ± 112.77 ops/s 🏆 +43.6% vs Akka
Akka (Future) 1,434.05 ± 528.46 ops/s Baseline
Cats apply (Future) 1,213.68 ± 322.90 ops/s -15.4% vs Akka

Key Findings

Cats.applyF is the clear winner!

  • ✅ 43.6% faster than Akka - Stays in F context, no Future conversion
  • ✅ 69.6% faster than Cats.apply - Avoids Dispatcher overhead for Future conversion
  • ✅ Lower variance (±112.77 vs ±528.46) - More predictable performance

Analysis

Why is Cats applyF Faster?

1. Batch Efficiency:

  • Batches of 1000 operations amortize overhead
  • Semaphore-based coordination scales well
  • No actor mailbox indirection

2. Pure F[_] Context:

// applyF: Stays in IO, no conversions
operations.sequence.unsafeRunSync()  // Single conversion at end

// vs Akka: Future for each operation
futures.map(Await.result(_))  // Many Future allocations

// vs apply: IO -> Future per operation  
futures.map(dispatcher.unsafeToFuture(_))  // Dispatcher overhead

3. Resource Management:

  • Pre-allocated semaphores (fixed bucket count)
  • Efficient semantic blocking vs actor mailboxes
  • Better CPU cache locality

Real-World Considerations

1. Batch Size Impact

With BatchSize = 1000 (realistic load):

  • ✅ Cats applyF dominates - +43.6% throughput
  • ✅ Semaphore coordination scales efficiently
  • ✅ Single unsafeRunSync at the end vs many Awaits

With single operations:

  • Overhead more visible but still comparable
  • Choose based on your architecture, not micro-optimizations

2. Concurrency Model

Akka:

// One actor per key (dynamic)
// Mailbox-based serialization
akkaSequentially(userId) {
  // Runs in actor context
  // Future per operation
}

Cats (bucket-based):

// Fixed buckets (CPU * 5)
// Semaphore-based serialization
catsSequentially.applyF(userId) {
  // Stays in F[_] context
  // More efficient composition
}

3. Performance Characteristics

Scenario Winner Reason
Batched operations Cats applyF +43.6% throughput
Pure FP workflow Cats applyF No context switching
Future-based code Akka Native Future support
High key count Cats Fixed memory (buckets)
Low key count Comparable Both efficient
I/O-bound tasks Cats applyF Better async composition

When to Use Each

✅ Use Cats applyF When (RECOMMENDED):

  • 🏆 Best performance - 43.6% faster than Akka
  • ✅ Pure Cats Effect application
  • ✅ Need functional composition
  • ✅ Want type safety and predictability
  • ✅ Batched or high-throughput workloads
  • ✅ I/O-bound operations
  • ✅ Starting a new project

Use Akka Sequentially When:

  • ✅ You already use Akka/Pekko ecosystem
  • ✅ Team familiar with actors
  • ✅ Existing Akka codebase (migration cost)
  • ✅ Simple Future-based code
  • ✅ Battle-tested solution needed

Use Cats apply (Future) When:

  • ⚠️ Migrating from Akka to Cats Effect
  • ⚠️ Need Sequentially trait compatibility
  • ⚠️ Interop with Future-based code
  • ❌ Not recommended for new code (use applyF instead)

Performance Tips

For Akka:

// Tune dispatcher
akka.actor.default-dispatcher {
  throughput = 100
  fork-join-executor {
    parallelism-min = 8
    parallelism-max = 64
  }
}

For Cats:

// Use applyF for best performance
sequentially.applyF(key) {
  IO {
    // Your code here
  }
}

// Avoid this in hot paths
sequentially(key) {
  // Creates IO.delay + Future conversion
}

Benchmark Details

Test Configuration

  • Warmup: 5 iterations, 10s each
  • Measurement: 5 iterations, 10s each
  • Batch Size: 1000 operations per iteration
  • Task: Empty operation {}
  • Key: Random keys for each operation
  • JVM: OpenJDK 21.0.2
  • Threads: 1 benchmark thread

What Was Measured

private val BatchSize = 1000

@Benchmark
def akkaFuture(): Unit = {
  implicit val ec = akkaSystem.dispatcher
  val futures = List.fill(BatchSize)(akkaSequentially(Random.nextInt()) {})
  Await.result(Future.sequence(futures), 10.seconds)
}

@Benchmark
def catsApplyF(): Unit = {
  val operations = List.fill(BatchSize)(
    catsSequentially.applyF(Random.nextInt())(IO.unit)
  )
  operations.sequence.unsafeRunSync()(runtime)
}

@Benchmark
def catsApplyFuture(): Unit = {
  implicit val ec = ExecutionContext.global
  val futures = List.fill(BatchSize)(
    catsSequentially(Random.nextInt()) {}(dispatcher)
  )
  Await.result(Future.sequence(futures), 10.seconds)
}

Key Insight: Batching 1000 operations amortizes Await.result and unsafeRunSync overhead, giving a realistic view of sustained throughput.

Running More Realistic Benchmarks

With I/O Simulation

Create a new benchmark:

@Benchmark
def withIO(): Unit = {
  // Simulate database call
  val io = IO.sleep(1.millis) *> IO.pure(42)
  
  // Measure Akka
  Await.result(akkaSequentially(0) {
    Thread.sleep(1)
    42
  }, 10.seconds)
  
  // vs Cats
  catsSequentially.applyF(0)(io).unsafeRunSync()
}

With Multiple Keys

val keys = Random.shuffle((0 until 1000).toList)

@Benchmark
def multipleKeys(): Unit = {
  val futures = keys.take(100).map { key =>
    sequentially(key) { /* work */ }
  }
  Await.result(Future.sequence(futures), 10.seconds)
}

Recommendations

For Production

  1. Benchmark your actual workload - not no-op operations
  2. Measure with realistic key distribution
  3. Include I/O operations in benchmarks
  4. Test under load with concurrent requests

Quick Decision Guide

Starting a new project?
├─ Yes → Use Cats applyF 🏆 (best performance + FP benefits)
└─ No, existing codebase
   ├─ Using Cats Effect? → Use Cats applyF (43.6% faster than Akka)
   ├─ Using Akka heavily? → Keep Akka (migration cost may not justify)
   └─ Future-based only? → Consider Akka (simpler) or Cats applyF (faster)

Running the Full Comparison

# Comprehensive comparison
./compare-applyF-vs-akka.sh

# Manual with more iterations
sbt "project benchmark" "Jmh/run -i 10 -wi 5 -f 2 SequentiallyCatsVsAkkaBenchmark"

Conclusion

Benchmark Results Summary

Performance Rankings:

  1. 🥇 Cats applyF: 2,058.68 ops/s (+43.6% vs Akka)
  2. 🥈 Akka: 1,434.05 ops/s (baseline)
  3. 🥉 Cats apply: 1,213.68 ops/s (-15.4% vs Akka)

Key Takeaways:

  • ✅ Cats applyF is the performance winner - Significantly faster in batched scenarios
  • ✅ Staying in F[_] context pays off - 69.6% faster than Future conversion
  • ✅ Lower variance - More predictable performance (±112.77 vs ±528.46)
  • ✅ Scales well - Semaphore-based coordination handles batches efficiently

Recommendations

For New Projects:

  • 🎯 Use Cats applyF - Best performance + functional composition

For Existing Akka Projects:

  • Keep Akka if migration cost is high
  • Consider Cats applyF for new features

For Migration:

  • Start with apply for compatibility
  • Migrate to applyF for performance gains

The Real Winner: Cats Effect + functional programming + measured performance! 🏆