WIP: add caching using KV

[karthik2804]

without caching in kv

bombardier -c 1 -n 1000 http://localhost:3000/cloud/data-redis
Bombarding http://localhost:3000/cloud/data-redis with 1000 request(s) using 1 connection(s)
 1000 / 1000 [=========================================================] 100.00% 68/s 14s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec        69.01      24.34     104.92
  Latency       14.48ms     1.66ms    25.97ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     2.20MB/s

with caching

bombardier -c 1 -n 1000 http://localhost:3000/cloud/data-postgres
Bombarding http://localhost:3000/cloud/data-postgres with 1000 request(s) using 1 connection(s)
 1000 / 1000 [=========================================================] 100.00% 998/s 1s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      1168.36     367.53    1642.22
  Latency        0.86ms     5.59ms   177.52ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    34.10MB/s

[itowlson]

I found this a bit hard to follow, but I think you are in a bind - there is no real loader layer where you can inject caching behaviour, so you need to inject caching into existing large functions, and it ends up bringing lots of tactical considerations around things like time parsing right up into the cache and application layers.

Ideally it would be good to start unpicking bits of the structure, and adding traits and helper methods (e.g. if cached.has_expired()) rather than operating directly on strings and serialisation types. But that would likely require a more significant refactor than you can fit in at this time. Something to think about for the longer term maybe...

[karthik2804]

@itowlson, I think I have addressed all the comments and I think this is ready for another round of review.

[mikkelhegn]

Has this been tested in the cloud? (Not sure if we can), but I'm curious where the performance benefits come from? Moving from generating things as the request comes in, to read something pre-generated from the KV Store? Is that main thing?

[karthik2804]

We can't test it on the cloud yet. You are correct in the source of performance improvements, by keeping a copy of rendered stuff and using it until the cache is invalidated, the entire render cycle is skipped.

[itowlson]

LGTM!