Commit dc8505c
authored
perf(macros): reduce redundant implicit searches in ctx.run() pipeline (#720)
Previously, ProtoQuill's `ctx.run()` macro expansion performed redundant
`Expr.summon` (implicit search) calls for GenericEncoder/GenericDecoder
resolution. For a 10-field case class, the pipeline issued ~30 implicit
searches; for 20 fields, ~60. Each search is expensive in Scala 3's macro
system, making this the primary compilation bottleneck (issue #619 documents
a single repository file taking 2m49s to compile).
This commit takes a two-pronged approach: optimizing the existing macro
pipeline, and introducing a new NamedTuple-based query API that sidesteps
the most expensive macro work entirely.
## Macro pipeline optimizations
Six coordinated changes reduce redundant implicit searches in the existing
`query[T]` / `ctx.run()` path:
1. **isKnownLeafType fast-path** (TypeExtensions.scala): Adds a cheap O(1)
`TypeRepr.=:=` check for 10 primitive types (String, Int, Long, Short,
Byte, Float, Double, Boolean, BigDecimal, Array[Byte]) that are
guaranteed to have encoders/decoders in every context. `java.util.Date`
is intentionally excluded because it has no encoder in the Cassandra
context. This bypasses expensive implicit search entirely for these
common types.
2. **GenericDecoder fast-path** (GenericDecoder.scala): Adds new first cases
in the `flatten` and `values` pattern matches that use `isKnownLeafType`
to skip the `Expr.summon` guard. Known scalar fields no longer trigger
implicit search just to determine leaf-vs-branch status.
3. **Decoder expression cache** (GenericDecoder.scala): Introduces a
`mutable.Map[String, Option[Expr[_]]]` cache threaded through the entire
decode pipeline (flatten, values, decode, decodeOptional, Summon.decoder,
Summon.nullChecker). The cache is created once at the `GenericDecoder.summon`
entry point. For 20 fields with 5 unique types, this reduces implicit
searches from ~60 to ~7 (~88% reduction).
4. **ElaborateStructure leaf cache** (ElaborateStructure.scala): Adds a
`mutable.Map[String, Boolean]` cache to `base()`, threaded through
`flatten()` and `collectFields()`, with isKnownLeafType as an initial
fast-path. Prevents repeated encoder/decoder summoning when determining
leaf-vs-branch status for the same type appearing in multiple fields.
5. **QuatMaking cache fix** (QuatMaking.scala): The existing caches
(`encodeableCache` and `quatCache`) were defined but completely bypassed—
`lookupIsEncodeable` and `lookupCache` called `computeEncodeable()` /
`computeQuat()` directly instead of using `getOrElseUpdate`. This commit
fixes both to actually use their caches. Also adds the isKnownLeafType
fast-path to `existsEncoderFor` to avoid 2 Expr.summon calls per
known primitive type. The cache comment now accurately describes the
object-level `getOrElseUpdate` mechanism (previously the comment
incorrectly referenced non-existent instance-level methods).
6. **Pre-materialized Option decoders** (Decoders.scala, Encoders.scala in
jdbc, cassandra, mirror, doobie contexts): Adds explicit `implicit val`
definitions for `Option[T]` encoders/decoders of common types. Previously,
`Option[String]` etc. required the compiler to derive them through
`optionDecoder[String]` via implicit search chains. Pre-materializing them
short-circuits this search.
## NamedTuple-based record API (io.getquill.record)
As proposed in #643, this introduces a `Selectable`-based `Record[C, W[_]]`
that uses `NamedTuple.Map[NamedTuple.From[C], W]` for type-level schema
derivation instead of macro AST inspection. The higher-kinded `W[_]`
parameter wraps each field type so the compiler sees the correct runtime
representation — for query building, `W = Col` where `type Col[A] = FieldExpr`,
so `Record[Person, Col]` has `Fields = (name: FieldExpr, age: FieldExpr)`,
matching what `selectDynamic` actually returns at runtime:
- `Record[C, W[_]]` extends `Selectable` with
`type Fields = NamedTuple.Map[NamedTuple.From[C], W]`, so field access
(e.g. `record.name`) is resolved entirely through the type system via
`selectDynamic` — no macro reflection needed, and no ClassCastException
from mismatched compile-time vs runtime types.
- `TypedEntityQuery[T]` wraps the query and produces the same AST nodes
(`Entity`, `Filter`, `Map`, `SortBy`, etc.) as the macro-based `query[T]`.
Note: `map` currently returns `TypedEntityQuery[T]` (same entity type)
rather than supporting type-changing projections — this is a known
limitation documented in the code, with the full fix planned as part of
a `Queryable[Q, R]` typeclass rearchitecture.
- `SchemaDeriving` extracts field names and types at the type level using
`NamedTuple.Names` and `NamedTuple.From` instead of recursive macro
`'[field *: fields]` pattern matching.
- `RecordCodec` uses `summonInline` to materialize all encoders/decoders in
a single traversal, replacing the per-field `Expr.summon` calls.
- `TypedQuerySpec` covers the API with 8 AST-level unit tests: entity
creation, filter/map/sortBy/take/drop AST construction, toQuoted
bridging, Record field access, and chained operation composition.
The new API is available as `typedQuery[T]` and bridges seamlessly into the
existing `ctx.run()` infrastructure via an implicit conversion to
`Quoted[EntityQuery[T]]`. The existing `query[T]` API is completely unchanged.
## Other changes
- **Scala 3.8.1 upgrade** with sbt 1.12.4, bringing compiler improvements
including given-loop prevention, parallelized JVM backend, pipelined builds,
and type-size normalization
- **JVM tuning** in CI: 8GB heap with ParallelGC (was 6GB with G1GC)
- **Java target fix**: Changed `-release` and `-target` from 21 to 17 to
match the JDK version used in CI (temurin 17)
- **Parser fix** for Scala 3.8 augmentString desugaring of toInt/toLong
- **Doobie upgrade**: RC5 → RC12, resolving incompatibility with newer
doobie releases
- **HikariCP 6.3.3**: Aligns Scala 3 version with the Scala 2.13 artifact,
resolving the version mismatch between the two
- **Other dependency updates**: ZIO 2.1.24, pprint 0.9.6, logback 1.5.32,
scalatest 3.2.19, scala-logging 3.9.6, and other test dependencies
- **@implicitNotFound annotations** on mappedEncoder/mappedDecoder for
clearer compile-time error messages when MappedEncoding instances are
missing
- **-Vprofile diagnostic flag** gated behind `-Dprofile=true` for
before/after measurement
All 265 tests in quill-sql (including new TypedQuerySpec) and 668 tests in
quill-sql-tests pass. No public API changes to query[T], ctx.run(), or
encoder/decoder interfaces.
Inspired-by: #643
Resolves: #619
Resolves: #650
Resolves: #621
Refs: #195
Authored-By: Colin K. Williams / li-nk.social <colin@li-nk.org>1 parent 4d953d7 commit dc8505c
23 files changed
Lines changed: 761 additions & 132 deletions
File tree
- .github/workflows
- project
- quill-cassandra/src/main/scala/io/getquill/context/cassandra/encoding
- quill-doobie/src/main/scala/io/getquill/doobie
- quill-jdbc/src/main/scala/io/getquill/context/jdbc
- quill-sql/src
- main/scala/io/getquill
- context/mirror
- generic
- metaprog
- parser
- quat
- record
- test/scala/io/getquill/record
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
16 | | - | |
| 15 | + | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| |||
86 | 86 | | |
87 | 87 | | |
88 | 88 | | |
89 | | - | |
| 89 | + | |
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
| |||
116 | 116 | | |
117 | 117 | | |
118 | 118 | | |
119 | | - | |
120 | | - | |
| 119 | + | |
| 120 | + | |
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
| |||
127 | 127 | | |
128 | 128 | | |
129 | 129 | | |
130 | | - | |
| 130 | + | |
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
134 | | - | |
| 134 | + | |
| 135 | + | |
135 | 136 | | |
136 | 137 | | |
137 | 138 | | |
| |||
153 | 154 | | |
154 | 155 | | |
155 | 156 | | |
| 157 | + | |
156 | 158 | | |
157 | 159 | | |
158 | 160 | | |
159 | 161 | | |
160 | 162 | | |
161 | 163 | | |
162 | | - | |
163 | | - | |
| 164 | + | |
| 165 | + | |
164 | 166 | | |
165 | 167 | | |
166 | 168 | | |
| |||
171 | 173 | | |
172 | 174 | | |
173 | 175 | | |
174 | | - | |
| 176 | + | |
175 | 177 | | |
176 | 178 | | |
177 | | - | |
| 179 | + | |
178 | 180 | | |
179 | 181 | | |
180 | 182 | | |
181 | | - | |
| 183 | + | |
182 | 184 | | |
183 | 185 | | |
184 | 186 | | |
| |||
202 | 204 | | |
203 | 205 | | |
204 | 206 | | |
205 | | - | |
206 | | - | |
| 207 | + | |
| 208 | + | |
207 | 209 | | |
208 | 210 | | |
209 | 211 | | |
| |||
264 | 266 | | |
265 | 267 | | |
266 | 268 | | |
267 | | - | |
| 269 | + | |
268 | 270 | | |
269 | | - | |
270 | | - | |
| 271 | + | |
| 272 | + | |
271 | 273 | | |
272 | | - | |
273 | | - | |
| 274 | + | |
| 275 | + | |
274 | 276 | | |
275 | 277 | | |
276 | 278 | | |
| |||
290 | 292 | | |
291 | 293 | | |
292 | 294 | | |
293 | | - | |
| 295 | + | |
294 | 296 | | |
295 | 297 | | |
296 | 298 | | |
| |||
299 | 301 | | |
300 | 302 | | |
301 | 303 | | |
| 304 | + | |
302 | 305 | | |
303 | 306 | | |
304 | 307 | | |
305 | 308 | | |
306 | | - | |
| 309 | + | |
| 310 | + | |
307 | 311 | | |
308 | | - | |
| 312 | + | |
| 313 | + | |
309 | 314 | | |
310 | 315 | | |
311 | 316 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
Lines changed: 16 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
64 | 80 | | |
Lines changed: 16 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
82 | 98 | | |
Lines changed: 19 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
| 94 | + | |
94 | 95 | | |
95 | 96 | | |
96 | | - | |
| 97 | + | |
97 | 98 | | |
98 | 99 | | |
99 | 100 | | |
| |||
108 | 109 | | |
109 | 110 | | |
110 | 111 | | |
| 112 | + | |
111 | 113 | | |
112 | 114 | | |
113 | | - | |
| 115 | + | |
114 | 116 | | |
115 | 117 | | |
116 | 118 | | |
| |||
126 | 128 | | |
127 | 129 | | |
128 | 130 | | |
129 | | - | |
130 | | - | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
131 | 134 | | |
132 | 135 | | |
133 | 136 | | |
134 | | - | |
| 137 | + | |
| 138 | + | |
135 | 139 | | |
136 | 140 | | |
137 | 141 | | |
| |||
175 | 179 | | |
176 | 180 | | |
177 | 181 | | |
| 182 | + | |
178 | 183 | | |
179 | 184 | | |
180 | | - | |
| 185 | + | |
181 | 186 | | |
182 | 187 | | |
183 | 188 | | |
| |||
219 | 224 | | |
220 | 225 | | |
221 | 226 | | |
| 227 | + | |
222 | 228 | | |
223 | 229 | | |
224 | 230 | | |
225 | 231 | | |
226 | | - | |
| 232 | + | |
227 | 233 | | |
228 | 234 | | |
229 | 235 | | |
| |||
234 | 240 | | |
235 | 241 | | |
236 | 242 | | |
237 | | - | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
238 | 249 | | |
239 | 250 | | |
240 | 251 | | |
| |||
Lines changed: 15 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
77 | 92 | | |
78 | 93 | | |
79 | 94 | | |
| |||
Lines changed: 15 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
85 | 100 | | |
86 | 101 | | |
87 | 102 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
48 | 59 | | |
49 | 60 | | |
50 | 61 | | |
| |||
Lines changed: 15 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
58 | 73 | | |
0 commit comments