Repository navigation
fix(objectql): bucket a declared Field.date by its calendar day under every timezone (#24618) - #24642
Conversation
… every timezone The in-memory rows path handed a date value to bucketDateKey with the reference zone, which read '2026-10-01' as UTC midnight and labelled it 2026-09-30 in every zone west of UTC, so it disagreed with the pushdown path. A groupBy field the object declares a date now goes through the date storage form and is labelled with no zone; datetime keeps the zone. Claude-Session: https://claude.ai/code/session_01AHeUBbe6S3JRawcpbLejXD Co-authored-by: Claude <noreply@anthropic.com>
…rd the fixture note Claude-Session: https://claude.ai/code/session_01AHeUBbe6S3JRawcpbLejXD Co-authored-by: Claude <noreply@anthropic.com>
…ot a link a published .d.ts cannot resolve Claude-Session: https://claude.ai/code/session_01AHeUBbe6S3JRawcpbLejXD Co-authored-by: Claude <noreply@anthropic.com>
…r-day pins Claude-Session: https://claude.ai/code/session_01AHeUBbe6S3JRawcpbLejXD Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 2 package(s): 1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 1 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 35 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin b4839773026eac47a3cc4eaf87b88ad1d6434838 && git checkout b4839773026eac47a3cc4eaf87b88ad1d6434838
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin b61353ffa04232421d8536d321441095511a62d9 bf7b57a5de104aa8640a3e76e867448c215a03a7 && git checkout -B drift-repro b61353ffa04232421d8536d321441095511a62d9 && git merge --no-ff bf7b57a5de104aa8640a3e76e867448c215a03a7
node scripts/docs-audit/affected-docs.mjs --json b61353ffa04232421d8536d321441095511a62d9
|
席位核对 · ACCEPT ·
|
Fixes #24618
Clause-②: no
What was wrong
A non-UTC
timezoneforcesengine.aggregateonto the in-memory rows path (ADR-0053 Phase 2). On that pathprojectGroupValuepassed every date-bucketed value tobucketDateKeytogether with the zone. It had nodate/datetimebranch.bucketDateKeytakes an INSTANT, so it read'2026-10-01'as2026-10-01T00:00Zand placed it on the zone's calendar. In every zone west of UTC, aField.datetherefore landed one day early. The SQL pushdown path, which runs only at UTC, labelled the stored day. Measured at bucketDateKey on the base973917c57:'2026-10-01'under America/New_York gives2026-09-30 / 2026-W40 / 2026-09 / 2026-Q3 / 2026. America/Los_Angeles gives the same. UTC and Asia/Tokyo give2026-10-01 / 2026-W40 / 2026-10 / 2026-Q4 / 2026. ADateat that day's UTC midnight shifts the same way.The fix
packages/objectql/src/in-memory-aggregation.ts:applyInMemoryAggregationalready receives the object's declared field map, becauseengine.aggregatepassesdeclaredFields. Two new internal helpers (neither is exported):calendarDayGroupFieldsreads which date-bucketedgroupByfields the object declares a calendar day. It reads them once per call, from the declaration, throughdeclaredFieldClasses(classdate, that is,CALENDAR_DATE_TYPES).bucketCalendarDayputs such a value throughtemporalStorageForm(value, 'date'), the one storage rule. It then labels the value withbucketDateKeyand no zone.datetime;fieldsargument, an undeclared field, or aformula.packages/core/src/utils/datetime.ts: comment only.bucketDateKey's docblock now says thatvalueis an instant, and that a caller bucketing adatepasses no zone..d.tsstill readsbucketDateKey(value, granularity, timezone?),bucketDateValue(value, granularity, timezone?)andapplyInMemoryAggregation(rows, ast, timezone?, fields?, reportWithheld?). Neither helper appears in any.d.tsas a declaration.Where it lands: the field type was already available inside
packages/objectql, sopackages/services/service-analyticsneeds no change. The cube path reaches this function throughObjectQLStrategy, thenctx.executeAggregate, thenengine.aggregate, thenapplyInMemoryAggregation(raw, ast, tz, declaredFields).dataset-executor.tspasses its timezone into the sameengine.aggregate.Fixture triage
packages/objectql/src/engine-aggregate-timezone.test.tsdeclaredclosed_atadate, but its rows hold instants such as2024-03-01T03:00:00.000Z, and its case pins the New York shift.datetime, the type of what the rows store. The file keeps pinning the routing it is about, and the new file pins thedatereading.packages/qa/dogfood/test/analytics-timezone.dogfood.test.tsbucketscreated_at. The registry injects that field as adatetime, so it is unaffected.Tests
New file
packages/objectql/src/in-memory-aggregation-date-calendar-day.test.ts, 30 cases. All expected labels are written as literals.(a) A declared
dateequals the stored days under America/New_York, America/Los_Angeles, UTC and Asia/Tokyo, at day, week, month, quarter and year. The fixture covers these edges:Also in (a):
{2026-10: 6, 2026-11: 4}and quarter{2026-Q4: 10}under New York;Datevalue at UTC midnight buckets by its UTC day;aliasdoes not change which field's declaration is read.(b) Control: a declared
datetimeunder New York still lands on the New York day.2026-10-01T02:00Zgives2026-09-30and quarter2026-Q3. Under UTC the same instants land on the UTC day.(c)
engine.aggregate: the pushdown route and the rows route give the same labels for adate. The pushdown route runs at UTC; a driver double answers the literal labels and counts its native calls. The rows route runs under each non-UTC zone.Ablation. The fix was committed first (
3e648b2eb).node scripts/ablation-replace.mjsthen replacedcalendarDayFields.has(field)withfalse: anchor 1 to 0, blob2f518a44toca64430f. After restore, the blob equals HEAD's andgit diff HEADis empty.engine-aggregate-timezone.test.ts. The 18 are the 13 New York / Los Angeles cases of (a) and the 5 of (c). For the door case, the received month under the mutation was{2026-09: 1, 2026-10: 6, 2026-11: 3}../in-memory-aggregation.jsrelatively, so the mutation ran againstsrcand needed no build leg.Real-SQL parity. This was a one-off measurement and is not committed.
@objectstack/driver-sql(better-sqlite3:memory:) built in this worktree, with@objectstack/objectqlbuilt atd789f7e23. Same 12 rows,due_datedeclareddate.driver.aggregate(the real SQL) againstapplyInMemoryAggregation(find(), ast, tz, fields), for New York, Los Angeles and Tokyo at all five granularities: 15/15 agree.engine.aggregateover the sameSqlDriver, UTC (real pushdown) against each non-UTC zone (rows route): 15/15 agree.find()presented thedatecolumn as astring. driver-memory'sfind()returned both a string write and aDatewrite as a'YYYY-MM-DD'string. It advertises no native granularity, so itsdatebuckets always take this path.Runs:
pnpm --filter @objectstack/objectql exec vitest run --project local --maxWorkers=2: 448 files / 8444 tests passed atd789f7e23.srcis unchanged since then; only the new test file's typing changed. Re-run atbf7b57a5d: the new file plus the routing file, 2 files / 32 tests passed.pnpm --filter @objectstack/objectql typecheck(tsc pluscheck:test-typecheck, whose program lists both test files) exited 0 atbf7b57a5d.pnpm --filter @objectstack/core typecheckand its local tests (106 files / 2580 tests) passed atd789f7e23.bf7b57a5d: 68 derived commands (node scripts/pm/dispatch-gates.mjs --commands), 67 exit 0.node scripts/check-engine-split-ratio.mjs --days 90exited 2: it refuses on a shallow clone, and the deepening fetch was cut off by the proxy (fatal: early EOF), so it is NOT MEASURED.--ranreconciliation: 68 derived, 68 run, 0 unrun. Not run locally, left to CI: therepovitest project (test:repo), the workspace type-check lanes, and the families the derivation names as outside its list.Not in this PR: the parity gate's non-UTC probe
packages/verify/src/date-bucket-parity.tsbelongs todomain:cliand was not edited. It callsapplyInMemoryAggregation(rows, ast)with no zone and no field map, so it cannot see this class. To see it, theoncolumn (Field.date) needs a second comparison: the same pushed-down answer againstapplyInMemoryAggregation(rows, ast, 'America/New_York', FIELDS), plus a positive-offset zone. Adatepushdown answer has no zone. The probe's existingonvalues2024-07-01and2025-01-01already cross a quarter edge and a year edge under New York. This half of the card's "expected fix shape" goes to that lane through the seat.Acceptance notes
packages/drivers/driver-memory/src/memory-analytics.ts,MemoryAnalyticsService.aggregateWithTimeBuckets, callsbucketDateKey(value, granularity, timezone)with nodate/datetimebranch. That is the same class of defect. No stock composition registersMemoryAnalyticsService: it is exported, andservice-analyticsnames it only as a fallback a host supplies. I measured no reach, so nothing is filed. Owner: none.engine.aggregatestill forces the rows route for any non-UTC zone, even when every date-bucketed item is a declareddate, whose pushdown answer does not depend on the zone. This is a performance note, not a defect, and nothing pulls on it.formulawhose result is a date stays on the zone path, because its declaration names no stored value class.Generated by Claude Code