The API layer was calling purifyObject() on the raw req.body without
handling arrays. When NightscoutKit sends [status], only the outer
array would be purified (no-op), not the individual status objects.
Now normalizes to array and purifies each devicestatus object,
matching the treatments pattern.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NightscoutKit (Loop) sends profiles wrapped in arrays: [profile].
The MongoDB driver migration changed insert() to insertOne(), breaking
array support.
Changes:
- API layer: normalize input to array, purify each item
- Storage layer: use insertMany() instead of insertOne()
- Tests: verify single, array, and empty array handling
This matches the proven pattern from treatments API.
Fixes array handling regression introduced in d46c5b41.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
UUID_HANDLING should ONLY affect UUID values in the _id field.
Previous commit incorrectly added server-side dedup for syncIdentifier
and uuid fields, which was never part of the original behavior.
Changes:
- upsertQueryFor(): Remove syncIdentifier/uuid as dedup keys
- Batch POST: Only fetch existing IDs by identifier, not by
syncIdentifier/uuid
- tests: Update TEST-CACHE-003/004 to document actual behavior
(duplicates occur without ObjectIdCache - this is by design)
- docs: Correct treatments-schema.md (syncIdentifier/uuid preserved,
not copied to identifier)
- docs: Remove external link from entries-schema.md
Loop carbs/doses rely on ObjectIdCache for dedup, not server-side logic.
This matches the original (pre-change) server behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
syncIdentifier and uuid fields are used for dedup, not copied to
identifier. Only UUID values in _id field are extracted to identifier.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
REQ-SYNC-072 scope correction: normalizeTreatmentId() should ONLY
handle UUID values in the _id field, not copy syncIdentifier or uuid
fields to identifier.
Changes:
- normalizeTreatmentId(): Only extract UUID from _id to identifier
- normalizeEntryId(): Same fix for entries collection
- upsertQueryFor(): Add syncIdentifier and uuid as dedup fallbacks
(fields are preserved, not copied to identifier)
- Batch POST: Fetch _id for docs deduped by syncIdentifier/uuid
Test updates:
- TEST-ID-003, TEST-V1-ID-004: Updated to expect identifier NOT copied
from syncIdentifier (scope fix)
Affected clients:
- Loop overrides (UUID _id → identifier): Still works
- Loop carbs/doses (syncIdentifier): Dedup works, no identifier copy
- xDrip+ (uuid): Dedup works, no identifier copy
- AAPS (identifier): Unchanged
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
UUID_HANDLING default changed to true in 15.0.7. Test now explicitly
sets UUID_HANDLING=false rather than deleting the env var.
742 passing, 1 pending.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Write normalization (syncIdentifier/uuid → identifier) is ALWAYS on
- UUID_HANDLING flag only controls READ path (GET/DELETE by UUID)
- Fix default: UUID_HANDLING=true (not false)
- Remove incorrect xDrip+ mention from entries (doesn't use UUID _id)
- Clarify that flag only affects API calls with UUID as _id parameter
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The feature only handles the specific case where a UUID is sent as
the _id field itself. It does NOT affect:
- AAPS (uses 'identifier' field)
- xDrip+ (uses 'uuid' field)
- Loop carbs/doses (uses 'syncIdentifier' field)
Only affects:
- Loop overrides (_id: syncIdentifier.uuidString)
- Trio CGM entries (_id: UUID)
See docs/10-domain/client-id-handling-deep-dive.md for full analysis.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Avoid implying all AID clients use the same pattern or that any
specific implementation is incorrect. Different clients have
divergent sync patterns - the feature accommodates this variety.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update default from false to true to enable AID client compatibility
out of the box:
- lib/server/env.js: readENVTruthy('UUID_HANDLING', true)
- README.md: Document UUID_HANDLING in Features section
- docs/example-template.env: Update comments, show true as default
Rationale:
- Loop, Trio, AAPS, xDrip+ use UUID sync patterns by default
- Before MongoDB 5.x, UUID _id didn't crash (just didn't CRUD properly)
- ObjectID users completely unaffected (quirk only triggers on UUID)
- Can set UUID_HANDLING=false for strict mode if needed
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
API returns _id as plain string "507f1f77bcf86cd799439011", not
MongoDB Extended JSON format {"$oid": "..."}.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document MongoDB pool settings and AUTH_FAIL_DELAY for test tuning.
Useful for CI or resource-constrained environments.
Refs: DOC-ENV-002
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add 6 tests verifying UUID_HANDLING env var behavior:
- UUID-OFF-001: GET by UUID returns empty (no crash)
- UUID-OFF-002: DELETE by UUID deletes nothing (no crash)
- UUID-ON-001: GET by UUID finds treatment via identifier
- UUID-ON-002: DELETE by UUID removes treatment via identifier
- UUID-ON-003: ObjectId still works normally
- UUID-ON-004: Non-matching UUID returns empty
Tests use clearModuleCache() to reload env.js with different flag values.
Refs: uuid-test-flag-off, uuid-test-flag-on, REQ-SYNC-072
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace Date.now() with the pre-captured 'now' variable in the
'set a pill to BWP with infos' test. This prevents timing drift
between when test data timestamps are set and when the sandbox
is initialized, eliminating flaky failures in CI environments.
Refs: BWP-TIME-001, GAP-TEST-001
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- env.js: Add UUID_HANDLING env var (default: false)
- query.js: Add UUID detection in normalizeIdValue()
- When UUID_HANDLING=true and _id is UUID, search by identifier field
- Returns searchByIdentifier flag to redirect query
- treatments.js: Move queryOpts inside query_for() for env access
- entries.js: Same pattern for entries collection
When UUID_HANDLING=true:
- GET /treatments/{uuid} searches by identifier field
- DELETE /treatments/{uuid} deletes by identifier field
- Same behavior for entries collection
When UUID_HANDLING=false (default):
- UUID _id values return empty results (safe, no crash)
- Maintains backwards compatibility
Refs: uuid-feature-flag, uuid-query-impl from uuid-identifier-lookup.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This test passes locally in ~50ms but occasionally times out at 30s
in constrained GitHub runners. Adding retries(2) allows it to recover
from transient CI resource contention.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The 'WebSocket dbAdd Array Handling Investigation' block was R&D to
understand insertOne behavior with arrays. The investigation concluded:
- MongoDB's insertOne([a,b]) creates single doc (not multiple)
- Fix: sequential processing via processNextItem() in websocket.js
Production tests now cover this behavior:
- 'dbAdd with array input for treatments - current behavior test'
- 'dbAdd with array input for devicestatus - current behavior test'
- 'dbAdd with array input for entries - current behavior test'
Removes 2 flaky investigative tests, keeps 729 production tests passing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Change from warning to process.exit(1) to prevent any possibility of
running destructive test operations against a production database.
Tests now fail immediately if NODE_ENV !== 'test', with clear instructions
on how to fix.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SAFETY-001: Fix tests/ci.test.env to use NODE_ENV=test instead of production
SAFETY-002: Add NODE_ENV check to tests/hooks.js with warning
SAFETY-003: Create tests/fixtures/test-guard.js with guarded deleteMany/drop helpers
This prevents deleteMany({}) from accidentally running against production
databases if test environment is misconfigured.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Trio/Loop upload CGM entries with UUID strings as _id field.
This caused MongoDB errors when re-uploading with different UUID
at same timestamp: "immutable field '_id'" error.
Fix:
- Add normalizeEntryId() to extract UUID from _id to identifier field
- Add upsertQueryFor() to strip non-ObjectId _id before $set
- Maintain sysTime+type as primary dedup key for CGM data integrity
- Add identifier to indexed fields
Tests:
- 3 baseline tests document current sysTime+type dedup behavior
- 6 UUID handling tests including the previously-failing scenario
Refs: GAP-SYNC-045, REQ-SYNC-072
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add 3 new tests for explicit identifier field handling:
- supports identifier field for AAPS-style treatments
- deduplicates by identifier on re-upload
- supports batch upload with identifiers
These complement existing UUID _id tests (Loop pattern) to cover
both AID client sync patterns.
Refs: REQ-SYNC-072, GAP-TREAT-012
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TEST-CACHE-001: POST carb → cache syncIdentifier → PUT with id
TEST-CACHE-002: POST dose → cache syncIdentifier → DELETE with id
TEST-CACHE-003: Cache miss (24hr expiry) → POST same syncIdentifier
TEST-CACHE-004: App restart (cache empty) → POST existing syncIdentifier
TEST-CACHE-005: Batch POST → verify response order → cache mapping
7 new tests validating Loop's ObjectIdCache behavior:
- syncIdentifier → ObjectId mapping
- Response order for batch operations
- Deduplication by syncIdentifier
- Hex string syncIdentifier handling
All 7 tests passing.
Refs: Loop ObjectIdCache.swift analysis
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TEST-GAP-001: Loop override POST with UUID _id
TEST-GAP-002: Loop override DELETE by UUID
TEST-GAP-003: Loop override UPDATE by UUID
TEST-GAP-004: Loop override re-POST (upsert)
12 new tests validating REQ-SYNC-072 behavior:
- UUID _id promoted to identifier field
- Server generates valid ObjectId for _id
- Updates/deletes work via identifier lookup
- Duplicate detection via identifier
- Batch and edge case handling
New fixtures:
- loop-override.js: Real Loop override payload patterns
All 12 tests passing.
Refs: GAP-TREAT-012, REQ-SYNC-072
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes#8450 - Loop Temporary Override sync breaks due to UUID _id handling
Option G Implementation:
- Extract client sync identity (identifier) from any source:
- Loop overrides: UUID in _id field → moved to identifier
- Loop carbs/doses: syncIdentifier → copied to identifier
- AAPS: identifier already present
- xDrip+: uuid → copied to identifier
- Server generates proper ObjectId for _id field
- Deduplication uses identifier (not _id) as primary key
- No database migration needed - gradual adoption
Changes:
- normalizeTreatmentId(): extracts client identity to identifier field
- upsertQueryFor(): identifier-first lookup, strips UUID _id for upsert
- create()/upsert()/save(): fetch _id from DB after update by identifier
- Added 'identifier' to indexedFields for efficient querying
- Updated UUID treatment test with full workflow coverage
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ben WestandCopilot
copilot-swe-agent[bot]andbewest
Andy LowandGitHub