Add validation for _id field in activity and food APIs:
- activity: POST, PUT, DELETE now validate _id format
- food: POST, PUT, DELETE now validate _id format
Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request
Previously:
- activity: 500 crash on invalid _id in save/remove
- food: silently replaced invalid _id with new ObjectId (data loss)
Tests added covering all validation cases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add validation for _id field in devicestatus API:
- POST: validates each document's _id before storage
- DELETE: validates _id parameter (allows wildcard '*')
Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request
Previously, invalid _id values were silently stored as strings instead
of ObjectIds, causing inconsistent data and query issues.
Tests added for all validation cases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add validation for _id field in profile API:
- POST: validates each document's _id before storage
- PUT: validates _id format before update
- DELETE: validates _id parameter before removal
Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request
This prevents 500 errors from BSONError when clients send
UUID-style _ids (e.g., NightscoutKit).
Tests added for all validation cases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The API layer was calling purifyObject() on the raw req.body without
handling arrays. When NightscoutKit sends [status], only the outer
array would be purified (no-op), not the individual status objects.
Now normalizes to array and purifies each devicestatus object,
matching the treatments pattern.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NightscoutKit (Loop) sends profiles wrapped in arrays: [profile].
The MongoDB driver migration changed insert() to insertOne(), breaking
array support.
Changes:
- API layer: normalize input to array, purify each item
- Storage layer: use insertMany() instead of insertOne()
- Tests: verify single, array, and empty array handling
This matches the proven pattern from treatments API.
Fixes array handling regression introduced in d46c5b41.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
UUID_HANDLING should ONLY affect UUID values in the _id field.
Previous commit incorrectly added server-side dedup for syncIdentifier
and uuid fields, which was never part of the original behavior.
Changes:
- upsertQueryFor(): Remove syncIdentifier/uuid as dedup keys
- Batch POST: Only fetch existing IDs by identifier, not by
syncIdentifier/uuid
- tests: Update TEST-CACHE-003/004 to document actual behavior
(duplicates occur without ObjectIdCache - this is by design)
- docs: Correct treatments-schema.md (syncIdentifier/uuid preserved,
not copied to identifier)
- docs: Remove external link from entries-schema.md
Loop carbs/doses rely on ObjectIdCache for dedup, not server-side logic.
This matches the original (pre-change) server behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
REQ-SYNC-072 scope correction: normalizeTreatmentId() should ONLY
handle UUID values in the _id field, not copy syncIdentifier or uuid
fields to identifier.
Changes:
- normalizeTreatmentId(): Only extract UUID from _id to identifier
- normalizeEntryId(): Same fix for entries collection
- upsertQueryFor(): Add syncIdentifier and uuid as dedup fallbacks
(fields are preserved, not copied to identifier)
- Batch POST: Fetch _id for docs deduped by syncIdentifier/uuid
Test updates:
- TEST-ID-003, TEST-V1-ID-004: Updated to expect identifier NOT copied
from syncIdentifier (scope fix)
Affected clients:
- Loop overrides (UUID _id → identifier): Still works
- Loop carbs/doses (syncIdentifier): Dedup works, no identifier copy
- xDrip+ (uuid): Dedup works, no identifier copy
- AAPS (identifier): Unchanged
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The feature only handles the specific case where a UUID is sent as
the _id field itself. It does NOT affect:
- AAPS (uses 'identifier' field)
- xDrip+ (uses 'uuid' field)
- Loop carbs/doses (uses 'syncIdentifier' field)
Only affects:
- Loop overrides (_id: syncIdentifier.uuidString)
- Trio CGM entries (_id: UUID)
See docs/10-domain/client-id-handling-deep-dive.md for full analysis.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Avoid implying all AID clients use the same pattern or that any
specific implementation is incorrect. Different clients have
divergent sync patterns - the feature accommodates this variety.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update default from false to true to enable AID client compatibility
out of the box:
- lib/server/env.js: readENVTruthy('UUID_HANDLING', true)
- README.md: Document UUID_HANDLING in Features section
- docs/example-template.env: Update comments, show true as default
Rationale:
- Loop, Trio, AAPS, xDrip+ use UUID sync patterns by default
- Before MongoDB 5.x, UUID _id didn't crash (just didn't CRUD properly)
- ObjectID users completely unaffected (quirk only triggers on UUID)
- Can set UUID_HANDLING=false for strict mode if needed
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- env.js: Add UUID_HANDLING env var (default: false)
- query.js: Add UUID detection in normalizeIdValue()
- When UUID_HANDLING=true and _id is UUID, search by identifier field
- Returns searchByIdentifier flag to redirect query
- treatments.js: Move queryOpts inside query_for() for env access
- entries.js: Same pattern for entries collection
When UUID_HANDLING=true:
- GET /treatments/{uuid} searches by identifier field
- DELETE /treatments/{uuid} deletes by identifier field
- Same behavior for entries collection
When UUID_HANDLING=false (default):
- UUID _id values return empty results (safe, no crash)
- Maintains backwards compatibility
Refs: uuid-feature-flag, uuid-query-impl from uuid-identifier-lookup.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Trio/Loop upload CGM entries with UUID strings as _id field.
This caused MongoDB errors when re-uploading with different UUID
at same timestamp: "immutable field '_id'" error.
Fix:
- Add normalizeEntryId() to extract UUID from _id to identifier field
- Add upsertQueryFor() to strip non-ObjectId _id before $set
- Maintain sysTime+type as primary dedup key for CGM data integrity
- Add identifier to indexed fields
Tests:
- 3 baseline tests document current sysTime+type dedup behavior
- 6 UUID handling tests including the previously-failing scenario
Refs: GAP-SYNC-045, REQ-SYNC-072
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes#8450 - Loop Temporary Override sync breaks due to UUID _id handling
Option G Implementation:
- Extract client sync identity (identifier) from any source:
- Loop overrides: UUID in _id field → moved to identifier
- Loop carbs/doses: syncIdentifier → copied to identifier
- AAPS: identifier already present
- xDrip+: uuid → copied to identifier
- Server generates proper ObjectId for _id field
- Deduplication uses identifier (not _id) as primary key
- No database migration needed - gradual adoption
Changes:
- normalizeTreatmentId(): extracts client identity to identifier field
- upsertQueryFor(): identifier-first lookup, strips UUID _id for upsert
- create()/upsert()/save(): fetch _id from DB after update by identifier
- Added 'identifier' to indexedFields for efficient querying
- Updated UUID treatment test with full workflow coverage
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Change engines.node from '^22.x || ^20.x' to '>=16.x'
- Change engines.npm from '>=10.x' to '>=8.x'
- Update runtime checkNodeVersion to allow Node 16+
- Creates overlap with previous release (^16.x || ^14.x)
This allows users on Node 16/18 to upgrade smoothly while
recommending Node 20 or 22 LTS for best support.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix logic in delay list initialization to correctly handle zero-value settings for authorization failure delays, and update test configurations to leverage this fix.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 605cec52-e52a-4520-8af0-466d9c971344
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: ee25d747-8c3a-444e-8dbf-e0229eee7d31
Replit-Helium-Checkpoint-Created: true
- devicestatus.js: Replace async.eachSeries + insertOne with insertMany
- entries.js: Replace forEach + updateOne with bulkWrite
- treatments.js: Replace async.eachSeries + replaceOne with bulkWrite
(preserves sequential fallback for preBolus treatments)
This improves performance for batch inserts and aligns with MongoDB
best practices per data-shape-requirements.md recommendations.
Update env.js to correctly parse PREDICTIONS_MAX_SIZE, allowing a value of 0 to disable truncation. Modify tests and documentation to reflect this change.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 81a6141f-c27c-4178-8cf3-6474e7b82918
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 92d3ef13-9728-43c6-8e8b-d435184fec07
Replit-Helium-Checkpoint-Created: true
Update rounding logic in `lib/sandbox.js` to incorporate an epsilon value for enhanced precision, addressing potential floating-point errors in insulin calculations.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: cf276ff5-4001-4b44-93fe-7c9786ee4327
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: f10d2278-3392-4fd0-ad73-36beee7aadb9
Replit-Helium-Checkpoint-Created: true
Introduce logic to truncate prediction arrays (IOB, COB, UAM, ZT) in devicestatus documents when they exceed a configured maximum size, preventing potential issues with large document handling in MongoDB. This includes updates to bootevent, devicestatus module, environment configuration, and new tests to verify truncation behavior.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: daa945f9-0872-4250-9868-a1245067293b
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: baff25d9-e915-44d9-87f0-f7583f443a30
Replit-Helium-Checkpoint-Created: true
Fixes test failure in api3.renderer.test.js "SEARCH should accept csv content type"
that was caused by MongoDB driver upgrade from 3.x to 5.x.
Changes:
1. lib/server/entries.js:
- Change from replaceOne() to updateOne() with $set operator
- MongoDB 3.x update() did partial updates, but 5.x replaceOne() does
full document replacement
- Using updateOne with $set preserves the original partial update behavior
- Prevents documents with same {sysTime, type} from replacing each other
2. tests/api3.renderer.test.js:
- Add database cleanup in before() hook to delete all entries
- Ensures test isolation from previous test files
- Previous tests (especially old API v1 tests) were leaving entries in DB
with undefined app/identifier fields that interfered with CSV rendering
The CSV test now passes - it expects exactly 2 documents but was getting
105 entries due to leftover test data from previous test runs.
Fixes two bugs in the entries API that caused test failures:
1. POST endpoints now return JSON arrays consistently
- Created format_post_response() middleware for POST requests
- Replaces format_entries() which is designed for GET with content negotiation
- Previously, POST requests without Accept header defaulted to text/plain handler
- This caused responses to fail or return empty objects instead of JSON arrays
- Now matches behavior of treatments and devicestatus APIs
2. Fixed callback never being called for empty array posts
- Added empty array check in lib/server/entries.js create() function
- Previously, empty array caused forEach loop to never execute
- Completion callback was inside forEach, so never triggered for empty input
- This caused 15 second timeouts on POST requests with empty arrays
All 26 tests in api.shape-handling.test.js now pass.
Files changed:
- lib/api/entries/index.js: Added format_post_response, updated POST routes
- lib/server/entries.js: Added empty array handling in create()
Removes the 15-second debounce from `lib/server/bootevent.js`, allowing `data-received` and `tick` events to trigger immediate data updates by calling `ctx.dataloader.update()` directly.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: ef8efcf8-f9e9-49f9-9441-59adbee38ad3
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 67966157-9e23-4065-b9a1-0ca67482bafc
Replit-Helium-Checkpoint-Created: true
Switching the way units is fetched to this way makes it similar to the other
code already in place that does something similar. This allows the tests to
pass with the existing fixtures.
Considering augmenting additional tests that explicitly tests whether the
relevant branch of code is taken.
Changing default SAGE warn and urgent threshold defaults to be useful for 10 day sensors rather than 7 day sensors from the G5 era. SAGE pill turns yellow at day 9 and red 4 hours before expiration of 10 days.
Once G6 is retired, consider setting WARN to 10 days and URGENT to 10.5 days.
Ben WestandCopilot
copilot-swe-agent[bot]andbewest
Andy Low
bewest
Ingo Reitz
Scott LeibrandandBen West
Jörg SchömerandBen West
Adam RadoczandBen West
dsnallfotandBen West
Ken FoxandGitHub