- Change 'FAILED' to 'ACTIVATED' - this is protective, not a failure
- Explain that tests WILL DELETE data in the database
- Explain the purpose: preventing accidental production data loss
- List all override options clearly
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ctx.entries module isn't always available depending on boot context.
Access the entries collection directly via ctx.store.db.collection().
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds multi-layer protection against running destructive tests on production:
1. Pre-flight check (hooks.js): Verifies NODE_ENV=test before any DB connection
2. Database name check: Requires 'test' substring in database name
3. Entry count threshold: Refuses if database has >100 entries (configurable)
Environment Variables:
- TEST_SAFETY_MAX_ENTRIES: Max entries before refusing (default: 100)
- TEST_SAFETY_REQUIRE_TEST_DB: Require 'test' in DB name (default: true)
- TEST_SAFETY_SKIP: Emergency bypass for all checks (default: false)
Files:
- tests/lib/production-safety.js: Core safety check module
- tests/00_production-safety.test.js: Runs first to gate test suite
- tests/production-safety.test.js: Unit tests for safety module
- tests/hooks.js: Updated to use new module
This addresses concerns about users with 'test' in production DB names
by adding the entry count threshold as a secondary safety measure.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wip/test improvements - extends same fixes for data shape across all remaining api surface areas and includes test coverage across the test matrix spectrum.
- Add test for single entry returns array with one item
- Add test for empty array returns empty result
- Validates response format consistency for entries API
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add test for single activity returns array with one item
- Add test for activity array returns array
- Add test for empty array returns empty array
- Rename test file to follow *.test.js convention
Validates array normalization behavior for activity API.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add tests for POST with array of foods
- Add tests for PUT with array of foods
- Add test for empty array returning empty array
- Fix PUT endpoint to normalize array input like POST
- Fix food.save() storage to handle arrays with bulkWrite
- Rename test file to follow *.test.js convention
Validates fix from ef7bff3d for complete array handling.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace sequential replaceOne calls with bulkWrite for batch upserts.
This improves performance when inserting multiple documents at once.
- activity.js: Use bulkWrite with replaceOne ops instead of forEach loop
- food.js: Same optimization for consistency
Both maintain exact same upsert behavior (query by _id+created_at or doc).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add array normalization to food API POST endpoint:
- API layer: normalize single object to array (like activity/profile)
- Storage layer: use replaceOne loop with upsert (same as activity pattern)
- Storage layer: accept both single object and array for backward compat
Previously POST /api/food/ with array input would crash:
insertOne([{...}]) → MongoDB error
Now supports both single object and array input consistently.
Response format is now array (matching treatments pattern).
Fixes#8447
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add validation for _id field in activity and food APIs:
- activity: POST, PUT, DELETE now validate _id format
- food: POST, PUT, DELETE now validate _id format
Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request
Previously:
- activity: 500 crash on invalid _id in save/remove
- food: silently replaced invalid _id with new ObjectId (data loss)
Tests added covering all validation cases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add validation for _id field in devicestatus API:
- POST: validates each document's _id before storage
- DELETE: validates _id parameter (allows wildcard '*')
Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request
Previously, invalid _id values were silently stored as strings instead
of ObjectIds, causing inconsistent data and query issues.
Tests added for all validation cases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add validation for _id field in profile API:
- POST: validates each document's _id before storage
- PUT: validates _id format before update
- DELETE: validates _id parameter before removal
Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request
This prevents 500 errors from BSONError when clients send
UUID-style _ids (e.g., NightscoutKit).
Tests added for all validation cases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The API layer was calling purifyObject() on the raw req.body without
handling arrays. When NightscoutKit sends [status], only the outer
array would be purified (no-op), not the individual status objects.
Now normalizes to array and purifies each devicestatus object,
matching the treatments pattern.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NightscoutKit (Loop) sends profiles wrapped in arrays: [profile].
The MongoDB driver migration changed insert() to insertOne(), breaking
array support.
Changes:
- API layer: normalize input to array, purify each item
- Storage layer: use insertMany() instead of insertOne()
- Tests: verify single, array, and empty array handling
This matches the proven pattern from treatments API.
Fixes array handling regression introduced in d46c5b41.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
UUID_HANDLING should ONLY affect UUID values in the _id field.
Previous commit incorrectly added server-side dedup for syncIdentifier
and uuid fields, which was never part of the original behavior.
Changes:
- upsertQueryFor(): Remove syncIdentifier/uuid as dedup keys
- Batch POST: Only fetch existing IDs by identifier, not by
syncIdentifier/uuid
- tests: Update TEST-CACHE-003/004 to document actual behavior
(duplicates occur without ObjectIdCache - this is by design)
- docs: Correct treatments-schema.md (syncIdentifier/uuid preserved,
not copied to identifier)
- docs: Remove external link from entries-schema.md
Loop carbs/doses rely on ObjectIdCache for dedup, not server-side logic.
This matches the original (pre-change) server behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
syncIdentifier and uuid fields are used for dedup, not copied to
identifier. Only UUID values in _id field are extracted to identifier.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
REQ-SYNC-072 scope correction: normalizeTreatmentId() should ONLY
handle UUID values in the _id field, not copy syncIdentifier or uuid
fields to identifier.
Changes:
- normalizeTreatmentId(): Only extract UUID from _id to identifier
- normalizeEntryId(): Same fix for entries collection
- upsertQueryFor(): Add syncIdentifier and uuid as dedup fallbacks
(fields are preserved, not copied to identifier)
- Batch POST: Fetch _id for docs deduped by syncIdentifier/uuid
Test updates:
- TEST-ID-003, TEST-V1-ID-004: Updated to expect identifier NOT copied
from syncIdentifier (scope fix)
Affected clients:
- Loop overrides (UUID _id → identifier): Still works
- Loop carbs/doses (syncIdentifier): Dedup works, no identifier copy
- xDrip+ (uuid): Dedup works, no identifier copy
- AAPS (identifier): Unchanged
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
UUID_HANDLING default changed to true in 15.0.7. Test now explicitly
sets UUID_HANDLING=false rather than deleting the env var.
742 passing, 1 pending.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Write normalization (syncIdentifier/uuid → identifier) is ALWAYS on
- UUID_HANDLING flag only controls READ path (GET/DELETE by UUID)
- Fix default: UUID_HANDLING=true (not false)
- Remove incorrect xDrip+ mention from entries (doesn't use UUID _id)
- Clarify that flag only affects API calls with UUID as _id parameter
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The feature only handles the specific case where a UUID is sent as
the _id field itself. It does NOT affect:
- AAPS (uses 'identifier' field)
- xDrip+ (uses 'uuid' field)
- Loop carbs/doses (uses 'syncIdentifier' field)
Only affects:
- Loop overrides (_id: syncIdentifier.uuidString)
- Trio CGM entries (_id: UUID)
See docs/10-domain/client-id-handling-deep-dive.md for full analysis.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Avoid implying all AID clients use the same pattern or that any
specific implementation is incorrect. Different clients have
divergent sync patterns - the feature accommodates this variety.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update default from false to true to enable AID client compatibility
out of the box:
- lib/server/env.js: readENVTruthy('UUID_HANDLING', true)
- README.md: Document UUID_HANDLING in Features section
- docs/example-template.env: Update comments, show true as default
Rationale:
- Loop, Trio, AAPS, xDrip+ use UUID sync patterns by default
- Before MongoDB 5.x, UUID _id didn't crash (just didn't CRUD properly)
- ObjectID users completely unaffected (quirk only triggers on UUID)
- Can set UUID_HANDLING=false for strict mode if needed
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
API returns _id as plain string "507f1f77bcf86cd799439011", not
MongoDB Extended JSON format {"$oid": "..."}.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document MongoDB pool settings and AUTH_FAIL_DELAY for test tuning.
Useful for CI or resource-constrained environments.
Refs: DOC-ENV-002
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add 6 tests verifying UUID_HANDLING env var behavior:
- UUID-OFF-001: GET by UUID returns empty (no crash)
- UUID-OFF-002: DELETE by UUID deletes nothing (no crash)
- UUID-ON-001: GET by UUID finds treatment via identifier
- UUID-ON-002: DELETE by UUID removes treatment via identifier
- UUID-ON-003: ObjectId still works normally
- UUID-ON-004: Non-matching UUID returns empty
Tests use clearModuleCache() to reload env.js with different flag values.
Refs: uuid-test-flag-off, uuid-test-flag-on, REQ-SYNC-072
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace Date.now() with the pre-captured 'now' variable in the
'set a pill to BWP with infos' test. This prevents timing drift
between when test data timestamps are set and when the sandbox
is initialized, eliminating flaky failures in CI environments.
Refs: BWP-TIME-001, GAP-TEST-001
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- env.js: Add UUID_HANDLING env var (default: false)
- query.js: Add UUID detection in normalizeIdValue()
- When UUID_HANDLING=true and _id is UUID, search by identifier field
- Returns searchByIdentifier flag to redirect query
- treatments.js: Move queryOpts inside query_for() for env access
- entries.js: Same pattern for entries collection
When UUID_HANDLING=true:
- GET /treatments/{uuid} searches by identifier field
- DELETE /treatments/{uuid} deletes by identifier field
- Same behavior for entries collection
When UUID_HANDLING=false (default):
- UUID _id values return empty results (safe, no crash)
- Maintains backwards compatibility
Refs: uuid-feature-flag, uuid-query-impl from uuid-identifier-lookup.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This test passes locally in ~50ms but occasionally times out at 30s
in constrained GitHub runners. Adding retries(2) allows it to recover
from transient CI resource contention.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The 'WebSocket dbAdd Array Handling Investigation' block was R&D to
understand insertOne behavior with arrays. The investigation concluded:
- MongoDB's insertOne([a,b]) creates single doc (not multiple)
- Fix: sequential processing via processNextItem() in websocket.js
Production tests now cover this behavior:
- 'dbAdd with array input for treatments - current behavior test'
- 'dbAdd with array input for devicestatus - current behavior test'
- 'dbAdd with array input for entries - current behavior test'
Removes 2 flaky investigative tests, keeps 729 production tests passing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Change from warning to process.exit(1) to prevent any possibility of
running destructive test operations against a production database.
Tests now fail immediately if NODE_ENV !== 'test', with clear instructions
on how to fix.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SAFETY-001: Fix tests/ci.test.env to use NODE_ENV=test instead of production
SAFETY-002: Add NODE_ENV check to tests/hooks.js with warning
SAFETY-003: Create tests/fixtures/test-guard.js with guarded deleteMany/drop helpers
This prevents deleteMany({}) from accidentally running against production
databases if test environment is misconfigured.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Trio/Loop upload CGM entries with UUID strings as _id field.
This caused MongoDB errors when re-uploading with different UUID
at same timestamp: "immutable field '_id'" error.
Fix:
- Add normalizeEntryId() to extract UUID from _id to identifier field
- Add upsertQueryFor() to strip non-ObjectId _id before $set
- Maintain sysTime+type as primary dedup key for CGM data integrity
- Add identifier to indexed fields
Tests:
- 3 baseline tests document current sysTime+type dedup behavior
- 6 UUID handling tests including the previously-failing scenario
Refs: GAP-SYNC-045, REQ-SYNC-072
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ben WestandGitHub
copilot-swe-agent[bot]andbewest
Andy LowandGitHub