fix: gate UUID write-path normalization on UUID_HANDLING flag and fix docs accuracy

Co-authored-by: bewest <394179+bewest@users.noreply.github.com>

CHANGELOG.md

@@ -7,47 +7,25 @@ All notable changes to cgm-remote-monitor are documented in this file.
 ### Added
 
 #### UUID/Identifier Handling (REQ-SYNC-072)
-- **Treatments API**: UUID values in `_id` field are now extracted to `identifier`
-  - Loop overrides: UUID in `_id` → extracted to `identifier`
-  - Loop carbs/doses: `syncIdentifier` field used for dedup (not copied)
-  - xDrip+: `uuid` field used for dedup (not copied)
-  - AAPS: `identifier` unchanged (already correct)
-- **Entries API**: CGM entries with UUID `_id` now handled correctly (GAP-SYNC-045)
-- **Deduplication**: Server uses `identifier`, `syncIdentifier`, or `uuid` for upsert matching
-- **Scope**: Only UUID values in `_id` field trigger normalization; other fields preserved
+
+- **`UUID_HANDLING` env var** (default: `true`): Feature flag that controls UUID `_id` normalization for treatments and entries.
+  - When `true`: UUID values sent as `_id` are extracted to the `identifier` field and a server-generated ObjectId is assigned. GET/DELETE by UUID are routed through the `identifier` field.
+  - When `false`: UUID `_id` values are stripped (UUID identity not preserved) and UUID-based queries return empty results.
+- **Treatments API**: Loop overrides with UUID `_id` are now normalized correctly, preventing duplicate records (Issue #8450).
+- **Entries API**: CGM entries (e.g., Trio) with UUID `_id` are now handled correctly.
+- **Scope**: Only UUID values in the `_id` field are affected. Other client identity fields (`syncIdentifier`, `uuid`, `identifier`) are preserved but not modified.
 
 #### Test Infrastructure
-- **NODE_ENV=test safety check**: Tests now refuse to run without `NODE_ENV=test`, preventing accidental production database destruction (GAP-SYNC-046)
-- New npm scripts: `test:unit`, `test:integration` for faster development cycles
-- Comprehensive test suite for UUID handling across v1 and v3 APIs
 
-### Changed
-
-#### Node.js Requirements
-- **Minimum**: Node.js 20 LTS (was Node 14/16)
-- **Tested**: Node 20, 22, 24
-- Updated `engines` field in package.json
-
-#### MongoDB Requirements
-- **Minimum**: MongoDB 4.4 (was 4.2)
-- **Tested**: MongoDB 4.4, 5.0, 6.0
-- Improved compatibility with MongoDB 5.x+ `$set` behavior
-
-### Fixed
-
-- **Loop Override Sync** (Issue #8450): Overrides with UUID `_id` no longer cause duplicate records
-- **mmol BG Display**: Fixed OpenAPS tooltips showing incorrect values for mmol/L users
-- **AAPS Entry Dedup**: Added retry logic for CI flakiness in concurrent upload tests
+- **NODE_ENV=test safety check**: Tests now refuse to run without `NODE_ENV=test`, preventing accidental production database modification.
+- Comprehensive test suite for UUID handling behavior across write and read paths.
 
 ### Documentation
 
-- Updated README.md with current Node.js and MongoDB requirements
-- Added "Running Tests Locally" section to CONTRIBUTING.md
-- Updated treatments schema documentation with identifier normalization behavior
-
-### Security
-
-- Enabled Traditional Chinese localization
+- Updated README.md with `UUID_HANDLING` and MongoDB pool configuration env vars.
+- Added entries schema documentation (`docs/data-schemas/entries-schema.md`).
+- Updated treatments schema documentation with identifier normalization behavior.
+- Added test environment variables reference to CONTRIBUTING.md.
 
 ---
 

README.md

@@ -252,7 +252,7 @@ To learn more about the Nightscout API, visit https://YOUR-SITE.com/api-docs/ or
     Setting it to `denied` will require a token from every visit, using `status-only` will enable api-secret based login.
   * `IMPORT_CONFIG` - Used to import settings and extended settings from a url such as a gist.  Structure of file should be something like: `{"settings": {"theme": "colors"}, "extendedSettings": {"upbat": {"enableAlerts": true}}}`
   * `TREATMENTS_AUTH` (`on`) - possible values `on` or `off`. Deprecated, if set to `off` the `careportal` role will be added to `AUTH_DEFAULT_ROLES`
-  * `UUID_HANDLING` (`true`) - Controls how UUID `_id` values are handled for treatments and entries. When `true` (default), if a client sends a UUID string as the `_id` field, it is moved to the `identifier` field and the server generates a proper ObjectId for `_id`. This only affects the specific case where a UUID is sent as `_id` (e.g., Loop overrides, Trio CGM entries). Set to `false` for strict mode where UUID `_id` values are rejected.
+  * `UUID_HANDLING` (`true`) - Controls how UUID `_id` values are handled for treatments and entries. When `true` (default), if a client sends a UUID string as the `_id` field, it is extracted to the `identifier` field (for sync deduplication) and the server generates a proper ObjectId for `_id`. Queries by UUID (`GET`/`DELETE`) are also routed through the `identifier` field. When `false`, UUID `_id` values are silently stripped on write (no identifier is preserved) and UUID-based queries return empty results. This only affects the specific case where a UUID is sent as `_id` (e.g., Loop overrides, Trio CGM entries).
 
 #### Data Rights
 

docs/data-schemas/entries-schema.md

@@ -52,14 +52,14 @@ The `entries` collection stores CGM (Continuous Glucose Monitor) sensor readings
 
 ### Identifier Field Normalization (REQ-SYNC-072)
 
-As of v15.0.7, the server **always** normalizes certain client sync identities into the `identifier` field on write:
+As of v15.0.7, the server normalizes UUID values in `_id` into the `identifier` field when `UUID_HANDLING=true` (default):
 
-| Client | Sends | Server Action |
-|--------|-------|---------------|
-| **Trio** | UUID in `_id` | Move to `identifier`, assign server ObjectId |
-| **Loop** (entries) | ObjectId (from cache) | Normal ObjectId behavior |
+| Client | Sends | Server Action (UUID_HANDLING=true) | Server Action (UUID_HANDLING=false) |
+|--------|-------|-------------------------------------|--------------------------------------|
+| **Trio** | UUID in `_id` | Move to `identifier`, assign server ObjectId | Strip `_id`, assign ObjectId (UUID not preserved) |
+| **Loop** (entries) | ObjectId (from cache) | Normal ObjectId behavior | Normal ObjectId behavior |
 
-**Note**: The write-path normalization happens **always**. The `UUID_HANDLING` env var only controls the **read path** (GET/DELETE by UUID).
+**Note**: The `UUID_HANDLING` env var controls **both** write-path normalization (identifier extraction) and read-path queries (GET/DELETE by UUID).
 
 **Important**: For entries, `sysTime + type` is ALWAYS the primary deduplication key. The `identifier` field is for client sync tracking only - it does NOT override the dedup logic.
 
@@ -83,34 +83,36 @@ This means:
 
 When a client sends a UUID as `_id`:
 
-1. **Extract**: UUID is copied to `identifier` field
+1. **Extract**: UUID is copied to `identifier` field (when `UUID_HANDLING=true`)
 2. **Strip**: Non-ObjectId `_id` is removed before database operation
 3. **Upsert**: Server uses `sysTime + type` for matching
 4. **Assign**: Server-generated ObjectId becomes final `_id`
 
-This prevents the MongoDB "immutable field '_id'" error while preserving client sync identity.
+This prevents the MongoDB "immutable field '_id'" error while preserving client sync identity (when enabled).
 
 ---
 
 ## UUID_HANDLING Feature Flag
 
-The `UUID_HANDLING` environment variable controls **read path** behavior (GET/DELETE by UUID). It does NOT affect write normalization, which always happens.
+The `UUID_HANDLING` environment variable controls both **write-path** normalization (identifier extraction) and **read-path** queries (GET/DELETE by UUID).
 
 When `UUID_HANDLING=true` (default):
 
 | Operation | Behavior |
 |-----------|----------|
+| POST/PUT with UUID `_id` | UUID moved to `identifier`, server assigns ObjectId |
 | GET by UUID | Searches by `identifier` field |
 | DELETE by UUID | Deletes by `identifier` field |
 
-When `UUID_HANDLING=false` (strict mode):
+When `UUID_HANDLING=false`:
 
 | Operation | Behavior |
 |-----------|----------|
+| POST/PUT with UUID `_id` | UUID `_id` stripped, ObjectId assigned (UUID not preserved) |
 | GET by UUID | Returns empty (no crash) |
 | DELETE by UUID | Deletes nothing (no crash) |
 
-**Note**: This only affects cases where a UUID is passed as the `_id` parameter in API calls (e.g., `GET /api/v1/entries/{uuid}`).
+**Note**: This only affects cases where a UUID is passed as the `_id` field (writes) or as the `_id` parameter in API calls (reads), e.g., `GET /api/v1/entries/{uuid}`.
 
 ---
 

docs/data-schemas/treatments-schema.md

@@ -123,22 +123,22 @@ The `treatments` collection stores all user interventions and system events rela
 
 ### Identifier Field Normalization (REQ-SYNC-072)
 
-As of v15.0.7, the server normalizes **UUID values in the `_id` field** to the `identifier` field:
+As of v15.0.7, the server normalizes **UUID values in the `_id` field** when `UUID_HANDLING=true` (default):
 
-| Client | Client Field | Server Action |
-|--------|--------------|---------------|
-| **Loop** (overrides) | UUID in `_id` | Move to `identifier`, assign server ObjectId |
-| **Loop** (carbs/doses) | `syncIdentifier` | Preserved as-is (no server-side dedup) |
-| **AAPS** | `identifier` | Unchanged (already correct) |
-| **xDrip+** | `uuid` | Preserved as-is (no server-side dedup) |
+| Client | Client Field | UUID_HANDLING=true | UUID_HANDLING=false |
+|--------|--------------|---------------------|----------------------|
+| **Loop** (overrides) | UUID in `_id` | Move to `identifier`, assign ObjectId | Strip `_id`, assign ObjectId (UUID not preserved) |
+| **Loop** (carbs/doses) | `syncIdentifier` | Preserved as-is | Preserved as-is |
+| **AAPS** | `identifier` | Unchanged (already correct) | Unchanged (already correct) |
+| **xDrip+** | `uuid` | Preserved as-is | Preserved as-is |
 
 **Scope:** Only UUID values in the `_id` field are affected. Other client identity fields (`syncIdentifier`, `uuid`) are preserved but NOT copied to `identifier`.
 
-**Note:** `UUID_HANDLING` env var controls both read and write paths for UUID `_id` handling.
+**UUID_HANDLING controls both write-path normalization and read-path queries** (GET/DELETE by UUID `_id`).
 
 **Deduplication Priority:** The server uses `identifier` or `_id` for upsert matching when present, falling back to `created_at + eventType` for legacy records.
 
-**Example - Loop Override Upload:**
+**Example - Loop Override Upload (UUID_HANDLING=true):**
 
 ```javascript
 // Client sends:

docs/example-template.env

@@ -19,11 +19,10 @@ AUTH_FAIL_DELAY=50
 # (e.g., Loop overrides, Trio CGM entries)
 # Does NOT affect clients using separate identifier/uuid fields (AAPS, xDrip+, etc.)
 #
-# When true (default): UUID in _id is moved to 'identifier' field, server generates ObjectId
-# - POST/PUT: UUID _id → identifier, new ObjectId assigned
-# - GET/DELETE: UUID _id searches by identifier field
-# When false: Strict mode - UUID _id values rejected on write, ignored on read
+# When true (default):
+# - POST/PUT: UUID _id → extracted to 'identifier' field, new ObjectId assigned
+# - GET/DELETE: UUID _id queries search by 'identifier' field
+# When false:
+# - POST/PUT: UUID _id is stripped, new ObjectId assigned (UUID identity not preserved)
+# - GET/DELETE: UUID _id queries return empty results (no crash)
 # UUID_HANDLING=true
-#
-# Set to false for strict ObjectId-only mode:
-# UUID_HANDLING=false

lib/server/entries.js

@@ -244,11 +244,12 @@ function storage (env, ctx) {
     // REQ-SYNC-072: Only handle UUID values in _id field
     // Scope: ONLY the _id field when value is a valid UUID
     if (typeof doc._id === 'string' && !OBJECT_ID_HEX_RE.test(doc._id)) {
-      // Non-ObjectId string in _id (UUID format) - move to identifier
-      if (!doc.identifier) {
+      // Non-ObjectId string in _id (UUID format)
+      // Only move to identifier when UUID_HANDLING is enabled
+      if (env.uuidHandling && !doc.identifier) {
         doc.identifier = doc._id;
       }
-      // Delete invalid _id so server generates ObjectId
+      // Always delete invalid _id so server generates ObjectId
       delete doc._id;
     } else if (Object.prototype.hasOwnProperty.call(doc, '_id') && doc._id !== null && doc._id !== '') {
       // Convert valid ObjectId strings to ObjectId objects

lib/server/env.js

@@ -77,8 +77,10 @@ function setSSL () {
   env.secureCspReportOnly = readENVTruthy("SECURE_CSP_REPORT_ONLY", false);
   
   // UUID handling for specific client patterns that send UUID as _id field
-  // When true (default): UUID _id values are moved to 'identifier' field, server generates ObjectId
-  // When false: UUID _id values are rejected on write, ignored on read (strict mode)
+  // When true (default): UUID _id values are extracted to 'identifier' field, server generates ObjectId
+  //   - Writes: UUID preserved as identifier, new ObjectId assigned
+  //   - Reads: GET/DELETE by UUID searches identifier field
+  // When false: UUID _id values are stripped (UUID not preserved), UUID-based queries return empty
   // Only affects cases where UUID is sent as _id (e.g., Loop overrides, Trio CGM entries)
   env.uuidHandling = readENVTruthy("UUID_HANDLING", true);
 }

lib/server/treatments.js

@@ -350,11 +350,12 @@ function storage (env, ctx) {
     // Scope: ONLY the _id field when value is a valid UUID
     // Does NOT touch syncIdentifier, uuid, or other client fields
     if (typeof obj._id === 'string' && !OBJECT_ID_HEX_RE.test(obj._id)) {
-      // Non-ObjectId string in _id (UUID format) - move to identifier
-      if (!obj.identifier) {
+      // Non-ObjectId string in _id (UUID format)
+      // Only move to identifier when UUID_HANDLING is enabled
+      if (env.uuidHandling && !obj.identifier) {
         obj.identifier = obj._id;
       }
-      // Delete invalid _id so server generates ObjectId
+      // Always delete invalid _id so server generates ObjectId
       delete obj._id;
     } else if (Object.prototype.hasOwnProperty.call(obj, '_id') && obj._id !== null && obj._id !== '') {
       // Convert valid ObjectId strings to ObjectId objects

tests/uuid-handling.test.js

@@ -129,7 +129,26 @@ describe('UUID_HANDLING=false (explicit)', function() {
         });
     });
   });
-});
+
+  it('UUID-OFF-003: POST with UUID _id strips UUID, does not copy to identifier', function(done) {
+    // When UUID_HANDLING=false, UUID in _id is stripped but NOT preserved as identifier
+    self.ctx.treatments.create([{
+      _id: TEST_UUID,
+      eventType: 'Note',
+      notes: 'UUID stripped write test',
+      created_at: new Date().toISOString()
+    }], function(err) {
+      should.not.exist(err);
+      // Treatment was created (no crash), but UUID was not preserved as identifier
+      self.ctx.treatments.list({}, function(err, results) {
+        should.not.exist(err);
+        results.length.should.equal(1);
+        // identifier should NOT be set from the UUID _id when flag is false
+        should.not.exist(results[0].identifier);
+        done();
+      });
+    });
+  });
 
 describe('UUID_HANDLING=true', function() {
   var self = this;
@@ -271,6 +290,26 @@ describe('UUID_HANDLING=true', function() {
         });
     });
   });
+
+  it('UUID-ON-005: POST with UUID _id extracts UUID to identifier', function(done) {
+    // When UUID_HANDLING=true, UUID in _id is extracted to identifier
+    self.ctx.treatments.create([{
+      _id: TEST_UUID,
+      eventType: 'Note',
+      notes: 'UUID write test',
+      created_at: new Date().toISOString()
+    }], function(err) {
+      should.not.exist(err);
+      // UUID should have been moved to identifier
+      self.ctx.treatments.list({}, function(err, results) {
+        should.not.exist(err);
+        results.length.should.equal(1);
+        // identifier should be set from the UUID _id
+        results[0].identifier.should.equal(TEST_UUID);
+        done();
+      });
+    });
+  });
 });
 
 // ============================================