bckelley/cgm-remote-monitor
1
0
Fork 0
mirror of https://github.com/bckelley/cgm-remote-monitor.git synced 2026-08-24 03:14:12 -05:00
Code Issues Packages Projects Releases Wiki Activity

fix: gate UUID write-path normalization on UUID_HANDLING flag and fix docs accuracy

Browse Source 
Co-authored-by: bewest <394179+bewest@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-03-17 23:44:47 +00:00
co-authored by bewest
parent d1c9f185ac
commit 002f34fcda
9 changed files with 95 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+13 -35
View File
@@ -7,47 +7,25 @@ All notable changes to cgm-remote-monitor are documented in this file.
### Added
#### UUID/Identifier Handling (REQ-SYNC-072)
- **Treatments API**: UUID values in `_id` field are now extracted to `identifier`
- Loop overrides: UUID in `_id` → extracted to `identifier`
- Loop carbs/doses: `syncIdentifier` field used for dedup (not copied)
- xDrip+: `uuid` field used for dedup (not copied)
- AAPS: `identifier` unchanged (already correct)
- **Entries API**: CGM entries with UUID `_id` now handled correctly (GAP-SYNC-045)
- **Deduplication**: Server uses `identifier`, `syncIdentifier`, or `uuid` for upsert matching
- **Scope**: Only UUID values in `_id` field trigger normalization; other fields preserved
- **`UUID_HANDLING` env var** (default: `true`): Feature flag that controls UUID `_id` normalization for treatments and entries.
- When `true`: UUID values sent as `_id` are extracted to the `identifier` field and a server-generated ObjectId is assigned. GET/DELETE by UUID are routed through the `identifier` field.
- When `false`: UUID `_id` values are stripped (UUID identity not preserved) and UUID-based queries return empty results.
- **Treatments API**: Loop overrides with UUID `_id` are now normalized correctly, preventing duplicate records (Issue #8450).
- **Entries API**: CGM entries (e.g., Trio) with UUID `_id` are now handled correctly.
- **Scope**: Only UUID values in the `_id` field are affected. Other client identity fields (`syncIdentifier`, `uuid`, `identifier`) are preserved but not modified.
#### Test Infrastructure
- **NODE_ENV=test safety check**: Tests now refuse to run without `NODE_ENV=test`, preventing accidental production database destruction (GAP-SYNC-046)
- New npm scripts: `test:unit`, `test:integration` for faster development cycles
- Comprehensive test suite for UUID handling across v1 and v3 APIs
### Changed
#### Node.js Requirements
- **Minimum**: Node.js 20 LTS (was Node 14/16)
- **Tested**: Node 20, 22, 24
- Updated `engines` field in package.json
#### MongoDB Requirements
- **Minimum**: MongoDB 4.4 (was 4.2)
- **Tested**: MongoDB 4.4, 5.0, 6.0
- Improved compatibility with MongoDB 5.x+ `$set` behavior
### Fixed
- **Loop Override Sync** (Issue #8450): Overrides with UUID `_id` no longer cause duplicate records
- **mmol BG Display**: Fixed OpenAPS tooltips showing incorrect values for mmol/L users
- **AAPS Entry Dedup**: Added retry logic for CI flakiness in concurrent upload tests
- **NODE_ENV=test safety check**: Tests now refuse to run without `NODE_ENV=test`, preventing accidental production database modification.
- Comprehensive test suite for UUID handling behavior across write and read paths.
### Documentation
- Updated README.md with current Node.js and MongoDB requirements
- Added "Running Tests Locally" section to CONTRIBUTING.md
- Updated treatments schema documentation with identifier normalization behavior
### Security
- Enabled Traditional Chinese localization
- Updated README.md with `UUID_HANDLING` and MongoDB pool configuration env vars.
- Added entries schema documentation (`docs/data-schemas/entries-schema.md`).
- Updated treatments schema documentation with identifier normalization behavior.
- Added test environment variables reference to CONTRIBUTING.md.
---
README.md
+1 -1
View File
@@ -252,7 +252,7 @@ To learn more about the Nightscout API, visit https://YOUR-SITE.com/api-docs/ or
Setting it to `denied` will require a token from every visit, using `status-only` will enable api-secret based login.
* `IMPORT_CONFIG` - Used to import settings and extended settings from a url such as a gist. Structure of file should be something like: `{"settings": {"theme": "colors"}, "extendedSettings": {"upbat": {"enableAlerts": true}}}`
* `TREATMENTS_AUTH` (`on`) - possible values `on` or `off`. Deprecated, if set to `off` the `careportal` role will be added to `AUTH_DEFAULT_ROLES`
* `UUID_HANDLING` (`true`) - Controls how UUID `_id` values are handled for treatments and entries. When `true` (default), if a client sends a UUID string as the `_id` field, it is moved to the `identifier` field and the server generates a proper ObjectId for `_id`. This only affects the specific case where a UUID is sent as `_id` (e.g., Loop overrides, Trio CGM entries). Set to `false` for strict mode where UUID `_id` values are rejected.
* `UUID_HANDLING` (`true`) - Controls how UUID `_id` values are handled for treatments and entries. When `true` (default), if a client sends a UUID string as the `_id` field, it is extracted to the `identifier` field (for sync deduplication) and the server generates a proper ObjectId for `_id`. Queries by UUID (`GET`/`DELETE`) are also routed through the `identifier` field. When `false`, UUID `_id` values are silently stripped on write (no identifier is preserved) and UUID-based queries return empty results. This only affects the specific case where a UUID is sent as `_id` (e.g., Loop overrides, Trio CGM entries).
#### Data Rights
docs/data-schemas/entries-schema.md
+13 -11
View File
@@ -52,14 +52,14 @@ The `entries` collection stores CGM (Continuous Glucose Monitor) sensor readings
### Identifier Field Normalization (REQ-SYNC-072)
As of v15.0.7, the server **always** normalizes certain client sync identities into the `identifier` field on write:
As of v15.0.7, the server normalizes UUID values in `_id` into the `identifier` field when `UUID_HANDLING=true` (default):
| Client | Sends | Server Action |
|--------|-------|---------------|
| **Trio** | UUID in `_id` | Move to `identifier`, assign server ObjectId |
| **Loop** (entries) | ObjectId (from cache) | Normal ObjectId behavior |
| Client | Sends | Server Action (UUID_HANDLING=true) | Server Action (UUID_HANDLING=false) |
|--------|-------|-------------------------------------|--------------------------------------|
| **Trio** | UUID in `_id` | Move to `identifier`, assign server ObjectId | Strip `_id`, assign ObjectId (UUID not preserved) |
| **Loop** (entries) | ObjectId (from cache) | Normal ObjectId behavior | Normal ObjectId behavior |
**Note**: The write-path normalization happens **always**. The `UUID_HANDLING` env var only controls the **read path** (GET/DELETE by UUID).
**Note**: The `UUID_HANDLING` env var controls **both** write-path normalization (identifier extraction) and read-path queries (GET/DELETE by UUID).
**Important**: For entries, `sysTime + type` is ALWAYS the primary deduplication key. The `identifier` field is for client sync tracking only - it does NOT override the dedup logic.
@@ -83,34 +83,36 @@ This means:
When a client sends a UUID as `_id`:
1. **Extract**: UUID is copied to `identifier` field
1. **Extract**: UUID is copied to `identifier` field (when `UUID_HANDLING=true`)
2. **Strip**: Non-ObjectId `_id` is removed before database operation
3. **Upsert**: Server uses `sysTime + type` for matching
4. **Assign**: Server-generated ObjectId becomes final `_id`
This prevents the MongoDB "immutable field '_id'" error while preserving client sync identity.
This prevents the MongoDB "immutable field '_id'" error while preserving client sync identity (when enabled).
---
## UUID_HANDLING Feature Flag
The `UUID_HANDLING` environment variable controls **read path** behavior (GET/DELETE by UUID). It does NOT affect write normalization, which always happens.
The `UUID_HANDLING` environment variable controls both **write-path** normalization (identifier extraction) and **read-path** queries (GET/DELETE by UUID).
When `UUID_HANDLING=true` (default):
| Operation | Behavior |
|-----------|----------|
| POST/PUT with UUID `_id` | UUID moved to `identifier`, server assigns ObjectId |
| GET by UUID | Searches by `identifier` field |
| DELETE by UUID | Deletes by `identifier` field |
When `UUID_HANDLING=false` (strict mode):
When `UUID_HANDLING=false`:
| Operation | Behavior |
|-----------|----------|
| POST/PUT with UUID `_id` | UUID `_id` stripped, ObjectId assigned (UUID not preserved) |
| GET by UUID | Returns empty (no crash) |
| DELETE by UUID | Deletes nothing (no crash) |
**Note**: This only affects cases where a UUID is passed as the `_id` parameter in API calls (e.g., `GET /api/v1/entries/{uuid}`).
**Note**: This only affects cases where a UUID is passed as the `_id` field (writes) or as the `_id` parameter in API calls (reads), e.g., `GET /api/v1/entries/{uuid}`.
---
docs/data-schemas/treatments-schema.md
+9 -9
View File
@@ -123,22 +123,22 @@ The `treatments` collection stores all user interventions and system events rela
### Identifier Field Normalization (REQ-SYNC-072)
As of v15.0.7, the server normalizes **UUID values in the `_id` field** to the `identifier` field:
As of v15.0.7, the server normalizes **UUID values in the `_id` field** when `UUID_HANDLING=true` (default):
| Client | Client Field | Server Action |
|--------|--------------|---------------|
| **Loop** (overrides) | UUID in `_id` | Move to `identifier`, assign server ObjectId |
| **Loop** (carbs/doses) | `syncIdentifier` | Preserved as-is (no server-side dedup) |
| **AAPS** | `identifier` | Unchanged (already correct) |
| **xDrip+** | `uuid` | Preserved as-is (no server-side dedup) |
| Client | Client Field | UUID_HANDLING=true | UUID_HANDLING=false |
|--------|--------------|---------------------|----------------------|
| **Loop** (overrides) | UUID in `_id` | Move to `identifier`, assign ObjectId | Strip `_id`, assign ObjectId (UUID not preserved) |
| **Loop** (carbs/doses) | `syncIdentifier` | Preserved as-is | Preserved as-is |
| **AAPS** | `identifier` | Unchanged (already correct) | Unchanged (already correct) |
| **xDrip+** | `uuid` | Preserved as-is | Preserved as-is |
**Scope:** Only UUID values in the `_id` field are affected. Other client identity fields (`syncIdentifier`, `uuid`) are preserved but NOT copied to `identifier`.
**Note:** `UUID_HANDLING` env var controls both read and write paths for UUID `_id` handling.
**UUID_HANDLING controls both write-path normalization and read-path queries** (GET/DELETE by UUID `_id`).
**Deduplication Priority:** The server uses `identifier` or `_id` for upsert matching when present, falling back to `created_at + eventType` for legacy records.
**Example - Loop Override Upload:**
**Example - Loop Override Upload (UUID_HANDLING=true):**
```javascript
// Client sends:
docs/example-template.env
+7 -8
View File
@@ -19,11 +19,10 @@ AUTH_FAIL_DELAY=50
# (e.g., Loop overrides, Trio CGM entries)
# Does NOT affect clients using separate identifier/uuid fields (AAPS, xDrip+, etc.)
#
# When true (default): UUID in _id is moved to 'identifier' field, server generates ObjectId
# - POST/PUT: UUID _id → identifier, new ObjectId assigned
# - GET/DELETE: UUID _id searches by identifier field
# When false: Strict mode - UUID _id values rejected on write, ignored on read
# UUID_HANDLING=true
#
# Set to false for strict ObjectId-only mode:
# UUID_HANDLING=false
# When true (default):
# - POST/PUT: UUID _id → extracted to 'identifier' field, new ObjectId assigned
# - GET/DELETE: UUID _id queries search by 'identifier' field
# When false:
# - POST/PUT: UUID _id is stripped, new ObjectId assigned (UUID identity not preserved)
# - GET/DELETE: UUID _id queries return empty results (no crash)
# UUID_HANDLING=true
lib/server/entries.js
+4 -3
View File
@@ -244,11 +244,12 @@ function storage (env, ctx) {
// REQ-SYNC-072: Only handle UUID values in _id field
// Scope: ONLY the _id field when value is a valid UUID
if (typeof doc._id === 'string' && !OBJECT_ID_HEX_RE.test(doc._id)) {
// Non-ObjectId string in _id (UUID format) - move to identifier
if (!doc.identifier) {
// Non-ObjectId string in _id (UUID format)
// Only move to identifier when UUID_HANDLING is enabled
if (env.uuidHandling && !doc.identifier) {
doc.identifier = doc._id;
}
// Delete invalid _id so server generates ObjectId
// Always delete invalid _id so server generates ObjectId
delete doc._id;
} else if (Object.prototype.hasOwnProperty.call(doc, '_id') && doc._id !== null && doc._id !== '') {
// Convert valid ObjectId strings to ObjectId objects
lib/server/env.js
+4 -2
View File
@@ -77,8 +77,10 @@ function setSSL () {
env.secureCspReportOnly = readENVTruthy("SECURE_CSP_REPORT_ONLY", false);
// UUID handling for specific client patterns that send UUID as _id field
// When true (default): UUID _id values are moved to 'identifier' field, server generates ObjectId
// When false: UUID _id values are rejected on write, ignored on read (strict mode)
// When true (default): UUID _id values are extracted to 'identifier' field, server generates ObjectId
// - Writes: UUID preserved as identifier, new ObjectId assigned
// - Reads: GET/DELETE by UUID searches identifier field
// When false: UUID _id values are stripped (UUID not preserved), UUID-based queries return empty
// Only affects cases where UUID is sent as _id (e.g., Loop overrides, Trio CGM entries)
env.uuidHandling = readENVTruthy("UUID_HANDLING", true);
}
lib/server/treatments.js
+4 -3
View File
@@ -350,11 +350,12 @@ function storage (env, ctx) {
// Scope: ONLY the _id field when value is a valid UUID
// Does NOT touch syncIdentifier, uuid, or other client fields
if (typeof obj._id === 'string' && !OBJECT_ID_HEX_RE.test(obj._id)) {
// Non-ObjectId string in _id (UUID format) - move to identifier
if (!obj.identifier) {
// Non-ObjectId string in _id (UUID format)
// Only move to identifier when UUID_HANDLING is enabled
if (env.uuidHandling && !obj.identifier) {
obj.identifier = obj._id;
}
// Delete invalid _id so server generates ObjectId
// Always delete invalid _id so server generates ObjectId
delete obj._id;
} else if (Object.prototype.hasOwnProperty.call(obj, '_id') && obj._id !== null && obj._id !== '') {
// Convert valid ObjectId strings to ObjectId objects
tests/uuid-handling.test.js
+40 -1
View File
@@ -129,7 +129,26 @@ describe('UUID_HANDLING=false (explicit)', function() {
});
});
});
});
it('UUID-OFF-003: POST with UUID _id strips UUID, does not copy to identifier', function(done) {
// When UUID_HANDLING=false, UUID in _id is stripped but NOT preserved as identifier
self.ctx.treatments.create([{
_id: TEST_UUID,
eventType: 'Note',
notes: 'UUID stripped write test',
created_at: new Date().toISOString()
}], function(err) {
should.not.exist(err);
// Treatment was created (no crash), but UUID was not preserved as identifier
self.ctx.treatments.list({}, function(err, results) {
should.not.exist(err);
results.length.should.equal(1);
// identifier should NOT be set from the UUID _id when flag is false
should.not.exist(results[0].identifier);
done();
});
});
});
describe('UUID_HANDLING=true', function() {
var self = this;
@@ -271,6 +290,26 @@ describe('UUID_HANDLING=true', function() {
});
});
});
it('UUID-ON-005: POST with UUID _id extracts UUID to identifier', function(done) {
// When UUID_HANDLING=true, UUID in _id is extracted to identifier
self.ctx.treatments.create([{
_id: TEST_UUID,
eventType: 'Note',
notes: 'UUID write test',
created_at: new Date().toISOString()
}], function(err) {
should.not.exist(err);
// UUID should have been moved to identifier
self.ctx.treatments.list({}, function(err, results) {
should.not.exist(err);
results.length.should.equal(1);
// identifier should be set from the UUID _id
results[0].identifier.should.equal(TEST_UUID);
done();
});
});
});
});
// ============================================