mirror of
https://github.com/bckelley/cgm-remote-monitor.git
synced 2026-08-24 03:14:12 -05:00
docs: update MongoDB modernization docs to reflect bulkWrite completion
This commit is contained in:
@@ -20,24 +20,28 @@ The MongoDB modernization requires ensuring that v1 API batch endpoints use `ins
|
||||
- Impact assessment is complete
|
||||
- We know exactly what clients expect
|
||||
|
||||
### ⚠️ Issues to Fix
|
||||
### ✅ Issues Fixed (January 2026)
|
||||
|
||||
1. **lib/server/treatments.js (line 18-30)**
|
||||
- Currently uses `async.eachSeries` with individual `replaceOne` per item
|
||||
- **Need:** Use `bulkWrite` for batch upsert semantics
|
||||
- **Impact:** Loop and Trio send arrays, expect batch insert behavior
|
||||
1. **lib/server/treatments.js** ✅ COMPLETED
|
||||
- Now uses `bulkWrite` with `replaceOne` + `upsert: true` for batch operations
|
||||
- Falls back to sequential processing for `preBolus` treatments (which create additional records)
|
||||
- **Commit:** e9417af5
|
||||
|
||||
2. **lib/server/entries.js (line 92-135)**
|
||||
- Currently uses `forEach` with individual `replaceOne` per item
|
||||
- **Need:** Use `bulkWrite` for batch upsert semantics
|
||||
- **Impact:** Loop and Trio send glucose arrays
|
||||
2. **lib/server/entries.js** ✅ COMPLETED
|
||||
- Now uses `bulkWrite` with `updateOne` + `$set` + `upsert: true`
|
||||
- **Commit:** e9417af5
|
||||
|
||||
3. **Response Ordering**
|
||||
- Current async callbacks may not preserve submission order
|
||||
- **Need:** Guarantee response array indices match submission indices
|
||||
- **Impact:** Loop's syncIdentifier→objectId cache mapping breaks otherwise
|
||||
3. **lib/server/devicestatus.js** ✅ COMPLETED
|
||||
- Now uses `insertMany` for batch inserts
|
||||
- **Commit:** e9417af5
|
||||
|
||||
4. **Write Result Format**
|
||||
4. **Response Ordering** ✅ RESOLVED
|
||||
- All batch operations use `ordered: true` to preserve submission order
|
||||
- Response array indices match submission array indices
|
||||
|
||||
### ⚠️ Remaining Issues
|
||||
|
||||
1. **Write Result Format**
|
||||
- MongoDB driver version differences in `insertedIds` format
|
||||
- **Need:** Translator utility to normalize across driver versions
|
||||
- **Impact:** Driver upgrades could break response format
|
||||
@@ -179,10 +183,11 @@ Understand the current flow:
|
||||
- [ ] Handle MongoDB 3.x, 4.x, 5.x differences
|
||||
- [ ] Unit tests for translator
|
||||
|
||||
### Priority 3: Update Storage Layer
|
||||
- [ ] Update `lib/server/treatments.js` to use bulkWrite
|
||||
- [ ] Update `lib/server/entries.js` to use bulkWrite
|
||||
- [ ] Ensure response ordering preserved
|
||||
### Priority 3: Update Storage Layer ✅ COMPLETED
|
||||
- [x] Update `lib/server/treatments.js` to use bulkWrite
|
||||
- [x] Update `lib/server/entries.js` to use bulkWrite
|
||||
- [x] Update `lib/server/devicestatus.js` to use insertMany
|
||||
- [x] Ensure response ordering preserved (using `ordered: true`)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -124,11 +124,15 @@
|
||||
|
||||
**Requirement:** When an array is POSTed to `/api/treatments`, use `insertMany`
|
||||
|
||||
**Current Status:** Arrays create multiple documents (validated by tests)
|
||||
**Current Status:** ✅ COMPLETED (January 2026)
|
||||
|
||||
**Note:** Current implementation uses `async.eachSeries` with individual `replaceOne` calls. While this creates multiple documents correctly, it's not using `insertMany` or `bulkWrite` as recommended.
|
||||
**Implementation:**
|
||||
- `lib/server/treatments.js` now uses `bulkWrite` with `replaceOne` + `upsert: true`
|
||||
- `lib/server/entries.js` now uses `bulkWrite` with `updateOne` + `$set` + `upsert: true`
|
||||
- `lib/server/devicestatus.js` now uses `insertMany`
|
||||
- All batch operations use `ordered: true` for response ordering guarantees
|
||||
|
||||
**Action Required:** Refactor to use `bulkWrite` for better performance and ordering guarantees.
|
||||
**Commit:** e9417af5
|
||||
|
||||
### Section 6.1.2: Response Format ✅
|
||||
|
||||
|
||||
@@ -437,9 +437,10 @@ npm list mongodb mongodb-legacy > mongodb-versions-baseline.txt
|
||||
**Key Question:** Where are arrays being handled?
|
||||
|
||||
**Finding (from code review):**
|
||||
- ✅ `lib/server/treatments.js` line 18-30: Handles arrays with `async.eachSeries` (sequential iteration)
|
||||
- ✅ `lib/server/entries.js` line 92-135: Handles arrays with `forEach` (parallel iteration)
|
||||
- ⚠️ **CRITICAL:** Both use `replaceOne` per item, NOT `insertMany`
|
||||
- ✅ `lib/server/treatments.js`: Now uses `bulkWrite` with `replaceOne` + `upsert: true` for batch operations
|
||||
- ✅ `lib/server/entries.js`: Now uses `bulkWrite` with `updateOne` + `$set` + `upsert: true`
|
||||
- ✅ `lib/server/devicestatus.js`: Now uses `insertMany` for batch inserts
|
||||
- ✅ **COMPLETED (January 2026):** All batch operations migrated to bulk MongoDB operations (commit e9417af5)
|
||||
|
||||
#### Task 2.1.2: Identify v1 vs v3 API Data Flow
|
||||
**Diagram to create:**
|
||||
@@ -449,7 +450,7 @@ POST /api/v1/treatments (array)
|
||||
→ lib/api/treatments/index.js:post_response (line 104-145)
|
||||
→ ctx.treatments.create(array)
|
||||
→ lib/server/treatments.js:create (line 11-38)
|
||||
→ async.eachSeries → replaceOne per item ⚠️ ISSUE: Should be insertMany
|
||||
→ bulkWrite with replaceOne + upsert ✅ FIXED
|
||||
|
||||
V3 API Flow:
|
||||
POST /api/v3/treatments (single object)
|
||||
@@ -473,19 +474,20 @@ res.json(created); // where created is array of objects from storage layer
|
||||
|
||||
### 2.2 Identify Critical Changes Needed
|
||||
|
||||
#### Issue 1: v1 API Must Use insertMany for Arrays
|
||||
**Current:** `async.eachSeries` with individual `replaceOne` calls
|
||||
**Required:** Single `insertMany` call for batch semantics
|
||||
**Impact:** Loop and Trio depend on batch insert behavior
|
||||
#### Issue 1: v1 API Must Use insertMany for Arrays ✅ COMPLETED
|
||||
**Previous:** `async.eachSeries` with individual `replaceOne` calls
|
||||
**Implemented:** `bulkWrite` with batch operations (commit e9417af5)
|
||||
**Impact:** Loop and Trio batch insert behavior now properly supported
|
||||
|
||||
**Affected Files:**
|
||||
- `lib/server/treatments.js` - create() and upsert() functions
|
||||
- `lib/server/entries.js` - create() function
|
||||
**Updated Files:**
|
||||
- `lib/server/treatments.js` - create() now uses bulkWrite
|
||||
- `lib/server/entries.js` - create() now uses bulkWrite
|
||||
- `lib/server/devicestatus.js` - create() now uses insertMany
|
||||
|
||||
#### Issue 2: Response Ordering Must Be Preserved
|
||||
**Current:** Results accumulated in callback order (might not match submission order)
|
||||
**Required:** Response array must match submission array indices
|
||||
**Impact:** Loop's syncIdentifier→objectId cache mapping will break if order changes
|
||||
#### Issue 2: Response Ordering Must Be Preserved ✅ COMPLETED
|
||||
**Previous:** Results accumulated in callback order (might not match submission order)
|
||||
**Implemented:** All bulk operations use `ordered: true`
|
||||
**Impact:** Response array indices now guaranteed to match submission array indices
|
||||
|
||||
#### Issue 3: Write Result Format Translation
|
||||
**Current:** Direct MongoDB write result exposed to clients?
|
||||
|
||||
Reference in New Issue
Block a user