docs: update MongoDB modernization docs to reflect bulkWrite completion

This commit is contained in:
Ben West
2026-01-19 13:49:36 -08:00
parent 9318a577b5
commit 367ebdacd3
3 changed files with 47 additions and 36 deletions
+23 -18
View File
@@ -20,24 +20,28 @@ The MongoDB modernization requires ensuring that v1 API batch endpoints use `ins
- Impact assessment is complete
- We know exactly what clients expect
### ⚠️ Issues to Fix
### Issues Fixed (January 2026)
1. **lib/server/treatments.js (line 18-30)**
- Currently uses `async.eachSeries` with individual `replaceOne` per item
- **Need:** Use `bulkWrite` for batch upsert semantics
- **Impact:** Loop and Trio send arrays, expect batch insert behavior
1. **lib/server/treatments.js** ✅ COMPLETED
- Now uses `bulkWrite` with `replaceOne` + `upsert: true` for batch operations
- Falls back to sequential processing for `preBolus` treatments (which create additional records)
- **Commit:** e9417af5
2. **lib/server/entries.js (line 92-135)**
- Currently uses `forEach` with individual `replaceOne` per item
- **Need:** Use `bulkWrite` for batch upsert semantics
- **Impact:** Loop and Trio send glucose arrays
2. **lib/server/entries.js** ✅ COMPLETED
- Now uses `bulkWrite` with `updateOne` + `$set` + `upsert: true`
- **Commit:** e9417af5
3. **Response Ordering**
- Current async callbacks may not preserve submission order
- **Need:** Guarantee response array indices match submission indices
- **Impact:** Loop's syncIdentifier→objectId cache mapping breaks otherwise
3. **lib/server/devicestatus.js** ✅ COMPLETED
- Now uses `insertMany` for batch inserts
- **Commit:** e9417af5
4. **Write Result Format**
4. **Response Ordering** ✅ RESOLVED
- All batch operations use `ordered: true` to preserve submission order
- Response array indices match submission array indices
### ⚠️ Remaining Issues
1. **Write Result Format**
- MongoDB driver version differences in `insertedIds` format
- **Need:** Translator utility to normalize across driver versions
- **Impact:** Driver upgrades could break response format
@@ -179,10 +183,11 @@ Understand the current flow:
- [ ] Handle MongoDB 3.x, 4.x, 5.x differences
- [ ] Unit tests for translator
### Priority 3: Update Storage Layer
- [ ] Update `lib/server/treatments.js` to use bulkWrite
- [ ] Update `lib/server/entries.js` to use bulkWrite
- [ ] Ensure response ordering preserved
### Priority 3: Update Storage Layer ✅ COMPLETED
- [x] Update `lib/server/treatments.js` to use bulkWrite
- [x] Update `lib/server/entries.js` to use bulkWrite
- [x] Update `lib/server/devicestatus.js` to use insertMany
- [x] Ensure response ordering preserved (using `ordered: true`)
---
@@ -124,11 +124,15 @@
**Requirement:** When an array is POSTed to `/api/treatments`, use `insertMany`
**Current Status:** Arrays create multiple documents (validated by tests)
**Current Status:** ✅ COMPLETED (January 2026)
**Note:** Current implementation uses `async.eachSeries` with individual `replaceOne` calls. While this creates multiple documents correctly, it's not using `insertMany` or `bulkWrite` as recommended.
**Implementation:**
- `lib/server/treatments.js` now uses `bulkWrite` with `replaceOne` + `upsert: true`
- `lib/server/entries.js` now uses `bulkWrite` with `updateOne` + `$set` + `upsert: true`
- `lib/server/devicestatus.js` now uses `insertMany`
- All batch operations use `ordered: true` for response ordering guarantees
**Action Required:** Refactor to use `bulkWrite` for better performance and ordering guarantees.
**Commit:** e9417af5
### Section 6.1.2: Response Format ✅
@@ -437,9 +437,10 @@ npm list mongodb mongodb-legacy > mongodb-versions-baseline.txt
**Key Question:** Where are arrays being handled?
**Finding (from code review):**
- ✅ `lib/server/treatments.js` line 18-30: Handles arrays with `async.eachSeries` (sequential iteration)
- ✅ `lib/server/entries.js` line 92-135: Handles arrays with `forEach` (parallel iteration)
- ⚠️ **CRITICAL:** Both use `replaceOne` per item, NOT `insertMany`
- ✅ `lib/server/treatments.js`: Now uses `bulkWrite` with `replaceOne` + `upsert: true` for batch operations
- ✅ `lib/server/entries.js`: Now uses `bulkWrite` with `updateOne` + `$set` + `upsert: true`
- ✅ `lib/server/devicestatus.js`: Now uses `insertMany` for batch inserts
- ✅ **COMPLETED (January 2026):** All batch operations migrated to bulk MongoDB operations (commit e9417af5)
#### Task 2.1.2: Identify v1 vs v3 API Data Flow
**Diagram to create:**
@@ -449,7 +450,7 @@ POST /api/v1/treatments (array)
→ lib/api/treatments/index.js:post_response (line 104-145)
→ ctx.treatments.create(array)
→ lib/server/treatments.js:create (line 11-38)
async.eachSeries → replaceOne per item ⚠️ ISSUE: Should be insertMany
bulkWrite with replaceOne + upsert ✅ FIXED
V3 API Flow:
POST /api/v3/treatments (single object)
@@ -473,19 +474,20 @@ res.json(created); // where created is array of objects from storage layer
### 2.2 Identify Critical Changes Needed
#### Issue 1: v1 API Must Use insertMany for Arrays
**Current:** `async.eachSeries` with individual `replaceOne` calls
**Required:** Single `insertMany` call for batch semantics
**Impact:** Loop and Trio depend on batch insert behavior
#### Issue 1: v1 API Must Use insertMany for Arrays ✅ COMPLETED
**Previous:** `async.eachSeries` with individual `replaceOne` calls
**Implemented:** `bulkWrite` with batch operations (commit e9417af5)
**Impact:** Loop and Trio batch insert behavior now properly supported
**Affected Files:**
- `lib/server/treatments.js` - create() and upsert() functions
- `lib/server/entries.js` - create() function
**Updated Files:**
- `lib/server/treatments.js` - create() now uses bulkWrite
- `lib/server/entries.js` - create() now uses bulkWrite
- `lib/server/devicestatus.js` - create() now uses insertMany
#### Issue 2: Response Ordering Must Be Preserved
**Current:** Results accumulated in callback order (might not match submission order)
**Required:** Response array must match submission array indices
**Impact:** Loop's syncIdentifier→objectId cache mapping will break if order changes
#### Issue 2: Response Ordering Must Be Preserved ✅ COMPLETED
**Previous:** Results accumulated in callback order (might not match submission order)
**Implemented:** All bulk operations use `ordered: true`
**Impact:** Response array indices now guaranteed to match submission array indices
#### Issue 3: Write Result Format Translation
**Current:** Direct MongoDB write result exposed to clients?