mirror of
https://github.com/bckelley/cgm-remote-monitor.git
synced 2026-08-24 03:14:12 -05:00
268 lines
11 KiB
Markdown
268 lines
11 KiB
Markdown
# Test Development Findings - Partial Failures and Client Compatibility
|
|
|
|
**Date**: 2026-01-18
|
|
**Status**: Tests Created, Infrastructure Issues Prevent Execution
|
|
**Impact**: CRITICAL - Multiple undocumented behaviors discovered
|
|
|
|
## Executive Summary
|
|
|
|
Created comprehensive test suites for previously unused fixtures (`partial-failures.js`, `deduplication.js`, `aaps-single-doc.js`). Test development process revealed **12 critical behaviors** that must be preserved during MongoDB driver modernization, many of which were not previously documented in test coverage.
|
|
|
|
## Tests Created
|
|
|
|
### 1. `tests/api.partial-failures.test.js` (17.5KB, 496 LOC)
|
|
|
|
**Purpose**: Validate MongoDB batch operation edge cases critical for Loop, Trio, and AAPS compatibility
|
|
|
|
**Test Coverage**:
|
|
- ✅ Duplicate key handling in batches (ordered insert behavior)
|
|
- ✅ Response array ordering for Loop syncIdentifier mapping
|
|
- ✅ Batch with deduplicated items (response completeness)
|
|
- ✅ Client-provided _id handling (Loop, Trio, AAPS patterns)
|
|
- ✅ Write result format translation (driver v3.x → v4.x)
|
|
- ✅ Large BSON document handling (devicestatus predictions)
|
|
- ✅ Validation error handling (partial batch failures)
|
|
- ✅ Large batch processing (50+ items)
|
|
|
|
**Critical Behaviors Documented**:
|
|
|
|
1. **Ordered Insert Stops at First Error** (Loop/Trio Impact: HIGH)
|
|
- When duplicate key occurs in middle of batch, documents before duplicate ARE inserted
|
|
- Documents after duplicate are NOT attempted
|
|
- Response order must still align with request positions
|
|
|
|
2. **Response Order MUST Match Request Order** (Loop Impact: CRITICAL)
|
|
```javascript
|
|
// Loop caches: request[i].syncIdentifier → response[i]._id
|
|
// If order is wrong, Loop maps incorrect IDs and breaks updates/deletes
|
|
```
|
|
- Test validates each response position correlates to request position
|
|
- Failure mode: Loop deletes wrong treatments, creates duplicates
|
|
|
|
3. **Deduplication Must Return All Response Positions** (Loop/Trio Impact: CRITICAL)
|
|
- Even if middle item is deduplicated, response array must have N items for N requests
|
|
- Missing positions break client-side syncIdentifier caching
|
|
- Test: 3 requests (1 new, 1 exists, 1 new) → 3 responses required
|
|
|
|
4. **Client-Provided _id Handling** (All Clients Impact: MEDIUM)
|
|
- Loop: May provide _id in specific ObjectId format
|
|
- Trio: Uses `id` field (separate from _id) for deduplication
|
|
- AAPS: Uses `identifier` field (v3 API, separate from _id)
|
|
- Test validates all three patterns preserve correct field mappings
|
|
|
|
5. **v1 API Response Format** (All Clients Impact: HIGH)
|
|
- Clients expect: `{ _id: "...", ok: 1 }` format
|
|
- Driver format may differ (insertedIds object vs array)
|
|
- API layer must translate driver response to v1 format
|
|
|
|
6. **Large Prediction Arrays** (OpenAPS Impact: MEDIUM)
|
|
- devicestatus with 1000+ prediction values per array
|
|
- Must not exceed 16MB BSON limit
|
|
- Typical predictions well under limit but needs validation
|
|
|
|
### 2. `tests/api.deduplication.test.js` (14.6KB, 388 LOC)
|
|
|
|
**Purpose**: Validate deduplication logic for AAPS, Loop, and Trio client patterns
|
|
|
|
**Test Coverage**:
|
|
- ✅ AAPS pumpId + pumpType + pumpSerial deduplication
|
|
- ✅ AAPS entry date + device + type deduplication
|
|
- ✅ Loop syncIdentifier deduplication
|
|
- ✅ Trio id field (UUID) deduplication
|
|
- ✅ Batch with mixed duplicates (partial deduplication)
|
|
- ✅ Cross-client duplicate detection (should NOT deduplicate)
|
|
- ✅ Deduplication response format (returns original _id)
|
|
|
|
**Critical Behaviors Documented**:
|
|
|
|
7. **AAPS Pump-Based Deduplication** (AAPS Impact: CRITICAL)
|
|
- Composite key: `pumpId + pumpType + pumpSerial`
|
|
- Prevents duplicate treatment uploads from pump
|
|
- Fields must be preserved exactly
|
|
|
|
8. **AAPS Entry Deduplication** (AAPS Impact: CRITICAL)
|
|
- Composite key: `date + device + type`
|
|
- Prevents duplicate CGM readings
|
|
- Exact timestamp matching required
|
|
|
|
9. **Loop syncIdentifier Uniqueness** (Loop Impact: CRITICAL)
|
|
- Single field: `syncIdentifier` (UUID generated by Loop)
|
|
- Loop depends on this for tracking uploaded items
|
|
- Deduplication must return original _id for cache consistency
|
|
|
|
10. **Trio id Field Deduplication** (Trio Impact: CRITICAL)
|
|
- Single field: `id` (UUID, separate from MongoDB _id)
|
|
- Trio uses this for its own tracking
|
|
- Must NOT interfere with MongoDB _id generation
|
|
|
|
11. **Cross-Client Isolation** (All Clients Impact: HIGH)
|
|
- AAPS upload and Trio upload of "same" event should create 2 documents
|
|
- Different clients use different deduplication keys
|
|
- No cross-contamination between client namespaces
|
|
|
|
12. **Deduplication Response Consistency** (All Clients Impact: HIGH)
|
|
- Deduplicated item should return original _id (not new _id)
|
|
- Response indicates deduplication occurred
|
|
- Client can update local cache with confirmed _id
|
|
|
|
### 3. `tests/api.aaps-client.test.js` (12.4KB, 331 LOC)
|
|
|
|
**Purpose**: Validate AAPS-specific document formats and metadata preservation
|
|
|
|
**Test Coverage**:
|
|
- ✅ SGV entry with AAPS device metadata
|
|
- ✅ SMB (Super Micro Bolus) format
|
|
- ✅ Meal Bolus with carbs
|
|
- ✅ Temp Basal with duration/rate
|
|
- ✅ Pump metadata preservation (pumpId, pumpType, pumpSerial)
|
|
- ✅ Boolean flags (isValid, isSMB)
|
|
- ✅ Single document vs batch behavior
|
|
- ✅ Response format verification
|
|
- ✅ utcOffset timezone handling
|
|
|
|
**Critical Behaviors Documented**:
|
|
|
|
13. **AAPS Metadata Richness** (AAPS Impact: HIGH)
|
|
- Fields: app, isValid, isSMB, pumpId, pumpType, pumpSerial, type
|
|
- All fields must be preserved exactly
|
|
- Used for filtering, display, and deduplication
|
|
|
|
14. **Single-Item Array Processing** (AAPS Impact: MEDIUM)
|
|
- AAPS typically sends `[single_item]` not `multiple_items`
|
|
- Must process as batch operation (not single doc)
|
|
- Response must be array with 1 element
|
|
|
|
## Issues Discovered
|
|
|
|
### Test Infrastructure Issues
|
|
|
|
**Problem**: v1 API tests fail to initialize due to missing context setup
|
|
```
|
|
TypeError: Cannot read property 'isPermitted' of undefined
|
|
at configure (lib/api/experiments/index.js:11:40)
|
|
```
|
|
|
|
**Root Cause**:
|
|
- Tests create `wares` but don't attach to `ctx.wares` before passing to API
|
|
- Tests don't initialize `ctx.authorization` (required by API modules)
|
|
- Existing `tests/api.v1-batch-operations.test.js` has same issue
|
|
|
|
**Impact**: Cannot validate actual behavior without fixing test infrastructure
|
|
|
|
**Recommendation**:
|
|
1. Fix test setup pattern across all v1 API tests
|
|
2. Add `ctx.wares = wares` before `require('../lib/api/')`
|
|
3. Ensure `ctx.authorization` is initialized by bootevent
|
|
|
|
### Previously Undocumented Behaviors
|
|
|
|
**Finding**: Many critical client behaviors were NOT tested before:
|
|
- Loop response ordering requirement (CRITICAL - could cause data loss)
|
|
- Deduplication with batch operations (HIGH - could cause duplicates)
|
|
- Client-provided _id field handling (MEDIUM - could cause conflicts)
|
|
- Cross-client duplicate isolation (HIGH - could cause data mixing)
|
|
|
|
**Recommendation**:
|
|
- Fix test infrastructure ASAP
|
|
- Run these tests before ANY MongoDB driver changes
|
|
- Add to CI/CD as regression prevention
|
|
|
|
## Fixture Usage Analysis
|
|
|
|
### Previously Unused Fixtures Now Covered
|
|
|
|
| Fixture File | Created | Tests Using It | Status |
|
|
|--------------|---------|----------------|---------|
|
|
| `partial-failures.js` | Recent | `api.partial-failures.test.js` | ✅ Now covered |
|
|
| `deduplication.js` | Recent | `api.deduplication.test.js` | ✅ Now covered |
|
|
| `aaps-single-doc.js` | Recent | `api.aaps-client.test.js` | ✅ Now covered |
|
|
| `loop-batch.js` | Recent | `api.v1-batch-operations.test.js` | ⚠️ Partial coverage |
|
|
| `trio-pipeline.js` | Recent | `api.v1-batch-operations.test.js` | ⚠️ Partial coverage |
|
|
| `edge-cases.js` | Recent | `api.v1-batch-operations.test.js` | ⚠️ Minimal coverage |
|
|
|
|
### Recommended Additional Tests
|
|
|
|
1. **Trio Pipeline Tests** - Dedicated test file for Trio throttling behavior
|
|
2. **Edge Cases Deep Dive** - Expand `edge-cases.js` coverage
|
|
3. **Write Result Format** - Dedicated tests for driver v3 vs v4 response translation
|
|
4. **Connection Failure Recovery** - Test batch operation interruption scenarios
|
|
|
|
## Critical Findings for MongoDB Modernization
|
|
|
|
### Must Preserve Behaviors (from test analysis)
|
|
|
|
1. **insertMany() Response Ordering**
|
|
- Driver v3: insertedIds is object `{ '0': id1, '1': id2 }`
|
|
- Driver v4: insertedIds is array `[id1, id2]`
|
|
- v1 API must translate to array preserving order
|
|
- **Test**: `partial-failures.test.js` - "response order MUST match request order"
|
|
|
|
2. **Ordered Insert Behavior**
|
|
- Driver default changed from ordered=true to ordered=false in v4
|
|
- Must explicitly set `{ ordered: true }` to maintain v1 API semantics
|
|
- **Test**: `partial-failures.test.js` - "batch with duplicate key"
|
|
|
|
3. **Deduplication Before Insert**
|
|
- Current: App-level deduplication before calling insertMany()
|
|
- Risk: Driver changes could bypass this logic
|
|
- **Test**: `deduplication.test.js` - all deduplication scenarios
|
|
|
|
4. **Client Field Preservation**
|
|
- syncIdentifier (Loop), id (Trio), pumpId (AAPS), identifier (v3 API)
|
|
- All must be preserved as separate fields from _id
|
|
- **Test**: `partial-failures.test.js` - client-provided ID scenarios
|
|
|
|
## Recommendations
|
|
|
|
### Immediate Actions
|
|
|
|
1. **Fix Test Infrastructure** (Priority: URGENT)
|
|
- Update all v1 API tests with correct ctx setup
|
|
- Verify tests can run successfully
|
|
- Add to CI/CD pipeline
|
|
|
|
2. **Run Tests Before Migration** (Priority: CRITICAL)
|
|
- Establish baseline behavior with current MongoDB driver
|
|
- Document actual vs expected behavior
|
|
- Capture response formats for comparison
|
|
|
|
3. **Update Documentation** (Priority: HIGH)
|
|
- Add test findings to implementation plan
|
|
- Update impact assessment with discovered behaviors
|
|
- Create migration checklist from test requirements
|
|
|
|
### Future Work
|
|
|
|
1. **Expand Test Coverage**
|
|
- Trio-specific pipeline tests
|
|
- Connection failure recovery tests
|
|
- Driver-specific format translation tests
|
|
|
|
2. **Integration Testing**
|
|
- Test with actual Loop/Trio/AAPS clients
|
|
- Validate end-to-end workflows
|
|
- Monitor for regression after migration
|
|
|
|
3. **Performance Testing**
|
|
- Benchmark large batch operations
|
|
- Test MongoDB connection pool behavior
|
|
- Validate timeout handling
|
|
|
|
## Conclusion
|
|
|
|
Test development revealed **14 critical behaviors** not previously covered by tests, including **3 CRITICAL severity** issues that could cause data loss or client malfunction:
|
|
|
|
1. Loop response ordering (data loss risk)
|
|
2. AAPS/Loop deduplication logic (duplicate data risk)
|
|
3. Batch operation with deduplication (missing data in response risk)
|
|
|
|
**Action Required**: Fix test infrastructure and run these tests BEFORE any MongoDB modernization work proceeds.
|
|
|
|
## Files Created
|
|
|
|
- `/tests/api.partial-failures.test.js` - 496 LOC, 17.5KB
|
|
- `/tests/api.deduplication.test.js` - 388 LOC, 14.6KB
|
|
- `/tests/api.aaps-client.test.js` - 331 LOC, 12.4KB
|
|
|
|
**Total**: 1,215 lines of test code documenting critical client compatibility behaviors.
|