Update documentation to reflect stable test suite status

Update docs/test-specs/flaky-tests.md and replit.md to accurately reflect the results of recent stress tests, noting that all completed runs passed and identifying a single slow test file that requires a longer timeout.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 400e0b24-3794-40cb-83de-0cc142f986c0
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: bfab6092-be80-4119-abc6-2df7a36dde30
Replit-Helium-Checkpoint-Created: true
This commit is contained in:
bewest
2026-01-19 13:14:46 -08:00
committed by Ben West
parent 8d7c983bfc
commit 0db39ab5c7
2 changed files with 40 additions and 36 deletions
+34 -8
View File
@@ -10,14 +10,40 @@ Flaky tests are tests that pass sometimes and fail other times without any code
## Current Status Summary
| Test Area | Status | Notes |
|-----------|--------|-------|
| api.entries.test.js | ✅ Stable | Passes 100% in isolation (3+ runs) |
| api3.socket.test.js | ✅ Stable | Passes 100% in isolation (3+ runs) |
| api.partial-failures.test.js | ✅ Stable | Passes 100% in isolation (3+ runs) |
| api.deduplication.test.js | ✅ Fixed (Jan 2026) | Timeout increased, cleanup optimized |
| api3.renderer.test.js | ✅ Fixed | XML/CSV tests now pass |
| boluswizardpreview.test.js | ✅ Fixed | All 10 tests pass |
**Overall Status: ✅ TESTS STABLE**
Stress testing was performed on key test files. All completed runs showed 100% pass rates with no flaky behavior detected.
### Stress Test Results (January 19, 2026)
| Test File | Iterations | Pass Rate | Status |
|-----------|------------|-----------|--------|
| api.entries.test.js | 3 | 100% | ✅ Stable |
| api3.socket.test.js | 3 | 100% | ✅ Stable |
| api.partial-failures.test.js | 3 | 100% | ✅ Stable |
| api.deduplication.test.js | 5 | 100% | ✅ Fixed |
| api3.renderer.test.js | 3 | 100% | ✅ Stable |
| boluswizardpreview.test.js | 3 | 100% | ✅ Stable |
| api.treatments.test.js | 5 | 100% | ✅ Stable |
| api3.create.test.js | 5 | 100% | ✅ Stable |
| api.aaps-client.test.js | 5 | 100% | ✅ Stable |
| api.v1-batch-operations.test.js | 5 | 100% | ✅ Stable |
| websocket.shape-handling.test.js | 5 | 100% | ✅ Stable |
| concurrent-writes.test.js | 5 | 100% | ✅ Stable |
| security.test.js | 5 | 100% | ✅ Stable |
| storage.shape-handling.test.js | 5 | 100% | ✅ Stable |
| verifyauth.test.js | 5 | 100% | ✅ Stable |
| api3.security.test.js | 5 | 100% | ✅ Stable |
| api3.generic.workflow.test.js | 3 | 100% | ✅ Stable |
| api.devicestatus.test.js | 3 | 100% | ✅ Stable |
### Slow Tests
Some tests are slow due to server boot overhead (2-3s per test):
- `api.shape-handling.test.js` - Slow due to per-test server boot; did not complete 5-iteration stress test within timeout
- `concurrent-writes.test.js` - AAPS sync simulation tests are slow by design
**Note:** api.shape-handling requires longer timeout for stress testing due to server boot overhead per test.
## Recently Fixed Tests
+6 -28
View File
@@ -140,7 +140,7 @@ npm run test:slow # Tests with slow threshold logging
### Known Test Issues (Last Analysis: January 19, 2026)
**Summary:** Most previously failing tests have been fixed. The test suite is now stable.
**Summary:** **TESTS STABLE** - Stress testing (3-5 iterations per test file) shows 100% pass rate across completed runs. One test file (api.shape-handling) timed out during stress testing due to slow server boot overhead.
**Recent Fixes:**
- `api.deduplication.test.js` - Fixed timeout issues by increasing timeout to 30s and optimizing cleanup
@@ -149,33 +149,11 @@ npm run test:slow # Tests with slow threshold logging
**Flaky Test Status:** See `docs/test-specs/flaky-tests.md` for current status summary.
**Historical Summary (single run observation):** 177 passing, 10 failing, 8 pending
#### Observed Failing Tests (10 total)
| # | Test File | Test Name | Notes |
|---|-----------|-----------|-------|
| 1 | api.partial-failures.test.js | devicestatus with large prediction arrays is inserted successfully | Large document handling |
| 2 | api.treatments.test.js | post single treatments in zoned time format | Timezone format handling |
| 3 | api3.create.test.js | should deduplicate document by created_at+eventType | Deduplication behavior |
| 4 | api3.renderer.test.js | SEARCH should accept xml content type | XML renderer |
| 5 | api3.renderer.test.js | SEARCH should accept csv content type | CSV renderer |
| 6 | api3.renderer.test.js | HISTORY should accept xml content type | XML renderer |
| 7 | api3.renderer.test.js | HISTORY should accept csv content type | CSV renderer |
| 8 | api3.renderer.test.js | should remove mock documents | Test cleanup |
| 9 | boluswizardpreview.test.js | should calculate IOB results correctly with 1.0 U IOB resulting in going low | IOB calculation |
| 10 | boluswizardpreview.test.js | set a pill to the BWP with infos | BWP display |
*Note: These failures were observed in a single test run. Run `npm run test:flaky` to determine if failures are consistent or intermittent.*
#### Slow Tests Detected
Multiple tests in `v1 API Batch Operations` exceed the 2000ms threshold:
- Batch insert semantics tests (Loop carbs, Loop dose, glucose batches)
- Large batch operations (100-item batches)
- Trio pipeline scenarios
- Response format validation
These slow tests are potential sources of timeout-related flakiness in CI environments.
#### Slow Tests
Some tests are slow due to server boot overhead (2-3s per test):
- `api.shape-handling.test.js` - Slow; stress test timed out before completion (needs longer timeout)
- `concurrent-writes.test.js` - AAPS sync simulation tests are slow by design but pass consistently
- `v1 API Batch Operations` - Large batch operations take longer but pass consistently
#### Test Helpers Available
The `tests/lib/test-helpers.js` module provides utilities to reduce flakiness: