Drive the real ProcessTimeRange + process_* handlers and the real
TandemSourceApi / NightscoutApi clients, mocking only the HTTP transport
(base_session / requests). Tandem responses are a small representative slice of
real captured pump-log events (verbatim, deviceAssignmentId redacted); tests
assert the exact Nightscout operations produced: full multi-type sync, api-secret
header, resume-alarm skip, dedup, pretend mode, and empty window.
Copy the exact eventSchema.json to events.json (camelCase keys) and fix
build_events.py's fieldNameFormat to preserve camelCase instead of collapsing
it via .title() (which turned schema keys like commandedRate into commandedrate).
Regenerate events.py so attributes are clean camelCase (commandedRate,
currentGlucoseDisplayValue, egvTimeStamp, bolusId, ...); acronyms follow the
schema's own casing (bg, iob, rssi). Update the battery transform and all
attribute references in the process handlers and tests to match.
Real Dexcom G6 (LidCgmDataGxb, eventCode 256) pump-logs JSON readings captured
from a live t:slim X2 account, exercising the production Events -> ProcessCGMReading
path (the existing G6 coverage used only the binary decoder). Covers steady/rising/
falling/high readings plus a SpecialLow (raw display 0 -> LOW sentinel 39).
Covers regular, extended (combo), and canceled boluses using complete event
groups captured verbatim from the live Tandem Source API (all messages of each
bolusId; deviceAssignmentId redacted), embedded inline as test class variables.
- process_basal / process_user_mode: use timedelta.total_seconds() instead of
.seconds so durations spanning >=24h (and negative deltas) are correct.
- process_cartridge: report cartridge fill from insulinVolume (v2Volume is 0 on
real pumps); treat tubing primeSize -1 as 'not recorded'; format cannula
primeSize with %.1f instead of %d.
- process_bolus: no longer drop the extended portion of a combo bolus. The
initial portion is emitted as before; the extended portion (LidBolexCompleted)
is added as a separate treatment at its completion time.
- check.py: return after a config ImportError instead of falling through to an
unbound-name NameError.
One test file per event type observed in captured Tandem Source
pump-logs responses (30 types), each asserting parse/dispatch,
field round-trip, and enum/bitmask/ratio resolution against real
capture fixtures.
Mirrors pump_events(): pages get_pump_logs across date windows and dedupes
clockChanges (LID_TIME_CHANGED/LID_DATE_CHANGED, codes 13/14) by
(sequenceGroup, sequenceNumber), returning them parsed via Events().
Delete the PumpMetadata TypedDict, _bff_pump_to_metadata and pump_metadata
transform layer. Callers now consume the raw BffPump dicts from get_pumper()
directly. The pump-local -> UTC date conversion is kept as a shared
naive_local_to_utc() helper, applied only at the call sites that compare a
pump date against real UTC (choose_device staleness/selection, autoupdate
timing).
Also expand pump_events JSON parse coverage: drive bolus (20), basal (279),
CGM (399) and alarm (5) events through pump_events(), asserting decoded
fields and enum members (previously only eventCode 16 was covered).
maxDateOfEvents and availableDataRange.start are pump-local naive
wall-clock strings, but consumers parse them with arrow.get() (assumes
UTC) and compare against arrow.utcnow()/time.time(). Normalize both to
true UTC at the adapter boundary via _naive_local_to_utc (interpreting
the naive value in TIMEZONE_NAME), fixing the off-by-offset staleness
warning and update-timing telemetry.
Split BffPump into a required base plus a total=False extension so the
always-present fields are typed required, and read algorithm (canonically
optional) via .get() to avoid a KeyError.
The bff pump-logs endpoint gives glucoseValueStatus + a raw display value; a
below/above-range reading (e.g. status SpecialLow with displayValue 38) is a
boundary indicator, not a measurement. Mirror the Tandem Source frontend
(CgmBuilder.determineGlucoseValue): map SpecialLow/precise<40 -> 39 and
SpecialHigh/precise>400 -> 401. Each sensor (G7/G6/FSL2/FSL3) is resolved
against its own glucoseValueStatus enum members rather than assuming the enums
are consistent across sensor types.
Also widen the real-JSON CGM tests to span glucose 38..361 (incl. the LOW
sentinel) and add a ProcessTimeRange basal JSON integration test.
Event(x) and Events(x) now accept either a raw binary event/stream or a
pump-logs JSON event dict / iterable of dicts, dispatching on input type,
replacing the separate Event_from_json/Events_from_json functions. Point
pump_events() and all tests at the unified entry points, and add real-JSON
sync tests for CGM readings, user-mode sleep/exercise, and alarms alongside
the existing binary-fixture tests.
Exercise Events_from_json -> ProcessCGMReading with real LID_CGM_DATA_G7
pump-logs events captured from a live account (device id redacted), asserting
sgv, egv-derived dateString, pump_event_id, and last-upload skip behavior.
The existing binary-fixture tests are kept alongside.
UpdateProfiles already sources settings.details via pump_metadata(); add
end-to-end compare_profiles tests over a real-shape PumpSettings confirming
the per-segment basal/carbratio/sens schedule, flat-cgm target_low/high, and
defaultProfile are translated correctly, and that a matching Nightscout
profile yields no change.
The settings.details blob differs from the old lastUpload.settings: profile
segments live under timeDependentSegments (was tDependentSegs), cgmSettings
is flat (highGlucoseAlertMgPerDl/lowGlucoseAlertMgPerDl, not nested per-alert
objects), and carbEntry is a string enum. Rework the dataclasses to the new
shape (verified against live t:slim X2 and Mobi accounts), keep a
tDependentSegs alias for segment consumers, and point tandemsource_profile_store
at the flat cgm fields. Values remain milliunit-scaled, so the NS translation
math is unchanged.
Live probes against t:slim X2 and Mobi accounts confirm the pump-logs
endpoint never returns event 81 (LID_DAILY_BASAL) and no returned event
carries battery data, so DEVICE_STATUS yields nothing on the new API (it
degrades gracefully). The endpoint also ignores the eventIds filter and
returns every event in the window; filtering is effectively client-side.
Event 81 (LidDailyBasal, battery) is not in Tandem's default id list; the
pump-logs endpoint may not return it. DEVICE_STATUS already fetches all
event types, and no-daily-basal-event already returns nothing — add a test
pinning that, plus a guard so an event 81 that arrives without battery
fields is skipped with a warning instead of raising on the percent math.
Fetch pre-parsed JSON events from get_pump_logs instead of decoding the
retired reportsfacade binary stream:
- page the requested range into inclusive windows of at most 28 days
(the endpoint caps each request at ~4 weeks), covering short ranges and
single days correctly
- dedupe events that span windows by (sequenceGroup, sequenceNumber)
- count but skip clockChanges (LID_TIME_CHANGED/LID_DATE_CHANGED are not
consumed by any processor)
- parse via Events_from_json
Remove the dead reportsfacade pump_events_raw() and the now-unused
pump_event_metadata()/PumpEventMetadata/LastUpload types.
Verify against real captured pump-logs events that build_from_json:
- resolves enum/dictionary fields from their raw ints (commandedRateSource,
alarmId, requestedAction, previousUserMode, glucoseValueStatus incl. 0)
- scales ratio fields (rate x0.1)
- folds bitmask arrays of set-bit indices back to the raw int the IntFlag
expects (activeSleepSchedule [0]->1, cgmDataType [0]->1,
egvInfoBitmask [0,5,6,7,8,11,12]->6625), and empty array->0
- keeps the raw.timestamp shim used by process_device_status and the raw
egvTimeStamp seconds used by ProcessCGMReading
Extend build_events.py (the codemod) so events.json stays the source of
truth: each generated event class gains build_from_json(event), mapping a
pump-logs JSON event's eventProperties onto its {field}[Raw] attrs
(matched by normalized field name), plus RawEvent.build_from_json() which
derives timestampRaw from pumpDateTime so eventTimestamp/seqNum/eventId
keep working. Add generic.Event_from_json()/Events_from_json() dispatchers
mirroring Event()/Events(); unknown eventCode -> bare RawEvent.
Enum/dictionary/ratio fields are raw ints (map directly onto {field}Raw so
existing properties resolve); bitmask fields arrive as arrays of set-bit
indices, converted to the raw int via _bitmask_arr_to_int. events.py is
regenerated; the byte-path build() is unchanged.
Tests: tests/eventparser/test_from_json.py.
Implement TandemSourceApi.get_pump_logs(device_id, ...) for GET
api/reports/bff/pump-logs/{assignmentId}?pumperId&startDate&endDate&eventIds,
the JSON events endpoint that replaces the base64 reportsfacade/pumpevents.
startDate/endDate are sent as {ymd}T00:00:00Z / {ymd}T23:59:59Z per the
captured request. Adds typed PumpLogsResponse/PumpLogEvent TypedDicts,
verified to cover every key across all captured pump-logs responses.
Tests: URL/path/param construction (order-independent parse_qs), default
vs custom vs empty eventIds, empty query dict passthrough, return
passthrough, and None-dates-default-to-today.
The device-id flow now carries the UUID assignmentId (as deviceId) end to
end; rename the leftover tconnectDeviceId local in check.py to match. The
deviceId flow is exercised by the existing choose_device, process and
update_profiles tests.
- tests/sync/tandemsource/test_update_profiles.py: settings sourcing from
pump_metadata() — matching deviceId with settings reaches
PumpSettings.from_dict (proven via sentinel, using the real BFF
settings.details shape); settings=None / no-match / empty return False
without parsing.
- test_tandemsource.py: adapter edge cases (availableDataRange/settings
keys absent, missing required key raises KeyError, Mobi Control-IQ+
passthrough) and DEFAULT_EVENT_IDS regression (55 ids, no dupes,
477/480/486 present).
Fixtures are inline and trimmed from the real captured responses.
Switch choose_device, check, update_profiles, process.py and cli_helpers
from the old pump_event_metadata() (reportsfacade) to the normalized
pump_metadata() (BFF): tconnectDeviceId -> deviceId (UUID), and
lastUpload.settings -> settings.
Handle the BFF returning never-uploaded pumps (maxDateWithEvents=None):
skip them in the most-recent auto-select and fall back to the first
pump; raise a clear NoDevicesFound on an empty account instead of an
opaque TypeError. Clean stale comments/docstrings in tandemsource.py.
Tests: add tests/sync/tandemsource/test_choose_device.py (11 cases:
explicit/auto/never-uploaded/empty/InvalidSerialNumber/stale-warning)
and tests/api/test_tandemsource.py (pump_metadata adapter mapping),
and update the test_process fixture to the new keys.
Introduce a typed PumpMetadata TypedDict and pump_metadata()/
_bff_pump_to_metadata() that adapt get_pumper().pumps[] into the stable
shape the sync code needs. Maps the new BFF fields to normalized names:
assignmentId -> deviceId (UUID), maxDateOfEvents -> maxDateWithEvents,
availableDataRange.start -> minDateWithEvents, settings.details ->
settings. Verified against the captured account response (7 pumps).
Additive only; consumers are migrated off pump_event_metadata() in a
follow-up commit.
Add TandemSourceApi.get_pumper() for GET api/reports/bff/pumper/{pumperId},
the new device-list endpoint that replaces pumpeventmetadata. Adds
strictly-typed TypedDicts (BffPumper, BffPump, AvailableDataRange,
PumpSettingsEnvelope) derived from the captured account response; verified
they exactly cover the real JSON keys. pumps[].assignmentId is the UUID
device id for the pump-logs endpoint; settings.details (typed as dict for
now) will be modeled by PumpSettings in a later step. Nullable/absent
fields use total=False + Optional.
After the client_id change, extract_jwt still validates the id_token
audience against TDC_OIDC_CLIENT_ID (the OIDC-standard case). But since
the token exchange wasn't captured, we can't be certain Tandem sets
aud=client_id on the id_token. If it doesn't, fall back to decoding with
verify_aud disabled (signature + issuer still verified) and log a
warning, rather than failing login outright.
The Tandem Source web app now authenticates with client_id
0oa4wnbvtladeyVZX4h7 (US); the old 0oa27ho9tpZE9Arjy4h7 no longer
appears in the current build. This value is used both for the OIDC
authorize/token requests and the id_token audience check, which stay
consistent. EU client_id left unchanged (no EU capture to verify).
Match the Tandem Source web app's getLogIDList() as observed in the live
GET api/reports/bff/pump-logs request (from the captured HAR). Adds the
FSL3 event ids 477 (join), 480 (data), 486 (stop); reorders to match the
frontend. No removals (was 52 ids, now 55).
Annotate the 11 event processors, the ProcessTimeRange orchestrator, and
UpdateProfiles. The shared processor interface is now typed:
__init__(tconnect, nightscout, tconnect_device_id, pretend, features),
enabled() -> bool, process(events, time_start, time_end) -> List[dict],
write(ns_entries) -> int. Converter helpers return Optional[dict] since
they fall through to None on type mismatch.
ProcessTimeRange.process() returns Tuple[int, Optional[int]] and its
tconnectDevice param reuses the PumpEventMetadata TypedDict. Client
params (TConnectApi/NightscoutApi/BaseEvent) use TYPE_CHECKING-guarded
imports with string forward refs to avoid import cycles at runtime.
Annotate NightscoutApi methods and the module-level date helpers. Adds a
DateLike alias (str | datetime | arrow.Arrow) for the timestamp/filter
params. Writers return None; the last_uploaded_* getters return
Optional[dict]; api_status/current_profile return dict.
Response shapes are kept as loose dict/Optional[dict] rather than
TypedDicts since the Nightscout API shape varies across versions; only
inputs (which we control) are tightly typed.
Annotate method signatures and dynamic JSON response types. Adds
TypedDicts for the responses whose shapes are confirmed from call sites
and logs:
- PumpEventMetadata (+ nested LastUpload) for pump_event_metadata()
- JwtClaims for the decoded id_token stored on jwtData; pumperId and
accountId are UUID strings, not ints
pump_events_raw() returns a base64 str (not Any). pumper_info() stays
Any since it has no callers and its shape is never logged. TypedDict is
imported with a typing_extensions fallback for Python 3.7.
api_headers() hardcoded Origin/Referer to tconnect.tandemdiabetes.com,
but requests target SOURCE_URL (source.tandemdiabetes.com /
source.eu.tandemdiabetes.com). The WAF enforces same-origin and
returned HTTP 403 ("The request is blocked"). Derive Origin/Referer
from SOURCE_URL so both US and EU regions match.
James Woglom