ProcessAlarm.skip_event() read event.alarmId on every EventClass.ALARM event,
but LidMalfunctionActivated (a sibling of LidAlarmActivated in that class) has
no alarmId, so a malfunction alarm crashed the sync with AttributeError.
- Narrow with isinstance before reading alarmId; malfunction events now upload
as "Malfunction" as intended, and sync continues.
- Type the alarm handlers against an explicit AlarmEvent union and add an
assert_never exhaustiveness guard, so a type checker rejects unguarded
subtype attribute access and flags any newly added ALARM event type.
- Fix a latent None-leak: alarm_to_nsentry now always returns a dict.
- Add mypy as a gradual-typing beachhead (setup.cfg [mypy], CI step, Pipfile
typecheck script), scoped to process_alarm.py. This configuration fails on
exactly the #156 class of bug.
- Add regression tests: malfunction processing, mixed alarm batches, the event
shape, and an AlarmEvent/EventClass.ALARM sync guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LpMTd5LzkqFLZTUKd3H8em
Patching autoupdate.time.time patches the global time.time, which
logging calls internally per record; a finite side_effect list gets
exhausted and raises StopIteration on Python 3.11. Use return_value.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
get() retries only HTTP 401 and 500, so any other API error propagates out of
the autoupdate loop and exits the process. When Tandem retired the
reportsfacade endpoints and pumpeventmetadata began returning 404 (#146), a
container with a restart policy would crash-loop. That is the worst possible
response to an API outage: the credentials cache dies with the process, so
every restart performs a full login against sso.tandemdiabetes.com. In my EU
deployment that was a fresh login roughly every two minutes for hours from a
single IP, which seems a good way to earn a WAF ban while already broken.
Transient network errors (DNS failures, timeouts, mid-stream disconnects,
urllib3 retry-budget exhaustion) have the same problem.
This keeps both failure families inside the loop and backs off exponentially:
30s doubling to a cap of AUTOUPDATE_DEFAULT_SLEEP_SECONDS (300s default),
reset on any successful poll. The cap reuses the existing poll interval, so a
failing API is never contacted more often than a healthy one. After three
consecutive failures the log escalates from WARNING to ERROR.
Staying alive forever would make a real outage silent on deployments whose
only alarm is the container dying, so after AUTOUPDATE_API_FAILURE_MINUTES
(default 45) of unbroken failure the process gives up and exits non-zero.
That is roughly one restart per hour during a genuine outage instead of one
every two minutes, while short blips stay silent. Set 0 to disable.
This is deliberately not gated on AUTOUPDATE_RESTART_ON_FAILURE, which covers
the pump-not-uploading watchdog where restarting achieves nothing (as the
existing TODO notes) and which many users therefore disable. An unreachable
API is a different failure and gets its own knob.
ApiLoginException stays fatal: bad credentials are not transient, and
retrying them in-process would hammer the login endpoint with attempts that
cannot succeed.
Also included:
- A defensive clamp so a negative rolling-average entry can never reach
time.sleep() and crash with ValueError.
- Tests covering the backoff sequence, reset-on-success, the sustained-failure
exit, the opt-out, and that login failures and programming errors still
propagate.
- README documentation for all nine AUTOUPDATE_* variables, none of which were
documented outside secret.py.
TConnectApi and TandemSourceApi defaulted their region parameter to a
hardcoded 'US', so callers that construct them without an explicit
region (e.g. tconnectsync-heroku's check_login path) sent EU accounts
to the US login endpoint, which rejected them with HTTP 401
account/invalid_credentials even when TCONNECT_REGION=EU was set.
Both classes (and fetch_oneshot) now fall back to the configured
TCONNECT_REGION when no region argument is given, so downstream
consumers honor the .env/environment configuration.
Adds top-level integration tests that configure the EU region via each
supported mechanism (TCONNECT_REGION environment variable, .env file,
and --region CLI flag) and drive the real downstream code -- including
the full main(['--check-login']) entrypoint -- against a mocked HTTP
layer with a real RS256-signed OIDC id_token. Only the EU endpoints are
registered, and the tests assert the EU login/token/jwks/pumper/
pump-logs endpoints are actually invoked and that no US host is ever
contacted.
Fixes#152
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DEvvZSWHo2dki5h1HikUU
Drive the real ProcessTimeRange + process_* handlers and the real
TandemSourceApi / NightscoutApi clients, mocking only the HTTP transport
(base_session / requests). Tandem responses are a small representative slice of
real captured pump-log events (verbatim, deviceAssignmentId redacted); tests
assert the exact Nightscout operations produced: full multi-type sync, api-secret
header, resume-alarm skip, dedup, pretend mode, and empty window.
Copy the exact eventSchema.json to events.json (camelCase keys) and fix
build_events.py's fieldNameFormat to preserve camelCase instead of collapsing
it via .title() (which turned schema keys like commandedRate into commandedrate).
Regenerate events.py so attributes are clean camelCase (commandedRate,
currentGlucoseDisplayValue, egvTimeStamp, bolusId, ...); acronyms follow the
schema's own casing (bg, iob, rssi). Update the battery transform and all
attribute references in the process handlers and tests to match.
Real Dexcom G6 (LidCgmDataGxb, eventCode 256) pump-logs JSON readings captured
from a live t:slim X2 account, exercising the production Events -> ProcessCGMReading
path (the existing G6 coverage used only the binary decoder). Covers steady/rising/
falling/high readings plus a SpecialLow (raw display 0 -> LOW sentinel 39).
Covers regular, extended (combo), and canceled boluses using complete event
groups captured verbatim from the live Tandem Source API (all messages of each
bolusId; deviceAssignmentId redacted), embedded inline as test class variables.
- process_basal / process_user_mode: use timedelta.total_seconds() instead of
.seconds so durations spanning >=24h (and negative deltas) are correct.
- process_cartridge: report cartridge fill from insulinVolume (v2Volume is 0 on
real pumps); treat tubing primeSize -1 as 'not recorded'; format cannula
primeSize with %.1f instead of %d.
- process_bolus: no longer drop the extended portion of a combo bolus. The
initial portion is emitted as before; the extended portion (LidBolexCompleted)
is added as a separate treatment at its completion time.
- check.py: return after a config ImportError instead of falling through to an
unbound-name NameError.
One test file per event type observed in captured Tandem Source
pump-logs responses (30 types), each asserting parse/dispatch,
field round-trip, and enum/bitmask/ratio resolution against real
capture fixtures.
Mirrors pump_events(): pages get_pump_logs across date windows and dedupes
clockChanges (LID_TIME_CHANGED/LID_DATE_CHANGED, codes 13/14) by
(sequenceGroup, sequenceNumber), returning them parsed via Events().
Delete the PumpMetadata TypedDict, _bff_pump_to_metadata and pump_metadata
transform layer. Callers now consume the raw BffPump dicts from get_pumper()
directly. The pump-local -> UTC date conversion is kept as a shared
naive_local_to_utc() helper, applied only at the call sites that compare a
pump date against real UTC (choose_device staleness/selection, autoupdate
timing).
Also expand pump_events JSON parse coverage: drive bolus (20), basal (279),
CGM (399) and alarm (5) events through pump_events(), asserting decoded
fields and enum members (previously only eventCode 16 was covered).
maxDateOfEvents and availableDataRange.start are pump-local naive
wall-clock strings, but consumers parse them with arrow.get() (assumes
UTC) and compare against arrow.utcnow()/time.time(). Normalize both to
true UTC at the adapter boundary via _naive_local_to_utc (interpreting
the naive value in TIMEZONE_NAME), fixing the off-by-offset staleness
warning and update-timing telemetry.
Split BffPump into a required base plus a total=False extension so the
always-present fields are typed required, and read algorithm (canonically
optional) via .get() to avoid a KeyError.
The bff pump-logs endpoint gives glucoseValueStatus + a raw display value; a
below/above-range reading (e.g. status SpecialLow with displayValue 38) is a
boundary indicator, not a measurement. Mirror the Tandem Source frontend
(CgmBuilder.determineGlucoseValue): map SpecialLow/precise<40 -> 39 and
SpecialHigh/precise>400 -> 401. Each sensor (G7/G6/FSL2/FSL3) is resolved
against its own glucoseValueStatus enum members rather than assuming the enums
are consistent across sensor types.
Also widen the real-JSON CGM tests to span glucose 38..361 (incl. the LOW
sentinel) and add a ProcessTimeRange basal JSON integration test.
Event(x) and Events(x) now accept either a raw binary event/stream or a
pump-logs JSON event dict / iterable of dicts, dispatching on input type,
replacing the separate Event_from_json/Events_from_json functions. Point
pump_events() and all tests at the unified entry points, and add real-JSON
sync tests for CGM readings, user-mode sleep/exercise, and alarms alongside
the existing binary-fixture tests.
Exercise Events_from_json -> ProcessCGMReading with real LID_CGM_DATA_G7
pump-logs events captured from a live account (device id redacted), asserting
sgv, egv-derived dateString, pump_event_id, and last-upload skip behavior.
The existing binary-fixture tests are kept alongside.
UpdateProfiles already sources settings.details via pump_metadata(); add
end-to-end compare_profiles tests over a real-shape PumpSettings confirming
the per-segment basal/carbratio/sens schedule, flat-cgm target_low/high, and
defaultProfile are translated correctly, and that a matching Nightscout
profile yields no change.
The settings.details blob differs from the old lastUpload.settings: profile
segments live under timeDependentSegments (was tDependentSegs), cgmSettings
is flat (highGlucoseAlertMgPerDl/lowGlucoseAlertMgPerDl, not nested per-alert
objects), and carbEntry is a string enum. Rework the dataclasses to the new
shape (verified against live t:slim X2 and Mobi accounts), keep a
tDependentSegs alias for segment consumers, and point tandemsource_profile_store
at the flat cgm fields. Values remain milliunit-scaled, so the NS translation
math is unchanged.
Live probes against t:slim X2 and Mobi accounts confirm the pump-logs
endpoint never returns event 81 (LID_DAILY_BASAL) and no returned event
carries battery data, so DEVICE_STATUS yields nothing on the new API (it
degrades gracefully). The endpoint also ignores the eventIds filter and
returns every event in the window; filtering is effectively client-side.
Event 81 (LidDailyBasal, battery) is not in Tandem's default id list; the
pump-logs endpoint may not return it. DEVICE_STATUS already fetches all
event types, and no-daily-basal-event already returns nothing — add a test
pinning that, plus a guard so an event 81 that arrives without battery
fields is skipped with a warning instead of raising on the percent math.
Fetch pre-parsed JSON events from get_pump_logs instead of decoding the
retired reportsfacade binary stream:
- page the requested range into inclusive windows of at most 28 days
(the endpoint caps each request at ~4 weeks), covering short ranges and
single days correctly
- dedupe events that span windows by (sequenceGroup, sequenceNumber)
- count but skip clockChanges (LID_TIME_CHANGED/LID_DATE_CHANGED are not
consumed by any processor)
- parse via Events_from_json
Remove the dead reportsfacade pump_events_raw() and the now-unused
pump_event_metadata()/PumpEventMetadata/LastUpload types.
Verify against real captured pump-logs events that build_from_json:
- resolves enum/dictionary fields from their raw ints (commandedRateSource,
alarmId, requestedAction, previousUserMode, glucoseValueStatus incl. 0)
- scales ratio fields (rate x0.1)
- folds bitmask arrays of set-bit indices back to the raw int the IntFlag
expects (activeSleepSchedule [0]->1, cgmDataType [0]->1,
egvInfoBitmask [0,5,6,7,8,11,12]->6625), and empty array->0
- keeps the raw.timestamp shim used by process_device_status and the raw
egvTimeStamp seconds used by ProcessCGMReading
Extend build_events.py (the codemod) so events.json stays the source of
truth: each generated event class gains build_from_json(event), mapping a
pump-logs JSON event's eventProperties onto its {field}[Raw] attrs
(matched by normalized field name), plus RawEvent.build_from_json() which
derives timestampRaw from pumpDateTime so eventTimestamp/seqNum/eventId
keep working. Add generic.Event_from_json()/Events_from_json() dispatchers
mirroring Event()/Events(); unknown eventCode -> bare RawEvent.
Enum/dictionary/ratio fields are raw ints (map directly onto {field}Raw so
existing properties resolve); bitmask fields arrive as arrays of set-bit
indices, converted to the raw int via _bitmask_arr_to_int. events.py is
regenerated; the byte-path build() is unchanged.
Tests: tests/eventparser/test_from_json.py.
Implement TandemSourceApi.get_pump_logs(device_id, ...) for GET
api/reports/bff/pump-logs/{assignmentId}?pumperId&startDate&endDate&eventIds,
the JSON events endpoint that replaces the base64 reportsfacade/pumpevents.
startDate/endDate are sent as {ymd}T00:00:00Z / {ymd}T23:59:59Z per the
captured request. Adds typed PumpLogsResponse/PumpLogEvent TypedDicts,
verified to cover every key across all captured pump-logs responses.
Tests: URL/path/param construction (order-independent parse_qs), default
vs custom vs empty eventIds, empty query dict passthrough, return
passthrough, and None-dates-default-to-today.
The device-id flow now carries the UUID assignmentId (as deviceId) end to
end; rename the leftover tconnectDeviceId local in check.py to match. The
deviceId flow is exercised by the existing choose_device, process and
update_profiles tests.
- tests/sync/tandemsource/test_update_profiles.py: settings sourcing from
pump_metadata() — matching deviceId with settings reaches
PumpSettings.from_dict (proven via sentinel, using the real BFF
settings.details shape); settings=None / no-match / empty return False
without parsing.
- test_tandemsource.py: adapter edge cases (availableDataRange/settings
keys absent, missing required key raises KeyError, Mobi Control-IQ+
passthrough) and DEFAULT_EVENT_IDS regression (55 ids, no dupes,
477/480/486 present).
Fixtures are inline and trimmed from the real captured responses.
Switch choose_device, check, update_profiles, process.py and cli_helpers
from the old pump_event_metadata() (reportsfacade) to the normalized
pump_metadata() (BFF): tconnectDeviceId -> deviceId (UUID), and
lastUpload.settings -> settings.
Handle the BFF returning never-uploaded pumps (maxDateWithEvents=None):
skip them in the most-recent auto-select and fall back to the first
pump; raise a clear NoDevicesFound on an empty account instead of an
opaque TypeError. Clean stale comments/docstrings in tandemsource.py.
Tests: add tests/sync/tandemsource/test_choose_device.py (11 cases:
explicit/auto/never-uploaded/empty/InvalidSerialNumber/stale-warning)
and tests/api/test_tandemsource.py (pump_metadata adapter mapping),
and update the test_process fixture to the new keys.
Introduce a typed PumpMetadata TypedDict and pump_metadata()/
_bff_pump_to_metadata() that adapt get_pumper().pumps[] into the stable
shape the sync code needs. Maps the new BFF fields to normalized names:
assignmentId -> deviceId (UUID), maxDateOfEvents -> maxDateWithEvents,
availableDataRange.start -> minDateWithEvents, settings.details ->
settings. Verified against the captured account response (7 pumps).
Additive only; consumers are migrated off pump_event_metadata() in a
follow-up commit.
Add TandemSourceApi.get_pumper() for GET api/reports/bff/pumper/{pumperId},
the new device-list endpoint that replaces pumpeventmetadata. Adds
strictly-typed TypedDicts (BffPumper, BffPump, AvailableDataRange,
PumpSettingsEnvelope) derived from the captured account response; verified
they exactly cover the real JSON keys. pumps[].assignmentId is the UUID
device id for the pump-logs endpoint; settings.details (typed as dict for
now) will be modeled by PumpSettings in a later step. Nullable/absent
fields use total=False + Optional.
After the client_id change, extract_jwt still validates the id_token
audience against TDC_OIDC_CLIENT_ID (the OIDC-standard case). But since
the token exchange wasn't captured, we can't be certain Tandem sets
aud=client_id on the id_token. If it doesn't, fall back to decoding with
verify_aud disabled (signature + issuer still verified) and log a
warning, rather than failing login outright.
The Tandem Source web app now authenticates with client_id
0oa4wnbvtladeyVZX4h7 (US); the old 0oa27ho9tpZE9Arjy4h7 no longer
appears in the current build. This value is used both for the OIDC
authorize/token requests and the id_token audience check, which stay
consistent. EU client_id left unchanged (no EU capture to verify).
Match the Tandem Source web app's getLogIDList() as observed in the live
GET api/reports/bff/pump-logs request (from the captured HAR). Adds the
FSL3 event ids 477 (join), 480 (data), 486 (stop); reorders to match the
frontend. No removals (was 52 ids, now 55).
Annotate the 11 event processors, the ProcessTimeRange orchestrator, and
UpdateProfiles. The shared processor interface is now typed:
__init__(tconnect, nightscout, tconnect_device_id, pretend, features),
enabled() -> bool, process(events, time_start, time_end) -> List[dict],
write(ns_entries) -> int. Converter helpers return Optional[dict] since
they fall through to None on type mismatch.
ProcessTimeRange.process() returns Tuple[int, Optional[int]] and its
tconnectDevice param reuses the PumpEventMetadata TypedDict. Client
params (TConnectApi/NightscoutApi/BaseEvent) use TYPE_CHECKING-guarded
imports with string forward refs to avoid import cycles at runtime.
Annotate NightscoutApi methods and the module-level date helpers. Adds a
DateLike alias (str | datetime | arrow.Arrow) for the timestamp/filter
params. Writers return None; the last_uploaded_* getters return
Optional[dict]; api_status/current_profile return dict.
Response shapes are kept as loose dict/Optional[dict] rather than
TypedDicts since the Nightscout API shape varies across versions; only
inputs (which we control) are tightly typed.
Annotate method signatures and dynamic JSON response types. Adds
TypedDicts for the responses whose shapes are confirmed from call sites
and logs:
- PumpEventMetadata (+ nested LastUpload) for pump_event_metadata()
- JwtClaims for the decoded id_token stored on jwtData; pumperId and
accountId are UUID strings, not ints
pump_events_raw() returns a base64 str (not Any). pumper_info() stays
Any since it has no callers and its shape is never logged. TypedDict is
imported with a typing_extensions fallback for Python 3.7.
Since the 2.0 migration to Tandem Source, the live sync path
(api.tandemsource + sync/tandemsource/*) no longer references the
legacy t:connect APIs. This removes that now-unreachable code.
Removed modules:
- api/controliq.py, api/ws2.py, api/android.py, api/webui.py
(the legacy controliq / tconnectws2 / android / webui clients)
- process.py (old process_time_range; already broken since it
imported sync submodules that no longer exist)
- parser/ciq_therapy_events.py, parser/tconnect.py (TConnectEntry)
- domain/therapy_event.py, domain/bolus.py, domain/device_settings.py,
domain/utility.py
Trimmed dead wiring from live modules:
- api/__init__.py: dropped the controliq/ws2/android/webui properties,
keeping only the tandemsource accessor
- check.py: removed the unused TConnectEntry import
- parser/nightscout.py: removed the unused legacy profile_store() plus
the now-orphaned tandem_to_ns_time / tandem_to_ns_time_seconds helpers
and InvalidTimeException (the live path uses tandemsource_profile_store)
Tests: removed suites covering the deleted modules; pared tests/api/fake.py
down to the TConnectApi fake still used by the tandemsource tests. README
"Tandem APIs" sections updated to reflect the single Tandem Source API.
Full test suite passes (48 passed, 1 skipped).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQNn3mBG1kXTAdQb9c2jfW
api_headers() hardcoded Origin/Referer to tconnect.tandemdiabetes.com,
but requests target SOURCE_URL (source.tandemdiabetes.com /
source.eu.tandemdiabetes.com). The WAF enforces same-origin and
returned HTTP 403 ("The request is blocked"). Derive Origin/Referer
from SOURCE_URL so both US and EU regions match.
ClaudeandJames Woglom
xannasavinandJames Woglom
Beshoy GirgisandJames Woglom