arrow's isoformat() ends a timestamp with its UTC offset, e.g.
'2026-07-16T00:00:00+02:00'. Placed raw into a query string, '+' is a
reserved character that servers decode as a space, so Nightscout receives
'2026-07-16T00:00:00 02:00' and answers 'could not parse as a valid ISO-8601
date'. Any user on a positive UTC offset hits this on every date-filtered
query.
The current workaround retries the request with 'T' replaced by a space
(t_to_space) and treats a failure as 'no previous entry'. That gets a
response, but it works around the symptom: the value is still mangled, every
affected query costs two round-trips, and a genuinely empty result is
indistinguishable from a rejected one.
Percent-encoding the value fixes the cause: the offset survives, the first
request succeeds, and the t_to_space fallback and its retry wrapper are no
longer needed and are removed.
Adds tests for time_range(): that a positive offset is encoded rather than
emitted raw, that the encoded value round-trips back to the original instant,
that negative offsets and 'Z' still parse, and the bounds/no-bounds cases.
There was no coverage of time_range() before. The encoding test fails on
master with '+' present in the query and passes with this change.
Running with this in my EU deployment since May.
ProcessAlarm.skip_event() read event.alarmId on every EventClass.ALARM event,
but LidMalfunctionActivated (a sibling of LidAlarmActivated in that class) has
no alarmId, so a malfunction alarm crashed the sync with AttributeError.
- Narrow with isinstance before reading alarmId; malfunction events now upload
as "Malfunction" as intended, and sync continues.
- Type the alarm handlers against an explicit AlarmEvent union and add an
assert_never exhaustiveness guard, so a type checker rejects unguarded
subtype attribute access and flags any newly added ALARM event type.
- Fix a latent None-leak: alarm_to_nsentry now always returns a dict.
- Add mypy as a gradual-typing beachhead (setup.cfg [mypy], CI step, Pipfile
typecheck script), scoped to process_alarm.py. This configuration fails on
exactly the #156 class of bug.
- Add regression tests: malfunction processing, mixed alarm batches, the event
shape, and an AlarmEvent/EventClass.ALARM sync guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LpMTd5LzkqFLZTUKd3H8em
Patching autoupdate.time.time patches the global time.time, which
logging calls internally per record; a finite side_effect list gets
exhausted and raises StopIteration on Python 3.11. Use return_value.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
get() retries only HTTP 401 and 500, so any other API error propagates out of
the autoupdate loop and exits the process. When Tandem retired the
reportsfacade endpoints and pumpeventmetadata began returning 404 (#146), a
container with a restart policy would crash-loop. That is the worst possible
response to an API outage: the credentials cache dies with the process, so
every restart performs a full login against sso.tandemdiabetes.com. In my EU
deployment that was a fresh login roughly every two minutes for hours from a
single IP, which seems a good way to earn a WAF ban while already broken.
Transient network errors (DNS failures, timeouts, mid-stream disconnects,
urllib3 retry-budget exhaustion) have the same problem.
This keeps both failure families inside the loop and backs off exponentially:
30s doubling to a cap of AUTOUPDATE_DEFAULT_SLEEP_SECONDS (300s default),
reset on any successful poll. The cap reuses the existing poll interval, so a
failing API is never contacted more often than a healthy one. After three
consecutive failures the log escalates from WARNING to ERROR.
Staying alive forever would make a real outage silent on deployments whose
only alarm is the container dying, so after AUTOUPDATE_API_FAILURE_MINUTES
(default 45) of unbroken failure the process gives up and exits non-zero.
That is roughly one restart per hour during a genuine outage instead of one
every two minutes, while short blips stay silent. Set 0 to disable.
This is deliberately not gated on AUTOUPDATE_RESTART_ON_FAILURE, which covers
the pump-not-uploading watchdog where restarting achieves nothing (as the
existing TODO notes) and which many users therefore disable. An unreachable
API is a different failure and gets its own knob.
ApiLoginException stays fatal: bad credentials are not transient, and
retrying them in-process would hammer the login endpoint with attempts that
cannot succeed.
Also included:
- A defensive clamp so a negative rolling-average entry can never reach
time.sleep() and crash with ValueError.
- Tests covering the backoff sequence, reset-on-success, the sustained-failure
exit, the opt-out, and that login failures and programming errors still
propagate.
- README documentation for all nine AUTOUPDATE_* variables, none of which were
documented outside secret.py.
TConnectApi and TandemSourceApi defaulted their region parameter to a
hardcoded 'US', so callers that construct them without an explicit
region (e.g. tconnectsync-heroku's check_login path) sent EU accounts
to the US login endpoint, which rejected them with HTTP 401
account/invalid_credentials even when TCONNECT_REGION=EU was set.
Both classes (and fetch_oneshot) now fall back to the configured
TCONNECT_REGION when no region argument is given, so downstream
consumers honor the .env/environment configuration.
Adds top-level integration tests that configure the EU region via each
supported mechanism (TCONNECT_REGION environment variable, .env file,
and --region CLI flag) and drive the real downstream code -- including
the full main(['--check-login']) entrypoint -- against a mocked HTTP
layer with a real RS256-signed OIDC id_token. Only the EU endpoints are
registered, and the tests assert the EU login/token/jwks/pumper/
pump-logs endpoints are actually invoked and that no US host is ever
contacted.
Fixes#152
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DEvvZSWHo2dki5h1HikUU
Drive the real ProcessTimeRange + process_* handlers and the real
TandemSourceApi / NightscoutApi clients, mocking only the HTTP transport
(base_session / requests). Tandem responses are a small representative slice of
real captured pump-log events (verbatim, deviceAssignmentId redacted); tests
assert the exact Nightscout operations produced: full multi-type sync, api-secret
header, resume-alarm skip, dedup, pretend mode, and empty window.
Copy the exact eventSchema.json to events.json (camelCase keys) and fix
build_events.py's fieldNameFormat to preserve camelCase instead of collapsing
it via .title() (which turned schema keys like commandedRate into commandedrate).
Regenerate events.py so attributes are clean camelCase (commandedRate,
currentGlucoseDisplayValue, egvTimeStamp, bolusId, ...); acronyms follow the
schema's own casing (bg, iob, rssi). Update the battery transform and all
attribute references in the process handlers and tests to match.
Real Dexcom G6 (LidCgmDataGxb, eventCode 256) pump-logs JSON readings captured
from a live t:slim X2 account, exercising the production Events -> ProcessCGMReading
path (the existing G6 coverage used only the binary decoder). Covers steady/rising/
falling/high readings plus a SpecialLow (raw display 0 -> LOW sentinel 39).
Covers regular, extended (combo), and canceled boluses using complete event
groups captured verbatim from the live Tandem Source API (all messages of each
bolusId; deviceAssignmentId redacted), embedded inline as test class variables.
One test file per event type observed in captured Tandem Source
pump-logs responses (30 types), each asserting parse/dispatch,
field round-trip, and enum/bitmask/ratio resolution against real
capture fixtures.
Mirrors pump_events(): pages get_pump_logs across date windows and dedupes
clockChanges (LID_TIME_CHANGED/LID_DATE_CHANGED, codes 13/14) by
(sequenceGroup, sequenceNumber), returning them parsed via Events().
Delete the PumpMetadata TypedDict, _bff_pump_to_metadata and pump_metadata
transform layer. Callers now consume the raw BffPump dicts from get_pumper()
directly. The pump-local -> UTC date conversion is kept as a shared
naive_local_to_utc() helper, applied only at the call sites that compare a
pump date against real UTC (choose_device staleness/selection, autoupdate
timing).
Also expand pump_events JSON parse coverage: drive bolus (20), basal (279),
CGM (399) and alarm (5) events through pump_events(), asserting decoded
fields and enum members (previously only eventCode 16 was covered).
maxDateOfEvents and availableDataRange.start are pump-local naive
wall-clock strings, but consumers parse them with arrow.get() (assumes
UTC) and compare against arrow.utcnow()/time.time(). Normalize both to
true UTC at the adapter boundary via _naive_local_to_utc (interpreting
the naive value in TIMEZONE_NAME), fixing the off-by-offset staleness
warning and update-timing telemetry.
Split BffPump into a required base plus a total=False extension so the
always-present fields are typed required, and read algorithm (canonically
optional) via .get() to avoid a KeyError.
The bff pump-logs endpoint gives glucoseValueStatus + a raw display value; a
below/above-range reading (e.g. status SpecialLow with displayValue 38) is a
boundary indicator, not a measurement. Mirror the Tandem Source frontend
(CgmBuilder.determineGlucoseValue): map SpecialLow/precise<40 -> 39 and
SpecialHigh/precise>400 -> 401. Each sensor (G7/G6/FSL2/FSL3) is resolved
against its own glucoseValueStatus enum members rather than assuming the enums
are consistent across sensor types.
Also widen the real-JSON CGM tests to span glucose 38..361 (incl. the LOW
sentinel) and add a ProcessTimeRange basal JSON integration test.
Event(x) and Events(x) now accept either a raw binary event/stream or a
pump-logs JSON event dict / iterable of dicts, dispatching on input type,
replacing the separate Event_from_json/Events_from_json functions. Point
pump_events() and all tests at the unified entry points, and add real-JSON
sync tests for CGM readings, user-mode sleep/exercise, and alarms alongside
the existing binary-fixture tests.
Exercise Events_from_json -> ProcessCGMReading with real LID_CGM_DATA_G7
pump-logs events captured from a live account (device id redacted), asserting
sgv, egv-derived dateString, pump_event_id, and last-upload skip behavior.
The existing binary-fixture tests are kept alongside.
UpdateProfiles already sources settings.details via pump_metadata(); add
end-to-end compare_profiles tests over a real-shape PumpSettings confirming
the per-segment basal/carbratio/sens schedule, flat-cgm target_low/high, and
defaultProfile are translated correctly, and that a matching Nightscout
profile yields no change.
The settings.details blob differs from the old lastUpload.settings: profile
segments live under timeDependentSegments (was tDependentSegs), cgmSettings
is flat (highGlucoseAlertMgPerDl/lowGlucoseAlertMgPerDl, not nested per-alert
objects), and carbEntry is a string enum. Rework the dataclasses to the new
shape (verified against live t:slim X2 and Mobi accounts), keep a
tDependentSegs alias for segment consumers, and point tandemsource_profile_store
at the flat cgm fields. Values remain milliunit-scaled, so the NS translation
math is unchanged.
Event 81 (LidDailyBasal, battery) is not in Tandem's default id list; the
pump-logs endpoint may not return it. DEVICE_STATUS already fetches all
event types, and no-daily-basal-event already returns nothing — add a test
pinning that, plus a guard so an event 81 that arrives without battery
fields is skipped with a warning instead of raising on the percent math.
Fetch pre-parsed JSON events from get_pump_logs instead of decoding the
retired reportsfacade binary stream:
- page the requested range into inclusive windows of at most 28 days
(the endpoint caps each request at ~4 weeks), covering short ranges and
single days correctly
- dedupe events that span windows by (sequenceGroup, sequenceNumber)
- count but skip clockChanges (LID_TIME_CHANGED/LID_DATE_CHANGED are not
consumed by any processor)
- parse via Events_from_json
Remove the dead reportsfacade pump_events_raw() and the now-unused
pump_event_metadata()/PumpEventMetadata/LastUpload types.
Verify against real captured pump-logs events that build_from_json:
- resolves enum/dictionary fields from their raw ints (commandedRateSource,
alarmId, requestedAction, previousUserMode, glucoseValueStatus incl. 0)
- scales ratio fields (rate x0.1)
- folds bitmask arrays of set-bit indices back to the raw int the IntFlag
expects (activeSleepSchedule [0]->1, cgmDataType [0]->1,
egvInfoBitmask [0,5,6,7,8,11,12]->6625), and empty array->0
- keeps the raw.timestamp shim used by process_device_status and the raw
egvTimeStamp seconds used by ProcessCGMReading
Extend build_events.py (the codemod) so events.json stays the source of
truth: each generated event class gains build_from_json(event), mapping a
pump-logs JSON event's eventProperties onto its {field}[Raw] attrs
(matched by normalized field name), plus RawEvent.build_from_json() which
derives timestampRaw from pumpDateTime so eventTimestamp/seqNum/eventId
keep working. Add generic.Event_from_json()/Events_from_json() dispatchers
mirroring Event()/Events(); unknown eventCode -> bare RawEvent.
Enum/dictionary/ratio fields are raw ints (map directly onto {field}Raw so
existing properties resolve); bitmask fields arrive as arrays of set-bit
indices, converted to the raw int via _bitmask_arr_to_int. events.py is
regenerated; the byte-path build() is unchanged.
Tests: tests/eventparser/test_from_json.py.
Implement TandemSourceApi.get_pump_logs(device_id, ...) for GET
api/reports/bff/pump-logs/{assignmentId}?pumperId&startDate&endDate&eventIds,
the JSON events endpoint that replaces the base64 reportsfacade/pumpevents.
startDate/endDate are sent as {ymd}T00:00:00Z / {ymd}T23:59:59Z per the
captured request. Adds typed PumpLogsResponse/PumpLogEvent TypedDicts,
verified to cover every key across all captured pump-logs responses.
Tests: URL/path/param construction (order-independent parse_qs), default
vs custom vs empty eventIds, empty query dict passthrough, return
passthrough, and None-dates-default-to-today.
- tests/sync/tandemsource/test_update_profiles.py: settings sourcing from
pump_metadata() — matching deviceId with settings reaches
PumpSettings.from_dict (proven via sentinel, using the real BFF
settings.details shape); settings=None / no-match / empty return False
without parsing.
- test_tandemsource.py: adapter edge cases (availableDataRange/settings
keys absent, missing required key raises KeyError, Mobi Control-IQ+
passthrough) and DEFAULT_EVENT_IDS regression (55 ids, no dupes,
477/480/486 present).
Fixtures are inline and trimmed from the real captured responses.
Switch choose_device, check, update_profiles, process.py and cli_helpers
from the old pump_event_metadata() (reportsfacade) to the normalized
pump_metadata() (BFF): tconnectDeviceId -> deviceId (UUID), and
lastUpload.settings -> settings.
Handle the BFF returning never-uploaded pumps (maxDateWithEvents=None):
skip them in the most-recent auto-select and fall back to the first
pump; raise a clear NoDevicesFound on an empty account instead of an
opaque TypeError. Clean stale comments/docstrings in tandemsource.py.
Tests: add tests/sync/tandemsource/test_choose_device.py (11 cases:
explicit/auto/never-uploaded/empty/InvalidSerialNumber/stale-warning)
and tests/api/test_tandemsource.py (pump_metadata adapter mapping),
and update the test_process fixture to the new keys.
Since the 2.0 migration to Tandem Source, the live sync path
(api.tandemsource + sync/tandemsource/*) no longer references the
legacy t:connect APIs. This removes that now-unreachable code.
Removed modules:
- api/controliq.py, api/ws2.py, api/android.py, api/webui.py
(the legacy controliq / tconnectws2 / android / webui clients)
- process.py (old process_time_range; already broken since it
imported sync submodules that no longer exist)
- parser/ciq_therapy_events.py, parser/tconnect.py (TConnectEntry)
- domain/therapy_event.py, domain/bolus.py, domain/device_settings.py,
domain/utility.py
Trimmed dead wiring from live modules:
- api/__init__.py: dropped the controliq/ws2/android/webui properties,
keeping only the tandemsource accessor
- check.py: removed the unused TConnectEntry import
- parser/nightscout.py: removed the unused legacy profile_store() plus
the now-orphaned tandem_to_ns_time / tandem_to_ns_time_seconds helpers
and InvalidTimeException (the live path uses tandemsource_profile_store)
Tests: removed suites covering the deleted modules; pared tests/api/fake.py
down to the TConnectApi fake still used by the tandemsource tests. README
"Tandem APIs" sections updated to reflect the single Tandem Source API.
Full test suite passes (48 passed, 1 skipped).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQNn3mBG1kXTAdQb9c2jfW
xannasavinandJames Woglom
ClaudeandJames Woglom
Beshoy GirgisandJames Woglom