v0.8.4

%s was missing from the format string. This enhances the message to
report the unexpected thearpy type along with the dump of the event that
trigger the exception.

.github/ISSUE_TEMPLATE/setup-help--tconnectsync-heroku.md

@@ -0,0 +1,53 @@
+---
+name: 'Setup help: tconnectsync-heroku'
+about: When experiencing an issue setting up tconnectsync-heroku
+title: ''
+labels: heroku, setup help
+assignees: ''
+
+---
+
+**Describe the problem**
+A clear and concise description of the issue you're experiencing.
+
+
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. ...
+2. ...
+
+
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+
+
+**Have you followed the [Troubleshooting steps in the README?](https://github.com/jwoglom/tconnectsync-heroku#troubleshooting)** (Yes or No)
+
+
+
+**Setup details**
+
+* **On what platform are you using the t:connect mobile app?** (Android or iOS)
+* **What version are you using of the t:connect mobile app?**
+
+
+
+**Heroku log output**
+If applicable, add the full output from the heroku logs in More > View Logs. ([See the instructions in the README.](https://github.com/jwoglom/tconnectsync-heroku#testing))
+
+
+
+**Check Login output**
+If applicable, [please follow the instructions to visit the `check_login` page](https://github.com/jwoglom/tconnectsync-heroku#testing), and copy and paste the output here.
+
+This file may contain sensitive details like your Nightscout URL and pump serial number. Copy the output into a text editor like Textedit or Notepad first and find-and-replace sensitive strings, like your Nightscout URL and pump serial number, if they appear.
+
+The output will also contain pump and CGM related information such as delivery events and blood sugars. If you would prefer to keep the contents private, send an email to tconnectsync<at>wogloms.net with the log output so @jwoglom can investigate.
+
+
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/setup-help--tconnectsync.md

@@ -0,0 +1,49 @@
+---
+name: 'Setup help: tconnectsync'
+about: When experiencing an issue setting up tconnectsync on Windows, Linux, or MacOS.
+title: ''
+labels: setup help
+assignees: ''
+
+---
+
+**Describe the problem**
+A clear and concise description of the issue you're experiencing.
+
+
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. ...
+2. ...
+
+
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+
+
+**Setup details**
+
+* **Operating system:** 
+* **tconnectsync version** (from `--version`): 
+* **On what platform are you using the t:connect mobile app?** (Android or iOS)
+* **What version are you using of the t:connect mobile app?**
+
+
+
+**Terminal output**
+If applicable, add the full output from your terminal from running tconnectsync. Copy the output into a text editor like Textedit or Notepad first and find-and-replace sensitive strings, like your Nightscout URL and pump serial number, if they appear.
+
+
+
+**Check Login output**
+If applicable, please run `--check-login` and upload the tconnectsync-check-output.log file as an attachment on the issue. This file should have sensitive details like your Nightscout URL and pump serial number automatically removed, but double-check before uploading.
+
+This log will contain pump and CGM related information such as delivery events and blood sugars. If you would prefer to keep the contents private, send an email to tconnectsync<at>wogloms.net with the log output so @jwoglom can investigate.
+
+
+
+**Additional context**
+Add any other context about the problem here.

.gitignore

@@ -8,3 +8,4 @@ build
 *.egg-info
 .env
 tconnectsync-check-output.log
+ignore_*

README.md

@@ -116,7 +116,10 @@ First, ensure that you have **Python 3** with **Pip** installed:
 
 * **On MacOS:** Open Terminal. Install [Homebrew](https://brew.sh/), and then run `brew install python3`
 * **On Linux:** Follow your distribution's specific instructions.
-  For Debian/Ubuntu based distros, `sudo apt install python3 python3-pip`
+  - For Debian/Ubuntu based distros, `sudo apt install python3 python3-pip`
+  - For CentOS/Rocky Linux 8: 
+    - `sudo dnf install python39-pip`
+    - `sudo alternatives --set python /usr/bin/python3.9` 
 * **On Windows:** Install Ubuntu under the [Windows Subsystem for Linux](https://ubuntu.com/wsl).
   Open the Ubuntu Terminal, then run `sudo apt install python3 python3-pip`.
   Perform the remainder of the steps under the Ubuntu environment.
@@ -126,6 +129,12 @@ Now install the `tconnectsync` package with pip:
 ```
 $ pip3 install tconnectsync
 ```
+To install into a user environment instead of system-wide for a more contained install:
+````
+$ pip3 install --user tconnectsync
+````
+  - This will place the tconnectsync binary file at ``/home/<username>/.local/bin/tconnectsync``
+
 
 If the pip3 command is not found, run `python3 -m pip install tconnectsync` instead.
 
@@ -377,22 +386,27 @@ invoking tconnectsync with no arguments via cron.
 
 If using Pipenv or a virtualenv, make sure that you either prefix the call to main.py with `pipenv run` or source the `bin/activate` file within the virtualenv, so that the proper dependencies are loaded. If not using any kind of virtualenv, you can instead just install the necessary dependencies as specified inside Pipfile globally.
 
-An example configuration in `/etc/crontab` which runs every 15 minutes:
+An system-wide example configuration in `/etc/crontab` which runs every 15 minutes, on the 15 minute mark:
 
 ```bash
 # m         h  dom mon dow user   command
 0,15,30,45  *  *   *   *   root   /path/to/tconnectsync/run.sh
 ```
 
+An example of a user crontab `crontab -e` if not running system-wide, which runs every 15 minutes:
+```
+*/15 * * * * /path/to/tconnectsync/run.sh
+```
+
 You can use one of the same `run.sh` files referenced above, but remove the `--auto-update` flag since you are handling the functionality for running the script periodically yourself.
 
 ## Tandem APIs
 
 This application utilizes three separate Tandem APIs for obtaining t:connect data, referenced here by the identifying part of their URLs:
 
-* [**controliq**](https://github.com/jwoglom/tconnectsync/blob/master/tconnectsync/api/controliq.py) - Contains Control:IQ related data, namely a timeline of all Basal events uploaded by the pump, separated by type (temp basals, algorithmically-updated basals, or profile-updated basals).
+* [**controliq**](https://github.com/jwoglom/tconnectsync/blob/master/tconnectsync/api/controliq.py) - Contains Control:IQ related data, namely a timeline of all Basal events uploaded by the pump, separated by type (temp basals, algorithmically-updated basals, or profile-updated basals). Additionally includes CGM and Bolus data.
 * [**android**](https://github.com/jwoglom/tconnectsync/blob/master/tconnectsync/api/android.py) - Used internally by the t:connect Android app, these API endpoints were discovered by reverse-engineering the Android app. Most of the API endpoints are used for uploading pump data, and tconnectsync uses one endpoint which returns the most recent event ID uploaded by the pump, so we know when more data has been uploaded.
-* [**tconnectws2**](https://github.com/jwoglom/tconnectsync/blob/master/tconnectsync/api/ws2.py) - More legacy than the others, this seems to power the bulk of the main t:connect website. It is used to retrieve a CSV export of non-ControlIQ basal data, as well as bolus and IOB data. (I haven't found any mentions of bolus or IOB data in the Control:IQ-specific API.)
+* [**tconnectws2**](https://github.com/jwoglom/tconnectsync/blob/master/tconnectsync/api/ws2.py) - More legacy than the others, this seems to power the bulk of the main t:connect website. It is used as a last resort due to severe performance issues with this API (see https://github.com/jwoglom/tconnectsync/issues/43). We can use it to retrieve a CSV export of non-ControlIQ basal data, as well as bolus and IOB data. It is only used for bolus data as a fallback, and for pump-reported IOB data if requested. Full tracking of pump events also uses a limited version of this API.
 
 I have only tested tconnectsync with a Tandem pump set in the US Eastern timezone. Tandem's (to us, undocumented) APIs are [a bit loose with timezones](https://github.com/jwoglom/tconnectsync/blob/master/tconnectsync/parser.py#L15), so please let me know if you notice any timezone-related bugs.
 ## Backfilling t:connect Data
@@ -406,3 +420,15 @@ python3 main.py --start-date 2020-01-01 --end-date 2020-03-01
 In order to bulk-import a lot of data, you may need to use shorter intervals, and invoke tconnectsync multiple times. Tandem's API endpoints occasionally return invalid data if you request too large of a data window which causes tconnectsync to error out mid-way through.
 
 One oddity when backfilling data is that the Control:IQ specific API endpoints return errors if they are queried before you updated your pump to utilize Control:IQ. This is [partially worked around in tconnectsync's code](https://github.com/jwoglom/tconnectsync/blob/d841c3811aeff3671d941a7d3ff4b80cce6a219e/main.py#L238), but you might need to update the logic if you did not switch to a Control:IQ enabled pump immediately after launch.
+
+## t:connect API Testing
+
+To test t:connect API endpoints in a Python shell, you can do something like the following:
+
+```python
+import tconnectsync
+tconnectsync.util.cli.enable_logging()
+api = tconnectsync.util.cli.get_api()
+# Make API calls, e.g.
+therapy_timeline = api.controliq.therapy_timeline('2022-08-01', '2022-08-10')
+```

codecov.yml

@@ -0,0 +1,6 @@
+coverage:
+  status:
+    project:
+      default:
+        target: 50%
+        threshold: null

setup.cfg

@@ -1,6 +1,6 @@
 [metadata]
 name = tconnectsync
-version = 0.7.2
+version = 0.8.4
 author = James Woglom 
 author_email = j@wogloms.net
 description = Syncs Tandem t:connect pump data to Nightscout for the t:slim X2

tconnectsync/api/controliq.py

@@ -14,9 +14,12 @@ class ControlIQApi:
     BASE_URL = 'https://tdcservices.tandemdiabetes.com/'
     LOGIN_URL = 'https://tconnect.tandemdiabetes.com/login.aspx?ReturnUrl=%2f'
 
+    LAST_CONFIRMED_SOFTWARE_VERSION = 't:connect 7.14.0.1'
+
     userGuid = None
     accessToken = None
     accessTokenExpiresAt = None
+    tconnect_software_ver = None
 
     def __init__(self, email, password):
         self.login(email, password)
@@ -47,6 +50,16 @@ class ControlIQApi:
             return True
 
     def _build_login_data(self, email, password, soup):
+        try:
+            version = soup.select_one("#footer_version").text.strip()
+            self.tconnect_software_ver = version
+            logger.info("Reported tconnect software version: %s" % version)
+            if version != self.LAST_CONFIRMED_SOFTWARE_VERSION:
+                logger.warn("Newer API version than last confirmed working. Saw %s and expected %s" % (version, self.LAST_CONFIRMED_SOFTWARE_VERSION))
+                logger.warn("If you experience any issues, please report them to https://github.com/jwoglom/tconnectsync")
+        except Exception:
+            logger.warn("Unable to find tconnect software version")
+            pass
         return {
             "__LASTFOCUS": "",
             "__EVENTTARGET": "ctl00$ContentBody$LoginControl$linkLogin",
@@ -105,6 +118,7 @@ class ControlIQApi:
 
     """
     Returns detailed basal event information and reasons for delivery suspension.
+    End-date inclusive: Returns data from 00:00 on start date to 23:59 on end date.
     """
     def therapy_timeline(self, start=None, end=None):
         startDate = parse_date(start)
@@ -125,10 +139,7 @@ class ControlIQApi:
         startDate = parse_date(start)
         endDate = parse_date(end)
 
-        return self.get('tconnect/controliq/api/summary/users/%s' % (self.userGuid), {
-            "startDate": startDate,
-            "endDate": endDate
-        })
+        return self.get('tconnect/controliq/api/summary/users/%s?startDate=%s&endDate=%s' % (self.userGuid, startDate, endDate), {})
     
     """
     Returns active account features, including the date when ControlIQ was enabled.

tconnectsync/api/webui.py

@@ -1,3 +1,4 @@
+from typing import List
 import requests
 import urllib
 import datetime
@@ -7,6 +8,10 @@ import logging
 
 from bs4 import BeautifulSoup
 
+from tconnectsync.domain.device_settings import Device, Profile, ProfileSegment
+from tconnectsync.util import removesuffix, removeprefix
+from tconnectsync.util.constants import MMOLL_TO_MGDL
+
 from .common import base_headers, ApiException
 
 logger = logging.getLogger(__name__)
@@ -92,12 +97,11 @@ class WebUIScraper:
                 settings_guid = settings_a.attrs['href'].split('?guid=')[1]
             
             if serial_number:
-                devices[serial_number] = {
-                    'name': device_name,
-                    'model_number': model_number,
-                    'status': status,
-                    'guid': settings_guid
-                }
+                devices[serial_number] = Device(
+                    name=device_name,
+                    model_number=model_number,
+                    status=status,
+                    guid=settings_guid)
         
         return devices
     
@@ -106,7 +110,7 @@ class WebUIScraper:
     Note that pump_guid is NOT the serial number of the pump, and
     should be obtained from my_devices()[str(serial_number)]['guid']
     """
-    def device_settings_from_guid(self, pump_guid):
+    def device_settings_from_guid(self, pump_guid: str) -> List[Profile]:
         profiles = []
         settings = {}
         r = self.get('myaccount/DeviceSettings.aspx?guid=%s' % pump_guid)
@@ -124,12 +128,33 @@ class WebUIScraper:
         
         return profiles, settings
     
-    def _parse_profile_tbl(self, tbl):
+    def _parse_profile_tbl(self, tbl) -> Profile:
         profile = {}
         profile["title"] = self.strip(tbl.select_one('.setting_title').text)
         profile["active"] = bool(tbl.find(text='Active at the time of upload'))
         profile["segments"] = []
 
+        def parse_basal_rate(rate) -> float:
+            return float(removesuffix(rate, ' u/hr'))
+        
+        def parse_factor(ratio) -> int:
+            return parse_bg_mgdl(removeprefix(ratio, '1u:'))
+
+        def parse_ratio(ratio) -> float:
+            return float(removesuffix(removeprefix(ratio, '1u:'), ' g'))
+        
+        def parse_bg_mgdl(bg) -> int:
+            if bg.endswith(' mg/dL'):
+                return float(removesuffix(bg, ' mg/dL'))
+            elif bg.endswith(' mmol/L'):
+                return float(removesuffix(bg, ' mmol/L')) * MMOLL_TO_MGDL
+            raise ValueError(bg)
+        
+        def hours_to_mins(text) -> int:
+            hrmin = removesuffix(text, " hours")
+            hr, min = hrmin.split(":", 1)
+            return int(min) + int(hr)*60
+
         for tr in tbl.select('tr'):
             # Skip header rows
             if tr.select_one('.setting_bg'):
@@ -149,19 +174,20 @@ class WebUIScraper:
                     t = "12:00 AM"
                 elif display_time == "Noon":
                     t = "12:00 PM"
+
                 segment = {
                     "display_time": display_time,
                     "time": t,
-                    "basal_rate": self.strip(tds[1].text),
-                    "correction_factor": self.strip(tds[2].text),
-                    "carb_ratio": self.strip(tds[3].text),
-                    "target_bg": self.strip(tds[4].text)
+                    "basal_rate": parse_basal_rate(self.strip(tds[1].text)),
+                    "correction_factor": parse_factor(self.strip(tds[2].text)),
+                    "carb_ratio": parse_ratio(self.strip(tds[3].text)),
+                    "target_bg_mgdl": parse_bg_mgdl(self.strip(tds[4].text))
                 }
-                profile["segments"].append(segment)
+                profile["segments"].append(ProfileSegment(**segment))
                 continue
             
             if tr.find(text='Calculated Total Daily Basal'):
-                profile["calculated_total_daily_basal"] = self.strip(tds[1].text)
+                profile["calculated_total_daily_basal"] = float(removesuffix(self.strip(tds[1].text), " units"))
                 continue
             
             # Last row
@@ -175,12 +201,12 @@ class WebUIScraper:
                     key = self.strip(key)
                     val = self.strip(val)
                     if key == 'Duration of Insulin':
-                        profile["insulin_duration"] = val
+                        profile["insulin_duration_min"] = hours_to_mins(val)
                     elif key == 'Carbohydrates':
-                        profile["carbohydrates"] = val
+                        profile["carbs_enabled"] = self.strip(val.lower()) == "on"
 
 
-        return profile
+        return Profile(**profile)
 
     def _parse_settings_tbl(self, tbl):
         outer_tr = tbl.select('tr')[2]

tconnectsync/api/ws2.py

@@ -21,14 +21,14 @@ class WS2Api:
         self.userGuid = userGuid
         self.session = base_session()
 
-    def get(self, endpoint):
-        r = self.session.get(self.BASE_URL + endpoint, headers=base_headers())
+    def get(self, endpoint, **kwargs):
+        r = self.session.get(self.BASE_URL + endpoint, headers=base_headers(), **kwargs)
         if r.status_code != 200:
             raise ApiException(r.status_code, "WS2 API HTTP %s response: %s" % (str(r.status_code), r.text))
         return r.text
 
-    def get_jsonp(self, endpoint):
-        r = self.session.get(self.BASE_URL + endpoint + '?callback=cb', headers=base_headers())
+    def get_jsonp(self, endpoint, **kwargs):
+        r = self.session.get(self.BASE_URL + endpoint + '?callback=cb', headers=base_headers(), **kwargs)
         if r.status_code != 200:
             raise ApiException(r.status_code, "WS2 API HTTP %s response: %s" % (str(r.status_code), r.text))
 
@@ -71,13 +71,16 @@ class WS2Api:
     Basal data does NOT appear for the specified time range if using Control-IQ.
     The ControlIQ API endpoints must be used for basal data instead.
     However, all other fields are still accessed via this endpoint.
+
+    This has its own built-in retry logic because Tandem's frontend serving
+    the API returns 500s when its backend times out.
     """
     def therapy_timeline_csv(self, start=None, end=None, tries=0):
         startDate = parse_date(start)
         endDate = parse_date(end)
 
         try:
-            req_text = self.get('therapytimeline2csv/%s/%s/%s?format=csv' % (self.userGuid, startDate, endDate))
+            req_text = self.get('therapytimeline2csv/%s/%s/%s?format=csv' % (self.userGuid, startDate, endDate), timeout=10)
         except ApiException as e:
             # This seems to occur as some kind of soft rate-limit.
             logger.warning("Received ApiException in therapy_timeline_csv: (retry count %d) %s" % (tries, e))
@@ -124,6 +127,8 @@ class WS2Api:
      - "manual"
      - "previous"
      - "alarm"
+    
+    End-date inclusive: Returns data from 00:00 on start date to 23:59 on end date.
 
     {"BasalSuspension":[{"EventDateTime":"/Date(EPOCH_MS-0000)/", "SuspendReason": "reason"}]}
     """
@@ -132,7 +137,7 @@ class WS2Api:
         endDate = parse_date(end)
         arg = "filterbasal/1" if filterbasal else ""
 
-        return self.get_jsonp('basalsuspension/%s/%s/%s/%s' % (self.userGuid, startDate, endDate, arg))
+        return self.get_jsonp('basalsuspension/%s/%s/%s/%s' % (self.userGuid, startDate, endDate, arg), timeout=10)
 
     """
     Returns info on BasalIQ in JSONP format.
@@ -141,4 +146,4 @@ class WS2Api:
         startDate = parse_date(start)
         endDate = parse_date(end)
 
-        return self.get_jsonp('basaliqtech/%s/%s/%s' % (self.userGuid, startDate, endDate))
+        return self.get_jsonp('basaliqtech/%s/%s/%s' % (self.userGuid, startDate, endDate), timeout=10)

tconnectsync/autoupdate.py

@@ -1,5 +1,6 @@
 import time
 import logging
+import datetime
 import sys
 
 from .process import process_time_range
@@ -67,7 +68,7 @@ class Autoupdate:
                             # (we can see the indexes increasing, so we know something's happening!)
                             if (now - last_action_or_start) >= 60 * self.secret.AUTOUPDATE_FAILURE_MINUTES:
                                 logger.error(AutoupdateFailureError(
-                                        "An event index change was recorded, but no new data was found via the API. " +
+                                        ("%s: An event index change was recorded, but no new data was found via the API. " % datetime.datetime.now()) +
                                         "The %s was %d minutes ago. This is a problem with tconnectsync." % 
                                         ("last processed event" if self.last_successful_process_time_range else "start of autoupdate", (now - last_action_or_start)//60)))
 
@@ -75,7 +76,7 @@ class Autoupdate:
                                     logger.error("Exiting with error code due to AUTOUPDATE_RESTART_ON_FAILURE")
                                     return 1
                             else:
-                                logger.warn(AutoupdateFailureWarning("An event index change was recorded, but no new data was found via the API. " +
+                                logger.warn(AutoupdateFailureWarning(("%s: An event index change was recorded, but no new data was found via the API. " % datetime.datetime.now()) +
                                             "The %s was %d minutes ago. Resetting TConnectApi to attempt to solve this problem." % 
                                             ("last processed event" if self.last_successful_process_time_range else "start of autoupdate", (now - last_action_or_start)//60)))
                                 
@@ -107,7 +108,7 @@ class Autoupdate:
                 # The most likely case here is that the pump isn't uploading right now.
                 if self.last_event_time and (now - self.last_event_time) >= 60 * self.secret.AUTOUPDATE_NO_DATA_FAILURE_MINUTES:
                     logger.error(AutoupdateNoEventIndexesDetectedError(
-                        "No new data event indexes have been detected for %d minutes. " % ((now - self.last_event_time)//60) +
+                        "%s: No new data event indexes have been detected for %d minutes. " % (datetime.datetime.now(), (now - self.last_event_time)//60) +
                         "The t:connect app might no longer be functioning."))
 
                     # TODO: restarting doesn't really help anything here.
@@ -124,11 +125,11 @@ class Autoupdate:
                 # above no indexes warning.
                 elif self.last_successful_process_time_range and (now - self.last_successful_process_time_range) >= 60 * self.secret.AUTOUPDATE_FAILURE_MINUTES:
                     logger.error(AutoupdateNoNewDataDetectedError(
-                        "No new data has been detected via the API for %d minutes. " % (now - self.last_successful_process_time_range)//60 +
+                        "%s: No new data has been detected via the API for %d minutes. " % (datetime.datetime.now(), now - self.last_successful_process_time_range)//60 +
                         "tconnectsync might not be functioning properly."))
 
                     if self.secret.AUTOUPDATE_RESTART_ON_FAILURE:
-                        logger.error("Exiting with error code due to AUTOUPDATE_RESTART_ON_FAILURE")
+                        logger.error("%s: Exiting with error code due to AUTOUPDATE_RESTART_ON_FAILURE" % datetime.datetime.now())
                         return 1
 
                 # Track how long we've been retrying

tconnectsync/check.py

@@ -21,7 +21,7 @@ Attempts to authenticate with each t:connect API,
 and returns the output of a sample API call from each.
 Also attempts to connect to the Nightscout API.
 """
-def check_login(tconnect, time_start, time_end, verbose=False, sanitize=False):
+def check_login(tconnect, time_start, time_end, verbose=False, sanitize=True):
     errors = 0
 
     loglines = []
@@ -76,6 +76,7 @@ def check_login(tconnect, time_start, time_end, verbose=False, sanitize=False):
     try:
         summary = tconnect.controliq.dashboard_summary(time_start, time_end)
         debug("ControlIQ dashboard summary: \n%s" % pformat(summary))
+        log("tconnect_software_ver: %s" % tconnect.controliq.tconnect_software_ver)
     except Exception as e:
         log("Error occurred querying ControlIQ API for dashboard_summary:")
         log(e)
@@ -110,27 +111,32 @@ def check_login(tconnect, time_start, time_end, verbose=False, sanitize=False):
     
     log("-----")
 
-    log("Logging in to t:connect WS2 API...")
+    log("Initializing t:connect WS2 API...")
+    ws2_loggedin = False
     try:
         summary = tconnect.ws2.basaliqtech(time_start, time_end)
         debug("WS2 basaliq status: \n%s" % pformat(summary))
+        ws2_loggedin = True
     except Exception as e:
-        log("Error occurred querying WS2 API:")
+        log("Error occurred querying WS2 API. This is okay so long as you are not using the PUMP_EVENTS or IOB sync features.")
         log(e)
         errors += 1
     
-    log("Querying WS2 therapy_timeline_csv...")
     lastReadingTime = None
-    try:
-        ttcsv = tconnect.ws2.therapy_timeline_csv(time_start, time_end)
-        debug("therapy_timeline_csv: \n%s" % pformat(ttcsv))
-        if ttcsv and "readingData" in ttcsv and len(ttcsv["readingData"]) > 0:
-            log("Last therapy_timeline_csv reading: \n%s" % pformat(ttcsv["readingData"][-1]))
-            lastReadingTime = TConnectEntry._datetime_parse(ttcsv["readingData"][-1]['EventDateTime'])
-    except Exception as e:
-        log("Error occurred querying WS2 therapy_timeline_csv:")
-        log(e)
-        errors += 1
+    if ws2_loggedin:
+        log("Querying WS2 therapy_timeline_csv...")
+        try:
+            ttcsv = tconnect.ws2.therapy_timeline_csv(time_start, time_end)
+            debug("therapy_timeline_csv: \n%s" % pformat(ttcsv))
+            if ttcsv and "readingData" in ttcsv and len(ttcsv["readingData"]) > 0:
+                log("Last therapy_timeline_csv reading: \n%s" % pformat(ttcsv["readingData"][-1]))
+                lastReadingTime = TConnectEntry._datetime_parse(ttcsv["readingData"][-1]['EventDateTime'])
+        except Exception as e:
+            log("Error occurred querying WS2 therapy_timeline_csv. This is okay so long as you are not using the PUMP_EVENTS or IOB sync features.")
+            log(e)
+            errors += 1
+    else:
+        log("Not able to log in to WS2 API, so skipping therapy_timeline_csv")
     
     log("-----")
 

tconnectsync/domain/bolus.py

@@ -0,0 +1,25 @@
+from dataclasses import dataclass, asdict
+
+
+@dataclass
+class Bolus:
+    description: str
+    complete: str # "1" / "0"
+    completion: str
+    request_time: str # _datetime_parse timestamp
+    completion_time: str # _datetime_parse timestamp
+    insulin: str
+    requested_insulin: str
+    carbs: str
+    bg: str # potentially ""
+    user_override: str
+    extended_bolus: str # "1" / "0"
+    bolex_completion_time: str
+    bolex_start_time: str
+
+    def to_dict(self):
+        return asdict(self)
+    
+    @property
+    def is_extended_bolus(self):
+        return self.extended_bolus == "1"

tconnectsync/domain/device_settings.py

@@ -0,0 +1,27 @@
+from dataclasses import dataclass
+from typing import List, Optional
+
+@dataclass
+class Device:
+    name: str
+    model_number: str
+    status: str
+    guid: Optional[str]
+
+@dataclass
+class ProfileSegment:
+    display_time: str # Identical to time except written out as Midnight or Noon
+    time: str
+    basal_rate: float # _ u/hr
+    correction_factor: int # 1u: _ mg/dL
+    carb_ratio: float # 1u: _ g
+    target_bg_mgdl: int
+
+@dataclass
+class Profile:
+    title: str
+    active: bool
+    segments: List[ProfileSegment]
+    calculated_total_daily_basal: float # in units
+    insulin_duration_min: int
+    carbs_enabled: bool

tconnectsync/domain/therapy_event.py

@@ -0,0 +1,431 @@
+import arrow
+
+from tconnectsync.domain.bolus import Bolus
+
+from ..secret import TIMEZONE_NAME
+
+
+def _datetime_parse(date):
+    # consistent format with ws2 endpoint
+    return arrow.get(date, tzinfo=TIMEZONE_NAME).format("YYYY-MM-DD HH:mm:ssZZ")
+
+class TherapyEvent:
+    type = None
+    eventDateTime = None
+    sourceRecId = None
+
+    def parse(self, json):
+        self.type = json['type']
+        self.eventDateTime = json['eventDateTime']
+        self.sourceRecId = json['sourceRecId']
+        self.rawJson = json
+
+class CGMTherapyEvent(TherapyEvent):
+    eventID = None
+    egv = None
+    """
+    {
+        "eventDateTime": "2022-07-21T00:00:08",
+        "eventID": 256,
+        "requestDateTime": "0001-01-01T00:00:00",
+        "type": "CGM",
+        "description": "EGV",
+        "sourceRecId": 0,
+        "eventTypeId": 0,
+        "deviceType": "t:slim X2 Insulin Pump",
+        "serialNumber": "xxx",
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0,
+        "egv": {
+            "estimatedGlucoseValue": 174,
+            "hypo": 0,
+            "belowTarget": 0,
+            "withinTarget": 1,
+            "aboveTarget": 0,
+            "hyper": 0
+        }
+    },
+    """
+    @classmethod
+    def parse(_, json):
+        self = CGMTherapyEvent()
+        TherapyEvent.parse(self, json)
+        self.eventID = json['eventID']
+        self.egv = json['egv']['estimatedGlucoseValue']
+        return self
+
+class BGTherapyEvent(TherapyEvent):
+    eventID = None
+    egv = None
+    """
+    {   
+        'bg': 160, # note in EGV
+        'cgmCalibration': 1, # not in EGV
+        'description': 'BG',
+        'deviceType': 't:slim X2 Insulin Pump',
+        'eventDateTime': '2022-08-20T07:25:24',
+        'eventTypeId': 16,
+        'indexId': 844955,
+        'interactive': 0,
+        'iob': 0.75,
+        'note': {   'active': False,
+                'eventId': 0, # different location than EGV
+                'eventTypeId': 16,
+                'id': 0,
+                'indexId': '',
+                'sourceRecordId': 0},
+        'requestDateTime': '0001-01-01T00:00:00',
+        'serialNumber': 'xxx',
+        'sourceRecId': 793549667,
+        'tempRateActivated': 0,
+        'tempRateCompleted': 0,
+        'tempRateId': 0,
+        'type': 'BG',
+        'uploadId': 748700213}
+    """    
+    @classmethod
+    def parse(_, json):
+        self = BGTherapyEvent()
+        TherapyEvent.parse(self, json)
+        self.eventID = json['note']['eventId']
+        # This is probably not how we want to provide CGM calibrations to Nightscout,
+        # but will just include it as egv data for now to keep the thing from crashing :)
+        self.egv = json['bg']
+        return self
+
+class BolusTherapyEvent(TherapyEvent):
+    bolusRequestOptions = None
+    REQUEST_AUTOMATIC = "Automatic Bolus/Correction"
+    REQUEST_STANDARD = "Standard"
+
+    bolusType = None
+    TYPE_AUTOMATIC = "Automatic Correction"
+    TYPE_CARB = "Carb"
+
+    carbSize = None
+    correctionBolusSize = None
+    foodBolusSize = None
+    
+    insulinDelivered = None
+    insulinRequested = None
+    completionDateTime = None
+
+    completionStatus = None
+    STATUS_COMPLETED = "Completed"
+
+    eventHistoryReportDetails = None
+
+    standardPercent = None
+    sourceRecId = None
+
+    @classmethod
+    def parse(_, json):
+        self = BolusTherapyEvent()
+        TherapyEvent.parse(self, json)
+        self.description = json.get("description")
+        self.complete = json.get("standard", {}).get("bolusIsComplete")
+        self.completion = json.get("standard", {}).get("completionStatusDesc")
+        self.request_time = json.get("requestDateTime")
+        self.completion_time = json.get("standard", {}).get("insulinDelivered", {}).get("completionDateTime")
+        # TODO: separate extended vs standard bolus into separate fields
+        self.insulin = json.get("standard", {}).get("insulinDelivered", {}).get("value")
+        self.requested_insulin = json.get("standard", {}).get("insulinRequested")
+        self.carbs = json.get("carbSize")
+        self.bg = json.get("bg")
+        self.user_override = json.get("userOverride")
+        self.extended_bolus = json.get("bolusRequestOptions") == "Extended"
+        if self.extended_bolus and self.complete:
+            # TODO(https://github.com/jwoglom/tconnectsync/issues/19): read more extended bolus info
+            self.complete = json.get("bolex", {}).get("extendedBolusIsComplete")
+            self.completion = json.get("bolex", {}).get("completionStatusDesc")
+            self.bolex_completion_time = json.get("bolex", {}).get("insulinDelivered", {}).get("completionDateTime")
+            self.bolex_start_time = json.get("bolex", {}).get("bolexStartDateTime")
+        else:
+            self.bolex_completion_time = ""
+            self.bolex_start_time = ""
+        return self
+    
+    def to_bolus(self):
+        return Bolus(
+            description=self.description,
+            complete="1" if self.complete else "0",
+            completion=self.completion or "",
+            request_time=_datetime_parse(self.request_time),
+            completion_time=_datetime_parse(self.completion_time),
+            insulin=str(self.insulin),
+            requested_insulin=str(self.requested_insulin),
+            carbs=str(self.carbs or "0"), # Nightscout expects non-empty carbs
+            bg=str(self.bg or ""),
+            user_override=str(self.user_override),
+            extended_bolus="1" if self.extended_bolus else "0",
+            bolex_completion_time=_datetime_parse(self.bolex_completion_time) if self.bolex_completion_time else "",
+            bolex_start_time=_datetime_parse(self.bolex_start_time) if self.bolex_start_time else ""
+        )
+
+    """
+    Correction:
+    {
+        "actualTotalBolusRequested": 2.9,
+        "bg": 254,
+        "bolusRequestOptions": "Automatic Bolus/Correction",
+        "bolusType": "Automatic Correction",
+        "carbSize": 0,
+        "correctionBolusSize": 2.9,
+        "correctionFactor": 30,
+        "declinedCorrection": 0,
+        "duration": 0,
+        "eventDateTime": "2022-07-21T11:53:08",
+        "eventHistoryReportDetails": "CF 1:30 - Carb Ratio 1:0 - Target BG 110",
+        "eventHistoryReportEventDesc": "Correction Bolus",
+        "foodBolusSize": 0,
+        "isQuickBolus": 0,
+        "note": {
+            "id": 0,
+            "indexId": "572946",
+            "eventTypeId": 64,
+            "sourceRecordId": 0,
+            "eventId": 0,
+            "active": false
+        },
+        "requestDateTime": "2022-07-21T11:53:08",
+        "standard": {
+            "insulinDelivered": {
+                "completionDateTime": "2022-07-21T11:55:24",
+                "value": 2.9
+            },
+            "foodDelivered": 0,
+            "correctionDelivered": 2.9,
+            "insulinRequested": 2.9,
+            "completionStatusId": 3,
+            "completionStatusDesc": "Completed",
+            "bolusIsComplete": 1,
+            "bolusRequestId": 3361,
+            "bolusCompletionId": 3361
+        },
+        "standardPercent": 100,
+        "targetBG": 110,
+        "userOverride": 0,
+        "type": "Bolus",
+        "description": "Automatic Bolus/Correction",
+        "sourceRecId": 1171791787,
+        "eventTypeId": 0,
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0
+    },
+    Standard:
+    {
+        "actualTotalBolusRequested": 4.17,
+        "bolusRequestOptions": "Standard",
+        "bolusType": "Carb",
+        "carbSize": 25,
+        "correctionBolusSize": 0,
+        "correctionFactor": 30,
+        "declinedCorrection": 0,
+        "duration": 0,
+        "eventDateTime": "2022-07-21T12:27:36",
+        "eventHistoryReportDetails": "CF 1:30 - Carb Ratio 1:6 - Target BG 110",
+        "eventHistoryReportEventDesc": "Food Bolus",
+        "foodBolusSize": 4.17,
+        "iob": 2.62,
+        "isQuickBolus": 0,
+        "note": {
+            "id": 0,
+            "indexId": "573042",
+            "eventTypeId": 64,
+            "sourceRecordId": 0,
+            "eventId": 0,
+            "active": false
+        },
+        "requestDateTime": "2022-07-21T12:27:36",
+        "standard": {
+            "insulinDelivered": {
+                "completionDateTime": "2022-07-21T12:29:21",
+                "value": 4.17
+            },
+            "foodDelivered": 4.17,
+            "correctionDelivered": 0,
+            "insulinRequested": 4.17,
+            "completionStatusId": 3,
+            "completionStatusDesc": "Completed",
+            "bolusIsComplete": 1,
+            "bolusRequestId": 3362,
+            "bolusCompletionId": 3362
+        },
+        "standardPercent": 100,
+        "targetBG": 110,
+        "userOverride": 0,
+        "type": "Bolus",
+        "description": "Standard",
+        "sourceRecId": 1171853319,
+        "eventTypeId": 0,
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0
+    },
+    Extended bolus incomplete:
+    {
+        "actualTotalBolusRequested": 0.4,
+        "bg": 131,
+        "bolex": {
+            "size": 0.2,
+            "bolexStartDateTime": "2022-08-09T23:20:04",
+            "iob": 0,
+            "completionStatusId": 0,
+            "extendedBolusIsComplete": 0,
+            "insulinRequested": 0,
+            "bolexCompletionId": 0
+        },
+        "bolusRequestOptions": "Extended",
+        "bolusType": "Carb",
+        "carbSize": 0,
+        "correctionBolusSize": 0.0,
+        "correctionFactor": 30.0,
+        "declinedCorrection": 0,
+        "duration": 15,
+        "eventDateTime": "2022-08-09T23:19:15",
+        "eventHistoryReportDetails": "CF 1:30 - Carb Ratio 1:6 - Target BG 110<br/>Override: Pump calculated Bolus = 0.0 units",
+        "eventHistoryReportEventDesc": "Food Bolus: 50&#37; Extended 15 mins",
+        "foodBolusSize": 0.0,
+        "iob": 5.87,
+        "isQuickBolus": 0,
+        "note": {
+            "id": 0,
+            "indexId": "631597",
+            "eventTypeId": 64,
+            "sourceRecordId": 0,
+            "eventId": 0,
+            "active": false
+        },
+        "requestDateTime": "2022-08-09T23:19:15",
+        "standard": {
+            "insulinDelivered": {
+                "completionDateTime": "2022-08-09T23:20:04",
+                "value": 0.2
+            },
+            "foodDelivered": 0.0,
+            "correctionDelivered": 0.0,
+            "insulinRequested": 0.2,
+            "completionStatusId": 3,
+            "completionStatusDesc": "Completed",
+            "bolusIsComplete": 1,
+            "bolusRequestId": 3636.0,
+            "bolusCompletionId": 3636.0
+        },
+        "standardPercent": 50.0,
+        "targetBG": 110,
+        "userOverride": 1,
+        "type": "Bolus",
+        "description": "Extended 50.00%/0.00",
+        "sourceRecId": 1209631944,
+        "eventTypeId": 0,
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0
+    }
+    Extended bolus (complete):
+    {
+        "actualTotalBolusRequested": 0.4,
+        "bg": 131,
+        "bolex": {
+            "size": 0.2,
+            "bolexStartDateTime": "2022-08-09T23:20:04",
+            "insulinDelivered": {
+                "completionDateTime": "2022-08-09T23:35:03",
+                "value": 0.2
+            },
+            "iob": 5.7,
+            "completionStatusId": 3.0,
+            "completionStatusDesc": "Completed",
+            "extendedBolusIsComplete": 1,
+            "insulinRequested": 0.2,
+            "bolexCompletionId": 16757133
+        },
+        "bolusRequestOptions": "Extended",
+        "bolusType": "Carb",
+        "carbSize": 0,
+        "correctionBolusSize": 0.0,
+        "correctionFactor": 30.0,
+        "declinedCorrection": 0,
+        "duration": 15,
+        "eventDateTime": "2022-08-09T23:19:15",
+        "eventHistoryReportDetails": "CF 1:30 - Carb Ratio 1:6 - Target BG 110<br/>Override: Pump calculated Bolus = 0.0 units",
+        "eventHistoryReportEventDesc": "Food Bolus: 50&#37; Extended 15 mins",
+        "foodBolusSize": 0.0,
+        "iob": 5.87,
+        "isQuickBolus": 0,
+        "note": {
+            "id": 0,
+            "indexId": "631597",
+            "eventTypeId": 64,
+            "sourceRecordId": 0,
+            "eventId": 0,
+            "active": false
+        },
+        "requestDateTime": "2022-08-09T23:19:15",
+        "standard": {
+            "insulinDelivered": {
+                "completionDateTime": "2022-08-09T23:20:04",
+                "value": 0.2
+            },
+            "foodDelivered": 0.0,
+            "correctionDelivered": 0.0,
+            "insulinRequested": 0.2,
+            "completionStatusId": 3,
+            "completionStatusDesc": "Completed",
+            "bolusIsComplete": 1,
+            "bolusRequestId": 3636.0,
+            "bolusCompletionId": 3636.0
+        },
+        "standardPercent": 50.0,
+        "targetBG": 110,
+        "userOverride": 1,
+        "type": "Bolus",
+        "description": "Extended 50.00%/0.00",
+        "sourceRecId": 1209631944,
+        "eventTypeId": 0,
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0
+    }
+    CGM Calibration (Therapy Event Type BG):
+    {   'bg': 160,
+        'cgmCalibration': 1,
+        'description': 'BG',
+        'deviceType': 't:slim X2 Insulin Pump',
+        'eventDateTime': '2022-08-20T07:25:24',
+        'eventTypeId': 16,
+        'indexId': 844955,
+        'interactive': 0,
+        'iob': 0.75,
+        'note': {   'active': False,
+                    'eventId': 0,
+                    'eventTypeId': 16,
+                    'id': 0,
+                    'indexId': '',
+                    'sourceRecordId': 0},
+        'requestDateTime': '0001-01-01T00:00:00',
+        'serialNumber': 'xxx',
+        'sourceRecId': 793549667,
+        'tempRateActivated': 0,
+        'tempRateCompleted': 0,
+        'tempRateId': 0,
+        'type': 'BG',
+        'uploadId': 0}
+    """

tconnectsync/domain/utility.py

@@ -0,0 +1,24 @@
+#!/usr/bin/env python3
+
+class Time:
+    def __init__(self, hour: int, min: int):
+        self.hour = hour
+        self.min = min
+    
+    @classmethod
+    def parse(cls, input):
+        if ' ' not in input:
+            raise ValueError('unable to parse time: %s' % input)
+
+        hrmin, ampm = input.split(' ')
+        
+        hr, min = hrmin.split(':')
+        hr = int(hr)
+        min = int(min)
+
+        if ampm.lower() == 'pm':
+            hr += 12
+        elif ampm.lower() != 'am':
+            raise ValueError('unable to parse time: %s' % input)
+        
+        return cls(hr, min)

tconnectsync/nightscout.py

@@ -1,19 +1,26 @@
-import sys
+import datetime
 import requests
 import hashlib
 import time
 import urllib.parse
+import arrow
 
 from urllib.parse import urljoin
 
 from .api.common import ApiException
 from .parser.nightscout import ENTERED_BY
 
-# try:
-#     from .secret import NS_URL, NS_SECRET
-# except Exception:
-#     print('Unable to import Nightscout secrets from secret.py')
-#     sys.exit(1)
+def format_datetime(date):
+        return arrow.get(date).isoformat()
+
+def time_range(field_name, start_time, end_time):
+	arg = ''
+	if start_time:
+		arg += '&find[%s][$gte]=%s' % (field_name, format_datetime(start_time))
+	if end_time:
+		arg += '&find[%s][$lte]=%s' % (field_name, format_datetime(end_time))
+	return arg
+
 
 class NightscoutApi:
 	def __init__(self, url, secret):
@@ -48,8 +55,9 @@ class NightscoutApi:
 		if r.status_code != 200:
 			raise ApiException(r.status_code, "Nightscout put response: %s" % r.text)
 
-	def last_uploaded_entry(self, eventType):
-		latest = requests.get(urljoin(self.url, 'api/v1/treatments?count=1&find[enteredBy]=' + urllib.parse.quote(ENTERED_BY) + '&find[eventType]=' + urllib.parse.quote(eventType) + '&ts=' + str(time.time())), headers={
+	def last_uploaded_entry(self, eventType, time_start=None, time_end=None):
+		dateFilter = time_range('created_at', time_start, time_end)
+		latest = requests.get(urljoin(self.url, 'api/v1/treatments?count=1&find[enteredBy]=' + urllib.parse.quote(ENTERED_BY) + '&find[eventType]=' + urllib.parse.quote(eventType) + dateFilter + '&ts=' + str(time.time())), headers={
 			'api-secret': hashlib.sha1(self.secret.encode()).hexdigest()
 		})
 		if latest.status_code != 200:
@@ -60,8 +68,9 @@ class NightscoutApi:
 			return j[0]
 		return None
 	
-	def last_uploaded_bg_entry(self):
-		latest = requests.get(urljoin(self.url, 'api/v1/entries.json?count=1&find[device]=' + urllib.parse.quote(ENTERED_BY) + '&ts=' + str(time.time())), headers={
+	def last_uploaded_bg_entry(self, time_start=None, time_end=None):
+		dateFilter = time_range('dateString', time_start, time_end)
+		latest = requests.get(urljoin(self.url, 'api/v1/entries.json?count=1&find[device]=' + urllib.parse.quote(ENTERED_BY) + dateFilter + '&ts=' + str(time.time())), headers={
 			'api-secret': hashlib.sha1(self.secret.encode()).hexdigest()
 		})
 		if latest.status_code != 200:
@@ -72,8 +81,9 @@ class NightscoutApi:
 			return j[0]
 		return None
 
-	def last_uploaded_activity(self, activityType):
-		latest = requests.get(urljoin(self.url, 'api/v1/activity?find[enteredBy]=' + urllib.parse.quote(ENTERED_BY) + '&find[activityType]=' + urllib.parse.quote(activityType) + '&ts=' + str(time.time())), headers={
+	def last_uploaded_activity(self, activityType, time_start=None, time_end=None):
+		dateFilter = time_range('created_at', time_start, time_end)
+		latest = requests.get(urljoin(self.url, 'api/v1/activity?find[enteredBy]=' + urllib.parse.quote(ENTERED_BY) + '&find[activityType]=' + urllib.parse.quote(activityType) + dateFilter + '&ts=' + str(time.time())), headers={
 			'api-secret': hashlib.sha1(self.secret.encode()).hexdigest()
 		})
 		if latest.status_code != 200:

tconnectsync/parser/ciq_therapy_events.py

@@ -0,0 +1,23 @@
+from tconnectsync.domain.therapy_event import BolusTherapyEvent, CGMTherapyEvent, BGTherapyEvent
+from tconnectsync.parser.tconnect import TConnectEntry
+
+import logging
+
+logger = logging.getLogger(__name__)
+
+def split_therapy_events(ciqTherapyEvents):
+    bolusEvents = []
+    cgmEvents = []
+    bgEvents = []
+    for e in ciqTherapyEvents['event']:
+        event = TConnectEntry.parse_therapy_event(e)
+        if isinstance(event, BolusTherapyEvent):
+            bolusEvents.append(event)
+        elif isinstance(event, CGMTherapyEvent): 
+            cgmEvents.append(event)
+        elif isinstance(event, BGTherapyEvent):
+            bgEvents.append(event)
+        
+    logger.debug("split_therapy_events: %d bolus, %d CGM, %d BG" % (len(bolusEvents), len(cgmEvents), len(bgEvents)))
+    # TODO: BG events (CGM Calibration) values are not currently returned from ciq_therapy_events.py
+    return bolusEvents, cgmEvents

tconnectsync/parser/nightscout.py

@@ -48,7 +48,7 @@ class NightscoutEntry:
         }
         if bg:
             if bg_type not in (NightscoutEntry.SENSOR, NightscoutEntry.FINGER):
-                raise InvalidBolusTypeException
+                raise InvalidBolusTypeException("bg_type: %s (%s)" % (bg_type, data))
 
             data.update({
                 "glucose": str(bg),

tconnectsync/parser/tconnect.py

@@ -1,6 +1,9 @@
 from os import stat
 import sys
 import arrow
+from tconnectsync.domain.bolus import Bolus
+
+from tconnectsync.domain.therapy_event import BolusTherapyEvent, CGMTherapyEvent, BGTherapyEvent
 
 try:
     from ..secret import TIMEZONE_NAME
@@ -105,7 +108,7 @@ class TConnectEntry:
         complete = is_complete(data["ExtendedBolusIsComplete"]) or is_complete(data["BolusIsComplete"])
         extended_bolus = ("extended" in data["Description"].lower())
 
-        return {
+        return Bolus(**{
             "description": data["Description"],
             "complete": "1" if complete else "",
             "completion": data["CompletionStatusDesc"] if not extended_bolus else data["BolexCompletionStatusDesc"],
@@ -120,7 +123,7 @@ class TConnectEntry:
             # Note: completion time can be empty if the extended bolus is in progress
             "bolex_completion_time": TConnectEntry._datetime_parse(data["BolexCompletionDateTime"]).format() if data["BolexCompletionDateTime"] and complete and extended_bolus else None,
             "bolex_start_time": TConnectEntry._datetime_parse(data["BolexStartDateTime"]).format() if data["BolexStartDateTime"] and complete and extended_bolus else None,
-        }
+        })
     
     @staticmethod
     def parse_reading_entry(data):
@@ -189,7 +192,18 @@ class TConnectEntry:
             "time": time.format(),
             "event_type": TConnectEntry.BASALSUSPENSION_EVENTS[data["SuspendReason"]]
         }
-
+    
+    # Parses an entry from controliq.therapy_events() and returns a TherapyEvent
+    @staticmethod
+    def parse_therapy_event(data):
+        if data["type"] == "Bolus":
+            return BolusTherapyEvent.parse(data)
+        elif data["type"] == "CGM":
+            return CGMTherapyEvent.parse(data)
+        elif data["type"] == "BG":
+            return BGTherapyEvent.parse(data)
+        
+        raise UnknownTherapyEventException(data)
 
 class UnknownCIQActivityEventException(Exception):
     def __init__(self, data):
@@ -197,4 +211,9 @@ class UnknownCIQActivityEventException(Exception):
 
 class UnknownBasalSuspensionEventException(Exception):
     def __init__(self, data):
-        super().__init__("Unknown basal suspension event type: %s" % data)
+        super().__init__("Unknown basal suspension event type: %s" % data)
+
+class UnknownTherapyEventException(Exception):
+    def __init__(self, data):
+        typ = data["type"]
+        super().__init__(f"Unknown therapy event type: {typ} in {data}")

tconnectsync/process.py

@@ -3,6 +3,8 @@ import datetime
 import arrow
 import time
 
+from tconnectsync.parser.ciq_therapy_events import split_therapy_events
+
 from .util import timeago
 from .api.common import ApiException
 from .sync.basal import (
@@ -29,6 +31,7 @@ from .sync.pump_events import (
 )
 from .parser.tconnect import TConnectEntry
 from .features import BASAL, BOLUS, IOB, BOLUS_BG, CGM, DEFAULT_FEATURES, PUMP_EVENTS
+from tconnectsync.sync import basal
 
 logger = logging.getLogger(__name__)
 
@@ -50,33 +53,80 @@ def process_time_range(tconnect, nightscout, time_start, time_end, pretend, feat
             ciqTherapyTimelineData = None
         else:
             raise e
+    
+    csvReadingData = None
+    csvIobData = None
+    csvBasalData = None
+    csvBolusData = None
 
-    logger.info("Downloading t:connect CSV data")
-    csvdata = tconnect.ws2.therapy_timeline_csv(time_start, time_end)
+    ciqBolusData = None
+    ciqReadingData = None
+    if BOLUS in features:
+        logger.info("Downloading t:connect therapy_events")
+        ciqTherapyEventsData = tconnect.controliq.therapy_events(time_start, time_end)
+        ciqBolusData, ciqReadingData = split_therapy_events(ciqTherapyEventsData)
 
-    readingData = csvdata["readingData"]
-    iobData = csvdata["iobData"]
-    csvBasalData = csvdata["basalData"]
-    bolusData = csvdata["bolusData"]
+        if ciqReadingData and len(ciqReadingData) > 0:
+            lastReading = ciqReadingData[-1].eventDateTime
+            lastReading = TConnectEntry._datetime_parse(lastReading)
+            logger.debug(ciqReadingData[-1])
+            logger.info("Last CGM reading from t:connect CIQ: %s (%s)" % (lastReading, timeago(lastReading)))
+        else:
+            logger.warning("No last CGM reading is able to be determined from CIQ")
+        
+        if ciqBolusData and len(ciqBolusData) > 0:
+            lastBolus = ciqBolusData[-1].eventDateTime
+            lastReading = TConnectEntry._datetime_parse(lastBolus)
+            logger.debug(ciqBolusData[-1].to_bolus())
+            logger.info("Last bolus from t:connect CIQ: %s (%s)" % (lastBolus, timeago(lastBolus)))
+
+    bolusFallingBack = (BOLUS in features and not ciqBolusData)
+    ciqFallingBack = (CGM in features and not ciqReadingData)
+    if bolusFallingBack or \
+       ciqFallingBack or \
+       BOLUS_BG in features or \
+       IOB in features:
+        logger.warn("Downloading t:connect CSV data")
+        if bolusFallingBack:
+            logger.warn("Falling back on WS2 CSV data source because BOLUS is an enabled feature and CIQ bolus data was empty!!")
+        if ciqFallingBack:
+            logger.warn("Falling back on WS2 CSV data source because CGM is an enabled feature and CIQ cgm data was empty!!")
+        if BOLUS_BG in features:
+            logger.warn("Falling back on WS2 CSV data source because BOLUS_BG is an enabled feature. " +
+                        "Please consider disabling this feature to improve synchronization reliability.")
+        if IOB in features:
+            logger.warn("Falling back on WS2 CSV data source because IOB is an enabled feature. " +
+                        "Please consider disabling this feature to improve synchronization reliability.")
+        
+        logger.warn("<!!> The WS2 data source is unreliable and may prevent timely synchronization")
+        csvdata = tconnect.ws2.therapy_timeline_csv(time_start, time_end)
+
+        csvReadingData = csvdata["readingData"]
+        csvIobData = csvdata["iobData"]
+        csvBasalData = csvdata["basalData"]
+        csvBolusData = csvdata["bolusData"]
+
+        if csvReadingData and len(csvReadingData) > 0:
+            lastReading = csvReadingData[-1]['EventDateTime'] if 'EventDateTime' in csvReadingData[-1] else 0
+            lastReading = TConnectEntry._datetime_parse(lastReading)
+            logger.debug(csvReadingData[-1])
+            logger.info("Last CGM reading from t:connect CSV: %s (%s)" % (lastReading, timeago(lastReading)))
+        else:
+            logger.warning("No last CGM reading is able to be determined from CSV")
 
-    if readingData and len(readingData) > 0:
-        lastReading = readingData[-1]['EventDateTime'] if 'EventDateTime' in readingData[-1] else 0
-        lastReading = TConnectEntry._datetime_parse(lastReading)
-        logger.debug(readingData[-1])
-        logger.info("Last CGM reading from t:connect: %s (%s)" % (lastReading, timeago(lastReading)))
-    else:
-        logger.warning("No last CGM reading is able to be determined")
 
     added = 0
 
-    cgmData = None
-    if CGM in features or BOLUS_BG in features:
-        logger.debug("Processing CGM events")
-        cgmData = process_cgm_events(readingData)
-    
-    if CGM in features:
-        logger.debug("Writing CGM events")
-        added += ns_write_cgm_events(nightscout, cgmData, pretend)
+    if csvReadingData:
+        cgmData = None
+        if CGM in features or BOLUS_BG in features:
+            logger.debug("Processing CGM events")
+            cgmData = process_cgm_events(csvReadingData)
+        
+        if CGM in features:
+            logger.debug("Writing CGM events")
+            added += ns_write_cgm_events(nightscout, cgmData, pretend, time_start=time_start, time_end=time_end)
+            logger.debug("Finished writing CGM events")
 
     if BASAL in features:
         basalEvents = process_ciq_basal_events(ciqTherapyTimelineData)
@@ -85,13 +135,20 @@ def process_time_range(tconnect, nightscout, time_start, time_end, pretend, feat
             add_csv_basal_events(basalEvents, csvBasalData)
         else:
             logger.debug("No CSV basal data found")
+        
+        if basalEvents and len(basalEvents) > 0:
+            logger.info("Last basal event from CIQ: %s" % basalEvents[-1])
 
-        added += ns_write_basal_events(nightscout, basalEvents, pretend=pretend)
+        logger.debug("Writing basal events")
+        added += ns_write_basal_events(nightscout, basalEvents, pretend=pretend, time_start=time_start, time_end=time_end)
+        logger.debug("Finished writing basal events")
     
     if PUMP_EVENTS in features:
         pumpEvents = process_ciq_activity_events(ciqTherapyTimelineData)
         logger.debug("CIQ activity events: %s" % pumpEvents)
 
+        logger.warn("Using WS2 data source for basalsuspension because PUMP_EVENTS is an enabled feature")
+        logger.warn("<!!> The WS2 data source is unreliable and may prevent timely synchronization")
         ws2BasalSuspension = tconnect.ws2.basalsuspension(time_start, time_end)
 
         bsPumpEvents = process_basalsuspension_events(ws2BasalSuspension)
@@ -99,15 +156,32 @@ def process_time_range(tconnect, nightscout, time_start, time_end, pretend, feat
 
         pumpEvents += bsPumpEvents
 
-        added += ns_write_pump_events(nightscout, pumpEvents, pretend=pretend)
+        logger.debug("Writing pump events")
+        added += ns_write_pump_events(nightscout, pumpEvents, pretend=pretend, time_start=time_start, time_end=time_end)
+        logger.debug("Finished writing basal events")
 
     if BOLUS in features:
-        bolusEvents = process_bolus_events(bolusData)
-        added += ns_write_bolus_events(nightscout, bolusEvents, pretend=pretend, include_bg=(BOLUS_BG in features))
+        bolusEvents = []
+        if ciqBolusData:
+            logger.info("Processing ciqBolusData (%d entries)" % len(ciqBolusData))
+            bolusEvents = process_bolus_events(ciqBolusData, source="ciq")
 
-    if IOB in features:
-        iobEvents = process_iob_events(iobData)
-        added += ns_write_iob_events(nightscout, iobEvents, pretend=pretend)
+        if csvBolusData and not bolusEvents:
+            logger.warn("Falling back on non-CIQ csvBolusData")
+            bolusEvents = process_bolus_events(csvBolusData, source="csv")
+            logger.debug("ciq bolusEvents: %s" % bolusEvents)
+        
+        logger.info("finalized bolusEvents: %s" % bolusEvents)
+        logger.debug("Writing bolus events")
+        added += ns_write_bolus_events(nightscout, bolusEvents, pretend=pretend, include_bg=(BOLUS_BG in features), time_start=time_start, time_end=time_end)
+        logger.debug("Finished writing bolus events")
+
+    if csvIobData:
+        if IOB in features:
+            iobEvents = process_iob_events(csvIobData)
+            logger.debug("Writing iob events")
+            added += ns_write_iob_events(nightscout, iobEvents, pretend=pretend)
+            logger.debug("Finished writing iob events")
 
     logger.info("Wrote %d events to Nightscout this process cycle" % added)
     return added

tconnectsync/secret.py

@@ -31,7 +31,7 @@ def get_bool(name, default):
 TCONNECT_EMAIL = get('TCONNECT_EMAIL', 'email@email.com')
 TCONNECT_PASSWORD = get('TCONNECT_PASSWORD', 'password')
 
-PUMP_SERIAL_NUMBER = get_number('PUMP_SERIAL_NUMBER', '11111111')
+PUMP_SERIAL_NUMBER = int(get_number('PUMP_SERIAL_NUMBER', '11111111'))
 
 NS_URL = get('NS_URL', 'https://yournightscouturl/')
 NS_SECRET = get('NS_SECRET', 'apisecret')

tconnectsync/sync/basal.py

@@ -104,9 +104,9 @@ def add_csv_basal_events(basalEvents, data):
 """
 Given processed basal data, adds basal events to Nightscout.
 """
-def ns_write_basal_events(nightscout, basalEvents, pretend=False):
+def ns_write_basal_events(nightscout, basalEvents, pretend=False, time_start=None, time_end=None):
     logger.debug("ns_write_basal_events: querying for last uploaded entry")
-    last_upload = nightscout.last_uploaded_entry(BASAL_EVENTTYPE)
+    last_upload = nightscout.last_uploaded_entry(BASAL_EVENTTYPE, time_start=time_start, time_end=time_end)
     last_upload_time = None
     if last_upload:
         last_upload_time = arrow.get(last_upload["created_at"])
@@ -121,7 +121,7 @@ def ns_write_basal_events(nightscout, basalEvents, pretend=False):
     for event in basalEvents:
         if last_upload_time and arrow.get(event["time"]) < last_upload_time:
             if pretend:
-                logger.info("Skipping basal event before last upload time: %s" % event)
+                logger.info("Skipping basal event before last upload time: %s (time range: %s - %s)" % (event, time_start, time_end))
             continue
 
         recent_needs_update = False

tconnectsync/sync/bolus.py

@@ -1,5 +1,6 @@
 import arrow
 import logging
+from tconnectsync.domain.bolus import Bolus
 
 from tconnectsync.sync.cgm import find_event_at
 
@@ -15,26 +16,32 @@ logger = logging.getLogger(__name__)
 """
 Given bolus data input from the therapy timeline CSV, converts it into a digestable format.
 """
-def process_bolus_events(bolusdata, cgmEvents=None):
+def process_bolus_events(bolusdata, cgmEvents=None, source=""):
     bolusEvents = []
 
     for b in bolusdata:
-        parsed = TConnectEntry.parse_bolus_entry(b)
-        if parsed["completion"] != "Completed":
-            if parsed["insulin"] and float(parsed["insulin"]) > 0:
+        parsed = None
+        if source == "ciq":
+            parsed = b.to_bolus()
+        else:
+            parsed = TConnectEntry.parse_bolus_entry(b)
+        
+        assert type(parsed) == Bolus
+        if parsed.completion != "Completed":
+            if parsed.insulin and float(parsed.insulin) > 0:
                 # Count non-completed bolus if any insulin was delivered (vs. the amount of insulin requested)
-                parsed["description"] += " (%s: requested %s units)" % (parsed["completion"], parsed["requested_insulin"])
+                parsed.description += " (%s: requested %s units)" % (parsed.completion, parsed.requested_insulin)
             else:
-                logger.warning("Skipping non-completed bolus data (was a bolus in progress?): %s parsed: %s" % (b, parsed))
+                logger.warning("Skipping non-completed %s bolus data (was a bolus in progress?): %s parsed: %s" % (source, b, parsed))
                 continue
 
-        if parsed["bg"] and cgmEvents:
-            requested_at = parsed["request_time"] if not parsed["extended_bolus"] else parsed["bolex_start_time"]
-            parsed["bg_type"] = guess_bolus_bg_type(parsed["bg"], requested_at, cgmEvents)
+        if parsed.bg and cgmEvents:
+            requested_at = parsed.request_time if not parsed.extended_bolus else parsed.bolex_start_time
+            parsed.bg_type = guess_bolus_bg_type(parsed.bg, requested_at, cgmEvents)
 
         bolusEvents.append(parsed)
 
-    bolusEvents.sort(key=lambda event: arrow.get(event["request_time"] if not event["extended_bolus"] else event["bolex_start_time"]))
+    bolusEvents.sort(key=lambda event: arrow.get(event.request_time if not event.is_extended_bolus else event.bolex_start_time))
 
     return bolusEvents
 
@@ -57,9 +64,9 @@ def guess_bolus_bg_type(bg, created_at, cgmEvents):
 """
 Given processed bolus data, adds bolus events to Nightscout.
 """
-def ns_write_bolus_events(nightscout, bolusEvents, pretend=False, include_bg=False, reading_events=None):
+def ns_write_bolus_events(nightscout, bolusEvents, pretend=False, include_bg=False, reading_events=None, time_start=None, time_end=None):
     logger.debug("ns_write_bolus_events: querying for last uploaded entry")
-    last_upload = nightscout.last_uploaded_entry(BOLUS_EVENTTYPE)
+    last_upload = nightscout.last_uploaded_entry(BOLUS_EVENTTYPE, time_start=time_start, time_end=time_end)
     last_upload_time = None
     if last_upload:
         last_upload_time = arrow.get(last_upload["created_at"])
@@ -72,27 +79,27 @@ def ns_write_bolus_events(nightscout, bolusEvents, pretend=False, include_bg=Fal
 
     add_count = 0
     for event in bolusEvents:
-        created_at = event["completion_time"] if not event["extended_bolus"] else event["bolex_start_time"]
+        created_at = event.completion_time if not event.is_extended_bolus else event.bolex_start_time
         if last_upload_time and arrow.get(created_at) <= last_upload_time:
             if pretend:
-                logger.info("Skipping basal event before last upload time: %s" % event)
+                logger.info("Skipping basal event before last upload time: %s (time range: %s - %s)" % (event, time_start, time_end))
             continue
 
-        if include_bg and event["bg"]:
+        if include_bg and event.bg:
             entry = NightscoutEntry.bolus(
-                bolus=event["insulin"],
-                carbs=event["carbs"],
+                bolus=event.insulin,
+                carbs=event.carbs,
                 created_at=created_at,
-                notes="{}{}{}".format(event["description"], " (Override)" if event["user_override"] == "1" else "", " (Extended)" if event["extended_bolus"] == "1" else ""),
-                bg=event["bg"],
-                bg_type=event["bg_type"]
+                notes="{}{}{}".format(event.description, " (Override)" if event.user_override == "1" else "", " (Extended)" if event.extended_bolus == "1" else ""),
+                bg=event.bg,
+                bg_type=event.bg_type
             )
         else:
             entry = NightscoutEntry.bolus(
-                bolus=event["insulin"],
-                carbs=event["carbs"],
+                bolus=event.insulin,
+                carbs=event.carbs,
                 created_at=created_at,
-                notes="{}{}{}".format(event["description"], " (Override)" if event["user_override"] == "1" else "", " (Extended)" if event["extended_bolus"] == "1" else "")
+                notes="{}{}{}".format(event.description, " (Override)" if event.user_override == "1" else "", " (Extended)" if event.extended_bolus == "1" else "")
             )
 
         add_count += 1

tconnectsync/sync/cgm.py

@@ -39,9 +39,9 @@ def find_event_at(cgmEvents, find_time):
 """
 Given processed CGM data, adds reading entries to Nightscout.
 """
-def ns_write_cgm_events(nightscout, cgmEvents, pretend=False):
+def ns_write_cgm_events(nightscout, cgmEvents, pretend=False, time_start=None, time_end=None):
     logger.debug("ns_write_cgm_events: querying for last uploaded entry")
-    last_upload = nightscout.last_uploaded_bg_entry()
+    last_upload = nightscout.last_uploaded_bg_entry(time_start=time_start, time_end=time_end)
     last_upload_time = None
     if last_upload:
         last_upload_time = arrow.get(last_upload["dateString"])
@@ -52,7 +52,7 @@ def ns_write_cgm_events(nightscout, cgmEvents, pretend=False):
         created_at = event["time"]
         if last_upload_time and arrow.get(created_at) <= last_upload_time:
             if pretend:
-                logger.info("Skipping CGM event before last upload time: %s" % event)
+                logger.info("Skipping CGM event before last upload time: %s (time range: %s - %s)" % (event, time_start, time_end))
             continue
 
         entry = NightscoutEntry.entry(

tconnectsync/sync/iob.py

@@ -24,9 +24,9 @@ def process_iob_events(iobdata):
 """
 Given processed IOB data, creates a single Nightscout activity definition to store IOB.
 """
-def ns_write_iob_events(nightscout, iobEvents, pretend=False):
+def ns_write_iob_events(nightscout, iobEvents, pretend=False, time_start=None, time_end=None):
     logger.debug("ns_write_iob_events: querying for last uploaded entry")
-    last_upload = nightscout.last_uploaded_activity(IOB_ACTIVITYTYPE)
+    last_upload = nightscout.last_uploaded_activity(IOB_ACTIVITYTYPE, time_start=time_start, time_end=time_end)
     last_upload_time = None
     if last_upload:
         last_upload_time = arrow.get(last_upload["created_at"])

tconnectsync/sync/pump_events.py

@@ -49,7 +49,7 @@ def process_basalsuspension_events(data):
 """
 Given processed pump event data (of various types), write them to Nightscout
 """
-def ns_write_pump_events(nightscout, pumpEvents, pretend=False):
+def ns_write_pump_events(nightscout, pumpEvents, pretend=False, time_start=None, time_end=None):
     count = 0
 
     siteChangeEvents = []
@@ -80,17 +80,17 @@ def ns_write_pump_events(nightscout, pumpEvents, pretend=False):
     logger.debug("sleepEvents: %s" % sleepEvents)
     logger.debug("activityEvents: %s" % activityEvents)
     
-    count += ns_write_pump_sitechange_events(nightscout, siteChangeEvents, pretend=pretend)
-    count += ns_write_empty_cart_events(nightscout, emptyCartEvents, pretend=pretend)
-    count += ns_write_user_suspended_events(nightscout, userSuspendedEvents, pretend=pretend)
+    count += ns_write_pump_sitechange_events(nightscout, siteChangeEvents, pretend=pretend, time_start=time_start, time_end=time_end)
+    count += ns_write_empty_cart_events(nightscout, emptyCartEvents, pretend=pretend, time_start=time_start, time_end=time_end)
+    count += ns_write_user_suspended_events(nightscout, userSuspendedEvents, pretend=pretend, time_start=time_start, time_end=time_end)
 
-    count += ns_write_exercise_activity_events(nightscout, exerciseEvents, pretend=pretend)
-    count += ns_write_sleep_activity_events(nightscout, sleepEvents, pretend=pretend)
-    count += ns_write_activity_events(nightscout, activityEvents, pretend=pretend)
+    count += ns_write_exercise_activity_events(nightscout, exerciseEvents, pretend=pretend, time_start=time_start, time_end=time_end)
+    count += ns_write_sleep_activity_events(nightscout, sleepEvents, pretend=pretend, time_start=time_start, time_end=time_end)
+    count += ns_write_activity_events(nightscout, activityEvents, pretend=pretend, time_start=time_start, time_end=time_end)
 
     return count
 
-def ns_write_pump_sitechange_events(nightscout, siteChangeEvents, pretend=False):
+def ns_write_pump_sitechange_events(nightscout, siteChangeEvents, pretend=False, time_start=None, time_end=None):
     return _ns_write_pump_events(
         nightscout, 
         siteChangeEvents, 
@@ -99,9 +99,11 @@ def ns_write_pump_sitechange_events(nightscout, siteChangeEvents, pretend=False)
             reason=event["event_type"]
         ),
         SITECHANGE_EVENTTYPE,
-        pretend=pretend)
+        pretend=pretend,
+        time_start=time_start,
+        time_end=time_end)
 
-def ns_write_empty_cart_events(nightscout, emptyCartEvents, pretend=False):
+def ns_write_empty_cart_events(nightscout, emptyCartEvents, pretend=False, time_start=None, time_end=None):
     return _ns_write_pump_events(
         nightscout, 
         emptyCartEvents, 
@@ -110,9 +112,11 @@ def ns_write_empty_cart_events(nightscout, emptyCartEvents, pretend=False):
             reason=event["event_type"]
         ),
         BASALSUSPENSION_EVENTTYPE,
-        pretend=pretend)
+        pretend=pretend,
+        time_start=time_start,
+        time_end=time_end)
 
-def ns_write_user_suspended_events(nightscout, userSuspendedEvents, pretend=False):
+def ns_write_user_suspended_events(nightscout, userSuspendedEvents, pretend=False, time_start=None, time_end=None):
     return _ns_write_pump_events(
         nightscout, 
         userSuspendedEvents, 
@@ -121,9 +125,11 @@ def ns_write_user_suspended_events(nightscout, userSuspendedEvents, pretend=Fals
             reason=event["event_type"]
         ),
         BASALSUSPENSION_EVENTTYPE,
-        pretend=pretend)
+        pretend=pretend,
+        time_start=time_start,
+        time_end=time_end)
 
-def ns_write_exercise_activity_events(nightscout, exerciseEvents, pretend=False):
+def ns_write_exercise_activity_events(nightscout, exerciseEvents, pretend=False, time_start=None, time_end=None):
     return _ns_write_pump_events(
         nightscout, 
         exerciseEvents, 
@@ -134,9 +140,11 @@ def ns_write_exercise_activity_events(nightscout, exerciseEvents, pretend=False)
             event_type=EXERCISE_EVENTTYPE
         ),
         EXERCISE_EVENTTYPE,
-        pretend=pretend)
+        pretend=pretend,
+        time_start=time_start,
+        time_end=time_end)
 
-def ns_write_sleep_activity_events(nightscout, sleepEvents, pretend=False):
+def ns_write_sleep_activity_events(nightscout, sleepEvents, pretend=False, time_start=None, time_end=None):
     return _ns_write_pump_events(
         nightscout, 
         sleepEvents, 
@@ -147,9 +155,11 @@ def ns_write_sleep_activity_events(nightscout, sleepEvents, pretend=False):
             event_type=SLEEP_EVENTTYPE
         ),
         SLEEP_EVENTTYPE,
-        pretend=pretend)
+        pretend=pretend,
+        time_start=time_start,
+        time_end=time_end)
 
-def ns_write_activity_events(nightscout, activityEvents, pretend=False):
+def ns_write_activity_events(nightscout, activityEvents, pretend=False, time_start=None, time_end=None):
     return _ns_write_pump_events(
         nightscout, 
         activityEvents, 
@@ -159,15 +169,17 @@ def ns_write_activity_events(nightscout, activityEvents, pretend=False):
             duration=event["duration_mins"]
         ),
         ACTIVITY_EVENTTYPE,
-        pretend=pretend)
+        pretend=pretend,
+        time_start=time_start,
+        time_end=time_end)
 
-def _ns_write_pump_events(nightscout, events, buildNsEventFunc, eventType, pretend=False):
+def _ns_write_pump_events(nightscout, events, buildNsEventFunc, eventType, pretend=False, time_start=None, time_end=None):
     if len(events) == 0:
         logger.debug("No %s events to process" % eventType)
         return 0
 
     logger.debug("ns_write_pump_events: querying for last %s" % eventType)
-    last_upload = nightscout.last_uploaded_entry(eventType)
+    last_upload = nightscout.last_uploaded_entry(eventType, time_start=time_start, time_end=time_end)
     last_upload_time = None
     if last_upload:
         last_upload_time = arrow.get(last_upload["created_at"])
@@ -192,7 +204,7 @@ def _ns_write_pump_events(nightscout, events, buildNsEventFunc, eventType, prete
             
             if skip:
                 if pretend:
-                    logger.info("Skipping %s event before last upload time: %s" % (eventType, event))
+                    logger.info("Skipping %s pump event before last upload time: %s (time range: %s - %s)" % (eventType, event, time_start, time_end))
                 continue
 
         entry = buildNsEventFunc(event)

tconnectsync/util/__init__.py

@@ -1,5 +1,8 @@
 import arrow
 
+from . import cli
+from . import constants
+
 def timeago(timestamp):
     seconds = (arrow.get() - arrow.get(timestamp)).total_seconds()
     fmt = '%s ago' if seconds >= 0 else 'in %s'
@@ -15,3 +18,14 @@ def timeago(timestamp):
     ret += '%d minutes' % (seconds//60)
 
     return fmt % ret
+
+# String methods only available in python 3.9+
+def removesuffix(input_string, suffix):
+    if suffix and input_string.endswith(suffix):
+        return input_string[:-len(suffix)]
+    return input_string
+
+def removeprefix(input_string, prefix):
+    if prefix and input_string.startswith(prefix):
+        return input_string[len(prefix):]
+    return input_string

tconnectsync/util/cli.py

@@ -0,0 +1,20 @@
+import logging
+
+"""
+Enables logging at the specified level for all loggers
+inside the tconnectsync package, and sets up a basicConfig
+to print those log messages to stderr.
+"""
+def enable_logging(level=logging.DEBUG):
+    logging.basicConfig()
+    for logger in logging.root.manager.loggerDict:
+        if logger.startswith('tconnectsync'):
+            logging.getLogger(logger).setLevel(level)
+
+"""
+Returns a TConnectApi object with default secret parameters.
+"""
+def get_api():
+    from ..api import TConnectApi
+    from ..secret import TCONNECT_EMAIL, TCONNECT_PASSWORD
+    return TConnectApi(TCONNECT_EMAIL, TCONNECT_PASSWORD)

tconnectsync/util/constants.py

@@ -0,0 +1,4 @@
+
+# http://www.soc-bdr.org/rds/authors/unit_tables_conversions_and_genetic_dictionaries/conversion_glucose_mg_dl_to_mmol_l/index_en.html
+MMOLL_TO_MGDL = 18.0182
+MGDL_TO_MMOLL = 0.0555

tests/api/test_controliq.py

@@ -221,12 +221,10 @@ class TestControlIQApi(unittest.TestCase):
         ciq = ControlIQApi()
         ciq.userGuid = 'aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee'
 
-        def fake_get(endpoint, query):
+        def fake_get(raw_endpoint, ignored_query):
+            endpoint, query = raw_endpoint.split("?")
             self.assertTrue(endpoint.endswith("summary/users/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"))
-            self.assertEqual(query, {
-                "startDate": "04-01-2021",
-                "endDate": "04-02-2021"
-            })
+            self.assertEqual(query, "startDate=04-01-2021&endDate=04-02-2021")
 
             return {"faked_json": True}
 

tests/api/test_webui.py

@@ -11,6 +11,7 @@ import requests_mock
 from bs4 import BeautifulSoup
 
 from tconnectsync.api.webui import WebUIScraper
+from tconnectsync.domain.device_settings import Device, Profile, ProfileSegment
 
 from .fake import ControlIQApi
 
@@ -293,30 +294,30 @@ class TestWebUIScraper(unittest.TestCase):
 
             devices = webui.my_devices()
             self.assertDictEqual(devices, {
-                '100001': {
+                '100001': Device(**{
                     'name': 't:slim X2™ Insulin Pump', 
                     'model_number': '001002717', 
                     'status': 'Activated —  Dec 30 2021', 
                     'guid': '00000000-0000-0000-0000-000000000001'
-                },
-                '10000002': {
+                }),
+                '10000002': Device(**{
                     'name': 't:slim X2™ Insulin Pump', 
                     'model_number': '001000354', 
                     'status': 'Activated —  Oct 26 2021', 
                     'guid': '00000000-0000-0000-0000-000000000002'
-                }, 
-                '100003': {
+                }), 
+                '100003': Device(**{
                     'name': 't:slim X2™ Insulin Pump', 
                     'model_number': '001000096', 
                     'status': 'Activated —  Nov 20 2017', 
                     'guid': '00000000-0000-0000-0000-000000000003'
-                }, 
-                'ABCDEFGH': {
+                }), 
+                'ABCDEFGH': Device(**{
                     'name': 'OneTouch Verio IQ', 
                     'model_number': 'VERIO IQ', 
                     'status': 'Activated —  Jan 17 2018', 
                     'guid': None
-                }})
+                })})
     
     PUMP_SETTINGS_HTML = """
 <!DOCTYPE html
@@ -956,42 +957,42 @@ class TestWebUIScraper(unittest.TestCase):
                 text=self.PUMP_SETTINGS_HTML)
 
             profiles, settings = webui.device_settings_from_guid('00000000-0000-0000-0000-000000000001')
-            self.assertListEqual(profiles, [{
+            self.assertListEqual(profiles, [Profile(**{
                 'title': 'A', 
                 'active': True, 
-                'segments': [{
+                'segments': [ProfileSegment(**{
                     'display_time': 'Midnight', 
                     'time': '12:00 AM', 
-                    'basal_rate': '0.800 u/hr', 
-                    'correction_factor': '1u:30 mg/dL', 
-                    'carb_ratio': '1u:6.0 g', 
-                    'target_bg': '110 mg/dL'
-                }, {
+                    'basal_rate': 0.800,
+                    'correction_factor': 30.0,
+                    'carb_ratio': 6.0,
+                    'target_bg_mgdl': 110
+                }), ProfileSegment(**{
                     'display_time': '6:00 AM', 
                     'time': '6:00 AM', 
-                    'basal_rate': '1.250 u/hr', 
-                    'correction_factor': '1u:30 mg/dL', 
-                    'carb_ratio': '1u:6.0 g', 
-                    'target_bg': '110 mg/dL'
-                }, {
+                    'basal_rate': 1.250,
+                    'correction_factor': 30,
+                    'carb_ratio': 6.0,
+                    'target_bg_mgdl': 110
+                }), ProfileSegment(**{
                     'display_time': '11:00 AM', 
                     'time': '11:00 AM', 
-                    'basal_rate': '1.000 u/hr', 
-                    'correction_factor': '1u:30 mg/dL', 
-                    'carb_ratio': '1u:6.0 g', 
-                    'target_bg': '110 mg/dL'
-                }, {
+                    'basal_rate': 1.000,
+                    'correction_factor': 30,
+                    'carb_ratio': 6.0,
+                    'target_bg_mgdl': 110
+                }), ProfileSegment(**{
                     'display_time': 'Noon', 
                     'time': '12:00 PM', 
-                    'basal_rate': '0.800 u/hr', 
-                    'correction_factor': '1u:30 mg/dL', 
-                    'carb_ratio': '1u:6.0 g', 
-                    'target_bg': '110 mg/dL'
-                }], 
-                'calculated_total_daily_basal': '21.65 units', 
-                'insulin_duration': '5:00 hours', 
-                'carbohydrates': 'On'
-            }])
+                    'basal_rate': 0.800,
+                    'correction_factor': 30,
+                    'carb_ratio': 6.0,
+                    'target_bg_mgdl': 110
+                })], 
+                'calculated_total_daily_basal': 21.65, 
+                'insulin_duration_min': 5*60,
+                'carbs_enabled': True
+            })])
 
             self.assertDictEqual(settings, {
                 'Alerts': {

tests/api/test_ws2.py

@@ -10,7 +10,7 @@ from tconnectsync.api.common import ApiException
 class TestWS2Api(unittest.TestCase):
     def fake_get_with_http_500(self, num_times):
         tries = 0
-        def fake_get(endpoint):
+        def fake_get(endpoint, **kwargs):
             nonlocal tries, num_times
             if "therapytimeline2csv" in endpoint:
                 if tries < num_times:
@@ -94,7 +94,7 @@ Report Generated On, 4/24/2021 7:50:04 PM
 
         rawData = self.RAW_DATA_FULL
 
-        def fake_get(endpoint):
+        def fake_get(endpoint, **kwargs):
             nonlocal rawData
             if endpoint == 'therapytimeline2csv/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/2021-04-01/2021-04-02?format=csv':
                 return rawData
@@ -111,7 +111,7 @@ Report Generated On, 4/24/2021 7:50:04 PM
 
         rawData = ""
 
-        def fake_get(endpoint):
+        def fake_get(endpoint, **kwargs):
             nonlocal rawData
             if endpoint == 'therapytimeline2csv/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/2021-04-01/2021-04-02?format=csv':
                 return rawData

tests/domain/test_therapy_event.py

@@ -0,0 +1,367 @@
+import dataclasses
+import unittest
+from tconnectsync.domain.bolus import Bolus
+
+from tconnectsync.domain.therapy_event import BolusTherapyEvent, CGMTherapyEvent
+
+class TestCGMTherapyEvent(unittest.TestCase):
+    maxDiff = None
+    sampleJson = {
+        "eventDateTime": "2022-07-21T00:00:08",
+        "eventID": 256,
+        "requestDateTime": "0001-01-01T00:00:00",
+        "type": "CGM",
+        "description": "EGV",
+        "sourceRecId": 0,
+        "eventTypeId": 0,
+        "deviceType": "t:slim X2 Insulin Pump",
+        "serialNumber": "xxx",
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0,
+        "egv": {
+            "estimatedGlucoseValue": 174,
+            "hypo": 0,
+            "belowTarget": 0,
+            "withinTarget": 1,
+            "aboveTarget": 0,
+            "hyper": 0
+        }
+    }
+
+    def test_parse_cgm(self):
+        e = CGMTherapyEvent.parse(self.sampleJson)
+        self.assertEqual(e.type, "CGM")
+        self.assertEqual(e.eventDateTime, "2022-07-21T00:00:08")
+        self.assertEqual(e.sourceRecId, 0)
+        self.assertEqual(e.eventID, 256)
+        self.assertEqual(e.egv, 174)
+
+class TestBolusTherapyEvent(unittest.TestCase):
+    maxDiff = None
+    standardJson = {
+        "actualTotalBolusRequested": 4.17,
+        "bolusRequestOptions": "Standard",
+        "bolusType": "Carb",
+        "carbSize": 25,
+        "correctionBolusSize": 0,
+        "correctionFactor": 30,
+        "declinedCorrection": 0,
+        "duration": 0,
+        "eventDateTime": "2022-07-21T12:27:36",
+        "eventHistoryReportDetails": "CF 1:30 - Carb Ratio 1:6 - Target BG 110",
+        "eventHistoryReportEventDesc": "Food Bolus",
+        "foodBolusSize": 4.17,
+        "iob": 2.62,
+        "isQuickBolus": 0,
+        "note": {
+            "id": 0,
+            "indexId": "573042",
+            "eventTypeId": 64,
+            "sourceRecordId": 0,
+            "eventId": 0,
+            "active": False
+        },
+        "requestDateTime": "2022-07-21T12:27:36",
+        "standard": {
+            "insulinDelivered": {
+                "completionDateTime": "2022-07-21T12:29:21",
+                "value": 4.17
+            },
+            "foodDelivered": 4.17,
+            "correctionDelivered": 0,
+            "insulinRequested": 4.17,
+            "completionStatusId": 3,
+            "completionStatusDesc": "Completed",
+            "bolusIsComplete": 1,
+            "bolusRequestId": 3362,
+            "bolusCompletionId": 3362
+        },
+        "standardPercent": 100,
+        "targetBG": 110,
+        "userOverride": 0,
+        "type": "Bolus",
+        "description": "Standard",
+        "sourceRecId": 1171853319,
+        "eventTypeId": 0,
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0
+    }
+
+    def test_standard_to_bolus(self):
+        e = BolusTherapyEvent.parse(self.standardJson)
+        self.assertIsNotNone(e)
+        b = e.to_bolus()
+        self.assertEqual(dataclasses.asdict(b), dataclasses.asdict(Bolus(
+            description="Standard",
+            complete="1",
+            completion="Completed",
+            request_time="2022-07-21 12:27:36-04:00",
+            completion_time="2022-07-21 12:29:21-04:00",
+            insulin="4.17",
+            requested_insulin="4.17",
+            carbs="25",
+            bg="",
+            user_override="0",
+            extended_bolus="0",
+            bolex_completion_time="",
+            bolex_start_time=""
+        )))
+
+    correctionJson = {
+        "actualTotalBolusRequested": 2.9,
+        "bg": 254,
+        "bolusRequestOptions": "Automatic Bolus/Correction",
+        "bolusType": "Automatic Correction",
+        "carbSize": 0,
+        "correctionBolusSize": 2.9,
+        "correctionFactor": 30,
+        "declinedCorrection": 0,
+        "duration": 0,
+        "eventDateTime": "2022-07-21T11:53:08",
+        "eventHistoryReportDetails": "CF 1:30 - Carb Ratio 1:0 - Target BG 110",
+        "eventHistoryReportEventDesc": "Correction Bolus",
+        "foodBolusSize": 0,
+        "isQuickBolus": 0,
+        "note": {
+            "id": 0,
+            "indexId": "572946",
+            "eventTypeId": 64,
+            "sourceRecordId": 0,
+            "eventId": 0,
+            "active": False
+        },
+        "requestDateTime": "2022-07-21T11:53:08",
+        "standard": {
+            "insulinDelivered": {
+                "completionDateTime": "2022-07-21T11:55:24",
+                "value": 2.9
+            },
+            "foodDelivered": 0,
+            "correctionDelivered": 2.9,
+            "insulinRequested": 2.9,
+            "completionStatusId": 3,
+            "completionStatusDesc": "Completed",
+            "bolusIsComplete": 1,
+            "bolusRequestId": 3361,
+            "bolusCompletionId": 3361
+        },
+        "standardPercent": 100,
+        "targetBG": 110,
+        "userOverride": 0,
+        "type": "Bolus",
+        "description": "Automatic Bolus/Correction",
+        "sourceRecId": 1171791787,
+        "eventTypeId": 0,
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0
+    }
+
+    def test_correction_to_bolus(self):
+        e = BolusTherapyEvent.parse(self.correctionJson)
+        self.assertIsNotNone(e)
+        b = e.to_bolus()
+        self.assertEqual(dataclasses.asdict(b), dataclasses.asdict(Bolus(
+            description="Automatic Bolus/Correction",
+            complete="1",
+            completion="Completed",
+            request_time="2022-07-21 11:53:08-04:00",
+            completion_time="2022-07-21 11:55:24-04:00",
+            insulin="2.9",
+            requested_insulin="2.9",
+            carbs="0",
+            bg="254",
+            user_override="0",
+            extended_bolus="0",
+            bolex_completion_time="",
+            bolex_start_time=""
+        )))
+    
+    extendedBolusIncompleteJson = {
+        "actualTotalBolusRequested": 0.4,
+        "bg": 131,
+        "bolex": {
+            "size": 0.2,
+            "bolexStartDateTime": "2022-08-09T23:20:04",
+            "iob": 0,
+            "completionStatusId": 0,
+            "extendedBolusIsComplete": 0,
+            "insulinRequested": 0,
+            "bolexCompletionId": 0
+        },
+        "bolusRequestOptions": "Extended",
+        "bolusType": "Carb",
+        "carbSize": 0,
+        "correctionBolusSize": 0.0,
+        "correctionFactor": 30.0,
+        "declinedCorrection": 0,
+        "duration": 15,
+        "eventDateTime": "2022-08-09T23:19:15",
+        "eventHistoryReportDetails": "CF 1:30 - Carb Ratio 1:6 - Target BG 110<br/>Override: Pump calculated Bolus = 0.0 units",
+        "eventHistoryReportEventDesc": "Food Bolus: 50&#37; Extended 15 mins",
+        "foodBolusSize": 0.0,
+        "iob": 5.87,
+        "isQuickBolus": 0,
+        "note": {
+            "id": 0,
+            "indexId": "631597",
+            "eventTypeId": 64,
+            "sourceRecordId": 0,
+            "eventId": 0,
+            "active": False
+        },
+        "requestDateTime": "2022-08-09T23:19:15",
+        "standard": {
+            "insulinDelivered": {
+                "completionDateTime": "2022-08-09T23:20:04",
+                "value": 0.2
+            },
+            "foodDelivered": 0.0,
+            "correctionDelivered": 0.0,
+            "insulinRequested": 0.2,
+            "completionStatusId": 3,
+            "completionStatusDesc": "Completed",
+            "bolusIsComplete": 1,
+            "bolusRequestId": 3636.0,
+            "bolusCompletionId": 3636.0
+        },
+        "standardPercent": 50.0,
+        "targetBG": 110,
+        "userOverride": 1,
+        "type": "Bolus",
+        "description": "Extended 50.00%/0.00",
+        "sourceRecId": 1209631944,
+        "eventTypeId": 0,
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0
+    }
+
+    def test_extended_bolus_incomplete_to_bolus(self):
+        e = BolusTherapyEvent.parse(self.extendedBolusIncompleteJson)
+        self.assertIsNotNone(e)
+        b = e.to_bolus()
+        self.assertEqual(dataclasses.asdict(b), dataclasses.asdict(Bolus(
+            description="Extended 50.00%/0.00",
+            complete="0",
+            completion="",
+            request_time="2022-08-09 23:19:15-04:00",
+            completion_time="2022-08-09 23:20:04-04:00",
+            insulin="0.2",
+            requested_insulin="0.2",
+            carbs="0",
+            bg="131",
+            user_override="1",
+            extended_bolus="1",
+            bolex_completion_time="",
+            bolex_start_time="2022-08-09 23:20:04-04:00"
+        )))
+    
+    extendedBolusJson = {
+        "actualTotalBolusRequested": 0.4,
+        "bg": 131,
+        "bolex": {
+            "size": 0.2,
+            "bolexStartDateTime": "2022-08-09T23:20:04",
+            "insulinDelivered": {
+                "completionDateTime": "2022-08-09T23:35:03",
+                "value": 0.2
+            },
+            "iob": 5.7,
+            "completionStatusId": 3.0,
+            "completionStatusDesc": "Completed",
+            "extendedBolusIsComplete": 1,
+            "insulinRequested": 0.2,
+            "bolexCompletionId": 16757133
+        },
+        "bolusRequestOptions": "Extended",
+        "bolusType": "Carb",
+        "carbSize": 0,
+        "correctionBolusSize": 0.0,
+        "correctionFactor": 30.0,
+        "declinedCorrection": 0,
+        "duration": 15,
+        "eventDateTime": "2022-08-09T23:19:15",
+        "eventHistoryReportDetails": "CF 1:30 - Carb Ratio 1:6 - Target BG 110<br/>Override: Pump calculated Bolus = 0.0 units",
+        "eventHistoryReportEventDesc": "Food Bolus: 50&#37; Extended 15 mins",
+        "foodBolusSize": 0.0,
+        "iob": 5.87,
+        "isQuickBolus": 0,
+        "note": {
+            "id": 0,
+            "indexId": "631597",
+            "eventTypeId": 64,
+            "sourceRecordId": 0,
+            "eventId": 0,
+            "active": False
+        },
+        "requestDateTime": "2022-08-09T23:19:15",
+        "standard": {
+            "insulinDelivered": {
+                "completionDateTime": "2022-08-09T23:20:04",
+                "value": 0.2
+            },
+            "foodDelivered": 0.0,
+            "correctionDelivered": 0.0,
+            "insulinRequested": 0.2,
+            "completionStatusId": 3,
+            "completionStatusDesc": "Completed",
+            "bolusIsComplete": 1,
+            "bolusRequestId": 3636.0,
+            "bolusCompletionId": 3636.0
+        },
+        "standardPercent": 50.0,
+        "targetBG": 110,
+        "userOverride": 1,
+        "type": "Bolus",
+        "description": "Extended 50.00%/0.00",
+        "sourceRecId": 1209631944,
+        "eventTypeId": 0,
+        "indexId": 0,
+        "uploadId": 0,
+        "interactive": 0,
+        "tempRateId": 0,
+        "tempRateCompleted": 0,
+        "tempRateActivated": 0
+    }
+
+    def test_extended_bolus_complete_to_bolus(self):
+        e = BolusTherapyEvent.parse(self.extendedBolusJson)
+        self.assertIsNotNone(e)
+        b = e.to_bolus()
+        self.assertEqual(dataclasses.asdict(b), dataclasses.asdict(Bolus(
+            description="Extended 50.00%/0.00",
+            complete="1",
+            completion="Completed",
+            request_time="2022-08-09 23:19:15-04:00",
+            completion_time="2022-08-09 23:20:04-04:00",
+            insulin="0.2",
+            requested_insulin="0.2",
+            carbs="0",
+            bg="131",
+            user_override="1",
+            extended_bolus="1",
+            bolex_completion_time="2022-08-09 23:35:03-04:00",
+            bolex_start_time="2022-08-09 23:20:04-04:00"
+        )))
+
+
+BOLUS_FULL_EXAMPLES = [
+    TestBolusTherapyEvent.standardJson,
+    TestBolusTherapyEvent.correctionJson,
+    TestBolusTherapyEvent.extendedBolusJson
+]

tests/nightscout_fake.py

@@ -20,7 +20,7 @@ class NightscoutApi(tconnectsync.nightscout.NightscoutApi):
     def put_entry(self, ns_format, entity):
         self.put_entries[entity].append(ns_format)
 
-    def last_uploaded_entry(self, eventType):
+    def last_uploaded_entry(self, eventType, start_date=None, end_date=None):
         raise NotImplementedError
 
     def last_uploaded_activity(self, activityType):

tests/parser/test_tconnect.py

@@ -1,6 +1,7 @@
 #!/usr/bin/env python3
 
 import unittest
+from tconnectsync.domain.bolus import Bolus
 from tconnectsync.parser.tconnect import TConnectEntry, UnknownBasalSuspensionEventException, UnknownCIQActivityEventException
 
 class TestTConnectEntryBasal(unittest.TestCase):
@@ -177,7 +178,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
     def test_parse_bolus_entry_std_correction(self):
         self.assertEqual(
             TConnectEntry.parse_bolus_entry(self.entryStdCorrection),
-            {
+            Bolus(**{
                 "description": "Standard/Correction",
                 "complete": "1",
                 "completion": "Completed",
@@ -191,7 +192,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
                 "extended_bolus": "",
                 "bolex_completion_time": None,
                 "bolex_start_time": None
-        })
+        }))
     
     entryStd = {
         "Type": "Bolus",
@@ -239,7 +240,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
     def test_parse_bolus_entry_std(self):
         self.assertEqual(
             TConnectEntry.parse_bolus_entry(self.entryStd),
-            {
+            Bolus(**{
                 "description": "Standard",
                 "complete": "1",
                 "completion": "Completed",
@@ -253,7 +254,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
                 "extended_bolus": "",
                 "bolex_completion_time": None,
                 "bolex_start_time": None
-        })
+        }))
     
     entryStdAutomatic = {
         "Type": "Bolus",
@@ -301,7 +302,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
     def test_parse_bolus_entry_std_automatic(self):
         self.assertEqual(
             TConnectEntry.parse_bolus_entry(self.entryStdAutomatic),
-            {
+            Bolus(**{
                 "description": "Automatic Bolus/Correction",
                 "complete": "1",
                 "completion": "Completed",
@@ -315,7 +316,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
                 "extended_bolus": "",
                 "bolex_completion_time": None,
                 "bolex_start_time": None
-        })
+        }))
     
     entryStdIncompleteZero = {
         "Type": "Bolus",
@@ -363,7 +364,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
     def test_parse_bolus_entry_std_incomplete_zero(self):
         self.assertEqual(
             TConnectEntry.parse_bolus_entry(self.entryStdIncompleteZero),
-            {
+            Bolus(**{
                 "description": "Standard",
                 "complete": "",
                 "completion": "User Aborted",
@@ -377,7 +378,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
                 "extended_bolus": "",
                 "bolex_completion_time": None,
                 "bolex_start_time": None
-            })
+            }))
     
     entryStdIncompletePartial = {
         "Type": "Bolus",
@@ -425,7 +426,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
     def test_parse_bolus_entry_std_incomplete_partial(self):
         self.assertEqual(
             TConnectEntry.parse_bolus_entry(self.entryStdIncompletePartial),
-            {
+            Bolus(**{
                 "description": "Standard/Correction",
                 "complete": "",
                 "completion": "Terminated by Alarm",
@@ -439,7 +440,7 @@ class TestTConnectEntryBolus(unittest.TestCase):
                 "extended_bolus": "",
                 "bolex_completion_time": None,
                 "bolex_start_time": None
-            })
+            }))
 
 class TestTConnectEntryReading(unittest.TestCase):
     entry1 = {

tests/sync/test_bolus.py

@@ -52,7 +52,7 @@ class TestBolusSync(unittest.TestCase):
         self.assertEqual(len(bolusEvents), len(bolusData))
 
         def set_bg_type(entry, type):
-            entry["bg_type"] = type
+            entry.bg_type = type
             return entry
 
         # Expect FINGER for bolus entries with a BG because there's no matching event with the same BG
@@ -111,7 +111,7 @@ class TestBolusSync(unittest.TestCase):
         self.assertEqual(len(bolusEvents), len(bolusData))
 
         def set_bg_type(entry, type):
-            entry["bg_type"] = type
+            entry.bg_type = type
             return entry
 
         expected = [
@@ -145,7 +145,7 @@ class TestBolusSync(unittest.TestCase):
         ]
 
         for e in partialEntries:
-            e["description"] += " (%s: requested %s units)" % (e["completion"], e["requested_insulin"])
+            e.description += " (%s: requested %s units)" % (e.completion, e.requested_insulin)
 
         self.assertListEqual(bolusEvents, [
             TConnectEntry.parse_bolus_entry(d) for d in stdData

tests/test_process.py

@@ -8,6 +8,7 @@ import copy
 from tconnectsync.process import process_time_range
 from tconnectsync.parser.nightscout import EXERCISE_EVENTTYPE, IOB_ACTIVITYTYPE, SLEEP_EVENTTYPE, NightscoutEntry
 from tconnectsync.features import BASAL, BOLUS, IOB, PUMP_EVENTS
+from tests.domain.test_therapy_event import BOLUS_FULL_EXAMPLES, TestCGMTherapyEvent
 
 from .api.fake import TConnectApi
 from .nightscout_fake import NightscoutApi
@@ -20,6 +21,11 @@ class TestProcessTimeRange(unittest.TestCase):
 
     def stub_therapy_timeline(self, time_start, time_end):
         return copy.deepcopy(TestBasalSync.base)
+    
+    def stub_ciq_therapy_events(self, time_start, time_end):
+        return {
+            "event": []
+        }
 
     def stub_therapy_timeline_csv(self, time_start, time_end):
         return {
@@ -32,10 +38,10 @@ class TestProcessTimeRange(unittest.TestCase):
     def stub_ws2_basalsuspension(self, time_start, time_end):
         return {"BasalSuspension": []}
 
-    def stub_last_uploaded_entry(self, event_type):
+    def stub_last_uploaded_entry(self, event_type, **kwargs):
         return None
 
-    def stub_last_uploaded_activity(self, activity_type):
+    def stub_last_uploaded_activity(self, activity_type, **kwargs):
         return None
 
     """No data in Nightscout. Uploads all basal data from tconnect."""
@@ -53,6 +59,7 @@ class TestProcessTimeRange(unittest.TestCase):
             return TestBasalSync.get_example_ciq_basal_events()
 
         tconnect.controliq.therapy_timeline = fake_therapy_timeline
+        tconnect.controliq.therapy_events = self.stub_ciq_therapy_events
         tconnect.ws2.therapy_timeline_csv = self.stub_therapy_timeline_csv
 
         nightscout = NightscoutApi()
@@ -88,6 +95,7 @@ class TestProcessTimeRange(unittest.TestCase):
             return TestBasalSync.get_example_ciq_basal_events()
 
         tconnect.controliq.therapy_timeline = fake_therapy_timeline
+        tconnect.controliq.therapy_events = self.stub_ciq_therapy_events
         tconnect.ws2.therapy_timeline_csv = self.stub_therapy_timeline_csv
 
         nightscout = NightscoutApi()
@@ -116,11 +124,12 @@ class TestProcessTimeRange(unittest.TestCase):
             return TestBasalSync.get_example_ciq_basal_events()
 
         tconnect.controliq.therapy_timeline = fake_therapy_timeline
+        tconnect.controliq.therapy_events = self.stub_ciq_therapy_events
         tconnect.ws2.therapy_timeline_csv = self.stub_therapy_timeline_csv
 
         nightscout = NightscoutApi()
 
-        def fake_last_uploaded_entry(event_type):
+        def fake_last_uploaded_entry(event_type, **kwargs):
             if event_type == "Temp Basal":
                 return {
                     "created_at": "2021-03-16 00:20:21-04:00",
@@ -159,11 +168,12 @@ class TestProcessTimeRange(unittest.TestCase):
             return TestBasalSync.get_example_ciq_basal_events()
 
         tconnect.controliq.therapy_timeline = fake_therapy_timeline
+        tconnect.controliq.therapy_events = self.stub_ciq_therapy_events
         tconnect.ws2.therapy_timeline_csv = self.stub_therapy_timeline_csv
 
         nightscout = NightscoutApi()
 
-        def fake_last_uploaded_entry(event_type):
+        def fake_last_uploaded_entry(event_type, **kwargs):
             if event_type == "Temp Basal":
                 return {
                     "created_at": "2021-03-16 00:20:21-04:00",
@@ -193,8 +203,8 @@ class TestProcessTimeRange(unittest.TestCase):
         })
         self.assertListEqual(nightscout.deleted_entries, [])
 
-    """No data in Nightscout. Uploads all bolus data from tconnect."""
-    def test_new_ciq_bolus_data(self):
+    """No data in Nightscout. Uploads all bolus data from tconnect via the WS2 API."""
+    def test_new_ciq_bolus_data_from_ws2(self):
         tconnect = TConnectApi()
 
         # datetimes are unused by the API fake
@@ -202,6 +212,7 @@ class TestProcessTimeRange(unittest.TestCase):
         end = datetime.datetime(2021, 4, 21, 12, 0)
 
         tconnect.controliq.therapy_timeline = self.stub_therapy_timeline
+        tconnect.controliq.therapy_events = self.stub_ciq_therapy_events
 
         bolusData = TestBolusSync.get_example_csv_bolus_events()
         def fake_therapy_timeline_csv(time_start, time_end):
@@ -231,6 +242,46 @@ class TestProcessTimeRange(unittest.TestCase):
         self.assertDictEqual(nightscout.put_entries, {})
         self.assertListEqual(nightscout.deleted_entries, [])
 
+    """No data in Nightscout. Uploads all bolus data from tconnect via CIQ therapy_events."""
+    def test_new_ciq_bolus_data_from_ciq_therapy_events(self):
+        tconnect = TConnectApi()
+
+        # datetimes are unused by the API fake
+        start = datetime.datetime(2022, 8, 9, 12, 0)
+        end = datetime.datetime(2021, 8, 10, 12, 0)
+
+        tconnect.controliq.therapy_timeline = self.stub_therapy_timeline
+
+        def fake_ciq_therapy_events(time_start, time_end):
+            return {
+                "event": BOLUS_FULL_EXAMPLES
+            }
+        tconnect.controliq.therapy_events = fake_ciq_therapy_events
+
+        tconnect.ws2.therapy_timeline_csv = self.stub_therapy_timeline_csv
+
+        nightscout = NightscoutApi()
+
+        nightscout.last_uploaded_entry = self.stub_last_uploaded_entry
+        nightscout.last_uploaded_activity = self.stub_last_uploaded_activity
+
+        process_time_range(tconnect, nightscout, start, end, pretend=False, features=[BOLUS, BASAL])
+
+        pprint.pprint(nightscout.uploaded_entries)
+        self.assertEqual(len(nightscout.uploaded_entries["treatments"]), len(BOLUS_FULL_EXAMPLES))
+        self.assertDictEqual(dict(nightscout.uploaded_entries), {
+            "treatments": [
+                # BGs are excluded because BG is not enabled in features for this test
+                NightscoutEntry.bolus(2.9, 0, "2022-07-21 11:55:24-04:00", notes="Automatic Bolus/Correction"),
+                NightscoutEntry.bolus(4.17, 25, "2022-07-21 12:29:21-04:00", notes="Standard"),
+                # NOTE: the extended bolus is 0.2+0.2 but we currently only surface standard
+                # BUG: inconsistency: we use the completion timestamp as the event timestamp for standard boluses,
+                # but the standard strand time for extended
+                NightscoutEntry.bolus(0.2, 0, "2022-08-09 23:20:04-04:00", notes="Extended 50.00%/0.00 (Override) (Extended)"),
+        ]})
+        self.assertDictEqual(nightscout.put_entries, {})
+        self.assertListEqual(nightscout.deleted_entries, [])
+
     """No data in Nightscout. Nothing should be updated in Nightscout without the BOLUS feature."""
     def test_bolus_data_not_updated_without_feature(self):
         tconnect = TConnectApi()
@@ -271,6 +322,7 @@ class TestProcessTimeRange(unittest.TestCase):
         end = datetime.datetime(2021, 4, 21, 12, 0)
 
         tconnect.controliq.therapy_timeline = self.stub_therapy_timeline
+        tconnect.controliq.therapy_events = self.stub_ciq_therapy_events
 
         iobData = TestIOBSync.get_example_csv_iob_events()
         def fake_therapy_timeline_csv(time_start, time_end):
@@ -307,6 +359,7 @@ class TestProcessTimeRange(unittest.TestCase):
         end = datetime.datetime(2021, 4, 21, 12, 0)
 
         tconnect.controliq.therapy_timeline = self.stub_therapy_timeline
+        tconnect.controliq.therapy_events = self.stub_ciq_therapy_events
 
         iobData = TestIOBSync.get_example_csv_iob_events()
         def fake_therapy_timeline_csv(time_start, time_end):
@@ -354,7 +407,7 @@ class TestProcessTimeRange(unittest.TestCase):
 
         nightscout.last_uploaded_entry = self.stub_last_uploaded_entry
 
-        def fake_last_uploaded_activity(activityType):
+        def fake_last_uploaded_activity(activityType, **kwargs):
             if activityType == IOB_ACTIVITYTYPE:
                 return iobData[0]
             return self.stub_last_uploaded_activity(activityType)
@@ -454,6 +507,7 @@ class TestProcessTimeRange(unittest.TestCase):
             }
 
         tconnect.controliq.therapy_timeline = fake_therapy_timeline
+        tconnect.controliq.therapy_events = self.stub_ciq_therapy_events
         tconnect.ws2.therapy_timeline_csv = self.stub_therapy_timeline_csv
         tconnect.ws2.basalsuspension = self.stub_ws2_basalsuspension
 
@@ -506,7 +560,7 @@ class TestProcessTimeRange(unittest.TestCase):
 
         nightscout = NightscoutApi()
 
-        def fake_last_uploaded_entry(event_type):
+        def fake_last_uploaded_entry(event_type, **kwargs):
             if event_type == "Sleep":
                 return {
                     "created_at": "2021-05-02 14:46:40-04:00",
@@ -519,7 +573,7 @@ class TestProcessTimeRange(unittest.TestCase):
                     "duration": 20,
                     "_id": "exercise"
                 }
-            return self.stub_last_uploaded_entry()
+            return self.stub_last_uploaded_entry(**kwargs)
 
         nightscout.last_uploaded_entry = fake_last_uploaded_entry
         nightscout.last_uploaded_activity = self.stub_last_uploaded_activity
@@ -628,7 +682,7 @@ class TestProcessTimeRange(unittest.TestCase):
 
         nightscout = NightscoutApi()
 
-        def fake_last_uploaded_entry(event_type):
+        def fake_last_uploaded_entry(event_type, **kwargs):
             if event_type == "Site Change":
                 return {
                     "created_at": "2021-12-04 16:18:10-05:00"
@@ -637,7 +691,7 @@ class TestProcessTimeRange(unittest.TestCase):
                 return {
                     "created_at": "2021-12-04 16:07:32-05:00"
                 }
-            return self.stub_last_uploaded_entry()
+            return self.stub_last_uploaded_entry(**kwargs)
 
         nightscout.last_uploaded_entry = fake_last_uploaded_entry
         nightscout.last_uploaded_activity = self.stub_last_uploaded_activity