version: bump to 0.9.6

This works in Windows, as-is. Updated the installation instructions with folder paths and scheduling instructions.

.github/workflows/publish-docker.yml

@@ -22,6 +22,16 @@ jobs:
           repository: jwoglom/tconnectsync/tconnectsync
           tag_with_ref: true
 
+      - name: Push latest tag to GitHub Packages
+        uses: docker/build-push-action@v1
+        with:
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+          registry: docker.pkg.github.com
+          repository: jwoglom/tconnectsync/tconnectsync
+          tags: latest
+          push: ${{ startsWith(github.ref, 'refs/tags/') }}
+
   push_to_dockerhub:
     name: Push Docker image to Docker Hub
     runs-on: ubuntu-latest

.github/workflows/python-package.yml

@@ -31,9 +31,13 @@ jobs:
     - name: Run pipenv check
       run: |
        # DDoS attacks in wheel and setuptools packages, not relevant
+       # root certificate store, not relevant
         pipenv check \
           --ignore 51499 \
-          --ignore 52495 
+          --ignore 52495 \
+          --ignore 52365 \
+          --ignore 59956 \
+          --ignore 58755
     - name: Lint with flake8
       run: |
         # stop the build if there are Python syntax errors or undefined names

README.md

@@ -55,7 +55,7 @@ The following synchronization features can be optionally enabled:
   * Sleep Mode (in Nightscout, appears with a start and end time)
 * `IOB`: Insulin-on-board data. Only the most recent IOB entry is saved to Nightscout, as an "activity". The Nightscout UI does not currently display this information. In order to read this value, you need to query the Nightscout activity API endpoint. If you don't know what that means, then there is no reason to enable this option.
 
-The following synchronization features are under development, [**but are not yet ready for use**](https://github.com/jwoglom/tconnectsync/issues/16):
+The following synchronization features are considered to be in alpha, and haven't been widely tested. If you want to use them, [set `ENABLE_TESTING_MODES=true` for them to show up](https://github.com/jwoglom/tconnectsync/blob/master/tconnectsync/features.py#L29):
 * `BOLUS_BG`: Adds BG readings which are associated with boluses on the pump into the Nightscout treatment object. It will determine whether the BG reading was automatically filled via the Dexcom connection on the pump or was manually entered by seeing if the BG reading matches the current CGM reading as known to the pump at that time. Support for this is nearly complete.
 * `CGM`: Adds Dexcom CGM readings from the pump to Nightscout as SGV (sensor glucose value) entries. This should only be used in a situation where xDrip/Dexcom Share/etc. is not used and the pump connection to the CGM will be the only source of CGM data to Nightscout. This requires additional testing before it should be considered ready.
 
@@ -122,9 +122,11 @@ First, ensure that you have **Python 3** with **Pip** installed:
   - For CentOS/Rocky Linux 8: 
     - `sudo dnf install python39-pip`
     - `sudo alternatives --set python /usr/bin/python3.9` 
-* **On Windows:** Install Ubuntu under the [Windows Subsystem for Linux](https://ubuntu.com/wsl).
-  Open the Ubuntu Terminal, then run `sudo apt install python3 python3-pip`.
-  Perform the remainder of the steps under the Ubuntu environment.
+* **On Windows:** 
+  - **With WSL:** Install Ubuntu under the [Windows Subsystem for Linux](https://ubuntu.com/wsl).
+    Open the Ubuntu Terminal, then run `sudo apt install python3 python3-pip`.
+    Perform the remainder of the steps under the Ubuntu environment.
+  - **Native:** Alternatively, you can run tconnectsync in native Windows with no modifications. However, this is less well-tested (open a GitHub issue if you experience any problems).
 
 Now install the `tconnectsync` package with pip:
 
@@ -136,7 +138,7 @@ To install into a user environment instead of system-wide for a more contained i
 $ pip3 install --user tconnectsync
 ````
   - This will place the tconnectsync binary file at ``/home/<username>/.local/bin/tconnectsync``
-
+  - For non-WSL Windows, it will be in ``<PYTHON DIRECTORY>\Lib\site-packages\tconnectsync``
 
 If the pip3 command is not found, run `python3 -m pip install tconnectsync` instead.
 
@@ -167,7 +169,7 @@ Move the `.env` file you created to the following folder:
 
 * **MacOS:** `/Users/<username>/.config/tconnectsync/.env`
 * **Linux:** `$HOME/.config/tconnectsync/.env`
-* **Windows:** `$HOME/.config/tconnectsync/.env` (inside WSL)
+* **Windows:** `$HOME/.config/tconnectsync/.env` (inside WSL) OR `C:\Users\<username>\.config\tconnectsync` (native Windows)
 
 ```
 $ tconnectsync --check-login
@@ -402,6 +404,20 @@ An example of a user crontab `crontab -e` if not running system-wide, which runs
 
 You can use one of the same `run.sh` files referenced above, but remove the `--auto-update` flag since you are handling the functionality for running the script periodically yourself.
 
+### For Native Windows 
+
+Create a batch file 'tconnectsync.bat' file containing:
+
+```
+python "C:\Users\<USERNAME>\AppData\Local\Programs\Python\<PYTHONVERSIONDIRECTORY>\Lib\site-packages\tconnectsync\main.py" --auto-update
+```
+
+If `python` does not exist in your path, specify the full path to `python.exe`.
+
+If main.py doesn't exist in `C:\Users\<USERNAME>\AppData\Local\Programs\Python\<PYTHONVERSIONDIRECTORY>\Lib\site-packages\tconnectsync\`, create it to match the copy in this repository.
+
+[Use Windows Task Scheduler](https://www.windowscentral.com/how-create-automated-task-using-task-scheduler-windows-10) to run this batch file on a scheduled basis.
+
 ## Tandem APIs
 
 This application utilizes three separate Tandem APIs for obtaining t:connect data, referenced here by the identifying part of their URLs:

setup.cfg

@@ -1,6 +1,6 @@
 [metadata]
 name = tconnectsync
-version = 0.9.3
+version = 0.9.6
 author = James Woglom 
 author_email = j@wogloms.net
 description = Syncs Tandem t:connect pump data to Nightscout for the t:slim X2

tconnectsync/__init__.py

@@ -18,11 +18,12 @@ try:
         TCONNECT_PASSWORD,
         NS_URL,
         NS_SECRET,
-        NS_SKIP_TLS_VERIFY
+        NS_SKIP_TLS_VERIFY,
+        PUMP_SERIAL_NUMBER
     )
     from . import secret
-except Exception:
-    print('Unable to read secret.py')
+except Exception as e:
+    print('Unable to read secrets from secret.py', e)
     sys.exit(1)
 
 
@@ -73,6 +74,16 @@ def main(*args, **kwargs):
     if time_end < time_start:
         raise Exception('time_start must be before time_end')
 
+
+    if TCONNECT_EMAIL == 'email@email.com':
+        logging.warn('NO USERNAME WAS PROVIDED. Ensure you have set TCONNECT_EMAIL appropriately.')
+    if TCONNECT_PASSWORD == 'password':
+        logging.warn('NO PASSWORD WAS PROVIDED. Ensure you have set TCONNECT_PASSWORD appropriately.')
+    if NS_URL == 'https://yournightscouturl/':
+        logging.warn('NO NIGHTSCOUT URL WAS PROVIDED. Ensure your have set NS_URL appropriately.')
+    if PUMP_SERIAL_NUMBER == '11111111':
+        logging.warn('NO PUMP SERIAL NUMBER WAS PROVIDED. Ensure you have set PUMP_SERIAL_NUMBER appropriately.')
+
     tconnect = TConnectApi(TCONNECT_EMAIL, TCONNECT_PASSWORD)
 
     nightscout = NightscoutApi(NS_URL, NS_SECRET, NS_SKIP_TLS_VERIFY)

tconnectsync/api/common.py

@@ -127,7 +127,7 @@ def split_days_range(start_a, end_a, days: int = 5) -> List[Tuple[str, str]]:
 class ApiException(Exception):
     def __init__(self, status_code, text, *args, **kwargs):
         self.status_code = status_code
-        super().__init__('%s (HTTP %s)' % (text, status_code), *args, **kwargs)
+        super().__init__('%s%s' % (text, ' (HTTP %s)' % status_code if status_code else ''), *args, **kwargs)
 
 class ApiLoginException(ApiException):
     pass

tconnectsync/api/controliq.py

@@ -5,7 +5,7 @@ import logging
 
 from bs4 import BeautifulSoup
 
-from ..util import timeago
+from ..util import timeago, cap_length
 from .common import parse_date, base_headers, base_session, ApiException, ApiLoginException
 
 logger = logging.getLogger(__name__)
@@ -14,7 +14,7 @@ class ControlIQApi:
     BASE_URL = 'https://tdcservices.tandemdiabetes.com/'
     LOGIN_URL = 'https://tconnect.tandemdiabetes.com/login.aspx?ReturnUrl=%2f'
 
-    LAST_CONFIRMED_SOFTWARE_VERSION = 't:connect 7.14.0.1'
+    LAST_CONFIRMED_SOFTWARE_VERSION = 't:connect 7.16.0.1'
 
     userGuid = None
     accessToken = None
@@ -34,12 +34,20 @@ class ControlIQApi:
             data = self._build_login_data(email, password, soup)
 
             req = s.post(self.LOGIN_URL, data=data, headers={'Referer': self.LOGIN_URL, **base_headers()}, allow_redirects=False)
+            # HTTP 200 is reported when credentials are incorrect
+            if req.status_code == 200:
+                login_error = self._find_login_error(req.text)
+                if not login_error:
+                    login_error = 'Check your login credentials.'
+                raise ApiLoginException(None, 'Error logging in to t:connect: %s' % login_error)
+
             if req.status_code != 302:
-                raise ApiLoginException(req.status_code, 'Error logging in to t:connect. Check your login credentials.')
+                raise ApiLoginException(req.status_code, 'Error logging in to t:connect')
 
 
             fwd = s.post(urllib.parse.urljoin(self.LOGIN_URL, req.headers['Location']), cookies=req.cookies, headers=base_headers())
             if fwd.status_code != 200:
+                logger.warn("Received non-HTTP 200: %s" % req.text)
                 raise ApiException(fwd.status_code, 'Error retrieving t:connect login cookies.')
 
             self.userGuid = req.cookies['UserGUID']
@@ -79,6 +87,14 @@ class ControlIQApi:
             "txtLoginPassword_ClientState": '{"enabled":true,"emptyMessage":"","validationText":"%s","valueAsString":"%s","lastSetTextBoxValue":"%s"}' % (password, password, password)
         }
 
+    def _find_login_error(self, text):
+        try:
+            soup = BeautifulSoup(text, features='lxml')
+            notice_error = soup.select_one(".notice_error").text.strip()
+            return notice_error
+        except Exception:
+            return None
+
     def needs_relogin(self):
         diff = (arrow.get(self.accessTokenExpiresAt) - arrow.get())
         return (diff.seconds <= 5 * 60)

tconnectsync/api/webui.py

@@ -30,10 +30,14 @@ class WebUIScraper:
         return self.controliq.needs_relogin()
 
     def _get(self, endpoint):
-        r = self.controliq.loginSession.get(self.BASE_URL + endpoint, headers=base_headers())
+        r = self.controliq.loginSession.get(self.BASE_URL + endpoint, headers=base_headers(), allow_redirects=True)
 
         if r.status_code != 200:
             raise ApiException(r.status_code, "WebUIScraper HTTP %s response: %s" % (str(r.status_code), r.text))
+
+        if 'login.aspx' in r.url:
+            raise ApiException(401, "WebUIScraper HTTP %s response for login page, returning 401: %s" % (str(r.status_code), r.url))
+
         return r
 
 

tconnectsync/util/__init__.py

@@ -28,4 +28,9 @@ def removesuffix(input_string, suffix):
 def removeprefix(input_string, prefix):
     if prefix and input_string.startswith(prefix):
         return input_string[len(prefix):]
-    return input_string
+    return input_string
+
+def cap_length(text, maxlen):
+    if not text or len(text) <= maxlen:
+        return text
+    return '%s[...]%s' % (text[:maxlen//2], text[maxlen//-2:])

tests/api/test_controliq.py

@@ -111,12 +111,65 @@ class TestControlIQApi(unittest.TestCase):
 
                 text=post_callback)
 
-            self.assertRaises(ApiLoginException, ciq.login, 'email@email.com', 'password')
+            self.assertRaisesRegex(ApiLoginException, 'Error logging in to t:connect: Check your login credentials.', ciq.login, 'email@email.com', 'password')
 
             self.assertIsNone(ciq.userGuid)
             self.assertIsNone(ciq.accessToken)
             self.assertIsNone(ciq.accessTokenExpiresAt)
 
+    def test_login_invalid_credentials_parsed_message(self):
+        ciq = ControlIQApi()
+        ciq.LOGIN_URL = RealControlIQApi.LOGIN_URL
+        ciq.login = lambda email, password: RealControlIQApi.login(ciq, email, password)
+
+        with requests_mock.Mocker() as m:
+            m.get('https://tconnect.tandemdiabetes.com/login.aspx?ReturnUrl=%2f',
+                request_headers=base_headers(),
+
+                text=self.LOGIN_HTML)
+
+            def post_callback(request, context):
+                context.status_code = 200
+                return '<html><body><div class="notice_error" id="literalMessage" style="">The email address or password you entered is invalid. Please re-enter and try again.</div></body></html>'
+
+            m.post('https://tconnect.tandemdiabetes.com/login.aspx?ReturnUrl=%2f',
+                request_headers={'Referer': ciq.LOGIN_URL, **base_headers()},
+
+                text=post_callback)
+
+            self.assertRaisesRegex(ApiLoginException, 'Error logging in to t:connect: The email address or password you entered is invalid. Please re-enter and try again.', ciq.login, 'email@email.com', 'password')
+
+            self.assertIsNone(ciq.userGuid)
+            self.assertIsNone(ciq.accessToken)
+            self.assertIsNone(ciq.accessTokenExpiresAt)
+
+    def test_login_unexpected_http_code(self):
+        ciq = ControlIQApi()
+        ciq.LOGIN_URL = RealControlIQApi.LOGIN_URL
+        ciq.login = lambda email, password: RealControlIQApi.login(ciq, email, password)
+
+        with requests_mock.Mocker() as m:
+            m.get('https://tconnect.tandemdiabetes.com/login.aspx?ReturnUrl=%2f',
+                request_headers=base_headers(),
+
+                text=self.LOGIN_HTML)
+
+            def post_callback(request, context):
+                context.status_code = 500
+                return '<html><body>...</body></html>'
+
+            m.post('https://tconnect.tandemdiabetes.com/login.aspx?ReturnUrl=%2f',
+                request_headers={'Referer': ciq.LOGIN_URL, **base_headers()},
+
+                text=post_callback)
+
+            self.assertRaisesRegex(ApiLoginException, 'Error logging in to t:connect \(HTTP 500\)', ciq.login, 'email@email.com', 'password')
+
+            self.assertIsNone(ciq.userGuid)
+            self.assertIsNone(ciq.accessToken)
+            self.assertIsNone(ciq.accessTokenExpiresAt)
+
+
     def fake_get_with_http_code(self, http_code, expected_endpoint, num_times):
         tries = 0
         def fake_get(endpoint, query):

tests/api/test_webui.py

@@ -18,7 +18,7 @@ from .fake import ControlIQApi
 from tconnectsync.api.controliq import ControlIQApi as RealControlIQApi
 from tconnectsync.api.common import ApiException, ApiLoginException, base_headers
 
-class TestWebUIScraper(unittest.TestCase):
+class TestWebUIScraperMyDevices(unittest.TestCase):
     maxDiff = None
 
     DEVICES_HTML = """
@@ -319,6 +319,64 @@ class TestWebUIScraper(unittest.TestCase):
                     'guid': None
                 })})
     
+    REDIRECT_HTML = """<html><head><title>Object moved</title></head><body>
+<h2>Object moved to <a href="/login.aspx?ReturnUrl=%2fmyaccount%2fmy_devices.aspx">here</a>.</h2>
+</body></html>"""
+
+    LOGIN_HTML = """<html></html>"""
+
+    def test_relogin_when_redirected_to_login_page(self):
+        ciq = ControlIQApi()
+        ciq.needs_relogin = lambda: False
+        ciq.loginSession = requests.Session()
+        ciq.LOGIN_URL = RealControlIQApi.LOGIN_URL
+        ciq._email = 'EMAIL'
+        ciq._password = 'PASSWORD'
+
+        login_times = []
+        def fake_login(email, password):
+            self.assertEqual(email, ciq._email)
+            self.assertEqual(password, ciq._password)
+
+            login_times.append(1)
+            return True
+
+        ciq.login = fake_login
+        webui = WebUIScraper(ciq)
+
+        ciq.login(ciq._email, ciq._password)
+        self.assertEqual(len(login_times), 1)
+
+        with requests_mock.Mocker() as m:
+            m.get('https://tconnect.tandemdiabetes.com/myaccount/my_devices.aspx',
+                request_headers=base_headers(),
+
+                status_code=302,
+                headers={'Location': 'https://tconnect.tandemdiabetes.com/login.aspx?ReturnUrl=%2fmyaccount%2fmy_devices.aspx'},
+                text=self.REDIRECT_HTML)
+
+            m.get('https://tconnect.tandemdiabetes.com/login.aspx?ReturnUrl=%2fmyaccount%2fmy_devices.aspx',
+                request_headers=base_headers(),
+
+                status_code=200,
+                text=self.LOGIN_HTML)
+
+            self.assertRaises(ApiException, webui.my_devices)
+            self.assertEqual(len(login_times), 2)
+
+        with requests_mock.Mocker() as m:
+            m.get('https://tconnect.tandemdiabetes.com/myaccount/my_devices.aspx',
+                request_headers=base_headers(),
+
+                text=self.DEVICES_HTML)
+
+            devices = webui.my_devices()
+            self.assertEqual(len(login_times), 2)
+
+
+class TestWebUIScraperPumpSettings(unittest.TestCase):
+    maxDiff = None
+
     PUMP_SETTINGS_HTML = """
 <!DOCTYPE html
     PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">