Updated examples.ods for COINBASE

This reverts commit e4d395240d.

README.md

@@ -2,13 +2,8 @@
 
 This is a Python based extension for LibreOffice Calc to make stock market, index and FX data available in Calc 
 spreadsheets - currently supporting Yahoo's and Financial Times' finance websites using old-fashioned web scraping. 
-Starting with version 3 of the extension, Google was dropped as source as its finance website was changed dramatically
-requiring a complete re-work of the logic to pull data of it. No one seems to have complained, so I guess not a lot of 
-people rely on it. If there is popular demand, I can take a look at adding it back. Alternatively I'm open for 
-suggestions to add another provider with global exchange and asset class coverage.
 
-Additionally, starting with version 3 of the extension, it contains all non-standard Python packages it uses 
-(dateutil, pytz, pyparsing). Consequently, it is no longer necessary to install anything to your system.
+Starting with version 3.1.0, we received a contribution to get crypto data directly from Coinbase
 
 ### Feedback requested:
 
@@ -21,11 +16,12 @@ under menu item: Tools, Extension Manager...
 
 Please make sure, not to rename the OXT file when downloading and before installing: LO will mess up the installation otherwise and the extension won't work.
 
-Getting data should be a simple as having this in a cell: 
+Getting data should be as simple as having this in a cell: 
 - `=GETREALTIME("IBM",21,"YAHOO")`
 - `=GETREALTIME("IBM:NYQ",21,"FT")`
 - `=GETREALTIME("EURUSD","LAST_PRICE","FT")`
 - `=GETHISTORIC("IBM",90,"2020-12-01","YAHOO")`
+- `=GETREALTIME("ETH-USD","LAST_PRICE","COINBASE")`
 
 Codes 21 and 90 stand for "last price" and "close" (see below), respectively. 
 Only Yahoo has historic data available.
@@ -43,10 +39,11 @@ List of example URLs for checking symbols. You can start from these and search f
 is available on the website but not from the extension, a good place to start when raising an issue is to include a similar 
 URL to compare results.
 
-|Website|Symbol|Example URL for Vodafone Group Plc UK |
-| :--- | :--- | :--- |
-|YAHOO|VOD.L|https://finance.yahoo.com/quote/VOD.L|
-|FT|VOD:LSE|https://markets.ft.com/data/equities/tearsheet/summary?s=VOD:LSE|
+| Website  | Symbol  | Example URL for Vodafone Group Plc UK                            |
+|:---------|:--------|:-----------------------------------------------------------------|
+| YAHOO    | VOD.L   | https://finance.yahoo.com/quote/VOD.L                            |
+| FT       | VOD:LSE | https://markets.ft.com/data/equities/tearsheet/summary?s=VOD:LSE |
+| COINBASE | ETH-EUR | https://api.exchange.coinbase.com/products/ETH-EUR/stats         |
 
 ### LibreOffice: using , (comma) vs ; (semicolon) to separate arguments in formula
 
@@ -58,47 +55,47 @@ Depending on your system's language and default LibreOffice settings, you maybe
 
 ### List of supported data points
 
-You can either specify numbers or names (lower or upper case) - not all bits are available from all sources and they are not necessarily consistent across sources either.
+You can either specify numbers or names (lower or upper case) - not all bits are available from all sources, and they are not necessarily consistent across sources either.
 
-|Name|Code|YAHOO|FT|YAHOO (historic)|Notes|
-| :--- | :--- | :---: | :---: | :---: | :---: |
-|PREV_CLOSE|5|Yes|Yes|||
-|OPEN|6|Yes|Yes|Yes||
-|CHANGE|7|Yes|Yes|||
-|LAST_PRICE_DATE|8|Yes|Yes| |yyyy-mm-dd|
-|LAST_PRICE_TIME|10|Yes|Yes|||
-|CHANGE_IN_PERCENT|11|Yes|Yes|||
-|LOW|14|Yes|Yes|Yes||
-|HIGH|16|Yes|Yes|Yes||
-|LAST_PRICE|21|Yes|Yes|||
-|BID|22|Yes|Yes|||
-|ASK|25|Yes|Yes|||
-|HIGH_52_WEEK|24|Yes|Yes|||
-|LOW_52_WEEK|26|Yes|Yes|||
-|MARKET_CAP|27|Yes|Yes|||
-|BIDSIZE|30|Yes|Yes|||
-|ASKSIZE|31|Yes|Yes|||
-|VOLUME|35|Yes|Yes|Yes||
-|AVG_DAILY_VOL_3MONTH|39|Yes|Yes|||
-|BETA|67|Yes|Yes|||
-|EPS|68|Yes|Yes|||
-|PE_RATIO|69|Yes|Yes|||
-|DIV|70|Yes|Yes|||
-|DIV_YIELD|71|Yes|Yes|||
-|EX_DIV_DATE|72|Yes|Yes| |yyyy-mm-dd|
-|PAYOUT_RATIO|73|Yes|No|||
-|EXPIRY_DATE|74|Yes (on options)|No| |yyyy-mm-dd|
-|SHARES_OUT|75|Yes|Yes| ||
-|FREE_FLOAT|76|Yes|Yes| ||
-|CLOSE|90|No|No|Yes||
-|ADJ_CLOSE|91|No|No|Yes||
-|SECTOR|98|Yes|Yes|||
-|INDUSTRY|99|Yes|Yes|||
-|TICKER|101|Yes|Yes|||
-|EXCHANGE|102|Yes|No|||
-|CURRENCY|103|Yes|Yes|||
-|NAME|104|Yes|Yes|||
-|TIMEZONE|105|Yes|Yes|||
+| Name                 | Code |      YAHOO       | FT  | YAHOO (historic) | COINBASE |   Notes    |
+|:---------------------|:-----|:----------------:|:---:|:----------------:|----------|:----------:|
+| PREV_CLOSE           | 5    |       Yes        | Yes |||
+| OPEN                 | 6    |       Yes        | Yes |       Yes        | Yes      ||
+| CHANGE               | 7    |       Yes        | Yes |||
+| LAST_PRICE_DATE      | 8    |       Yes        | Yes |                  |          | yyyy-mm-dd |
+| LAST_PRICE_TIME      | 10   |       Yes        | Yes |||
+| CHANGE_IN_PERCENT    | 11   |       Yes        | Yes |||
+| LOW                  | 14   |       Yes        | Yes |       Yes        | Yes      ||
+| HIGH                 | 16   |       Yes        | Yes |       Yes        | Yes      ||
+| LAST_PRICE           | 21   |       Yes        | Yes |                  | Yes      |
+| BID                  | 22   |       Yes        | Yes |||
+| ASK                  | 25   |       Yes        | Yes |||
+| HIGH_52_WEEK         | 24   |       Yes        | Yes |||
+| LOW_52_WEEK          | 26   |       Yes        | Yes |||
+| MARKET_CAP           | 27   |       Yes        | Yes |||
+| BIDSIZE              | 30   |       Yes        | Yes |||
+| ASKSIZE              | 31   |       Yes        | Yes |||
+| VOLUME               | 35   |       Yes        | Yes |       Yes        | Yes      ||
+| AVG_DAILY_VOL_3MONTH | 39   |       Yes        | Yes |||
+| BETA                 | 67   |       Yes        | Yes |||
+| EPS                  | 68   |       Yes        | Yes |||
+| PE_RATIO             | 69   |       Yes        | Yes |||
+| DIV                  | 70   |       Yes        | Yes |||
+| DIV_YIELD            | 71   |       Yes        | Yes |||
+| EX_DIV_DATE          | 72   |       Yes        | Yes |                  |          | yyyy-mm-dd |
+| PAYOUT_RATIO         | 73   |       Yes        | No  |||
+| EXPIRY_DATE          | 74   | Yes (on options) | No  |                  |          | yyyy-mm-dd |
+| SHARES_OUT           | 75   |       Yes        | Yes |                  |          ||
+| FREE_FLOAT           | 76   |       Yes        | Yes |                  |          ||
+| CLOSE                | 90   |        No        | No  |       Yes        |          ||
+| ADJ_CLOSE            | 91   |        No        | No  |       Yes        |          ||
+| SECTOR               | 98   |       Yes        | Yes |||
+| INDUSTRY             | 99   |       Yes        | Yes |||
+| TICKER               | 101  |       Yes        | Yes |                  | Yes      |
+| EXCHANGE             | 102  |       Yes        | No  |||
+| CURRENCY             | 103  |       Yes        | Yes |                  | Yes      |
+| NAME                 | 104  |       Yes        | Yes |||
+| TIMEZONE             | 105  |       Yes        | Yes |||
 
 ### Dealing with missing data:
 
@@ -133,8 +130,8 @@ python3 -m unittest discover src
 
 ### Tested with:
 - Windows 10 / LibreOffice Calc 7.1.2.2 / Python 3.8.8
-- Ubuntu 20.04 / LibreOffice Calc 6.4.3.2 / Python 3.8.2
-- MacOS 10.15.7 / LibeOffice Calc 7.2.0.4 / Python 3.8.10
+- Ubuntu 20.04.5 / LibreOffice Calc 6.4.7.2 / Python 3.8.10
+- MacOS 10.15.7 / LibreOffice Calc 7.2.0.4 / Python 3.8.10
 
 (Previous versions)
 - Debian 10.3  / LibreOffice Calc 6.1.5.2 / Python 3.7.3

compile.sh

@@ -5,8 +5,20 @@ set -o nounset
 set -o pipefail
 set -o noclobber
 
-export PATH=$PATH:/usr/lib/libreoffice/sdk/bin
-export PATH=$PATH:/usr/lib/libreoffice/program
+if [[ $OSTYPE == "darwin"* ]]; then
+  echo MacOS
+  # Assuming both are installed in the applications folder
+  # Required some steps to make it work for MacOS M1, mind the `find` call which could return more than one (shouldn't)
+  # install_name_tool -change @__VIA_LIBRARY_PATH__/libreglo.dylib $(find /Applications -name "libreglo.dylib") /Applications/LibreOffice7.4_SDK/bin/idlc
+  # install_name_tool -change @__VIA_LIBRARY_PATH__/libuno_sal.dylib.3 $(find /Applications -name "libuno_sal.dylib.3") /Applications/LibreOffice7.4_SDK/bin/idlc
+  # install_name_tool -change @__VIA_LIBRARY_PATH__/libuno_salhelpergcc3.dylib.3 $(find /Applications -name "libuno_salhelpergcc3.dylib.3") /Applications/LibreOffice7.4_SDK/bin/idlc
+  # codesign --force -s - $(find /Applications -name "idlc")
+  export PATH=$PATH:/Applications/LibreOffice7.4_SDK/bin
+  export PATH=$PATH:/Applications/LibreOffice.app/Contents/MacOS
+else
+  export PATH=$PATH:/usr/lib/libreoffice/sdk/bin
+  export PATH=$PATH:/usr/lib/libreoffice/program
+fi
 
 # Setup build directories
 
@@ -37,6 +49,7 @@ cp -f "${PWD}"/src/tz.py "${PWD}"/build/
 cp -f "${PWD}"/src/financials_ft.py "${PWD}"/build/
 cp -f "${PWD}"/src/financials_google.py "${PWD}"/build/
 cp -f "${PWD}"/src/financials_yahoo.py "${PWD}"/build/
+cp -f "${PWD}"/src/financials_coinbase.py "${PWD}"/build/
 
 # this copies python modules dateutil, pytz, pyparsing to extension so it doesn't have to be installed by user
 

src/financials.py

@@ -56,6 +56,7 @@ import six
 from datacode import Datacode
 import financials_google as google
 import financials_yahoo as yahoo
+import financials_coinbase as coinbase
 import financials_ft as ft
 
 implementation_name = "com.financials.getinfo.python.FinancialsImpl"  # as defined in Financials.xcu
@@ -96,6 +97,7 @@ class FinancialsImpl(unohelper.Base, Financials):
         self.ctx = ctx
         self.google = google.createInstance(ctx)
         self.yahoo = yahoo.createInstance(ctx)
+        self.coinbase = coinbase.createInstance(ctx)
         self.ft = ft.createInstance(ctx)
 
     @profile
@@ -141,6 +143,8 @@ class FinancialsImpl(unohelper.Base, Financials):
                 s = self.yahoo.getRealtime(ticker, datacode)
             elif source == 'FT':
                 s = self.ft.getRealtime(ticker, datacode)
+            elif source == 'COINBASE':
+                s = self.coinbase.getRealtime(ticker, datacode)
             else:
                 s = 'Source \'{}\' not supported'.format(source)
 

src/financials_coinbase.py

@@ -0,0 +1,109 @@
+#  financials_coinbase.py
+#
+#  license: GNU LGPL
+#
+#  This library is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU Lesser General Public
+#  License as published by the Free Software Foundation; either
+#  version 3 of the License, or (at your option) any later version.
+
+
+import csv
+import datetime
+import logging
+import os
+import pprint
+import re
+import time
+import json
+
+import dateutil.parser
+import pytz
+
+import jsonParser
+from baseclient import BaseClient, HttpException
+from datacode import Datacode
+
+logger = logging.getLogger(__name__)
+
+
+# logger.setLevel(logging.DEBUG)
+
+class Coinbase(BaseClient):
+    def __init__(self, ctx):
+        super().__init__()
+
+        self.crumb = None
+        self.realtime = {}
+        self.js = jsonParser.jsonObject
+
+    def getRealtime(self, ticker, datacode):
+
+        """
+        Retrieve realtime data for ticker from Coinbase and cache it for further lookups
+
+        :param ticker: the ticker symbol e.g. ETH-EUR
+        :param datacode: the requested datacode, not all are supported
+        :return:
+        """
+
+        # remove white space
+        ticker = "".join(ticker.split())
+
+        # use cached value for up to 60 seconds
+        if ticker in self.realtime:
+            tick = self.realtime[ticker]
+            if time.time() - 60 < tick[Datacode.TIMESTAMP]:
+                return self._return_value(tick, datacode)
+            else:
+                del self.realtime[ticker]
+
+        url = 'https://api.exchange.coinbase.com/products/{}/stats'.format(ticker)
+
+        try:
+            text = self.urlopen(url, redirect=True, data=None, headers=None)
+        except BaseException as e:
+            logger.exception("BaseException ticker=%s datacode=%s", ticker, datacode)
+            return 'Coinbase.getRealtime({}, {}) - urlopen: {}'.format(ticker, datacode, e)
+
+        try:
+            with open(os.path.join(self.basedir, 'coinbase-{}.json'.format(ticker)), "w", encoding="utf-8") as text_file:
+                print(f"<!-- '{url}' -->\r\n\r\n{text}", file=text_file)
+        except BaseException:
+            logger.exception("BaseException ticker=%s datacode=%s", ticker, datacode)
+
+        try:
+            results = json.loads(text)
+
+        except BaseException as e:
+            logger.exception("BaseException ticker=%s datacode=%s", ticker, datacode)
+            return 'Coinbase.getRealtime({}, {}) - crumb: {}'.format(ticker, datacode, e)
+
+        try:
+            price = results['last']
+
+            if not price:
+                return 'Could not find price for \'{}\''.format(ticker)
+
+            if ticker not in self.realtime:
+                self.realtime[ticker] =  self.get_ticker()
+
+            tick = self.realtime[ticker]
+
+            tick[Datacode.TIMESTAMP] = time.time()
+            tick[Datacode.LAST_PRICE] = float(price)
+            tick[Datacode.OPEN] = float(results['open'])
+            tick[Datacode.HIGH] = float(results['high'])
+            tick[Datacode.LOW] = float(results['low'])
+            tick[Datacode.VOLUME] =  float(results['volume'])
+            tick[Datacode.TICKER] = ticker.split('-', 1)[0]
+            tick[Datacode.CURRENCY] = ticker.split('-', 1)[1]
+
+        except BaseException as e:
+            logger.exception("BaseException ticker=%s datacode=%s", ticker, datacode)
+            return 'Coinbase.getRealtime({}, {}) - process: {}'.format(ticker, datacode, e)
+
+        return self._return_value(self.realtime[ticker], datacode)
+
+def createInstance(ctx):
+    return Coinbase(ctx)

src/generate_metainfo.py

@@ -14,7 +14,7 @@ import os
 cur_dir = os.getcwd()
 
 addin_id = "com.financials.getinfo"
-addin_version = "3.0.9"
+addin_version = "3.1.0"
 addin_displayname = "Financial Market Extension"
 addin_publisher_link = "https://github.com/cmallwitz/Financials-Extension"
 addin_publisher_name = "The Publisher"

src/test_coinbase.py

@@ -0,0 +1,37 @@
+#  test_yahoo.py
+#
+#  license: GNU LGPL
+#
+#  This library is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU Lesser General Public
+#  License as published by the Free Software Foundation; either
+#  version 3 of the License, or (at your option) any later version.
+
+import argparse
+import logging
+import os
+import pathlib
+import sys
+import unittest
+
+logging.basicConfig(level=logging.ERROR, format="%(asctime)s %(name)s %(levelname)s %(message)s")
+
+import financials
+from datacode import Datacode
+import testutils
+
+financials = financials.createInstance(None)
+
+
+class Test(unittest.TestCase):
+
+    def test_currency(self):
+        s = financials.getRealtime('ETH-EUR', Datacode.LAST_PRICE.value, 'COINBASE')
+        self.assertEqual(float, type(s), 'test_currency LAST_PRICE')
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('unittest_args', nargs='*')
+    args = parser.parse_args()
+    unit_argv = [sys.argv[0]] + args.unittest_args
+    unittest.main(argv=unit_argv)

src/test_ft.py

@@ -311,40 +311,40 @@ class Test(unittest.TestCase):
         self.assertEqual('General Industrials', s, 'test_TY_equity INDUSTRY {}'.format(s))
 
     def test_TLV_equity(self):
-        s = financials.getRealtime('DELT:TLV', 'LAST_PRICE', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'LAST_PRICE', 'FT')
         self.assertEqual(float, type(s), 'test_TLV_equity LAST_PRICE {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'OPEN', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'OPEN', 'FT')
         self.assertEqual(float, type(s), 'test_TLV_equity OPEN {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'LOW', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'LOW', 'FT')
         self.assertEqual(float, type(s), 'test_TLV_equity LOW {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'HIGH', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'HIGH', 'FT')
         self.assertEqual(float, type(s), 'test_TLV_equity HIGH {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'LOW_52_WEEK', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'LOW_52_WEEK', 'FT')
         self.assertEqual(float, type(s), 'test_TLV_equity LOW_52_WEEK {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'HIGH_52_WEEK', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'HIGH_52_WEEK', 'FT')
         self.assertEqual(float, type(s), 'test_TLV_equity HIGH_52_WEEK {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'MARKET_CAP', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'MARKET_CAP', 'FT')
         self.assertEqual(float, type(s), 'test_TLV_equity MARKET_CAP {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'VOLUME', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'VOLUME', 'FT')
         self.assertEqual(float, type(s), 'test_TLV_equity VOLUME {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'CURRENCY', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'CURRENCY', 'FT')
         self.assertEqual('ILa', s, 'test_TLV_equity CURRENCY')
 
-        s = financials.getRealtime('DELT:TLV', 'SECTOR', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'SECTOR', 'FT')
         self.assertEqual(str, type(s), 'test_TLV_equity SECTOR {}'.format(s))
-        self.assertEqual('Consumer Discretionary', s, 'test_TY_equity SECTOR {}'.format(s))
+        self.assertEqual('Financials', s, 'test_TY_equity SECTOR {}'.format(s))
 
-        s = financials.getRealtime('DELT:TLV', 'INDUSTRY', 'FT')
+        s = financials.getRealtime('LUMI:TLV', 'INDUSTRY', 'FT')
         self.assertEqual(str, type(s), 'test_TLV_equity INDUSTRY {}'.format(s))
-        self.assertEqual('Personal Goods', s, 'test_TY_equity INDUSTRY {}'.format(s))
+        self.assertEqual('Banks', s, 'test_TY_equity INDUSTRY {}'.format(s))
 
     def test_index(self):
         s = financials.getRealtime('INX:IOM', 'ticker', 'FT')