Bug 1574835

Summary: units_cur broken (again)
Product: [Fedora] Fedora Reporter: Martin Tarenskeen <m.tarenskeen>
Component: unitsAssignee: Kamil Dudka <kdudka>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 28CC: jci.redhat, kdudka, m.tarenskeen
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: units-2.16-5.fc29 units-2.14-6.fc27 units-2.16-5.fc28 units-2.18-1.fc28 units-2.18-1.fc29 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-11-09 05:25:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Martin Tarenskeen 2018-05-04 06:29:38 UTC
Description of problem:
units_cur is broken (again)

Version-Release number of selected component (if applicable):
units-2.16-4.fc28.x86_64

How reproducible:
always

Steps to Reproduce:
1. units_cur

Actual results:

Traceback (most recent call last):
  File "/usr/bin/units_cur", line 261, in <module>
    rates = [ data['resource']['fields']['price'] for data in webdata]
  File "/usr/bin/units_cur", line 261, in <listcomp>
    rates = [ data['resource']['fields']['price'] for data in webdata]
KeyError: 'price'


Expected results:
/usr/share/units/currency.units updated with new currency exchange values

Additional info:
Error was already there in Fedora 27

Comment 1 Kamil Dudka 2018-05-04 08:47:42 UTC
The error was not there the last time I tried the script.  Something must have changed on the web service that units takes the data from.

Comment 2 Kamil Dudka 2018-05-11 10:11:58 UTC
*** Bug 1576928 has been marked as a duplicate of this bug. ***

Comment 3 John Inzinna 2018-05-24 16:21:40 UTC
units_cur makes the assumption that the json document returned from yahoo will contain a set of data for every resource element, for example:
            {
                "resource": {
                    "classname": "Quote",
                    "fields": {
                        "name": "USD/ECS",
                        "price": "25000.000000",
                        "symbol": "ECS=X",
                        "ts": "1487964442",
                        "type": "currency",
                        "utctime": "2017-02-24T19:27:22+0000",
                        "volume": "0"
                    }
                }
            }

Currently, yahoo is returning an empty resource:
            {
                "resource": {
                    "classname": "Quote",
                    "fields": {

                    }
                }
            },

This can be addressed by making units_cur slightly more robust by allowing for an exception when creating the currency exchange lists.  I made this patch to units_cur (from units-2.16-4.fc28):

261,262c261,270
< rates = [ data['resource']['fields']['price'] for data in webdata]
< codes = [ data['resource']['fields']['symbol'][0:3] for data in webdata]
---
> rates = []
> codes = []
> for data in webdata:
>   try:
>     rate = data['resource']['fields']['price']
>     code = data['resource']['fields']['symbol'][0:3]
>     rates.append( rate )
>     codes.append( code )
>   except Exception:
>     pass

This allows for skipping any values that cause an exception trying to grab the expected data.

Comment 4 Kamil Dudka 2018-05-28 10:24:46 UTC
Thanks!  I have notified the upstream maintainer about the bug and proposed fix.

Comment 5 Kamil Dudka 2018-05-28 12:41:27 UTC
I have picked the upstream fix for this:

https://src.fedoraproject.org/cgit/rpms/units.git/commit/?id=8200b417

Comment 6 Fedora Update System 2018-05-28 13:18:23 UTC
units-2.14-6.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-d787458160

Comment 7 Fedora Update System 2018-05-28 13:18:28 UTC
units-2.16-5.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-8e52c5a761

Comment 8 Fedora Update System 2018-05-28 14:25:20 UTC
units-2.14-6.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-d787458160

Comment 9 Fedora Update System 2018-05-28 15:50:02 UTC
units-2.16-5.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-8e52c5a761

Comment 10 Fedora Update System 2018-06-06 12:58:22 UTC
units-2.14-6.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 11 Fedora Update System 2018-06-06 13:31:17 UTC
units-2.16-5.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 12 Martin Tarenskeen 2018-08-28 08:54:31 UTC
The problem (or a similar problem) has returned :-(

Error connecting to currency server:
404 Client Error: Not Found for url: https://finance.yahoo.com/webservice/v1/symbols/allcurrencies/quote?format=json.

I'm using units_cur from units-2.17-5.fc28.x86_64 now.

Comment 13 Kamil Dudka 2018-08-28 09:10:28 UTC
(In reply to Martin Tarenskeen from comment #12)
> Error connecting to currency server:
> 404 Client Error: Not Found for url:
> https://finance.yahoo.com/webservice/v1/symbols/allcurrencies/
> quote?format=json.

Looks like Yahoo! does not provide the currency rate data on that URL any more.

Comment 14 Martin Tarenskeen 2018-10-29 10:27:33 UTC
This bug has been fixed upstream in version 2.18

Comment 15 Kamil Dudka 2018-10-29 12:18:39 UTC
Indeed.  Let's update to 2.18 to fix this!

Comment 16 Fedora Update System 2018-10-29 12:33:47 UTC
units-2.18-1.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-d28abe8ef5

Comment 17 Fedora Update System 2018-10-29 12:33:54 UTC
units-2.18-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-5d8cba8f94

Comment 18 Fedora Update System 2018-10-31 17:30:36 UTC
units-2.18-1.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-d28abe8ef5

Comment 19 Fedora Update System 2018-10-31 18:51:14 UTC
units-2.18-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-5d8cba8f94

Comment 20 Fedora Update System 2018-11-09 05:25:19 UTC
units-2.18-1.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2018-11-09 06:01:50 UTC
units-2.18-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.