Bug 1706032 - python-boto FTBFS with Python 3.8 / nonresponsive maintainer
Summary: python-boto FTBFS with Python 3.8 / nonresponsive maintainer
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-boto
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Gwyn Ciesla
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1737800
Blocks: PYTHON38 1731534
TreeView+ depends on / blocked
 
Reported: 2019-05-03 12:19 UTC by Miro Hrončok
Modified: 2019-08-24 18:06 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-08-24 18:06:12 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Full log from Copr (391.74 KB, text/plain)
2019-05-03 12:19 UTC, Miro Hrončok
no flags Details
2.49.0 mock log (205.09 KB, text/plain)
2019-08-13 14:52 UTC, Gwyn Ciesla
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github boto boto issues 3879 0 None None None 2019-08-13 14:52:12 UTC

Description Miro Hrončok 2019-05-03 12:19:09 UTC
Created attachment 1562365 [details]
Full log from Copr

python-boto 2.45.0-12.fc31 fails to build with Python 3.8:

+ /usr/bin/python2 tests/test.py default
..............................................................................................................S.......................................................S...............................................S...S...............................................................................................................................................................................................................................................................................................................................................................................S..........................................................................S.........................................................................................................................................................................................................................................................................................SS....................................................................................................................
nose command: tests/test.py -a !notdefault tests/unit
----------------------------------------------------------------------
Ran 1061 tests in 4.254s

OK (SKIP=8)
+ /usr/bin/python3 tests/test.py default
/builddir/build/BUILD/boto-2.45.0/boto/connection.py:672: SyntaxWarning: invalid escape sequence \w
  '(?:(?P<user>[\w\-\.]+):(?P<pass>.*)@)?'
/builddir/build/BUILD/boto-2.45.0/boto/connection.py:673: SyntaxWarning: invalid escape sequence \w
  '(?P<host>[\w\-\.]+)'
/builddir/build/BUILD/boto-2.45.0/boto/connection.py:674: SyntaxWarning: invalid escape sequence \d
  '(?::(?P<port>\d+))?'
/builddir/build/BUILD/boto-2.45.0/boto/https_connection.py:80: SyntaxWarning: invalid escape sequence \.
  host_re = host.replace('.', '\.').replace('*', '[^.]*')
/builddir/build/BUILD/boto-2.45.0/boto/cloudfront/distribution.py:512: SyntaxWarning: invalid escape sequence \*
  """
/builddir/build/BUILD/boto-2.45.0/boto/cloudsearchdomain/layer1.py:79: SyntaxWarning: invalid escape sequence \`
  """
/builddir/build/BUILD/boto-2.45.0/boto/iam/connection.py:1114: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if tld is 'default':
/builddir/build/BUILD/boto-2.45.0/tests/unit/rds/test_connection.py:642: SyntaxWarning: invalid escape sequence \R
  logfile_sample = """
.....................................................................................................................................................................................................................S...S................................................................................................................................................................................................................................................................................................................................E.............................................S............................................................................S.............................................................................................................................................................................................................................................................................E.E.........SS....................................................................................................................
======================================================================
ERROR: test_item_lookup (tests.unit.ecs.test_connection.TestECSConnection)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/boto-2.45.0/tests/unit/ecs/test_connection.py", line 53, in test_item_lookup
    item_set = self.service_connection.item_lookup(
  File "/builddir/build/BUILD/boto-2.45.0/boto/ecs/__init__.py", line 105, in item_lookup
    return self.get_response('ItemLookup', params)
  File "/builddir/build/BUILD/boto-2.45.0/boto/ecs/__init__.py", line 78, in get_response
    xml.sax.parseString(body.encode('utf-8'), h)
  File "/usr/lib64/python3.8/xml/sax/__init__.py", line 48, in parseString
    parser.parse(inpsrc)
  File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 111, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib64/python3.8/xml/sax/xmlreader.py", line 125, in parse
    self.feed(buffer)
  File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 217, in feed
    self._parser.Parse(data, isFinal)
  File "/builddir/build/BUILD/Python-3.8.0a3/Modules/pyexpat.c", line 459, in EndElement
  File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 336, in end_element
    self._cont_handler.endElement(name)
  File "/builddir/build/BUILD/boto-2.45.0/boto/handler.py", line 41, in endElement
    self.nodes[-1][1].endElement(name, self.current_text, self.connection)
  File "/builddir/build/BUILD/boto-2.45.0/boto/ecs/item.py", line 136, in endElement
    self.curItem.endElement(name, value, connection)
  File "/builddir/build/BUILD/boto-2.45.0/boto/ecs/item.py", line 70, in endElement
    self._xml.write("%s</%s>" % (cgi.escape(value).replace("&amp;amp;", "&amp;"), name))
AttributeError: module 'cgi' has no attribute 'escape'
-------------------- >> begin captured logging << --------------------
boto: DEBUG: Using access key provided by client.
boto: DEBUG: Using secret key provided by client.
boto: DEBUG: Method: GET
boto: DEBUG: Path: /onca/xml
boto: DEBUG: Data: 
boto: DEBUG: Headers: {}
boto: DEBUG: Host: ecs.amazonaws.com
boto: DEBUG: Port: 443
boto: DEBUG: Params: {'ItemId': '0316067938', 'ResponseGroup': 'Reviews', 'Service': 'AWSECommerceService', 'Operation': 'ItemLookup', 'Version': '2010-11-01'}
boto: DEBUG: establishing HTTPS connection: host=ecs.amazonaws.com, kwargs={'timeout': 70, 'port': 443}
boto: DEBUG: Token: None
boto: DEBUG: using _calc_signature_2
boto: DEBUG: query string: AWSAccessKeyId=aws_access_key_id&ItemId=0316067938&Operation=ItemLookup&ResponseGroup=Reviews&Service=AWSECommerceService&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2019-05-03T00%3A37%3A04Z&Version=2010-11-01
boto: DEBUG: string_to_sign: GET
ecs.amazonaws.com
/onca/xml
AWSAccessKeyId=aws_access_key_id&ItemId=0316067938&Operation=ItemLookup&ResponseGroup=Reviews&Service=AWSECommerceService&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2019-05-03T00%3A37%3A04Z&Version=2010-11-01
boto: DEBUG: len(b64)=44
boto: DEBUG: base64 encoded digest: b'Pd82Yk/fVDQrlV+yx+l0duvpVN+whCWjFtUKnRwkiEo='
boto: DEBUG: query_string: AWSAccessKeyId=aws_access_key_id&ItemId=0316067938&Operation=ItemLookup&ResponseGroup=Reviews&Service=AWSECommerceService&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2019-05-03T00%3A37%3A04Z&Version=2010-11-01 Signature: b'Pd82Yk/fVDQrlV+yx+l0duvpVN+whCWjFtUKnRwkiEo='
boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.45.0 Python/3.8.0a3 Linux/4.16.13-300.fc28.x86_64', 'Content-Length': '0', 'Host': 'ecs.amazonaws.com'}
boto: DEBUG: Response headers: []
boto: DEBUG: 
            <Items>
              <Request>
              <IsValid>True</IsValid>
              <ItemLookupRequest>
                <ItemId>B00008OE6I</ItemId>
              </ItemLookupRequest>
              </Request>
              <Item>
                <ASIN>B00008OE6I</ASIN>
                <ItemAttributes>
                <Manufacturer>Canon</Manufacturer>
                <ProductGroup>Photography</ProductGroup>
                <Title>Canon PowerShot S400 4MP Digital Camera w/ 3x Optical Zoom</Title>
               </ItemAttributes>
              </Item>
            </Items>
        
--------------------- >> end captured logging << ---------------------

======================================================================
ERROR: test_constructor (tests.unit.utils.test_utils.TestPassword)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/boto-2.45.0/tests/unit/utils/test_utils.py", line 104, in test_constructor
    password.set('foo')
  File "/builddir/build/BUILD/boto-2.45.0/boto/utils.py", line 780, in set
    self.str = self.hashfunc(value).hexdigest()
  File "/builddir/build/BUILD/boto-2.45.0/tests/unit/utils/test_utils.py", line 101, in <lambda>
    hmac_hashfunc = lambda msg: hmac.new(b'mysecretkey', msg)
  File "/usr/lib64/python3.8/hmac.py", line 146, in new
    return HMAC(key, msg, digestmod)
  File "/usr/lib64/python3.8/hmac.py", line 49, in __init__
    raise ValueError('`digestmod` is required.')
ValueError: `digestmod` is required.

======================================================================
ERROR: test_hmac (tests.unit.utils.test_utils.TestPassword)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/boto-2.45.0/tests/unit/utils/test_utils.py", line 93, in test_hmac
    self.clstest(HMACPassword)
  File "/builddir/build/BUILD/boto-2.45.0/tests/unit/utils/test_utils.py", line 61, in clstest
    self.assertNotEquals(password, 'foo')
  File "/usr/lib64/python3.8/unittest/case.py", line 1394, in deprecated_func
    return original_func(*args, **kwargs)
  File "/usr/lib64/python3.8/unittest/case.py", line 901, in assertNotEqual
    if not first != second:
  File "/builddir/build/BUILD/boto-2.45.0/boto/utils.py", line 790, in __eq__
    return str(self.hashfunc(other).hexdigest()) == str(self.str)
  File "/builddir/build/BUILD/boto-2.45.0/tests/unit/utils/test_utils.py", line 88, in hmac_hashfunc
    return hmac.new(b'mysecretkey', msg)
  File "/usr/lib64/python3.8/hmac.py", line 146, in new
    return HMAC(key, msg, digestmod)
  File "/usr/lib64/python3.8/hmac.py", line 49, in __init__
    raise ValueError('`digestmod` is required.')
ValueError: `digestmod` is required.

----------------------------------------------------------------------
Ran 1061 tests in 3.413s

FAILED (SKIP=6, errors=3)

There are two problems here:



AttributeError: module 'cgi' has no attribute 'escape'

https://docs.python.org/3.8/whatsnew/3.8.html#api-and-feature-removals

> parse_qs, parse_qsl, and escape are removed from cgi module. They are deprecated from Python 3.2 or older.

https://docs.python.org/3.7/library/cgi.html#cgi.escape

> Deprecated since version 3.2: This function is unsafe because quote is false by default, and therefore deprecated. Use html.escape() instead.




ValueError: `digestmod` is required.

https://docs.python.org/3.7/library/hmac.html#hmac.new

> Deprecated since version 3.4, will be removed in version 3.8: MD5 as implicit default digest for digestmod is deprecated.


Full log attached.

Comment 1 Miro Hrončok 2019-05-29 12:30:50 UTC
Among others, this block a rebuild of heat-cfntools. Any chance you could prioritize this please?

Comment 2 Miro Hrončok 2019-07-10 10:37:06 UTC
Please respond.

Comment 3 Garrett Holmstrom 2019-07-26 01:38:14 UTC
As much as I hate to admit it, my dev bandwidth for working on Fedora cloud stuff has been next to none lately and I haven't needed to use python-boto in a while.  Are you interested in helping out with this package?

Comment 4 Gwyn Ciesla 2019-08-13 14:52:42 UTC
Created attachment 1603407 [details]
2.49.0 mock log

Comment 5 Gwyn Ciesla 2019-08-13 14:53:08 UTC
Still fails on latest upstream, logged upstream.

Comment 6 Ben Cotton 2019-08-13 17:00:23 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 7 Ben Cotton 2019-08-13 17:03:14 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Comment 8 Miro Hrončok 2019-08-13 17:15:54 UTC
Python 3.8 will happen in Fedora 32, changing back to rawhide.

Comment 9 Miro Hrončok 2019-08-14 22:24:37 UTC
The coordinated rebuild of Python 3.8 has started in the `f32-python` side tag.

If you figure out how to rebuild this package, please don't rebuild it in regular rawhide, but use the side tag instead:

    on branch master:
    $ fedpkg build --target=f32-python

To wait for a build to show up in the side tag, do:

    $ koji wait-repo f32-python --build=<nvr>

Where <nvr> is name-version-release of the source package, e.g. python-foo-1.1-2.fc32.

An updated mock config is posted at:
http://copr.fedorainfracloud.org/coprs/g/python/python3.8/

Note that it will take a while before the essential packages are rebuilt, so don't expect all your dependencies to be available right away.

Thanks. Let us know if you need up to date info, or if you have any questions.



PS this message is mass posted to all the bugs that block the PYTHON38 bug. If this is also a Fedora 31 FTBFS bug and you manage to fix it, you can do a f31 build as usual:

    on branch f31:
    $ fedpkg build

Comment 10 Miro Hrončok 2019-08-21 16:35:22 UTC
The f32-python side tag has been merged. In order to rebuild the package, do it in regular rawhide, but please wait until python3-3.8 is tagged:

  $ koji wait-repo f32-build --build python3-3.8.0~b3-3.fc32


If your built already started in f32-python, after it is finished, please tag it to rawhide with:

  $ koji tag-build f32-pending <nvr>

For example:

  $ koji tag-build f32-pending libreoffice-6.3.0.4-3.fc32

Thanks!

(This comment is mass posted to all bugzillas blocking the PYTHON38 tracking bug.)

Comment 11 Miro Hrončok 2019-08-21 17:29:23 UTC
(Python 3.8 has landed in the rawhide buildroot.)

Comment 12 Gwyn Ciesla 2019-08-24 18:06:12 UTC
Works if I disable tests.


Note You need to log in before you can comment on or make changes to this bug.