Bug 1595331

Summary: python-peewee FTFBS with Python 3.7
Product: [Fedora] Fedora Reporter: Viliam Križan <vkrizan>
Component: python-peeweeAssignee: Viliam Križan <vkrizan>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: cstratak, mhroncok, vkrizan
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-peewee-2.10.2-5.fc29 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-26 20:39:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1565020    

Description Viliam Križan 2018-06-26 15:57:13 UTC
Description of problem: python-peewee FTFBS with Python 3.7.

RuntimeError: generator raised StopIteration
  https://www.python.org/dev/peps/pep-0479/
  Solution: return instead (works with 2.7 as well) 


~~~
======================================================================
ERROR: test_iterator (playhouse.tests.test_query_results.TestQueryResultWrapper)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2353, in iterator
    yield self.iterate()
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2345, in iterate
    raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_query_results.py", line 111, in test_iterator
    usernames = [u.username for u in qr.iterator()]
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_query_results.py", line 111, in <listcomp>
    usernames = [u.username for u in qr.iterator()]
RuntimeError: generator raised StopIteration
======================================================================
ERROR: test_iterator_extended (playhouse.tests.test_query_results.TestQueryResultWrapper)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2353, in iterator
    yield self.iterate()
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2345, in iterate
    raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_query_results.py", line 158, in test_iterator_extended
    for user in qr.iterator():
RuntimeError: generator raised StopIteration
======================================================================
ERROR: test_iterator_query_method (playhouse.tests.test_query_results.TestQueryResultWrapper)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2353, in iterator
    yield self.iterate()
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2345, in iterate
    raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_query_results.py", line 131, in test_iterator_query_method
    usernames = [u.username for u in qr.iterator()]
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_query_results.py", line 131, in <listcomp>
    usernames = [u.username for u in qr.iterator()]
RuntimeError: generator raised StopIteration
======================================================================
ERROR: test_dump_field_types (playhouse.tests.test_csv_utils.TestCSVDump)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2353, in iterator
    yield self.iterate()
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2345, in iterate
    raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_csv_utils.py", line 265, in test_dump_field_types
    self.assertCSV(query, expected)
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_csv_utils.py", line 202, in assertCSV
    final_buf = dump_csv(query, buf, **kwargs)
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/csv_utils.py", line 341, in dump_csv
    for row in query.tuples().iterator():
RuntimeError: generator raised StopIteration
======================================================================
ERROR: test_dump_functions (playhouse.tests.test_csv_utils.TestCSVDump)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2353, in iterator
    yield self.iterate()
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2345, in iterate
    raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_csv_utils.py", line 237, in test_dump_functions
    self.assertCSV(query, expected)
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_csv_utils.py", line 202, in assertCSV
    final_buf = dump_csv(query, buf, **kwargs)
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/csv_utils.py", line 341, in dump_csv
    for row in query.tuples().iterator():
RuntimeError: generator raised StopIteration
======================================================================
ERROR: test_dump_simple (playhouse.tests.test_csv_utils.TestCSVDump)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2353, in iterator
    yield self.iterate()
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2345, in iterate
    raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_csv_utils.py", line 212, in test_dump_simple
    self.assertCSV(User.select().order_by(User.id), expected)
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_csv_utils.py", line 202, in assertCSV
    final_buf = dump_csv(query, buf, **kwargs)
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/csv_utils.py", line 341, in dump_csv
    for row in query.tuples().iterator():
RuntimeError: generator raised StopIteration
======================================================================
ERROR: test_column_preservation (playhouse.tests.test_dataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2353, in iterator
    yield self.iterate()
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2345, in iterate
    raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_dataset.py", line 62, in test_column_preservation
    data = [(row['book_id'] or '', row['bookId'] or '') for row in books]
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_dataset.py", line 62, in <listcomp>
    data = [(row['book_id'] or '', row['bookId'] or '') for row in books]
RuntimeError: generator raised StopIteration
======================================================================
ERROR: test_magic_methods (playhouse.tests.test_dataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2353, in iterator
    yield self.iterate()
  File "/builddir/build/BUILD/peewee-2.10.2/peewee.py", line 2345, in iterate
    raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_dataset.py", line 206, in test_magic_methods
    users = sorted([u for u in user], key=operator.itemgetter('username'))
  File "/builddir/build/BUILD/peewee-2.10.2/playhouse/tests/test_dataset.py", line 206, in <listcomp>
    users = sorted([u for u in user], key=operator.itemgetter('username'))
RuntimeError: generator raised StopIteration
----------------------------------------------------------------------
~~~

Comment 1 Miro Hrončok 2018-06-26 16:09:50 UTC
The generator should return instead.

See https://www.python.org/dev/peps/pep-0479/

Comment 2 Miro Hrončok 2018-06-26 16:11:01 UTC
(Sorry, it took me a while to realize you already say that in the begging of the text.)

Comment 3 Viliam Križan 2018-06-26 17:38:25 UTC
(In reply to Miro Hrončok from comment #2)
> (Sorry, it took me a while to realize you already say that in the begging of
> the text.)

No problem =)


For some reason the scratch build hangs on running the test suite:
https://koji.fedoraproject.org/koji/taskinfo?taskID=27883743

Comment 4 Miro Hrončok 2018-06-26 18:08:30 UTC
You cannot convert all raise StopIteration to returns. You should amke sure that generators (functions that yield) never raise it. Sometimes that involves stuff like:


            try:
                yield self.iterate()
            except StopIteration:
                return

I'm trying a different patch in https://koji.fedoraproject.org/koji/taskinfo?taskID=27885548

Should I push if it works, or would you like to review?

Comment 5 Miro Hrončok 2018-06-26 18:12:12 UTC
Here it is anyway:


diff --git a/peewee.py b/peewee.py
index cd29348..44e7601 100644
--- a/peewee.py
+++ b/peewee.py
@@ -2350,7 +2350,10 @@ class QueryResultWrapper(object):
 
     def iterator(self):
         while True:
-            yield self.iterate()
+            try:
+                yield self.iterate()
+            except StopIteration:
+                return
 
     def next(self):
         if self._idx < self._ct:
diff --git a/playhouse/_speedups.pyx b/playhouse/_speedups.pyx
index 184ee07..a9a297c 100644
--- a/playhouse/_speedups.pyx
+++ b/playhouse/_speedups.pyx
@@ -213,7 +213,10 @@ cdef class _QueryResultWrapper(object):
 
     def iterator(self):
         while True:
-            yield self.iterate()
+            try:
+                yield self.iterate()
+            except StopIteration:
+                return
 
     def __next__(self):
         cdef object inst


---------


Your patch made it iterate forever, because iterate (not a generator) never raised an exception and instead returned None, while iterator (a generator) was yielding Nones forever.

Comment 6 Miro Hrončok 2018-06-26 18:21:43 UTC
ppc64le fails with unrelated problem.

Comment 7 Viliam Križan 2018-06-26 20:04:32 UTC
Thanks Miro!  Feel free to push it.

Comment 8 Miro Hrončok 2018-06-26 20:20:07 UTC
I've pushed it and started another build, but the ppc64le problem will probably hit us again. https://koji.fedoraproject.org/koji/taskinfo?taskID=27887082

Comment 9 Viliam Križan 2018-06-26 20:25:18 UTC
Yeah the ppc64le have problems with concurrency.  Sometimes it fails.  Would `ExcludeArch` help?  I guess we do not need any PPC architecture.

Comment 10 Miro Hrončok 2018-06-26 20:39:31 UTC
It built.  ¯\_(ツ)_/¯ 

Excluding the arch would put unnecessary burden on noarch packages that (build)require pythonX-peewee.