Summary says it all, last working aarch64 build was https://koji.fedoraproject.org/koji/buildinfo?buildID=805232 Sep 28, 2016 All builds on aarch64 since then have failed. Presumably build hangs populating buildroot (build.log is empty), latest root.log stops at "DEBUG util.py:435: Running transaction" Tail end of latest root.log appended. Some folks on -devel list theorized that it was either ecl or time at fault, but that appears to be not the case, as this latest scratch build https://koji.fedoraproject.org/koji/taskinfo?taskID=18549458 Drops both of those from the buildroot, and it is still hanging (somewhere) ---------------------------------- DEBUG util.py:435: Installing: DEBUG util.py:435: aajohan-comfortaa-fonts noarch 2.004-7.fc26 build 150 k DEBUG util.py:435: desktop-file-utils aarch64 0.23-3.fc26 build 76 k DEBUG util.py:435: emacs-filesystem noarch 1:25.2-0.1.rc2.fc26 build 66 k DEBUG util.py:435: fontconfig aarch64 2.12.1-4.fc26 build 249 k DEBUG util.py:435: fontpackages-filesystem noarch 1.44-18.fc26 build 14 k DEBUG util.py:435: freetype aarch64 2.7.1-2.fc26 build 360 k DEBUG util.py:435: gcl aarch64 2.6.12-7.fc26 build 5.5 M DEBUG util.py:435: gcl-selinux aarch64 2.6.12-7.fc26 build 35 k DEBUG util.py:435: libX11 aarch64 1.6.4-6.fc26 build 611 k DEBUG util.py:435: libX11-common noarch 1.6.4-6.fc26 build 165 k DEBUG util.py:435: libXau aarch64 1.0.8-7.fc26 build 33 k DEBUG util.py:435: libXft aarch64 2.3.2-5.fc26 build 62 k DEBUG util.py:435: libXrender aarch64 0.9.10-2.fc26 build 30 k DEBUG util.py:435: libpng aarch64 2:1.6.28-2.fc26 build 119 k DEBUG util.py:435: libselinux-utils aarch64 2.6-4.fc27 build 165 k DEBUG util.py:435: libxcb aarch64 1.12-3.fc26 build 214 k DEBUG util.py:435: perl aarch64 4:5.24.1-390.fc27 build 6.0 M DEBUG util.py:435: perl-Carp noarch 1.40-366.fc26 build 28 k DEBUG util.py:435: perl-Errno aarch64 1.25-390.fc27 build 67 k DEBUG util.py:435: perl-Exporter noarch 5.72-367.fc26 build 32 k DEBUG util.py:435: perl-File-Path noarch 2.12-366.fc26 build 33 k DEBUG util.py:435: perl-IO aarch64 1.36-390.fc27 build 133 k DEBUG util.py:435: perl-PathTools aarch64 3.63-367.fc26 build 87 k DEBUG util.py:435: perl-Scalar-List-Utils aarch64 3:1.47-2.fc26 build 64 k DEBUG util.py:435: perl-Socket aarch64 4:2.024-2.fc26 build 56 k DEBUG util.py:435: perl-Text-Tabs+Wrap noarch 2013.0523-366.fc26 build 22 k DEBUG util.py:435: perl-Unicode-Normalize aarch64 1.25-366.fc26 build 76 k DEBUG util.py:435: perl-constant noarch 1.33-368.fc26 build 23 k DEBUG util.py:435: perl-libs aarch64 4:5.24.1-390.fc27 build 1.4 M DEBUG util.py:435: perl-macros aarch64 4:5.24.1-390.fc27 build 64 k DEBUG util.py:435: perl-parent noarch 1:0.236-2.fc26 build 18 k DEBUG util.py:435: perl-threads aarch64 1:2.15-1.fc26 build 58 k DEBUG util.py:435: perl-threads-shared aarch64 1.55-1.fc26 build 45 k DEBUG util.py:435: policycoreutils aarch64 2.6-4.fc27 build 703 k DEBUG util.py:435: python2 aarch64 2.7.13-5.fc26 build 97 k DEBUG util.py:435: python2-appdirs noarch 1.4.0-10.fc26 build 21 k DEBUG util.py:435: python2-libs aarch64 2.7.13-5.fc26 build 5.9 M DEBUG util.py:435: python2-packaging noarch 16.8-5.fc27 build 46 k DEBUG util.py:435: python2-pip noarch 9.0.1-9.fc27 build 1.8 M DEBUG util.py:435: python2-pyparsing noarch 2.1.10-3.fc26 build 139 k DEBUG util.py:435: python2-setuptools noarch 34.3.0-1.fc26 build 442 k DEBUG util.py:435: python2-six noarch 1.10.0-8.fc26 build 35 k DEBUG util.py:435: tcl aarch64 1:8.6.6-2.fc26 build 1.0 M DEBUG util.py:435: tk aarch64 1:8.6.6-1.fc26 build 1.5 M DEBUG util.py:435: Transaction Summary DEBUG util.py:435: ================================================================================ DEBUG util.py:435: Install 44 Packages DEBUG util.py:435: Total download size: 28 M DEBUG util.py:435: Installed size: 115 M DEBUG util.py:435: Downloading Packages: DEBUG util.py:435: -------------------------------------------------------------------------------- DEBUG util.py:435: Total 2.1 MB/s | 28 MB 00:13 DEBUG util.py:435: Running transaction check DEBUG util.py:435: Transaction check succeeded. DEBUG util.py:435: Running transaction test DEBUG util.py:435: Transaction test succeeded. DEBUG util.py:435: Running transaction
sorry, I linked the wrong build, this one (still still going): https://koji.fedoraproject.org/koji/taskinfo?taskID=18549867
I was able to reproduce this locally with Fedora 25 + updates on an aarch64 system (AMD Seattle). It hangs after installing the last of the build requirements when running 'mock -r fedora-rawhide-aarch64 ...' strace on the dnf child process of mock shows it's just waiting on a futex: futex(0xffffa5c736c0, FUTEX_WAIT, 2, NULL
I installed some debuginfo packages and got a backtrace on dnf while it was waiting on the futex: ~]# pgrep -laf dnf 19472 /usr/libexec/system-python /usr/bin/dnf builddep --installroot /var/lib/... ~]# gdb /usr/libexec/system-python -p 19472 ... (gdb) bt #0 0x0000ffffa85f0a3c in __lll_lock_wait (futex=0xffff9fa936c0, private=128) at lowlevellock.c:46 #1 0x0000ffffa85ec24c in __GI___pthread_rwlock_rdlock (rwlock=0xffff9fa936c0) at pthread_rwlock_rdlock.c:135 #2 0x0000ffffa60e1b08 in __db_pthread_mutex_readlock ( env=env@entry=0xaaaadea7ded0, mutex=<optimized out>) at ../../src/mutex/mut_pthread.c:511 #3 0x0000ffffa61d9340 in __memp_alloc (dbmp=0xffff00000000, dbmp@entry=0xaaaadc5776b0, infop=<optimized out>, mfp=0x0, mfp@entry=0xffff9f9424d0, len=281470681743360, len@entry=0, offsetp=0xaaaadea6d180, offsetp@entry=0x0, retp=0xaaaadeab27e0, retp@entry=0xffffcbe8f2b0) at ../../src/mp/mp_alloc.c:278 #4 0x0000ffffa61dc2b8 in __memp_fget (dbmfp=dbmfp@entry=0xaaaadea6d180, pgnoaddr=0x34e46b98191fb600, pgnoaddr@entry=0xffffcbe8f488, ip=0x0, ip@entry=0xffff9fae05f0, txn=<optimized out>, txn@entry=0xaaaadc5a0bf0, flags=<optimized out>, flags@entry=0, addrp=0x2b50000001a, addrp@entry=0xffffcbe8f460) at ../../src/mp/mp_fget.c:641 #5 0x0000ffffa61a40c8 in __db_goff (dbc=<optimized out>, dbt=0xffffcbe8f488, tlen=<optimized out>, pgno=<optimized out>, bpp=<optimized out>, bpsz=<optimized out>) at ../../src/db/db_overflow.c:173 #6 0x0000ffffa618ec68 in __dbc_iget (dbc=0xaaaadea76400, key=0xffffcbe8f460, data=0xffffcbe8f488, flags=<optimized out>) at ../../src/db/db_cam.c:1132 #7 0x0000ffffa618ee64 in __dbc_get (dbc=dbc@entry=0xaaaadea76400, key=key@entry=0xffffcbe8f460, data=data@entry=0xffffcbe8f488, flags=flags@entry=26) at ../../src/db/db_cam.c:770 #8 0x0000ffffa619c1f4 in __dbc_get_pp (dbc=0xaaaadea76400, key=0xffffcbe8f460, data=0xffffcbe8f488, flags=26) at ../../src/db/db_iface.c:2361 #9 0x0000ffffa6285b9c in dbiCursorGet (dbc=0xaaaadeabb8b0, key=0xffffcbe8f460, data=0xffffcbe8f488, flags=26) at backend/db3.c:646 #10 0x0000ffffa6285cd4 in db3_pkgdbGet (dbi=<optimized out>, dbc=0xaaaadeabb8b0, hdrNum=<optimized out>, hdrBlob=0xffffcbe8f520, hdrLen=0xffffcbe8f514) at backend/db3.c:1322 #11 0x0000ffffa628fb28 in rpmdbNextIterator (mi=mi@entry=0xaaaadea81e50) at rpmdb.c:1577 #12 0x0000ffffa62b6fa0 in rpmteDBHeader (te=te@entry=0xaaaadc6d78a0) at rpmte.c:520 #13 0x0000ffffa62b80b8 in rpmteOpen (reload_fi=0, te=0xaaaadc6d78a0) at rpmte.c:565 #14 rpmteProcess (te=0xaaaadc6d78a0, goal=goal@entry=PKG_POSTTRANS) at rpmte.c:755 #15 0x0000ffffa62bba60 in runTransScripts (ts=ts@entry=0x0, goal=goal@entry=PKG_POSTTRANS) at transaction.c:1201 #16 0x0000ffffa62be69c in rpmtsRun (ts=<optimized out>, okProbs=okProbs@entry=0x0, ignoreSet=<optimized out>) at transaction.c:1546 #17 0x0000ffffa5d23a68 in rpmts_Run (s=0xffffa573aee8, args=<optimized out>, kwds=<optimized out>) at rpmts-py.c:575 #18 0x0000ffffa86d2094 in PyCFunction_Call ( func=<built-in method run of TransactionSet object at remote 0xffffa573aee8>, args=(<method at remote 0xffff9fe7c3c8>, '', 64), kwds=<optimized out>) at /usr/src/debug/Python-3.5.3/Objects/methodobject.c:98 #19 0x0000ffffa8689cb0 in PyObject_Call ( func=<built-in method run of TransactionSet object at remote 0xffffa573aee8>, arg=(<method at remote 0xffff9fe7c3c8>, '', 64), kw=0x0) at /usr/src/debug/Python-3.5.3/Objects/abstract.c:2166 #20 0x0000ffffa87421b4 in PyEval_CallObjectWithKeywords ( func=func@entry=<built-in method run of TransactionSet object at remote 0xffffa573aee8>, arg=arg@entry=(<method at remote 0xffff9fe7c3c8>, '', 64), kw=kw@entry=0x0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4653 #21 0x0000ffffa86a3e08 in methoddescr_call (descr=<optimized out>, args=(<method at remote 0xffff9fe7c3c8>, '', 64), kwds=0x0) at /usr/src/debug/Python-3.5.3/Objects/descrobject.c:250 #22 0x0000ffffa8689cb0 in PyObject_Call ( func=<method_descriptor at remote 0xffffa5d601f8>, arg=(<TransactionSet(_probFilter=64) at remote 0xffffa573aee8>, <method at remote 0xffff9fe7c3c8>, '', 64), kw=0x0) at /usr/src/debug/Python-3.5.3/Objects/abstract.c:2166 #23 0x0000ffffa8743f28 in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0xffffcbe8fbe0, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:5009 #24 call_function (oparg=<optimized out>, pp_stack=0xffffcbe8fbe0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4805 #25 PyEval_EvalFrameEx ( f=Frame 0xaaaadebaa428, for file /usr/lib/python3.5/site-packages/dnf/base.py, line 678, in _run_transaction (self=<BaseCli(cmd_conf=<CmdConf(downloadonly=False) at remote 0xffffa7f69a58>, _logging=<Logging(stdout_handler=<StreamHandler(lock=<_thread.RLock at remote 0xffffa7fc3d20>, filters=[<MaxLevelFilter(max_level=30) at remote 0xffffa57dd278>], level=20, formatter=None, stream=<_io.TextIOWrapper at remote 0xffffa808e630>, _name=None) at remote 0xffffa57b70f0>, stderr_handler=<StreamHandler(lock=<_thread.RLock at remote 0xffffa57fc870>, filters=[], level=30, formatter=None, stream=<_io.TextIOWrapper at remote 0xffffa808e708>, _name=None) at remote 0xffffa57dd2b0>, presetup=<function at remote 0xffffa5a0e2f0>, setup=<function at remote 0xffffa5a0e488>) at remote 0xffffa59ace80>, output=<Output(conf=<YumConf(__opt281473460923304='normal', __opt281473460921624=0, __opt281473460922128=False, __opt281473460923976=None, __opt281473460921400=None, __opt281473460924032=None, __opt281473460924368=True, __opt281473460...(truncated), f@entry=Frame 0xffff9fe7a240, for file /usr/lib64/python3.5/site-packages/rpm/transaction.py, line 103, in run (self=<TransactionSet(_probFilter=64) at remote 0xffffa573aee8>, callback=<method at remote 0xffff9fe7c3c8>, data=''), throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #26 0x0000ffffa874a714 in fast_function (nk=<optimized out>, na=<optimized out>, n=3, pp_stack=0xffffcbe8fd60, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4876 #27 call_function (oparg=<optimized out>, pp_stack=0xffffcbe8fd60) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4803 Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0xe2 in position 0: invalid continuation byte: #28 PyEval_EvalFrameEx (f=, f@entry=<unknown at remote 0x2>, throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #29 0x0000ffffa874b3d0 in _PyEval_EvalCodeWithName ( _co=<code at remote 0xffffa74c3b70>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=0, kws=0xaaaadc6d3a38, kwcount=kwcount@entry=1, defs=0xffffa74c38a0, defcount=0, kwdefs=0x0, closure=0x0, name='_run_transaction', qualname='Base._run_transaction') at /usr/src/debug/Python-3.5.3/Python/ceval.c:4091 #30 0x0000ffffa87480ac in fast_function (nk=1, na=<optimized out>, n=3, pp_stack=0xffffcbe8ff70, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4886 #31 call_function (oparg=<optimized out>, pp_stack=0xffffcbe8ff70) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4803 Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0xe2 in position 0: invalid continuation byte: #32 PyEval_EvalFrameEx (f=, f@entry=[], throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #33 0x0000ffffa874b3d0 in _PyEval_EvalCodeWithName ( _co=<code at remote 0xffffa74c38a0>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=0, kws=0xaaaadc55ae58, kwcount=kwcount@entry=0, defs=0xffffa59aded0, defcount=1, kwdefs=0x0, closure=0x0, name='do_transaction', qualname='Base.do_transaction') at /usr/src/debug/Python-3.5.3/Python/ceval.c:4091 #34 0x0000ffffa87480ac in fast_function (nk=0, na=<optimized out>, n=2, pp_stack=0xffffcbe90180, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4886 #35 call_function (oparg=<optimized out>, pp_stack=0xffffcbe90180) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4803 Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0xe2 in position 0: invalid continuation byte: #36 PyEval_EvalFrameEx (f=, f@entry=<unknown at remote 0xaaaadc4d9a48>, throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #37 0x0000ffffa874b3d0 in _PyEval_EvalCodeWithName ( _co=<code at remote 0xffffa59aded0>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=43690, kws=0xaaaadc6d5190, kwcount=kwcount@entry=1, defs=0xaaaadc0d4500, defcount=1, kwdefs=0x0, closure=(<cell at remote 0xffffa599e678>,), name='do_transaction', qualname='BaseCli.do_transaction') at /usr/src/debug/Python-3.5.3/Python/ceval.c:4091 #38 0x0000ffffa87480ac in fast_function (nk=1, na=<optimized out>, n=3, pp_stack=0xffffcbe90390, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4886 #39 call_function (oparg=<optimized out>, pp_stack=0xffffcbe90390) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4803 #40 PyEval_EvalFrameEx (f=<unknown at remote 0xaaaadc4bb380>, f@entry=Frame 0xaaaadc6d4fd8, for file /usr/lib/python3.5/site-packages/dnf/cli/main.py, line 149, in resolving (cli=<Cli(repo_setopts={}, cmdstring='dnf builddep --installroot /var/lib/mock/fedora-rawhide-aarch64/root/ --releasever 27 /var/lib/mock/fedora-rawhide-aarch64/root//builddir/build/SRPMS/maxima-5.39.0-4.fc27.src.rpm --setopt=tsflags=nocontexts ', cli_commands={'erase': <type at remote 0xaaaadc4b44a8>, 'builddep': <type at remote 0xaaaadc539328>, 'repo-pkgs': <type at remote 0xaaaadc4b08a8>, 'upgrade': <type at remote 0xaaaadc4be738>, 'debuginfo-install': <type at remote 0xaaaadc53fc68>, 'mark': <type at remote 0xaaaadc4b87d8>, 'makecache': <type at remote 0xaaaadc4b7788>, 'updateinfo': <type at remote 0xaaaadc4bdf08>, 'search': <type at remote 0xaaaadc4bc8c8>, 'update-to': <type at remote 0xaaaadc4bf208>, 'provides': <type at remote 0xaaaadc48f5b8>, 'downgrade': <type at remote 0xaaaadc4b3a98>, 'upgrade-to': <type at remote 0xaaaadc4bf208>, 'repo-packages': <type at remote 0xaaaadc4b08a8>, 'copr': <type at...(truncated), throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #41 0x0000ffffa874a714 in fast_function (nk=<optimized out>, na=<optimized out>, n=2, pp_stack=0xffffcbe90510, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4876 #42 call_function (oparg=<optimized out>, pp_stack=0xffffcbe90510) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4803 Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0x94 in position 0: invalid start byte: #43 PyEval_EvalFrameEx (f=, f@entry=Frame 0xaaaadc4d9888, for file /usr/lib/python3.5/site-packages/dnf/cli/main.py, line 120, in _main (base=<BaseCli(cmd_conf=<CmdConf(downloadonly=False) at remote 0xffffa7f69a58>, _logging=<Logging(stdout_handler=<StreamHandler(lock=<_thread.RLock at remote 0xffffa7fc3d20>, filters=[<MaxLevelFilter(max_level=30) at remote 0xffffa57dd278>], level=20, formatter=None, stream=<_io.TextIOWrapper at remote 0xffffa808e630>, _name=None) at remote 0xffffa57b70f0>, stderr_handler=<StreamHandler(lock=<_thread.RLock at remote 0xffffa57fc870>, filters=[], level=30, formatter=None, stream=<_io.TextIOWrapper at remote 0xffffa808e708>, _name=None) at remote 0xffffa57dd2b0>, presetup=<function at remote 0xffffa5a0e2f0>, setup=<function at remote 0xffffa5a0e488>) at remote 0xffffa59ace80>, output=<Output(conf=<YumConf(__opt281473460923304='normal', __opt281473460921624=0, __opt281473460922128=False, __opt281473460923976=None, __opt281473460921400=None, __opt281473460924032=None, __opt281473460924368=True, __opt281473460920504=...(truncated), throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #44 0x0000ffffa874a714 in fast_function (nk=<optimized out>, na=<optimized out>, n=2, pp_stack=0xffffcbe90690, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4876 #45 call_function (oparg=<optimized out>, pp_stack=0xffffcbe90690) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4803 #46 PyEval_EvalFrameEx ( f=Frame 0xaaaadc4aa258, for file /usr/lib/python3.5/site-packages/dnf/cli/main.py, line 174, in user_main (args=['builddep', '--installroot', '/var/lib/mock/fedora-rawhide-aarch64/root/', '--releasever', '27', '/var/lib/mock/fedora-rawhide-aarch64/root//builddir/build/SRPMS/maxima-5.39.0-4.fc27.src.rpm', '--setopt=tsflags=nocontexts'], exit_code=True), f@entry=Frame 0xaaaadc4bb4f8, for file /usr/lib/python3.5/site-packages/dnf/cli/main.py, line 60, in main (args=['builddep', '--installroot', '/var/lib/mock/fedora-rawhide-aarch64/root/', '--releasever', '27', '/var/lib/mock/fedora-rawhide-aarch64/root//builddir/build/SRPMS/maxima-5.39.0-4.fc27.src.rpm', '--setopt=tsflags=nocontexts'], base=<BaseCli(cmd_conf=<CmdConf(downloadonly=False) at remote 0xffffa7f69a58>, _logging=<Logging(stdout_handler=<StreamHandler(lock=<_thread.RLock at remote 0xffffa7fc3d20>, filters=[<MaxLevelFilter(max_level=30) at remote 0xffffa57dd278>], level=20, formatter=None, stream=<_io.TextIOWrapper at remote 0xffffa808e630>, _name=None) at remote 0xffffa57b70f0>, stderr_handler=<StreamHandler(lock=<_thread.RLock at remote 0xffffa57fc870>, filters=[], level=30, formatter=None, stream=<_io.TextIOWrapper at remote 0xffffa808e708>, _name=None) at remote 0xffffa57dd2b0>, presetup=<function at remote 0xffffa5a0e2f0>, setup=<function at remote 0xffffa5a0e488>) at remote 0xffffa59ace80>, output=<Outp...(truncated), throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #47 0x0000ffffa874a714 in fast_function (nk=<optimized out>, na=<optimized out>, n=1, pp_stack=0xffffcbe90810, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4876 #48 call_function (oparg=<optimized out>, pp_stack=0xffffcbe90810) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4803 #49 PyEval_EvalFrameEx (f=<unknown at remote 0xffffa7fd47b8>, f@entry=0x0, throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #50 0x0000ffffa874b3d0 in _PyEval_EvalCodeWithName ( _co=<code at remote 0xffffa59c6810>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=0, kws=0xaaaadc144b80, kwcount=kwcount@entry=1, defs=0xffffa7fd4930, defcount=1, kwdefs=0x0, closure=0x0, name='user_main', qualname='user_main') at /usr/src/debug/Python-3.5.3/Python/ceval.c:4091 #51 0x0000ffffa87480ac in fast_function (nk=1, na=<optimized out>, n=3, pp_stack=0xffffcbe90a20, func=<optimized out>) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4886 #52 call_function (oparg=<optimized out>, pp_stack=0xffffcbe90a20) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4803 #53 PyEval_EvalFrameEx (f=<unknown at remote 0xffffa805d210>, f@entry=<unknown at remote 0xffffcbe90a90>, throwflag=throwflag@entry=0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:3304 #54 0x0000ffffa874b3d0 in _PyEval_EvalCodeWithName ( _co=_co@entry=<code at remote 0xffffa7fd4930>, globals=globals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, locals=locals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, args=args@entry=0x0, argcount=65535, argcount@entry=0, kws=kws@entry=0x0, kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0, kwdefs=kwdefs@entry=0x0, closure=closure@entry=0x0, name=name@entry=0x0, qualname=qualname@entry=0x0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4091 #55 0x0000ffffa874b4e4 in PyEval_EvalCodeEx ( _co=_co@entry=<code at remote 0xffffa7fd4930>, globals=globals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, locals=locals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0, kwdefs=kwdefs@entry=0x0, closure=closure@entry=0x0) at /usr/src/debug/Python-3.5.3/Python/ceval.c:4112 #56 0x0000ffffa874b51c in PyEval_EvalCode ( co=co@entry=<code at remote 0xffffa7fd4930>, globals=globals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, locals=locals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}) at /usr/src/debug/Python-3.5.3/Python/ceval.c:797 #57 0x0000ffffa876ad60 in run_mod (mod=mod@entry=0xaaaadc136288, filename=filename@entry='/usr/bin/dnf', globals=globals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, locals=locals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, flags=flags@entry=0xffffcbe90c40, arena=arena@entry=0xaaaadc19c0d0) at /usr/src/debug/Python-3.5.3/Python/pythonrun.c:991 #58 0x0000ffffa876d290 in PyRun_FileExFlags (fp=fp@entry=0xaaaadc134ea0, filename_str=filename_str@entry=0xffffa80b0ec0 "/usr/bin/dnf", start=start@entry=257, globals=globals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, locals=locals@entry={'here': '/usr/bin', '__builtins__': <module at remote 0xffffa80e35e8>, 'suppress_keyboard_interrupt_message': <function at remote 0xffffa7f3a378>, '__file__': '/usr/bin/dnf', '__spec__': None, '__name__': '__main__', 'main': <module at remote 0xffffa59aff48>, '__doc__': None, '__package__': None, 'sys': <module at remote 0xffffa8073e08>, 'unicode_literals': <_Feature(compiler_flag=131072, mandatory=(3, 0, 0, 'alpha', 0), optional=(2, 6, 0, 'alpha', 2)) at remote 0xffffa7f5de10>, '__cached__': None, '__loader__': <SourceFileLoader(path='/usr/bin/dnf', name='__main__') at remote 0xffffa8007da0>}, closeit=closeit@entry=1, flags=flags@entry=0xffffcbe90c40) at /usr/src/debug/Python-3.5.3/Python/pythonrun.c:944 #59 0x0000ffffa876d42c in PyRun_SimpleFileExFlags (fp=0xaaaadc134ea0, fp@entry=0x37495df3, filename=0xffffa80b0ec0 "/usr/bin/dnf", closeit=closeit@entry=1, flags=0xffffcbe90c40, flags@entry=0xffffcbe90cd0) at /usr/src/debug/Python-3.5.3/Python/pythonrun.c:411 #60 0x0000ffffa876d880 in PyRun_AnyFileExFlags (fp=fp@entry=0x37495df3, filename=<optimized out>, closeit=closeit@entry=1, flags=flags@entry=0xffffcbe90cd0) at /usr/src/debug/Python-3.5.3/Python/pythonrun.c:80 #61 0x0000ffffa878380c in run_file (p_cf=0xffffcbe90cd0, filename=0x34e46b98191fb600 <error: Cannot access memory at address 0x34e46b98191fb600>, fp=0x37495df3) at /usr/src/debug/Python-3.5.3/Modules/main.c:318 #62 Py_Main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/Python-3.5.3/Modules/main.c:768 #63 0x0000aaaaba570d04 in main (argc=9, argv=0xffffcbe90e58) at /usr/src/debug/Python-3.5.3/Programs/python.c:65 (gdb)
I tried breaking it down to see which of the install-dependencies is causing the hang, and the culprit appears to be gcl: mock -r fedora-rawhide-aarch64 --clean mock -r fedora-rawhide-aarch64 --init for x in aajohan-comfortaa-fonts desktop-file-utils emacs-filesystem \ fontconfig fontpackages-filesystem freetype gcl gcl-selinux \ libX11 libX11-common libXau libXft libXrender libpng \ libselinux-utils libxcb perl perl-Carp perl-Errno perl-Exporter \ perl-File-Path perl-IO perl-PathTools perl-Scalar-List-Utils \ perl-Socket perl-Text-Tabs+Wrap perl-Unicode-Normalize \ perl-constant perl-libs perl-macros perl-parent perl-threads \ perl-threads-shared policycoreutils python2 python2-appdirs \ python2-libs python2-packaging python2-pip python2-pyparsing \ python2-setuptools python2-six tcl tk do echo "::::: mock installing $x" mock -r fedora-rawhide-aarch64 --install $x done When I ran the above commands, it hung when it got to gcl. ::::: mock installing aajohan-comfortaa-fonts ::::: mock installing desktop-file-utils ::::: mock installing emacs-filesystem ... ::::: mock installing gcl And while it was stuck here, I verified it was stuck in a futex again: ~]# pgrep -laf dnf 19472 dnf 21383 /usr/libexec/system-python /usr/bin/dnf --installroot /var/lib/mock/fedora-rawhide-aarch64/root/ --releasever 27 --disableplugin=local --setopt=deltarpm=false install gcl --setopt=tsflags=nocontexts ~]# strace -p 21383 strace: Process 21383 attached futex(0xffffa81c36c0, FUTEX_WAIT, 2, NULL
I can reproduce the hang with just 3 commands now: mock -r fedora-rawhide-aarch64 --clean mock -r fedora-rawhide-aarch64 --init mock -r fedora-rawhide-aarch64 --install gcl Changing component to gcl.
fyi, at least until gcl is happy again, maxima.spec: %changelog * Thu Mar 23 2017 Rex Dieter <rdieter> - 5.39.0-5 - aarch64: drop gcl (#1435395), use ecl default - drop BR: time (unused)
I verified your changes allow maxima to build for me, and the build time was decent at about 37 minutes. $ fedpkg pull $ fedpkg srpm $ mock -r fedora-rawhide-aarch64 --resultdir=/home/rpmbuild/mock-rawhide \ maxima-5.39.0-5.fc26.src.rpm ... Wrote: /builddir/build/RPMS/maxima-5.39.0-5.fc27.aarch64.rpm Wrote: /builddir/build/RPMS/maxima-gui-5.39.0-5.fc27.aarch64.rpm Wrote: /builddir/build/RPMS/maxima-src-5.39.0-5.fc27.aarch64.rpm Wrote: /builddir/build/RPMS/maxima-runtime-ecl-5.39.0-5.fc27.aarch64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.OElQgV + umask 022 + cd /builddir/build/BUILD + cd maxima-5.39.0 + /usr/bin/rm -rf /builddir/build/BUILDROOT/maxima-5.39.0-5.fc27.aarch64 + exit 0 Finish: rpmbuild maxima-5.39.0-5.fc26.src.rpm Finish: build phase for maxima-5.39.0-5.fc26.src.rpm INFO: Done(fedpkg/maxima/maxima-5.39.0-5.fc26.src.rpm) Config(fedora-rawhide-aarch64) 36 minutes 48 seconds INFO: Results and/or logs in: /home/rpmbuild/mock-rawhide INFO: Cleaning up build root ('cleanup_on_success=True') Start: clean chroot Finish: clean chroot Finish: run
I'm the gcl maintainer. I cannot debug this. It only happens on aarch64, and I have no access to any aarch64 machine. Can somebody give me such access? In any case, the evidence points to something going wrong after installation, therefore during execution of the %post scripts, right? I see only two possible candidates: %post for the main package: /sbin/fixfiles -R gcl restore || : %post for gcl-selinux: /usr/sbin/semodule -i %{_datadir}/selinux/packages/gcl/gcl.pp 2> /dev/null || : /sbin/restorecon -R %{_libdir}/maxima 2> /dev/null || : I don't know why any of those commands would hang, but again, since this is an aarch64-only phenomenon, I either need access to an aarch64 machine, or I need somebody who has such access to investigate further for me.
The culprit cannot possibly be gcl. The hang is happening at install time. No gcl component is running at that time. As noted in comment 8, the culprit can only be the contents of one of the scripts, and those run only fixfiles, semodule, and restorecon. Therefore, one of THOSE is hanging. Since all 3 belong to the policycoreutils package, I am reassigning this bug to that component. I would have been happy to debug this, but I asked both here and on -devel list for access to an aarch64 machine and got no response. If those with the necessary capabilities won't respond, there is absolutely nothing I can do.
Months go by... Policycoreutils maintainers, do you have any intention of looking at this bug, or of providing access to aarch64 hardware?
I tried install gcl on Fedora 28 aarch64 and it worked. Then I reverted the patch from https://bugzilla.redhat.com/show_bug.cgi?id=1435395#c6 and tried to build maxima and it was built - https://koji.fedoraproject.org/koji/taskinfo?taskID=25717536 Even maxima-5.39.0-3.fc26 which originaly failed can be built now - https://koji.fedoraproject.org/koji/taskinfo?taskID=25721648 Is it possible that in the mean time it's fixed by some update? Is this still reproducible?
This message is a reminder that Fedora 26 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '26'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 26 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.