Bug 1297808

Summary: vdsm-4.17.17 fails make distcheck
Product: [oVirt] vdsm Reporter: Sandro Bonazzola <sbonazzo>
Component: BuildAssignee: Marcin Sobczyk <msobczyk>
Status: CLOSED DEFERRED QA Contact: Lukas Svaty <lsvaty>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.17.17CC: bugs, mperina, pkliczew, sbonazzo
Target Milestone: ---Keywords: CodeChange
Target Release: ---Flags: sbonazzo: ovirt-4.3-
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-01-22 11:28:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Sandro Bonazzola 2016-01-12 14:16:01 UTC
Steps to Reproduce:
1. $ git checkout v4.17.17
2. $ ./autogen.sh --system --enable-hooks
3. $ make distcheck

Actual results:
Making all in rpc
make[3]: Entering directory '/NotBackedUp/sbonazzo/upstream/vdsm/vdsm-4.17.17/_build/sub/vdsm/rpc'
  Generate vdsm-api.html
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=../../../../vdsm/rpc/../lib \
    /usr/bin/python ../../../../vdsm/rpc/process-schema.py \
              ../../../../vdsm/rpc/vdsmapi-schema.json ../../../../vdsm/rpc/vdsm-api.html
Traceback (most recent call last):
  File "../../../../vdsm/rpc/process-schema.py", line 372, in <module>
    main()
  File "../../../../vdsm/rpc/process-schema.py", line 368, in main
    create_doc(symbols, output)
  File "../../../../vdsm/rpc/process-schema.py", line 292, in create_doc
    with open(filename, 'w') as f:
IOError: [Errno 13] Permission denied: '../../../../vdsm/rpc/vdsm-api.html'
Makefile:692: recipe for target 'vdsm-api.html' failed
make[3]: *** [vdsm-api.html] Error 1
make[3]: Leaving directory '/NotBackedUp/sbonazzo/upstream/vdsm/vdsm-4.17.17/_build/sub/vdsm/rpc'
Makefile:895: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/NotBackedUp/sbonazzo/upstream/vdsm/vdsm-4.17.17/_build/sub/vdsm'
Makefile:559: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/NotBackedUp/sbonazzo/upstream/vdsm/vdsm-4.17.17/_build/sub'
Makefile:768: recipe for target 'distcheck' failed
make: *** [distcheck] Error 1

Expected results:
make distcheck should work

Additional info:
Not blocking releases on this since make dist works, but this is an alarm on bad packaging of vdsm.

Comment 1 Yaniv Bronhaim 2016-01-14 10:28:48 UTC
try make clean before - 
make clean; ./autogen.sh --system; make dist check

works for me

Comment 2 Sandro Bonazzola 2016-01-14 14:57:05 UTC
$ git checkout v4.17.17
$ make clean
make: *** No rule to make target 'clean'.  Stop.

as expected...

$ git checkout v4.17.17
$ ./autogen.sh --system --enable-hooks
$ make clean
$ make distcheck

finish with:

Making all in rpc
make[3]: Entering directory '/NotBackedUp/sbonazzo/upstream/vdsm/vdsm-4.17.17/_build/sub/vdsm/rpc'
  Generate vdsm-api.html
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=../../../../vdsm/rpc/../lib \
    /usr/bin/python ../../../../vdsm/rpc/process-schema.py \
              ../../../../vdsm/rpc/vdsmapi-schema.json ../../../../vdsm/rpc/vdsm-api.html
Traceback (most recent call last):
  File "../../../../vdsm/rpc/process-schema.py", line 372, in <module>
    main()
  File "../../../../vdsm/rpc/process-schema.py", line 368, in main
    create_doc(symbols, output)
  File "../../../../vdsm/rpc/process-schema.py", line 292, in create_doc
    with open(filename, 'w') as f:
IOError: [Errno 13] Permission denied: '../../../../vdsm/rpc/vdsm-api.html'
Makefile:692: recipe for target 'vdsm-api.html' failed
make[3]: *** [vdsm-api.html] Error 1
make[3]: Leaving directory '/NotBackedUp/sbonazzo/upstream/vdsm/vdsm-4.17.17/_build/sub/vdsm/rpc'
Makefile:895: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/NotBackedUp/sbonazzo/upstream/vdsm/vdsm-4.17.17/_build/sub/vdsm'
Makefile:559: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/NotBackedUp/sbonazzo/upstream/vdsm/vdsm-4.17.17/_build/sub'
Makefile:768: recipe for target 'distcheck' failed
make: *** [distcheck] Error 1


Note I'm not root.

Comment 3 Yaniv Bronhaim 2016-01-17 08:44:43 UTC
yes.. dist check and distcheck are different

Piotr, please check the issue

Comment 4 Piotr Kliczewski 2016-01-20 09:57:05 UTC
After fixing the issue with vdsm-api.html generation there is still issue with make distcheck:

if [ -x "/usr/bin/python3" ]; then \
    PYTHONDONTWRITEBYTECODE=1 /usr/bin/python3 -m compileall \
	contrib/logdb contrib/profile-stats init/daemonAdapter vdsm/get-conf-item vdsm/set-conf-item vdsm/supervdsmServer vdsm/vdsm vdsm/vdsm-restore-net-config vdsm/storage/curl-img-wrap vdsm/storage/fc-scan vdsm-tool/vdsm-tool  \
	`find . -name '*.py'`; \
else \
    echo "Warning: skipping python3 syntax check"; \
fi
Listing 'contrib/logdb'...
Can't list 'contrib/logdb'
Listing 'contrib/profile-stats'...
Can't list 'contrib/profile-stats'
Listing 'init/daemonAdapter'...
Can't list 'init/daemonAdapter'
Listing 'vdsm/get-conf-item'...
Can't list 'vdsm/get-conf-item'
Listing 'vdsm/set-conf-item'...
Can't list 'vdsm/set-conf-item'
Listing 'vdsm/supervdsmServer'...
Can't list 'vdsm/supervdsmServer'
Listing 'vdsm/vdsm'...
Can't list 'vdsm/vdsm'
Listing 'vdsm/vdsm-restore-net-config'...
Can't list 'vdsm/vdsm-restore-net-config'
Listing 'vdsm/storage/curl-img-wrap'...
Can't list 'vdsm/storage/curl-img-wrap'
Listing 'vdsm/storage/fc-scan'...
Can't list 'vdsm/storage/fc-scan'
Listing 'vdsm-tool/vdsm-tool'...
Can't list 'vdsm-tool/vdsm-tool'
Compiling './lib/vdsm/tool/validate_ovirt_certs.py'...
Compiling './lib/vdsm/config.py'...
Compiling './lib/vdsm/dsaversion.py'...
Compiling './lib/vdsm/constants.py'...
Compiling './vdsm/sos/vdsm.py'...
Compiling './vdsm/dumpStorageTable.py'...
Compiling './tests/crossImportsTests.py'...
python -c 'import pyflakes; print("pyflakes-%s" % pyflakes.__version__)'
pyflakes-0.8.1
( find . -path './.git' -prune -type f -o \
	-name '*.py' && \
	echo contrib/logdb contrib/profile-stats init/daemonAdapter vdsm/get-conf-item vdsm/set-conf-item vdsm/supervdsmServer vdsm/vdsm vdsm/vdsm-restore-net-config vdsm/storage/curl-img-wrap vdsm/storage/fc-scan vdsm-tool/vdsm-tool  ) | xargs /usr/bin/pyflakes
contrib/logdb: No such file or directory
contrib/profile-stats: No such file or directory
init/daemonAdapter: No such file or directory
vdsm/get-conf-item: No such file or directory
vdsm/set-conf-item: No such file or directory
vdsm/supervdsmServer: No such file or directory
vdsm/vdsm: No such file or directory
vdsm/vdsm-restore-net-config: No such file or directory
vdsm/storage/curl-img-wrap: No such file or directory
vdsm/storage/fc-scan: No such file or directory
vdsm-tool/vdsm-tool: No such file or directory
Makefile:984: recipe for target 'pyflakes' failed
make[1]: *** [pyflakes] Error 123
make[1]: Leaving directory '/home/pkliczewski/git/bug/vdsm/vdsm-4.17.999/_build/sub'
Makefile:754: recipe for target 'distcheck' failed
make: *** [distcheck] Error 1

Comment 5 Yaniv Kaul 2017-09-04 14:04:55 UTC
Still relevant?

Comment 6 Piotr Kliczewski 2017-09-04 14:15:41 UTC
Yes, make distcheck still is not working. No work done around this change for long time.

Comment 7 Martin Perina 2019-01-22 11:28:02 UTC
To properly support distcheck we would need to develop special Python loader, because VDSM has a lot of generated Python files. Due to insufficient resources closing for now