Bug 1647715 - Coccinelle default install can't find its own standard library
Summary: Coccinelle default install can't find its own standard library
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: coccinelle
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Richard W.M. Jones
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-08 08:24 UTC by John Friend
Modified: 2018-11-18 03:55 UTC (History)
1 user (show)

Fixed In Version: coccinelle-1.0.7-4.fc28 coccinelle-1.0.7-4.fc29
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-18 02:56:50 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
shell script for repro (126 bytes, application/x-shellscript)
2018-11-08 08:24 UTC, John Friend
no flags Details
works-for-me spec file (18.92 KB, text/x-matlab)
2018-11-08 08:28 UTC, John Friend
no flags Details
Patch spec file, now at 1.0.7-3, cleanups and added python functionality test (2.51 KB, patch)
2018-11-08 11:50 UTC, John Friend
no flags Details | Diff
Add missing env variable for python test to succeed (1.60 KB, patch)
2018-11-08 13:35 UTC, John Friend
no flags Details | Diff
Install log with (benign?) cpio warnings aboout missing files (14.17 KB, text/plain)
2018-11-08 13:42 UTC, John Friend
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1632023 0 unspecified CLOSED coccinelle-1.0.7 is available 2021-02-22 00:41:40 UTC

Description John Friend 2018-11-08 08:24:25 UTC
Created attachment 1503274 [details]
shell script for repro

Description of problem:



Version-Release number of selected component (if applicable):
1.7.0, now in rawhide after RHBZ-1632023

How reproducible:
Always

Steps to Reproduce:
1. Install coccinelle
2. run attached repro.sh
3. se warnings of "missing" files

Actual results:
$ ./repro.sh
warning: Can't find macro file: /usr/bin/../lib/coccinelle/standard.h
warning: Can't find default iso file: /usr/bin/../lib/coccinelle/standard.iso
HANDLING: foo.c
Hello World!

Expected results:
$ ./repro.sh
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: foo.c
Hello World!

Additional info: 

IIUC, upstream planned for spatch binary to live in COCCINELLE_HOME="/usr/lib64/coccinelle/", 
not in /bin. Instead the script/spatch.sh should be called, which sets COCCINELLE_HOME and 
then calls the binary which uses the env to find its standard library.

I'm trying to get fc29 to have a working-out-of-the-box coccinelle 1.7.0 in fc29, 
with working python support (thanks to 1632023). It seems to have been broken
in one way or another for a long time. 

Coccinelle is/should be used heavily in the kernel community where it is used
to statically find bugs in code, some of which may have security implications.
The kernel crows is notoriously curmudgeonly, so it's important to remove as
much friction as possible in order to get people to use the tools available.
So, please, let's fix this.

Comment 1 John Friend 2018-11-08 08:28:11 UTC
Created attachment 1503277 [details]
works-for-me spec file

- installs spatch bin in /usr/lib64/coccinelle/, and installs wrapper scripts
scripts/spatch.sh as /usr/bin/spatch
- Fix autotools version mismatch on fc29 (wants 1.15, complains of 1.16 being installed)

Comment 2 Richard W.M. Jones 2018-11-08 09:02:54 UTC
It's easier if you supply patches to the spec file.  Anyway
I have integrated that into the Rawhide spec and built something.

https://koji.fedoraproject.org/koji/taskinfo?taskID=30734682

Comment 3 John Friend 2018-11-08 10:53:57 UTC
Fine, but what about fc29???

Comment 4 Richard W.M. Jones 2018-11-08 10:55:42 UTC
The package now fully works in Rawhide and you've checked that?

Comment 5 John Friend 2018-11-08 11:49:08 UTC
I didn't realize that's what you were asking for. You just closed the ticket without a comment.

I'm attaching a third fix, as a patch against your most recent version. I built
it on f29 and it works (I applied it as part of a local fedpkg clone of your repo). 

Please accept these changes. If you'd like me to test again once you queue a build, I'm glad to do so, but can only do so on f29 (which is what I run).
Then, I would really like to see this pushed f29 updates, which is my goal here.

Comment 6 John Friend 2018-11-08 11:50:21 UTC
Created attachment 1503305 [details]
Patch spec file, now at 1.0.7-3, cleanups and added python functionality test

Comment 7 John Friend 2018-11-08 11:53:47 UTC
To clarify, 1.0.7-2 works as well for me (when built on f29), but because you had
to manually merge my spec file, some cleanups were required, hence the new patch , which hopefully involves just a "git am" for you.

Comment 8 Richard W.M. Jones 2018-11-08 12:50:01 UTC
OK please try this package which contains a version of your
latest specfile patch:

https://koji.fedoraproject.org/koji/taskinfo?taskID=30737051

If this works I will backport to F29.

Comment 9 John Friend 2018-11-08 13:35:51 UTC
Created attachment 1503326 [details]
Add missing env variable for python test to succeed

Comment 10 John Friend 2018-11-08 13:39:34 UTC
Getting closer.

fedpkg local doesn't isolate the build, so the python test succeeded because I had 1.0.7-3 already installed. Fix attached.

Rebuilt with fedpkg mock (on f29), installed and tested files with and without python support. All ok.

There's a bunch of warnings during the  install phase
cpio: buffer.ml: Cannot stat: No such file or directory
cpio: bundles/parmap/filename.ml: Cannot stat: No such file or directory
cpio: bundles/parmap/list.ml: Cannot stat: No such file or directory
cpio: bundles/parmap/printf.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/buffer.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/bytes.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/filename.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/hashtbl.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/list.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/printf.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/stdcompat__option.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/stdlib.ml: Cannot stat: No such file or directory
cpio: bundles/pyml/string.ml: Cannot stat: No such file or directory
cpio: bytes.ml: Cannot stat: No such file or directory
cpio: filename.ml: Cannot stat: No such file or directory
cpio: format.ml: Cannot stat: No such file or directory
cpio: hashtbl.ml: Cannot stat: No such file or directory
cpio: lexing.ml: Cannot stat: No such file or directory
cpio: list.ml: Cannot stat: No such file or directory
cpio: map.ml: Cannot stat: No such file or directory
cpio: marshal.ml: Cannot stat: No such file or directory
cpio: otherlibs/dynlink/natdynlink.ml: Cannot stat: No such file or directory
cpio: parmap/parmap.ml: Cannot stat: No such file or directory
cpio: parsing_c/lexer_c.ml: Cannot stat: No such file or directory
cpio: parsing_cocci/lexer_cli.ml: Cannot stat: No such file or directory
cpio: parsing_cocci/lexer_cocci.ml: Cannot stat: No such file or directory
cpio: parsing_cocci/lexer_script.ml: Cannot stat: No such file or directory
cpio: printf.ml: Cannot stat: No such file or directory
cpio: pyml-current/py.ml: Cannot stat: No such file or directory
cpio: pyml-current/pyutils.ml: Cannot stat: No such file or directory
cpio: random.ml: Cannot stat: No such file or directory
cpio: set.ml: Cannot stat: No such file or directory
cpio: stdlib.ml: Cannot stat: No such file or directory
cpio: str.ml: Cannot stat: No such file or directory
cpio: string.ml: Cannot stat: No such file or directory
cpio: tools/spgen/source/spgen_lexer.ml: Cannot stat: No such file or directory

I think these have been there since forever, it doesn't seem to break anything,
in particular rpmbuild doesn't find ant missing files for the %install.

So, even unfixed, I think this is fully functional and a net improvement.

Comment 11 John Friend 2018-11-08 13:42:04 UTC
Created attachment 1503341 [details]
Install log with (benign?) cpio warnings aboout missing files

Comment 12 Richard W.M. Jones 2018-11-08 13:54:45 UTC
Yes the cpio warnings are benign.  They come from the RPM debuginfo
generation step because it doesn't know how to find the source
files from the OCaml library path (indeedsome of the sources files
don't even exist).  In any case it can be ignored.

I have updated the PYTHONPATH logic (note I'd already fixed that in the -3)
build.  If there's nothing else then as far as I know it should
work on Fedora 29?

Comment 13 John Friend 2018-11-08 14:06:09 UTC
Ah, so you did. 

Yes, I think we're good to go on fedora 29.

Comment 14 Fedora Update System 2018-11-08 14:18:44 UTC
coccinelle-1.0.7-4.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-15cfaca5d4

Comment 15 Fedora Update System 2018-11-08 14:19:43 UTC
coccinelle-1.0.7-4.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-0cac07f63a

Comment 16 John Friend 2018-11-08 14:32:41 UTC
great, thanks.

Comment 17 Fedora Update System 2018-11-09 07:44:59 UTC
coccinelle-1.0.7-4.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-0cac07f63a

Comment 18 Fedora Update System 2018-11-09 07:50:00 UTC
coccinelle-1.0.7-4.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-15cfaca5d4

Comment 19 John Friend 2018-11-16 22:46:15 UTC
bump. This is ready to be pushed from testing to updates.

Comment 20 Fedora Update System 2018-11-18 02:56:50 UTC
coccinelle-1.0.7-4.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2018-11-18 03:55:21 UTC
coccinelle-1.0.7-4.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.