Bug 2083958 - Libdir does not contain /include directory (bad layout of installed files?)
Summary: Libdir does not contain /include directory (bad layout of installed files?)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: ghc9.2
Version: 36
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jens Petersen
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-11 06:17 UTC by Alexey Radkov
Modified: 2022-06-12 02:05 UTC (History)
1 user (show)

Fixed In Version: ghc9.2-9.2.3-10.fc36
Clone Of:
Environment:
Last Closed: 2022-06-12 02:05:13 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Alexey Radkov 2022-05-11 06:17:22 UTC
Description of problem:

In ghc9.2

# ghc-9.2.2 --print-libdir
/usr/lib64/ghc-9.2.2/lib

The /include directory locates at one level up /usr/lib64/ghc-9.2.2/include. The GHC 9.2 reference says:
 
> --print-libdir
> 
>     Print the path to GHC’s library directory. This is the top of the directory tree containing GHC’s libraries, interfaces, and include files (usually something like /usr/local/lib/ghc-5.04 on Unix). This is the value of $libdir in the package configuration file (see Packages).

So, it seems that $libdir should contain /include, and I used this when configuring a C project that included HsFFI.h.

Notice that neither Fedora GHC modules 8.10, 9.0 and 9.2 nor the standard rpm packages ghc8.10 and ghc9.0 have this issue: their $libdir do not end with extra /lib, and thus they contain the /include dir, e.g. in ghc module 9.2

# ghc --print-libdir
/usr/lib64/ghc-9.2.1

or in ghc9.0

# ghc-9.0.2 --print-libdir
/usr/lib64/ghc-9.0.2

As well (but of lesser importance to me), I used $(ghc-9.0.2 --print-libdir)/rts to access directory with rts libs, in ghc9.2 it is hidden in a nested directory and has a suffix added: /usr/lib64/ghc-9.2.2/lib/x86_64-linux-ghc-9.2.2/rts-1.0.2.


Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
ls $(ghc-9.2.2 --print-libdir)/include

Actual results:
ls: cannot access '/usr/lib64/ghc-9.2.2/lib/include': No such file or directory

Expected results:
Cmm.h               HsFFI.h     Rts.h     Stg.h          ghcconfig.h    ghcversion.h  stg/
DerivedConstants.h  MachDeps.h  RtsAPI.h  ghcautoconf.h  ghcplatform.h  rts/

Additional info:

Comment 1 Jens Petersen 2022-05-11 09:31:45 UTC
Thanks for reporting

I think this is because ghc9.2 is built with the new ghc Hadrian buildsystem,
and I had to bundle libffi instead of using the system due to some build config issues.

But we just fixed this for the coming 9.4 release.

Let me see if I can backport the fix to 9.2 as well.

Comment 2 Jens Petersen 2022-05-11 09:34:21 UTC
Though it is possible the change of directory layout might still affect you.

Comment 3 Jens Petersen 2022-05-11 09:35:21 UTC
It might help me if you are able to share your exact build steps
so that I can test it locally too.

Comment 4 Alexey Radkov 2022-05-11 09:40:16 UTC
Thanks!

Btw, disregard my second complain about rts library path, i did not actually use it correctly.

Comment 5 Jens Petersen 2022-05-11 09:56:32 UTC
Also if you want you can try the new ghc9.4 alpha1 package.
I just pushed it for updates-testing, though you can install directly from koji:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1964732

Eg I used koji-tool like this:

$ koji-tool install ghc9.4 -x '*prof' -x '*doc' -x compiler-default

Comment 6 Jens Petersen 2022-05-11 09:58:46 UTC
But it kind of sounds like your problem is with the newer directory structure
not the bundling of the libffi library. If so I am less sure how to fix that.

Comment 7 Alexey Radkov 2022-05-11 10:43:33 UTC
(In reply to Jens Petersen from comment #3)
> It might help me if you are able to share your exact build steps
> so that I can test it locally too.

Not sure that it would be simple to reproduce the build steps as it requires installing Nginx.

The issue arises when I build nginx-haskell-module (https://github.com/lyokha/nginx-haskell-module). This module does not link against any Haskell library at compile time (it links them in run-time via custom Haskell handlers - i.e. plain FFI libraries - using dlopen()), however it #include <HsFFI.h> to access GHC type declarations.

Nginx provides a simple mechanism to build Makefiles via configure script by writing file named *config* associated with the Nginx module. After putting there line

> ngx_module_incs="$(ghc --print-libdir)/include"

the configure scripts creates a C flag like -I/path/to/GHC/libdir/include when running

> ./configure --add-module=$HOME/devel/nginx-haskell-module

And obviously, make breaks with error message

> In file included from /home/lyokha/devel/nginx-haskell-module/src/ngx_http_haskell_module.c:17:
> /home/lyokha/devel/nginx-haskell-module/src/ngx_http_haskell_module.h:25:10: fatal error: HsFFI.h: No such file or directory
>    25 | #include <HsFFI.h>
>       |          ^~~~~~~~~
> compilation terminated.

I also built docker images for Fedora 35 and ghc8.10, ghc9.0 and ghc9.2 (from https://raw.githubusercontent.com/lyokha/nginx-haskell-module/master/docker/Dockerfile), but for ghc9.2 I needed to put line

> ln -s /usr/lib64/ghc-9.2.2/include /usr/lib64/ghc-9.2.2/lib

before the ./configure to embed the /include directory into the directory that $(ghc --print-libdir) prints.

OTOH, I also made github CI action to build the module with GHC 9.2 as well (https://raw.githubusercontent.com/lyokha/nginx-haskell-module/master/.github/workflows/ci.yml), however it builds GHC installed by ghcup from https://github.com/haskell/actions on Ubuntu, as I see in build logs

> Installing ghc version 9.2.2
>   /opt/hostedtoolcache/ghcup/0.1.17.6/x64/ghcup set ghc 9.2.2
>   [ Info  ] GHC 9.2.2 successfully set as default version
>   
>   
>   Found ghc 9.2.2 in cache at path /home/runner/.ghcup/ghc/9.2.2/bin. Setup successful.

 And it worked just fine (i.e. it seems that it installed GHC directories in the *expected* way).

Comment 8 Alexey Radkov 2022-05-21 12:37:03 UTC
I filed the bug at GHC's: https://gitlab.haskell.org/ghc/ghc/-/issues/21609

Comment 9 Jens Petersen 2022-05-23 11:35:09 UTC
Thanks again for reporting this

(In reply to Alexey Radkov from comment #7)
> The issue arises when I build nginx-haskell-module
> (https://github.com/lyokha/nginx-haskell-module). This module does not link
> against any Haskell library at compile time (it links them in run-time via
> custom Haskell handlers - i.e. plain FFI libraries - using dlopen()),
> however it #include <HsFFI.h> to access GHC type declarations.

Ah I see, cool - that is interesting!
I haven't really seen such usage much before.

> Nginx provides a simple mechanism to build Makefiles via configure script by
> writing file named *config* associated with the Nginx module. After putting
> there line
> 
> > ngx_module_incs="$(ghc --print-libdir)/include"

Does it help if you append /../include instead of /include ?

> I also built docker images for Fedora 35 and ghc8.10, ghc9.0 and ghc9.2
> (from
> https://raw.githubusercontent.com/lyokha/nginx-haskell-module/master/docker/
> Dockerfile), but for ghc9.2 I needed to put line
> 
> > ln -s /usr/lib64/ghc-9.2.2/include /usr/lib64/ghc-9.2.2/lib
> 
> before the ./configure to embed the /include directory into the directory
> that $(ghc --print-libdir) prints.

I guess I can try to add such a symlink to the package for back-compat.
Thanks for filing the upstream issue.

> OTOH, I also made github CI action to build the module with GHC 9.2 as well
> (https://raw.githubusercontent.com/lyokha/nginx-haskell-module/master/.
> github/workflows/ci.yml), however it builds GHC installed by ghcup from
> https://github.com/haskell/actions on Ubuntu, as I see in build logs
> 
> > Installing ghc version 9.2.2
> >   /opt/hostedtoolcache/ghcup/0.1.17.6/x64/ghcup set ghc 9.2.2
> >   [ Info  ] GHC 9.2.2 successfully set as default version
> >   
> >   Found ghc 9.2.2 in cache at path /home/runner/.ghcup/ghc/9.2.2/bin. Setup successful.
> 
>  And it worked just fine (i.e. it seems that it installed GHC directories in
> the *expected* way).

I just checked with the upstream ghcup maintainer and apparently the ghc 9.2.2 bindists
it provides are only built with Hadrian for Alpine and MacOS. So that explains the discrepancy.

I am hoping this may get fixed for 9.2.3, but that may take some time.

I will try to add a symlink later when I have time to rebuild ghc9.2 (and ghc9.4).
Sorry for the inconvenience, though I would like to ship 9.2+ built with the newer Hadrian system.

Comment 10 Alexey Radkov 2022-05-23 11:56:02 UTC
> Does it help if you append /../include instead of /include ?

Yes, it helps, but it does not look reliable as the path may change in other releases of GHC.

> I guess I can try to add such a symlink to the package for back-compat.
> Thanks for filing the upstream issue.

The issue was already assigned and bound to 9.4.1 milestone, I think that the best option would be waiting until its resolution.

> Sorry for the inconvenience, though I would like to ship 9.2+ built with the newer Hadrian system.

Not a problem at all, I will customize paths in the config until the issue gets fixed in the upstream.

Thanks! I really appreciate your work in bringing GHC to Fedora!

Comment 11 Jens Petersen 2022-05-23 17:13:24 UTC
Thanks, Alexey

I am pushing ghc9.2-9.2.2-9.fc37 to Rawhide,
which among other things adds an include/ symlink.

Comment 12 Alexey Radkov 2022-05-23 17:18:53 UTC
Thank you!

Comment 13 Jens Petersen 2022-05-26 14:08:21 UTC
> I am pushing ghc9.2-9.2.2-9.fc37 to Rawhide

Unfortunately the build is failing on s390x
(due to libffi changes I made I think)
so it may have to wait for 9.2.3.

Comment 14 Fedora Update System 2022-06-03 15:10:17 UTC
FEDORA-2022-09a95704d9 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-09a95704d9

Comment 15 Fedora Update System 2022-06-04 02:08:14 UTC
FEDORA-2022-09a95704d9 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-09a95704d9`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-09a95704d9

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Alexey Radkov 2022-06-07 17:31:03 UTC
Looks that it had better solution (from my side): see https://gitlab.haskell.org/ghc/ghc/-/issues/21609#note_435800.

Command

$ ghc --print-libdir

has a better variant for finding include files:

$ ghc-pkg field rts include-dirs  --simple-output

This prints identical set of include files containing HsFFI.h and ghcversion.h as well. So I switched to this command and tested this for GHC versions 8.0 through 9.4, and this worked well. So, symbolic links are probably not needed as ghc-pkg is able to retrieve necessary information without them.

Comment 17 Jens Petersen 2022-06-08 10:08:57 UTC
Great, thanks Alexey

Then I guess I can drop the symlink from ghc9.2

Comment 18 Alexey Radkov 2022-06-08 10:41:15 UTC
Yes, the new command works well for all versions of GHC. Thank you very much!

Comment 19 Fedora Update System 2022-06-12 02:05:13 UTC
FEDORA-2022-09a95704d9 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.