Bug 1409363 (llvm3.9) - Review Request: llvm3.9 - Versioned LLVM 3.9
Summary: Review Request: llvm3.9 - Versioned LLVM 3.9
Keywords:
Status: CLOSED NEXTRELEASE
Alias: llvm3.9
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kevin Fenzi
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-01 11:24 UTC by Milan Bouchet-Valat
Modified: 2017-07-10 11:37 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-07-10 11:37:24 UTC
Type: Bug
Embargoed:
kevin: fedora-review+


Attachments (Terms of Use)

Description Milan Bouchet-Valat 2017-01-01 11:24:57 UTC
Spec URL: https://nalimilan.fedorapeople.org/llvm3.9.spec
SRPM URL: https://nalimilan.fedorapeople.org/llvm3.9-3.9.1-1.fc24.src.rpm
Description: LLVM is a compiler infrastructure designed for compile-time,
link-time, runtime, and idle-time optimization of programs from
arbitrary programming languages.  The compiler infrastructure includes
mirror sets of programming tools as well as libraries with equivalent
functionality.
Fedora Account System Username: nalimilan

This package is based on the llvm .spec file, with a patch to allow installing files to versioned directories and several patches needed by Julia. I'm going to need it soon to package Julia 0.6.0 in rawhide.

AFAIK, according to the rules (and previous experience in Bug 1352215), no review is needed. But I still need a Bug # to file a SCM admin request.

Comment 1 Milan Bouchet-Valat 2017-01-01 11:28:56 UTC
I've just filed an SCM admin request.

Comment 2 Kevin Fenzi 2017-01-02 20:54:30 UTC
Package request has been denied with the reason: The package must be reviewed and fedora-review flag set to + before you can request the package.

Comment 3 Orion Poplawski 2017-01-02 21:11:30 UTC
llvm3.9.spec:35: W: mixed-use-of-spaces-and-tabs (spaces: line 35, tab: line 10)

I think -static is missing:

Requires:       %{name}-devel%{?_isa} = %{version}-%{release}

No %license?

Comment 4 Milan Bouchet-Valat 2017-01-02 22:48:26 UTC
Kevin: I'm fine with going through a review, but I was told the contrary for llvm3.7, and the rules at https://fedoraproject.org/wiki/Packaging:ReviewGuidelines#Package_Review_Process seem to imply otherwise. Maybe they could be made more explicit?

Orion: Thanks. These issues actually come from the original llvm .spec file. I've also noticed another issue with Requires(posttrans) being repeated instead of Requires(postun). All of them are fixed in the new version; they should be added to llvm.spec too.

I've also disabled generating HTML docs: since the recent Sphinx update, the build fails with obscure errors. This package isn't intended for general development anyway, people would better use llvm for that.

Finally, I've added patches needed by Julia.

Spec URL: https://nalimilan.fedorapeople.org/llvm3.9.spec
SRPM URL: https://nalimilan.fedorapeople.org/llvm3.9-3.9.1-2.fc24.src.rpm

Comment 5 Kevin Fenzi 2017-01-02 23:09:56 UTC
Ah, right you are, but thats a bit unclear to me. I guess it implies that whoever is processing the scm request has to check those things. "The package MUST be properly named according to the naming guidelines and MUST NOT conflict with all other versions of the same package. If these requirements are not met, an exemption is required as above." 

I don't have time to check those now, but can try and do so later, or someone else can look and finish processing this. Sorry for the delay.

Comment 6 Milan Bouchet-Valat 2017-01-03 13:14:42 UTC
No worries.

Here's the Koji build on all arches: https://koji.fedoraproject.org/koji/taskinfo?taskID=17155580

Comment 7 Jens Petersen 2017-01-04 02:16:35 UTC
Just for the record llvm 3.9.0 is already in Rawhide since end of Nov.

Or maybe do you still need llvm3.9 for current releases?

Comment 8 Milan Bouchet-Valat 2017-01-04 09:55:42 UTC
Yes, for now rawhide has 3.9.0, but I expect F26 to ship with 4.0, which will be released in February. So indeed for the next few weeks it's going to be redundant, but I figured I'd better start the review process early.

Comment 9 Kevin Fenzi 2017-01-08 19:25:47 UTC
Package request has been approved: https://admin.fedoraproject.org/pkgdb/package/rpms/llvm3.9

Comment 10 Jeremy Newton 2017-01-13 17:41:42 UTC
Will there also be a clang3.9? I believe clang was recently separated from llvm, so a clang counterpart maybe in order, although I'm not sure if this is useful.

As well, this could also be branched onto F25 to provide better performance and functionality to mesa but would require a clang3.9 counterpart for building IIRC, see bug #1381109 for details.

Comment 11 Milan Bouchet-Valat 2017-01-13 19:17:54 UTC
No, it doesn't contain clang. Indeed it's a separate package since 3.7 AFAICT, so somebody will have to package it separately. Note that since the port to CMake, versioned install directories are no longer supported, so that will likely require some work. The patches I use for llvm3.9 are being reviewed at https://reviews.llvm.org/D28234

I'll push the contents of llvm3.9 after that upstream review.

Comment 12 Orion Poplawski 2017-02-06 23:35:42 UTC
This package doesn't build for me:

+ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-extra-options=-fno-devirtualize --with-extra-ld-options=-Wl,-Bsymbolic --libdir=/usr/lib64/llvm3.9 --includedir=/usr/include/llvm3.9 --disable-polly --disable-libcpp --enable-cxx11 --enable-clang-arcmt --enable-clang-static-analyzer --enable-clang-rewriter --enable-optimized --disable-profiling --disable-assertions --disable-werror --disable-expensive-checks --enable-debug-runtime --enable-keep-symbols --enable-jit --enable-docs --disable-doxygen --enable-threads --enable-pthreads --enable-zlib --enable-pic --enable-shared --enable-timestamps --enable-backtraces --enable-targets=x86,powerpc,arm,aarch64,cpp,nvptx,systemz,r600 --enable-bindings=none --enable-libffi --enable-ltdl-install --with-binutils-include=/usr/include
################################################################################
################################################################################
The LLVM project no longer supports building with configure & make.

Please migrate to the CMake-based build system.
For more information see: http://llvm.org/docs/CMake.html
################################################################################
################################################################################

Comment 13 Orion Poplawski 2017-02-06 23:37:27 UTC
I don't think the content of https://nalimilan.fedorapeople.org/llvm3.9-3.9.1-2.fc24.src.rpm is correct.

Comment 14 Milan Bouchet-Valat 2017-02-08 09:28:25 UTC
Are you sure? I just downloaded the SRPM again, and the .spec file calls %cmake, not ./configure.

Update on progress: the patch is still being reviewed upstream. If that takes too long, I'll push the package with the current patch, but for now there's no hurry (at least for me).

Comment 15 Orion Poplawski 2017-02-08 23:46:08 UTC
My bad - I got confused with too many different versions at the same time.

I've been playing around with trying to build a versioned clang with the versioned llvm packages (you've seen my commits to llvm3.7 towards that).  I'm starting to think it would be better to install into a prefix (%_libdir/%name) and then move/symlink from there rather than the reverse in order to get the output of llvm-config right and satisfy some assumptions of the clang package.

Comment 16 Milan Bouchet-Valat 2017-02-09 11:15:51 UTC
I don't understand what you mean. We don't currently move libraries around, we just pass the needed paths to configure/cmake. My patch against the CMake system ensures llvm-config returns the correct paths, else it couldn't be accepted upstream.

Comment 17 Orion Poplawski 2017-02-09 16:21:40 UTC
Here are the issues I've encountered trying to use llvm3.X, including building clang3.X with llvm3.X:

- clang wants to find 'llvm-config', but currently this is 'llvm-config-%{_isa_bits}-3.X'.  I modified llvm3.7 to make llvm-config available in %_libdir/%name/bin, so you can just add that to PATH.  This helps with other tools as well.

- clang wants to find LLVMConfing.cmake in `llvm-config --prefix`/share/llvm/cmake, but:

# llvm-config-64-3.9 --prefix
/usr

which makes it look in /usr/share/llvm/cmake, which is wrong.  While looking at packaging clang3.X I've ended up installing into %_libdir/%name so that --prefix can work.  See bug #1420512 for my current package.

Alternatively I can patch clang's CMakeLists.txt to work, and perhaps it's worth working with clang upstream to find a solution.

- I'm a little concerned about the overlap with alternatives between the various llvm packages.  In some ways it's nice to install any one and have access to llvm-config, but the priority is all the same so it seems indeterminate as to which becomes active.

Comment 18 Orion Poplawski 2017-02-09 16:53:07 UTC
And now looking at 3.9 - 

/usr/lib64/cmake/llvm3.9/LLVMExports.cmake:set(_IMPORT_PREFIX "/usr")

leads to errors like:

CMake Error at //usr/lib64/cmake/llvm3.9/LLVMExports.cmake:888 (message):
  The imported target "llvm-tblgen" references the file

     "/usr/bin/llvm-tblgen"

  but this file does not exist.  Possible reasons include:

Another reason I think we need to install into a separate prefix.

Comment 19 Orion Poplawski 2017-02-09 21:30:30 UTC
Well, my attempt at installing into will not work with current llvm-config if we want to keep the llvm-%{major_version} names in /usr/bin.  Multi-lib requires those names to the binaries (since links in /usr/bin would conflict), but llvm-config now determines the prefix by resolving the path and going up a directory.  If we dropped the versioned names we could keep the binaries in %_libdir/%name/bin.

Comment 20 Milan Bouchet-Valat 2017-02-09 22:22:20 UTC
I haven't had the time to look in depth to this, but it looks like the incorrect LLVMConfing.cmake path should be fixed upstream. I think there's even a CMake variable for this, which I might even have fixed in recent versions of my patch.

OTOH, indeed we will have to stop renaming files under /usr/bin, and instead put them under a specific directory, if we want Clang to be able to find them just using llvm-config.

Comment 21 Orion Poplawski 2017-02-09 22:34:21 UTC
Here is my current spec that installs into prefix:

https://www.cora.nwra.com/~orion/fedora/llvm3.9.spec

I've also kept the versioned binaries for everything except llvm-config.  With this I'm able to build clang3.9 with:

export PATH=%{_libdir}/llvm%{major_version}/bin:$PATH

Comment 22 Milan Bouchet-Valat 2017-02-11 15:43:19 UTC
I don't like the idea of putting everything under /lib64/llvm3.9. That probably doesn't follow the FHS, even if we add symlinks. It took me quite some work to improve LLVM's CMake build system, so why not make Clang use that? All the required support is there.

For example, it looks like Clang should use LLVM_CMAKE_DIR to find where CMake files have been installed. If you need to get that information from llvm-config instead, we can add an argument for it. Or you could just hardcode it since it's not going to change and the LLVM version is hardcoded too.

Comment 23 Orion Poplawski 2017-02-13 19:19:59 UTC
It's really not that different from how we handle MPI packages: http://fedoraproject.org/wiki/Packaging:MPI

What would really be good to do is to have clang's CMakeLists.txt do:

find_package(LLVM 3.9 REQUIRED)
list(APPEND CMAKE_MODULE_PATH "${LLVM_CMAKE_DIR}")

but this will require more fixes to LLVMConfig.cmake.

Comment 24 Orion Poplawski 2017-02-13 22:57:57 UTC
Here's my current version:

https://www.cora.nwra.com/~orion/fedora/llvm3.9.spec
https://www.cora.nwra.com/~orion/fedora/llvm3.9-3.9.1-4.el7.src.rpm

* Fri Feb 10 2017 Orion Poplawski <orion.com> - 3.9.1-4
- Add patch to add sonames to libraries
- Make -static require ncurses-devel

* Thu Feb 9 2017 Orion Poplawski <orion.com> - 3.9.1-3
- Install into libdir prefix

This sets a SOVERSION for libraries, and by extension for clang since it uses llvm's cmake macros.  This prevents the versioned llvm or clang from loading the wrong libraries.

This also updates the install_dirs.patch to allow clang to be able to do find_package(LLVM).

I'm doing some builds locally and at https://copr.fedorainfracloud.org/coprs/orion/llvm_versioned/

Comment 25 Orion Poplawski 2017-02-13 23:48:10 UTC
My updated install_dirs.patch doesn't quite work with your current -2 llvm package.  I get:

CMake Error at /usr/lib64/cmake/llvm3.9/LLVMConfig.cmake:165 (include):
  include could not find load file:

    /usr//usr/lib64/cmake/llvm3.9/LLVMExports.cmake

because LLVM_INSTALL_PACKAGE_DIR is being specified as a complete path and the LLVMConfig.cmake file doesn't support that.

        -DLLVM_INSTALL_PACKAGE_DIR=%{_libdir}/cmake/%{name} \

But I don't think that should be too hard to fix.

Comment 26 Milan Bouchet-Valat 2017-02-23 13:13:22 UTC
Sorry for the delay. I don't really care about where things are installed as long as I can build Julia with the package. Can you upload your version of the package to git? Then I'll need to add some more patches for 4.0 backports.

Comment 27 Orion Poplawski 2017-02-23 15:21:22 UTC
Done.  Thanks.

Comment 28 Milan Bouchet-Valat 2017-02-23 16:58:54 UTC
Thanks. I've pushed my patches. Do you need anything more, or should we trigger a build?

Comment 29 Orion Poplawski 2017-02-23 17:10:40 UTC
I think I'm good, so go ahead.

Comment 30 Milan Bouchet-Valat 2017-07-10 11:37:24 UTC
llvm3.9 is in F26 for some time now.


Note You need to log in before you can comment on or make changes to this bug.