Bug 2450007 - Review Request: nvidia-container-toolkit - Build and run containers leveraging NVIDIA GPUs
Summary: Review Request: nvidia-container-toolkit - Build and run containers leveragin...
Keywords:
Status: ASSIGNED
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Tom.Rix
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2026-03-21 19:39 UTC by Jonathan Steffan
Modified: 2026-05-23 01:51 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed:
Type: ---
Embargoed:


Attachments (Terms of Use)
The .spec file difference from Copr build 10304665 to 10311924 (1.84 KB, patch)
2026-04-10 15:18 UTC, Fedora Review Service
no flags Details | Diff
The .spec file difference from Copr build 10311924 to 10317718 (1.16 KB, patch)
2026-04-13 11:03 UTC, Fedora Review Service
no flags Details | Diff
The .spec file difference from Copr build 10452587 to 10456018 (745 bytes, patch)
2026-05-13 14:46 UTC, Fedora Review Service
no flags Details | Diff
The .spec file difference from Copr build 10456018 to 10461429 (326 bytes, patch)
2026-05-14 22:24 UTC, Fedora Review Service
no flags Details | Diff
The .spec file difference from Copr build 10461429 to 10501619 (509 bytes, patch)
2026-05-22 01:18 UTC, Fedora Review Service
no flags Details | Diff
The .spec file difference from Copr build 10501619 to 10504408 (3.77 KB, patch)
2026-05-23 01:51 UTC, Fedora Review Service
no flags Details | Diff

Description Jonathan Steffan 2026-03-21 19:39:44 UTC
Spec URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit.spec
SRPM URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit-1.19.0-1.fc45.src.rpm
Description: Build and run containers leveraging NVIDIA GPUs.
Fedora Account System Username: jsteffan

Comment 1 Fedora Review Service 2026-03-21 19:48:21 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10250149
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10250149-nvidia-container-toolkit/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 2 Mikel Olasagasti Uranga 2026-04-09 10:33:54 UTC
- Add explicit BuildRequires on make

> %make_build

- Why custom __golang_extldflags are required? Add a comment

> %global __golang_extldflags -Wl,-z,lazy -Wl,--export-dynamic

- Current template for the %prep part does the following:

> %prep
> %goprep -p1
> tar -xf %{S:1}

While you're doing:

> %goprep -A
> %setup -q -T -D -a1 %{forgesetupargs}
> tar xf %{SOURCE3}
> %autopatch -p1

Feels a pre 1.19 go2rpm template was used and later reapplied somehow. It's purely cosmetic request, but if you could normalize it would be better.

Comment 3 Mikel Olasagasti Uranga 2026-04-09 10:34:18 UTC
[fedora-review-service-build]

Comment 4 Fedora Review Service 2026-04-09 10:43:01 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10304665
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10304665-nvidia-container-toolkit/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 5 Mikel Olasagasti Uranga 2026-04-09 11:47:38 UTC
- Checking upstream Makefile, NVIDIA sets to:

> EXTLDFLAGS = -Wl,--export-dynamic -Wl,--unresolved-symbols=ignore-in-object-files -Wl,-z,lazy

https://github.com/NVIDIA/nvidia-container-toolkit/blob/main/Makefile#L63

- Version ldflags are not set:

> -X $(CLI_VERSION_PACKAGE).gitCommit=$(GIT_COMMIT) -X $(CLI_VERSION_PACKAGE).version=$(CLI_VERSION)"

Export the following after "%global gomodulesmode GO111MODULE=on"

> export GO_LDFLAGS="-X github.com/NVIDIA/nvidia-container-toolkit/internal/info.version=%{version}  \
>                   -X github.com/NVIDIA/nvidia-container-toolkit/internal/info.gitCommit=%{commit}"

The problem is that gitCommit/commit would need to be modified on each version update. You can have something else written there like "unkonwn" or "Fedora". Check how is printed with the upstream version and if it has to have any known format.

Comment 6 Jonathan Steffan 2026-04-10 15:06:50 UTC
Changelog:

- Update __golang_extldflags to match upstream, link to upstream
- Set info.version and info.gitCommit
- Add make BR

Spec URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit.spec
SRPM URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit-1.19.0-1.fc45.src.rpm

Why I'm doing the awkwardly looking %prep is because of an odd mix between forge, go macros, what I need to patch, and the added sources. I'll experiment a bit to see where this is breaking down.

Comment 7 Fedora Review Service 2026-04-10 15:18:38 UTC
Created attachment 2136591 [details]
The .spec file difference from Copr build 10304665 to 10311924

Comment 8 Fedora Review Service 2026-04-10 15:18:42 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10311924
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10311924-nvidia-container-toolkit/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 9 Jonathan Steffan 2026-04-12 17:25:53 UTC
Changelog:

- Remove patch, instead copy in staged required binary to expected location: https://github.com/NVIDIA/nvidia-container-toolkit/issues/1758
- Update %prep to be more modern

Spec URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit.spec
SRPM URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit-1.19.0-1.fc45.src.rpm

Comment 10 Mikel Olasagasti Uranga 2026-04-13 10:54:25 UTC
[fedora-review-service-build]

Comment 11 Fedora Review Service 2026-04-13 11:03:37 UTC
Created attachment 2136942 [details]
The .spec file difference from Copr build 10311924 to 10317718

Comment 12 Fedora Review Service 2026-04-13 11:03:40 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10317718
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10317718-nvidia-container-toolkit/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 13 Tom.Rix 2026-05-11 19:52:44 UTC
(In reply to Mikel Olasagasti Uranga from comment #5)
> - Checking upstream Makefile, NVIDIA sets to:
> 
> > EXTLDFLAGS = -Wl,--export-dynamic -Wl,--unresolved-symbols=ignore-in-object-files -Wl,-z,lazy
> 
> https://github.com/NVIDIA/nvidia-container-toolkit/blob/main/Makefile#L63
> 
rpmlint is also complaining a lot about symbols.  What resolves these symbols ?

Comment 14 Tom.Rix 2026-05-11 20:14:00 UTC
Source3:        https://github.com/NVIDIA/dgx-selinux/archive/%{dgx_selinux_commit}/dgx-selinux-%{dgx_selinux_commit}.tar.gz

From the readme
https://github.com/NVIDIA/dgx-selinux/blob/master/README.md
This is expected to run on bare metal, not just containers.

I think this needs its own package.
And it can follow the selinux packaging policy
https://fedoraproject.org/wiki/PackagingDrafts/SELinux_Independent_Policy

Comment 15 Jonathan Steffan 2026-05-12 14:38:06 UTC
(In reply to Tom.Rix from comment #13)
> (In reply to Mikel Olasagasti Uranga from comment #5)
> > - Checking upstream Makefile, NVIDIA sets to:
> > 
> > > EXTLDFLAGS = -Wl,--export-dynamic -Wl,--unresolved-symbols=ignore-in-object-files -Wl,-z,lazy
> > 
> > https://github.com/NVIDIA/nvidia-container-toolkit/blob/main/Makefile#L63
> > 
> rpmlint is also complaining a lot about symbols.  What resolves these
> symbols ?

They are resolved at runtime by the linker:

  Library location (pkg/lookup/library.go): A First() combinator tries three strategies in order,
  returning the first hit:
  1. Direct dlopen attempt
  2. Predefined filesystem search paths
  3. Parse /etc/ld.so.cache

  ld.so.cache parsing (internal/ldcache/ldcache.go): The binary cache is mmap-ed directly. Each entry2
  struct has a Flags field with ELF architecture bits (0x0300 = x86_64, 0x0a00 = aarch64, etc.) used to
  filter 32-bit vs 64-bit libs. No ldconfig subprocess — raw struct parsing.

  dlopen/dlsym (vendor/github.com/NVIDIA/go-nvml/pkg/dl/dl.go): CGO wrapper around libc dlopen/dlsym.
  Flags: RTLD_LAZY | RTLD_GLOBAL | RTLD_NODELETE. Every call is wrapped with runtime.LockOSThread() for
  thread safety.

  Symlink chain resolution (pkg/lookup/symlinks/symlink.go): Once a library path is found, the full
  symlink chain is walked (e.g., libcuda.so → libcuda.so.1 → libcuda.so.550.1.0) and all aliases are
  collected — so containers get every soname variant.

  Versioned symbol negotiation (internal/nvsandboxutils/lib.go): After dlopen, updateVersionedSymbols()
  probes for _v2/_v3 suffixed symbols via dlsym. If found, the function pointer is upgraded — forward
  compatibility without ABI breaks.

  Summary: locate via ldcache mmap → dlopen → dlsym with version probing → expose full symlink chain.

Comment 16 Jonathan Steffan 2026-05-12 14:38:50 UTC
(In reply to Tom.Rix from comment #14)
> Source3:       
> https://github.com/NVIDIA/dgx-selinux/archive/%{dgx_selinux_commit}/dgx-
> selinux-%{dgx_selinux_commit}.tar.gz
> 
> From the readme
> https://github.com/NVIDIA/dgx-selinux/blob/master/README.md
> This is expected to run on bare metal, not just containers.
> 
> I think this needs its own package.
> And it can follow the selinux packaging policy
> https://fedoraproject.org/wiki/PackagingDrafts/SELinux_Independent_Policy

Agreed. I'll drop it and add a new policy package for inclusion.

Comment 17 Jonathan Steffan 2026-05-12 14:57:32 UTC
Changelog:

- Drop the additional SELinux policy, keep the boolean management

Spec URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit.spec
SRPM URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit-1.19.0-1.fc45.src.rpm

Comment 18 Fedora Review Service 2026-05-13 03:46:55 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10452587
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10452587-nvidia-container-toolkit/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 19 Jonathan Steffan 2026-05-13 14:36:14 UTC
Changelog:

- Fix the SELinux changes (oops don't remove the %files section :))

Spec URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit.spec
SRPM URL: https://jsteffan.fedorapeople.org/nvidia-container-toolkit/nvidia-container-toolkit-1.19.0-1.fc45.src.rpm

Comment 20 Fedora Review Service 2026-05-13 14:46:32 UTC
Created attachment 2141077 [details]
The .spec file difference from Copr build 10452587 to 10456018

Comment 21 Fedora Review Service 2026-05-13 14:46:34 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10456018
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10456018-nvidia-container-toolkit/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 22 Tom.Rix 2026-05-14 19:00:41 UTC
> 
>   Symlink chain resolution (pkg/lookup/symlinks/symlink.go): Once a library
> path is found, the full
>   symlink chain is walked (e.g., libcuda.so → libcuda.so.1 →
> libcuda.so.550.1.0) and all aliases are
>   collected — so containers get every soname variant.
> 

cuda is of course not open source.
What are the legal issues for this method ?
I am hoping you can point to a past ruling saying it is ok.

Comment 23 Jonathan Steffan 2026-05-14 22:02:43 UTC
(In reply to Tom.Rix from comment #22)
> > 
> >   Symlink chain resolution (pkg/lookup/symlinks/symlink.go): Once a library
> > path is found, the full
> >   symlink chain is walked (e.g., libcuda.so → libcuda.so.1 →
> > libcuda.so.550.1.0) and all aliases are
> >   collected — so containers get every soname variant.
> > 
> 
> cuda is of course not open source.
> What are the legal issues for this method ?
> I am hoping you can point to a past ruling saying it is ok.

Dynamically linking is the licensing demarcation point. Everything shipped in this package is licensed with an approved license and should be fine for Fedora. If the relevant DSOs are available in the probed locations, they are exposed inside the container. If not, nothing happens and the CDI configuration is unable to be assembled.

Comment 24 Jonathan Steffan 2026-05-14 22:03:31 UTC
For the approval, well it's already approved as it's already been included in Fedora: https://src.fedoraproject.org/rpms/golang-github-nvidia-container-toolkit

Comment 26 Fedora Review Service 2026-05-14 22:24:07 UTC
Created attachment 2141321 [details]
The .spec file difference from Copr build 10456018 to 10461429

Comment 27 Fedora Review Service 2026-05-14 22:24:10 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10461429
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10461429-nvidia-container-toolkit/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 28 Tom.Rix 2026-05-18 14:42:18 UTC
(In reply to Jonathan Steffan from comment #24)
> For the approval, well it's already approved as it's already been included
> in Fedora:
> https://src.fedoraproject.org/rpms/golang-github-nvidia-container-toolkit

Why are we doing a package review then ?

Comment 29 Jonathan Steffan 2026-05-18 14:45:30 UTC
(In reply to Tom.Rix from comment #28) 
> Why are we doing a package review then ?

It's a new package name and a new spec, because we are changing what the package ships. In addition, this new package uses vendoring.

Comment 30 Tom.Rix 2026-05-18 20:32:25 UTC
I am going to recommend the cuda part to be removed.

For licensing I was specifically looking for an nvidia and cuda references that would allow wrapping of cuda in fedora.  If we do this and zluda or similar hook it up their own cuda wrapper, will fedora or its users be sued ?

Comment 31 Jonathan Steffan 2026-05-18 21:24:37 UTC
I'm not sure I follow. If we don't think this content is permissive in Fedora, I need to file a ticket to get it approved? It would be silly to block this code IMHO.

Comment 33 Fedora Review Service 2026-05-22 01:18:08 UTC
Created attachment 2142275 [details]
The .spec file difference from Copr build 10461429 to 10501619

Comment 34 Fedora Review Service 2026-05-22 01:18:10 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10501619
(failed)

Build log:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10501619-nvidia-container-toolkit/builder-live.log.gz

Please make sure the package builds successfully at least for Fedora Rawhide.

- If the build failed for unrelated reasons (e.g. temporary network
  unavailability), please ignore it.
- If the build failed because of missing BuildRequires, please make sure they
  are listed in the "Depends On" field


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 36 Fedora Review Service 2026-05-23 01:51:33 UTC
Created attachment 2142420 [details]
The .spec file difference from Copr build 10501619 to 10504408

Comment 37 Fedora Review Service 2026-05-23 01:51:36 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/10504408
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2450007-nvidia-container-toolkit/fedora-rawhide-x86_64/10504408-nvidia-container-toolkit/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.


Note You need to log in before you can comment on or make changes to this bug.