Description of problem: Quarkus native image invoking in a container fails with podman 2.x. Last worked with a version < 2.x. See also: https://groups.google.com/g/quarkus-dev/c/8LVoqk4G5uc/m/M5MAVopLAwAJ Version-Release number of selected component (if applicable): podman-2.0.1-1.fc32.x86_64 How reproducible: 100% Steps to Reproduce: $ sudo dnf install java-11-openjdk-devel $ wget https://downloads.apache.org/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz $ tar -xf apache-maven-3.6.3-bin.tar.gz $ PATH=$(pwd)/apache-maven-3.6.3/bin:$PATH $ JAVA_HOME=/usr/lib/jvm/java-11-openjdk mvn --version Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f) Maven home: /home/sgehwolf/Documents/trash/quarkus-reproducer/apache-maven-3.6.3 Java version: 11.0.8-ea, vendor: N/A, runtime: /usr/lib/jvm/java-11-openjdk-11.0.8.5-0.1.ea.fc32.x86_64 Default locale: en_CA, platform encoding: UTF-8 OS name: "linux", version: "5.7.7-200.fc32.x86_64", arch: "amd64", family: "unix" $ JAVA_HOME=/usr/lib/jvm/java-11-openjdk mvn io.quarkus:quarkus-maven-plugin:1.5.2.Final:create \ -DprojectGroupId=org.acme \ -DprojectArtifactId=getting-started \ -DclassName="org.acme.getting.started.GreetingResource" \ -Dpath="/hello" $ cd getting-started $ JAVA_HOME=/usr/lib/jvm/java-11-openjdk mvn package -Pnative -Dquarkus.native.container-build=true -Dquarkus.native.container-runtime=podman Actual results: [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Pulling image quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 Trying to pull quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11... Getting image source signatures Copying blob 116562948826 skipped: already exists Copying blob 57de4da701b5 skipped: already exists Copying blob cf0f3ebe9f53 [--------------------------------------] 0.0b / 0.0b Copying config 363b397474 done Writing manifest to image destination Storing signatures 363b397474a55e1fcd4a87101f817fbca634d4bf52cb00c81895e3768dc1ea86 [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM Version 19.3.1 CE [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] podman run -v /home/sgehwolf/Documents/trash/quarkus-reproducer/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z --env LANG=C --userns=keep-id --rm quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Duser.language=en -J-Dfile.encoding=UTF-8 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -H:+JNI -jar getting-started-1.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:-IncludeAllTimeZones -H:EnableURLProtocols=http --no-server -H:-UseServiceLoaderFeature -H:+StackTrace getting-started-1.0-SNAPSHOT-runner [getting-started-1.0-SNAPSHOT-runner:25] classlist: 3,926.67 ms [getting-started-1.0-SNAPSHOT-runner:25] (cap): 794.55 ms [getting-started-1.0-SNAPSHOT-runner:25] setup: 2,203.46 ms 07:09:37,935 INFO [org.jbo.threads] JBoss Threads version 3.1.1.Final [getting-started-1.0-SNAPSHOT-runner:25] (typeflow): 17,927.66 ms [getting-started-1.0-SNAPSHOT-runner:25] (objects): 17,690.59 ms [getting-started-1.0-SNAPSHOT-runner:25] (features): 730.37 ms [getting-started-1.0-SNAPSHOT-runner:25] analysis: 38,200.71 ms [getting-started-1.0-SNAPSHOT-runner:25] (clinit): 708.81 ms [getting-started-1.0-SNAPSHOT-runner:25] universe: 1,985.66 ms [getting-started-1.0-SNAPSHOT-runner:25] (parse): 2,365.89 ms [getting-started-1.0-SNAPSHOT-runner:25] (inline): 5,591.25 ms [getting-started-1.0-SNAPSHOT-runner:25] (compile): 27,734.04 ms [getting-started-1.0-SNAPSHOT-runner:25] compile: 37,630.81 ms [getting-started-1.0-SNAPSHOT-runner:25] image: 3,177.51 ms [getting-started-1.0-SNAPSHOT-runner:25] write: 344.81 ms Fatal error: java.lang.RuntimeException: java.lang.RuntimeException: host C compiler or linker does not seem to work: java.lang.RuntimeException: returned 1 Running command: cc -v -o /project/getting-started-1.0-SNAPSHOT-runner -z noexecstack -Wl,--gc-sections -Wl,--dynamic-list -Wl,/tmp/SVM-7754259141576949641/exported_symbols.list -Wl,-x -L/tmp/SVM-7754259141576949641 -L/opt/graalvm/lib -L/opt/graalvm/lib/svm/clibraries/linux-amd64 /tmp/SVM-7754259141576949641/getting-started-1.0-SNAPSHOT-runner.o /opt/graalvm/lib/libnet.a /opt/graalvm/lib/libjava.a /opt/graalvm/lib/libzip.a /opt/graalvm/lib/libnio.a /opt/graalvm/lib/libextnet.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libffi.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libstrictmath.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libjvm.a /opt/graalvm/lib/svm/clibraries/linux-amd64/liblibchelper.a -lm -lpthread -ldl -lz -lrt Using built-in specs. COLLECT_GCC=cc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper OFFLOAD_TARGET_NAMES=nvptx-none OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux Thread model: posix gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC) COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/8/:/usr/libexec/gcc/x86_64-redhat-linux/8/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/8/:/usr/lib/gcc/x86_64-redhat-linux/ LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/8/:/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/8/../../../:/lib/:/usr/lib/ COLLECT_GCC_OPTIONS='-v' '-o' '/project/getting-started-1.0-SNAPSHOT-runner' '-z' 'noexecstack' '-L/tmp/SVM-7754259141576949641' '-L/opt/graalvm/lib' '-L/opt/graalvm/lib/svm/clibraries/linux-amd64' '-mtune=generic' '-march=x86-64' /usr/libexec/gcc/x86_64-redhat-linux/8/collect2 -plugin /usr/libexec/gcc/x86_64-redhat-linux/8/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper -plugin-opt=-fresolution=/tmp/ccg8yY7k.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --no-add-needed --eh-frame-hdr --hash-style=gnu -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /project/getting-started-1.0-SNAPSHOT-runner -z noexecstack /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/8/crtbegin.o -L/tmp/SVM-7754259141576949641 -L/opt/graalvm/lib -L/opt/graalvm/lib/svm/clibraries/linux-amd64 -L/usr/lib/gcc/x86_64-redhat-linux/8 -L/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/8/../../.. --gc-sections --dynamic-list /tmp/SVM-7754259141576949641/exported_symbols.list -x /tmp/SVM-7754259141576949641/getting-started-1.0-SNAPSHOT-runner.o /opt/graalvm/lib/libnet.a /opt/graalvm/lib/libjava.a /opt/graalvm/lib/libzip.a /opt/graalvm/lib/libnio.a /opt/graalvm/lib/libextnet.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libffi.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libstrictmath.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libjvm.a /opt/graalvm/lib/svm/clibraries/linux-amd64/liblibchelper.a -lm -lpthread -ldl -lz -lrt -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-redhat-linux/8/crtend.o /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crtn.o /usr/bin/ld: cannot open output file /project/getting-started-1.0-SNAPSHOT-runner: Permission denied collect2: error: ld returned 1 exit status at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:600) at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006) at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:462) at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:315) at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:454) at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:115) at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:479) Caused by: java.lang.RuntimeException: host C compiler or linker does not seem to work: java.lang.RuntimeException: returned 1 Running command: cc -v -o /project/getting-started-1.0-SNAPSHOT-runner -z noexecstack -Wl,--gc-sections -Wl,--dynamic-list -Wl,/tmp/SVM-7754259141576949641/exported_symbols.list -Wl,-x -L/tmp/SVM-7754259141576949641 -L/opt/graalvm/lib -L/opt/graalvm/lib/svm/clibraries/linux-amd64 /tmp/SVM-7754259141576949641/getting-started-1.0-SNAPSHOT-runner.o /opt/graalvm/lib/libnet.a /opt/graalvm/lib/libjava.a /opt/graalvm/lib/libzip.a /opt/graalvm/lib/libnio.a /opt/graalvm/lib/libextnet.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libffi.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libstrictmath.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libjvm.a /opt/graalvm/lib/svm/clibraries/linux-amd64/liblibchelper.a -lm -lpthread -ldl -lz -lrt Using built-in specs. COLLECT_GCC=cc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper OFFLOAD_TARGET_NAMES=nvptx-none OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux Thread model: posix gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC) COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/8/:/usr/libexec/gcc/x86_64-redhat-linux/8/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/8/:/usr/lib/gcc/x86_64-redhat-linux/ LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/8/:/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/8/../../../:/lib/:/usr/lib/ COLLECT_GCC_OPTIONS='-v' '-o' '/project/getting-started-1.0-SNAPSHOT-runner' '-z' 'noexecstack' '-L/tmp/SVM-7754259141576949641' '-L/opt/graalvm/lib' '-L/opt/graalvm/lib/svm/clibraries/linux-amd64' '-mtune=generic' '-march=x86-64' /usr/libexec/gcc/x86_64-redhat-linux/8/collect2 -plugin /usr/libexec/gcc/x86_64-redhat-linux/8/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper -plugin-opt=-fresolution=/tmp/ccg8yY7k.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --no-add-needed --eh-frame-hdr --hash-style=gnu -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /project/getting-started-1.0-SNAPSHOT-runner -z noexecstack /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/8/crtbegin.o -L/tmp/SVM-7754259141576949641 -L/opt/graalvm/lib -L/opt/graalvm/lib/svm/clibraries/linux-amd64 -L/usr/lib/gcc/x86_64-redhat-linux/8 -L/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/8/../../.. --gc-sections --dynamic-list /tmp/SVM-7754259141576949641/exported_symbols.list -x /tmp/SVM-7754259141576949641/getting-started-1.0-SNAPSHOT-runner.o /opt/graalvm/lib/libnet.a /opt/graalvm/lib/libjava.a /opt/graalvm/lib/libzip.a /opt/graalvm/lib/libnio.a /opt/graalvm/lib/libextnet.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libffi.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libstrictmath.a /opt/graalvm/lib/svm/clibraries/linux-amd64/libjvm.a /opt/graalvm/lib/svm/clibraries/linux-amd64/liblibchelper.a -lm -lpthread -ldl -lz -lrt -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-redhat-linux/8/crtend.o /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crtn.o /usr/bin/ld: cannot open output file /project/getting-started-1.0-SNAPSHOT-runner: Permission denied collect2: error: ld returned 1 exit status at com.oracle.svm.hosted.image.NativeBootImageViaCC.write(NativeBootImageViaCC.java:388) at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:652) at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:445) at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) Error: Image build request failed with exit status 1 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 02:04 min (Wall Clock) [INFO] Finished at: 2020-07-08T09:10:47+02:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:1.5.2.Final:build (default) on project getting-started: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors [ERROR] [error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: java.lang.RuntimeException: Failed to build native image [ERROR] at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:358) [ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [ERROR] at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:932) [ERROR] at io.quarkus.builder.BuildContext.run(BuildContext.java:277) [ERROR] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [ERROR] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2046) [ERROR] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1578) [ERROR] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452) [ERROR] at java.base/java.lang.Thread.run(Thread.java:834) [ERROR] at org.jboss.threads.JBossThread.run(JBossThread.java:479) [ERROR] Caused by: java.lang.RuntimeException: Image generation failed. Exit code: 1 [ERROR] at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.imageGenerationFailed(NativeImageBuildStep.java:374) [ERROR] at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:344) [ERROR] ... 12 more [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException Expected results: [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Pulling image quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 Trying to pull quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11... Getting image source signatures Copying blob 57de4da701b5 skipped: already exists Copying blob cf0f3ebe9f53 skipped: already exists Copying config 363b397474 done Writing manifest to image destination Storing signatures 363b397474a55e1fcd4a87101f817fbca634d4bf52cb00c81895e3768dc1ea86 [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM Version 19.3.1 CE [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] podman run -v /home/sgehwolf/Documents/trash/quarkus-reproducer/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z --env LANG=C --userns=keep-id --rm quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Duser.language=en -J-Dfile.encoding=UTF-8 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -H:+JNI -jar getting-started-1.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:-IncludeAllTimeZones -H:EnableURLProtocols=http --no-server -H:-UseServiceLoaderFeature -H:+StackTrace getting-started-1.0-SNAPSHOT-runner [getting-started-1.0-SNAPSHOT-runner:26] classlist: 3,688.99 ms [getting-started-1.0-SNAPSHOT-runner:26] (cap): 1,181.71 ms [getting-started-1.0-SNAPSHOT-runner:26] setup: 2,518.34 ms 07:13:44,009 INFO [org.jbo.threads] JBoss Threads version 3.1.1.Final [getting-started-1.0-SNAPSHOT-runner:26] (typeflow): 16,768.13 ms [getting-started-1.0-SNAPSHOT-runner:26] (objects): 16,296.23 ms [getting-started-1.0-SNAPSHOT-runner:26] (features): 673.63 ms [getting-started-1.0-SNAPSHOT-runner:26] analysis: 35,442.21 ms [getting-started-1.0-SNAPSHOT-runner:26] (clinit): 712.25 ms [getting-started-1.0-SNAPSHOT-runner:26] universe: 2,032.64 ms [getting-started-1.0-SNAPSHOT-runner:26] (parse): 2,765.49 ms [getting-started-1.0-SNAPSHOT-runner:26] (inline): 5,469.40 ms [getting-started-1.0-SNAPSHOT-runner:26] (compile): 34,601.81 ms [getting-started-1.0-SNAPSHOT-runner:26] compile: 44,611.09 ms [getting-started-1.0-SNAPSHOT-runner:26] image: 3,175.00 ms [getting-started-1.0-SNAPSHOT-runner:26] write: 464.09 ms [getting-started-1.0-SNAPSHOT-runner:26] [total]: 92,243.99 ms [INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 102093ms [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:48 min (Wall Clock) [INFO] Finished at: 2020-07-08T09:14:58+02:00 [INFO] ------------------------------------------------------------------------ Additional info: Works for me with this podman version: $ rpm -q podman podman-1.8.2-2.fc32.x86_64
Is this rootless or rootful? What podman command are you using to execute this? rpm -q podman fuse-overlayfs crun conmon
(In reply to Daniel Walsh from comment #1) > Is this rootless or rootful? I've tried both. Fails the same way in rootless and rootful mode. > What podman command are you using to execute this? The reproducer prints this when you try that yourself. I see this: [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] podman run -v /home/sgehwolf/Documents/openjdk/quarkus/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z --env LANG=C --userns=keep-id --rm quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Duser.language=en -J-Dfile.encoding=UTF-8 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -H:+JNI -jar getting-started-1.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:-IncludeAllTimeZones -H:EnableURLProtocols=http --no-server -H:-UseServiceLoaderFeature -H:+StackTrace getting-started-1.0-SNAPSHOT-runner > rpm -q podman fuse-overlayfs crun conmon $ rpm -q podman fuse-overlayfs crun conmon podman-2.0.1-1.fc32.x86_64 fuse-overlayfs-1.1.1-1.fc32.x86_64 crun-0.14-2.fc32.x86_64 conmon-2.0.18-1.fc32.x86_64
Since you are getting permission denied, first I would look in /var/log/audit/audit.log to see if there are any AVC messages about containers or any seccomp messages? You could try to build --privileged to make sure it is an access issue. If that works, then you could try to build again using --cap-add all Success would tell us it is a seccomp or a missing capability. --security-opt seccomp=unconfined Success would tell us if this is a seccomp issue.
(In reply to Daniel Walsh from comment #3) > Since you are getting permission denied, first I would look in > /var/log/audit/audit.log to see if there are any AVC messages about > containers or any seccomp messages? Not that I could see. I've tried to build with selinux in permissive mode yesterday which failed too. > You could try to build --privileged to make sure it is an access issue. Running the podman command with '--privileged=true' fails in the same way. > If that works, then you could try to build again using > > --cap-add all Fails as well. > Success would tell us it is a seccomp or a missing capability. > > --security-opt seccomp=unconfined > > Success would tell us if this is a seccomp issue. Fails as well. Note that the image mounts a volume from the host and attempts to write to it (file '/project/getting-started-1.0-SNAPSHOT-runner').
Ok this looks like something is wrong with User Namespace $ podman run --usern=keep-id -v /home/sgehwolf/Documents/trash/quarkus-reproducer/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z getting-started-1.0-SNAPSHOT-runner ls -ld /project
ld -ld /home/sgehwolf/Documents/trash/quarkus-reproducer/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar
$ podman run -ti --entrypoint /bin/bash -v /home/sgehwolf/Documents/openjdk/quarkus/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z --env LANG=C --userns=keep-id --rm quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 [quarkus@2e5522a089a9 project]$ ls -ld /project drwxrwxr-x. 3 15263 15263 4096 Jul 8 13:50 /project $ ls -ld /home/sgehwolf/Documents/openjdk/quarkus/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar drwxrwxr-x. 3 sgehwolf sgehwolf 4096 Jul 8 15:50 /home/sgehwolf/Documents/openjdk/quarkus/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar [sgehwolf@t580-laptop getting-started]$ id uid=15263(sgehwolf) gid=15263(sgehwolf) groups=15263(sgehwolf),10(wheel),135(mock) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 With podman 1.8 I see: $ podman run -ti --entrypoint /bin/bash -v /home/sgehwolf/Documents/openjdk/quarkus/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z --env LANG=C --userns=keep-id --rm quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 bash-4.4$ ls -ld /project drwxrwxr-x. 3 15263 15263 4096 Jul 8 13:50 /project However, with podman 2.0 the userid of the image (quarkus user) is being used. With podman 1.8 my host user id is being used: With podman 1.8: $ podman run -ti --entrypoint /bin/bash -v /home/sgehwolf/Documents/openjdk/quarkus/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z --env LANG=C --userns=keep-id --rm quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 bash-4.4$ id uid=15263(15263) gid=15263 groups=15263 With podman 2.0: $ podman run -ti --entrypoint /bin/bash -v /home/sgehwolf/Documents/openjdk/quarkus/getting-started/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z --env LANG=C --userns=keep-id --rm quay.io/quarkus/ubi-quarkus-native-image:19.3.1-java11 [quarkus@f2c8da887a18 project]$ id uid=1001(quarkus) gid=1001(quarkus) groups=1001(quarkus) Which explains the permission denied. From 'man podman-run': --userns=auto|host|keep-id|container:id|ns:namespace Set the user namespace mode for the container. It defaults to the PODMAN_USERNS environment variable. An empty value means user namespaces are disabled. keep-id: creates a user namespace where the current rootless user's UID:GID are mapped to the same values in the container. This option is ignored for containers created by the root user.
This might be fixed in podman 2.0.2 I just tried a quick test of cat /tmp/Dockerfile FROM registry.fedoraproject.org/fedora:32 RUN useradd test USER test $ podman build -t test /tmp STEP 1: FROM registry.fedoraproject.org/fedora:32 STEP 2: RUN useradd test --> Using cache e88621936855fb828b0dc9d9568c40a41203e42572ff660b509239cb6c3239e3 STEP 3: USER test --> Using cache bad01f82b8381f1351f1629e9e6f6ad5bb735b96c5fbb9c7972d5bea72c2574c STEP 4: COMMIT test --> bad01f82b83 bad01f82b8381f1351f1629e9e6f6ad5bb735b96c5fbb9c7972d5bea72c2574c $ id -u 3267 $ podman run --userns keep-id test id uid=1000(test) gid=1000(test) groups=1000(test) $ podman run test id uid=1000(test) gid=1000(test) groups=1000(test)
https://bodhi.fedoraproject.org/updates/FEDORA-2020-19924b556e
I was able to download it with dnf -y update --enablerepo=updates-testing podman
(In reply to Daniel Walsh from comment #8) > This might be fixed in podman 2.0.2 Unfortunately no. Not for me. > I just tried a quick test of > > cat /tmp/Dockerfile > FROM registry.fedoraproject.org/fedora:32 > RUN useradd test > USER test > > $ podman build -t test /tmp > STEP 1: FROM registry.fedoraproject.org/fedora:32 > STEP 2: RUN useradd test > --> Using cache > e88621936855fb828b0dc9d9568c40a41203e42572ff660b509239cb6c3239e3 > STEP 3: USER test > --> Using cache > bad01f82b8381f1351f1629e9e6f6ad5bb735b96c5fbb9c7972d5bea72c2574c > STEP 4: COMMIT test > --> bad01f82b83 > bad01f82b8381f1351f1629e9e6f6ad5bb735b96c5fbb9c7972d5bea72c2574c > > $ id -u > 3267 > $ podman run --userns keep-id test id > uid=1000(test) gid=1000(test) groups=1000(test) Isn't this supposed to keep UID 3267? Not use UID 1000 (the test user in the container image).
$ rpm -q podman podman-2.0.2-1.fc32.x86_64 $ id -u 15263 $ podman run -ti --rm --userns=keep-id -v $(pwd)/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z rhbz1854738 [test@5ea41ceb1387 /]$ id -u 1000 [test@5ea41ceb1387 /]$ touch /project/test.txt touch: cannot touch '/project/test.txt': Permission denied With podman 1.8 I have this (podman-1.8.2-2.fc32.x86_64): $ id -u 15263 [sgehwolf@t580-laptop getting-started]$ podman run -ti --rm --userns=keep-id -v $(pwd)/target/getting-started-1.0-SNAPSHOT-native-image-source-jar:/project:z rhbz1854738 bash-5.0$ id -u 15263 bash-5.0$ touch /project/test.txt I've image rhbz1854738 created this way: $ cat Dockerfile FROM fedora:32 RUN useradd test USER test VOLUME /project $ podman build -t rhbz1854738 . STEP 1: FROM fedora:32 Getting image source signatures Copying blob 1657ffead824 done Copying config eb7134a03c done Writing manifest to image destination Storing signatures STEP 2: RUN useradd test --> d9cf0f3531c STEP 3: USER test --> 48acb3c7d28 STEP 4: VOLUME /project STEP 5: COMMIT rhbz1854738 --> 1f3ba4f44aa
No the image will still run as the user of the image. If you want to run as your user you need to specify it. All keep-uid does is map your UID to the same UID inside of the container. IE My uid is 3267 If I run --keep-uid, then there will be a 3267 mapped to 3267 inside of the container, but if the container is set to run as UID 1000 then it will run as UID 1000, no matter what. podman 1.8 looks like it had a bug and did not work correctly.
OK so it boils down to getting permissions correctly set up before writing to the volume mount in the quarkus native builder image. This issue was somewhat helpful for getting something working: https://github.com/containers/podman/issues/3990 Anyway, I've filed this quarkus issue: https://github.com/quarkusio/quarkus/issues/10637 Feel free to close.
Thanks, Dan, for helping us tracking it down.
For the record it is now fixed by https://github.com/quarkusio/quarkus/pull/11100