Bug 2018949

Summary: podman multi stage build failing when using with --pull flag
Product: Red Hat Enterprise Linux 8 Reporter: Suhaas Bhat <subhat>
Component: podmanAssignee: Jindrich Novy <jnovy>
Status: CLOSED ERRATA QA Contact: Joy Pu <ypu>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 8.4CC: arajan, bbaude, cpippin, dwalsh, jligon, jnovy, lsm5, mheon, nalin, pthomas, tsweeney, umohnani, vrothber, ypu
Target Milestone: rcKeywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: podman-3.4.3-0.2.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-10 13:27:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Suhaas Bhat 2021-11-01 10:54:52 UTC
Description of problem:
podman multi stage build is failing and giving below error :

Error: error creating build container: pull policy is always but image has been referred to by ID 

Version-Release number of selected component (if applicable):
podman-3.2.3-0.11.module+el8.4.0+12050+ef972f71.x86_64

How reproducible:
Fully reproducible

Steps to Reproduce:

Dockerfile 
-------------------
FROM ubuntu:focal as deployer
FROM ubi8/ubi:latest as builder
FROM deployer
-------------------

Build Command : podman build -f Dockerfile . --force-rm --no-cache --pull 

Actual results:
Build fails

Expected results:
Successful Build

Additional info:

----------------------------------
func (r *Runtime) Pull(ctx context.Context, name string, pullPolicy config.PullPolicy, options *PullOptions) ([]*Image, error) {
        logrus.Debugf("Pulling image %s (policy: %s)", name, pullPolicy)

        if options == nil {
                options = &PullOptions{}
        }

        var possiblyUnqualifiedName string // used for short-name resolution
        ref, err := alltransports.ParseImageName(name)
        if err != nil {
                // If the image clearly refers to a local one, we can look it up directly.
                // In fact, we need to since they are not parseable
                    if strings.HasPrefix(name, "sha256:") || (len(name) == 64 && !strings.ContainsAny(name, "/.:@")) {
                        if pullPolicy == config.PullPolicyAlways {
                                return nil, errors.Errorf("pull policy is always but image has been referred to by ID (%s)", name)
                        }


// If we looked up the image by ID, we cannot really pull from anywhere.
        if localImage != nil && strings.HasPrefix(localImage.ID(), imageName) {
                switch pullPolicy {
                case config.PullPolicyAlways:
                        return nil, errors.Errorf("pull policy is always but image has been referred to by ID (%s)", imageName)
                default:
                        return []string{resolvedImageName}, nil

--------------------------------------

Comment 1 Tom Sweeney 2021-11-02 21:10:32 UTC
I'm going to keep this assigned to Valentin, but if you have time to take a look at this BZ this week Aditya, that would be good.

Comment 2 Valentin Rothberg 2021-11-08 09:29:14 UTC
This has been fixed in Podman 3.4.  Please let me know if we need backports or wait for 3.4 to hit RHEL.

Comment 3 Tom Sweeney 2021-11-08 15:47:19 UTC
@vrothber if you could add a comment with a pointer to a PR/commit for this, that would be great, thanks!

Comment 4 Valentin Rothberg 2021-11-09 09:09:50 UTC
`git bisect` revealed the following commit to fix the reproducer: https://github.com/containers/buildah/commit/14965cde13f9

Comment 5 Jindrich Novy 2021-11-09 09:16:50 UTC
Nalin, do you mind having a quick look at this one please?

Comment 6 Nalin Dahyabhai 2021-11-09 14:54:32 UTC
The commit Valentin points to in https://bugzilla.redhat.com/show_bug.cgi?id=2018949#c4 was in 1.22, but not 1.21, which is what podman 3.2 used.  podman 3.3 used buildah 1.23, so this should be fixed in podman 3.3 or later.

Comment 7 Nalin Dahyabhai 2021-11-09 14:55:22 UTC
Correction: podman 3.3 used buildah 1.22, but it should be fixed there regardless.

Comment 8 Jindrich Novy 2021-11-09 15:10:57 UTC
Thanks Nalin, we have podman-3.3.x in 8.5 which is about to get released now. Targeting 8.6 then to not to miss QE on this one.

Can we get qa ack please?

Comment 11 Joy Pu 2022-01-17 07:03:15 UTC
Can reproduce with podman-3.2.3-0.11.module+el8.4.0+12050+ef972f71.x86_64.
And test with podman-3.4.5-0.4.module+el8.6.0+13880+7c3afcf8.x86_64 the build process works well as expected. So set this to verified. Details:
# podman build -f Dockerfile . --force-rm --no-cache --pull 
[1/3] STEP 1/1: FROM quay.io/libpod/busybox:latest AS deployer
Trying to pull quay.io/libpod/busybox:latest...
Getting image source signatures
Copying blob 9758c28807f2 skipped: already exists  
Copying config f0b02e9d09 done  
Writing manifest to image destination
Storing signatures
--> f0b02e9d092
[2/3] STEP 1/1: FROM ubi8/ubi:latest AS builder
Trying to pull registry.access.redhat.com/ubi8/ubi:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob adffa6963146 skipped: already exists  
Copying blob 26f1167feaf7 skipped: already exists  
Copying config fca12da1dc done  
Writing manifest to image destination
Storing signatures
--> fca12da1dc3
[3/3] STEP 1/1: FROM f0b02e9d092d905d0d87a8455a1ae3e9bb47b4aa3dc125125ca5cd10d6441c9f
[3/3] COMMIT
--> f0b02e9d092
f0b02e9d092d905d0d87a8455a1ae3e9bb47b4aa3dc125125ca5cd10d6441c9f

Comment 15 errata-xmlrpc 2022-05-10 13:27:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1762