Bug 1706128 - [OSP15] openstack tripleo container image prepare command failed with using tripleo-modify-image - "tripleo_common.image.exception.ImageNotFoundException: Not found image: https://registry.access.redhat.com/v2/rhosp15/openstack-nova-compute/manifest/15
Summary: [OSP15] openstack tripleo container image prepare command failed with using ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-tripleoclient
Version: 15.0 (Stein)
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: beta
: 15.0 (Stein)
Assignee: Steve Baker
QA Contact: Artem Hrechanychenko
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-03 15:27 UTC by Artem Hrechanychenko
Modified: 2019-11-11 20:31 UTC (History)
9 users (show)

Fixed In Version: python-tripleoclient-11.4.1-0.20190611084233.f36d899.el8ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-21 11:21:45 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1831152 0 None None None 2019-05-31 01:34:51 UTC
OpenStack gerrit 663010 0 'None' MERGED Drop the port from DockerInsecureRegistryAddress 2021-01-18 09:54:59 UTC
Red Hat Product Errata RHEA-2019:2811 0 None None None 2019-09-21 11:21:56 UTC

Description Artem Hrechanychenko 2019-05-03 15:27:41 UTC
Description of problem:
[stack@undercloud-0 ~]$ sudo openstack tripleo container image prepare -e /home/stack/containers-prepare-parameter-copy.yaml | sudo tee /home/stack/prepare_output.yaml
Completed upload for image registry.access.redhat.com/rhceph/rhceph-3-rhel7:latest
...
...
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-zaqar-wsgi:20190423.1
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 30, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/container_image.py", line 681, in take_action
    cleanup=parsed_args.cleanup)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py", line 207, in container_images_prepare_multi
    uploader.upload()
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 201, in upload
    uploader.run_tasks()
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1629, in run_tasks
    result = uploader.upload_image(first_task)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 938, in upload_image
    session=source_session
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 319, in iter
    return fut.result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 361, in call
    result = fn(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1049, in _fetch_manifest
    raise ImageNotFoundException('Not found image: %s' % url)
tripleo_common.image.exception.ImageNotFoundException: Not found image: https://registry.access.redhat.com/v2/rhosp15/openstack-nova-compute/manifests/15.0
Not found image: https://registry.access.redhat.com/v2/rhosp15/openstack-nova-compute/manifests/15.0


[stack@undercloud-0 ~]$ cat containers-prepare-parameter-copy.yaml 
# Generated with the following on 2019-05-02T15:46:34.815555
#
#   openstack tripleo container image prepare default --output-env-file /home/stack/containers-prepare-parameter.yaml --local-push-destination
#

parameter_defaults:
  ContainerImagePrepare:
  - push_destination: true
    set:
      name_prefix: openstack-
      name_suffix: ''
      namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
      neutron_driver: ovn
      openshift_asb_namespace: registry.access.redhat.com/openshift3
      openshift_asb_tag: v3.11
      openshift_cluster_monitoring_image: ose-cluster-monitoring-operator
      openshift_cluster_monitoring_namespace: registry.access.redhat.com/openshift3
      openshift_cluster_monitoring_tag: v3.11
      openshift_cockpit_image: registry-console
      openshift_cockpit_namespace: registry.access.redhat.com/openshift3
      openshift_cockpit_tag: v3.11
      openshift_configmap_reload_image: ose-configmap-reloader
      openshift_configmap_reload_namespace: registry.access.redhat.com/openshift3
      openshift_configmap_reload_tag: v3.11
      openshift_etcd_image: etcd
      openshift_etcd_namespace: registry.access.redhat.com/rhel7
      openshift_etcd_tag: latest
      openshift_gluster_block_image: rhgs-gluster-block-prov-rhel7
      openshift_gluster_image: rhgs-server-rhel7
      openshift_gluster_namespace: registry.access.redhat.com/rhgs3
      openshift_gluster_tag: v3.11
      openshift_grafana_namespace: registry.access.redhat.com/openshift3
      openshift_grafana_tag: v3.11
      openshift_heketi_image: rhgs-volmanager-rhel7
      openshift_heketi_namespace: registry.access.redhat.com/rhgs3
      openshift_heketi_tag: v3.11
      openshift_kube_rbac_proxy_image: ose-kube-rbac-proxy
      openshift_kube_rbac_proxy_namespace: registry.access.redhat.com/openshift3
      openshift_kube_rbac_proxy_tag: v3.11
      openshift_kube_state_metrics_image: ose-kube-state-metrics
      openshift_kube_state_metrics_namespace: registry.access.redhat.com/openshift3
      openshift_kube_state_metrics_tag: v3.11
      openshift_namespace: registry.access.redhat.com/openshift3
      openshift_oauth_proxy_tag: v3.11
      openshift_prefix: ose
      openshift_prometheus_alertmanager_tag: v3.11
      openshift_prometheus_config_reload_image: ose-prometheus-config-reloader
      openshift_prometheus_config_reload_namespace: registry.access.redhat.com/openshift3
      openshift_prometheus_config_reload_tag: v3.11
      openshift_prometheus_node_exporter_tag: v3.11
      openshift_prometheus_operator_image: ose-prometheus-operator
      openshift_prometheus_operator_namespace: registry.access.redhat.com/openshift3
      openshift_prometheus_operator_tag: v3.11
      openshift_prometheus_tag: v3.11
      openshift_tag: v3.11
      tag: 20190423.1
# BEGIN ANSIBLE MANAGED BLOCK
  - includes:
    - nova-compute
    namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
    modify_role: tripleo-modify-image
    modify_append_tag: "-hotfix"
    modify_vars:
     tasks_from: rpm_install.yml
     rpms_path: /home/stack/nova-compute-modify-dir
# END ANSIBLE MANAGED BLOCK


but running the same command without modify_role: tripleo-modify-image works

[stack@undercloud-0 ~]$ sudo openstack tripleo container image prepare -e /home/stack/containers-prepare-parameter.yaml | sudo tee /home/stack/prepare_output.yaml

http://pastebin.test.redhat.com/760451

Version-Release number of selected component (if applicable):
RHOS_TRUNK-15.0-RHEL-8-20190423.n.1
ansible-role-tripleo-modify-image-1.0.1-0.20190422122515.f1dfdc6.el8ost.noarch

How reproducible:
always

Steps to Reproduce:
1.Deploy OSP15 Overcloud with 1controller+1compute
2.Prepare rpm for injecting into new image
3.Create copy of containers-prepare-parameter and add next lines:
- includes:
    - nova-compute
    modify_role: tripleo-modify-image
    modify_append_tag: "-hotfix"
    modify_vars:
     tasks_from: rpm_install.yml
     rpms_path: /home/stack/nova-compute-modify-dir

4. run  sudo openstack tripleo container image prepare -e /home/stack/containers-prepare-parameter-copy.yaml | sudo tee /home/stack/prepare_output.yaml

Actual results:
Not found image: https://registry.access.redhat.com/v2/rhosp15/openstack-nova-compute/manifests/15.0

Expected results:
passed and provide list of images for containers

Additional info:

Comment 3 Artem Hrechanychenko 2019-05-17 06:21:52 UTC
Hi,
any updates about that bz?
that bz block feature testing of https://bugzilla.redhat.com/show_bug.cgi?id=1626139

Comment 5 Steve Baker 2019-05-17 21:04:30 UTC
I see an error in the ansible generated config

# BEGIN ANSIBLE MANAGED BLOCK
  - includes:
    - nova-compute
    namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
    modify_role: tripleo-modify-image
    modify_append_tag: "-hotfix"
    modify_vars:
     tasks_from: rpm_install.yml
     rpms_path: /home/stack/nova-compute-modify-dir
# END ANSIBLE MANAGED BLOCK

should be

# BEGIN ANSIBLE MANAGED BLOCK
  - includes:
    - nova-compute
    set:
      namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
      modify_role: tripleo-modify-image
      modify_append_tag: "-hotfix"
      modify_vars:
       tasks_from: rpm_install.yml
       rpms_path: /home/stack/nova-compute-modify-dir
# END ANSIBLE MANAGED BLOCK

Comment 6 Artem Hrechanychenko 2019-05-18 17:50:21 UTC
(In reply to Steve Baker from comment #5)
> I see an error in the ansible generated config
> 
> # BEGIN ANSIBLE MANAGED BLOCK
>   - includes:
>     - nova-compute
>     namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
>     modify_role: tripleo-modify-image
>     modify_append_tag: "-hotfix"
>     modify_vars:
>      tasks_from: rpm_install.yml
>      rpms_path: /home/stack/nova-compute-modify-dir
> # END ANSIBLE MANAGED BLOCK
> 
> should be
> 
> # BEGIN ANSIBLE MANAGED BLOCK
>   - includes:
>     - nova-compute
>     set:
>       namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
>       modify_role: tripleo-modify-image
>       modify_append_tag: "-hotfix"
>       modify_vars:
>        tasks_from: rpm_install.yml
>        rpms_path: /home/stack/nova-compute-modify-dir
> # END ANSIBLE MANAGED BLOCK




Hi,
in case of 
# BEGIN ANSIBLE MANAGED BLOCK
  - includes:
    - nova-compute
    set:
      modify_role: tripleo-modify-image
      modify_append_tag: "-hotfix"
      modify_vars:
        tasks_from: rpm_install.yml
        rpms_path: /home/stack/nova-compute-modify-dir


I didn't get errors but also didn't get updated data

also I didn't use "set:" in OSP14 and successfully worked with tripleo-modify-image
(undercloud) [stack@undercloud-0 ~]$ cat prepare_output.yaml |grep compute
  DockerCeilometerComputeImage: 192.168.24.1:8787/rhosp15/openstack-ceilometer-compute:20190509.1
  DockerNovaComputeImage: registry.access.redhat.com/rhosp15/openstack-nova-compute:15.0
  DockerNovaComputeIronicImage: registry.access.redhat.com/rhosp15/openstack-nova-compute-ironic:15.0
  DockerNovaLibvirtConfigImage: registry.access.redhat.com/rhosp15/openstack-nova-compute:15.0

Comment 8 Artem Hrechanychenko 2019-05-27 10:23:55 UTC
Change to Urgent and set blocker because that issue block a regression test and at the same time OSP15 feature testing

Comment 9 Emilien Macchi 2019-05-27 14:45:22 UTC
Did you miss "namespace" in the last comment?

Comment 10 Artem Hrechanychenko 2019-05-27 14:49:07 UTC
Nope, 
we don't set namespace in OSP14,15 automation
I just tried to play.
generally we use next template

- name: include tripleo-modify-image using rpm method in containers-prepare-parameter-copy.yaml
  blockinfile:
    path: "~/containers-prepare-parameter-copy.yaml"
    block: |1+
       - includes:
         - {{ test.tripleo.modify.container }}
         modify_role: tripleo-modify-image
         modify_append_tag: "-hotfix"
         modify_vars:
          tasks_from: rpm_install.yml
          rpms_path: /home/stack/{{ test.tripleo.modify.container }}-modify-dir
  when: test.tripleo.modify.method == "rpm"

- name: include tripleo-modify-image using yum-update method in containers-prepare-parameter-copy.yaml
  blockinfile:
    path: "~/containers-prepare-parameter-copy.yaml"
    block: |1+
       - includes:
         - {{ test.tripleo.modify.container }}
         modify_role: tripleo-modify-image
         modify_append_tag: "-hotfix"
         modify_vars:
          tasks_from: yum_update.yml
          compare_host_packages: true
          yum_repos_dir_path: /etc/yum.repos.d
  when: test.tripleo.modify.method == "yum-update"

- name: include tripleo-modify-image using modify-image method in containers-prepare-parameter-copy.yaml
  blockinfile:
    path: "~/containers-prepare-parameter-copy.yaml"
    block: |1+
       - includes:
         - {{ test.tripleo.modify.container }}
         modify_role: tripleo-modify-image
         modify_append_tag: "-hotfix"
         modify_vars:
          tasks_from: modify_image.yml
          modify_dir_path: /home/stack/{{ test.tripleo.modify.container }}-modify-dir
  when: test.tripleo.modify.method == "modify-image"

Comment 11 Artem Hrechanychenko 2019-05-27 14:59:09 UTC
Sorry, miscommunication from me:

We use default container-repare-paramater.yaml where we inject modify-image part

example from job itself:
https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/df/view/deployment/job/DFG-df-deployment-15-virthost-1cont_1comp-lvm-ipv4-geneve-tripleo-modify-image-yum-RHELOSP-41500/3/artifact/undercloud-0.tar.gz

# Generated with the following on 2019-05-27T06:53:02.228713
#
#   openstack tripleo container image prepare default --output-env-file /home/stack/containers-prepare-parameter.yaml --local-push-destination
#

parameter_defaults:
  ContainerImagePrepare:
  - push_destination: true
    set:
      ceph_image: rhceph-4.0-rhel8
      ceph_namespace: docker-registry.upshift.redhat.com/ceph
      ceph_tag: latest
      name_prefix: openstack-
      name_suffix: ''
      namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
      neutron_driver: ovn
      tag: 20190523.1
# BEGIN ANSIBLE MANAGED BLOCK
  - includes:
    - nova-compute
    modify_role: tripleo-modify-image
    modify_append_tag: "-hotfix"
    modify_vars:
     tasks_from: yum_update.yml
     compare_host_packages: true
     yum_repos_dir_path: /etc/yum.repos.d
# END ANSIBLE MANAGED BLOCK

Comment 13 Steve Baker 2019-05-27 21:35:18 UTC
Oh apologies, the modify_* attributes should not be inside the set: block, but namespace, name_prefix, name_suffix, tag etc should be inside a set: if you want to override the default. You likely want to end up with something like this:

parameter_defaults:
  ContainerImagePrepare:
  - push_destination: true
    set:
      ceph_image: rhceph-4.0-rhel8
      ceph_namespace: docker-registry.upshift.redhat.com/ceph
      ceph_tag: latest
      name_prefix: openstack-
      name_suffix: ''
      namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
      neutron_driver: ovn
      tag: 20190523.1
# BEGIN ANSIBLE MANAGED BLOCK
  - push_destination: true
    set:
      name_prefix: openstack-
      name_suffix: ''
      namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
      tag: 20190523.1
    includes:
    - nova-compute
    modify_role: tripleo-modify-image
    modify_append_tag: "-hotfix"
    modify_vars:
     tasks_from: yum_update.yml
     compare_host_packages: true
     yum_repos_dir_path: /etc/yum.repos.d
# END ANSIBLE MANAGED BLOCK

So the ansible managed block adds an entry which is identical to the one above it, except for:
* excludes ceph_*, neutron_driver because they are not relevant to nova-compute
* includes nova-compute so this is the only image modified
* adds modify_* to do a yum update on image
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-compute:20190523.1
  generating image:
  rhosp15/openstack-nova-compute:20190523.1-hotfix
  and pushing that to the local repository


Upstream docs are here:
https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/container_image_prepare.html

Also there is an argument for writing out the env file so you shouldn't need a sudo tee, you can just do this

sudo openstack tripleo container image prepare -e /home/stack/containers-prepare-parameter-copy.yaml --output-env-file /home/stack/prepare_output.yaml

Comment 14 Artem Hrechanychenko 2019-05-28 12:40:15 UTC
Still doesn't work for me
[stack@undercloud-0 ~]$ cat containers-prepare-parameter-copy.yaml 
# Generated with the following on 2019-05-28T06:41:14.903170
#
#   openstack tripleo container image prepare default --output-env-file /home/stack/containers-prepare-parameter.yaml --local-push-destination
#

parameter_defaults:
  ContainerImagePrepare:
  - push_destination: true
    set:
      ceph_image: rhceph-3-rhel7
      ceph_namespace: registry.access.redhat.com/rhceph
      ceph_tag: latest
      name_prefix: openstack-
      name_suffix: ''
      namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
      neutron_driver: ovn
      tag: 20190523.1

# BEGIN ANSIBLE MANAGED BLOCK
  - push_destination: true
    set:
      name_prefix: openstack-
      name_suffix: ''
      namespace: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15
      tag: 20190523.1
    includes:
    - nova-compute
    modify_role: tripleo-modify-image
    modify_append_tag: "-hotfix"
    modify_vars:
     tasks_from: yum_update.yml
     compare_host_packages: true
     yum_repos_dir_path: /etc/yum.repos.d
# END ANSIBLE MANAGED BLOCK

[stack@undercloud-0 ~]$ sudo openstack tripleo container image prepare -e /home/stack/containers-prepare-parameter-copy.yaml --output-env-file /home/stack/prepare_output.yaml
Completed upload for image registry.access.redhat.com/rhceph/rhceph-3-rhel7:latest
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-aodh-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-aodh-evaluator:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-aodh-listener:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-aodh-notifier:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-barbican-keystone-listener:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-barbican-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ceilometer-central:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-barbican-worker:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ceilometer-compute:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ceilometer-notification:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-cinder-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-cinder-backup:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-cinder-volume:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-cinder-scheduler:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-cron:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-collectd:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-etcd:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ec2-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-fluentd:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-glance-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-gnocchi-metricd:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-gnocchi-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-gnocchi-statsd:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-haproxy:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-heat-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-heat-api-cfn:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-heat-engine:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-horizon:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-conductor:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-inspector:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-pxe:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-iscsid:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-neutron-agent:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-keepalived:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-keystone:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-manila-scheduler:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-manila-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mariadb:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-manila-share:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mistral-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-memcached:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mistral-engine:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mistral-executor:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mistral-event-engine:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-dhcp-agent:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-l3-agent:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-metadata-agent:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-server-ovn:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-openvswitch-agent:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-metadata-agent-ovn:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-compute:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-compute-ironic:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-conductor:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-consoleauth:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-libvirt:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-novncproxy:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-placement-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-scheduler:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-octavia-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-octavia-health-manager:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-octavia-worker:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-octavia-housekeeping:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ovn-controller:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ovn-nb-db-server:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ovn-sb-db-server:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ovn-northd:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-qdrouterd:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-panko-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-rabbitmq:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-redis:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-sahara-api:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-sahara-engine:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-sensu-client:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-skydive-agent:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-skydive-analyzer:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-swift-account:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-swift-object:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-swift-container:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-swift-proxy-server:20190523.1
Completed upload for image brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-zaqar-wsgi:20190523.1
error checking if pulling from registry for "docker://192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1" is blocked: unable to parse the registries configuration (/etc/containers/registries.conf): invalid URL '192.168.24.1:8787': parse 192.168.24.1:8787: first path segment in URL cannot contain colon
Error pulling image:
buildah pull 192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1
None
error checking if pulling from registry for "docker://192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1" is blocked: unable to parse the registries configuration (/etc/containers/registries.conf): invalid URL '192.168.24.1:8787': parse 192.168.24.1:8787: first path segment in URL cannot contain colon
Error pulling image:
buildah pull 192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1
None
error checking if pulling from registry for "docker://192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1" is blocked: unable to parse the registries configuration (/etc/containers/registries.conf): invalid URL '192.168.24.1:8787': parse 192.168.24.1:8787: first path segment in URL cannot contain colon
Error pulling image:
buildah pull 192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1
None
error checking if pulling from registry for "docker://192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1" is blocked: unable to parse the registries configuration (/etc/containers/registries.conf): invalid URL '192.168.24.1:8787': parse 192.168.24.1:8787: first path segment in URL cannot contain colon
Error pulling image:
buildah pull 192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1
None
error checking if pulling from registry for "docker://192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1" is blocked: unable to parse the registries configuration (/etc/containers/registries.conf): invalid URL '192.168.24.1:8787': parse 192.168.24.1:8787: first path segment in URL cannot contain colon
Error pulling image:
buildah pull 192.168.24.1:8787/rhosp15/openstack-nova-compute:20190523.1
None
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/container_image.py", line 656, in take_action
    cleanup=parsed_args.cleanup)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py", line 213, in container_images_prepare_multi
    uploader.upload()
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 201, in upload
    uploader.run_tasks()
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1638, in run_tasks
    result = uploader.upload_image(first_task)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 974, in upload_image
    self._copy_registry_to_local(t.target_image_source_tag_url)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 331, in iter
    raise retry_exc.reraise()
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 167, in reraise
    raise self.last_attempt.result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 361, in call
    result = fn(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1313, in _copy_registry_to_local
    raise ImageUploaderException('Pulling image failed')
tripleo_common.image.exception.ImageUploaderException: Pulling image failed
Pulling image failed

Comment 15 Steve Baker 2019-05-28 22:32:23 UTC
OK, this now looks correct but you're now hitting a problem Wes has also seen where /etc/containers/registries.conf is incorrectly populated. I think the registries.conf issue needs a separate bz.

Setting needinfo for Wes for a bz number.

Comment 16 wes hayutin 2019-05-31 01:37:53 UTC
sorry too many meetings.  Thanks for opening https://bugs.launchpad.net/tripleo/+bug/1831152 Steve :)

Comment 17 Artem Hrechanychenko 2019-06-18 13:28:35 UTC
VERIFIED
python3-tripleoclient-11.4.1-0.20190611084233.f36d899.el8ost.noarch

Comment 21 errata-xmlrpc 2019-09-21 11:21:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811


Note You need to log in before you can comment on or make changes to this bug.