Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1896436

Summary: Sahara Manifest not found in container image tests
Product: Red Hat OpenStack Reporter: David Rosenfeld <drosenfe>
Component: openstack-tripleo-commonAssignee: Adriano Petrich <apetrich>
Status: CLOSED DUPLICATE QA Contact: David Rosenfeld <drosenfe>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 16.2 (Train)CC: aschultz, mburns, michele, slinaber
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-10 23:38:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David Rosenfeld 2020-11-10 14:37:40 UTC 
Description of problem: In OSP 16.2 all three container image modify tests are failing. The error in logs is:

requests.exceptions.HTTPError: 404 Client Error: NOT FOUND for url: https://rhos-qe-mirror-rdu2.usersys.redhat.com:5002/v2/rh-osbs/rhosp16-openstack-sahara-api/manifests/16.2_20201103.2


Version-Release number of selected component (if applicable): RHOS-16.2-RHEL-8-20201105.n.0


How reproducible: Every time one of the container image modification Jenkins jobs is executed ex: DFG-df-deployment-16.2-virthost-1cont_1comp_3ceph-ceph-ipv4-geneve-tripleo-modify-image-yum-RHELOSP-41500


Steps to Reproduce:
1. Execute Jenkins job: DFG-df-deployment-16.2-virthost-1cont_1comp_3ceph-ceph-ipv4-geneve-tripleo-modify-image-yum-RHELOSP-41500
2.
3.

Actual results: Container image modification fails with error above.


Expected results: Container image modification is successful.


Additional info:
Comment 1 David Rosenfeld 2020-11-10 14:38:21 UTC 
Logs to a failing test(see: ir-tripleo-config-changes.log): 

https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/DFG/view/df/view/deployment/job/DFG-df-deployment-16.2-virthost-1cont_1comp_3ceph-ceph-ipv4-geneve-tripleo-modify-image-yum-RHELOSP-41500/5/

Seen in logs:

[rh-osbs/rhosp16-openstack-barbican-api] Writing chunk 1 for sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e
[rh-osbs/rhosp16-openstack-barbican-api] Written 29080262 bytes for sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e
[/rh-osbs/rhosp16-openstack-barbican-api] Done fetching layer sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e from registry
[rh-osbs/rhosp16-openstack-barbican-api] Layer written successfully /var/lib/image-serve/v2/rh-osbs/rhosp16-openstack-barbican-api/blobs/sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e.gz
[rh-osbs/rhosp16-openstack-barbican-api] Provided digest: sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e, Calculated digest: sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e
[rh-osbs/rhosp16-openstack-barbican-api] Done exporting image layer sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e
Tracking processed layer sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e for local scope
Released lock on layer sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e
[/rh-osbs/rhosp16-openstack-barbican-api] Upload complete for layer sha256:bf1cc38a5068347cbc563186867871ffd899fa5720d8dd8976bdf16546b93b7e
[/rh-osbs/rhosp16-openstack-barbican-api] Waiting for next job: 6 of 6 complete
[/rh-osbs/rhosp16-openstack-barbican-api] Completed 6 jobs
[/rh-osbs/rhosp16-openstack-barbican-api] Uploading config with digest: sha256:3f0bfd32be77a9c768fd48da5bcb377500b044de522b7614982215dc6c8f22b3
https://rhos-qe-mirror-rdu2.usersys.redhat.com:5002 "GET /v2/rh-osbs/rhosp16-openstack-barbican-api/blobs/sha256:3f0bfd32be77a9c768fd48da5bcb377500b044de522b7614982215dc6c8f22b3 HTTP/1.1" 200 4012
Rebuilding /var/lib/image-serve/v2/_catalog
[rh-osbs/rhosp16-openstack-barbican-api] Rebuilding /var/lib/image-serve/v2/rh-osbs/rhosp16-openstack-barbican-api/tags
[/rh-osbs/rhosp16-openstack-barbican-api] Finished copying image
[rhos-qe-mirror-rdu2.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-barbican-api:16.2_20201103.2] Completed upload for image
Exception occured while running the command
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1791, in _fetch_manifest
    timeout=30
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 412, in get
    **kwargs)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 319, in iter
    return fut.result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 361, in call
    result = fn(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 391, in _action
    request=req)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 256, in check_status
    request.raise_for_status()
  File "/usr/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: NOT FOUND for url: https://rhos-qe-mirror-rdu2.usersys.redhat.com:5002/v2/rh-osbs/rhosp16-openstack-sahara-api/manifests/16.2_20201103.2

During handling of the above exception, another exception occurred:
Comment 3 Alex Schultz 2020-11-10 23:38:27 UTC 

*** This bug has been marked as a duplicate of bug 1886730 ***