Bug 1411084 - deploy 3rd party artifacts are broken
Summary: deploy 3rd party artifacts are broken
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z4
: 10.0 (Newton)
Assignee: Tim Rozet
QA Contact: Gurenko Alex
URL:
Whiteboard:
: 1411085 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-08 10:07 UTC by Alon Dotan
Modified: 2017-09-06 17:09 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-09-06 17:09:30 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1649303 0 None None None 2017-01-10 15:38:45 UTC
OpenStack gerrit 409822 0 None MERGED Fix bug when using multiple DeployArtifactURLs 2020-11-25 20:37:52 UTC
Red Hat Product Errata RHBA-2017:2654 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 director Bug Fix Advisory 2017-09-06 20:55:36 UTC

Description Alon Dotan 2017-01-08 10:07:07 UTC
Description of problem:
deploy 3rd party rpms are broken,
if try to deploy more then one additional artifact the deployment failed,

the bug is in puppet/deploy-artifacts.sh

code snip:
if [ -n "$artifact_urls" ]; then
  for URL in $(echo $artifact_urls | sed -e "s| |\n|g" | sort -u); do
    curl --globoff -o $TMP_DATA/file_data "$artifact_urls"


the bug is in the curl command,
the command using $artifact_urls instead $URL, so if there is only one artifact url artifact_urls contains single url, but if more then curl gets multi url line as argument  

Version-Release number of selected component (if applicable):


How reproducible:
deploy overcloud with -e TEMPLATE_DIR/puppet/deploy-artifacts.yaml

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Tim Rozet 2017-01-10 15:38:07 UTC
This was fixed by https://review.openstack.org/#/c/409822/

Comment 2 Emilien Macchi 2017-03-01 21:12:38 UTC
Closing it. This script is not supported by OSP Director, therefore it was fixed upstream so users can pull a new version of the script.

Comment 3 Emilien Macchi 2017-03-01 21:14:20 UTC
*** Bug 1411085 has been marked as a duplicate of this bug. ***

Comment 4 Alon Dotan 2017-03-02 09:48:17 UTC
(In reply to Emilien Macchi from comment #2)
> Closing it. This script is not supported by OSP Director, therefore it was
> fixed upstream so users can pull a new version of the script.

this script is packed in the rpms that redhat installed for the director,
we didnt imported it from the community our self's,
there is any other scripts that redhat supply and not supporting in the OSP director?

Comment 5 Alex Schultz 2017-03-24 19:47:53 UTC
This seems to have been captured as part of BZ#1430726

Comment 9 Gurenko Alex 2017-06-26 20:10:56 UTC
Can't make it to work. I'm using build 2017-06-19.1 and openstack-tripleo-heat-templates-5.2.0-20.

 What is the correct usage of this feature? 

I'm passing extra-config.yaml with following content:

parameter_defaults:
  DeployArtifactURLs:
    - https://dl.fedoraproject.org/pub/epel/7/x86_64/h/htop-2.0.2-1.el7.x86_64.rpm

but package is not installed on the overcloud nodes after the deployment. No errors are found in logs and deployment completes successfully.

Comment 10 Gurenko Alex 2017-06-27 12:48:14 UTC
So, I'm been looking into this and found the problem. Right now deploy-artifacts.yaml that contains this code:

resources:
  DeployArtifacts:
    type: OS::Heat::SoftwareConfig
    properties:
      group: script
      inputs:
        - name: artifact_urls
          default: {list_join: [' ', {get_param: DeployArtifactURLs}]}
      config: {get_file: ./deploy-artifacts.sh}

is doing export of the artifac_urls that supposed to be used by ./deploy-artifacts.sh in following section:

if [ -n "$artifact_urls" ]; then
...
else
  echo "No artifact_urls was set. Skipping..."
fi

and this code while run manually goes to echo "No artifact_urls was set. Skipping..." all the time.

So with a little playing around (http://pastebin.test.redhat.com/497947) and looking into man bash I've found this: 

(from man bash) Array variables may not (yet) be exported.

so artifact_urls needs to be passed to the script in some other way.

Comment 12 Alex Schultz 2017-07-03 19:41:11 UTC
FYI, using this to install rpms doesn't work (try tar.gz instead).  RPMs installs were fixed in master by https://review.openstack.org/#/c/472847/.  We actually use this under the covers with the upload-puppet-modules (via upload-swift-artifacts) helper script. http://hardysteven.blogspot.com/2016/08/tripleo-deploy-artifacts-and-puppet.html


parameters_defaults:
  DeployArtifactURLs:
    - http://url1/file1.tar.gz
    - http://url2/file2.tar.gz

Comment 13 Gurenko Alex 2017-08-15 12:15:54 UTC
(In reply to Alex Schultz from comment #12)
> FYI, using this to install rpms doesn't work (try tar.gz instead).  RPMs
> installs were fixed in master by https://review.openstack.org/#/c/472847/. 
> We actually use this under the covers with the upload-puppet-modules (via
> upload-swift-artifacts) helper script.
> http://hardysteven.blogspot.com/2016/08/tripleo-deploy-artifacts-and-puppet.
> html
> 
> 
> parameters_defaults:
>   DeployArtifactURLs:
>     - http://url1/file1.tar.gz
>     - http://url2/file2.tar.gz

So, I've been trying today to pass following:

    DeployArtifactURLs:
      - 'https://nmap.org/dist/nmap-7.60.tgz'

 and I was expecting to see this file, well anywhere, but no traces whatsoever, can my comment above still be valid?

Comment 14 Alex Schultz 2017-08-15 15:12:07 UTC
This bug was for multiple files. But what you would see with that file is that it is extracted to /. So there should be a /nmap-7.60 directory on the node with the contents of the tgz extracted into it.  The actual .tgz gets deleted during the installation.

Comment 16 Gurenko Alex 2017-08-15 15:40:42 UTC
(In reply to Alex Schultz from comment #14)
> This bug was for multiple files. But what you would see with that file is
> that it is extracted to /. So there should be a /nmap-7.60 directory on the
> node with the contents of the tgz extracted into it.  The actual .tgz gets
> deleted during the installation.

Yeah, I've re-trying from scratch now, I've tried it manually today and it worked, but I've been looking into wrong location for results, it's just now I saw that it's going to /. Is it a good idea to unpack to /?
 Anyway, re-trying with multiple archives now and Steve pushed patch fixing rpm installation as well.

Comment 17 Alex Schultz 2017-08-15 18:58:27 UTC
That's how it's always worked because tars are relative.  That part is unchanged in this bug, it's just being able to provide more than one that was broken

Comment 18 Gurenko Alex 2017-08-16 05:48:45 UTC
Yes, I've successfully retried it. Both archives ended up at /.

Comment 21 errata-xmlrpc 2017-09-06 17:09:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2654


Note You need to log in before you can comment on or make changes to this bug.