Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2088529 - ForemanCustomScript in Host provisioned on Azure CR fails with `command not found`
Summary: ForemanCustomScript in Host provisioned on Azure CR fails with `command not f...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Compute Resources - Azure
Version: 6.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: 6.13.0
Assignee: Chris Roberts
QA Contact: addubey
URL: https://projects.theforeman.org/issue...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-19 15:54 UTC by Jitendra Yejare
Modified: 2023-05-03 13:21 UTC (History)
6 users (show)

Fixed In Version: rubygem-foreman_azure_rm-2.2.7-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-03 13:21:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 35481 0 Normal Assigned ForemanCustomScript in Host provisioned on Azure CR fails with `command not found` 2022-09-06 18:59:28 UTC
Github theforeman foreman_azure_rm pull 158 0 None open Fixes #35481 - Remove hard coded sudo command in script extension 2022-09-06 18:59:28 UTC
Red Hat Issue Tracker SAT-11117 0 None None None 2022-09-08 20:05:46 UTC
Red Hat Product Errata RHSA-2023:2097 0 None None None 2023-05-03 13:21:34 UTC

Description Jitendra Yejare 2022-05-19 15:54:49 UTC
Created attachment 1881401 [details]
[Screenshots] Azure Portal VM

Description of problem:
When Satellite provisions host on Azure CR, a `ForemanCustomScript` Extension that  Satellite adds to the Azure host is throwing an error `Enable succeeded: [stout] [stderr] /bin/sh: azurerootpasswd: command not found`.

This failure probably delayed the VM read operation from Azure python client and failed all QE tests for Azure Provisioning.

Version-Release number of selected component (if applicable):
Satellite 6.11 | 6.10 | 6.9

How reproducible:
Always

Steps to Reproduce:
1. Create an Azure Compute Resource.
2. Create an Image in Azure CR with marketplace image `marketplace://RedHat:RHEL:7-RAW:latest` (BTW, the issue is reproducible with all the images that Satellite supports to provision the host from)
3. Using the image above provision an azure host, and make sure the azure host has the following compute attributes:

script_command = 'touch /var/tmp/text.txt',
script_uris = `https://raw.githubusercontent.com/SatelliteQE/robottelo/master/tests/foreman/data/uri.sh`

Actual results:
1. The VM is being provisioned to Azure but the ForemanCustomScript extention post VM deployment is failing with the error `Enable succeeded: [stout] [stderr] /bin/sh: azurerootpasswd: command not found. Note `azurerootpasswd` is the password set as a password in compute attributes.
2. The script command in compute attributes was executed successfully.
3. But, The custom script in `script_uris` in compute attributes is not executed and seems the failure is for that reason only.

Expected results:
The VM provisions successfully with the post-deployment script (both script command and script uris) executed successfully.

Additional info:
Attachments with screenshots from Azure portal showing failure from VMs `ForemanCustomScript` extention.

Comment 2 Jitendra Yejare 2022-05-24 13:43:19 UTC
@bbuckingham No, It's something working in earlier z-versions of 6.9 - 6.10 and especially seeing from earlier snaps of 6.11 and the same issue in 6.9 and 6.10 at the same time.

Comment 3 Jitendra Yejare 2022-05-24 14:05:51 UTC
Hey Brad,

Just run the 6.11 automation and this issue is not reproducible now :)  TBH, I have seen it passing earlier as well once in 6.11 snaps and again starts failing.

Not sure what's the case in 6.10 and 6.9, but I will post to you on that!

Comment 4 Ron Lavi 2022-05-26 11:44:46 UTC
Hey @jyejare@redhat.com, any news with the other versions?
I am suggesting to remove it from 6.11.z as it wasn't reproduced anymore.

Comment 5 Jitendra Yejare 2022-05-26 14:14:57 UTC
@rlavi It reproduced again yesterday on snap 22 of Sat 6.11. The passing is very infrequent/intermittent but the failure is very frequent in the last 22 snaps!

So I will still keep it open for 6.11 and rest of the versions!

Comment 6 Chris Roberts 2022-05-26 15:36:16 UTC
Are you setting the password in the virtual machine tab or when you create an image in the compute resource?

Comment 7 Jitendra Yejare 2022-05-27 08:24:16 UTC
@chrobert I am setting the password from Virtual Machine, but not in the image in the Azure compute resource!

Comment 12 Chris Roberts 2022-08-31 19:11:25 UTC
Made it past the /bin/sh: azurerootpasswd: command not found`

It looks like 5 years ago we hard coded this line in the plugin:

https://github.com/theforeman/foreman_azure_rm/blob/master/app/models/foreman_azure_rm/azure_rm.rb#L297

Looking at the if/else block, even if script was blank we would make it true which is a another bug in itself(my pr will fix this) because each vm would have the linux script extension added to it.

Looking at how Azure does things now they do not use that file anymore instead in /etc/sudoers.d we have this

[root@kari-elofson ~]# cat /etc/sudoers.d/90-cloud-init-users 
# Created by cloud-init v. 19.4 on Tue, 03 Aug 2021 14:30:54 +0000

# User rules for azureuser
azureuser ALL=(ALL) NOPASSWD:ALL

# User rules for azureuser
azureuser ALL=(ALL) NOPASSWD:ALL

I tested removing that line of code and we do not create the linux script extension on vms now and I do not see the error posted in the BZ when looking at /var/lib/waagent//custom-script/download/0/stderr/out

The issue I am seeing now is that it's downloading the uri.sh but it's not actually echoing anything into /var/tmp/text.txt. We are making the file but it's not doing the echo. When I try to do it as the normal user when logging into the Azure vm, I don't get any errors and it echos fine to the file.

I am going to keep debugging and see if that is an issue with us or if something else changed on the Azure side. First issue is solved at least and will make a PR when I am done looking at the 2nd part of the issue.

Comment 13 Chris Roberts 2022-08-31 19:18:00 UTC
Looking at this we have to actually call the script we are downloading:

https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/custom-script-linux#property-values (see commandtoexecute) Since this was working before I am guessing something prob changed in this extension.

Going to try and adjust my command to create the /var/tmp/text and then run the script. If I can get that to work I will post my command here and in the pr testing steps. Hopefully that is the last piece to this puzzle.

Comment 14 Chris Roberts 2022-08-31 21:16:54 UTC
Able to get the QE uri.sh script running with this as the command:

sudo sh /var/lib/waagent/custom-script/download/0/uri.sh

Will start to work on a PR and link it up when it's done.

Comment 15 Bryan Kearney 2022-09-08 20:04:08 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/35481 has been resolved.

Comment 21 errata-xmlrpc 2023-05-03 13:21:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.13 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2097


Note You need to log in before you can comment on or make changes to this bug.