Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1601793

Summary: [cloud-init][Aliyun] cloud-init failure on Aliyun BMS
Product: Red Hat Enterprise Linux 7 Reporter: Chen Shi <cheshi>
Component: cloud-initAssignee: Eduardo Otubo <eterrell>
Status: CLOSED NOTABUG QA Contact: Vratislav Hutsky <vhutsky>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.5CC: cheshi, huzhao, jgreguske, linl, ribarry, wshi, xiliang, xiqin
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-05-13 13:08:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
cloud-init 18.2 status log
none
/var/log/messages
none
/var/log/cloud-init.log none

Description Chen Shi 2018-07-17 09:09:40 UTC
Description of problem:
cloud init failure log can be seen in system log on Aliyun BMS.

Version-Release number of selected component (if applicable):
cloud-init-18.2-1.el7.x86_64
cloud-init-0.7.9-24.el7.x86_64
cloud-init-0.7.9-9.el7_4.1.x86_64

RHEL Version:
RHEL7.5
RHEL7.4

How reproducible:
100%

Steps to Reproduce:
1. Create Instance on Aliyun BMS
2. Check the current cloud-init version:
cloud-init-0.7.9-24.el7.x86_64
3. Check /var/log/messages, we can see the following log: [1]
Apr 26 16:03:06 AliYunOS systemd: Stopped Initial cloud-init job (pre-networking).
Apr 26 16:03:06 AliYunOS systemd: Stopping Initial cloud-init job (pre-networking)...
Jul 16 14:19:17 AliYunOS systemd: Cannot add dependency job for unit cloud-init-upgrade.service, ignoring: Unit not found.
Jul 16 14:19:18 AliYunOS systemd: Starting Initial cloud-init job (pre-networking)...
Jul 16 14:19:20 AliYunOS systemd: Started Initial cloud-init job (pre-networking).
Jul 16 14:19:26 AliYunOS systemd: Starting Initial cloud-init job (metadata service crawler)...
Jul 16 14:19:26 AliYunOS systemd: cloud-init.service: main process exited, code=exited, status=1/FAILURE
Jul 16 14:19:26 AliYunOS systemd: Failed to start Initial cloud-init job (metadata service crawler).
Jul 16 14:19:26 AliYunOS systemd: Unit cloud-init.service entered failed state.
Jul 16 14:19:26 AliYunOS systemd: cloud-init.service failed.
4. Upgrade cloud-init to the latest build in brew and reboot the instance.
5. Check the current cloud-init version:
cloud-init-18.2-1.el7.x86_64
6. Check /var/log/messages, we can see the following log:
Jul 16 14:25:22 AliYunOS systemd: Stopped Initial cloud-init job (pre-networking).
Jul 16 14:25:22 AliYunOS systemd: Stopping Initial cloud-init job (pre-networking)...
Jul 16 14:27:50 AliYunOS systemd: Cannot add dependency job for unit cloud-init-upgrade.service, ignoring: Unit not found.
Jul 16 14:27:51 AliYunOS systemd: Starting Initial cloud-init job (pre-networking)...
Jul 16 14:27:53 AliYunOS systemd: Started Initial cloud-init job (pre-networking).
Jul 16 14:27:56 AliYunOS systemd: Starting Initial cloud-init job (metadata service crawler)...
Jul 16 14:27:56 AliYunOS systemd: cloud-init.service: main process exited, code=exited, status=1/FAILURE
Jul 16 14:27:56 AliYunOS systemd: Failed to start Initial cloud-init job (metadata service crawler).
Jul 16 14:27:56 AliYunOS systemd: Unit cloud-init.service entered failed state.
Jul 16 14:27:56 AliYunOS systemd: cloud-init.service failed.
7. Check the service status by `systemctl status` and we can see only cloud-init-local.service is active: [2][3]
● cloud-init.service - Initial cloud-init job (metadata service crawler)
   Loaded: loaded (/usr/lib/systemd/system/cloud-init.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2018-07-16 14:27:56 CST; 30min ago
--
● cloud-init-local.service - Initial cloud-init job (pre-networking)
   Loaded: loaded (/usr/lib/systemd/system/cloud-init-local.service; enabled; vendor preset: disabled)
   Active: active (exited) since Mon 2018-07-16 14:27:53 CST; 30min ago
--
● cloud-final.service - Execute cloud user/final scripts
   Loaded: loaded (/usr/lib/systemd/system/cloud-final.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2018-07-16 14:27:57 CST; 30min ago
--
● cloud-config.service - Apply the settings specified in cloud-config
   Loaded: loaded (/usr/lib/systemd/system/cloud-config.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2018-07-16 14:27:56 CST; 30min ago

Actual results:
Something wrong with cloud-init services.

Expected results:
cloud-init service should run without error.

Additional info:
1. This issue can be reproduced on ebmc4.8xlarge and ebmg5.24xlarge.
2. This issue can be reproduced with RHEL7.4 (cloud-init-0.7.9-9.el7_4.1.x86_64).
3. This issue can be reproduced with RHEL7.5 (cloud-init-0.7.9-24.el7.x86_64).
4. This issue can be reproduced with the latest cloud-init build in brew (cloud-init-18.2-1.el7.x86_64).
5. From Vitaly:
   [Vitaly] This is likely a result of missing patches in cloud-init. In 7.6 we're rebasing to 18.2, could you please grad this version from brew and test with it? If we're still seeing errors we'll likely need a BZ and Alibaba should work upstream to address these issues.

[1] More details can be found in the system log attached.
[2] More details can be found in the log attached.
[3] /var/log/cloud-init.log is also attached for your information.

Comment 2 Chen Shi 2018-07-17 09:11:58 UTC
Created attachment 1459368 [details]
cloud-init 18.2 status log

Comment 3 Chen Shi 2018-07-17 09:13:00 UTC
Created attachment 1459369 [details]
/var/log/messages

Comment 4 Chen Shi 2018-07-17 09:13:45 UTC
Created attachment 1459370 [details]
/var/log/cloud-init.log