Bug 1630964 - Streaming raw images only works for whole disk images
Summary: Streaming raw images only works for whole disk images
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ironic
Version: 15.0 (Stein)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: Upstream M2
: 15.0 (Stein)
Assignee: Dmitry Tantsur
QA Contact: mlammon
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-09-19 15:24 UTC by Dmitry Tantsur
Modified: 2019-09-26 10:46 UTC (History)
3 users (show)

Fixed In Version: openstack-ironic-12.1.1-0.20190427020357.d537f13.el8ost openstack-ironic-python-agent-3.5.1-0.20190206013117.d525f8a.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-21 11:18:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 605707 0 None None None 2018-10-26 10:54:47 UTC
OpenStack gerrit 610013 0 None None None 2018-10-26 10:55:43 UTC
Red Hat Bugzilla 1607779 0 high CLOSED [RFE] Direct deploy without Swift 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHEA-2019:2811 0 None None None 2019-09-21 11:19:12 UTC
Storyboard 2003809 0 None None None 2018-09-19 15:24:44 UTC

Internal Links: 1607779

Description Dmitry Tantsur 2018-09-19 15:24:45 UTC
This feature greatly reduces the memory requirement, since it avoids converting an image in memory. However, for some reason it only handles whole disk images.

Comment 5 mlammon 2019-07-23 21:44:23 UTC
In this scenario we have deploy method. image download source is swift.  
This test verify image is streamed with having the image format set to raw with an actual raw image. 
This was a pre-existing environment which just deployed direct

 
This assumes you have downloaded the director images (i.e   
for i in /usr/share/rhosp-director-images/overcloud-full-latest-15.0.tar /usr/share/rhosp-director-images/ironic-python-agent-latest-15.0.tar; do tar -xvf $i; done )

overcloud-full.qcow2 

# If you need to convert our standard qcow2 partition image to raw image
qemu-img convert -f qcow2 -O raw overcloud-full.qcow2 overcloud-full.img

# You can verify you image format
(undercloud) [stack@undercloud-0 raw_images]$ qemu-img info overcloud-full.img
image: overcloud-full.img
file format: raw
virtual size: 3.6G (3859480576 bytes)
disk size: 2.1G

(undercloud) [stack@undercloud-0 ~]$ openstack image list
+--------------------------------------+------------------------+--------+
| ID                                   | Name                   | Status |
+--------------------------------------+------------------------+--------+
| edf0b746-114d-4590-bb16-cda51de2c277 | bm-deploy-kernel       | active |
| 5b41c05c-6723-497e-a3fa-80aa2a780930 | bm-deploy-ramdisk      | active |
| de281dd1-f0db-479b-8a8a-2549af164c82 | overcloud-full-initrd  | active |
| 3a0a05b4-6932-4266-9d87-8699a6fa90fa | overcloud-full-vmlinuz | active |
+--------------------------------------+------------------------+--------+

(undercloud) [stack@undercloud-0 raw_images]$ openstack image create --disk-format raw --file overcloud-full.img overcloud-full --property kernel_id=edf0b746-114d-4590-bb16-cda51de2c277 --property ramdisk_id=5b41c05c-6723-497e-a3fa-80aa2a780930
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field            | Value                                                                                                                                                                                                                                                                                                                                                                         |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum         | e22b91ea3ac1e6fc83c2e13e45bf8dad                                                                                                                                                                                                                                                                                                                                              |
| container_format | bare                                                                                                                                                                                                                                                                                                                                                                          |
| created_at       | 2019-07-23T20:46:46Z                                                                                                                                                                                                                                                                                                                                                          |
| disk_format      | raw                                                                                                                                                                                                                                                                                                                                                                           |
| file             | /v2/images/99087c73-3fad-4610-a57c-e1ecdae777bf/file                                                                                                                                                                                                                                                                                                                          |
| id               | 99087c73-3fad-4610-a57c-e1ecdae777bf                                                                                                                                                                                                                                                                                                                                          |
| min_disk         | 0                                                                                                                                                                                                                                                                                                                                                                             |
| min_ram          | 0                                                                                                                                                                                                                                                                                                                                                                             |
| name             | overcloud-full                                                                                                                                                                                                                                                                                                                                                                |
| owner            | 26c53cbd36724bd7a1f7b2df03af8ea8                                                                                                                                                                                                                                                                                                                                              |
| properties       | direct_url='swift+config://ref1/glance/99087c73-3fad-4610-a57c-e1ecdae777bf', kernel_id='edf0b746-114d-4590-bb16-cda51de2c277', os_hash_algo='sha512', os_hash_value='676548a7af34bee44e1286256e089849f5cf3ffca2ae0fa43357085215fcfb4114e5969cc4fcbbffb38eb77904144cfd65a04344776978b2c553f65aa4d39238', os_hidden='False', ramdisk_id='5b41c05c-6723-497e-a3fa-80aa2a780930' |
| protected        | False                                                                                                                                                                                                                                                                                                                                                                         |
| schema           | /v2/schemas/image                                                                                                                                                                                                                                                                                                                                                             |
| size             | 3859480576                                                                                                                                                                                                                                                                                                                                                                    |
| status           | active                                                                                                                                                                                                                                                                                                                                                                        |
| tags             |                                                                                                                                                                                                                                                                                                                                                                               |
| updated_at       | 2019-07-23T20:47:48Z                                                                                                                                                                                                                                                                                                                                                          |
| virtual_size     | None                                                                                                                                                                                                                                                                                                                                                                          |
| visibility       | shared                                                                                                                                                                                                                                                                                                                                                                        |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

To verify in the deploy journalctl deploy logs here:
/var/log/containers/ironic/deploy/

untar one of the node file
546dd5cd-1824-498c-bf05-f9e2eb07b553_3fe766b2-2b7f-4991-9473-d374ae9e6708_2019-07-23-15-15-56.tar.gz 

 [root@undercloud-0 tmp]# ll
total 580
-rw-r--r--. 1 root root  51613 Jul 23 20:55 546dd5cd-1824-498c-bf05-f9e2eb07b553_3fe766b2-2b7f-4991-9473-d374ae9e6708_2019-07-23-15-15-56.tar.gz
-rw-r--r--. 1 root root   1951 Jul 23 20:55 df
-rw-r--r--. 1 root root   1615 Jul 23 20:55 ip_addr
-rw-r--r--. 1 root root    275 Jul 23 20:55 iptables
-rw-r--r--. 1 root root 524196 Jul 23 20:55 journal
-rw-r--r--. 1 root root    184 Jul 23 20:55 ps

 [root@undercloud-0 tmp]# grep -r "Image streamed onto device" journal
Jul 23 11:15:32 host-192-168-24-14 ironic-python-agent[703]: 2019-07-23 11:15:32.438 703 INFO ironic_python_agent.extensions.standby [-] Image streamed onto device /dev/vda2 in 18.671968460083008 seconds


2) Next we need to verify with ImageDownloadSource set for http opposed to using swift

Comment 9 mlammon 2019-07-29 16:44:45 UTC
Env:
openstack-ironic-12.1.2-0.20190715172459.c1b18ca.el8ost.src.rpm
openstack-ironic-python-agent-3.6.2-0.20190703190413.18637c7.el8ost.src.rpm
 

ImageDownloadSource: http has now been verified using Ironic in UC with deploying OC nodes.
It is verified looking at the log.
"Jul 29 11:22:59 host-192-168-24-11 ironic-python-agent[678]: 2019-07-29 11:22:59.607 678 INFO ironic_python_agent.extensions.standby [-] Image streamed onto device /dev/vda2 in 14.087767124176025 seconds"


Configurations:

Note: using undercloud_custom_env.yaml 

(undercloud) [stack@undercloud-0 ~]$ cat undercloud.conf
[DEFAULT]
# Network interface on the Undercloud that will be handling the PXE
# boots and DHCP for Overcloud instances. (string value)
local_interface = eth0
local_ip = 192.168.24.1/24
undercloud_public_host = 192.168.24.2
undercloud_admin_host = 192.168.24.3
#TODO: use release >= 10 when RHBZ#1633193 is resolved
undercloud_ntp_servers=clock1.rdu2.redhat.com
container_images_file=/home/stack/containers-prepare-parameter.yaml
container_insecure_registries = brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888
undercloud_timezone = UTC
undercloud_service_certificate = /etc/pki/instack-certs/undercloud.pem
hieradata_override = /home/stack/hiera_override.yaml
custom_env_files = /home/stack/undercloud_custom_env.yaml
[ctlplane-subnet]
local_subnet = ctlplane-subnet
cidr = 192.168.24.0/24
dhcp_start = 192.168.24.5
dhcp_end = 192.168.24.24
gateway = 192.168.24.1
inspection_iprange = 192.168.24.100,192.168.24.120
masquerade = true
#TODO(skatlapa): add param to override masq



(undercloud) [stack@undercloud-0 ~]$ cat /home/stack/undercloud_custom_env.yaml
parameter_defaults:
   IronicDefaultDeployInterface: direct
   IronicImageDownloadSource: http

Comment 12 errata-xmlrpc 2019-09-21 11:18:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811


Note You need to log in before you can comment on or make changes to this bug.