Bug 1281845

The RHEV-H Next installer will provide the flexibility to choose a custom layout which only needs to suite a few requirements.

With this flexibility it will be possible to choose a custom partitioning layout where the local storage domain can be preserved upon reinstallation of RHEV-H.
I.e. the local storage domain can be put onto a separate LV.

However, this is not done automatically, it needs to be considered during the initial installation.

Example:
1. During the first installation
   - Use a thon LVVM layout
   - Create a separate volume for /localsd
2. Install
3. Reinstall
   - Clear volumes, except for the volume used for /localsd

Does this flow meet the requirements?

This can now be tested with upstream oVirt Node Next builds.

That's the reply from the customer:

This new feature is a very interesting improvement towards the feature we are requesting.

We think also that would be very useful that the installer warn you before overwriting a local storage, as this kind of operations could be critical in environments using local storage.

Thanks Alexandros, that sound slike another RFE.

Let's take this RFE to track that there is the ability to keep local SDs.
An additional RFE should be filed to get a warning during reinstallation, if there is the risk of destroying an local SD.

Thank you for the reply Fabian.
I have filed:

Bug 1327248 - [RFE] Warning during rhev-h reinstallation, if there is the risk of destroying a local SD.
https://bugzilla.redhat.com/show_bug.cgi?id=1327248

Hi Fabian,

According #c4 & 6, I did the following testing.

1. Anaconda interactive installation on local machine, automatic partition, installer will pop-up warn info before overwriting a local storage like attachment "anaconda-auto-manual-partition" show.

2. Anaconda interactive installation on local machine, manual partition, installer still can pop-up warn info before overwriting a local storage.

3. ks installation on local machine, no warning info, but I don't think it is necessary.

Test version:
rhev-hypervisor7-ng-4.0-20160616.0.x86_64
imgbased-0.7.0-0.1.el7ev.noarch
redhat-release-rhev-hypervisor-4.0-0.7.el7.x86_64

So can I verify this bug with above test scenarios?

Thanks.

Created attachment 1168944 [details]
anaconda-auto-manual-partition

Chen, to me the testing in comment 10 rather looks like the verification of bug 1327248, see comment 8.


To verify this bug, you the test steps described in comment 4.

In other words again (and a bit different than comment 4):

1. Install NGN manually
   Create one VG for NGN installatin
   Create another separate VG with an LV for /localsd
2. Add NGN to Engine
3. Create local SD on /localsd
4. Ensure setup is working
5. Reinstall NGN
   - Boot from ISO
   - Remove the VG used for NGN installation
   - Create a new VG for NGN installation
   - KEEP THE /localsd VG (but no mountpoint set)
6. Add host to engine
7. Setup /localsd
8. Host into maintenance
9. Mount LV from separate VG to /localsd
10. Host tp up again

After 10: DC with localsd should be UP

(In reply to Fabian Deutsch from comment #12)
> Chen, to me the testing in comment 10 rather looks like the verification of
> bug 1327248, see comment 8.
> 
> 
> To verify this bug, you the test steps described in comment 4.
> 
> In other words again (and a bit different than comment 4):
> 
> 1. Install NGN manually
>    Create one VG for NGN installatin
>    Create another separate VG with an LV for /localsd
> 2. Add NGN to Engine
> 3. Create local SD on /localsd
> 4. Ensure setup is working
> 5. Reinstall NGN
>    - Boot from ISO
>    - Remove the VG used for NGN installation
>    - Create a new VG for NGN installation
>    - KEEP THE /localsd VG (but no mountpoint set)
> 6. Add host to engine
> 7. Setup /localsd
> 8. Host into maintenance
> 9. Mount LV from separate VG to /localsd
> 10. Host tp up again
> 
> After 10: DC with localsd should be UP

This bug is blocked by bug 1356087. I will verify this bug after 1356087 fixed.

Fabian, let's do not consider the bug 1356087 in comment 13, just consider the comment 12.

For step 3 in comment 12, the host initialized the meta data on this LV.
For step 7 in comment 12, I don't quite understand what the specific operation is, does it need to attach the /localsd to new host? 
For step 9 in comment 12, still not sure what should do.
For "After 10:" in comment 12, base on step 7 and step 9, I don't think DC with /localsd can be up unless the metadata is cleared. but if the meta data is cleared, why we preserved this LV during installation?

Let's try to clarify the requirement here:
Would the customer like to _re-use_ the local domain with initial metadata? Does that means if during installation, the local storage domain is preserved, after installation, customer can UP the previous local storage domain and all initial meta data can be recognized?

According to comment 14, I have to reassign this bug to get the attention.

Let me clarify my steps and also reply to comment 14:

1. Install NGN manually
   Create one VG for NGN installatin
   Create another separate VG with an LV for OR use a separate disk for /localsd
2. Create a new cluster (for local SD) and add NGN to to this cluster
3. Create local SD on /localsd
4. Ensure this setup is working
5. Reinstall NGN
   - Boot from ISO
   - Remove the VG used for NGN installation
   - Create a new VG for NGN installation
   - KEEP THE /localsd VG (but no mountpoint set)
6. Create a new cluster and add the reinstalled host to this cluster
7. Recover /localsd using:
7.a Mount the existing localsds VG OR separate device to /localsd_old
7.b mkdir /localsd
7.c Import the /localsd_old into /localsd as described here: http://www.ovirt.org/develop/release-management/features/storage/importstoragedomain/

This procedur should
a) Keep/Not delete the plain data on the localsd VG or the separate device
b) Allow th euser to use the localsd again after reinstallation

Ying, does this make sense?

Julie, I wonder if this is something for the docs, or rather for a kbase.

(In reply to Fabian Deutsch from comment #17)
> Let me clarify my steps and also reply to comment 14:
> 
> 1. Install NGN manually
>    Create one VG for NGN installatin
>    Create another separate VG with an LV for OR use a separate disk for
> /localsd
> 2. Create a new cluster (for local SD) and add NGN to to this cluster
> 3. Create local SD on /localsd
> 4. Ensure this setup is working
> 5. Reinstall NGN
>    - Boot from ISO
>    - Remove the VG used for NGN installation
>    - Create a new VG for NGN installation
>    - KEEP THE /localsd VG (but no mountpoint set)
> 6. Create a new cluster and add the reinstalled host to this cluster
> 7. Recover /localsd using:
> 7.a Mount the existing localsds VG OR separate device to /localsd_old
> 7.b mkdir /localsd
> 7.c Import the /localsd_old into /localsd as described here:
> http://www.ovirt.org/develop/release-management/features/storage/
> importstoragedomain/

Thanks Fabian.
I knew we could export to NFS domain and import the VM/template again before. 
But step 7, I still need to consider it tomorrow. 


> 
> This procedur should
> a) Keep/Not delete the plain data on the localsd VG or the separate device
> b) Allow th euser to use the localsd again after reinstallation
> 
> Ying, does this make sense?

The expectations are correct, keep the plain data and user can reuse the data again after reinstallation.

Hi, 

Here the steps I made:

Create a virtual machine with 2 hard-disks

#1) Installed redhat-virtualization-host-4.0-20160812.0.x86_64.liveimg.squashfs via kickstart

    Boot the iso: RHEL-7.2-20151030.0-Server-x86_64-boot.iso added to the grub 
    lines:
              ks=http://192.168.125.1/user.cfg

#2) Created directory in the Hypervisor
    # mkdir /data

    # fdisk /dev/sdb (created partition 1)
    # mkfs.ext4 /dev/sdb1
    # mount -t ext4 /dev/sdb1 /data
    (Added as well in fstab to auto-mount)

    # chown 36:36 /data

#3) In the RHEV Admin page:
     - Created Datacenter/Cluster for local storage
     - Registered/Approved the Host to the new Datacenter/Cluster
     - Created Local Storage specifying /data in the Storage tab

* At this point Host is UP and Storage is UP

Rebooted the host to do the re-installation:

Manual re-install:
======================
- Anaconda complains about having storage already used (/dev/sda and /dev/sdb) and asked to click in the storage icon to resolve it. I have clicked and DESELECT the /dev/sdb storage to avoid any issue with my local storage.


Automatic re-install:
=======================
-   Boot the iso: RHEL-7.2-20151030.0-Server-x86_64-boot.iso added to the grub 
    lines:
              ks=http://192.168.125.1/user.cfg


   * Now, I have changed the kickstart to change clearpart option to only touch 
     the sda partitions.
     <snip>
     clearpart --all --drives=sda
     </snip>

After the reinstall
========================
I can see the local storage data available in the /dev/sdb1:

# mount /dev/sdb1 /mnt
# ls /mnt/
__DIRECT_IO_TEST__  e44d9940-9fde-47c6-be0d-d78cf3f0adfd  lost+found


==== user.cfg ==================
#Authconfig set
authconfig --enableshadow --passalgo=md5

#Keyboard type set
keyboard us

#Default language set
lang en_US
liveimg --url=http://192.168.125.1/redhat-virtualization-host-4.0-20160812.0.x86_64.liveimg.squashfs

#Bootloader location set
bootloader --location=mbr

#Root password set
rootpw --plaintext redhat

#Clear partitions before disk part
clearpart --all

#Auto partition in thinp mode
autopart --type=thinp

%post --erroronfail
imgbase layout --init
imgbase --experimental volume --create /var 4G
%end

#Text mode for installer
text

#Reboot system after install
reboot

To import the existing domain:

- In the node, prepare the /data path again:

  # mkdir /data
  # mount /dev/sdb1 /data
  <The previous storage still there>
  # chown 36:36 /data


- Registered the node into RHEVM (4.0.2.6-0.1.el7ev)
- Created Datacenter/Cluster for local storage
- Create a new local storage in /mnt 
  (I couldn't import directly the previous one)

< At this moment everything should be up >

- Now in RHEVM, go to storage and click import and provide /data
  and later active it in the DataCenter tab -> Storage.

Finally, if you put the temporary local storage domain from /mnt in maintenance mode it will change the Data (master) storage to the imported domain.

I hope this test helps to clarify.

Thanks Douglas for detail on this bug, we will check this TestOnly bug on 4.0 again to make sure all works well or not during QE testing, the ETA: Aug 21.

Chen, the steps in comment 21 and comment 22 are very clearly steps to us, could we try to this today? It is RFE bug, we would like to make it into GA once pass QE. 

For comment 21 and comment 22, you also need create VM on previous local storage /data, and _after_ reinstalled, and /data storage is up on Host again, then try to start this VM, if VM can be started after reinstallaion host, then we can verify this bug if we do not consider the bug 1356087 in comment 13.

Hi Douglas,
   Thank you for detail with steps to reproduce.And follow your steps,i tested successfully.

Here the steps I made:
     My testing environment is the machine with storage "/dev/sda" and the USB drive "/dev/sdb".

#1) Installed redhat-virtualization-host-4.0-20160817.0.x86_64.liveimg.squashfs via GUI with storage "/dev/sda",and "/dev/sdb" keep.

  
#2) Created directory in the Hypervisor
    # mkdir /data

    # fdisk /dev/sdb (created partition 1)
    # mkfs.ext4 /dev/sdb1
    # mount -t ext4 /dev/sdb1 /data
    (Added as well in fstab to auto-mount)

    # chown 36:36 /data
     

#3) In the RHEV Admin page:
     - Created Datacenter/Cluster for local storage
     - Registered/Approved the Host to the new Datacenter/Cluster
     - Created Local Storage specifying /data in the Storage tab

* At this point Host is UP and Storage is UP,and the output of using cmd "ls /data":
[root@dhcp-8-194 /]# ls /data                                                   
0f26c6a8-7617-4ba6-8ab4-1a12cfa12ddf  __DIRECT_IO_TEST__  lost+found 

#4)Rebooted the host to do the re-installation:
     Reinstalled redhat-virtualization-host-4.0-20160817.0.x86_64.liveimg.squashfs via GUI with storage "/dev/sda",and "/dev/sdb" keep.


#5)import the existing domain:

- In the node, prepare the /data path again:

  # mkdir /data
  # mount /dev/sdb1 /data
  <The previous storage still there>
  # chown 36:36 /data

The output of using cmd "ls /data":
[root@dhcp-8-194 /]# ls /data                                                   
0f26c6a8-7617-4ba6-8ab4-1a12cfa12ddf  __DIRECT_IO_TEST__  lost+found 


- Registered the node into RHEVM (4.0.2.6-0.1.el7ev)
- Created Datacenter/Cluster for local storage
- Create a new local storage in /mnt (chown 36:36 /mnt)
  
< At this moment everything should be up >

Now in RHEVM, go to storage and click import and provide /data
  and it will display some information about storage,then the import storage is maintenance,later active it in the DataCenter tab -> Storage.

Finally, i put the temporary local storage domain from /mnt in maintenance mode it will change the Data (master) storage to the imported domain.

Add:some pictures about storage information listed on attachment.

Thank you for all efforts.
Yihui Zhao

Created attachment 1191844 [details]
activate.png

Created attachment 1191846 [details]
data_terminal.png

Created attachment 1191847 [details]
local_rhevm.png

Created attachment 1191848 [details]
maintenance.png

Created attachment 1191849 [details]
rhevm.data.png

Created attachment 1191850 [details]
rhevm_import.png

Just a note, the bug in comment 13 is considered to be an enhancement.
But in today's state it is possible - as described in comment 21 - to setup storage to allow keeping local storage domain data.

Thanks yzhao for the detail testing, cancel the needinfo flag.

Verify this bug according #c25.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1702.html

Hello Douglas/All,

Thanks for the Enhancement. Customer is happy but came up with one more request.
This enhancement is working for RHVN to RHVN, but is it working from RHEV-H (3.6) to RHVN? Please let me know if not so i can open a Bug (and if there is any possibility that we will implement it to help users migrate easier from 3.6 to 4)

(In reply to Alexandros Gkesos from comment #38)
> Hello Douglas/All,
> 
> Thanks for the Enhancement. Customer is happy but came up with one more
> request.
> This enhancement is working for RHVN to RHVN, but is it working from RHEV-H
> (3.6) to RHVN? Please let me know if not so i can open a Bug (and if there
> is any possibility that we will implement it to help users migrate easier
> from 3.6 to 4)

From 3.6 to 4 we must check a valid path and test it. We are currently looking at a few options, but it will take some time before there is a result. There is a specific bug report, please see: bz#1320556.

Hi Douglas,

Can you please review current documentation we have on the subject and confirm it is accurate?

First, we have the admin guide section about local storage, with the 2 notes on top related to this bug:
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/html/administration_guide/sect-preparing_and_adding_local_storage

Second, we have a KCS, that the first note is pointing to:
https://access.redhat.com/solutions/2804081

Thank you!

(In reply to Marina from comment #41)
> Hi Douglas,
> 
> Can you please review current documentation we have on the subject and
> confirm it is accurate?
> 
> First, we have the admin guide section about local storage, with the 2 notes
> on top related to this bug:
> https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/
> html/administration_guide/sect-preparing_and_adding_local_storage
> 
> Second, we have a KCS, that the first note is pointing to:
> https://access.redhat.com/solutions/2804081
> 
Looks good Marina, we are working to document the steps via:
https://bugzilla.redhat.com/show_bug.cgi?id=1421437