Bug 1472999

Summary: ValueError: device is already in tree
Product: [Fedora] Fedora Reporter: larsehauge
Component: python-blivetAssignee: Blivet Maintenance Team <blivet-maint-list>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 26CC: anaconda-maint-list, blivet-maint-list, dlehman, dwt, g.kaviyarasu, jkonecny, jonathan, mkolman, rvykydal, sbueno, vanmeeuwen+fedora, vponcova, vtrefny
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: abrt_hash:a6be6482f134a9b6db36de07b3a5b5328e542c4835164e9dd0100c5001244149;VARIANT_ID=workstation;
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-03 17:28:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
File: anaconda-tb
none
File: anaconda.log
none
File: environ
none
File: journalctl
none
File: lsblk_output
none
File: nmcli_dev_list
none
File: os_info
none
File: program.log
none
File: storage.log
none
File: ifcfg.log
none
Requested logs generated by anaconda crash reporter
none
Archive of VM log files from failed f26 install described in previous post. none

Description larsehauge 2017-07-19 19:44:21 UTC
Description of problem:
Having trouble installing both Fedora 26 from USB and Fedora 25 from DVD.

This report is from the USB with Fedora 26.
The installer is started, but the error message with unknown error comes as soon as user input is required.

I tried to update the BIOS after failing to install Fedora. 
However that also failed, but luckly the computer rebooted as normal afterwards.
Not sure if these to issues are connected.

Any assistance would be highly appreciated.

Lars

Version-Release number of selected component:
anaconda-core-26.21.11-1.fc26.x86_64

The following was filed automatically by anaconda:
anaconda 26.21.11-1 exception report
Traceback (most recent call first):
  File "/usr/lib/python3.6/site-packages/blivet/devicetree.py", line 148, in _add_device
    raise ValueError("device is already in tree")
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/populator/helpers/partition.py", line 111, in run
    self._devicetree._add_device(device)
  File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 306, in handle_device
    device = helper_class(self, info).run()
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 154, in _add_slave_devices
    self.handle_device(slave_info)
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/populator/helpers/mdraid.py", line 55, in run
    self._devicetree._add_slave_devices(self.data)
  File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 306, in handle_device
    device = helper_class(self, info).run()
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/populator/helpers/mdraid.py", line 216, in run
    self._devicetree.handle_device(array_info, update_orig_fmt=True)
  File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 345, in handle_format
    helper_class(self, info, device).run()
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 318, in handle_device
    self.handle_format(info, device)
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 519, in _populate
    self.handle_device(dev)
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 454, in populate
    self._populate()
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/blivet.py", line 271, in reset
    self.devicetree.populate(cleanup_only=cleanup_only)
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 45, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/osinstall.py", line 1175, in storage_initialize
    storage.reset()
  File "/usr/lib64/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.6/site-packages/pyanaconda/threads.py", line 251, in run
    threading.Thread.run(self)
ValueError: device is already in tree

Additional info:
addons:         com_redhat_kdump
cmdline:        /usr/libexec/system-python  /sbin/anaconda --liveinst --method=livecd:///dev/mapper/live-base
cmdline_file:   BOOT_IMAGE=/images/pxeboot/vmlinuz root=live:CDLABEL=Fedora-WS-Live-26-1-5 rd.live.image quiet
executable:     /sbin/anaconda
hashmarkername: anaconda
kernel:         4.11.8-300.fc26.x86_64
other involved packages: python3-blivet-2.1.9-1.fc26.noarch, system-python-libs-3.6.1-8.fc26.x86_64
product:        Fedora
release:        Fedora release 26 (Twenty Six)
type:           anaconda
version:        26

Comment 1 larsehauge 2017-07-19 19:44:28 UTC
Created attachment 1301305 [details]
File: anaconda-tb

Comment 2 larsehauge 2017-07-19 19:44:30 UTC
Created attachment 1301306 [details]
File: anaconda.log

Comment 3 larsehauge 2017-07-19 19:44:31 UTC
Created attachment 1301307 [details]
File: environ

Comment 4 larsehauge 2017-07-19 19:44:34 UTC
Created attachment 1301308 [details]
File: journalctl

Comment 5 larsehauge 2017-07-19 19:44:35 UTC
Created attachment 1301309 [details]
File: lsblk_output

Comment 6 larsehauge 2017-07-19 19:44:37 UTC
Created attachment 1301310 [details]
File: nmcli_dev_list

Comment 7 larsehauge 2017-07-19 19:44:39 UTC
Created attachment 1301311 [details]
File: os_info

Comment 8 larsehauge 2017-07-19 19:44:41 UTC
Created attachment 1301312 [details]
File: program.log

Comment 9 larsehauge 2017-07-19 19:44:43 UTC
Created attachment 1301313 [details]
File: storage.log

Comment 10 larsehauge 2017-07-19 19:44:44 UTC
Created attachment 1301314 [details]
File: ifcfg.log

Comment 11 larsehauge 2017-07-20 17:25:23 UTC
Solved it by disconnecting the other drives physically (disconnecting SATA).
Was then able to install Fedora. 

Closing the bug report.

Comment 12 Dennis W. Tokarski 2017-10-19 14:31:09 UTC
Similar problem has been detected:

Just start the installer for f26. Local dvd iso, netinstall, doesn't matter.
No user interaction is required.

When the first language selection window appears there will be a
brief pause--just a few seconds--then the unexpected error message
dialog pops up.

As usual since about f14, anaconda is puking on lvm-on-md. Some
releases have worked, most have not, and on a few it's the presence
of a luks layer in there somewhere which causes it to choke.  I have
lvm-on-luks-on-md (raid6).

Strangely, when I very wisely tested this scenario on a vm it worked.
Thus encouraged, I began the installation on real hardware and this
is the result.

This exact bug seems to have been reported in July. When I can get
logged into Bugzilla I'll look that up and put a pointer here.

addons:         com_redhat_docker, com_redhat_kdump
cmdline:        /usr/libexec/system-python  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-S-dvd-x86_64-26 quiet
hashmarkername: anaconda
kernel:         4.11.8-300.fc26.x86_64
package:        anaconda-26.21.11-1
product:        Fedora
reason:         ValueError: device is already in tree
release:        Cannot get release name.
version:        26

Comment 13 Dennis W. Tokarski 2017-10-19 14:45:28 UTC
Ah, I see the installer bug reporter dropped this into the existing bug report.

This really should not have been closed. Having to rip open the system and physically disconnect random drives until the installer starts to work is >>NOT<< a solution.

Comment 14 Jiri Konecny 2017-10-20 08:37:15 UTC
Hello Dennis,

First, we didn't close the bug reporter does.

Second, in general it is not that easy support everything users can create. There are really big number of configurations users can create and everything is still evolving so please bear with us.

I'm opening this issue and changing components to the storage library.

Comment 15 David Lehman 2017-10-20 15:16:42 UTC
Dennis, can you attach the logs from your failed installation?

Comment 16 Dennis W. Tokarski 2017-10-20 16:26:19 UTC
Created attachment 1341336 [details]
Requested logs generated by anaconda crash reporter

This log set is one I had saved from the boot attempt just prior to the one which generated the auto-report. Nothing was changed between the two occasions and the observed behavior was identical.

Oh btw, this machine has two raid6 arrays of four drives each. The production set is 4x1TB and the other is 4x250GB. The latter is just used for as-needed bulk storage. The logs I just provided ran on that configuration.

I have since followed lars' inspiration and removed the SATA interface card for the bulk store array, leaving just the production array and a couple of DVD drives. Rebooting the installer produced the same result.

Comment 17 David Lehman 2017-10-20 17:21:34 UTC
The root cause of the problem you are having, Dennis, is that sda and sdi both have the same "UUID" value of "aaaaaaaa", which leads to partitions sda1 and sdi1 both having the same UUID: "aaaaaaaa-1". There is a requirement that UUIDs actually be unique. I don't know off the top of my head how to change one of them to satisfy this requirement.

Comment 18 David Lehman 2017-10-20 17:22:44 UTC
To clarify, I'm referring to the partition table UUID, on which the partition UUIDs are based.

Comment 19 David Lehman 2017-10-20 18:13:10 UTC
The original reporter apparently cloned the partition table on one drive across at least two others without modifying the duplicates so their UUIDs were unique.

Comment 20 Dennis W. Tokarski 2017-10-28 22:17:07 UTC
OK, now that's just bizarre. Not the UUID uniqueness requirement,
that's entirely reasonable. It's also new with f25, by the way,
which explains why I've never run into this before.

No, what was bugging me here is how my partition UUIDs came to
be 0xaaaaaaaa in the first place. I could just--barely!--accept
that I hit that lucky one in 2^32 chance on one of them. But
that and two others as well? No. Just no.

I'm pretty sure what happened is this...

This is a multiboot system (was f18/16/14) and what's now the
bulk store array used to be the production array. When the
4x1TB hardware arrived and it came time to install f20, I
would have hooked up the new drives to the f18 system and
tested them with badblocks.  Probably at some point once it
finished I started another round and then decided not to
wait for it, interrupting part way into the write-pattern-
0xAA pass.

Then I would have manually partitioned the first 1TB drive
and cloned the remainders with

   sfdisk-d /dev/sde | sfdisk /dev/sdf ... then g and h.

When this is done using f18 sfdisk, the label-id is not
copied, at least not if the destination is already non-0,
and neither is a new one generated. So there I had three
drives with all 0xaa for UUIDs. And no installation I've
ever done since has cared until now. Which is fine.

I made up a VM to test this scenario, tried the f26 install
on it, and got the result reported here. So far so good.
Then I used hexedit to give the last three drives unique
UUIDs, after which the installation showed the first
language selection display, blanked the screen, and did
nothing more. On vc 1 at the bottom of the screen it
said "Pane is dead".

Since editing the UUIDs didn't seem to cause any harm 
I tried this on the bare metal machine. There the install
finished without issue, leaving me with a shiny new
f26 system. No idea why that worked and the VM didn't.

So, my immediate problem here is solved now that you've
identified that crucial clue. Thanks for that!

There remains the mystery of why the similarly configured
VM wouldn't install and what the different failure mode
means.

I will attach the VM logs in case you care to pursue that.

Comment 21 Dennis W. Tokarski 2017-10-28 22:26:21 UTC
Created attachment 1344884 [details]
Archive of VM log files from failed f26 install described in previous post.

I've also prepared an archive containing these logs,
a README, and the qcow2 images of the four drives
used by the VM as well as the xml machine definition.

It's 6+ GB though. If you want to look at this I can
push it up to a host with a faster connection than
I've got. It'll take a while to make available.

Let me know if you want it.

Comment 22 Fedora End Of Life 2018-05-03 08:42:58 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.