Bug 1842440 - nbdkit cannot read the password interactively when started from virt-v2v
Summary: nbdkit cannot read the password interactively when started from virt-v2v
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: nbdkit
Version: 8.3
Hardware: x86_64
OS: Unspecified
medium
medium
Target Milestone: rc
: 8.3
Assignee: Richard W.M. Jones
QA Contact: mxie@redhat.com
URL:
Whiteboard:
Depends On: 1809231
Blocks: 1838425
TreeView+ depends on / blocked
 
Reported: 2020-06-01 09:27 UTC by Pino Toscano
Modified: 2020-11-17 17:49 UTC (History)
12 users (show)

Fixed In Version: nbdkit-1.20.3-1.module+el8.3.0+6969+c553f4a0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1838425
Environment:
Last Closed: 2020-11-17 17:49:13 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
vddk-input-password-manually.log (6.89 KB, text/plain)
2020-06-01 09:29 UTC, Pino Toscano
no flags Details
strace (1.74 MB, text/plain)
2020-06-01 09:38 UTC, Richard W.M. Jones
no flags Details

Description Pino Toscano 2020-06-01 09:27:39 UTC
+++ This bug was initially created as a clone of Bug #1838425 +++

Verify the bug with below builds:
virt-v2v-1.42.0-4.module+el8.3.0+6798+ad6e66be.x86_64
libguestfs-1.42.0-2.module+el8.3.0+6798+ad6e66be.x86_64
libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64
qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64
nbdkit-1.20.2-1.module+el8.3.0+6764+cc503f20.x86_64
VMware-vix-disklib-7.0.0-15832853.x86_64.tar.gz

Steps:
1.Convert a guest from VMware via vddk and don't set any option about password in v2v command line. Input the correct password according to the prompt, but nbdkit can't recognize the password

# virt-v2v  -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA  esx6.7-rhel8.2-x86_64 
[   0.0] Opening the source -i libvirt -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 esx6.7-rhel8.2-x86_64 -it vddk  -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA
Enter root's password for 10.73.73.141: 
password: 
nbdkit: error: could not read password from stdin: Inappropriate ioctl for device
virt-v2v: error: nbdkit did not start up.  There may be errors printed by 
nbdkit above.

If the messages above are not sufficient to diagnose the problem then add 
the ‘virt-v2v -v -x’ options and examine the debugging output 
carefully.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

Comment 1 Pino Toscano 2020-06-01 09:29:46 UTC
Created attachment 1694051 [details]
vddk-input-password-manually.log

Relevant log from bug 1842440.

Comment 2 Richard W.M. Jones 2020-06-01 09:38:04 UTC
Created attachment 1694057 [details]
strace

This is the strace from reproducing the bug.
I've removed passwords from the file.

When the error occurs, with my annotations:

4147927 fstat(1, {st_mode=S_IFCHR|0666, st_rdev=makedev(0x1, 0x3), ...}) = 0

isatty (https://github.com/libguestfs/nbdkit/blob/a18e1566702a247b3c344044fd2d206a042a5c32/server/public.c#L438)

4147927 ioctl(1, TCGETS, 0x7ffdbd08cc90) = -1 ENOTTY (Inappropriate ioctl for device)
4147927 ioctl(0, TCGETS, 0x7ffdbd08d410) = -1 ENOTTY (Inappropriate ioctl for device)
4147927 fstat(0, {st_mode=S_IFCHR|0666, st_rdev=makedev(0x1, 0x3), ...}) = 0
4147927 ioctl(0, TCGETS, 0x7ffdbd08d280) = -1 ENOTTY (Inappropriate ioctl for device)

tcgetattr/tcsetattr (https://github.com/libguestfs/nbdkit/blob/a18e1566702a247b3c344044fd2d206a042a5c32/server/public.c#L440)

4147927 brk(NULL)                       = 0x561af1f71000
4147927 brk(0x561af1f92000)             = 0x561af1f92000
4147927 read(0, "", 4096)               = 0

getline (https://github.com/libguestfs/nbdkit/blob/a18e1566702a247b3c344044fd2d206a042a5c32/server/public.c#L446)
Note that there was no failure, but errno was polluted by the earlier errors.
We then save the incorrect errno.

4147927 ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0

tcsetattr (https://github.com/libguestfs/nbdkit/blob/a18e1566702a247b3c344044fd2d206a042a5c32/server/public.c#L451)

4147927 write(2, "\33[1;31m", 7)        = 7
4147927 write(2, "nbdkit: ", 8)         = 8
4147927 write(2, "error: ", 7)          = 7
4147927 write(2, "could not read password from stdin: Inappropriate ioctl for device", 66) = 66
4147927 write(2, "\n", 1)               = 1
4147927 write(2, "\33[0m", 4)           = 4
4147927 write(1, "password: \n", 11)    = 11

Delayed password prompt.  This would have been flushed and printed in two
parts presumably if stdin was connected to anything, but I suspect stdin
must be /dev/null.

Comment 3 Richard W.M. Jones 2020-06-01 09:44:43 UTC
One line reproducer:

$ nbdkit ssh host=localhost /nosuchfile password=- --run 'qemu-img info $nbd' </dev/null
password: 
nbdkit: error: could not read password from stdin: Inappropriate ioctl for device

Note that this doesn't address the question of why stdin == /dev/null
when nbdkit is run from virt-v2v, but I'll look at that issue separately.

Comment 4 Richard W.M. Jones 2020-06-01 10:32:02 UTC
Patches posted:

https://www.redhat.com/archives/libguestfs/2020-June/msg00005.html

but this is not the whole story with this bug.

Comment 5 Richard W.M. Jones 2020-06-01 17:22:09 UTC
Also needs this virt-v2v patch:

https://www.redhat.com/archives/libguestfs/2020-June/msg00013.html

Actually this on its own is sufficient, but it would be nice to have
the nbdkit patches from comment 4 as well.

Comment 7 Richard W.M. Jones 2020-06-09 11:39:56 UTC
The fix has been fully backported (for nbdkit) to nbdkit 1.20.3, so
this bug will be fixed by the next rebase (bug 1809231).

Comment 11 mxie@redhat.com 2020-06-11 10:50:34 UTC
Verify the bug with below builds:
virt-v2v-1.42.0-4.module+el8.3.0+6798+ad6e66be.x86_64
libguestfs-1.42.0-2.module+el8.3.0+6798+ad6e66be.x86_64
libvirt-6.4.0-1.module+el8.3.0+6881+88468c00.x86_64
qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64
nbdkit-1.20.3-1.module+el8.3.0+6969+c553f4a0.x86_64


Steps:
1.Convert a guest from VMware via vddk and don't set any option about password in v2v command line. Input the correct password for all password prompts, then v2v conversion can finish successfully

# virt-v2v  -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA  esx6.7-rhel8.2-x86_64 
[   0.0] Opening the source -i libvirt -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 esx6.7-rhel8.2-x86_64 -it vddk  -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA
Enter root's password for 10.73.73.141: 
password: 
[  15.0] Creating an overlay to protect the source from being modified
[  15.9] Opening the overlay
[  22.3] Inspecting the overlay
[  46.6] Checking for sufficient free disk space in the guest
[  46.6] Estimating space required on target for each disk
[  46.6] Converting Red Hat Enterprise Linux 8.2 (Ootpa) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 203.3] Mapping filesystem data to avoid copying unused and blank areas
[ 204.7] Closing the overlay
[ 205.0] Assigning disks to buses
[ 205.0] Checking if the guest needs BIOS or UEFI to boot
[ 205.0] Initializing the target -o libvirt -os default
[ 205.0] Copying disk 1/1 to /var/lib/libvirt/images/esx6.7-rhel8.2-x86_64-sda (raw)
    (100.00/100%)
[ 549.7] Creating output metadata
[ 549.7] Finishing off


2.Convert a guest from VMware via vddk and don't set any option about password in v2v command line. Input wrong password for the first password prompt, v2v conversion is failed with correct error
# virt-v2v  -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA  esx6.7-rhel8.2-x86_64 
[   0.0] Opening the source -i libvirt -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 esx6.7-rhel8.2-x86_64 -it vddk  -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA
Enter root's password for 10.73.73.141: 
virt-v2v: error: exception: libvirt: VIR_ERR_INTERNAL_ERROR: VIR_FROM_ESX: 
internal error: HTTP response code 500 for call to 'Login'. Fault: 
ServerFaultCode - Cannot complete login due to an incorrect user name or 
password.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]


3.Convert a guest from VMware via vddk and don't set any option about password in v2v command line. Input correct password for the first password prompt but input wrong password for the second password prompt, but v2v error is not 'incorrect user name or password'


# virt-v2v  -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA  esx6.7-rhel8.2-x86_64 
[   0.0] Opening the source -i libvirt -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 esx6.7-rhel8.2-x86_64 -it vddk  -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA
Enter root's password for 10.73.73.141: 
password: 
[   7.8] Creating an overlay to protect the source from being modified
nbdkit: vddk[1]: error: VixDiskLib_Open: [esx6.7-matrix] esx6.7-rhel8.2-x86_64/esx6.7-rhel8.2-x86_64.vmdk: Insufficient permissions in the host operating system
qemu-img: /var/tmp/v2vovl7f88f9.qcow2: Requested export not available
Could not open backing image to determine size.
virt-v2v: error: qemu-img command failed, see earlier errors

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]


4.Convert a guest from VMware without vddk and don't set any option about password in v2v command line. Input the correct password for all password prompts, v2v conversion can continue normally

# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-rhel8.2-x86_64 
[   0.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-rhel8.2-x86_64
Enter root's password for 10.73.198.169: 
Enter host password for user 'root':
Enter host password for user 'root':
[  19.4] Creating an overlay to protect the source from being modified
[  20.0] Opening the overlay
[  53.6] Inspecting the overlay
[ 135.7] Checking for sufficient free disk space in the guest
[ 135.7] Estimating space required on target for each disk
[ 135.7] Converting Red Hat Enterprise Linux 8.2 (Ootpa) to run on KVM
^C


5.Convert a guest from VMware without vddk and don't set any option about password in v2v command line. Input correct password for the first password prompt but input wrong password for the second or third password prompt, v2v conversion is failed with correct error
# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-rhel8.2-x86_64 
[   0.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-rhel8.2-x86_64
Enter root's password for 10.73.198.169: 
Enter host password for user 'root':
Enter host password for user 'root':
'curl' -q --max-redirs '5' --globoff --head --silent --url 'https://10.73.198.169/folder/esx7.0-rhel8.2-x86%5f64/esx7.0-rhel8.2-x86%5f64-flat.vmdk?dcPath=data&dsName=esx7.0-matrix' --user <hidden> --insecure
HTTP/2 401 
date: Thu, 11 Jun 2020 10:37:33 GMT
set-cookie: vmware_soap_session="2a5a0db8d8caf6ee004e1ccb54853044b7222d7e"; Path=/; HttpOnly; Secure; 
content-security-policy: block-all-mixed-content
strict-transport-security: max-age=31536000
www-authenticate: Basic realm="VMware HTTP server"
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1
content-length: 0
x-envoy-upstream-service-time: 5086
server: envoy

virt-v2v: error: vcenter: incorrect username or password

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]


Hi rjones,

   Please help to check the result of step3, is it a problem?

Comment 12 Richard W.M. Jones 2020-06-11 15:28:44 UTC
My opinion is that step 3 isn't really a problem, and this bug is
verified.

Ideally people would use -ip always, and when virt-v2v is consumed by
other programs like IMS those programs should always be using -ip.  The
ability to ask for the password is just a fail-over for people using
virt-v2v on the command line.

Comment 13 mxie@redhat.com 2020-06-12 13:36:54 UTC
Thanks rjones, move the bug from ON_QA to VERIFIED according to comment11 and comment12.

Comment 16 errata-xmlrpc 2020-11-17 17:49:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5137


Note You need to log in before you can comment on or make changes to this bug.