RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1199403 - migration fails after hot-unplugging a virtconsole that is not the last one.
Summary: migration fails after hot-unplugging a virtconsole that is not the last one.
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.1
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Amit Shah
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-06 07:01 UTC by Luyao Huang
Modified: 2016-03-28 08:32 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-04-21 07:13:24 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Luyao Huang 2015-03-06 07:01:46 UTC 
Description of problem:
migrate will fail when migrate a vm after hot-unplug a console and the error should be improved

Version-Release number of selected component (if applicable):

libvirt-1.2.8-16.el7_1.1.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.1.x86_64

How reproducible:
100%

Steps to Reproduce:
 
1.prepare a happy running vm have settings like this:
# virsh dumpxml test3

    <console type='pty' tty='/dev/pts/18'>
      <source path='/dev/pts/18'/>
      <target type='virtio' port='0'/>
      <alias name='console0'/>
    </console>
    <console type='pty'>
      <source path='/dev/pts/19'/>
      <target type='virtio' port='1'/>
      <alias name='console1'/>
    </console>
    <console type='pty'>
      <source path='/dev/pts/20'/>
      <target type='virtio' port='2'/>
      <alias name='console2'/>
    </console>

2.hot-unplug the middle of this three console:

# cat virtioconsole.xml
    <console type='pty'>
      <source path='/dev/pts/19'/>
      <target type='virtio' port='1'/>
      <alias name='console1'/>
    </console>

# virsh detach-device test3 virtioconsole.xml
Device detached successfully

3.recheck the xml in source:

    <console type='pty' tty='/dev/pts/18'>
      <source path='/dev/pts/18'/>
      <target type='virtio' port='0'/>
      <alias name='console0'/>
    </console>
    <console type='pty'>
      <source path='/dev/pts/20'/>
      <target type='virtio' port='2'/>
      <alias name='console2'/>
    </console>

4.migrate to target:
# virsh migrate test3 qemu+ssh://10.66.6.12/system --live
error: Domain not found: no domain with matching name 'test3'

5.check the log in target:

2015-03-04 09:27:36.901+0000: 19330: error : qemuMonitorIO:701 : internal error: early end of file from monitor: possible problem:
qemu: warning: error while loading state for instance 0x0 of device '0000:00:05.0/virtio-console'
2015-03-04T09:27:36.862587Z qemu-kvm: load of migration failed: Invalid argument

6.check the xml which use to generate qemu process in target (before qemu died):

    <console type='pty' tty='/dev/pts/2'>
      <source path='/dev/pts/2'/>
      <target type='virtio' port='0'/>
      <alias name='console0'/>
    </console>
    <console type='pty'>
      <source path='/dev/pts/3'/>
      <target type='virtio' port='1'/>
      <alias name='console1'/>
    </console>


Actual results:
migrate will fail when migrate a vm after hot-unplug a console and the error should be improved

and test if we hotunplug the last virtio console won't meet this issue.

Expected results:
1. in step 4 error message should be improved and we can catch the log from qemu log
2. migrate should success


infomation:
Comment 1 Jiri Denemark 2015-03-06 08:30:33 UTC 
Yes, the error message should be improved, but we already track that in bug 1090093.

So we only need to fix the issue with chardev unplug. It seems we remove the device from domain definition even though QEMU still thinks the device is present, which should never happen since we started using device-detached events.
Comment 2 Ján Tomko 2015-03-06 11:44:40 UTC 
Libvirt does receive the event when detaching the device:
2015-03-06 11:33:04.122+0000: 11977: debug : qemuMonitorJSONIOProcessLine:183 : Line [{"timestamp": {"seconds": 1425641584, "microseconds": 122228}, "event": "DEVICE_DELETED", "data": {"device": "console1", "path": "/machine/peripheral/console1"}}]

But it does not seem to be removed completely, beacuse the destination QEMU still fails:

qemu: warning: error while loading state for instance 0x0 of device '0000:00:08.0/virtio-console'
2015-03-06T11:34:48.057625Z qemu-kvm: load of migration failed: Invalid argument


If I unplug console #2, then console #3 (leaving just console #1), migration works.
Comment 3 Amit Shah 2015-04-02 09:21:24 UTC 
We don't recommend using virtio-console; and we surely don't want people running multiple virtio-consoles.

Does this reproduce even with virtio-serial channels?

Is this a regression?  Can you please tell which was the last version that worked fine?
Comment 4 Luyao Huang 2015-04-02 09:54:27 UTC 
(In reply to Amit Shah from comment #3)
> We don't recommend using virtio-console; and we surely don't want people
> running multiple virtio-consoles.
> 
> Does this reproduce even with virtio-serial channels?
> 

Cannot reproduce with virtio-serial channels.

> Is this a regression?  Can you please tell which was the last version that
> worked fine?

test with rhel7.0 qemu-kvm-rhev still can meet this issue.

I will test with qemu-kvm tomorrow.
Comment 5 Luyao Huang 2015-04-03 08:18:17 UTC 
(In reply to Luyao Huang from comment #4)
> (In reply to Amit Shah from comment #3)
> > We don't recommend using virtio-console; and we surely don't want people
> > running multiple virtio-consoles.
> > 
> > Does this reproduce even with virtio-serial channels?
> > 
> 
> Cannot reproduce with virtio-serial channels.
> 
> > Is this a regression?  Can you please tell which was the last version that
> > worked fine?
> 
> test with rhel7.0 qemu-kvm-rhev still can meet this issue.
> 
> I will test with qemu-kvm tomorrow.

Test with qemu-kvm-1.5.3-87.el7 and qemu-kvm-1.5.3-60.el7_0.11.x86_64, still found this issue. So i don't think it is a regression.
Comment 6 Amit Shah 2015-04-21 06:46:38 UTC 
Since virtio-serial ports don't exhibit this behaviour, and only virtio-console ports do, this has to be something the guest does with console ports.  We don't recommend testing more than one console port anyway, and console ports are a low priority.

So unless this is breaking something important, it will be closed wontfix.
Comment 7 Luyao Huang 2015-04-21 07:00:33 UTC 
(In reply to Amit Shah from comment #6)
> Since virtio-serial ports don't exhibit this behaviour, and only
> virtio-console ports do, this has to be something the guest does with
> console ports.  We don't recommend testing more than one console port
> anyway, and console ports are a low priority.
> 
> So unless this is breaking something important, it will be closed wontfix.

Okay, this issue was found during test some corner case, so it is not a big issue and i think we can close this bug.
Comment 8 Amit Shah 2015-04-21 07:13:24 UTC 
Thanks.
Note You need to log in before you can comment on or make changes to this bug.