Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be available on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1199403 - migration fails after hot-unplugging a virtconsole that is not the last one.
Summary: migration fails after hot-unplugging a virtconsole that is not the last one.
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.1
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Amit Shah
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-06 07:01 UTC by Luyao Huang
Modified: 2016-03-28 08:32 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-04-21 07:13:24 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Luyao Huang 2015-03-06 07:01:46 UTC
Description of problem:
migrate will fail when migrate a vm after hot-unplug a console and the error should be improved

Version-Release number of selected component (if applicable):

libvirt-1.2.8-16.el7_1.1.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.1.x86_64

How reproducible:
100%

Steps to Reproduce:
 
1.prepare a happy running vm have settings like this:
# virsh dumpxml test3

    <console type='pty' tty='/dev/pts/18'>
      <source path='/dev/pts/18'/>
      <target type='virtio' port='0'/>
      <alias name='console0'/>
    </console>
    <console type='pty'>
      <source path='/dev/pts/19'/>
      <target type='virtio' port='1'/>
      <alias name='console1'/>
    </console>
    <console type='pty'>
      <source path='/dev/pts/20'/>
      <target type='virtio' port='2'/>
      <alias name='console2'/>
    </console>

2.hot-unplug the middle of this three console:

# cat virtioconsole.xml
    <console type='pty'>
      <source path='/dev/pts/19'/>
      <target type='virtio' port='1'/>
      <alias name='console1'/>
    </console>

# virsh detach-device test3 virtioconsole.xml
Device detached successfully

3.recheck the xml in source:

    <console type='pty' tty='/dev/pts/18'>
      <source path='/dev/pts/18'/>
      <target type='virtio' port='0'/>
      <alias name='console0'/>
    </console>
    <console type='pty'>
      <source path='/dev/pts/20'/>
      <target type='virtio' port='2'/>
      <alias name='console2'/>
    </console>

4.migrate to target:
# virsh migrate test3 qemu+ssh://10.66.6.12/system --live
error: Domain not found: no domain with matching name 'test3'

5.check the log in target:

2015-03-04 09:27:36.901+0000: 19330: error : qemuMonitorIO:701 : internal error: early end of file from monitor: possible problem:
qemu: warning: error while loading state for instance 0x0 of device '0000:00:05.0/virtio-console'
2015-03-04T09:27:36.862587Z qemu-kvm: load of migration failed: Invalid argument

6.check the xml which use to generate qemu process in target (before qemu died):

    <console type='pty' tty='/dev/pts/2'>
      <source path='/dev/pts/2'/>
      <target type='virtio' port='0'/>
      <alias name='console0'/>
    </console>
    <console type='pty'>
      <source path='/dev/pts/3'/>
      <target type='virtio' port='1'/>
      <alias name='console1'/>
    </console>


Actual results:
migrate will fail when migrate a vm after hot-unplug a console and the error should be improved

and test if we hotunplug the last virtio console won't meet this issue.

Expected results:
1. in step 4 error message should be improved and we can catch the log from qemu log
2. migrate should success


infomation:

Comment 1 Jiri Denemark 2015-03-06 08:30:33 UTC
Yes, the error message should be improved, but we already track that in bug 1090093.

So we only need to fix the issue with chardev unplug. It seems we remove the device from domain definition even though QEMU still thinks the device is present, which should never happen since we started using device-detached events.

Comment 2 Ján Tomko 2015-03-06 11:44:40 UTC
Libvirt does receive the event when detaching the device:
2015-03-06 11:33:04.122+0000: 11977: debug : qemuMonitorJSONIOProcessLine:183 : Line [{"timestamp": {"seconds": 1425641584, "microseconds": 122228}, "event": "DEVICE_DELETED", "data": {"device": "console1", "path": "/machine/peripheral/console1"}}]

But it does not seem to be removed completely, beacuse the destination QEMU still fails:

qemu: warning: error while loading state for instance 0x0 of device '0000:00:08.0/virtio-console'
2015-03-06T11:34:48.057625Z qemu-kvm: load of migration failed: Invalid argument


If I unplug console #2, then console #3 (leaving just console #1), migration works.

Comment 3 Amit Shah 2015-04-02 09:21:24 UTC
We don't recommend using virtio-console; and we surely don't want people running multiple virtio-consoles.

Does this reproduce even with virtio-serial channels?

Is this a regression?  Can you please tell which was the last version that worked fine?

Comment 4 Luyao Huang 2015-04-02 09:54:27 UTC
(In reply to Amit Shah from comment #3)
> We don't recommend using virtio-console; and we surely don't want people
> running multiple virtio-consoles.
> 
> Does this reproduce even with virtio-serial channels?
> 

Cannot reproduce with virtio-serial channels.

> Is this a regression?  Can you please tell which was the last version that
> worked fine?

test with rhel7.0 qemu-kvm-rhev still can meet this issue.

I will test with qemu-kvm tomorrow.

Comment 5 Luyao Huang 2015-04-03 08:18:17 UTC
(In reply to Luyao Huang from comment #4)
> (In reply to Amit Shah from comment #3)
> > We don't recommend using virtio-console; and we surely don't want people
> > running multiple virtio-consoles.
> > 
> > Does this reproduce even with virtio-serial channels?
> > 
> 
> Cannot reproduce with virtio-serial channels.
> 
> > Is this a regression?  Can you please tell which was the last version that
> > worked fine?
> 
> test with rhel7.0 qemu-kvm-rhev still can meet this issue.
> 
> I will test with qemu-kvm tomorrow.

Test with qemu-kvm-1.5.3-87.el7 and qemu-kvm-1.5.3-60.el7_0.11.x86_64, still found this issue. So i don't think it is a regression.

Comment 6 Amit Shah 2015-04-21 06:46:38 UTC
Since virtio-serial ports don't exhibit this behaviour, and only virtio-console ports do, this has to be something the guest does with console ports.  We don't recommend testing more than one console port anyway, and console ports are a low priority.

So unless this is breaking something important, it will be closed wontfix.

Comment 7 Luyao Huang 2015-04-21 07:00:33 UTC
(In reply to Amit Shah from comment #6)
> Since virtio-serial ports don't exhibit this behaviour, and only
> virtio-console ports do, this has to be something the guest does with
> console ports.  We don't recommend testing more than one console port
> anyway, and console ports are a low priority.
> 
> So unless this is breaking something important, it will be closed wontfix.

Okay, this issue was found during test some corner case, so it is not a big issue and i think we can close this bug.

Comment 8 Amit Shah 2015-04-21 07:13:24 UTC
Thanks.


Note You need to log in before you can comment on or make changes to this bug.