Bug 1352556

viostor test plan - all good. 

Sign-off.
Vadim.

However I have a couple of small notes:
1. "2 Driver Mapping Table" - in 7.3 we should have Win10 directory for
    Win10 & WS2016 drivers.
2. "5 Function Test Scenarios - "scsi=off|on" flag is almost irrelevant for Windows, since Windows virtio-blk driver doesn't honour this flag.
3. "5 Function Test Scenarios - some Hot plug\unplug tests will fail if we go through hibernate\resume cycles. It's a known problem.

(In reply to Vadim Rozenfeld from comment #2)
> viostor test plan - all good. 
> 
> Sign-off.
> Vadim.
> 
> However I have a couple of small notes:
> 1. "2 Driver Mapping Table" - in 7.3 we should have Win10 directory for
>     Win10 & WS2016 drivers.
> 2. "5 Function Test Scenarios - "scsi=off|on" flag is almost irrelevant for
> Windows, since Windows virtio-blk driver doesn't honour this flag.
> 3. "5 Function Test Scenarios - some Hot plug\unplug tests will fail if we
> go through hibernate\resume cycles. It's a known problem.

Hi Vadim,

Thanks for your reply, I will modify the test plan according to your notes.

BR
Yu Wang

vioscsi test plan - looks good.

Sign-off.
Vadim.

Just like in the viostor case, Win10 directory should be used for installing drivers on WS2016.

Best regards,
Vadim.

Hi Yan,

Could you please review the test plan for netkvm?

test plan refer to the attachment.


Thanks
Yu Wang

Both balloon and virtio-serial tests are fine.

Sign-off.
Vadim.

Some thoughts regarding to balloon and vioserial functional testing. It is worth
running some of the functional tests (like balloon inflating/deflating and disable/enable cycles under heavy load or memory overcommitment condition.
The same also true for vioserial as well, except for that in this case we should 
check read/write instead of inflating/deflating).

Both balloon and virtio-serial tests are fine.

Sign-off.
Vadim.

Some thoughts regarding to balloon and vioserial functional testing. It is worth
running some of the functional tests (like balloon inflating/deflating and disable/enable cycles under heavy load or memory overcommitment condition.
The same also true for vioserial as well, except for that in this case we should 
check read/write instead of inflating/deflating).

In general looks OK. I have several comments\questions:

1. Vadim - will we have binary for Windows 10 only (due to signature)? In this case driver map should change. And we will have Win10 directory for Windows 10 and Server 2016.

2. We need to test the device\driver with different configurations:

* Muti-queue \ Single queue

* Legacy virtio \ virtio 1.0 - this variation relates to all drivers, not only networking.

(In reply to Yan Vugenfirer from comment #8)
> In general looks OK. I have several comments\questions:
> 
> 1. Vadim - will we have binary for Windows 10 only (due to signature)? In
> this case driver map should change. And we will have Win10 directory for
> Windows 10 and Server 2016.

Good point. Current only virtio-input, balloon, virtio-serial, virtio-blk and virtio-scsi have Server10_x64 OS mask specified. I will check and fix the rest.
  
> 
> 2. We need to test the device\driver with different configurations:
> 
> * Muti-queue \ Single queue
> 
> * Legacy virtio \ virtio 1.0 - this variation relates to all drivers, not
> only networking.

(In reply to Yan Vugenfirer from comment #8)
> In general looks OK. I have several comments\questions:
> 
> 1. Vadim - will we have binary for Windows 10 only (due to signature)? In
> this case driver map should change. And we will have Win10 directory for
> Windows 10 and Server 2016.
> 
> 2. We need to test the device\driver with different configurations:
> 
> * Muti-queue \ Single queue
> 
> * Legacy virtio \ virtio 1.0 - this variation relates to all drivers, not
> only networking.

Yes, we tested with muti-queue and single queue for different guest OS (generally random test), the same test matrix as virtio/virtio1.0. And we will add virtio/virtio 1.0 to all the virtio drivers.


Thanks 
Yu Wang

(In reply to Yu Wang from comment #10)
> (In reply to Yan Vugenfirer from comment #8)
> > In general looks OK. I have several comments\questions:
> > 
> > 1. Vadim - will we have binary for Windows 10 only (due to signature)? In
> > this case driver map should change. And we will have Win10 directory for
> > Windows 10 and Server 2016.
> > 
> > 2. We need to test the device\driver with different configurations:
> > 
> > * Muti-queue \ Single queue
> > 
> > * Legacy virtio \ virtio 1.0 - this variation relates to all drivers, not
> > only networking.
> 
> Yes, we tested with muti-queue and single queue for different guest OS
> (generally random test), the same test matrix as virtio/virtio1.0. And we
> will add virtio/virtio 1.0 to all the virtio drivers.
> 
> 
> Thanks 
> Yu Wang

Then the test plan is good from my point of view.

Sign-off.

Hi Gal,

Could you please review the test plan for viorng and pvpanic?

Thanks
Yu Wang

PVPanic:

- Does the driver mapping for Win8 is current? I thought Win2016 should be Win10.

RNG:

- Driver mapping table is missing Win10.
- "[virtio-win][viorng] assign illegal values to viorng parameter" does it belong to the driver tests? It is handled by QEMU.

Other than these looks okay. Thanks.

(In reply to Gal Hammer from comment #13)
> PVPanic:
> 
> - Does the driver mapping for Win8 is current? I thought Win2016 should be
> Win10.

QE have updated this to win10 on our side, thanks for reminder.

> 
> RNG:
> 
> - Driver mapping table is missing Win10.
> - "[virtio-win][viorng] assign illegal values to viorng parameter" does it
> belong to the driver tests? It is handled by QEMU.

Okey, QE will remove this case, and change driver mapping table.

> 
> Other than these looks okay. Thanks.

(In reply to Vadim Rozenfeld from comment #6)
> Both balloon and virtio-serial tests are fine.
> 
> Sign-off.
> Vadim.
> 
> Some thoughts regarding to balloon and vioserial functional testing. It is
> worth
> running some of the functional tests (like balloon inflating/deflating and
> disable/enable cycles under heavy load or memory overcommitment condition.

for balloon inflating/deflating, we have case called "Enlarge/Evict guest's memory", is that satisfy your requirement?

for disable/enable cycles under heavy load or memory overcommitment condition , we will add a case like"disable/enable balloon in a loop during guest play videos." Is that ok?

> The same also true for vioserial as well, except for that in this case we
> should 
> check read/write instead of inflating/deflating).

for serial, we have a case like  "transfer 4M file via serial port", and will add "disable/enable during transfer 4M file via serial port" , is that ok? or any other suggestion?

(In reply to Yu Wang from comment #15)
> (In reply to Vadim Rozenfeld from comment #6)
> > Both balloon and virtio-serial tests are fine.
> > 
> > Sign-off.
> > Vadim.
> > 
> > Some thoughts regarding to balloon and vioserial functional testing. It is
> > worth
> > running some of the functional tests (like balloon inflating/deflating and
> > disable/enable cycles under heavy load or memory overcommitment condition.
> 
> for balloon inflating/deflating, we have case called "Enlarge/Evict guest's
> memory", is that satisfy your requirement?
> 
> for disable/enable cycles under heavy load or memory overcommitment
> condition , we will add a case like"disable/enable balloon in a loop during
> guest play videos." Is that ok?
> 
> > The same also true for vioserial as well, except for that in this case we
> > should 
> > check read/write instead of inflating/deflating).
> 
> for serial, we have a case like  "transfer 4M file via serial port", and
> will add "disable/enable during transfer 4M file via serial port" , is that
> ok? or any other suggestion?

All good.
Thanks,
Vadim.

Sorry, this fell off my radar.

In comment 2 (viostor plan), Vadim writes:
>"2 Driver Mapping Table" - in 7.3 we should have Win10 directory for
>   Win10 & WS2016 drivers.

In comment 4 (vioscsi plan), Vadim writes:
>Just like in the viostor case, Win10 directory should be used for installing >drivers on WS2016.

To make it easy for RHEV and RHOS, the virtio-win rpm packing scripts will put the windows 10 drivers in two directories, one for Win10 and one for WS2016. The files in each directory will be identical (hard links). RHEV actually takes the virtio-win rpm and uses it as input to their own packaging algorithms. The algorithms for determining which drivers to install are keyed off the OS.

The additional WS2016 directory does not invalidate anything in the test plan or the earlier feedback from Vadim. However, QE may want to confirm that the files in the two directories are identical.

P.S. This assumes that the signed drivers returned by the Microsoft signing process actually makes a distinction between Win10 and WS2016. Since I haven't yet seen a signed driver, I don't know what it will look like. If it does not report WS2016 as a separate OS, then my comments above don't apply. In any case, there is still no impact to the test plan.

(In reply to Jeff Nelson from comment #17)
> Sorry, this fell off my radar.
> 
> In comment 2 (viostor plan), Vadim writes:
> >"2 Driver Mapping Table" - in 7.3 we should have Win10 directory for
> >   Win10 & WS2016 drivers.
> 
> In comment 4 (vioscsi plan), Vadim writes:
> >Just like in the viostor case, Win10 directory should be used for installing >drivers on WS2016.
> 
> To make it easy for RHEV and RHOS, the virtio-win rpm packing scripts will
> put the windows 10 drivers in two directories, one for Win10 and one for
> WS2016. The files in each directory will be identical (hard links). RHEV
> actually takes the virtio-win rpm and uses it as input to their own
> packaging algorithms. The algorithms for determining which drivers to
> install are keyed off the OS.
> 
> The additional WS2016 directory does not invalidate anything in the test
> plan or the earlier feedback from Vadim. However, QE may want to confirm
> that the files in the two directories are identical.
> 
> P.S. This assumes that the signed drivers returned by the Microsoft signing
> process actually makes a distinction between Win10 and WS2016. Since I
> haven't yet seen a signed driver, I don't know what it will look like. If it
> does not report WS2016 as a separate OS, then my comments above don't apply.
> In any case, there is still no impact to the test plan.

That's all good. When speaking about using drivers from Win10 directory for both Win10 and WS2016 platforms I was just referencing to pre-whql build where we usually give names to subdirectories after workstation platforms (Wnet and Wlh is the only exception from this rule). But we definitely need to maintain two different directories in RPM - one for Win10, and another one for WS2016.

Vadim.

Speaking of driver mapping tables, why is vioscsi\Install\Win7 still unused? The table lists vioscsi\Install\Wlh as the version that should be installed on Win7 and 2008R2. Same question raised in BZ #1325078, comments 12 and 13.

(In reply to Ladi Prosek from comment #19)
> Speaking of driver mapping tables, why is vioscsi\Install\Win7 still unused?
> The table lists vioscsi\Install\Wlh as the version that should be installed
> on Win7 and 2008R2. Same question raised in BZ #1325078, comments 12 and 13.

Good catch. You are absolutely right. Before 7,3 there were no difference between Wlh and Win7 (as well as Win8) versions of virtio-scsi driver. Nowadays, with introducing multiqueue support, the situation has been changed toward spiting one single version suitable for Win7 and Wlh into two different drivers.

(In reply to Vadim Rozenfeld from comment #20)
> (In reply to Ladi Prosek from comment #19)
> > Speaking of driver mapping tables, why is vioscsi\Install\Win7 still unused?
> > The table lists vioscsi\Install\Wlh as the version that should be installed
> > on Win7 and 2008R2. Same question raised in BZ #1325078, comments 12 and 13.
> 
> Good catch. You are absolutely right. Before 7,3 there were no difference
> between Wlh and Win7 (as well as Win8) versions of virtio-scsi driver.
> Nowadays, with introducing multiqueue support, the situation has been
> changed toward spiting one single version suitable for Win7 and Wlh into two
> different drivers.

Thanks, this change does not affect only the test plan though. We need to make sure that the rpm's are packaged correctly. Right now Win2008/vioscsi.sys, Win2008R2/vioscsi.sys, and Win7/vioscsi.sys are all hardlinks to the same file. Should I open a BZ to track it? Jeff?

Also, vioscsi is not the only such driver. Any reason why we shouldn't fix viostor, vioserial, and balloon as well?

change status to verified as all test plan are signed-off