Bug 1290802 - Automatic test plan scheduling for WirelessX tests fail on Intel 8260ac WLAN card
Automatic test plan scheduling for WirelessX tests fail on Intel 8260ac WLAN ...
Status: CLOSED NOTABUG
Product: Red Hat Certification Program
Classification: Red Hat
Component: redhat-certification-hardware (Show other bugs)
1.0
x86_64 Linux
high Severity high
: ---
: ---
Assigned To: brose
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-11 08:46 EST by Rainer Koenig
Modified: 2017-04-18 21:24 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-04-18 21:24:15 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Outputs of the WirelessAC test (34.69 KB, text/plain)
2015-12-11 08:46 EST, Rainer Koenig
no flags Details

  None (edit)
Description Rainer Koenig 2015-12-11 08:46:45 EST
Created attachment 1104689 [details]
Outputs of the WirelessAC test

Description of problem:
Currently doing a hardware certification, see 
https://hardware.redhat.com/show.cgi?id=1283991 
for details. 

The problem is that for the tests on an Intel 8260ac WLAN adapter the automatic schelduling of the Wireless test creates a testplan entry for "WirelessG".

Version-Release number of selected component (if applicable):
redhat-certification-hardware-1.7.1-20151113.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1. Kickstart RHEL7.2 plus rhcert suite
2. Register System in Web UI 
3. Look at the test plan

Actual results:
For the wireless card a test "WirelessG" is selected.

Expected results:
The test suite should plan for a "WirelessAC" test.

Additional info:
The system is also connected via wired Gigabit Ehternet to the same subnet as via WLAN. The IP adsress for the connection between the test server and the system under test is the wired IP adsress. 

The reviewer pointed out in comment #34 
https://hardware.redhat.com/show.cgi?id=1283991#c34
that the schedule is based on the output of the
command 

iw <interfacename> link

and following this algorithm based on the tx-bitrate reported:

if speedValue < 11.1:
                    return "b"
                elif speedValue < 54.1:
                    return "g"
                elif speedValue < 300.0:
                    return "n"
                else:
                    return "ac"

And yes, the bitrates I saw were all lower than what was necessary for scheduling an "ac" test.

So I got really so much desperate that I installed Windows 10 on that machine to see how this engineering sample of the 8260ac card behaves on our competitiors operating system. Well, here I get a connection info of a bit above 800 Mbit/sec. So I tried a download of the RHEL 7.2 iso (3.8 GBytes) from my server and it took around 80 seconds.

Then I switched back to a fresh new RHEL 7.2 setup. And I tried the same download on RHEL 7.2 and I got a download time of around 60 seconds. So RHEL 7.2 seems to be as fast as Windows, even faster. This was the first WTF moment.

After the download via wget was done I did an "iw wlp1s0 link" and got an even bigger surprise:

[root@localhost tmp]# iw wlp1s0 link
Connected to f0:79:59:cf:6d:14 (on wlp1s0)
	SSID: LinuxTest5G
	freq: 5180
	RX: 33183134 bytes (2792509 packets)
	TX: 11547993 bytes (134220 packets)
	signal: -30 dBm
	tx bitrate: 866.7 MBit/s VHT-MCS 9 80MHz short GI VHT-NSS 2

	bss flags:	short-slot-time
	dtim period:	3
	beacon int:	100

Wow. 866.7 Mbit/s. That's a number sufficient to schedule a "WirelessAC" test. 

So let me see that again. The same "iw wlp1s0 link" command issued a few seconds later
gave the following ouput:

[root@localhost ~]# iw wlp1s0 link
Connected to f0:79:59:cf:6d:14 (on wlp1s0)
	SSID: LinuxTest5G
	freq: 5180
	RX: 36574837 bytes (2795133 packets)
	TX: 11614513 bytes (134777 packets)
	signal: -27 dBm
	tx bitrate: 6.0 MBit/s

	bss flags:	short-slot-time
	dtim period:	3
	beacon int:	100

Oops. tx bitrate at 6.0 Mbit. 

Second WTF moment today.

But what if....
...the adapter is very very smart and adjusts the bitrate according to the traffic? When there is no WLAN traffic there is usally no need to maintain full speed. 

I tried again to schedule a test, but now I did the registration of the system in the GUI while the download of the RHEL 7.2 ISO image was on the way. 

As a result now the tset suite schedules for a "WirelessAC" test. I performed this test. It looks like "PASS" except that in the runtime log it looks like the test is looking at the tx-bitrate again after all the traffic is done and deciding, that the device doesn't support 802.11ac. 

BTW: When looking at the test results in the GUI, the WirelessAC test completly disappeared. But in the results on the test system its still in /var/log/rhcert/... and I can download the results and submit it to the certification. 

So final conclusion from my side:
a) automatic scheduling for the Intel 8260ac adapter fails because without workload it goes down to low bitrates 
b) WirelessAC test fails as well because it looks at the link speed in a low traffic phase of the test and gets a too low result for 802.11ac.
Comment 1 Christian Horn 2015-12-11 10:18:44 EST
@Rainer, thanks a lot for looking so deep into this.
Idea for a workaround would be to have a process fetch data in a loop, while the wireless test is running.  So also when it finishes and in the lest test did fall back to 6mbit, it would stay up.

Changing the rate is default procedure, I think, have seen that here on the laptop often.  Can possibly disabled as driver option.

This is blocking a certification now.
Has AC-wireless been subject to hwcert suite testing?
Have we already certified wireless cards on rhel7.2?
Comment 2 brose 2016-02-01 13:34:13 EST
We will be releasing a change to detection/planning of 802.11 interfaces which does not rely on the speed.

The test itself has not been changed.

Is Christian's idea of turning off the fluctuations feasible for the purposes of certification?
Comment 3 Rainer Koenig 2016-02-02 01:50:46 EST
From my point of view everything is fine as long as we find a way to pass the certification tests. I really don't mind to "hack" a bit the environment so that the test is fine.

Another thing to think about is how Wireless<X> tests match to the testplan in the hardware catalog. If the testplan shows WLAN and the results are WirelessG or WirelessAC then I can't match the results to the item in the test plan.

Note You need to log in before you can comment on or make changes to this bug.