Bug 2271375 - Performance tests fail
Summary: Performance tests fail
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 40
Hardware: aarch64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-03-25 10:53 UTC by Stanislav Graf
Modified: 2025-05-16 08:00 UTC (History)
17 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2025-05-16 08:00:50 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Stanislav Graf 2024-03-25 10:53:02 UTC
1. Please describe the problem:

Latest 6.8.1 kernel works great, no issues, except of performance test suite. I assume this is a bug within the suite.

$ sudo ./runtests.sh -t performance
Test suite called with performance
gmake[2]: Warning: File '../bin/aarch64-linux-gnu/lib_tcp.o' has modification time 307156 s in the future
lib_timing.c: In function ‘touch’:
lib_timing.c:1626:17: error: type defaults to ‘int’ in declaration of ‘psize’ [-Wimplicit-int]
 1626 |         static  psize;
      |                 ^~~~~
gmake[2]: *** [Makefile:240: ../bin/aarch64-linux-gnu/lib_timing.o] Error 1
make[1]: *** [Makefile:114: lmbench] Error 2
make: *** [Makefile:20: build] Error 2
gmake[2]: Warning: File '../bin/aarch64-linux-gnu/lib_tcp.o' has modification time 307156 s in the future
lib_timing.c: In function ‘touch’:
lib_timing.c:1626:17: error: type defaults to ‘int’ in declaration of ‘psize’ [-Wimplicit-int]
 1626 |         static  psize;
      |                 ^~~~~
gmake[2]: *** [Makefile:240: ../bin/aarch64-linux-gnu/lib_timing.o] Error 1
make[1]: *** [Makefile:114: lmbench] Error 2
make: *** [Makefile:23: results] Error 2
./performance/lmbench3                                           PASS    

Test suite complete                                              PASS    

Your log file is being submitted...
{"message":"Upload successful!"}
The following information is not submitted with your log;
it is for informational purposes only.
Vulnerability status:
/sys/devices/system/cpu/vulnerabilities/gather_data_sampling:Not affected
/sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected
/sys/devices/system/cpu/vulnerabilities/l1tf:Not affected
/sys/devices/system/cpu/vulnerabilities/mds:Not affected
/sys/devices/system/cpu/vulnerabilities/meltdown:Not affected
/sys/devices/system/cpu/vulnerabilities/mmio_stale_data:Not affected
/sys/devices/system/cpu/vulnerabilities/reg_file_data_sampling:Not affected
/sys/devices/system/cpu/vulnerabilities/retbleed:Not affected
/sys/devices/system/cpu/vulnerabilities/spec_rstack_overflow:Not affected
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user pointer sanitization
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Not affected
/sys/devices/system/cpu/vulnerabilities/srbds:Not affected
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected


2. What is the Version-Release number of the kernel:
6.8.1-300.fc40.aarch64

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :
Don't know

4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:
Yes, 100% reproducibility with ./runtests.sh -t performance

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:
yes, looks like the same error
6.9.0-0.rc0.20240322git8e938e398669.14.fc41.aarch64

6. Are you running any modules that not shipped with directly Fedora's kernel?:
No

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.
command generates empty file

Reproducible: Always

Comment 1 itrymybest80 2024-06-02 14:21:00 UTC
Test suite called with performance
lib_timing.c: I funktion ”touch”:
lib_timing.c:1626:17: fel: typen sätts skönsmässigt till ”int” i deklarationen av ”psize” [-Wimplicit-int]
 1626 |         static  psize;
      |                 ^~~~~
gmake[2]: *** [Makefile:240: ../bin/x86_64-linux-gnu/lib_timing.o] Fel 1
make[1]: *** [Makefile:114: lmbench] Fel 2
make: *** [Makefile:20: build] Fel 2
lib_timing.c: I funktion ”touch”:
lib_timing.c:1626:17: fel: typen sätts skönsmässigt till ”int” i deklarationen av ”psize” [-Wimplicit-int]
 1626 |         static  psize;
      |                 ^~~~~
gmake[2]: *** [Makefile:240: ../bin/x86_64-linux-gnu/lib_timing.o] Fel 1
make[1]: *** [Makefile:114: lmbench] Fel 2
make: *** [Makefile:23: results] Fel 2
./performance/lmbench3                                           PASS    

Test suite complete                                              PASS

kernel-6.8.12-300.fc40.x86_64 [error]
kernel-6.9.3-200.fc40.x86_64 [error]
kernel-6.10.0-0.rc1.20240531git4a4be1ad3a6e.21.fc41.x86_64 [error]

Default performance test(s) pass, but the performance test(s) will always end prematurely.

Comment 2 Gilbert Fernandes 2024-06-18 18:42:36 UTC
I am sorry to report that I have been able to run that performance suite without error.
Fedora 40, up to date.
Also checked my test suite is up to date using git pull --rebase :

commit 6ca1b0d46fa3ece0a8b24bad49ef83da531576a0 (HEAD -> master, origin/master, origin/HEAD)
Author: Justin M. Forbes <jforbes>
Date:   Thu Mar 21 15:29:52 2024 -0500

    Update the anonymous commithook for the new kerneltest update
    
    Signed-off-by: Justin M. Forbes <jforbes>

I did run the test exactly like you did :

gf@aesir:~/kernel-test$ sudo ./runtests.sh -t performance
[sudo] password for gf: 
Test suite called with performance
Using config in CONFIG.aesir
Tue Jun 18 08:14:48 PM CEST 2024
Latency measurements
Tue Jun 18 08:16:39 PM CEST 2024
Calculating file system latency
Tue Jun 18 08:16:41 PM CEST 2024
Local networking
Tue Jun 18 08:20:23 PM CEST 2024
Bandwidth measurements
Tue Jun 18 08:38:28 PM CEST 2024
Calculating context switch overhead
Tue Jun 18 08:38:32 PM CEST 2024
./performance/lmbench3                                           PASS    

Test suite complete                                              PASS    

Your log file is being submitted...
fedora_submit.py -u <fasuser> [-p <password>] -l <logfile>
The following information is not submitted with your log;
it is for informational purposes only.
Vulnerability status:
/sys/devices/system/cpu/vulnerabilities/gather_data_sampling:Not affected
/sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected
/sys/devices/system/cpu/vulnerabilities/l1tf:Not affected
/sys/devices/system/cpu/vulnerabilities/mds:Not affected
/sys/devices/system/cpu/vulnerabilities/meltdown:Not affected
/sys/devices/system/cpu/vulnerabilities/mmio_stale_data:Not affected
/sys/devices/system/cpu/vulnerabilities/reg_file_data_sampling:Not affected
/sys/devices/system/cpu/vulnerabilities/retbleed:Not affected
/sys/devices/system/cpu/vulnerabilities/spec_rstack_overflow:Vulnerable: Safe RET, no microcode
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Mitigation: Speculative Store Bypass disabled via prctl
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: usercopy/swapgs barriers and __user pointer sanitization
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected
/sys/devices/system/cpu/vulnerabilities/srbds:Not affected
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected

No error did occur.
I did saw the RAM used go up to almost 50 Gb (of the 64 Gb) and the CPUs did have to work a little with the fan to get until the end.

But no error.

Linux aesir 6.9.4-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jun 12 13:33:34 UTC 2024 x86_64 GNU/Linux

Comment 3 itrymybest80 2024-07-03 23:32:17 UTC
https://pagure.io/kernel-tests/issue/53 not sure if related or not, but it would be nice to complete your "performance test" again; apologies if it turns out to be off-topic.

Comment 4 Gilbert Fernandes 2024-07-04 08:20:19 UTC
To answer the "Did it work previously in Fedora?" part :

I did grab the changes to the git repository. And ran the standard then performance test again.
I do not see errors on the Fedora 41 + all updates available applied through dnf.

Linux aesir 6.9.7-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jun 27 18:11:45 UTC 2024 x86_64 GNU/Linux
NAME="Fedora Linux"
VERSION="40 (Workstation Edition)"

performance test :

gf@aesir:~/kernel-test$ sudo ./runtests.sh -t performance
[sudo] password for gf: 
Test suite called with performance
Using config in CONFIG.aesir
Thu Jul  4 09:49:56 AM CEST 2024
Latency measurements
Thu Jul  4 09:51:53 AM CEST 2024
Calculating file system latency
Thu Jul  4 09:51:55 AM CEST 2024
Local networking
Thu Jul  4 09:55:35 AM CEST 2024
Bandwidth measurements
Thu Jul  4 10:14:01 AM CEST 2024
Calculating context switch overhead
Thu Jul  4 10:14:05 AM CEST 2024
./performance/lmbench3                                           PASS    

Test suite complete                                              PASS    

Your log file is being submitted...
/home/gf/kernel-test/logs/kernel-test-1720079395.log.txt
Please visit https://id.fedoraproject.org/openidc/Authorization?scope=openid+email+profile+https%3A%2F%2Fid.fedoraproject.org%2Fscope%2Fgroups+https%3A%2F%2Fid.fedoraproject.org%2Fscope%2Fagreements+https%3A%2F%2Fid.fedoraproject.org%2Fscope%2Ffas-attributes&response_type=code&client_id=kerneltest-client&redirect_uri=[REDIRECT_URI]&response_mode=query to grant authorization
127.0.0.1 - - [04/Jul/2024 10:17:37] "GET /?code=[CODE] HTTP/1.1" 200 47
{"message":"Upload successful!"}

The following information is not submitted with your log;
it is for informational purposes only.
Vulnerability status:
/sys/devices/system/cpu/vulnerabilities/gather_data_sampling:Not affected
/sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected
/sys/devices/system/cpu/vulnerabilities/l1tf:Not affected
/sys/devices/system/cpu/vulnerabilities/mds:Not affected
/sys/devices/system/cpu/vulnerabilities/meltdown:Not affected
/sys/devices/system/cpu/vulnerabilities/mmio_stale_data:Not affected
/sys/devices/system/cpu/vulnerabilities/reg_file_data_sampling:Not affected
/sys/devices/system/cpu/vulnerabilities/retbleed:Not affected
/sys/devices/system/cpu/vulnerabilities/spec_rstack_overflow:Vulnerable: Safe RET, no microcode
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Mitigation: Speculative Store Bypass disabled via prctl
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: usercopy/swapgs barriers and __user pointer sanitization
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected
/sys/devices/system/cpu/vulnerabilities/srbds:Not affected
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected

No error.
The OpenID part worked flawlessly too.

Comment 5 itrymybest80 2024-08-06 14:45:14 UTC
I'm able to complete the performance test with this patch: https://pagure.io/kernel-tests/pull-request/54

Comment 6 Aoife Moloney 2025-04-25 10:22:30 UTC
This message is a reminder that Fedora Linux 40 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 40 on 2025-05-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '40'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 40 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 7 Stanislav Graf 2025-04-25 12:47:00 UTC
I haven't seen this issue when I participated in kernel test week for 6.13 or 6.14. Performance tests are now passing for me.

Comment 8 Aoife Moloney 2025-05-16 08:00:50 UTC
Fedora Linux 40 entered end-of-life (EOL) status on 2025-05-13.

Fedora Linux 40 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.