Bug 1651348

Well, I haven't hit this problem, although I have hit a bunch of other ones. I'm on a mustang now with kernel 4.20.0-0.rc5.git2, qemu 3.0.91 (qemu-3.1.0-0.1.rc1.fc30).

I'm on a b0 though, I wonder if that has something to do with it? There are a couple problems with the A3's (AFAIK, and I don't think fedora is carrying any of the workarounds). Let me see if I can reproduce it on an older mustang.

Actually, I have reproduced it on the mustang now.

So, this is almost assuredly caused by https://github.com/qemu/qemu/commit/0f8d06f16c9d1041d728d09d464462ebe713c662.

Which is odd, as I've been struggling to understand how it works, given that on all the machines I've tested aa64pfr0_el1 is being trapped by the kernel and emulated to userspace, with the single detail that its _NOT_ indicating aarch32 support at any exception levels. That is because the kernel is sanitizing the feature registers, but because its not marked FTR_VISIBLE the default value being exported is aarch64 only support.

Well, that isn't happening here:

(gdb) fin
Run till exit from #0  kvm_arm_get_host_cpu_features (
    ahcf=ahcf@entry=0xaaaaab804870 <arm_host_cpu_features>)
    at /home/rth/qemu/qemu/target/arm/kvm64.c:483
0x0000aaaaaaec76b8 in kvm_arm_set_cpu_features_from_host (cpu=0xffffbc529010)
    at /home/rth/qemu/qemu/target/arm/kvm.c:155
155	            return;
Value returned is $3 = true
(gdb) p/x arm_host_cpu_features.isar.id_aa64pfr0
$4 = 0x122

However, 

$ grep VERSION /etc/os-release 
VERSION="18.10 (Cosmic Cuttlefish)"
$ uname -a
Linux chuckanut 4.18.0-11-generic #12-Ubuntu SMP Tue Oct 23 19:24:51 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
$ head -8 /proc/cpuinfo 
processor	: 0
BogoMIPS	: 100.00
Features	: fp asimd evtstrm cpuid
CPU implementer	: 0x50
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 1

I'll see about trying a kernel newer than 4.18, and see if that's where the change came in.

We have missed the boat for fixing this for the qemu-3.1 release.  Which is annoying.
This should probably get an upstream launchpad bug report too.

Hmm (i'm not a qemu expert), so some of what this was based on was dumping the values from arm_cpu_realizefn, where it turns out ARM_FEATURE_AARCH64 isn't even set because -cpu host,aarch64=off disables it. So that code isn't even executing!

Hence when I saw that cpu_isar_feature equal to 0, I assumed that was because id_aa64pfr0 was 0x11, which is what you get if its read from userspace due to the HIDDEN/STRICT settings for the emulation. But dumping it, its also, so right before the assert() we have:

arm_feature(ARM_FEATURE_AARCH64)=0
cpu_isar_feature = 0
id_aa64pfr0=0

if dumped like:

"arm_feature(ARM_FEATURE_AARCH64)=%d cpu_isar_feature = %d , id_aa64pfr0=%lX\n",arm_feature(&cpu->env, ARM_FEATURE_AARCH64), cpu_isar_feature(aa64_aa32, cpu), cpu->isar.id_aa64pfr0

and run with qemu-system-aarch64 machine virt-3.1,accel=kvm -cpu host,aarch64=off

Ok, knowing the command line options helps some,
but I still don't replicate either the assert or
what you're seeing.


(gdb) run -machine virt-3.1,accel=kvm -cpu host,aarch64=off
Thread 1 "qemu-system-aar" hit Breakpoint 2, arm_cpu_realizefn (
    dev=0xffffbc256010, errp=0xffffffffe670)
    at /home/rth/qemu/qemu/target/arm/cpu.c:906
906	        assert(no_aa32 || cpu_isar_feature(arm_div, cpu));
(gdb) p/x cpu->isar
$3 = {id_isar0 = 0x2101110, id_isar1 = 0x13112111, id_isar2 = 0x21232042, 
  id_isar3 = 0x1112131, id_isar4 = 0x10142, id_isar5 = 0x1, id_isar6 = 0x0, 
  mvfr0 = 0x10110222, mvfr1 = 0x12111111, mvfr2 = 0x43, id_aa64isar0 = 0x0, 
  id_aa64isar1 = 0x0, id_aa64pfr0 = 0x122, id_aa64pfr1 = 0x0}
(gdb) call isar_feature_arm_div(&cpu->isar)
$4 = true


The contents of id_aa64pfr0 should be completely ignored for aarch64=off.

I do still see kvm_arm_set_cpu_features_from_host being called in
order to grab the other id registers to fill in -cpu host.

Ok, I've been really confused about where id_aa6iasr0 has been comming from, and know I know whats going on (part of the clue was your line numbers/printing).

Upstream qemu head has a bunch of code to set the isar registers that (AFAIK) is missing from the release fedora is running. Looks like your commit was incompletely backported?

OMG, now I feel foolish.  The clue was right here:

qemu-system-aarch64-3.1.0-0.1.rc1.fc30.aarch64
                              ^^^

The fix for the bug that you're seeing was included in 3.1.0-rc2.
The final 3.1.0 release came out today, fwiw.

I should have probably pulled qemu forward earlier to see if the problem exists. Anyway, it seems that its using KVM_GET_ONE_REG which returns the correct value for aa64pfr0 too.

3.1.0 GA is built for rawhide now