Bug 2044172
| Summary: | [RHEL9] Enable virtio-mem as tech-preview on ARM64 libvirt | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Guowen Shan <gshan> |
| Component: | libvirt | Assignee: | Michal Privoznik <mprivozn> |
| libvirt sub component: | General | QA Contact: | Virtualization Bugs <virt-bugs> |
| Status: | CLOSED CURRENTRELEASE | Docs Contact: | |
| Severity: | medium | ||
| Priority: | unspecified | CC: | dhildenb, eric.auger, gshan, lijin, mprivozn, mzhan, virt-maint, yalzhang, yihyu, zhenyzha |
| Version: | 9.1 | Keywords: | FutureFeature, TestOnly, Triaged |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | aarch64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-8.3.0-1.el9 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-02-16 22:21:09 UTC | Type: | Feature Request |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 2044162 | ||
| Bug Blocks: | 1924294, 2047797 | ||
|
Description
Guowen Shan
2022-01-24 07:28:16 UTC
This depends on the work to enable virtio-mem in QEMU, which is tracked by bug 2044162 Gavin, David, Michal, Is virtio-mem support in libvirt arch specific? Does ARM need any special treatment? If not, then maybe this BZ should be TestOnly? Good point. There's nothing arch specific in libvirt. And quick glance over qemu code does not show any arch specific signs either. Making this TestOnly then. At least nothing specific for virtio-mem-pci I think, so it should be fine for aarch64. Once we want to support s390x via virtio-mem-ccw, we'll need libvirt extensions. Michael, could you help to set ITM/DTM so that QA can schedule time to verify? The depedent patchsets have been merged. Bug 2044162 - [RHEL9.1] Enable virtio-mem as tech-preview on ARM64 QEMU (Fixed in qemu-kvm-7.0.0-2.el9) Bug 2044155 - [RHEL9.1] Enable virtio-mem as tech-preview on ARM64 kernel (Fixed in kernel-5.14.0-99.el9) Thanks, Gavin Perfect, so QEMU part is all done then. Since this is TestOnly bug, it can be switched to ON_QA. Let me do that. Env:
Host:
libvirt-8.3.0-1.el9.aarch64
qemu-kvm-7.0.0-4.el9.aarch64
kernel-5.14.0-101.el9.aarch64
Guest:
kernel-5.14.0-101.el9.aarch64
Preparation:
Add kernel option "memhp_default_state=online_movable"
to guest kernel line before test.
```
[root@localhost ~]# grubby --info=/boot/vmlinuz-5.14.0-101.el9.aarch64
index=0
kernel="/boot/vmlinuz-5.14.0-101.el9.aarch64"
args="ro console=tty0 console=ttyS0,115200 reboot=pci biosdevname=0 crashkernel=2G-:448M rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap memhp_default_state=online_movable"
root="/dev/mapper/rhel-root"
initrd="/boot/initramfs-5.14.0-101.el9.aarch64.img"
title="Red Hat Enterprise Linux (5.14.0-101.el9.aarch64) 9.1 (Plow)"
id="68e17a52352b4af48625845bda73c680-5.14.0-101.el9.aarch64"
```
Case 1. Basic test with virtio-mem devices
s1. Boot guest with two numa nodes and two virtio-mem devices:
```
<maxMemory slots='32' unit='KiB'>20971520</maxMemory>
<memory unit='KiB'>20971520</memory>
<currentMemory unit='KiB'>7340032</currentMemory>
<vcpu placement='static'>4</vcpu>
<numatune>
<memory mode='strict' nodeset='0-3'/>
</numatune>
---snip---
<cpu mode='host-passthrough' check='none'>
<topology sockets='1' dies='1' cores='4' threads='1'/>
<feature policy='require' name='sve'/>
<numa>
<cell id='0' cpus='0-1' memory='2097152' unit='KiB' memAccess='shared'/>
<cell id='1' cpus='2-3' memory='2097152' unit='KiB' memAccess='shared'/>
</numa>
</cpu>
---snip---
<devices>
---snip---
<memory model='virtio-mem'>
<target>
<size unit='KiB'>8388608</size>
<node>0</node>
<block unit='KiB'>2048</block>
<requested unit='KiB'>1048576</requested>
<current unit='KiB'>1048576</current>
</target>
<alias name='virtiomem0'/>
<address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
</memory>
<memory model='virtio-mem'>
<target>
<size unit='KiB'>8388608</size>
<node>1</node>
<block unit='KiB'>2048</block>
<requested unit='KiB'>2097152</requested>
<current unit='KiB'>2097152</current>
</target>
<alias name='virtiomem1'/>
<address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
</memory>
</devices>
```
s2. Start Guest and check qemu cmdline, memory-devices
```
-device {"driver":"virtio-mem-pci","node":0,"block-size":2097152,"requested-size":1073741824,"memdev":"memvirtiomem0","id":"virtiomem0","bus":"pci.8","addr":"0x0"} -object {"qom-type":"memory-backend-file","id":"memvirtiomem1","mem-path":"/dev/shm/libvirt/qemu/2-fj-kvm-vm/virtiomem1","share":true,"reserve":false,"size":8589934592,"host-nodes":[0,1,2,3],"policy":"bind"} -device {"driver":"virtio-mem-pci","node":1,"block-size":2097152,"requested-size":2147483648,"memdev":"memvirtiomem1","id":"virtiomem1","bus":"pci.9","addr":"0x0"}
```
```
# virsh qemu-monitor-command --hmp fj-kvm-vm info memory-devices
Memory device [virtio-mem]: "virtiomem0"
memaddr: 0x140000000
node: 0
requested-size: 1073741824
size: 1073741824
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem0
Memory device [virtio-mem]: "virtiomem1"
memaddr: 0x340000000
node: 1
requested-size: 2147483648
size: 2147483648
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem1
```
s3. Check guest numa info
```
# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1
node 0 size: 2996 MB
node 0 free: 2821 MB
node 1 cpus: 2 3
node 1 size: 3532 MB
node 1 free: 3330 MB
node distances:
node 0 1
0: 10 20
1: 20 10
```
s4. Resize virtio-mem, grow both to maximum
# virsh update-memory-device fj-kvm-vm --live --alias virtiomem0 --requested-size 8G
# virsh update-memory-device fj-kvm-vm --live --alias virtiomem1 --requested-size 8G
# virsh qemu-monitor-command --hmp fj-kvm-vm info memory-devices
Memory device [virtio-mem]: "virtiomem0"
memaddr: 0x140000000
node: 0
requested-size: 8589934592
size: 8589934592
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem0
Memory device [virtio-mem]: "virtiomem1"
memaddr: 0x340000000
node: 1
requested-size: 8589934592
size: 8589934592
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem1
s5. Check guest memoryinfo
[root@localhost ~]# free -h
total used free shared buff/cache available
Mem: 19Gi 481Mi 18Gi 8.0Mi 139Mi 18Gi
Swap: 1.0Gi 0B 1.0Gi
[root@localhost ~]# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1
node 0 size: 10164 MB
node 0 free: 9863 MB
node 1 cpus: 2 3
node 1 size: 9676 MB
node 1 free: 9356 MB
node distances:
node 0 1
0: 10 20
1: 20 10
s6. Use the hot added memory by 'numactl -m <NUMA NODE> <application>'
for two purposes: (a) the hot added memory can be accessed. (b)
the data on the hot added memory can be migrated in the scenario
of hot remove (shrink).
[root@localhost ~]# mkdir -p /tmp/numa_test
[root@localhost ~]# numactl -m 0 dd if=/dev/urandom of=/tmp/numa_test/test bs=1k count=5242880
5242880+0 records in
5242880+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 106.959 s, 50.2 MB/s
[root@localhost ~]# rm -rf /tmp/numa_test/*
[root@localhost ~]# numactl -m 1 dd if=/dev/urandom of=/tmp/numa_test/test bs=1k count=5242880
5242880+0 records in
5242880+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 98.4798 s, 54.5 MB/s
# shrink virt-mem device to 0
# virsh update-memory-device fj-kvm-vm --live --alias virtiomem0 --requested-size 0G
# virsh update-memory-device fj-kvm-vm --live --alias virtiomem1 --requested-size 0G
# virsh qemu-monitor-command --hmp fj-kvm-vm info memory-devices
Memory device [virtio-mem]: "virtiomem0"
memaddr: 0x140000000
node: 0
requested-size: 0
size: 0
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem0
Memory device [virtio-mem]: "virtiomem1"
memaddr: 0x340000000
node: 1
requested-size: 0
size: 0
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem1
root@localhost ~]# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1
node 0 size: 2026 MB
node 0 free: 1880 MB
node 1 cpus: 2 3
node 1 size: 1447 MB
node 1 free: 1288 MB
node distances:
node 0 1
0: 10 20
1: 20 10
[root@localhost ~]# free -h
total used free shared buff/cache available
Mem: 3.4Gi 189Mi 3.1Gi 8.0Mi 112Mi 3.0Gi
Swap: 1.0Gi 0B 1.0Gi
Case2. Test hotplug virtio-mem device
Preparation: Same as above but remove all virto-mem devices from guest.
s1. Start guest
s2. prepare 2 virtio-mem device xml
# cat virtio-mem0.xml
<memory model='virtio-mem'>
<target>
<size unit='KiB'>8388608</size>
<node>0</node>
<block unit='KiB'>2048</block>
<requested unit='KiB'>1048576</requested>
<current unit='KiB'>1048576</current>
</target>
<alias name='virtiomem0'/>
</memory>
# cat virtio-mem1.xml
<memory model='virtio-mem'>
<target>
<size unit='KiB'>8388608</size>
<node>1</node>
<block unit='KiB'>2048</block>
<requested unit='KiB'>1048576</requested>
<current unit='KiB'>1048576</current>
</target>
<alias name='virtiomem1'/>
</memory>
s3. Attach virtio-mem devices
# virsh attach-device fj-kvm-vm --file virtio-mem0.xml --persistent
Device attached successfully
# virsh attach-device fj-kvm-vm --file virtio-mem1.xml --persistent
Device attached successfully
# virsh qemu-monitor-command --hmp fj-kvm-vm info memory-devices
Memory device [virtio-mem]: "virtiomem0"
memaddr: 0x140000000
node: 0
requested-size: 1073741824
size: 1073741824
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem0
Memory device [virtio-mem]: "virtiomem1"
memaddr: 0x340000000
node: 1
requested-size: 1073741824
size: 1073741824
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem1
# guest dmesg info
---snip---
[ 496.605819] pcieport 0000:00:01.7: pciehp: Slot(0-7): Attention button pressed
[ 496.608639] pcieport 0000:00:01.7: pciehp: Slot(0-7) Powering on due to button press
[ 496.611702] pcieport 0000:00:01.7: pciehp: Slot(0-7): Card present
[ 496.614165] pcieport 0000:00:01.7: pciehp: Slot(0-7): Link Up
[ 496.772990] pci 0000:08:00.0: [1af4:1058] type 00 class 0x00ff00
[ 496.775269] pci 0000:08:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 496.779390] pci 0000:08:00.0: BAR 4: assigned [mem 0x8000e00000-0x8000e03fff 64bit pref]
[ 496.781849] pcieport 0000:00:01.7: PCI bridge to [bus 08]
[ 496.783585] pcieport 0000:00:01.7: bridge window [io 0x8000-0x8fff]
[ 496.791015] pcieport 0000:00:01.7: bridge window [mem 0x10e00000-0x10ffffff]
[ 496.796624] pcieport 0000:00:01.7: bridge window [mem 0x8000e00000-0x8000ffffff 64bit pref]
[ 496.806395] virtio-pci 0000:08:00.0: enabling device (0000 -> 0002)
[ 496.834460] virtio_mem virtio7: start address: 0x140000000
[ 496.836065] virtio_mem virtio7: region size: 0x200000000
[ 496.837585] virtio_mem virtio7: device block size: 0x200000
[ 496.839184] virtio_mem virtio7: nid: 0
[ 496.840278] virtio_mem virtio7: memory block size: 0x8000000
[ 496.841926] virtio_mem virtio7: subblock size: 0x1000000
[ 496.843777] TECH PREVIEW: virtio_mem may not be fully supported.
Please review provided documentation for limitations.
[ 496.847419] virtio_mem virtio7: plugged size: 0x0
[ 496.848772] virtio_mem virtio7: requested size: 0x40000000
[ 496.886942] Built 2 zonelists, mobility grouping on. Total pages: 892517
[ 496.888953] Policy zone: Normal
[ 501.778392] pcieport 0000:00:02.0: pciehp: Slot(0-8): Attention button pressed
[ 501.781314] pcieport 0000:00:02.0: pciehp: Slot(0-8) Powering on due to button press
[ 501.784912] pcieport 0000:00:02.0: pciehp: Slot(0-8): Card present
[ 501.787276] pcieport 0000:00:02.0: pciehp: Slot(0-8): Link Up
[ 501.943002] pci 0000:09:00.0: [1af4:1058] type 00 class 0x00ff00
[ 501.945271] pci 0000:09:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 501.949366] pci 0000:09:00.0: BAR 4: assigned [mem 0x8001000000-0x8001003fff 64bit pref]
[ 501.951838] pcieport 0000:00:02.0: PCI bridge to [bus 09]
[ 501.953620] pcieport 0000:00:02.0: bridge window [io 0x9000-0x9fff]
[ 501.960109] pcieport 0000:00:02.0: bridge window [mem 0x11000000-0x111fffff]
[ 501.965271] pcieport 0000:00:02.0: bridge window [mem 0x8001000000-0x80011fffff 64bit pref]
[ 501.974220] virtio-pci 0000:09:00.0: enabling device (0000 -> 0002)
[ 501.983759] virtio_mem virtio8: start address: 0x340000000
[ 501.985337] virtio_mem virtio8: region size: 0x200000000
[ 501.986858] virtio_mem virtio8: device block size: 0x200000
[ 501.988450] virtio_mem virtio8: nid: 1
[ 501.989763] virtio_mem virtio8: memory block size: 0x8000000
[ 501.991410] virtio_mem virtio8: subblock size: 0x1000000
[ 501.993524] virtio_mem virtio8: plugged size: 0x0
[ 501.994884] virtio_mem virtio8: requested size: 0x40000000
[ 502.042476] Built 2 zonelists, mobility grouping on. Total pages: 1154662
[ 502.044819] Policy zone: Normal
s4. Resize virtio-mem, grow both to maximum
# virsh update-memory-device fj-kvm-vm --live --alias virtiomem0 --requested-size 8G
# virsh update-memory-device fj-kvm-vm --live --alias virtiomem1 --requested-size 8G
# virsh qemu-monitor-command --hmp fj-kvm-vm info memory-devices
Memory device [virtio-mem]: "virtiomem0"
memaddr: 0x140000000
node: 0
requested-size: 8589934592
size: 8589934592
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem0
Memory device [virtio-mem]: "virtiomem1"
memaddr: 0x340000000
node: 1
requested-size: 8589934592
size: 8589934592
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem1
[root@localhost ~]# free -h
total used free shared buff/cache available
Mem: 19Gi 481Mi 18Gi 8.0Mi 220Mi 18Gi
Swap: 1.0Gi 0B 1.0Gi
[root@localhost ~]# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1
node 0 size: 10164 MB
node 0 free: 9810 MB
node 1 cpus: 2 3
node 1 size: 9676 MB
node 1 free: 9328 MB
node distances:
node 0 1
0: 10 20
1: 20 10
s5. Use the hot added memory by 'numactl -m <NUMA NODE> <application>'
for two purposes: (a) the hot added memory can be accessed. (b)
the data on the hot added memory can be migrated in the scenario
of hot remove (shrink).
[root@localhost ~]# numactl -m 0 dd if=/dev/urandom of=/tmp/numa_test/test bs=1k count=5242880
5242880+0 records in
5242880+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 104.339 s, 51.5 MB/s
[root@localhost ~]# rm -rf /tmp/numa_test/*
[root@localhost ~]# numactl -m 1 dd if=/dev/urandom of=/tmp/numa_test/test bs=1k count=5242880
5242880+0 records in
5242880+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 105.022 s, 51.1 MB/s
[root@localhost ~]# memhog -r10 4G --membind 0
[root@localhost ~]# memhog -r10 4G --membind 1
# virsh update-memory-device fj-kvm-vm --live --alias virtiomem0 --requested-size 0G
# virsh update-memory-device fj-kvm-vm --live --alias virtiomem1 --requested-size 0G
# virsh qemu-monitor-command --hmp fj-kvm-vm info memory-devices
Memory device [virtio-mem]: "virtiomem0"
memaddr: 0x140000000
node: 0
requested-size: 0
size: 134217728
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem0
Memory device [virtio-mem]: "virtiomem1"
memaddr: 0x340000000
node: 1
requested-size: 0
size: 4697620480
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem1
[root@localhost ~]# free -h
total used free shared buff/cache available
Mem: 3.4Gi 227Mi 3.0Gi 8.0Mi 140Mi 3.0Gi
Swap: 1.0Gi 0B 1.0Gi
[root@localhost ~]# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1
node 0 size: 1988 MB
node 0 free: 1762 MB
node 1 cpus: 2 3
node 1 size: 1484 MB
node 1 free: 1365 MB
node distances:
node 0 1
0: 10 20
1: 20 10
Case3. Test with virtio-mem + vIOMMU Emm... I don't know how to set virtio-mem + iommu in libvirt xml. I tried common way to protest virtio-mem with iommu like ``` 153 <memory model='virtio-mem'> 154 <target> 155 <size unit='KiB'>8388608</size> 156 <node>0</node> 157 <block unit='KiB'>2048</block> 158 <requested unit='KiB'>1048576</requested> 159 </target> 160 <driver iommu='on'> 161 <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> 162 </memory> ``` But it can't pass libvirt xml format check. Does the libvirt support set iommu for memory device? (In reply to Yiding Liu (Fujitsu) from comment #9) > Case3. Test with virtio-mem + vIOMMU > > Emm... I don't know how to set virtio-mem + iommu in libvirt xml. > s///g(In reply to Yiding Liu (Fujitsu) from comment #9) > Case3. Test with virtio-mem + vIOMMU > > Emm... I don't know how to set virtio-mem + iommu in libvirt xml. > > I tried common way to protest virtio-mem with iommu like s/protest/protect/g > ``` > 153 <memory model='virtio-mem'> > 154 <target> > 155 <size unit='KiB'>8388608</size> > 156 <node>0</node> > 157 <block unit='KiB'>2048</block> > 158 <requested unit='KiB'>1048576</requested> > 159 </target> > 160 <driver iommu='on'> > 161 <address type='pci' domain='0x0000' bus='0x08' slot='0x00' > function='0x0'/> > 162 </memory> > ``` > But it can't pass libvirt xml format check. Does the libvirt support set > iommu > for memory device? The basic functions of virtio-mem works, so set it as VERIFIED. (In reply to Yiding Liu (Fujitsu) from comment #9) > But it can't pass libvirt xml format check. Does the libvirt support set > iommu > for memory device? No. I haven't implemented that. Is that needed? (In reply to Michal Privoznik from comment #11) > (In reply to Yiding Liu (Fujitsu) from comment #9) > > > But it can't pass libvirt xml format check. Does the libvirt support set > > iommu > > for memory device? > > No. I haven't implemented that. Is that needed? Hi, Michal. The qemu-kvm has supported it, so I think it is needed in libvirt. BTW. Fujitsu has no requirement for it. Any priority is ok for us if you will implement it in libvirt. @Gavin, what do you think. Case3. virtio-iommu + virtio-mem test
I checked upstream libvirt and the upstream also doesn't support set iommu
for virtio-mem device.
1. Add virtio iommu + virtio-mem device
```
<memory model='virtio-mem'>
<target>
<size unit='KiB'>8388608</size>
<node>0</node>
<block unit='KiB'>2048</block>
<requested unit='KiB'>1048576</requested>
</target>
<address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
</memory>
<memory model='virtio-mem'>
<target>
<size unit='KiB'>8388608</size>
<node>1</node>
<block unit='KiB'>2048</block>
<requested unit='KiB'>2097152</requested>
</target>
<address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
</memory>
<iommu model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</iommu>
```
2. Start guest
Check dmesg and pci device
```
[root@localhost ~]# lspci
--snip---
08:00.0 Unclassified device [00ff]: Red Hat, Inc. Device 1058 (rev 01)
09:00.0 Unclassified device [00ff]: Red Hat, Inc. Device 1058 (rev 01)
[root@localhost ~]# dmesg | grep iommu
---snip---
[ 3.333485] virtio-pci 0000:08:00.0: Adding to iommu group 1
[ 3.342953] virtio-pci 0000:09:00.0: Adding to iommu group 2
```
3. Test virtio-mem deive
```
[root@hpe-apollo80-01-n01 ~]# virsh qemu-monitor-command --hmp fj-kvm-vm info memory-devices
Memory device [virtio-mem]: "virtiomem0"
memaddr: 0x140000000
node: 0
requested-size: 1073741824
size: 1073741824
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem0
Memory device [virtio-mem]: "virtiomem1"
memaddr: 0x340000000
node: 1
requested-size: 2147483648
size: 2147483648
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem1
[root@hpe-apollo80-01-n01 ~]# virsh update-memory-device fj-kvm-vm --live --alias virtiomem0 --requested-size 8G
[root@hpe-apollo80-01-n01 ~]# virsh update-memory-device fj-kvm-vm --live --alias virtiomem1 --requested-size 8G
[root@localhost ~]# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1
node 0 size: 3034 MB
node 0 free: 2817 MB
node 1 cpus: 2 3
node 1 size: 3495 MB
node 1 free: 3340 MB
node distances:
node 0 1
0: 10 20
1: 20 10
[root@localhost ~]# free -h
total used free shared buff/cache available
Mem: 19Gi 514Mi 18Gi 8.0Mi 141Mi 18Gi
Swap: 1.0Gi 0B 1.0Gi
[root@localhost ~]# mkdir -p /tmp/numa_test
[root@localhost ~]# numactl -m 0 dd if=/dev/urandom of=/tmp/numa_test/test bs=1k count=5242880
5242880+0 records in
5242880+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 110.641 s, 48.5 MB/s
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# rm -rf /tmp/numa_test/*
[root@localhost ~]# numactl -m 1 dd if=/dev/urandom of=/tmp/numa_test/test bs=1k count=5242880
5242880+0 records in
5242880+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 107.677 s, 49.9 MB/s
[root@localhost ~]# memhog -r10 4G --membind 0
[root@localhost ~]# memhog -r10 4G --membind 1
[root@hpe-apollo80-01-n01 ~]# virsh update-memory-device fj-kvm-vm --live --alias virtiomem0 --requested-size 0G
[root@hpe-apollo80-01-n01 ~]# virsh update-memory-device fj-kvm-vm --live --alias virtiomem1 --requested-size 0G
[root@hpe-apollo80-01-n01 ~]# virsh qemu-monitor-command --hmp fj-kvm-vm info memory-devices
Memory device [virtio-mem]: "virtiomem0"
memaddr: 0x140000000
node: 0
requested-size: 0
size: 2147483648
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem0
Memory device [virtio-mem]: "virtiomem1"
memaddr: 0x340000000
node: 1
requested-size: 0
size: 5368709120
max-size: 8589934592
block-size: 2097152
memdev: /objects/memvirtiomem1
[root@localhost ~]# free -h
total used free shared buff/cache available
Mem: 3.6Gi 212Mi 2.6Gi 8.0Mi 742Mi 3.1Gi
Swap: 1.0Gi 0B 1.0Gi
[root@localhost ~]# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1
node 0 size: 2202 MB
node 0 free: 2001 MB
node 1 cpus: 2 3
node 1 size: 1447 MB
node 1 free: 693 MB
node distances:
node 0 1
0: 10 20
1: 20 10
```
Yiding, thanks for your further testing and checking. The result is as expected from my mind. As we discussed in today's meeting, lets create another separate bug, to support virtio-iommu for virtio-mem device, if Michal agrees. Michael, please let Yiding know your preference. (In reply to Guowen Shan from comment #14) > Yiding, thanks for your further testing and checking. The result > is as expected from my mind. As we discussed in today's meeting, > lets create another separate bug, to support virtio-iommu for > virtio-mem device, if Michal agrees. Michael, please let Yiding > know your preference. Yep, I agree. The basic functionality is in RHEL-9 and that's what this bug reflects. https://bugzilla.redhat.com/show_bug.cgi?id=2095091(In reply to Michal Privoznik from comment #15) > (In reply to Guowen Shan from comment #14) > > Yiding, thanks for your further testing and checking. The result > > is as expected from my mind. As we discussed in today's meeting, > > lets create another separate bug, to support virtio-iommu for > > virtio-mem device, if Michal agrees. Michael, please let Yiding > > know your preference. > > Yep, I agree. The basic functionality is in RHEL-9 and that's what this bug > reflects. Done. https://bugzilla.redhat.com/show_bug.cgi?id=2095091 Closing as CURRENTRELEASE as RHEL 9.1 GA'ed in November 2022 |