Bug 1712243 - [downstream clone - 4.3.4] Engine generates too many updates to vm_dynamic table due to the session change
Summary: [downstream clone - 4.3.4] Engine generates too many updates to vm_dynamic ta...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.2.8-2
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.3.5
: 4.3.5
Assignee: Shmuel Melamud
QA Contact: Liran Rotenberg
URL:
Whiteboard:
Depends On: 1693628
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-21 06:58 UTC by RHV bug bot
Modified: 2019-08-12 11:53 UTC (History)
5 users (show)

Fixed In Version: ovirt-engine-4.3.5.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1693628
Environment:
Last Closed: 2019-08-12 11:53:28 UTC
oVirt Team: Virt
Target Upstream Version:
Embargoed:
lrotenbe: testing_plan_complete+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 4015571 0 Troubleshoot None Engine database size grows dramatically due to vm_dymaic table updates. 2019-05-21 06:58:16 UTC
Red Hat Product Errata RHEA-2019:2431 0 None None None 2019-08-12 11:53:36 UTC
oVirt gerrit 99028 0 master MERGED core: Persist the guest session state into the DB 2020-01-29 08:50:39 UTC
oVirt gerrit 100190 0 ovirt-engine-4.3 MERGED core: Persist the guest session state into the DB 2020-01-29 08:50:39 UTC

Description RHV bug bot 2019-05-21 06:58:01 UTC
+++ This bug is a downstream clone. The original bug is: +++
+++   bug 1693628 +++
======================================================================

Description of problem:
When analyzing changes in data provided by vdsm about the VM we check if the data is different than the data stored in the DB if so then we update the DB.

One of the fields provided by the vdsm is the session state (UserLoggedOn, LoggedOff, Locked, ...). We evaluate the state as changed as in the beginning the state in the DB is Unknown and we try to update the DB, but the VmDynamic.updateRuntimeData does not update the data for the session. So we end up in updating the DB All the time.

Version-Release number of selected component (if applicable):
4.2.8

How reproducible:
100%

Steps to Reproduce:
1. Stop guest agent on a RHEL guest system
2. Run echo '{"__name__": "session-logon"}' > /dev/virtio-ports/ovirt-guest-agent.0
3. vdsm-client VM getStats vmID=VMUUID | grep -i session

You should see:

        "session": "UserLoggedOn", 

4. Run sql
select session from vm_dynamic where vm_guid='VMUUID';

You will always see '0'

Actual results:
Always 0 (Unknown)

Expected results:
DB is updated

Additional info:
This is very critical for gbig environment where the DB updates can generate hundreds of updated to the DB every hour.

(Originally by Roman Hodain)

Comment 1 RHV bug bot 2019-05-21 06:58:03 UTC
can you please add versions of qemu-guest-agent and ovirt-guest-agent running in that guest?

(Originally by michal.skrivanek)

Comment 2 RHV bug bot 2019-05-21 06:58:05 UTC
and the entire output of vdsm-client VM getStats vmID=VMUUID please

(Originally by michal.skrivanek)

Comment 3 RHV bug bot 2019-05-21 06:58:07 UTC
I do not have any agent installed as I trigger the change manually by the echo command. The issue is not agent related.
The vdsm-client out is  [1]. This should fix the problem:

    https://gerrit.ovirt.org/#/c/99028/


[1]:
# vdsm-client VM getStats vmID=a7664445-87ce-4492-8f9e-1b376e881696 
[
    {
        "displayInfo": [
            {
                "tlsPort": "5901", 
                "ipAddress": "xxxxxxx", 
                "port": "5900", 
                "type": "spice"
            }
        ], 
        "memUsage": "0", 
        "acpiEnable": "true", 
        "vmId": "a7664445-87ce-4492-8f9e-1b376e881696", 
        "guestIPs": "xxxxxxxx", 
        "session": "UserLoggedOn", 
        "netIfaces": [
            {
                "name": "eth0", 
                "inet6": [
                    "fe80::546f:7dff:feec:0", 
                    "2620:52:0:25c0:546f:7dff:feec:0"
                ], 
                "inet": [
                    "xxxxxxxx"
                ], 
                "hw": "56:6f:7d:ec:00:00"
            }
        ], 
        "timeOffset": "-1", 
        "memoryStats": {
            "swap_out": 0, 
            "majflt": 0, 
            "minflt": 546, 
            "mem_cached": "682104", 
            "mem_free": "897244", 
            "mem_buffers": "2108", 
            "swap_in": 0, 
            "pageflt": 546, 
            "mem_total": "1813960", 
            "mem_unused": "897244"
        }, 
        "balloonInfo": {
            "balloon_max": "2097152", 
            "balloon_cur": "2097152", 
            "balloon_target": "2097152", 
            "balloon_min": "2097152"
        }, 
        "pauseCode": "NOERR", 
        "disksUsage": [
            {
                "path": "/", 
                "total": "6641680384", 
                "used": "1494687744", 
                "fs": "xfs"
            }, 
            {
                "path": "/boot", 
                "total": "1063256064", 
                "used": "151805952", 
                "fs": "xfs"
            }
        ], 
        "network": {
            "vnet0": {
                "macAddr": "56:6f:7d:ec:00:00", 
                "rxDropped": "1947", 
                "tx": "10083143", 
                "txDropped": "0", 
                "rxErrors": "0", 
                "rx": "1067332937", 
                "txErrors": "0", 
                "state": "unknown", 
                "sampleTime": 4630469.1, 
                "speed": "1000", 
                "name": "vnet0"
            }
        }, 
        "vmType": "kvm", 
        "cpuUser": "5.70", 
        "elapsedTime": "92635", 
        "vmJobs": {}, 
        "cpuSys": "1.00", 
        "appsList": [
            "ovirt-guest-agent-common-1.0.14-3.el7ev", 
            "kernel-3.10.0-957.el7"
        ], 
        "guestOs": "3.10.0-957.el7.x86_64", 
        "vmName": "test", 
        "guestFQDN": "unused", 
        "hash": "-4853325321511922985", 
        "lastLogin": 1553765872.655569, 
        "cpuUsage": "861670000000", 
        "vcpuPeriod": 100000, 
        "lastLogout": 1553762889.5671, 
        "lastUser": "Unknown", 
        "guestTimezone": {
            "zone": "Europe/Prague", 
            "offset": 60
        }, 
        "vcpuQuota": "-1", 
        "guestContainers": [], 
        "kvmEnable": "true", 
        "disks": {
            "hdc": {
                "readLatency": "0", 
                "writtenBytes": "0", 
                "writeOps": "0", 
                "apparentsize": "0", 
                "readOps": "12", 
                "writeLatency": "0", 
                "readBytes": "368", 
                "flushLatency": "0", 
                "readRate": "0.0", 
                "truesize": "0", 
                "writeRate": "0.0"
            }, 
            "sda": {
                "readLatency": "0", 
                "writtenBytes": "2590196224", 
                "writeOps": "638134", 
                "apparentsize": "3221225472", 
                "readOps": "116890", 
                "writeLatency": "3483024", 
                "imageID": "f8a12645-1501-45e5-b59b-cca1fa0a54df", 
                "readBytes": "496647680", 
                "flushLatency": "90658937", 
                "readRate": "0.0", 
                "truesize": "3221225472", 
                "writeRate": "14848.0"
            }
        }, 
        "monitorResponse": "0", 
        "guestOsInfo": {
            "kernel": "3.10.0-957.el7.x86_64", 
            "type": "linux", 
            "version": "7.6", 
            "distribution": "Red Hat Enterprise Linux Server", 
            "arch": "x86_64", 
            "codename": "Maipo"
        }, 
        "username": "root", 
        "guestName": "unused", 
        "status": "Up", 
        "guestCPUCount": 1, 
        "vcpuCount": "1", 
        "clientIp": "xxxxxxxxxxx", 
        "statusTime": "4630469100"
    }
]

(Originally by Roman Hodain)

Comment 4 RHV bug bot 2019-05-21 06:58:09 UTC
(In reply to Roman Hodain from comment #3)
> I do not have any agent installed as I trigger the change manually by the
> echo command. The issue is not agent related.
> The vdsm-client out is  [1]. This should fix the problem:
> 
>     https://gerrit.ovirt.org/#/c/99028/

it indeed should!

(Originally by michal.skrivanek)

Comment 5 Liran Rotenberg 2019-06-23 14:41:48 UTC
Verified on:
ovirt-engine-4.3.5.1-0.1.el7.noarch
vdsm-4.30.17-1.el7ev.x86_64

Steps:
1. Stop guest agent on a RHEL guest system
2. Run echo '{"__name__": "session-logon"}' > /dev/virtio-ports/ovirt-guest-agent.0
3. vdsm-client VM getStats vmID=VMUUID | grep -i session
4. Check the engine's DB
$ su - postgres
$ psql -d engine
# select session from vm_dynamic where vm_guid='VMUUID';

I tried both with the qemu-guest-agent on and off, and with step one set as session-logon, session-logoff.

Results:
$ vdsm-client VM getStats vmID=VMUUID | grep -i session
    "session": "UserLoggedOn",
engine=# select session from vm_dynamic where vm_guid='VMUUID';
 session 
---------
       1
(1 row)

Comment 7 Liran Rotenberg 2019-07-14 07:41:49 UTC
Verified in comment #5.

Comment 9 errata-xmlrpc 2019-08-12 11:53:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2431


Note You need to log in before you can comment on or make changes to this bug.