I updaed swtpm dist-git package to 0.2.0-1.20191018git9227cf4.

Danilo, is it part of the rhel av 8.1.1 module already?

curl -s "https://mbs.engineering.redhat.com/module-build-service/1/module-builds/?name=virt&stream=8.1&state=5&base_module_br_stream=el8.1.1" | grep swtpm
"swtpm-0.2.0-1.20191018git9227cf4.module+el8.1.1+4483+2f45aaa2", 

So yes.

Hi  Marc-Andre,

I encountered some issues when try to verify this bug. Could you help check pls?

Pkgs:
swtpm-0.2.0-1.20191018git9227cf4.module+el8.1.1+4483+2f45aaa2.x86_64
swtpm-tools-0.2.0-1.20191018git9227cf4.module+el8.1.1+4483+2f45aaa2.x86_64
libvirt-5.6.0-7.module+el8.1.1+4483+2f45aaa2.x86_64
qemu-kvm-4.1.0-14.module+el8.1.1+4632+a8269660.x86_64
edk2-ovmf-20190308git89910a39dcfd-6.el8.noarch

1. # swtpm socket --print-capabilities
{ "type": "swtpm", "features": [ "tpm-send-command-header", "cmdarg-seccomp", "cmdarg-key-fd", "cmdarg-pwd-fd" ] }

# swtpm_setup --print-capabilities
{ "type": "swtpm_setup", "features": [ "cmdarg-keyfile-fd", "cmdarg-pwdfile-fd" ] }

2.# virsh secret-list
 UUID                                   Usage
-----------------------------------------------------------
 5172c66b-c6b9-415a-af4d-ac47f11c3ac0   vtpm VTPM_example

# virsh secret-dumpxml 5172c66b-c6b9-415a-af4d-ac47f11c3ac0
<secret ephemeral='no' private='yes'>
  <uuid>5172c66b-c6b9-415a-af4d-ac47f11c3ac0</uuid>
  <description>sample vTPM secret</description>
  <usage type='vtpm'>
    <name>VTPM_example</name>
  </usage>
</secret>

3. Try to edit guest with:
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'>
        <encryption secret='VTPM_example'/>
      </backend>
    </tpm>
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]: 

4.# virsh dumpxml rhel8.1-ovmf |grep 'tpm m' -A5
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'>
        <encryption secret='5172c66b-c6b9-415a-af4d-ac47f11c3ac0'/>
      </backend>
    </tpm>

# virsh start rhel8.1-ovmf
error: Failed to start domain rhel8.1-ovmf
error: internal error: qemu unexpectedly closed the monitor: qemu_madvise: Invalid argument
madvise doesn't support MADV_DONTDUMP, but dump_guest_core=off specified
2019-12-06T08:49:11.571701Z qemu-kvm: tpm-emulator: TPM result for CMD_INIT: 0x101 operation failed

5. If enable dump_guest_core = 1 in /etc/libvirt/qemu.conf:
# systemctl restart libvirtd

# virsh start rhel8.1-ovmf
error: Failed to start domain rhel8.1-ovmf
error: internal error: qemu unexpectedly closed the monitor: 2019-12-06T08:52:51.013632Z qemu-kvm: tpm-emulator: TPM result for CMD_INIT: 0x101 operation failed


Actual results:
1. In step3, secret usage name can not be used by vtpm encryption, not align with https://libvirt.org/formatsecret.html#vTPMUsageType:
"The vTPM secret can then be used by UUID or by this usage name via the <encryption> element of a tpm when using an emulator."

2. In step4, vtpm encryption meets MADV_DONTDUMP error when dump_guest_core disabled. Could you give a description about why pls?

3. In step5, UEFI guest with encrypted vtpm can not be started.


Additional info:
1. This UEFI guest with vtpm(without encryption) can be started and vtpm works well.
2. Another BIOS guest with this encrypted vtpm can be started.


Thank you in advance.

Created attachment 1642633 [details]
comment7_qemu.log

(In reply to yanqzhan from comment #7)

> Actual results:
> 1. In step3, secret usage name can not be used by vtpm encryption, not align
> with https://libvirt.org/formatsecret.html#vTPMUsageType:
> "The vTPM secret can then be used by UUID or by this usage name via the
> <encryption> element of a tpm when using an emulator."

Good catch,

Documentation is wrong, I sent a patch upstream ("doc: vtpm only support secrets by UUID at this point"). Support by usage name is lacking.

I am not sure it is worth a backport, let see the discussion upstream.

> 
> 2. In step4, vtpm encryption meets MADV_DONTDUMP error when dump_guest_core
> disabled. Could you give a description about why pls?

This is unrelated (I'll need to dig for that bug elsewhere) 

> 3. In step5, UEFI guest with encrypted vtpm can not be started.
>


That one is more surprising, could you check /var/log/swtpm/libvirt/qemu/$VMNAME-swtpm.log as well as system log for selinux errors and such?
 
I am trying to reproduce on a rhel8 vm, but if I can access your setup that may also help!

thanks

(In reply to Marc-Andre Lureau from comment #11)
> > 3. In step5, UEFI guest with encrypted vtpm can not be started.
> 
> That one is more surprising, could you check
> /var/log/swtpm/libvirt/qemu/$VMNAME-swtpm.log as well as system log for
> selinux errors and such?
>  
> I am trying to reproduce on a rhel8 vm, but if I can access your setup that
> may also help!
> 
> thanks

Hi Marc-Andre,

I found the root cause: VM with an unencrypted vtpm was started before, if now add encryption on it, the tpmstate file blocks its start. Thank you for the reminder of checking swtpm.log.

Pkgs info:
swtpm-0.2.0-1.20191018git9227cf4.module+el8.1.1+4483+2f45aaa2.x86_64
swtpm-tools-0.2.0-1.20191018git9227cf4.module+el8.1.1+4483+2f45aaa2.x86_64
libvirt-daemon-5.6.0-10.module+el8.1.1+5131+a6fe889c.x86_64
qemu-kvm-4.1.0-19.module+el8.1.1+5172+e3ff58a1.x86_64

Reproduce steps:
1. start the vm with unencrypted vtpm:
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'/>
    </tpm>

# virsh start rhel8.1-ovmf
# ps aux|grep swtpm
tss      13769  0.2  0.0  23632  3380 ?        Ss   01:21   0:00 /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/var/run/libvirt/qemu/swtpm/27-rhel8.1-ovmf-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/8e7d9ada-9d61-4b42-a8dc-b4e4a9d8cc16/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log --tpm2 --pid file=/var/run/libvirt/qemu/swtpm/27-rhel8.1-ovmf-swtpm.pid
 ... -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm -chardev socket,id=chrtpm,path=/var/run/libvirt/qemu/swtpm/27-rhel8.1-ovmf-swtpm.sock -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0 ...

2. shutdown the vm and add encryption on it:
# virsh shutdown rhel8.1-ovmf

    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'>
        <encryption secret='472bad1d-19e6-4363-ae76-edfcedebc737'/>
      </backend>
    </tpm>

3. Try to Restart vm:
# virsh start rhel8.1-ovmf 
error: Failed to start domain rhel8.1-ovmf
error: internal error: process exited while connecting to monitor: qemu_madvise: Invalid argument
madvise doesn't support MADV_DONTDUMP, but dump_guest_core=off specified
2019-12-17T06:00:35.612368Z qemu-kvm: tpm-emulator: TPM result for CMD_INIT: 0x101 operation failed

# tail -f /var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log 
Wrong decryption key. Need 128 bit key.
SWTPM_NVRAM_LoadData: Error from SWTPM_NVRAM_GetDecryptedData rc = 40
Wrong decryption key. Need 128 bit key.
SWTPM_NVRAM_LoadData: Error from SWTPM_NVRAM_GetDecryptedData rc = 40
libtpms/tpm2: Entering failure mode; code: 8, location: NvPowerOn line 126
Error: Could not initialize libtpms.
Error: Could not initialize the TPM
Data client disconnected

4. Remove a tpmstate file and try to start vm again:
# rm -f /var/lib/libvirt/swtpm/8e7d9ada-9d61-4b42-a8dc-b4e4a9d8cc16/tpm2/tpm2-00.permall 
# virsh start rhel8.1-ovmf 
Domain rhel8.1-ovmf started

# ps aux|grep swtpm
tss      13266  0.0  0.0  23632  3488 ?        Ss   01:01   0:00 /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/var/run/libvirt/qemu/swtpm/26-rhel8.1-ovmf-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/8e7d9ada-9d61-4b42-a8dc-b4e4a9d8cc16/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log --tpm2 --pid file=/var/run/libvirt/qemu/swtpm/26-rhel8.1-ovmf-swtpm.pid --key pwdfd=38,mode=aes-256-cbc --migration-key pwdfd=41,mode=aes-256-cbc
 ...-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm -chardev socket,id=chrtpm,path=/var/run/libvirt/qemu/swtpm/26-rhel8.1-ovmf-swtpm.sock -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0 ...

Additional info:
1. It's nothing about firmware(uefi,bios), machine type(q35,i440fx), because I tried all of them this time.
2. The qemu_madvise issues only happens on q35 guests, but dump_guest_core need not be set after rm the tpmstate file.
3. The swtpm.log when convert from a encrypted vtpm to an unencrypted one is:
# tail -f /var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log 
Could not find plain data in byte stream.
SWTPM_NVRAM_LoadData: Error from SWTPM_NVRAM_GetDecryptedData rc = 9
Could not find plain data in byte stream.
SWTPM_NVRAM_LoadData: Error from SWTPM_NVRAM_GetDecryptedData rc = 9
libtpms/tpm2: Entering failure mode; code: 8, location: NvPowerOn line 126
Error: Could not initialize libtpms.
Error: Could not initialize the TPM
Data client disconnected


Is this expected? Or shall I open a libvirt bug to help clean the tpm2-00.permall file?

Thanks.

Keep comment12 issue and needinfo open, verify other aspects of this bug.

Pkgs:
swtpm-0.2.0-1.20191018git9227cf4.module+el8.1.1+4483+2f45aaa2.x86_64
libvirt-daemon-5.6.0-10.module+el8.1.1+5131+a6fe889c.x86_64
qemu-kvm-4.1.0-19.module+el8.1.1+5172+e3ff58a1.x86_64

Steps:
Pre-step:
Define a secret and set value per https://libvirt.org/formatsecret.html#vTPMUsageType.
<secret ephemeral='no' private='yes'>
  <uuid>472bad1d-19e6-4363-ae76-edfcedebc737</uuid>
  <description>sample vTPM secret</description>
  <usage type='vtpm'>
    <name>VTPM_example</name>
  </usage>
</secret>

S1. Start guest with encrypted vtpm
1. Start one guest:
# virsh dumpxml rhel8.1-ovmf |grep encrypt  -4
..    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'>
        <encryption secret='472bad1d-19e6-4363-ae76-edfcedebc737'/>
      </backend>
      <alias name='tpm0'/>
    </tpm>

# ps aux|grep swtpm|grep rhel8.1-ovmf
system_u:system_r:svirt_t:s0:c192,c873 tss 15405 0.0  0.0 23632  3352 ?        Ss   Dec17   0:00 /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/var/run/libvirt/qemu/swtpm/33-rhel8.1-ovmf-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/8e7d9ada-9d61-4b42-a8dc-b4e4a9d8cc16/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log --tpm2 --pid file=/var/run/libvirt/qemu/swtpm/33-rhel8.1-ovmf-swtpm.pid --key pwdfd=38,mode=aes-256-cbc --migration-key pwdfd=41,mode=aes-256-cbc
system_u:system_r:svirt_t:s0:c192,c873 qemu 15409 ... -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm -chardev socket,id=chrtpm,path=/var/run/libvirt/qemu/swtpm/33-rhel8.1-ovmf-swtpm.sock -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0 ...

Check vtpm usage in guest OS:
[root@localhost ~]# tpm2_getrandom 7
0x69 0xB2 0xF3 0xD5 0xEE 0x12 0x33

2. Start another guest with same vtpm device(same secret):
# virsh start vm2-ovmf
Domain vm2-ovmf started
 
system_u:system_r:svirt_t:s0:c517,c778 tss 32368 0.0  0.0 23684  3440 ?        Ss   04:04   0:00 /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/var/run/libvirt/qemu/swtpm/42-vm2-ovmf-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/190d73e7-7d6f-4fe0-9926-3ef82506af69/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/vm2-ovmf-swtpm.log --tpm2 --pid file=/var/run/libvirt/qemu/swtpm/42-vm2-ovmf-swtpm.pid --key pwdfd=40,mode=aes-256-cbc --migration-key pwdfd=43,mode=aes-256-cbc
system_u:system_r:svirt_t:s0:c517,c778 qemu 32372 ... -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm -chardev socket,id=chrtpm,path=/var/run/libvirt/qemu/swtpm/42-vm2-ovmf-swtpm.sock -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0 ...

[root@localhost ~]# tpm2_getrandom 6
0xAA 0xA1 0x45 0x20 0x1D 0xB1


S2. Save to file
1. managedsave and start
# virsh managedsave rhel8.1-ovmf

Domain rhel8.1-ovmf state saved by libvirt

# virsh start rhel8.1-ovmf
Domain rhel8.1-ovmf started

# ps aux|grep swtpm|grep rhel8.1-ovmf
system_u:system_r:svirt_t:s0:c23,c107 tss 1864  0.0  0.0  23632  3496 ?        Ss   05:32   0:00 /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/var/run/libvirt/qemu/swtpm/45-rhel8.1-ovmf-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/8e7d9ada-9d61-4b42-a8dc-b4e4a9d8cc16/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log --tpm2 --pid file=/var/run/libvirt/qemu/swtpm/45-rhel8.1-ovmf-swtpm.pid --key pwdfd=38,mode=aes-256-cbc --migration-key pwdfd=43,mode=aes-256-cbc
system_u:system_r:svirt_t:s0:c23,c107 qemu 1868...-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm -chardev socket,id=chrtpm,path=/var/run/libvirt/qemu/swtpm/45-rhel8.1-ovmf-swtpm.sock -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0…

[root@localhost ~]# tpm2_getrandom 7
0x0D 0xE8 0xBA 0xDE 0x1B 0x5F 0x80

2. Change secret value between managedsave and start:
# virsh managedsave rhel8.1-ovmf

(1)# MYSECRET=`printf %s "new sesame" | base64`
# virsh secret-set-value 472bad1d-19e6-4363-ae76-edfcedebc737 $MYSECRET
Secret value set

# virsh start rhel8.1-ovmf
error: Failed to start domain rhel8.1-ovmf
error: internal error: qemu unexpectedly closed the monitor: qemu_madvise: Invalid argument
madvise doesn't support MADV_DONTDUMP, but dump_guest_core=off specified
2019-12-19T11:31:29.374411Z qemu-kvm: tpm-emulator: TPM result for CMD_INIT: 0x101 operation failed

# tail -f /var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log
Verification of HMAC failed. Data integrity is compromised
SWTPM_NVRAM_LoadData: Error from SWTPM_NVRAM_GetDecryptedData rc = 33
Verification of HMAC failed. Data integrity is compromised
SWTPM_NVRAM_LoadData: Error from SWTPM_NVRAM_GetDecryptedData rc = 33
libtpms/tpm2: Entering failure mode; code: 8, location: NvPowerOn line 126
Error: Could not initialize libtpms.
Error: Could not initialize the TPM
Data client disconnected

(2)Change secret value back:
# MYSECRET=`printf %s "open sesame" | base64`
# virsh secret-set-value 472bad1d-19e6-4363-ae76-edfcedebc737 $MYSECRET
Secret value set

# virsh start rhel8.1-ovmf
Domain rhel8.1-ovmf started


S3. Migration
1. Define a same secret with same value on target, migrate to target:
(1)# virsh migrate rhel8.1-ovmf --live qemu+ssh://10.66.*.*/system --verbose
Migration: [100 %]

Target
system_u:system_r:svirt_t:s0:c488,c974 tss 8971 0.0  0.0  23632  3456 ?        Ss   06:20   0:00 /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/var/run/libvirt/qemu/swtpm/127-rhel8.1-ovmf-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/8e7d9ada-9d61-4b42-a8dc-b4e4a9d8cc16/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log --tpm2 --pid file=/var/run/libvirt/qemu/swtpm/127-rhel8.1-ovmf-swtpm.pid --key pwdfd=41,mode=aes-256-cbc --migration-key pwdfd=44,mode=aes-256-cbc
system_u:system_r:svirt_t:s0:c488,c974 qemu 8976...mdev emulator,id=tpm-tpm0,chardev=chrtpm -chardev socket,id=chrtpm,path=/var/run/libvirt/qemu/swtpm/127-rhel8.1-ovmf-swtpm.sock -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0…

    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'>
        <encryption secret='472bad1d-19e6-4363-ae76-edfcedebc737'/>
      </backend>
      <alias name='tpm0'/>
    </tpm>

[root@localhost ~]# tpm2_getrandom 7
0x14 0x51 0x8A 0xD1 0xD9 0x73 0x4C

(2)migrate back:
# virsh migrate rhel8.1-ovmf --live qemu+ssh://lenovo-***/system --verbose
Migration: [100 %]

2. Change target secret value before migration:
(1)Target
# MYSECRET=`printf %s "new sesame" | base64`
# virsh secret-set-value 472bad1d-19e6-4363-ae76-edfcedebc737  $MYSECRET
Secret value set

Source
# virsh migrate rhel8.1-ovmf --live qemu+ssh://10.66.*.*/system --verbose
Migration: [100 %]error: internal error: qemu unexpectedly closed the monitor: qemu_madvise: Invalid argument
madvise doesn't support MADV_DONTDUMP, but dump_guest_core=off specified
2019-12-19T11:42:59.949536Z qemu-kvm: tpm-emulator: Setting the stateblob (type 1) failed with a TPM error 0x21 decryption error
2019-12-19T11:42:59.956099Z qemu-kvm: error while loading state for instance 0x0 of device 'tpm-emulator'
2019-12-19T11:43:00.126603Z qemu-kvm: load of migration failed: Input/output error

# tail -f /var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log
Verification of HMAC failed. Data integrity is compromised
SWTPM_NVRAM_LoadData: Error from SWTPM_NVRAM_GetDecryptedData rc = 33
Verification of HMAC failed. Data integrity is compromised
SWTPM_NVRAM_LoadData: Error from SWTPM_NVRAM_GetDecryptedData rc = 33
libtpms/tpm2: Entering failure mode; code: 8, location: NvPowerOn line 126
Error: Could not initialize libtpms.
Error: Could not initialize the TPM
Data client disconnected

Libvirtd.log
2019-12-19 11:43:01.705+0000: 3875: debug : qemuMigrationSrcRestoreDomainState:149 : Restoring pre-migration state due to migration error

[target]
# tail -f /var/log/swtpm/libvirt/qemu/rhel8.1-ovmf-swtpm.log
Verification of HMAC failed. Data integrity is compromised
Decrypting the permall blob with the migration key failed; res = 33
Data client disconnected

Libvirtd.log
2019-12-19 11:48:28.043+0000: 10055: error : qemuMonitorIORead:604 : Unable to read from monitor: Connection reset by peer
2019-12-19 11:48:28.044+0000: 10055: error : qemuProcessReportLogError:2098 : internal error: qemu unexpectedly closed the monitor: qemu_madvise: Invalid argument
2019-12-19 11:48:28.680+0000: 10060: error : virProcessRunInFork:1170 : internal error: child reported (status=125):
2019-12-19 11:48:28.686+0000: 10060: error : virProcessRunInFork:1170 : internal error: child reported (status=125): internal error: child reported (status=125):

(2)Change secret value back:
Target:
# MYSECRET=`printf %s "open sesame" | base64`
# virsh secret-set-value 472bad1d-19e6-4363-ae76-edfcedebc737  $MYSECRET
Secret value set

Source:
# virsh migrate rhel8.1-ovmf --live qemu+ssh://10.66.*.*/system --verbose
Migration: [100 %]

And on target, vtpm works well in guest os.

S4: Other regression checkpoints
1. domrename the guest with encrypted vtpm, and start it.
Guest start succeed.

2. remove encrypted vtpm from guest by virsh edit  or virsh define.
Succeed without error

3. lifecycle testing for guest with encrypted vtpm
create, define, start, save, restore, suspend, resume, undefine --nvram.
Succeed without error.

4. restart libvirtd while guest with encrypted vtpm is running.
guest running and vtpm works well in guest os.

5. plug/unplug
error: Operation not supported: live attach of device 'tpm' is not supported
error: Operation not supported: persistent attach of device 'tpm' is not supported
error: Operation not supported: live detach of device 'tpm' is not supported
error: Operation not supported: persistent detach of device 'tpm' is not supported

S5: Negative testing  for xml
1. Edit encryption without secret
         <encryption/>
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]:

2. Edit secret with invalid uuid pattern
        <encryption secret='472bad1d-19e6-4363-ae76-edfcedebc73'/>  <==missing 1 number

error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]:

3. Using non-existent secret
        <encryption secret='472bad1d-19e6-4363-ae76-edfcedebc730'/>
# virsh start rhel8.1-ovmf
error: Failed to start domain rhel8.1-ovmf
error: Secret not found: no secret with matching uuid '472bad1d-19e6-4363-ae76-edfcedebc730'


S6. Session mode with unprivileged user
Pre-steps refer to bz1519016#c24 step0.
1.$ virsh start rhel8.1-ovmf-session
Domain rhel8.1-ovmf-session started

$ ps aux|grep swtpm
test      9376  0.4  0.0  23632  3328 ?        Ss   21:21   0:00 /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/home/test/.cache/libvirt/qemu/run/swtpm/8-rhel8.1-ovmf-session-swtpm.sock,mode=0600 --tpmstate dir=/home/test/.config/libvirt/qemu/swtpm/57479029-b056-453c-a303-ff538c9d3e7b/tpm2,mode=0600 --log file=/home/test/.cache/libvirt/qemu/log/rhel8.1-ovmf-session-swtpm.log --tpm2 --pid file=/home/test/.cache/libvirt/qemu/run/swtpm/8-rhel8.1-ovmf-session-swtpm.pid --key pwdfd=33,mode=aes-256-cbc --migration-key pwdfd=36,mode=aes-256-cbc
...-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm -chardev socket,id=chrtpm,path=/home/test/.cache/libvirt/qemu/run/swtpm/8-rhel8.1-ovmf-session-swtpm.sock -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0…

$ virsh dumpxml rhel8.1-ovmf-session |grep 'tpm m' -A5
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'>
        <encryption secret='fcda2043-ec4c-46ad-93bf-c4ac1567185a'/>
      </backend>
      <alias name='tpm0'/>
    </tpm>

$ virsh console rhel8.1-ovmf-session
[root@localhost ~]# tpm2_getrandom 8
0xE7 0x26 0xA9 0xE8 0xD2 0xA4 0x72 0x93

(In reply to yanqzhan from comment #12)

Marc-Andre's reply via email:

Switching between encrypted and unencrypted is unsupported. This
should behave similarly to disk encryption support.

There was an attempt to check XML configuration changes after start in
libvirt, but it was reverted
(https://libvirt.org/git/?p=libvirt.git;a=commit;h=d8326cb8826170019c74018511d76c58665ab282).
I suppose this is now consistent with other encrypted usage.

> yanqzhan: And I'm not quite sure about when vtpm encrypted and decrypted when guest start, I don't know how the secret is used to encrypt and decrypt in this only one action.

The TPM state is encrypted on creation, the key is passed to
swtpm_setup & swtpm via filedescriptor by libvirt.

(In reply to yanqzhan from comment #15)

> Marc-Andre's reply via email:
> 
> Switching between encrypted and unencrypted is unsupported. This
> should behave similarly to disk encryption support.
> 
> There was an attempt to check XML configuration changes after start in
> libvirt, but it was reverted
> (https://libvirt.org/git/?p=libvirt.git;a=commit;
> h=d8326cb8826170019c74018511d76c58665ab282).
> I suppose this is now consistent with other encrypted usage.
> 

Hi,

I still have a question: how to distinguish whether it's a new vtpm device?

For disk encryption support, user can remove the disk, and add a new disk with new password as he like. 

But for vtpm, the state file uses guest's uuid as dir name:
$ virsh domuuid rhel8.1-ovmf-session
57479029-b056-453c-a303-ff538c9d3e7b
$  ls /home/test/.config/libvirt/qemu/swtpm/
57479029-b056-453c-a303-ff538c9d3e7b  

So no matter how many times you remove the vtpm device, once you add a vtpm at next time, it uses the same state file. 
Combining with the guest start issue, it means if guest ever started with an encrypted vtpm, it can never use other forms(unecrypted or with different password) of vtpm in the future. 
For physical machine, user can insert and pull tpm chip on it. Is this like if pull a tpm chip up, the machine can never use other tpm chips in the future?

Now can only make new vtpm work by removing the state file, but end-users can not do that in general.

(In reply to yanqzhan from comment #16)
> For physical machine, user can insert and pull tpm chip on it. Is this like
> if pull a tpm chip up, the machine can never use other tpm chips in the
> future?

I think you could in theory swap TPM chips, although I don't think it's a common operation, but more like a hack.

> Now can only make new vtpm work by removing the state file, but end-users
> can not do that in general.

In general, TPM devices are not meant to be swappable, because the machine may hold its secrets there.

But I am not sure if the state file is left around on purpose, or we should remove it when the device is removed from the domain definition.

Probably it's the same problem as with volumes, you don't want to remove storage by mistake, so it's best to leave it there until the VM is actually undefined.

Thanks Marc-Andre. Then mark this bug as verified.

Additional scenarios:
After the state file of encrypted vtpm generated after first start
1. Change to use another secret(different password):
# virsh secret-list 
 UUID                                   Usage
------------------------------------------------------------
 183fe401-9bf1-45e9-bc4d-fa4042249f49   vtpm VTPM_example2
 472bad1d-19e6-4363-ae76-edfcedebc737   vtpm VTPM_example

# MYSECRET=`printf %s "hello redhat" | base64`
#  virsh secret-set-value   183fe401-9bf1-45e9-bc4d-fa4042249f49  $MYSECRET

# virsh dumpxml rhel8.1-ovmf |grep encrypt
        <encryption secret='183fe401-9bf1-45e9-bc4d-fa4042249f49'/>

# virsh start rhel8.1-ovmf 
error: Failed to start domain rhel8.1-ovmf
error: internal error: qemu unexpectedly closed the monitor: ...
qemu_madvise: Invalid argument
madvise doesn't support MADV_DONTDUMP, but dump_guest_core=off specified
2020-01-03T08:12:21.849734Z qemu-kvm: tpm-emulator: TPM result for CMD_INIT: 0x101 operation failed

2. Use other secret but with same password:
# MYSECRET=`printf %s "open sesame" | base64`
# virsh secret-set-value 183fe401-9bf1-45e9-bc4d-fa4042249f49  $MYSECRET

# virsh start rhel8.1-ovmf 
Domain rhel8.1-ovmf started

This BZ was not attached to an advisory and therefore was not closed when RHEL AV was shipped. Correcting this now by marking the BZ CLOSED CURRENTRELEASE.