RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2090219 - Not able to install windows 11 OS with vTPM in spec (disable FIPS)
Summary: Not able to install windows 11 OS with vTPM in spec (disable FIPS)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: swtpm
Version: 9.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Marc-Andre Lureau
QA Contact: Qinghua Cheng
URL: https://github.com/stefanberger/libtp...
Whiteboard:
Depends On:
Blocks: 2089301 2097939 2097947 2152915
TreeView+ depends on / blocked
 
Reported: 2022-05-25 11:58 UTC by Qinghua Cheng
Modified: 2022-12-13 14:52 UTC (History)
22 users (show)

Fixed In Version: swtpm-0.7.0-3.20211109gitb79fd91.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2097939 2097947 2152915 (view as bug list)
Environment:
Last Closed: 2022-11-15 10:20:50 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-123326 0 None None None 2022-05-25 12:07:15 UTC
Red Hat Product Errata RHSA-2022:8100 0 None None None 2022-11-15 10:21:04 UTC

Description Qinghua Cheng 2022-05-25 11:58:52 UTC
Description of problem:

Even with tpm: {} in VM spec, Windows 11 installer still gives error "This PC can't run Windows 11"

Version-Release number of selected component (if applicable):

RHEL 8.6
kernel: 4.18.0-372.10.1.el8_6.x86_64
qemu-kvm: qemu-kvm-6.2.0-11.module+el8.6.0+14712+f96656d3.x86_64

swtpm-0.7.0-1.20211109gitb79fd91.module+el8.6.0+13879+1439f356.x86_64
libtpms-0.9.1-0.20211126git1ff6fe1f43.module+el8.6.0+13879+1439f356.x86_64

How reproducible:
100%

Steps to Reproduce:
1. On host fips-mode-setup --enable
2. Reboot host
3. Try to install a Win11 guest

Actual results:
Got "This PC can't run Windows 11" message. Win11 installer did not start the installation. 

Expected results:
Win11 guest vm installed and works. 

Additional info:

Comment 5 Yanqiu Zhang 2022-05-26 08:12:30 UTC
Reproduces for Linux guest.

Pkgs on host:
libtpms-0.9.1-0.20211126git1ff6fe1f43.el9.x86_64
swtpm-0.7.0-2.20211109gitb79fd91.el9.x86_64
libvirt-8.3.0-1.el9.x86_64
qemu-kvm-7.0.0-3.el9.x86_64
edk2-ovmf-20220221gitb24306f15d-1.el9.noarch
openssl-3.0.1-27.el9_0.x86_64

Steps:
1. Prepare a Linux guest with vtpm device, works well (when no FIPS enabled on host). 
# virsh dumpxml avocado-vt-vm1 |grep /tpm -B8
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'>
        <encryption secret='255cedc2-b6a6-4d98-b7d1-be6008676d52'/>
        <active_pcr_banks>
          <sha256/>
        </active_pcr_banks>
      </backend>
      <alias name='tpm0'/>
    </tpm>
[root@guest ~]# ls /dev|grep tpm
tpm0
tpmrm0
[root@guest ~]# tpm2_getrandom 8 --hex
ddd2f6bb7d18a678

# virsh shutdown avocado-vt-vm1 
Domain 'avocado-vt-vm1' is being shutdown

2.Enable FIPS on host and reboot host:
# fips-mode-setup --enable
Kernel initramdisks are being regenerated. This might take some time.
Setting system policy to FIPS
Note: System-wide crypto policies are applied on application start-up.
It is recommended to restart the system for the change of policies
to fully take place.
FIPS mode will be enabled.
Please reboot the system for the setting to take effect.

# reboot

# fips-mode-setup --check
FIPS mode is enabled.


3. Start the guest again, check vtpm status:
# virsh start avocado-vt-vm1 
Domain 'avocado-vt-vm1' started

# virsh console avocado-vt-vm1 
...
[root@guest ~]# ls /dev|grep tpm
(nothing output)

[root@guest ~]# tpm2_getrandom 8 --hex
ERROR:tcti:src/tss2-tcti/tcti-device.c:440:Tss2_Tcti_Device_Init() Failed to open specified TCTI device file /dev/tpmrm0: No such file or directory 
...

4. Check swtpm log, failure at TestSymmetricAlgorithm shows:
# vim /var/log/swtpm/libvirt/qemu/avocado-vt-vm1-swtpm.log
  The TPM's state will be encrypted using a key derived from a passphrase (fd).
Starting vTPM reconfiguration as tss:tss @ Thu 26 May 2022 03:49:13 AM EDT
Successfully activated PCR banks sha256 among sha1,sha256,sha384,sha512.
Successfully authored TPM state.
Ending vTPM manufacturing @ Thu 26 May 2022 03:49:13 AM EDT
libtpms/tpm2: Entering failure mode; code: 6, location: TestSymmetricAlgorithm line 230
libtpms/tpm2: TPM2_Process: Entered failure mode through command:
80 01 00 00 00 0b 00 00 01 43 00

Comment 10 Qianqian Zhu 2022-05-31 09:29:39 UTC
FYI: How to install Win11 without vTPM and UEFI
https://docs.google.com/document/d/1YIIWnhVmIFoAdh6_kz8pAF3ZJ5YyjDpVpAZAT3hqKxI/edit

Note: This is just workaround not officially support.

Comment 12 Stefan Berger 2022-06-07 11:14:52 UTC
My guess is that enablement of FIPS mode disables algorithms during runtime that were previously available to the TPM. We cannot just disable those algorithms for the TPM since their support is compiled into libtpms. We can also not easily disable them during compile time without breaking the upgrade path for VMs who have previously used such algorithms and now would have those eliminated because libtpms all of a sudden doesn't support them anymore and applications that were previously using them cannot use encrypted data anymore.

Comment 13 Marc-Andre Lureau 2022-06-07 11:55:23 UTC
(In reply to Stefan Berger from comment #12)
> My guess is that enablement of FIPS mode disables algorithms during runtime
> that were previously available to the TPM. We cannot just disable those
> algorithms for the TPM since their support is compiled into libtpms. We can
> also not easily disable them during compile time without breaking the
> upgrade path for VMs who have previously used such algorithms and now would
> have those eliminated because libtpms all of a sudden doesn't support them
> anymore and applications that were previously using them cannot use
> encrypted data anymore.

Can we have a --disable-camellia, --disable-algX... (and related --print-capabilities), so new VM can be setup with a more limited set of algorithms?

Do you know what is the minmal subset of algorithm a TPM2 must implement and where this information is?

I wish to have the list of enabled/disabled algorhithms in FIPS mode (https://csrc.nist.gov/publications/detail/fips/140/2/final doesn't seem to help me much, I guess I'll dig openssl code...)

Comment 14 Stefan Berger 2022-06-07 12:24:56 UTC
I am not sure what this would mean to the TCG TPM2 code to runtime-disable algorithms and what potential trouble we could run into with this, let alone how to thoroughly test this so that it doesn't have any unwanted side effects...

Comment 15 Daniel Berrangé 2022-06-07 15:47:44 UTC
(In reply to Marc-Andre Lureau from comment #13)

> Can we have a --disable-camellia, --disable-algX... (and related
> --print-capabilities), so new VM can be setup with a more limited set of
> algorithms?
> 
> Do you know what is the minmal subset of algorithm a TPM2 must implement and
> where this information is?
> 
> I wish to have the list of enabled/disabled algorhithms in FIPS mode
> (https://csrc.nist.gov/publications/detail/fips/140/2/final doesn't seem to
> help me much, I guess I'll dig openssl code...)

Ignoring FIPS mode for a minute, the global distro crypto-policies configuration can disable algorithms per-host too. IOW, merely disabling non-FIPS algorithms at build time feels insufficiently flexible, and we need to have a way to dynamically select algorithms based on what's available on the host at the time of vTPM instantiation for each VM.

Comment 16 Simo Sorce 2022-06-07 16:16:26 UTC
I would suggest also to not put too much value on "compatibility".

If the vTPM is started with a restricted selection of algorithms and the VM need to access one that is not available, an error condition will happen.

The operator will then be in a position to rollback/re-enable whatever change led to the issue.

If a migration fails, the migration will be rolled back, re-assessed and re-planned.

It is fine.

Comment 17 Daniel Berrangé 2022-06-07 16:21:06 UTC
Also in terms of supportability, I think it would be desirable if the TPM self-tests can be triggered when '/usr/bin/swtpm' is first launched, such that it will immediately exit with an error message if any crypto algorithms implied by the TPM config are unavailable. The intent would be that 'virsh start $GUEST' should get a failure to start swtpm, with a clear message about missing algorithms, rather than us successfully launching the guest with a broken TPM.

This kind of debuggability improvement would be useful right now, but even more important if we get dynamic configuration of algorithms selected at swtpm_create time, so we can easily detect if the VM is later launched on a different and/or upgraded and/or reconfigured host which is missing algorithms that were originally available at swtpm_create time.

Comment 18 Stefan Berger 2022-06-08 01:14:29 UTC
(In reply to Qinghua Cheng from comment #0)
> Description of problem:
> 
> Even with tpm: {} in VM spec, Windows 11 installer still gives error "This
> PC can't run Windows 11"
> 
> Version-Release number of selected component (if applicable):
> 
> RHEL 8.6
> kernel: 4.18.0-372.10.1.el8_6.x86_64
> qemu-kvm: qemu-kvm-6.2.0-11.module+el8.6.0+14712+f96656d3.x86_64
> 
> swtpm-0.7.0-1.20211109gitb79fd91.module+el8.6.0+13879+1439f356.x86_64
> libtpms-0.9.1-0.20211126git1ff6fe1f43.module+el8.6.0+13879+1439f356.x86_64
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. On host fips-mode-setup --enable
> 2. Reboot host
> 3. Try to install a Win11 guest
> 

Btw, does the installation of Win11 work on a non FIPS-enabled system and can the VM then use the TPM?

Comment 19 Daniel Berrangé 2022-06-08 08:15:15 UTC
(In reply to Stefan Berger from comment #18)
> (In reply to Qinghua Cheng from comment #0)
> > Description of problem:

> > Steps to Reproduce:
> > 1. On host fips-mode-setup --enable
> > 2. Reboot host
> > 3. Try to install a Win11 guest
> > 
> 
> Btw, does the installation of Win11 work on a non FIPS-enabled system and
> can the VM then use the TPM?

Windows 11 installs and runs fine in KVM guests with vTPM + OOVMF enabled on a non-FIPS enabled system. I don't know what exactly Win11 does with the TPM, but the installer won't even let you start the installation if you don't have the vTPM present, so I assume it is doing something with it.

Comment 20 Stefan Berger 2022-06-08 11:04:29 UTC
(In reply to Daniel Berrangé from comment #17)
> Also in terms of supportability, I think it would be desirable if the TPM
> self-tests can be triggered when '/usr/bin/swtpm' is first launched, such
> that it will immediately exit with an error message if any crypto algorithms
> implied by the TPM config are unavailable. The intent would be that 'virsh

The TPM2 self test put the device into failure mode for first encounteref compiled-in malfunctioning algorithm. I don't think we should change the behavior of the device in that respect.

Comment 21 Daniel Berrangé 2022-06-08 12:43:19 UTC
(In reply to Stefan Berger from comment #20)
> (In reply to Daniel Berrangé from comment #17)
> > Also in terms of supportability, I think it would be desirable if the TPM
> > self-tests can be triggered when '/usr/bin/swtpm' is first launched, such
> > that it will immediately exit with an error message if any crypto algorithms
> > implied by the TPM config are unavailable. The intent would be that 'virsh
> 
> The TPM2 self test put the device into failure mode for first encounteref
> compiled-in malfunctioning algorithm. I don't think we should change the
> behavior of the device in that respect.

I'm not suggesting we change the existing self test behaviour, just that 
we augment it with further checks at swtpm startup. Even if we filter 
algorithms exposed, based on what's available at time of swtpm_setup,
we can still get problems later at runtime if crypto policies changed
again. Both when QEMU is started, and at any point while QEMU is running,
so a runtime self test will always be valuable, in addition to any test
at swtpm process startup.

In a virtual env the cause of any self test failure is highly likely to
be an host / cloud administrator mistake. Leaving the failure to be
detected by the guest OS at runtime when they trigger TPM self tests
is not an effective way for the host/cloud admin to learn of their
mistake. We want to be able to discover mistakes at the earliest
opportunity, and VM startup is a preferable place to validate it
from POV of being able to get this known to the host admin. 

This would also allow guest owner orchestration tools to cope with the
problem. eg if a user's attempt to boot their VMs fails due to the
swtpm startup discovering a problem, their tools can transparently
re-spawn the VM on another host minimizing disruption.

Comment 22 Marc-Andre Lureau 2022-06-08 13:52:43 UTC
We may want to backport https://github.com/stefanberger/swtpm/pull/704 for now. It calls FIPS_mode_set(0) early and fail with a more explicit error.

Comment 23 Simo Sorce 2022-06-08 20:33:03 UTC
FIPS_mode_set() does not exist on RHEL-9 which uses OpenSSL 3 and has a completely different architecture for FIPS mode.

Comment 24 Stefan Berger 2022-06-08 21:18:22 UTC
(In reply to Simo Sorce from comment #23)
> FIPS_mode_set() does not exist on RHEL-9 which uses OpenSSL 3 and has a
> completely different architecture for FIPS mode.

That's why I am using the new EVP API calls EVP_default_properties_is_fips_enabled(NULL) and  EVP_default_properties_enable_fips(NULL, 0) for OpenSSL 3.0.

Comment 25 Stefan Berger 2022-06-09 11:47:11 UTC
Please test the patch here: https://github.com/stefanberger/swtpm/pull/704

Let me know whether this resolves the issue on RHEL 8.6. Thanks.

Comment 27 Yanqiu Zhang 2022-06-10 03:47:37 UTC
Test results for linux guest:
On rhel9.1:
swtpm-0.7.0-3.20211109gitb79fd91.el9.x86_64
libtpms-0.9.1-0.20211126git1ff6fe1f43.el9.x86_64
libvirt-8.4.0-1.el9.x86_64
qemu-kvm-7.0.0-5.el9.x86_64
edk2-ovmf-20220221gitb24306f15d-1.el9.noarch
openssl-3.0.1-27.el9_0.x86_64

# fips-mode-setup --check
FIPS mode is enabled.

xml same as comment5.
1. swtpm log:
...
  The TPM's state will be encrypted using a key derived from a passphrase (fd).
Starting vTPM reconfiguration as tss:tss @ Thu 09 Jun 2022 11:28:48 PM EDT
Warning: Disabled OpenSSL FIPS mode
Successfully activated PCR banks sha256 among sha1,sha256,sha384,sha512.
Successfully authored TPM state.
Ending vTPM manufacturing @ Thu 09 Jun 2022 11:28:48 PM EDT
Warning: Disabled OpenSSL FIPS mode
2. guest vtpm:
[root@localhost ~]# ls /dev|grep tpm
tpm0
tpmrm0
[root@localhost ~]# tpm2_getrandom --hex 8
ee978a1504c2cf85

On rhel8.6:
swtpm-0.7.0-2.20211109gitb79fd91.module+el8.6.0+13725+61ae1949.x86_64
libtpms-0.9.1-0.20211126git1ff6fe1f43.module+el8.6.0+14480+c0a3aa0f.x86_64
libvirt-8.0.0-5.module+el8.6.0+14480+c0a3aa0f.x86_64
qemu-kvm-6.2.0-11.module+el8.6.0+14707+5aa4b42d.x86_64
edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch
openssl-1.1.1k-5.el8_5.x86_64

# fips-mode-setup --check
FIPS mode is enabled.

a xml used:
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'/>
      <alias name='tpm0'/>
    </tpm>
1. swtpm log:
...
Successfully authored TPM state.
Ending vTPM manufacturing @ Thu 09 Jun 2022 11:24:55 PM EDT
Warning: Disabled OpenSSL FIPS mode
2. guest vtpm:
[root@localhost ~]# ls /dev|grep tpm
tpm0
tpmrm0
[root@localhost ~]# tpm2_getrandom --hex 8
1eabf2f355f38730

Comment 28 Qinghua Cheng 2022-06-10 06:10:56 UTC
On RHEL 8.6 

qemu-kvm: qemu-kvm-6.2.0-11.module+el8.6.0+15404+4d757596.1.x86_64

libtpms: libtpms-0.9.1-0.20211126git1ff6fe1f43.module+el8.6.0+14480+c0a3aa0f.x86_64
swtpm: swtpm-0.7.0-2.20211109gitb79fd91.module+el8.6.0+13725+61ae1949.x86_64
openssl: openssl-1.1.1k-6.el8_5.x86_64
edk2: edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch

# fips-mode-setup --check
FIPS mode is enabled.

Win11 guest installed successfully, and check tpm device status in the guest:

tpm.msc shows:
TPM device is ready to use.

Comment 29 Daniel Berrangé 2022-06-10 12:43:54 UTC
Note that if we backport the proposed upstream workaround patches to RHEL, then an installation of RHEL that uses swtpm with a VM is of course no longer able to be considered FIPS compliant.

Having swtpm TPM installed is not the end of the world, but introduces risk that a user will unwittingly use TPM making their VM non-FIPs compliant. Fortunately most virt mgmt apps don't enable TPM by default historically, though with virt-install/virt-manager we have started adding TPM whenever UEFI is used now. Users can override this, but it will be easy to miss this fact. So this is definitely sub-optimal from a RHEL FIPS compliance POV.

If we go down this route, we need to open a new bug to track the fact that swtpm is not FIPS compliant and work towards a proper solution on a pretty timely basis.

Comment 30 Amnon Ilan 2022-06-15 07:35:53 UTC
(In reply to Daniel Berrangé from comment #29)
..
> If we go down this route, we need to open a new bug to track the fact that
> swtpm is not FIPS compliant and work towards a proper solution on a pretty
> timely basis.

What is the plan/direction for a longer-term proper solution?

Comment 31 Daniel Berrangé 2022-06-15 08:03:54 UTC
(In reply to Amnon Ilan from comment #30)
> (In reply to Daniel Berrangé from comment #29)
> ..
> > If we go down this route, we need to open a new bug to track the fact that
> > swtpm is not FIPS compliant and work towards a proper solution on a pretty
> > timely basis.
> 
> What is the plan/direction for a longer-term proper solution?

Full dynamic algorithm selection at time of provisioning a new vTPM for a VM.

Comment 32 Stefan Berger 2022-06-15 10:15:48 UTC
(In reply to Daniel Berrangé from comment #31)
> (In reply to Amnon Ilan from comment #30)
> > (In reply to Daniel Berrangé from comment #29)
> > ..
> > > If we go down this route, we need to open a new bug to track the fact that
> > > swtpm is not FIPS compliant and work towards a proper solution on a pretty
> > > timely basis.
> > 
> > What is the plan/direction for a longer-term proper solution?
> 
> Full dynamic algorithm selection at time of provisioning a new vTPM for a VM.

I have some work now in this direction. The TPM 2 code needs to be instrumented to support runtime-disablement of algorithms via a profile that expresses the enabled algorithms in some sort of language whose verbs correspond to compile-time options for the TPM 2 code. For example, the compile time option ALG_SHA512 allows to  enable support for SHA-512 in the TPM 2 during build and 'sha512' as a verb, when given, allows to enable the SHA-512 algorithm and SHA-512 PCR banks during runtime then. This profile can be set upon creation of a TPM 2 instance and will then become part of the TPM 2's state so that it can migrate along with the TPM 2 and is always there. What I want to prevent is that users have control over the profile after a TPM has been created enabling them to disable or enable algorithms at will, so that's why this would become part of the TPM 2's 'state' and then cannot be changed anymore so that the applications inside a VM have a reliable set of algorithms provided by the TPM 2. The problem with the change to the TPM 2's state in turn is that the state blob now becomes incompatible with older version that cannot read it anymore due to this additional profile in the state blob. So this breaks migration from a newer vTPM to an older one. Also, libtpms needs new API call(s) to support the setting of the profile.

Discussion here: https://github.com/stefanberger/swtpm/issues/703

Comment 33 Daniel Berrangé 2022-06-15 17:31:34 UTC
(In reply to Stefan Berger from comment #32)
> (In reply to Daniel Berrangé from comment #31)
> > (In reply to Amnon Ilan from comment #30)
> > > (In reply to Daniel Berrangé from comment #29)
> > > ..
> > > > If we go down this route, we need to open a new bug to track the fact that
> > > > swtpm is not FIPS compliant and work towards a proper solution on a pretty
> > > > timely basis.
> > > 
> > > What is the plan/direction for a longer-term proper solution?
> > 
> > Full dynamic algorithm selection at time of provisioning a new vTPM for a VM.
> 
> For example, the
> compile time option ALG_SHA512 allows to  enable support for SHA-512 in the
> TPM 2 during build and 'sha512' as a verb, when given, allows to enable the
> SHA-512 algorithm and SHA-512 PCR banks during runtime then. This profile
> can be set upon creation of a TPM 2 instance and will then become part of
> the TPM 2's state so that it can migrate along with the TPM 2 and is always
> there. 

>        What I want to prevent is that users have control over the profile
> after a TPM has been created enabling them to disable or enable algorithms
> at will, so that's why this would become part of the TPM 2's 'state' and
> then cannot be changed anymore so that the applications inside a VM have a
> reliable set of algorithms provided by the TPM 2. The problem with the
> change to the TPM 2's state in turn is that the state blob now becomes
> incompatible with older version that cannot read it anymore due to this
> additional profile in the state blob. So this breaks migration from a newer
> vTPM to an older one. Also, libtpms needs new API call(s) to support the
> setting of the profile.

New VMs will have the dynamically filtered set of algorithms. If they
attempted to migrate to an old host, this would end up running on a
swtpm without dynamic filtering and thus result in a guest ABI change.
IOW, the inability todo backwards migration with the changed state
is potentially a desirable feature.

Having said that, I wonder if we want to be able to control whether
the TPM algorithms are statically set vs dynamically set, with a
property in libvirt XML config. eg to allow creation of a VM which
explicitly has the older static algorithm set to enable backwards
migration to take place.

Or maybe it is necessary to have a knob that controls the TPM state
format version, to future proof us for future possible changes in
the TPM state that would impact migratability ?

Comment 34 Stefan Berger 2022-06-15 19:09:16 UTC
> Or maybe it is necessary to have a knob that controls the TPM state
> format version, to future proof us for future possible changes in
> the TPM state that would impact migratability ?

I'd rather not do this. Typically the state blobs change when new algorithms are enabled. Let's say version N added a new key type to the TPM 2 and now such a key is loaded into the TPM and the new key needs to be marshalled for migration. You cannot migrate to N-1 since then the key would be missing and the application in the VM would not be able to use the key on version N-1 and decrypt its data anymore. I also want to have one format for the state that is *testable*.

Comment 35 Daniel Berrangé 2022-06-16 08:07:54 UTC
(In reply to Stefan Berger from comment #34)
> > Or maybe it is necessary to have a knob that controls the TPM state
> > format version, to future proof us for future possible changes in
> > the TPM state that would impact migratability ?
> 
> I'd rather not do this. Typically the state blobs change when new algorithms
> are enabled. Let's say version N added a new key type to the TPM 2 and now
> such a key is loaded into the TPM and the new key needs to be marshalled for
> migration. You cannot migrate to N-1 since then the key would be missing and
> the application in the VM would not be able to use the key on version N-1
> and decrypt its data anymore. I also want to have one format for the state
> that is *testable*.

That scenario described is fine to block backwards migrate, since the VM is relying on features that don't exist in older versions.

Consider though that the TPM code adds support for a new algorithm, but due to crypto policies on the host, the algorithm does NOT get enabled with the vTPM is provisioned. In that scenario there's no feature based reason why backwards migration could not be allowed, since the old TPM impl would still support every feature the vTPM has actually exposed to the guest.

In terms of testing, surely you already need to be testing with multiple versions of the state, since the new vTPM can be run using a state that was forwards migrated from an old host. I assuming it doesn't in-place convert the incoming migrate state to the new format, since they would then prevent the VM moving back to the original host later.

Comment 36 Stefan Berger 2022-06-16 11:26:05 UTC
(In reply to Stefan Berger from comment #34)
> > Or maybe it is necessary to have a knob that controls the TPM state
> > format version, to future proof us for future possible changes in
> > the TPM state that would impact migratability ?
> 
> I'd rather not do this. Typically the state blobs change when new algorithms
> are enabled. Let's say version N added a new key type to the TPM 2 and now

Actually it's not just algorithms, it's also when new commands are added to the TPM 2 that influence auditing state (bitfields) and potentially other internal state.

Comment 37 Stefan Berger 2022-06-16 11:34:05 UTC
(In reply to Daniel Berrangé from comment #35)
> In terms of testing, surely you already need to be testing with multiple
> versions of the state, since the new vTPM can be run using a state that was
> forwards migrated from an old host. I assuming it doesn't in-place convert
> the incoming migrate state to the new format, since they would then prevent
> the VM moving back to the original host later.

It does convert the state to the new format. Newer code can read all older formats but cannot write all the older formats since the newer code already filled in default values for whatever was missing in the older state and doesn't track whether the default values changed or something like that so that it could write out older state. This is a complex crypto device with thousands of lines of code and more than 100 (TPM2) commands that is hard to test just to begin with. If we now open up flexibility to configure it nearly at will we also loose test suites that are not prepared for this kind of flexibility. 

Also in the past bugfixes that affected the state didn't allow to fall back to older state anymore. So we have algorithms, commands and bugfixes influencing the state of the TPM 2.

Comment 38 Marc-Andre Lureau 2022-06-16 11:42:19 UTC
The tested series ("disable fips") from comment 26 is now merged upstream.

Shall we use this bug to "disable-fips" on rhel9.1 (cloning for 8.7 as well), and clone a new bug for "FIPS compatibility"?

Comment 39 Qinghua Cheng 2022-06-17 03:00:56 UTC
Cloned bz 2097939 for FIPS enabled scenario. 

Cloned bz 2097947 for rhel 8.7.

Comment 44 Yanqiu Zhang 2022-06-27 14:02:07 UTC
Verified for linux guest on:
swtpm-0.7.0-3.20211109gitb79fd91.el9.x86_64
libtpms-0.9.1-2.20211126git1ff6fe1f43.el9.x86_64
libvirt-8.4.0-3.el9.x86_64
qemu-kvm-7.0.0-6.el9.x86_64
edk2-ovmf-20220526git16779ede2d36-2.el9.noarch

Steps on a fips enabled host:
# fips-mode-setup --check
FIPS mode is enabled.

Start guest with vtpm device successfully, and find warning about FIPS in swtpm log:
...The TPM's state will be encrypted using a key derived from a passphrase (fd).
Starting vTPM reconfiguration as tss:tss @ Mon 27 Jun 2022 09:26:36 AM EDT
Warning: Disabled OpenSSL FIPS mode
Successfully activated PCR banks sha256 among sha1,sha256,sha384,sha512.
Successfully authored TPM state.
Ending vTPM manufacturing @ Mon 27 Jun 2022 09:26:36 AM EDT
Warning: Disabled OpenSSL FIPS mode

Regression test of vtpm auto cases passed on the host:
#  avocado run --vt-type libvirt --test-runner=runner --vt-machine-type q35 virtual_devices.tpm_device..emulator
 (01/52) type_specific.io-github-autotest-libvirt.virtual_devices.tpm_device.normal_test.tpm-crb_model.emulator.plain.pcrbank_default.basic: PASS (70.91 s)
...
 (52/52) type_specific.io-github-autotest-libvirt.virtual_devices.tpm_device.negative_test.emulator.pcrbank_test.managedsave_modify: PASS (80.64 s)
RESULTS    : PASS 50 | ERROR 0 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
The 2 failed by existing bz2100314

Comment 45 Qinghua Cheng 2022-06-28 01:32:39 UTC
Verified for Windows guests:

kernel: 5.14.0-114.el9.x86_64
qemu: qemu-kvm-7.0.0-6.el9.x86_64
libtpms-0.9.1-2.20211126git1ff6fe1f43.el9.x86_64
swtpm-0.7.0-3.20211109gitb79fd91.el9.x86_64
edk2-ovmf-20220526git16779ede2d36-1.el9.noarch

# fips-mode-setup --check
FIPS mode is enabled.

Win11 guest works installed an works normally. 

vtpm regression test PASS. No new bug found.

Set bug verified.

Comment 46 ncist2011 2022-08-31 03:20:29 UTC
host: centos7
kernel: 3.10.0-957.10.5.el7.x86_64
libvirt: libvirt-5.5.0-6.50.x86_64
qemu: qemu-kvm-4.2.0-29.52.el7.5.x86_64
libtpms: libtpms-0.9.5-1.el7.x86_64
swtpm: swtpm-0.7.4-1.el7.x86_64
uefi: OVMF-20180508-6.gitee3198e672e2.el7.noarch

guest: windows11.iso

guest xml:

<devices>
  <tpm model='tpm-tis'>
    <backend type='emulator' version='2.0'>
    </backend>
  </tpm>
</devices>


Actual results:
Got "This PC can't run Windows 11" message. Win11 installer did not start the installation. 

Expected results:
Win11 guest vm installed and works. 

What should i do?

Comment 47 ncist2011 2022-08-31 04:02:37 UTC
(In reply to ncist2011 from comment #46)
> host: centos7
> kernel: 3.10.0-957.10.5.el7.x86_64
> libvirt: libvirt-5.5.0-6.50.x86_64
> qemu: qemu-kvm-4.2.0-29.52.el7.5.x86_64
> libtpms: libtpms-0.9.5-1.el7.x86_64
> swtpm: swtpm-0.7.4-1.el7.x86_64
> uefi: OVMF-20180508-6.gitee3198e672e2.el7.noarch
> 
> guest: windows11.iso
> 
> guest xml:
> 
> <devices>
>   <tpm model='tpm-tis'>
>     <backend type='emulator' version='2.0'>
>     </backend>
>   </tpm>
> </devices>
> 
> 
> Actual results:
> Got "This PC can't run Windows 11" message. Win11 installer did not start
> the installation. 
> 
> Expected results:
> Win11 guest vm installed and works. 
> 
> What should i do?



swtpm log:

Starting vTPM manufacturing as tss:tss @ Wed 31 Aug 2022 11:48:24 AM CST
Successfully created RSA 2048 EK with handle 0x81010001.
  Invoking /usr/bin/swtpm_localca --type ek --ek 9799bb9a79b1f4409a43347d2117b00d7c551dc71ffdcaaf4ff0af4d6a097cda2175de2fa8e5086fb0bbf62418539e650e047fceabd396ca08ffd127596a1a310a24d1ca0403ddd0b2d61a868f0ea217fe02b0b2a9af5fe2c60a8095a0acff44987376789a4ba80241486214ff9f40c3785a464794ad11f231a124a6b23567a493f0e2b0441dd01e4943eb98bf5da94c48434cf99f5068eb828845170e7ea78aa075922921a6344ab155b12b4491008e4b65ce386cdf411d6e88c22fe6f47798d5ddaa5b3c18de828c169769466eca8dfd843ddc52e099e0327e157632f49a2794f9bcea7cecf7befddcc894c3b120a10fe0c393a0b2199badd8b675fb531ed1 --dir /tmp/swtpm_setup.certs.E0Y9Q1 --logfile /var/log/swtpm/libvirt/qemu/instance-00014826-swtpm.log --vmid instance-00014826:4e2fe573-9c11-4ec2-a556-9291eedbeb4e --tpm-spec-family 2.0 --tpm-spec-level 0 --tpm-spec-revision 164 --tpm-manufacturer id:00001014 --tpm-model swtpm --tpm-version id:20191023 --tpm2 --configfile /etc/swtpm-localca.conf --optsfile /etc/swtpm-localca.options
Successfully created EK certificate locally.
  Invoking /usr/bin/swtpm_localca --type platform --ek 9799bb9a79b1f4409a43347d2117b00d7c551dc71ffdcaaf4ff0af4d6a097cda2175de2fa8e5086fb0bbf62418539e650e047fceabd396ca08ffd127596a1a310a24d1ca0403ddd0b2d61a868f0ea217fe02b0b2a9af5fe2c60a8095a0acff44987376789a4ba80241486214ff9f40c3785a464794ad11f231a124a6b23567a493f0e2b0441dd01e4943eb98bf5da94c48434cf99f5068eb828845170e7ea78aa075922921a6344ab155b12b4491008e4b65ce386cdf411d6e88c22fe6f47798d5ddaa5b3c18de828c169769466eca8dfd843ddc52e099e0327e157632f49a2794f9bcea7cecf7befddcc894c3b120a10fe0c393a0b2199badd8b675fb531ed1 --dir /tmp/swtpm_setup.certs.E0Y9Q1 --logfile /var/log/swtpm/libvirt/qemu/instance-00014826-swtpm.log --vmid instance-00014826:4e2fe573-9c11-4ec2-a556-9291eedbeb4e --tpm-spec-family 2.0 --tpm-spec-level 0 --tpm-spec-revision 164 --tpm-manufacturer id:00001014 --tpm-model swtpm --tpm-version id:20191023 --tpm2 --configfile /etc/swtpm-localca.conf --optsfile /etc/swtpm-localca.options
Successfully created platform certificate locally.
Successfully created NVRAM area 0x1c00002 for RSA 2048 EK certificate.
Successfully created NVRAM area 0x1c08000 for platform certificate.
Successfully created ECC EK with handle 0x81010016.
  Invoking /usr/bin/swtpm_localca --type ek --ek x=9cb2bc87d6348f7578de72950a3f24daef97079a53c38c66622685e4c3d259b3f467c7b1b3e5e3b5d1d5a7901dc5b56d,y=5a5dd714c7c7d848979f95229c1907cf5b7abb7174d06e69ae4d14b59f01ea0a9dd8a736c6b2a127491e9b37cf64346f,id=secp384r1 --dir /tmp/swtpm_setup.certs.E0Y9Q1 --logfile /var/log/swtpm/libvirt/qemu/instance-00014826-swtpm.log --vmid instance-00014826:4e2fe573-9c11-4ec2-a556-9291eedbeb4e --tpm-spec-family 2.0 --tpm-spec-level 0 --tpm-spec-revision 164 --tpm-manufacturer id:00001014 --tpm-model swtpm --tpm-version id:20191023 --tpm2 --configfile /etc/swtpm-localca.conf --optsfile /etc/swtpm-localca.options
Successfully created EK certificate locally.
Successfully created NVRAM area 0x1c00016 for ECC EK certificate.
Successfully activated PCR banks sha256 among sha1,sha256,sha384,sha512.
Successfully authored TPM state.
Ending vTPM manufacturing @ Wed 31 Aug 2022 11:48:25 AM CST


windows11 guest:

/usr/libexec/qemu-kvm -name guest=instance-00014826,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-337-instance-00014826/master-key.aes -machine pc-q35-rhel8.2.0,accel=kvm,usb=off,dump-guest-core=off -cpu cpu64-rhel6,abm=off,aes=on,apic=on,arat=on,avx=on,clflush=on,cmov=on,cx16=on,cx8=on,dca=on,de=on,dtes64=on,erms=on,est=on,f16c=on,fpu=on,fsgsbase=on,fxsr=on,ht=on,lahf-lm=on,lm=on,mca=on,mce=on,mmx=on,monitor=on,msr=on,mtrr=on,nx=on,pae=on,pat=on,pbe=on,pcid=on,pclmulqdq=on,pdcm=on,pdpe1gb=on,pge=on,popcnt=on,pse=on,pse36=on,rdrand=on,rdtscp=on,sep=on,smep=on,smx=on,spec-ctrl=on,ss=on,ssbd=on,sse=on,sse2=on,sse4.1=on,sse4.2=on,ssse3=on,syscall=on,tm=on,tm2=on,tsc=on,vme=on,vmx=on,xsave=on,xsaveopt=on,xtpr=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0xfff,hv-vpindex,hv-synic,hv-stimer -drive file=/usr/share/OVMF/OVMF_CODE.secboot.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/instance-00014826_VARS.fd,if=pflash,format=raw,unit=1 -m size=8388608k,slots=128,maxmem=536870912k -overcommit mem-lock=off -smp 8,maxcpus=128,sockets=128,cores=1,threads=1 -numa node,nodeid=0,cpus=0-127,mem=8192 -uuid 4e2fe573-9c11-4ec2-a556-9291eedbeb4e -smbios type=1,manufacturer=Huayun,product=Huayun S1,version=2022.5.8-1.el7.centos,serial=4c4c4544-0050-3410-8052-c7c04f473332,uuid=4e2fe573-9c11-4ec2-a556-9291eedbeb4e,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=35,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-pci-bridge,id=pci.3,bus=pci.1,addr=0x0 -device pcie-root-port,port=0x12,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x6 -device qemu-xhci,id=usb,bus=pci.4,addr=0x0 -device virtio-serial-pci,id=virtio-serial0,bus=pci.5,addr=0x0 -drive file=/maxta//cinder/volumes/volume_620/volume-481aa160-460c-49a9-ab77-85678bc6cd69,format=raw,if=none,id=drive-virtio-disk0,cache=none,discard=unmap,aio=native -device virtio-blk-pci,scsi=off,num-queues=1,bus=pci.6,addr=0x0,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2,write-cache=on,serial=481aa160-460c-49a9-ab77-85678bc6cd69 -drive file=/maxta//cinder/volumes/volume_641/volume-07d502e5-0f84-4a10-aefe-bf4a0e99e3ae,format=raw,if=none,id=drive-sata0-0-0,readonly=on,discard=unmap -device ide-cd,bus=ide.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -netdev tap,fds=36:37,id=hostnet0,vhost=on,vhostfds=38:39 -device virtio-net-pci,mq=on,vectors=6,netdev=hostnet0,id=net0,mac=fa:16:3e:25:0a:2c,bus=pci.2,addr=0x0 -chardev file,id=charserial0,path=/var/lib/nova/instances/4e2fe573-9c11-4ec2-a556-9291eedbeb4e/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -chardev socket,id=charchannel0,fd=40,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.1 -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm -chardev socket,id=chrtpm,path=/var/run/libvirt/qemu/swtpm/337-instance-00014826-swtpm.sock -device tpm-tis,tpmdev=tpm-tpm0,id=tpm0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 118.37.12.10:1 -device cirrus-vga,id=video0,bus=pcie.0,addr=0x1 -device AC97,id=sound0,bus=pci.3,addr=0x1 -device virtio-balloon-pci,id=balloon0,bus=pci.7,addr=0x0,deflate-on-oom=on -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on


swtpm socket:

/usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/var/run/libvirt/qemu/swtpm/337-instance-00014826-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/4e2fe573-9c11-4ec2-a556-9291eedbeb4e/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/instance-00014826-swtpm.log --tpm2 --pid file=/var/run/libvirt/qemu/swtpm/337-instance-00014826-swtpm.pid

Comment 48 Marc-Andre Lureau 2022-08-31 06:43:31 UTC
@ncist2011, use tpm-crb (not tpm-tis)

Comment 49 ncist2011 2022-09-02 03:18:47 UTC
(In reply to Marc-Andre Lureau from comment #48)
> @ncist2011, use tpm-crb (not tpm-tis)

First, i boot windows11 guest with legacy(not uefi)
then, disable secureboot check like this:

REG ADD HKLM\SYSTEM\Setup\LabConfig /v BypassSecureBootCheck /t REG_DWORD /d 1

win11 guest installed successfully.

So, i suspect the uefi(OVMF-20180508-6.gitee3198e672e2.el7.noarch) is incompatible with vTPM.

Comment 51 errata-xmlrpc 2022-11-15 10:20:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: swtpm security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:8100

Comment 52 yalzhang@redhat.com 2022-12-12 09:09:56 UTC
Hi Marc, I have checked that this issue is fixed on rhel 9.1.0(this bug), and did not backport to rhel 9.0.0.z. But it is fixed on rhel 8.7(Bug 2097947), and cloned to 8.6.z(Bug 2109568). 
So do we need to backport this fix to rhel 9.0.0.z?

Comment 53 Marc-Andre Lureau 2022-12-12 13:11:15 UTC
Yes, we should do the backport for 9.0z. thanks

Comment 54 John Ferlan 2022-12-12 14:08:46 UTC
Requesting z-stream in order to ensure the 8.6.z-stream resolved bug 2109568 does not have a regression when/if migrate to RHEL 9.0.z


Note You need to log in before you can comment on or make changes to this bug.