Created attachment 1606430 [details]
failing case Te.wtl

Lots of history can be found starting at https://bugzilla.redhat.com/show_bug.cgi?id=1519013#c45

Created attachment 1618147 [details]
Test results on win 2019

pkg info for win 2019 test evn:

qemu-kvm-4.1.0-7.module+el8.1.0+4177+896cb282.x86_64
kernel-4.18.0-141.el8.x86_64
edk2-ovmf-20190308git89910a39dcfd-6.el8.noarch
libtpms-0.6.1-0.20190121git9dc915572b.module+el8.1.0+3523+b348b848.2.x86_64
swtpm-0.1.0-1.20190425gitca85606.module+el8.1.0+3966+4a23dca1.1.x86_64

BTW: Can you please run the test in Windows Server 2019 as well? Thanks.

(In reply to FuXiangChun from comment #1)
> Created attachment 1606430 [details]
> failing case Te.wtl

There are several failing test cases. Can you please attach the whole test package?

You can also gather logs from MS TPM driver during the test by following those steps:
https://docs.microsoft.com/en-us/previous-versions/windows/hardware/hck/hh998628(v=vs.85)?redirectedfrom=MSDN

(In reply to Yan Vugenfirer from comment #8)
> (In reply to FuXiangChun from comment #1)
> > Created attachment 1606430 [details]
> > failing case Te.wtl
> 
> There are several failing test cases. Can you please attach the whole test
> package?

Looking at those failing test cases it's hard to make sense of the reason why they are failing and what is actually being tested. Often the error code is 0x0 which makes this look like a success.

I downloaded hlk playlist and re-ran test cases from the playlist. 

When I run some test cases with secure boot enabled = on, windows can not boot. So I have to set secure boot = off to boot windows, and run the rest test cases. 

And PCR7 configuration is in Not Available status. 

You could find the whole package here: 
http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/bug1744045/win2019vtpm1.hlkx

FYI: PCR7 is related to BIOS/firmware measurements. Is "And PCR7 configuration is in Not Available status. " a reported failure from HLK?

Get PCR7 status information from: 

open msinfo32 and check the following:

    Secure boot State: ON (ON by default unless exclusively asked to set it to OFF)

    PCR7 configuration: Bound or Binding possible

Please also refer the following link:

https://docs.microsoft.com/en-us/windows-hardware/test/hlk/testref/tpm-system-fundamentals-testing-prerequisites#before-you-run-the-tpm-system-fundamentals-tests

Hi developer,

I run svvp test with TPM device, there are 8 cases about TPM, only 2 passed. Could you help check if my config is wrong or they are real bugs. Thanks in advance.

Steps:
1. Create socket file and start it in host.
# mkdir  /tmp/mytpm
# python3 -c "import socket as s; sock = s.socket(s.AF_UNIX); sock.bind('/tmp/guest-swtpm.sock')"
# /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/tmp/swtpm/guest-swtpm.sock,mode=0600 --tpmstate dir=/tmp/mytpm,mode=0600 --tpm2 


2.boot up win2019 guest with tmp device(q35 + ovmf), and enable secure boot.

-blockdev node-name=file_ovmf_code,driver=file,read-only=on,filename=/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd \
-blockdev node-name=file_ovmf_vars,driver=file,filename=OVMF_VARS-2.secboot.fd \
-machine pflash0=file_ovmf_code,pflash1=file_ovmf_vars \

-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
-chardev socket,id=chrtpm,path=/tmp/swtpm/guest-swtpm.sock \
-device tpm-crb,tpmdev=tpm-tpm0,id=tpm0 \

3.check tpm is ready in guest with tpm.msc tool.

4.open msinfo32 and check the following:
Secure boot State: ON 
PCR7 configuration: Elevation Required to View

4. run tpm related jobs in hlk



Results:

Pssed -2
TPM 2.0 TCG Physical Presence Interface 1.3 Test
TPM 2.0 Attestation Test

Failed -6
1. TPM 2.0 EK Certificate Tests
Error checking for EK Certificate. If you have an fTPM, make sure the test device has network access. Check with the TPM manufacturer about getting an EK Certificate. HR: 0x80070032
2. TPM 2.0 - Supplemental test
Interrupt resource for TPM device not present.
Interrupt support could not be initialized by TPM driver.
3. TPM 2.0 Core Provisioning Test
Information: 262144 (0x40000) //means: The EK Certificate was not read from the TPM NV Ram and stored in the registry.  
INFORMATION_EK_CERTIFICATE
4. TPM 2.0 Platform Crypto Provider Key Storage Provider Test
Endorsement Key (EK) certificate not found. Some systems require internet connectivity to locate the EK certificate
5. TPM 2.0 UEFI Preboot Interface Test
----Reading the initial bit state FAILED: Status 0xc0000225
----Setting both the MOR bit and the DAD bit FAILED: Status 0xc000000d
----Clearing both the MOR bit and the DAD bit FAILED: Status 0xc0000225
https://docs.microsoft.com/en-us/windows-hardware/test/hlk/testref/9a9fcb0b-2b95-4360-9f06-7022e68a6c71
6. BitLocker Tpm+PIN+ USB and Recovery Password tests
Cannot Find Pattern "\\HLK-1809\tests\AMD64\nttest\basetest\ngscb\tpm20\Provision.ps1"
Error Code: 0x80070002 (The system cannot find the file specified)

all logs are in the following link:
http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/bug1744045/TPM-SVVP-win2019.hlkx

pkg:
libtpms-0.7.0-1.20191018gitdc116933b7.module+el8.2.0+4793+b09dd2fb.x86_64
libtpms-0.7.0-1.20191018gitdc116933b7.module+el8.2.0+4793+b09dd2fb.x86_64
qemu-kvm-4.2.0-21.module+el8.2.1+6586+8b7713b9.x86_64
kernel-4.18.0-193.el8.x86_64
virtio-win-1.9.11-1.el8.iso

TPM 2 tests (non-UEFI related ones) that are not working may be related to:

- TPM Manufacturer: The manufacturer string "IBM " contains a space at the end, which I was going to remove (and make updates for libtpms 0.6.2 and libtpms 0.7.1) . This is the reason why one of the subtests is failing -- due to the space at the end.

- (Endorsement key & platform) certificate authority: The vTPM instances are created upon start of a VM by launching swtpm_setup the very first time the VM is started. Libvirt would do this automatically. The TPM 2 gets certificates that are created by a certificate authority that has either been setup by the user or been created on the fly. Neither one of those certificate authorities will be a well known certificate authority known to issue TPM 2 certificates. Manufacturers of TPM 2's have known certificate authorities and those ones are known to Microsoft for example. So we cannot pass tests related to known certificate authorities.

- The vTPM returns a unknown version identifier: There's not much we can do about this unless Microsoft was to take the identifier and put it into their list of accepted TPM identifiers. I am not sure what the identifier is constructed of but it may contain TPM 2 'firmware' date and version information that may then subsequently change with updates to the TPM 2 code used by libtpms.

- Emulated hardware is not providing interrupts: Neither the CRB nor the TIS interface in QEMU are providing interrupts to notify drivers about events. Both are successfully used in polling mode by Linux and also Windows. I think there's nothing wrong with using the device in polling mode.

- There's a TPM based virtual smart card test that is failing for unknown reasons. I am not sure why that is since I don't know what is going on inside this test, so it's hard to say why the test is failing or on what level.

This should be 'it.' for non-UEFI related TPM 2 tests. No other tests should be failing assuming the VM is running with EDK2/UEFI and one follows the prompt for clearing the TPM 2 via F12 key when asked for by EDK2.

(In reply to xiagao from comment #14)
> Hi developer,
> 
> I run svvp test with TPM device, there are 8 cases about TPM, only 2 passed.
> Could you help check if my config is wrong or they are real bugs. Thanks in
> advance.
> 
> Steps:
> 1. Create socket file and start it in host.
> # mkdir  /tmp/mytpm
> # python3 -c "import socket as s; sock = s.socket(s.AF_UNIX);
> sock.bind('/tmp/guest-swtpm.sock')"
> # /usr/bin/swtpm socket --daemon --ctrl
> type=unixio,path=/tmp/swtpm/guest-swtpm.sock,mode=0600 --tpmstate
> dir=/tmp/mytpm,mode=0600 --tpm2 
> 
> 
> 2.boot up win2019 guest with tmp device(q35 + ovmf), and enable secure boot.
> 
> -blockdev
> node-name=file_ovmf_code,driver=file,read-only=on,filename=/usr/share/edk2/
> ovmf/OVMF_CODE.secboot.fd \
> -blockdev
> node-name=file_ovmf_vars,driver=file,filename=OVMF_VARS-2.secboot.fd \
> -machine pflash0=file_ovmf_code,pflash1=file_ovmf_vars \
> 
> -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
> -chardev socket,id=chrtpm,path=/tmp/swtpm/guest-swtpm.sock \
> -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0 \

Can you not use libvirt to define and start a VM. It will launch swtpm_setup automatically to create the certificates and inject them into the TPM 2. This tool simulates some TPM 2 manufacturing steps.


> 
> 3.check tpm is ready in guest with tpm.msc tool.
> 
> 4.open msinfo32 and check the following:
> Secure boot State: ON 
> PCR7 configuration: Elevation Required to View
> 
> 4. run tpm related jobs in hlk
> 
> 
> 
> Results:
> 
> Pssed -2
> TPM 2.0 TCG Physical Presence Interface 1.3 Test
> TPM 2.0 Attestation Test
> 
> Failed -6
> 1. TPM 2.0 EK Certificate Tests
> Error checking for EK Certificate. If you have an fTPM, make sure the test
> device has network access. Check with the TPM manufacturer about getting an
> EK Certificate. HR: 0x80070032

The only failure should be related to the known certificate authority (CA). You have to run swtpm_setup *before* you start the swtpm. Libvirt would do this for you.

swtpm_setup --tpm2 --tpmstate /tmp/mytpm --create-ek-cert --create-platform-cert [ --overwrite ]


> 2. TPM 2.0 - Supplemental test
> Interrupt resource for TPM device not present.
> Interrupt support could not be initialized by TPM driver.

Neither CRB nor TIS interfaces provide interrupts. Both work fine in polling mode.

> 3. TPM 2.0 Core Provisioning Test
> Information: 262144 (0x40000) //means: The EK Certificate was not read from
> the TPM NV Ram and stored in the registry.  
> INFORMATION_EK_CERTIFICATE

Anything relate to EK certificate (other than known CA) should go away once you run swtpm_setup before you start swtpm -- or just use libvirt.

> 4. TPM 2.0 Platform Crypto Provider Key Storage Provider Test
> Endorsement Key (EK) certificate not found. Some systems require internet
> connectivity to locate the EK certificate

Anything relate to EK certificate (other than known CA) should go away once you run swtpm_setup before you start swtpm -- or just use libvirt.

Hi Stefan,
Could you help me out about the following quesitons?Thanks you.

After run swtpm_setup *before* starting the swtpm.
- TPM 2.0 EK Certificate Tests failed with following error.
fail error:
The below will always fail and is required to be filtered.
If an error is not being filtered, then the Authority Key Identifier is not known by Microsoft.
Please contact tpmkit with this log for assistance with this error.
 WexTraceInfo ThreadId=2772 ProcessId=664 TimeStamp=2314840443 LogSessionId=1  


- TPM 2.0 - Supplemental test
From comment 16, you mean it's normal for qemu, so in order to pass this job in svvp test, 
if we should ask for errata from MSFT.


- TPM 2.0 UEFI Preboot Interface Test
disable secure boot, failed with this error.
----Reading the initial bit state FAILED: Status 0xc0000225
----Setting both the MOR bit and the DAD bit FAILED: Status 0xc000000d
----Clearing both the MOR bit and the DAD bit FAILED: Status 0xc0000225
enable secure boot, failed with this error
Exit code 1. Error: An error has occurred setting the element data.
The value is protected by Secure Boot policy and cannot be modified or deleted.

Do you have any idea about this case?


- BitLocker Tpm+PIN+ USB and Recovery Password tests
Cannot Find Pattern "\\HLK-1809\tests\AMD64\nttest\basetest\ngscb\tpm20\Provision.ps1"
Error Code: 0x80070002 (The system cannot find the file specified)

And this one ?

all logs:
http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/bug1744045/TPM-SVVP-win2019-2.hlkx

BR,
xiaoling

Created attachment 1688903 [details]
TPM Hardware Interface test - screen log

Which revision of TPM are you testing? I'm looking at one of the tests 
('TPM 2.0 - Hardware Interface') and during running the test I'm getting 
information that "This LibTester is only aware of TPM revisions up to 1.46. 
Some test case may fail". Note that this message is not stored in logs; testing 
framework is displaying this for a short perdiod of time on 
the guest screen. So it looks like that HLK 1903 doesn't support TPM interface 
revision greater then 1.46

I've dumped messages visible on the guest screen and attached.

(In reply to Marek Kedzierski from comment #20)
> Which revision of TPM are you testing? I'm looking at one of the tests 
> ('TPM 2.0 - Hardware Interface') and during running the test I'm getting 
> information that "This LibTester is only aware of TPM revisions up to 1.46. 
> Some test case may fail". Note that this message is not stored in logs;
> testing 
> framework is displaying this for a short perdiod of time on 
> the guest screen. So it looks like that HLK 1903 doesn't support TPM
> interface 
> revision greater then 1.46
> 
> I've dumped messages visible on the guest screen and attached.

I am just fixing the Elliptic Curve error you are seeing there. I queued the patches in libtpms's stable-0.6.0.next, stable-0.7.0.next, and master.next trees.

(In reply to xiagao from comment #18)
> Hi Stefan,
> Could you help me out about the following quesitons?Thanks you.
> 
> After run swtpm_setup *before* starting the swtpm.
> - TPM 2.0 EK Certificate Tests failed with following error.
> fail error:
> The below will always fail and is required to be filtered.
> If an error is not being filtered, then the Authority Key Identifier is not
> known by Microsoft.
> Please contact tpmkit with this log for assistance with this
> error.
>  WexTraceInfo ThreadId=2772 ProcessId=664 TimeStamp=2314840443
> LogSessionId=1  

This is the one related to the known CA, so yes, this one will always fail since we don't have a known certificate authority for the vTPMs we create.

> 
> 
> - TPM 2.0 - Supplemental test
> From comment 16, you mean it's normal for qemu, so in order to pass this job
> in svvp test, 
> if we should ask for errata from MSFT.

Are you referring to the test checking whether interrupt is being used? If so, yes, it will fail. Windows ( & Linux) has to use the TIS and CRB interfaces in polling mode since we don't provide interrupts.

> 
> 
> - TPM 2.0 UEFI Preboot Interface Test
> disable secure boot, failed with this error.
> ----Reading the initial bit state FAILED: Status 0xc0000225
> ----Setting both the MOR bit and the DAD bit FAILED: Status 0xc000000d
> ----Clearing both the MOR bit and the DAD bit FAILED: Status 0xc0000225
> enable secure boot, failed with this error
> Exit code 1. Error: An error has occurred setting the element data.
> The value is protected by Secure Boot policy and cannot be modified or
> deleted.
> 
> Do you have any idea about this case?

I have not seen those errors under this test. The error I have seen are related to ParseAndLogFile.


> 
> 
> - BitLocker Tpm+PIN+ USB and Recovery Password tests
> Cannot Find Pattern
> "\\HLK-1809\tests\AMD64\nttest\basetest\ngscb\tpm20\Provision.ps1"
> Error Code: 0x80070002 (The system cannot find the file specified)
> 
> And this one ?

I don't know what is supposed to create this file or why it is missing. I don't have an answer.

(In reply to Stefan Berger from comment #21)
> (In reply to Marek Kedzierski from comment #20)
> > Which revision of TPM are you testing? I'm looking at one of the tests 
> > ('TPM 2.0 - Hardware Interface') and during running the test I'm getting 
> > information that "This LibTester is only aware of TPM revisions up to 1.46. 
> > Some test case may fail". Note that this message is not stored in logs;
> > testing 
> > framework is displaying this for a short perdiod of time on 
> > the guest screen. So it looks like that HLK 1903 doesn't support TPM
> > interface 
> > revision greater then 1.46
> > 
> > I've dumped messages visible on the guest screen and attached.
> 
> I am just fixing the Elliptic Curve error you are seeing there. I queued the
> patches in libtpms's stable-0.6.0.next, stable-0.7.0.next, and master.next
> trees.

It looks like that a lot TPM 2.0 tests are failing because of that elliptic curve.
One more question: are there huge differences between TPM revision 1.46 and 1.59?
I'm asking because as I said only revisions <= 1.46 were supported by HLK 1903.
Is there a possiblity to switch easily to TPM 1.46 to make sure that interface meets HLK 
requirements? Currently it is hard to say which tests are failing (if any?)
because of revision 1.59. Another option is to use new HLK 2004 and see which TPM revision
is supported there.

(In reply to Marek Kedzierski from comment #23)
> 
> It looks like that a lot TPM 2.0 tests are failing because of that elliptic
> curve.

If you could try to recompile libtpms from the sources of these trees here - whichever one is closest to the one you were using. Yhey have the elliptic curve fixes:

https://github.com/stefanberger/libtpms/commits/stable-0.7.0.next
https://github.com/stefanberger/libtpms/commits/stable-0.6.0.next

./configure --prefix=/usr --with-tpm2 --with-openssl
make 
make check
sudo make install

(make sure the libtpms symbolic links in /lib64/ point to the one you installed if you end up doing this)

> One more question: are there huge differences between TPM revision 1.46 and
> 1.59?

1.59 is a superset of functionality compared to 1.46, possibly some fixes, but not too many. A test for 1.46 will therefore only test a subset of 1.59 functionality.

> I'm asking because as I said only revisions <= 1.46 were supported by HLK
> 1903.

It *shouldn't matter* unless it refuses to test based on revision number.

> Is there a possiblity to switch easily to TPM 1.46 to make sure that
> interface meets HLK 
> requirements? Currently it is hard to say which tests are failing (if any?)
> because of revision 1.59. Another option is to use new HLK 2004 and see
> which TPM revision
> is supported there.

There is no easy way to switch to a certain revision. I make the libtpms releases without coordination with others so libtpms releases just have some revision of TPM 2 code. But really, it shouldn't matter when it comes to test suites that test subsets of a revision possibly implementing a superset of functionality. Or so I think at the moment. :-)

(In reply to Stefan Berger from comment #24)
> (In reply to Marek Kedzierski from comment #23)
> > 
> > It looks like that a lot TPM 2.0 tests are failing because of that elliptic
> > curve.
> 
> If you could try to recompile libtpms from the sources of these trees here -
> whichever one is closest to the one you were using. Yhey have the elliptic
> curve fixes:
> 
> https://github.com/stefanberger/libtpms/commits/stable-0.7.0.next
> https://github.com/stefanberger/libtpms/commits/stable-0.6.0.next
> 
> ./configure --prefix=/usr --with-tpm2 --with-openssl
> make 
> make check
> sudo make install
> 
> (make sure the libtpms symbolic links in /lib64/ point to the one you
> installed if you end up doing this)

I compiled your branch (0.7.0.next) and ran two of failig tests again ('Hardware Interface' and
'NV Storage, Policy, Error Handling and Attestation') It took some time, but both were passed!
I'll run rest of the ramainig TPM 2.0 tests next week and let you know.  

> 
> > One more question: are there huge differences between TPM revision 1.46 and
> > 1.59?
> 
> 1.59 is a superset of functionality compared to 1.46, possibly some fixes,
> but not too many. A test for 1.46 will therefore only test a subset of 1.59
> functionality.
> 
> > I'm asking because as I said only revisions <= 1.46 were supported by HLK
> > 1903.
> 
> It *shouldn't matter* unless it refuses to test based on revision number.
> 
> > Is there a possiblity to switch easily to TPM 1.46 to make sure that
> > interface meets HLK 
> > requirements? Currently it is hard to say which tests are failing (if any?)
> > because of revision 1.59. Another option is to use new HLK 2004 and see
> > which TPM revision
> > is supported there.
> 
> There is no easy way to switch to a certain revision. I make the libtpms
> releases without coordination with others so libtpms releases just have some
> revision of TPM 2 code. But really, it shouldn't matter when it comes to
> test suites that test subsets of a revision possibly implementing a superset
> of functionality. Or so I think at the moment. :-)

(In reply to Stefan Berger from comment #16)
> TPM 2 tests (non-UEFI related ones) that are not working may be related to:
> 
> - TPM Manufacturer: The manufacturer string "IBM " contains a space at the
> end, which I was going to remove (and make updates for libtpms 0.6.2 and
> libtpms 0.7.1) . This is the reason why one of the subtests is failing --
> due to the space at the end.
> 
> - (Endorsement key & platform) certificate authority: The vTPM instances are
> created upon start of a VM by launching swtpm_setup the very first time the
> VM is started. Libvirt would do this automatically. The TPM 2 gets
> certificates that are created by a certificate authority that has either
> been setup by the user or been created on the fly. Neither one of those
> certificate authorities will be a well known certificate authority known to
> issue TPM 2 certificates. Manufacturers of TPM 2's have known certificate
> authorities and those ones are known to Microsoft for example. So we cannot
> pass tests related to known certificate authorities.
> 
> - The vTPM returns a unknown version identifier: There's not much we can do
> about this unless Microsoft was to take the identifier and put it into their
> list of accepted TPM identifiers. I am not sure what the identifier is
> constructed of but it may contain TPM 2 'firmware' date and version
> information that may then subsequently change with updates to the TPM 2 code
> used by libtpms.
> 
> - Emulated hardware is not providing interrupts: Neither the CRB nor the TIS
> interface in QEMU are providing interrupts to notify drivers about events.
> Both are successfully used in polling mode by Linux and also Windows. I
> think there's nothing wrong with using the device in polling mode.

I want to make a general comment regarding the functionality required by MS certifications.
While there are many cases where the device can be functional both on Linux on Windows without implementing specific functionality required by WHQL tests, still if we want to achieve WHQL certification for such a device we most probably will have to implement such functionality. There is a way to ask for exceptions, but they should be well justified. An example of such exception is wake up on lan functionality for paravirtualized network device).
Regarding TPM - in order to pass SVVP certification (hypervisor certification) we will have to enable TPM for VMs.
And part of the system certification policy is that all the devices in the system must be individually certified.

> 
> - There's a TPM based virtual smart card test that is failing for unknown
> reasons. I am not sure why that is since I don't know what is going on
> inside this test, so it's hard to say why the test is failing or on what
> level.
> 
> This should be 'it.' for non-UEFI related TPM 2 tests. No other tests should
> be failing assuming the VM is running with EDK2/UEFI and one follows the
> prompt for clearing the TPM 2 via F12 key when asked for by EDK2.

(In reply to Yan Vugenfirer from comment #26)

> 
> I want to make a general comment regarding the functionality required by MS
> certifications.
> While there are many cases where the device can be functional both on Linux
> on Windows without implementing specific functionality required by WHQL
> tests, still if we want to achieve WHQL certification for such a device we
> most probably will have to implement such functionality. There is a way to
> ask for exceptions, but they should be well justified. An example of such
> exception is wake up on lan functionality for paravirtualized network
> device).

When it comes to interrupt support for TIS, this is something we can probably do much easier than for example having a well known CA for certificates or having HLK know and accept the identifier of the software TPM that may change with every upgrade of the TPM 2 code. The TPM manufacturers have the known CAs for issuing certificates for the endorsement key and for creating the platform certificate, but for a software TPM created on the fly there is no known CA. Basically every user or organization would have to have its CA that others would have to trust. When it comes to the TPM identifier, this is something we would have to publish when a new release is made for libtpms and for MS to accept. But there's still the hurdle of the CA...

I have actually tried to enable the interrupts for the TPM TIS and sent related patches to the Linux TPM driver community. It backfired because some Lenovo machines received an interrupt storm and couldn't deal with the TIS having interrupts. I ended up withdrawing the Linux patches and never published the QEMU patches. Also, I think there are lots of machines out in the field working with TPM in polling mode... actually from what it looked like this is the majority.

> Regarding TPM - in order to pass SVVP certification (hypervisor
> certification) we will have to enable TPM for VMs.
> And part of the system certification policy is that all the devices in the
> system must be individually certified.
>

(In reply to Stefan Berger from comment #27)
> (In reply to Yan Vugenfirer from comment #26)
> 
> > 
> > I want to make a general comment regarding the functionality required by MS
> > certifications.
> > While there are many cases where the device can be functional both on Linux
> > on Windows without implementing specific functionality required by WHQL
> > tests, still if we want to achieve WHQL certification for such a device we
> > most probably will have to implement such functionality. There is a way to
> > ask for exceptions, but they should be well justified. An example of such
> > exception is wake up on lan functionality for paravirtualized network
> > device).
> 
> When it comes to interrupt support for TIS, this is something we can
> probably do much easier than for example having a well known CA for
> certificates or having HLK know and accept the identifier of the software
> TPM that may change with every upgrade of the TPM 2 code. The TPM
> manufacturers have the known CAs for issuing certificates for the
> endorsement key and for creating the platform certificate, but for a
> software TPM created on the fly there is no known CA. Basically every user
> or organization would have to have its CA that others would have to trust.
> When it comes to the TPM identifier, this is something we would have to
> publish when a new release is made for libtpms and for MS to accept. But
> there's still the hurdle of the CA...

This is also an interesting question. Here in Red Hat we already have a certificate that is created by well-known certificate authority. Each time we build Windows drivers release we are signing the binaries with this key. Can it be used as an endorsement key during the test? Unfortunately, my knowledge of providing initial keys to SW TPM is limited.

> 
> I have actually tried to enable the interrupts for the TPM TIS and sent
> related patches to the Linux TPM driver community. It backfired because some
> Lenovo machines received an interrupt storm and couldn't deal with the TIS
> having interrupts. I ended up withdrawing the Linux patches and never
> published the QEMU patches. Also, I think there are lots of machines out in
> the field working with TPM in polling mode... actually from what it looked
> like this is the majority.
> 
> > Regarding TPM - in order to pass SVVP certification (hypervisor
> > certification) we will have to enable TPM for VMs.
> > And part of the system certification policy is that all the devices in the
> > system must be individually certified.
> >

(In reply to Yan Vugenfirer from comment #28)

> This is also an interesting question. Here in Red Hat we already have a
> certificate that is created by well-known certificate authority. Each time
> we build Windows drivers release we are signing the binaries with this key.
> Can it be used as an endorsement key during the test? Unfortunately, my
> knowledge of providing initial keys to SW TPM is limited.
> 

I would say no. Hardware TPMs get their certificates during manufacturing and the vendors create the TPM 2 endorsement key and sign it with their well known CA while keeping the CA in a safe place (HW device). These CAs are also likely used for no other purpose than signing TPM 2 certificates. There's no such CA for dynamically created vTPM instances.

(In reply to Stefan Berger from comment #29)
> (In reply to Yan Vugenfirer from comment #28)
> 
> > This is also an interesting question. Here in Red Hat we already have a
> > certificate that is created by well-known certificate authority. Each time
> > we build Windows drivers release we are signing the binaries with this key.
> > Can it be used as an endorsement key during the test? Unfortunately, my
> > knowledge of providing initial keys to SW TPM is limited.
> > 
> 
> I would say no. Hardware TPMs get their certificates during manufacturing
> and the vendors create the TPM 2 endorsement key and sign it with their well
> known CA while keeping the CA in a safe place (HW device). These CAs are
> also likely used for no other purpose than signing TPM 2 certificates.
> There's no such CA for dynamically created vTPM instances.

Might it make sense in the initial phase to use passthrough for certification purposes (SVVP)?

(In reply to Yan Vugenfirer from comment #30)

> 
> Might it make sense in the initial phase to use passthrough for
> certification purposes (SVVP)?

One would be testing a completely different type of device than what would (have to) be used in the end.

I wonder, though, whether other hypervisor vendors' vTPMs are certified and what exception they received. They must have the same problem. I don't see any vendor selling a hypervisor with the same well known CA to all its customers.

Removing needinfo. 
The failures under investigation.

(In reply to Stefan Berger from comment #10)
> (In reply to Yan Vugenfirer from comment #8)
> > (In reply to FuXiangChun from comment #1)
> > > Created attachment 1606430 [details]
> > > failing case Te.wtl
> > 
> > There are several failing test cases. Can you please attach the whole test
> > package?
> 
> Looking at those failing test cases it's hard to make sense of the reason
> why they are failing and what is actually being tested. Often the error code
> is 0x0 which makes this look like a success.

I took a look at [attachment 1606430 [details]] and compared it results without elliptic curves fix.
I think all of them failed because of elliptic curves issue.

Test results with HLK and libtpms are available here now: https://github.com/stefanberger/libtpms/wiki/Testing-of-libtpms-Functionality

Please find the latest test results here:

http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/bug1744045/libtpms_rebase_win2019_cert.hlkx

Test environment:

RHEL 8.3 
kernel: 4.18.0-221.el8.x86_64
qemu-kvm: qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64
libtpms: libtpms-0.7.2-1.20200527git7325acb477.module+el8.3.0+7068+4e1b8df5.x86_64
swtpm: swtpm-0.3.0-1.20200218git74ae43b.module+el8.3.0+7102+0b79cc3a.x86_64
edk2: edk2-ovmf-20200602gitca407c7246bf-1.el8.noarch

windows guest: win2019 en_windows_server_2019_updated_may_2020_x64_dvd_5651846f.iso

Steps:
1. swtpm_setup --tpm2 --tpmstate /tmp/mytpm --create-ek-cert --create-platform-cert --overwrite
2. /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/tmp/guest-swtpm.sock,mode=0600 --tpmstate dir=/tmp/mytpm,mode=0600 --tpm2
3. Start VM with vtpm device by qemu-kvm cmd:
    ...
    -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
    -chardev socket,id=chrtpm,path=/tmp/guest-swtpm.sock \
    -device tpm-crb,tpmdev=tpm-tpm0,id=tpm0 \
    ...

Yuri - given that this bz is a blocker for bug 1774158 assigned to you - can you help interpret the results here?

Hi Marc-Andre,


We know that some HLK tests still fail therefore, we decided to split the bug into two. 
This one will include the one that pass, and the ones that fail and need to be fixed will in BZ #1858821

Therefore, I think that we should move this bug to ON_QA.

I have finished HLK test on windows 2019 guest, the results are as following:

There are 4 test case failed:
DevFund Broker Ttest
Operate In Server Core Test
TPM - Auxiliary Test
TPM 2.0 EK Certificate Tests

Full test logs can be found here: http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/bug1744045/win2019-vtpm-vbl-1.hlkx

My environment:

RHEL 8.3
qemu-kvm-5.0.0-2.module+el8.3.0+7379+0505d6ca.x86_64
kernel-4.18.0-228.el8.x86_64
edk2-ovmf-20200602gitca407c7246bf-2.el8.noarch
swtpm-0.3.0-1.20200218git74ae43b.module+el8.3.0+7102+0b79cc3a.x86_64
libtpms-0.7.2-1.20200527git7325acb477.module+el8.3.0+7068+4e1b8df5.x86_64
Guest : windows server 2019 (Q35 + OVMF + vTPM)
HLK version:10.1.17763.1

I will post win10 guest test results here when the test run is done. 

Hi Marc-Andre, 

Can you help evaluate the result and see if the passed ones are correct and failed cases will be in BZ #1858821 ?

Thanks!

(In reply to Qinghua Cheng from comment #43)
> I have finished HLK test on windows 2019 guest, the results are as following:
> 
> There are 4 test case failed:
> DevFund Broker Ttest
> Operate In Server Core Test
> TPM - Auxiliary Test
> TPM 2.0 EK Certificate Tests

Are "DefFund Broker Ttest" and "Operate In Server Core Test" TPM related tests? There are tons of tests that have TPM in the name, so I am wondering.

For the other two tests I have this explanation:

TPM Auxiliary Test: VerifySpecVersion
Comment: TPM 2.0 revisions do not match the one expected by HLK; this is not a TPM 2.0 functionality issue. HLK seems to assume a certain revision of the TPM specs to be implemented. Microsoft and I do not coordinate the releases of libtpms and HLK so they could match some sort of expectations on revisions. So I don't think this is an actual issue.


TPM 2.0 EK Certificate Tests: VerifyEkCertisKnownAuthority
Comment: Software TPMs do not/cannot have a known CA like vendors of hardware TPMs use for TPM 2 certificates; this is not a TPM 2.0 functionality issue

https://github.com/stefanberger/libtpms/wiki/Testing-of-libtpms-Functionality

(In reply to Stefan Berger from comment #44)
> (In reply to Qinghua Cheng from comment #43)
> > I have finished HLK test on windows 2019 guest, the results are as following:
> > 
> > There are 4 test case failed:
> > DevFund Broker Ttest
> > Operate In Server Core Test
> > TPM - Auxiliary Test
> > TPM 2.0 EK Certificate Tests
> 
> Are "DefFund Broker Ttest" and "Operate In Server Core Test" TPM related
> tests? There are tons of tests that have TPM in the name, so I am wondering.

Hi Stefan,

Thanks for your review. 

These two test cases are automatically loaded with the TPM 2.0 test suite. There are totally 64 test cases were loaded and executed. Please refer to http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/bug1744045/win2019-vtpm-vbl-1.hlkx for full test lists and logs. 

If not all test cases are required, please let me know.

Thanks,
Qinghua Cheng

> 
> For the other two tests I have this explanation:
> 
> TPM Auxiliary Test: VerifySpecVersion
> Comment: TPM 2.0 revisions do not match the one expected by HLK; this is not
> a TPM 2.0 functionality issue. HLK seems to assume a certain revision of the
> TPM specs to be implemented. Microsoft and I do not coordinate the releases
> of libtpms and HLK so they could match some sort of expectations on
> revisions. So I don't think this is an actual issue.
> 
> 
> TPM 2.0 EK Certificate Tests: VerifyEkCertisKnownAuthority
> Comment: Software TPMs do not/cannot have a known CA like vendors of
> hardware TPMs use for TPM 2 certificates; this is not a TPM 2.0
> functionality issue
> 
> https://github.com/stefanberger/libtpms/wiki/Testing-of-libtpms-Functionality

Hi, 

Most of the mentioned TPM tests are NOT related to TPM.
The tests that are checking vTPM implementation are those between 
'Auxiliary..' and 'TPM 2.0 Quality'.  

Regarding 'DevFund Broker Test':
This test emulates accessing the vTPM device from a container application. The test configures
some security settings and then tries to open the device. Access is NOT blocked so 
this test fails. The same behavior is observed with Hyper-V vTPM 
implementation (the test also fails there). This is not related to vTPM implementation.

Regarding 'Operate in Server Core':
This tests checks if OS installation type is 'Server Core' (InstallationType in 
 SOFTWARE\Microsoft\Windows NT\CurrentVersion). Because installation used 
for testing was normal DataCenter installation, the InstallationType is set to 'Server'
and test fails. This is also not related to vTPM.

Thanks, 

Marek

(In reply to Marek Kedzierski from comment #46)
> Hi, 
> 
> Most of the mentioned TPM tests are NOT related to TPM.
> The tests that are checking vTPM implementation are those between 
> 'Auxiliary..' and 'TPM 2.0 Quality'.  

Thanks for your info.

Dose it mean we only select and run test cases between 'Auxiliary..' and 'TPM 2.0 Quality' for vTPM test is ok?

> 
> Regarding 'DevFund Broker Test':
> This test emulates accessing the vTPM device from a container application.
> The test configures
> some security settings and then tries to open the device. Access is NOT
> blocked so 
> this test fails. The same behavior is observed with Hyper-V vTPM 
> implementation (the test also fails there). This is not related to vTPM
> implementation.
> 
> Regarding 'Operate in Server Core':
> This tests checks if OS installation type is 'Server Core' (InstallationType
> in 
>  SOFTWARE\Microsoft\Windows NT\CurrentVersion). Because installation used 
> for testing was normal DataCenter installation, the InstallationType is set
> to 'Server'
> and test fails. This is also not related to vTPM.
> 
> Thanks, 
> 
> Marek

HLK test results on Win10 guest:

RHEL 8.3
kernel: 4.18.0-233.el8.x86_64
qemu-kvm: qemu-kvm-5.1.0-2.module+el8.3.0+7652+b30e6901.x86_64
edk2: edk2-ovmf-20200602gitca407c7246bf-3.el8.noarch
libtpms: libtpms-0.7.2-1.20200527git7325acb477.module+el8.3.0+7068+4e1b8df5.x86_64
swtpm: swtpm-0.3.0-1.20200218git74ae43b.module+el8.3.0+7648+42900458.x86_64
win10 guest: 2004 Enterprise edition
HLK: 10.0.19041.1

Run vtpm related test cases:

Totally 13 test cases, 2 failed: 
  
TPM Auxiliary Test
TPM 2.0 EK Certificate Tests

Full log: http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/bug1744045/win10vtpmtestcases.hlkx

Hi Marek, 

Could you help review test results with Win10 guest in comment 48 ?

If there are no other comments on the test results for Win2019 in Comment 38 and Win10 in Comment 48, I will move this bug to verified. 

Thanks,
Qinghua

(In reply to Qinghua Cheng from comment #49)
> Hi Marek, 
> 
> Could you help review test results with Win10 guest in comment 48 ?
> 
> If there are no other comments on the test results for Win2019 in Comment 38
> and Win10 in Comment 48, I will move this bug to verified. 
> 
> Thanks,
> Qinghua

Hi Qinghua, 

Regarding comment 48 I don't have any questions. Status of tests is known.

I've reviewed tests from Comment 38, I've found one strange crash of HLK testing application
during performance tests. I've not seen this before. Could you try to run this test again?

Thanks, 

Marek

(In reply to Marek Kedzierski from comment #50)
> (In reply to Qinghua Cheng from comment #49)
> > Hi Marek, 
> > 
> > Could you help review test results with Win10 guest in comment 48 ?
> > 
> > If there are no other comments on the test results for Win2019 in Comment 38
> > and Win10 in Comment 48, I will move this bug to verified. 
> > 
> > Thanks,
> > Qinghua
> 
> Hi Qinghua, 
> 
> Regarding comment 48 I don't have any questions. Status of tests is known.
> 
> I've reviewed tests from Comment 38, I've found one strange crash of HLK
> testing application
> during performance tests. I've not seen this before. Could you try to run
> this test again?
> 
> Thanks, 
> 
> Marek


Hi Marek,

Sorry I pointed to a wrong link for win2019 test results. The correct one should be Comment 43. Could you help review it again.

Thanks,
Qinghua

(In reply to Qinghua Cheng from comment #51)
> (In reply to Marek Kedzierski from comment #50)
> > (In reply to Qinghua Cheng from comment #49)
> > > Hi Marek, 
> > > 
> > > Could you help review test results with Win10 guest in comment 48 ?
> > > 
> > > If there are no other comments on the test results for Win2019 in Comment 38
> > > and Win10 in Comment 48, I will move this bug to verified. 
> > > 
> > > Thanks,
> > > Qinghua
> > 
> > Hi Qinghua, 
> > 
> > Regarding comment 48 I don't have any questions. Status of tests is known.
> > 
> > I've reviewed tests from Comment 38, I've found one strange crash of HLK
> > testing application
> > during performance tests. I've not seen this before. Could you try to run
> > this test again?
> > 
> > Thanks, 
> > 
> > Marek
> 
> 
> Hi Marek,
> 
> Sorry I pointed to a wrong link for win2019 test results. The correct one
> should be Comment 43. Could you help review it again.
> 
There is no problem.

> Thanks,
> Qinghua

Regarding Comment 43 - I've already provided information in Comment 46 and Stefan Berger provided 
information in Comment 44.

Thanks, 

Marek

(In reply to Marek Kedzierski from comment #52)
> (In reply to Qinghua Cheng from comment #51)
> > (In reply to Marek Kedzierski from comment #50)
> > > (In reply to Qinghua Cheng from comment #49)
> > > > Hi Marek, 
> > > > 
> > > > Could you help review test results with Win10 guest in comment 48 ?
> > > > 
> > > > If there are no other comments on the test results for Win2019 in Comment 38
> > > > and Win10 in Comment 48, I will move this bug to verified. 
> > > > 
> > > > Thanks,
> > > > Qinghua
> > > 
> > > Hi Qinghua, 
> > > 
> > > Regarding comment 48 I don't have any questions. Status of tests is known.
> > > 
> > > I've reviewed tests from Comment 38, I've found one strange crash of HLK
> > > testing application
> > > during performance tests. I've not seen this before. Could you try to run
> > > this test again?
> > > 
> > > Thanks, 
> > > 
> > > Marek
> > 
> > 
> > Hi Marek,
> > 
> > Sorry I pointed to a wrong link for win2019 test results. The correct one
> > should be Comment 43. Could you help review it again.
> > 
> There is no problem.
> 
> > Thanks,
> > Qinghua
> 
> Regarding Comment 43 - I've already provided information in Comment 46 and
> Stefan Berger provided 
> information in Comment 44.
> 
> Thanks, 
> 
> Marek

Thanks for Marek and Stefan's review and comments. 

Based on Comment 44, Comment 46 and Comment 50. I moved this bug to verified.