Bug 2334600 - swtpm: Internal ABI mismatch due to inconsistent _FILE_OFFSET_BITS definition
Summary: swtpm: Internal ABI mismatch due to inconsistent _FILE_OFFSET_BITS definition
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: swtpm
Version: rawhide
Hardware: i686
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Stefan Berger
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-12-27 19:33 UTC by Stefan Berger
Modified: 2024-12-27 22:47 UTC (History)
15 users (show)

Fixed In Version:
Clone Of:
Environment:
If this bug requires documentation, please select an appropriate Doc Type value.
Last Closed: 2024-12-27 22:47:44 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Stefan Berger 2024-12-27 19:33:40 UTC
I have been running into an issue when building my swtpm package on Fedora for rawhide on i686 but not on any other CPU type. The following error occurs during the builds ( https://koji.fedoraproject.org/koji/tasks?state=all&owner=stefanberger&view=tree&method=all&order=-id ):

tpmlib.h:76:7: error: type of 'tpmlib_handle_tcg_tpm2_cmd_header' does not match original declaration [-Werror=lto-type-mismatch]
   76 | off_t tpmlib_handle_tcg_tpm2_cmd_header(const unsigned char *command,
      |       ^
tpmlib.c:576:7: note: return value type mismatch
  576 | off_t tpmlib_handle_tcg_tpm2_cmd_header(const unsigned char *command,
      |       ^
tpmlib.c:576:7: note: 'tpmlib_handle_tcg_tpm2_cmd_header' was previously declared here
tpmlib.c:576:7: note: code may be misoptimized unless '-fno-strict-aliasing' is used


When inspecting the two files it shows that the prototype and function signature are identical:

https://github.com/stefanberger/swtpm/blob/stable-0.10/src/swtpm/tpmlib.c#L576

off_t tpmlib_handle_tcg_tpm2_cmd_header(const unsigned char *command,
                                        uint32_t command_length,
                                        TPM_MODIFIER_INDICATOR *locality)

https://github.com/stefanberger/swtpm/blob/stable-0.10/src/swtpm/tpmlib.h#L76

off_t tpmlib_handle_tcg_tpm2_cmd_header(const unsigned char *command,
                                        uint32_t command_length,
                                        TPM_MODIFIER_INDICATOR *locality);

I cannot recreate the issue locally trying to build this file with the same command line that fails on the build machine.


Reproducible: Always

Steps to Reproduce:
You may have to build this on Fedora. I could not reproduce the issue in a rawhide VM.

Comment 1 Florian Weimer 2024-12-27 20:33:32 UTC
Autotools try to define _FILE_OFFSET_BITS=64 for this package:

Applying this patch:

diff --git a/src/swtpm/tpmlib.h b/src/swtpm/tpmlib.h
index 90f9cb869f932846..4562a283f84b5df7 100644
--- a/src/swtpm/tpmlib.h
+++ b/src/swtpm/tpmlib.h
@@ -43,6 +43,10 @@
 
 #include <libtpms/tpm_library.h>
 
+#if _FILE_OFFSET_BITS != 64
+#error inconsistent _FILE_OFFSET_BITS
+#endif
+
 const char *tpmlib_get_blobname(uint32_t blobtype);
 enum TPMLIB_StateType tpmlib_blobtype_to_statetype(uint32_t blobtype);
 TPM_RESULT tpmlib_register_callbacks(struct libtpms_callbacks *cbs);

Gives me:

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../.. -I../../include -I../../include -I../../include/swtpm -I../../src/utils -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection -Wreturn-type -Wsign-compare -Wswitch-enum -Wmissing-prototypes -Wall -Werror -Wformat -Wformat-security -Wshadow -I/usr/include/p11-kit-1 -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection -Wreturn-type -Wsign-compare -Wswitch-enum -Wmissing-prototypes -Wall -Werror -Wformat -Wformat-security -Wshadow -I/usr/include/p11-kit-1 -fstack-protector-strong -Wstack-protector -D_FORTIFY_SOURCE=2 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/sysprof-6 -pthread -I/usr/include/json-glib-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -DWITH_GZFILEOP -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/sysprof-6 -pthread -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection -Wreturn-type -Wsign-compare -Wswitch-enum -Wmissing-prototypes -Wall -Werror -Wformat -Wformat-security -Wshadow -I/usr/include/p11-kit-1 -c mainloop.c  -fPIC -DPIC -o .libs/libswtpm_libtpms_la-mainloop.o
In file included from mainloop.c:60:
tpmlib.h:47:2: error: #error inconsistent _FILE_OFFSET_BITS
   47 | #error inconsistent _FILE_OFFSET_BITS
      |  ^~~~~

Therefore, the definition of off_t is inconsistent across compilation units, and the reported ABI problem is real.

If tpmlib.h is an installed header, it really should use uint64_t instead of off_t, so that the ABI does not vary based on compiler flags.

Comment 2 Stefan Berger 2024-12-27 21:25:17 UTC
Thanks. I thought something with the compiler is wrong since the failing build is for swtpm-0.10.0-4.fc42 while swtpm-0.10.0-3.fc42 still worked fine.

https://koji.fedoraproject.org/koji/buildinfo?buildID=2585235

Comment 3 Stefan Berger 2024-12-27 22:47:44 UTC
Rawhide packages patched and build succeeds: https://koji.fedoraproject.org/koji/tasks?owner=stefanberger&state=active&view=tree&method=all&order=-id


Note You need to log in before you can comment on or make changes to this bug.