Bug 2521313

Created attachment 2155280 [details]
Dockerfile, Dockerfile.f43, bisect_pydantic.py

Description of problem:

matrix-synapse fails to start on Fedora 44, crashing immediately at import
time with a pydantic v1-compat ConfigError, before any configuration file
is even read. This affects the systemd service as well as any invocation
of `python3 -m synapse.app.homeserver` (including --version, generate-config,
etc.), since the crash happens while importing synapse.config.homeserver.

Root cause: synapse/config/mas.py in matrix-synapse 1.137.0 (the version
currently shipped by Fedora 44) still imports from synapse._pydantic_compat
and uses the pydantic v1 compatibility shim (`@validator`, etc.). That shim
is broken under Python 3.14 due to an upstream pydantic bug:
pydantic/pydantic#12363 (https://github.com/pydantic/pydantic/issues/12363),
which was fixed in pydantic 2.13.0b2 via pydantic/pydantic#12367.

However, Synapse itself already fixed this independently: upstream release
1.142.0rc1 (2025-11-04) shipped both "Add support for Python 3.14"
(element-hq/synapse#19055, #19134) and "Update pydantic to v2"
(element-hq/synapse#19071) in the same changeset. That migration rewrote
synapse/config/mas.py to import directly from `pydantic` (v2 native,
`@model_validator`) instead of the v1 shim, which removes the crash
entirely, independent of which pydantic version is installed.

Confirmed via a control test: Fedora 43's "updates" repo already carries
matrix-synapse-1.147.1-1.fc43 (well past the 1.142.0 migration). Running
it under the same Python 3.14 + python3-pydantic-2.12.5 combination as
Fedora 44 does NOT crash -- it proceeds normally past the import and only
fails later due to a missing config file (expected, since none was
provided). This isolates the cause to the matrix-synapse package version,
not the Python or pydantic version.

I also confirmed via `dnf repoquery matrix-synapse` against both the
`fedora` and `updates` repos for Fedora 44 that 1.137.0-2.fc44 is
currently the *only* build available -- Fedora 44 has not picked up any
of the 10 upstream releases between 1.137.0 and 1.147.1 that Fedora 43's
updates repo already has. So this is not a local configuration issue;
Fedora 44's matrix-synapse package is simply behind.


Version-Release number of selected component (if applicable):

matrix-synapse-1.137.0-2.fc44
python3-pydantic-2.12.5-3.fc44
python3-3.14.7 (python3 --version: Python 3.14.7)


How reproducible:

Always. 100% reproducible on a clean Fedora 44 install/container with no
special configuration.


Steps to Reproduce:
1. On a clean Fedora 44 system/container: dnf -y install matrix-synapse
2. python3 -m synapse.app.homeserver --config-path=/etc/synapse/homeserver.yaml
3. Observe the traceback below (no homeserver.yaml is even required to
   trigger this, since the crash occurs at import time before config
   parsing begins).

Control test showing this is version-specific to matrix-synapse, not to
Python/pydantic:
4. On a clean Fedora 43 container (same Python 3.14.7, same
   python3-pydantic-2.12.5, but matrix-synapse-1.147.1-1.fc43 from F43's
   updates repo): dnf -y install matrix-synapse
5. python3 -m synapse.app.homeserver --config-path=/etc/synapse/homeserver.yaml
6. This does NOT crash with the ConfigError -- it proceeds past all
   imports and fails only with FileNotFoundError for the (intentionally
   absent) config file.

A minimal Dockerfile reproducer is attached (Dockerfile), along with a
pydantic-version bisection script (bisect_pydantic.py) that independently
confirms the pydantic-side bug and its fix boundary.


Actual results:

Traceback (most recent call last):
  File "<frozen runpy>", line 203, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/lib64/python3.14/site-packages/synapse/app/homeserver.py", line 34, in <module>
    from synapse.api.urls import (
    ...<7 lines>...
    )
  File "/usr/lib64/python3.14/site-packages/synapse/api/urls.py", line 30, in <module>
    from synapse.config.homeserver import HomeServerConfig
  File "/usr/lib64/python3.14/site-packages/synapse/config/homeserver.py", line 39, in <module>
    from .mas import MasConfig
  File "/usr/lib64/python3.14/site-packages/synapse/config/mas.py", line 34, in <module>
    class MasConfigModel(ParseModel):
    ...<21 lines>...
            return v
  File "/usr/lib/python3.14/site-packages/pydantic/v1/main.py", line 221, in __new__
    inferred = ModelField.infer(
        name=var_name,
    ...<3 lines>...
        config=config,
    )
  File "/usr/lib/python3.14/site-packages/pydantic/v1/fields.py", line 504, in infer
    return cls(
        name=name,
    ...<7 lines>...
        field_info=field_info,
    )
  File "/usr/lib/python3.14/site-packages/pydantic/v1/fields.py", line 434, in __init__
    self.prepare()
    ~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/site-packages/pydantic/v1/fields.py", line 544, in prepare
    self._set_default_and_type()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/site-packages/pydantic/v1/fields.py", line 576, in _set_default_and_type
    raise errors_.ConfigError(f'unable to infer type for attribute "{self.name}"')
pydantic.v1.errors.ConfigError: unable to infer type for attribute "secret"

systemd reports:
synapse.service: Main process exited, code=exited, status=1/FAILURE
synapse.service: Failed with result 'exit-code'.
Failed to start synapse.service - Synapse Matrix homeserver.


Expected results:

synapse starts normally, as it did on Fedora 43 with the same
homeserver.yaml and no other configuration changes -- and as it does with
matrix-synapse-1.147.1-1.fc43 from Fedora 43's own updates repo under an
identical Python 3.14 / pydantic 2.12.5 environment.


Additional info:

Recommended fix: update the matrix-synapse package in Fedora 44 to
>=1.142.0 (ideally matching Fedora 43's already-available 1.147.1-1.fc43
build), which no longer touches the pydantic v1 compatibility shim in
synapse/config/mas.py.

Separately, bumping python3-pydantic to >=2.13.0 addresses the underlying
pydantic v1-shim bug directly and would also fix this for the current
matrix-synapse-1.137.0 build, but it is not sufficient by itself as a
long-term fix, since Synapse versions prior to 1.142.0 depend on the
(now-unsupported-on-Python-3.14) pydantic.v1 shim throughout the
codebase, not just in mas.py. Updating matrix-synapse is the more direct
and complete fix.

Supporting pydantic bisection (independent of the matrix-synapse-version
finding above, but confirms the underlying pydantic bug and its fix
boundary): I bisected pydantic releases directly against PyPI under a
plain Python 3.14 venv (script attached: bisect_pydantic.py):

  pydantic 2.12.4    -> crashes
  pydantic 2.13.0b1  -> crashes
  pydantic 2.13.0b2  -> works

This matches pydantic/pydantic#12367, which fixed pydantic/pydantic#12363.

There is no config-level workaround, since the failure occurs before
configuration is read.

I also checked whether affected users have any package-level workaround
available today, and found none:

- Cross-installing Fedora 43's matrix-synapse-1.147.1-1.fc43 RPM directly
  onto a Fedora 44 system fails dependency resolution:
    nothing provides python3.14dist(pyrsistent) >= 0.18 needed by
    matrix-synapse-1.147.1-1.fc43.x86_64
  (Fedora 44's python3-pyrsistent is older than what 1.147.1 requires.)

- There is no python3-pydantic build >=2.13.0 available anywhere for
  Python 3.14 to swap in as a smaller, single-package fix: Fedora 44's
  `updates-testing` repo still only has 2.12.5-3.fc44, and while Rawhide
  has python3-pydantic-2.13.4-5.fc45, it is already built against
  python(abi) = 3.15 (Rawhide has moved past Python 3.14), so it cannot
  be installed on Fedora 44 either:
    nothing provides python(abi) = 3.15 needed by
    python3-pydantic-2.13.4-5.fc45.noarch

So until Fedora ships a fix, affected users have no supported way to get
a working matrix-synapse via dnf/rpm alone -- the only workarounds are
running Synapse from a pip-installed venv or a container outside the
system package manager, bypassing Fedora packaging entirely. This makes
this bug effectively a full outage for any Fedora 44 matrix-synapse user
who updates, with no in-place package-manager fix available.

Originally noticed after updating a working Fedora 43 install to Fedora 44
today; matrix-synapse had been running fine prior to the upgrade.

Attachments:
  - Dockerfile: minimal reproducer against the real Fedora 44 RPM packages
  - bisect_pydantic.py: standalone script that bisects PyPI pydantic
    releases under Python 3.14 to find the crashing/working boundary