I tried a rawhide install with btrfs and inst.sdboot option.
But reboot failed because of a systemd switch-root error.
I've found a error in the loader entry : some btrfs flags are missing.
Adding rootflags=subvol=root solves the issue.
This was noticed by Steve Capper as well, and I have a fix i'm about to PR.
There is a PR here: https://github.com/rhinstaller/anaconda/pull/5172
If accepted probably should eventually be backported to f39 (where it works as well).
*** Bug 2240102 has been marked as a duplicate of this bug. ***
Proposing as a Final FE, as anaconda only merges FE/blocker fixes, and this fixes an easily-encountered bug in an F39 Change that cannot be fixed with an update (as it's in the installer).
Note: the PR for this is now https://github.com/rhinstaller/anaconda/pull/5194 , it was split out of the other one.
Discussed during the 2023-09-25 blocker review meeting: 
The decision to classify this bug as an "AcceptedFreezeException (Final)" was made as this is an issue in an F39 Change that cannot be fixed with a post-release update, and the fix is pretty isolated to the systemd-boot code in the installer so can't break anything else.
FEDORA-2023-e8775823be has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-e8775823be
FEDORA-2023-e8775823be has been pushed to the Fedora 39 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-e8775823be`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-e8775823be
See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
When trying to install F39 with inst.sdboot, this update of Anaconda ends up with an error, see https://bugzilla.redhat.com/show_bug.cgi?id=2244794
FEDORA-2023-e8775823be has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.