Switchdesk seems to have a couple of problems Telsa found Firstly if you select "this desktop only" it generates a .Xclients-localhost.localdomain:0.0 file. If you dump the environment from startx of a local display DISPLAY is ":0" not ":0.0" so it fails to find it. Secondly if you create a local one it seems to also delete the Xclients.default file which means all your other desktops (and due the bug above) all your desktops stop working
it's fixed in 4.0.0, which is available in rawhide.