The Xclients files used by switchdesk should include a test to exec the
appropriate session
manager under ssh-agent, if ssh-agent is present.  This would make the use
of ssh (with passphrases remembered for each session) easier.

(Note: I'm posting a similar enhancement request for xinitrc, as its
Xclients file should also be modified, for consistency's sake...)