Dave, is this just a fact of the install or can this be controlled. 
The extra button is just ridiculous for people who don't have an
Exchange server.  Of course I'm sure there's no way for the plugin to
get installed automatically if someone selects to use an exchange server.

Unfortunately as the code stands the button will always be there if
you have the evolution-connector package installed.  I've looked at
making it only appear if you have an Exchange account added, but the
code changes are non-trivial and upstream don't want them (they say it
will be fixed in Evolution 2.2).

I agree that this looks ridiculous, so I'll try to do a
minimally-invasive patch to fix this...

Ok, lets remove the connector by default since it's technically
infeasable to hide the extra button, but we'll keep it in core.  

Re-assigning to the distribution component and the amazing notting!

Fixed, will be in future trees.