xvkbd is built without $RPM_OPT_FLAGS, see gcc invocations in the build log: http://kojipkgs.fedoraproject.org/packages/xvkbd/3.1/3.fc13/data/logs/x86_64/build.log See bug 496968 for more info. (Cc'ing reviewer.)
xvkbd-3.1-4.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/xvkbd-3.1-4.fc12
Seems fixed in 3.1-4.