Installing perl-Image-Info package pulls in various X11 packages like libX11 and libxcb. That all only because perl-Image-Info requires rgb for rgb.txt database that is provided by xorg-x11-server-utils file. If the /usr/share/X11/rgb.txt file were in a separate sub-package, perl-Image-Info that is no X11 client would not pull in X11 libraries. Reading xorg-x11-server-utils.spec reveals that xorg-x11-server-utils is a bundle of 16 different upstream projects. Moreover none of the three files (showrgb, rgb.txt, showrgb.1) produced by rgb sources require X11 libraries. I recommend to split the rgb into a sub-package or more preferably to package rgb as a standalone package. If you don't have for it I can do it for you.
There was no word against in last four years. I'm going to subpackage the rgb.