Description of problem: ## What we need? Today, all the translators which are present in glusterfs, which needs to be built for glusterfs, has to be in glusterfs repository for it to be building easily, and working smoothly. One of the major reason is that, glusterd is in same repo, which manages the volfile generation, and hence it makes sense! By keeping volgen as a separate package (https://github.com/gluster/glusterd2/issues/1190) one can technically have no dependency on any project, but need to 'place' the translator in the proper path when they build it, and add their translator in the template. If we allow developers to have a project for developing translators independent of glusterfs codebase (ie, using glusterfs-devel package as a dependency), it will allow the project to proceed faster, adapted well. ## What is the problem? The challenge I am facing to 'host' a separate project for a new translator is that, the header files installed in the system are not having a proper dependency graph! * I can't just add '#include <glusterfs/xlator.h>' (or similar) and start writing a translator. * There are many macros (including GF_HOST_OS) which are defined through configure.ac, but are not added in a .h file (not in config.h too). The problem is, even though we build a specific '-devel' package with a fixed set of variables, that is not captured, and I have to add all those things again in configure.ac of new project. * Some libglusterfs header files depend on .h files generated from RPC directory, which gets installed in ${includedir}/glusterfs/rpc/${filename}.h, but are called as '#include "${filename}.h"` in glusterfs codebase. Problem with this is, I have to provide '-Ipath' option to build, which is not intuitive. * Some variables, like 'uuid_t' are not properly resolved even if I include compat-uuid or uuid.h from include path! With all these problems, I am yet not successful in hosting my own translator in a separate repo. ## What is required? I just want to write a new translator or say compile 'playground/template.so' as a translator in a separate repo, by making glusterfs-devel package as a dependency, and technically, I should be able to achieve it, without a lot of issues. Version-Release number of selected component (if applicable): master How reproducible: 100% Actual results: Not easy to build this xlators outside glusterfs repo! Expected results: Should be able to do it easily! Additional info: This helps us to move the translator code we remove while fixing bug https://bugzilla.redhat.com/show_bug.cgi?id=1635688 in a separate repo, so, interested users can continue to use it, and we can continue to keep glusterfs repo with only maintained codebase!
I bashed together an example autoconf and rpm .spec file for building the experimental/posix2 xlator outside the tree. You can check it out at https://github.com/gluster/glusterfs-posix2. The xlator sources are unchanged. There are some very small diffs between the in-tree Makefile.am files and the out-of-tree Makefile.am files, but on the whole they're minor. E.g. one of diffs is that for this quick-and-dirty example I didn't include linking with the .sym file. Instructions for compiling the xlator and building an RPM for it are in the INSTALL file. (You probably want to build an rpm to install, otherwise a non-rpm install will install to /usr/local by default. Or you can use options to configure to get it to install to /usr/lib64/glusterfs/4.1.5/.... instead of /usr/local/...) IMO it should be fairly straightforward to drop in a different xlator in the xlators tree and build. It doesn't deal with the issue of automatically adding the xlator to a volume graph. (That would obviously be a lot more work.)
> it should be fairly straightforward to drop in a different xlator in the xlators tree and build. But it should be expected that each xlator will require (minor) tweaks to the configure.ac, much like we have to do with glusterfs' configure.ac. There should also be a glusterfs.pc as part of the glusterfs-devel package. Using the glusterfs-api.pc and glusterfs-api-devel is a bit of a hack.
Kaleb, this is good! I am OK with stating the bare-minimum requirements to do this. Technically, we can even host a playground repo, which people can use to clone and develop on! That is fine. Currently I am still facing some issues to build glusterfs-posix2 repo. (mainly with uuid.h inclusion, some error (on Fedora27). Let me do few more round of tests, and we should be good to go. In general, this is encouraging :-) Thanks for this!
Works fine for me on F27. Did you install libuuid-devel? I guess configure(.ac) should check for that.
REVIEW: https://review.gluster.org/21385 (build/packaging/core: add glusterfs.pc(.in)) posted (#3) for review on master by Kaleb KEITHLEY
For me, with the changes to handle uuid.h properly, the build is working fine! Happy with the status so far on this one!
REVIEW: https://review.gluster.org/22944 (glusterfs-fops: fix the modularity) posted (#1) for review on master by Amar Tumballi
I was trying to get libglusterfs totally independent of rpc linking. There are some issues, which can be resolved easily, but one major issue is 'client_t', which is not an easy one. Need more ideas, and discussions on this one.
REVIEW: https://review.gluster.org/22944 (glusterfs-fops: fix the modularity) merged (#3) on master by Amar Tumballi
REVIEW: https://review.gluster.org/23015 (libglusterfs: remove dependency of rpc) posted (#1) for review on master by Amar Tumballi
REVIEW: https://review.gluster.org/23015 (libglusterfs: remove dependency of rpc) merged (#4) on master by Amar Tumballi
This bug is moved to https://github.com/gluster/glusterfs/issues/1078, and will be tracked there from now on. Visit GitHub issues URL for further details