Description of problem: After installing gluster on Ubuntu 12.04 from the 3.5 PPA, geo-replication fails to start. root@sm-dev-moo:~ # gluster volume geo-replication test-moo 192.168.33.13::test-bow create push-pem force The hook-script (/var/lib/glusterd/hooks/1/gsync-create/post/S56glusterd-geo-rep-create-post.sh) required for push-pem is not present. Please install the hook-script and retry geo-replication command failed root@sm-dev-moo:~ # Investigation of the source build indicates that several files are not being copied from the build. The below is generated by adding "--failed-missing" to the dh_install command in the rules file. Some missing/unpackaged files are legit and replaced with more Ubuntu specific files, but minimally we are missing usr/share/glusterfs/scripts which has some geo-rep helpers. Also, following the specfile we seem to be missing all hooks, specifically the hook for setting up geo-replication, but evidently all of them - https://github.com/gluster/glusterfs/blob/master/glusterfs.spec.in#L916 === missing file list dh_install: etc/init.d/glusterd exists in debian/tmp but is not installed to anywhere dh_install: etc/glusterfs/glusterfs-georep-logrotate exists in debian/tmp but is not installed to anywhere dh_install: etc/glusterfs/glusterfs-logrotate exists in debian/tmp but is not installed to anywhere dh_install: etc/glusterfs/gluster-rsyslog-7.2.conf exists in debian/tmp but is not installed to anywhere dh_install: etc/glusterfs/gluster-rsyslog-5.8.conf exists in debian/tmp but is not installed to anywhere dh_install: etc/glusterfs/logger.conf.example exists in debian/tmp but is not installed to anywhere dh_install: usr/include/glusterfs/gfchangelog/changelog.h exists in debian/tmp but is not installed to anywhere dh_install: usr/include/glusterfs/api/glfs-handles.h exists in debian/tmp but is not installed to anywhere dh_install: usr/include/glusterfs/api/glfs.h exists in debian/tmp but is not installed to anywhere dh_install: usr/share/man/man8/glusterd.8 exists in debian/tmp but is not installed to anywhere dh_install: usr/share/man/man8/gluster.8 exists in debian/tmp but is not installed to anywhere dh_install: usr/share/man/man8/mount.glusterfs.8 exists in debian/tmp but is not installed to anywhere dh_install: usr/share/man/man8/glusterfsd.8 exists in debian/tmp but is not installed to anywhere dh_install: usr/share/man/man8/glusterfs.8 exists in debian/tmp but is not installed to anywhere dh_install: usr/share/doc/glusterfs/glusterfs.vim exists in debian/tmp but is not installed to anywhere dh_install: usr/share/doc/glusterfs/benchmarking/rdd.c exists in debian/tmp but is not installed to anywhere dh_install: usr/share/doc/glusterfs/benchmarking/glfs-bm.c exists in debian/tmp but is not installed to anywhere dh_install: usr/share/doc/glusterfs/benchmarking/local-script.sh exists in debian/tmp but is not installed to anywhere dh_install: usr/share/doc/glusterfs/benchmarking/launch-script.sh exists in debian/tmp but is not installed to anywhere dh_install: usr/share/doc/glusterfs/benchmarking/README exists in debian/tmp but is not installed to anywhere dh_install: usr/share/glusterfs/scripts/slave-upgrade.sh exists in debian/tmp but is not installed to anywhere dh_install: usr/share/glusterfs/scripts/gsync-upgrade.sh exists in debian/tmp but is not installed to anywhere dh_install: usr/share/glusterfs/scripts/gsync-sync-gfid exists in debian/tmp but is not installed to anywhere dh_install: usr/share/glusterfs/scripts/get-gfid.sh exists in debian/tmp but is not installed to anywhere dh_install: usr/share/glusterfs/scripts/generate-gfid-file.sh exists in debian/tmp but is not installed to anywhere dh_install: usr/share/glusterfs/scripts/post-upgrade-script-for-quota.sh exists in debian/tmp but is not installed to anywhere dh_install: usr/share/glusterfs/scripts/pre-upgrade-script-for-quota.sh exists in debian/tmp but is not installed to anywhere dh_install: usr/bin/fusermount-glusterfs exists in debian/tmp but is not installed to anywhere === Version-Release number of selected component (if applicable): sm@sm-dev-floor:~ $ dpkg -l | grep gluster ii glusterfs-client 3.5.2-ubuntu1~precise1 clustered file-system (client package) ii glusterfs-common 3.5.2-ubuntu1~precise1 GlusterFS common libraries and translator modules ii glusterfs-examples 3.2.5-1ubuntu1 example files for the glusterfs server and client ii glusterfs-server 3.5.2-ubuntu1~precise1 clustered file-system (server package) sm@sm-dev-floor:~ $ How reproducible: Always Steps to Reproduce: 1. install gluster 3.5 2. setup some volumes 2. try to setup geo replication via https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_distributed_geo_rep.md Actual results: See problem description for error Expected results: Geo-replication setup competes Additional info:
Created attachment 929422 [details] first pass patch adding a quick patch to add required files for geo-rep and some libs, def. not correct.
Hi Nathan, Thanks for reporting the issue. It came up in IRC once before but I wasnt able to figure out a solution. I'll try your patch & see if I can get it working in the packages. Your help is much appreciated. Regards, -louis
Created attachment 932454 [details] working debian dir here's my latest debian dir with updates install and rules files.
Why install everything in /usr/include/glusterfs? Did you need some header file for something? I normally only include glfs.h since that's the libgfapi other apps use. Just curious if there's something else useful in there. Aside from that little curiosity it all looks great to me. Thanks again for your contribution! I might not have figured this out on my own. You really helped a lot of people with this. All the best, -louis
I've been testing out geo-rep with 3.5.2 on ubuntu trusty & debian jessie and I'm running into an issue with the location of gsyncd. It gets installed into /usr/lib/x86_64-linux-gnu/glusterfs but the generated config says it's location is /nonexistent/gsyncd (which obviously doesn't exist). Geo-rep fails to start because of this. I'm stumped at this point. I'll review the issue on IRC this week and possibly open a new bug if necessary.
Re: #4 - I figured that was an oversight, and assumed all the header files needed to get shipped, mainly because there was some changelog related stuff in there. Re: #5 - I think the /nonexistent/gsyncd is fixed in 3.6 -- there's a few threads about it. If you look at the georep logs on both sides, it's trying to call /usr/libexec/glusterfs/gsyncd which doesn't exist in Ubuntu 12.04. I just made a /usr/libexec and symlinked glsuterfs to /usr/lib/x86_64-linux-gnu/glusterfs/ and it was able to start after that. I'll try to clean up my internal ansible gluster playbook and geo-replciation module next week which helps speed the setup and testing a bit. -n
I was wondering if you ran into this /nonexistent thing too. Thanks for confirming. I'll bring it up with the devs when I see them in IRC next (probably tuesday). I'm sure they'll have good insight. As for the headers, I just left them out by default until people asked (rightly so) for the libgfapi file, glfs.h, which is needed to build anything that uses libgfapi (samba vfs, qemu, etc). We discussed in IRC recenly the idea of putting headers in a glusterfs-dev package, but I don't think it's worth the effort. People are happy enough with just glfs.h in glusterfs-common as far as I know. In your opinion, should I publish new packages with the post hook script right away, even though geo-rep still doesnt work because of the gsyncd path problem? Or wait (days probably) until that's fixed too, and it all works out of the box? -louis
Yeah, I would, because at least people have a chance of making geo-rep work with the packages.
These packages are still completely broken? Rad...
This bug is getting closed because the 3.5 is marked End-Of-Life. There will be no further updates to this version. Please open a new bug against a version that still receives bugfixes if you are still facing this issue in a more current release.