Bug 978518

Summary: Server subpackage for ZooKeeper
Product: [Fedora] Fedora Reporter: Jeffrey C. Ollie <jeff>
Component: zookeeperAssignee: Jeffrey C. Ollie <jeff>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: java-sig-commits, jeff, kawai, matt, pmackinn, puntogil, rrati, tstclair
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-09-14 07:45:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Patch to zookeeper package implementing server subpackage none

Description Jeffrey C. Ollie 2013-06-26 18:06:37 UTC
Created attachment 765724 [details]
Patch to zookeeper package implementing server subpackage

Here's a patch that adds a server subpackage to the ZooKeeper package
in Fedora.  It's working well for me so far, with a small Hadoop/HBase setup and a couple of custom apps that make use of ZooKeeper.  I also did a couple of scratch builds:

F19: http://koji.fedoraproject.org/koji/taskinfo?taskID=5540309
F20: http://koji.fedoraproject.org/koji/taskinfo?taskID=5540247

Comment 1 gil cattaneo 2013-06-26 18:29:29 UTC
if you want become co-maintainer I'll be happy to give you permission to make you make the changes you requested.
but there is a problem:
Zookepeer should be configured for run under/with Hadhoop (not available at the moment, see https://fedoraproject.org/wiki/Features/Hadoop for the progress)
and as in src/package/rpm/spec/zookeper.spec

the group is hadhoop and not zookeper in %%pre server
regards

Comment 2 Jeffrey C. Ollie 2013-06-26 18:51:05 UTC
I don't see what running as user hadoop brings vs running as zookeeper. I'm only using Hadoop 1.1.2 and haven't even looked at 2.x...  Is there some new requirement in 2.x that ZooKeeper run as the same user as Hadoop?  I'm running a small Hadoop cluster with ZooKeeper running as a different user and it seems to be working.  I also have some custom code (totally unrelated to Hadoop) that uses ZooKeeper directly.

Comment 3 Hiroaki KAWAI 2013-07-17 02:42:26 UTC
+1 for this patch. I want to zookeeper only, too. Hadoop 2.x is just a service that may use zookeeper. In the current package, startup script is not used at all. So zookeeper server would not start (and any service including Hadoop would not be able to use zookeeper).

By the way, is there any reason not to use src/packages/rpm/spec/zookeeper.spec in the zookeeper-3.4.5 package? "hadoop" in that file is just a user name.

Comment 4 Jeffrey C. Ollie 2013-07-17 23:56:42 UTC
(In reply to Hiroaki KAWAI from comment #3)
> +1 for this patch. I want to zookeeper only, too. Hadoop 2.x is just a
> service that may use zookeeper. In the current package, startup script is
> not used at all. So zookeeper server would not start (and any service
> including Hadoop would not be able to use zookeeper).

I have commit rights now, so I'll be adding the server subpackage soon.

> By the way, is there any reason not to use
> src/packages/rpm/spec/zookeeper.spec in the zookeeper-3.4.5 package?
> "hadoop" in that file is just a user name.

I never looked at that spec file, but Fedora has a specific set of packaging standards.  One standard that many upstream Java spec files fail on is the requirement to compile everything from source, and not to rely on pre-packaged .jar files that may have been included with the upstream .tar.gz.

Comment 5 Timothy St. Clair 2013-07-18 16:05:42 UTC
+1 to the patch, and zookeeper user. 

&& Jeff is correct about the .spec comment. 

I've pushed f19, and am awaiting update on the acl's for devel
http://koji.fedoraproject.org/koji/taskinfo?taskID=5628074

Once devel acls are done, I can merge through.

Comment 6 Timothy St. Clair 2013-07-18 16:15:39 UTC
Will also compare against https://github.com/skottler/zookeeper-rpms

Comment 7 Timothy St. Clair 2013-07-18 18:51:53 UTC
One change that I think needs to be made is around ownership of config settings:

--------------------------------------
%files server
%attr(0750,zookeeper,zookeeper) %dir %{_sysconfdir}/zookeeper
%attr(0640,zookeeper,zookeeper) %ghost %config(noreplace) %{_sysconfdir}/zookeeper/zoo.cfg
%attr(0640,zookeeper,zookeeper) %{_sysconfdir}/zookeeper/zoo_sample.cfg
%attr(0640,zookeeper,zookeeper) %config(noreplace) %{_sysconfdir}/zookeeper/log4j.properties

%attr(0750,zookeeper,zookeeper) %dir %{_localstatedir}/log/zookeeper
%attr(0750,zookeeper,zookeeper) %dir %{_sharedstatedir}/zookeeper
%attr(0750,zookeeper,zookeeper) %dir %{_sharedstatedir}/zookeeper/data
%attr(0640,zookeeper,zookeeper) %ghost %{_sharedstatedir}/zookeeper/data/myid
%attr(0750,zookeeper,zookeeper) %dir %{_sharedstatedir}/zookeeper/log
%{_unitdir}/zookeeper.service
--------------------------------------

Typically %{_sysconfig} is %defattr(-,root,root,-) for obvious reasons.

Comment 8 Jeffrey C. Ollie 2013-07-22 14:56:42 UTC
The problem with having the configuration files owned by root is that they need to be world readable so that the zookeeper user can read them. I don't think that there's anything sensitive in the files right now but I'd rather start out with a safe default.

Comment 9 Timothy St. Clair 2013-07-22 15:09:23 UTC
world readable is standard operating procedure for daemons which don't contain system security information.

Comment 10 gil cattaneo 2013-07-22 15:19:30 UTC
(In reply to Jeffrey C. Ollie from comment #8)
> The problem with having the configuration files owned by root is that they
> need to be world readable so that the zookeeper user can read them. I don't
> think that there's anything sensitive in the files right now but I'd rather
> start out with a safe default.

[!]: Each %files section contains %defattr if rpm < 4.4
     Note: %defattr present but not needed

     ---> please remove

see https://bugzilla.redhat.com/show_bug.cgi?id=823122#c19
with the changes have introduced a problem.
if the problem persist i'm going to remove them personally
regards

Comment 12 Timothy St. Clair 2013-07-22 16:13:16 UTC
That was me and my legacy EL mindset, I removed in F19 and I saw you updated rawhide.  I walked through and updated the permissions, it should be good.  Someone want to double check and verify?

Comment 13 gil cattaneo 2013-07-22 16:18:36 UTC
found another problem in zookeeper spec file:

$ rpmlint zookeeper.spec
zookeeper.spec: E: specfile-error warning: bogus date in %changelog: Tue Jun 15 2013 Timothy St. Clair <tstclair> - 3.4.5-8
zookeeper.spec: E: specfile-error warning: bogus date in %changelog: Tue Jun 15 2013 Jeffrey C. Ollie <jeff> - 3.4.5-7
should be

* Sat Jun 15 2013 Timothy St. Clair <tstclair> - 3.4.5-8
- cleanup file ownership properties.

* Sat Jun 15 2013 Jeffrey C. Ollie <jeff> - 3.4.5-7
- add server subpackage

regards

Comment 14 Timothy St. Clair 2013-07-22 16:23:50 UTC
fixed, and pushed.

Comment 15 Timothy St. Clair 2013-07-23 21:02:23 UTC
Any objections for a bohdi update?  I was thinking about running through it tomorrow...

Comment 16 Timothy St. Clair 2013-07-24 16:24:21 UTC
Given the ticket was targeted to rawhide, I will hold off any f19 update unless specifically requested.