Bug 1008189 (varnish-agent) - Review Request: varnish-agent - Administration agent for Varnish Cache
Summary: Review Request: varnish-agent - Administration agent for Varnish Cache
Keywords:
Status: CLOSED ERRATA
Alias: varnish-agent
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Sandro Mani
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-15 17:58 UTC by Dridi Boukelmoune
Modified: 2013-12-19 07:17 UTC (History)
1 user (show)

Fixed In Version: varnish-agent-2.1-5.fc19
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-12-19 07:17:37 UTC
Type: ---
Embargoed:
manisandro: fedora-review+
gwync: fedora-cvs+


Attachments (Terms of Use)

Description Dridi Boukelmoune 2013-09-15 17:58:07 UTC
Spec URL: https://bitbucket.org/dridi/fedora_packages/downloads/varnish-agent.spec
SRPM URL: https://bitbucket.org/dridi/fedora_packages/downloads/varnish-agent-2.1-1.fc19.src.rpm
Description:
Varnish Agent software that runs on all caches managed by Varnish
Administration Console (VAC).

Fedora Account System Username: dridi

Comment 1 Sandro Mani 2013-09-20 12:22:34 UTC
Remarks:
- The actual tarball name for the SOURCE0 URL seems to be vagent2-%{version}.tar.gz (i.e. try running wget --content-disposition https://github.com/varnish/vagent2/archive/2.1.tar.gz)

- rm -rf %{buildroot} is unnecessary nowadays

- %{_sysconfdir}/varnish is unowned

- fedora-review tells [!]: Package should not use obsolete m4 macros
Looks like you would need to rename AM_CONFIG_HEADER to AM_CONFIG_HEADERS (note the final S) to get rid of this one.


- rpmlint: missing-call-to-setgroups /usr/bin/varnish-agent
Explanation for which is:
"This executable is calling setuid and setgid without setgroups or initgroups.
There is a high probability this mean it didn't relinquish all groups, and this
would be a potential security issue to be fixed. Seek POS36-C on the web for
details about the problem."
I'm not sure if you're supposed to take any action this.

Overall, looks good!

Comment 2 Dridi Boukelmoune 2013-09-20 12:58:46 UTC
(In reply to Sandro Mani from comment #1)
> Remarks:
> - The actual tarball name for the SOURCE0 URL seems to be
> vagent2-%{version}.tar.gz (i.e. try running wget --content-disposition
> https://github.com/varnish/vagent2/archive/2.1.tar.gz)

How can I solve that ?

> - rm -rf %{buildroot} is unnecessary nowadays

It should be removed from the default template for rpmdev-newspec then. To whom should I report that ?

> - %{_sysconfdir}/varnish is unowned

It is owned by the varnish package, which should be automatically required.

> - fedora-review tells [!]: Package should not use obsolete m4 macros
> Looks like you would need to rename AM_CONFIG_HEADER to AM_CONFIG_HEADERS
> (note the final S) to get rid of this one.

Should I send a patch upstream ?

> - rpmlint: missing-call-to-setgroups /usr/bin/varnish-agent
> Explanation for which is:
> "This executable is calling setuid and setgid without setgroups or
> initgroups.
> There is a high probability this mean it didn't relinquish all groups, and
> this
> would be a potential security issue to be fixed. Seek POS36-C on the web for
> details about the problem."
> I'm not sure if you're supposed to take any action this.

The process takes a user/group (defaults to nobody) and then calls daemon(3) to run unprivileged. But that's pretty much all I know from the man :/ (I haven't read the source code).

Comment 3 Sandro Mani 2013-09-20 14:42:04 UTC
>> Remarks:
>> - The actual tarball name for the SOURCE0 URL seems to be
>> vagent2-%{version}.tar.gz (i.e. try running wget --content-disposition
>> https://github.com/varnish/vagent2/archive/2.1.tar.gz)
> How can I solve that ?

Just use https://github.com/varnish/vagent2/archive/vagent2-%{version}.tar.gz as Source0.

>
>> - rm -rf %{buildroot} is unnecessary nowadays
> It should be removed from the default template for rpmdev-newspec then. To whom
> should I report that ?

The package is rpmdevtools.

>
>> - %{_sysconfdir}/varnish is unowned
> It is owned by the varnish package, which should be automatically required.
No, only varnish-libs is required. You will need to add an explicit Requries for varnish.
>
>> - fedora-review tells [!]: Package should not use obsolete m4 macros
>> Looks like you would need to rename AM_CONFIG_HEADER to AM_CONFIG_HEADERS
>> (note the final S) to get rid of this one.
> Should I send a patch upstream ?

Yeah I would.

>
>> - rpmlint: missing-call-to-setgroups /usr/bin/varnish-agent
>> Explanation for which is:
>> "This executable is calling setuid and setgid without setgroups or
>> initgroups.
>> There is a high probability this mean it didn't relinquish all groups, and
>> this
>> would be a potential security issue to be fixed. Seek POS36-C on the web for
>> details about the problem."
>> I'm not sure if you're supposed to take any action this.
> The process takes a user/group (defaults to nobody) and then calls daemon(3) to
> run unprivileged. But that's pretty much all I know from the man :/ (I haven't
> read the source code).

According to https://www.securecoding.cert.org/confluence/display/seccode/POS36-C.+Observe+correct+revocation+order+while+relinquishing+privileges , the program needs to call setgid(getgid()) before setuid(getuid()). In the source (src/main.c), this actually seems to be the case, but possibly the conditional block around setgid is confusing rpmlint? So I'd say it actually looks ok!

Comment 4 Dridi Boukelmoune 2013-09-20 14:53:03 UTC
(In reply to Sandro Mani from comment #3)
> >> Remarks:
> >> - The actual tarball name for the SOURCE0 URL seems to be
> >> vagent2-%{version}.tar.gz (i.e. try running wget --content-disposition
> >> https://github.com/varnish/vagent2/archive/2.1.tar.gz)
> > How can I solve that ?
> 
> Just use
> https://github.com/varnish/vagent2/archive/vagent2-%{version}.tar.gz as
> Source0.

404
https://github.com/varnish/vagent2/archive/vagent2-2.1.tar.gz as

Github uses the tag in the archive URL, that is to say "2.1".

> >> - %{_sysconfdir}/varnish is unowned
> > It is owned by the varnish package, which should be automatically required.
> No, only varnish-libs is required. You will need to add an explicit Requries
> for varnish.

Good catch! And the agent is actually pretty useless without varnish installed.

I'll send new spec and srpm with everything fixed.

Comment 5 Sandro Mani 2013-09-20 14:56:55 UTC
Uh you are right about the URL, sorry about that.

Comment 6 Dridi Boukelmoune 2013-10-28 22:30:06 UTC
Hi,

It's been a long time. I've been busy with other stuff, and I hit an issue with this package but didn't know how I would solve it.

I've packaged varnish-agent so that it would work out of the box once installed. The problem is that my latest submission doesn't, because it cannot authenticate on the varnishd server (it uses a shared secret file).

I was considering 3 solutions:
- patch varnish-agent
- bump version to 2.2 (not released yet, wouldn't fix it as of today)
- change permissions or ownership on the secret file in the varnish package

I really didn't want to do the third, and didn't want to patch varnish-agent without a test suite. Hopefully varnish-agent has a test suite, so I've patched it for the 2.1 version I'm packaging and sent a pull request [1] upstream for the 2.2 version (including the AC_CONFIG_HEADERS fix).

Spec URL: https://bitbucket.org/dridi/fedora_packages/downloads/varnish-agent.spec
SRPM URL: https://bitbucket.org/dridi/fedora_packages/downloads/varnish-agent-2.1-2.fc19.src.rpm

[1] https://github.com/varnish/vagent2/pull/94

Comment 7 Sandro Mani 2013-10-31 17:42:15 UTC
Hi, sorry for the somewhat late reply.

Problem: does not build, see
http://koji.fedoraproject.org/koji/taskinfo?taskID=6120510
./autogen.sh is returning a bad status because of
"automake: warnings are treated as errors". You need to either fix the warnings (recommended) or remove -Werror  from configure.ac

Two other details:
- You should add a comment to the patches, explaining briefly what they do and adding upstream bug links if they exist
- Why CFLAGS="%{optflags}" ? %configure takes care of that generating makefiles with the appropriate flags

Comment 8 Dridi Boukelmoune 2013-11-01 13:30:25 UTC
(In reply to Sandro Mani from comment #7)
> Problem: does not build, see
> http://koji.fedoraproject.org/koji/taskinfo?taskID=6120510
> ./autogen.sh is returning a bad status because of
> "automake: warnings are treated as errors". You need to either fix the
> warnings (recommended) or remove -Werror  from configure.ac

I have failing koji builds too (f19-f20), but it's the test suite that fails the builds:
http://koji.fedoraproject.org/koji/taskinfo?taskID=6122538
https://koji.fedoraproject.org/koji/taskinfo?taskID=6122308
http://koji.fedoraproject.org/koji/taskinfo?taskID=6122551

Is it possible to browse koji build's root ? I'd like to read the test logs.

I'll take a look at the automake issue, it only happens on rawhide so far.

Comment 9 Sandro Mani 2013-11-01 13:35:17 UTC
It is not possible to browse the koji roots, but you can
- build it locally with mock and browser it there
- cat the logs from the spec files, then the output appears in the build.log file

The reason automake only fails on rawhide is that it has the new automake-1.14, while the rest still have 1.13.

Comment 10 Dridi Boukelmoune 2013-11-02 00:12:30 UTC
The ARM build uncovered what looked like a race condition in the test suite. After some digging with the help of dgilmore it looks more serious than just the test suite. There's either a bug in varnish itself, or in this package.

It'll take a little longer than I expected...

I have also added the comment for the patch set, moved the optflags to the %configure command line, and fixed the automake issue.

Comment 11 Dridi Boukelmoune 2013-12-07 22:40:23 UTC
I finally have a new submission!

Spec URL: https://bitbucket.org/dridi/fedora_packages/downloads/varnish-agent.spec
SRPM URL: https://bitbucket.org/dridi/fedora_packages/downloads/varnish-agent-2.1-4.fc19.src.rpm

ChangeLog:
- Added a patch to make the test suites work offline
- Added a patch to disable the ban test suite
- Removed the default -S option from the service

http://koji.fedoraproject.org/koji/taskinfo?taskID=6268014

The race condition was just the tip of the iceberg. There were more issues in the test suite: unnecessary access to the Internet.

The race condition in itself doesn't seem to belong to this package. Varnish has a background thread that handles cache invalidation based on "bans". Some ban-related tests could fail between the moment a ban was created, and the moment it was checked.

The problem is it still happens even when you disable the background thread, so my guess is that it's a bug in varnish, not the agent. This issue has already been discussed in the pull request mentioned in comment 6 and upstream is working on it.

The relative slowness of the ARM builders emphasizes this issue. For this reason, I have disabled the "ban" test suite until we (probably upstream) find out what's wrong.

Comment 12 Sandro Mani 2013-12-07 23:04:40 UTC
Some remarks, full review below:

- The CFLAGS="%{optflags}" in %configure CFLAGS="%{optflags}" is not necessary, the %configure macro already includes the CFLAGS, see rpm -E %configure.

- [!]: Patches link to upstream bugs/comments/lists or are otherwise justified. => the patch descriptions could be improved

- Would updating to 2.2.0 RC2 remove some downstream patches?

- Non-applied patches:
varnish-agent.src: W: patch-not-applied Patch0: vagent.secret_privileges.patch
varnish-agent.src: W: patch-not-applied Patch1: vagent.configure.patch
varnish-agent.src: W: patch-not-applied Patch2: vagent.automake.patch
varnish-agent.src: W: patch-not-applied Patch3: vagent.test_suite.patch
varnish-agent.src: W: patch-not-applied Patch4: vagent.disable_ban_tests.patch

Overall, remove the CFLAGS="%{optflags}" and look at the non-applied patches issue, then I'm happy :)


Full review:

Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
[ ] = Manual review needed



===== MUST items =====

C/C++:
[x]: Package does not contain kernel modules.
[x]: Package contains no static executables.
[x]: Package does not contain any libtool archives (.la)
[x]: Rpath absent or only used for internal libs.

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses found:
     "BSD (3 clause)", "BSD (2 clause)", "Unknown or generated". 10 files have
     unknown license. Detailed output of licensecheck in
     /home/sandro/.Data/Desktop/1008189-varnish-agent/licensecheck.txt
[x]: %build honors applicable compiler flags or justifies otherwise.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[-]: Package contains desktop file if it is a GUI application.
[-]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[ ]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Requires correct, justified where necessary.
[x]: Spec file is legible and written in American English.
[x]: Package contains systemd file(s) if in need.
[x]: Useful -debuginfo package or justification otherwise.
[x]: Package is not known to require an ExcludeArch tag.
[-]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
     Note: Documentation size is 10240 bytes in 2 files.
[x]: Package complies to the Packaging Guidelines
[x]: Package successfully compiles and builds into binary rpms on at least one
     supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).
[x]: If (and only if) the source package includes the text of the license(s)
     in its own file, then that file, containing the text of the license(s)
     for the package is included in %doc.
[x]: Package requires other packages for directories it uses.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: All build dependencies are listed in BuildRequires, except for any that
     are listed in the exceptions section of Packaging Guidelines.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: %config files are marked noreplace or the reason is justified.
[x]: Each %files section contains %defattr if rpm < 4.4
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package use %makeinstall only when make install' ' DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: No %config files under /usr.
[x]: Package do not use a name that already exist
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as provided
     in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local

===== SHOULD items =====

Generic:
[-]: If the source package does not include license text(s) as a separate file
     from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[?]: Package functions as described.
[!]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[!]: Patches link to upstream bugs/comments/lists or are otherwise justified.
[-]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[?]: Package should compile and build into binary rpms on all supported
     architectures.
[x]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed files.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: Dist tag is present (not strictly required in GL).
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: Fully versioned dependency in subpackages if applicable.
[x]: Uses parallel make %{?_smp_mflags} macro.
[x]: SourceX tarball generation or download is documented.
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).
[x]: Large data in /usr/share should live in a noarch subpackage if package is
     arched.
[x]: Package should not use obsolete m4 macros
[x]: Spec file according to URL is the same as in SRPM.


Rpmlint
-------
Checking: varnish-agent-2.1-4.fc21.x86_64.rpm
          varnish-agent-2.1-4.fc21.src.rpm
varnish-agent.x86_64: E: missing-call-to-setgroups /usr/bin/varnish-agent
varnish-agent.x86_64: W: only-non-binary-in-usr-lib
varnish-agent.src:103: W: macro-in-%changelog %{version}
varnish-agent.src: W: patch-not-applied Patch0: vagent.secret_privileges.patch
varnish-agent.src: W: patch-not-applied Patch1: vagent.configure.patch
varnish-agent.src: W: patch-not-applied Patch2: vagent.automake.patch
varnish-agent.src: W: patch-not-applied Patch3: vagent.test_suite.patch
varnish-agent.src: W: patch-not-applied Patch4: vagent.disable_ban_tests.patch
2 packages and 0 specfiles checked; 1 errors, 7 warnings.




Rpmlint (installed packages)
----------------------------
# rpmlint varnish-agent
varnish-agent.x86_64: E: missing-call-to-setgroups /usr/bin/varnish-agent
varnish-agent.x86_64: W: only-non-binary-in-usr-lib
1 packages and 0 specfiles checked; 1 errors, 1 warnings.
# echo 'rpmlint-done:'



Requires
--------
varnish-agent (rpmlib, GLIBC filtered):
    /bin/sh
    config(varnish-agent)
    libc.so.6()(64bit)
    libmicrohttpd.so.10()(64bit)
    libnsl.so.1()(64bit)
    libpthread.so.0()(64bit)
    libvarnishapi.so.1()(64bit)
    libvarnishapi.so.1(LIBVARNISHAPI_1.0)(64bit)
    rtld(GNU_HASH)
    systemd
    varnish



Provides
--------
varnish-agent:
    config(varnish-agent)
    varnish-agent
    varnish-agent(x86-64)



Source checksums
----------------
https://github.com/varnish/vagent2/archive/2.1.tar.gz :
  CHECKSUM(SHA256) this package     : 50f8a3dddba82935308ce9dba04059d0cbbc2b1e10a1df8936a14d9df347abfd
  CHECKSUM(SHA256) upstream package : 50f8a3dddba82935308ce9dba04059d0cbbc2b1e10a1df8936a14d9df347abfd


Generated by fedora-review 0.5.0 (920221d) last change: 2013-08-30
Command line :/usr/bin/fedora-review -b 1008189
Buildroot used: fedora-rawhide-x86_64
Active plugins: Generic, Shell-api, C/C++
Disabled plugins: Java, Python, SugarActivity, Perl, R, PHP, Ruby
Disabled flags: EPEL5, EXARCH, DISTTAG

Comment 13 Dridi Boukelmoune 2013-12-08 09:02:28 UTC
(In reply to Sandro Mani from comment #12)
> Some remarks, full review below:
> 
> - The CFLAGS="%{optflags}" in %configure CFLAGS="%{optflags}" is not
> necessary, the %configure macro already includes the CFLAGS, see rpm -E
> %configure.

Done.

> - [!]: Patches link to upstream bugs/comments/lists or are otherwise
> justified. => the patch descriptions could be improved
> 
> - Would updating to 2.2.0 RC2 remove some downstream patches?

Some have been integrated indeed. I was considering the RC2 on rawhide after a stable version landed in Fedora.

> - Non-applied patches:
> varnish-agent.src: W: patch-not-applied Patch0:
> vagent.secret_privileges.patch
> varnish-agent.src: W: patch-not-applied Patch1: vagent.configure.patch
> varnish-agent.src: W: patch-not-applied Patch2: vagent.automake.patch
> varnish-agent.src: W: patch-not-applied Patch3: vagent.test_suite.patch
> varnish-agent.src: W: patch-not-applied Patch4:
> vagent.disable_ban_tests.patch

False positive, the %autosetup macro applies the patches. Please check the logs of the koji build from comment 11.

> Overall, remove the CFLAGS="%{optflags}" and look at the non-applied patches
> issue, then I'm happy :)

Spec URL: https://bitbucket.org/dridi/fedora_packages/downloads/varnish-agent.spec
SRPM URL: https://bitbucket.org/dridi/fedora_packages/downloads/varnish-agent-2.1-5.fc19.src.rpm

Comment 14 Sandro Mani 2013-12-08 17:11:55 UTC
All ok, approved!

Comment 15 Dridi Boukelmoune 2013-12-08 21:10:58 UTC
Remember that I still owe you one review for this one, so do not hesitate to ask me when you need a reviewer :)

Comment 16 Dridi Boukelmoune 2013-12-08 21:13:01 UTC
New Package SCM Request
=======================
Package Name: varnish-agent
Short Description: Administration agent for Varnish Cache
Owners: dridi
Branches: f19 f20
InitialCC:

Comment 17 Gwyn Ciesla 2013-12-09 13:32:15 UTC
Git done (by process-git-requests).

Comment 18 Fedora Update System 2013-12-09 15:13:58 UTC
varnish-agent-2.1-5.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/varnish-agent-2.1-5.fc20

Comment 19 Fedora Update System 2013-12-09 15:15:08 UTC
varnish-agent-2.1-5.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/varnish-agent-2.1-5.fc19

Comment 20 Fedora Update System 2013-12-09 20:24:26 UTC
varnish-agent-2.1-5.fc20 has been pushed to the Fedora 20 testing repository.

Comment 21 Fedora Update System 2013-12-19 07:17:37 UTC
varnish-agent-2.1-5.fc20 has been pushed to the Fedora 20 stable repository.

Comment 22 Fedora Update System 2013-12-19 07:17:53 UTC
varnish-agent-2.1-5.fc19 has been pushed to the Fedora 19 stable repository.


Note You need to log in before you can comment on or make changes to this bug.