Bug 977646

Summary: Review Request: mylvmbackup - Utility for creating MySQL backups via LVM snapshots
Product: [Fedora] Fedora Reporter: Christopher Meng <i>
Component: Package ReviewAssignee: David Dick <ddick>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: ddick, lenz, Marcin.Dulak, pahan
Target Milestone: ---Flags: ddick: fedora-review+
gwync: fedora-cvs+
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: mylvmbackup-0.15-2.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-07-26 00:04:26 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Christopher Meng 2013-06-25 04:54:31 UTC
Spec URL: http://cicku.me/mylvmbackup.spec
SRPM URL: http://cicku.me/mylvmbackup-0.14-1.fc20.src.rpm
Description: mylvmbackup is a tool for quickly creating full physical backups of a MySQL 
server's data files. To perform a backup, mylvmbackup obtains a read lock on 
all tables and flushes all server caches to disk, makes an LVM snapshot of 
the volume containing the MySQL data directory, and unlocks the tables again. 
The snapshot process takes only a small amount of time. When it is done, the 
server can continue normal operations, while the actual file backup proceeds.
Fedora Account System Username: cicku

Comment 1 marcindulak 2013-06-26 10:53:09 UTC
Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
[ ] = Manual review needed


Issues:
=======
- No %config files under /usr.
  Note: %config(noreplace) /usr/share/mylvmbackup/*.pm
  See: http://fedoraproject.org/wiki/Packaging/Guidelines#Configuration_files

  ----> I see this is how mylvmbackup is packaged upstream
        https://build.opensuse.org/package/show?package=mylvmbackup&project=home%3ALenzGr , but we can't use %config under /usr in Fedora.
        A solution could be to use, e.g.:
        hooksdir=/etc/mylvmbackup/hooks in /etc/mylvmbackup.conf
        and create that dir in spec.
        I guess one should communicate this choice upstream.

        Another comment: the upstream build.opensuse.org and the current spec
        share some similarities - if you based on upstream - include this
        information in changelog.

===== MUST items =====

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[x]: Package contains desktop file if it is a GUI application.
[x]: Development files must be in a -devel package
[ ]: Package requires other packages for directories it uses.

  ----> /etc/mylvmbackup.conf refers to /etc/my.cnf, and this is provided by:
        el5: Requires: mysql
        el6, f17-f18: Requires: mysql-libs
        f19-: Requires: mariadb-libs

[x]: Package uses nothing in %doc for runtime.
[x]: Package is not known to require ExcludeArch.
[ ]: Package complies to the Packaging Guidelines
[x]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses found:
     "Unknown or generated". 3 files have unknown license. Detailed output of
     licensecheck in /home/mock/977646-mylvmbackup/licensecheck.txt

  ----> false positive due to /usr/share/mylvmbackup/*.pm files

[x]: Package consistently uses macro is (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[x]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[ ]: Requires correct, justified where necessary.

  ----> see "Package requires other packages for directories it uses." above

[x]: Spec file is legible and written in American English.
[x]: Package contains systemd file(s) if in need.
[x]: Large documentation must go in a -doc subpackage.
     Note: Documentation size is 61440 bytes in 5 files.
[x]: All build dependencies are listed in BuildRequires, except for any that
     are listed in the exceptions section of Packaging Guidelines.
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[ ]: %config files are marked noreplace or the reason is justified.

  ----> see "No %config files under /usr." above

[x]: Each %files section contains %defattr if rpm < 4.4
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Fully versioned dependency in subpackages, if present.
[x]: If (and only if) the source package includes the text of the license(s)
     in its own file, then that file, containing the text of the license(s)
     for the package is included in %doc.
[x]: Package use %makeinstall only when make install' ' DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package do not use a name that already exist
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as provided
     in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local
[x]: Package successfully compiles and builds into binary rpms on at least one
     supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).

Perl:
[ ]: Package contains the mandatory BuildRequires and Requires:.

===== SHOULD items =====

Generic:
[x]: If the source package does not include license text(s) as a separate file
     from upstream, the packager SHOULD query upstream to include it.
[ ]: Final provides and requires are sane (see attachments).
[x]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[ ]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[ ]: Package should compile and build into binary rpms on all supported
     architectures.
[ ]: %check is present and all tests pass.
[ ]: Packages should try to preserve timestamps of original installed files.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: Dist tag is present.
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: SourceX tarball generation or download is documented.
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define.

===== EXTRA items =====

Generic:
[x]: Large data in /usr/share should live in a noarch subpackage if package is
     arched.
[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).
[x]: Spec file according to URL is the same as in SRPM.


Rpmlint
-------
Checking: mylvmbackup-0.14-1.fc20.noarch.rpm
mylvmbackup.noarch: W: non-etc-or-var-file-marked-as-conffile /usr/share/mylvmbackup/logerr.pm
mylvmbackup.noarch: W: non-etc-or-var-file-marked-as-conffile /usr/share/mylvmbackup/backupfailure.pm
mylvmbackup.noarch: W: non-etc-or-var-file-marked-as-conffile /usr/share/mylvmbackup/preflush.pm

  ----> see "No %config files under /usr." above

mylvmbackup.noarch: E: non-readable /etc/mylvmbackup.conf 0600L

  ----> This is due to /etc/mylvmbackup.conf potentially containing
        sensitive information (mysql password, ...). 
        There is a "--password=string" option to mylvmbackup,
        but in case someone writes password into /etc/mylvmbackup.conf
        it's safer to keep the permission as they are now (0600).

mylvmbackup.noarch: E: incorrect-fsf-address /usr/bin/mylvmbackup

  ----> incorrect postal address of FSF
        "In all cases, upstream should be informed about this. This is the only requirement with respect to this error."
        http://fedoraproject.org/wiki/Common_Rpmlint_issues#incorrect-fsf-address
        Please write to https://launchpad.net/~mylvmbackup-discuss
        or https://bugs.launchpad.net/mylvmbackup

1 packages and 0 specfiles checked; 2 errors, 3 warnings.




Rpmlint (installed packages)
----------------------------
# rpmlint mylvmbackup
mylvmbackup.noarch: W: non-etc-or-var-file-marked-as-conffile /usr/share/mylvmbackup/logerr.pm
mylvmbackup.noarch: W: non-etc-or-var-file-marked-as-conffile /usr/share/mylvmbackup/backupfailure.pm
mylvmbackup.noarch: W: non-etc-or-var-file-marked-as-conffile /usr/share/mylvmbackup/preflush.pm
mylvmbackup.noarch: E: non-readable /etc/mylvmbackup.conf 0600L
mylvmbackup.noarch: E: incorrect-fsf-address /usr/bin/mylvmbackup

1 packages and 0 specfiles checked; 2 errors, 3 warnings.
# echo 'rpmlint-done:'



Requires
--------
mylvmbackup (rpmlib, GLIBC filtered):
    config(mylvmbackup)
    perl(:MODULE_COMPAT_5.16.3)
    perl(Date::Format)
    perl(Sys::Hostname)
    perl(strict)



Provides
--------
mylvmbackup:
    config(mylvmbackup)
    mylvmbackup
    perl(backupfailure)
    perl(logerr)
    perl(preflush)



Source checksums
----------------
http://www.lenzg.net/mylvmbackup/mylvmbackup-0.14.tar.gz :
  CHECKSUM(SHA256) this package     : a979082f525f5b0b44bd09169938f2b5d8394fc403fc8b6a6e8b809d7c1a5724
  CHECKSUM(SHA256) upstream package : a979082f525f5b0b44bd09169938f2b5d8394fc403fc8b6a6e8b809d7c1a5724


Generated by fedora-review 0.4.1 (b2e211f) last change: 2013-04-29
Buildroot used: fedora-rawhide-i386
Command line :/usr/bin/fedora-review -m fedora-rawhide-i386 -b 977646

Comment 2 Christopher Meng 2013-06-27 03:22:41 UTC
1) These 3 pm files are scripts, is it ok to put them under /etc?

2) /etc/mylvmbackup.conf has permision 0600L because it contains the MySQL root user's password. I don't think we want to expose that to any user on the system.

3) I will pull in mariadb-server instead of -libs because this is a script for backup, not just need a file. In order to backup database we need to have a running databases.

I know the problem, and I think upstream should create a new feature that reading such information from a file instead of script itself, right?

Comment 3 marcindulak 2013-06-27 09:16:33 UTC
(In reply to Christopher Meng from comment #2)
> 1) These 3 pm files are scripts, is it ok to put them under /etc?

are they actually used, or only provided as examples of hooks?
If they are just examples then let's store them under docs, e.g:
%doc hooks
and we don't need to change hooksdir=/etc/mylvmbackup/hooks in /etc/mylvmbackup.conf

If they are necessary for mylvmbackup then it's OK to have such files under /etc
(there are already several executables there: find /etc -perm 755 -type f),
my suggestion is as in comment #1

> 
> 2) /etc/mylvmbackup.conf has permision 0600L because it contains the MySQL
> root user's password. I don't think we want to expose that to any user on
> the system.
> 
> 3) I will pull in mariadb-server instead of -libs because this is a script
> for backup, not just need a file. In order to backup database we need to
> have a running databases.

OK, mariadb-server pulls mariadb-libs as dependency
What about EL6: mysql-server?
I guess having mylvmbackup in EPEL is more interesting than Fedora.

> 
> I know the problem, and I think upstream should create a new feature that
> reading such information from a file instead of script itself, right?

There is a "--password=string" option to mylvmbackup, but that
just moves sensitive data to a script that calls it,
instead of having it in /etc/mylvmbackup.conf
Encryption would be desirable here.

Comment 4 Lenz Grimmer 2013-07-04 10:15:28 UTC
Hi,

thanks for the review and for looking into including mylvmbackup in your distribution.

(In reply to Marcin.Dulak from comment #3)
> (In reply to Christopher Meng from comment #2)
> > 1) These 3 pm files are scripts, is it ok to put them under /etc?
> 
> are they actually used, or only provided as examples of hooks?

These are just examples (empty templates), a user can either edit or replace them with actual scripts.

> If they are just examples then let's store them under docs, e.g:
> %doc hooks

I'm fine with that.

> and we don't need to change hooksdir=/etc/mylvmbackup/hooks in
> /etc/mylvmbackup.conf

How about changing hooksdir to /usr/share/mylvmbackup instead and adding this empty directory to the spec file (in addition to putting the hooks in the docs directory? This might be a more appropriate place than /etc/
 
> If they are necessary for mylvmbackup then it's OK to have such files under
> /etc
> (there are already several executables there: find /etc -perm 755 -type f),
> my suggestion is as in comment #1

They are not necessary, these hooks can be used to implement additional functionality as needed. We just need to ensure that RPM does not mangle any of these hooks in case the user has modified them.

> > 2) /etc/mylvmbackup.conf has permision 0600L because it contains the MySQL
> > root user's password. I don't think we want to expose that to any user on
> > the system.
> > 
> > 3) I will pull in mariadb-server instead of -libs because this is a script
> > for backup, not just need a file. In order to backup database we need to
> > have a running databases.
> 
> OK, mariadb-server pulls mariadb-libs as dependency
> What about EL6: mysql-server?
> I guess having mylvmbackup in EPEL is more interesting than Fedora.

Define "interesting" :)

I think it would make sense having in both, but I'll continue to provide my own RPM builds from the SUSE build service as well.

> > I know the problem, and I think upstream should create a new feature that
> > reading such information from a file instead of script itself, right?
> 
> There is a "--password=string" option to mylvmbackup, but that
> just moves sensitive data to a script that calls it,
> instead of having it in /etc/mylvmbackup.conf
> Encryption would be desirable here.

There are other options how a MySQL client like mylvmbackup can store the password - http://dev.mysql.com/doc/refman/5.6/en/password-security-user.html provides alternative options to storing it in the configuration file. This is mentioned in the mylvmbackup man page as well.

Comment 5 marcindulak 2013-07-04 14:33:28 UTC
(In reply to Lenz Grimmer from comment #4)
> Hi,
> 
> thanks for the review and for looking into including mylvmbackup in your
> distribution.
> 
> (In reply to Marcin.Dulak from comment #3)
> > (In reply to Christopher Meng from comment #2)
> > > 1) These 3 pm files are scripts, is it ok to put them under /etc?
> > 
> > are they actually used, or only provided as examples of hooks?
> 
> These are just examples (empty templates), a user can either edit or replace
> them with actual scripts.
> 
> > If they are just examples then let's store them under docs, e.g:
> > %doc hooks
> 
> I'm fine with that.
> 
> > and we don't need to change hooksdir=/etc/mylvmbackup/hooks in
> > /etc/mylvmbackup.conf
> 
> How about changing hooksdir to /usr/share/mylvmbackup instead and adding
> this empty directory to the spec file (in addition to putting the hooks in
> the docs directory? This might be a more appropriate place than /etc/

yes, that sounds good. In this way if user creates the hook files
under /usr/share/mylvmbackup they will be preserved
without being controlled by rpm.

>  
> > If they are necessary for mylvmbackup then it's OK to have such files under
> > /etc
> > (there are already several executables there: find /etc -perm 755 -type f),
> > my suggestion is as in comment #1
> 
> They are not necessary, these hooks can be used to implement additional
> functionality as needed. We just need to ensure that RPM does not mangle any
> of these hooks in case the user has modified them.
> 
> > > 2) /etc/mylvmbackup.conf has permision 0600L because it contains the MySQL
> > > root user's password. I don't think we want to expose that to any user on
> > > the system.
> > > 
> > > 3) I will pull in mariadb-server instead of -libs because this is a script
> > > for backup, not just need a file. In order to backup database we need to
> > > have a running databases.
> > 
> > OK, mariadb-server pulls mariadb-libs as dependency
> > What about EL6: mysql-server?
> > I guess having mylvmbackup in EPEL is more interesting than Fedora.
> 
> Define "interesting" :)
> 
> I think it would make sense having in both, but I'll continue to provide my
> own RPM builds from the SUSE build service as well.
> 
> > > I know the problem, and I think upstream should create a new feature that
> > > reading such information from a file instead of script itself, right?
> > 
> > There is a "--password=string" option to mylvmbackup, but that
> > just moves sensitive data to a script that calls it,
> > instead of having it in /etc/mylvmbackup.conf
> > Encryption would be desirable here.
> 
> There are other options how a MySQL client like mylvmbackup can store the
> password -
> http://dev.mysql.com/doc/refman/5.6/en/password-security-user.html provides
> alternative options to storing it in the configuration file. This is
> mentioned in the mylvmbackup man page as well.

Comment 6 marcindulak 2013-09-02 08:36:56 UTC
Hi, any progress here?

Comment 7 Pavel Alexeev 2014-04-25 12:42:15 UTC
Christopher, if you are still interesting I could review it.

Comment 8 Christopher Meng 2014-07-01 16:15:42 UTC
NEW SPEC URL: http://us-la.cicku.me/mylvmbackup.spec
NEW SRPM URL: http://us-la.cicku.me/mylvmbackup-0.15-1.fc21.src.rpm

Comment 9 David Dick 2014-07-09 11:48:09 UTC
Please add the following

BR perl(lib)
BR perl(MIME::Lite)
BR perl(Config::IniFiles)
BR perl(Date::Format)
BR perl(DBD::mysql)
BR perl(DBI)
BR perl(diagnostics)
BR perl(Fcntl)
BR perl(File::Basename)
BR perl(File::Copy)
BR perl(File::Path)
BR perl(File::Temp)
BR perl(Getopt::Long)
BR perl(strict)
BR perl(Sys::Hostname)
BR perl(Sys::Syslog)

Also, while upstream has provided no tests to allow us to check the package works as expected on different archs, if you add a basic check section, such as

%check
perl -c %{name}

that will help with ensuring the binary will at least compile correctly.

If you add 

%dir %{_datadir}/%{name}

to %files and 

mkdir -p %{buildroot}%{_datadir}/%{name}

to the end of the %install section, that will provide a directory for users to place their own hooks.

i will continue this review tomorrow.

Comment 10 David Dick 2014-07-10 09:05:52 UTC
I was too harsh on upstream in my previous comment.

%check
make syntaxcheck

will accomplish the desired result.

The following additional BRs are also required

BR /usr/bin/pod2man
BR /usr/bin/pod2html

I was thinking about other possible targets for the /usr/share/mylvmbackup as /etc/mylvmbackup, however, i think that /usr/share/mylvmbackup is the best alternative for user supplied perl libraries.

Comment 11 Christopher Meng 2014-07-14 03:39:29 UTC
(In reply to David Dick from comment #10)
> I was too harsh on upstream in my previous comment.
> 
> %check
> make syntaxcheck

Added, although I think it's useless...

> The following additional BRs are also required
> 
> BR /usr/bin/pod2man
> BR /usr/bin/pod2html

Done.

> I was thinking about other possible targets for the /usr/share/mylvmbackup
> as /etc/mylvmbackup, however, i think that /usr/share/mylvmbackup is the
> best alternative for user supplied perl libraries.

I delete the pm files (because they are configuration files, I personally don't want users to lost them during the transaction), therefore just empty the folder and add the folder in the %files while placing hooks underneath the %_pkgdocdir.

NEW SPEC URL: http://us-la.cicku.me/mylvmbackup.spec
NEW SRPM URL: http://us-la.cicku.me/mylvmbackup-0.15-2.fc22.src.rpm

Comment 12 David Dick 2014-07-14 08:37:29 UTC
(In reply to Christopher Meng from comment #11)
> I delete the pm files (because they are configuration files, I personally
> don't want users to lost them during the transaction), therefore just empty
> the folder and add the folder in the %files while placing hooks underneath
> the %_pkgdocdir.

Okay.

I've missed one more BR which you'll need to add to pass syntaxcheck

BR perl(File::Copy::Recursive)

Also, the license appears to be GPLv2+ (see mylvmbackup line 7-8)

Sorry for missing these points on my first review.

Comment 13 Christopher Meng 2014-07-15 02:19:03 UTC
Thanks, fixed.

New Package SCM Request
=======================
Package Name: mylvmbackup
Short Description: Utility for creating MySQL backups via LVM snapshots
Upstream URL: http://www.lenzg.net/mylvmbackup/
Owners: cicku
Branches: f19 f20 f21 el6 epel7

Comment 14 Gwyn Ciesla 2014-07-15 12:34:33 UTC
Git done (by process-git-requests).

Comment 15 Fedora Update System 2014-07-16 01:31:37 UTC
mylvmbackup-0.15-2.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/mylvmbackup-0.15-2.fc20

Comment 16 Fedora Update System 2014-07-16 01:31:46 UTC
mylvmbackup-0.15-2.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/mylvmbackup-0.15-2.fc19

Comment 17 Fedora Update System 2014-07-16 01:31:56 UTC
mylvmbackup-0.15-2.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mylvmbackup-0.15-2.el6

Comment 18 Fedora Update System 2014-07-16 03:51:17 UTC
mylvmbackup-0.15-2.el6 has been pushed to the Fedora EPEL 6 testing repository.

Comment 19 Fedora Update System 2014-07-26 00:04:26 UTC
mylvmbackup-0.15-2.fc20 has been pushed to the Fedora 20 stable repository.

Comment 20 Fedora Update System 2014-07-28 03:27:01 UTC
mylvmbackup-0.15-2.fc19 has been pushed to the Fedora 19 stable repository.

Comment 21 Fedora Update System 2014-08-01 16:42:53 UTC
mylvmbackup-0.15-2.el6 has been pushed to the Fedora EPEL 6 stable repository.