Bug 1173126 - can't build successive packages with --no-clean
Summary: can't build successive packages with --no-clean
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: mock
Version: el6
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Michael Simacek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1173563 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-12-11 13:39 UTC by Dave Love
Modified: 2015-02-20 00:33 UTC (History)
11 users (show)

Fixed In Version: mock-1.2.6-1.el6
Clone Of:
Environment:
Last Closed: 2015-01-21 23:09:05 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
fedora-review log (4.86 KB, text/plain)
2015-01-17 11:58 UTC, Florian "der-flo" Lehner
no flags Details

Description Dave Love 2014-12-11 13:39:31 UTC
Description of problem:

I don't know if this is intentional or not in the current epel-testing version:

In the previous version, you could build one package and build a second in
the same root directly with --no-clean to save time.  That now fails with

ERROR: Expected to find single rebuilt srpm, found 2.

Version-Release number of selected component (if applicable):
1.2.3-1

Comment 1 Miroslav Suchý 2014-12-11 13:44:49 UTC
That is intentional. I do not want to support concurent usage of the same buildroot. The builds may affect each other and therefore the build is not reproducible any more.
If you want to run two builds in parallel, then use:
  mock --uniqueext=build1 package1.src.rpm
  mock --uniqueext=build2 package2.src.rpm

Or you can run mockchain which automatize this.

Comment 2 Michael Simacek 2014-12-11 14:36:03 UTC
I'm sure the reporter didn't mean concurrently, but in succession, because I'm also having the same problem. And I would consider this to be supported use case, since it preserves the build stability if used correctly (if the BuildRequires of the second package are the same or subset of the first). The problem is a consequence of #483486. I think a solution to have both this and nosrc builds would be to delete the builddir except the SOURCES directory. The SRPMS, RPMS, SPECS, BUILD and BUILDROOT should be deleted. I will make the patch if you agree.

Comment 3 Dave Love 2014-12-11 14:44:15 UTC
For what it's worth, I didn't mean parallel builds (which didn't work in
the previous version as far as I know), just building one and re-using
the root some time after it's finished.  (Of course, I realize it can
make things non-reproducible.)  Experimentally, --uniqueext doesn't do
the same thing, though it's not clear from the doc -- you still get the
dependencies reinstalled.

Comment 4 Miroslav Suchý 2014-12-11 15:10:22 UTC
*nod* removing /builddir/build/{SRPM,RPMS,SPECS,BUILD,BUILDROOT,originals}/* sounds good to me. Well removing  /builddir/build/SRPM would be sufficient as well.

Feel free Michael to make the patch.

Comment 5 Miroslav Suchý 2014-12-12 13:08:47 UTC
*** Bug 1173563 has been marked as a duplicate of this bug. ***

Comment 6 Vít Ondruch 2014-12-12 13:46:58 UTC
(In reply to Miroslav Suchý from comment #4)
> Well removing  /builddir/build/SRPM would be sufficient

This is not enough of course, because it leaves the resulting rpms in RPMS directory.

Comment 7 Michael Simacek 2014-12-12 14:12:32 UTC
I already sent patch upstream, I'm including the commit message to clarify the proposed solution:

With --no-clean, delete /builddir except build/SOURCES [RHBZ#1173126]
In order to be able to have the ability to execute more successive builds with --no-clean, it's necessary to cleanup build directories from the previous build. Now the whole /builddir is deleted except SOURCES directory which is retained to support nosrc builds. This is further configurable with new config option 'exclude_from_homedir_cleanup', which contains a list of paths relative to /builddir that will be excluded from deletion in prebuild.


To put it simply, by default it deletes everything in the /builddir except /builddir/build/SOURCES. And this is further configurable for people who want to delete everything/keep more files.

Comment 8 Vít Ondruch 2014-12-12 14:25:05 UTC
Could you please clarify why to keep SOURCES? What is it nosrc build?

Comment 9 Miroslav Suchý 2014-12-12 17:45:38 UTC
> because it leaves the resulting rpms in RPMS directory.

but that is not a problem. Only problem is populated SRPM directory, because mock then does not know for sure which src.rpm should be used.

> Could you please clarify why to keep SOURCES? What is it nosrc build?

See bug 483486. For example see:
http://developerblog.redhat.com/2014/12/10/how-to-package-proprietary-software/
I must admit this was new to me as well.

> To put it simply, by default it deletes everything in the /builddir except /builddir/build/SOURCES.

I would suggest to remove only /builddir/build/* except SOURCES. The fact that previously whole /builddir was deleted is just coincidence. It was bug in code (it was removed when user was removed) and not intentional. If user want to clean it he should not use --no-clean at first place.

Comment 10 Vít Ondruch 2014-12-15 11:48:49 UTC
(In reply to Miroslav Suchý from comment #9)
> I would suggest to remove only /builddir/build/* except SOURCES. The fact
> that previously whole /builddir was deleted is just coincidence. It was bug
> in code (it was removed when user was removed) and not intentional. If user
> want to clean it he should not use --no-clean at first place.

Actually this is preferred solution for me as well, since it keeps bash history in case I need to used shell in between builds.

Comment 11 Michael Simacek 2014-12-16 14:19:11 UTC
(In reply to Miroslav Suchý from comment #9)
> > because it leaves the resulting rpms in RPMS directory.
> 
> but that is not a problem. Only problem is populated SRPM directory, because
> mock then does not know for sure which src.rpm should be used.

That will cause all the RPMS being copied to the resultdir, so even those which were built during previous runs. Not a big problem though, there are more important issues.

> 
> > Could you please clarify why to keep SOURCES? What is it nosrc build?
> 
> See bug 483486. For example see:
> http://developerblog.redhat.com/2014/12/10/how-to-package-proprietary-
> software/
> I must admit this was new to me as well.
> 
> > To put it simply, by default it deletes everything in the /builddir except /builddir/build/SOURCES.
> 
> I would suggest to remove only /builddir/build/* except SOURCES. The fact
> that previously whole /builddir was deleted is just coincidence. It was bug
> in code (it was removed when user was removed) and not intentional. If user
> want to clean it he should not use --no-clean at first place.

I believe it was intentional. There are programs that put their caches, generated files or other data in $HOME (usually hidden files/dirs) and some of them are run during build of packages. Therefore keeping those files threaten the build stability.


(In reply to Vít Ondruch from comment #10)
> (In reply to Miroslav Suchý from comment #9)
> > I would suggest to remove only /builddir/build/* except SOURCES. The fact
> > that previously whole /builddir was deleted is just coincidence. It was bug
> > in code (it was removed when user was removed) and not intentional. If user
> > want to clean it he should not use --no-clean at first place.
> 
> Actually this is preferred solution for me as well, since it keeps bash
> history in case I need to used shell in between builds.

The proposed solution makes it configurable, so keeping the bash history could be enabled with
config_opts['exclude_from_homedir_cleanup'].append('.bash_history')

But it's probably a good idea to have .bash_history there by default.

Comment 12 Vít Ondruch 2014-12-16 14:25:03 UTC
(In reply to Michael Simacek from comment #11)
> (In reply to Miroslav Suchý from comment #9)
> > > because it leaves the resulting rpms in RPMS directory.
> > 
> > but that is not a problem. Only problem is populated SRPM directory, because
> > mock then does not know for sure which src.rpm should be used.
> 
> That will cause all the RPMS being copied to the resultdir, so even those
> which were built during previous runs. Not a big problem though, there are
> more important issues.

Yes, that was my original concern

 
> > > Could you please clarify why to keep SOURCES? What is it nosrc build?
> > 
> > See bug 483486. For example see:
> > http://developerblog.redhat.com/2014/12/10/how-to-package-proprietary-
> > software/
> > I must admit this was new to me as well.
> > 
> > > To put it simply, by default it deletes everything in the /builddir except /builddir/build/SOURCES.
> > 
> > I would suggest to remove only /builddir/build/* except SOURCES. The fact
> > that previously whole /builddir was deleted is just coincidence. It was bug
> > in code (it was removed when user was removed) and not intentional. If user
> > want to clean it he should not use --no-clean at first place.
> 
> I believe it was intentional. There are programs that put their caches,
> generated files or other data in $HOME (usually hidden files/dirs) and some
> of them are run during build of packages. Therefore keeping those files
> threaten the build stability.

This might be the reason. But there are also other locations, where some build output may remain IMO. Not a big concern for me.

> (In reply to Vít Ondruch from comment #10)
> > (In reply to Miroslav Suchý from comment #9)
> > > I would suggest to remove only /builddir/build/* except SOURCES. The fact
> > > that previously whole /builddir was deleted is just coincidence. It was bug
> > > in code (it was removed when user was removed) and not intentional. If user
> > > want to clean it he should not use --no-clean at first place.
> > 
> > Actually this is preferred solution for me as well, since it keeps bash
> > history in case I need to used shell in between builds.
> 
> The proposed solution makes it configurable, so keeping the bash history
> could be enabled with
> config_opts['exclude_from_homedir_cleanup'].append('.bash_history')
> 
> But it's probably a good idea to have .bash_history there by default.

+1 for .bash_history

Comment 13 Sergio Basto 2015-01-06 19:06:06 UTC
Hi,   
1 - this is not Fedora EPEL bug, is fedora bug. 
2 - bug #1171601 is a consequence of this bug . 
3 - bug #1168116 seems also a consequence of this bug . 



(In reply to Miroslav Suchý from comment #4)
> *nod* removing /builddir/build/{SRPM,RPMS,SPECS,BUILD,BUILDROOT,originals}/*
> sounds good to me. Well removing  /builddir/build/SRPM would be sufficient
> as well.
> 
> Feel free Michael to make the patch.

The patch is revert the code that prevent cleaning  /builddir/build/ before --no-clean --rebuild

Comment 14 Michael Simacek 2015-01-07 10:52:14 UTC
Applied upstream as commits 405a623, 3698470. Behavior is as described in #c7. I also sent followup patch to exclude .bash_history.

Comment 15 Fedora Update System 2015-01-16 09:56:42 UTC
mock-1.2.4-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/mock-1.2.4-1.fc21

Comment 16 Fedora Update System 2015-01-16 09:57:20 UTC
mock-1.2.4-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/mock-1.2.4-1.fc20

Comment 17 Fedora Update System 2015-01-16 10:01:06 UTC
mock-1.2.4-1.el7 has been submitted as an update for Fedora EPEL 7.
https://admin.fedoraproject.org/updates/mock-1.2.4-1.el7

Comment 18 Fedora Update System 2015-01-16 10:01:44 UTC
mock-1.2.4-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.2.4-1.el6

Comment 19 Fedora Update System 2015-01-16 19:10:24 UTC
Package mock-1.2.4-1.el7:
* should fix your issue,
* was pushed to the Fedora EPEL 7 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=epel-testing mock-1.2.4-1.el7'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2015-0312/mock-1.2.4-1.el7
then log in and leave karma (feedback).

Comment 20 Florian "der-flo" Lehner 2015-01-17 11:58:41 UTC
Created attachment 981002 [details]
fedora-review log

hi!

As i mentioned in https://admin.fedoraproject.org/updates/FEDORA-2015-0784/mock-1.2.4-1.fc21 here is the log:

$ mock --version
1.2.4

Comment 21 Björn Esser (besser82) 2015-01-17 15:04:46 UTC
(In reply to Florian "der-flo" Lehner from comment #20)
> Created attachment 981002 [details]
> fedora-review log
> 
> hi!
> 
> As i mentioned in
> https://admin.fedoraproject.org/updates/FEDORA-2015-0784/mock-1.2.4-1.fc21
> here is the log:
> 
> $ mock --version
> 1.2.4

Could you please post your `fedora-review.log`, too?  It's located inside "/home/$USER/.cache/fedora-review.log"

Comment 22 Miroslav Suchý 2015-01-19 07:19:01 UTC
That is bug in fedora-review. See bug 1151943.

Comment 23 Fedora Update System 2015-01-21 23:09:05 UTC
mock-1.2.4-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 24 Fedora Update System 2015-01-27 03:04:35 UTC
mock-1.2.4-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 25 Fedora Update System 2015-01-28 15:20:28 UTC
mock-1.2.5-1.el7 has been submitted as an update for Fedora EPEL 7.
https://admin.fedoraproject.org/updates/mock-1.2.5-1.el7

Comment 26 Fedora Update System 2015-01-28 15:21:44 UTC
mock-1.2.5-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.2.5-1.el6

Comment 27 Fedora Update System 2015-01-28 15:22:55 UTC
mock-1.2.5-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/mock-1.2.5-1.fc20

Comment 28 Fedora Update System 2015-01-28 15:24:06 UTC
mock-1.2.5-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/mock-1.2.5-1.fc21

Comment 29 Fedora Update System 2015-02-02 17:22:28 UTC
mock-1.2.5-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 30 Fedora Update System 2015-02-03 04:42:30 UTC
mock-1.2.6-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.2.6-1.el6

Comment 31 Fedora Update System 2015-02-03 04:43:06 UTC
mock-1.2.6-1.el7 has been submitted as an update for Fedora EPEL 7.
https://admin.fedoraproject.org/updates/mock-1.2.6-1.el7

Comment 32 Fedora Update System 2015-02-03 04:43:29 UTC
mock-1.2.6-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/mock-1.2.6-1.fc20

Comment 33 Fedora Update System 2015-02-03 04:44:05 UTC
mock-1.2.6-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/mock-1.2.6-1.fc21

Comment 34 Fedora Update System 2015-02-09 05:29:20 UTC
mock-1.2.6-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 35 Fedora Update System 2015-02-15 02:55:35 UTC
mock-1.2.6-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 36 Fedora Update System 2015-02-20 00:32:59 UTC
mock-1.2.6-1.el7 has been pushed to the Fedora EPEL 7 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 37 Fedora Update System 2015-02-20 00:33:49 UTC
mock-1.2.6-1.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.