Bug 1900371 - Can't clone mozilla-central with mercurial-5.4-4
Summary: Can't clone mozilla-central with mercurial-5.4-4
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mercurial
Version: 33
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Mads Kiilerich
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-11-22 15:50 UTC by Bill Gianopoulos
Modified: 2020-12-26 01:36 UTC (History)
7 users (show)

Fixed In Version: mercurial-5.4.2-3.fc33
Clone Of:
Environment:
Last Closed: 2020-12-26 01:36:24 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Bill Gianopoulos 2020-11-22 15:50:41 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Bill Gianopoulos 2020-11-22 15:55:52 UTC
OK so 

Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.  attempt to "hg clone https://hg.mozilla.org/moailla-central"


Actual results

hangs at the :applying file changes" step forever using 100% of cpu.


Expected results:

Clone should proceed normally

Comment 2 Bill Gianopoulos 2020-11-22 15:57:33 UTC
Downgrading to version 5.2-5.fc33 from koji fixes the issue.

Comment 3 Bill Gianopoulos 2020-11-22 16:13:15 UTC
OK change that to

hg clone https://hg.mozilla.org/mozilla-central

Comment 4 Bill Gianopoulos 2020-11-23 17:13:10 UTC
This works just as badly with 5.4-1.eln102.x86_64.

There does not seem to be a version on koji between 5.2-5.fc33 and 5.4-1.eln102.

Comment 5 Bill Gianopoulos 2020-11-24 16:10:48 UTC
This works find under mercurial 5.6 on Windows.

Comment 6 Fedora Admin user for bugzilla script actions 2020-11-26 14:53:22 UTC
This package has changed maintainer in the Fedora.
Reassigning to the new maintainer of this component.

Comment 7 Ondřej Pohořelský 2020-12-02 13:20:57 UTC
This is fixed in my WIP PR to drop Python 2 dependency.

https://src.fedoraproject.org/rpms/mercurial/pull-request/13
https://copr.fedorainfracloud.org/coprs/opohorel/mercurial/packages/

Comment 8 Mads Kiilerich 2020-12-03 16:22:46 UTC
@Ondřej: what is your theory of what is causing the problem and how it should be fixed?

Comment 9 Ondřej Pohořelský 2020-12-04 11:21:51 UTC
(In reply to Mads Kiilerich from comment #8)
> @Ondřej: what is your theory of what is causing the problem and how it
> should be fixed?

To be honest, I don't know. When this bug was reported I already had rebased Mercurial to release 5.6.
I only tested if I can reproduce this bug on my version, which I can't.

This leads me to think that it might be upstream issue that was fixed by rebase.

Comment 10 Mads Kiilerich 2020-12-04 12:12:20 UTC
I would be careful to claim it fixed based on one failed attempt at reproducing. I have no doubt that other users have been able to clone that repo with 5.4 .

Can anybody else reproduce the problem?

Also, it is not feasible to update f33 to 5.6 .

@wgianopoulos can you reproduce with https://koji.fedoraproject.org/koji/buildinfo?buildID=1655255 ?

Comment 11 Bill Gianopoulos 2020-12-04 16:10:58 UTC
I already said it works with 5.6 bak in comment 5.  I does not just fail in cloning it fails if i use the same clone bundle under 5.4 under Linux and works under windows where i am running 5.6.  SO this is not a network issue.

Comment 12 Bill Gianopoulos 2020-12-04 16:12:18 UTC
It fails during applying the bundle to freshly init'd directory.

Comment 13 Mads Kiilerich 2020-12-04 18:35:59 UTC
Thanks for clarifying comment 5. I understood it as saying that it worked with 5.6 on another machine under Windows.

f34 is now using 5.6.1 . Great if that solves your problem.

It is not feasible to update f33 to 5.6, so confirmation with that version has limited value for f33. If 5.4.2 also works for you, that will be the solution in f33. Thus, testing of that version is relevant. If not, we will need a better understanding of the problem before we can see if we can find a solution.

Exactly how do you reproduce with bundle (to rule out network issues)? Like
  hg init mozilla-central && cd mozilla-central && hg incoming -q --bundle ../mozilla-central.hg https://hg.mozilla.org/mozilla-central && hg pull ../mozilla-central.hg
?

4+ GB download takes some time here and is hard to iterate on ...

Comment 14 Mads Kiilerich 2020-12-04 21:04:35 UTC
I gave up trying to reproduce after downloading 23 GB bundle and running out of disk space.

Comment 15 Bill Gianopoulos 2020-12-05 13:06:13 UTC
OK first well sort of on different machine under windows on the same hardware but all of my f33 systems dual boot.  so same hardware running windows using the same bundle file.

As far as what do a do with the bundle file?  

hg init mozilla-central
cd mozilla-central
hg unbundle ../mozilla-central.hg (that is the name i gave the bundle file)

at this point it gets to adding file changes, goes to the end of the progress meter all changes added and then hangs there forever using 100% of CPU.

Had that worked correct next steps would be:

echo '[paths]' >> .hg/hgrc
echo 'default = https://hg.mozilla.org/mozilla-central' >> .hg/hgr
hg pull
hg update

Next  why is it unfeasible to provide version 5.6 on f33?
Is this a technical issue, like too many pre-requisites?

Or is it political, like too late to get it approved?

Will the f34 package work on f44?


Also thought I made this clear earlier.  The 5.2 build I am running is the newest f33 build available on koji that does not have this issue.  I already tried earlier 5.4 builds and they get the same failure.

Also since this is obviously an upstream issue, could we ask the upstream people what is different between 5.6 and 5.4 that "fixed" this?  Might be a simple patch you could add to a 5.4 build.

Comment 16 Bill Gianopoulos 2020-12-05 13:13:10 UTC
(In reply to Bill Gianopoulos from comment #15)
 it approved?
> 
> Will the f34 package work on f44?
                               ^^^
                               f33

Comment 17 Bill Gianopoulos 2020-12-05 16:29:01 UTC
I have a report form someone else that this works find with Mercurial 5.3.2

Comment 18 Bill Gianopoulos 2020-12-05 16:36:34 UTC
So perhaps a change added between 5.3.2 and 5.4 that could be backed out.

Comment 19 Bill Gianopoulos 2020-12-05 18:20:53 UTC
OK so same machine same OS failure under mercurial 5.4  works fine with either version 5.2 or version 5.6.1!

Comment 20 Bill Gianopoulos 2020-12-05 18:26:43 UTC
(In reply to Bill Gianopoulos from comment #19)
> OK so same machine same OS failure under mercurial 5.4  works fine with
> either version 5.2 or version 5.6.1!

Oh should have said under f34 rawhide!

Comment 21 Mads Kiilerich 2020-12-07 23:20:01 UTC
I have done some testing and made some observations:

I don't see "can't clone" or "hangs forever". It is just sometimes slow while updating phases. Cloning a 10 GB repo will of course be slow, but instead of something like ½ hour (plus download time) on my slow machine, it spends an additional hour without visible progress. Inspection with gdb shows it is doing work in mercurial/phases.py registernew.

Also, cloning with py3 is "fast". It is only slow on py2.

Users are thus generally able to clone mozilla-central with patience or by using mercurial-py3.

Even without considering the 5.6 upgrade, Ondřej's proposal would avoid the problem by dropping py2 support. Testing with the proposed changes will be with py3 ... which doesn't have a problem anyway. The same can already be achieved in f33 with 5.4: just use mercurial-py3 instead of mercurial-py2.

Upstream https://www.mercurial-scm.org/wiki/Release5.6 mentions fixes related to phases.registernew. (The regression seems to have been introduced in https://www.mercurial-scm.org/repo/hg/rev/fdc802f29b2c , released in 5.4 .) The fix in https://www.mercurial-scm.org/repo/hg/rev/29a259be6424 (released in 5.6) seems to pinpoint and fix the performance issue.

I have included this 29a259be6424 patch in mercurial-5.4.2-3.fc33 on https://koji.fedoraproject.org/koji/taskinfo?taskID=57002325 . Please test and report back.


Mercurial has a very stable command line API, but the python API is internal and unstable between major versions (like 5.5 to 5.6). Major updates will break other packages in Fedora and extensions installed by users. Major versions also have the general risk of feature development introducing bugs or "unexpected changes". Major updates and should only be done in fedora Rawhide - not rolled out to otherwise happy users of released Fedora versions. Mercurial does however have a very strict bugfix-only policy on release branches, and it is thus generally safe to make minor updates in Fedora releases. It is thus only feasible to update Fedora 33 to 5.4.2 (which is the last version in the 5.4 series).

Comment 22 Bill Gianopoulos 2020-12-07 23:42:40 UTC
Have you tried cloning mozilla(In reply to Mads Kiilerich from comment #21)
> I have done some testing and made some observations:
> 
> I don't see "can't clone" or "hangs forever". It is just sometimes slow
> while updating phases. Cloning a 10 GB repo will of course be slow, but
> instead of something like ½ hour (plus download time) on my slow machine, it
> spends an additional hour without visible progress. Inspection with gdb
> shows it is doing work in mercurial/phases.py registernew.
> 
> Also, cloning with py3 is "fast". It is only slow on py2.
> 
> Users are thus generally able to clone mozilla-central with patience or by
> using mercurial-py3.
> 
> Even without considering the 5.6 upgrade, Ondřej's proposal would avoid the
> problem by dropping py2 support. Testing with the proposed changes will be
> with py3 ... which doesn't have a problem anyway. The same can already be
> achieved in f33 with 5.4: just use mercurial-py3 instead of mercurial-py2.
> 
> Upstream https://www.mercurial-scm.org/wiki/Release5.6 mentions fixes
> related to phases.registernew. (The regression seems to have been introduced
> in https://www.mercurial-scm.org/repo/hg/rev/fdc802f29b2c , released in 5.4
> .) The fix in https://www.mercurial-scm.org/repo/hg/rev/29a259be6424
> (released in 5.6) seems to pinpoint and fix the performance issue.
> 
> I have included this 29a259be6424 patch in mercurial-5.4.2-3.fc33 on
> https://koji.fedoraproject.org/koji/taskinfo?taskID=57002325 . Please test
> and report back.
> 
> 
> Mercurial has a very stable command line API, but the python API is internal
> and unstable between major versions (like 5.5 to 5.6). Major updates will
> break other packages in Fedora and extensions installed by users. Major
> versions also have the general risk of feature development introducing bugs
> or "unexpected changes". Major updates and should only be done in fedora
> Rawhide - not rolled out to otherwise happy users of released Fedora
> versions. Mercurial does however have a very strict bugfix-only policy on
> release branches, and it is thus generally safe to make minor updates in
> Fedora releases. It is thus only feasible to update Fedora 33 to 5.4.2
> (which is the last version in the 5.4 series).

Have you actually tried to clone mozilla-central using Mercurial 5.4 or are you just talking out of your hat?

Comment 23 Bill Gianopoulos 2020-12-09 12:18:20 UTC
Upgrading to version 5.4.2-3.fc33 now available on koji fixes this.

Comment 24 Bill Gianopoulos 2020-12-09 12:43:35 UTC
(In reply to Bill Gianopoulos from comment #22)
> Have you tried cloning mozilla(In reply to Mads Kiilerich from comment #21)
> > I have done some testing and made some observations:
> > 
> > I don't see "can't clone" or "hangs forever". It is just sometimes slow
> > while updating phases. Cloning a 10 GB repo will of course be slow, but
> > instead of something like ½ hour (plus download time) on my slow machine, it
> > spends an additional hour without visible progress. Inspection with gdb
> > shows it is doing work in mercurial/phases.py registernew.
> > 
> > Also, cloning with py3 is "fast". It is only slow on py2.
> > 
> > Users are thus generally able to clone mozilla-central with patience or by
> > using mercurial-py3.
> > 
> > Even without considering the 5.6 upgrade, Ondřej's proposal would avoid the
> > problem by dropping py2 support. Testing with the proposed changes will be
> > with py3 ... which doesn't have a problem anyway. The same can already be
> > achieved in f33 with 5.4: just use mercurial-py3 instead of mercurial-py2.
> > 
> > Upstream https://www.mercurial-scm.org/wiki/Release5.6 mentions fixes
> > related to phases.registernew. (The regression seems to have been introduced
> > in https://www.mercurial-scm.org/repo/hg/rev/fdc802f29b2c , released in 5.4
> > .) The fix in https://www.mercurial-scm.org/repo/hg/rev/29a259be6424
> > (released in 5.6) seems to pinpoint and fix the performance issue.
> > 
> > I have included this 29a259be6424 patch in mercurial-5.4.2-3.fc33 on
> > https://koji.fedoraproject.org/koji/taskinfo?taskID=57002325 . Please test
> > and report back.
> > 
> > 
> > Mercurial has a very stable command line API, but the python API is internal
> > and unstable between major versions (like 5.5 to 5.6). Major updates will
> > break other packages in Fedora and extensions installed by users. Major
> > versions also have the general risk of feature development introducing bugs
> > or "unexpected changes". Major updates and should only be done in fedora
> > Rawhide - not rolled out to otherwise happy users of released Fedora
> > versions. Mercurial does however have a very strict bugfix-only policy on
> > release branches, and it is thus generally safe to make minor updates in
> > Fedora releases. It is thus only feasible to update Fedora 33 to 5.4.2
> > (which is the last version in the 5.4 series).
> 
> Have you actually tried to clone mozilla-central using Mercurial 5.4 or are
> you just talking out of your hat?


Sorry I was a bit terse here!  Thank you for fixing!

Comment 25 Bill Gianopoulos 2020-12-09 13:13:50 UTC
(In reply to Bill Gianopoulos from comment #24)

> Sorry I was a bit terse here!  Thank you for fixing!

I think the word I was looking for here was snarky, rather than terse.  But I do apologize.  My frustration got the better of me.

Comment 26 Fedora Update System 2020-12-09 17:00:35 UTC
FEDORA-2020-21c79fcc13 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-21c79fcc13

Comment 27 Fedora Update System 2020-12-11 00:04:38 UTC
FEDORA-2020-21c79fcc13 has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-21c79fcc13`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-21c79fcc13

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 28 Fedora Update System 2020-12-26 01:36:24 UTC
FEDORA-2020-21c79fcc13 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.