Bug 1572765 - RFE: support second pass install of packages on live media
Summary: RFE: support second pass install of packages on live media
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Anaconda Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-27 19:15 UTC by David Cantrell
Modified: 2018-05-02 18:05 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)

Description David Cantrell 2018-04-27 19:15:28 UTC
The live media install method is useful, but it would be handy to offer a set of additional software that could be installed once the live image has been transferred to the target system.  For example, the install media could contain two additional yum repos.  The user could could select additional packages from those repos (depsolving against the repos + the installed base of the live image) and anaconda could just perform a yum install on the target after transferring the live image.

Comment 1 Martin Kolman 2018-04-27 20:15:04 UTC
(In reply to David Cantrell from comment #0)
> The live media install method is useful, but it would be handy to offer a
> set of additional software that could be installed once the live image has
> been transferred to the target system.  For example, the install media could
> contain two additional yum repos.
Yeah, that would keep the installation process offline & we would not need to make sure network is available before the additional packages can be installed.

Still, the repos would need to be somewhere on the live media other than the base image or else they would end up on the target system when we rsync the base image.

Also not sure if Live Media Creator can do such image layout.

>  The user could could select additional
I'm wondering how the UX for this would look like. The current software spoke is based on comps, so I wonder if comps could be used for this as well? Otherwise some other mechanism of grouping packages to logical groups would be needed & likely also a new spoke.

> packages from those repos (depsolving against the repos + the installed base
> of the live image) and anaconda could just perform a yum install on the
> target after transferring the live image.
We actually have some mid-term plans for uninstalling the Anaconda packages from the target system after live installation as they are generally not needed, can be quite sizable when all dependencies are included & all the extra packages make subsequent system updates longer.

The idea of also installing packages from on-image repos is relatively closely related to this.

Comment 2 Jiri Konecny 2018-05-02 06:18:30 UTC
(In reply to Martin Kolman from comment #1)
> (In reply to David Cantrell from comment #0)
> > The live media install method is useful, but it would be handy to offer a
> > set of additional software that could be installed once the live image has
> > been transferred to the target system.  For example, the install media could
> > contain two additional yum repos.
> Yeah, that would keep the installation process offline & we would not need
> to make sure network is available before the additional packages can be
> installed.

I don't think we should make those only offline repositories. I would prefer that if we want to add additional user repositories to the Live installation make this properly with possibility to add any repository as with the netinst installation.

Also when we are thinking about this, why not to create Live installation with DNF payload and media stored repository? Maybe I'm missing something but what is the benefit of the Live installation rsync payload over the DNF payload? Making Live inst with DNF payload could simplify uninstallation of packages and also this case.

Another problem I see is mixing payloads, we basically don't know what is inside of Live image so it could be harder to prevent conflicts and the performance of the installation will go down because there would be basically two transactions and the second can change packages in the first. 

> 
> Still, the repos would need to be somewhere on the live media other than the
> base image or else they would end up on the target system when we rsync the
> base image.
> 
> Also not sure if Live Media Creator can do such image layout.
> 
> >  The user could could select additional
> I'm wondering how the UX for this would look like. The current software
> spoke is based on comps, so I wonder if comps could be used for this as
> well? Otherwise some other mechanism of grouping packages to logical groups
> would be needed & likely also a new spoke.

I would definitely stay with existing solution than creating a new one. IMHO comps should be fine here.

Comment 3 Martin Kolman 2018-05-02 13:02:14 UTC
(In reply to Jiri Konecny from comment #2)
> (In reply to Martin Kolman from comment #1)
> > (In reply to David Cantrell from comment #0)
> > > The live media install method is useful, but it would be handy to offer a
> > > set of additional software that could be installed once the live image has
> > > been transferred to the target system.  For example, the install media could
> > > contain two additional yum repos.
> > Yeah, that would keep the installation process offline & we would not need
> > to make sure network is available before the additional packages can be
> > installed.
> 
> I don't think we should make those only offline repositories. I would prefer
> that if we want to add additional user repositories to the Live installation
> make this properly with possibility to add any repository as with the
> netinst installation.
There are several reasons to use only offline repositories (at least initially):
- no need to show and QA the source spoke on Live images (the source spoke might also confuse users as in most cases will not be needed)
- keeps the live installation offline, otherwise we would need to require network connectivity if any non-local repos are added, complicating the overall logic some more

There was a discussion on DevConf with the localization and Fedora Workstation people about Anaconda installing langpacks from online repositories, which resulted in the functionality being moved to Gnome Initial Setup instead so that Anaconda does not need to handle online package installation on live images. GIS runs on the target system and does some things that require network, so it's much better fit to handle langpack installation as well.

Not saying online package installation is impossible, but there are some significant pecularities on top of offline package installation that would have to be solved.

> 
> Also when we are thinking about this, why not to create Live installation
> with DNF payload and media stored repository? Maybe I'm missing something
> but what is the benefit of the Live installation rsync payload over the DNF
> payload?
There are some:
- you get exactly the same thing installed that you see in the live environment
- the installation is faster as we are just transferring files vs running a transaction for couple hundred packages with all the related overhead (unpacking of packages, checksumming, scriptlets, verification, dependency resolving, etc.)
- less combinations (we regularly have issues with some environment & group combinations not being installable due to packaging conflicts)
- smaller image size
-> the image would be at least twice as big if just the live image content was included as RPMs
-> the image would be much bigger (basically size of DVD + size of live image) if we provided the same options as on the DVD image (different DEs, etc.)

>Making Live inst with DNF payload could simplify uninstallation of
> packages and also this case.
> 
> Another problem I see is mixing payloads, we basically don't know what is
> inside of Live image so it could be harder to prevent conflicts
Yep, that's definitely an issue if this was a default-on feature - there could be basically *anything* in the live base image (technically there might not even be DNF/rpm).

> and the
> performance of the installation will go down because there would be
> basically two transactions and the second can change packages in the first.
It depends - if we are just removing or adding some packages, it should not take much longer. Also if the user does not exercise the option to add/remove stuff, we could just skip the second transaction.

BTW, do some of the stakeholders who maintain the current live deliverables know about this RFE or have requested such functionality in the past ? 

The Fedora live image most people likely use is the Fedora Workstation one and the Fedora Workstation group has been rather trending to make the installation process as minimal as possible rather than to add more features there. So it might be prudent to ask them if they would actually be interested in something like this than to work on something that will actually not be used in the end.

Comment 4 David Cantrell 2018-05-02 18:00:25 UTC
(In reply to Martin Kolman from comment #1)
> (In reply to David Cantrell from comment #0)
> > The live media install method is useful, but it would be handy to offer a
> > set of additional software that could be installed once the live image has
> > been transferred to the target system.  For example, the install media could
> > contain two additional yum repos.
> Yeah, that would keep the installation process offline & we would not need
> to make sure network is available before the additional packages can be
> installed.

Correct.

> Still, the repos would need to be somewhere on the live media other than the
> base image or else they would end up on the target system when we rsync the
> base image.

Correct, which would be something that LMC would need to expand to handle.

> Also not sure if Live Media Creator can do such image layout.

Yes, see above.

> >  The user could could select additional
> I'm wondering how the UX for this would look like. The current software
> spoke is based on comps, so I wonder if comps could be used for this as
> well? Otherwise some other mechanism of grouping packages to logical groups
> would be needed & likely also a new spoke.

Right, this is a problem.  I don't think relying on comps is necessarily a good idea here.  The use case I'm thinking about is a user that wants to create a common base image for a system (the live image) and then add in a second pass repo so they can select among some applications to add.  The user knows what application choices they want, so I would say those should become the options presented in the UX.  The second pass repo constructed would be depsolved against the live image so any additional runtime packages required for the selected packages would be in the second pass repo.  The exposed choices to the user are only the things they are thinking about.

For example, let's say the user creates a base system as a live image.  Then they add a second repo containing httpd, fldigi, and pv.  The only choices they want to see at install time are those packages, implying they want the runtime dependencies required for those.

Probably the easiest way to implement this would be to generate a comps file specifically for this second pass repo as its constructed.  That's something Composer could do.  (just thinking out loud)

> > packages from those repos (depsolving against the repos + the installed base
> > of the live image) and anaconda could just perform a yum install on the
> > target after transferring the live image.
> We actually have some mid-term plans for uninstalling the Anaconda packages
> from the target system after live installation as they are generally not
> needed, can be quite sizable when all dependencies are included & all the
> extra packages make subsequent system updates longer.
> 
> The idea of also installing packages from on-image repos is relatively
> closely related to this.

Right.

Comment 5 David Cantrell 2018-05-02 18:03:51 UTC
(In reply to Jiri Konecny from comment #2)
> (In reply to Martin Kolman from comment #1)
> > (In reply to David Cantrell from comment #0)
> > > The live media install method is useful, but it would be handy to offer a
> > > set of additional software that could be installed once the live image has
> > > been transferred to the target system.  For example, the install media could
> > > contain two additional yum repos.
> > Yeah, that would keep the installation process offline & we would not need
> > to make sure network is available before the additional packages can be
> > installed.
> 
> I don't think we should make those only offline repositories. I would prefer
> that if we want to add additional user repositories to the Live installation
> make this properly with possibility to add any repository as with the
> netinst installation.

Adding additional repos by network doesn't work for all use cases.  The desire here is to create a customized self-contained install image.

> Also when we are thinking about this, why not to create Live installation
> with DNF payload and media stored repository? Maybe I'm missing something
> but what is the benefit of the Live installation rsync payload over the DNF
> payload? Making Live inst with DNF payload could simplify uninstallation of
> packages and also this case.

The advantage from my point of view is the speed.  DNF and yum are slow.  If the amount of time spent depsolving in dnf can be reduced, I feel that is a win.  The use case here is that the core part of every installed system will be the same, so start with the live payload and once that's on the system perform a second pass installation from the other repo.

And why do that?  Well, they may want to install from this customized media and not select any of those optional second pass packages.

> Another problem I see is mixing payloads, we basically don't know what is
> inside of Live image so it could be harder to prevent conflicts and the
> performance of the installation will go down because there would be
> basically two transactions and the second can change packages in the first. 

We would know what's in the live image because we would be constructing it.

> > Still, the repos would need to be somewhere on the live media other than the
> > base image or else they would end up on the target system when we rsync the
> > base image.
> > 
> > Also not sure if Live Media Creator can do such image layout.
> > 
> > >  The user could could select additional
> > I'm wondering how the UX for this would look like. The current software
> > spoke is based on comps, so I wonder if comps could be used for this as
> > well? Otherwise some other mechanism of grouping packages to logical groups
> > would be needed & likely also a new spoke.
> 
> I would definitely stay with existing solution than creating a new one. IMHO
> comps should be fine here.

See my comps idea above.

Comment 6 David Cantrell 2018-05-02 18:05:47 UTC
(In reply to Martin Kolman from comment #3)
> (In reply to Jiri Konecny from comment #2)
> > (In reply to Martin Kolman from comment #1)
> > > (In reply to David Cantrell from comment #0)
> > > > The live media install method is useful, but it would be handy to offer a
> > > > set of additional software that could be installed once the live image has
> > > > been transferred to the target system.  For example, the install media could
> > > > contain two additional yum repos.
> > > Yeah, that would keep the installation process offline & we would not need
> > > to make sure network is available before the additional packages can be
> > > installed.
> > 
> > I don't think we should make those only offline repositories. I would prefer
> > that if we want to add additional user repositories to the Live installation
> > make this properly with possibility to add any repository as with the
> > netinst installation.
> There are several reasons to use only offline repositories (at least
> initially):
> - no need to show and QA the source spoke on Live images (the source spoke
> might also confuse users as in most cases will not be needed)
> - keeps the live installation offline, otherwise we would need to require
> network connectivity if any non-local repos are added, complicating the
> overall logic some more
> 
> There was a discussion on DevConf with the localization and Fedora
> Workstation people about Anaconda installing langpacks from online
> repositories, which resulted in the functionality being moved to Gnome
> Initial Setup instead so that Anaconda does not need to handle online
> package installation on live images. GIS runs on the target system and does
> some things that require network, so it's much better fit to handle langpack
> installation as well.
> 
> Not saying online package installation is impossible, but there are some
> significant pecularities on top of offline package installation that would
> have to be solved.
> 
> > 
> > Also when we are thinking about this, why not to create Live installation
> > with DNF payload and media stored repository? Maybe I'm missing something
> > but what is the benefit of the Live installation rsync payload over the DNF
> > payload?
> There are some:
> - you get exactly the same thing installed that you see in the live
> environment
> - the installation is faster as we are just transferring files vs running a
> transaction for couple hundred packages with all the related overhead
> (unpacking of packages, checksumming, scriptlets, verification, dependency
> resolving, etc.)
> - less combinations (we regularly have issues with some environment & group
> combinations not being installable due to packaging conflicts)
> - smaller image size
> -> the image would be at least twice as big if just the live image content
> was included as RPMs
> -> the image would be much bigger (basically size of DVD + size of live
> image) if we provided the same options as on the DVD image (different DEs,
> etc.)
> 
> >Making Live inst with DNF payload could simplify uninstallation of
> > packages and also this case.
> > 
> > Another problem I see is mixing payloads, we basically don't know what is
> > inside of Live image so it could be harder to prevent conflicts
> Yep, that's definitely an issue if this was a default-on feature - there
> could be basically *anything* in the live base image (technically there
> might not even be DNF/rpm).
> 
> > and the
> > performance of the installation will go down because there would be
> > basically two transactions and the second can change packages in the first.
> It depends - if we are just removing or adding some packages, it should not
> take much longer. Also if the user does not exercise the option to
> add/remove stuff, we could just skip the second transaction.
> 
> BTW, do some of the stakeholders who maintain the current live deliverables
> know about this RFE or have requested such functionality in the past ? 

Don't know who those are.

> The Fedora live image most people likely use is the Fedora Workstation one
> and the Fedora Workstation group has been rather trending to make the
> installation process as minimal as possible rather than to add more features
> there. So it might be prudent to ask them if they would actually be
> interested in something like this than to work on something that will
> actually not be used in the end.

This RFE is for a Composer use case.


Note You need to log in before you can comment on or make changes to this bug.