Bug 1699234 - environment-modules causes severe bash slow-down
Summary: environment-modules causes severe bash slow-down
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: environment-modules
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jan Synacek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-12 08:01 UTC by Tim Niemueller
Modified: 2020-11-12 12:41 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-27 23:13:43 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tim Niemueller 2019-04-12 08:01:06 UTC
Description of problem:
When environment-modules is installed, the overall system performance for bash-driven workloads is reduced severely. See the following comparison:

# F29, environment-modules NOT installed
> time bash -c exit

real	0m0,003s
user	0m0,000s
sys	0m0,004s


# F29, environment-modules is installed
> time bash -c exit

real	0m0,074s
user	0m0,017s
sys	0m0,008s


This is more than a 20x slow-down (this is on an older dual-core laptop). This might not seem crucial for the raw numbers presented, but we have noticed a tremendous slow-down of our build process due to this (https://github.com/fawkesrobotics/fawkes/issues/73). We use bash as the Makefile shell interpreter for more features and flexibility. If environment-modules is installed, simply cleaning the working copy (make clean) takes considerably longer. Aside that this is noticeable in several other operations.


Version-Release number of selected component (if applicable):
environment-modules-4.2.3-1.fc29.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Run "time bash -c exit"
2. Install environment-modules
3. Run "time bash -c exit"

Actual results:
Observe the large increase in run-time of this simple command.

Expected results:
No noticeable slow-down.

Additional info:
On my system environment-modules seems to have been pulled in by Sphinx. I would prefer if environment-modules was strictly optional as to avoid its performance penalty and still be able to use the desired applications.

It seems that the large impact occurred after F27. Was there a major upgrade to environment-modules starting with F28, or have other factors contributed to it being so slow now? I assume that I had it installed most of the time, but we noticed this slowdown only with F28 and later.

Comment 1 Tim Niemueller 2019-04-12 09:16:43 UTC
It seems the upgrade to modules-4.0.0 happened in F28: https://src.fedoraproject.org/rpms/environment-modules/c/a663cb8c0c29726b2626caf0990f8689ce8a4afa?branch=master

In https://modules.readthedocs.io/en/stable/MIGRATING.html#migrating-from-v3-2-to-v4-0 it reads:
"Major evolution occurs with this v4.0 release as the traditional module command implemented in C is replaced by the native Tcl version. This full Tcl rewrite of the Modules package was started in 2002 and has now reached maturity to take over the binary version. This flavor change enables to refine and push forward the module concept."

So switching from a C-written tool to TCL may have caused the slow-down. In that case, it would be nice to make environment-modules entirely optional in order to be able to have a system without it at all times.

Or do you see other options, Jan?

Comment 2 Xavier Delaruelle 2019-04-12 18:48:37 UTC
The slow-down you describe is not related by the fact Modules is now fully written in Tcl rather C. What is new in Modules version 4 is that its initialization scripts automatically set the BASH_ENV variable and make it point to the Modules init script in order to make the "module" command also available for non-interactive environment.

So every time bash is spawned, it evaluates the Modules initialization script. This is what causes the slow down you notice.

I will work for next feature release of Modules on an installation option to disable the use of the BASH_ENV variable. Such an option may be interesting to set in the Fedora/EL package.

Comment 3 Tim Niemueller 2019-04-12 19:08:31 UTC
Thanks for the clarification, Xavier, appreciated. Looking forward to a fix landing in Fedora.

It might still be an idea only to only suggest environment-modules rather than require it, if it is not essential for the functioning of the package in question.

Comment 4 Xavier Delaruelle 2019-08-15 17:31:33 UTC
New release of environment-modules introduces the ability to control whether or not shell startup file should be set to ensure module command is defined once shell has been initialized. When enabled, the ENV and BASH_ENV environment variables are set, when module function is defined, to the Modules bourne shell initialization script. When disabled both ENV and BASH_ENV are left untouched. Prior version 4.3.0 this "set shell startup file" mechanism was enabled by default.

The 4.3.0 package built for Fedora (bug #1733752) sets the `--disable-set-shell-startup` configure option which disables the mechanism by default. If a sysadmin want the mecanishm enabled, he/she could enable it by adding the `module config set_shell_startup 1` line to the /etc/environment-modules/initrc configuration file.

This change should fix the issue described here.

Comment 5 Ben Cotton 2019-10-31 18:57:09 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 6 Ben Cotton 2019-11-27 23:13:43 UTC
Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.