Bug 1923745 - texlive-context %transfiletriggerin does a recursive search from / that can take a very long time
Summary: texlive-context %transfiletriggerin does a recursive search from / that can t...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: texlive-base
Version: 33
Hardware: Unspecified
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Tom "spot" Callaway
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-02-01 18:51 UTC by Anssi Hannula
Modified: 2021-02-11 01:52 UTC (History)
1 user (show)

Fixed In Version: texlive-base-20200327-21.fc33 texlive-base-20200327-21.fc32
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-11 01:42:51 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Anssi Hannula 2021-02-01 18:51:49 UTC
Description of problem:

texlive-context has %transfiletriggerin that runs
/usr/bin/mtxrun --generate &> /dev/null || :

The search path that mtxrun uses seems to include ".", and the rpm script is run with '/' as the working directory, causing mtxrun to perform a recursive search starting from the root, and stat()in every file in the tree.

It seems that the recursive search always stops at "/dev/fd/3/" with ENOENT (No such file or directory), so the issue only becomes noticeable if there is a large amount of files before "/dev" - at least in my case the directories are processed alphabetically, so this means directories that sort before "dev" in /.

I happen to have a very large remote file share mounted at /delta that has tens of millions of files.
To make the matter worse, the remote file system has symlink loops and mtxrun has no protection from those, causing the rpm installation process to get stuck for days without completion, continously using CPU and network resources.

I verified the issue on a clean Fedora 33 installation.


Version-Release number of selected component (if applicable):
20200327-20.fc33

How reproducible:
Always


Steps to Reproduce:
1. Install texlive-context and strace.
2. Run to emulate the transfiletriggerin script:
  sudo sh -c '(cd / && strace -e %file /usr/bin/mtxrun --generate)'

Actual results:
The process stat()s every file from / up to /dev/fd/3/, including recursively processing directories and mount points and file system loops.

Expected results:
The process should not perform searches from file system root.


Additional info:

Comment 1 Tom "spot" Callaway 2021-02-01 18:54:35 UTC
Clearly, the solution is to umount both /delta and /dev. ;)

I'll figure out how to make it run more narrowly (probably only from /usr).

Comment 2 Tom "spot" Callaway 2021-02-02 17:59:35 UTC
I think this will do the trick:

export TEXMFLOCAL=/usr/share/texlive/texmf-local

When I set that, it doesn't search through . anymore.

I've added it to the context transfiletriggerin in this build, please test and let me know if it resolves the issue on your end:

https://koji.fedoraproject.org/koji/taskinfo?taskID=61140967

Comment 3 Anssi Hannula 2021-02-02 19:29:40 UTC
Thanks, that build seems to work OK for me as well.

Comment 4 Fedora Update System 2021-02-02 19:59:21 UTC
FEDORA-2021-969de5378f has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2021-969de5378f

Comment 5 Fedora Update System 2021-02-02 19:59:21 UTC
FEDORA-2021-c08dcba23d has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-c08dcba23d

Comment 6 Fedora Update System 2021-02-03 01:31:23 UTC
FEDORA-2021-969de5378f has been pushed to the Fedora 32 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-969de5378f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-969de5378f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2021-02-03 02:30:15 UTC
FEDORA-2021-c08dcba23d has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-c08dcba23d`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-c08dcba23d

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2021-02-11 01:42:51 UTC
FEDORA-2021-c08dcba23d has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 9 Fedora Update System 2021-02-11 01:52:47 UTC
FEDORA-2021-969de5378f has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.