Description of problem: Using latex in Docker on Rawhide gives an "I can't find the format file `latex.fmt'!" error. Version-Release number of selected component (if applicable): texlive-2020-32.fc34.src.rpm How reproducible: Easy in docker Steps to Reproduce: 1. cat Dockerfile FROM registry.fedoraproject.org/fedora:rawhide RUN dnf -y update RUN dnf -y install texlive RUN printf '\\documentclass{article}\n\\begin{document}\ntest\n\\end{document}' > test.tex RUN latex test.tex 2. docker build . Actual results: his is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=latex) restricted \write18 enabled. kpathsea: Running mktexfmt latex.fmt mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order): mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes: mktexfmt: /root/.texlive2020/texmf-config/web2c/fmtutil.cnf mktexfmt [INFO]: writing formats under /root/.texlive2020/texmf-var/web2c mktexfmt [INFO]: did not find entry for byfmt=latex, skipped mktexfmt [INFO]: total formats: 0 mktexfmt [INFO]: exiting with status 0 I can't find the format file `latex.fmt'! Expected results: Works. Additional info:
Sidenote F33 (texlive-9:2020-31.fc33) works as expected: Output written on test.dvi (1 page, 244 bytes). Transcript written on test.log. Dockerfile: FROM registry.fedoraproject.org/fedora:33 RUN dnf -y update RUN dnf -y install texlive RUN printf '\\documentclass{article}\n\\begin{document}\ntest\n\\end{document}' > test.tex RUN latex test.tex
I tested texlive-9:2020-32.fc33 (from https://bodhi.fedoraproject.org/updates/FEDORA-2020-fb18e81c97) on F33 as well and it works as well. So my guess it must be something else that triggers the issue.
I just tested this with podman (since that's part of Fedora and should be identical) and I could not reproduce your rawhide failure. Perhaps rawhide was in a weird state that day? 1. cat Dockerfile FROM registry.fedoraproject.org/fedora:rawhide RUN dnf -y update RUN dnf -y install texlive RUN printf '\\documentclass{article}\n\\begin{document}\ntest\n\\end{document}'> test.tex RUN latex test.tex 2. podman build . Result: STEP 5: RUN latex test.tex This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=latex) restricted \write18 enabled. kpathsea: Running mktexfmt latex.fmt mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order): mktexfmt: /usr/share/texlive/texmf-dist/web2c/fmtutil.cnf mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes: mktexfmt: /root/.texlive2020/texmf-config/web2c/fmtutil.cnf mktexfmt [INFO]: writing formats under /root/.texlive2020/texmf-var/web2c mktexfmt [INFO]: --- remaking latex with pdftex mktexfmt: running `pdftex -ini -jobname=latex -progname=latex -translate-file=cp227.tcx *latex.ini' ... This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (INITEX) restricted \write18 enabled. (/usr/share/texlive/texmf-dist/web2c/cp227.tcx) entering extended mode (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/latex.ini (/usr/share/texlive/texmf-dist/tex/generic/tex-ini-files/pdftexconfig.tex) (/usr/share/texlive/texmf-dist/tex/latex/base/latex.ltx (/usr/share/texlive/texmf-dist/tex/latex/base/texsys.cfg) ./texsys.aux found \@currdir set to: ./. Assuming \openin and \input have the same search path. Defining UNIX/DOS style filename parser. catcodes, registers, parameters, LaTeX2e <2020-02-02> patch level 5 hacks, control, par, spacing, files, font encodings, lengths, ==================================== Local config file fonttext.cfg used ==================================== (/usr/share/texlive/texmf-dist/tex/latex/base/fonttext.cfg (/usr/share/texlive/texmf-dist/tex/latex/base/fonttext.ltx === Don't modify this file, use a .cfg file instead === (/usr/share/texlive/texmf-dist/tex/latex/base/omlenc.def) (/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.def) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.def) (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.def) (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def) (/usr/share/texlive/texmf-dist/tex/latex/base/ts1cmr.fd) (/usr/share/texlive/texmf-dist/tex/latex/base/t1cmr.fd) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1cmr.fd) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1cmss.fd) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1cmtt.fd))) ==================================== Local config file fontmath.cfg used ==================================== (/usr/share/texlive/texmf-dist/tex/latex/base/fontmath.cfg (/usr/share/texlive/texmf-dist/tex/latex/base/fontmath.ltx === Don't modify this file, use a .cfg file instead === (/usr/share/texlive/texmf-dist/tex/latex/base/omlcmm.fd) (/usr/share/texlive/texmf-dist/tex/latex/base/omscmsy.fd) (/usr/share/texlive/texmf-dist/tex/latex/base/omxcmex.fd) (/usr/share/texlive/texmf-dist/tex/latex/base/ucmr.fd))) ==================================== Local config file preload.cfg used ===================================== (/usr/share/texlive/texmf-dist/tex/latex/base/preload.cfg (/usr/share/texlive/texmf-dist/tex/latex/base/preload.ltx)) page nos., x-ref, environments, center, verbatim, math definitions, boxes, title, sectioning, contents, floats, footnotes, index, bibliography, output, =========================================== Local configuration file hyphen.cfg used =========================================== (/usr/share/texlive/texmf-dist/tex/generic/babel/hyphen.cfg (/usr/share/texlive/texmf-dist/tex/generic/babel/switch.def) (/usr/share/texlive/texmf-dist/tex/generic/hyphen/hyphen.tex) (/usr/share/texlive/texmf-dist/tex/generic/hyphen/dumyhyph.tex) (/usr/share/texlive/texmf-dist/tex/generic/hyphen/zerohyph.tex)) (/usr/share/texlive/texmf-dist/tex/latex/base/utf8.def (/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.dfu)) (/usr/share/texlive/texmf-dist/tex/latex/base/ltexpl.ltx (/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3.ltx (/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3-code.tex (/usr/share/texlive/texmf-dist/tex/latex/l3kernel/l3deprecation.def)))) ) ) Beginning to dump on file latex.fmt (preloaded format=latex 2020.11.16) 17444 strings of total length 297830 233795 memory locations dumped; current usage is 165&226828 15747 multiletter control sequences \font\nullfont=nullfont \font\OMX/cmex/m/n/10=cmex10 \font\tenln=line10 \font\tenlnw=linew10 \font\tencirc=lcircle10 \font\tencircw=lcirclew10 \font\OT1/cmr/m/n/5=cmr5 \font\OT1/cmr/m/n/7=cmr7 \font\OT1/cmr/m/n/10=cmr10 \font\OML/cmm/m/it/5=cmmi5 \font\OML/cmm/m/it/7=cmmi7 \font\OML/cmm/m/it/10=cmmi10 \font\OMS/cmsy/m/n/5=cmsy5 \font\OMS/cmsy/m/n/7=cmsy7 \font\OMS/cmsy/m/n/10=cmsy10 \font\c__fp_exp_intarray=cmr10 at 0.00002pt \font\c__fp_trig_intarray=cmr10 at 0.00003pt \font\g__regex_charcode_intarray=cmr10 at 0.00005pt \font\g__regex_catcode_intarray=cmr10 at 0.00006pt \font\g__regex_balance_intarray=cmr10 at 0.00008pt \font\g__regex_state_active_intarray=cmr10 at 0.00009pt \font\g__regex_thread_state_intarray=cmr10 at 0.0001pt \font\g__regex_submatch_prev_intarray=cmr10 at 0.00012pt \font\g__regex_submatch_begin_intarray=cmr10 at 0.00014pt \font\g__regex_submatch_end_intarray=cmr10 at 0.00015pt 532331 words of font info for 24 preloaded fonts 14 hyphenation exceptions Hyphenation trie of length 6081 has 183 ops out of 35111 2 for language 1 181 for language 0 0 words of pdfTeX memory 0 indirect objects No pages of output. Transcript written on latex.log. mktexfmt [INFO]: log file copied to: /root/.texlive2020/texmf-var/web2c/pdftex/latex.log mktexfmt [INFO]: /root/.texlive2020/texmf-var/web2c/pdftex/latex.fmt installed. mktexfmt [INFO]: successfully rebuilt formats: 1 mktexfmt [INFO]: not selected formats: 20 mktexfmt [INFO]: total formats: 21 mktexfmt [INFO]: exiting with status 0 entering extended mode (./test.tex LaTeX2e <2020-02-02> patch level 5 L3 programming layer <2020-04-06> (/usr/share/texlive/texmf-dist/tex/latex/base/article.cls Document Class: article 2019/12/20 v1.4l Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo)) (/usr/share/texlive/texmf-dist/tex/latex/l3backend/l3backend-dvips.def) No file test.aux. [1] (./test.aux) ) Output written on test.dvi (1 page, 244 bytes). Transcript written on test.log.
Strange, the issue persists for me. I was also able to reproduce it using Docker in Github Actions, using input: https://github.com/junghans/rawhide-latex/blob/main/.github/workflows/latex.yml Full log here: https://github.com/junghans/rawhide-latex/runs/1408882973?check_suite_focus=true
Okay, that's really odd. Your package transaction log is _identical_ to what I get locally, but the result is clearly different. It is not finding /usr/share/texlive/texmf-dist/web2c/fmtutil.cnf (which is owned by texlive-kpathsea). My podman run shows this key difference: kpathsea: Running mktexfmt latex.fmt mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order): mktexfmt: /usr/share/texlive/texmf-dist/web2c/fmtutil.cnf mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes: mktexfmt: /root/.texlive2020/texmf-config/web2c/fmtutil.cnf Can you amend your Dockerfile to this and rerun? FROM registry.fedoraproject.org/fedora:rawhide RUN dnf -y update RUN dnf -y install texlive RUN cat /usr/share/texlive/texmf-dist/web2c/fmtutil.cnf RUN /usr/bin/fmtutil-sys --all || : RUN printf '\\documentclass{article}\n\\begin{document}\ntest\n\\end{document}' > test.tex RUN latex test.tex Hopefully those logs will show more about where the issue is. The fmtutil-sys call is already part of the transactional file trigger for texlive-kpathsea, so it should be redundant, BUT all the executions in the triggers have their output silenced, so if that call fails in a unique way, then we have a better lead. There is an issue where fmtutil-sys tries to generate the optex bits and fails, but that _should not_ be causing this issue. Nevertheless, I'm fixing that in rawhide right now.
Thanks, here is the log with the two extra commands: https://github.com/junghans/rawhide-latex/runs/1409411796?check_suite_focus=true
I also made an actual docker build here: https://github.com/junghans/rawhide-latex/runs/1409471864?check_suite_focus=true
Okay, so /usr/share/texlive/texmf-dist/web2c/fmtutil.cnf exists. Please try: FROM registry.fedoraproject.org/fedora:rawhide RUN dnf -y update RUN dnf -y install texlive RUN /usr/bin/kpsewhich -debug 2 fmtutil.cnf RUN /usr/bin/fmtutil-sys --all || : RUN printf '\\documentclass{article}\n\\begin{document}\ntest\n\\end{document}'> test.tex RUN latex test.tex
Here you go: https://github.com/junghans/rawhide-latex/runs/1413647912?check_suite_focus=true FYI, I made it so you can just edit the Dockerfile on GitHub using: https://github.com/junghans/rawhide-latex/edit/main/.github/workflows/Dockerfile This will trigger a PR, which will then run the docker build.
Step 4/8 : RUN /usr/bin/kpsewhich -debug 2 fmtutil.cnf ---> Running in 59a3ddebd602 The command '/bin/sh -c /usr/bin/kpsewhich -debug 2 fmtutil.cnf' returned a non-zero code: 1 That is both bad and entirely unhelpful. :/ Not your fault, obviously.
Okay. A few rounds of debugging and this is what I've learned: In the %transfiletriggerin for texlive-kpathsea, it runs: /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr 2> /dev/null || : That command is supposed to go across the texlive dirs in /usr and generate the "ls-R" files. These files are what texlive then uses to find all of its other files. While we can't see the output from that trigger (because it is going to /dev/null), we can see that these ls-R files are not being generated: Step 6/12 : RUN ls -l /usr/share/texlive/texmf-dist/ls-R 3379 ---> Running in 7df79b40dda0 3380 ls: cannot access '/usr/share/texlive/texmf-dist/ls-R': No such file or directory When I try to explicitly run that command in the Dockerfile... it fails: Step 5/12 : RUN /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr --verbose 3370 ---> Running in 4ea843e52813 3371 mktexlsr: /usr/share/texlive/texmf-config: directory not writable. Skipping... 3372 mktexlsr: /usr/share/texlive/texmf-dist: directory not writable. Skipping... 3373 mktexlsr: /usr/share/texlive/texmf-local//: directory not writable. Skipping... 3374 mktexlsr: /usr/share/texlive/texmf-var: directory not writable. Skipping... 3375 mktexlsr: Done. 3376 Best guess? Docker has /usr mounted read-only or otherwise restricts writes outside of the dnf transaction. I have no idea _why_ this is happening, but the directories are not writable, and this seems to be the heart of the issue. Without those ls-R files, nothing else in texlive will find what it needs to work right. I also don't know why this doesn't happen with podman, or why this works in Fedora 33/32.
Maybe... something isn't working right with triggers in rawhide? I'm more inclined to point at docker at this point though.
Docker is not supported in Rawhide, it is not even available. This bug is a texlive bug not a package Fedora does not ship
It might be worth trying to use podman in the github action, AFIAK podman is available on ubuntu. I gonna try to test this on Fedora using docker, to see if this is related.
(In reply to Daniel Walsh from comment #13) > Docker is not supported in Rawhide, it is not even available. > This bug is a texlive bug not a package Fedora does not ship I only switched to docker as https://hub.docker.com/_/fedora says so, but I made an issue on github now. (In reply to Clement Verna from comment #14) > It might be worth trying to use podman in the github action, AFIAK podman is > available on ubuntu. > > I gonna try to test this on Fedora using docker, to see if this is related. That might be ok in some cases, but for the jobs.<job_id>.container option one cannot not really opt to run podman instead of docker.
(In reply to Clement Verna from comment #14) > I gonna try to test this on Fedora using docker, to see if this is related. Any update on this?
(In reply to Christoph Junghans from comment #16) > (In reply to Clement Verna from comment #14) > > I gonna try to test this on Fedora using docker, to see if this is related. > > Any update on this? Yeah the build fails with ``` [core@localhost ~]$ sudo docker build -t test . Sending build context to Docker daemon 7.168kB Step 1/8 : FROM registry.fedoraproject.org/fedora:rawhide ---> 771dcbc7e672 Step 2/8 : RUN dnf -y update ---> Using cache ---> 9192c39df926 Step 3/8 : RUN dnf -y install texlive ---> Using cache ---> 8feec3f65e5d Step 4/8 : RUN /usr/bin/kpsewhich -debug 2 fmtutil.cnf ---> Running in a1c33e1a5d3a The command '/bin/sh -c /usr/bin/kpsewhich -debug 2 fmtutil.cnf' returned a non-zero code: 1 ``` looking at /usr/share/texlive with Docker ``` [root@d3115e9cc241 /]# ls -l /usr/share/texlive/ total 15964 -rw-r--r--. 1 root root 2098 Nov 17 15:26 LICENSE.CTAN -rw-r--r--. 1 root root 5267 Nov 17 15:26 LICENSE.TL -rw-r--r--. 1 root root 182 Nov 17 15:26 README -rw-r--r--. 1 root root 250 Nov 17 15:26 README.usergroups drwxr-xr-x. 2 root root 162 Nov 21 10:08 fmtutil.cnf.d -rw-r--r--. 1 root root 1783 Nov 17 15:26 index.html drwxr-xr-x. 2 root root 4096 Nov 21 10:07 licenses drwxr-xr-x. 2 root root 4096 Nov 21 10:07 readme-html.dir -rw-r--r--. 1 root root 350 Nov 17 15:26 release-texlive.txt -rw-r--r--. 1 root root 16309069 Nov 17 15:13 texlive.tlpdb drwxr-xr-x. 3 root root 19 Nov 21 10:07 texmf-config drwxr-xr-x. 15 root root 182 Nov 21 10:08 texmf-dist drwxr-xr-x. 3 root root 26 Nov 21 10:07 texmf-local lrwxrwxrwx. 1 root root 14 Nov 17 15:13 texmf-var -> /var/lib/texmf drwxr-xr-x. 6 root root 97 Nov 21 10:08 tlpkg ``` and with podman ``` [root@05870bef34a5 /]# ls -l /usr/share/texlive/ total 15964 -rw-r--r--. 1 root root 2098 Nov 17 15:26 LICENSE.CTAN -rw-r--r--. 1 root root 5267 Nov 17 15:26 LICENSE.TL -rw-r--r--. 1 root root 182 Nov 17 15:26 README -rw-r--r--. 1 root root 250 Nov 17 15:26 README.usergroups drwxr-xr-x. 2 root root 162 Nov 21 10:31 fmtutil.cnf.d -rw-r--r--. 1 root root 1783 Nov 17 15:26 index.html drwxr-xr-x. 2 root root 4096 Nov 21 10:30 licenses drwxr-xr-x. 2 root root 4096 Nov 21 10:30 readme-html.dir -rw-r--r--. 1 root root 350 Nov 17 15:26 release-texlive.txt -rw-r--r--. 1 root root 16309069 Nov 17 15:13 texlive.tlpdb drwxr-xr-x. 3 root root 35 Nov 21 10:39 texmf-config drwxr-xr-x. 15 root root 35 Nov 21 10:39 texmf-dist drwxr-xr-x. 3 root root 35 Nov 21 10:39 texmf-local lrwxrwxrwx. 1 root root 14 Nov 17 15:13 texmf-var -> /var/lib/texmf drwxr-xr-x. 6 root root 97 Nov 21 10:31 tlpkg ```
Dug up a little bit more, /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr is using the test command to check if the a file/directory is writtable using Docker and using bash ``` [root@1bf79b706073 /]# test -w /usr/share/texlive/texmf-config/ [root@1bf79b706073 /]# echo $? 0 File is writable ``` using docker and using sh (mktexlsr is using #!/usr/bin/sh) ``` sh-5.0# test -w /usr/share/texlive/texmf-config/ sh-5.0# echo $? 1 File is not writable sh-5.0# ls -l /usr/share/texlive/ total 15964 -rw-r--r--. 1 root root 2098 Nov 17 15:26 LICENSE.CTAN -rw-r--r--. 1 root root 5267 Nov 17 15:26 LICENSE.TL -rw-r--r--. 1 root root 182 Nov 17 15:26 README -rw-r--r--. 1 root root 250 Nov 17 15:26 README.usergroups drwxr-xr-x. 2 root root 162 Nov 21 11:42 fmtutil.cnf.d -rw-r--r--. 1 root root 1783 Nov 17 15:26 index.html drwxr-xr-x. 2 root root 4096 Nov 21 11:41 licenses drwxr-xr-x. 2 root root 4096 Nov 21 11:41 readme-html.dir -rw-r--r--. 1 root root 350 Nov 17 15:26 release-texlive.txt -rw-r--r--. 1 root root 16309069 Nov 17 15:13 texlive.tlpdb drwxr-xr-x. 3 root root 31 Nov 21 11:55 texmf-config drwxr-xr-x. 15 root root 194 Nov 21 11:55 texmf-dist drwxr-xr-x. 3 root root 38 Nov 21 11:55 texmf-local lrwxrwxrwx. 1 root root 14 Nov 17 15:13 texmf-var -> /var/lib/texmf drwxr-xr-x. 6 root root 97 Nov 21 11:42 tlpkg ``` Now doing the same with podman and bash ``` [root@d10a5ef8afe5 /]# test -w /usr/share/texlive/texmf-config/ [root@d10a5ef8afe5 /]# echo $? 0 [root@d10a5ef8afe5 /]# ``` and podman and sh ``` sh-5.0# test -w /usr/share/texlive/texmf-config/ sh-5.0# echo $? 0 sh-5.0# ``` So something is making sh not work correctly when using docker.
Very strange, but why is it working in the F33 container then? It seems the version of bash (that provides sh) and coreutils (that provides test) are the same on both, F33 and Rawhide.
https://bugzilla.redhat.com/show_bug.cgi?id=1900021 seems to be related.
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle. Changing version to 34.
I believe this has been fixed in Docker upstream, closing.