Skip to content
Snippets Groups Projects
Commit 46e75fc9 authored by Ben Gamari's avatar Ben Gamari
Browse files

hadrian: Mitigate mktexfmt race

At least some versions of Texlive's `mktexfmt` utility cannot be invoked
concurrently in their initial run since they fail to handle failure of
`mkdir` due to racing. Specifically, we see

```
| Run Xelatex: users_guide.tex => /tmp/extra-dir-9616886274866
| Run Xelatex: Haddock.tex => /tmp/extra-dir-9616886274869
This is XeTeX, Version 3.14159265-2.6-0.999992 (TeX Live 2020) (preloaded format=xelatex)
 restricted \write18 enabled.
kpathsea: Running mktexfmt xelatex.fmt
mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order):
mktexfmt:   /usr/share/texlive/texmf-dist/web2c/fmtutil.cnf
mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes:
mktexfmt:   /builds/ghc/ghc/tmp-home/.texlive2020/texmf-config/web2c/fmtutil.cnf
/usr/bin/mktexfmt: mkdir(/builds/ghc/ghc/tmp-home/.texlive2020/texmf-var/web2c/) failed for tree /builds/ghc/ghc/tmp-home/.texlive2020/texmf-var/web2c: File exists at /usr/share/texlive/tlpkg/TeXLive/TLUtils.pm line 937.
I can't find the format file `xelatex.fmt'!
```

That is two `mktexfmt` invocations (for the user's guide and haddock
builds) attempted to create `$HOME/texlive2020/texmf-var/web2c` and
raced. One of the two `mkdir`'s consequently failed, bringing down the
entire build.

We avoid this by ensuring that the first `xelatex` invocation is always
performed serially.

Fixes #25564.
parent dfd1db48
No related branches found
No related tags found
1 merge request!13703hadrian: Mitigate mktexfmt race
Pipeline #104284 passed
......@@ -326,11 +326,27 @@ getPkgDocTarget root path =
-- | Build all PDF documentation
buildPdfDocumentation :: Rules ()
buildPdfDocumentation = mapM_ buildSphinxPdf docPaths
buildPdfDocumentation = do
-- Note [Avoiding mktexfmt race]
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- We must ensure that the *first* xelatex invocation in the
-- build is performed serially (that is, not concurrently with
-- any other xelatex invocations) as mktexfmt does not handle
-- racing `mkdir` calls gracefully. However, we assume that
-- subsequent invocations are safe to run concurrently since the
-- initial call will have created the requisite directories (namely
-- $HOME/.texlive2020/texmf-var/web2c).
--
-- Fixes #25564.
let maxConcurrentReaders = 1000
xelatexMutex <- newResource "xelatex-mutex" maxConcurrentReaders
let rs = [(xelatexMutex, 1)]
mapM_ (buildSphinxPdf rs) docPaths
-- | Compile a Sphinx ReStructured Text package to LaTeX
buildSphinxPdf :: FilePath -> Rules ()
buildSphinxPdf path = do
buildSphinxPdf :: [(Resource, Int)] -> FilePath -> Rules ()
buildSphinxPdf rs path = do
root <- buildRootRules
root -/- pdfRoot -/- path <.> "pdf" %> \file -> do
......@@ -344,7 +360,8 @@ buildSphinxPdf path = do
checkSphinxWarnings dir
-- LaTeX "fixed point"
build $ target docContext Xelatex [path <.> "tex"] [dir]
-- See Note [Avoiding mktexfmt race] above.
buildWithResources rs $ target docContext Xelatex [path <.> "tex"] [dir]
build $ target docContext Xelatex [path <.> "tex"] [dir]
build $ target docContext Xelatex [path <.> "tex"] [dir]
build $ target docContext Makeindex [path <.> "idx"] [dir]
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment