Matplotlib tex-macro in description

Question

I'm using matplotlib to generate pgf files. Based on those, I use standalone tex files which only include necessary settings and the afore-built pgfs. In this scenario, I'm getting errors when using custom tex-macros for descriptions in my plot files.

Here an example pgf generator:

import matplotlib as mpl
mpl.use("pgf")
mpl.rcParams.update({
    "pgf.texsystem": "pdflatex",
    "pgf.preamble": [
        #r"\newcommand{\foo}{foo}",
        r"\usepackage{import}",
        r'\subimport{./}{foo.tex}'
    ]
})

import matplotlib.pyplot as plt
plt.figure(figsize=(4.5,2.5))
plt.plot(range(5))
plt.xlabel(r'\foo{}')
plt.savefig('foo.pgf')

that can be used in a dir with the following foo.tex file:

\newcommand{\foo}{foo}

Running this results in the following error:

ValueError: Error processing '\foo{}'
LaTeX Output:
! Undefined control sequence.
<argument> ....000000}{12.000000}\selectfont \foo
                                                  {}
<*> ...ze{10.000000}{12.000000}\selectfont \foo{}}

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on texput.log.

Please note, that this is an error generated by matplotlib and not of compiling my standalone files. Also note, that the error goes away when the \foo macro is provided as part of the pgf.preamble (the line commented out) instead. I checked the pgf produced by this variant and indeed it uses \foo{}.

I'm having trouble narrowing the problem further down. Here my concrete questions:

1. Why does matplotlib invoke pdflatex at all? I'm generating pgf output and thus pdflatex should not be necessary. (For the reference: I straced the script above and indeed know that pdflatex is being called.)
2. Is there a way of preserving the temporary file that matplotlib tries to compile? The error references texput.log by (of course) that file doesn't exist afterwards.
3. Why can't I use a macro in a label which is provided in another tex file?

Answer 1

tl;dr Including tex-files in the pgf.preamble of matplotlib requires absolute paths.

---

For the future, I recommend the following pdflatex "replacement script" for purposes of debugging:

#!/usr/bin/env bash
MAIN=/usr/bin/pdflatex
cat /dev/stdin | tee /some/abs/path/to/dbg.tex | "${MAIN}" "${@}"

Make sure to save it as pdflatex, make sure it's executable, make sure /usr/bin/pdflatex is your actual pdflatex and ensure this wrapper is found first in yout PATH (cf. which pdflatex). When running the python generator, we preserve the final tex input in the dbg.tex. That answers (2).

Considering the output, we see:

\documentclass{minimal}
\usepackage{import}
\subimport{./}{foo.tex}

\begin{document}
text $math \mu$
\typeout{pgf_backend_query_start}
\sbox0{\sffamily\fontsize{10.000000}{12.000000}\selectfont lp}
\typeout{\the\wd0,\the\ht0,\the\dp0}
\sbox0{\sffamily\fontsize{10.000000}{12.000000}\selectfont 0}
\typeout{\the\wd0,\the\ht0,\the\dp0}
\sbox0{\sffamily\fontsize{10.000000}{12.000000}\selectfont 1}
\typeout{\the\wd0,\the\ht0,\the\dp0}
\sbox0{\sffamily\fontsize{10.000000}{12.000000}\selectfont 2}
\typeout{\the\wd0,\the\ht0,\the\dp0}
\sbox0{\sffamily\fontsize{10.000000}{12.000000}\selectfont 3}
\typeout{\the\wd0,\the\ht0,\the\dp0}
\sbox0{\sffamily\fontsize{10.000000}{12.000000}\selectfont 4}
\typeout{\the\wd0,\the\ht0,\the\dp0}
\sbox0{\sffamily\fontsize{10.000000}{12.000000}\selectfont \foo{}}
\typeout{\the\wd0,\the\ht0,\the\dp0}

I don't know what that should be useful for. But I'm guessing matplotlib is trying to adjust the font-setup for which it tries compiling this "test" document. That (sort of) answers (1).

Now the conclusion (obvious in hindsight): matplotlib compiles this sample document in a temporary directory. Clearly there is no foo.tex available in this directory, so the subimport fails. From that point onwards it is obvious that \foo will not be available.

Though not the cleanest solution, this can be fixed by including foo.tex via an absolute path. Working python generator to finally answer (3):

import matplotlib as mpl
import pathlib

mpl.use("pgf")
mpl.rcParams.update({
    "pgf.texsystem": "pdflatex",
    "pgf.preamble": [
        r"\usepackage{import}",
        f'\subimport{{{pathlib.Path.cwd().resolve()}/}}{{foo.tex}}']
})

import matplotlib.pyplot as plt
plt.figure(figsize=(4.5,2.5))
plt.plot(range(5))
plt.xlabel(r'\foo{}')
plt.savefig('foo.pgf')

(I use python3 and pathlib. For python2 we would rather fall back to os.getcwd.)