Is it possible to use GPU acceleration on compiling large LaTeX documents?
tikzexternalize in parallel
I thought I might as well expand my answer a bit regarding the parallelization of externalizing tikzpicture
s. I recommend reading the section of pgfman for more details (p.607 onwards).
Disclaimer: I'm a Linux guy; haven't tried this on other platforms. But this question suggests that it can be done, apparently.
Anyway: Let's make a very simple example document with quite a few tikzpicture
s in it. Note the mode=list and make
in the call to tikzexternalize
. That's the important bit.
\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{external}
\tikzexternalize[prefix=tikz-,mode=list and make]
\begin{document}
\begin{tikzpicture}
\draw (0,0) -- (1,1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (1,0) -- (1,1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (0,2) -- (1,1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (0,3) -- (1,1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (0,0) -- (2,1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (0,0) -- (5,1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (0,0) -- (2,-1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (0,0) -- (1,6);
\end{tikzpicture}
\begin{tikzpicture}
\draw (0,1) -- (7,1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (0,-1) -- (5,1);
\end{tikzpicture}
\begin{tikzpicture}
\draw (-1,0) -- (1,0);
\end{tikzpicture}
\end{document}
Run
pdflatex myfile
or your engine of preference.
Then you will have a new file in your directory, myfile.makefile
. Execute that with
make -j<PARALLEL_JOB_COUNT> -f myfile.makefile
And that will build your tikzpicture
s in parallel batches of <PARALLEL_JOB_COUNT>
For four, this would be
make -j4 -f myfile.makefile
Use your number of cores as a guideline for how many parallel jobs to run.
If all has gone well, you can then compile your document again and include all the tikzpicture
s:
pdflatex myfile
And all should be well.
As I mentioned in the comment to TeXnician's answer, I was able to cut down compilation time for my thesis (lots of pgfplot
s) from about 90 minutes to roughly 13 on a clean directory (well, 16 minutes, actually, counting the two 90-second runs before and after generating the tikzpicture
s, but still -- pretty impressive gain, methinks). Of course, more cores are better here; I have 12 on that machine.
Aside from compilation time, another thing I found hugely beneficial to this process came about in the workflow though: You can easily force a rebuild of individual tikzpictures from the commandline. If you have your individual tikzpicture
s in separate files, this means you basically get a very similar workflow as if you'd used standalone
for your tikzpicture
s. So, for example, if we want to build tikz-main-figure5.pdf
in this case, we can issue:
make -B -f main.makefile tikz-main-figure5.pdf
The -B
option forces make
to rebuild the target -- usually needed in these cases if you change your tikzpicture
without then compiling your main document again, because then the m5dsum
will not get updated and make
will think nothing has changed. You can of course also just delete the .pdf
file and recompile without the -B
switch, but this is how I usually did it during those long hours late at night when I was fiddling with my plots and trying to get them to look right. I had the TikZ pdf open in one window, and the editor with the source code in another, and a shortcut for the recompilation, and that was really quite convenient to work with.
GPU note
Leaving aside the things which have already been mentioned about the compilation of a TeX document being a heavily serialized job in many ways, you would probably need to write your own TeX implementation to run on your GPU, or a wrapper which can run the x86 TeX on your GPU. Neither of these are what I'd call a trivial process, and given that in many scenarios the benefits would probably be marginal (or negative) at best, I doubt it's worth it.
Update: Benchmarks and Background
Since a compile time of 90 minutes seems as ludicrous to me as it probably does to most here, I've dug up the project again and have been fiddling around with it a bit. The current numbers are as follows:
- Building all
tikzpicture
s from scratch (most of which arepgfplot
s), sequentially: 57 minutes - Building the
tikzpicture
s with 12 jobs in parallel: 7:30 minutes
The whole shebang consists of
- 61 output pdf files for the
tikzpicture
s - 256,000 lines of CSV data, 130,000 of which is 3 columns, the rest mostly 2 columns
- 237
\addplot
commands, the data for which is usually read in via\pgfplotstableread
commands
At the moment, I see two primary areas for optimization. Since they are also generally valid for speeding up the compilation of tikzpicture
s, I hope this is still sufficiently on-topic to mention:
- Reducing the number of data points, and
- removing any
filter/.code
commands, moving their work to the tool which generates the data in the first place.
So:
- Yes, there is potential for optimization here.
- I'm fine with that. My point was that parallel compilation of
tikzpicture
s can bring significant reductions in overall compile time and is relatively easy to accomplish if you're not commandline-averse. - Also, it's not as bad as it sounds, since I rarely have to compile all
tikzpicture
s from scratch -- usually I just update one thing, and then the whole document compiles in about a 40 seconds (could be further reduced with\includeonly
, but I usually don't bother). Or if I'm working on a specifictikzpicture
, I only ever need to rebuild that, which is also quite acceptable (see the workflow notes above).
If anyone wants to have a go at it themselves (or be appalled at some LaTeX heresy I probably committed), the document can be found here (full pdf versions available in the snapshots directory). It's a bit of a convoluted setup because I also use a separate build
directory and getting that to work properly with tikzexternalize
is a bit of a story in and of itself. But if anyone wants to have a look at an actual use-case for a non-tiny project with tikzexternalize
, it might be of interest.
This question can be understood in two ways:
- You want to compile your LaTeX document using CPU and GPU: This is effectively the same question as if you would like to use another core (i.e. multi-core setup) for compiling the document, that's not possible.
- You want to use the GPU exclusively: I doubt that this would be highly effective as usually GPUs have some optimized operations, mostly concerning graphics, but are not that much faster for "ordinary" tasks (they may even be slower for some operations). Apart from that it would be a really hard task to use the GPU instead of the CPU as you would need to translate TeX's CPU instructions into GPU instructions beforehand and have your OS (and probably also graphics driver) allow these operations.
What you can do in your situation:
- If you have many TikZ figures you can use externalization which makes compiling faster (except for the first time). Just including PDFs is very fast.
- You could also create a format which would "pre-compile" the preamble in some way and therefore speeds things up (except when creating the format, thanks to Skillmon for pointing this out).