How to tell Arara to skip redundant steps?
Here's my proposal:
% arara: lualatex: { draft: yes, shell: yes }
% arara: biber if changed (toFile('TestBibliography.bib'))
% arara: --> || found ('log', 'Please \\(re\\)run Biber')
% arara: makeglossaries if changed ('glo') || missing ('gls')
% arara: makeindex if changed ('idx') || missing ('ind')
% arara: lualatex until !found('log', '\\(?(R|r)e\\)?run (to get|LaTeX)')
Let's go one by one.
% arara: lualatex: { draft: yes, shell: yes }
This one will run LuaLaTeX in draft mode with -shell-escape enabled. The draft mode makes LuaLaTeX not generate the .pdf file, thus it won't include graphics. This run of LuaLaTeX just generates auxiliary files which are needed for the other programs. This saves a couple of seconds in compilation time. The -shell-escape is optional; I needed it, but if you aren't using any package that requires it you can remove that (I was using for externalisation of TikZ pictures).
% arara: biber if changed (toFile('TestBibliography.bib'))
% arara: --> || found ('log', 'Please \\(re\\)run Biber')
This directive will run Biber only if the .log file says so. BibLaTeX tells you when you need to (re)run Biber, so you can rely on that to know when. What BibLaTeX can't tell is when you change the .bib file, so I added an alternative condition, which is if I changed my .bib file (since the name isn't the same as the master .tex file, I needed to make it into a file reference with toFile('TestBibliography.bib')).
A couple of observations here: The % arara: --> line is a continuation line. It means that what follows the --> belongs to the previous line. It's used just for clarity of the code. It's documented in page 20 of the manual. Thus, the directive above could be written:
% arara: biber if changed (toFile('TestBibliography.bib')) || found ('log', 'Please \\(re\\)run Biber')
without change in meaning.
The syntax for found (an other commands like that) is:
found(<string extension or file reference>,<regular expression>)
The <string extension> in this case is 'log', and the <regular expression> is 'Please \\(re\\)run Biber'. In regular expressions, a pair of parentheses form a group, so (re)run does not match the literal string (re)run in the .log file, so you have to escape the parentheses with backslashes (\(re\)) so that they mean literal parentheses. However in Java (in which language Arara is written) a \\ translates to a single \1, so you need to escape the backslash as well. So, in a directive \\(re\\)run will be read by Arara as \(re\)run (first escaping level), and then will be passed to the regular expression engine which will understand \(re\)run as the literal string (re)run in the log file. Phew :)
[1]: The literal string "\\" is a single backslash. In regular expressions, the backslash is also an escape character. The regular expression "\\" matches a single backslash. This regular expression as a Java string, becomes "\\\\".
% arara: makeglossaries if changed ('glo') || missing ('gls')
% arara: makeindex if changed ('idx') || missing ('ind')
This directive will run makeglossaries and makeindex only if the input files .glo and .idx, respectively, changed (which already evaluates to true if the file didn't exist before, which is the case of the first run), or if the output file for these tools, .gls and .ind, did't exist before, which covers the situations where you'd need to run these tools.
% arara: lualatex until !found('log', '\\(?(R|r)e\\)?run (to get|LaTeX)')
Finally, this directive will run LuaLaTeX as many times as necessary until there are no messages in the .log file saying either of these strings:
Rerun LaTeX
rerun LaTeX
(Re)run LaTeX
(re)run LaTeX
Rerun to get
rerun to get
(Re)run to get
(re)run to get
which cover most of the messages I could find in my .log file. If you add another package which requires multiple runs and uses a different message you'd need to adapt it. For instance, if the package said (in the .log) “Execute LaTeX again” (which does not match any of the patterns above), then you could change that directive to:
% arara: lualatex until !found('log', '\\(?(R|r)e\\)?run (to get|LaTeX)')
% arara: --> && !found('log','Execute LaTeX again')
or something like that. It's virtually impossible to consider all the cases form all LaTeX packages, so you need to tailor the directive according to your document, but the rerun LaTeX one is pretty generic.
Remarks on this one: Again, '\\(?(R|r)e\\)?run (to get|LaTeX)' is a regular expression (if you know these, you can skip this part). Remember, Arara eats one level of escaping, so the above translates to '\(?(R|r)e\)?run (to get|LaTeX)'. This regular expression matches:
\(? | A (optional) literal '('
(R|r) | One of 'R' or 'r'
e | The letter 'e'
\)? | A (optional) literal ')'
run | The string 'run ' (note the trailing space)
(to get|LaTeX) | Either 'to get' or 'LaTeX'
which translates to the cases listed above.
With your sample document, the first run of arara test.tex yields:
phelype@phelype ~/testing> arara test.tex
__ _ _ __ __ _ _ __ __ _
/ _` | '__/ _` | '__/ _` |
| (_| | | | (_| | | | (_| |
\__,_|_| \__,_|_| \__,_|
Processing 'test.tex' (size: 30 KB, last modified: 05/07/2019
12:05:44), please wait.
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
(Biber) The Biber reference management software ......... SUCCESS
(MakeGlossaries) The MakeGlossaries software ............ SUCCESS
(MakeIndex) The MakeIndex software ...................... SUCCESS
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
Total: 6.75 seconds
and further runs (with no modification of the sources) yield:
phelype@phelype ~/testing> arara test.tex
__ _ _ __ __ _ _ __ __ _
/ _` | '__/ _` | '__/ _` |
| (_| | | | (_| | | | (_| |
\__,_|_| \__,_|_| \__,_|
Processing 'test.tex' (size: 30 KB, last modified: 05/07/2019
12:05:44), please wait.
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
Total: 2.69 seconds
From the second run on, you get the first run of LuaLaTeX in draft mode to read the input .tex file and generate the files for Biber, Makeglossaries, and Makeindex (which are the same, so these tools aren't run), and a second run to produce the .pdf file.
In arara rules can be executed conditionally using checks on the various files. These checks are described in the current manual (version 4.0) Section 6.1 - Files.
Some useful methods are missing and changed, which check if files are missing or changed (unsurprisingly). The methods can be used in two ways: by providing a file extension, in which case the name of the main file is prepended to the extension, or by providing a full filename using the syntax toFile('yourfile.ext'). Another useful method is found, which does a string search in the provided file and returns true if the string is found. This can be used to search the main log file, or any other file if necessary, for messages indicating that a rerun is needed.
For the document in the question the following rules can be used for example:
% arara: lualatex
% arara: biber if missing('bbl') || found('log', 'Citation')
% arara: makeglossaries if missing('gls') || changed('glo') || changed(toFile('TestGlossary.tex'))
% arara: makeindex if changed('idx')
% arara: lualatex if found('log', 'No file ') || found('log', 'undefined references') || found('log', 'Rerun required') || found('log', 'Rerun to get cross-references')
% arara: lualatex
The biber rule now means run Biber if there is no bbl file (which is the file that contains the compiled bibliography) or if there are messages about citations in the log (indicating new references that are not yet included in the compiled bibliography). Note that the first syntax variant is used here, so for Test.tex the command missing('bbl') is expanded to check if Test.bbl is missing and similary log is expanded to Test.log.
The makeglossaries rule checks if the gls file exists (the output of the makeglossaries command or if the glo file is changed (written by lualatex when a glsadd command is encountered) or if the input file with the glossaries is changed. This last check is actually not really useful because the file can change without new glossary items being added to the main text and new entries can be added without changing the file, but it serves as a demonstration of the toFile() construct.
The makeindex rule checks for changes in the idx file that contains index entries.
The second lualatex rule checks for various phrases in the log file that indicate that a rerun is needed. Note that string matching on the log file is an indirect method for checking the need for reruns which may result in necessary reruns being missed or in redundant reruns being performed. The packages and tools may write different information in the log in different situations, or after updates etc., and the strings may be present in the log file for completely unrelated reasons, for example in \typeout commands. Arara itself does not seem to provide functionality for robust rerun checking - there are some examples in the manual but these all use log matching.
The rules above result in the following three runs:
run 1
__ _ _ __ __ _ _ __ __ _
/ _` | '__/ _` | '__/ _` |
| (_| | | | (_| | | | (_| |
\__,_|_| \__,_|_| \__,_|
Processing 'archange.tex' (size: 1 KB, last modified: 05/07/2019
13:29:24), please wait.
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
(Biber) The Biber reference management software ......... SUCCESS
(MakeGlossaries) The MakeGlossaries software ............ SUCCESS
(MakeIndex) The MakeIndex software ...................... SUCCESS
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
Total: 4.53 seconds
run 2
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
(MakeGlossaries) The MakeGlossaries software ............ SUCCESS
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
Total: 2.73 seconds
run 3
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
(LuaLaTeX) LuaLaTeX engine .............................. SUCCESS
Total: 2.62 seconds