LaTeX3-sensitive editors
Update (2017-02-14)
Thanks to the time spent on the answer below, and to some new features in WinEdt 10.2, now WinEdt 10.2 is really "LaTeX3-sensitive".
WinEdt 10.2 incorporates the highlighting scheme in this answer (with many improvements) and features many other functionalities to support LaTeX3 (e.g. command completion for commands and environments defined with \NewDocumentCommand, \NewDocumentEnvironment and alike).
Complete Highlighting Scheme for WinEdt
This is a highlighting scheme for expl3 in WinEdt 8.
New: It is now available as an add-on: LaTeX3
Highlighting LaTeX3 guards in .dtx files
Add the following lines in Switches.ini just before the switch SWITCH="DTX Single Guard"
SWITCH="DTX LaTeX3 Guard"
ENABLED=1
MODE_FILTER="DTX"
START="DTX %"
STOP=">"
HIGHLIGHT_START=1
STEP_OVER_STOP=1
HIGHLIGHT_STOP=1
SCOPE=1
DOMINANT_PRIORITY=0
STRICT_PRIORITY=1
PRIORITY=9
INDENTED=0
BOLN_ONLY=1
CASE_SENSITIVE=1
START1_TRIGGER=""
START2_TRIGGER="<@@="
STOP1_TRIGGER=""
STOP2_TRIGGER=""
TEXT_COLOR=5
BACKGROUND_RGB="$F8F8F8"
TRANSPARENT=144
DEFAULT_FONT=1
Result:
Collecting LaTeX3 guards in the "Tree"
Add the following lines in Tree.ini inside the branch BRANCH="Guards-DTX", just between the lines END="</?>" and ITEM="<?>"
ITEM="<@@=?>"
MODE_FILTER="DTX"
CASE_SENSITIVE=1
LINE_START="%"
BEGINNING_OF_LINE_ONLY=2
CURRENT_DOCUMENT_ONLY=1
ALL_OPENED_DOCUMENTS=0
COMPLETE_PROJECT_TREE=1
ICON="ArrowPurple"
LEVEL=0
CAPTION="%?"
MAX_LINE_SPAN=1
ON_CTRL_CLICK_MACRO="GlobalMark;TreeTrack(2);"
ON_CLICK_MACRO="TreeTrack(2,1);"
ON_DBL_CLICK_MACRO="TreeTrack(2,2);"
ACTION="Find"
IMAGE="Find"
MACRO="TreeTrack(2);"
END="<@@=?>"
Result:
Highlighting LaTeX3 commands
Add the following lines in FilterSets.ini just before the set FILTER_SET="~AlphaNumeric"
FILTER_SET="~Alpha@_:"
ENABLED=0
MODE_FILTER=""
SET=:~(Alpha+["@_:"])
BEFORE=""
AFTER=""
BOLN_NOT_OK=0
EOLN_NOT_OK=0
STRICT_PRIORITY=0
PRIORITY=0
DEFAULT_FONT=1
Then, in Switches.ini, in the switch SWITCH="\", replace the line
STOP="~Alpha@"
with
STOP="~Alpha@_:"
Finally, in Keywords.ini, replace all occurrences of
AFTER="~Alpha@"
with
AFTER="~Alpha@_:"
Result:
Highlighting LaTeX3 Kernel keywords
Add the following lines in Switches.ini just before the switch SWITCH="\"
SWITCH="LaTeX3 Kernel Engine Prefixes"
ENABLED=1
MODE_FILTER="TeX"
START="\"
STOP="~Alpha@_:"
HIGHLIGHT_START=1
STEP_OVER_STOP=0
HIGHLIGHT_STOP=0
SCOPE=0
DOMINANT_PRIORITY=0
STRICT_PRIORITY=0
PRIORITY=5
INDENTED=0
BOLN_ONLY=0
CASE_SENSITIVE=1
START1_TRIGGER=""
START2_TRIGGER="luatex_if_engine"
START2_TRIGGER="pdftex_if_engine"
START2_TRIGGER="xetex_if_engine"
STOP1_TRIGGER=""
STOP2_TRIGGER=""
TEXT_COLOR=5
DEFAULT_FONT=1
BOLD=1
SWITCH="LaTeX3 Kernel Prefixes"
ENABLED=1
MODE_FILTER="TeX"
START="\"
STOP="~Alpha@_:"
HIGHLIGHT_START=1
STEP_OVER_STOP=0
HIGHLIGHT_STOP=0
SCOPE=0
DOMINANT_PRIORITY=0
STRICT_PRIORITY=0
PRIORITY=5
INDENTED=0
BOLN_ONLY=0
CASE_SENSITIVE=1
START1_TRIGGER=""
START2_TRIGGER="alloc_"
START2_TRIGGER="bool_"
START2_TRIGGER="box_"
START2_TRIGGER="cctab_"
START2_TRIGGER="char_"
START2_TRIGGER="chk_"
START2_TRIGGER="clist_"
START2_TRIGGER="codedoc_"
START2_TRIGGER="coffin_"
START2_TRIGGER="color_"
START2_TRIGGER="cs_"
START2_TRIGGER="dim_"
START2_TRIGGER="driver_"
START2_TRIGGER="else_"
START2_TRIGGER="etex_"
START2_TRIGGER="exp_"
START2_TRIGGER="expl_"
START2_TRIGGER="fi_"
START2_TRIGGER="fp_"
START2_TRIGGER="group_"
START2_TRIGGER="hbox_"
START2_TRIGGER="hcoffin_"
START2_TRIGGER="if_"
START2_TRIGGER="insert_"
START2_TRIGGER="int_"
START2_TRIGGER="kernel_"
START2_TRIGGER="keys_"
START2_TRIGGER="keyval_"
START2_TRIGGER="lua_"
START2_TRIGGER="luatex_"
START2_TRIGGER="mode_"
START2_TRIGGER="muskip_"
START2_TRIGGER="or_"
START2_TRIGGER="pdftex_"
START2_TRIGGER="peek_"
START2_TRIGGER="prop_"
START2_TRIGGER="quark_"
START2_TRIGGER="reverse_"
START2_TRIGGER="seq_"
START2_TRIGGER="skip_"
START2_TRIGGER="str_"
START2_TRIGGER="tex_"
START2_TRIGGER="tl_"
START2_TRIGGER="token_"
START2_TRIGGER="use_"
START2_TRIGGER="vbox_"
START2_TRIGGER="vcoffin_"
START2_TRIGGER="xetex_"
STOP1_TRIGGER=""
STOP2_TRIGGER=""
TEXT_COLOR=4
DEFAULT_FONT=1
BOLD=1
SWITCH="LaTeX3 File Handling"
ENABLED=1
MODE_FILTER="TeX"
START="\"
STOP="~Alpha@_:"
HIGHLIGHT_START=1
STEP_OVER_STOP=0
HIGHLIGHT_STOP=0
SCOPE=0
DOMINANT_PRIORITY=0
STRICT_PRIORITY=0
PRIORITY=5
INDENTED=0
BOLN_ONLY=0
CASE_SENSITIVE=1
START1_TRIGGER=""
START2_TRIGGER="file_"
START2_TRIGGER="g_file_"
START2_TRIGGER="ior_"
START2_TRIGGER="iow_"
START2_TRIGGER="l_ior_"
START2_TRIGGER="l_iow_"
STOP1_TRIGGER=""
STOP2_TRIGGER=""
TEXT_COLOR=2
DEFAULT_FONT=1
BOLD=1
SWITCH="LaTeX3 Package Messages"
ENABLED=1
MODE_FILTER="TeX"
START="\"
STOP="~Alpha@_:"
HIGHLIGHT_START=1
STEP_OVER_STOP=0
HIGHLIGHT_STOP=0
SCOPE=0
DOMINANT_PRIORITY=0
STRICT_PRIORITY=0
PRIORITY=5
INDENTED=0
BOLN_ONLY=0
CASE_SENSITIVE=1
START1_TRIGGER=""
START2_TRIGGER="msg_"
STOP1_TRIGGER=""
STOP2_TRIGGER=""
TEXT_COLOR=9
DEFAULT_FONT=1
ITALIC=1
SWITCH="LaTeX3 NoOp Functions"
ENABLED=1
MODE_FILTER="TeX"
START="\"
STOP="~Alpha@_:"
HIGHLIGHT_START=1
STEP_OVER_STOP=0
HIGHLIGHT_STOP=0
SCOPE=0
DOMINANT_PRIORITY=0
STRICT_PRIORITY=0
PRIORITY=5
INDENTED=0
BOLN_ONLY=0
CASE_SENSITIVE=1
START1_TRIGGER=""
START2_TRIGGER="prg_"
START2_TRIGGER="scan_"
STOP1_TRIGGER=""
STOP2_TRIGGER=""
TEXT_COLOR=3
DEFAULT_FONT=1
BOLD=1
Then, add the following lines in Keywords.ini just before the group KEYWORD_GROUP="TeX Units"
KEYWORD_GROUP="LaTeX3 Keywords"
ENABLED=1
MODE_FILTER="TeX"
BEFORE="\ (single)"
AFTER="~Alpha@_:"
BOLN_NOT_OK=1
EOLN_NOT_OK=0
STRICT_PRIORITY=0
PRIORITY=6
CASE_SENSITIVE=1
TEXT_COLOR=0
DEFAULT_FONT=1
BOLD=1
UNDERLINE=1
LIST="END_LIST"
ExplSyntaxOff
ExplSyntaxOn
END_LIST
KEYWORD_GROUP="LaTeX3 Keywords..."
ENABLED=1
MODE_FILTER="TeX"
BEFORE="\ (single)"
AFTER="~Alpha@_:"
BOLN_NOT_OK=1
EOLN_NOT_OK=0
STRICT_PRIORITY=0
PRIORITY=6
CASE_SENSITIVE=1
TEXT_COLOR=4
DEFAULT_FONT=1
BOLD=1
LIST="END_LIST"
DeclareDocumentCommand
DeclareDocumentEnvironment
NewDocumentCommand
NewDocumentEnvironment
ProvideDocumentCommand
ProvideDocumentEnvironment
RenewDocumentCommand
RenewDocumentEnvironment
END_LIST
Result:
Highlight LaTeX3 properties & constants
This is the best that can be done for this one.
Add the following lines in FilterSets.ini just before the set FILTER_SET="="
FILTER_SET="."
ENABLED=0
MODE_FILTER=""
SET=:["."]
BEFORE=""
AFTER=""
BOLN_NOT_OK=0
EOLN_NOT_OK=0
STRICT_PRIORITY=0
PRIORITY=0
DEFAULT_FONT=1
Then, add the following lines in Switches.ini just before the switch SWITCH="\" (after the newly defined switch SWITCH="LaTeX3 Kernel Prefixes")
SWITCH="LaTeX3 Properties"
ENABLED=1
MODE_FILTER="TeX"
START="."
STOP=":"
HIGHLIGHT_START=1
STEP_OVER_STOP=1
HIGHLIGHT_STOP=1
SCOPE=0
DOMINANT_PRIORITY=0
STRICT_PRIORITY=0
PRIORITY=5
INDENTED=0
BOLN_ONLY=0
CASE_SENSITIVE=1
START1_TRIGGER=""
START2_TRIGGER="code"
START2_TRIGGER="choice"
START2_TRIGGER="choice_code"
START2_TRIGGER="generate_choices"
START2_TRIGGER="initial"
START2_TRIGGER="default"
START2_TRIGGER="meta"
START2_TRIGGER="alloc_"
START2_TRIGGER="bool_"
START2_TRIGGER="box_"
START2_TRIGGER="cctab_"
START2_TRIGGER="char_"
START2_TRIGGER="chk_"
START2_TRIGGER="clist_"
START2_TRIGGER="codedoc_"
START2_TRIGGER="coffin_"
START2_TRIGGER="color_"
START2_TRIGGER="cs_"
START2_TRIGGER="dim_"
START2_TRIGGER="driver_"
START2_TRIGGER="else_"
START2_TRIGGER="etex_"
START2_TRIGGER="exp_"
START2_TRIGGER="expl_"
START2_TRIGGER="fi_"
START2_TRIGGER="file_"
START2_TRIGGER="fp_"
START2_TRIGGER="group_"
START2_TRIGGER="hbox_"
START2_TRIGGER="hcoffin_"
START2_TRIGGER="if_"
START2_TRIGGER="insert_"
START2_TRIGGER="int_"
START2_TRIGGER="ior_"
START2_TRIGGER="iow_"
START2_TRIGGER="kernel_"
START2_TRIGGER="keys_"
START2_TRIGGER="keyval_"
START2_TRIGGER="lua_"
START2_TRIGGER="luatex_"
START2_TRIGGER="mode_"
START2_TRIGGER="msg_"
START2_TRIGGER="muskip_"
START2_TRIGGER="or_"
START2_TRIGGER="pdftex_"
START2_TRIGGER="peek_"
START2_TRIGGER="prg_"
START2_TRIGGER="prop_"
START2_TRIGGER="quark_"
START2_TRIGGER="reverse_"
START2_TRIGGER="scan_"
START2_TRIGGER="seq_"
START2_TRIGGER="skip_"
START2_TRIGGER="str_"
START2_TRIGGER="tex_"
START2_TRIGGER="tl_"
START2_TRIGGER="token_"
START2_TRIGGER="use_"
START2_TRIGGER="vbox_"
START2_TRIGGER="vcoffin_"
START2_TRIGGER="xetex_"
STOP1_TRIGGER=""
STOP2_TRIGGER="n"
STOP2_TRIGGER="N"
STOP2_TRIGGER="x"
TEXT_COLOR=1
DEFAULT_FONT=1
ITALIC=1
SWITCH="LaTeX3 Kernel Constants"
ENABLED=1
MODE_FILTER="TeX"
START="\"
STOP="~Alpha@_:"
HIGHLIGHT_START=1
STEP_OVER_STOP=0
HIGHLIGHT_STOP=0
SCOPE=0
DOMINANT_PRIORITY=0
STRICT_PRIORITY=0
PRIORITY=5
INDENTED=0
BOLN_ONLY=0
CASE_SENSITIVE=1
START1_TRIGGER=""
START2_TRIGGER="c_"
STOP1_TRIGGER=""
STOP2_TRIGGER=""
TEXT_COLOR=6
DEFAULT_FONT=1
ITALIC=1
Result:
For the record, here's my font-lock settings for Emacs with LaTeX3. To make use of these, I define a latex3-mode which is derived from the inbuild latex-mode. (Note: I don't use AucTeX, I use the simple TeX modes.) The idea of the font-locking is similar to Joseph's in that it adds more matches for highlighting. I've added a specials for core functions (probably not included them all as yet). There's also a distinction between macros with a _ and ones without (such as ordinary LaTeX2e macros/functions and the xparse ones). I could probably do with defining a few more faces to further distinguish, but this is currently working well enough for me.
(defconst latex3-font-lock-keywords
(eval-when-compile
(let* (;; Commands relevant to data structures
(specials "\\(bool\\|char\\|clist\\|cs\\|exp\\|file\\|group\\|keys\\|prop\\|q\\|seq\\|tl\\)_[a-zA-Z_:]+")
(general "\\([a-zA-Z_:]+\\)")
(camel "\\([a-zA-Z@]+\\)")
(nocamel "\\(?:[^a-zA-Z@_:]\\)")
(slash "\\\\")
)
(list
(list (concat "\\(" slash specials "\\)")
1 'font-lock-function-name-face)
(list (concat "\\(" slash camel "\\)" nocamel) 1 'font-lock-function-name-face)
(list (concat "\\(" slash general "\\)") 1 'font-lock-variable-name-face)
)
)
)
"Extra commands to highlight in LaTeX3 modes."
)
(define-derived-mode latex3-mode latex-mode "LaTeX3"
"Major mode to edit LaTeX3 files."
(set (make-local-variable 'font-lock-defaults)
'((tex-font-lock-keywords latex3-font-lock-keywords)
nil nil ((?$ . "\"")) nil
;; Who ever uses that anyway ???
(font-lock-mark-block-function . mark-paragraph)
(font-lock-syntactic-face-function
. tex-font-lock-syntactic-face-function)
(font-lock-unfontify-region-function
. tex-font-lock-unfontify-region)
(font-lock-syntactic-keywords
. tex-font-lock-syntactic-keywords)
(parse-sexp-lookup-properties . t)))
)
(add-hook 'latex3-mode-hook (function (lambda () (setq ispell-parser 'tex))))
(add-hook 'latex3-mode-hook '(lambda () (flyspell-mode 0)))
Here's a screenshot (where you can see that I should add box and vbox to my list of special functions):
TeXworks uses a simple regex-based approach to syntax highlighting, with the information stored in the file syntax-patterns.txt inside the folder TeXworks/configuration, which lives in a system-dependent location. I have a set of patterns for working with .dtx (LaTeX documented source) files, which include highlighting for expl3 code:
[LaTeX DTX]
# comments
red Y \^\^A.*
# Guards
orange N %<(?:[A-Za-z0-9!\|]+|.)>
limegreen N %<\*(?:[A-Za-z0-9!\|]+|.)>
crimson N %</(?:[A-Za-z0-9!\|]+|.)>
darkviolet N %<@@=(?:[A-Za-z]+|.)>
# special characters
darkred N \^\^\^\^\^[0-9a-z]{5}
darkred N \^\^\^\^[0-9a-z]{4}
darkred N \^\^\^[0-9a-z]{3}
darkred N \^\^[0-9a-z]{2}
darkred N [$#^_{}&]
gray N ^%%.*
gray N ^%
# Macrocode
green N \\(?:begin|end)\{macrocode\}
# LaTeX environments
darkgreen N \\(?:begin|end)\s*\{[^}]*\}
# control sequences
blue N \\(?:[A-Za-z@:_]+|.)
(The colour scheme is based on that used by WinEdt for .dtx editing, as I used to use WinEdt.)
The key line is the last one, as regex \\(?:[A-Za-z@:_]+|.) matches both expl3 and 'standard' LaTeX2e macro names. If you wanted to only highlight expl3 code-level macros in one colour, you could go with something like
# functions
blue N \\(?:[A-Za-z_]+):(?:[A-Za-z_:]+|.)
# Variables
blue N \\(?:[A-Za-z_]+)_(?:[A-Za-z_]+|.)
which then won't highlight document/design level (just letter in the name) or LaTeX2e internal (@ in the name) control sequences. You could of course get more sophisticated and give them all different highlighting! You could also extend the regexes in the way Andrew Stacey does in his answer to differentiate expl3 core and 'additional' names, using a more sophisticated set of regexes.
Note that I've also extended the .dtx guards recognised to cover the %<@@=...> one that l3doscrip adds to the 'standard' set, for marking internal names. It's covered by the line
darkviolet N %<@@=(?:[A-Za-z]+|.)>
where I had to find a new colour to fit in with those that were already in use!