What to be aware when using new-style package?
I will give a FAQ-style presentation of some things I became aware of while transitioning one of my packages to this format. Leonid's description of the new package format is required reading before looking at this FAQ!
How are contexts assigned to symbols in the package?
In traditional packages, symbols are looked up from $ContextPath
, and if not found, created in $Context
. It is always the current value of $ContextPath
and $Context
that determines what a symbol name in the package file refers to.
BeginPackage
and Begin
simply manipulate $ContextPath
and $Context
. When they are evaluated, the value of these system variables will change. End
and EndPackage
change it back. The process is dynamic, and happens as each line in the package file is evaluated one by one.
In new-style packages, the context of each symbol is decided before the contents of the package file are evaluated. Mathematica will scan all files in the package directory for the Package
, PackageExport
, PackageImport
, PackageScope
directives, and uses them to pre-determine the context of each symbol that appears in the package.
The package contents are evaluated only after this has happened.
Which contexts are symbols created in?
A private symbol in a file named Foo
that is part of the package MyApp`
will go in the context MyApp`Foo`PackagePrivate`
.
A package scope symbol, declared with PackageScope
, will go in MyApp`PackageScope`
.
An exported symbol, declared with PackageExport
, will go in MyApp`
.
How do Package
, PackageExport
, PackageImport
, PackageScope
evaluate?
They do not evaluate at all. They are not expressions and hence they also cannot be terminated by a ;
which is a short form for CompoundExpression
.
They behave more like directives than symbols. They simply signal to the parser how to assign contexts to each symbol in the file. This assignment of contexts takes place before any evaluation is done.
This means that e.g. this is not valid:
PackageExport /@ {"symbol1", "symbol2"}
Instead we must write
PackageExport["symbol1"]
PackageExport["symbol2"]
PackageExport
never evaluates as a symbol.
In what order are files loaded?
A new-style package is usually made up by multiple files, each containing Package["MyApp`"]
. If the loading of one of these files is triggered with Get
or Needs
, all other files that belong to the MyApp`
package will be loaded.
After the first file was loaded, the rest will be loaded in alphabetical order.
The first file would either have the same name as the package itself (e.g. MyApp.m
for MyApp`
) or it would be explicitly loaded in Kernel/init.m
.
What are valid file names?
Since file names are mapped to context names, file names must also be valid context names. This means that _
or a space cannot be used in file names. File names may not start with a digit.
It is a natural thought to try to control loading order by prepending digits to file names. But such names are not valid.
What is the value of $Context
and $ContextPath
during package loading?
In Mathematica 11.0 and later, the value of $Context
and $ContextPath
correspond to the file that is currently being loaded.
For example, if the current file is Main.m
and the package name is MyApp`
, then
$Context === "MyApp`Main`PackagePrivate`"
$ContextPath === {"MyApp`PackageScope`", "MyApp`", "System`"}
However, in Mathematica 10.x, $Context
and $ContextPath
do not change at all during loading. They retain the value they had before package loading (e.g. $Context === Global`
and the usual $ContextPath
).
What this means in practice is that in Mathematica 10.x, ToExpression["x"]
would create the symbol x
in the Global`
context (or whatever was the context before package loading) instead of the package's private context.
The same is true for <*expr*>
included in StringTemplate
s.
Finally, if the package loads another file with Get
, all symbols in that file will be created in Global`
, not in the package's private context.
What is the value of $InputFile
and $Input
when loading new-style packages?
In Mathematica 11.0 and later, $InputFile
is always set to the specific file that is currently being loaded. It is different for each file that makes up the package.
In Mathematica 10.x, $InputFile
is always set to the first file of the package, regardless of which file is currently being loaded.
$Input
is always set to the name of the first file, in all versions between M10.0–M11.3.
Can we give a definition to Package
?
Each file that is part of the package must include
Package["MyContext`"]
As explained before Package
is just a directive, and does not have a definition. We might, in principle, give it a definition, and hope that it will evaluate in each package file.
This, however, does not happen. Package
and related directives are removed from the file before its contents are evaluated.
I would add to this that it is undocumented for a reason, unfinished, and may break in the future.