...one of the most highly
regarded and expertly designed C++ library projects in the
world.
— Herb Sutter and Andrei
Alexandrescu, C++
Coding Standards
The Jam/MR Executable
jam [ -a ] [ -n ] [ -v ] [ -q ] [ -d debug ] [ -f jambase ] [ -j jobs ] [ -o actionsfile ] [ -s var=value ] [ -t target ] [ target ... ]
Jam is a program construction tool, like make(1).
Jam recursively builds target files from source files, using dependency information and updating actions expressed in the Jambase file, which is written in jam's own interpreted language. The default Jambase is compiled into jam and provides a boilerplate for common use, relying on a user-provide file "Jamfile" to enumerate actual targets and sources.
The Jambase is described in the Jambase Reference and the document Using Jamfiles and Jambase.
If target is provided on the command line, jam builds target; otherwise jam builds the target 'all'.
Jam may be invoked with the following options:
-a
| Build all targets anyway, even if they are up-to-date. |
-d n
| Enable cummulative debugging levels from 1 to n.
Interesting values are:
|
-d +n
| Enable debugging level n. |
-d 0
| Turn off all debugging levels. Only errors are not suppressed. |
-f jambase
| Read jambase instead of using the built-in Jambase. Only one -f flag is permitted, but the jambase may explicitly include other files. |
-j n
| Run up to n shell commands concurrently (UNIX and NT only). The default is 1. |
-n
| Don't actually execute the updating actions, but do everything else. This changes the debug level default to -d2. |
-o file
| Write the updating actions to the specified file instead of running them (or outputting them, as on the Mac). |
-q
| Quit quickly (as if an interrupt was received) as soon as any target fails. |
-s var=value
| Set the variable var to value, overriding both internal variables and variables imported from the environment. |
-t target
| Rebuild target and everything that depends on it, even if it is up-to-date. |
-v
| Print the version of jam and exit. |
Jam has four phases of operation: start-up, parsing, binding, and updating.
Upon start-up, jam imports environment variable settings into jam variables. Environment variables are split at blanks with each word becoming an element in the variable's list of values. Environment variables whose names end in PATH are split at $(SPLITPATH) characters (e.g., ":" for Unix).
To set a variable's value on the command line, overriding the variable's environment value, use the -s option. To see variable assignments made during jam's execution, use the -d+7 option.
In the parsing phase, jam reads and parses the Jambase file, by default the built-in one. It is written in the jam language. See Language below. The last action of the Jambase is to read (via the "include" rule) a user-provided file called "Jamfile".
Collectively, the purpose of the Jambase and the Jamfile is to
name built target and source files, construct the dependency
graph among them, and associate build actions with targets.
The Jambase defines boilerplate rules and variable assignments,
and the Jamfile uses these to specify the actual relationship
among the target and source files. See the Jambase Reference and the document Using Jamfiles and Jambase for information.
Binding
File target names are given as absolute or relative path names in the filesystem. If the path name is absolute, it is bound as is. If the path name is relative, it is normally bound as is, and thus relative to the current directory. This can be modified by the settings of the $(SEARCH) and $(LOCATE) variables, which enable jam to find and build targets spread across a directory tree. See SEARCH and LOCATE Variables below.
During the binding phase, jam also performs header file scanning, where it looks inside source files for the implicit dependencies on other files caused by C's #include syntax. This is controlled by the special variables $(HDRSCAN) and $(HDRRULE). The result of the scan is formed into a rule invocation, with the scanned file as the target and the found included file names as the sources. Note that this is the only case where rules are invoked outside the parsing phase. See HDRSCAN and HDRRULE Variables below.
After binding, jam again recursively descends the dependency graph, this time executing the update actions for each target marked for update during the binding phase. If a target's updating actions fail, then all other targets which depend on that target are skipped.
The -j flag instructs jam to build more than one target
at a time. If there are multiple actions on a single target,
they are run sequentially.
LANGUAGE
Jam treats its input files as whitespace-separated tokens, with two exceptions: double quotes (") can enclose whitespace to embed it into a token, and everything between the matching curly braces ({}) in the definition of a rule action is treated as a single string. A backslash (\) can escape a double quote, or any single whitespace character.
Jam requires whitespace (blanks, tabs, or newlines) to surround all tokens, including the colon (:) and semicolon (;) tokens.
Jam keywords (an mentioned in this document) are reserved and generally must be quoted with double quotes (") to be used as arbitrary tokens, such as variable or target names.
The essential jam data entity is a target. Built targets are files to be updated. Source targets are the files used in updating built targets. Built targets and source targets are collectively referred to as file targets, and frequently built targets are source targets for other built targets. Pseudotargets are symbols which represent dependencies on other targets, but which are not themselves associated with any real file.
A file target's identifier is generally the file's name, which can be absolutely rooted, relative to the directory of jam's invocation, or simply local (no directory). Most often it is the last case, and the actual file path is bound using the $(SEARCH) and $(LOCATE) special variables. See SEARCH and LOCATE Variables below. A local filename is optionally qualified with grist, a string value used to assure uniqueness. A file target with an identifier of the form file(member) is a library member (usually an ar(1) archive on UNIX).
The basic jam language entity is called a rule. A rule is defined in two parts: the procedure and the actions. The procedure is a body of jam statements to be run when the rule is invoked; the actions are the OS shell commands to execute when updating the built targets of the rule.
Rules can return values, which can be expanded into a list with "[ rule args ... ]". A rule's value is the value of its last statement, though only the following statements have values: 'if' (value of the leg chosen), 'switch' (value of the case chosen), set (value of the resulting variable), and 'return' (value of its arguments). Note that 'return' doesn't actually cause a return, i.e., is a no-op unless it is the last statement of the last block executed within rule body.
The jam statements for defining and invoking rules are as follows:
|
A rule is invoked with values in field1 through fieldN. They may be referenced in the procedure's statements as $(1) through $(N) (9 max), and the first two only may be referenced in the action's commands as $(1) and $(2). $(<) and $(>) are synonymous with $(1) and $(2).
Rules fall into two categories: updating rules (with actions), and pure procedure rules (without actions). Updating rules treat arguments $(1) and $(2) as built targets and sources, respectively, while pure procedure rules can take arbitrary arguments.
When an updating rule is invoked, its updating actions are added to those associated with its built targets ($(1)) before the rule's procedure is run. Later, to build the targets in the updating phase, commands are passed to the OS command shell, with $(1) and $(2) replaced by bound versions of the target names. See Binding above.
Rule invokation may be indirected through a variable:
|
The following action modifiers are understood:
|
Jam has eleven built-in rules, all of which are pure procedure rules without updating actions. They are in three groups: the first builds the dependency graph; the second modifies it; and the third are just utility rules.
|
The six rules ALWAYS, LEAVES, NOCARE, NOTFILE, NOUPDATE, and TEMPORARY modify the dependency graph so that jam treats the targets differently during its target binding phase. See Binding above. Normally, jam updates a target if it is missing, if its filesystem modification time is older than any of its dependencies (recursively), or if any of its dependencies are being updated. This basic behavior can be changed by invoking the following rules:
|
The two rules ECHO and EXIT are utility rules, used only in jam's parsing phase.
|
The GLOB rule does filename globbing.
|
The MATCH rule does pattern matching.
|
Jam has several simple flow-of-control statements:
|
Jam variables are lists of zero or more elements, with each element being a string value. An undefined variable is indistinguishable from a variable with an empty list, however, a defined variable may have one more elements which are null strings. All variables are referenced as $(variable).
Variables are either global or target-specific. In the latter case, the variable takes on the given value only during the updating of the specific target.
A variable is defined with:
|
The first two forms set variable globally. The third and forth forms set a target-specific variable. The = operator replaces any previous elements of variable with elements; the += operation adds elements to variable's list of elements. The final two forms are synonymous: they set variable globally, but only if it was previously unset.
Variables referenced in updating commands will be replaced with their values; target-specific values take precedence over global values. Variables passed as arguments ($(1) and $(2)) to actions are replaced with their bound values; the "bind" modifier can be used on actions to cause other variables to be replaced with bound values. See Action Modifiers above.
Jam variables are not re-exported to the environment of the shell that executes the updating actions, but the updating actions can reference jam variables with $(variable).
During parsing, jam performs variable expansion on each token that is not a keyword or rule name. Such tokens with embedded variable references are replaced with zero or more tokens. Variable references are of the form $(v) or $(vm), where v is the variable name, and m are optional modifiers.
Variable expansion in a rule's actions is similar to variable expansion in statements, except that the action string is tokenized at whitespace regardless of quoting.
The result of a token after variable expansion is the product of the components of the token, where each component is a literal substring or a list substituting a variable reference. For example:
|
The variable name and modifiers can themselves contain a variable reference, and this partakes of the product as well:
|
Because of this product expansion, if any variable reference in a token is undefined, the result of the expansion is an empty list. If any variable element is a null string, the result propagates the non-null elements:
|
A variable element's string value can be parsed into grist and filename-related components. Modifiers to a variable are used to select elements, select components, and replace components. The modifiers are:
[n]
| Select element number n (starting at 1). If the variable contains fewer than n elements, the result is a zero-element list. |
[n-m]
| Select elements number n through m. |
[n-]
| Select elements number n through the last. |
:B
| Select filename base. |
:S
| Select (last) filename suffix. |
:M
| Select archive member name. |
:D
| Select directory path. |
:P
| Select parent directory. |
:G
| Select grist. |
:U
| Replace lowercase characters with uppercase. |
:L
| Replace uppercase characters with lowercase. |
:chars
| Select the components listed in chars. |
:G=grist
| Replace grist with grist. |
:D=path
| Replace directory with path. |
:B=base
| Replace the base part of file name with base. |
:S=suf
| Replace the suffix of file name with suf. |
:M=mem
| Replace the archive member name with mem. |
:R=root
| Prepend root to the whole file name, if not already rooted. |
:E=value
| Assign value to the variable if it is unset. |
:J=joinval
| Concatentate list elements into single element, separated by joinval. |
On VMS, $(var:P) is the parent directory of $(var:D).
This section discusses variables that have special meaning to
jam.
SEARCH and LOCATE Variables
These two variables control the binding of file target names to locations in the file system. Generally, $(SEARCH) is used to find existing sources while $(LOCATE) is used to fix the location for built targets.
Rooted (absolute path) file targets are bound as is. Unrooted file target names are also normally bound as is, and thus relative to the current directory, but the settings of $(LOCATE) and $(SEARCH) alter this:
Both $(SEARCH) and $(LOCATE) should be set target-specific and
not globally. If they were set globally, jam would use
the same paths for all file binding, which is not likely to
produce sane results. When writing your own rules, especially
ones not built upon those in Jambase, you may need to set
$(SEARCH) or $(LOCATE) directly. Almost all of the rules defined
in Jambase set $(SEARCH) and $(LOCATE) to sensible values for
sources they are looking for and targets they create, respectively.
HDRSCAN and HDRRULE Variables
These two variable control header file scanning. $(HDRSCAN) is an egrep(1) pattern, with ()'s surrounding the file name, used to find file inclusion statements in source files. Jambase uses $(HDRPATTERN) as the pattern for $(HDRSCAN). $(HDRRULE) is the name of a rule to invoke with the results of the scan: the scanned file is the target, the found files are the sources. This is the only place where jam invokes a rule through a variable setting.
Both $(HDRSCAN) and $(HDRRULE) must be set for header file scanning to take place, and they should be set target-specific and not globally. If they were set globally, all files, including executables and libraries, would be scanned for header file include statements.
The scanning for header file inclusions is not exact, but it is at least dynamic, so there is no need to run something like makedepend(GNU) to create a static dependency file. The scanning mechanism errs on the side of inclusion (i.e., it is more likely to return filenames that are not actually used by the compiler than to miss include files) because it can't tell if #include lines are inside #ifdefs or other conditional logic. In Jambase, HdrRule applies the NOCARE rule to each header file found during scanning so that if the file isn't present yet doesn't cause the compilation to fail, jam won't care.
Also, scanning for regular expressions only works where the included file name is literally in the source file. It can't handle languages that allow including files using variable names (as the Jam language itself does).
A number of Jam built-in variables can be used to identify runtime platform:
OS | OS identifier string |
OSPLAT | Underlying architecture, when applicable |
MAC | true on MAC platform |
NT | true on NT platform |
OS2 | true on OS2 platform |
UNIX | true on Unix platforms |
VMS | true on VMS platform |
JAMDATE | Time and date at jam start-up. |
JAMUNAME | Ouput of uname(1) command (Unix only) |
JAMVERSION | jam version, currently "2.3" |
When jam executes a rule's action block, it forks and execs a shell, passing the action block as an argument to the shell. The invocation of the shell can be controlled by $(JAMSHELL). The default on Unix is, for example:
JAMSHELL = /bin/sh -c % ;
The % is replaced with the text of the action block.
Jam does not directly support building in parallel across multiple hosts, since that is heavily dependent on the local environment. To build in parallel across multiple hosts, you need to write your own shell that provides access to the multiple hosts. You then reset $(JAMSHELL) to reference it.
Just as jam expands a % to be the text of the rule's action block, it expands a ! to be the multi-process slot number. The slot number varies between 1 and the number of concurrent jobs permitted by the -j flag given on the command line. Armed with this, it is possible to write a multiple host shell. For example:
|
In addition to generic error messages, jam may emit one of the following:
|
The -j flag can cause jam to get confused when single actions update more than one target at a time. jam may proceed as if the targets were built even though they are still under construction.
For parallel building to be successful, the dependencies among files must be properly spelled out, as targets tend to get built in a quickest-first ordering. Also, beware of un-parallelizable commands that drop fixed-named files into the current directory, like yacc(1) does.
With the -j flag, errors from failed commands can get staggeringly mixed up.
A poorly set $(JAMSHELL) is likely to result in silent failure.
Jam documentation and source are available from the Perforce Public Depot.
Jam's author is Christopher Seiwald (seiwald@perforce.com). Documentation is provided by Perforce Software, Inc.
Copyright 1993-2002 Christopher Seiwald and Perforce Software, Inc.
Comments to info@perforce.com
Last updated: April 1, 2002
$Id: Jam.html,v 1.6 2003/09/01 08:04:33 vladimir_prus Exp $