Tooltree is a project structured as a set of packages that exist as siblings immediately under the tooltree directory. The goal is to structure a project — a versioned collection of assets, such a git repository — as a set of smaller components on which development can proceed independently.
Dependencies between packages are minimized and managed. Packages import selected packages and they export a directory that will be provided to other packages when they import it. This allows a package to change its internal organization substantially while still supporting the same contracts with other packages.
Each package has a Makefile that describes how it is built. Here are some build command examples that work across Tooltree packages. The following commands deal with assets within the current package:
make Build/update the `default` target. make clean Delete build products. make graph Show dependences between assets built by `make`.
The following commands deal with assets outside of the current package:
make imports Build/update all imported packages make deep Build/update imported packages *and* this package. make clean imports Delete build products for imported packages. make 'Graph(imports)' Show imported packages and their relationships.
The top-level Makefile is itself an empty package that includes other packages for the convenience of validating the entire project.
See any of tooltree/*/Makefile for examples. Here are the key elements of a tooltree Makefile:
Minion: Each package in tooltree has a Makefile that defines Minion targets and then, at the bottom, includes build/tooltree.mk.
tooltree.mk includes minion.mk after defining a number of variables, functions, and Minion classes used across multiple packages. Some of these definitions support the modular, packaage-based structure of tooltree.
Package name: Each package has a name, defined by the thisPackage variable. This defaults to the name of the directory containing the package, so packages within the tooltree directory structure rely on this default definition.
Properties: Each package has properties that describe its imports and the directory it exports. See Package Descriptions, below, for what these definitions look like. For packages within tooltree, these are defined in tooltree.mk.
Imports: For each imported package P, our Makefile can use $(package.P) to refer to the directory exported by P.
Imported Makefiles: When a package Makefile includes makefiles that are exported by other packages, it sets the includeImports variable to a list of paths describing their locations. There are a few reasons for using this mechanism rather than directly including them:
Convenience and modularity. The first element in each path is the name of the exporting package, which will be replaced with that package's export directory. For example, 'p1/defs.mk' expands to $(package.p1)/defs.mk, which in turn will expand to something like ../p1/.out/exports/defs.mk, depending on how p1's export directory is defined.
These makefiles have to be included after minion.mk has been loaded because $(package.PKG) variables generally include $(VOUTDIR) or $V, which are defined or defaulted by Minion.
These makefiles might not exist until imported packages are built. This mechanism allows some essential build commands like make imports and make help to proceed even when the imported makefiles do not exist, while still providing important error checking when more complex targets are selected.
Packages are defined with the following variables:
$(package.NAME.dir) = the package directory, where its Makefile resides.
$(package.NAME.imports) = other packages that must be built before this package is built.
$(package.NAME.outdir) = a path relative to DIR that names a directory that will contain exports (build results) after the package is built. If empty or undefined, there is no build step and the entire package is exported.
Tooltree.mk will compute the following for each package imported by the package currently being built:
$(package.NAME) = path to the export directory.
To leverage tooltree, an external project's Makefile at a minimum will declare its own package name (with thisPackage = NAME), declare the packages it imports (with package.NAME.imports = ...), define includeImports if it uses imported makfiles, and finally include tooltree.mk at the bottom.
For example:
Alias(default).in = LuaExe(prog.lua) thisPackage = root package.root.imports = build-lua includeImports = build-lua/build-lua.mk include <PATH-TO-TOOLTREE>/build/tooltree.mk
Note: While thisPackage will default to the name of the directory that contains it, this default should probably be avoided in the case of top-level makefiles because that directory name may vary.
If the project is itself structured as multiple packages, it can provide a make include file that can be used by all its packages, analogously to how tooltree.mk is used within Tooltree. It would describe all the packages in its project and then include tooltree.mk.
For example, the project-level include file might look like this:
# projectX.mk # this can be included from Makefiles in other directories... Xdir := $(dir $(lastword $(MAKEFILE_LIST))) package.foo.dir := $(Xdir)foo package.foo.outdir = $(VOUTDIR)/exports package.foo.imports = build-lua package.bar.dir := $(Xdir)bar/ package.bar.outdir = . package.bar.imports = foo include $(Xdir)/<PATH_TO_TOOLTREE>/tooltree.mk
Its own package makefiles would resemble those in tooltree:
# projectX/foo/Makefile Alias(default).in = Ship(exports) exports = LuaExe(foo) includeImports = build-lua/build-lua.mk include ../projectX.mk
This new build directory comes along with a rewrite of the build system that was motivated by the following goals:
Make it easier for external projects to leverage Tooltree packages.
Simplify the build system — reduce the number of “moving parts”.
Use Minion instead of Crank for builds.
Remove unused packages.
In more detail, the changes comprise the following:
The following packages have been dropped: p4x, pakman, simp4, cdep, ctools, crank.
Two-phase, two-level builds have been abandoned.
Formerly, a “make configure” step at the top level of the project built a project configuration file that described the entirety of the build, determining what a subsequent “make all” would do, both at the project and package level. Instead of this, all potential targets are always available, and the default targets at project and package level are hardcoded.
SUBDIR/Package files are no longer supported. This information, along with the configuration data formerly in the top-level Makefile (where packages are located, what variants are built) is now in build/tooltree.mk, so it can be leveraged by external makefiles.
Package-level makefiles can build the entire project, and the top-level makefile is just a package that builds a selected set of packages.
Many build features have been dropped.
This includes support for Windows/Cygwin builds, support for multiple toolchains, cross-target builds, and valgrind builds.
These features introduced a fair amopunt of complexity in the build system and is no longer actively used. In cross-target builds, for example, the MDB package's “release” variant could make use of both the “host” (build machine) variant and the “release” (target) variant of the Lua package. This is not currently supported in the Minion-based builds.
The original Crank sources, in all their glory, are still there in the history, and on the “crank” branch.
Minion replaces Crank.
Minion retains the “functional OO” flavor of Crank, but is more refined. It addresses the most common pain points of Crank.
Minion supports more readable syntax for property definitions — {prop} vs. $(call .,prop)— and more powerful, function-like syntax for constructing instances from other instances — as in Copy(CExe(foo.c),dir:exports).
Minion understands dependencies between instances, because inputs and dependencies can be specified as instance names, not just file names. These instance names are automatically translated to output file names for use in command-line contexts (e.g. {^} and {@}). In Crank, intermediate files would often have have to be explicitly named, and it leaned on classes not just as types, but as collections of files.
Minion makefile goals are explicit and easy to trace. In Crank, aliases were defined as side effects via the prereqOf property. Class names were available as phony targets, and were implicitly a preqreq of “all” (the default goal).
Minion makefiles are purely functional, relying on subclassing to customize build behavior. They do not override Minion's variable definitions; minion.mk is included only at the end of the user makefile. Crank makefiles would often need to override or modify property definitions, which introduced brittle dependencies on class implementations.
Cached (pre-compiled) rules are on by default in Crank, so makefiles have to use <wildcard> and <shell> to avoid consistency problems. In Minion, caching must be explicitly enabled, and small makefiles are plenty fast without it.
This Crank example:
CC += $(call <wildcard>,*.c) # include rules to compile these Exe += prog Exe[prog].prereqOf = prog # `prog` goal defined here (& elsewhere?) Exe[prog].in = $(call get,out,CC) # accepts files, not items
In Minion is:
Alias(prog).in = CExe(@*.c)