Tools and generators

| Home | Libraries | People | FAQ | More

## Tools and generators

This section will describe how Boost.Build can be extended to support new tools.

For each additional tool, a Boost.Build object called generator must be created. That object has specific types of targets which it accepts an produces. Using that information, Boost.Build is able to automatically invoke the generator. For example, if you declare a generator which takes a target of the type `D` and produces a target of the type `OBJ`, when placing a file with extention `.d` in a list of sources will cause Boost.Build to invoke your generator, and then to link the resulting object file into an application. (Of course, this requires that you specify that the `.d` extension corresponds to the `D` type.)

Each generator should be an instance of a class derived from the `generator` class. In the simplest case, you don't need to create a derived class, but simply create an instance of the `generator` class. Let's review the example we've seen in the introduction.

    import generators ;
    generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
    actions inline-file
    {
        "./inline-file.py" $(<) $(>)
    }

We declare a standard generator, specifying its id, the source type and the target type. When invoked, the generator will create a target of type `CPP` which will have the source target of type `VERBATIM` as the only source. But what command will be used to actually generate the file? In bjam, actions are specified using named "actions" blocks and the name of the action block should be specified when creating targets. By convention, generators use the same name of the action block as their own id. So, in above example, the "inline-file" actions block will be use to convert the source into the target.

There are two primary kinds of generators: standard and composing, which are registered with the `generators.register-standard` and the `generators.register-composing` rules, respectively. For example:

    generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
    generators.register-composing mex.mex : CPP LIB : MEX ;

The first generators takes a single source of type `VERBATIM` and produces a result. The second generator takes any number of sources, which can have either the `CPP` or the `LIB` type. Composing generators are typically used for generating top-level target type. For example, the first generator invoked when building an `exe` target is a composing generator corresponding to the proper linker.

You should also know about two specific function for registering generators: `generators.register-c-compiler` and `generators.register-linker`. The first sets up header dependecy scanning for C files, and the seconds handles various complexities like searched libraries. For that reason, you should always use those functions when adding support for compilers and linkers.

(Need a note about UNIX)

### Custom generator classes

The standard generators allows you to specify source and target types, action, and a set of flags. If you need anything more complex, you need to create a new generator class with your own logic. Then, you have to create an instance of that class and register it. Here's an example how you can create your own generator class:

    class custom-generator : generator
    {
        rule __init__ ( * : * )
        {
            generator.__init__ $(1) : $(2) : $(3) : $(4) : $(5) : $(6) : $(7) : $(8) : $(9) ;
        }
    }

    generators.register
      [ new custom-generator verbatim.inline-file : VERBATIM : CPP ] ;

This generator will work exactly like the `verbatim.inline-file` generator we've defined above, but it's possible to customize the behaviour by overriding methods of the `generator` class.

There are two methods of interest. The `run` methods is responsible for overall process - it takes a number of source targets, converts them the the right types, and creates the result. The `generated-targets` method is called when all sources are converted to the right types to actually create the result.

The `generated-target` method can be overridden when you want to add additional properties to the generated targets or use additional sources. For example (which is real), you have a tool for analysing programs, which should be given a name of executable and the list of all sources. Naturally, you don't want to list all source files manually. Here's how the `generated-target` method can find the list of sources automatically:

    class itrace-generator : generator {
    ....
        rule generated-targets ( sources + : property-set : project name ? )
        {
            local leafs ;
            local temp = [ virtual-target.traverse $(sources[1]) : : include-sources ] ;
            for local t in $(temp)
            {
                if ! [ $(t).action ]
                {
                    leafs += $(t) ;
                }
            }
            return [ generator.generated-targets $(sources) $(leafs)
              : $(property-set) : $(project) $(name) ] ;
        }
    }
    generators.register [ new itrace-generator nm.itrace : EXE : ITRACE ] ;

The `generated-targets` rule will be called with a single source target of type `EXE`. The call to the `virtual-target.traverse` will return all targets the executable depends on, and we further find files which are not produced from anything. The found targets are added to the sources.

The `run` method can be overriden to completely customize the way generator works. In particular, the conversion of sources to the desired types can be completely customized. Here's another real example. Tests for the Boost Python library usually consist of two parts: a Python program and a C++ file. The C++ file is compiled to Python extension which is loaded by the Python program. But in the likely case that both files have the same name, the created Python extension must be renamed. Otherwise, Python program will import itself, not the extension. Here's how it can be done:

    rule run ( project name ? : property-set : sources * : multiple ? )
    {
        local python ;
        for local s in $(sources)
        {
            if [ $(s).type ] = PY
            {
                python = $(s) ;
            }
        }

        local libs ;
        for local s in $(sources)
        {
            if [ type.is-derived [ $(s).type ] LIB ]
            {
                libs += $(s) ;
            }
        }

        local new-sources ;
        for local s in $(sources)
        {
            if [ type.is-derived [ $(s).type ] CPP ]
            {
                local name = [ $(s).name ] ;
                if $(name) = [ $(python).name ]
                {
                    name = $(name)_ext ;
                }
                new-sources += [ generators.construct $(project) $(name) :
                  PYTHON_EXTENSION : $(property-set) : $(s) $(libs) ] ;
            }
        }

        result = [ construct-result $(python) $(new-sources) : $(project) $(name)
                     : $(property-set) ] ;
    }

First, we separate all source into python files, libraries and C++ sources. For each C++ source we create a separate Python extension by calling `generators.construct` and passing the C++ source and the libraries. At this point, we also change the extension's name, if necessary.