Hi everyone! I am trying to come up with a composable, configurable system for improving build recipes. Part of the goal of Tidepool is for package definitions to be easy to write and maintain. In order to do that, we need better solutions than manually operating compilers via injected command line strings. To solve this, I am working on a design for Builders and Capabilities. Let’s talk about those and what I am envisioning for their use.
Capabilities
Let’s start with Capabilities. A capability represents a single-purpose helper which can be snapped into package definitions. Something such as “run make” or “do JavaScript tests”. These can be added and removed as needed to compose a full set of build operations. You may combine capabilities for a C compiler, a build runner, artifact installation, or anything else you need for your package.
# package.nix
{ config }:
{
capabilities = {
inherit (config.capabilities) make;
};
}
Each of these capabilities needs to, like packages, support versioning as well as multiple instances of themselves.
# package.nix
{ config }:
{
capabilities = {
make = config.capabilities.make.versions."1.2.3";
makeSubProject = config.capabilities.make.extend {
path = "/subdir";
};
};
}
Allowing version selection and customization like this is vital for allowing capabilities to compose and be useful for packages that may otherwise have unique restrictions.
Capabilities may provide dependencies as well as build hooks just like packages do. These merge with the dependencies and hooks of the package definition they are used on when built. Certain considerations need to be made for multiple instances of a given capability when different dependency versions may be used. For example, two versions of the same capability may provide a C compiler and append a hook which sets $CC in the build environment. There needs to be a way to disable this behavior directly on the capability.
Builders
A builder is a comprehensive module which is used to fully build a package, producing its resulting artifact. Builders help collect capabilities and necessary dependencies into a single abstraction which can be applied to a package, removing complexity from the individual package definitions. For example, you may use a “C Builder” to build a C project.
# package.nix
{ config }:
{
builder = config.builders.c;
}
Similar to capabilities, builders must also support versioning and configuration.
# package.nix
{ config }:
{
builder = config.builders.c.versions."1.2.3";
}
# package.nix
{ config }:
{
builder = config.builders.c.extend {
toolchain = "gnu";
};
}
By supporting versioning on both capabilities and builders, we are able to freeze package versions in-place. This makes Tidepool uniquely capable of preserving package history and allowing for discovery and use of older package versions than existing solutions make possible.
Importantly, builders and capabilities can both be used or ignored in whatever capacity the user prefers. If a builder does not support something that a particular package needs, then a package can apply capabilities or custom scripts as needed to its own package definition.
Takeaways
- Builders are the user-friendly abstraction for “help me build a package for X kind of project”
- Capabilities are the composable bits that actually provide build steps and tools
- Everything supports versioning so that it can safely be frozen to retain older versions of packages without worrying about breakage
- Everything supports configuration/extension in standard Tidepool fashion
- Making a new package definition should, in most cases, be as simple as setting
builderandsrc.