The build system I'm currently working does support multi-repository builds, but is a pure build system: it builds from files already available (potentially defined by git tree of some repository not necessarily checked out, but still available locally), but does not fetch new sources. In this way, the fetch-build separation is maintained that ensures all files are persistent locally before the build starts (ensuring that the project can be rebuilt later, allowing auditing of the source code, etc).
Nevertheless, it is often convenient to fetch and then build. And for complex repository set ups (involving many repositories) we need a fetch tool anyway, so let's also make that tool (called just-mr) a launcher for the actual build tool: it fetches the needed sources, sets up the configuration, and execve(2)s the requested tool (usually the build tool) passing on the remaining arguments after having forwarded the appropriate options.
Now, for fetching sources, there are typical cases that should (and are) built into the fetch tool, like getting an archive (with fixed blob id). However, no tool can have specialized knowledge of any version control system on earth (let alone all the other ways of getting sources), so there is generic way for all but the most-common cases, the "git tree" repository: you specify the root by a git tree identifier and provide a command (together with fixed environment variables) that, when executed in an empty directory (with the specified environment), will generate that tree (potentially in some subdirectory). Typically, that command is an appropriate call to a different version control system, something like ["cvs", "-d", "...", "export", "-D", "...", "desired-module"]. The ahead-of-time description of the result in form of the tree identifier allows to realize that the sources are already available without ever going to the network. Fixing of the environment variables avoids surprises and fetches that only sometimes work (or only in my shell or on my machine).
Now, if the repository is not public, some form of credentials have to be provided to the foreign version control system. In the case of ssh this typically means setting SSH_AUTH_SOCK to the path where the ssh agent listens. That path, however, is anything but constant, so (as opposed to the {"CVS_RSH": "ssh"} environement variable setting) it does not make sense to put it in any repository description or configuration file. Still, we have to pass it to the otherwise well isolated invocation of the foreign build system. For that, justbuild has another universal escape hatch: the local launcher. A command is (on unix) given by a list of strings (technically plus a path, but using the zeroth argument is fine, and here actually simplifies things); the launcher is another list of strings with which the command is prepended before we exec (chaing the usual unix way). So using ["env", "SSH_AUTH_SOCK=..."] as a launcher, we get what we want. (The launcher gets forwarded to just, but in the rc file we can set another -L option to override).
But always specifying that launcher with the appropriate value by hand on the command line when calling just-mr is inconvenient. So I ended up writing a launcher for just-mr (a launcher for just).
#!/usr/bin/env python3
import json
import os
import pathlib
import sys
ENV_KEEP = ["SSH_AUTH_SOCK"]
launcher = ["env", "PATH=%s/.nix-profile/bin" % (pathlib.Path.home(),)]
for x in ENV_KEEP:
if x in os.environ:
launcher.append("%s=%s" % (x, os.environ[x]))
cmd = ["just-mr", "-L", json.dumps(launcher)]
cmd.extend(sys.argv[1:])
print("Exec %r" % (cmd,), file=sys.stderr)
os.execvp("just-mr", cmd)
While there are good reasons for any of those indirections, it is still amazing how quickly that chain of execs grew.
Update: In the action of a "git tree" repository, environment variables can be inherited since 541ba9586b8a3aabe90f2dd01d994428dfab4cb9 implemented the respective design.