2024/02/21: Describing chained pipes

In the build system I'm currently working on, the repository description used by the tool basically acts as a lock file, i.e., it cotains the precise trees/commits of the dependencies, but is generated from a more high-level description. For trees, there is a clear need for a script generating the tree description, but for importing git repositories, just-import-git does exactly what is needed.

Nevertheles, I found it convenient to still move the information of which repositories to import out of the update script, also because in that way, the scripts can reside under the subdirectory bin of my repository, whereas all the dependency-related information can reside in etc. So, now I describe the repositories to import as a list, with each entry describing one repository to import given by

encoded, in the obvious way, as a list. Such a decription might, e.g., look as follows. 
[ ["upstream rules", "https://github.com/just-buildsystem/rules-cc"]
, [ "rules-typesetting src"
  , "https://github.com/just-buildsystem/rules-typesetting"
  , "--plain"
  ]
]
And the actual update is done by a trivial python script, chained into the update script in the usual way. 
#!/usr/bin/env python3

import json
import subprocess
import sys
import tempfile

def log(*args, **kwargs) -> None:
    print(*args, file=sys.stderr, **kwargs)

def fail(s, exit_code):
    log(f"Error: {s}")
    sys.exit(exit_code)

def run_cmd(cmd, stdin=subprocess.DEVNULL):
    result = subprocess.run(cmd, stdout=subprocess.PIPE, stdin=stdin)
    if result.returncode != 0:
        fail("Command %s failed" % (cmd,))
    return result.stdout.decode('utf-8')

def update(repos, to_import):
    for desc in to_import:
        cmd = [
            "just-import-git", "-C", "-",
            "--as", desc[0]
        ] + desc[2:] + [desc[1]]
        log("%r --> %r" % (desc,cmd))
        with tempfile.TemporaryFile() as f:
            f.write(json.dumps(repos).encode('utf-8'))
            f.seek(0)
            repos = json.loads(run_cmd(cmd, stdin=f))
    return repos

if __name__ == "__main__":
    repos = json.load(sys.stdin)
    with open(sys.argv[1]) as f:
        to_import = json.load(f)
    repos = update(repos, to_import)
    print(json.dumps(repos))
download