Improve our release workflow

The problem

As I outlined in both these posts (see posts linked at the bottom) we need to work on a workflow that allows us to properly utilise upstream’s caching.

In another post I suggested that we make releases to unstable at 7PM UTC since this was when I evaluated that most people would be awake and ready to triage a potential onslaught of issues.

But what happened in actuality is that we started rebuilding packages from source, for me one of those was erlang which lead my machine to crash a few times lol. And we had others report packages like TPM2.

So what is the fix?

My proposal is that we figure out some kind of workflow that similar to upstream nixpkgs has a staging step. The biggest issue though with this is keeping track of what commit we should track as the “staging” commit since it would have to be a upstream commit until we get our own hydra and caching.

Some background reading

1 Like

disclaimer: Not being to horribly familiar with how upstream nixpkgs does this, I’m just wondering if we can tap into the output of staging? With the intention - due to lack of aux infrastructure - of just skipping over the problem for now.

The other solution (indirectly included in my words) could be to provide aux as a pre and post upstream staging variant (for now). I’m not sure how much work that would be, see my initial disclaimer. Plz discard idea if unreasonable.

The issue with that is how many commits we have on top of master. Since if we are just “tapping into staging” we would be losing our commits. But the idea is not completely dead, I shall do some reading into how upstream handles staging and hopefully we can match their release schedule.

2 Likes

So after taking a look. We are NOT going to have a unstable release today so we can fall 1 day behind on master. Why you might be thinking. Well in nixos/nixpkgs the staging branch is roughly 1 day behind master and then nixos-unstable and nixpkgs-unstable are roughly 1 day behind staging (2 days behind master).

In theory this means that we can piggyback upon upstream’s binary cache for that bit longer.

7 Likes