Tidepool eval performance: where PR #40 leaves us

General
1

Hi all,

Following up on the evaluation speed question raised in To Module Or Not, That Is The Question back in May 2024, now that some work has landed against it.

I had a look through the merged PR #40 (“refactor: eval performant target”) along with the refactor/fixed-targets and refactor/use-platform-specs branches, and the current tidepool/README.md from PR #18. From what I can tell, #40 is a targeted fix rather than the structural fixed-point redesign discussed in that original thread: .extend isn’t memoised in Nix, so several foundation stage1 packages (bash, binutils, gcc and others) were being re-evaluated from scratch by every dependent that called .extend on them (binutils alone reportedly went from 139 evaluations down to 4). The fix adds a -passthrough version file per affected package that calls .extend once, with other call sites referencing that instead.

That’s a genuinely useful win, but as far as I can see it’s currently scoped to src/packages/foundation/, and the broader modules-vs-fixed-points question doesn’t look fully resolved yet. refactor/fixed-targets (already merged) turned out to be about collapsing the build.host.target triple to build.host, which is unrelated to that discussion; refactor/use-platform-specs is unmerged and its branch looks stale relative to current main, so I didn’t want to assume it’s still active without a maintainer confirming that.

Given that, I wondered whether a profiling pass might be useful before any further structural changes, partly to establish what the .extend fix in #40 has actually bought, and partly to see whether the same re-evaluation pattern exists anywhere it hasn’t been patched yet (src/packages/aux/ and src/packages/core/, including Lix, look like they haven’t had the same treatment). Roughly:

  • Isolating nix eval from build time, with NIX_SHOW_STATS=1 or similar, on a few packages of varying dependency depth
  • Comparing a package evaluated once vs. twice in the same process, to check whether the same un-memoised .extend pattern shows up outside foundation/
  • Plotting eval time against package-set size to see whether it’s flat per-package now or still creeping up

If that sounds useful, I’d be happy to put together a small benchmark fixture to support it. One thing I noticed while looking around is that there doesn’t currently seem to be any CI in the labs repo, so an eval-time check on each PR would need that as a prerequisite — happy to help scope that too, if it’s wanted.

Please do correct me if I’ve misread the current state of any of this.

Thanks,

Chris

2

Hey Chris! Performance is a tough thing with the constraints of the module system. The real realization was “pinning” the many variants of a package to an accessible attribute, allowing Nix to properly memoize. This change was done a while ago now and had to do with the underlying package module implementation itself (the branches you mentioned are at a higher level and trying to deal with the scripting of builds). Are you still noticing eval time issues on Tidepool main? It should be very fast to eval, at least it is on both my desktop and laptop.