On the future of our nixpkgs fork

[getchoo]

well…this has got a lot of activity in the last day

i would rather not spam a ton of replies, so this is just going to be one big response to most of the feedback here. bear with me

@minion

I think Aux, and therefore the roadmap, started in a panicked time when we genuinely thought that the foundation’s response would cause a decent amount of nixpkgs contributors to pack up and leave there.

i share a similar sentiment. a lot of the ideas being thrown around feel like a knee jerk reaction to the drama, and not exactly a calculated response. i think we can learn from lix here

@isabel

We start rebranding as an alternate nixpkgs provider and thus os

i think this is a really good way to put this, and it could work well in our future collaboration with lix. we could easily have them be the alternative cppnix implementation, while we provide the alternative nix{pkgs,os}

@Nairou

But we’d still have the synchronization issue with nixpkgs even after that, right?

to an extent, yes. i think this will be lessened though, as we should ever attempt to reach the same size as nixpkgs. personally, i imagine that us kinda building a “new” nixpkgs from the ground up can give us a good opportunity to see what’s really needed in the repository. this would gives us an easier time in maintenance, allow for faster iteration with things like staging merges, less duplicated work for niche packages, and in general a better experience for users with more packages being better maintained. some prior art in this aspect would be a distro i use to work on a bit, solus. they have (or had? i sadly haven’t kept up with them in a while) an interesting policy of new package inclusions also requiring a justification for it being added

@Nairou

Instead of blindly pulling the entirety of nixpkgs into a second repo, we’d instead have to piecemeal updates from certain packages in nixpkgs into the appropriate aux repo.

now assuming we don’t try to keep consistent parity with nixpkgs (and have less packages in general), i don’t imagine would be that big of a deal. i would also find the duplicated effort here a bit more worth it, as we’re doing something completely new – not just giving users yet another monorepo

@Nairou

Will the development of this sorting tool hold up usability of package availability?

this is again assuming we will just pull all changes from nixpkgs. i think this would be a misstep

for changes we do pull? i think many could be manual, or just done with a r-ryantm equivalent

@Nairou

Will new packages in nixpkgs remain unavailable until an appropriate aux repo has been selected for it?

this again assumes all new packages added to nixpkgs will likewise be added to aux’s repositories, so in assuming they aren’t as i said before, this would in a way be true

for packages that are accepted to aux, i don’t think choosing an appropriate repo would take much time. something in pythonPackages for example will just go into auxolotl/python 9 times out of 10

@Nairou

Would aux benefit from a more gradual change, where packages slowly get migrated out of auxolotl into individual repos?

i don’t think this is a very good idea, as if we do this for long enough i would be concerned about the effort required to move away from it – especially as new packages are added. this could easily create an ever moving goalpost where each time we move packages, we will have more added that will need to be moved later on as well

Could even do something like AUR in Arch, where the entirety of nixpkgs is in a “community” repo

i have a lot of opinions on the aur and many of it’s issues (as an arch user for many years and still an active aur maintainer), but for our purposes the only one i will mention is moderation. i would much rather be proactive in avoiding potential issues with poor quality or genuinely malicious packages

i would much rather see the current nixpkgs approach taken in each repository, where anyone can contribute or maintain a package, but these changes do need to be approved by a committer (a SIG member in our case) ahead of time

@srxl

Do we want to be “Nixpkgs compatible”? Or do we want to be our own thing entirely? Personally, I don’t think striving for compatibility is a useful goal, since Nixpkgs is a decently fast moving target. Less fast on versioned releases, sure, but it’s still a target that we don’t have control over, and will very heavily influence our decisions if we choose to target it. Plus, if we are just a Nixpkgs-compatible package set, then we need to ask “Why should someone choose us over Nixpkgs”? We need to have some distinguishing factor from Nixpkgs that people look at and say “Oh, that’s interesting, I want to try this instead”.

i completely agree here. i would also like to note that even if we aren’t directly nixpkgs compatible, it’s not like packages from either could be mixed and matched with overrides or package options in modules

@srxl

Bring in Nixpkgs as a dependency of Core, and use that to allow other SIGs to bootstrap their own work

i am a bit iffy on this, as it could easily put a pretty massive burden on core if dependent SIGs are overzealous in what they use from nixpkgs. i can get how the current synchronous approach of core → language SIGs → broader/DE SIGs → top-level can be annoying for those who need to just wait on getting their work start, though. maybe a good solution here is to adopt nixpkgs as a dependency like you said, but ensure that all packages used must be in the build closure of the standard install iso – as based on the roadmap i originally proposed here, that should be the real benchmark of a beta release for example

@srxl

Eventually, Core drops their nixpkgs dependency, and we become our own completely isolated project and package set

but yeah assuming this happens, i would be find with nixpkgs being a dependency for a bit. i would just make sure this can easily happen ofc

@VlinkZ

I also believe that in the time being, even once we have a functioning fully aux system, having a nixpkgs export from core may be a good idea (although using flakes a user could just add the input themselves).

i would be against this in color simply because it would mean nixpkgs itself would become a dependency of all other repositories, which would one again kinda defeat the point of moving away from a monorepo imo. maybe we could do this with top-level as that should be the repo people on an auxOS should be using…but like you said, users can just add the input themselves (with flakes or something like npins)

@VlinkZ

Infrastructure: If we go in the direction proposed and build up our package set gradually rather than by forking nixpkgs, I believe that work on infrastructure such as CI and a binary cache can start a lot sooner given the smaller scale

yeah, especially with us using a pure nixpkgs as a temporary base while core is worked on

@danderson

This makes me a little sad, because independence from nixpkgs is still a big goal of mine

this is still my goal here honestly. even though nixpkgs may be used as a temporary bootstrap tool, i would very much rather it not become a permanent dependency on our infrastructure – because then how are we really an alternative?

@danderson

I’m not looking forward to rebuilding nixpkgs from the ground up, but if that’s the long-term goal anyway, and nixpkgs endures despite my fears… then yeah, no sense in creating a clone of all that work and then also try to do new things, we should do just the new things.

and this will be hard, definitely; i think this is a good spot to learn from some of the possible mistakes of nixpkgs though. as i said before, creating a more curated experience with our repositories might be able to make a better experience for both users and developers. the aux i see in the future isn’t another 90,000 package large repository, but something more sustainable (and comprehensible) for a smaller community

@Melkor333

It’s just a few days now and despite the soft-fork being more effort, less reward than the hard-fork, it’s a statement

statements are important, i agree. where i disagree though is that a fork of nixpkgs itself is required for this. if anything, i find moving away from a monorepo entirely and separating concerns into smaller, but more independent communities (away from the control of a central group of committers) to be a much more impactful

@AngryAnt

The majority of the mentioned issues with the fork are tied to the execution - not to the goal.

i would argue much of the issues are actually inherit to the goal itself. having both a nixpkgs fork and individual repositories will always be a massive undertaking. likewise, directly inheriting work from nixpkgs upstream while also accepting PRs to our fork will always create a weird situation for contributors

@AngryAnt

As an example, a fork which only provides current stable branch, synced at some frequency actually sustainable for this project, is still an entirely valid fork.

valid? definitely. but useful? you would have a hard time convincing me at least

only having the current stable (23.11) is an instant no go for me, as i use unstable more often on my devices. this also removes the more fast tracked -small branches for those willing to possibly compile more, or the regular nixpkgs-unstable branch used by a vast majority of non-nixos consumers. having a singular branch that is still basically (only that branch of) nixpkgs but slightly outdated isn’t a very appealing offer

@AngryAnt

I understand the desire to always be the objectively superior choice to anything nix

this was never my goal honestly. even though i am very interested in the idea of individual repos for example, i wouldn’t consider it objectively better for a user at the end of the day. my desire is to be an alternative to nixpkgs that some may prefer, not a definite upgrade

and finally, thank you all for your feedback here! i'll admit i was a bit nervous about the response i may receive due to how drastic this change was -- and purposefully waited a bit for things to settle -- but the discussions here have been great. keep it up guys :)