Our new plan going forewards

General
1

What does @minion’s recent announcement mean for us?

It is time to create a new and more in depth plan for the time between and including the creation of our proof of concept repos and getting a partially usable system.

Significant feedback from @committee_security is requested, as security will be a significant concern throughout this time.

Bellow is my sample plan and some important parts of each step. These steps are hardly final and will need a lot of feedback, so please feel free to add your plans and ideas.

  1. Decide the layout of the repos, will we be using the skeleton repo by @Jeff or shall we continue with the layout of our current proof of concept repos e.g. GitHub - auxolotl/javascript.

  2. Create a solid set of rules.

    • In the matrix chat I suggested something similar to brew’s limitations (GitHub repo: <30 forks, <30 watchers and <75 stars), the limitations should be platform dependant of course.
    • We should also require update scripts, or create a larger program that can handle this for us, that could run by CI, or manually by a user.
    • Yearly maintainer audits?

  3. Start packaging:

    1. major packages. e.g. nodejs, cargo, gnome and the kernel
    2. notable packages. e.g. niri, gum and jq
    3. widely used but not so important packages. e.g. catppuccin-gtk and paprius icons. No package here should be a blocker from moving onto step 4, and should allow us to start work on step 4 whilst this is still ongoing

  4. The clean up stage.

    • At this point we should have a working VM where most functions a daily driver are usable.
    • Ensure cross repo standards are met
3 Likes
2

In the matrix chat I suggested something similar to brew’s limitations (GitHub repo: <30 forks, <30 watchers and <75 stars), the limitations should be platform dependant of course.

What would this kind of limitation limit? Whether or not the software is allowed to be packaged for Aux?

3

I guess the idea is to only devote limited dev/maintainer resources to sufficiently notable software; the brew limitations @isabel referred to seem to be here: brew/docs/Acceptable-Casks.md at master · Homebrew/brew · GitHub

3 Likes
4

Yes. But also to answer why. One of nixpkgs biggest issues is maintainability, you can see on their repology, nixpkgs unstable repository information - Repology, not an insignificant amount of packages are out of date. And this can be a significant security risk with some packages and a large struggle for keeping packages up to the latest.

3 Likes
5

This is totally understandable! It just wasn’t clear to me what you meant, wording was not explicit about that. :slight_smile:

1 Like
6

Personally, I think the proof of concept repositories are a better path - starting with the nixpkgs-skeleton is just going to lead to the same problems as nixpkgs down the line.

I do think there’s a better way to accomplish this than flakes - in the longer term maybe we/lix can deal with the limitations that prevent IFD in upstream nixpkgs. But in the short term flakes are probably still better than git submodules.

I had been doing some early work on an alternative to search.nixos.org - I think it would be helpful to decide on a schema for meta attributes early on (even if that’s just writing down the upstream one), possibly using something like yants to check it.

5 Likes
7

Haven’t posted much about it yet, but working on a proof of concept for core here: add stdenv by vlinkz · Pull Request #2 · auxolotl/core · GitHub.
Still isn’t able to build nix, but basics like stdenv, python3, and bash build. Right now I’m building on an idea similar to nixpkgs by-name, but with a few key changes that allow for packaging hopefully all packages under this structure. From my pr:

by-name
├── gd
│   ├── gd
│   │   ├── default.nix
│   │   └── packages.nix
│   └── gdbm
│       └── default.nix
...

In this case, package gdbm will be evaluated as gdbm = callPackage ./by-name/gdbm { }
Packages with a packages.nix file, such as gd will be imported as import "${path}/packages.nix" { inherit lib noSysDirs config overlays; } res self super. This idea allows for most (all?) packages and package sets to be placed in by-name.

Now that some stuff is in a working state, I’d appreciate any feedback on the structure and implementation. You can test with nix build github:auxolotl/core/stdenv#auxPackages.x86_64-linux.stdenv or any other package (still lots of missing dependencies for some).

3 Likes
8

Happy to help!
The current proposal does not seem to touch on security topics (yet I guess), are there any questions and/ or clarifications I can help with right now?

1 Like
9

Without “picking” the structure I proposed. I think it would be good to simply start bootstrapping from the bottom up, like @VlinkZ is doing.

We are explorers in uncharted territory. We can propose layouts to guide our next-step. But in terms of “committing” or “deciding”, I think we should only commit to one step and not a whole layout.

We build lib, then core, then try to design maintainer repos (like the existing JavaScript repo). Each sig team can have the goal of trying to get their main package (like rustc for rust) working using only aux core to both get an idea of the complexity and get an idea of “who goes first” because some sig teams might effectively depend on another sig team finishing their work.

That ^ is probably going to change peoples perspectives (including me) on things. Will some of the sig teams need to use nixpkgs as a crutch, can we even use nixpkgs as a crutch? (Will we be able to overlay core, or will it just be overlaying the after-core stuff, if we only overlay after-core stuff is that going to be a problem, etc).

We can iterate from there with security and testing. For example, trying different designs for automating testing, and trying designs that let us patch projects with security fixes without rebuilding everything, and trying to do an overlay on nixpkgs and seeing if its viable or if we hit tons of problems.

5 Likes