Prefix.dev Blog

You might be interested in Pixi: https://prefix.dev/ It uses uv under the hood for Python dependencies, while allowing you to also manage Conda dependencies in the same manifest (pixi.toml). The ergonomics are really nice and intuitive imo, and we're on our way to replace our Poetry and Conda usage with only Pixi for Python/C++ astrodynamics projects. The workspace-centric approach along with native lockfiles made most of our package management issues go away. I highly recommend it! (Not affiliated anyhow, other than contributing with a simple PR for fun)

Can recommend using conda, more specifically mambaforge/micromamba (no licensing issues when used at work). This works way better than pip, as it does more checks/dependency checking, so it does not break as easily as pip, though this makes it definitely way slower when installing something. It also supports updating your environment to the newest versions of all packages. It's no silver bullet and mixing it with pip leads to even more breakages, but there is pixi [0] which aims to support interop between pypi and conda packages [0] https://prefix.dev/

There is also pixi, which is evolving really fast. Brings the best of conda and looks like a integrations with pypi is ready to be released. https://prefix.dev/

My completely unvarnished thoughts, in the hope that they are useful: I had one JIRA-ticket-worth of stuff to do on a conda environment, and was going to try to use pixi, but IIRC I got confused about how to use the environment.yml and went back to conda grudgingly. I still have pixi installed on my machine, and when I look through the list of subcommands, it does seem to probably have a better UX than conda. When I go to https://prefix.dev, the "Get Started Quickly" section has what looks like a terminal window, but the text inside is inscrutable. What do they various lines mean? There's directories, maybe commands, check boxes... I don't get it. It doesn't look like a shell despite the Terminal wrapping box. Below that I see that there's a pixi.toml, but I don't really want a new toml or yml file, there's enough repository lice to confuse new people on projects already. Any time spent educating on packaging is time not spent on discovery, and is an impediment to onboarding.

We're working hard to get ROS out of dependency hell - https://prefix.dev/blog/reproducible-package-management-for-robotics Would love to hear your thoughts.

Hah, your comment persuaded me to look at that page, and honestly I also can't tell what it even is. I think it's supposed to be an alternative to `mise` and `asdf`, but it mostly mentions various Python tools? And doesn't seem to have an overview of what's available to install through Pixi? Then I clicked a link and got to https://prefix.dev/ ... > pixi is a fast software package manager built on top of the existing conda ecosystem. Spins up development environments quickly on Windows, macOS and Linux. Oh, build on top of conda. I am so going to stay the hell as far away from that as possible!

Try Pixi (https://prefix.dev/). It uses uv for Python while also managing your other libraries from Conda. It has a migration path from Conda.

There is also Pixi from the same developer group (http://prefix.dev). It still uses the Conda ecosystem but it provides a Nix shell-esque environment as well as lock files. It basically maximizes the reproducibility of a development environment without the need for containers. No need for crazy remoting plugins in your IDE, or devcontainers. Pixi is also written in Rust so it should be at a similar performance as Mamba.

I'm happy to see uv is adopted in Pixi, which is a new personal favorite: https://prefix.dev/blog/uv_in_pixi Reasons for liking pixi, over e.g. poetry: - Like poetry, it keeps everything in a project-local definition file and environment But unlike poetry, it also: - Can install python itself, even ancient versions - Can install conda packages - Is extremely fast

Pixi [1]. A lot of people recommend UV, which pixi uses under the hood to install pip packages [2]. The difference between UV and pixi is that pixi is better tooling for working with Conda repositories, and UV focuses in working with pip packages. Pixi allows installing not only pip packages but anything that has been packaged for Conda, which includes a lot of non-python stuff (C/C++ libraries and more). For instance, I use it to install the Go compiler per-project, since using a global installation tends to grow "polluted" with every single go package that you have ever installed. You could even use it to install Node, NPM or ruby per-project, it just works. It also supports lock files [3]. -- 1: https://pixi.sh/latest/ 2: https://prefix.dev/blog/uv_in_pixi 3: https://pixi.sh/latest/features/lockfile/#why-a-lock-file

For conda there is mamba [0], a drop-in replacement that's really fast. By the way, the creator of mamba started his own company at https://prefix.dev/ They want to essentially leverage the conda(-forge) infrastructure to build a new cross-platform, cross-language, cargo-like package manager: pixi [0] https://github.com/mamba-org/mamba

Cool! We're also working on improving conda environments and the interaction with them. If you would like to have a talk find us through https://prefix.dev

For as much improvement as there has been with what can be distributed via PyPI, there are still some domains that have gnarlier dependencies than wheels happily handle alone, and you either need to reach for the system package manager (and loose the ability to really control the dependency environment from that mismatch), or take advantage of the Conda ecosystem. My org does a lot of work combining machine learning with oceanographic and climate modeling, which are both domains that have deep dependency chains that don't always mesh well, especially as our researchers mix in R and other languages as the same time, and the Conda ecosystem helps us a ton with that, but there are issues that `conda` and `mamba` don't help us out with. Pixi takes a swing at some of what the Conda ecosystem hasn't been great at (at least without a lot of manual custom ceremony) that Cargo, Poetry, Pipenv, PDM, and other dependency and workflow management tools have demonstrated can be done such as lock files, cross platform dependency management, task running, and defining multiple related environments. What's really cool when you have a mix of projects, Pixi can work almost entirely PyPI native out of a `pyproject.toml`, other than installing Python from Conda-Forge, so you can mix and match environments but stay with the same tool. https://prefix.dev/blog/using_python_projects_with_pixi docs: https://pixi.sh/latest/advanced/pyproject_toml/

Pixi uses uv for resolving pypi deps: https://prefix.dev/blog/uv_in_pixi If you look closely, pixi used `resolvo to power `rip` then switched from a `rip` solver to a `uv` solver

How about prefix then? https://prefix.dev/blog/uv_in_pixi

prefix-dev/pixi is a successor to Mamba and Conda which wraps libsolv like dnf. https://github.com/prefix-dev/pixi OpenSUSE/libsolv: https://github.com/openSUSE/libsolv OpenSUSE/zypper may have wrapped or may still wrap libsolv? TIL libsolv also supports arch .apk and .deb; there's a deb2solv utility to create libsolv .solv files: https://github.com/openSUSE/libsolv/blob/master/tools/deb2solv.c "[mamba-org/rattler:] A new memory-safe SAT solver for package management in Rust (port of libsolv)" (2024) https://prefix.dev/blog/the_new_rattler_resolver https://news.ycombinator.com/item?id=37101862 Pixi is integrating uv: https://github.com/astral-sh/uv/issues/1572#issuecomment-1949979306 astral-sh/uv [1] solves versions with PubGrub [2], which was written for Dart: [1] https://github.com/astral-sh/uv [2] https://github.com/pubgrub-rs/pubgrub uv's benchmarks are run with hyperfine, a CLI benchmarking tool: https://github.com/sharkdp/hyperfine astral-sh/rye was mitsuhiko/rye, and rye wraps uv with "fallback to unearth [resolvelib] and pip-tools": https://github.com/astral-sh/rye https://lucumr.pocoo.org/2024/2/15/rye-grows-with-uv/ So, TIL it looks like rye for python and pixi for conda (python, r, rust, go,) are the latest tools for python packaging, not apt packaging. fpm is one way to create deb packages from python virtualenvs for install with the new APT solver, which resulted in this research.