Ramblings on dependency management

By now most languages come with their own dependency manager which is either provided directly by the language toolchain (Rust’s crates) or is provided by 3rd parties but have become so embedded in the community that they can rightfully call themselves the standard (Apache Maven for Java and npm for JavaScript).

The dependency management is, depending on the language, one of the most-used tools after the compiler/interpreter and as such, its importance should not be underestimated. I have tried loving Haskell a few times now, but it’s dependency management – my personal hell – turned it into an immediate pass. I will only subject myself to this if you pay me for it, not in my free-time.

A bad example

Elm enters the ring. Elm is… opinionated. That’s not a bad thing at all. It’s dependency management sure is though. It combines the worst from all other dependency managers into one solution that feels only slightly less clunky than C’s “let the system package manager handle it”.

This is by no means meant as an attack. Elm largely was a one-man-show started and run by Evan Czaplicki who wrote the transpiler and the entire ecosystem. So it’s only prudent to cut him some slack here. Unfortunately, with his disappearance, Elm has quickly died down. While I would love for this to change (shoutout to Lamdera), I don’t currently see a wider movement that addresses the languages stagnation, and as such I have my doubts¹.

Super-centralisation

Elm has the Elm package registry. The Elm package registry only accepts GitHub, a platform known for not being Open-Source and using user’s code to train AI models while gleefully disregarding licensing terms². Would you prefer to have your projects hosted somewhere else? Sucks to suck. GitHub or GitOut.

But we can just add the dependency using a link to the repo, right? Just like with cargo or npm. Right? The answer is a resounding no. You just can’t.

Add to that the risk of the Elm registry “disappearing” for whatever reason, and this is a recipe to make the language full-on unusable over night with the only remedy being someone else setting up their own package registry and distributes their own modified elm binaries. Of course this also means no self-hosted private registries for enterprises.

No integrity check

Jeroen Engels, Co-Host of the Elm Radio podcast, recently informed me that “Elm does have [integrity checks]”³. While it does in fact verify that the uploaded version of the package matches the one published on GitHub, there is no verification of the download that I could find in the Install command⁴. I may be paranoid, but this alone would likely disqualify it from being used in sensitive environments like medical software or in the military sector (the latter isn’t too bad of a trade-off). Just seeing a *.lock file or a checksum raises my confidence that the builds are actually reproducible.

On this, I was mistaken. The install subcommand does not install the package at all, it’s the compiler itself that fetches packages and verfies them. Thank you to Jeroen for pointing that out.

Use SemVer!

One of the de-facto standards of software development that has recently come under scrutiny⁵, is SemVer. While a number of alternatives like CalVer exist, they are unsupported by Elm. This seems like not much of an issue, but that also means:

no development packages

Elm requires packages that are uploaded to it’s registry to be on a stable version. This entails that distribution of development packages has to happen through Subtrees/-modules which overall just feels incredibly hacky (and not the fun kind where you make your microwave play music).

This is annoying, especially if you want to “evolve” the package that might be useful for others alongside the development of a private project or simply feel like a different versioning schema fits your code better.

No maintenance tools

Oh no! Your package has a glaring security issue and there is no way anyone should use it? Hope your users are following you on your social media, you actually make a post about it, and they read it.⁶ Because the Elm registry is a dead-drop. You can’t retract versions, no way to mark them as vulnerable, the only way to update the description is releasing a new version.

Elm packaging feels more like using a hand-grenade: pull the pin, throw, and be done with it. Too bad, that that is not always how it should work.

No separated namespaces

Remember C not having namespaces? Elm has that too! Take the excellent NoRedInk/elm-json-decode-pipeline which just defines a subpackage under Json.Decode, which is usually defined in the “standard library”. While the care and skill of Elm developers ensured that I have not yet encountered issues like conflicting packages, it is not at all impossible or unlikely, if the language suddenly reanimates.

Honourable mention: No Licensing information

Oops, the package you’re relying on is GPL’d… you definitely clicked your way to the source repository before using it in production, right? While this is mainly a problem with the central registry, it remains a potential trouble-maker and a legal landmine.

The issue here is caused by two things:

1. The package registry does not show it
2. The license isn’t even part of the package information

Thanks to Jeroen (again), for pointing that out. This field was merely absent in the default elm.json for applications. This was a mistake.

Introducing: mpn – Moritz’ Packaging Network

Let’s imagine how I would change Elm’s package manager if given the keys to the kingdom and disregarding all backwards compatibility.

Edit: actually not too bad, compatibility-wise.

Lessons learnt from other languages

When looking at languages, it’s always a nice idea to copy someone else’s homework. If it’s a problem, it has likely been solved by someone else already, so why not use that to form our own solution.

Why it should probably have a registry

Ideally, we would take a bite from Go’s packaging where one simply imports a git repo, but this has some issues, as soon as we add package caching and the possibility of disappearing repositories. On the bright side: without cache, it would significantly reduce the traffic associated with running the language infrastructure and there would be no reason to always specify the package registry to use.

Moving the Elmers(?) over to a repository base approach seems a bit idealistic though. If centralised package managers have shown one thing, then that they are a popular solution. I personally can not empathise, but it would be strange to deny reality here where significantly smarter people than me have opted for registries.

Okay, but at least have repositories as first-class citizens

One thing that I would like to have is the ability to just embed a git repo. It’s great for developing packages and helps in making the internet just a bit less centralised again.

Having it cached locally, would also allow for quick checks for updates. They are just one git-pull away. Also makes authentication easier for private dependencies. It automatically uses git’s authentication procedures.

Just add a remote file

While we’re at it: One thing zig does, which is quite nice is allowing to import archives from the web. While this has the vibes of the guy in a tan trenchcoat in a dimly-lit alleyway approaching you with “I got some prime dependencies for you”, it is great if an older version can only be found on archive.org.

Adding a checksum

Having different remotes, requires the presence of another kind of checksum. I’ve opted for hashing the output of git archive --format tar.gz in the below example.

Allow for custom registries

Setting up your own registry should be possible, at the very least. Relying on a third party is not ideal, and in some cases the inability to setup your own registry could be a dealbreaker for some organisations that emphasize tight code controls.

Should Codeberg decide to run their own registry, it would be crazy not to embrace them.

Package maintenance through a file

Having a simple file in your project root that contains the package metadata has some advantages: it’s automatically versioned, mistakes an quickly be remedied, and it can be used as a common standard across indices and when directly accessing repos. As for disadvantages: it’s cluttering the directory. Not exactly a huge disadvantage in my books.

Format-wise, a normal data serialisation format would be fine. No need to be like Go and make up a new format.

Import-Aliases

What’s a nice aspect of packages from central repositories? Clearly the short names! With multiple dependencies and potential for conflict because multiple repositories define the same name, this is an issue that can be addressed with import aliases. Just ensure that the namespace is unique and you have got yourself a solution.

The new Elm packaging system

Now, with all that we have learnt combined, we can setup “mpn” as Elm’s new dependency management. Which has significantly increased complexity, in exchange for a long list of features.

elm.json

With all our new changes, it’s unavoidable that the main dependency file grows a bit. I would argue that it remains readable. I have taken the liberty of indenting the file with tabs.

{
	"type": "application",
	"source-directories": [
		"src"
	],
	"elm-version": "0.20.0",
	"dependencies": {
		"direct": {
			"ElmJsonDecodePipeline": {
				"version": "1.0.1",
				"package-index": "elmpkg",
				"name": "NoRedInk/elm-json-decode-pipeline",
				"checksum": "SHA256:92524c7cd931b547b2bf39bc7ce2488b0d66ddff517889307bab2603a3c80552"
			},
			"OurCoolProprietaryLibrary": {
				"version": "1.0.0",
				"package-index": "private-index",
				"name": "our-cool-proprietary-library",
				"trust-remote-hashes": true
			},
			"Elm": {
				"subpackages": {
					"Browser": {
						"version": "1.0.2"
					},
					"Bytes": {
						"version": "1.0.8"
					},
				},
				"default": {
					"package-index": "stdlib",
					"trust-remote-hashes": true
				}
			},
			"Round": {
				"version": "fe126fed",
				"source": "git:git@github.com:myrho/elm-round.git",
				"checksum": "BLAKE2:a5c065462278451df90447d848b6d6bd523425d01e78da2d2dcb800cf7eeeb55fdbf9a553a9518ba538a491e6273e3b2b3857a87ab953834f4c18de51b5463c3"
			},
			"Colors": {
				"version": "",
				"source": "https://git.sr.ht/~mpldr/cie-color-diff.elm/archive/1.0.0.tar.gz",
				"checksum": "SHA256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
			},
		},
	},
	"indirect-dependencies": {
		// omitted for brevity. No namespaces defined.
	},
}

Packages can be installed using

• elm install NoRedInk/elm-json-decode-pipeline
• elm index add private-index elm.pkgs.dev && elm install --from private-index our-cool-proprietary-library
• elm install --stdlib Browser Bytes
• elm install --git git@github.com:myrho/elm-round.git (uses latest tag or current origin/HEAD)
• elm install --as Colors https://git.sr.ht/~mpldr/cie-color-diff.elm/archive/1.0.0.tar.gz

For using the packages, the import path is simply prefixed with their import alias:

 module Archiv exposing (main)
 
-import Browser exposing (..)
-import Browser.Navigation exposing (..)
+import Elm.Browser exposing (..)
+import Elm.Browser.Navigation exposing (..)

If the user for whatever reason decides to make the import path ambiguous, it is also within their power to easily resolve that conflict by either changing the import alias or by renaming their conflicting file(s). The import alias itself can be auto-generated from the last path-element or a preferred import alias can be provided in the package-definition file.

Package metadata

If you wish to publish your package, you can add some metadata to a file called elm-package.toml. Why TOML? Mainly because of multi-line text in descriptions.

[package]
name = "CIE-color-diff.elm"
description = """
allows diffing colors
while hiding the science behind it
"""
recommended_versions = [
	"v1.5.2",
	"v2.0.4"
]
recommended_alias = "Colors"
licenses = [
	"AGPL-3.0-or-later",
	"CC0-1.0"
]

[version: 2.0.3]
retracted = true
retraction_reason = "Syntax-Error in Color.elm makes package uncompilable"

[version: 1.5.1]
known_vulnerabilities = [
	"CVE-2025-XXX"
]

This can be fetched from package registries or git repositories at a regular interval and users can be informed about known vulnerabilities or if they are using a retracted version.

Things I did not touch upon

While these are quite a few drastic changes, there are still more things to consider that simply don’t make sense to implement short-term.

Attestation

Being able to review code and publish your results is an obvious benefit for the security of the ecosystem as a whole. Make it a signed file in a repository and it can easily be integrated with registries or a subcommand to check for audits. Trust would have to be established manually, but that to me seems like a feature. I wouldn’t want a review by user1968431 to have the same weight as one by Google.

How to store artifacts/sources

I don’t like the approach of throwing dependencies in a subdirectory like node_modules. I do see the value in vendoring dependencies if one wishes to do so, but having this as a default that is essentially never checked into version control seems like a waste. And that’s without even touching on the extra storage required. Go and Rust seem to use the user-directory, which seems like a sensible place to put them. Might I suggest $XDG_CACHE_HOME?