Ramblings on dependency management
By now most languages come with their own dependency manager which is either provided directly by the language toolchain (Rust’s crates) or is provided by 3rd parties but have become so embedded in the community that they can rightfully call themselves the standard (Apache Maven for Java and npm for JavaScript).
The dependency management is, depending on the language, one of the most-used tools after the compiler/interpreter and as such, its importance should not be underestimated. I have tried loving Haskell a few times now, but it’s dependency management – my personal hell – turned it into an immediate pass. I will only subject myself to this if you pay me for it, not in my free-time.
A bad example
Elm enters the ring. Elm is… opinionated. That’s not a bad thing at all. It’s dependency management sure is though. It combines the worst from all other dependency managers into one solution that feels only slightly less clunky than C’s “let the system package manager handle it”.
This is by no means meant as an attack. Elm largely was a one-man-show started and run by Evan Czaplicki who wrote the transpiler and the entire ecosystem. So it’s only prudent to cut him some slack here. Unfortunately, with his disappearance, Elm has quickly died down. While I would love for this to change (shoutout to Lamdera), I don’t currently see a wider movement that addresses the languages stagnation, and as such I have my doubts1.
Super-centralisation
Elm has the Elm package registry. The Elm package registry only accepts GitHub, a platform known for not being Open-Source and using user’s code to train AI models while gleefully disregarding licensing terms2. Would you prefer to have your projects hosted somewhere else? Sucks to suck. GitHub or GitOut.
But we can just add the dependency using a link to the repo, right? Just like
with cargo
or npm
. Right? The answer is a resounding no. You just can’t.
Add to that the risk of the Elm registry “disappearing” for whatever reason, and this is a recipe to make the language full-on unusable over night with the only remedy being someone else setting up their own package registry and distributes their own modified elm binaries. Of course this also means no self-hosted private registries for enterprises.
No integrity check
Jeroen Engels, Co-Host of the Elm Radio podcast, recently informed me that “Elm
does have [integrity checks]”3. While it does in fact verify that the
uploaded version of the package matches the one published on GitHub, there is
no verification of the download that I could find in the Install command4. I
may be paranoid, but this alone would likely disqualify it from being used in
sensitive environments like medical software or in the military sector (the
latter isn’t too bad of a trade-off). Just seeing a *.lock
file or a checksum
raises my confidence that the builds are actually reproducible.
On this, I was mistaken. The install
subcommand does not install the package
at all, it’s the compiler itself that fetches packages and verfies them. Thank
you to Jeroen for pointing that out.
Use SemVer!
One of the de-facto standards of software development that has recently come under scrutiny5, is SemVer. While a number of alternatives like CalVer exist, they are unsupported by Elm. This seems like not much of an issue, but that also means:
no development packages
Elm requires packages that are uploaded to it’s registry to be on a stable version. This entails that distribution of development packages has to happen through Subtrees/-modules which overall just feels incredibly hacky (and not the fun kind where you make your microwave play music).
This is annoying, especially if you want to “evolve” the package that might be useful for others alongside the development of a private project or simply feel like a different versioning schema fits your code better.
No maintenance tools
Oh no! Your package has a glaring security issue and there is no way anyone should use it? Hope your users are following you on your social media, you actually make a post about it, and they read it.6 Because the Elm registry is a dead-drop. You can’t retract versions, no way to mark them as vulnerable, the only way to update the description is releasing a new version.
Elm packaging feels more like using a hand-grenade: pull the pin, throw, and be done with it. Too bad, that that is not always how it should work.
No separated namespaces
Remember C not having namespaces? Elm has that too! Take the excellent
NoRedInk/elm-json-decode-pipeline
which just defines a subpackage under Json.Decode
, which is usually defined
in the “standard library”. While the care and skill of Elm developers ensured
that I have not yet encountered issues like conflicting packages, it is not at
all impossible or unlikely, if the language suddenly reanimates.
Honourable mention: No Licensing information
Oops, the package you’re relying on is GPL’d… you definitely clicked your way
to the source repository before using it in production, right? While this is
mainly a problem with the central registry, it remains a potential
trouble-maker and a legal landmine.
The issue here is caused by two things:
The package registry does not show itThe license isn’t even part of the package information
Thanks to Jeroen (again), for pointing that out. This field was merely absent in the default elm.json for applications. This was a mistake.
Introducing: mpn – Moritz’ Packaging Network
Let’s imagine how I would change Elm’s package manager if given the keys to the kingdom and disregarding all backwards compatibility.
Edit: actually not too bad, compatibility-wise.
Lessons learnt from other languages
When looking at languages, it’s always a nice idea to copy someone else’s homework. If it’s a problem, it has likely been solved by someone else already, so why not use that to form our own solution.
Why it should probably have a registry
Ideally, we would take a bite from Go’s packaging where one simply imports a git repo, but this has some issues, as soon as we add package caching and the possibility of disappearing repositories. On the bright side: without cache, it would significantly reduce the traffic associated with running the language infrastructure and there would be no reason to always specify the package registry to use.
Moving the Elmers(?) over to a repository base approach seems a bit idealistic though. If centralised package managers have shown one thing, then that they are a popular solution. I personally can not empathise, but it would be strange to deny reality here where significantly smarter people than me have opted for registries.
Okay, but at least have repositories as first-class citizens
One thing that I would like to have is the ability to just embed a git repo. It’s great for developing packages and helps in making the internet just a bit less centralised again.
Having it cached locally, would also allow for quick checks for updates. They are just one git-pull away. Also makes authentication easier for private dependencies. It automatically uses git’s authentication procedures.
Just add a remote file
While we’re at it: One thing zig does, which is quite nice is allowing to import archives from the web. While this has the vibes of the guy in a tan trenchcoat in a dimly-lit alleyway approaching you with “I got some prime dependencies for you”, it is great if an older version can only be found on archive.org.
Adding a checksum
Having different remotes, requires the presence of another kind of checksum.
I’ve opted for hashing the output of git archive --format tar.gz
in the below
example.
Allow for custom registries
Setting up your own registry should be possible, at the very least. Relying on a third party is not ideal, and in some cases the inability to setup your own registry could be a dealbreaker for some organisations that emphasize tight code controls.
Should Codeberg decide to run their own registry, it would be crazy not to embrace them.
Package maintenance through a file
Having a simple file in your project root that contains the package metadata has some advantages: it’s automatically versioned, mistakes an quickly be remedied, and it can be used as a common standard across indices and when directly accessing repos. As for disadvantages: it’s cluttering the directory. Not exactly a huge disadvantage in my books.
Format-wise, a normal data serialisation format would be fine. No need to be like Go and make up a new format.
Import-Aliases
What’s a nice aspect of packages from central repositories? Clearly the short names! With multiple dependencies and potential for conflict because multiple repositories define the same name, this is an issue that can be addressed with import aliases. Just ensure that the namespace is unique and you have got yourself a solution.
The new Elm packaging system
Now, with all that we have learnt combined, we can setup “mpn” as Elm’s new dependency management. Which has significantly increased complexity, in exchange for a long list of features.
elm.json
With all our new changes, it’s unavoidable that the main dependency file grows a bit. I would argue that it remains readable. I have taken the liberty of indenting the file with tabs.
{
"type": "application",
"source-directories": [
"src"
],
"elm-version": "0.20.0",
"dependencies": {
"direct": {
"ElmJsonDecodePipeline": {
"version": "1.0.1",
"package-index": "elmpkg",
"name": "NoRedInk/elm-json-decode-pipeline",
"checksum": "SHA256:92524c7cd931b547b2bf39bc7ce2488b0d66ddff517889307bab2603a3c80552"
},
"OurCoolProprietaryLibrary": {
"version": "1.0.0",
"package-index": "private-index",
"name": "our-cool-proprietary-library",
"trust-remote-hashes": true
},
"Elm": {
"subpackages": {
"Browser": {
"version": "1.0.2"
},
"Bytes": {
"version": "1.0.8"
},
},
"default": {
"package-index": "stdlib",
"trust-remote-hashes": true
}
},
"Round": {
"version": "fe126fed",
"source": "git:git@github.com:myrho/elm-round.git",
"checksum": "BLAKE2:a5c065462278451df90447d848b6d6bd523425d01e78da2d2dcb800cf7eeeb55fdbf9a553a9518ba538a491e6273e3b2b3857a87ab953834f4c18de51b5463c3"
},
"Colors": {
"version": "",
"source": "https://git.sr.ht/~mpldr/cie-color-diff.elm/archive/1.0.0.tar.gz",
"checksum": "SHA256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
},
},
},
"indirect-dependencies": {
// omitted for brevity. No namespaces defined.
},
}
Packages can be installed using
elm install NoRedInk/elm-json-decode-pipeline
elm index add private-index elm.pkgs.dev && elm install --from private-index our-cool-proprietary-library
elm install --stdlib Browser Bytes
elm install --git git@github.com:myrho/elm-round.git
(uses latest tag or currentorigin/HEAD
)elm install --as Colors https://git.sr.ht/~mpldr/cie-color-diff.elm/archive/1.0.0.tar.gz
For using the packages, the import path is simply prefixed with their import alias:
module Archiv exposing (main)
-import Browser exposing (..)
-import Browser.Navigation exposing (..)
+import Elm.Browser exposing (..)
+import Elm.Browser.Navigation exposing (..)
If the user for whatever reason decides to make the import path ambiguous, it is also within their power to easily resolve that conflict by either changing the import alias or by renaming their conflicting file(s). The import alias itself can be auto-generated from the last path-element or a preferred import alias can be provided in the package-definition file.
Package metadata
If you wish to publish your package, you can add some metadata to a file called
elm-package.toml
. Why TOML? Mainly because of multi-line text in
descriptions.
[package]
name = "CIE-color-diff.elm"
description = """
allows diffing colors
while hiding the science behind it
"""
recommended_versions = [
"v1.5.2",
"v2.0.4"
]
recommended_alias = "Colors"
licenses = [
"AGPL-3.0-or-later",
"CC0-1.0"
]
[version: 2.0.3]
retracted = true
retraction_reason = "Syntax-Error in Color.elm makes package uncompilable"
[version: 1.5.1]
known_vulnerabilities = [
"CVE-2025-XXX"
]
This can be fetched from package registries or git repositories at a regular interval and users can be informed about known vulnerabilities or if they are using a retracted version.
Things I did not touch upon
While these are quite a few drastic changes, there are still more things to consider that simply don’t make sense to implement short-term.
Attestation
Being able to review code and publish your results is an obvious benefit for the security of the ecosystem as a whole. Make it a signed file in a repository and it can easily be integrated with registries or a subcommand to check for audits. Trust would have to be established manually, but that to me seems like a feature. I wouldn’t want a review by user1968431 to have the same weight as one by Google.
How to store artifacts/sources
I don’t like the approach of throwing dependencies in a subdirectory like
node_modules
. I do see the value in vendoring dependencies if one wishes to
do so, but having this as a default that is essentially never checked into
version control seems like a waste. And that’s without even touching on the
extra storage required. Go and Rust seem to use the user-directory, which seems
like a sensible place to put them. Might I suggest $XDG_CACHE_HOME
?
if the Elm community could prove me wrong here, I would be very happy. ↩︎
I understand that this is an ongoing discussion and philosophical debate, but it might’ve been a good idea by M$ execs to address those in advance. Would’ve made a dent in the profits though, so of course we can’t be bothered by small things like license terms. ↩︎
https://fosstodon.org/@jfmengels@mastodon.cloud/114686864070044878 ↩︎
https://github.com/elm/compiler/blob/2f6dd29258e880dbb7effd57a829a0470d8da48b/terminal/src/Install.hs ↩︎
At least I assume that the average Elm dev isn’t subscribed to too many mailing lists. ↩︎