Module cargo::core::resolver::encode

source ·
Expand description

Definition of how to encode a Resolve into a TOML Cargo.lock file

This module contains all machinery necessary to parse a Resolve from a Cargo.lock as well as serialize a Resolve to a Cargo.lock.

Changing Cargo.lock

In general Cargo is quite conservative about changing the format of Cargo.lock. Usage of new features in Cargo can change Cargo.lock at any time, but otherwise changing the serialization of Cargo.lock is a difficult operation to do that we typically avoid.

The main problem with changing the format of Cargo.lock is that it can cause quite a bad experience for end users who use different versions of Cargo. If every PR to a project oscillates between the stable channel’s encoding of Cargo.lock and the nightly channel’s encoding then that’s a pretty bad experience.

We do, however, want to change Cargo.lock over time. (and we have!). To do this the rules that we currently have are:

  • Add support for the new format to Cargo. This involves code changes in Cargo itself, likely by adding a new variant of ResolveVersion and branching on that where necessary. This is accompanied with tests in the lockfile_compat module.

    • Do not update ResolveVersion::default(). The new lockfile format will not be used yet.

    • Preserve the new format if found. This means that if Cargo finds the new version it’ll keep using it, but otherwise it continues to use whatever format it previously found.

  • Wait a “long time”. This is at least until the changes here hit stable Rust. Often though we wait a little longer to let the changes percolate into one or two older stable releases.

  • Change the return value of ResolveVersion::default() to the new format. This will cause new lock files to use the latest encoding as well as causing any operation which updates the lock file to update to the new format.

This migration scheme in general means that Cargo we’ll get support for a new format into Cargo ASAP, but it won’t be exercised yet (except in Cargo’s own tests). Eventually when stable/beta/nightly all have support for the new format (and maybe a few previous stable versions) we flip the switch. Projects on nightly will quickly start seeing changes, but stable/beta/nightly will all understand this new format and will preserve it.

While this does mean that projects’ Cargo.lock changes over time, it’s typically a pretty minimal effort change that’s just “check in what’s there”.

Historical changes to Cargo.lock

Listed from most recent to oldest, these are some of the changes we’ve made to Cargo.lock’s serialization format:

  • A version marker is now at the top of the lock file which is a way for super-old Cargos (at least since this was implemented) to give a formal error if they see a lock file from a super-future Cargo. Additionally as part of this change the encoding of git dependencies in lock files changed where branch = "master" is now encoded with branch=master instead of with nothing at all.

  • The entries in dependencies arrays have been shortened and the checksum field now shows up directly in [[package]] instead of always at the end of the file. The goal of this change was to ideally reduce merge conflicts being generated on Cargo.lock. Updating a version of a package now only updates two lines in the file, the checksum and the version number, most of the time. Dependency edges are specified in a compact form where possible where just the name is listed. The version/source on dependency edges are only listed if necessary to disambiguate which version or which source is in use.

  • A comment at the top of the file indicates that the file is a generated file and contains the special symbol @generated to indicate to common review tools that it’s a generated file.

  • A [root] entry for the “root crate” has been removed and instead now included in [[package]] like everything else.

  • All packages from registries contain a checksum which is a sha256 checksum of the tarball the package is associated with. This is all stored in the [metadata] table of Cargo.lock which all versions of Cargo since 1.0 have preserved. The goal of this was to start recording checksums so mirror sources can be verified.

Other oddities about Cargo.lock

There’s a few other miscellaneous weird things about Cargo.lock that you may want to be aware of when reading this file:

  • All packages have a source listed to indicate where they come from. For path dependencies, however, no source is listed. There’s no way we could emit a filesystem path name and have that be portable across systems, so all packages from a path are not listed with a source. Note that this also means that all packages with path sources must have unique names.

  • The [metadata] table in Cargo.lock is intended to be a generic mapping of strings to strings that’s simply preserved by Cargo. This was a very early effort to be forward compatible against changes to Cargo.lock’s format. This is nowadays sort of deemed a bad idea though and we don’t really use it that much except for checksums historically. It’s not really recommended to use this.

  • The actual literal on-disk serialiation is found in src/cargo/ops/lockfile.rs which basically renders a toml::Value in a special fashion to make sure we have strict control over the on-disk format.

Structs

Functions

Type Aliases