Introduction
Thank you for your interest in contributing to Cargo! This guide provides an overview of how to contribute to Cargo, how to dive into the code, and how the testing infrastructure works.
There are many ways to contribute, such as helping other users, filing issues, improving the documentation, fixing bugs, and working on small and large features.
If you have a general question about Cargo or its internals, feel free to ask on Zulip.
This guide assumes you have some familiarity with Rust, and how to use Cargo, rustup, and general development tools like git.
Please also read the Rust Code of Conduct.
Issue Tracker
Cargo’s issue tracker is located at https://github.com/rust-lang/cargo/issues/. This is the primary spot where we track bugs and small feature requests. See Process for more about our process for proposing changes.
Filing issues
We can’t fix what we don’t know about, so please report problems liberally. This includes problems with understanding the documentation, unhelpful error messages, and unexpected behavior.
If you think that you have identified an issue with Cargo that might compromise its users’ security, please do not open a public issue on GitHub. Instead, we ask you to refer to Rust’s security policy.
Opening an issue is as easy as following this link. There are several templates for different issue kinds, but if none of them fit your issue, don’t hesitate to modify one of the templates, or click the Open a blank issue link.
The Rust tools are spread across multiple repositories in the Rust organization. It may not always be clear where to file an issue. No worries! If you file in the wrong tracker, someone will either transfer it to the correct one or ask you to move it. Some other repositories that may be relevant are:
rust-lang/rust
— Home for therustc
compiler andrustdoc
.rust-lang/rustup
— Home for therustup
toolchain installer.rust-lang/rustfmt
— Home for therustfmt
tool, which also includescargo fmt
.rust-lang/rust-clippy
— Home for theclippy
tool, which also includescargo clippy
.rust-lang/crates.io
— Home for the crates.io website.
Issues with cargo fix
can be tricky to know where they should be filed,
since the fixes are driven by rustc
, processed by rustfix
, and the
front-interface is implemented in Cargo. Feel free to file in the Cargo issue
tracker, and it will get moved to one of the other issue trackers if
necessary.
Issue labels
Issue labels are very helpful to identify the types of issues and which category they are related to. The Cargo team typically manages assigning labels. The labels use a naming convention with short prefixes and colors to indicate the kind of label:
-
Yellow, A-prefixed labels state which area of the project an issue relates to.
-
Light purple, C-prefixed labels represent the category of an issue. In particular, C-feature-request marks proposals for new features. If an issue is C-feature-request, but is not Feature accepted or I-nominated, then it was not thoroughly discussed, and might need some additional design or perhaps should be implemented as an external subcommand first. Ping @rust-lang/cargo if you want to send a PR for such issue.
-
Dark purple, Command-prefixed labels mean the issue has to do with a specific cargo command.
-
Green, E-prefixed labels indicate the level of experience or effort necessary to fix the issue. E-mentor issues also have some instructions on how to get started. Generally, all of the E-prefixed labels are issues that are ready for someone to contribute to!
-
Red, I-prefixed labels indicate the importance of the issue. The I-nominated label indicates that an issue has been nominated for prioritizing at the next triage meeting.
-
Purple gray, O-prefixed labels are the operating system or platform that this issue is specific to.
-
Orange, P-prefixed labels indicate a bug’s priority.
-
S-prefixed labels are “status” labels, typically used for PRs, but can also indicate an issue is S-blocked.
-
The light orange relnotes label marks issues that should be highlighted in the Rust release notes of the next release.
-
Dark blue, Z-prefixed labels are for unstable, nightly features.
Process
This chapter gives an overview of how Cargo comes together, and how you can be a part of that process.
See the Working on Cargo chapter for an overview of the contribution process.
Please read the guidelines below before working on an issue or new feature.
Due to limited review capacity, the Cargo team is not accepting new features or major changes at this time. Please consult with the team before opening a new PR. Only issues that have been explicitly marked as accepted will be reviewed.
Cargo team
Cargo is managed by a team of volunteers. The Cargo Team reviews all changes, and sets the direction for the project.
The team meets on a weekly basis on a video chat. If you are interested in participating, feel free to contact us on Zulip.
Roadmap
The Roadmap Project Board is used for tracking major initiatives. This gives an overview of the things the team is interested in and thinking about.
The RFC Project Board is used for tracking RFCs.
Working on small bugs
Issues labeled with the E-help-wanted, E-easy, or E-mentor labels are typically issues that the Cargo team wants to see addressed, and are relatively easy to get started with. If you are interested in one of those, and it has not already been assigned to someone, leave a comment. See Issue assignment below for assigning yourself.
If there is a specific issue that you are interested in, but it doesn’t have
one of the E-
labels, leave a comment on the issue. If a Cargo team member
has the time to help out, they will respond to help with the next steps.
Working on large bugs
Some issues may be difficult to fix. They may require significant code changes, or major design decisions. The E-medium and E-hard labels can be used to tag such issues. These will typically involve some discussion with the Cargo team on how to tackle it.
Working on small features
Small feature requests are typically managed on the issue tracker. Features that the Cargo team have approved will have the Feature accepted label or the E-mentor label. If there is a feature request that you are interested in, feel free to leave a comment expressing your interest. If a Cargo team member has the time to help out, they will respond to help with the next steps. Keep in mind that the Cargo team has limited time, and may not be able to help with every feature request. Most of them require some design work, which can be difficult. Check out the design principles chapter for some guidance.
Working on large features
Cargo follows the Rust model of evolution. Major features usually go through an RFC process. Therefore, before opening a feature request issue create a Pre-RFC thread on the internals forum to get preliminary feedback.
Implementing a feature as a custom subcommand is encouraged as it helps demonstrate the demand for the functionality and is a great way to deliver a working solution faster as it can iterate outside of Cargo’s release cadence.
See the unstable chapter for how new major features are typically implemented.
Bots and infrastructure
The Cargo project uses several bots:
- GitHub Actions are used to automatically run all tests for each PR.
- triagebot automatically assigns reviewers for PRs, see Assignment for how to configure.
- bors is used to merge PRs. See The merging process.
- triagebot is used for assigning issues to non-members, see Issue assignment.
- rfcbot is used for making asynchronous decisions by team members.
Issue assignment
Normally, if you plan to work on an issue that has been marked with one of the
E-
tags or Feature accepted, it is sufficient just to leave a comment that
you are working on it. We also have a bot that allows you to formally “claim”
an issue by entering the text @rustbot claim
in a comment. See the
Assignment docs on how this works.
Working on Cargo
This chapter gives an overview of how to build Cargo, make a change, and submit a Pull Request.
- Before hacking on Cargo.
- Check out the Cargo source.
- Building Cargo.
- Making a change.
- Writing and running tests.
- Submitting a Pull Request.
- The merging process.
Before hacking on Cargo
We encourage people to discuss their design before hacking on code. This gives the Cargo team a chance to know your idea more. Sometimes after a discussion, we even find a way to solve the problem without coding! Typically, you file an issue or start a thread on the internals forum before submitting a pull request. Please read the process of how features and bugs are managed in Cargo.
Checkout out the source
We use the “fork and pull” model described here, where contributors push changes to their personal fork and create pull requests to bring those changes into the source repository. Cargo uses git and GitHub for all development.
- Fork the
rust-lang/cargo
repository on GitHub to your personal account (see GitHub docs). - Clone your fork to your local machine using
git clone
(see GitHub docs) - It is recommended to start a new branch for the change you want to make. All Pull Requests are made against the master branch.
Building Cargo
Cargo is built by…running cargo
! There are a few prerequisites that you
need to have installed:
rustc
andcargo
need to be installed. Cargo is expected to build and test with the current stable, beta, and nightly releases. It is your choice which to use. Nightly is recommended, since some nightly-specific tests are disabled when using the stable release. But using stable is fine if you aren’t working on those.- A C compiler (typically gcc, clang, or MSVC).
- git
- Unix:
- pkg-config
- OpenSSL (
libssl-dev
on Ubuntu,openssl-devel
on Fedora)
- macOS:
- OpenSSL (homebrew is recommended to install the
openssl
package)
- OpenSSL (homebrew is recommended to install the
If you can successfully run cargo build
, you should be good to go!
Running Cargo
You can use cargo run
to run cargo itself, or you can use the path directly
to the cargo binary, such as target/debug/cargo
.
If you are using rustup
, beware that running the binary directly can cause
issues with rustup overrides. Usually, when cargo
is executed as part of
rustup, the toolchain becomes sticky (via an environment variable), and all
calls to rustc
will use the same toolchain. But when cargo
is not run via
rustup, the toolchain may change based on the directory. Since Cargo changes
the directory for each compilation, this can cause different calls to rustc
to use different versions. There are a few workarounds:
- Don’t use rustup overrides.
- Use
rustup run target/debug/cargo
to executecargo
. - Set the
RUSTC
environment variable to a specificrustc
executable (not the rustup wrapper). - Create a custom toolchain. This is a bit of a hack, but you can create a
directory in the rustup
toolchains
directory, and create symlinks for all the files and directories in there to your toolchain of choice (such as nightly), except for thecargo
binary, which you can symlink to yourtarget/debug/cargo
binary in your project directory.
Normally, all development is done by running Cargo’s test suite, so running it directly usually isn’t required. But it can be useful for testing Cargo on more complex projects.
Making a change
Some guidelines on working on a change:
- All code changes are expected to comply with the formatting suggested by
rustfmt
. You can userustup component add rustfmt
to installrustfmt
and usecargo fmt
to automatically format your code. - Include tests that cover all non-trivial code. See the Testing chapter for more about writing and running tests.
- All code should be warning-free. This is checked during tests.
Submitting a Pull Request
After you have committed your work, and pushed it to GitHub, you can open a Pull Request
- Push your commits to GitHub and create a pull request against Cargo’s
master
branch. - Include a clear description of what the change is and why it is being made.
- Use GitHub’s keywords in the description to automatically link to an issue
if the PR resolves the issue. For example
Closes #1234
will link issue #1234 to the PR. When the PR is merged, GitHub will automatically close the issue.
@rustbot
will automatically assign a reviewer for the PR. It
may take at least a few days for someone to respond. If you don’t get a
response in over a week, feel free to ping the assigned reviewer.
When your PR is submitted, GitHub automatically runs all tests. The GitHub interface will show a green checkmark if it passes, or a red X if it fails. There are links to the logs on the PR page to diagnose any issues. The tests typically finish in under 30 minutes.
The reviewer might point out changes deemed necessary. Large or tricky changes may require several passes of review and changes.
Status labeling
PRs will get marked with labels like S-waiting-on-review
or S-waiting-on-author
to indicate their status.
The @rustbot
bot can be used by anyone to adjust the labels.
If a PR gets marked as S-waiting-on-author
, and you have pushed new changes that you would like to be reviewed, you can write a comment on the PR with the text @rustbot ready
.
The bot will switch the labels on the PR.
More information about these commands can be found at the shortcuts documentation.
The merging process
After a reviewer has approved your PR, they will issue a command to the bors
bot (also known as “Homu”, the software that powers @bors
). Bors will
create a temporary branch with your PR, and run all tests. Only if all tests
pass will it merge the PR to master. If it fails, the bot will leave a comment
on the PR. This system ensures that the master branch is always in a good
state, and that merges are processed one at a time. The Homu queue
dashboard shows the current merge queue. Cargo’s queue is rarely
busy, but a busy project like the rust repo is constantly full.
Assuming everything works, congratulations! It may take at least a week for the changes to arrive on the nightly channel. See the release chapter for more information on how Cargo releases are made.
Release process
Cargo is released with rustc
using a “train model”. After a
change lands in Cargo’s master branch, it will be synced with the
rust-lang/rust repository by a Cargo team member, which happens about once a
week. If there are complications, it can take longer. After it is synced and
merged, the changes will appear in the next nightly release, which is usually
published around 00:30 UTC.
After changes are in the nightly release, they will make their way to the stable release anywhere from 6 to 12 weeks later, depending on when during the cycle it landed.
The current release schedule is posted on the Rust Forge. See the release process for more details on how Rust’s releases are created. Rust releases are managed by the Release team.
Build process
The build process for Cargo is handled as part of building Rust. Every PR on
the rust-lang/rust repository creates a full collection of release artifacts
for every platform. The code for this is in the dist
bootstrap module.
Every night at 00:00 UTC, the artifacts from the most recently merged PR are
promoted to the nightly release channel. A similar process happens for beta
and stable releases.
Submodule updates
Cargo is tracked in the rust-lang/rust repository using a git submodule. It is updated manually about once a week by a Cargo team member. However, anyone is welcome to update it as needed.
@ehuss has a tool called subup to automate the process of updating the submodule, updating the lockfile, running tests, and creating a PR. Running the tests ahead-of-time helps avoid long cycle times waiting for bors if there are any errors. Subup will also provide a message to include in the PR with a list of all PRs it covers. Posting this in the PR message also helps create reference links on each Cargo PR to the submodule update PR to help track when it gets merged.
The following is an example of the command to run in a local clone of rust-lang/rust to run a certain set of tests of things that are likely to get broken by a Cargo update:
subup --up-branch update-cargo \
--commit-message "Update cargo" \
--test="src/tools/linkchecker tidy \
src/tools/cargo \
src/tools/rustfmt \
src/tools/cargo
If doing a beta backport, the command is similar, but needs to point to the correct branches:
subup --up-branch update-beta-cargo \
--rust-branch beta \
--set-config rust.channel=beta \
--commit-message "[beta] Update cargo" \
--test="src/tools/linkchecker tidy \
src/tools/cargo \
src/tools/rustfmt \
rust-1.66.0:src/tools/cargo
Version updates
Shortly after each major release, a Cargo team member will post a PR to update
Cargo’s version in Cargo.toml
. Cargo’s library is permanently unstable, so
its version number starts with a 0
. The minor version is always 1 greater
than the Rust release it is a part of, so cargo 0.49.0 is part of the 1.48
Rust release. The CHANGELOG is also usually updated at this time.
Also, any version-specific checks that are no longer needed can be removed. For example, some tests are disabled on stable if they require some nightly behavior. Once that behavior is available on the new stable release, the checks are no longer necessary. (I usually search for the word “nightly” in the testsuite directory, and read the comments to see if any of those nightly checks can be removed.)
Sometimes Cargo will have a runtime check to probe rustc
if it supports a
specific feature. This is usually stored in the TargetInfo
struct. If this
behavior is now stable, those checks should be removed.
Cargo has several other packages in the crates/
directory. If any of these
packages have changed, the version should be bumped before the beta
release. It is rare that these get updated. Bumping these as-needed helps
avoid churning incompatible version numbers. This process should be improved
in the future!
@ehuss has a tool called cargo-new-release to automate the process of doing a version bump. It runs through several steps:
- Creates a branch
- Updates the version numbers
- Creates a changelog for anything on the master branch that is not part of beta
- Creates a changelog for anything on the beta branch
It opens a browser tab for every PR in order to review each change. It places each PR in the changelog with its title, but usually every PR should be rewritten to explain the change from the user’s perspective. Each PR should also be categorized as an Addition, Change, Fix, or Nightly-only change. Most PRs are deleted, since they are not relevant to users of Cargo. For example, remove all PRs related to Cargo internals, infrastructure, documentation, error changes, refactorings, etc. Usually about half of the PRs get removed. This process usually takes @ehuss about an hour to finish.
Docs publishing
Docs are automatically published during the Rust release process. The nightly channel’s docs appear at https://doc.rust-lang.org/nightly/cargo/. Once nightly is promoted to beta, those docs will appear at https://doc.rust-lang.org/beta/cargo/. Once the stable release is made, it will appear on https://doc.rust-lang.org/cargo/ (which is the “current” stable) and the release-specific URL such as https://doc.rust-lang.org/1.46.0/cargo/.
The code that builds the documentation is located in the doc
bootstrap
module.
crates.io publishing
Cargo’s library is published to crates.io as part of the stable release
process. This is handled by the Release team as part of their process. There
is a publish.py
script that in theory should help with this process. The
test and build tool crates aren’t published.
Beta backports
If there is a regression or major problem detected during the beta phase, it may be necessary to backport a fix to beta. The process is documented in the Beta Backporting page.
Stable backports
In (hopefully!) very rare cases, a major regression or problem may be reported after the stable release. Decisions about this are usually coordinated between the Release team and the Cargo team. There is usually a high bar for making a stable patch release, and the decision may be influenced by whether or not there are other changes that need a new stable release.
The process here is similar to the beta-backporting process. The
rust-lang/cargo branch is the same as beta (rust-1.XX.0
). The
rust-lang/rust branch is called stable
.
Unstable features
Most new features should go through the unstable process. This means that the feature will only be usable on the nightly channel, and requires a specific opt-in by the user. Small changes can skip this process, but please consult with the Cargo team first.
Unstable feature opt-in
For features that require behavior changes or new syntax in Cargo.toml
, then
it will need a cargo-features
value placed at the top of Cargo.toml
to
enable it. The process for doing adding a new feature is described in the
features
module. Code that implements the feature will need to manually
check that the feature is enabled for the current manifest.
For features that add new command-line flags, config options, or environment
variables, then the -Z
flags will be needed to enable them. The features
module also describes how to add these. New flags should use the
fail_if_stable_opt
method to check if the -Z unstable-options
flag has
been passed.
Unstable documentation
Every unstable feature should have a section added to the unstable chapter describing how to use the feature.
Tracking issues
Each unstable feature should get a tracking issue. These issues are typically created when a PR is close to being merged, or soon after it is merged. Use the tracking issue template when creating a tracking issue.
Larger features should also get a new label in the issue tracker so that when issues are filed, they can be easily tied together.
Pre-Stabilization
Once an unstable feature is “complete”, the search for users to test
and give feedback begins. Testing notes should be written up to give users an
idea of how to test the new feature. An example being the
workspace inheritance testing notes for workspace inheritance. Once testing
notes have been written up you should make posts in various rust communities
(rust subreddit, users, internals, etc). Example posts made for workspace
inheritance: reddit post, users post, internals post. The unstable feature
should also be added to This Week in Rust. This should be done by adding the
label call-for-testing
to the RFC for the feature and making a comment with a
link to the testing notes and the tracking issue (as needed). If there is not an
RFC, a pull request should be made to the TWiR repo adding the feature to the
Call for Testing
section (example).
Stabilization
After some period of time, typically measured in months, the feature can be considered to be stabilized. The feature should not have any significant known bugs or issues, and any design concerns should be resolved.
The stabilization process depends on the kind of feature. For smaller features, you can leave a comment on the tracking issue expressing interest in stabilizing it. It can usually help to indicate that the feature has received some real-world testing, and has exhibited some demand for broad use.
For larger features that have not gone through the RFC process, then an RFC to call for stabilization might be warranted. This gives the community a final chance to provide feedback about the proposed design.
For a small feature, or one that has already gone through the RFC process, a Cargo Team member may decide to call for a “final comment period” using rfcbot. This is a public signal that a major change is being made, and gives the Cargo Team members an opportunity to confirm or block the change. This process can take a few days or weeks, or longer if a concern is raised.
Once the stabilization has been approved, the person who called for stabilization should prepare a PR to stabilize the feature. This PR should:
- Flip the feature to stable in the
features
module. - Remove any unstable checks that aren’t automatically handled by the feature system.
- Move the documentation from the unstable chapter into the appropriate places in the Cargo book and man pages.
- Remove the
-Z
flags and help message if applicable. - Update all tests to remove nightly checks.
- Tag the PR with relnotes label if it seems important enough to highlight in the Rust release notes.
Design Principles
The purpose of Cargo is to formalize a canonical Rust workflow, by automating the standard tasks associated with distributing software. Cargo simplifies structuring a new project, adding dependencies, writing and running unit tests, and more.
Cargo is not intended to be a general-purpose build tool. Ideally, it should be easy to integrate it within another build tool, though admittedly that is not as seamless as desired.
Stability and compatibility
Backwards compatibility
Cargo strives to remain backwards compatible with projects created in previous versions. The CLI interface also strives to remain backwards compatible, such that the commands and options behave the same. That being said, changes in behavior, and even outright breakage are sometimes done in limited situations. The following outlines some situations where backwards-incompatible changes are made:
- Anything that addresses a security concern.
- Dropping support for older platforms and tooling. Cargo follows the Rust tiered platform support.
- Changes to resolve possibly unsafe or unreliable behavior.
None of these changes should be taken lightly, and should be avoided if possible, or possibly with some transition period to alert the user of the potential change.
Behavior is sometimes changed in ways that have a high confidence that it won’t break existing workflows. Almost every change carries this risk, so it is often a judgment call balancing the benefit of the change with the perceived possibility of its negative consequences.
At times, some changes fall in the gray area, where the current behavior is undocumented, or not working as intended. These are more difficult judgment calls. The general preference is to balance towards avoiding breaking existing workflows.
Support for older registry APIs and index formats may be dropped, if there is high confidence that there aren’t any active registries that may be affected. This has never (to my knowledge) happened so far, and is unlikely to happen in the future, but remains a possibility.
In all of the above, a transition period may be employed if a change is known to cause breakage. A warning can be issued to alert the user that something will change, and provide them with an alternative to resolve the issue (preferably in a way that is compatible across versions if possible).
Cargo is only expected to work with the version of the related Rust tools
(rustc
, rustdoc
, etc.) that it is released with. As a matter of choice,
the latest nightly works with the most recent stable release, but that is
mostly to accommodate development of Cargo itself, and should not be expected
by users.
Forwards compatibility
Additionally, Cargo strives a limited degree of forwards compatibility. Changes should not egregiously prevent older versions from working. This is mostly relevant for persistent data, such as on-disk files and the registry interface and index. It also applies to a lesser degree to the registry API.
Changes to Cargo.lock
require a transition time, where the new format is not
automatically written when the lock file is updated. The transition time
should not be less than 6 months, though preferably longer. New projects may
use the new format in a shorter time frame.
Changes to Cargo.toml
can be made in any release. This is because the user
must manually modify the file, and opt-in to any new changes. Additionally,
Cargo will usually only issue a warning about new fields it doesn’t
understand, but otherwise continue to function.
Changes to cache files (such as artifacts in the target
directory, or cached
data in Cargo’s home directory) should not prevent older versions from
running, but they may cause older versions to recreate the cache, which may
result in a performance impact.
Changes to the registry index should not prevent older versions from working. Generally, older versions ignore new fields, so the format should be easily extensible. Changes to the format or interpretation of existing fields should be done very carefully to avoid preventing older versions of Cargo from working. In some cases, this may mean that older versions of Cargo will not be able to select a newly published crate, but it shouldn’t prevent them from working at all. This level of compatibility may not last forever, but the exact time frame for such a change has not yet been decided.
The registry API may be changed in such a way to prevent older versions of Cargo from working. Generally, compatibility should be retained for as long as possible, but the exact length of time is not specified.
Simplicity and layers
Standard workflows should be easy and consistent. Each knob that is added has a high cost, regardless if it is intended for a small audience. Layering and defaults can help avoid the surface area that the user needs to be concerned with. Try to avoid small functionalities that may have complex interactions with one another.
Implementing a Change
This chapter gives an overview of what you need to know in making a change to cargo.
If you feel something is missing that would help you, feel free to ask on Zulip.
Architecture Overview
See the
nightly docs
for an overview of cargo
s architecture and links out to further details.
New Subcommands
Cargo is a single binary composed of a set of clap
subcommands. All
subcommands live in src/bin/cargo/commands
directory.
src/bin/cargo/main.rs
is the entry point.
Each subcommand, such as src/bin/cargo/commands/build.rs
, usually performs
the following:
- Parse the CLI flags. See the
command_prelude
module for some helpers to make this easier. - Load the config files.
- Discover and load the workspace.
- Calls the actual implementation of the subcommand which resides in
src/cargo/ops
.
If the subcommand is not found in the built-in list, then Cargo will
automatically search for a subcommand named cargo-{NAME}
in the users PATH
to execute the subcommand.
Console Output
All of Cargo’s output should go through the Shell
struct. You can normally
obtain the Shell
instance from the Config
struct. Do not use the std
println!
macros.
Most of Cargo’s output goes to stderr. When running in JSON mode, the output goes to stdout.
It is important to properly handle errors when writing to the console.
Informational commands, like cargo list
, should ignore any errors writing
the output. There are some drop_print
macros that are intended to make
this easier.
Messages written during compilation should handle errors, and abort the build
if they are unable to be displayed. This is generally automatically handled in
the JobQueue
as it processes each message.
Errors
Cargo uses anyhow
for managing errors. This makes it convenient to “chain”
errors together, so that Cargo can report how an error originated, and what it
was trying to do at the time.
Error helpers are implemented in the errors
module. Use the
InternalError
error type for errors that are not expected to happen. This
will print a message to the user to file a bug report.
The binary side of Cargo uses the CliError
struct to wrap the process exit
code. Usually Cargo exits with 101 for an error, but some commands like cargo test
will exit with different codes.
Style
Some guidelines for Cargo’s output:
- Keep the normal output brief. Cargo is already fairly noisy, so try to keep the output as brief and clean as possible.
- Good error messages are very important! Try to keep them brief and to the point, but good enough that a beginner can understand what is wrong and can figure out how to fix. It is a difficult balance to hit! Err on the side of providing extra information.
- When using any low-level routines, such as
std::fs
, always add error context about what it is doing. For example, reading from a file should include context about which file is being read if there is an error. - Cargo’s error style is usually a phrase, starting with a lowercase letter. If there is a longer error message that needs multiple sentences, go ahead and use multiple sentences. This should probably be improved sometime in the future to be more structured.
Filesystem
Cargo tends to get run on a very wide array of file systems. Different file systems can have a wide range of capabilities, and Cargo should strive to do its best to handle them. Some examples of issues to deal with:
- Not all file systems support locking. Cargo tries to detect if locking is supported, and if not, will ignore lock errors. This isn’t ideal, but it is difficult to deal with.
- The
fs::canonicalize
function doesn’t work on all file systems (particularly some Windows file systems). If that function is used, there should be a fallback if it fails. This function will also return\\?\
style paths on Windows, which can have some issues (such as some tools not supporting them, or having issues with relative paths). - Timestamps can be unreliable. The
fingerprint
module has a deeper discussion of this. One example is that Docker cache layers will erase the fractional part of the time stamp. - Symlinks are not always supported, particularly on Windows.
Debugging
Logging
Cargo uses the env_logger
crate to display debug log messages. The
CARGO_LOG
environment variable can be set to enable debug logging, with a
value such as trace
, debug
, or warn
. It also supports filtering for
specific modules. Feel free to use the standard log
macros to help with
diagnosing problems.
# Outputs all logs with levels debug and higher
CARGO_LOG=debug cargo generate-lockfile
# Don't forget that you can filter by module as well
CARGO_LOG=cargo::core::resolver=trace cargo generate-lockfile
# This will print lots of info about the download process. `trace` prints even more.
CARGO_HTTP_DEBUG=true CARGO_LOG=cargo::ops::registry=debug cargo fetch
# This is an important command for diagnosing fingerprint issues.
CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build
Architecture Overview
This chapter gives a very high-level overview of Cargo’s architecture. This is intended to give you links into the code which is hopefully commented with more in-depth information.
If you feel something is missing that would help you, feel free to ask on Zulip.
Codebase Overview
See nightly docs
Packages and Resolution
See nightly docs
Compilation
See nightly docs
Files
See nightly docs
Tests
Cargo has an extensive test suite. Most of it is implemented as integration
tests in the testsuite
directory. There are several other tests:
- Unit tests are scattered throughout.
- The dependency resolver has its own set of tests in the
resolver-tests
directory. - All of the packages in the
crates
directory have their own set of tests. - The
build-std
test is for the build-std feature. It is separate since it has some special requirements. - Documentation has a variety of tests, such as link validation, and the SemVer chapter validity checks.
Running Tests
Using cargo test
is usually sufficient for running the full test suite. This
can take a few minutes, so you may want to use more targeted flags to pick the
specific test you want to run, such as cargo test --test testsuite -- check::check_success
.
Running nightly tests
Some tests only run on the nightly toolchain, and will be ignored on other channels. It is recommended that you run tests with both nightly and stable to ensure everything is working as expected.
Some of the nightly tests require the rustc-dev
and llvm-tools-preview
rustup components installed. These components include the compiler as a
library. This may already be installed with your nightly toolchain, but if it
isn’t, run rustup component add rustc-dev llvm-tools-preview --toolchain=nightly
.
Running cross tests
Some tests exercise cross compiling to a different target. This will require
you to install the appropriate target. This typically is the 32-bit target of
your host platform. For example, if your host is a 64-bit
x86_64-unknown-linux-gnu
, then you should install the 32-bit target with
rustup target add i686-unknown-linux-gnu
. If you don’t have the alternate
target installed, there should be an error message telling you what to do. You
may also need to install additional tools for the target. For example, on Ubuntu
you should install the gcc-multilib
package.
If you can’t install an alternate target, you can set the
CFG_DISABLE_CROSS_TESTS=1
environment variable to disable these tests. The
Windows cross tests only support the MSVC toolchain.
Running build-std tests
The build-std
tests are disabled by default, but you can run them by setting
the CARGO_RUN_BUILD_STD_TESTS=1
environment variable and running cargo test --test build-std
. This requires the nightly channel, and also requires the
rust-src
component installed with rustup component add rust-src --toolchain=nightly
.
Running with gitoxide
as default git backend in tests
By default, the git2
backend is used for most git operations. As tests need to explicitly
opt-in to use nightly features and feature flags, adjusting all tests to run with nightly
and -Zgitoxide
is unfeasible.
This is why the private environment variable named __CARGO_USE_GITOXIDE_INSTEAD_OF_GIT2
can be
set while running tests to automatically enable the -Zgitoxide
flag implicitly, allowing to
test gitoxide
for the entire cargo test suite.
Running public network tests
Some (very rare) tests involve connecting to the public internet.
These tests are disabled by default,
but you can run them by setting the CARGO_PUBLIC_NETWORK_TESTS=1
environment variable.
Additionally our CI suite has a smoke test for fetching dependencies.
For most contributors, you will never need to bother with this.
Running container tests
Tests marked with container_test
involve running Docker to test more complex configurations.
These tests are disabled by default,
but you can run them by setting the CARGO_CONTAINER_TESTS=1
environment variable.
You will need to have Docker installed and running to use these.
Note: Container tests mostly do not work on Windows.
- The SSH tests require ssh-agent, but the two versions of ssh-agent on Windows are not suitable for testing.
- The Microsoft version of ssh-agent runs as a global service, and can’t be isolated per test.
- The mingw/cygwin one can’t be accessed from a Windows executable like cargo.
- Pageant similarly does not seem to have a way to isolate it (and I’m not certain it can be driven completely from the command-line).
The tests also can’t run on Windows CI because the Docker that is preinstalled doesn’t support Linux containers, and setting up Windows containers is a pain.
macOS should work with Docker installed and running, but unfortunately the tests are not run on CI because Docker is not available.
Writing Tests
The following focuses on writing an integration test. However, writing unit tests is also encouraged!
Testsuite
Cargo has a wide variety of integration tests that execute the cargo
binary
and verify its behavior, located in the testsuite
directory. The
support
crate and snapbox
contain many helpers to make this process easy.
There are two styles of tests that can roughly be categorized as
- functional tests
- The fixture is programmatically defined
- The assertions are regular string comparisons
- Easier to share in an issue as a code block is completely self-contained
- More resilient to insignificant changes though ui tests are easy to update when a change does occur
- ui tests
- The fixture is file-based
- The assertions use file-backed snapshots that can be updated with an env variable
- Easier to review the expected behavior of the command as more details are included
- Easier to get up and running from an existing project
- Easier to reason about as everything is just files in the repo
These tests typically work by creating a temporary “project” with a
Cargo.toml
file, executing the cargo
binary process, and checking the
stdout and stderr output against the expected output.
Functional Tests
Generally, a functional test will be placed in tests/testsuite/<command>.rs
and will look roughly like:
#[cargo_test]
fn <description>() {
let p = project()
.file("src/main.rs", r#"fn main() { println!("hi!"); }"#)
.build();
p.cargo("run --bin foo")
.with_stderr(
"\
[COMPILING] foo [..]
[FINISHED] [..]
[RUNNING] `target/debug/foo`
",
)
.with_stdout("hi!")
.run();
}
}
The #[cargo_test]
attribute is used in place of #[test]
to inject some setup code.
ProjectBuilder
via project()
:
- Each project is in a separate directory in the sandbox
- If you do not specify a
Cargo.toml
manifest usingfile()
, one is automatically created with a project name offoo
usingbasic_manifest()
.
Execs
via p.cargo(...)
:
- This executes the command and evaluates different assertions
- See
support::compare
for an explanation of the string pattern matching. Patterns are used to make it easier to match against the expected output.
- See
#[cargo_test]
attribute
The #[cargo_test]
attribute injects code which does some setup before starting the test.
It will create a filesystem “sandbox” under the “cargo integration test” directory for each test, such as /path/to/cargo/target/tmp/cit/t123/
.
The sandbox will contain a home
directory that will be used instead of your normal home directory.
The #[cargo_test]
attribute takes several options that will affect how the test is generated.
They are listed in parentheses separated with commas, such as:
#[cargo_test(nightly, reason = "-Zfoo is unstable")]
The options it supports are:
nightly
— This will cause the test to be ignored if not running on the nightly toolchain. This is useful for tests that use unstable options inrustc
orrustdoc
. These tests are run in Cargo’s CI, but are disabled in rust-lang/rust’s CI due to the difficulty of updating both repos simultaneously. Areason
field is required to explain why it is nightly-only.build_std_real
— This is a “real”-Zbuild-std
test (in thebuild_std
integration test). This only runs on nightly, and only if the environment variableCARGO_RUN_BUILD_STD_TESTS
is set (these tests on run on Linux).build_std_mock
— This is a “mock”-Zbuild-std
test (which uses a mock standard library). This only runs on nightly, and is disabled for windows-gnu.requires_
— This indicates a command that is required to be installed to be run. For example,requires_rustfmt
means the test will only run if the executablerustfmt
is installed. These tests are always run on CI. This is mainly used to avoid requiring contributors from having every dependency installed.>=1.64
— This indicates that the test will only run with the given version ofrustc
or newer. This can be used when a newrustc
feature has been stabilized that the test depends on. If this is specified, areason
is required to explain why it is being checked.public_network_test
— This tests contacts the public internet. These tests are disabled unless theCARGO_PUBLIC_NETWORK_TESTS
environment variable is set. Use of this should be extremely rare, please avoid using it if possible. The hosts it contacts should have a relatively high confidence that they are reliable and stable (such as github.com), especially in CI. The tests should be carefully considered for developer security and privacy as well.container_test
— This indicates that it is a test that uses Docker. These tests are disabled unless theCARGO_CONTAINER_TESTS
environment variable is set. This requires that you have Docker installed. The SSH tests also assume that you have OpenSSH installed. These should work on Linux, macOS, and Windows where possible. Unfortunately these tests are not run in CI for macOS or Windows (no Docker on macOS, and Windows does not support Linux images). Seecrates/cargo-test-support/src/containers.rs
for more on writing these tests.ignore_windows="reason"
— Indicates that the test should be ignored on windows for the given reason.
Testing Nightly Features
If you are testing a Cargo feature that only works on “nightly” Cargo, then
you need to call masquerade_as_nightly_cargo
on the process builder and pass
the name of the feature as the reason, like this:
p.cargo("build").masquerade_as_nightly_cargo(&["print-im-a-teapot"])
If you are testing a feature that only works on nightly rustc (such as
benchmarks), then you should use the nightly
option of the cargo_test
attribute, like this:
#[cargo_test(nightly, reason = "-Zfoo is unstable")]
This will cause the test to be ignored if not running on the nightly toolchain.
Specifying Dependencies
You should not write any tests that use the network such as contacting crates.io. Typically, simple path dependencies are the easiest way to add a dependency. Example:
let p = project()
.file("Cargo.toml", r#"
[package]
name = "foo"
version = "1.0.0"
[dependencies]
bar = {path = "bar"}
"#)
.file("src/lib.rs", "extern crate bar;")
.file("bar/Cargo.toml", &basic_manifest("bar", "1.0.0"))
.file("bar/src/lib.rs", "")
.build();
If you need to test with registry dependencies, see
support::registry::Package
for creating packages you can depend on.
If you need to test git dependencies, see support::git
to create a git
dependency.
Cross compilation
There are some utilities to help support tests that need to work against a target other than the host. See Running cross tests for more an introduction on cross compilation tests.
Tests that need to do cross-compilation should include this at the top of the test to disable it in scenarios where cross compilation isn’t available:
if cargo_test_support::cross_compile::disabled() {
return;
}
The name of the target can be fetched with the cross_compile::alternate()
function. The name of the host target can be fetched with
cargo_test_support::rustc_host()
.
The cross-tests need to distinguish between targets which can build versus
those which can actually run the resulting executable. Unfortunately, macOS is
currently unable to run an alternate target (Apple removed 32-bit support a
long time ago). For building, x86_64-apple-darwin
will target
x86_64-apple-ios
as its alternate. However, the iOS target can only execute
binaries if the iOS simulator is installed and configured. The simulator is
not available in CI, so all tests that need to run cross-compiled binaries are
disabled on CI. If you are running on macOS locally, and have the simulator
installed, then it should be able to run them.
If the test needs to run the cross-compiled binary, then it should have something like this to exit the test before doing so:
if cargo_test_support::cross_compile::can_run_on_host() {
return;
}
UI Tests
UI Tests are a bit more spread out and generally look like:
tests/testsuite/<command>/mod.rs
:
mod <case>;
tests/testsuite/<command>/<case>/mod.rs
:
use cargo_test_support::prelude::*;
use cargo_test_support::compare::assert_ui;
use cargo_test_support::Project;
use cargo_test_support::curr_dir;
#[cargo_test]
fn case() {
let project = Project::from_template(curr_dir!().join("in"));
let project_root = project.root();
let cwd = &project_root;
snapbox::cmd::Command::cargo_ui()
.arg("run")
.arg_line("--bin foo")
.current_dir(cwd)
.assert()
.success()
.stdout_matches_path(curr_dir!().join("stdout.log"))
.stderr_matches_path(curr_dir!().join("stderr.log"));
assert_ui().subset_matches(curr_dir!().join("out"), &project_root);
}
Then populate
tests/testsuite/<command>/<case>/in
with the project’s directory structuretests/testsuite/<command>/<case>/out
with the files you want verifiedtests/testsuite/<command>/<case>/stdout.log
with nothingtests/testsuite/<command>/<case>/stderr.log
with nothing
#[cargo_test]
:
- This is used in place of
#[test]
- This attribute injects code which does some setup before starting the
test, creating a filesystem “sandbox” under the “cargo integration test”
directory for each test such as
/path/to/cargo/target/cit/t123/
- The sandbox will contain a
home
directory that will be used instead of your normal home directory
Project
:
- The project is copied from a directory in the repo
- Each project is in a separate directory in the sandbox
Command
via Command::cargo_ui()
:
- Set up and run a command.
OutputAssert
via Command::assert()
:
- Perform assertions on the result of the
Command
Assert
via assert_ui()
:
- Verify the command modified the file system as expected
Updating Snapshots
The project, stdout, and stderr snapshots can be updated by running with the
SNAPSHOTS=overwrite
environment variable, like:
$ SNAPSHOTS=overwrite cargo test
Be sure to check the snapshots to make sure they make sense.
Testing Nightly Features
If you are testing a Cargo feature that only works on “nightly” Cargo, then
you need to call masquerade_as_nightly_cargo
on the process builder and pass
the name of the feature as the reason, like this:
snapbox::cmd::Command::cargo()
.masquerade_as_nightly_cargo(&["print-im-a-teapot"])
If you are testing a feature that only works on nightly rustc (such as
benchmarks), then you should use the nightly
option of the cargo_test
attribute, like this:
#[cargo_test(nightly, reason = "-Zfoo is unstable")]
This will cause the test to be ignored if not running on the nightly toolchain.
Platform-specific Notes
When checking output, use /
for paths even on Windows: the actual output
of \
on Windows will be replaced with /
.
Be careful when executing binaries on Windows. You should not rename, delete, or overwrite a binary immediately after running it. Under some conditions Windows will fail with errors like “directory not empty” or “failed to remove” or “access is denied”.
Debugging tests
In some cases, you may need to dig into a test that is not working as you expect, or you just generally want to experiment within the sandbox environment. The general process is:
-
Build the sandbox for the test you want to investigate. For example:
cargo test --test testsuite -- features2::inactivate_targets
. -
In another terminal, head into the sandbox directory to inspect the files and run
cargo
directly.-
The sandbox directories start with
t0
for the first test.cd target/tmp/cit/t0
-
Set up the environment so that the sandbox configuration takes effect:
export CARGO_HOME=$(pwd)/home/.cargo
-
Most tests create a
foo
project, so head into that:cd foo
-
-
Run whatever cargo command you want. See Running Cargo for more details on running the correct
cargo
process. Some examples:/path/to/my/cargo/target/debug/cargo check
- Using a debugger like
lldb
orgdb
:lldb /path/to/my/cargo/target/debug/cargo
- Set a breakpoint, for example:
b generate_root_units
- Run with arguments:
r check
Benchmarking and Profiling
Internal profiler
Cargo has a basic, hierarchical profiler built-in. The environment variable
CARGO_PROFILE
can be set to an integer which specifies how deep in the
profile stack to print results for.
# Output first three levels of profiling info
CARGO_PROFILE=3 cargo generate-lockfile
Benchmarking
Benchsuite
Head over to the benches
directory for more
information about the benchmarking suite.
Informal benchmarking
The overhead for starting a build should be kept as low as possible (preferably, well under 0.5 seconds on most projects and systems). Currently, the primary parts that affect this are:
- Running the resolver.
- Querying the index.
- Checking git dependencies.
- Scanning the local project.
- Building the unit dependency graph.
One way to test this is to use hyperfine. This is a tool that can be used to
measure the difference between different commands and settings. Usually this
is done by measuring the time it takes for cargo build
to finish in a large
project where the build is fresh (no actual compilation is performed). Just
run cargo build
once before using hyperfine.
Crater
Crater is a tool for compiling and running tests for every crate on crates.io (and a few on GitHub). It is mainly used for checking the extent of breakage when implementing potentially breaking changes and ensuring lack of breakage by running beta vs stable compiler versions.
Essentially it runs some cargo
command on every crate twice; once against the “start” toolchain and again against the “end” toolchain.
For example, “start” could be the stable release, and “end” could be beta.
If it passes in “start” but fails with “end”, then that is reported as a regression.
There is a bot called craterbot which is used to run crater on hardware managed by the rust-lang organization.
Crater is run by the release team during the beta cycle. If there are any regressions that look like they are caused by Cargo, they should contact the Cargo team to decide how to handle it.
Running crater
If you have a change that you want to test before the beta release, or you want to test behavior that is not normally exercised by crater, you can do a manual run of crater. Roughly the steps are:
-
Create a branch with your changes.
In your clone of cargo, make the changes to incorporate whatever new thing you want to test and push it to a branch on your fork on GitHub.
-
Get a clone of https://github.com/rust-lang/rust
-
Create a branch in your rust-lang/rust clone to add your changes.
-
Change the
src/tools/cargo
submodule to point to your new branch.Modify
.gitmodules
to point to your clone and branch of cargo with the changes you want to test. For example:git submodule set-url src/tools/cargo https://github.com/ehuss/cargo.git git submodule set-branch --branch my-awesome-feature src/tools/cargo git submodule update --remote src/tools/cargo git add .gitmodules src/tools/cargo git commit
-
Create an PR on rust-lang/rust.
Push your submodule changes to GitHub and make a PR. Start the PR title with
[EXPERIMENT]
to make it clear what the PR is for and assign yourself or @ghost. -
Make a “try” build.
A “try” build creates a full release of x86_64-unknown-linux-gnu and stores it on rust-lang servers. This can be done with a comment
@bors try
on the PR (all Cargo team members should have permission to do this). -
Run crater.
Look at the craterbot docs to determine the command that you want to run. There are different modes like
check-only
,build-and-test
,rustdoc
, etc.You can also choose how many crates to run against. If you are uncertain if your cargo changes will work correctly, it might be a good idea to run against
top-100
first to check its behavior. This will run much faster. You can do a full run afterwards.After the try build finishes (which should take a couple hours), ask someone to make a crater run. The Cargo team does not have that permission, so just ask someone on Zulip. They will need to write a comment to
@craterbot
with the command that you have specified. -
Wait.
Crater can take anywhere from a few hours to a few weeks to run depending on how long the craterbot queue is and which mode you picked and the priority of your job. When the crater run finishes, craterbot will post a comment to the PR with a link to a report of the results.
-
Investigate the report.
Look through the report which contains links to build logs for any regressions or errors.
-
Close the PR.
Whenever you are done doing crater runs, close your PR.
Advanced crater modes
Crater only has a few built-in modes, such as running cargo check
or cargo test
.
You can pass extra flags with +cargoflags
.
More complex tests can be accomplished by customizing Cargo to perform whatever actions you want.
Since crater essentially runs cargo check
, you can modify the check
command to perform whichever actions you want.
For example, to test cargo fix --edition
, this commit intercepted cargo check
and modified it to instead:
- Only run on crates with the 2018 edition.
- Run
cargo fix --edition
. - Modify the manifest to switch to the 2021 edition.
- Run
cargo check
to verify.
If you need to compare the before and after of a command that is not part of crater’s built-in modes, that can be more difficult. Two possible options:
- Work with the infra team to add a new mode.
- Build two custom try builds.
Each one should modify the
cargo check
command as described above. The “start” build should perform whichever action you want with an otherwise unmodified cargo. The “end” build should perform whichever action you want with your modified cargo. Then, in the@craterbot
command, specify the start and end hashes of the two try builds.
Limitations
There are some limitations of crater to consider when running Cargo:
- A crater run without regressions is not a green light to move forward.
- A large portion of Rust code is not tested, such as closed-source projects or things otherwise not collected by crater.
- Many crates can’t build in crater’s environment or are otherwise broken.
- Some crates have flaky tests.
- Crater runs in an isolated environment.
- It only runs on Linux x86-64.
- It does not have network access.
- The crate source is in a read-only mount.
- Crater does several steps before running the test (using its own copy of the stable toolchain):
- It generates a lockfile using
generate-lockfile
and includes-Zno-index-update
to prevent index updates (which makes it run much faster). - All dependencies are downloaded ahead-of-time with
cargo fetch
.
- It generates a lockfile using
- The built-in modes pass several flags to cargo such as
--frozen
or--message-format=json
. It will sometimes use--all-targets
and sometimes not. Check the crater source for more details on how it works.