Skip to content

Conversation

@ranger-ross
Copy link
Member

PR for crater run of rust-lang/cargo#16336
Before running, we should probably get a sanity check from @epage that there is nothing I missed.

See Zulip Thread

r? @Mark-Simulacrum

@rustbot
Copy link
Collaborator

rustbot commented Dec 10, 2025

Some changes occurred in src/tools/cargo

cc @ehuss

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 10, 2025
@rustbot
Copy link
Collaborator

rustbot commented Dec 10, 2025

⚠️ Warning ⚠️

@ehuss
Copy link
Contributor

ehuss commented Dec 10, 2025

Do you happen to know the crater command you want to run? (The command syntax is documented at https://github.com/rust-lang/crater/blob/master/docs/bot-usage.md)

@Kivooeo
Copy link
Member

Kivooeo commented Dec 10, 2025

Let's do try first and then run crater in check-only mode, at least this is how I understand this process

@bors try

@rust-bors

This comment has been minimized.

rust-bors bot added a commit that referenced this pull request Dec 10, 2025
…, r=<try>

Crater run for cargo build-dir changes
@ehuss
Copy link
Contributor

ehuss commented Dec 10, 2025

@Kivooeo IIUC, that's not going to be sufficient for this because the new layout is not enabled by default. There may be some special settings like environment variables that will be needed to test this properly. And as for the mode, I suspect we want to build, not check, because we'll want to see if build scripts or proc-macros are doing any shenanigans that are required for linking correctly. (Or maybe test I don't know.)

@rust-bors
Copy link

rust-bors bot commented Dec 10, 2025

☀️ Try build successful (CI)
Build commit: 3117848 (311784882baa42a9638298cb325fa01a062c9d0f, parent: 198328ad7960b1bece0dc48496bff6c62dd5d339)

@epage
Copy link
Contributor

epage commented Dec 10, 2025

For this, you can either set -Zbuild-dir-new-layout or CARGO_BUILD_DIR_LAYOUT_V2=true.

Note that x.py can't build with this enabled, see #t-infra/bootstrap > Preparing for the new `build-dir` layout @ 💬

I guess build-only is desirable over check-only to ensure -sys crates link?

A full build-and-test run would help uncover how much people's tests are relying on internals (e.g. I recently fixed rustfmts tests).

@Mark-Simulacrum
Copy link
Member

@craterbot run mode=build-and-test start=master#198328ad7960b1bece0dc48496bff6c62dd5d339 end=try#311784882baa42a9638298cb325fa01a062c9d0f+cargoflags=-Zbuild-dir-new-layout crates=random-1000 p=1

Small run just to make sure it mostly works.

@craterbot
Copy link
Collaborator

👌 Experiment pr-149852 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 13, 2025
@bors
Copy link
Collaborator

bors commented Dec 13, 2025

☔ The latest upstream changes (presumably #149934) made this pull request unmergeable. Please resolve the merge conflicts.

@craterbot
Copy link
Collaborator

🚧 Experiment pr-149852 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment pr-149852 is completed!
📊 2 regressed and 0 fixed (1000 total)
📊 0 spurious results on the retry-regessed-list.txt, consider a retry1 if this is a significant amount.
📰 Open the summary report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

Footnotes

  1. re-run the experiment with crates=https://crater-reports.s3.amazonaws.com/pr-149852/retry-regressed-list.txt

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Dec 17, 2025
@epage
Copy link
Contributor

epage commented Dec 17, 2025

Example from https://github.com/ikorason/headr at 7f4ae60bfd5efd605d9a7e2572ba482b3ac02551

[INFO] [stdout] ---- dies_bad_lines stdout ----
[INFO] [stdout] Error: CargoError { cause: Some(NotFoundError { path: "/opt/rustwide/target/debug/build/headr/757f0ead74b5dd7c/headr" }) }
  • Using assert_cmds deprecated Command::cargo_bin which I believe will fail even with a split target-dir / build-dir

Example from chute-kun 0.1.0

[INFO] [stdout] thread 'init_config_writes_default_toml_to_env_path' (150) panicked at tests/cli_init_config_test.rs:12:47:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: CargoError { cause: Some(NotFoundError { path: "/opt/rustwide/target/debug/build/chute-kun/f8f797d1537351f6/chute" }) }
  • Using assert_cmds deprecated Command::cargo_bin which I believe will fail even with a split target-dir / build-dir

@epage
Copy link
Contributor

epage commented Dec 18, 2025

Opened issues for assert_cmd's dependents that use the deprecated cargo_bin:

(skipped all with download counts below 100,000)

epage added a commit to epage/cargo that referenced this pull request Dec 19, 2025
This also makes artifact deps available for consistency purposes.

I originally proposed this for when moving Cargo off of Cargo's own
internals but went with a different solution.

Re-visiting this because `assert_cmd` has 2300+ dependents and it seems
like making `assert_cmd::cargo_bin` work would be a better use of time
than migrating all of those users to `assert_cmd::cargo_bin!`.
For example, within the top 130 dependents (100,000 downloads or more),
I found 66 that used `assert_cmd::cargo_bin`
(rust-lang/rust#149852).

> The reason to make `CARGO_BIN_EXE` set only at build-time was to
> address the concern of manually running the test executable.
> It's not
> too common for tests to look at any runtime environment variables
> (that I know of), so it usually just works (like running
> `gdb target/debug/.../test-xxx`).
> The concern was that if it were set at
> runtime it would move more down the path where directly running the test
> executable doesn't "just work".

See https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/cargo_bin_exe.20and.20tests/near/513638220

However,
- This is user opt-in
- Users can run with `-vv` to see the env variables that are set
- Users can run `CARGO_TARGET_..._RUNNER=gdb`
- We can notify the `cargo nextest`, the main third-party test runner,
  about this.  It should be easy to support as they have their own
  version of this, see https://nexte.st/docs/configuration/env-vars/?h=nextest_bin_exe#environment-variables-nextest-sets
@epage
Copy link
Contributor

epage commented Dec 19, 2025

Realized a better route for this might be to move forward with making CARGO_BIN_EXE_* available at runtime, which I had previously proposed. cargo_bin could be updated to check that and people will just need to cargo update assert_cmd and it will just work. There will be a span of Cargo versions where CARGO_BUILD_BUILD_DIR will break the tests.

First step is rust-lang/cargo#16421

@Skgland
Copy link
Contributor

Skgland commented Dec 19, 2025

making CARGO_BIN_EXE_* available at runtime, which I had previously proposed. cargo_bin could be updated to check that

Unless I'm mistaken it already does that.

Command::cargo_bin calls crate::cargo::cargo_bin_cmd1, which calls crate::cargo::cargo_bin2, which calls crate::cargo::cargo_bin_str which checks the env var3

Footnotes

  1. https://docs.rs/crate/assert_cmd/latest/source/src/cmd.rs#68

  2. https://docs.rs/crate/assert_cmd/latest/source/src/cargo.rs#155

  3. https://docs.rs/crate/assert_cmd/latest/source/src/cargo.rs#247-248

@epage
Copy link
Contributor

epage commented Dec 19, 2025

which checks the env var3

That is a bug and it always falls back to the unwrap_or_else case.

@Skgland
Copy link
Contributor

Skgland commented Dec 19, 2025

which checks the env var3

That is a bug and it always falls back to the unwrap_or_else case.

Well, yes, currently it always falls back, as the env var isn't currently set at runtime, but if your suggestion is implemented to also set the env var at runtime then it should use the value of the env var without requiring changes to the assert_cmd crate, so dependents of assert_cmd shouldn't need to cargo update assert_cmd and it should just work as is.

@epage
Copy link
Contributor

epage commented Dec 19, 2025

True. That bug was not copied over to snapbox and they would need the update for those who haven't moved off of it yet.

epage added a commit to epage/cargo that referenced this pull request Dec 19, 2025
This also makes artifact deps available for consistency purposes.

I originally proposed this for when moving Cargo off of Cargo's own
internals but went with a different solution.

Re-visiting this because `assert_cmd` has 2300+ dependents and it seems
like making `assert_cmd::cargo_bin` work would be a better use of time
than migrating all of those users to `assert_cmd::cargo_bin!`.
For example, within the top 130 dependents (100,000 downloads or more),
I found 66 that used `assert_cmd::cargo_bin`
(rust-lang/rust#149852).

> The reason to make `CARGO_BIN_EXE` set only at build-time was to
> address the concern of manually running the test executable.
> It's not
> too common for tests to look at any runtime environment variables
> (that I know of), so it usually just works (like running
> `gdb target/debug/.../test-xxx`).
> The concern was that if it were set at
> runtime it would move more down the path where directly running the test
> executable doesn't "just work".

See https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/cargo_bin_exe.20and.20tests/near/513638220

However,
- This is user opt-in
- Users can run with `-vv` to see the env variables that are set
- Users can run `CARGO_TARGET_..._RUNNER=gdb`
- We can notify the `cargo nextest`, the main third-party test runner,
  about this.  It should be easy to support as they have their own
  version of this, see https://nexte.st/docs/configuration/env-vars/?h=nextest_bin_exe#environment-variables-nextest-sets
@Mark-Simulacrum
Copy link
Member

Note that the crater run above was partial -- should we run a full one across all crates? Or does the above results already have enough value that we want to tweak before scheduling a full run?

@epage
Copy link
Contributor

epage commented Dec 20, 2025

Waiting on a cargo feature to be merged to reduce known issues.

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants