rbe_configs_gen fails with Bazel 8.

[veblush]

bazel-toolchains/rbe_configs_gen doesn't appear to work with Bazel 8. This used to work with prior Bazel versions such as 7.4.1. You can find the script I used (here)

$ third_party/toolchains/generate_linux_rbe_configs.sh                  
++ dirname third_party/toolchains/generate_linux_rbe_configs.sh                                           
+ cd third_party/toolchains/../..                                                                         
++ pwd                                                                                                    
+ REPO_ROOT=/usr/local/google/home/veblush/git/grpc   
++ mktemp -d
+ TEMP_DIR=/tmp/tmp.daBRqT16k7
+ pushd /tmp/tmp.daBRqT16k7
/tmp/tmp.daBRqT16k7 ~/git/grpc
+ git clone https://github.com/bazelbuild/bazel-toolchains.git
Cloning into 'bazel-toolchains'...
remote: Enumerating objects: 8047, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 8047 (delta 5), reused 1 (delta 1), pack-reused 8036 (from 2)
Receiving objects: 100% (8047/8047), 1.94 MiB | 33.61 MiB/s, done.
Resolving deltas: 100% (4868/4868), done.
+ cd bazel-toolchains
+ docker run --rm -v /tmp/tmp.daBRqT16k7/bazel-toolchains:/srcdir -w /srcdir golang:1.23 go build -o rbe_configs_gen ./cmd/rbe_configs_gen/rbe_configs_gen.go
Unable to find image 'golang:1.23' locally
1.23: Pulling from library/golang
fd0410a2d1ae: Pull complete 
bf571be90f05: Pull complete 
684a51896c82: Pull complete 
941421258b3c: Pull complete 
06f05ace1117: Pull complete 
a103a99567f1: Pull complete 
4f4fb700ef54: Pull complete 
Digest: sha256:9820aca42262f58451f006de3213055974b36f24b31508c1baa73c967fcecb99
Status: Downloaded newer image for golang:1.23
go: downloading google.golang.org/genproto v0.0.0-20200527145253-8367513e4ece
go: downloading github.com/googleapis/gax-go/v2 v2.0.5
go: downloading cloud.google.com/go v0.57.0
go: downloading github.com/bazelbuild/bazelisk v1.7.4
go: downloading github.com/golang/protobuf v1.4.1
go: downloading github.com/coreos/go-semver v0.3.0
go: downloading google.golang.org/grpc v1.29.1
go: downloading github.com/hashicorp/go-version v1.2.1
go: downloading github.com/mitchellh/go-homedir v1.1.0
go: downloading google.golang.org/protobuf v1.24.0
go: downloading golang.org/x/net v0.0.0-20201110031124-69a78807bb2b
go: downloading golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f
go: downloading google.golang.org/api v0.22.0
go: downloading golang.org/x/text v0.3.4
go: downloading golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
go: downloading go.opencensus.io v0.22.3
go: downloading github.com/google/go-cmp v0.5.2
go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
+ popd
~/git/grpc
+ RBE_CONFIGS_GEN_TOOL_PATH=/tmp/tmp.daBRqT16k7/bazel-toolchains/rbe_configs_gen
+ LINUX_RBE_DOCKERFILE_DIR=tools/dockerfile/test/rbe_ubuntu2004
++ cat tools/dockerfile/test/rbe_ubuntu2004.current_version
+ LINUX_RBE_DOCKER_IMAGE=us-docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004:087453aa2851cb7b77b581c0b366ca4b53efa862@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68
+ BAZEL_VERSION=8.0.0
+ CONFIG_OUTPUT_PATH=third_party/toolchains/rbe_ubuntu2004_bazel8
+ rm -rf /usr/local/google/home/veblush/git/grpc/third_party/toolchains/rbe_ubuntu2004_bazel8
+ /tmp/tmp.daBRqT16k7/bazel-toolchains/rbe_configs_gen --bazel_version=8.0.0 --toolchain_container=us-docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004:087453aa2851cb7b77b581c0b366ca4b53efa862@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68 --output_src_root=/usr/local/google/home/veblush/git/grpc --output_config_path=third_party/toolchains/rbe_ubuntu2004_bazel8 --exec_os=linux --target_os=linux --generate_java_configs=false
2025/01/16 14:14:16 rbe_configs_gen.go \
2025/01/16 14:14:16 --toolchain_container="us-docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004:087453aa2851cb7b77b581c0b366ca4b53efa862@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68" \
2025/01/16 14:14:16 --exec_os="linux" \
2025/01/16 14:14:16 --target_os="linux" \
2025/01/16 14:14:16 --bazel_version="8.0.0" \
2025/01/16 14:14:16 --output_src_root="/usr/local/google/home/veblush/git/grpc" \
2025/01/16 14:14:16 --output_config_path="third_party/toolchains/rbe_ubuntu2004_bazel8" \
2025/01/16 14:14:16 --generate_java_configs=false \
2025/01/16 14:14:16 rbeconfigsgen.Options:
2025/01/16 14:14:16 BazelVersion="8.0.0"
2025/01/16 14:14:16 ToolchainContainer="us-docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004:087453aa2851cb7b77b581c0b366ca4b53efa862@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68"
2025/01/16 14:14:16 ExecOS="linux"
2025/01/16 14:14:16 TargetOS="linux"
2025/01/16 14:14:16 DockerPlatform=""
2025/01/16 14:14:16 OutputTarball=""
2025/01/16 14:14:16 OutputSourceRoot="/usr/local/google/home/veblush/git/grpc"
2025/01/16 14:14:16 OutputConfigPath="third_party/toolchains/rbe_ubuntu2004_bazel8"
2025/01/16 14:14:16 OutputManifest=""
2025/01/16 14:14:16 PlatformParams={ExecConstraints: [@platforms//os:linux @platforms//cpu:x86_64 @bazel_tools//tools/cpp:clang], TargetConstraints: [@platforms//os:linux @platforms//cpu:x86_64], CppToolchainTarget: "", ToolchainContainer: "", OSFamily: "Linux"}
2025/01/16 14:14:16 GenCPPConfigs=true
2025/01/16 14:14:16 CPPConfigTargets=[@local_config_cc//...]
2025/01/16 14:14:16 CPPConfigRepo="local_config_cc"
2025/01/16 14:14:16 CppBazelCmd="build"
2025/01/16 14:14:16 CppGenEnv=map[ABI_LIBC_VERSION:glibc_2.19 ABI_VERSION:clang BAZEL_COMPILER:clang BAZEL_HOST_SYSTEM:i686-unknown-linux-gnu BAZEL_TARGET_CPU:k8 BAZEL_TARGET_LIBC:glibc_2.19 BAZEL_TARGET_SYSTEM:x86_64-unknown-linux-gnu CC:clang CC_TOOLCHAIN_NAME:linux_gnu_x86]
2025/01/16 14:14:16 CppGenEnvJSON=""
2025/01/16 14:14:16 GenJavaConfigs=false
2025/01/16 14:14:16 JavaUseLocalRuntime=false
2025/01/16 14:14:16 TempWorkDir=""
2025/01/16 14:14:16 Cleanup=true
2025/01/16 14:14:16 Running: 'docker pull us-docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004:087453aa2851cb7b77b581c0b366ca4b53efa862@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68'
2025/01/16 14:14:17 Running: 'docker inspect --format={{index .RepoDigests 0}} us-docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004:087453aa2851cb7b77b581c0b366ca4b53efa862@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68'
2025/01/16 14:14:17 Resolved toolchain image "us-docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004:087453aa2851cb7b77b581c0b366ca4b53efa862@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68" to fully qualified reference "docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68".
2025/01/16 14:14:17 Running: 'docker create --rm docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68 sleep infinity'
2025/01/16 14:14:17 Created container ID d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f for toolchain container image docker.pkg.dev/grpc-testing/testing-images-public/rbe_ubuntu2004@sha256:b3eb1a17b7b091e3c5648a803076b2c40601242ff91c04d55997af6641305f68.
2025/01/16 14:14:17 Running: 'docker start d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f'
2025/01/16 14:14:18 Running: 'docker exec d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f mkdir /workdir'
2025/01/16 14:14:19 Running: 'docker cp /tmp/rbeconfigsgen_4198098463/bazelisk d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f:/workdir/bazelisk'
2025/01/16 14:14:19 Running: 'docker exec -w /workdir d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f chmod +x /workdir/bazelisk'
2025/01/16 14:14:19 Running: 'docker exec -w /workdir d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f mkdir /workdir/cpp_configs_project'
2025/01/16 14:14:19 Running: 'docker exec -w /workdir/cpp_configs_project d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f touch WORKSPACE BUILD.bazel'
2025/01/16 14:14:19 Running: 'docker exec -w /workdir/cpp_configs_project -e USE_BAZEL_VERSION=8.0.0 -e ABI_LIBC_VERSION=glibc_2.19 -e BAZEL_COMPILER=clang -e BAZEL_TARGET_CPU=k8 -e BAZEL_TARGET_SYSTEM=x86_64-unknown-linux-gnu -e CC=clang -e ABI_VERSION=clang -e BAZEL_HOST_SYSTEM=i686-unknown-linux-gnu -e BAZEL_TARGET_LIBC=glibc_2.19 -e CC_TOOLCHAIN_NAME=linux_gnu_x86 d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f /workdir/bazelisk build @local_config_cc//...'
2025/01/16 14:14:25 Output: 2025/01/16 22:14:19 Downloading https://releases.bazel.build/8.0.0/release/bazel-8.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Computing main repo mapping: 
WARNING: --enable_bzlmod is set, but no MODULE.bazel file was found at the workspace root. Bazel will create an empty MODULE.bazel file. Please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. For more details, please refer to https://github.com/bazelbuild/bazel/issues/18958.
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
WARNING: Target pattern parsing failed.
ERROR: Skipping '@local_config_cc//...': No repository visible as '@local_config_cc' from main repository
ERROR: No repository visible as '@local_config_cc' from main repository
INFO: Elapsed time: 4.908s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
2025/01/16 14:14:25 Running: 'docker stop -t 0 d86e0f3ef997d67ef0ae3f1d46863baa465a1454f2605a9d7eb91b088a856c2f'
2025/01/16 14:14:25 Config generation failed: Config generation failed: failed to generate C++ configs: Bazel was unable to build the C++ config generation targets in the toolchain container: exit status 1

[dougthor42]

+1. I think the root cause might be the fact that a lot (all?) of the previously-bundled rules/toolchains were split out in 8.0:

Many rules that were bundled with Bazel are now split into their own repositories, as part of the Starlarkification effort. This includes:

[dougthor42]

For those finding this in the future:

So far I haven't been able to get rbe_configs_gen to work directly with Bazel 8.x, but I was able to manually hack what it generates for 7.x so that things work with 8.x. The changes are:

Add bazel_dep(name = "rules_cc", version="0.2.0") to your MODULE.bazel. This can be dev_dependency=True.

Then adjust @bazel_tools stuff because those things were migrated to external repos. Note that @bazel_tools//tools/cpp:toolchain_type was not migrated.

--- a/config/cc/armeabi_cc_toolchain_config.bzl
+++ b/config/cc/armeabi_cc_toolchain_config.bzl
@@ -15,7 +15,7 @@
 """A Starlark cc_toolchain configuration rule"""
 
 load(
-    "@bazel_tools//tools/cpp:cc_toolchain_config_lib.bzl",
+    "@rules_cc//cc:cc_toolchain_config_lib.bzl",
     "feature",
     "tool_path",
 )

--- a/config/cc/cc_toolchain_config.bzl
+++ b/config/cc/cc_toolchain_config.bzl
@@ -14,9 +14,9 @@
 
 """A Starlark cc_toolchain configuration rule"""
 
-load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "ACTION_NAMES")
+load("@rules_cc//cc:action_names.bzl", "ACTION_NAMES")
 load(
-    "@bazel_tools//tools/cpp:cc_toolchain_config_lib.bzl",
+    "@rules_cc//cc:cc_toolchain_config_lib.bzl",
     "action_config",
     "artifact_name_pattern",
     "env_entry",

--- a/config/config/BUILD
+++ b/config/config/BUILD
@@ -23,7 +23,7 @@ toolchain(
     exec_compatible_with = [
         "@platforms//os:linux",
         "@platforms//cpu:x86_64",
-        "@bazel_tools//tools/cpp:clang",
+        "@rules_cc//cc/private/toolchain:clang",
     ],
     target_compatible_with = [
         "@platforms//os:linux",
@@ -40,7 +40,7 @@ platform(
     constraint_values = [
         "@platforms//os:linux",
         "@platforms//cpu:x86_64",
-        "@bazel_tools//tools/cpp:clang",
+        "@rules_cc//cc/private/toolchain:clang",
     ],
     exec_properties = {