Tracking issue: resource exhaustion cases

[cdstanford]

Tracking issues for crates causing exhaustion of resources (stack/memory/etc.) for the current binary.

See: 9518569, 83ddaec

• on top10: syn crash

• on top100: still investigating idna and futures-util stack overflows

Update (2024-08-27): All crates OK on top10 and top100.

Still seeing some crashes on top10000 crates, e.g.

$ cargo run --bin scan data/packages/tryhard     
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
     Running `target/debug/scan data/packages/tryhard`
thread 'main' panicked at /Users/caleb/.cargo/registry/src/index.crates.io-6f17d22bba15001f/chalk-recursive-0.93.0/src/fixed_point/stack.rs:51:13:
overflow depth reached
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[cdstanford]

Tracking CSV: data/crate-lists/known_crashes.csv

[cdstanford]

Crash on idna -- shows up only on scan_all, not on scan for some reason.

% cargo run --release --bin scan_all -- data/crate-lists/known_crashes.csv kc -n 1  
    Finished release [optimized] target(s) in 0.16s
     Running `target/release/scan_all data/crate-lists/known_crashes.csv kc -n 1`
[2023-11-11T18:52:41Z WARN  cargo_scan::scan_stats] Scan crashed, skipping crate: data/packages/syn
50% complete

thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow
zsh: abort      cargo run --release --bin scan_all -- data/crate-lists/known_crashes.csv kc -

Further info on the rust-analyzer issue: rust-lang/rust-analyzer#15873

Backtrace:

#0  0x00007ffff76b950b in _int_malloc (av=av@entry=0x7fffd8000030, bytes=bytes@entry=4) at malloc.c:3839
#1  0x00007ffff76bacad in __GI___libc_malloc (bytes=4) at malloc.c:3329
#2  0x0000555555d7df8e in <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::hb41cdb5deae7e7de ()
#3  0x0000555555bcb94f in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#4  0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#5  0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#6  0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#7  0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#8  0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#9  0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#10 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#11 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#12 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#13 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#14 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#15 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#16 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#17 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#18 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#19 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#20 0x0000555555bcb870 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
…
#7200 0x0000555555bcb770 in ra_ap_hir_ty::mir::borrowck::ever_initialized_map::dfs::h8cd5d4310d767a9f ()
#7201 0x0000555555bcbe4b in ra_ap_hir_ty::mir::borrowck::mutability_of_locals::hba525a970b1f346d ()
#7202 0x0000555555b873cf in <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold::hf643a2707da45619 ()
#7203 0x0000555555d84e2a in <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::hf86859ff1ec06fa1 ()
#7204 0x0000555555c0ec8b in core::iter::adapters::try_process::hb1a6ca047a711b06 ()
#7205 0x0000555555bcaa43 in ra_ap_hir_ty::mir::borrowck::borrowck_query::h357bad0a7674f275 ()
#7206 0x0000555555b1e65f in salsa::runtime::Runtime::execute_query_implementation::h77d2ff2ad3db514a ()
#7207 0x0000555555cfbea4 in salsa::derived::slot::Slot<Q,MP>::read_upgrade::hc5e8188eaad65a77 ()
#7208 0x0000555555d30797 in salsa::derived::slot::Slot<Q,MP>::read::h99414e96809d084d ()
#7209 0x0000555555bab7eb in <salsa::derived::DerivedStorage<Q,MP> as salsa::plumbing::QueryStorageOps<Q>>::try_fetch::h7fc8a5dfbc1f6b7b ()
#7210 0x0000555555bccd5b in <DB as ra_ap_hir_ty::db::HirDatabase>::borrowck::__shim::he3b6df767df7b6fd ()
#7211 0x00005555558c3ad4 in <DB as ra_ap_hir_ty::db::HirDatabase>::borrowck::hb464843fa18e085b ()
#7212 0x0000555555993517 in ra_ap_hir::DefWithBody::diagnostics::h1f0689a8a59eb3ef ()
#7213 0x000055555598c1b3 in ra_ap_hir::ModuleDef::diagnostics::h02f4137f8ba61937 ()
#7214 0x000055555598ce41 in ra_ap_hir::Module::diagnostics::hce741c1ba0ea1e71 ()
#7215 0x0000555555894a06 in ra_ap_ide_diagnostics::diagnostics::h33df8498272e49b2 ()
#7216 0x0000555555890a59 in salsa::Cancelled::catch::h3c9deead3624cb6c ()
#7217 0x0000555555886bb5 in ra_ap_ide::Analysis::diagnostics::hf5a6c6ab257c0f98 ()
#7218 0x000055555573df6f in cargo_scan::resolution::name_resolution::ResolverImpl::new::ha94577dd2587845a ()
#7219 0x00005555557402f8 in cargo_scan::resolution::resolve::FileResolver::new::hc1b4eaba8cfe377c ()
#7220 0x000055555573be22 in cargo_scan::scanner::scan_file::h80130fe46bd9504e ()
#7221 0x000055555573c216 in cargo_scan::scanner::try_scan_file::hfa5e8554797191aa ()
#7222 0x000055555573c9ef in cargo_scan::scanner::scan_crate_with_sinks::h98ead6d5212ea6b9 ()
#7223 0x000055555574a24d in cargo_scan::audit_file::AuditFile::scan_with_sinks::h91d5ae165a6f895b ()
#7224 0x000055555574a3ec in cargo_scan::audit_file::AuditFile::new_caller_checked_default_with_sinks_and_results::h7f70860d857976ec ()
#7225 0x000055555574f250 in cargo_scan::scan_stats::get_crate_stats::h09e65ad4da9206f1 ()
#7226 0x000055555574ee65 in cargo_scan::scan_stats::get_crate_stats_default::h9e3ca13835bf59e2 ()
#7227 0x00005555556f2629 in <F as threadpool::FnBox>::call_box::hc22dfd90e5514964 ()
#7228 0x000055555571d650 in std::sys_common::backtrace::__rust_begin_short_backtrace::h8191077cb2020d77 ()
#7229 0x00005555557193da in core::ops::function::FnOnce::call_once{{vtable.shim}}::h36184550a97ce966 ()
#7230 0x00005555564029d5 in alloc::boxed::{impl#47}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> ()
    at library/alloc/src/boxed.rs:2007
#7231 alloc::boxed::{impl#47}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2007
#7232 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#7233 0x00007ffff76aa9eb in start_thread (arg=<optimized out>) at pthread_create.c:444
#7234 0x00007ffff772e7cc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

[cdstanford]

Another stack overflow crash on futures-util

[cdstanford]

update: stack overflows fixed for now! Disabled the offending code in
30d47eb

[cdstanford]

Found another stack overflow on tryhard, disabled for now in 74ca0cd