CLI omits ~53% of Rust files as 'isolated extraction failed' while the core library extracts them fine — silently partial index

[emanzx]

Summary

code2graph-cli reports reason=extraction-error detail=isolated extraction failed for roughly half of all Rust files in a project — but the core library extracts those exact same files successfully.

The result is a silently partial graph: index reports success, the cache is populated, and queries return plausible-looking answers computed from ~47% of the code.

Measured on two independent large Rust codebases, including this repo:

project	.rs files	omitted by CLI
a mid-size workspace (mae8-v2)	169	97	57%
nodedb @ v0.4.0	4556	2416	53%

Every omission is reason=extraction-error — none are the size/count limits.

The core library is fine — it's the CLI path

Same file, two surfaces:

$ code2graph index . --allow-partial --no-cache
indexed 0 files; 0 changed, 0 deleted; partial
omitted files=1
omitted src/recency.rs reason=extraction-error detail=isolated extraction failed

// direct library call, same file
code2graph::extract_path("src/recency.rs", &src)
// => Ok(FileFacts { symbols: 2, references: 17 })   in ~60ms

So extraction itself succeeds. Something between extract and the CLI's acceptance of the result is rejecting it.

Reproduction

Any of the ~2400 affected files in this repo will do; here is a self-contained 27-line one (from mae8-v2, Apache-2.0) that fails on its own in an otherwise empty project:

// recency — content-time helper for sort correctness.
//
// Ported verbatim from mae8-mcp/src/tools/timesort.rs::content_time_secs.
//
// After a v1→v2 migration every migrated entry's `indexed_ms` ≈ the migration
// moment, so sorting by `indexed_ms` collapses "recent" lists to
// migration-order rather than real chronology. The real event-times live in
// `extra` fields. `content_time_secs` derives the best available event-time per
// entry kind, falling back to the store write-time when none is present.

use crate::entry::types::Entry;
use nodedb_types::value::Value;

/// Best available "content time" for recency sorting, in unix SECONDS.
/// Prefers the entry's real event-time from `extra` (per-kind conventions),
/// falling back to the store write-time `indexed_ms` (ms → s) when none is set.
pub fn content_time_secs(entry: &Entry) -> i64 {
    // Priority: the most event-meaningful field per kind. All are unix seconds.
    for key in ["session_date_unix", "anointed_at_unix", "gist_at_unix", "created_at_unix"] {
        if let Some(Value::Integer(t)) = entry.extra.get(key)
            && *t > 0
        {
            return *t;
        }
    }
    entry.indexed_ms.map(|ms| (ms / 1000) as i64).unwrap_or(0)
}

mkdir -p repro/src && cd repro
# save the above as src/recency.rs
code2graph index . --allow-partial --no-cache
#   omitted src/recency.rs reason=extraction-error detail=isolated extraction failed

What I ruled out

• Not the resolution tier — --tier name and --tier scope omit an identical 249 files.
• Not project size or cross-file resolution — a single file alone in a fresh project still fails.
• Not the size/count limits — 100% of omissions report extraction-error, never file-too-large / total-bytes-limit / file-count-limit.
• Not a specific syntax feature (as far as I could tell) — I hand-rebuilt the failing file's constructs in isolation (let-chains in both cond && let and let … && cond forms, multi-line formatting, let-else, nested enum patterns inside a for, async + generics, non-ASCII in comments and string literals). Every reconstruction extracted fine. So it does not appear to be an obvious grammar gap — which is what made me suspect the CLI-side path rather than the extractor.
• Not stale — installed from 93b9709c, current main at the time of testing.

Where I'd look

In cli/src/refresh/prepare.rs, the post-extraction arm:

Ok(mut value) => {
    packages.enrich_file_facts(&mut value);
    if validate_file_facts(std::slice::from_ref(&value)).is_err() {
        // -> OmissionReason::ExtractionError

OmissionReason::ExtractionError is reachable from two distinct causes — a worker returning WorkerErrorCode::Extraction, and a locally-successful extraction whose facts fail validate_file_facts after enrich_file_facts has mutated them. Given the library extracts these files cleanly, the second path (enrichment producing facts that then fail validation) looks like the likelier candidate — but the two are indistinguishable from the outside, because the underlying error is discarded before it reaches the operator.

Two suggestions

1. Fix the underlying rejection (wherever it lands).
2. Independently: stop discarding the reason. Right now both causes collapse into detail=isolated extraction failed, so an operator cannot tell a worker crash from a validation rejection, and cannot see which validation rule fired. Surfacing the real error (behind a flag if it must stay quiet by default) would have made this diagnosable without reading the source. The same applies to the aggregate: index currently reports overall success while dropping half the project — omissions are only visible if you pass --allow-partial and read the output. A partial index that looks complete is the risky part; a loud summary line (indexed N, omitted M (X%)) on every run would surface it.

Notes

Happy to test a patch, run the repro against other codebases, or provide the full omission list from either project. Also glad to be told I'm holding it wrong — if there's a flag or configuration that avoids this, I couldn't find it in --help or the docs.