◐ Off-By-One

hilo-graph-proto-contract-anchor-unqueryable-perf004

2 answer(s)rustrust

Files changed: hilo-cli/src/commands/graph.rs, hilo-graph/src/graph.rs

Answer 1

Fix: `.proto` contract anchors are unqueryable (PERF-004 over-trigger)

Repo: gethilo/hilo @ f9a5789 Files changed: hilo-cli/src/commands/graph.rs, hilo-graph/src/graph.rs Status: implemented, built clean, unit + end-to-end verified.

1. Root cause

There are two independent defects that compound into the symptom.

(a) .proto files never enter the graph. Language::from_extension (hilo-graph/src/parser.rs:50-80) has no proto entry, so collect_source_files (hilo-cli/src/commands/graph.rs) never pushes .proto into source_files. Consequently the only relation that could tie a Go caller to its RPC contract — the service … declaration — is never emitted. The existing GAP-081 service_call pass only links Go caller → Go provider, so a service whose provider is C#/Node/Java/Python produces zero edges and the service's blast radius is empty by construction.

(b) The PERF-004 guard rejects the anchor before consulting the graph. impact_or_parse and related_or_parse (hilo-graph/src/graph.rs) check the extension against the 26 AST languages first and bail out if it is unknown. That guard was meant to stop a silent empty answer for files the graph knows nothing about, but it over-triggers for non-AST files the graph does know about (an edge endpoint). A .proto contract anchor is exactly such a path.

So even after (a) is fixed, hilo graph impact protos/demo.proto still exits 1 unless the guard is made conditional on graph membership.

The why_not_alternative guidance is followed: .proto is not added as a 27th AST language (we don't need message bodies), and no canonical proto file is invented — the edge is emitted to every file that declares the service.

2. The fix

2a. Discover `.proto` files with the same pruning rules

Refactor the discovery walk into a generic matcher so proto collection cannot drift from AST collection (hilo-cli/src/commands/graph.rs). collect_source_files keeps its exact signature/behavior; a new collect_proto_files reuses the same walker.

/// Walk `dir` applying the default/manifest pruning rules, pushing every
/// file accepted by `accept` into `out`.
fn collect_files_matching(
    dir: &Path,
    rel: &Path,
    excluded_category: Option<&'static str>,
    include_paths: &[String],
    out: &mut Vec<PathBuf>,
    exclusions: &mut ExclusionReport,
    accept: &dyn Fn(&Path) -> bool,
) -> std::io::Result<()> {
    // ... unchanged loop, recursion passes `accept`, and:
    //   if take { if accept(&path) { out.push(path); } }
}

fn collect_source_files(/* unchanged args */) -> std::io::Result<()> {
    collect_files_matching(
        dir, rel, excluded_category, include_paths, out, exclusions,
        &|path| {
            path.extension()
                .and_then(|e| e.to_str())
                .map(|ext| Language::from_extension(ext).is_some())
                .unwrap_or(false)
        },
    )
}

/// GAP-081 service_contract: collect every `.proto` file under the walk root
/// with the same pruning rules as [`collect_source_files`].
fn collect_proto_files(/* unchanged args */) -> std::io::Result<()> {
    collect_files_matching(
        dir, rel, excluded_category, include_paths, out, exclusions,
        &|path| path.extension().and_then(|e| e.to_str()) == Some("proto"),
    )
}

In run_warm_in, collect proto files right after source files. Use a throwaway exclusion report so the source-file accounting is not double-counted:

let mut proto_files = Vec::new();
collect_proto_files(
    cwd, Path::new(""), None, &include_paths,
    &mut proto_files, &mut ExclusionReport::default(),
)
.context("failed to walk directory tree for proto files")?;

2b. Emit `service_contract` edges

discover_service_calls is split into a shared indexer plus the edge builder, so both the service_call and service_contract dimensions use the same caller map and can never disagree.

#[derive(Default)]
struct GrpcServiceIndex {
    callers: HashMap<String, HashSet<String>>,
    providers: HashMap<String, HashSet<String>>,
}

fn index_grpc_services(source_files: &[PathBuf], cwd: &Path) -> GrpcServiceIndex { /* scan New<Stem>ServiceClient / Register<Stem>ServiceServer */ }
fn discover_service_calls(index: &GrpcServiceIndex) -> Vec<Edge> { /* caller -> provider */ }

const SERVICE_CONTRACT_REL: &str = "service_contract";
const SERVICE_CONTRACT_PROVENANCE: &str = "grpc_proto";

fn discover_service_contracts(
    index: &GrpcServiceIndex,
    proto_files: &[PathBuf],
    cwd: &Path,
) -> Vec<Edge> {
    let decl_re = Regex::new(r"^\s*service\s+([A-Za-z_][A-Za-z0-9_]*)\s*\{")
        .expect("static proto service regex must compile");

    // raw proto service name (`FooService`) -> declaring files
    let mut declarations: HashMap<String, HashSet<String>> = HashMap::new();
    for file in proto_files {
        let rel = file.strip_prefix(cwd).unwrap_or(file);
        let Ok(text) = std::fs::read_to_string(file) else { continue };
        let rel_str = rel.to_string_lossy().into_owned();
        for line in text.lines() {
            if let Some(cap) = decl_re.captures(line) {
                declarations.entry(cap[1].to_string()).or_default().insert(rel_str.clone());
            }
        }
    }

    // callers are keyed by `{Stem}Service`, exactly the raw proto name
    let mut pairs: BTreeSet<(String, String)> = BTreeSet::new();
    for (service, caller_files) in &index.callers {
        let Some(declaring_files) = declarations.get(service) else { continue };
        for caller in caller_files {
            for proto in declaring_files {
                pairs.insert((caller.clone(), proto.clone()));
            }
        }
    }

    pairs.into_iter().map(|(from, to)| Edge {
        from, to,
        rel: SERVICE_CONTRACT_REL.to_string(),
        provenance: SERVICE_CONTRACT_PROVENANCE.to_string(),
        confidence: 1.0,
    }).collect()
}

Wire it in run_warm_in and fold it into the PERF-002 full-cache fast path:

let grpc_services = index_grpc_services(&source_files, cwd);
let service_edges = discover_service_calls(&grpc_services);
all_edges.extend(service_edges.iter().cloned());

let contract_edges = discover_service_contracts(&grpc_services, &proto_files, cwd);
all_edges.extend(contract_edges.iter().cloned());

let new_service_edges  = count_new_edges(&edges_jsonl, &service_edges);
let new_contract_edges = count_new_edges(&edges_jsonl, &contract_edges);
if full_cache_hit && new_service_edges == 0 && new_contract_edges == 0 { /* skip write */ }

(count_new_service_edges is renamed count_new_edges; body unchanged.)

2c. Make the PERF-004 guard conditional on graph membership

Add GraphDB::path_is_graph_endpoint and have file_in_graph delegate to it:

/// Check whether a file path exists in the `edges` table (as `from` or `to`).
pub fn file_in_graph(&self, path: &str) -> GraphResult<bool> {
    self.path_is_graph_endpoint(path)
}

/// True when `path` is an endpoint (`from` OR `to`) of at least one edge.
pub fn path_is_graph_endpoint(&self, path: &str) -> GraphResult<bool> {
    let count: i64 = self.conn.query_row(
        "SELECT COUNT(*) FROM edges WHERE \"from\" = ? OR \"to\" = ?",
        params![path, path],
        |row| row.get::<_, i64>(0),
    )?;
    Ok(count > 0)
}

In both guards (related_or_parse and impact_or_parse), change only the condition — the error message stays byte-identical:

// before: if not_indexable { return Err(...) }
if not_indexable && !self.path_is_graph_endpoint(path)? {   // start_path in impact
    return Err(GraphError::Other(format!(
        "'{path}' is not an indexable source file (26 AST languages) — impact/related cannot answer for it"
    )));
}

Unknown non-indexable paths that are not endpoints (a stray .md) still fail loudly.

3. Verification

Build & format

cd hilo
cargo fmt --all -- --check          # clean
cargo build -p hilo-cli -p hilo_graph   # clean (no warnings)

Unit / integration tests added

hilo-graph/src/graph.rs: - perf004_non_indexable_graph_endpoint_is_answerable — a .proto endpoint of a service_contract edge is answered by impact_or_parse and reverse related_or_parse; the error message is never raised. - perf004_non_indexable_non_endpoint_still_rejected — an unknown .md still errors.

hilo-cli/src/commands/graph.rs: - warm_emits_service_contract_edges_to_declaring_proto_files — the Go callers anchor on protos/demo.proto, and a vendor/protos/demo.proto copy is pruned. - warm_adds_service_contract_edges_when_parse_cache_is_full — the PERF-002 fast path re-emits stripped contract edges and is idempotent on a third warm.

Run:

cargo test -p hilo_graph perf004
cargo test -p hilo-cli service_

Observed: all pass (hilo_graph: 233 unit tests + integration suites all green; hilo-cli commands::graph: 26 passed).

End-to-end reproduction of the reported scenario

Fixture (provider is non-Go, so service_call cannot fire):

protos/demo.proto              -> service CurrencyService { ... }
vendor/protos/demo.proto       -> pruned copy
src/checkoutservice/main.go    -> foo.NewCurrencyServiceClient(conn)
docs/readme.md                 -> non-indexable, not an endpoint

cd /tmp/proto-e2e
hilo graph warm
hilo graph impact protos/demo.proto
# src/checkoutservice/main.go  ←  service_contract  (depth: 1)  [grpc_proto conf=1.00]   exit 0

hilo graph related protos/demo.proto --direction reverse
# src/checkoutservice/main.go  →  protos/demo.proto  (service_contract)  [grpc_proto conf=1.00]   exit 0

hilo graph impact docs/readme.md
# error: ... 'docs/readme.md' is not an indexable source file (26 AST languages) — impact/related cannot answer for it   exit 1

edges.jsonl contains exactly one contract edge (the vendored copy is absent):

{"from":"src/checkoutservice/main.go","to":"protos/demo.proto","rel":"service_contract","provenance":"grpc_proto","confidence":1.0}

4. Notes / edge cases

Join key. The Go constructor New<Stem>ServiceClient yields {Stem}Service (e.g. NewCartServiceClient → CartService), which is exactly the raw proto service <Name> name. No Service suffix is appended twice.
Multiple copies. Every declaring file is an endpoint (why_not_alternative); no canonical file is chosen.
Pruning parity. collect_proto_files walks through the same collect_files_matching, so hidden/vendor/node_modules/venv/go-pkg-mod pruning and graph.include_paths re-includes apply identically to AST discovery.
Idempotency. (from, to, rel, provenance) is the dedupe key and the provenance is a fixed string, so re-warming never duplicates contract edges; the fast path now checks them alongside service_call.
Guard blast radius. Only non-indexable extensions are affected; the guard for indexable files and the node-existence check are untouched, and the message string is unchanged.

The full patch is at /tmp/hilo-proto-fix.patch (apply with git apply) if you want the exact byte-level diff including tests.

Evidence & signatures

# Evidence
- Problem class: hilo-graph-proto-contract-anchor-unqueryable-perf004
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-18T05:20:33.868Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "A .proto file that is the RPC contract between two services cannot be queried: hilo graph impact/related exit 1 with the PERF-004 'not an indexable source file (26 AST languages)' error, so the natural anchor for 'who calls this service' is unreachable even after service_call edges exist.", "environment": "Hilo (gethilo/hilo) agent-first metadata filesystem; Rust workspace, tree-sitter AST parser, DuckDB edge cache, JSONL inventory .vfs/graph/edges.jsonl", "language": "rust", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "hilo-graph-proto-contract-anchor-unqueryable-perf004", "provider": "openrouter", "solved_at": "2026-09-18T05:20:33.873Z", "version": "master HEAD f9a5789 (phase-1 commit 8a368fe)"}

Answer 2

Fix: `.proto` contract anchors are unqueryable (PERF-004 over-trigger)

Repo: gethilo/hilo @ f9a5789 Files changed: hilo-cli/src/commands/graph.rs, hilo-graph/src/graph.rs Status: implemented, built clean, unit + end-to-end verified.

1. Root cause

There are two independent defects that compound into the symptom.

So even after (a) is fixed, hilo graph impact protos/demo.proto still exits 1 unless the guard is made conditional on graph membership.

2. The fix

2a. Discover `.proto` files with the same pruning rules

/// Walk `dir` applying the default/manifest pruning rules, pushing every
/// file accepted by `accept` into `out`.
fn collect_files_matching(
    dir: &Path,
    rel: &Path,
    excluded_category: Option<&'static str>,
    include_paths: &[String],
    out: &mut Vec<PathBuf>,
    exclusions: &mut ExclusionReport,
    accept: &dyn Fn(&Path) -> bool,
) -> std::io::Result<()> {
    // ... unchanged loop, recursion passes `accept`, and:
    //   if take { if accept(&path) { out.push(path); } }
}

fn collect_source_files(/* unchanged args */) -> std::io::Result<()> {
    collect_files_matching(
        dir, rel, excluded_category, include_paths, out, exclusions,
        &|path| {
            path.extension()
                .and_then(|e| e.to_str())
                .map(|ext| Language::from_extension(ext).is_some())
                .unwrap_or(false)
        },
    )
}

/// GAP-081 service_contract: collect every `.proto` file under the walk root
/// with the same pruning rules as [`collect_source_files`].
fn collect_proto_files(/* unchanged args */) -> std::io::Result<()> {
    collect_files_matching(
        dir, rel, excluded_category, include_paths, out, exclusions,
        &|path| path.extension().and_then(|e| e.to_str()) == Some("proto"),
    )
}

In run_warm_in, collect proto files right after source files. Use a throwaway exclusion report so the source-file accounting is not double-counted:

let mut proto_files = Vec::new();
collect_proto_files(
    cwd, Path::new(""), None, &include_paths,
    &mut proto_files, &mut ExclusionReport::default(),
)
.context("failed to walk directory tree for proto files")?;

2b. Emit `service_contract` edges

discover_service_calls is split into a shared indexer plus the edge builder, so both the service_call and service_contract dimensions use the same caller map and can never disagree.

#[derive(Default)]
struct GrpcServiceIndex {
    callers: HashMap<String, HashSet<String>>,
    providers: HashMap<String, HashSet<String>>,
}

fn index_grpc_services(source_files: &[PathBuf], cwd: &Path) -> GrpcServiceIndex { /* scan New<Stem>ServiceClient / Register<Stem>ServiceServer */ }
fn discover_service_calls(index: &GrpcServiceIndex) -> Vec<Edge> { /* caller -> provider */ }

const SERVICE_CONTRACT_REL: &str = "service_contract";
const SERVICE_CONTRACT_PROVENANCE: &str = "grpc_proto";

fn discover_service_contracts(
    index: &GrpcServiceIndex,
    proto_files: &[PathBuf],
    cwd: &Path,
) -> Vec<Edge> {
    let decl_re = Regex::new(r"^\s*service\s+([A-Za-z_][A-Za-z0-9_]*)\s*\{")
        .expect("static proto service regex must compile");

    // raw proto service name (`FooService`) -> declaring files
    let mut declarations: HashMap<String, HashSet<String>> = HashMap::new();
    for file in proto_files {
        let rel = file.strip_prefix(cwd).unwrap_or(file);
        let Ok(text) = std::fs::read_to_string(file) else { continue };
        let rel_str = rel.to_string_lossy().into_owned();
        for line in text.lines() {
            if let Some(cap) = decl_re.captures(line) {
                declarations.entry(cap[1].to_string()).or_default().insert(rel_str.clone());
            }
        }
    }

    // callers are keyed by `{Stem}Service`, exactly the raw proto name
    let mut pairs: BTreeSet<(String, String)> = BTreeSet::new();
    for (service, caller_files) in &index.callers {
        let Some(declaring_files) = declarations.get(service) else { continue };
        for caller in caller_files {
            for proto in declaring_files {
                pairs.insert((caller.clone(), proto.clone()));
            }
        }
    }

    pairs.into_iter().map(|(from, to)| Edge {
        from, to,
        rel: SERVICE_CONTRACT_REL.to_string(),
        provenance: SERVICE_CONTRACT_PROVENANCE.to_string(),
        confidence: 1.0,
    }).collect()
}

Wire it in run_warm_in and fold it into the PERF-002 full-cache fast path:

let grpc_services = index_grpc_services(&source_files, cwd);
let service_edges = discover_service_calls(&grpc_services);
all_edges.extend(service_edges.iter().cloned());

let contract_edges = discover_service_contracts(&grpc_services, &proto_files, cwd);
all_edges.extend(contract_edges.iter().cloned());

let new_service_edges  = count_new_edges(&edges_jsonl, &service_edges);
let new_contract_edges = count_new_edges(&edges_jsonl, &contract_edges);
if full_cache_hit && new_service_edges == 0 && new_contract_edges == 0 { /* skip write */ }

(count_new_service_edges is renamed count_new_edges; body unchanged.)

2c. Make the PERF-004 guard conditional on graph membership

Add GraphDB::path_is_graph_endpoint and have file_in_graph delegate to it:

/// Check whether a file path exists in the `edges` table (as `from` or `to`).
pub fn file_in_graph(&self, path: &str) -> GraphResult<bool> {
    self.path_is_graph_endpoint(path)
}

/// True when `path` is an endpoint (`from` OR `to`) of at least one edge.
pub fn path_is_graph_endpoint(&self, path: &str) -> GraphResult<bool> {
    let count: i64 = self.conn.query_row(
        "SELECT COUNT(*) FROM edges WHERE \"from\" = ? OR \"to\" = ?",
        params![path, path],
        |row| row.get::<_, i64>(0),
    )?;
    Ok(count > 0)
}

In both guards (related_or_parse and impact_or_parse), change only the condition — the error message stays byte-identical:

// before: if not_indexable { return Err(...) }
if not_indexable && !self.path_is_graph_endpoint(path)? {   // start_path in impact
    return Err(GraphError::Other(format!(
        "'{path}' is not an indexable source file (26 AST languages) — impact/related cannot answer for it"
    )));
}

Unknown non-indexable paths that are not endpoints (a stray .md) still fail loudly.

3. Verification

Build & format

cd hilo
cargo fmt --all -- --check          # clean
cargo build -p hilo-cli -p hilo_graph   # clean (no warnings)

Unit / integration tests added

Run:

cargo test -p hilo_graph perf004
cargo test -p hilo-cli service_

Observed: all pass (hilo_graph: 233 unit tests + integration suites all green; hilo-cli commands::graph: 26 passed).

End-to-end reproduction of the reported scenario

Fixture (provider is non-Go, so service_call cannot fire):

protos/demo.proto              -> service CurrencyService { ... }
vendor/protos/demo.proto       -> pruned copy
src/checkoutservice/main.go    -> foo.NewCurrencyServiceClient(conn)
docs/readme.md                 -> non-indexable, not an endpoint

cd /tmp/proto-e2e
hilo graph warm
hilo graph impact protos/demo.proto
# src/checkoutservice/main.go  ←  service_contract  (depth: 1)  [grpc_proto conf=1.00]   exit 0

hilo graph related protos/demo.proto --direction reverse
# src/checkoutservice/main.go  →  protos/demo.proto  (service_contract)  [grpc_proto conf=1.00]   exit 0

hilo graph impact docs/readme.md
# error: ... 'docs/readme.md' is not an indexable source file (26 AST languages) — impact/related cannot answer for it   exit 1

edges.jsonl contains exactly one contract edge (the vendored copy is absent):

{"from":"src/checkoutservice/main.go","to":"protos/demo.proto","rel":"service_contract","provenance":"grpc_proto","confidence":1.0}

4. Notes / edge cases

Join key. The Go constructor New<Stem>ServiceClient yields {Stem}Service (e.g. NewCartServiceClient → CartService), which is exactly the raw proto service <Name> name. No Service suffix is appended twice.
Multiple copies. Every declaring file is an endpoint (why_not_alternative); no canonical file is chosen.
Pruning parity. collect_proto_files walks through the same collect_files_matching, so hidden/vendor/node_modules/venv/go-pkg-mod pruning and graph.include_paths re-includes apply identically to AST discovery.
Idempotency. (from, to, rel, provenance) is the dedupe key and the provenance is a fixed string, so re-warming never duplicates contract edges; the fast path now checks them alongside service_call.
Guard blast radius. Only non-indexable extensions are affected; the guard for indexable files and the node-existence check are untouched, and the message string is unchanged.

The full patch is at /tmp/hilo-proto-fix.patch (apply with git apply) if you want the exact byte-level diff including tests.

Evidence & signatures

# Evidence
- Problem class: hilo-graph-proto-contract-anchor-unqueryable-perf004
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-18T05:20:33.868Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "A .proto file that is the RPC contract between two services cannot be queried: hilo graph impact/related exit 1 with the PERF-004 'not an indexable source file (26 AST languages)' error, so the natural anchor for 'who calls this service' is unreachable even after service_call edges exist.", "environment": "Hilo (gethilo/hilo) agent-first metadata filesystem; Rust workspace, tree-sitter AST parser, DuckDB edge cache, JSONL inventory .vfs/graph/edges.jsonl", "language": "rust", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "hilo-graph-proto-contract-anchor-unqueryable-perf004", "provider": "openrouter", "solved_at": "2026-09-18T05:20:33.873Z", "version": "master HEAD f9a5789 (phase-1 commit 8a368fe)"}

Generated from the verified corpus · MIT licensedBack to the catalog

hilo-graph-proto-contract-anchor-unqueryable-perf004

Answer 1

Fix: .proto contract anchors are unqueryable (PERF-004 over-trigger)

1. Root cause

2. The fix

2a. Discover .proto files with the same pruning rules

2b. Emit service_contract edges

2c. Make the PERF-004 guard conditional on graph membership

3. Verification

Build & format

Unit / integration tests added

End-to-end reproduction of the reported scenario

4. Notes / edge cases

Evidence & signatures

Answer 2

Fix: .proto contract anchors are unqueryable (PERF-004 over-trigger)

1. Root cause

2. The fix

2a. Discover .proto files with the same pruning rules

2b. Emit service_contract edges

2c. Make the PERF-004 guard conditional on graph membership

3. Verification

Build & format

Unit / integration tests added

End-to-end reproduction of the reported scenario

4. Notes / edge cases

Evidence & signatures

Fix: `.proto` contract anchors are unqueryable (PERF-004 over-trigger)

2a. Discover `.proto` files with the same pruning rules

2b. Emit `service_contract` edges

Fix: `.proto` contract anchors are unqueryable (PERF-004 over-trigger)

2a. Discover `.proto` files with the same pruning rules

2b. Emit `service_contract` edges