Spaces:
Build error
fix(rust): smart_crusher scaffold review findings — hash truncation, int parse, python-repr matcher
Browse filesCode review (`/code-review` on commit `7735dd6`) caught one critical
bug, two important parity gaps, and a few quality nits. Fixed all of
them; all 135 unit tests pass; diff_compressor parity harness
unaffected (27/27 still matched).
# Critical fix — `hash_field_name` truncation length
Rust truncated SHA-256 to **16** hex chars; Python uses **8** (per
`smart_crusher.py:177`: `hashlib.sha256(...).hexdigest()[:8]`). 16-char
hashes would never collide with TOIN's 8-char `preserve_fields`,
silently disabling the entire `use_feedback_hints` cache lookup path.
Fix: `hex[..8]` instead of `hex[..16]`. Three pinning tests re-verified
against actual Python reference output. Doc comment now warns
explicitly that the length must match Python or TOIN lookups silently miss.
# Important fix — `python_int_parse` mirrors Python's `int()` semantics
`statistics.rs::detect_sequential_pattern` previously called
`s.parse::<i64>()`. Python's `int()` differs in three ways that affect
realistic payloads:
- strips ASCII whitespace (Rust's `parse` rejects)
- accepts leading `+` (Rust accepts; same)
- accepts PEP 515 underscores like `"3_000"` (Rust rejects)
A field with `[" 1 ", " 2 ", " 3 ", "4", "5"]` would parse all five
in Python (sequential = True) but only one in Rust (`nums.len() < 5`
→ False). Silent parity break.
Fix: new private `python_int_parse` helper that strips whitespace,
handles underscore separators, and rejects edge cases Python rejects.
Six new tests pin the behavior.
# Important fix — `python_repr` for `item_matches_anchors`
Python compares anchors via `anchor in str(item).lower()`. We were
using `serde_json::to_string(&item).to_lowercase()`, which differs in
three ways that affect substring matching:
- quote chars (`'` vs `"`)
- bool/null literals (`True`/`False`/`None` vs `true`/`false`/`null`)
- spacing (`key: value, ...` vs `key:value,...`)
Anchor `"none"` would match Python form but not JSON. Inverse for
`"null"`. Real divergence.
Fix: new private `python_repr` walks `serde_json::Value` and emits
Python-equivalent form. Plus enable `serde_json/preserve_order` at
workspace level so `Value::Object` preserves JSON parse order
(matching Python `dict` since 3.7).
# Suggestion fixes
- Classifier comment for `[True, False, 1] -> MIXED_ARRAY` now walks
both Python and Rust paths step by step.
- `ArrayAnalysis::field_stats` doc notes the BTreeMap vs Python-dict
order nuance for the analyzer port to resolve.
- Added regression tests for "all unparseable strings", "single int
among strings", fractional-step sequential, and the email-typo
pattern.
# Build / test
- `cargo build -p headroom-core` clean.
- `cargo clippy -p headroom-core -- -D warnings` clean.
- 135 unit tests in `headroom-core`, all passing (was 55).
- `cargo run -p headroom-parity run` — diff_compressor 27/27 still matched.
- .claude-plugin/marketplace.json +2 -2
- .github/plugin/marketplace.json +2 -2
- Cargo.lock +1 -0
- Cargo.toml +6 -1
- crates/headroom-core/src/transforms/smart_crusher/anchors.rs +190 -28
- crates/headroom-core/src/transforms/smart_crusher/classifier.rs +15 -4
- crates/headroom-core/src/transforms/smart_crusher/hashing.rs +34 -20
- crates/headroom-core/src/transforms/smart_crusher/statistics.rs +158 -3
- crates/headroom-core/src/transforms/smart_crusher/types.rs +23 -4
- plugins/headroom-agent-hooks/.claude-plugin/plugin.json +1 -1
- plugins/headroom-agent-hooks/.github/plugin/plugin.json +1 -1
|
@@ -5,14 +5,14 @@
|
|
| 5 |
},
|
| 6 |
"metadata": {
|
| 7 |
"description": "Headroom marketplace for Claude Code and GitHub Copilot CLI plugins.",
|
| 8 |
-
"version": "0.
|
| 9 |
},
|
| 10 |
"plugins": [
|
| 11 |
{
|
| 12 |
"name": "headroom",
|
| 13 |
"source": "./plugins/headroom-agent-hooks",
|
| 14 |
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
|
| 15 |
-
"version": "0.
|
| 16 |
"author": {
|
| 17 |
"name": "Headroom Contributors",
|
| 18 |
"url": "https://github.com/chopratejas/headroom"
|
|
|
|
| 5 |
},
|
| 6 |
"metadata": {
|
| 7 |
"description": "Headroom marketplace for Claude Code and GitHub Copilot CLI plugins.",
|
| 8 |
+
"version": "0.11.0"
|
| 9 |
},
|
| 10 |
"plugins": [
|
| 11 |
{
|
| 12 |
"name": "headroom",
|
| 13 |
"source": "./plugins/headroom-agent-hooks",
|
| 14 |
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
|
| 15 |
+
"version": "0.11.0",
|
| 16 |
"author": {
|
| 17 |
"name": "Headroom Contributors",
|
| 18 |
"url": "https://github.com/chopratejas/headroom"
|
|
@@ -5,14 +5,14 @@
|
|
| 5 |
},
|
| 6 |
"metadata": {
|
| 7 |
"description": "Headroom marketplace for Claude Code and GitHub Copilot CLI plugins.",
|
| 8 |
-
"version": "0.
|
| 9 |
},
|
| 10 |
"plugins": [
|
| 11 |
{
|
| 12 |
"name": "headroom",
|
| 13 |
"source": "./plugins/headroom-agent-hooks",
|
| 14 |
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
|
| 15 |
-
"version": "0.
|
| 16 |
"author": {
|
| 17 |
"name": "Headroom Contributors",
|
| 18 |
"url": "https://github.com/chopratejas/headroom"
|
|
|
|
| 5 |
},
|
| 6 |
"metadata": {
|
| 7 |
"description": "Headroom marketplace for Claude Code and GitHub Copilot CLI plugins.",
|
| 8 |
+
"version": "0.11.0"
|
| 9 |
},
|
| 10 |
"plugins": [
|
| 11 |
{
|
| 12 |
"name": "headroom",
|
| 13 |
"source": "./plugins/headroom-agent-hooks",
|
| 14 |
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
|
| 15 |
+
"version": "0.11.0",
|
| 16 |
"author": {
|
| 17 |
"name": "Headroom Contributors",
|
| 18 |
"url": "https://github.com/chopratejas/headroom"
|
|
@@ -2198,6 +2198,7 @@ version = "1.0.149"
|
|
| 2198 |
source = "registry+https://github.com/rust-lang/crates.io-index"
|
| 2199 |
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
|
| 2200 |
dependencies = [
|
|
|
|
| 2201 |
"itoa",
|
| 2202 |
"memchr",
|
| 2203 |
"serde",
|
|
|
|
| 2198 |
source = "registry+https://github.com/rust-lang/crates.io-index"
|
| 2199 |
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
|
| 2200 |
dependencies = [
|
| 2201 |
+
"indexmap",
|
| 2202 |
"itoa",
|
| 2203 |
"memchr",
|
| 2204 |
"serde",
|
|
@@ -26,7 +26,12 @@ authors = ["Headroom Maintainers"]
|
|
| 26 |
|
| 27 |
[workspace.dependencies]
|
| 28 |
serde = { version = "1", features = ["derive"] }
|
| 29 |
-
serde_json
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
bytes = "1"
|
| 31 |
thiserror = "1"
|
| 32 |
tracing = "0.1"
|
|
|
|
| 26 |
|
| 27 |
[workspace.dependencies]
|
| 28 |
serde = { version = "1", features = ["derive"] }
|
| 29 |
+
# `preserve_order` makes `serde_json::Value::Object` use IndexMap so JSON
|
| 30 |
+
# parse order is preserved through Value→string→Value round-trips. The
|
| 31 |
+
# smart_crusher port relies on this to match Python's `str(dict)` output,
|
| 32 |
+
# which preserves insertion order; otherwise BTreeMap's sorted-key default
|
| 33 |
+
# would diverge from Python on every multi-key object.
|
| 34 |
+
serde_json = { version = "1", features = ["preserve_order"] }
|
| 35 |
bytes = "1"
|
| 36 |
thiserror = "1"
|
| 37 |
tracing = "0.1"
|
|
@@ -111,43 +111,117 @@ pub fn extract_query_anchors(text: &str) -> HashSet<String> {
|
|
| 111 |
anchors
|
| 112 |
}
|
| 113 |
|
| 114 |
-
///
|
|
|
|
| 115 |
///
|
| 116 |
-
///
|
| 117 |
-
///
|
| 118 |
-
///
|
| 119 |
-
/// lowercasing — this isn't byte-identical to Python's `str(dict)`
|
| 120 |
-
/// (Python uses single quotes, JSON uses double; Python's bool is
|
| 121 |
-
/// `True`/`False`, JSON's is `true`/`false`), so for cross-language
|
| 122 |
-
/// parity we need a string form that matches Python's. We document this
|
| 123 |
-
/// gap and fix it in the analyzer integration.
|
| 124 |
///
|
| 125 |
-
///
|
| 126 |
-
///
|
| 127 |
-
///
|
| 128 |
-
///
|
|
|
|
| 129 |
///
|
| 130 |
-
///
|
| 131 |
-
///
|
| 132 |
-
///
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
///
|
| 134 |
-
///
|
| 135 |
-
///
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
pub fn item_matches_anchors(item: &Value, anchors: &HashSet<String>) -> bool {
|
| 140 |
if anchors.is_empty() {
|
| 141 |
return false;
|
| 142 |
}
|
| 143 |
|
| 144 |
-
// Python: `str(item).lower()`.
|
| 145 |
-
//
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
};
|
| 150 |
-
|
| 151 |
anchors.iter().any(|a| item_str.contains(a))
|
| 152 |
}
|
| 153 |
|
|
@@ -248,4 +322,92 @@ mod tests {
|
|
| 248 |
let anchors: HashSet<String> = ["xyz123".to_string()].into_iter().collect();
|
| 249 |
assert!(!item_matches_anchors(&json!({"a": "b"}), &anchors));
|
| 250 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 251 |
}
|
|
|
|
| 111 |
anchors
|
| 112 |
}
|
| 113 |
|
| 114 |
+
/// Serialize a `serde_json::Value` to a string matching Python's
|
| 115 |
+
/// `str()` of the equivalent native value.
|
| 116 |
///
|
| 117 |
+
/// Used by `item_matches_anchors` because Python compares anchors via
|
| 118 |
+
/// `anchor in str(item).lower()` and `str(dict)` differs from
|
| 119 |
+
/// `json.dumps(dict)` in three ways that affect substring matching:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
///
|
| 121 |
+
/// | Aspect | Python `str(dict)` | `serde_json::to_string` |
|
| 122 |
+
/// |------------------|------------------------------|-------------------------|
|
| 123 |
+
/// | String quotes | single `'` | double `"` |
|
| 124 |
+
/// | Booleans / null | `True`, `False`, `None` | `true`, `false`, `null` |
|
| 125 |
+
/// | Spacing | `key: value`, `a, b` | `key:value`, `a,b` |
|
| 126 |
///
|
| 127 |
+
/// All three matter for anchor matching:
|
| 128 |
+
/// - An anchor `"name': 'a"` extracted from a user phrase like
|
| 129 |
+
/// `find {'name': 'alice'}` would match Python's serialization but
|
| 130 |
+
/// never the JSON form.
|
| 131 |
+
/// - An anchor `"true"` (lowercased from `"True"`) matches both, but
|
| 132 |
+
/// the unlowercased version `"True"` is in Python output and not
|
| 133 |
+
/// JSON. Lowercasing both sides handles this.
|
| 134 |
+
/// - An anchor `"name: alice"` (with the space) would match Python
|
| 135 |
+
/// but never JSON.
|
| 136 |
///
|
| 137 |
+
/// Output is then lowercased upstream (matching Python's `.lower()`)
|
| 138 |
+
/// so True/False/None case is normalized away after that step.
|
| 139 |
+
fn python_repr(value: &Value) -> String {
|
| 140 |
+
let mut out = String::new();
|
| 141 |
+
write_python_repr(&mut out, value);
|
| 142 |
+
out
|
| 143 |
+
}
|
| 144 |
+
|
| 145 |
+
fn write_python_repr(out: &mut String, value: &Value) {
|
| 146 |
+
match value {
|
| 147 |
+
Value::Null => out.push_str("None"),
|
| 148 |
+
Value::Bool(true) => out.push_str("True"),
|
| 149 |
+
Value::Bool(false) => out.push_str("False"),
|
| 150 |
+
Value::Number(n) => {
|
| 151 |
+
// Python `str(int)` and `str(float)` produce minimal forms.
|
| 152 |
+
// `serde_json::Number`'s `Display` matches Python for ints
|
| 153 |
+
// (`5`) but for floats it can write `1.0` while Python may
|
| 154 |
+
// write `1.0` too — close enough for substring matching
|
| 155 |
+
// since anchor strings rarely contain numeric literals
|
| 156 |
+
// beyond the digit prefix.
|
| 157 |
+
out.push_str(&n.to_string());
|
| 158 |
+
}
|
| 159 |
+
Value::String(s) => {
|
| 160 |
+
// Python `repr(s)` chooses single or double quotes
|
| 161 |
+
// depending on content. Default preference is single
|
| 162 |
+
// quotes; switches to double if the string contains a
|
| 163 |
+
// single quote and no double. We emit single quotes
|
| 164 |
+
// always — this matches the dominant case (no quotes in
|
| 165 |
+
// the string) and is what Python does for `str(dict)` of
|
| 166 |
+
// most realistic data. The rare case where Python would
|
| 167 |
+
// switch to double quotes is documented as a known parity
|
| 168 |
+
// gap in `python_repr_string_with_single_quote_drift`.
|
| 169 |
+
out.push('\'');
|
| 170 |
+
out.push_str(s);
|
| 171 |
+
out.push('\'');
|
| 172 |
+
}
|
| 173 |
+
Value::Array(items) => {
|
| 174 |
+
out.push('[');
|
| 175 |
+
for (i, item) in items.iter().enumerate() {
|
| 176 |
+
if i > 0 {
|
| 177 |
+
out.push_str(", ");
|
| 178 |
+
}
|
| 179 |
+
write_python_repr(out, item);
|
| 180 |
+
}
|
| 181 |
+
out.push(']');
|
| 182 |
+
}
|
| 183 |
+
Value::Object(map) => {
|
| 184 |
+
out.push('{');
|
| 185 |
+
// Python preserves insertion order in `dict.__str__` (since
|
| 186 |
+
// Python 3.7). We require the workspace `serde_json` to be
|
| 187 |
+
// built with `preserve_order` so `serde_json::Map` uses
|
| 188 |
+
// `IndexMap` instead of the default `BTreeMap` — see the
|
| 189 |
+
// comment on `serde_json` in the workspace `Cargo.toml`.
|
| 190 |
+
// Without that feature, this iteration is sorted-by-key
|
| 191 |
+
// and silently diverges from Python on every multi-key
|
| 192 |
+
// object.
|
| 193 |
+
for (i, (k, v)) in map.iter().enumerate() {
|
| 194 |
+
if i > 0 {
|
| 195 |
+
out.push_str(", ");
|
| 196 |
+
}
|
| 197 |
+
out.push('\'');
|
| 198 |
+
out.push_str(k);
|
| 199 |
+
out.push('\'');
|
| 200 |
+
out.push_str(": ");
|
| 201 |
+
write_python_repr(out, v);
|
| 202 |
+
}
|
| 203 |
+
out.push('}');
|
| 204 |
+
}
|
| 205 |
+
}
|
| 206 |
+
}
|
| 207 |
+
|
| 208 |
+
/// Check if a JSON value matches any query anchors.
|
| 209 |
+
///
|
| 210 |
+
/// Direct port of `item_matches_anchors` (Python `smart_crusher.py:152-168`).
|
| 211 |
+
/// Python uses `str(item).lower()` which produces Python's repr-like
|
| 212 |
+
/// representation. We mirror that via `python_repr` rather than
|
| 213 |
+
/// `serde_json::to_string` so substring matching has the same surface
|
| 214 |
+
/// as Python (single quotes, `True`/`False`/`None`, spaced commas/colons).
|
| 215 |
pub fn item_matches_anchors(item: &Value, anchors: &HashSet<String>) -> bool {
|
| 216 |
if anchors.is_empty() {
|
| 217 |
return false;
|
| 218 |
}
|
| 219 |
|
| 220 |
+
// Python: `str(item).lower()`. `python_repr` produces the same
|
| 221 |
+
// single-quoted, space-after-colon, `True`/`False`/`None` form
|
| 222 |
+
// that Python's `str()` does; lowercase normalizes the bool/null
|
| 223 |
+
// case to match Python's downstream `.lower()` call.
|
| 224 |
+
let item_str = python_repr(item).to_lowercase();
|
|
|
|
|
|
|
| 225 |
anchors.iter().any(|a| item_str.contains(a))
|
| 226 |
}
|
| 227 |
|
|
|
|
| 322 |
let anchors: HashSet<String> = ["xyz123".to_string()].into_iter().collect();
|
| 323 |
assert!(!item_matches_anchors(&json!({"a": "b"}), &anchors));
|
| 324 |
}
|
| 325 |
+
|
| 326 |
+
#[test]
|
| 327 |
+
fn hostname_blocklist_drops_e_g() {
|
| 328 |
+
// S5 in code review: pin that "e.g" in input doesn't surface as
|
| 329 |
+
// an anchor. Direct match against the regex confirms "e.g" itself
|
| 330 |
+
// matches before the blocklist filters it.
|
| 331 |
+
let anchors = extract_query_anchors("see e.g for example");
|
| 332 |
+
assert!(!anchors.contains("e.g"));
|
| 333 |
+
// Sanity: a normal hostname still passes through.
|
| 334 |
+
let anchors = extract_query_anchors("connect to api.example.com");
|
| 335 |
+
assert!(anchors.contains("api.example.com"));
|
| 336 |
+
}
|
| 337 |
+
|
| 338 |
+
#[test]
|
| 339 |
+
fn email_typo_pattern_still_matches_real_emails() {
|
| 340 |
+
// S4 in code review: the Python `[A-Z|a-z]` typo doesn't break
|
| 341 |
+
// real email matching — pin that explicitly.
|
| 342 |
+
let anchors = extract_query_anchors("contact alice@example.com today");
|
| 343 |
+
assert!(anchors.contains("alice@example.com"));
|
| 344 |
+
let anchors = extract_query_anchors("ping bob@SUB.EXAMPLE.IO");
|
| 345 |
+
assert!(anchors.contains("bob@sub.example.io"));
|
| 346 |
+
}
|
| 347 |
+
|
| 348 |
+
// ---------- python_repr (used by item_matches_anchors) ----------
|
| 349 |
+
|
| 350 |
+
#[test]
|
| 351 |
+
fn python_repr_matches_python_str_for_dict() {
|
| 352 |
+
// Python: `str({'name': 'Alice', 'ok': True, 'count': 5, 'val': None})`
|
| 353 |
+
// = `"{'name': 'Alice', 'ok': True, 'count': 5, 'val': None}"`
|
| 354 |
+
// (insertion order — Python's dict preserves it since 3.7).
|
| 355 |
+
//
|
| 356 |
+
// Workspace `Cargo.toml` enables serde_json's `preserve_order`
|
| 357 |
+
// feature, so `json!` macro and `serde_json::from_str` both
|
| 358 |
+
// preserve key insertion order. Without that feature the test
|
| 359 |
+
// below would fail.
|
| 360 |
+
let v = json!({"name": "Alice", "ok": true, "count": 5, "val": null});
|
| 361 |
+
let r = python_repr(&v);
|
| 362 |
+
assert_eq!(
|
| 363 |
+
r,
|
| 364 |
+
"{'name': 'Alice', 'ok': True, 'count': 5, 'val': None}"
|
| 365 |
+
);
|
| 366 |
+
}
|
| 367 |
+
|
| 368 |
+
#[test]
|
| 369 |
+
fn python_repr_list_uses_space_after_comma() {
|
| 370 |
+
// Python: `str([1, 2, 'abc', True])` = `"[1, 2, 'abc', True]"`.
|
| 371 |
+
let v = json!([1, 2, "abc", true]);
|
| 372 |
+
assert_eq!(python_repr(&v), "[1, 2, 'abc', True]");
|
| 373 |
+
}
|
| 374 |
+
|
| 375 |
+
#[test]
|
| 376 |
+
fn python_repr_nested() {
|
| 377 |
+
let v = json!({"a": [1, {"b": "c"}]});
|
| 378 |
+
assert_eq!(python_repr(&v), "{'a': [1, {'b': 'c'}]}");
|
| 379 |
+
}
|
| 380 |
+
|
| 381 |
+
#[test]
|
| 382 |
+
fn item_matches_anchor_with_python_none_form() {
|
| 383 |
+
// I3 fix in review: Python `str({'val': None}).lower()` produces
|
| 384 |
+
// `{'val': none}`. With the old JSON-based matcher, the same
|
| 385 |
+
// input would serialize as `{"val":null}` and an anchor "none"
|
| 386 |
+
// would never match. With `python_repr` the serialization is
|
| 387 |
+
// `{'val': None}` → lowercased to `{'val': none}` → contains "none".
|
| 388 |
+
let anchors: HashSet<String> = ["none".to_string()].into_iter().collect();
|
| 389 |
+
assert!(item_matches_anchors(&json!({"val": null}), &anchors));
|
| 390 |
+
}
|
| 391 |
+
|
| 392 |
+
#[test]
|
| 393 |
+
fn item_matches_anchor_avoids_json_null_token() {
|
| 394 |
+
// Inverse of the above: an anchor "null" must NOT match a Python-
|
| 395 |
+
// null repr (which writes `none`). Pre-fix code would erroneously
|
| 396 |
+
// match because of `serde_json::to_string`'s `null` literal.
|
| 397 |
+
let anchors: HashSet<String> = ["null".to_string()].into_iter().collect();
|
| 398 |
+
assert!(!item_matches_anchors(&json!({"val": null}), &anchors));
|
| 399 |
+
}
|
| 400 |
+
|
| 401 |
+
#[test]
|
| 402 |
+
fn python_repr_string_with_single_quote_drift() {
|
| 403 |
+
// Documented parity gap: Python's `repr` switches to double
|
| 404 |
+
// quotes if the string contains a single quote. We always use
|
| 405 |
+
// single quotes. Pin the gap so future changes are intentional.
|
| 406 |
+
let v = json!({"k": "it's fine"});
|
| 407 |
+
// Our output: `{'k': 'it's fine'}` (broken Python repr — Python
|
| 408 |
+
// would emit `{'k': "it's fine"}`).
|
| 409 |
+
assert_eq!(python_repr(&v), "{'k': 'it's fine'}");
|
| 410 |
+
// Substring matching for typical anchors still works because
|
| 411 |
+
// they don't reference the quote chars themselves.
|
| 412 |
+
}
|
| 413 |
}
|
|
@@ -173,10 +173,21 @@ mod tests {
|
|
| 173 |
|
| 174 |
#[test]
|
| 175 |
fn bool_with_number_is_mixed_not_bool_or_number() {
|
| 176 |
-
// Python's `[True, False, 1]`
|
| 177 |
-
//
|
| 178 |
-
//
|
| 179 |
-
//
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 180 |
let items = vec![json!(true), json!(false), json!(1)];
|
| 181 |
assert_eq!(classify_array(&items), ArrayType::MixedArray);
|
| 182 |
}
|
|
|
|
| 173 |
|
| 174 |
#[test]
|
| 175 |
fn bool_with_number_is_mixed_not_bool_or_number() {
|
| 176 |
+
// Python's `[True, False, 1]` walks like this:
|
| 177 |
+
// types == {bool, int} (because bool is an int subclass)
|
| 178 |
+
// has_bool = True
|
| 179 |
+
// `types <= {bool, int}` is True, so the bool-array gate is
|
| 180 |
+
// considered, but the inner `all(isinstance(i, bool))` check
|
| 181 |
+
// is False (because of the `1`), so does NOT return BOOL_ARRAY.
|
| 182 |
+
// `types == {dict}` False. `types == {str}` False.
|
| 183 |
+
// `types <= {int, float} and not has_bool` — has_bool is True,
|
| 184 |
+
// so the number-array gate fails too. `types == {list}` False.
|
| 185 |
+
// Falls through to MIXED_ARRAY.
|
| 186 |
+
//
|
| 187 |
+
// Rust matches by side effect of separate `Bool`/`Number` enum
|
| 188 |
+
// variants: the bool-array gate fails because `has_number` is
|
| 189 |
+
// True; the number-array gate fails because `has_bool` is True.
|
| 190 |
+
// Final: MIXED_ARRAY. Same outcome via different code path.
|
| 191 |
let items = vec![json!(true), json!(false), json!(1)];
|
| 192 |
assert_eq!(classify_array(&items), ArrayType::MixedArray);
|
| 193 |
}
|
|
@@ -1,23 +1,35 @@
|
|
| 1 |
//! Field-name hashing for cache keys.
|
| 2 |
//!
|
| 3 |
-
//! Direct port of `_hash_field_name` (Python `smart_crusher.py:171-
|
| 4 |
-
//! Used to
|
| 5 |
-
//!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
|
| 7 |
use sha2::{Digest, Sha256};
|
| 8 |
|
| 9 |
-
/// SHA-256 of the UTF-8 bytes, hex-encoded, truncated to
|
| 10 |
///
|
| 11 |
-
/// Python equivalent: `hashlib.sha256(field_name.encode()).hexdigest()[:
|
| 12 |
-
///
|
| 13 |
-
///
|
| 14 |
pub fn hash_field_name(field_name: &str) -> String {
|
| 15 |
let mut hasher = Sha256::new();
|
| 16 |
hasher.update(field_name.as_bytes());
|
| 17 |
let digest = hasher.finalize();
|
| 18 |
-
//
|
|
|
|
| 19 |
let hex = format!("{:x}", digest);
|
| 20 |
-
hex[..
|
| 21 |
}
|
| 22 |
|
| 23 |
#[cfg(test)]
|
|
@@ -25,22 +37,22 @@ mod tests {
|
|
| 25 |
use super::*;
|
| 26 |
|
| 27 |
#[test]
|
| 28 |
-
fn
|
| 29 |
-
// Verified against Python: hashlib.sha256(b"customer_id").hexdigest()[:
|
| 30 |
-
assert_eq!(hash_field_name("customer_id"), "
|
| 31 |
}
|
| 32 |
|
| 33 |
#[test]
|
| 34 |
fn empty_string() {
|
| 35 |
-
// Verified against Python: hashlib.sha256(b"").hexdigest()[:
|
| 36 |
-
assert_eq!(hash_field_name(""), "
|
| 37 |
}
|
| 38 |
|
| 39 |
#[test]
|
| 40 |
fn unicode_field_name() {
|
| 41 |
-
// Verified against Python: hashlib.sha256("café".encode()).hexdigest()[:
|
| 42 |
// UTF-8 bytes for "café" are 63 61 66 c3 a9 — must encode same way.
|
| 43 |
-
assert_eq!(hash_field_name("café"), "
|
| 44 |
}
|
| 45 |
|
| 46 |
#[test]
|
|
@@ -50,9 +62,11 @@ mod tests {
|
|
| 50 |
}
|
| 51 |
|
| 52 |
#[test]
|
| 53 |
-
fn
|
| 54 |
-
// Always exactly
|
| 55 |
-
|
| 56 |
-
|
|
|
|
|
|
|
| 57 |
}
|
| 58 |
}
|
|
|
|
| 1 |
//! Field-name hashing for cache keys.
|
| 2 |
//!
|
| 3 |
+
//! Direct port of `_hash_field_name` (Python `smart_crusher.py:171-177`).
|
| 4 |
+
//! Used to look up TOIN-anonymized `preserve_fields` — TOIN stores
|
| 5 |
+
//! field names as **SHA-256[:8]** for privacy (per Python doc-comment
|
| 6 |
+
//! at `smart_crusher.py:174-175`), so cache lookups will silently miss
|
| 7 |
+
//! if the truncation length drifts.
|
| 8 |
+
//!
|
| 9 |
+
//! # 16 vs 8 — got it wrong once, now pinned
|
| 10 |
+
//!
|
| 11 |
+
//! The first version of this file used `[:16]` based on a misread of
|
| 12 |
+
//! the Python source. Code review caught the discrepancy: Python uses
|
| 13 |
+
//! `[:8]`. Cache lookups against TOIN's 8-char hashes would have
|
| 14 |
+
//! silently missed every field, defeating the entire `use_feedback_hints`
|
| 15 |
+
//! path. Fixed here; the tests now pin against Python `[:8]` reference
|
| 16 |
+
//! values verified via `python3 -c "...hexdigest()[:8]"`.
|
| 17 |
|
| 18 |
use sha2::{Digest, Sha256};
|
| 19 |
|
| 20 |
+
/// SHA-256 of the UTF-8 bytes, hex-encoded, truncated to **8** chars.
|
| 21 |
///
|
| 22 |
+
/// Python equivalent: `hashlib.sha256(field_name.encode()).hexdigest()[:8]`.
|
| 23 |
+
/// Lowercase hex — both Python `hexdigest()` and Rust's `sha2` default
|
| 24 |
+
/// to lowercase, so this is consistent without manual case-coercion.
|
| 25 |
pub fn hash_field_name(field_name: &str) -> String {
|
| 26 |
let mut hasher = Sha256::new();
|
| 27 |
hasher.update(field_name.as_bytes());
|
| 28 |
let digest = hasher.finalize();
|
| 29 |
+
// Truncate to first 8 hex chars (4 bytes of digest). MUST match
|
| 30 |
+
// Python's `[:8]` — see module-level note above.
|
| 31 |
let hex = format!("{:x}", digest);
|
| 32 |
+
hex[..8].to_string()
|
| 33 |
}
|
| 34 |
|
| 35 |
#[cfg(test)]
|
|
|
|
| 37 |
use super::*;
|
| 38 |
|
| 39 |
#[test]
|
| 40 |
+
fn matches_python_sha256_truncated_to_8() {
|
| 41 |
+
// Verified against Python: hashlib.sha256(b"customer_id").hexdigest()[:8]
|
| 42 |
+
assert_eq!(hash_field_name("customer_id"), "1e38d67d");
|
| 43 |
}
|
| 44 |
|
| 45 |
#[test]
|
| 46 |
fn empty_string() {
|
| 47 |
+
// Verified against Python: hashlib.sha256(b"").hexdigest()[:8]
|
| 48 |
+
assert_eq!(hash_field_name(""), "e3b0c442");
|
| 49 |
}
|
| 50 |
|
| 51 |
#[test]
|
| 52 |
fn unicode_field_name() {
|
| 53 |
+
// Verified against Python: hashlib.sha256("café".encode()).hexdigest()[:8]
|
| 54 |
// UTF-8 bytes for "café" are 63 61 66 c3 a9 — must encode same way.
|
| 55 |
+
assert_eq!(hash_field_name("café"), "850f7dc4");
|
| 56 |
}
|
| 57 |
|
| 58 |
#[test]
|
|
|
|
| 62 |
}
|
| 63 |
|
| 64 |
#[test]
|
| 65 |
+
fn output_length_is_8() {
|
| 66 |
+
// Always exactly 8 hex chars regardless of input length.
|
| 67 |
+
// This must match Python's `[:8]`; if you change it, every TOIN
|
| 68 |
+
// preserve-field lookup silently misses.
|
| 69 |
+
assert_eq!(hash_field_name("a").len(), 8);
|
| 70 |
+
assert_eq!(hash_field_name(&"x".repeat(1000)).len(), 8);
|
| 71 |
}
|
| 72 |
}
|
|
@@ -82,6 +82,53 @@ pub fn calculate_string_entropy(s: &str) -> f64 {
|
|
| 82 |
}
|
| 83 |
}
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
/// Detect if numeric values form a sequential pattern (like IDs:
|
| 86 |
/// 1, 2, 3, ...).
|
| 87 |
///
|
|
@@ -136,9 +183,10 @@ pub fn detect_sequential_pattern(values: &[Value], check_order: bool) -> bool {
|
|
| 136 |
}
|
| 137 |
Value::String(s) => {
|
| 138 |
// Python: `try: nums.append(int(v))`. `int("3.14")` raises
|
| 139 |
-
//
|
| 140 |
-
//
|
| 141 |
-
|
|
|
|
| 142 |
nums.push(parsed as f64);
|
| 143 |
// BUG #2 fix: do NOT set had_non_string_numeric.
|
| 144 |
// If we later find this is the ONLY source of numeric
|
|
@@ -372,4 +420,111 @@ mod tests {
|
|
| 372 |
let v: Vec<Value> = (1..=10).map(|i| json!(i as f64)).collect();
|
| 373 |
assert!(detect_sequential_pattern(&v, true));
|
| 374 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 375 |
}
|
|
|
|
| 82 |
}
|
| 83 |
}
|
| 84 |
|
| 85 |
+
/// Parse a string the way Python's built-in `int()` does for plain
|
| 86 |
+
/// integer literals. Used by `detect_sequential_pattern` to mirror
|
| 87 |
+
/// `int(v)` behavior exactly.
|
| 88 |
+
///
|
| 89 |
+
/// Python's `int()` accepts:
|
| 90 |
+
/// - leading/trailing ASCII whitespace (stripped)
|
| 91 |
+
/// - leading sign (`+` or `-`)
|
| 92 |
+
/// - PEP 515 underscore digit separators (e.g. `"3_000"` → `3000`)
|
| 93 |
+
///
|
| 94 |
+
/// Rust's `str::parse::<i64>()` rejects all of those. If we used the
|
| 95 |
+
/// raw `parse`, real-world payloads with `" 5 "` or `"+5"` would
|
| 96 |
+
/// silently disagree with Python on whether the field is "numeric",
|
| 97 |
+
/// which changes sequential classification and breaks fixtures.
|
| 98 |
+
///
|
| 99 |
+
/// We deliberately do NOT support Python's other `int()` features
|
| 100 |
+
/// (base prefixes like `"0x10"`, scientific notation via `int(float(s))`,
|
| 101 |
+
/// etc.) because the Python `_detect_sequential_pattern` call site
|
| 102 |
+
/// uses the default-base `int()` overload — those paths are
|
| 103 |
+
/// unreachable.
|
| 104 |
+
fn python_int_parse(s: &str) -> Option<i64> {
|
| 105 |
+
// Python: `int()` strips ASCII whitespace from both ends.
|
| 106 |
+
let trimmed = s.trim();
|
| 107 |
+
if trimmed.is_empty() {
|
| 108 |
+
return None;
|
| 109 |
+
}
|
| 110 |
+
// Python: drop PEP 515 underscores between digits. The implementation
|
| 111 |
+
// is more careful than this (rejects leading/trailing/double underscores),
|
| 112 |
+
// but for our use-case any string with valid digits + underscore separators
|
| 113 |
+
// is what we want to accept. Edge cases like `"_5_"` will fail the
|
| 114 |
+
// i64::parse call below, matching Python's behavior of rejecting them.
|
| 115 |
+
let cleaned: String = if trimmed.contains('_') {
|
| 116 |
+
// Reject patterns Python rejects: leading/trailing underscore,
|
| 117 |
+
// double underscores. Otherwise strip them out.
|
| 118 |
+
let bytes = trimmed.as_bytes();
|
| 119 |
+
let starts_or_ends = bytes[0] == b'_'
|
| 120 |
+
|| *bytes.last().unwrap() == b'_'
|
| 121 |
+
|| trimmed.contains("__");
|
| 122 |
+
if starts_or_ends {
|
| 123 |
+
return None;
|
| 124 |
+
}
|
| 125 |
+
trimmed.replace('_', "")
|
| 126 |
+
} else {
|
| 127 |
+
trimmed.to_string()
|
| 128 |
+
};
|
| 129 |
+
cleaned.parse::<i64>().ok()
|
| 130 |
+
}
|
| 131 |
+
|
| 132 |
/// Detect if numeric values form a sequential pattern (like IDs:
|
| 133 |
/// 1, 2, 3, ...).
|
| 134 |
///
|
|
|
|
| 183 |
}
|
| 184 |
Value::String(s) => {
|
| 185 |
// Python: `try: nums.append(int(v))`. `int("3.14")` raises
|
| 186 |
+
// and Rust's plain `parse::<i64>` differs from `int()` on
|
| 187 |
+
// edges like leading whitespace and PEP 515 underscores.
|
| 188 |
+
// `python_int_parse` mirrors Python exactly — see fn doc.
|
| 189 |
+
if let Some(parsed) = python_int_parse(s) {
|
| 190 |
nums.push(parsed as f64);
|
| 191 |
// BUG #2 fix: do NOT set had_non_string_numeric.
|
| 192 |
// If we later find this is the ONLY source of numeric
|
|
|
|
| 420 |
let v: Vec<Value> = (1..=10).map(|i| json!(i as f64)).collect();
|
| 421 |
assert!(detect_sequential_pattern(&v, true));
|
| 422 |
}
|
| 423 |
+
|
| 424 |
+
#[test]
|
| 425 |
+
fn sequential_fractional_unit_step() {
|
| 426 |
+
// Floats with non-integer values but constant unit step. avg_diff
|
| 427 |
+
// = 1.0, all diffs in [0.5, 2.0], should be sequential. (Suggestion
|
| 428 |
+
// S6 in code review — pins float arithmetic doesn't drift.)
|
| 429 |
+
let v: Vec<Value> = vec![
|
| 430 |
+
json!(1.5),
|
| 431 |
+
json!(2.5),
|
| 432 |
+
json!(3.5),
|
| 433 |
+
json!(4.5),
|
| 434 |
+
json!(5.5),
|
| 435 |
+
];
|
| 436 |
+
assert!(detect_sequential_pattern(&v, true));
|
| 437 |
+
}
|
| 438 |
+
|
| 439 |
+
#[test]
|
| 440 |
+
fn bug2_all_unparseable_strings_returns_false() {
|
| 441 |
+
// S3 in code review: explicit test for the all-strings case where
|
| 442 |
+
// none parse. Falls out of `nums.len() < 5` already, but pinning
|
| 443 |
+
// the behavior protects against future refactors.
|
| 444 |
+
let v: Vec<Value> = vec![
|
| 445 |
+
json!("abc"),
|
| 446 |
+
json!("def"),
|
| 447 |
+
json!("ghi"),
|
| 448 |
+
json!("jkl"),
|
| 449 |
+
json!("mno"),
|
| 450 |
+
];
|
| 451 |
+
assert!(!detect_sequential_pattern(&v, true));
|
| 452 |
+
}
|
| 453 |
+
|
| 454 |
+
#[test]
|
| 455 |
+
fn bug2_single_int_among_strings_still_detects() {
|
| 456 |
+
// S3 in code review: validates that the BUG #2 gate fires on
|
| 457 |
+
// "ANY non-string numeric", not "majority". One real int among
|
| 458 |
+
// string-encoded numerics should be enough to count as sequential.
|
| 459 |
+
let v: Vec<Value> = vec![
|
| 460 |
+
json!("001"),
|
| 461 |
+
json!("002"),
|
| 462 |
+
json!(3), // <-- the unambiguous numeric
|
| 463 |
+
json!("004"),
|
| 464 |
+
json!("005"),
|
| 465 |
+
json!("006"),
|
| 466 |
+
];
|
| 467 |
+
assert!(detect_sequential_pattern(&v, true));
|
| 468 |
+
}
|
| 469 |
+
|
| 470 |
+
// ---------- python_int_parse ----------
|
| 471 |
+
|
| 472 |
+
#[test]
|
| 473 |
+
fn python_int_parse_basic() {
|
| 474 |
+
assert_eq!(python_int_parse("5"), Some(5));
|
| 475 |
+
assert_eq!(python_int_parse("-5"), Some(-5));
|
| 476 |
+
assert_eq!(python_int_parse("+5"), Some(5));
|
| 477 |
+
}
|
| 478 |
+
|
| 479 |
+
#[test]
|
| 480 |
+
fn python_int_parse_strips_whitespace() {
|
| 481 |
+
// Python: `int(" 5 ") == 5`. Rust's plain parse fails on this.
|
| 482 |
+
assert_eq!(python_int_parse(" 5 "), Some(5));
|
| 483 |
+
assert_eq!(python_int_parse("\t-3\n"), Some(-3));
|
| 484 |
+
}
|
| 485 |
+
|
| 486 |
+
#[test]
|
| 487 |
+
fn python_int_parse_underscores() {
|
| 488 |
+
// PEP 515 — Python: `int("3_000") == 3000`.
|
| 489 |
+
assert_eq!(python_int_parse("3_000"), Some(3000));
|
| 490 |
+
assert_eq!(python_int_parse("1_000_000"), Some(1_000_000));
|
| 491 |
+
}
|
| 492 |
+
|
| 493 |
+
#[test]
|
| 494 |
+
fn python_int_parse_underscore_edge_cases_rejected() {
|
| 495 |
+
// Python rejects these (raises ValueError); we mirror by
|
| 496 |
+
// returning None.
|
| 497 |
+
assert_eq!(python_int_parse("_5"), None);
|
| 498 |
+
assert_eq!(python_int_parse("5_"), None);
|
| 499 |
+
assert_eq!(python_int_parse("3__000"), None);
|
| 500 |
+
}
|
| 501 |
+
|
| 502 |
+
#[test]
|
| 503 |
+
fn python_int_parse_rejects_floats() {
|
| 504 |
+
// Python: `int("3.14")` raises. Mirror by returning None.
|
| 505 |
+
assert_eq!(python_int_parse("3.14"), None);
|
| 506 |
+
}
|
| 507 |
+
|
| 508 |
+
#[test]
|
| 509 |
+
fn python_int_parse_rejects_non_numeric() {
|
| 510 |
+
assert_eq!(python_int_parse("abc"), None);
|
| 511 |
+
assert_eq!(python_int_parse(""), None);
|
| 512 |
+
assert_eq!(python_int_parse(" "), None);
|
| 513 |
+
}
|
| 514 |
+
|
| 515 |
+
#[test]
|
| 516 |
+
fn sequential_with_whitespace_padded_strings_via_python_int_parse() {
|
| 517 |
+
// I1 fix in code review: real fixtures may carry whitespace-padded
|
| 518 |
+
// numeric strings. With the python_int_parse helper, mixed real-int
|
| 519 |
+
// + whitespace-padded-string fields still detect correctly.
|
| 520 |
+
let v: Vec<Value> = vec![
|
| 521 |
+
json!(1),
|
| 522 |
+
json!(" 2 "),
|
| 523 |
+
json!(3),
|
| 524 |
+
json!(" 4 "),
|
| 525 |
+
json!(5),
|
| 526 |
+
json!(6),
|
| 527 |
+
];
|
| 528 |
+
assert!(detect_sequential_pattern(&v, true));
|
| 529 |
+
}
|
| 530 |
}
|
|
@@ -123,10 +123,29 @@ impl CrushabilityAnalysis {
|
|
| 123 |
/// Complete analysis of an array.
|
| 124 |
///
|
| 125 |
/// Mirrors `ArrayAnalysis` at `smart_crusher.py:887-897`. `field_stats`
|
| 126 |
-
///
|
| 127 |
-
///
|
| 128 |
-
///
|
| 129 |
-
///
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 130 |
#[derive(Debug, Clone)]
|
| 131 |
pub struct ArrayAnalysis {
|
| 132 |
pub item_count: usize,
|
|
|
|
| 123 |
/// Complete analysis of an array.
|
| 124 |
///
|
| 125 |
/// Mirrors `ArrayAnalysis` at `smart_crusher.py:887-897`. `field_stats`
|
| 126 |
+
/// and `constant_fields` use `BTreeMap` for sorted-by-key iteration.
|
| 127 |
+
///
|
| 128 |
+
/// # Sort vs insertion order — known parity nuance
|
| 129 |
+
///
|
| 130 |
+
/// Python's `dict` preserves insertion order, and `_analyze_field` is
|
| 131 |
+
/// called once per key as it appears in `items[0].keys()` (i.e., JSON
|
| 132 |
+
/// parse order). With `serde_json/preserve_order` enabled at the
|
| 133 |
+
/// workspace level, `serde_json::Map` is an `IndexMap` and parse order
|
| 134 |
+
/// matches Python.
|
| 135 |
+
///
|
| 136 |
+
/// `BTreeMap` here gives sorted-key iteration — which differs from
|
| 137 |
+
/// Python's parse-order `dict`. This matters only if downstream code
|
| 138 |
+
/// observes the iteration order of `field_stats` (e.g., when emitting
|
| 139 |
+
/// debug output, picking a "first" field, or computing strategy
|
| 140 |
+
/// strings that include field names).
|
| 141 |
+
///
|
| 142 |
+
/// During the analyzer port (Stage 3c.1 commit 2), we'll either:
|
| 143 |
+
/// 1. Switch this to `IndexMap` if any code path observes order, OR
|
| 144 |
+
/// 2. Document that Python's order-sensitive paths get rewritten to
|
| 145 |
+
/// iterate sorted, then mirror that in Rust.
|
| 146 |
+
///
|
| 147 |
+
/// Tracked in the design doc at
|
| 148 |
+
/// `~/Desktop/SmartCrusher-Architecture-Improvements.md`.
|
| 149 |
#[derive(Debug, Clone)]
|
| 150 |
pub struct ArrayAnalysis {
|
| 151 |
pub item_count: usize,
|
|
@@ -1,6 +1,6 @@
|
|
| 1 |
{
|
| 2 |
"name": "headroom",
|
| 3 |
-
"version": "0.
|
| 4 |
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
|
| 5 |
"author": {
|
| 6 |
"name": "Headroom Contributors",
|
|
|
|
| 1 |
{
|
| 2 |
"name": "headroom",
|
| 3 |
+
"version": "0.11.0",
|
| 4 |
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
|
| 5 |
"author": {
|
| 6 |
"name": "Headroom Contributors",
|
|
@@ -1,6 +1,6 @@
|
|
| 1 |
{
|
| 2 |
"name": "headroom",
|
| 3 |
-
"version": "0.
|
| 4 |
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
|
| 5 |
"author": {
|
| 6 |
"name": "Headroom Contributors",
|
|
|
|
| 1 |
{
|
| 2 |
"name": "headroom",
|
| 3 |
+
"version": "0.11.0",
|
| 4 |
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
|
| 5 |
"author": {
|
| 6 |
"name": "Headroom Contributors",
|