JSON report reference

The field-level reference for the JSON report, schema version 0.2. The schema version identifies the report format. For a fixed package version, the same data and configuration produce byte-identical JSON when no deadline is set. Code that reads reports should check the schema version and tolerate unknown keys.

The top level:

Field	Type	Contents
`schema_version`	string	`"0.2"`
`overview`	object	`n_rows`, `n_columns`, `headline`, `dtype_counts`, `role_counts`, `top_findings` (top 5 claims)
`columns`	object	one card per column, keyed by name (Column cards)
`findings`	list	selected findings in rank order
`groups`	list	connected findings over shared columns; no redundancy claim
`selection`	object	`candidates`, `verified_candidates`, `distinct_statements`, `selected`
`coverage`	object	per-pass `status` and `detail` (Coverage, budgets, and deadlines)

Finding fields¶

Field	Type	Meaning
`id`	string	`category-<fnv1a64>` of the canonical statement — stable across reruns, independent of rank and detector
`category`	string	one of `missingness`, `concentration`, `column_roles`, `uniqueness`, `duplicates`, `formats`, `dependencies`, `conditional_rules`, `associations`, `gaps`, `intervals`, `ordering`, `changes`
`type`	string?	`functional` / `predictive` (dependencies), `if_then` (rules), `statistical` (associations), `residual_level` (changes)
`subject`	object	`columns` (all involved); dependencies add `inputs` and `target`; comparisons add `unit`
`specification`	object	`form` (`value_mapping`, `affine`, `piecewise_linear`), `conditions` and `outcome` for rules, `definition` for comparison settings such as tolerances and calendar references
`claim`	string	text summary of the result
`evidence`	list	one record per contributing method — see below
`interestingness`	float	deterministic selection priority in $[0,1]$ ; not a probability
`interpretation`	string?	a suggested implication, when one exists

Predicates (rule conditions and outcomes) carry column, operator (eq, ne, lt, le, gt, ge), a typed scalar value, and an optional transform (day_index for date comparisons). Numeric values stay numbers; conjunctions are lists.

Evidence records¶

Field	Type	Meaning
`method`	object	`detector` (registry id), `algorithm`, `provenance` (implementation source)
`basis`	string	`measured`, `estimated`, or `inferred`
`scope`	object	`sampling` (`full` / `sample` / `unspecified`), `population_rows`, `rows_evaluated`, `missing_values` policy
`validation`	object	`method` (`direct_comparison`, `cross_validation`, `diagnostic_holdout`, `diagnostic_partition`, `none`), `training_rows`, `evaluation_rows`, `folds`, `used_for_selection`
`metrics`	object	everything the method measured — counts, rates, coefficients, thresholds; detector-specific keys stay detector-specific

When methods produce the same normalized statement, the report combines their evidence in one finding. It retains each method’s counts and rates without averaging them. For example, a rule miner may count matches in its training rows while the shared evaluator counts matches in the diagnostic partition. Use each record’s scope and validation to identify the rows behind its metrics.

A complete finding¶

The example below generates a finding from the installed package:

import json

import polars as pl

import onceover

df = pl.DataFrame({
    "quantity": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] * 30,
})
df = df.with_columns((2 * pl.col("quantity")).alias("double_quantity"))

report = onceover.profile(df)
identity = next(f for f in report.findings
                if f["specification"].get("form") == "affine")
print(json.dumps(identity, indent=2))

{
  "id": "dependencies-b29bbce12b7cad97",
  "category": "dependencies",
  "type": "functional",
  "subject": {
    "columns": [
      "double_quantity",
      "quantity"
    ],
    "inputs": [
      "quantity"
    ],
    "target": "double_quantity"
  },
  "specification": {
    "form": "affine",
    "definition": {
      "a": 2.0,
      "absolute_tolerance": 1e-09,
      "b": 0.0,
      "missing_values": "exclude incomplete inputs and target",
      "relative_tolerance": 1e-09,
      "tolerance": "1e-9 absolute + 1e-9 relative"
    }
  },
  "claim": "double_quantity = 2.000000\u00b7quantity within tolerance on all 300 non-missing row pairs",
  "evidence": [
    {
      "method": {
        "detector": "identities",
        "algorithm": "ols_residual_scan",
        "provenance": "pairwise OLS + exact residual scan (BHUNT/BACON lineage, onceover v0.1)"
      },
      "basis": "measured",
      "scope": {
        "sampling": "full",
        "population_rows": 300,
        "rows_evaluated": 300,
        "missing_values": "see detector scope"
      },
      "validation": {
        "method": "direct_comparison",
        "used_for_selection": false
      },
      "metrics": {
        "a": 2.0,
        "absolute_tolerance": 1e-09,
        "b": 0.0,
        "exact_fraction": 1.0,
        "matching_rows": 300,
        "n": 300,
        "relative_tolerance": 1e-09,
        "tolerance": "1e-9 absolute + 1e-9 relative",
        "violations": 0
      }
    }
  ],
  "interestingness": 0.8695,
  "detector_interestingness": 0.75,
  "interpretation": "double_quantity is derived from quantity; drop it or treat as a check column"
}

Selection defaults¶

The default selection policy retains at most 100 findings across all detectors, including at most 20 conditional rules, 2 rules per outcome, and 3 predictive dependencies per target. To qualify for selection, a rule must meet the shared evaluator’s thresholds: ≥ 30 eligible rows, ≥ 30 antecedent matches, ≥ 20 joint matches, conditional rate above baseline by ≥ 0.02 absolute or lift ≥ 1.5. Python exposes max_findings and max_rules; the Rust SelectionPolicy configures the rest.

Versioning¶

schema_version identifies the format expected by a reader. Version 0.2 replaced detector-named finding kinds with semantic statements during pre-release development. Readers should check the version before interpreting fields and tolerate unknown keys within a supported format.