A reported rule may have been found by more than one method. To evaluate that rule, check the rows and success counts used by each method. This example shows where those numbers appear in a report.
A shipping rule¶
The fixture contains 4,000 orders. Premium orders usually ship express; standard orders usually ship ground. PRISM and association mining search for rules in these columns. After discovery, the rule evaluator counts how often each rule holds on a diagnostic subset of the table.
import json
import random
import polars as pl
import onceover
rng = random.Random(11)
n = 4000
segment = ["premium" if rng.random() < 0.5 else "std" for _ in range(n)]
shipping = [
"express" if (rng.random() < 0.95 if s == "premium" else rng.random() < 0.05)
else "ground"
for s in segment
]
df = pl.DataFrame({"segment": segment, "shipping": shipping})
report = onceover.profile(df)
rule = next(f for f in report.findings if f["category"] == "conditional_rules")
print("claim:", rule["claim"])
print("conditions:", json.dumps(rule["specification"]["conditions"]))
print("outcome:", json.dumps(rule["specification"]["outcome"]))
print("detectors contributing evidence:", [e["method"]["detector"] for e in rule["evidence"]])claim: When segment = "premium", shipping = "express" holds in 362 of 380 matching diagnostic rows (95.26%; baseline 47.28%)
conditions: [{"column": "segment", "operator": "eq", "value": "premium"}]
outcome: {"column": "shipping", "operator": "eq", "value": "express"}
detectors contributing evidence: ['association_rules', 'prism', 'rule_evaluator']
The selected rule is segment = "premium" → shipping = "express".
Among the 380 premium orders in the diagnostic subset, 362 ship express:
95.26%, compared with 47.28% express shipping across that subset.
PRISM, association mining, and the rule evaluator appear in the output
because they contributed to this finding. Their metrics serve different
purposes. PRISM scores conditions while constructing a rule; association
mining measures support and lift on its training and holdout rows. The
reported rule rate comes from the common diagnostic evaluation. For that
rate, read the conditional_rate, joint_count, and antecedent_count
metrics in the rule_evaluator record.
Rows and validation¶
An evidence record identifies the method and contains its metrics. Use these fields when deciding what the metrics describe:
| Field | How to read it |
|---|---|
scope.population_rows | Number of rows in the input table |
scope.rows_evaluated | Number of rows used for this record’s measurement |
scope.sampling | Whether those rows cover the full table or a sample |
scope.missing_values | How missing values affected eligibility or scoring |
validation.method | The check used, such as direct comparison or cross-validation |
validation.used_for_selection | Whether the check also helped choose a reported result |
For example, a model may use a holdout to choose when to stop fitting. Its holdout score then helped select the model. Before using that model on new data, evaluate it on another dataset that was not used for those choices.
The basis field distinguishes a direct measurement (measured), a
sample or model estimate (estimated), and an interpretation suggested by
other measurements (inferred). An identifier-like column role is an
interpretation: a high distinct count alone does not tell you what the
values mean.
When results are combined¶
Two methods can report the same rule. The report combines them when the category, type, columns, conditions, and outcome match. It stores their evidence under one finding instead of repeating the rule in the list.
Different conditions remain separate. For example, X → Y differs from
Y → X, and a rule about rows where X = a differs from a mapping checked
across every observed value of X.
When combining candidates, the implementation uses the description from the candidate with the larger evaluated row count. It retains the metrics from both candidates. Do not add their sample counts: the same rows may have contributed to both measurements.
Ranking and report limits¶
The selector scores findings using validation quality, the reported effect, the number of inputs and conditions, and whether a formula or mapping is specified. These scores determine which findings to inspect first; they are not probabilities that the findings are correct.
The default report contains at most 100 findings and 20 conditional rules.
It also limits predictive findings to three per target and rules to two
per outcome, such as shipping = "express". Python exposes max_findings
and max_rules; Rust’s SelectionPolicy exposes the other limits.
report.data["selection"] reports how many candidates were generated,
how many remained after verification, how many distinct findings they
represented, and how many were included in the report.
print(json.dumps(report.data["selection"], indent=2))
from collections import Counter
print("selected by category:", dict(Counter(f["category"] for f in report.findings))){
"candidates": 15,
"verified_candidates": 15,
"distinct_statements": 9,
"selected": 9
}
selected by category: {'dependencies': 2, 'duplicates': 1, 'conditional_rules': 4, 'formats': 2}
The output shows 15 candidates, nine distinct findings, and nine included in the report. Some candidates described the same rules; the report did not need to remove any distinct findings to meet its limits.
The shipping rate above describes the diagnostic rows. Those rows may already have been used during rule discovery, so test the rule on new orders before assuming it will perform as well there.
If an expected finding is absent, check coverage for skipped passes, sample sizes, and search limits. The JSON reference lists all finding and evidence fields.