Skip to content

Save & query results

Persist capability runs to an analytics store so you can compare results across runs, datasets, and capabilities with SQL.

1. Create a store

from checkmaite.core.analytics_store import AnalyticsStore, ParquetBackend

store = AnalyticsStore(ParquetBackend("./analytics_store"))

The Parquet backend writes plain Parquet files, one table per capability, under the given directory.

2. Write runs

Pass the runs returned by capability.run(...) to store.write(...):

store.write([bias_run, feasibility_run])
store.list_tables()

Writing the same run again is a no-op, so re-executing a notebook doesn't duplicate rows.

3. Query and join

store.query_sql(...) returns a Polars DataFrame:

store.query_sql("SELECT dataset_id, ber_upper, ber_lower FROM dataeval_feasibility")

Join capability tables on shared columns such as dataset_id:

store.query_sql(
    "SELECT b.dataset_id, b.balance_mean, f.ber_upper "
    "FROM dataeval_bias b "
    "JOIN dataeval_feasibility f USING (dataset_id)"
)

To filter by model, metric, or any other entity, join through the auto-populated runs table:

store.query_sql(
    "SELECT e.* FROM maite_evaluation e "
    "JOIN runs r ON e.run_uid = r.run_uid "
    "WHERE r.entity_type = 'model' AND r.entity_id = 'resnet50'"
)

Going further