EDA & Statistics
EDA worksheets are interactive exploratory-data-analysis surfaces on a dataset — univariate summaries, correlation matrices, scatter-and-fit, and a library of statistical test cards. Unlike a recipe, a worksheet computes on demand and is not part of the build graph; it is where an analyst pokes at a dataset before deciding what to model.
Worksheets persist per dataset (honeyframe.dataset_worksheets) and read through the governed statistics API, so PII masking applies to every value a worksheet shows.
Univariate & bivariate
- Univariate — per-column summaries: count, nulls, distinct, min/max, mean, quantiles, and a distribution sketch.
- Correlation matrix — pairwise correlation across the numeric columns you select.
- Scatter + fit (
scatter_fit) — a scatter of two columns with a fitted trend line.
Sampling
Statistics run over a sample so exploration stays fast on large tables. The sampling picker offers first, random, or last; the worksheet notes which sample a result was computed on.
Statistical test cards
Beyond summaries, a worksheet offers named statistical tests, each rendered as a card with its statistic, p-value (where applicable), and a plot:
| Card | Use |
|---|---|
| Chi-squared | Independence between two categorical columns. |
| Welch's t-test | Difference in means between two groups (unequal variance). |
| Kolmogorov–Smirnov | Whether two samples come from the same distribution. |
| CDF | Empirical cumulative distribution of a column. |
| Fit distribution | Fit a column to a candidate distribution and report goodness-of-fit. |
| ACF | Autocorrelation of a time-ordered series at increasing lags. |
| Mann–Kendall | Monotonic trend test for a time series. |
| Quantiles | Quantile table / plot for a numeric column. |
API
The statistics surface is served by the /api/statistics router (univariate, correlation matrix, scatter_fit, and the test cards). Worksheet state is saved so a worksheet reopens where you left it.
Relation to profiling and recipes
- The
generate_statisticsPrepare processor / recipe materializes per-column profiling (fill rate, null/distinct counts, min/max) as a dataset you can build on and schedule — use it when you want profiling in the pipeline rather than an ad-hoc worksheet. - The Catalog column profile (median, p25/p75, IQR, sample stddev) is the cached, at-a-glance version surfaced on the asset's Profile tab.
Use worksheets to explore, generate_statistics to persist, and the catalog profile for a quick governed snapshot.