Skip to content

Add matched-null validation for cluster counts - #938

Open
haomeng797-ship-it wants to merge 4 commits into
easystats:mainfrom
haomeng797-ship-it:codex/test-clusters
Open

Add matched-null validation for cluster counts#938
haomeng797-ship-it wants to merge 4 commits into
easystats:mainfrom
haomeng797-ship-it:codex/test-clusters

Conversation

@haomeng797-ship-it

Copy link
Copy Markdown
Member

Closes easystats/easystats#479.

This adds test_clusters(), which compares a selected cluster count with counts from matched-null data generated by matchednull. The default pipeline standardizes the data and uses mclust::Mclust() to select the number of components by BIC; custom scalar-returning clustering functions are also supported.

The public API keeps the main controls explicit, while options such as the copula and parallel evaluation can be passed through ... to matchednull::matched_null_test().

I added tests for custom functions, reproducibility, null and positive controls, argument forwarding, and input validation.

  • 22 targeted tests passed
  • lintr passed
  • package build passed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new public helper, test_clusters(), to validate an observed (or user-supplied) cluster count against matched-null reference data generated via matchednull, with a default mclust::Mclust() (BIC) pipeline and support for custom scalar-returning cluster functions.

Changes:

  • Introduces test_clusters() (exported) with roxygen documentation and matched-null integration.
  • Adds a focused test suite covering custom functions, reproducibility, controls, argument forwarding, and input validation.
  • Updates package metadata/documentation (NEWS entry, DESCRIPTION Suggests, WORDLIST, Rd generation, NAMESPACE export).

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
R/test_clusters.R Adds the new test_clusters() API plus internal validation/standardization helpers.
tests/testthat/test-test_clusters.R Adds test coverage for the new function’s expected behavior and input checks.
NEWS.md Documents the new user-facing function in the changelog.
NAMESPACE Exports test_clusters() as part of the public API.
man/test_clusters.Rd Generated documentation for test_clusters().
DESCRIPTION Adds matchednull (>= 0.2.1) to Suggests for the new functionality/tests.
inst/WORDLIST Adds “Meng” to the spelling whitelist for the new reference entry.
Files not reviewed (1)
  • man/test_clusters.Rd: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread R/test_clusters.R
Comment on lines +86 to +88
insight::check_if_installed("mclust")
mclustBIC <- mclust::mclustBIC
max_components <- min(as.integer(n_max), nrow(x) - 1L)
Comment thread DESCRIPTION
loo,
MASS,
Matrix,
matchednull (>= 0.2.1),
@strengejacke

Copy link
Copy Markdown
Member

Thanks, looks good to me! A version bump (4th digits, to increase dev version) in the DESCRIPTION is missing (or overwritten by merge main into this branch), else I have no other comments. @DominiqueMakowski anything you would like to add?

@haomeng797-ship-it

Copy link
Copy Markdown
Member Author

Thanks! I bumped the development version to 0.17.1.11.

@DominiqueMakowski

DominiqueMakowski commented Aug 25, 2026

Copy link
Copy Markdown
Member

This is a bit clunky

#' # Any scalar-returning clustering pipeline can be tested.
#' pick_two <- function(data) 2
#' test_clusters(iris[, 1:4], cluster_function = pick_two, iterations = 19)

Rather than having the whole Mclust complication (which is used to get a number), why not streamline the api and just have the n_clusters argument that takes either a number, or a list of numbers to test multiple solutions.

The pipeline could then be like

n <- n_clusters(data)
# this suggests 3
test_clusters(data, n_clusters = c(2, 3, 5))

or am i missing somethign

@haomeng797-ship-it

Copy link
Copy Markdown
Member Author

Thanks for pointing that out. I agree that the pick_two() example is awkward. The function argument is still needed, though, because the same cluster-selection procedure has to run on every matched-null dataset. Supplying fixed values through n_clusters would not specify how a cluster count should be obtained from each null dataset.

I’ll replace the example with a real clustering pipeline and clarify this in the documentation.

@DominiqueMakowski

Copy link
Copy Markdown
Member

You mean the function needs to be aware of the clustering method?

Then why not (I'm just brainstorming I dont have strong opinions) making the function a method that runs on a cluster_analysis() result? So the workflow would go like

n_clusters(data)
Rez <- cluster_analysis(...l

Test_clusters(Rez)
``|

It would actually test against a specific solution 

@haomeng797-ship-it

Copy link
Copy Markdown
Member Author

Yes that’s what I mean. test_clusters() needs some way to rerun the clustering method on each matched-null dataset, rather than only knowing the number selected for the original data.

I like the idea of making it a method for a cluster_analysis() result. That would probably feel more natural here. We’d just need the result object to retain the method and arguments, so the same analysis can be rerun on the null datasets.

The only thing I want to preserve is that the current test evaluates the selection process, rather than just one fixed solution.
I’ll look at what cluster_analysis() stores and see how the two could fit together!

@DominiqueMakowski

Copy link
Copy Markdown
Member

I’ll look at what cluster_analysis() stores

if need be we can add more attributes to store more info :) we do that quite commonly across the easyverse

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: matched-null validation for the easystats clustering workflow

4 participants