Xiao, Zhang, Lai and Liao's 2023 MetricEval explicitly transfers Cronbach & Meehl's measurement-theory vocabulary onto embedding-based NLG metrics, independently of Steck/Ekanadham/Kallus

measurement-theoryconstruct-validitycosine-similaritynlg-evaluationembeddingscronbachmeehl

Xiao, Zhang, Lai and Liao (2023) propose "MetricEval," a framework importing measurement theory wholesale into the evaluation of natural-language-generation (NLG) metrics, explicitly including "embedding-based metrics (e.g., BERTScore, MoverScore)" — the same family of cosine-similarity-adjacent scores Steck, Ekanadham and Kallus warn about in claim-cosine-similarity-of-embeddings-can-be-arbitrary. Their framing: "Key to measurement theory is the distinction between the observed score on a test... and the true score on the general construct (Cronbach and Meehl, 1955) that the test is theorized to measure... The gap between the observed and true scores is referred to as measurement error." Applied to NLG evaluation, a benchmark metric's numeric output is an "observed score" standing in for an "unobservable capability" (e.g., summarization quality) the metric is only theorized, not guaranteed, to track — see claim-cronbach-meehl-1955-construct-validity-names-general-substitution-pattern.

This shows the measurement-theory naming this capture went looking for is not confined to psychometrics or bibliometrics: as of 2023, researchers explicitly re-apply Cronbach & Meehl's 1955 vocabulary to the identical class of scalar metric (embedding-based similarity scores) that Steck, Ekanadham and Kallus's 2024 warning concerns — independently, and without citing Steck et al. or the Garfield/Seglen bibliometrics literature at all. It is the modern bridge between the two older literatures (claim-robinson-1950-ecological-fallacy-names-garfield-seglen-mechanism, claim-cronbach-meehl-1955-construct-validity-names-general-substitution-pattern) and the vault's own embedding-false-friend diagnostic cluster.