The code and prose in this chapter were written by Claude Code (an AI coding agent) under human direction. They have not been line-by-line verified by a human and must not be published or cited as-is.
WarningPreliminary & unvetted analysis
These are exploratory analyses of preliminary, in-progress LEVANTE data. Results have not been reviewed or validated and require checking by the LEVANTE Data Coordinating Center before any use.
Purpose
A QA screen for differential item functioning across languages on the five core tasks calibrated with a multigroup_site scalar model (so item difficulties are on a common scale): Math, Pattern Matching (matrix), Shape Rotation (mental rotation), Hearts & Flowers, Memory. (ToM and TROG were done separately; the by_language tasks — Vocab, ROAR — need the anchored approach and aren’t included here.)
Method (as for TROG): fit a Rasch model separately per language on each child’s first session (German = Leipzig, Spanish = Bogotá + rural, English = Canada), center each language’s difficulties, and compare. Large cross-language gaps flag items that may be mistranslated or mis-keyed.
Code
source(here::here("common.R"))library(mirt)sc <-read_rds(here("data/scores_all_sites.rds"))lang_of <-function(ds) dplyr::case_when( ds =="pilot_mpieva_de_main"~"German", stringr::str_detect(ds, "uniandes_co") ~"Spanish", ds =="pilot_western_ca_main"~"English", TRUE~NA_character_)fit_lang <-function(trd, L) { m <- trd |>filter(lang == L) |>distinct(run_id, item_uid, .keep_all =TRUE) |>select(run_id, item_uid, correct) |>pivot_wider(names_from = item_uid, values_from = correct) |>select(-run_id) |>as.matrix() m <- m[, colMeans(!is.na(m)) >0.03, drop =FALSE] m <- m[, apply(m, 2, \(x) length(unique(x[!is.na(x)])) >1), drop =FALSE]if (ncol(m) <5) return(NULL) fit <-mirt(m, 1, itemtype ="Rasch", verbose =FALSE, technical =list(NCYCLES =2000)) d <-coef(fit, simplify =TRUE)$items[, "d"]tibble(item_uid =names(d), b_c = (-d) -mean(-d), lang = L)}tasks <-c("egma-math","matrix-reasoning","mental-rotation","hearts-and-flowers","memory-game")
Mental Rotation is essentially invariant (r ≈ 0.96–0.99 — as expected for a non-verbal task); Memory and Pattern Matching are high (≈ 0.94–0.95 / mixed); Math and Hearts & Flowers are a bit lower (≈ 0.81–0.92), reflecting a handful of items rather than systemic drift. Crucially, none approaches the broken-item extreme seen in TROG (a 6.5-logit gap).
Flagged items (largest cross-language gaps)
Code
wide |>filter(maxgap >1) |>arrange(desc(maxgap)) |>mutate(across(c(German,Spanish,English,maxgap), \(x) round(x,2))) |>select(task_label, item_uid, German, Spanish, English, maxgap) |>head(20) |> knitr::kable(caption ="Items with > 1-logit max cross-language difficulty gap (centered)")
Items with > 1-logit max cross-language difficulty gap (centered)
wide |>filter(!is.na(German), !is.na(Spanish)) |>ggplot(aes(x = Spanish, y = German)) +geom_abline(slope =1, intercept =0, lty =3, color ="grey50") +geom_point(alpha =0.5, color ="#268bd2", size =1) + ggrepel::geom_text_repel(data =~filter(.x, maxgap >1.3),aes(label =str_remove(item_uid, "^(math_|matrix_|mrot_|mg_)")),size =2.2, color ="firebrick", max.overlaps =12) +facet_wrap(~task_label, scales ="free") +labs(x ="Item difficulty — Spanish (centered)",y ="Item difficulty — German (centered)",title ="Cross-language item difficulty: German vs Spanish",subtitle ="On-diagonal = invariant; labeled (red) = max pairwise gap > 1.3 logits")
Verdict
These five tasks are broadly cross-language invariant. Mental Rotation is near-perfect (non-verbal), Memory and Pattern Matching strong; Math and Hearts & Flowers a little lower but driven by specific items, not systemic drift.
No broken items on the scale of the TROG German embedding item. The largest gaps here (~1.5–2.2 logits) are worth a look but are not catastrophic; several are short Memory spans and a few Math items.
Hearts & Flowers: the weakest pair is German–Spanish (r ≈ 0.81), with heart_start carrying the most DIF — consistent with start trials being noisy/low-information (see tasks/hearts_and_flowers.qmd).
Action: spot-check the > 1.5-logit items (esp. the top Math and Pattern Matching items) for translation/key issues; otherwise these tasks are sound for cross-site use. Vocab and the ROAR by_language tasks remain to be screened with the anchored multigroup method.
