Interactive Validation App for 'quallmer' R package quallmer.app version 0.1.0 Companion package to 'quallmer' providing an interactive 'shiny' application for manual coding, reviewing large language model (LLM) generated annotations, and computing inter-rater reliability metrics. Supports three modes: blind manual coding, LLM output validation, and agreement calculation. Computes standard reliability metrics including Krippendorff's alpha (Krippendorff 2019 <doi:10.4135/9781071878781>), Cohen's kappa, Fleiss' kappa (Fleiss 1971 <doi:10.1037/h0031619>), intraclass correlation coefficient (ICC), and percent agreement for nominal, ordinal, interval, and ratio data. Also computes gold-standard validation metrics including accuracy, precision, recall, and F1 scores following Sokolova and Lapalme (2009 <doi:10.1016/j.ipm.2009.03.002>). Software Depends: R (>= 3.5) Imports: quallmer (>= 0.3.0), shiny, bslib, dplyr, tidyr, irr, htmltools, cli, stats, utils Suggests: testthat (>= 3.0.0) Seraphine F. Maerz <seraphine.maerz@unimelb.edu.au> Comprehensive R Archive Network (CRAN) Seraphine F. Maerz [aut, cre] (ORCID: <https://orcid.org/0000-0002-7173-9617>), Kenneth Benoit [aut] (ORCID: <https://orcid.org/0000-0002-0797-564X>) MIT + file LICENSE (https://CRAN.R-project.org/package=quallmer.app/LICENSE) 2026-03-08 application/tgz https://CRAN.R-project.org/package=quallmer.app doi:10.32614/CRAN.package.quallmer.app