get.R2.RdThe function is able to calculate the McFadden pseudo-\(R^{2}\) (\(R^{2}\)) for all items after
fitting CDM or directly.
get.R2(Y = NULL, Q = NULL, CDM.obj = NULL, model = "GDINA")A required N × I matrix or data.frame consisting of the responses of N
individuals to I items. Missing values should be coded as NA.
A required binary I × K matrix containing the attributes not required or required,
coded as 0 or 1, to master the items. The ith row of the matrix is a binary indicator vector
indicating which attributes are not required (coded as 0) and which attributes are required
(coded as 1) to master item i.
An object of class CDM.obj. Can can be NULL, but when it is not NULL, it
enables rapid verification of the Q-matrix without the need for parameter estimation.
@seealso CDM.
Type of model to fit; can be "GDINA", "LCDM", "DINA", "DINO",
"ACDM", "LLM", or "rRUM". Default = "GDINA".
An object of class matrix, which consisted of \(R^{2}\) for each item and each possible attribute mastery pattern.
The McFadden pseudo-\(R^{2}\) ( McFadden in 1974) serves as a definitive model-fit index, quantifying the proportion of variance explained by the observed responses. Comparable to the squared multiple-correlation coefficient in linear statistical models, this coefficient of determination finds its application in logistic regression models. Specifically, in the context of the CDM, where probabilities of accurate item responses are predicted for each examinee, the McFadden pseudo-\(R^{2}\) provides a metric to assess the alignment between these predictions and the actual responses observed. Its computation is straightforward, following the formula: $$ R_{i}^{2} = 1 - \frac{\log(L_{im}}{\log(L_{i0})} $$ where \(\log(L_{im}\) is the log-likelihood of the model, and \(\log(L_{i0})\) is the log-likelihood of the null model. If there were \(N\) examinees taking a test comprising \(I\) items, then \(\log(L_{im})\) would be computed as: $$ \log(L_{im}) = \sum_{p}^{N} \log \sum_{l=1}^{2^{K^\ast}} \pi(\alpha_{l}^{\ast} | X_{p}) P_{i}(\alpha_{l}^{\ast})^{X_{pi}} (1-P_{i}(\alpha_{l}^{\ast}))^{1-X_{pi}} $$ where \(\pi(\alpha_{l}^{\ast} | X_{p})\) is the posterior probability of examinee \(p\) with attribute profle \(\alpha_{l}^{\ast}\) when their response vector is \(\mathbf{X}_{p}\), and \(X_{pi}\) is examinee \(p\)'s response to item \(i\). Let \(X_{i}^{mean}\) be the average probability of correctly responding to item \(i\) across all \(N\) examinees; then \(\log(L_{i0}\) could be computed as: $$ \log(L_{i0}) = \sum_{p}^{N} \log {X_{i}^{mean}}^{X_{pi}} {(1-X_{i}^{mean})}^{1-X_{pi}} $$
McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembka (Ed.), Frontiers in economics (pp.105–142). Academic Press.
Najera, P., Sorrel, M. A., de la Torre, J., & Abad, F. J. (2021). Balancing ft and parsimony to improve Q-matrix validation. British Journal of Mathematical and Statistical Psychology, 74, 110–130. DOI: 10.1111/bmsp.12228.
Qin, H., & Guo, L. (2023). Using machine learning to improve Q-matrix validation. Behavior Research Methods. DOI: 10.3758/s13428-023-02126-0.
library(Qval)
set.seed(123)
## generate Q-matrix and data
K <- 3
I <- 20
example.Q <- sim.Q(K, I)
IQ <- list(
P0 = runif(I, 0.0, 0.2),
P1 = runif(I, 0.8, 1.0)
)
example.data <- sim.data(Q = example.Q, N = 500, IQ = IQ, model = "GDINA", distribute = "horder")
#> distribute = horder
#> model = GDINA
#> number of attributes: 3
#> number of items: 20
#> num of examinees: 500
#> average of P0 = 0.083
#> average of P1 = 0.894
#> theta_mean = -0.055 , theta_sd = 0.996
#> a = 1.5 1.5 1.5
#> b = -1.5 1.5 0
## calculate PVAF directly
PVAF <-get.PVAF(Y = example.data$dat, Q = example.Q)
#>
Iter = 1 Max. abs. change = 0.57951 Deviance = 10238.69
Iter = 2 Max. abs. change = 0.09509 Deviance = 8501.84
Iter = 3 Max. abs. change = 0.03995 Deviance = 8478.24
Iter = 4 Max. abs. change = 0.01792 Deviance = 8477.13
Iter = 5 Max. abs. change = 0.02604 Deviance = 8477.02
Iter = 6 Max. abs. change = 0.01142 Deviance = 8476.97
Iter = 7 Max. abs. change = 0.01740 Deviance = 8476.96
Iter = 8 Max. abs. change = 0.00686 Deviance = 8476.95
Iter = 9 Max. abs. change = 0.00466 Deviance = 8476.94
Iter = 10 Max. abs. change = 0.00253 Deviance = 8476.94
Iter = 11 Max. abs. change = 0.00030 Deviance = 8476.94
Iter = 12 Max. abs. change = 0.00014 Deviance = 8476.94
Iter = 13 Max. abs. change = 0.00016 Deviance = 8476.94
Iter = 14 Max. abs. change = 0.00760 Deviance = 8476.94
Iter = 15 Max. abs. change = 0.00246 Deviance = 8476.94
Iter = 16 Max. abs. change = 0.00502 Deviance = 8476.94
Iter = 17 Max. abs. change = 0.00017 Deviance = 8476.94
Iter = 18 Max. abs. change = 0.00109 Deviance = 8476.94
Iter = 19 Max. abs. change = 0.00004 Deviance = 8476.94
print(PVAF)
#> 000 100 010 001 110 101 011
#> item 1 -Inf 0.05200795 0.97617096 0.18597232 0.9803500 0.23900194 0.99588964
#> item 2 -Inf 0.86258484 0.03768251 0.31301239 0.8673840 0.98691938 0.32148058
#> item 3 -Inf 0.23538077 0.12153176 0.87250475 0.3079287 0.97444297 0.88197677
#> item 4 -Inf 0.05118094 0.09645885 0.99944525 0.1444479 0.99977914 0.99955172
#> item 5 -Inf 0.61967444 0.04634396 0.40114822 0.6340752 0.99292783 0.40218598
#> item 6 -Inf 0.05150476 0.08521455 0.99863037 0.1404493 0.99872632 0.99889345
#> item 7 -Inf 0.04302786 0.07413417 0.99720362 0.1172823 0.99794531 0.99857592
#> item 8 -Inf 0.02645312 0.99707114 0.10788852 0.9975597 0.14388347 0.99828838
#> item 9 -Inf 0.04978628 0.10729370 0.99870662 0.1541773 0.99928813 0.99944277
#> item 10 -Inf 0.11329682 0.60988009 0.38267804 0.7404337 0.44888604 0.96863356
#> item 11 -Inf 0.16252795 0.78512355 0.24785470 0.9700515 0.34201830 0.88499107
#> item 12 -Inf 0.08412574 0.52565713 0.68491774 0.6140911 0.70190926 0.99303516
#> item 13 -Inf 0.99543822 0.02468354 0.04060490 0.9970518 0.99778609 0.06769133
#> item 14 -Inf 0.04892994 0.07984576 0.99743751 0.1302842 0.99788185 0.99867379
#> item 15 -Inf 0.99641298 0.02098946 0.06173904 0.9971472 0.99679660 0.07085647
#> item 16 -Inf 0.04307850 0.80827365 0.20902293 0.8605611 0.23803789 0.99502425
#> item 17 -Inf 0.11898439 0.80125584 0.25400368 0.9206240 0.31229694 0.93674309
#> item 18 -Inf 0.19312807 0.09111948 0.91186601 0.2681952 0.97890622 0.92431327
#> item 19 -Inf 0.01166348 0.99302358 0.07554560 0.9972778 0.09136379 0.99406882
#> item 20 -Inf 0.26878163 0.73646434 0.22459899 0.9125336 0.41706919 0.79645483
#> 111
#> item 1 1
#> item 2 1
#> item 3 1
#> item 4 1
#> item 5 1
#> item 6 1
#> item 7 1
#> item 8 1
#> item 9 1
#> item 10 1
#> item 11 1
#> item 12 1
#> item 13 1
#> item 14 1
#> item 15 1
#> item 16 1
#> item 17 1
#> item 18 1
#> item 19 1
#> item 20 1
## caculate PVAF after fitting CDM
example.CDM.obj <- CDM(example.data$dat, example.Q, model="GDINA")
#>
Iter = 1 Max. abs. change = 0.57951 Deviance = 10238.69
Iter = 2 Max. abs. change = 0.09509 Deviance = 8501.84
Iter = 3 Max. abs. change = 0.03995 Deviance = 8478.24
Iter = 4 Max. abs. change = 0.01792 Deviance = 8477.13
Iter = 5 Max. abs. change = 0.02604 Deviance = 8477.02
Iter = 6 Max. abs. change = 0.01142 Deviance = 8476.97
Iter = 7 Max. abs. change = 0.01740 Deviance = 8476.96
Iter = 8 Max. abs. change = 0.00686 Deviance = 8476.95
Iter = 9 Max. abs. change = 0.00466 Deviance = 8476.94
Iter = 10 Max. abs. change = 0.00253 Deviance = 8476.94
Iter = 11 Max. abs. change = 0.00030 Deviance = 8476.94
Iter = 12 Max. abs. change = 0.00014 Deviance = 8476.94
Iter = 13 Max. abs. change = 0.00016 Deviance = 8476.94
Iter = 14 Max. abs. change = 0.00760 Deviance = 8476.94
Iter = 15 Max. abs. change = 0.00246 Deviance = 8476.94
Iter = 16 Max. abs. change = 0.00502 Deviance = 8476.94
Iter = 17 Max. abs. change = 0.00017 Deviance = 8476.94
Iter = 18 Max. abs. change = 0.00109 Deviance = 8476.94
Iter = 19 Max. abs. change = 0.00004 Deviance = 8476.94
PVAF <-get.PVAF(CDM.obj = example.CDM.obj)
print(PVAF)
#> 000 100 010 001 110 101 011
#> item 1 -Inf 0.05200795 0.97617096 0.18597232 0.9803500 0.23900194 0.99588964
#> item 2 -Inf 0.86258484 0.03768251 0.31301239 0.8673840 0.98691938 0.32148058
#> item 3 -Inf 0.23538077 0.12153176 0.87250475 0.3079287 0.97444297 0.88197677
#> item 4 -Inf 0.05118094 0.09645885 0.99944525 0.1444479 0.99977914 0.99955172
#> item 5 -Inf 0.61967444 0.04634396 0.40114822 0.6340752 0.99292783 0.40218598
#> item 6 -Inf 0.05150476 0.08521455 0.99863037 0.1404493 0.99872632 0.99889345
#> item 7 -Inf 0.04302786 0.07413417 0.99720362 0.1172823 0.99794531 0.99857592
#> item 8 -Inf 0.02645312 0.99707114 0.10788852 0.9975597 0.14388347 0.99828838
#> item 9 -Inf 0.04978628 0.10729370 0.99870662 0.1541773 0.99928813 0.99944277
#> item 10 -Inf 0.11329682 0.60988009 0.38267804 0.7404337 0.44888604 0.96863356
#> item 11 -Inf 0.16252795 0.78512355 0.24785470 0.9700515 0.34201830 0.88499107
#> item 12 -Inf 0.08412574 0.52565713 0.68491774 0.6140911 0.70190926 0.99303516
#> item 13 -Inf 0.99543822 0.02468354 0.04060490 0.9970518 0.99778609 0.06769133
#> item 14 -Inf 0.04892994 0.07984576 0.99743751 0.1302842 0.99788185 0.99867379
#> item 15 -Inf 0.99641298 0.02098946 0.06173904 0.9971472 0.99679660 0.07085647
#> item 16 -Inf 0.04307850 0.80827365 0.20902293 0.8605611 0.23803789 0.99502425
#> item 17 -Inf 0.11898439 0.80125584 0.25400368 0.9206240 0.31229694 0.93674309
#> item 18 -Inf 0.19312807 0.09111948 0.91186601 0.2681952 0.97890622 0.92431327
#> item 19 -Inf 0.01166348 0.99302358 0.07554560 0.9972778 0.09136379 0.99406882
#> item 20 -Inf 0.26878163 0.73646434 0.22459899 0.9125336 0.41706919 0.79645483
#> 111
#> item 1 1
#> item 2 1
#> item 3 1
#> item 4 1
#> item 5 1
#> item 6 1
#> item 7 1
#> item 8 1
#> item 9 1
#> item 10 1
#> item 11 1
#> item 12 1
#> item 13 1
#> item 14 1
#> item 15 1
#> item 16 1
#> item 17 1
#> item 18 1
#> item 19 1
#> item 20 1