get.R2.RdThe function is able to calculate the McFadden pseudo-\(R^{2}\) (\(R^{2}\)) for all items after
fitting CDM or directly.
get.R2(
Y = NULL,
Q = NULL,
att.str = NULL,
CDM.obj = NULL,
mono.constraint = FALSE,
model = "GDINA"
)A required \(N\) × \(I\) matrix or data.frame consisting of the responses of N individuals
to \(N\) × \(I\) items. Missing values need to be coded as NA.
A required binary \(I\) × \(K\) matrix containing the attributes not required or required
master the items. The ith row of the matrix is a binary indicator vector indicating which
attributes are not required (coded by 0) and which attributes are required (coded by 1) to master
item \(i\).
Specify attribute structures. NULL, by default, means there is no structure. Attribute structure
needs be specified as a list - which will be internally handled by att.structure function.
See examples. It can also be a matrix giving all permissible attribute profiles.
An object of class CDM.obj. Can be NULL, but when it is not NULL, it
enables rapid verification of the Q-matrix without the need for parameter estimation.
@seealso CDM.
Logical indicating whether monotonicity constraints should be fulfilled in estimation.
Default = FALSE.
Type of model to fit; can be "GDINA", "LCDM", "DINA", "DINO",
"ACDM", "LLM", or "rRUM". Default = "GDINA".
An object of class matrix, which consisted of \(R^{2}\) for each item and each possible attribute mastery pattern.
The McFadden pseudo-\(R^{2}\) (McFadden, 1974) serves as a definitive model-fit index, quantifying the proportion of variance explained by the observed responses. Comparable to the squared multiple-correlation coefficient in linear statistical models, this coefficient of determination finds its application in logistic regression models. Specifically, in the context of the CDM, where probabilities of accurate item responses are predicted for each examinee, the McFadden pseudo-\(R^{2}\) provides a metric to assess the alignment between these predictions and the actual responses observed. Its computation is straightforward, following the formula: $$ R_{i}^{2} = 1 - \frac{\log(L_{im})}{\log(L_{i0})} $$ where \(\log(L_{im})\) is the log-likelihood of the model, and \(\log(L_{i0})\) is the log-likelihood of the null model. If there were \(N\) examinees taking a test comprising \(I\) items, then \(\log(L_{im})\) would be computed as: $$ \log(L_{im}) = \sum_{p}^{N} \log \sum_{l=1}^{2^{K^\ast}} \pi(\boldsymbol{\alpha}_{l}^{\ast} | \boldsymbol{X}_{p}) P_{i}(\boldsymbol{\alpha}_{l}^{\ast})^{X_{pi}} \left[ 1-P_{i}(\boldsymbol{\alpha}_{l}^{\ast}) \right] ^{(1-X_{pi})} $$ where \(\pi(\boldsymbol{\alpha}_{l}^{\ast} | \boldsymbol{X}_{p})\) is the posterior probability of examinee \(p\) with attribute mastery pattern \(\boldsymbol{\alpha}_{l}^{\ast}\) when their response vector is \(\boldsymbol{X}_{p}\), and \(X_{pi}\) is examinee \(p\)'s response to item \(i\). Let \(\bar{X}_{i}\) be the average probability of correctly responding to item \(i\) across all \(N\) examinees; then \(\log(L_{i0})\) could be computed as: $$ \log(L_{i0}) = \sum_{p}^{N} \log {\bar{X}_{i}}^{X_{pi}} {(1-\bar{X}_{i})}^{(1-X_{pi})} $$
McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembka (Ed.), Frontiers in economics (pp.105–142). Academic Press.
Najera, P., Sorrel, M. A., de la Torre, J., & Abad, F. J. (2021). Balancing ft and parsimony to improve Q-matrix validation. British Journal of Mathematical and Statistical Psychology, 74, 110–130. DOI: 10.1111/bmsp.12228.
Qin, H., & Guo, L. (2023). Using machine learning to improve Q-matrix validation. Behavior Research Methods. DOI: 10.3758/s13428-023-02126-0.
# \donttest{
library(Qval)
set.seed(123)
## generate Q-matrix and data
K <- 3
I <- 20
Q <- sim.Q(K, I)
IQ <- list(
P0 = runif(I, 0.0, 0.2),
P1 = runif(I, 0.8, 1.0)
)
data <- sim.data(Q = Q, N = 500, IQ = IQ, model = "GDINA", distribute = "horder")
#> distribute = horder
#> model = GDINA
#> number of attributes: 3
#> number of items: 20
#> num of examinees: 500
#> average of P0 = 0.083
#> average of P1 = 0.894
#> theta_mean = -0.055 , theta_sd = 0.996
#> a = 1.5 1.5 1.5
#> b = -1.5 1.5 0
## calculate R2 directly
R2 <-get.R2(Y = data$dat, Q = Q)
#>
Iter = 1 Max. abs. change = 0.58226 Deviance = 10238.69
Iter = 2 Max. abs. change = 0.09628 Deviance = 8499.25
Iter = 3 Max. abs. change = 0.03447 Deviance = 8474.14
Iter = 4 Max. abs. change = 0.01418 Deviance = 8473.11
Iter = 5 Max. abs. change = 0.01087 Deviance = 8473.00
Iter = 6 Max. abs. change = 0.00870 Deviance = 8472.98
Iter = 7 Max. abs. change = 0.00678 Deviance = 8472.96
Iter = 8 Max. abs. change = 0.00521 Deviance = 8472.96
Iter = 9 Max. abs. change = 0.00398 Deviance = 8472.95
Iter = 10 Max. abs. change = 0.00303 Deviance = 8472.95
Iter = 11 Max. abs. change = 0.00230 Deviance = 8472.95
Iter = 12 Max. abs. change = 0.00174 Deviance = 8472.95
Iter = 13 Max. abs. change = 0.00131 Deviance = 8472.95
Iter = 14 Max. abs. change = 0.00099 Deviance = 8472.95
Iter = 15 Max. abs. change = 0.00075 Deviance = 8472.95
Iter = 16 Max. abs. change = 0.00056 Deviance = 8472.95
Iter = 17 Max. abs. change = 0.00042 Deviance = 8472.95
Iter = 18 Max. abs. change = 0.00032 Deviance = 8472.95
Iter = 19 Max. abs. change = 0.00024 Deviance = 8472.95
Iter = 20 Max. abs. change = 0.00018 Deviance = 8472.95
Iter = 21 Max. abs. change = 0.00014 Deviance = 8472.95
Iter = 22 Max. abs. change = 0.00010 Deviance = 8472.95
Iter = 23 Max. abs. change = 0.00008 Deviance = 8472.95
print(R2)
#> 000 100 010 001 110 101 011
#> item 1 0 0.01687504 0.30950008 0.05616181 0.30992650 0.07231454 0.31772871
#> item 2 0 0.36107634 0.01544152 0.12313741 0.36298109 0.43391740 0.12771177
#> item 3 0 0.07112106 0.03612654 0.25918458 0.09295401 0.30354235 0.26418985
#> item 4 0 0.03774422 0.06736935 0.85351062 0.10422207 0.85679557 0.85433823
#> item 5 0 0.32838492 0.02760602 0.22970691 0.33880749 0.52236570 0.23018914
#> item 6 0 0.03537067 0.05523055 0.76782002 0.09479502 0.76806713 0.76971939
#> item 7 0 0.02592157 0.04171859 0.65304143 0.06780609 0.65505092 0.65581563
#> item 8 0 0.01929698 0.73409734 0.07241102 0.73300997 0.09614703 0.74105140
#> item 9 0 0.01807026 0.03587836 0.36332903 0.05231119 0.36402050 0.36359018
#> item 10 0 0.07302217 0.30029774 0.21522472 0.36795647 0.25182273 0.50334356
#> item 11 0 0.05293293 0.22654792 0.07114991 0.28797398 0.10312816 0.25840838
#> item 12 0 0.03458177 0.17622498 0.26878724 0.20421304 0.27532169 0.36946609
#> item 13 0 0.67881388 0.01856466 0.02692421 0.67960624 0.68129055 0.05298808
#> item 14 0 0.02753211 0.04269419 0.59776244 0.07172353 0.59890664 0.60015470
#> item 15 0 0.62635137 0.01383779 0.03723270 0.62937883 0.62682255 0.04516506
#> item 16 0 0.02275655 0.38196755 0.10219768 0.40556681 0.11482351 0.48529266
#> item 17 0 0.03956205 0.22723400 0.07542647 0.26003826 0.09624686 0.26451346
#> item 18 0 0.06162062 0.02626763 0.27487781 0.08354494 0.30232155 0.28125742
#> item 19 0 0.00467149 0.41816175 0.02846726 0.42066985 0.03470523 0.42056814
#> item 20 0 0.04430078 0.11699546 0.03374550 0.14720593 0.06866455 0.12616238
#> 111
#> item 1 0.3191832
#> item 2 0.4436235
#> item 3 0.3148209
#> item 4 0.8579827
#> item 5 0.5277256
#> item 6 0.7750553
#> item 7 0.6607743
#> item 8 0.7414880
#> item 9 0.3642480
#> item 10 0.5341595
#> item 11 0.2981604
#> item 12 0.3760319
#> item 13 0.6802713
#> item 14 0.6039863
#> item 15 0.6355330
#> item 16 0.4890572
#> item 17 0.2884573
#> item 18 0.3129482
#> item 19 0.4238890
#> item 20 0.1640917
## calculate R2 after fitting CDM
CDM.obj <- CDM(data$dat, Q, model="GDINA")
#>
Iter = 1 Max. abs. change = 0.58226 Deviance = 10238.69
Iter = 2 Max. abs. change = 0.09628 Deviance = 8499.25
Iter = 3 Max. abs. change = 0.03447 Deviance = 8474.14
Iter = 4 Max. abs. change = 0.01418 Deviance = 8473.11
Iter = 5 Max. abs. change = 0.01087 Deviance = 8473.00
Iter = 6 Max. abs. change = 0.00870 Deviance = 8472.98
Iter = 7 Max. abs. change = 0.00678 Deviance = 8472.96
Iter = 8 Max. abs. change = 0.00521 Deviance = 8472.96
Iter = 9 Max. abs. change = 0.00398 Deviance = 8472.95
Iter = 10 Max. abs. change = 0.00303 Deviance = 8472.95
Iter = 11 Max. abs. change = 0.00230 Deviance = 8472.95
Iter = 12 Max. abs. change = 0.00174 Deviance = 8472.95
Iter = 13 Max. abs. change = 0.00131 Deviance = 8472.95
Iter = 14 Max. abs. change = 0.00099 Deviance = 8472.95
Iter = 15 Max. abs. change = 0.00075 Deviance = 8472.95
Iter = 16 Max. abs. change = 0.00056 Deviance = 8472.95
Iter = 17 Max. abs. change = 0.00042 Deviance = 8472.95
Iter = 18 Max. abs. change = 0.00032 Deviance = 8472.95
Iter = 19 Max. abs. change = 0.00024 Deviance = 8472.95
Iter = 20 Max. abs. change = 0.00018 Deviance = 8472.95
Iter = 21 Max. abs. change = 0.00014 Deviance = 8472.95
Iter = 22 Max. abs. change = 0.00010 Deviance = 8472.95
Iter = 23 Max. abs. change = 0.00008 Deviance = 8472.95
R2 <-get.R2(CDM.obj = CDM.obj)
print(R2)
#> 000 100 010 001 110 101 011
#> item 1 0 0.01687504 0.30950008 0.05616181 0.30992650 0.07231454 0.31772871
#> item 2 0 0.36107634 0.01544152 0.12313741 0.36298109 0.43391740 0.12771177
#> item 3 0 0.07112106 0.03612654 0.25918458 0.09295401 0.30354235 0.26418985
#> item 4 0 0.03774422 0.06736935 0.85351062 0.10422207 0.85679557 0.85433823
#> item 5 0 0.32838492 0.02760602 0.22970691 0.33880749 0.52236570 0.23018914
#> item 6 0 0.03537067 0.05523055 0.76782002 0.09479502 0.76806713 0.76971939
#> item 7 0 0.02592157 0.04171859 0.65304143 0.06780609 0.65505092 0.65581563
#> item 8 0 0.01929698 0.73409734 0.07241102 0.73300997 0.09614703 0.74105140
#> item 9 0 0.01807026 0.03587836 0.36332903 0.05231119 0.36402050 0.36359018
#> item 10 0 0.07302217 0.30029774 0.21522472 0.36795647 0.25182273 0.50334356
#> item 11 0 0.05293293 0.22654792 0.07114991 0.28797398 0.10312816 0.25840838
#> item 12 0 0.03458177 0.17622498 0.26878724 0.20421304 0.27532169 0.36946609
#> item 13 0 0.67881388 0.01856466 0.02692421 0.67960624 0.68129055 0.05298808
#> item 14 0 0.02753211 0.04269419 0.59776244 0.07172353 0.59890664 0.60015470
#> item 15 0 0.62635137 0.01383779 0.03723270 0.62937883 0.62682255 0.04516506
#> item 16 0 0.02275655 0.38196755 0.10219768 0.40556681 0.11482351 0.48529266
#> item 17 0 0.03956205 0.22723400 0.07542647 0.26003826 0.09624686 0.26451346
#> item 18 0 0.06162062 0.02626763 0.27487781 0.08354494 0.30232155 0.28125742
#> item 19 0 0.00467149 0.41816175 0.02846726 0.42066985 0.03470523 0.42056814
#> item 20 0 0.04430078 0.11699546 0.03374550 0.14720593 0.06866455 0.12616238
#> 111
#> item 1 0.3191832
#> item 2 0.4436235
#> item 3 0.3148209
#> item 4 0.8579827
#> item 5 0.5277256
#> item 6 0.7750553
#> item 7 0.6607743
#> item 8 0.7414880
#> item 9 0.3642480
#> item 10 0.5341595
#> item 11 0.2981604
#> item 12 0.3760319
#> item 13 0.6802713
#> item 14 0.6039863
#> item 15 0.6355330
#> item 16 0.4890572
#> item 17 0.2884573
#> item 18 0.3129482
#> item 19 0.4238890
#> item 20 0.1640917
# }