Ich bin gerade auf diese Arbeit gestoßen , in der beschrieben wird, wie die Wiederholbarkeit (auch bekannt als Zuverlässigkeit, auch bekannt als Intraclass-Korrelation) einer Messung über Mixed-Effects-Modellierung berechnet wird. Der R-Code wäre:

#fit the model
fit = lmer(dv~(1|unit),data=my_data)

#obtain the variance estimates
vc = VarCorr(fit)
residual_var = attr(vc,'sc')^2
intercept_var = attr(vc$id,'stddev')[1]^2

#compute the unadjusted repeatability
R = intercept_var/(intercept_var+residual_var)

#compute n0, the repeatability adjustment
n = as.data.frame(table(my_data$unit))
    k = nrow(n)
    N = sum(n$Freq)
n0 = (N-(sum(n$Freq^2)/N))/(k-1)

#compute the adjusted repeatability
Rn = R/(R+(1-R)/n0)

Ich glaube, dass dieser Ansatz auch verwendet werden kann, um die Zuverlässigkeit von Effekten (dh den Summenkontrasteffekt einer Variablen mit zwei Stufen) zu berechnen, wie in:

#make sure the effect variable has sum contrasts
contrasts(my_data$iv) = contr.sum

#fit the model
fit = lmer(dv~(iv|unit)+iv,data=my_data)

#obtain the variance estimates
vc = VarCorr(fit)
residual_var = attr(vc,'sc')^2
effect_var = attr(vc$id,'stddev')[2]^2

#compute the unadjusted repeatability
R = effect_var/(effect_var+residual_var)

#compute n0, the repeatability adjustment
n = as.data.frame(table(my_data$unit,my_data$iv))
k = nrow(n)
N = sum(n$Freq)
    n0 = (N-(sum(n$Freq^2)/N))/(k-1)

#compute the adjusted repeatability
Rn = R/(R+(1-R)/n0)

Drei Fragen:

Sind die obigen Berechnungen zum Erhalten der Punktschätzung der Wiederholbarkeit eines Effekts sinnvoll?
Wenn ich mehrere Variablen habe, deren Wiederholbarkeit ich schätzen möchte, lmer(dv~(iv1+iv2|unit)+iv1+iv2scheint das Hinzufügen aller Variablen zur gleichen Anpassung (z. B. ) höhere Wiederholbarkeitsschätzungen zu ergeben, als das Erstellen eines separaten Modells für jeden Effekt. Für mich ist dies rechnerisch sinnvoll, da durch die Einbeziehung mehrerer Effekte die Restvarianz tendenziell verringert wird. Ich bin jedoch nicht sicher, dass die resultierenden Wiederholbarkeitsschätzungen gültig sind. Sind sie?
Das oben zitierte Papier legt nahe, dass Likelihood-Profiling mir helfen könnte, Konfidenzintervalle für die Wiederholbarkeitsschätzungen zu erhalten. Soweit ich das beurteilen kann, werden jedoch confint(profile(fit))nur Intervalle für die Intercept- und Effect-Varianzen angegeben, während ich zusätzlich das Intervall für die Berechnung der Residualvarianz benötige das Intervall für die Wiederholbarkeit, nein?

mixed-model reliability intraclass-correlation repeatability spss factor-analysis survey modeling cross-validation error curve-fitting mediation correlation clustering sampling machine-learning probability classification metric r project-management optimization svm python dataset quality-control checking clustering distributions anova factor-analysis exponential poisson-distribution generalized-linear-model deviance machine-learning k-nearest-neighbour r hypothesis-testing t-test r variance levenes-test bayesian software bayesian-network regression repeated-measures least-squares change-scores variance chi-squared variance nonlinear-regression regression-coefficients multiple-comparisons p-value r statistical-significance excel sampling sample r distributions interpretation goodness-of-fit normality-assumption probability self-study distributions references theory time-series clustering econometrics binomial hypothesis-testing variance t-test paired-comparisons statistical-significance ab-test r references hypothesis-testing t-test normality-assumption wilcoxon-mann-whitney central-limit-theorem t-test data-visualization interactive-visualization goodness-of-fit

— Mike Lawrence
quelle

Ich denke, ich kann Ihre Fragen zumindest hinsichtlich der nicht angepassten Wiederholbarkeitsschätzungen, dh der klassischen klasseninternen Korrelationen (ICCs), beantworten. Was die "angepassten" Wiederholbarkeitsschätzungen anbelangt, habe ich das von Ihnen verknüpfte Papier überflogen und nicht wirklich gesehen, wo sich die Formel, die Sie anwenden, im Papier befindet? Basierend auf dem mathematischen Ausdruck scheint es sich um die Wiederholbarkeit von Mittelwerten zu handeln (und nicht um Einzelwerte). Aber es ist nicht klar, dass dies ein kritischer Teil Ihrer Frage ist, deshalb werde ich es ignorieren.

(1.) Sind die obigen Berechnungen zum Erhalten der Punktschätzung der Wiederholbarkeit eines Effekts sinnvoll?

Ja, der von Ihnen vorgeschlagene Ausdruck ist sinnvoll, aber eine geringfügige Änderung Ihrer vorgeschlagenen Formel ist erforderlich. Im Folgenden zeige ich, wie Sie Ihren vorgeschlagenen Wiederholungskoeffizienten ableiten können. Ich hoffe, dass dies sowohl die begriffliche Bedeutung des Koeffizienten verdeutlicht als auch zeigt, warum es wünschenswert wäre, ihn geringfügig zu ändern.

Zunächst nehmen wir den Wiederholungskoeffizienten in Ihrem ersten Fall und klären, was er bedeutet und woher er kommt. Das Verständnis dieses Sachverhalts hilft uns, den komplizierteren zweiten Fall zu verstehen.

Nur zufällige Abschnitte

$i$ $j$

y_{i j} = β_{0} + u_{0 j} + e_{i j},

$y_{ij} = \beta_0 + u_{0j} + e_{ij},$

u_{0 j}

$u_{0j}$

σ_{u_{0}}^{2}

$\sigma^2_{u_0}$

e_{i j}

$e_{ij}$

σ_{e}^{2}

$\sigma^2_e$

$x$ $y$

c o r r = \frac{c o v (x, y)}{\sqrt{v a r (x) v a r (y)}} .

$corr = \frac{cov(x, y)}{\sqrt{var(x)var(y)}}.$

The expression for ICC / repeatability coefficient then comes from letting the two random variables $x$ and $y$ be two observations drawn from the same $j$ group,

I C C = \frac{c o v (β_{0} + u_{0 j} + e_{i_{1} j}, β_{0} + u_{0 j} + e_{i_{2} j})}{\sqrt{v a r (β_{0} + u_{0 j} + e_{i_{1} j}) v a r (β_{0} + u_{0 j} + e_{i_{2} j})}},

$ICC = \frac{cov(\beta_0 + u_{0j} + e_{i_1j}, \beta_0 + u_{0j} + e_{i_2j})}{\sqrt{var(\beta_0 + u_{0j} + e_{i_1j})var(\beta_0 + u_{0j} + e_{i_2j})}},$ and if you simplify this using the definitions given above and the properties of variances/covariances (a process which I will not show here, unless you or others would prefer that I did), you end up with

I C C = \frac{σ_{u_{0}}^{2}}{σ_{u_{0}}^{2} + σ_{e}^{2}} .

$ICC = \frac{\sigma^2_{u_0}}{\sigma^2_{u_0} + \sigma^2_e}.$ What this means is that the ICC or "unadjusted repeatability coefficient" in this case has a simple interpretation as the expected correlation between a pair observations from the same cluster (net of the fixed effects, which in this case is just the grand mean). The fact that the ICC is also interpretable as a proportion of variance in this case is coincidental; that interpretation is not true in general for more complicated ICCs. The interpretation as some sort of correlation is what is primary.

Random intercepts and random slopes

Now for the second case, we have to first clarify what precisely is meant by "the reliability of effects (i.e. sum contrast effect of a variable with 2 levels)" -- your words.

First we lay out the model. The mixed model for the $i$ th response in the $j$ th group under the $k$ th level of a contrast-coded predictor $x$ is

y_{i j k} = β_{0} + β_{1} x_{k} + u_{0 j} + u_{1 j} x_{k} + e_{i j k},

$y_{ijk} = \beta_0 + \beta_1x_k + u_{0j} + u_{1j}x_k + e_{ijk},$ where the random intercepts have variance

σ_{u_{0}}^{2}

$\sigma^2_{u_0}$ , the random slopes have variance

σ_{u_{1}}^{2}

$\sigma^2_{u_1}$ , the random intercepts and slopes have covariance

σ_{u_{01}}

$\sigma_{u_{01}}$ , and the residuals

e_{i j}

$e_{ij}$ have variance

σ_{e}^{2}

$\sigma^2_e$ .

So what is the "repeatability of an effect" under this model? I think a good candidate definition is that it is the expected correlation between two pairs of difference scores computed within the same $j$ cluster, but across different pairs of observations $i$ .

So the pair of difference scores in question would be (remember that we assumed $x$ is contrast coded so that $|x_1|=|x_2|=x$ ):

y_{i_{1} j k_{2}} - y_{i_{1} j k_{1}} = (β_{0} - β_{0}) + β_{1} (x_{k_{2}} - x_{k_{1}}) + (u_{0 j} - u_{0 j}) + u_{1 j} (x_{k_{2}} - x_{k_{1}}) + (e_{i_{1} j k_{2}} - e_{i_{1} j k_{1}}) = 2 x β_{1} + 2 x u_{1 j} + e_{i_{1} j k_{2}} - e_{i_{1} j k_{1}}

$y_{i_1jk_2}-y_{i_1jk_1}=(\beta_0-\beta_0)+\beta_1(x_{k_2}-x_{k_1})+(u_{0j}-u_{0j})+u_{1j}(x_{k_2}-x_{k_1})+(e_{i_1jk_2}-e_{i_1jk_1}) \\=2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1}$ and

y_{i_{2} j k_{2}} - y_{i_{2} j k_{1}} = 2 x β_{1} + 2 x u_{1 j} + e_{i_{2} j k_{2}} - e_{i_{2} j k_{1}} .

$y_{i_2jk_2}-y_{i_2jk_1}=2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1}.$

Plugging these into the correlation formula gives us

I C C = \frac{c o v (2 x β_{1} + 2 x u_{1 j} + e_{i_{1} j k_{2}} - e_{i_{1} j k_{1}}, 2 x β_{1} + 2 x u_{1 j} + e_{i_{2} j k_{2}} - e_{i_{2} j k_{1}})}{\sqrt{v a r (2 x β_{1} + 2 x u_{1 j} + e_{i_{1} j k_{2}} - e_{i_{1} j k_{1}}) v a r (2 x β_{1} + 2 x u_{1 j} + e_{i_{2} j k_{2}} - e_{i_{2} j k_{1}})}},

$ICC = \frac{cov(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1}, 2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1})}{\sqrt{var(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1})var(2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1})}},$ which simplifies down to

I C C = \frac{2 x^{2} σ_{u_{1}}^{2}}{2 x^{2} σ_{u_{1}}^{2} + σ_{e}^{2}} .

$ICC = \frac{2x^2\sigma^2_{u_1}}{2x^2\sigma^2_{u_1} + \sigma^2_e}.$ Notice that the ICC is technically a function of

x

$x$ ! However, in this case

x

$x$ can only take 2 possible values, and the ICC is identical at both of these values.

As you can see, this is very similar to the repeatability coefficient that you proposed in your question, the only difference is that the random slope variance must be appropriately scaled if the expression is to be interpreted as an ICC or "unadjusted repeatability coefficient." The expression that you wrote works in the special case where the $x$ predictor is coded $\pm\frac{1}{\sqrt{2}}$ , but not in general.

(2.) When I have multiple variables whose repeatability I want to estimate, adding them all to the same fit (e.g. lmer(dv~(iv1+iv2|unit)+iv1+iv2) seems to yield higher repeatability estimates than creating a separate model for each effect. This makes sense computationally to me, as inclusion of multiple effects will tend to decrease the residual variance, but I'm not positive that the resulting repeatability estimates are valid. Are they?

I believe that working through a similar derivation as presented above for a model with multiple predictors with their own random slopes would show that the repeatability coefficient above would still be valid, except for the added complication that the difference scores we are conceptually interested in would now have a slightly different definition: namely, we are interested in the expected correlation of the differences between adjusted means after controlling for the other predictors in the model.

If the other predictors are orthogonal to the predictor of interest (as in, e.g., a balanced experiment), I would think the ICC / repeatability coefficient elaborated above should work without any modification. If they are not orthogonal then you would need to modify the formula to take account of this, which could get complicated, but hopefully my answer has given some hints about what that might look like.

— Jake Westfall
quelle

You are right Jake. The adjusted ICC referes to the section VII. EXTRAPOLATED REPEATABILITY AND HERITABILITY in the linked paper. The authors write It is important to distinguish between the repeatability of individual measurements $R$ and the repeatability of measurement means $R_n$ .

— Gabra

Berechnung der Wiederholbarkeit von Effekten aus einem früheren Modell

Nur zufällige Abschnitte

Random intercepts and random slopes