Varying coefficient meta-analytic methods for alpha reliability.
- 2 December 2010
- journal article
- research article
- Published by American Psychological Association (APA) in Psychological Methods
- Vol. 15 (4), 368-385
- https://doi.org/10.1037/a0020142
Abstract
The conventional fixed-effects (FE) and random-effects (RE) confidence intervals that are used to assess the average alpha reliability across multiple studies have serious limitations. The FE method, which is based on a constant coefficient model, assumes equal reliability coefficients across studies and breaks down under minor violations of this assumption. The RE method, which is based on a random coefficient model, assumes that the selected studies are a random sample from a normally distributed superpopulation. The RE method performs poorly in typical meta-analytic applications where the studies have not been randomly sampled from a normally distributed superpopulation or have been randomly sampled from a nonnormal superpopulation. A new confidence interval for the average reliability coefficient of a specific measurement scale is based on a varying coefficient statistical model and is shown to perform well under realistic conditions of reliability heterogeneity and nonrandom sampling of studies. New methods are proposed for assessing reliability moderator effects. The proposed methods are especially useful in meta-analyses that involve a small number of carefully selected studies for the purpose of obtaining a more accurate reliability estimate or to detect factors that moderate the reliability of a scale.Keywords
This publication has 17 references indexed in Scilit:
- A different paradigm for the initial colonisation of SahulArchaeology in Oceania, 2020
- Empirical vs natural weighting in random effects meta‐analysisStatistics in Medicine, 2010
- Meta-analytic interval estimation for standardized and unstandardized mean differences.Psychological Methods, 2009
- Meta-analytic interval estimation for bivariate correlations.Psychological Methods, 2008
- Toward using confidence intervals to compare correlations.Psychological Methods, 2007
- Perceived public stigma and the willingness to seek counseling: The mediating roles of self-stigma and attitudes toward counseling.Journal of Counseling Psychology, 2007
- Measuring the self-stigma associated with seeking psychological help.Journal of Counseling Psychology, 2006
- Confidence intervals for the amount of heterogeneity in meta‐analysisStatistics in Medicine, 2006
- Meta-analysis of coefficient alpha.Psychological Methods, 2006
- Approximate confidence interval for standard deviation of nonnormal distributionsComputational Statistics & Data Analysis, 2004