Abstract:

The ability to repeat the results obtained by statistical findings in academic studies is impossible if the universe, the method of sampling and the process of analysis are not clearly defined. In this study, the factor analysis included in the articles published between 2010 and 2017 in the four magazines recognized as prestigious in education research in Turkey, has been studied in terms of the repetitiveness of the results. A total of 275 articles have been analyzed. In 77.8% of the studies, the definition of the universe was not clear, in 50.9% the method of sampling was not specified or appropriate sampling was used, and in 76% the data lost was not mentioned. About half of the studies are critical in terms of model-data compliance. The research has not been identified whether the structure of the data (continuous/category) has not been taken into account. The study has shown that it is not possible to repeat the results.

Keywords:

Abstract:

In quantitative research, an attempt to reproduce previously reported results requires at least a transparent definition of the population, sampling method, and the analyses procedures used in the prior studies. Focusing on the articles published between 2010 and 2017 by the four prestigious educational research journals in Turkey, this study aimed to investigate the reproducibility of the factor analysis results from a theoretical perspective. A total of 275 articles were subject to descriptive content analysis. Results showed that 77.8% of the studies did not include an explicit definition of the population under interest, and in 50.9% of the studies, the sampling method was either not clear or reported to be convenience sampling. Moreover, information about the missing data or a missing data dealing technique was absent in the 76% of the articles. Approximately, half of the studies were found to have inadequate model fit. Furthermore, in almost all studies, it could not be determined whether the item types (i.e., levels of measurement scales) were taken into consideration during the analyses. In conclusion, the majority of the investigated factor analysis results were evaluated to be non-reproducible in practice.