2011University of the Witwatersrand, Johannesburg Institutional Repository on DSpace (University of the Witwatersrand, Johannesburg)Requires access

THE VALIDITY OF EXPLORATORY

Kevin Richard Maddern

Open publisher page 0 citations

Abstract

The purpose of this research was to determine the impact that ordinal data (Likert type responses) has on principle axis method exploratory factor analysis, and whether these results could be improved through first rescaling the ordinal data using the distribution fitting method proposed by Stacey (2005), or using analysis that ignores the inter scale distances by using Spearman-rank correlation analysis, prior to performing exploratory factor analysis. This research was a simulation study. It simulated the continuous attitudes/beliefs of respondents given a known (unobserved) factor structure, and then used these to create responses on an ordinal scale. The study compared the robustness of exploratory factor analysis using varimax rotation when applied; directly to the ordinal data, to the Spearman-rank correlation matrix of the same ordinal data, and to the transformed ordinal data using a distribution fitting algorithmic approach. Comparison was done between these methods and the known factors of the contrived ‘latent’ continuous population from which the ordinal data was derived. The research found that discretisation creates both a negative bias and an increase in the variance in the factor loading results. The amount of negative bias and variance increase both as the number of categories used in the response scale decrease, and as the underlying communalities in the factor structure decrease. Given various forms of scale distortion introduced during the discretisation process (skew, kurtosis, and bimodality), it was found that both the negative bias and the variance of the results will increase even further as the scale distortion increases. These scale distortions are akin to the various biases that a group of respondents may exhibit when completing an ordinal scale questionnaire. Skewed distortion was found to be the most destructive, followed by bimodal distortion. Although no improvement was found in the results when using normal distribution fitting algorithm prior to exploratory factor analysis, it was found that, in cases of skewed or bimodal distortion, performing exploratory factor analysis on the Spearman-rank correlation matrix of the iii ordinal data was superior to performing the factor analysis directly on the ordinal data. These results are relevant to many fields of research, including business, given the prolific use of ordinal scales to gather data due to their simplicity and ease of use, and given the need by researchers to simplify or further analyse the ordinal data using multivariate techniques intended for interval level data (such as exploratory factor analysis). It was found that researchers need to take care in the selection of the response scale for their analysis, since it is shown that the correct choice in category size is the most effective way to ensure that data loss due to discretisation error is minimised. Data loss was shown to constitute both a strengthening and a weakening in the observed factor loading when compared to the unobserved (intended) factor structure. The risk is therefore shown that a researcher may unknowingly discard good data, or include spurious data, through the incorrect use of the response scale. These results begin to guide the researcher to avoid these pitfalls.

About this research paper

What this paper is about

The purpose of this research was to determine the impact that ordinal data (Likert type responses) has on principle axis method exploratory factor analysis, and whether these results could be improved through first rescaling the ordinal data using the distribution fitting method proposed by Stacey (2005), or using analysis that ignores the inter scale distances by using Spearman-rank correlation analysis, prior to performing exploratory factor analysis. This research was a simulation study. It simulated the continuous attitudes/beliefs of respondents given a known (unobserved) factor structure, and then used these to create responses on an ordinal scale. The study compared the robustness of exploratory factor analysis using varimax rotation when applied; directly to the ordinal data, to the Spearman-rank correlation matrix of the same ordinal data, and to the transformed ordinal data using a distribution fitting algorithmic approach. Comparison was done between these methods and the known factors of the contrived ‘latent’ continuous population from which the ordinal data was derived. The research found that discretisation creates both a negative bias and an increase in the variance in the factor loading results. The amount of negative bias and variance increase both as the number of categories used in the response scale decrease, and as the underlying communalities in the factor structure decrease. Given various forms of scale distortion introduced during the discretisation process (skew, kurtosis, and bimodality), it was found that both the negative bias and the variance of the results will increase even further as the scale distortion increases. These scale distortions are akin to the various biases that a group of respondents may exhibit when completing an ordinal scale questionnaire. Skewed distortion was found to be the most destructive, followed by bimodal distortion. Although no improvement was found in the results when using normal distribution fitting algorithm prior to exploratory factor analysis, it was found that, in cases of skewed or bimodal distortion, performing exploratory factor analysis on the Spearman-rank correlation matrix of the iii ordinal data was superior to performing the factor analysis directly on the ordinal data. These results are relevant to many fields of research, including business, given the prolific use of ordinal scales to gather data due to their simplicity and ease of use, and given the need by researchers to simplify or further analyse the ordinal data using multivariate techniques intended for interval level data (such as exploratory factor analysis). It was found that researchers need to take care in the selection of the response scale for their analysis, since it is shown that the correct choice in category size is the most effective way to ensure that data loss due to discretisation error is minimised. Data loss was shown to constitute both a strengthening and a weakening in the observed factor loading when compared to the unobserved (intended) factor structure. The risk is therefore shown that a researcher may unknowingly discard good data, or include spurious data, through the incorrect use of the response scale. These results begin to guide the researcher to avoid these pitfalls.

Why it matters

A significance statement is not available in the OpenAlex record.

Key contribution

A contribution statement is not available in the OpenAlex record.

Method / approach

Method details are not available in the OpenAlex metadata.

Main findings

Findings are not separately available in the OpenAlex metadata.

Limitations

Limitations are not available in the OpenAlex metadata.

Applications

Application details are not available in the OpenAlex metadata.

Available abstract

The purpose of this research was to determine the impact that ordinal data (Likert type responses) has on principle axis method exploratory factor analysis, and whether these results could be improved through first rescaling the ordinal data using the distribution fitting method proposed by Stacey (2005), or using analysis that ignores the inter scale distances by using Spearman-rank correlation analysis, prior to performing exploratory factor analysis. This research was a simulation study. It simulated the continuous attitudes/beliefs of respondents given a known (unobserved) factor structure, and then used these to create responses on an ordinal scale. The study compared the robustness of exploratory factor analysis using varimax rotation when applied; directly to the ordinal data, to the Spearman-rank correlation matrix of the same ordinal data, and to the transformed ordinal data using a distribution fitting algorithmic approach. Comparison was done between these methods and the known factors of the contrived ‘latent’ continuous population from which the ordinal data was derived. The research found that discretisation creates both a negative bias and an increase in the variance in the factor loading results. The amount of negative bias and variance increase both as the number of categories used in the response scale decrease, and as the underlying communalities in the factor structure decrease. Given various forms of scale distortion introduced during the discretisation process (skew, kurtosis, and bimodality), it was found that both the negative bias and the variance of the results will increase even further as the scale distortion increases. These scale distortions are akin to the various biases that a group of respondents may exhibit when completing an ordinal scale questionnaire. Skewed distortion was found to be the most destructive, followed by bimodal distortion. Although no improvement was found in the results when using normal distribution fitting algorithm prior to exploratory factor analysis, it was found that, in cases of skewed or bimodal distortion, performing exploratory factor analysis on the Spearman-rank correlation matrix of the iii ordinal data was superior to performing the factor analysis directly on the ordinal data. These results are relevant to many fields of research, including business, given the prolific use of ordinal scales to gather data due to their simplicity and ease of use, and given the need by researchers to simplify or further analyse the ordinal data using multivariate techniques intended for interval level data (such as exploratory factor analysis). It was found that researchers need to take care in the selection of the response scale for their analysis, since it is shown that the correct choice in category size is the most effective way to ensure that data loss due to discretisation error is minimised. Data loss was shown to constitute both a strengthening and a weakening in the observed factor loading when compared to the unobserved (intended) factor structure. The risk is therefore shown that a researcher may unknowingly discard good data, or include spurious data, through the incorrect use of the response scale. These results begin to guide the researcher to avoid these pitfalls.

Key concepts: Psychology

Related papers

Back to paper searchBrowse research topicsOriginal source
THE VALIDITY OF EXPLORATORY — Research Paper | ScholarLens