New statistical method: Construct analysis!

We thought we’d share news about an interesting analysis method we recently used… Performing construct analysis is a method you can use when analyzing survey data. First you must build “constructs” from among your survey questions, or items. Each construct may refer directly back to an overarching goal of the program you are evaluating, such as “Improve school climate.” You simply select all the survey items used to measure improvements in school climate to create your construct. Next, to perform analysis on your constructs, the survey items within your construct must be “collapsed” or summed to create a new variable. The guiding principal behind this method is that, essentially, any one item can not tell you the whole story on complex topics or concepts. By using multiple items to assess your goal, you place less pressure on any single survey item to answer your research question. The Improve Group recently used this technique doing survey analysis for a client. This research project had a few characteristics that were especially conducive to this type of analysis: post/retrospective pre-test survey data; a survey asking girls about specific topics they had learned in the program; and items that specifically aligned with overall client goals. In our case, we created two groups for each construct: “After” and “Before.” We then performed t-tests to determine whether there was statistical significance in the difference between after and before for each construct as a whole. We had to keep in mind that the final difference scores (between “Before” and “After”) represent the mean of individual items. You may want to look at scores of individual items within a construct for consistency. If one or two items within a construct had much less or much greater change than the other items, it may skew the total mean score for the overall construct. We would like to thank James Riedel at GSUSA for asking us to perform this type of analysis. In addition, we would like to thank Sanford Weisberg of the University of Minnesota’s Department of Statistics and Robert Eichinger of Cornferry International and Loeminger Enterprises who kindly and graciously provided us with advice and assistance on this statistical project.