Module 5: Analyzing Qualitative and Quantitative Data

MODULE 5: ANALYZING QUALITATIVE AND QUANTITATIVE DATA

Module 5 describes techniques for analyzing qualitative and quantitative data collected for an evaluation. Qualitative data are often collected from depth interviews and focus groups conducted at site visits, while quantitative data are often analyzed using statistical techniques, like Student’s t-test (Newcomer and Conger, 2010). T-tests are used to compare the means of grouped data for continuous variables. Data can be grouped to represent characteristics of populations or samples. The central limit theorem provides the theoretical foundation to make statistical inferences from samples to populations, from which they were derived (Berman and Wang 2012). Paired samples t-tests are used to compare the performance by a group between two periods of a program, by administering a pretest and a posttest.

There are four t-test assumptions:

1. One variable is continuous, while another variable is dichotomous.
2. Both variable distributions have equivalent variances.
3. Both variable distributions are normally distributed.
4. The observations are independent (except for paired samples).

Levene’s test measures the equality of variances. The null hypothesis is that both variables have equal variances. To determine whether a variable is normally distributed, the Kolmogorov-Smirnov test is used for samples with more than 50 observations, while the Shapiro-Wilk test is used for samples with 50 observations or less. For departures from normality, a possible solution is to transform the variable of interest to alter the shape of its distribution. Mann-Whitney and Wilcoxon tests offer nonparametric alternatives for hypothesis testing whenever the four assumptions for conducting t-tests are violated. Robustness is the extent to which test results remain unaffected by departures from test assumptions. Harris, et al. (2014) applied the paired samples t-test in their evaluation of the “Your Life, Your Health!” (YLYH) program. Self-rated health (Stanford, 2014) was measured before and after the YLYH program.

References

Berman, E.M. and X. Wang. (2012). Essential Statistics for Public Managers and Policy Analysts, 3rd Ed. Thousand Oaks, CA: Sage Publications, Inc.

Harris, R.A., K.K. Frye, M.W. Estrade, and W.J. Hicks. (2013). “Your Life, Your Health! Chronic Disease Self-Management Education in Louisiana.” The International Journal of Health, Wellness and Society. Volume 3, Issue 2, pp. 1-12.

Newcomer, K.E. and D. Conger. (2010). “Using Statistics in Evaluation.” In Wholey, J.S., Harry P. Hatry, H.P., Newcomer, K.E., Eds. Handbook of Practical Program Evaluation, 3rd Ed. Hoboken, NJ: Jossey-Bass.

Stanford Patient Education Research Center. (2014). Self-Rated Health. Palo Alto, CA: Stanford University. http://patienteducation.stanford.edu/research/