What is the Chi-square Test of Independence?
This statistical test examines whether there is an association between two categorical variables. It does this by comparing the frequencies of a variable in each group compared to the expected frequency.
Expected frequency = the frequency we would expect if there was NO relationship between the groups.
So basically, if the observed frequencies differ substantially from the expected frequencies, the chi-square statistic becomes large, which may indicate a statistically significant relationship.
We run this test to see if we can reject the null hypothesis.
The null hypothesis of the chi-square test of independence states that the two variables are independent ( there is no association between them in the population)
The null hypothesis of the chi-square test of independence states that the two variables are independent ( there is no association between them in the population).
For example, in my worked example I’ll be examining data about COVID lockdown and whether people participated in exercise prior and during lockdown.
So now we now what the purpose of the test is, how do we actually run and interpret the chi-square Test Of Independence in Stata?
How to run the test in Stata
Step 1: Check variables
The first step is always to check your variables.
To run the chi-square Test of Independence you need:
- 2 categorical variables (coded as numbers or frequencies)
For this example, I am comparing the categories of Gender and Exercise Prior To Lockdown. Essentially, I’m looking at whether there is a relationship between Gender and what groups of people were already exercising prior to the COVID 19 lockdown.
Use the describe command to get an overview of your variables.
You can check your variables in the data editor. I can see in my data that the categories of Gender have been coded numerically as 1 and 2. 1= Male 2= Female.
I can also see that exercise prior to lockdown has been recorded as ‘Yes/No’ responses. These have also been coded numerically as 1= Yes 2= No.
You don’t need to guess this. Each data set you get from class/ an exam will introduce the variables. If it’s your own data…you will know what the variables stand for. Because you will have created the labels!
Not sure which test you should be using? Download my simple flowchart here.
Then, check your variables for any abnormalities.
Use the tab var command in stata.
This step is super quick and easy. You’re just doing a fast scan for anything obviously wrong in your data. This could be a really small sample size. Missing data. Etc.
As you can see, my data looks normal.
2. Check the assumptions
The next thing you’ll want to check is the expected frequencies.
Why?
The general rule of thumb is that expected frequencies should be at least 5 in most cells. If many cells have expected counts below 5, results may not be reliable.
You can check this using the command:
tab var1 var2, chi2 expected
If you look at the key, it shows you where the data is in the table. I can see that the top values are the actual frequency. The expected frequencies are the numbers underneath in the cell.
Straight away, I can see that all my expected values are indeed greater than 5. Yay. We can run the test!
Step 3. Run the test!
To run the chi-square Test of Independence, use the command:
tab var1 var2, chi2
If you want to generate the effect size use, this command:
tab var1 var 2. chi2 V
The V here stands for: Cramer’s V
And there you have it! You’re done. Running the test at least…
Interpreting the results
So you’ve run the test…now what? What do all those numbers at the bottom mean?
You will see 3 values:
- The chi-square test statistic (χ2)
- The p value
- Cramer’s V
What each value tells you:
χ2/ chi-square test statistic
This number tells you how far your observed counts differ from the expected counts (if the variables were independent).
You can see my χ2= 0.0196
This is tiny. Basically, our results were pretty much what we would expect.
The 1 in the brackets just refers to the degrees of freedom.
P value
This value tells you whether your results were statistically significant…or pretty much the same as chance.
My p = .889
This means that if there were truly no relationship between the variables, there is an 88.9% probability of observing a chi-square value this large or larger due to random sampling variability/ chance.
Since:
p > .05 → not significant
So we fail to reject the null hypothesis.
Cramér’s V
This is the effect size. Effect sizes measure the strength of the association between variables. I have a more detailed explanation here.
But essentially, my V = .014
Tiny number. Pretty much no association.
Writing up your results (APA7th edition style)
Finally, it’s time to write up your results so you can share what you found.
If I was writing up my results I would say:
A Chi-square test of independence was conducted to examine the association between gender and exercise prior to lockdown. There was no significant association between the variables, χ²(1, N = 100) = 0.02, p = .889, V = .01. The effect size was negligible, indicating no meaningful relationship between gender and exercise behaviour prior to lockdown.
Need help writing up your results according to APA 7th edition formatting? I have a simple table with templates and guides.
That’s it! You now know how to do a chi-square Test of Independence in Stata. Go try it out for yourself.
