06. Chi-square analysis

Module items¶

R Script file code¶

[[Copy the code]] below ➜ Paste into [[RStudio console]] ➜ Hit Enter

source(url("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/data/0-packages-data.R")); 
(function(f="06-chisquare.R"){if(!file.exists(f)){download.file("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/modules/06.-chi-square-analysis/06-chisquare.R",f,mode="wb");file.edit(f)}else{download.file("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/modules/06.-chi-square-analysis/06-chisquare.R",gsub(".R","-original.R",f),mode="wb");file.edit(gsub(".R","-original.R",f))}})()

When this R script file opens in a new tab, [[Save R script file|save your previous R script file(s)]], and
- Close the previous tabs (R Script files), which you can find later in the [[Files tab]].

Assignment¶

Chi-square
- See: [[How to submit an assignment]]

Sample assignment¶

Sample: Chi-square

Reading¶

Barnes, Sally, and Cathy Lewin. 2006. “An Introduction to Inferential Statistics: Testing for Differences and Relationships.” Pp. 226–35 in Research methods in the social sciences, edited by B. Somekh and C. Lewin. London: Sage.

Learning outcomes¶

Define the purpose and function of chi-square analysis
Identify situations where a chi-square test is the appropriate statistical method
Differentiate between factor and outcome variables
Learn the p-value to determine the statistical significance of a relationship
Learn how to generate and interpret a chi-square analysis table

Chi-square basics¶

Relationship between preferred pet and respondents’ age group
- Do you think there is a relationship between preferred pet and respondents’ age group?
- In other words, do you think the preferred pet is influenced by respondents’ age group?
  - Age group Cat Dog Total
    
    Younger 207
    41.4% 293
    58.6% 500
    100%
    
    Older 267
    53.4% 233
    46.6% 500
    100%
    
    Total 474
    47.4% 526
    52.6% 1000
    100%
    - Show the answer
      
      While the table shows differences in preferences between age groups (e.g., younger people prefer dogs more than older people), we CANNOT conclude that the differences are statistically significant.
      
      To assess whether these differences are due to chance or a real association, we need a statistical test like the chi-square test.
The [[chi-square]] test is used to discover if there is a relationship between:
- Two [[categorical]] variables.
- The chi-square test can only compare categorical variables.
  - It cannot make comparisons between continuous variables or between categorical and continuous variables.
If we consider the "relationship between preferred pet and respondents’ age group" table, we need to ask the following question:
- In this table, what seems to change what?
  - Someone's preferred pet will not change their age group. However, the opposite could be the case: someone's age group may influence their preferred pet. Therefore:
    - Age group is the [[factor variable]],
    - Preferred pet is the [[outcome variable]].

Statistical significance and p-value¶

This, and all the analyses in the following modules, will use statistical significance, for example, if age group actually influence the preferred pet, meaning if those percentage differences are negligible or not. [[Statistical significance]]:
- A measure of whether the research findings are meaningful. In other words, if the factor variable causes a change in the outcome variable in a statistically significant way. We will determine this using [[p-value]]:
  - A p-value is a measure of the probability that an observed difference could have occurred just by random chance.
  - The lower the p-value, the greater the statistical significance of the observed difference.
    - All p-values are between 0 and 1;
      - The most reliable studies have p-values very close to 0.
      - A p-value of 0.001 means that there is a 1 in 1000 probability that the results are due to chance and do not reflect a real difference.
      - A p-value of 0.05 means there is a 5 in 100 probability that the results are due to chance.
      - When a p-value is 0.05 or below, the result is considered to be "statistically significant."
        
        We refer to statistical significance as p < 0.05.

How to check statistical significance using the [[p-value]]?¶

[[Is my p-value less than 0.05?]]
- To determine [[statistical significance]]
  - We can use this website: www.whichnumberislarger.com.
    - Enter your p-value in the first box and 0.05 in the second box.
      - If your p-value is less than 0.05, the result is statistically significant.
      - If your p-value is greater than 0.05, the result is not statistically significant.
[[Check asterisks]]
- Another way to determine [[statistical significance]]:
  - Our tables and figures use asterisks (*) to indicate statistical significance.
  - If you see **at least one asterisk (*)**, the result is statistically significant at the 0.05 level.
    - The number of asterisks (*) indicates the significance level
      
      (1) * means p < 0.05
      
      (2) ** means p < 0.01
      
      (3) *** means p < 0.001

[[Chi-square]] specifics¶

When both [[factor variable]] and [[outcome variable]] are [[categorical]], we conduct chi-square.
- For example, we could wonder if income groups have a statistically significant effect on life satisfaction level:
  - Factor variable: Income groups [(1) Low, (2) Medium, (3)High],
  - Outcome variable: Life satisfaction level [(1)Not satisfied, (2) Moderately satisfied, (3) Satisfied],
    - If the p-value that chi-square test generates is less than 0.05, we will assume that someone's income group would determine their life satisfaction level.

GSS example 1: Significant p-value¶

The relationship between (degree and health)¶

We wonder if respondents' education degree have a statistically significant influence on their perceived personal health quality.

flowchart LR
    subgraph F["Factor variable (Categorical)"]
        A[Education degree<br/>0: Less than high school<br/> 1: High school<br/> 2: Associate/junior college<br/> 3: Bachelor's<br/> 4: Graduate]
    end

    subgraph O["Outcome variable (Categorical)"]
        B[Health quality<br/>1: Excellent<br/> 2: Very good<br/> 3: Good<br/> 4: Fair<br/> 5: Poor]
    end

    A ==>|May have an effect on| B

Find the variables in Variables in GSS page¶

We want to make sure that degree and health are categorical variables.

[[Search]] the variable names, degree and health, in Variables in GSS page.

Variable name	Variable label	Variable type	Question wording and response categories
`degree`	Respondents' education degree	Ordinal	Do you have less than high school, high school, associate/junior college, bachelor's, or graduate degree? (0: Less than high school; 1: High school; 2: Associate/junior college; 3: Bachelor's; 4: Graduate)
`health`	Perceived personal health quality	Ordinal ✅ RECODE	Would you say that in general your health is Excellent, Very good, Good, Fair, or Poor? (1: Excellent; 2: Very Good; 3: Good; 4: Fair; 5: Poor)

[[Chi-square]] #code¶

[[Model code]]

sjt.xtab(gss$factor_here, gss$outcome_here, show.row.prc = TRUE) 

[[Working code]]
- 1
  sjt.xtab(gss$degree, gss$health, show.row.prc = TRUE)
  - Line 1: We will put degree here ➜ factor_here and health here ➜ outcome_here.
    - [[Factor variable]] first, [[outcome variable]] second.
      - Now, [[find this working code in the R script file]], and
        
        [[Highlighting and running|Highlight and run]] the working code in this module's R script file to generate the output shown below, which will appear in the [[viewer tab]] of RStudio.

[[Chi-square]] #output¶

Perceived personal health quality by respondents' education degree

Respondents' education degree	Excellent	Very Good	Good	Fair	Total
Less than high school	39 11.1%	140 40%	134 38.3%	37 10.6%	350 100%
High school	242 13.4%	955 52.8%	520 28.7%	93 5.1%	1810 100%
Associate/junior college	55 15.3%	211 58.6%	81 22.5%	13 3.6%	360 100%
Bachelor's	188 21.9%	504 58.6%	148 17.2%	20 2.3%	860 100%
Graduate	139 24.1%	338 58.6%	91 15.8%	9 1.6%	577 100%
Total	663 16.8%	2148 54.3%	974 24.6%	172 4.3%	3957 100%

χ² = 201.239, df = 12, Cramer's V = 0.130, ==p=0.000==

[[Chi-square]] #interpretation significant (p < 0.05)¶

Significant (p < 0.05) chi-square interpretation sample

The respondents' education degree variable has an effect on perceived personal health quality since the p-value is less than 0.05.

We can conclude that respondents with less than high school, high school, associate/junior college, bachelor's, and graduate degree have substantially different perceived personal health quality.
Significant (p < 0.05) chi-square interpretation template

The [[variable label]] (of [[factor variable]]) variable has an effect on [[variable label]] (of [[outcome variable]]) since the [[p-value]] is less than 0.05.

We can conclude that [[value label]] 1 (of factor variable) and value label 2 (of factor variable)... have/are/feel... substantially different variable label (of outcome variable).
Interpretation explanation
- The [[p-value]] tells us whether the relationship between the two variables is statistically significant.
  - When the p-value is less than 0.05, we conclude that the groups have different values on the outcome variable.
  - When the p-value is greater than 0.05, we conclude that the groups have similar values on the outcome variable.
- (1) First sentence: After the variable label of the factor variable, we add the word "variable" in our interpretation:
  - "The respondents' education degree variable has an effect on perceived personal health quality..."
    - Since this is a significant chi-square result, we say the factor variable has an effect on the outcome variable.
- (2) Second sentence: We explain what the significant chi-square result means.
  - We list the value labels of the factor variable and compare them on the outcome variable.
    - For example: "respondents with less than high school, high school, associate/junior college, bachelor's, and graduate degree..."

GSS Example 2: Nonsignificant p-value¶

The relationship between (sex and happy)¶

We wonder if respondents' sex have a statistically significant influence on their perceived personal health quality.

flowchart LR
    subgraph F["Factor variable (Categorical)"]
        A[Sex<br/>1=Male<br/> 2: Female]
    end

    subgraph O["Outcome variable (Categorical)"]
        B[Happiness<br/>1=Very happy<br/> 2: Pretty happy<br/> 3: Not too happy]
    end

    A ==>|May have an effect on| B

Find the variables in Variables in GSS page¶

We want to make sure that sex and happy are categorical variables.

[[Search]] the variable names, sex and happy, in Variables in GSS page.

Variable name	Variable label	Variable type	Question wording and response categories
`sex`	Respondents' sex	Binary	What's your sex? (1: Male; 2: Female)
`happy`	Happiness level	Ordinal, RECODE	Would you say that you are very happy, pretty happy, or not too happy? (1: Very happy; 2: Pretty happy; 3: Not too happy)

[[Chi-square]] #code¶

[[Model code]]

sjt.xtab(gss$factor_here, gss$outcome_here, show.row.prc = TRUE) 

[[Working code]]
- 1
  sjt.xtab(gss$sex, gss$happy, show.row.prc = TRUE)
  - Line 1: We will put sex here ➜ factor_here and happy here ➜ outcome_here.
    - [[Factor variable]] first, [[outcome variable]] second.
      - Now, [[find this working code in the R script file]], and
        
        [[Highlighting and running|Highlight and run]] the working code in this module's R script file to generate the output shown below, which will appear in the [[viewer tab]] of RStudio.

[[Chi-square]] #output¶

Happiness level by respondents' sex

Respondents' sex	Very happy	Pretty happy	Not too happy	Total
Male	350 19.8%	1025 57.8%	397 22.4%	1772 100%
Female	441 20.5%	1264 58.7%	450 20.9%	2155 100%
Total	791 20.1%	2289 58.3%	847 21.6%	3927 100%

χ² = 1.399, df = 2, Cramer's V = 0.019, ==p=0.497==

[[Chi-square]] #interpretation nonsignificant (p > 0.05)¶

Nonsignificant (p > 0.05) chi-square interpretation sample

The respondents' sex variable has no effect on happiness level since the p-value is greater than 0.05.

We can conclude that males and females have similar happiness level.
Nonsignificant (p > 0.05) chi-square interpretation template

The [[variable label]] (of [[factor variable]]) variable has an no effect on [[variable label]] (of [[outcome variable]]) since the [[p-value]] is greater than 0.05.

We can conclude that [[value label]] 1 (of factor variable) and value label 2 (of factor variable)... have/are/feel... similar variable label (of outcome variable).
Interpretation explanation
- The [[p-value]] tells us whether the relationship between the two variables is statistically significant.
  - When the p-value is less than 0.05, we conclude that the groups have different values on the outcome variable.
  - When the p-value is greater than 0.05, we conclude that the groups have similar values on the outcome variable.
- (1) First sentence: After the variable label of the factor variable, we add the word "variable" in our interpretation:
  - "The respondents' sex variable has no effect on happiness level..."
    - Since this is a nonsignificant chi-square result, we say the factor variable has no effect on the outcome variable.
- (2) Second sentence: We explain what the nonsignificant chi-square result means.
  - We list the value labels of the factor variable and compare them on the outcome variable.
    - For example: "males and females..."

Age group	Cat	Dog	Total
Younger	207 41.4%	293 58.6%	500 100%
Older	267 53.4%	233 46.6%	500 100%
Total	474 47.4%	526 52.6%	1000 100%