LO: Interpret basic statistical concepts such as confidence intervals, p-values, hypothesis testing and relationship to clinical significance

Statistical inference

• Use of sample data to draw conclusions about the wider population (while accounting for uncertainty)
• Sampling variation (random error): natural variation in results that occurs because a study examines a sample (subset) of a population rather than the whole population
  

Confidence intervals

• Statistical theory used to derive an interval - the confidence interval (CI) which tells us about the precision/uncertainty around the estimate
• Narrow confidence interval - more precise/less uncertain
• Wider confidence intercal - less precise/more uncertain
• Provides a range of plausible values for the population parameter
• % confidence interval is a range of values around a particular statistic expected to contain the true population value of that statistic

Factors affecting confidence interval width

• Higher degree confidence interval - wider range
• Smaller sample - greater uncertainty so wider range
• Sample variation - greater outcome variation increases uncertainty

Null values

• Value that represents 'no effect' or 'no difference'
• Difference in means - 0
• Risk ratio or odds ratio - 1

Hypothesis testing

• Null hypothesis (H₀): no difference/effect in the population
• Alternate hypothesis (H₁): a difference/effect in the population
• Assume H₀ as a baseline
• How compatible are the observed data with H₀
• Quantify with a p-value

P-values

• P-value: probability of obtaining a result at least as extreme as that observed
• Probability - values between 0 and 1
• Smaller the p-value the less compatible the observed data aligns with the null hypothesis
• Absence of evidence $\ne$ evidence of absence
• Lack of statistically significant evidence does not (robustly) show there is no effect
• Hard to prove something does not impact negatively

Statistical significance

• For a pre-specified threshold ($\alpha$) a result is said to be 'statistically significant' if the p-value is smaller than threshold
• Pre-specification avoids arbitrary selection to get a significant result
• Threshold is often 0.05, no science behind this just vibes

• (1-$\alpha$) x 100% confidence interval and p-values intuitively relate to statistical significance

Clinical significance

• Statistical significance: is the result sufficiently incompatible with H₀
• Clinical significance: is the size of the effect large enough to make a meaningful difference to patients or clinical practice

Managing error rates

• Probability of Type I error set by $\alpha$
  • $\alpha$=0.05 - if H₀ true, 5% chance of incorrectly rejecting it
• More tests performed - greater chance of at least one false-positive result
• Probability of Type II error often denoted by $\beta$
  • Often talk about power = 1 - $\beta$
  • Higher power means a greater chance of detecting a true effect if there is one