Statistical testing

Statistical testing is a crucial topic in data analysis and research. Here's what trusted experts have shared about it:

1. Importance of Effect Sizes in Big Data: With large data sets, conventional tests of statistical significance might become meaningless as even trivial results can appear significant. Instead, focus on effect sizes and practical implications for more meaningful analysis 1.

2. Understanding Statistical Inference: Statistical tests help determine if the observed differences in data are meaningful or due to chance. For instance, in comparing two groups, a t test or chi-square test can help establish if differences are statistically significant enough to reject the null hypothesis 2 3.

   • 

     1

     Statistical Significance Debate

     00:13:11
   • 

     2

     Statistical Significance Explained

     00:43:23
   • 

     3

     Statistical Inference Explained

     00:50:58
   • 

     4

     Choosing the Right Statistical Test

     00:18:31
   • 

     5

     Understanding Statistical Significance

     00:01:38
   • 

     6

     Experimenting for Better Results

     00:09:42

3. Choosing the Right Statistical Test: The choice of statistical test depends on the data type and research question. For instance, when dealing with ordinal variables, non-parametric tests like the Wilcoxon-Mann-Whitney test can be appropriate instead of t-tests 4.

4. Limitations of Statistical Significance: Statistically significant results don't guarantee that an observed pattern isn't noise. Instead, they indicate the probability of observing such a pattern if there was no real effect. This distinction is crucial for accurate interpretation of research findings 5.

For a deeper dive into these concepts, you can explore the following clips from relevant episodes:

• Practicing and Communicating Data Science with Jeff Stanton on the Data Skeptic 1.
• on the Data Skeptic 6.
• Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths on the EconTalk 5.
• Episode 79: Basic Concepts in Statistics on the The Science of Everything Podcast 2 3.