Hypothesis Testing is a statistical tool, that allows the resemblance of two or more process attributes – mean, median, and standard deviation. It provides a method to determine differences.

  • An important part of a conclusion reached based on random sampling (statistical inference)
    • A hypothesis is a statement that we want to verify using data
      • is there a difference?
      • has there been a change?
  • Null and Alternative hypotheses are formed
    • Level of risk and confidence required
  • Experimentation is conducted and interpret the results
    • Fail to reject the null hypothesis
      • No change. No difference, or…
    • Reject the null hypothesis
      • There is a change, a difference

What might we “Test” for?

We will be testing for a Change or Difference in Process…

  • Central Tendency
    • Mean, Median, Mode
  • Variation
    • Variance, Standard Deviation
  • Proportion
    • % (ratio, proportion)
  • Frequency (of occurrence)
    • Distribution of count/frequency

Testing Protocol

  • Null” hypothesis (H0)
    • This is a statement validating the status quo – there will be no change of significance observed. Any differences detected are purely due to chance and not a change in the process.
    • Symbols:
      • = (equals)
      • < (not less than)
      • > (not greater than)
  • “Alternative” hypothesis (H1)
    • This is a statement that there will be a difference in statistical significance detected, there has been a change.
    • Symbols:
      • <> (doesn’t equal)
      • < (is less than)
      • > (is greater than)

The hypotheses are complementary to each other. If one is true, the other is not true, and vice versa.

If the p is low, the null must go!

When performing any statistical test, the outcome is based on sampling from the population, therefore there is room for an error. Most statistical tests are run with a 95% confidence level, indicating that there is a 5% chance of making an error.

The decision of whether to simply accept or reject the null hypothesis is predicated on the calculated p value. If the p value is a smaller amount than or adequate to a preassigned significance level (normally set at 5%), then we reject the null hypothesis and accept the alternative.

A p value is going to be calculated by the statistical software when running a hypothesis test.

Types of Hypothesis Testing

There are many different types of hypothesis tests, they can be divided into two main categories:

  • Parametric tests
    • Makes inferences about parameters like mean and variance
    • Based on assumptions of specific distributions (ex. “normal” or “t” distributions)
  • Non-parametric tests
    • Makes inferences about frequency distribution like median, distribution type
    • Usually include sign and rank tests (a type of “math” used)
    • Do not require assumptions of normality (but do have some assumptions… always check them!)

Depending on the type of data you have collected, as well as if it’s normal or non-normal, there are several hypothesis tests available for comparing process characteristics.

Hypothesis test decision tree

Leave a Reply