+34 616 71 29 85 carsten@dataz4s.com

The p-value

The p-value stands for probability value. It expresses the probability of getting a statistic at least as extreme as the one we are calculating for. Say we have an assumed mean (µ) of 10 and a sample mean (x̄) of 12. The p-value will express the probability of getting at least 12 assuming that the mean is 10?

 

 

The p-value in hypothesis tests

The p-value is typically applied in hypothesis testing (also called significance testing). When conducting a hypothesis test, we pre-define a significance level (α). The p-value is now applied to determine whether to reject or fail to reject the null hypothesis.

Say, we set α = 0.05. Then:

  • if we get a p-value below 0.05, we will reject the null hypothesis.
  • if we get a p-value greater than or equal to 0.05, we will fail to reject the null hypothesis

So:

  • P-value < α => Reject H0
  • P-value α => Fail to reject H0

 

When the p-value < α

When the p-value is below α, we will state that there is evidence enough to support our alternative hypothesis, and we reject the null hypothesis. The values that return p-values below α are also determined as significant.

Let’s take the example from above with an assumed mean (µ) = 10 and a sample mean of (x̄) 12. Say this returns a p-value of 0.03. This means that there is 3% chance that we would get 12 or more assuming that the mean is 10.

Prior to the conducting of our hypothesis test we had set the significance level (α) to 0.05 and as 0.03 < 0.05, we will reject the null hypothesis. Thus, we conclude that there is sufficient evidence in order to prove our alternative hypothesis stating that the mean (µ) is greater than 10.

We can also say that 12 is significant with a p-value of 0.03, and we therefore reject the null hypothesis.

Had we set α = 0.01 in this case, we would have failed to reject as 0.03 > 0.01. The conclusion would have been that there is sufficient evidence in to support our alternative hypothesis test at α = 0.01.

 

 When the p-value ≥ α

When the p-value is greater than or equal to our significance level (α), we will state that there is no evidence to support our alternative hypothesis, and we reject the null hypothesis. The result is determined as not significant.

 

Visualizing the p-value

In this graph the p-value is visualized together with the critical x-value (typically a sample mean) and the critical z-value and the alpha: 

The p-value visualized

 

Debate on the p-value

There is an ongoing debate about the use of p-value and hypothesis testing for inferential statistics.

In hypothesis testing the significance level is a threshold that we set up for ourselves prior to the calculation of the test statistics.

Say that we are about to run a hypothesis test and that we set α = 0.05. We essentially say: “We will reject the null hypothesis for values that return p-values below 0.05”. But, if we get a p-value of 0.051, extremely close to the significance level of 0.05, we will failing to reject H0.

Had the p-value been just 0.002 less, we would have rejected H0. Such an extremely small difference, in this case of 0.002, lead to the opposite desiction. The two only decisions we can take are reject and fail to reject. And they, obviously, lead to opposite conclusions. We determine that a result is not significant, when it is only 0.002 from being “significant”.

The p-value debate

So, part of the debate on this lack of nuanced concluding in hypothesis tests. Professionals from scientific environment state the p-value should be the foundation for inference and not only on the hypotheses and the related significance level.

 

P-value in Excel

The =NORM.S.INV and the =T.INV functions return the critical values in the normal distribution and the Student’s t-distribution respectively:

The p-value and critical Z in Excel

The =NORM.S.DIST and the =T.DIST functions return the p-values in the normal distribution and the Student’s t-distribution respectively:

The p-value and critical t in Excel

 

Learning statistics

Some of my preferred learning materials on the p-value:

 

 

Carsten Grube

Carsten Grube

Freelance Data Analyst

0 Comments

Submit a Comment

+34 616 71 29 85

Call me

Spain: Ctra. 404, km 2, 29100 Coín, Malaga

...........

Denmark: c/o Musvitvej 4, 3660 Stenløse

Drop me a line

What are you working on just now? Can I help you, and can you help me? 

About me

Learning statistics. Doing statistics. Freelance since 2005. Dane. Living in Spain. With my Spanish wife and two children. 

What they say

20 years in sales, analysis, journalism and startups. See what my customers and partners say about me.