Probability & Significance
What is meant by Probability and Significance?
- Psychologists are interested in finding out if the results of their studies show real differences or correlations, or if the results are due to chance factors (e.g. the probability of the IV having affected the DV)
- To determine whether results are significant and not due to chance factors, researchers use a measure of the level of significance
- Researchers must decide how large an effect or relationship is required to conclude that the observed result is unlikely to be due to chance; this decision is reflected in the level of significance applied to the data
- The level of significance is expressed as a decimal value where 'p' stands for the probability that chance factors are responsible for the results.
- For most purposes in psychology, the 5% level of significance is appropriate which is expressed as p < 0.05 (i.e. the probability of chance factors producing the observed result is less than or equal to 5%)
- The research will then use statistical tables to find the critical value which will determine whether or not they can reject the null hypothesis
Inferential statistics enable us to draw inferences about the population whereas descriptive statistics can only tell us about the sample taken from that population
The Use of Statistical Tables
- Once the researcher has conducted a statistical test they have an observed value which is used to determine whether results are significant
- This observed value needs to be compared to the critical value in the statistical tables
- Each statistical test has its own critical values table i.e. the critical values table for the Mann-Whitney test is different to the table for the Wilcoxon test
- In order to find out whether or not the observed value is significant the researcher must ask the following questions which will help them to use the critical values table properly:
|
Is the test one-tailed or two-tailed i.e. does the alternative hypothesis predict the direction of difference or does it simply state that ‘there will be a difference’? |
|
What is the N value i.e. how many participants are in the sample? |
|
Which level of significance is being applied i.e. the standard is 0.05 |
Maguire’s (2000) research using London taxi drivers clearly gets the thumbs up for passing the p < 0.05 test
Exam Tip
Make sure that you are precise when writing an exam response on probability and significance. Students sometimes mistakenly report that the 0.5 level is used as the baseline standard for determining significance when of course it is the 0.05 level. 0.5 is a 50% probability of the observed results being due to chance which is of course far too high and would mean that the researcher could not claim a significant result
What are Type I and Type II Errors?
- A Type I Error occurs when the null hypothesis is rejected when it should have been accepted i.e. the researcher claims that the results are significant when in fact they are not (also known as a ‘false positive’)
- A Type I Error is more likely to happen when the researcher uses a probability value that is too high e.g. 0.1 rather than 0.05
- A Type II Error occurs when the null hypothesis is accepted when it should have been rejected i.e. the researcher claims that the results are not significant when in fact they are (also known as a ‘false negative’)
- A Type II Error is more likely to happen when the researcher uses a probability value that is too low e.g. 0.01 instead of 0.05
- Using a 0.05 significance level guards against making either a Type I or a Type II Error
Worked example
LEVEL: 3 MARKS
Dr Stats has conducted a study which looks at whether or not a diet of oranges and beef can improve IQ compared to a normal diet. After conducting the study Dr Stats concluded that oranges and beef do indeed increase IQ significantly using a significance level of 0.10.
With reference to the study outlined in the stem above explain why Dr Stats is likely to have made a Type I Error in his conclusion and suggest how he might rectify this. [3]
AO2 = 3 marks
For full marks the answer should include all of the following:
A Type I Error is likely to have occurred because the significance level of 0.10 has been set too high. Dr Stats could rectify the error by setting his probability level at 0.05 which means that the probability of chance factors affecting the result is 5% or less.
