I’ve been meaning to get around to posting about Stuart Hurlbert and Cecilia Lombardi’s recent paper (2009; Ann. Zool. Fennici 46: 311–349) on the use of p-values in drawing scientific conclusions… but thankfully Jarrett Byrnes over at i’m a chordata! urochordata! wrote such a great post about it that all I need to do is point you over to his place. Just so you know what you’re getting into, Hurlbert & Lombardi provide a convincing argument against the sanctity of the canonical alpha value of 0.05 and against the use of alpha values and ‘statistically significant’ in general. Instead they recommend (quoting Jarrett):

1) Report a p-value for a test. 2) Do not assign it significance, but rather refer to the level of support it gives for rejecting a null – strong, weak, moderate, practically non-existent. Make sure this statement of support is grounded in the design and power of the experiment. Suspend judgement on rejecting a null if the p value is high, as p-value testing is NOT the same as giving evidence FOR a null (something so many of us forget). 3) Use this in accumulation with other lines of evidence to draw a conclusion about a research hypothesis.

Go check out the full post. It’s well worth the read.