Patri's First law of statistics

Patri Friedman writes in Catallarchy:

Correlation implies causation when convenient; otherwise, there are confounding factors.

Jonathan stole my words in a comment to Patri's post:

I am going to shamelessly steal that line sometime.

Commenter Joel writes:

If someone has an awesome and easy and effective method of testing causation, I’d sure like to hear about it.

Patri responds:

Joel - it’s called a "prospective study". Also known as a randomized trial.

But a randomized trial does not eliminate confounding. To make sense of the results you have to perform a correlation of sorts (call it analysis of variance, regression or whatever). You test whether two groups of randomly chosen objects that differ in the level of some factor (because you purposefully manipulate this factor) also differ in a response variable. The problem is that your manipulation of the focal factor may have the unwanted side effect of changing the value of a second factor. You can try to refine your method of manipulation, but you can never be sure that you have attained perfection. Also, the two groups of randomly allocated objects may randomly differ in the value of a second factor. You can make this very improbable by, among other things, having a very large number of objects; but you cannot make it impossible.

You can never infer causality with certainty.