hank_denver said:

The boring version of this is the one that works, and the boring version is: measure a baseline, change one variable, wait, measure again under the…

Filing a mild objection. Mild because I might be wrong; an objection because nobody has addressed the case that does not fit. Filing an objection on the n=1 problem. Personal experience is genuine evidence about one person and very weak evidence about anybody else, and this thread keeps promoting the first into the second.

hank_denver said:

The boring version of this is the one that works, and the boring version is: measure a baseline, change one variable, wait, measure again under the…

That holds where the measurement is reliable. Where it is noisy — and a lot of what gets tracked here is noisy — the same reasoning produces confident nonsense.

hank_denver said:

The boring version of this is the one that works, and the boring version is: measure a baseline, change one variable, wait, measure again under the…

This is exactly what I could not find anywhere else. Sending this to two other people who asked me the same thing last week.

From the other side of the consultation, briefly. The mechanism and the magnitude are separate questions. Agreeing that something happens says nothing about whether it happens enough to act on.

Closing the loop on my own question.

For the archive: it took about six weeks rather than the two I expected, and the difference between those two numbers was the whole of my problem.