The spring roll problem: things to keep in mind about hypothesis testing

My sister loves my spring rolls, and when I make spring rolls, I usually put vermicelli in. But there are two types of vermicelli in our house, one type is for making vermicelli soup, and the other type is for the spring rolls. A few times, we’re out of the type for spring rolls, so I use the one for making vermicelli soup as the ingredients for my spring rolls. Well, the one that I usually use for spring rolls is cheaper than the one for making vermicelli soup.

After hearing the story of the lady tasting tea, my sister started to boast that she could detect the type of vermicelli that I put into the spring rolls. She wants me to make spring rolls with expensive vermicelli only. I told her that’s just how she feels because I usually tell her if I make spring rolls with expensive vermicelli. She said no, she could really detect it and I could conduct hypothesis testing if I wanted to. I told her: “I know you’re lucky and most of the time you usually make correct guesses in multiple-choice questions or games as if you have a sixth sense. But in statistics, not being rejected doesn’t mean that is true. It would simply mean that there’s not enough evidence against . That’s one of the reasons why statisticians develop many test procedures for the same testing problem. If a test does not reject the null hypothesis, it may be rejected by another test! Also, the sample size and significance value may affect the test result as well. Because you’re very lucky, I need for the test. Also, experimenting with a small sample may not lead to rejection of , but a larger sample size may. So you need to try 30 spring rolls and pick out 15 rolls that have the expensive vermicelli for me. I’ll conduct hypothesis testing based on that data.”

She replied happily: “That’s okay! That means I can try more spring rolls and since my teeth print is on those rolls, I don’t think anybody will want them except me!”

So we had the bet and if I lose, I’ll only make spring rolls with expensive vermicelli and I’ll owe her 50 bowls of grilled meat-spring roll noodle and 20 lettuce rolls dishes:

She may think that she would have more spring rolls. But now I started to cheat using science. A commonly encountered assumption in hypothesis testing is that the observations are independent. Some tests are robust to this assumption, but some are not. If she eats continuously without drinking water between the taste of spring rolls, the feeling in her mouth may get messed up and she is more likely to pick at random instead. This probably will favor me.

One way to improve the independence between the observations is to get some water after each spring roll taste. Yet why should I tell her that?

The rest of the experiment is to be set up similarly to the lady-tasting tea problem. But I won’t tell you the result because I don’t want to shame my sister. (Honestly, really honestly…)

But anyway, just remember that:

– not being rejected doesn’t mean that is true.
– The sample size may affect the result of the test.
– If a test does not reject the null hypothesis, it can be rejected by another test.
– Some tests are robust to the independence assumption, but not all tests are.