In class last week, we talked about the scientific method and how one might put together an experiment to ask a simple question. Students selected one of a group of about a dozen superstitions and imagined that no one ha ever done any experimentation to determine whether these superstitions were true or not.

The exercise mimicked to things that scientists have to do in the real world. One is writing (and reviewing) grants. Because much of what scientists study requires a significant amount of resources to perform, it is common to write a grant for money to do the work. A fairly efficient system has been worked out to administrate how these awards are made where grants are submitted to an agency, where other scientists evaluate the proposition and rank them with a score that politicians use to determine a cutoff for funding (the F word!)

This type of review is fairly similar to the way that publications are evaluated to determine what merits (or does not merit) publication. In the case of publication reviews, reviewers write up their responses which can be returned to the scientists seeking to publish. These comments can be used to improve the publication prior to acceptance (unfortunately, for the applicant, this step does not occur in grant application).

In order to combine these two experiences, my students wrote up their proposals, submitted them to other students for review, and then rewrote their work in the light of these suggestions. One thing I brought up, but did not answer in any way was sample size determination. (I was relieved when no one asked for clarification about this in class because I wanted them to think about it themselves). The real answer to this question is more within the realm of statistics than science, so with respect to my class, I don’t want to actually answer it (phew!) so much as point to places where more info can be found. One good page on this topic can be found at  the concept stew website or at wikipedia.

New penny. Doesn’t it look fake? maybe we should test it to make sure it’s reliable.

Perhaps students might also think about these questions:

1. How many coin flips are required to determine fairness?

           good one! would you believe that you need about 10,000 tosses to get an answer with reasonable reliability?

2. How much soup do you need to sample in order to know whether it needs salt?

3. How many subjects should be examined to determine if a new drug is safe?

Just something to think about. I have a love/hate relationship with statistics myself. I love it, but I also hate how deep you have to go in order to get a good answer. That’s life, I guess.