When we do research, we generally have some group of individuals in mind -- college students, animals in a forest, residents of a community, etc. The group that we study is our sample. In most cases we are interested in the larger group, of which our sample is a part. That larger group is called the population. A sample is a subset of a population.

Sample selection affects the extent to which one can generalize to a population. Sampling has to do with the credibility of research findings, particularly with regard to external validity. External validity is the degree to which we can generalize our research findings, extending them beyond our single study.

Two sources of error

Ideally, a sample should represent the population from which it is drawn. There are two problems that lead to non-representative, and therefore invalid samples, which in turn reduce the credibility of the research findings.

Sampling error (chance variation)

Chance refers to events beyond one's control. Chance variation is always present to some degree. Uncontrolled factors such as temperature, time of day, restlessness, and innumerable other factors will influence participants' responses and will also influence the degree to which a sample mirrors its population.

---

If we draw multiple samples from a population, we will discover that the samples are not identical. The good news is that much of the chance variation is random -- samples leaning in one direction on some characteristics will be balanced by others leaning in another direction. A difference among samples that is due to random variation (chance) is called sampling error. Sampling error cannot be eliminated, but it can be estimated by using statistics.

---

Sample bias (constant error) Sample bias is a source of error that cannot be estimated or fixed in any way by statistics. It is error that results from factors other than chance. It is termed constant error because it is not random. It is a bias in a particular direction. Running subjects in an experiment in the evening when they are more likely to be tired, than earlier in the day, introduces a bias that will affect the results in a non-random way. Similarly, systematic observation at only a single time of day is likely to introduce a bias.

Sometimes researchers correct for constant error by handicapping. For example, in the evening experiment situation, looking at the means of samples tested when more rested, and making an adjustment to the group that seems to be showing a constant error. However, this is a risky procedure, and it is better to design the study to either exclude or to measure factors that might bias the results.

Sampling techniques are extremely important in dealing with both of these sources of error. We eliminate sample bias by 1) good research design, and 2) drawing samples in an unbiased way, so that the only variation besides the variables we are manipulating is due to chance.

Continue to Types of samples