101 Concepts for the Level I Exam

Concept 13: Selection of Sample Size & Sampling Biases

Increasing the sample size reduces the standard error and gives us narrower confidence intervals. However, while increasing sample size we must consider two things:

Cost involved: Compare the cost of getting more data to the potential benefits of increasing precision.
Risk of sampling from a different population: In the process of increasing sample size if we get data from a different population, then the accuracy will not improve.

Biases observed in sampling methods are:

Data-mining bias: Analyzing the same data repeatedly, till a pattern is identified. To avoid this bias test the pattern on out of sample data.
Sample selection bias: Excluding certain assets from the analysis due to unavailability of data. A type of sample selection bias is the survivorship bias, in which companies are excluded from analysis because they have gone out of business.
Look-ahead bias: Analyzing past data using information that became available now.
Time-period bias: If the selected time period is too short, the results may not be useful. If the time period is too long, then the results may not consider major structural changes in the economy.