Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Survey sampling
open-in-new
<h2 id="probability-sampling">Probability sampling</h2> <p>In a probability sample (also called "scientific" or "random" sample) each member of the target population has a known and non-zero probability of inclusion in the sample.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> A survey based on a probability sample can in theory produce statistical measurements of the target population that are <a href="/facts/Unbiased/geuh5KrE">unbiased</a>, because the expected value of the sample mean is equal to the population mean, E(ȳ)=μ, or have a measurable sampling error, which can be expressed as a <a href="/facts/Confidence_interval/NS8mG5UE">confidence interval</a> or <a href="/facts/Margin_of_error/BtnDQ0Iu">margin of error</a>.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a><a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> </p><p>A probability-based survey sample is created by constructing a list of the target population, called the <a href="/facts/Sampling_frame/LhrXV2FS">sampling frame</a>, a randomized process for selecting units from the sample frame, called a selection procedure, and a method of contacting selected units to enable them to complete the survey, called a data collection method or mode.<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> For some target populations this process may be easy; for example, sampling the employees of a company by using payroll lists. However, in large, disorganized populations simply constructing a suitable sample frame is often a complex and expensive task. </p><p>Common methods of conducting a probability sample of the household population in the United States are Area Probability Sampling, Random Digit Dial telephone sampling, and more recently, Address-Based Sampling.<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> </p><p>Within probability sampling, there are specialized techniques such as <a href="/facts/Stratified_sampling/CvAUCegT">stratified sampling</a> and <a href="/facts/Cluster_sampling/2PX8m0fd">cluster sampling</a> that improve the precision or efficiency of the sampling process without altering the fundamental principles of probability sampling. </p><p>Stratification is the process of dividing members of the population into homogeneous subgroups before sampling, based on auxiliary information about each sample unit. The strata should be mutually exclusive: every element in the population must be assigned to only one stratum. The strata should also be collectively exhaustive: no population element can be excluded. Then methods such as <a href="/facts/Simple_random_sample/vDnCHnNx">simple random sampling</a> or <a href="/facts/Systematic_sampling/KQhBEuXI">systematic sampling</a> can be applied within each stratum. Stratification often improves the representativeness of the sample by reducing sampling error. </p> <h2 id="bias-in-probability-sampling">Bias in probability sampling</h2> <p class="note">Main article: <a href="/facts/Sampling_bias/nrr4qR2T">Sampling bias</a></p> <p>Bias in surveys is undesirable, but often unavoidable. The major types of bias that may occur in the sampling process are: </p> <ul><li><a href="/facts/Non-response_bias/y9G9e8dl">Non-response bias</a>: When individuals or households selected in the survey sample cannot or will not complete the survey there is the potential for bias to result from this non-response. Nonresponse bias occurs when the observed value deviates from the population parameter due to differences between respondents and nonrespondents.<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a></li> <li><a href="/facts/Response_bias/KNaQzWlL">Response bias</a>: This is not the opposite of non-response bias, but instead relates to a possible tendency of respondents to give inaccurate or untruthful answers for various reasons.</li> <li>Selection Bias: Selection bias occurs when some units have a differing probability of selection that is unaccounted for by the researcher. For example, some households have multiple phone numbers making them more likely to be selected in a telephone survey than households with only one phone number. This selection bias would be corrected by applying a survey weight equal to [1/(# of phone numbers)] to each household.</li> <li><a href="/facts/Self-selection_bias/wjGk6FqC">Self-selection bias</a>: A type of bias in which individuals voluntarily select themselves into a group, thereby potentially biasing the response of that group.</li> <li><a href="/facts/Participation_bias/y9G9e8dl">Participation bias</a>: Bias that arises due to the characteristics of those who choose to participate in a survey or poll.</li> <li>Coverage bias: Coverage bias can occur when population members do not appear in the sample frame (undercoverage). Coverage bias occurs when the observed value deviates from the population parameter due to differences between covered and non-covered units. Telephone surveys suffer from a well known source of coverage bias because they cannot include households without telephones.</li></ul> <h2 id="non-probability-sampling">Non-probability sampling</h2> <p>Many surveys are not based on probability samples, but rather on finding a suitable collection of respondents to complete the survey. Some common examples of non-probability sampling are:<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a> </p> <ul><li>Judgement Samples: A researcher decides which population members to include in the sample based on his or her judgement. The researcher may provide some alternative justification for the representativeness of the sample. The underlying assumption is that the investigator will select units that are characteristic of the population. This method can be subjected to researcher's biases and perception.<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a></li> <li>Snowball Samples: Often used when a target population is rare. Members of the target population recruit other members of the population for the survey.</li> <li><a href="/facts/Quota_sampling/rcDcc2fr">Quota Samples</a>: The sample is designed to include a designated number of people with certain specified characteristics. For example, 100 coffee drinkers. This type of sampling is common in non-probability market research surveys.</li> <li><a href="/facts/Convenience_sample/zQWCuDJs">Convenience Samples</a>: The sample is composed of whatever persons can be most easily accessed to fill out the survey.</li></ul> <p>In non-probability samples the relationship between the target population and the survey sample is immeasurable and potential bias is unknowable. Sophisticated users of non-probability survey samples tend to view the survey as an experimental condition, rather than a tool for population measurement, and examine the results for internally consistent relationships. </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Sample_size_determination/sf8dAFzq">Sample size determination</a></li> <li><a href="/facts/Sampling_(statistics)/kIb01xdL">Sampling (statistics)</a></li> <li><a href="/facts/Total_survey_error/MvcSRpEP">Total survey error</a></li></ul> <h2 id="further-reading">Further reading</h2> <p>The textbook by Groves et alia provides an overview of survey methodology, including recent literature on questionnaire development (informed by <a href="/facts/Cognitive_psychology/GnzIgx2I">cognitive psychology</a>) : </p> <ul><li><a href="/facts/Robert_M._Groves/Mh0txHYX">Robert Groves</a>, et alia. <i>Survey methodology</i> (2010) Second edition of the (2004) first edition <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-471-48348-6.</li></ul> <p>The other books focus on the <a href="/facts/Statistical_theory/7s5ye1mp">statistical theory</a> of survey sampling and require some knowledge of basic statistics, as discussed in the following textbooks: </p> <ul><li><a href="/facts/David_S._Moore/eLchg4Qw">David S. Moore</a> and George P. McCabe (February 2005). "<i>Introduction to the practice of statistics</i>" (5th edition). W.H. Freeman & Company. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-7167-6282-X.</li> <li><a href="/facts/David_Freedman_(statistician)/9765ju56">Freedman, David</a>; Pisani, Robert; Purves, Roger (2007). <a href="https://web.archive.org/web/20080706153959/http://www2.wwnorton.com/college/titles/math/stat4/comment.htm"><i>Statistics</i></a> (4th ed.). <a href="/facts/New_York_City/YlyvFmVC">New York</a>: <a href="/facts/W._W._Norton_%26_Company/3cwCSubC">Norton</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-0-393-92972-0. Archived from <a href="http://www.wwnorton.com/college/titles/math/stat4/comment.htm">the original</a> on 2008-07-06.</li></ul> <p>The elementary book by Scheaffer et alia uses quadratic equations from high-school algebra: </p> <ul><li>Scheaffer, Richard L., William Mendenhal and R. Lyman Ott. <i>Elementary survey sampling</i>, Fifth Edition. Belmont: Duxbury Press, 1996.</li></ul> <p>More mathematical statistics is required for Lohr, for Särndal et alia, and for Cochran (classic): </p> <ul><li><a href="/facts/William_Gemmell_Cochran/XAUEqpw6">Cochran, William G.</a> (1977). <i>Sampling techniques</i> (Third ed.). Wiley. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-471-16240-X.</li> <li><a href="/facts/Sharon_Lohr/Q1Gkgq36">Lohr, Sharon L.</a> (1999). <i>Sampling: Design and analysis</i>. Duxbury. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-534-35361-4.</li> <li>Särndal, Carl-Erik; Swensson, Bengt; Wretman, Jan (1992). <i>Model assisted survey sampling</i>. Springer-Verlag. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-387-40620-4.</li></ul> <p>The historically important books by Deming and Kish remain valuable for insights for social scientists (particularly about the U.S. census and the <a href="/facts/University_of_Michigan_Institute_for_Social_Research/1vsooEtL">Institute for Social Research</a> at the <a href="/facts/University_of_Michigan/6e1Ea5xh">University of Michigan</a>): </p> <ul><li><a href="/facts/W._Edwards_Deming/ytaxg6sv">Deming, W. Edwards</a> (1966). <a href="https://archive.org/details/sometheoryofsamp00will"><i>Some Theory of Sampling</i></a>. <a href="/facts/Dover_Publications/RwjvlSFv">Dover Publications</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-486-64684-X. <a href="/facts/OCLC_(identifier)/Yqad8waq">OCLC</a> <a href="https://search.worldcat.org/oclc/166526">166526</a>.</li> <li><a href="/facts/Leslie_Kish/oy1EYwHC">Kish, Leslie</a> (1995) <i>Survey Sampling</i>, Wiley, <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-471-10949-5</li></ul> <h2 id="external-links">External links</h2> Wikimedia Commons has media related to Survey sampling. <ul><li><a href="https://cran.r-project.org/view=OfficialStatistics">CRAN Task View Survey Methodology</a></li> <li><a href="http://whatisasurvey.info">What is a Survey?</a> Booklet published by National Opinion Research Center and The American Statistical Association</li> <li><a href="http://www.osra.org/itlpj/bartlettkotrlikhiggins.pdf"><i>Journal of Information Technology Learning and Performance</i> article Organizational Research: Determining Sample Size in Survey Research</a></li> <li><a href="http://badanalysis.wordpress.com/research-guides/sampling/">Sample Design and Confidence Intervals</a></li> <li><a href="http://www.m-s-g.com/Web/genesys/resources.aspx">Survey Sampling Methods</a></li> <li><a href="https://web.archive.org/web/20161120041655/http://www.statcan.gc.ca/edu/power-pouvoir/ch13/nonprob/5214898-eng.htm">Non-probability sampling</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>"Non-Probability Sampling - AAPOR". www.aapor.org. Retrieved 2020-05-24. <a href="https://www.aapor.org/Education-Resources/Reports/Non-Probability-Sampling.aspx" target="_blank">https://www.aapor.org/Education-Resources/Reports/Non-Probability-Sampling.aspx</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Weisberg, Herbert F. (2005), The Total Survey Error Approach, University of Chicago Press: Chicago. p.231. <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>"Archived copy" (PDF). Office of Management and Budget. Retrieved 2009-06-17 – via National Archives. <a href="https://obamawhitehouse.archives.gov/omb/inforeg/statpolicy/standards_stat_surveys.pdf" target="_blank">https://obamawhitehouse.archives.gov/omb/inforeg/statpolicy/standards_stat_surveys.pdf</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Lohr. Brewer. Swedes <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Richard Valliant, Alan H. Dorfman, and Richard M. Royall (2000), Finite Population Sampling and Inference: A Prediction Approach, Wiley, New York, p. 19 <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Salant, Priscilla, I. Dillman, and A. Don. How to conduct your own survey. No. 300.723 S3. 1994. <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Kish, L. (1965), Survey Sampling, New York: Wiley. p. 20 <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Kish, L. (1965), Survey Sampling, New York: Wiley. p.59 <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>"Why Sampling Works - AAPOR". <a href="http://www.aapor.org/Education-Resources/For-Researchers/Poll-Survey-FAQ/Why-Sampling-Works.aspx" target="_blank">http://www.aapor.org/Education-Resources/For-Researchers/Poll-Survey-FAQ/Why-Sampling-Works.aspx</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>Groves et al., Survey Methodology, Wiley: New York. <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>Michael W. Link, Michael P. Battaglia, Martin R. Frankel, Larry Osborn, and Ali H. Mokdad, A Comparison of Address-Based Sampling (ABS) Versus Random-Digit Dialing (RDD) for General Population Surveys; Public Opinion Q, Spring 2008; 72: 6 - 27. <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>"Glossary - NCES Statistical Standards". nces.ed.gov. <a href="https://nces.ed.gov/StatProg/2002/glossary.asp" target="_blank">https://nces.ed.gov/StatProg/2002/glossary.asp</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>"Survey Sampling Methods". www.statpac.com. <a href="https://www.statpac.com/surveys/sampling.htm" target="_blank">https://www.statpac.com/surveys/sampling.htm</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>Government of Canada, Statistics Canada; Government of Canada, Statistics Canada (28 January 2009). "Learning resources: Statistics: Power from data! Non-probability sampling". www150.statcan.gc.ca. <a href="https://www150.statcan.gc.ca/n1/edu/power-pouvoir/ch13/nonprob/5214898-eng.htm" target="_blank">https://www150.statcan.gc.ca/n1/edu/power-pouvoir/ch13/nonprob/5214898-eng.htm</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> </ol>