Sample Survey Methods

This unit gives the designs for choosing a sample and the matching estimators, then the sampling distributions (z ,\ t ,\ \chi^{2}) used to make confidence statements. Every number below was recomputed by bca/tools/check_stats.py.

6.1 Types of sampling

Probability sampling: simple random, stratified random, systematic, cluster, multistage, PPS. Non-probability: convenience, judgement (purposive), quota, snowball: no valid sampling-error estimate.

Notation: population size N, sample size n ,\ f = \frac{n}{N}, population mean Ȳ and variance S² = Σ(Yᵢ − Ȳ)²/(N − 1), sample mean ȳ and variance s^{2}.

6.2 Simple random sampling (SRS)

Every possible sample of size n has the same chance. Drawn without replacement (SRSWOR) by lottery or random numbers (a unit cannot appear twice) or with replacement (SRSWR) (a unit may repeat). Number of possible samples: C (N ,\ n) for SRSWOR, N^{n} for SRSWR.

	SRSWOR	SRSWR
Estimate of mean	ȳ = \frac{\sum y}{n} (unbiased)	ȳ
Variance of ȳ	\frac{(1 - f) S^{2}}{n}	\frac{\sigma^{2}}{n} = \frac{(N - 1) S^{2}}{N n}
Estimated	\frac{(1 - f) s^{2}}{n}	\frac{s^{2}}{n}
Estimate of total Y	Ŷ = Nȳ	Nȳ
Variance of total	\frac{N^{2} (1 - f) s^{2}}{n}	\frac{N^{2} s^{2}}{n}

(1 - f) is the finite population correction; sampling without replacement is more precise.

6.3 Stratified random sampling

Divide the population into non-overlapping, homogeneous strata (by district, class, income band), then draw an SRS in each. Estimate: ȳ_st = Σ W_h ȳ_h, W_{h} = \frac{N_{h}}{N}.

Var(ȳ_st) = Σ W_h² (1 − f_h) s_h²/n_h. It is more precise than SRS when strata are internally similar and differ from one another, guarantees representation of subgroups, and allows separate estimates per stratum.

Allocation of n: proportional n_{h} = \frac{n N_{h}}{N}; equal; optimum/Neyman n_{h} \propto N_{h} S_{h} (larger or more variable strata get more; with different costs ∝ \frac{N_{h} S_{h}}{\sqrt{c_{h}}}).

6.4 Systematic sampling

Select every k-th unit after a random start: k = \frac{N}{n} (the sampling interval); choose a random start r from 1 \ldots k; the sample is r ,\ r + k ,\ r + 2 k ,\ \ldots.

Simple, spreads the sample evenly over an ordered list, no need for a full renumbering. Danger: if the list has a periodic pattern matching k (every 7th day of sales data) the sample is biased.

Equivalent to SRS if the list is in random order.

6.5 Cluster and multistage sampling

Cluster sampling: divide the population into clusters (villages, schools, wards), randomly select some clusters, and survey all units in them. Cheap and practical when no unit list exists and units are scattered.

Clusters are internally heterogeneous ideally; the higher the similarity within clusters (intra-cluster correlation) the less precise it is than SRS of the same size.

Multistage sampling samples in stages: districts → wards → households → persons; each stage uses SRS, systematic or PPS sampling.

It needs a list only for chosen units at each stage, and is the standard design for national surveys such as the Nepal Living Standards Survey and Demographic and Health Survey.

6.6 Probability proportional to size (PPS) sampling

When clusters differ in size, selecting each with probability proportional to its size removes the bias of giving small villages the same chance as big ones.

With p_{i} = \frac{M_{i}}{\sum M}, the unbiased Hansen–Hurwitz estimator for the total with n draws is Ŷ = \frac{\left( \frac{1}{n} \right) \sum y_{i}}{p_{i}}, and its variance is estimated from the spread of the \frac{y_{i}}{p_{i}} values.

6.7 Ratio and regression methods of estimation

If an auxiliary variable x correlated with y is known for the whole population (total X, mean \bar{X}), it improves precision. Ratio estimator: R̂ = ȳ/x̄; ȳ_R = R̂ \bar{X}, Ŷ_R = R̂ X.

It is better than ȳ when y is roughly proportional to x (the line through the origin).

Regression estimator: ȳ_lr = ȳ + b (\bar{X} - \bar{x}), with b the sample regression coefficient of y on x; best when y is linearly related to x (with any intercept), and never worse than the ratio estimator in large samples. Both are slightly biased but precise.

Ratio and regression methods also apply within strata (separate or combined estimators). Estimating the population total and its variance: Ŷ = Nȳ under SRS; Ŷ_st = Σ N_h ȳ_h under stratification with Var(Ŷ_st) = Σ N_h²(1 − f_h)s_h²/n_h.

6.8 Sampling distributions: z, t and χ²

The sampling distribution of a statistic is its distribution over all possible samples. The central limit theorem: for large n the distribution of \bar{x} is approximately normal with mean μ and standard error \frac{\sigma}{\sqrt{n}}, whatever the population shape.

Distribution	When used	Statistic
z (standard normal)	mean, σ known or n large (\geq 30); proportions	z = \frac{\bar{x} - \mu}{\frac{\sigma}{\sqrt{n}}}
Student's t	mean, σ unknown, small n; df = n - 1	t = \frac{\bar{x} - \mu}{\frac{s}{\sqrt{n}}}
\chi^{2} (chi-square)	tests of independence/goodness of fit; variance	\chi^{2} = \frac{\sum (O - E)^{2}}{E}; df depends on the test

The t distribution is bell-shaped but heavier-tailed than z and approaches z as df grows.

Critical values (5% level): z = 1.645 (one-tail), 1.96 (two-tail), 2.576 (1% two-tail); t two-tail 0.05: df 9 \to 2.262 ,\ 14 \to 2.145 ,\ 15 \to 2.131 ,\ 19 \to 2.093 ;\ \chi^{2} at 5%: df 1 \to 3.841 ,\ 2 \to 5.991 ,\ 3 \to 7.815 ,\ 5 \to 11.070.

(a) W. S. Gosset, 1908

(b) Gosset plaque, Guinness Storehouse, Dublin

(c) t (df = 3, red) versus normal (blue)

6.10 Eight sampling designs and tools, and what each is used for

1. Simple random sampling. Every unit has the same chance, like drawing lots, so the estimate is unbiased and easy to analyse. It suits a small, well-listed population such as 200 students of one batch. It is the baseline against which other designs are compared.
2. Stratified sampling. The population is split into alike groups (strata) such as Terai, hill and mountain, and each is sampled separately. This guarantees that every group is represented and often gives a smaller variance than simple random sampling. A national household survey in Nepal is naturally stratified by ecological region.
3. Systematic sampling. Pick a random start and then every k-th unit, such as every 20th voter on a list. It is quick, spreads the sample evenly and needs no random-number table. It fails if the list repeats with the same period as k, for example a weekly pattern with k = 7.
4. Cluster and multistage sampling. Whole villages, wards or schools are chosen at random and then units inside them are surveyed. It cuts travel cost enormously when no list of individuals exists. The price is a larger error, because people in one ward resemble each other.
5. PPS sampling. Big clusters are given a bigger chance of selection, in proportion to size. A village with 600 households is then six times more likely to be drawn than one with 100. The estimator divides each total by its selection probability to stay unbiased.
6. Ratio and regression estimation. When an easy-to-know variable such as farm area is correlated with the target, say production, it sharpens the estimate. The ratio estimator uses the known population total of the helper variable. It gave 31.25 instead of 30 in the worked example of Section 6.7.
7. The z and t distributions. They give confidence intervals and tests for a mean. Use z when σ is known or n is large, and t when σ is unknown and n is small, as Gosset showed. A software tester comparing the load times of 12 pages uses t.
8. The chi-square test. It compares observed counts with expected counts. A college can test whether language preference differs by gender, or whether a die or random-number generator is fair. The decision rule is the shaded tail of Figure 6.3.

Formula and fact sheet

• SRSWOR: Var(ȳ) = \frac{(1 - f) s^{2}}{n}; total Ŷ = Nȳ, Var = \frac{N^{2} (1 - f) s^{2}}{n}. SRSWR: \frac{s^{2}}{n}.
• Stratified: ȳ_st = ΣW_hȳ_h; Var = \frac{\sum W_{h}^{2} (1 - f_{h}) s_{h}^{2}}{n_{h}}; Neyman n_{h} \propto N_{h} S_{h}.
• Systematic: k = \frac{N}{n}, random start. PPS: Ŷ = \frac{\left( \frac{1}{n} \right) \sum y_{i}}{p_{i}}. Ratio: ȳ_R = (ȳ/x̄)X̄.
• SE of mean \frac{\sigma}{\sqrt{n}} ;\ z = 1.96 (95%); t with df n - 1 ;\ \chi^{2} = \frac{\sum (O - E)^{2}}{E}, df = (r - 1) (c - 1).

Chapter summary

• SRS is the baseline; stratification gains precision from homogeneous strata; systematic sampling is simple but pattern-sensitive; cluster and multistage designs cut cost; PPS handles unequal clusters.
• Ratio and regression estimators exploit an auxiliary variable.
• The CLT justifies z ;\ t replaces z for small n with unknown \sigma ;\ \chi^{2} tests counts.

Practice

1. N = 500, an SRSWOR of 50 gives ȳ = 40 and s = 6. Find the estimated variance of ȳ, the 95% CI, and the estimated total with its SE.
2. Two strata of sizes 400 and 600 have s = 5 and 10. Find the Neyman allocation for n = 100.
3. Select a systematic sample of 8 from N = 80 with random start 5.
4. Differentiate stratified and cluster sampling.
5. A sample ȳ = 25 with \bar{x} = 20, population mean \bar{X} = 22. Find the ratio estimate.
6. n = 25 ,\ \bar{x} = 100 ,\ s = 15: 95% CI (t₀.₀₂₅ with 24 df = 2.064).
7. Test independence for \begin{bmatrix} 40 & 10 \\ 20 & 30 \end{bmatrix} at 5%.
8. When is t used instead of z?

Answers and worked solutions

1. f = 0.1; Var = 0.9 \times \frac{36}{50} = 0.648, S E = 0.805 ;\ C I 40 \pm 1.96 \times 0.805 = (38.42, 41.58); total 500 \times 40 = 20,000, S E = 500 \times 0.805 = 402.5.

2. N_{1} S_{1} = 2,000; N_{2} S_{2} = 6,000: n_{1} = 100 × 2,000/8,000 = 25, n_{2} = 75.

3. k = 10; units 5, 15, 25, 35, 45, 55, 65, 75.

4. Stratified: sample within every homogeneous stratum, improves precision. Cluster: sample whole clusters, reduces cost, usually less precise.

5. R̂ = \frac{25}{20} = 1.25; estimate = 1.25 \times 22 = 27.5.

6. S E = \frac{15}{5} = 3 ;\ C I = 100 \pm 2.064 \times 3 = (93.81, 106.19).

7. Row totals 50, 50; column totals 60, 40; expected: 30 ,\ \frac{20}{30} ,\ 20. \chi^{2} = (10^{2} / 30 + 10^{2} / 20) \times 2 = (3.333 + 5) \times 2 = 16.67 > 3.841 → reject independence.

8. When σ is unknown and the sample is small (n < 30), the data being roughly normal.

Model answers: medium (5 marks) and long (10 marks) questions

Q1 (5 marks). A population has N = 800 students. A simple random sample without replacement of n = 40 gives mean marks ȳ = 65 and s = 10. Estimate the variance and standard error of the mean, a 95% confidence interval, and the population total with its standard error.

Introduction. Under SRSWOR the sample mean is an unbiased estimate of the population mean and its variance includes the finite population correction (1 - f), where f = \frac{n}{N}.

Sampling fraction. f = 40/800 = 0.05, so 1 - f = 0.95.

Variance and SE. Est. Var(ȳ) = \frac{(1 - f) s^{2}}{n} = 0.95 × 100/40 = 2.375, so SE(ȳ) = \sqrt{2.375} = 1.541.

Confidence interval. ȳ ± 1.96 S E = 65 \pm 1.96 \times 1.541 = 65 \pm 3.02, that is (61.98, 68.02).

Total. Ŷ = Nȳ = 800 \times 65 = 52,000. SE(Ŷ) = N × SE(ȳ) = 800 \times 1.541 = 1,232.9.

Interpretation and conclusion. We are 95% confident the true mean mark lies between about 62 and 68. Without the correction the SE would be 1.581, a little too large, so ignoring (1 - f) overstates the uncertainty.

The estimates are valid only if the 40 students were truly chosen at random from a complete list (Figure 6.1a).

Q2 (5 marks). A population has two strata: N_{1} = 600 with s_{1} = 4 and N_{2} = 400 with s_{2} = 12. Allocate a sample of n = 50 by (a) proportional and (b) Neyman allocation, and compare the variances.

Introduction. Stratified sampling draws a separate random sample from each stratum. The allocation decides how many units come from each.

\ast (a) Proportional allocation* n_{h} = \frac{n N_{h}}{N}. n_{1} = 50 × 600/1000 = 30 and n_{2} = 50 × 400/1000 = 20.

\ast (b) Neyman (optimum) allocation* n_{h} \propto N_{h} S_{h}. N_{1} S_{1} = 2,400 and N_{2} S_{2} = 4,800, total 7,200. n_{1} = 50 × 2400/7200 = 16.7 ≈ 17; n_{2} = 50 × 4800/7200 = 33.3 ≈ 33.

Variance of the stratified mean Var = \frac{\sum W_{h}^{2} (1 - f_{h}) s_{h}^{2}}{n_{h}} with W_{1} = 0.6 and W_{2} = 0.4.

Allocation	n_{1} ,\ n_{2}	Var(ȳ_st)	SE
Proportional	30, 20	1.2768	1.130
Neyman	17, 33	0.9698	0.985

Conclusion. Neyman allocation gives a 24% smaller variance because it samples more heavily from the highly variable stratum 2. It is the best design when the strata differ strongly in variability; proportional allocation is simpler and nearly as good when they do not.

Q3 (10 marks). Explain the z ,\ t and chi-square distributions with their uses. A sample of 9 response times has mean 52 ms and s = 6 ms. Test H_{0}: \mu = 48 ms against H_{1}: \mu \neq 48 ms at the 5% level and give a 95% confidence interval. Compare with a z-test.

Introduction. The sampling distribution of a statistic is its distribution over all possible samples.

By the central limit theorem the sample mean is approximately normal with standard error \frac{\sigma}{\sqrt{n}} for large n (Figure 6.2). Three families are used for inference.

The three distributions (Table).

Distribution	Use	Statistic	Shape
z (standard normal)	mean with known σ or n \geq 30; proportions	z = \frac{\bar{x} - \mu}{\frac{\sigma}{\sqrt{n}}}	symmetric bell
Student's t	mean, σ unknown, small n; df = n - 1	t = \frac{\bar{x} - \mu}{\frac{s}{\sqrt{n}}}	symmetric, heavier tails
\chi^{2}	goodness of fit, independence, variance	\chi^{2} = \frac{\sum (O - E)^{2}}{E}	skewed right, df-dependent

Key points. The t curve approaches the normal as df grows (Plate 6.1 ) ;\ \chi^{2} is never negative and its decision tail is on the right (Figure 6.3).

Hypothesis test (t-test).

1. Hypotheses: H_{0}: \mu = 48 ;\ H_{1}: \mu \neq 48 (two-tailed).
2. Statistic: S E = \frac{6}{\sqrt{9}} = 2. t = \frac{52 - 48}{2} = 2.0, with df = 8.
3. Critical value: t₀.₀₂₅ with 8 df = 2.306.
4. Decision: |2.0| < 2.306, so we do not reject H_{0}. The two-tailed p-value is 0.0805, greater than 0.05.

Confidence interval. 52 \pm 2.306 \times 2 = 52 \pm 4.61, that is (47.39, 56.61). The value 48 lies inside, which agrees with the test.

*Comparison with z . \ast If σ were wrongly taken as 6 (known), z = 2.0 and the critical value is 1.96, so the same data would reject H_{0} (p = 0.0455).

The z-test is too eager with only 9 observations because it ignores the extra uncertainty of estimating σ; that is exactly why Gosset invented the t distribution.

Conclusion. With a small sample and unknown σ, use t. Here the evidence (52 ms against 48 ms) is suggestive but not statistically significant at the 5% level; a larger sample would be needed to settle it.