Chance & simulation

Bootstrap Confidence Interval Lab

The nonparametric bootstrap of one sample of 2 to 30 values, generated from a seeded Normal, Uniform or Exponential parent or placed by the visitor: up to 1,000 resamples with replacement from one seeded stream, the mean or the median, the type 7 percentile interval beside the basic interval and the t-interval, the bootstrap standard error and bias and the exact plug-in standard error, a 30-bin histogram with a normal reference, pick counts of the first five resamples, the width at n = 5, 10, 20 and 30 against a 1/sqrt(n) guide, the median's share and distinct medians, and five scored challenges.

Last updated

Calculator overview

Inputs and outputs

This summary comes from the calculator's published input and output contract.

Inputs

Normal parent standard deviation (required)
About this input

sigma of the Normal parent, from 0.1 to 1000.

Default 10 Range 0.1 to 1000
Uniform parent high end (required)
About this input

b, the high end of the Uniform parent, from -1000 to 1000 and above a.

Default 100 Range -1000 to 1000
Exponential parent mean (required)
About this input

The mean of the Exponential parent, from 0.1 to 1000.

Default 10 Range 0.1 to 1000
Normal parent mean (required)
About this input

mu of the Normal parent, from -1000 to 1000.

Default 50 Range -1000 to 1000
Resampling seed (required)
About this input

The seed of the resampling stream, a whole number from 1 to 2147483646. Changing only this seed shows the Monte Carlo error of the interval.

Default 4900 Range 1 to 2147483646
Sample size (required)
About this input

n for a generated sample, a whole number from 2 to 30; the sample is always the first n generated values. Hidden with My data, where n is the number of points placed.

Default 20 Range 2 to 30
Uniform parent low end (required)
About this input

a, the low end of the Uniform parent, from -1000 to 1000 and below b.

Default 0 Range -1000 to 1000
Number of resamples (required)
About this input

B, the number of resamples: 100, 200, 500 or 1000 resamples.

Default 1000 resamples Allowed 100 resamples, 200 resamples, 500 resamples, 1000 resamples
Your prediction
When omitted Blank
About this input

Your prediction in challenges 1 and 4. Challenge 1: how many times wider the interval is with 5 values than with 20, a ratio such as 2. Challenge 4: the bootstrap standard error of the mean, in the units of the data values. Leave it blank until you predict; the results stay hidden until then, and 0 counts as a prediction. A number from 0 to 1000.

Default Not set Range 0 to 1000
Your data points (row i is point i; leave a row blank when it is not used)
When omitted Blank
About this input

Your points (My data): row i is point i, a number from -1000 to 1000 or blank. Blank rows are skipped and the order of the rows does not matter; at least 2 points are needed. Placed and dragged on the canvas (snapped to 0.1). Optional: the shipped default uses a generated sample. Checked even when the source is Generated sample.

Default 0 rows
ColumnRange or allowed values
Value -1000 to 1000
Statistic (required)
About this input

The statistic each resample computes: Mean or Median.

Default Mean Allowed Mean, Median
Challenge (required)
About this input

Selects a challenge: 0 for none, 1 to 5 for a predefined challenge, which fixes the settings except what it leaves to you.

Default 0 Range 0 to 5
Confidence level (required)
About this input

The confidence level of every interval: 80, 90, 95 or 99 percent.

Default 95 percent Allowed 80 percent, 90 percent, 95 percent, 99 percent
Parent distribution (required)
About this input

The parent a generated sample is drawn from: Normal, Uniform or Exponential (the canvas tabs). Not used with My data.

Default Normal Allowed Normal, Uniform, Exponential
Data seed (required)
About this input

The seed of the data stream, a whole number from 1 to 2147483646: the same seed gives the same sample. Not used with My data.

Default 2027 Range 1 to 2147483646
Data source (required)
About this input

Generated sample draws the values from the parent; My data uses the points you place on the number line (the canvas chip).

Default Generated sample Allowed Generated sample, My data

Outputs

Percentile interval, low end
About this output

The low end of the percentile interval: the type 7 order statistic of the B resample statistics at (100 - L)/200. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Percentile interval, high end
About this output

The high end of the percentile interval. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
How many times each sorted point was picked in resamples 1 to 5
About this output

How many times each sorted point was picked in resamples 1 to 5 (each column adds up to n). Blank while challenge 4 awaits a prediction.

No unit declared
Resamples used
About this output

B, the number of resamples used. Blank when the inputs are not valid.

No unit declared
Exact plug-in standard error of the mean
About this output

The exact standard error of the bootstrap mean: the population-form standard deviation of the data (divisor n) over sqrt(n). Mean only. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Parent's true value
About this output

The parent's true mean (or, for the median of an Exponential parent, its true median). Generated data only.

No unit declared
Normal parent standard deviation used
About this output

sigma used, with a generated Normal sample only.

No unit declared
Normal parent mean used
About this output

mu used, with a generated Normal sample only.

No unit declared
Uniform parent high end used
About this output

b used, with a generated Uniform sample only.

No unit declared
Parent used
About this output

The parent the generated sample came from. Blank with My data or when the inputs are not valid.

No unit declared
Uniform parent low end used
About this output

a used, with a generated Uniform sample only.

No unit declared
Resampling seed used
About this output

The resampling seed used. Blank when the inputs are not valid.

No unit declared
t-interval, high end
About this output

The high end of the t-interval. Mean only. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
t critical value
About this output

The two-sided t critical value at the level with n - 1 degrees of freedom. Mean only. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
t-interval, low end
About this output

The low end of the classic t-interval: the sample mean minus t times s/sqrt(n). Mean only. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
The interval width at n = 5, 10, 20 and 30 (500 resamples each) against a 1/sqrt(n) guide
About this output

The percentile interval at n = 5, 10, 20 and 30 (the first n generated values, 500 resamples each, the same stream): its ends, width, the t-interval width, the width times sqrt(n) and the 1/sqrt(n) guide through their average. Generated data only; columns 2 to 7 are blank while challenge 1 or 4 awaits a prediction and while challenge 2 is not met.

No unit declared
t-interval width
About this output

The width of the t-interval. Mean only. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Sample statistic
About this output

The statistic of the sample itself: its mean or its median. Blank when the inputs are not valid.

No unit declared
Sample mean
About this output

The mean of the sample. Blank when the inputs are not valid.

No unit declared
The first 500 resamples: each resample's statistic and its histogram bin (1 to 30)
About this output

The first 500 resamples (or B when fewer): each resample's statistic and its histogram bin. The canvas drops them into the histogram. Blank while challenge 4 awaits a prediction.

No unit declared
Resamples with the sample's median (percent)
About this output

The percent of resamples whose median equals the sample's median exactly (median only). Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Sample size used
About this output

n, the number of data values used. Blank when the inputs are not valid.

No unit declared
Sample standard deviation
About this output

The standard deviation of the sample (divisor n - 1). Blank when the inputs are not valid.

No unit declared
Challenge feedback
About this output

The prompt while challenge 1 or 4 awaits a prediction, the met text when met, otherwise a hint; blank with no challenge.

No unit declared
Bootstrap standard error
About this output

The bootstrap standard error: the standard deviation of the B resample statistics (divisor B - 1). Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Challenge goal
About this output

The goal of the selected challenge; blank with no challenge.

No unit declared
Challenge score
About this output

100 when the selected challenge is met, else 0.

No unit declared
Challenge: the value scored
About this output

The value the selected challenge scores: the width ratio (challenge 1), the interval width (2), the sample statistic (3), the plug-in standard error (4) or the median share (5). Blank with no challenge, when the inputs are not valid, or while challenge 1 or 4 awaits a prediction.

No unit declared
Mean of the resample statistics
About this output

The mean of the B resample statistics. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Basic interval, low end
About this output

The low end of the basic (reverse percentile) interval: twice the sample statistic minus the percentile high end. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Basic interval, high end
About this output

The high end of the basic interval: twice the sample statistic minus the percentile low end. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
The 30-bin histogram of all the resample statistics, with a normal reference
About this output

The 30-bin histogram of all B resample statistics (the smallest and largest at the centres of bins 1 and 30), with the count a normal curve with the bootstrap mean and standard error would give. Blank while challenge 4 awaits a prediction.

No unit declared
Bootstrap bias estimate
About this output

The bootstrap bias estimate: the mean of the resample statistics minus the sample statistic. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Statistic used
About this output

The statistic the calculation used. Blank when the inputs are not valid.

No unit declared
Challenge result
About this output

No challenge selected, MET or NOT MET.

No unit declared
Model version
About this output

The model version.

No unit declared
Model Status
About this output

NOT VALID with the reason when an input cannot be used; CHECK when every data value is the same, when an interval end rests on the 5 most extreme resamples, or when the sample has fewer than 10 values; otherwise OK.

No unit declared
Formula module versions
About this output

The formula modules the model is built from, with their versions.

No unit declared
Interval contains the parent value
About this output

Yes when the percentile interval contains the parent value, else No. Generated data only. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Exponential parent mean used
About this output

The Exponential mean used, with a generated Exponential sample only.

No unit declared
Percentile interval width
About this output

The width of the percentile interval. Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
Data source used
About this output

The data source the calculation used (the challenge value when a challenge is selected). Blank when the inputs are not valid.

No unit declared
Data seed used
About this output

The data seed used. Blank with My data or when the inputs are not valid.

No unit declared
The sample: each row's value, the sorted values and each row's sorted position
About this output

The sample: each row's value, the sorted values 1 to n and each row's position in the sorted order (for the pick-count badge on a placed point).

No unit declared
Confidence level used (percent)
About this output

The confidence level used, in percent. Blank when the inputs are not valid.

No unit declared
Distinct medians
About this output

How many different medians occurred among the B resamples (median with an odd n only). Blank when the inputs are not valid, or while challenge 4 awaits a prediction.

No unit declared
This page is provided by LogicCommons for informational purposes only. Results are analysis outputs computed from the inputs you supply and are not professional advice. Verify all inputs and results independently.

LogicCommons is in beta. If a result, label, or reference looks wrong, tell us here; we read every message.

Methodology

Purpose and model boundary

This lab shows how the nonparametric bootstrap builds a confidence interval from one sample. The sample has 2 to 30 values. It is either drawn from a seeded Normal, Uniform or Exponential parent, or made of points you place on a number line. The model then draws 100, 200, 500 or 1,000 resamples, each one n values picked at random with replacement from the sample, and computes the chosen statistic, the mean or the median, of every resample. From those resample statistics it returns the percentile interval, the basic (reverse percentile) interval, the bootstrap standard error and bias, and for the mean the classic t-interval and the exact plug-in standard error. It also returns a 30-bin histogram of the resample statistics with a normal reference, how often each point was picked in the first five resamples, and, for a generated sample, the interval width at n = 5, 10, 20 and 30 against a 1 over the square root of n guide. Five scored challenges use the same model.

It is a lab for learning what a bootstrap interval is and how it behaves: why it narrows as the sample grows, why a skewed sample gives a lopsided interval, why the median's bootstrap is lumpy, and how the interval moves a little when only the resampling seed changes and a lot when the data change. It is not a statistics tool for real decisions. It offers no BCa, studentized or smoothed bootstrap, handles one sample of at most 30 values, and only the mean and the median.

The model is the calculation authority. The page sends the values you set to the model and draws only what it returns: the sample strip with its pick badges, the resample statistics dropping into the histogram, the normal reference curve, the interval bands, the parent line and the width-by-size chart with its guide. The page computes no resample, interval or standard error itself, and nothing on the page draws a random number.

Inputs and units

The data, every statistic and every interval end are in the units of the data; the lab has no physical unit. Counts are whole numbers and shares are percentages. Every input except the point list is required and is checked on its entered value, even when the chosen data source, parent or statistic does not use it.

Input Meaning and unit
Data source Generated sample (default) draws the values from the parent; My data uses the points you place.
Parent distribution Normal (default), Uniform or Exponential: the parent a generated sample is drawn from.
Normal parent mean μ, from -1,000 to 1,000 (default 50), in the units of the data.
Normal parent standard deviation σ, from 0.1 to 1,000 (default 10).
Uniform parent low end, high end a and b, each from -1,000 to 1,000, with a below b (defaults 0 and 100).
Exponential parent mean From 0.1 to 1,000 (default 10).
Sample size n for a generated sample, a whole number from 2 to 30 (default 20). The sample is always the first n generated values, so raising n extends the sample rather than replacing it.
Your data points My data: up to 30 points, each a number from -1,000 to 1,000 or blank. Blank rows are skipped, the order of the points does not matter, and at least 2 points are needed; n is the number of points. Optional: the default uses a generated sample. The points are checked even when the source is Generated sample.
Statistic Mean (default) or Median: what each resample computes.
Number of resamples B: 100, 200, 500 or 1,000 resamples (default 1,000).
Confidence level L: 80, 90, 95 (default) or 99 percent, used by every interval.
Data seed The start of the data stream, a whole number from 1 to 2,147,483,646 (default 2027). The same seed always gives the same sample. Not used with My data.
Resampling seed The start of the resampling stream, a whole number from 1 to 2,147,483,646 (default 4900). Changing only this seed shows the Monte Carlo error of the interval.
Challenge 0 for free exploration (default), or 1 to 5 to load a challenge.
Your prediction Challenges 1 and 4 only: blank (default) until you predict, then a number from 0 to 1,000. Challenge 1 asks for a ratio; challenge 4 for a standard error in the units of the data. An entered 0 counts as a prediction.

Where each input is set:

  • On the canvas only: the data source (the Generated sample and My data chip), the parent distribution (its tabs), and your data points. With My data you click the number line to add a point, drag a point along it (snapped to 0.1), or drag it off the line to remove it.
  • In the input panel: the active parent's parameters and the sample size (both hidden with My data), the statistic, the number of resamples, the confidence level, the two seeds (the data seed hidden with My data), the challenge number, and your prediction, typed in a box.

Governing relationships

The generator. The model implements Park and Miller's minimal standard generator (MINSTD) with the multiplier 48271. Starting from s₀ = the seed, each state is

sₖ = MOD(48271 · sₖ₋₁, 2³¹ - 1), and a draw is u = s / (2³¹ - 1),

which lies strictly between 0 and 1. Both streams, the data stream from the data seed and the resampling stream from the resampling seed, discard their first two outputs, so the first draw used is s₃ and not a value close to the seed. The data stream's draw k is therefore s₍ₖ₊₂₎. Every product stays below 2⁴⁸, so every state is an exact whole number.

The resampling stream by jump-ahead. Resample b (b = 1 to 1,000) owns 30 consecutive draws of the resampling stream, draws 30(b - 1) + 1 to 30b, whatever n and B are. Its draws are reached directly, without stepping through the ones before: with Aⱼ = 48271ʲ reduced modulo 2³¹ - 1, the row start is R₁ = A₂ · seed and Rᵦ = A₃₀ · Rᵦ₋₁, and draw j of the row is dⱼ = Aⱼ · Rᵦ, each reduced modulo 2³¹ - 1. This gives exactly the same numbers as stepping the generator one draw at a time. Only the first n draws of a row are used.

The generated sample. Value i (i = 1 to 30, always all 30) uses the data stream's draws 2i - 1 and 2i, called u₁ and u₂:

  • Normal, by the Box-Muller transform (cosine branch): x = μ + σ · √(-2 ln u₁) · cos(2π u₂).
  • Uniform: x = a + (b - a) · u₁.
  • Exponential, by inversion: x = -(mean) · ln(1 - u₁).

Every value, generated or placed, is recorded to 0.0001 (rounded to the nearest ten-thousandth, halves away from zero). The sample of size n is the first n generated values, or the placed points.

One resample. The sample is sorted, x₍₁₎ ≤ … ≤ x₍ₙ₎. Draw j of the resample picks the sorted position INT(n · dⱼ / (2³¹ - 1)) + 1, from 1 to n, each with probability 1/n. The n picks are made with replacement, so a point can be picked several times or not at all.

  • Mean: the resample statistic is the mean of the n picked values.
  • Median: with kLo = INT((n + 1)/2) and kHi = INT(n/2) + 1, it is the average of the kLo-th and kHi-th smallest picked values (the middle value for odd n, where kLo = kHi).

The same rule applied to the sample itself gives the sample statistic θ̂.

Bootstrap summaries. With t₁ … t_B the B resample statistics:

  • bootstrap mean = (t₁ + … + t_B) / B, and bootstrap bias = bootstrap mean - θ̂;
  • bootstrap standard error = the standard deviation of t₁ … t_B with divisor B - 1;
  • exact plug-in standard error (mean only) = √(Σ(xᵢ - x̄)²) / n, the standard deviation of the data with divisor n, over √n. This is the exact standard deviation of the bootstrap distribution of the mean, the value the bootstrap standard error approaches as B grows;
  • sample standard deviation s, with divisor n - 1.

Percentile interval (sample quantile definition 7). With p = (100 - L)/200 at the low end and 1 - p at the high end, sort the resample statistics t₍₁₎ ≤ … ≤ t₍B₎ and set h = (B - 1)p + 1, j = INT(h) and r = h - j. The end is t₍ⱼ₎ + r · (t₍ⱼ₊₁₎ - t₍ⱼ₎), a linear interpolation between neighbouring order statistics. The width is the high end minus the low end.

Basic (reverse percentile) interval. Low end = 2θ̂ - (percentile high end); high end = 2θ̂ - (percentile low end). It reflects the percentile interval about the sample statistic.

t-interval (mean only). x̄ ± t · s/√n, where t is the two-sided critical value of Student's t distribution at the level L with n - 1 degrees of freedom. Its width is 2t · s/√n.

Parent value and coverage (generated data only). The parent's true value is μ for the Normal parent, (a + b)/2 for the Uniform parent and the mean for the Exponential parent. These are the parent's mean, and for the two symmetric parents also its median. For the median of an Exponential parent it is the true median, (mean) · ln 2. "Interval contains the parent value" reads Yes when the percentile low end ≤ the parent value ≤ the percentile high end.

Histogram. With tmin and tmax the smallest and largest of all B statistics and D = tmax - tmin, the 30 bins each have width w = D/29 and are placed so that tmin and tmax sit at the centres of bins 1 and 30: bin r runs from tmin + (r - 1.5)w to tmin + (r - 0.5)w. A statistic t falls in bin 1 + INT((t - tmin)/w + 0.5), kept within 1 to 30. The normal reference of a bin is B times the probability a normal curve with the bootstrap mean and standard error gives the bin. When every statistic is equal (D = 0), all B fall in bin 15 and there is no normal reference.

Width by sample size (generated data only). For each size m = 5, 10, 20 and 30, the model takes the first m generated values from the same parent, resamples them 500 times with resampling rows 1 to 500 of the same stream, and computes the percentile interval at the chosen level and statistic. It reports the ends, the width, the t-interval width 2t · s/√m (mean only), the width times √m, and the guide: the average of the four values of width times √m, divided by √m. If the width falls like 1 over √m, the width times √m stays roughly flat and the widths follow the guide.

Median diagnostics (median only). The median share is the percentage of resamples whose median equals the sample's median exactly. For odd n, where every resample median is one of the data values, the number of distinct medians counts the different values that occurred.

Calculation sequence

The model works through the relationships above in this order:

  1. It checks the entered values in the order of the status list under Errors and warnings.
  2. It fixes the values used: your inputs, or in a challenge the challenge's own values with only the inputs it leaves to you.
  3. It generates all 30 values from the data stream (or reads the placed points), records them to 0.0001, takes the first n, and sorts them.
  4. It computes the sample statistic, mean and standard deviation.
  5. It reaches each of the 1,000 resampling rows by jump-ahead, turns its first n draws into sorted positions, and computes the statistic of each of the first B resamples, with its histogram bin.
  6. It computes the bootstrap mean, standard error and bias, the plug-in standard error, the percentile, basic and t intervals, the parent value and whether the interval contains it.
  7. It builds the histogram, the pick counts of the first five resamples, the width-by-size series and the median diagnostics.
  8. It sets the status, scores a selected challenge, and withholds what a challenge holds back.

Outputs and interpretation

  • Percentile interval, low and high ends: the interval the bootstrap gives directly, from the order statistics of the resample statistics; and its width. In the units of the data.
  • Bootstrap standard error: how much the resample statistic varies from resample to resample.
  • Sample statistic, sample mean, sample standard deviation: the statistic of the sample itself, its mean, and its standard deviation (divisor n - 1).
  • Mean of the resample statistics and bootstrap bias estimate: the bias is the mean of the resample statistics minus the sample statistic. For the mean it is near 0 and differs from 0 only by Monte Carlo error.
  • Exact plug-in standard error (mean only): the value the bootstrap standard error would reach with unlimited resamples. Comparing the two shows the Monte Carlo error of B resamples.
  • Basic interval, low and high ends: the percentile interval reflected about the sample statistic. For a skewed sample the two intervals lean opposite ways.
  • t critical value, t-interval ends and width (mean only; blank for the median): the classic interval from the sample mean and standard deviation, for comparison with the bootstrap.
  • Parent's true value and Interval contains the parent value (generated data only): Yes or No. At 95 percent a No happens by chance in roughly one sample in twenty, and more often for small samples.
  • Distinct medians (median, odd n) and resamples with the sample's median (median, percent).
  • Settings used: the data source, parent and its parameters, statistic, sample size, number of resamples, level and both seeds of the run; in a challenge these are the challenge's own values. A value that does not apply reads blank (for example the data seed with My data).
  • The sample: each row's value, the sorted values and each row's sorted position.
  • Pick counts: how many times each sorted point was picked in resamples 1 to 5; each column adds up to n.
  • The first 500 resamples: each resample's statistic and its histogram bin (fewer rows when B is smaller).
  • Histogram: the low and high end, count and normal reference of each of the 30 bins, over all B resample statistics; the counts add up to B.
  • Width by sample size: the four rows of the series described above.
  • Model version and formula module versions.

Reading the canvas: the strip shows the sorted sample; during the first five resamples each point carries a badge with the number of times it was picked, and unpicked points are dimmed. The resample statistics then drop into their bins until the histogram shows all B counts, with the normal reference curve over it. The percentile interval, the t-interval (mean only) and, at the Advanced level, the basic interval are drawn as bands at the model's ends, with a line at the parent value. The width-by-size chart plots width against sample size, with the width on a logarithmic scale, and draws the guide line. The page shows fewer results at the Explore level and more at Experiment and Advanced; the level never changes a calculation.

What to take from the lab: the bootstrap treats the sample as a stand-in for the parent, so its interval is only as good as the sample. The width shrinks roughly like 1 over the square root of n. Changing only the resampling seed moves the ends a little (Monte Carlo error), while new data move them more (sampling error). Small samples give intervals that tend to be too narrow, and the median's bootstrap can only land on a few data values.

Challenges

Choosing a challenge number from 1 to 5 loads that challenge. It fixes its own data source, parent, sample size, statistic, 1,000 resamples, level and seeds, and leaves you only the input it names. The result reads MET (score 100) or NOT MET (score 0), with the goal, the value scored and a short feedback text: the met text, or a hint while it is not met.

  1. Compare widths. Predict how many times wider the 95 percent interval is with 5 data values than with 20 from the same parent (a Normal parent, mean 50, standard deviation 10; data seed 101; resampling seed 777). Open: your prediction. Scored on the width ratio, the width at n = 5 over the width at n = 20 in the width-by-size series: MET when your prediction is within 25 percent of the ratio.
  2. Narrow enough. Choose a sample size of 20 or fewer whose 95 percent interval for the mean is narrower than 8 (data seed 4242). Open: the sample size. Scored on the percentile interval width: MET when n is at most 20 and the width is below 8.
  3. A miss. At 90 percent about 1 interval in 10 misses the parent mean of 50. Find a data seed whose interval misses it (n = 20). Open: the data seed. Scored on the sample statistic and coverage: MET when the percentile interval does not contain 50.
  4. Predict the error. Predict the bootstrap standard error of the mean of these 16 values to within 10 percent (data seed 1616). Open: your prediction. Scored against the exact plug-in standard error, not the resampled one: MET when your prediction is within 10 percent of it.
  5. Median lumps. Place 11 points so that at least half of the resamples have exactly the same median as your sample (My data, the median, starting from the points 1 to 11). Open: the points. Scored on the median share: MET when there are exactly 11 points, not all equal, and at least 50 percent of the resamples have the sample's median.

Challenges 1 and 4 hold back the answer until you predict. While your prediction is blank, the challenge reads NOT MET with the prompt "Enter your prediction first", and the value scored is blank.

  • In challenge 1 the width-by-size series (every column but the sample size) is blank; everything else shows.
  • In challenge 4 the resampling results are blank: the mean of the resample statistics, both standard errors, the bias, the percentile and basic intervals and the width, the t outputs, whether the interval contains the parent value, the median diagnostics, the first 500 resamples, the pick counts, the histogram and the width-by-size series. The sample, its statistic, mean and standard deviation stay shown, because the prediction is made from them.

Once you enter a number, 0 included, every result appears and the challenge is scored. In challenge 2, while the challenge is not met, the width-by-size series is also blank, since it would show the answer. On the canvas, challenges 1, 2 and 4 hide the width-by-size chart, and challenge 4 also leaves the resample statistics and the interval bands undrawn: their spread is the standard error you are predicting, and the reveal is in Results. Your prediction changes nothing outside challenges 1 and 4.

Validation and status logic

The status reads the first match in the list under Errors and warnings. The first nine NOT VALID checks apply to the entered values; the check on the number of placed points and the three CHECK items apply to the values used, so a challenge's own settings decide them. The CHECK items apply inside challenges too. A CHECK does not stop the calculation: every result is still shown. A challenge number that is not a whole number from 0 to 5 counts as no challenge selected.

Assumptions and limitations

  • One sample of 2 to 30 values; 100, 200, 500 or 1,000 resamples; levels of 80, 90, 95 or 99 percent. Only the mean and the median; the percentile, basic and t intervals only.
  • The percentile interval tends to run too narrow for small samples, and for n below 10 the status says so.
  • The bootstrap of a median is lumpy: a resample median can only be one of a few data values (for odd n, one of the data values themselves), so the histogram has gaps and the ends sit on data values.
  • With few resamples or a high level an end rests on a handful of the most extreme resample statistics and moves a lot with the resampling seed. In the width-by-size series at 99 percent the ends rest on the 3 most extreme of its 500 statistics.
  • The width-by-size series uses the first 5, 10, 20 and 30 values of the same generated sample and 500 resamples at every size, whatever the number of resamples chosen.
  • Values are recorded to 0.0001; a placed value with more decimals is used to the nearest 0.0001. The canvas snaps points to 0.1.
  • The generator is deterministic and reproducible: the same seeds give the same sample and resamples. It is a small linear congruential generator, fine for teaching and wrong for cryptography or serious simulation. Seed 2,147,483,646 mirrors seed 1 (every state is 2³¹ - 1 minus the other).
  • An interval end, a challenge band or the parent value comparison sitting within about one part in a billion of its threshold may read either way.

Restrictions and non-computing states

Every input except the point list is required and is checked on its entered value even when the chosen source, parent or statistic does not use it. With My data at least 2 points are needed.

Under NOT VALID every result and every table is blank, except the model and module versions, the challenge result (No challenge selected, or NOT MET when a challenge is selected), a score of 0, and a selected challenge's goal and hint.

In challenges 1 and 4, until you predict, and in challenge 2 while it is not met, results are held back as described under Challenges. With My data the parent outputs, the coverage and the width-by-size series (except the sample sizes) are blank, because there is no parent.

Errors and warnings

The status messages, first match wins:

  • NOT VALID: choose a data source from the list: the data source is not Generated sample or My data.
  • NOT VALID: choose a parent distribution from the list: the parent is not one of the three.
  • NOT VALID: choose Mean or Median as the statistic
  • NOT VALID: choose the number of resamples and the confidence level from the lists
  • NOT VALID: each seed must be a whole number from 1 to 2147483646: a seed is blank, not whole or out of range.
  • NOT VALID: the sample size must be a whole number from 2 to 30
  • NOT VALID: the Normal parent needs a mean from -1000 to 1000 and a standard deviation from 0.1 to 1000, the Uniform parent a low end below its high end (both from -1000 to 1000), and the Exponential parent a mean from 0.1 to 1000: every parent's parameters are checked, not only the chosen parent's.
  • NOT VALID: point k must be a number from -1000 to 1000, or leave it blank: k is the first point that is not blank and not such a number. The points are checked even with a generated sample.
  • NOT VALID: the challenge number must be a whole number from 0 to 5, and the prediction a number from 0 to 1000
  • NOT VALID: with My data, place at least 2 points
  • CHECK: every data value is the same, so every resample is too and the interval has no width: every interval end equals the sample statistic and the standard errors and bias are 0.
  • CHECK: at this level each end of the percentile interval falls within the 5 most extreme of the resample statistics, so it moves a lot when only the resampling seed changes; use more resamples or a lower level: 100 resamples at 95 or 99 percent, or 200 or 500 resamples at 99 percent.
  • CHECK: fewer than 10 data values; the percentile interval tends to be too narrow for samples this small
  • OK: the run is computed with nothing to flag.

References

This laboratory is an educational model of the nonparametric bootstrap. It resamples one sample with replacement, computes the mean or the median of each resample, and forms the percentile interval from the order statistics of the resample statistics, using sample quantile definition 7 (linear interpolation between neighbouring order statistics), beside the basic (reverse percentile) interval and the classic t-interval for a mean. Its random numbers come from the Park-Miller minimal standard (MINSTD) generator, a multiplicative linear congruential generator with modulus 2^31 - 1 and the multiplier 48271, each stream discarding its first two outputs, so every run is exact and repeatable from its seeds. A Normal parent is sampled by the Box-Muller transform and an Exponential parent by inversion.

Sources and further reading

  • The bootstrap: B. Efron, "Bootstrap methods: another look at the jackknife", The Annals of Statistics 7(1), 1979, pages 1-26. doi:10.1214/aos/1176344552
  • The percentile and basic bootstrap confidence intervals: A. C. Davison and D. V. Hinkley, Bootstrap Methods and their Application, Cambridge University Press, 1997, chapter 5, Confidence Intervals. Cambridge University Press
  • The sample quantile definitions, of which the percentile interval uses definition 7: R. J. Hyndman and Y. Fan, "Sample quantiles in statistical packages", The American Statistician 50(4), 1996, pages 361-365. Author's page
  • The t-interval for a mean: NIST/SEMATECH, e-Handbook of Statistical Methods, section 1.3.5.2, Confidence Limits for the Mean. NIST
  • The minimal standard generator: S. K. Park and K. W. Miller, "Random number generators: good ones are hard to find", Communications of the ACM 31(10), 1988, pages 1192-1201; and the multiplier 48271, in S. K. Park, K. W. Miller and P. K. Stockmeyer, "Technical correspondence", Communications of the ACM 36(7), 1993. doi:10.1145/63039.63042, doi:10.1145/159544.376068
  • Normal values from two uniform draws: G. E. P. Box and M. E. Muller, "A note on the generation of random normal deviates", The Annals of Mathematical Statistics 29(2), 1958, pages 610-611. doi:10.1214/aoms/1177706645

No table or text from any of these publications is reproduced, and none of their authors or publishers has reviewed or endorses this laboratory.

This laboratory is for learning. Its results hold only within its stated ranges; a bootstrap interval from one small sample is only as good as that sample, and with few resamples its ends move with the resampling seed.

Links were current at build time. If one moves, search the document's title rather than reporting a broken link as a defect.

Results are informational and not professional advice. See our Terms of Use. Powered by SpreadsheetWeb · About · Privacy · Terms