Resampling Probability Estimates
For the Difference Between the Means of Two Independent Samples
With
- Enter the values for samples A and B into the designated cells, then click the 'Calculate' button to calculate the means of the two samples and the difference between them, M_{a}—M_{b}. For purposes of comparison, this action will also cause a t-test to be performed.
- Then click the 'Resample' button. With each click of this button, the programming will produce 1000 random re-sortings of the values you have entered, on each occasion calculating and keeping track of the "resampled" value of M_{a}—M_{b}. The probabilities estimated on the basis of this process represent the proportions of resampled M_{a}—M_{b} values whose distance from zero is as great as or greater than the observed value of M_{a}—M_{b}. If you click the "Resample' button more than once, the estimated probabilities are cumulative. [Note that each resampling will take a moment and that processing time increases exponentially as a function of n_{a}+n_{b}].
- Caveat. My personal view is that probability estimates based on resampling should be used with great caution, because the potential array of resampled values of M_{a}—M_{b} is a constrained subset of the full array of values that might be drawn from the original source population. The constraint derives from the fact that what you are really doing in resampling is shuffling and re-shuffling the very same values of A and B. In most cases involving the difference between the means of two independent samples, the probabilities estimated on the basis of resampling will be larger than those indicated by a t-test. For small values of n_{a} and n_{b}, the resampling probabilities correspond fairly closely with those that would be yielded by a randomization test for independent samples. (The randomization test is also based on the concept of shuffling and re-shuffling the same values of A and B.)
- For n_{a}= and n_{b}=, the total number of unique resortings of the entered values of A and B is
Data Entry:
P_{t-test}
| one-tailed
| |
two-tailed
| |