text:
Analysis of variance
From Wikipedia, the free encyclopedia
Jump to: navigation, search
Analysis of variance (ANOVA) is a collection of statistical models used to analyze the differences among group means and their associated procedures (such as "variation" among and between groups), developed by statistician and evolutionary biologist Ronald Fisher. In the ANOVA setting, the observed variance in a particular variable is partitioned into components attributable to different sources of variation. In its simplest form, ANOVA provides a statistical test of whether or not the means of several groups are equal, and therefore generalizes the t-test to more than two groups. ANOVAs are useful for comparing (testing) three or more means (groups or variables) for statistical significance. It is conceptually similar to multiple two-sample t-tests, but is more conservative (results in less type I error) [1] and is therefore suited to a wide range of practical problems.
Contents
[hide]
1 History
2 Motivating example
3 Background and terminology
3.1 Design-of-experiments terms
4 Classes of models
4.1 Fixed-effects models
4.2 Random-effects models
4.3 Mixed-effects models
5 Assumptions
5.1 Textbook analysis using a normal distribution
5.2 Randomization-based analysis
5.2.1 Unit-treatment additivity
5.2.2 Derived linear model
5.2.3 Statistical models for observational data
5.3 Summary of assumptions
6 Characteristics
7 Logic
7.1 Partitioning of the sum of squares
7.2 The F-test
7.3 Extended logic
8 For a single factor
9 For multiple factors
10 Worked numeric examples
11 Associated analysis
11.1 Preparatory analysis
11.1.1 The number of experimental units
11.1.2 Power analysis
11.1.3 Effect size
11.2 Follow-up analysis
11.2.1 Model confirmation
11.2.2 Follow-up tests
12 Study designs
13 Cautions
14 Generalizations
14.1 Connection to linear regression
14.1.1 Example
15 See also
16 Footnotes
17 Notes
18 References
19 Further reading
20 External links
History[edit]
While the analysis of variance reached fruition in the 20th century, antecedents extend centuries into the past according to Stigler.[2] These include hypothesis testing, the partitioning of sums of squares, experimental techniques and the additive model. Laplace was performing hypothesis testing in the 1770s.[3] The development of least-squares methods by Laplace and Gauss circa 1800 provided an improved method of combining observations (over the existing practices then used in astronomy and geodesy). It also initiated much study of the contributions to sums of squares. Laplace soon knew how to estimate a variance from a residual (rather than a total) sum of squares.[4] By 1827 Laplace was using least squares methods to address ANOVA problems regarding measurements of atmospheric tides.[5] Before 1800 astronomers had isolated observational errors resulting from reaction times (the "personal equation") and had developed methods of reducing the errors.[6] The experimental methods used in the study of the personal equation were later accepted by the emerging field of psychology [7] which developed strong (full factorial) experimental methods to which randomization and blinding were soon added.[8] An eloquent non-mathematical explanation of the additive effects model was available in 1885.[9]
Ronald Fisher introduced the term variance and proposed its formal analysis in a 1918 article The Correlation Between Relatives on the Supposition of Mendelian Inheritance.[10] His first application of the analysis of variance was published in 1921.[11] Analysis of variance became widely known after being included in Fisher's 1925 book Statistical Methods for Research Workers.
Randomization models were developed by several researchers. The first was published in Polish by Jerzy Neyman in 1923.[12]
One of the attributes of ANOVA which ensured its early popularity was computational elegance. The structure of the additive model allows solution for the additive coefficients by simple algebra rather than by matrix calculations. In the era of mechanical calculators this simplicity was critical. The determination of statistical significance also required access to tables of the F function which were supplied by early statistics texts.
Motivating example[edit]
No fit.
Fair fit
Very good fit
The analysis of variance can be used as an exploratory tool to explain observations. A dog show provides an example. A dog show is not a random sampling of the breed: it is typically limited to dogs that are adult, pure-bred, and exemplary. A histogram of dog weights from a show might plausibly be rather complex, like the yellow-orange distribution shown in the illustrations. Suppose we wanted to predict the weight of a dog based on a certain set of characteristics of each dog. Before we could do that, we would need to explain the distribution of weights by dividing the dog population into groups based on those characteristics. A successful grouping will split dogs such that (a) each group has a low variance of dog weights (meaning the group is relatively homogeneous) and (b) the mean of each group is distinct (if two groups have the same mean, then it isn't reasonable to conclude that the groups are, in fact, separate in any meaningful way).
In the illustrations to the right, each group is identified as X1, X2, etc. In the first illustration, we divide the dogs according to the product (interaction) of two binary groupings: young vs old, and short-haired vs long-haired (thus, group 1 is young, short-haired dogs, group 2 is young, long-haired dogs, etc.). Since the distributions of dog weight within each of the groups (shown in blue) has a large variance, and since the means are very close across groups, grouping dogs by these characteristics does not produce an effective way to explain the variation in dog weights: knowing which group a dog is in does not allow us to make any reasonable statements as to what that dog's weight is likely to be. Thus, this grouping fails to fit the distribution we are trying to explain (yellow-orange).
An attempt to explain the weight distribution by grouping dogs as (pet vs working breed) and (less athletic vs more athletic) would probably be somewhat more successful (fair fit). The heaviest show dogs are likely to be big strong working breeds, while breeds kept as pets tend to be smaller and thus lighter. As shown by the second illustration, the distributions have variances that are considerably smaller than in the first case, and the means are more reasonably distinguishable. However, the significant overlap of distributions, for example, means that we cannot reliably say that X1 and X2 are truly distinct (i.e., it is perhaps reasonably likely that splitting dogs according to the flip of a coin—by pure chance—might produce distributions that look similar).
An attempt to explain weight by breed is likely to produce a very good fit. All Chihuahuas are light and all St Bernards are heavy. The difference in weights between Setters and Pointers does not justify separate breeds. The analysis of variance provides the formal tools to justify these intuitive judgments. A common use of the method is the analysis of experimental data or the development of models. The method has some advantages over correlation: not all of the data must be numeric and one result of the method is a judgment in the confidence in an explanatory relationship.
Background and terminology[edit]
ANOVA is a particular form of statistical hypothesis testing heavily used in the analysis of experimental data. A test result (calculated from the null hypothesis and the sample) is called statistically significant if it is deemed unlikely to have occurred by chance, assuming the truth of the null hypothesis. A statistically significant result, when a probability (p-value) is less than a threshold (significance level), justifies the rejection of the null hypothesis, but only if the priori probability of the null hypothesis is not high.
In the typical application of ANOVA, the null hypothesis is that all groups are simply random samples of the same population. For example, when studying the effect of different treatments on similar samples of patients, the null hypothesis would be that all treatments have the same effect (perhaps none). Rejecting the null hypothesis would imply that different treatments result in altered effects.
By construction, hypothesis testing limits the rate of Type I errors (false positives) to a significance level. Experimenters also wish to limit Type II errors (false negatives). The rate of Type II errors depends largely on sample size (the rate will increase for small numbers of samples), significance level (when the standard of proof is high, the chances of overlooking a discovery are also high) and effect size (a smaller effect size is more prone to Type II error).
The terminology of ANOVA is largely from the statistical design of experiments. The experimenter adjusts factors and measures responses in an attempt to determine an effect. Factors are assigned to experimental units by a combination of randomization and blocking to ensure the validity of the results. Blinding keeps the weighing impartial. Responses show a variability that is partially the result of the effect and is partially random error.
ANOVA is the synthesis of several ideas and it is used for multiple purposes. As a consequence, it is difficult to define concisely or precisely.
"Classical ANOVA for balanced data does three things at once:
As exploratory data analysis, an ANOVA is an organization of an additive data decomposition, and its sums of squares indicate the variance of each component of the decomposition (or, equivalently, each set of terms of a linear model).
Comparisons of mean squares, along with an F-test ... allow testing of a nested sequence of models.
Closely related to the ANOVA is a linear model fit with coefficient estimates and standard errors."[13]
In short, ANOVA is a statistical tool used in several ways to develop and confirm an explanation for the observed data.
Additionally:
It is computationally elegant and relatively robust against violations of its assumptions.
ANOVA provides industrial strength (multiple sample comparison) statistical analysis.
It has been adapted to the analysis of a variety of experimental designs.
As a result: ANOVA "has long enjoyed the status of being the most used (some would say abused) statistical technique in psychological research."[14] ANOVA "is probably the most useful technique in the field of statistical inference."[15]
ANOVA is difficult to teach, particularly for complex experiments, with split-plot designs being notorious.[16] In some cases the proper application of the method is best determined by problem pattern recognition followed by the consultation of a classic authoritative test.[17]
Design-of-experiments terms[edit]
(Condensed from the NIST Engineering Statistics handbook: Section 5.7. A Glossary of DOE Terminology.)[18]
Balanced design
An experimental design where all cells (i.e. treatment combinations) have the same number of observations.
Blocking
A schedule for conducting treatment combinations in an experimental study such that any effects on the experimental results due to a known change in raw materials, operators, machines, etc., become concentrated in the levels of the blocking variable. The reason for blocking is to isolate a systematic effect and prevent it from obscuring the main effects. Blocking is achieved by restricting randomization.
Design
A set of experimental runs which allows the fit of a particular model and the estimate of effects.
DOE
Design of experiments. An approach to problem solving involving collection of data that will support valid, defensible, and supportable conclusions.[19]
Effect
How changing the settings of a factor changes the response. The effect of a single factor is also called a main effect.
Error
Unexplained variation in a collection of observations. DOE's typically require understanding of both random error and lack of fit error.
Experimental unit
The entity to which a specific treatment combination is applied.
Factors
Process inputs that an investigator manipulates to cause a change in the output.
Lack-of-fit error
Error that occurs when the analysis omits one or more important terms or factors from the process model. Including replication in a DOE allows separation of experimental error into its components: lack of fit and random (pure) error.
Model
Mathematical relationship which relates changes in a given response to changes in one or more factors.
Random error
Error that occurs due to natural variation in the process. Random error is typically assumed to be normally distributed with zero mean and a constant variance. Random error is also called experimental error.
Randomization
A schedule for allocating treatment material and for conducting treatment combinations in a DOE such that the conditions in one run neither depend on the conditions of the previous run nor predict the conditions in the subsequent runs.[nb 1]
Replication
Performing the same treatment combination more than once. Including replication allows an estimate of the random error independent of any lack of fit error.
Responses
The output(s) of a process. Sometimes called dependent variable(s).
Treatment
A treatment is a specific combination of factor levels whose effect is to be compared with other treatments.
Classes of models[edit]
There are three classes of models used in the analysis of variance, and these are outlined here.
Fixed-effects models[edit]
Main article: Fixed effects model
The fixed-effects model (class I) of analysis of variance applies to situations in which the experimenter applies one or more treatments to the subjects of the experiment to see whether the response variable values change. This allows the experimenter to estimate the ranges of response variable values that the treatment would generate in the population as a whole.
Random-effects models[edit]
Main article: Random effects model
Random effects model (class II) is used when the treatments are not fixed. This occurs when the various factor levels are sampled from a larger population. Because the levels themselves are random variables, some assumptions and the method of contrasting the treatments (a multi-variable generalization of simple differences) differ from the fixed-effects model.[20]
Mixed-effects models[edit]
Main article: Mixed model
A mixed-effects model (class III) contains experimental factors of both fixed and random-effects types, with appropriately different interpretations and analysis for the two types.
Example: Teaching experiments could be performed by a college or university department to find a good introductory textbook, with each text considered a treatment. The fixed-effects model would compare a list of candidate texts. The random-effects model would determine whether important differences exist among a list of randomly selected texts. The mixed-effects model would compare the (fixed) incumbent texts to randomly selected alternatives.
Defining fixed and random effects has proven elusive, with competing definitions arguably leading toward a linguistic quagmire.[21]
Assumptions[edit]
The analysis of variance has been studied from several approaches, the most common of which uses a linear model that relates the response to the treatments and blocks. Note that the model is linear in parameters but may be nonlinear across factor levels. Interpretation is easy when data is balanced across factors but much deeper understanding is needed for unbalanced data.
Textbook analysis using a normal distribution[edit]
The analysis of variance can be presented in terms of a linear model, which makes the following assumptions about the probability distribution of the responses:[22][23][24][25]
Independence of observations – this is an assumption of the model that simplifies the statistical analysis.
Normality – the distributions of the residuals are normal.
Equality (or "homogeneity") of variances, called homoscedasticity — the variance of data in groups should be the same.
The separate assumptions of the textbook model imply that the errors are independently, identically, and normally distributed for fixed effects models, that is, that the errors ( ε {\displaystyle \varepsilon } ) are independent and
ε ∼ N ( 0 , σ 2 ) . {\displaystyle \varepsilon \thicksim N(0,\sigma ^{2}).\,}
Randomization-based analysis[edit]
See also: Random assignment and Randomization test
In a randomized controlled experiment, the treatments are randomly assigned to experimental units, following the experimental protocol. This randomization is objective and declared before the experiment is carried out. The objective random-assignment is used to test the significance of the null hypothesis, following the ideas of C. S. Peirce and Ronald Fisher. This design-based analysis was discussed and developed by Francis J. Anscombe at Rothamsted Experimental Station and by Oscar Kempthorne at Iowa State University.[26] Kempthorne and his students make an assumption of unit treatment additivity, which is discussed in the books of Kempthorne and David R. Cox.[citation needed]
Unit-treatment additivity[edit]
In its simplest form, the assumption of unit-treatment additivity[nb 2] states that the observed response y i , j {\displaystyle y_{i,j}} from experimental unit i {\displaystyle i} when receiving treatment j {\displaystyle j} can be written as the sum of the unit's response y i {\displaystyle y_{i}} and the treatment-effect t j {\displaystyle t_{j}} , that is [27][28][29]
y i , j = y i + t j . {\displaystyle y_{i,j}=y_{i}+t_{j}.}
The assumption of unit-treatment additivity implies that, for every treatment j {\displaystyle j} , the j {\displaystyle j} th treatment has exactly the same effect t j {\displaystyle t_{j}} on every experiment unit.
The assumption of unit treatment additivity usually cannot be directly falsified, according to Cox and Kempthorne. However, many consequences of treatment-unit additivity can be falsified. For a randomized experiment, the assumption of unit-treatment additivity implies that the variance is constant for all treatments. Therefore, by contraposition, a necessary condition for unit-treatment additivity is that the variance is constant.
The use of unit treatment additivity and randomization is similar to the design-based inference that is standard in finite-population survey sampling.
Derived linear model[edit]
Kempthorne uses the randomization-distribution and the assumption of unit treatment additivity to produce a derived linear model, very similar to the textbook model discussed previously.[30] The test statistics of this derived linear model are closely approximated by the test statistics of an appropriate normal linear model, according to approximation theorems and simulation studies.[31] However, there are differences. For example, the randomization-based analysis results in a small but (strictly) negative correlation between the observations.[32][33] In the randomization-based analysis, there is no assumption of a normal distribution and certainly no assumption of independence. On the contrary, the observations are dependent!
The randomization-based analysis has the disadvantage that its exposition involves tedious algebra and extensive time. Since the randomization-based analysis is complicated and is closely approximated by the approach using a normal linear model, most teachers emphasize the normal linear model approach. Few statisticians object to model-based analysis of balanced randomized experiments.
Statistical models for observational data[edit]
However, when applied to data from non-randomized experiments or observational studies, model-based analysis lacks the warrant of randomization.[34] For observational data, the derivation of confidence intervals must use subjective models, as emphasized by Ronald Fisher and his followers. In practice, the estimates of treatment-effects from observational studies generally are often inconsistent. In practice, "statistical models" and observational data are useful for suggesting hypotheses that should be treated very cautiously by the public.[35]
Summary of assumptions[edit]
The normal-model based ANOVA analysis assumes the independence, normality and homogeneity of the variances of the residuals. The randomization-based analysis assumes only the homogeneity of the variances of the residuals (as a consequence of unit-treatment additivity) and uses the randomization procedure of the experiment. Both these analyses require homoscedasticity, as an assumption for the normal-model analysis and as a consequence of randomization and additivity for the randomization-based analysis.
However, studies of processes that change variances rather than means (called dispersion effects) have been successfully conducted using ANOVA.[36] There are no necessary assumptions for ANOVA in its full generality, but the F-test used for ANOVA hypothesis testing has assumptions and practical limitations which are of continuing interest.
Problems which do not satisfy the assumptions of ANOVA can often be transformed to satisfy the assumptions. The property of unit-treatment additivity is not invariant under a "change of scale", so statisticians often use transformations to achieve unit-treatment additivity. If the response variable is expected to follow a parametric family of probability distributions, then the statistician may specify (in the protocol for the experiment or observational study) that the responses be transformed to stabilize the variance.[37] Also, a statistician may specify that logarithmic transforms be applied to the responses, which are believed to follow a multiplicative model.[28][38] According to Cauchy's functional equation theorem, the logarithm is the only continuous transformation that transforms real multiplication to addition.[citation needed]
Characteristics[edit]
ANOVA is used in the analysis of comparative experiments, those in which only the difference in outcomes is of interest. The statistical significance of the experiment is determined by a ratio of two variances. This ratio is independent of several possible alterations to the experimental observations: Adding a constant to all observations does not alter significance. Multiplying all observations by a constant does not alter significance. So ANOVA statistical significance result is independent of constant bias and scaling errors as well as the units used in expressing observations. In the era of mechanical calculation it was common to subtract a constant from all observations (when equivalent to dropping leading digits) to simplify data entry.[39][40] This is an example of data coding.
Logic[edit]
The calculations of ANOVA can be characterized as computing a number of means and variances, dividing two variances and comparing the ratio to a handbook value to determine statistical significance. Calculating a treatment effect is then trivial, "the effect of any treatment is estimated by taking the difference between the mean of the observations which receive the treatment and the general mean".[41]
Partitioning of the sum of squares[edit]
Main article: Partition of sums of squares
ANOVA uses traditional standardized terminology. The definitional equation of sample variance is s 2 = 1 n − 1 ∑ ( y i − y ¯ ) 2 {\displaystyle s^{2}=\textstyle {\frac {1}{n-1}}\sum (y_{i}-{\bar {y}})^{2}} , where the divisor is called the degrees of freedom (DF), the summation is called the sum of squares (SS), the result is called the mean square (MS) and the squared terms are deviations from the sample mean. ANOVA estimates 3 sample variances: a total variance based on all the observation deviations from the grand mean, an error variance based on all the observation deviations from their appropriate treatment means, and a treatment variance. The treatment variance is based on the deviations of treatment means from the grand mean, the result being multiplied by the number of observations in each treatment to account for the difference between the variance of observations and the variance of means.
The fundamental technique is a partitioning of the total sum of squares SS into components related to the effects used in the model. For example, the model for a simplified ANOVA with one type of treatment at different levels.
S S Total = S S Error + S S Treatments {\displaystyle SS_{\text{Total}}=SS_{\text{Error}}+SS_{\text{Treatments}}}
The number of degrees of freedom DF can be partitioned in a similar way: one of these components (that for error) specifies a chi-squared distribution which describes the associated sum of squares, while the same is true for "treatments" if there is no treatment effect.
D F Total = D F Error + D F Treatments {\displaystyle DF_{\text{Total}}=DF_{\text{Error}}+DF_{\text{Treatments}}}
See also Lack-of-fit sum of squares.
The F-test[edit]
Main article: F-test
The F-test is used for comparing the factors of the total deviation. For example, in one-way, or single-factor ANOVA, statistical significance is tested for by comparing the F test statistic
F = variance between treatments variance within treatments {\displaystyle F={\frac {\text{variance between treatments}}{\text{variance within treatments}}}}
F = M S Treatments M S Error = S S Treatments / ( I − 1 ) S S Error / ( n T − I ) {\displaystyle F={\frac {MS_{\text{Treatments}}}{MS_{\text{Error}}}}={{SS_{\text{Treatments}}/(I-1)} \over {SS_{\text{Error}}/(n_{T}-I)}}}
where MS is mean square, I {\displaystyle I} = number of treatments and n T {\displaystyle n_{T}} = total number of cases
to the F-distribution with I − 1 {\displaystyle I-1} , n T − I {\displaystyle n_{T}-I} degrees of freedom. Using the F-distribution is a natural candidate because the test statistic is the ratio of two scaled sums of squares each of which follows a scaled chi-squared distribution.
The expected value of F is 1 + n σ Treatment 2 / σ Error 2 {\displaystyle 1+{n\sigma _{\text{Treatment}}^{2}}/{\sigma _{\text{Error}}^{2}}} (where n is the treatment sample size) which is 1 for no treatment effect. As values of F increase above 1, the evidence is increasingly inconsistent with the null hypothesis. Two apparent experimental methods of increasing F are increasing the sample size and reducing the error variance by tight experimental controls.
There are two methods of concluding the ANOVA hypothesis test, both of which produce the same result:
The textbook method is to compare the observed value of F with the critical value of F determined from tables. The critical value of F is a function of the degrees of freedom of the numerator and the denominator and the significance level (α). If F ≥ FCritical, the null hypothesis is rejected.
The computer method calculates the probability (p-value) of a value of F greater than or equal to the observed value. The null hypothesis is rejected if this probability is less than or equal to the significance level (α).
The ANOVA F-test is known to be nearly optimal in the sense of minimizing false negative errors for a fixed rate of false positive errors (i.e. maximizing power for a fixed significance level). For example, to test the hypothesis that various medical treatments have exactly the same effect, the F-test's p-values closely approximate the permutation test's p-values: The approximation is particularly close when the design is balanced.[31][42] Such permutation tests characterize tests with maximum power against all alternative hypotheses, as observed by Rosenbaum.[nb 3] The ANOVA F–test (of the null-hypothesis that all treatments have exactly the same effect) is recommended as a practical test, because of its robustness against many alternative distributions.[43][nb 4]
Extended logic[edit]
ANOVA consists of separable parts; partitioning sources of variance and hypothesis testing can be used individually. ANOVA is used to support other statistical tools. Regression is first used to fit more complex models to data, then ANOVA is used to compare models with the objective of selecting simple(r) models that adequately describe the data. "Such models could be fit without any reference to ANOVA, but ANOVA tools could then be used to make some sense of the fitted models, and to test hypotheses about batches of coefficients."[44] "[W]e think of the analysis of variance as a way of understanding and structuring multilevel models—not as an alternative to regression but as a tool for summarizing complex high-dimensional inferences ..."[44]
For a single factor[edit]
Main article: One-way analysis of variance
The simplest experiment suitable for ANOVA analysis is the completely randomized experiment with a single factor. More complex experiments with a single factor involve constraints on randomization and include completely randomized blocks and Latin squares (and variants: Graeco-Latin squares, etc.). The more complex experiments share many of the complexities of multiple factors. A relatively complete discussion of the analysis (models, data summaries, ANOVA table) of the completely randomized experiment is available.
For multiple factors[edit]
Main article: Two-way analysis of variance
ANOVA generalizes to the study of the effects of multiple factors. When the experiment includes observations at all combinations of levels of each factor, it is termed factorial. Factorial experiments are more efficient than a series of single factor experiments and the efficiency grows as the number of factors increases.[45] Consequently, factorial designs are heavily used.
The use of ANOVA to study the effects of multiple factors has a complication. In a 3-way ANOVA with factors x, y and z, the ANOVA model includes terms for the main effects (x, y, z) and terms for interactions (xy, xz, yz, xyz). All terms require hypothesis tests. The proliferation of interaction terms increases the risk that some hypothesis test will produce a false positive by chance. Fortunately, experience says that high order interactions are rare.[46][verification needed] The ability to detect interactions is a major advantage of multiple factor ANOVA. Testing one factor at a time hides interactions, but produces apparently inconsistent experimental results.[45]
Caution is advised when encountering interactions; Test interaction terms first and expand the analysis beyond ANOVA if interactions are found. Texts vary in their recommendations regarding the continuation of the ANOVA procedure after encountering an interaction. Interactions complicate the interpretation of experimental data. Neither the calculations of significance nor the estimated treatment effects can be taken at face value. "A significant interaction will often mask the significance of main effects."[47] Graphical methods are recommended to enhance understanding. Regression is often useful. A lengthy discussion of interactions is available in Cox (1958).[48] Some interactions can be removed (by transformations) while others cannot.
A variety of techniques are used with multiple factor ANOVA to reduce expense. One technique used in factorial designs is to minimize replication (possibly no replication with support of analytical trickery) and to combine groups when effects are found to be statistically (or practically) insignificant. An experiment with many insignificant factors may collapse into one with a few factors supported by many replications.[49]
Worked numeric examples[edit]
Several fully worked numerical examples are available. A simple case uses one-way (a single factor) analysis. A more complex case uses two-way (two-factor) analysis.
Associated analysis[edit]
Some analysis is required in support of the design of the experiment while other analysis is performed after changes in the factors are formally found to produce statistically significant changes in the responses. Because experimentation is iterative, the results of one experiment alter plans for following experiments.
Preparatory analysis[edit]
The number of experimental units[edit]
In the design of an experiment, the number of experimental units is planned to satisfy the goals of the experiment. Experimentation is often sequential.
Early experiments are often designed to provide mean-unbiased estimates of treatment effects and of experimental error. Later experiments are often designed to test a hypothesis that a treatment effect has an important magnitude; in this case, the number of experimental units is chosen so that the experiment is within budget and has adequate power, among other goals.
Reporting sample size analysis is generally required in psychology. "Provide information on sample size and the process that led to sample size decisions."[50] The analysis, which is written in the experimental protocol before the experiment is conducted, is examined in grant applications and administrative review boards.
Besides the power analysis, there are less formal methods for selecting the number of experimental units. These include graphical methods based on limiting the probability of false negative errors, graphical methods based on an expected variation increase (above the residuals) and methods based on achieving a desired confident interval.[51]
Power analysis[edit]
Power analysis is often applied in the context of ANOVA in order to assess the probability of successfully rejecting the null hypothesis if we assume a certain ANOVA design, effect size in the population, sample size and significance level. Power analysis can assist in study design by determining what sample size would be required in order to have a reasonable chance of rejecting the null hypothesis when the alternative hypothesis is true.[52][53][54][55]
Effect size[edit]
Main article: Effect size
Several standardized measures of effect have been proposed for ANOVA to summarize the strength of the association between a predictor(s) and the dependent variable (e.g., η2, ω2, or ƒ2) or the overall standardized difference (Ψ) of the complete model. Standardized effect-size estimates facilitate comparison of findings across studies and disciplines. However, while standardized effect sizes are commonly used in much of the professional literature, a non-standardized measure of effect size that has immediately "meaningful" units may be preferable for reporting purposes.[56]
Follow-up analysis[edit]
It is always appropriate to carefully consider outliers. They have a disproportionate impact on statistical conclusions and are often the result of errors.
Model confirmation[edit]
It is prudent to verify that the assumptions of ANOVA have been met. Residuals are examined or analyzed to confirm homoscedasticity and gross normality.[57] Residuals should have the appearance of (zero mean normal distribution) noise when plotted as a function of anything including time and modeled data values. Trends hint at interactions among factors or among observations. One rule of thumb: "If the largest standard deviation is less than twice the smallest standard deviation, we can use methods based on the assumption of equal standard deviations and our results will still be approximately correct."[58]
Follow-up tests[edit]
A statistically significant effect in ANOVA is often followed up with one or more different follow-up tests. This can be done in order to assess which groups are different from which other groups or to test various other focused hypotheses. Follow-up tests are often distinguished in terms of whether they are planned (a priori) or post hoc. Planned tests are determined before looking at the data and post hoc tests are performed after looking at the data.
Often one of the "treatments" is none, so the treatment group can act as a control. Dunnett's test (a modification of the t-test) tests whether each of the other treatment groups has the same mean as the control.[59]
Post hoc tests such as Tukey's range test most commonly compare every group mean with every other group mean and typically incorporate some method of controlling for Type I errors. Comparisons, which are most commonly planned, can be either simple or compound. Simple comparisons compare one group mean with one other group mean. Compound comparisons typically compare two sets of groups means where one set has two or more groups (e.g., compare average group means of group A, B and C with group D). Comparisons can also look at tests of trend, such as linear and quadratic relationships, when the independent variable involves ordered levels.
Following ANOVA with pair-wise multiple-comparison tests has been criticized on several grounds.[56][60] There are many such tests (10 in one table) and recommendations regarding their use are vague or conflicting.[61][62]
Study designs[edit]
There are several types of ANOVA. Many statisticians base ANOVA on the design of the experiment,[63] especially on the protocol that specifies the random assignment of treatments to subjects; the protocol's description of the assignment mechanism should include a specification of the structure of the treatments and of any blocking. It is also common to apply ANOVA to observational data using an appropriate statistical model.[citation needed]
Some popular designs use the following types of ANOVA:
One-way ANOVA is used to test for differences among two or more independent groups (means),e.g. different levels of urea application in a crop, or different levels of antibiotic action on several bacterial species,[64] or different levels of effect of some medicine on groups of patients. Typically, however, the one-way ANOVA is used to test for differences among at least three groups, since the two-group case can be covered by a t-test.[65] When there are only two means to compare, the t-test and the ANOVA F-test are equivalent; the relation between ANOVA and t is given by F = t2.
Factorial ANOVA is used when the experimenter wants to study the interaction effects among the treatments.
Repeated measures ANOVA is used when the same subjects are used for each treatment (e.g., in a longitudinal study).
Multivariate analysis of variance (MANOVA) is used when there is more than one response variable.
Cautions[edit]
Balanced experiments (those with an equal sample size for each treatment) are relatively easy to interpret; Unbalanced experiments offer more complexity. For single factor (one way) ANOVA, the adjustment for unbalanced data is easy, but the unbalanced analysis lacks both robustness and power.[66] For more complex designs the lack of balance leads to further complications. "The orthogonality property of main effects and interactions present in balanced data does not carry over to the unbalanced case. This means that the usual analysis of variance techniques do not apply. Consequently, the analysis of unbalanced factorials is much more difficult than that for balanced designs."[67] In the general case, "The analysis of variance can also be applied to unbalanced data, but then the sums of squares, mean squares, and F-ratios will depend on the order in which the sources of variation are considered."[44] The simplest techniques for handling unbalanced data restore balance by either throwing out data or by synthesizing missing data. More complex techniques use regression.
ANOVA is (in part) a significance test. The American Psychological Association holds the view that simply reporting significance is insufficient and that reporting confidence bounds is preferred.[56]
While ANOVA is conservative (in maintaining a significance level) against multiple comparisons in one dimension, it is not conservative against comparisons in multiple dimensions.[68]
Generalizations[edit]
ANOVA is considered to be a special case of linear regression[69][70] which in turn is a special case of the general linear model.[71] All consider the observations to be the sum of a model (fit) and a residual (error) to be minimized.
The Kruskal–Wallis test and the Friedman test are nonparametric tests, which do not rely on an assumption of normality.[72][73]
Connection to linear regression[edit]
Below we make clear the connection between multi-way ANOVA and linear regression. Linearly re-order the data so that k th {\displaystyle k^{\text{th}}} observation is associated with a response y k {\displaystyle y_{k}} and factors Z k , b {\displaystyle Z_{k,b}} where b ∈ { 1 , 2 , … , B } {\displaystyle b\in \{1,2,\ldots ,B\}} denotes the different factors and B {\displaystyle B} is the total number of factors. In one-way ANOVA B = 1 {\displaystyle B=1} and in two-way ANOVA B = 2 {\displaystyle B=2} . Furthermore, we assume the b t h {\displaystyle b^{th}} factor has I b {\displaystyle I_{b}} levels. Now, we can one-hot encode the factors into the ∑ b = 1 B I b {\displaystyle \sum _{b=1}^{B}I_{b}} dimensional vector v k {\displaystyle v_{k}} .
The one-hot encoding function g b : I b ↦ { 0 , 1 } I b {\displaystyle g_{b}:I_{b}\mapsto \{0,1\}^{I_{b}}} is defined such that the i t h {\displaystyle i^{th}} entry of g b ( Z k , b ) {\displaystyle g_{b}(Z_{k,b})} is
g b ( Z k , b ) i = { 1 if i = Z k , b 0 otherwise {\displaystyle g_{b}(Z_{k,b})_{i}={\begin{cases}1&{\text{if }}i=Z_{k,b}\\0&{\text{otherwise}}\end{cases}}}
The vector v k {\displaystyle v_{k}} is the concatenation of all of the above vectors for all b {\displaystyle b} . Thus, v k = [ g 1 ( Z k , 1 ) , g 2 ( Z k , 2 ) , … , g B ( Z k , B ) ] {\displaystyle v_{k}=[g_{1}(Z_{k,1}),g_{2}(Z_{k,2}),\ldots ,g_{B}(Z_{k,B})]} . In order to obtain a fully general B {\displaystyle B} -way interaction ANOVA we must also concatenate every additional interaction term in the vector v k {\displaystyle v_{k}} and then add an intercept term. Let that vector be x k {\displaystyle x_{k}} .
With this notation in place, we now have the exact connection with linear regression. We simply regress response y k {\displaystyle y_{k}} against the vector X k {\displaystyle X_{k}} . However, there is a concern about identifiability. In order to overcome such issues we assume that the sum of the parameters within each set of interactions is equal to zero. From here, one can use F-statistics or other methods to determine the relevance of the individual factors.
Example[edit]
We can consider the 2-way interaction example where we assume that the first factor has 2 levels and the second factor has 3 levels.
Define a i = 1 {\displaystyle a_{i}=1} if Z k , 1 = i {\displaystyle Z_{k,1}=i} and b i = 1 {\displaystyle b_{i}=1} if Z k , 2 = i {\displaystyle Z_{k,2}=i} , i.e. a {\displaystyle a} is the one-hot encoding of the first factor and b {\displaystyle b} is the one-hot encoding of the second factor.
With that,
X k = [ a 1 , a 2 , b 1 , b 2 , b 3 , a 1 × b 1 , a 1 × b 2 , a 1 × b 3 , a 2 × b 1 , a 2 × b 2 , a 2 × b 3 , 1 ] {\displaystyle X_{k}=[a_{1},a_{2},b_{1},b_{2},b_{3},a_{1}\times b_{1},a_{1}\times b_{2},a_{1}\times b_{3},a_{2}\times b_{1},a_{2}\times b_{2},a_{2}\times b_{3},1]}
where the last term is an intercept term. For a more concrete example suppose that
Z k , 1 = 2 Z k , 2 = 1 {\displaystyle {\begin{aligned}Z_{k,1}&=2\\Z_{k,2}&=1\end{aligned}}}
Then,
X k = [ 0 , 1 , 1 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 1 ] {\displaystyle X_{k}=[0,1,1,0,0,0,0,0,1,0,0,1]}
See also[edit]
Wikimedia Commons has media related to Analysis of variance.
AMOVA (analysis of molecular variance)
Analysis of covariance (ANCOVA)
ANORVA (analysis of rhythmic variance)
ANOVA on ranks
ANOVA-simultaneous component analysis
Explained variation
Mixed-design analysis of variance
Multivariate analysis of variance (MANOVA)
One-way analysis of variance
Permutational analysis of variance
Repeated measures ANOVA
Two-way analysis of variance
Variance decomposition
Footnotes[edit]
Jump up ^ Randomization is a term used in multiple ways in this material. "Randomization has three roles in applications: as a device for eliminating biases, for example from unobserved explanatory variables and selection effects: as a basis for estimating standard errors: and as a foundation for formally exact significance tests." Cox (2006, page 192) Hinkelmann and Kempthorne use randomization both in experimental design and for statistical analysis.
Jump up ^ Unit-treatment additivity is simply termed additivity in most texts. Hinkelmann and Kempthorne add adjectives and distinguish between additivity in the strict and broad senses. This allows a detailed consideration of multiple error sources (treatment, state, selection, measurement and sampling) on page 161.
Jump up ^ Rosenbaum (2002, page 40) cites Section 5.7 (Permutation Tests), Theorem 2.3 (actually Theorem 3, page 184) of Lehmann's Testing Statistical Hypotheses (1959).
Jump up ^ The F-test for the comparison of variances has a mixed reputation. It is not recommended as a hypothesis test to determine whether two different samples have the same variance. It is recommended for ANOVA where two estimates of the variance of the same sample are compared. While the F-test is not generally robust against departures from normality, it has been found to be robust in the special case of ANOVA. Citations from Moore & McCabe (2003): "Analysis of variance uses F statistics, but these are not the same as the F statistic for comparing two population standard deviations." (page 554) "The F test and other procedures for inference about variances are so lacking in robustness as to be of little use in practice." (page 556) "[The ANOVA F test] is relatively insensitive to moderate nonnormality and unequal variances, especially when the sample sizes are similar." (page 763) ANOVA assumes homoscedasticity, but it is robust. The statistical test for homoscedasticity (the F-test) is not robust. Moore & McCabe recommend a rule of thumb.
Notes[edit]
Jump up ^ Diez, David M; Barr, Christopher D; Cetinkaya-Rundel, Mine (2017). OpenIntro Statistics (3rd ed.). OpenIntro. Retrieved 11 November 2017.
Jump up ^ Stigler (1986)
Jump up ^ Stigler (1986, p 134)
Jump up ^ Stigler (1986, p 153)
Jump up ^ Stigler (1986, pp 154–155)
Jump up ^ Stigler (1986, pp 240–242)
Jump up ^ Stigler (1986, Chapter 7 - Psychophysics as a Counterpoint)
Jump up ^ Stigler (1986, p 253)
Jump up ^ Stigler (1986, pp 314–315)
Jump up ^ The Correlation Between Relatives on the Supposition of Mendelian Inheritance. Ronald A. Fisher. Philosophical Transactions of the Royal Society of Edinburgh. 1918. (volume 52, pages 399–433)
Jump up ^ On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample. Ronald A. Fisher. Metron, 1: 3-32 (1921)
Jump up ^ Scheffé (1959, p 291, "Randomization models were first formulated by Neyman (1923) for the completely randomized design, by Neyman (1935) for randomized blocks, by Welch (1937) and Pitman (1937) for the Latin square under a certain null hypothesis, and by Kempthorne (1952, 1955) and Wilk (1955) for many other designs.")
Jump up ^ Gelman (2005, p 2)
Jump up ^ Howell (2002, p 320)
Jump up ^ Montgomery (2001, p 63)
Jump up ^ Gelman (2005, p 1)
Jump up ^ Gelman (2005, p 5)
Jump up ^ "Section 5.7. A Glossary of DOE Terminology". NIST Engineering Statistics handbook. NIST. Retrieved 5 April 2012.
Jump up ^ "Section 4.3.1 A Glossary of DOE Terminology". NIST Engineering Statistics handbook. NIST. Retrieved 14 Aug 2012.
Jump up ^ Montgomery (2001, Chapter 12: Experiments with random factors)
Jump up ^ Gelman (2005, pp. 20–21)
Jump up ^ Snedecor, George W.; Cochran, William G. (1967). Statistical Methods (6th ed.). p. 321.
Jump up ^ Cochran & Cox (1992, p 48)
Jump up ^ Howell (2002, p 323)
Jump up ^ Anderson, David R.; Sweeney, Dennis J.; Williams, Thomas A. (1996). Statistics for business and economics (6th ed.). Minneapolis/St. Paul: West Pub. Co. pp. 452–453. ISBN 0-314-06378-1.
Jump up ^ Anscombe (1948)
Jump up ^ Kempthorne (1979, p 30)
^ Jump up to: a b Cox (1958, Chapter 2: Some Key Assumptions)
Jump up ^ Hinkelmann and Kempthorne (2008, Volume 1, Throughout. Introduced in Section 2.3.3: Principles of experimental design; The linear model; Outline of a model)
Jump up ^ Hinkelmann and Kempthorne (2008, Volume 1, Section 6.3: Completely Randomized Design; Derived Linear Model)
^ Jump up to: a b Hinkelmann and Kempthorne (2008, Volume 1, Section 6.6: Completely randomized design; Approximating the randomization test)
Jump up ^ Bailey (2008, Chapter 2.14 "A More General Model" in Bailey, pp. 38–40)
Jump up ^ Hinkelmann and Kempthorne (2008, Volume 1, Chapter 7: Comparison of Treatments)
Jump up ^ Kempthorne (1979, pp 125–126, "The experimenter must decide which of the various causes that he feels will produce variations in his results must be controlled experimentally. Those causes that he does not control experimentally, because he is not cognizant of them, he must control by the device of randomization." "[O]nly when the treatments in the experiment are applied by the experimenter using the full randomization procedure is the chain of inductive inference sound. It is only under these circumstances that the experimenter can attribute whatever effects he observes to the treatment and the treatment only. Under these circumstances his conclusions are reliable in the statistical sense.")
Jump up ^ Freedman[full citation needed]
Jump up ^ Montgomery (2001, Section 3.8: Discovering dispersion effects)
Jump up ^ Hinkelmann and Kempthorne (2008, Volume 1, Section 6.10: Completely randomized design; Transformations)
Jump up ^ Bailey (2008)
Jump up ^ Montgomery (2001, Section 3-3: Experiments with a single factor: The analysis of variance; Analysis of the fixed effects model)
Jump up ^ Cochran & Cox (1992, p 2 example)
Jump up ^ Cochran & Cox (1992, p 49)
Jump up ^ Hinkelmann and Kempthorne (2008, Volume 1, Section 6.7: Completely randomized design; CRD with unequal numbers of replications)
Jump up ^ Moore and McCabe (2003, page 763)
^ Jump up to: a b c Gelman (2008)
^ Jump up to: a b Montgomery (2001, Section 5-2: Introduction to factorial designs; The advantages of factorials)
Jump up ^ Belle (2008, Section 8.4: High-order interactions occur rarely)
Jump up ^ Montgomery (2001, Section 5-1: Introduction to factorial designs; Basic definitions and principles)
Jump up ^ Cox (1958, Chapter 6: Basic ideas about factorial experiments)
Jump up ^ Montgomery (2001, Section 5-3.7: Introduction to factorial designs; The two-factor factorial design; One observation per cell)
Jump up ^ Wilkinson (1999, p 596)
Jump up ^ Montgomery (2001, Section 3-7: Determining sample size)
Jump up ^ Howell (2002, Chapter 8: Power)
Jump up ^ Howell (2002, Section 11.12: Power (in ANOVA))
Jump up ^ Howell (2002, Section 13.7: Power analysis for factorial experiments)
Jump up ^ Moore and McCabe (2003, pp 778–780)
^ Jump up to: a b c Wilkinson (1999, p 599)
Jump up ^ Montgomery (2001, Section 3-4: Model adequacy checking)
Jump up ^ Moore and McCabe (2003, p 755, Qualifications to this rule appear in a footnote.)
Jump up ^ Montgomery (2001, Section 3-5.8: Experiments with a single factor: The analysis of variance; Practical interpretation of results; Comparing means with a control)
Jump up ^ Hinkelmann and Kempthorne (2008, Volume 1, Section 7.5: Comparison of Treatments; Multiple Comparison Procedures)
Jump up ^ Howell (2002, Chapter 12: Multiple comparisons among treatment means)
Jump up ^ Montgomery (2001, Section 3-5: Practical interpretation of results)
Jump up ^ Cochran & Cox (1957, p 9, "[T]he general rule [is] that the way in which the experiment is conducted determines not only whether inferences can be made, but also the calculations required to make them.")
Jump up ^ One-way/single factor ANOVA. Biomedical Statistics Archived 7 November 2014 at the Wayback Machine.
Jump up ^ "The Probable Error of a Mean". Biometrika. 6: 1–25. 1908. doi:10.1093/biomet/6.1.1.
Jump up ^ Montgomery (2001, Section 3-3.4: Unbalanced data)
Jump up ^ Montgomery (2001, Section 14-2: Unbalanced data in factorial design)
Jump up ^ Wilkinson (1999, p 600)
Jump up ^ Gelman (2005, p.1) (with qualification in the later text)
Jump up ^ Montgomery (2001, Section 3.9: The Regression Approach to the Analysis of Variance)
Jump up ^ Howell (2002, p 604)
Jump up ^ Howell (2002, Chapter 18: Resampling and nonparametric approaches to data)
Jump up ^ Montgomery (2001, Section 3-10: Nonparametric methods in the analysis of variance)
References[edit]
Anscombe, F. J. (1948). "The Validity of Comparative Experiments". Journal of the Royal Statistical Society. Series A (General). 111 (3): 181–211. doi:10.2307/2984159. JSTOR 2984159. MR 0030181.
Bailey, R. A. (2008). Design of Comparative Experiments. Cambridge University Press. ISBN 978-0-521-68357-9. Pre-publication chapters are available on-line.
Belle, Gerald van (2008). Statistical rules of thumb (2nd ed.). Hoboken, N.J: Wiley. ISBN 978-0-470-14448-0.
Cochran, William G.; Cox, Gertrude M. (1992). Experimental designs (2nd ed.). New York: Wiley. ISBN 978-0-471-54567-5.
Cohen, Jacob (1988). Statistical power analysis for the behavior sciences (2nd ed.). Routledge ISBN 978-0-8058-0283-2
Cohen, Jacob (1992). "Statistics a power primer". Psychological Bulletin. 112 (1): 155–159. doi:10.1037/0033-2909.112.1.155. PMID 19565683.
Cox, David R. (1958). Planning of experiments. Reprinted as ISBN 978-0-471-57429-3
Cox, D. R. (2006). Principles of statistical inference. Cambridge New York: Cambridge University Press. ISBN 978-0-521-68567-2.
Freedman, David A.(2005). Statistical Models: Theory and Practice, Cambridge University Press. ISBN 978-0-521-67105-7
Gelman, Andrew (2005). "Analysis of variance? Why it is more important than ever". The Annals of Statistics. 33: 1–53. doi:10.1214/009053604000001048.
Gelman, Andrew (2008). "Variance, analysis of". The new Palgrave dictionary of economics (2nd ed.). Basingstoke, Hampshire New York: Palgrave Macmillan. ISBN 978-0-333-78676-5.
Hinkelmann, Klaus & Kempthorne, Oscar (2008). Design and Analysis of Experiments. I and II (Second ed.). Wiley. ISBN 978-0-470-38551-7.
Howell, David C. (2002). Statistical methods for psychology (5th ed.). Pacific Grove, CA: Duxbury/Thomson Learning. ISBN 0-534-37770-X.
Kempthorne, Oscar (1979). The Design and Analysis of Experiments (Corrected reprint of (1952) Wiley ed.). Robert E. Krieger. ISBN 0-88275-105-0.
Lehmann, E.L. (1959) Testing Statistical Hypotheses. John Wiley & Sons.
Montgomery, Douglas C. (2001). Design and Analysis of Experiments (5th ed.). New York: Wiley. ISBN 978-0-471-31649-7.
Moore, David S. & McCabe, George P. (2003). Introduction to the Practice of Statistics (4e). W H Freeman & Co. ISBN 0-7167-9657-0
Rosenbaum, Paul R. (2002). Observational Studies (2nd ed.). New York: Springer-Verlag. ISBN 978-0-387-98967-9
Scheffé, Henry (1959). The Analysis of Variance. New York: Wiley.
Stigler, Stephen M. (1986). The history of statistics : the measurement of uncertainty before 1900. Cambridge, Mass: Belknap Press of Harvard University Press. ISBN 0-674-40340-1.
Wilkinson, Leland (1999). "Statistical Methods in Psychology Journals; Guidelines and Explanations". American Psychologist. 5 (8): 594–604. CiteSeerX 10.1.1.120.4818  . doi:10.1037/0003-066X.54.8.594.
Further reading[edit]
This article's further reading may not follow Wikipedia's content policies or guidelines. Please improve this article by removing less relevant or redundant publications with the same point of view; or by incorporating the relevant publications into the body of the article through appropriate citations. (November 2014) (Learn how and when to remove this template message)
Box, G. e. p. (1953). "Non-Normality and Tests on Variances". Biometrika. Biometrika Trust. 40 (3/4): 318–335. doi:10.1093/biomet/40.3-4.318. JSTOR 2333350.
Box, G. E. P. (1954). "Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification". The Annals of Mathematical Statistics. 25 (2): 290. doi:10.1214/aoms/1177728786.
Box, G. E. P. (1954). "Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, II. Effects of Inequality of Variance and of Correlation Between Errors in the Two-Way Classification". The Annals of Mathematical Statistics. 25 (3): 484. doi:10.1214/aoms/1177728717.
Caliński, Tadeusz; Kageyama, Sanpei (2000). Block designs: A Randomization approach, Volume I: Analysis. Lecture Notes in Statistics. 150. New York: Springer-Verlag. ISBN 0-387-98578-6.
Christensen, Ronald (2002). Plane Answers to Complex Questions: The Theory of Linear Models (Third ed.). New York: Springer. ISBN 0-387-95361-2.
Cox, David R. & Reid, Nancy M. (2000). The theory of design of experiments. (Chapman & Hall/CRC). ISBN 978-1-58488-195-7
Fisher, Ronald (1918). "Studies in Crop Variation. I. An examination of the yield of dressed grain from Broadbalk" (PDF). Journal of Agricultural Science. 11: 107–135. doi:10.1017/S0021859600003750. Archived from the original (PDF) on 12 June 2001.
Freedman, David A.; Pisani, Robert; Purves, Roger (2007) Statistics, 4th edition. W.W. Norton & Company ISBN 978-0-393-92972-0
Hettmansperger, T. P.; McKean, J. W. (1998). Edward Arnold, ed. Robust nonparametric statistical methods. Kendall's Library of Statistics. Volume 5 (First ed.). New York: John Wiley & Sons, Inc. pp. xiv+467 pp. ISBN 0-340-54937-8. MR 1604954.
Lentner, Marvin; Thomas Bishop (1993). Experimental design and analysis (Second ed.). P.O. Box 884, Blacksburg, VA 24063: Valley Book Company. ISBN 0-9616255-2-X.
Tabachnick, Barbara G. & Fidell, Linda S. (2007). Using Multivariate Statistics (5th ed.). Boston: Pearson International Edition. ISBN 978-0-205-45938-4
Wichura, Michael J. (2006). The coordinate-free approach to linear models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press. pp. xiv+199. ISBN 978-0-521-86842-6. MR 2283455.
Phadke, Madhav S. (1989). Quality Engineering using Robust Design. New Jersey: Prentice Hall PTR. ISBN 0-13-745167-9.
External links[edit]
Wikiversity has learning resources about Analysis of variance
SOCR ANOVA Activity and interactive applet.
Examples of all ANOVA and ANCOVA models with up to three treatment factors, including randomized block, split plot, repeated measures, and Latin squares, and their analysis in R (University of Southampton)
NIST/SEMATECH e-Handbook of Statistical Methods, section 7.4.3: "Are the means equal?"
Analysis of variance: Introduction
[show]
v
t
e
Statistics
Outline
Index
[show]
Descriptive statistics
Continuous data
Center
Mean
arithmetic
geometric
harmonic
Median
Mode
Dispersion
Variance
Standard deviation
Coefficient of variation
Percentile
Range
Interquartile range
Shape
Central limit theorem
Moments
Skewness
Kurtosis
L-moments
Count data
Index of dispersion
Summary tables
Grouped data
Frequency distribution
Contingency table
Dependence
Pearson product-moment correlation
Rank correlation
Spearman's rho
Kendall's tau
Partial correlation
Scatter plot
Graphics
Bar chart
Biplot
Box plot
Control chart
Correlogram
Fan chart
Forest plot
Histogram
Pie chart
Q–Q plot
Run chart
Scatter plot
Stem-and-leaf display
Radar chart
[show]
Data collection
Study design
Population
Statistic
Effect size
Statistical power
Sample size determination
Missing data
Survey methodology
Sampling
stratified
cluster
Standard error
Opinion poll
Questionnaire
Controlled experiments
Design
control
optimal
Controlled trial
Randomized
Random assignment
Replication
Blocking
Interaction
Factorial experiment
Uncontrolled studies
Observational study
Natural experiment
Quasi-experiment
[show]
Statistical inference
Statistical theory
Population
Statistic
Probability distribution
Sampling distribution
Order statistic
Empirical distribution
Density estimation
Statistical model
Lp space
Parameter
location
scale
shape
Parametric family
Likelihood (monotone)
Location–scale family
Exponential family
Completeness
Sufficiency
Statistical functional
Bootstrap
U
V
Optimal decision
loss function
Efficiency
Statistical distance
divergence
Asymptotics
Robustness
Frequentist inference
Point estimation
Estimating equations
Maximum likelihood
Method of moments
M-estimator
Minimum distance
Unbiased estimators
Mean-unbiased minimum-variance
Rao–Blackwellization
Lehmann–Scheffé theorem
Median unbiased
Plug-in
Interval estimation
Confidence interval
Pivot
Likelihood interval
Prediction interval
Tolerance interval
Resampling
Bootstrap
Jackknife
Testing hypotheses
1- & 2-tails
Power
Uniformly most powerful test
Permutation test
Randomization test
Multiple comparisons
Parametric tests
Likelihood-ratio
Wald
Score
Specific tests
Z (normal)
Student's t-test
F
Goodness of fit
Chi-squared
Kolmogorov–Smirnov
Anderson–Darling
Lilliefors
Jarque–Bera
Normality (Shapiro–Wilk)
Likelihood-ratio test
Model selection
Cross validation
AIC
BIC
Rank statistics
Sign
Sample median
Signed rank (Wilcoxon)
Hodges–Lehmann estimator
Rank sum (Mann–Whitney)
Nonparametric anova
1-way (Kruskal–Wallis)
2-way (Friedman)
Ordered alternative (Jonckheere–Terpstra)
Bayesian inference
Bayesian probability
prior
posterior
Credible interval
Bayes factor
Bayesian estimator
Maximum posterior estimator
[hide]
Correlation
Regression analysis
Correlation
Pearson product-moment
Partial correlation
Confounding variable
Coefficient of determination
Regression analysis
Errors and residuals
Regression model validation
Mixed effects models
Simultaneous equations models
Multivariate adaptive regression splines (MARS)
Linear regression
Simple linear regression
Ordinary least squares
General linear model
Bayesian regression
Non-standard predictors
Nonlinear regression
Nonparametric
Semiparametric
Isotonic
Robust
Heteroscedasticity
Homoscedasticity
Generalized linear model
Exponential families
Logistic (Bernoulli) / Binomial / Poisson regressions
Partition of variance
Analysis of variance (ANOVA, anova)
Analysis of covariance
Multivariate ANOVA
Degrees of freedom
[show]
Categorical / Multivariate / Time-series / Survival analysis
Categorical
Cohen's kappa
Contingency table
Graphical model
Log-linear model
McNemar's test
Multivariate
Regression
Manova
Principal components
Canonical correlation
Discriminant analysis
Cluster analysis
Classification
Structural equation model
Factor analysis
Multivariate distributions
Elliptical distributions
Normal
Time-series
General
Decomposition
Trend
Stationarity
Seasonal adjustment
Exponential smoothing
Cointegration
Structural break
Granger causality
Specific tests
Dickey–Fuller
Johansen
Q-statistic (Ljung–Box)
Durbin–Watson
Breusch–Godfrey
Time domain
Autocorrelation (ACF)
partial (PACF)
Cross-correlation (XCF)
ARMA model
ARIMA model (Box–Jenkins)
Autoregressive conditional heteroskedasticity (ARCH)
Vector autoregression (VAR)
Frequency domain
Spectral density estimation
Fourier analysis
Wavelet
Survival
Survival function
Kaplan–Meier estimator (product limit)
Proportional hazards models
Accelerated failure time (AFT) model
First hitting time
Hazard function
Nelson–Aalen estimator
Test
Log-rank test
[show]
Applications
Biostatistics
Bioinformatics
Clinical trials / studies
Epidemiology
Medical statistics
Engineering statistics
Chemometrics
Methods engineering
Probabilistic design
Process / quality control
Reliability
System identification
Social statistics
Actuarial science
Census
Crime statistics
Demography
Econometrics
National accounts
Official statistics
Population statistics
Psychometrics
Spatial statistics
Cartography
Environmental statistics
Geographic information system
Geostatistics
Kriging
Category
Portal
Commons
WikiProject
[show]
v
t
e
Design of experiments
Scientific
method
Scientific experiment
Statistical design
Control
Internal and external validity
Experimental unit
Blinding
Optimal design: Bayesian
Random assignment
Randomization
Restricted randomization
Replication versus subsampling
Sample size
Treatment
and blocking
Treatment
Effect size
Contrast
Interaction
Confounding
Orthogonality
Blocking
Covariate
Nuisance variable
Models
and inference
Linear regression
Ordinary least squares
Bayesian
Random effect
Mixed model
Hierarchical model: Bayesian
Analysis of variance (Anova)
Cochran's theorem
Manova (multivariate)
Ancova (covariance)
Compare means
Multiple comparison
Designs

Completely
randomized
Factorial
Fractional factorial
Plackett-Burman
Taguchi
Response surface methodology
Polynomial and rational modeling
Box-Behnken
Central composite
Block
Generalized randomized block design (GRBD)
Latin square
Graeco-Latin square
Orthogonal array
Latin hypercube
Repeated measures design
Crossover study
Randomized controlled trial
Sequential analysis
Sequential probability ratio test
Glossary
Category
Statistics portal
Statistical outline
Statistical topics
Statistics portal
[show]
v
t
e
Public health
General
Auxology
Biological hazard
Chief Medical Officer
Cultural competence
Deviance
Environmental health
Euthenics
Genomics
Globalization and disease
Health economics
Health literacy
Health policy
Health system
Health care reform
Public health law
Maternal health
Medical anthropology
Medical sociology
Mental health
Pharmaceutical policy
Public health laboratory
Reproductive health
Social psychology
Sociology of health and illness
Preventive healthcare
Behavior change
Theories
Family planning
Health promotion
Human nutrition
Preventive nutrition
Hygiene
Food safety
Hand washing
Infection control
Oral hygiene
Occupational safety and health
Human factors and ergonomics
Hygiene
Injury prevention
Medicine
Nursing
Patient safety
Organization
Pharmacovigilance
Safe sex
Sanitation
Emergency
Fecal–oral transmission
Open defecation
Sanitary sewer
Waterborne diseases
Smoking cessation
Vaccination
Vector control
Population health
Biostatistics
Child mortality
Community health
Epidemiology
Global health
Health impact assessment
Health system
Infant mortality
Open-source healthcare software
Public health informatics
Social determinants of health
Health equity
Race and health
Social medicine
Biological and
epidemiological statistics
Case-control study
Randomized controlled trial
Relative risk
Statistical hypothesis testing
Analysis of variance (ANOVA)
Regression analysis
ROC curve
Student's t-test
Z-test
Statistical software
Infectious and epidemic
disease prevention
Asymptomatic carrier
Epidemics
List
Notifiable diseases
List
Public health surveillance
Disease surveillance
Quarantine
Sexually transmitted infection
Tropical disease
Vaccine trial
Food hygiene and
safety management
Food
Additive
Chemistry
Engineering
Microbiology
Processing
Safety
Safety scandals
Genetically modified food
Good agricultural practice
Good manufacturing practice
HACCP
ISO 22000
Health behavioral
sciences
Diffusion of innovations
Health belief model
Health communication
Health psychology
Positive deviance
PRECEDE-PROCEED model
Social cognitive theory
Social norms approach
Theory of planned behavior
Transtheoretical model
Organizations,
education
and history
Organizations
Europe
Centre for Disease Prevention and Control
Committee on the Environment, Public Health and Food Safety
India
Ministry of Health and Family Welfare
U.S.
Centers for Disease Control and Prevention
City and county health departments
Council on Education for Public Health
Public Health Service
World Health Organization
World Toilet Organization
Education
Health education
Higher education
Bachelor of Science in Public Health
Doctor of Public Health
Professional degrees of public health
Schools of public health
History
Sara Josephine Baker
Samuel Jay Crumbine
Carl Rogers Darnall
Joseph Lister
Margaret Sanger
John Snow
Typhoid Mary
Germ theory of disease
Social hygiene movement
Category
Commons
WikiProject
Retrieved from "https://en.wikipedia.org/w/index.php?title=Analysis_of_variance&oldid=809838199"
Categories:
Analysis of variance
Design of experiments
Statistical tests
Parametric statistics
Hidden categories:
Articles needing more detailed references
Webarchive template wayback links
Use dmy dates from June 2013
All articles with unsourced statements
Articles with unsourced statements from August 2011
Articles with unsourced statements from October 2013
All pages needing factual verification
Wikipedia articles needing factual verification from December 2014
Articles with unsourced statements from May 2011
Wikipedia spam cleanup from November 2014
Wikipedia further reading cleanup
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Article
Talk
Variants
Views
Read
Edit
View history
More
Search
Navigation
Main page
Contents
Featured content
Current events
Random article
Donate to Wikipedia
Wikipedia store
Interaction
Help
About Wikipedia
Community portal
Recent changes
Contact page
Tools
What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Wikidata item
Cite this page
Print/export
Create a book
Download as PDF
Printable version
In other projects
Wikimedia Commons
Languages
العربية
Azərbaycanca
Български
Català
Čeština
Deutsch
Eesti
Ελληνικά
Español
Euskara
فارسی
Français
Galego
ગુજરાતી
한국어
Հայերեն
हिन्दी
Bahasa Indonesia
Italiano
עברית
Basa Jawa
Latviešu
Magyar
Bahasa Melayu
Nederlands
日本語
Norsk
Norsk nynorsk
Occitan
Polski
Português
Русский
Slovenščina
Basa Sunda
Svenska
தமிழ்
తెలుగు
Тоҷикӣ
Türkçe
Українська
中文
Edit links
This page was last edited on 11 November 2017, at 19:34.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.
Privacy policy
About Wikipedia
Disclaimers
Contact Wikipedia
Developers
Cookie statement
Mobile view
Enable previews

key: html

type: local

value: div

attrs: id class

values: mw-page-base noprint

values: mw-head-base noprint

value: a

attrs: id class lang

values: firstHeading firstHeading en

values: siteSub noprint

values: jump-to-nav mw-jump

value: b

attrs: href title

attrs: href class title

values:
/wiki/Statistical_test
mw-redirect
Statistical test

value: i

values:
/wiki/Type_I_error
mw-redirect
Type I error

values: Statistical Methods for Research Workers

attrs: alt src width height class srcset data-file-width data-file-height

values: Edit section: Background and terminology

values:
/wiki/Type_I_errors
mw-redirect
Type I errors

values:
/wiki/Type_II_errors
mw-redirect
Type II errors

values:
/wiki/Blind_experiment
mw-redirect
Blind experiment

values: Edit section: Design-of-experiments terms

values:
/wiki/Response_variable
mw-redirect
Response variable

values: Edit section: Textbook analysis using a normal distribution

values:
/wiki/Statistical_independence
mw-redirect
Statistical independence

values:
/wiki/Residual_(statistics)
mw-redirect
Residual (statistics)

values: vertical-align: -0.338ex; width:1.094ex; height:1.676ex;

values: vertical-align: -0.838ex; width:13.771ex; height:3.176ex;

values: Edit section: Randomization-based analysis

values:
/wiki/Randomization_test
mw-redirect
Randomization test

values:
/wiki/Francis_J._Anscombe
mw-redirect
Francis J. Anscombe

values:
/wiki/Rothamsted_Experimental_Station
mw-redirect
Rothamsted Experimental Station

values:
/wiki/David_R._Cox
mw-redirect
David R. Cox

values: This claim needs references to reliable sources. (August 2011)

values:
noprint Inline-Template Template-Fact
white-space:nowrap;

values: vertical-align: -1.005ex; width:3.106ex; height:2.343ex;

values: vertical-align: -0.338ex; width:0.813ex; height:2.176ex;

values: vertical-align: -0.671ex; margin-left: -0.016ex; width:0.985ex; height:2.509ex;

values: vertical-align: -0.671ex; width:1.957ex; height:2.009ex;

values: vertical-align: -1.005ex; width:1.767ex; height:2.676ex;

values: vertical-align: -1.005ex; width:13.447ex; height:2.676ex;

values: Edit section: Statistical models for observational data

values: This claim needs references to reliable sources. (October 2013)

values: Edit section: Partitioning of the sum of squares

values: vertical-align: -1.338ex; width:21.08ex; height:3.676ex;

values:
/wiki/Sum_of_squares_(statistics)
mw-redirect
Sum of squares (statistics)

values: vertical-align: -0.671ex; width:31.765ex; height:2.509ex;

values: SS_\text{Total} = SS_\text{Error} + SS_\text{Treatments}

values: vertical-align: -0.671ex; width:33.25ex; height:2.509ex;

values: DF_\text{Total} = DF_\text{Error} + DF_\text{Treatments}

values: vertical-align: -2.005ex; width:34.849ex; height:5.509ex;

values: F = \frac{\text{variance between treatments}}{\text{variance within treatments}}

values: vertical-align: -2.671ex; width:41.942ex; height:6.509ex;

values: F = \frac{MS_\text{Treatments}}{MS_\text{Error}} = {{SS_\text{Treatments} / (I-1)} \over {SS_\text{Error} / (n_T-I)}}

values: vertical-align: -0.338ex; width:1.182ex; height:2.176ex;

values: vertical-align: -0.671ex; width:2.802ex; height:2.009ex;

values: vertical-align: -0.338ex; width:5.206ex; height:2.176ex;

values: vertical-align: -0.671ex; width:6.835ex; height:2.509ex;

values: vertical-align: -1.005ex; width:21.396ex; height:3.176ex;

values:
1 + {n \sigma^2_\text{Treatment}} /
{\sigma^2_\text{Error}}

values:
/wiki/Permutation_test
mw-redirect
Permutation test

values: The material near this tag needs to be fact-checked with the cited source(s). (December 2014)

values: Edit section: The number of experimental units

values:
/wiki/Post-hoc_analysis
mw-redirect
Post-hoc analysis

values:
/wiki/Experimental_design
mw-redirect
Experimental design

values: This claim needs references to reliable sources. (May 2011)

values:
/wiki/One-way_ANOVA
mw-redirect
One-way ANOVA

values:
/wiki/Repeated_measures
mw-redirect
Repeated measures

values:
/wiki/Dependent_variable
mw-redirect
Dependent variable

values:
/wiki/Multiple_comparisons
mw-redirect
Multiple comparisons

values:
/wiki/Kruskal%E2%80%93Wallis_test
mw-redirect
Kruskal–Wallis test

values: Edit section: Connection to linear regression

values: vertical-align: -0.338ex; width:3.022ex; height:2.676ex;

values: vertical-align: -0.671ex; width:2.246ex; height:2.009ex;

values: vertical-align: -1.005ex; width:3.872ex; height:2.843ex;

values: vertical-align: -0.838ex; width:16.579ex; height:2.843ex;

values: vertical-align: -0.338ex; width:1.774ex; height:2.176ex;

values: vertical-align: -0.338ex; width:6.056ex; height:2.176ex;

values: vertical-align: -0.338ex; width:2.796ex; height:2.676ex;

values: vertical-align: -0.671ex; width:1.979ex; height:2.509ex;

values: vertical-align: -3.171ex; width:5.731ex; height:7.509ex;

values: vertical-align: -0.671ex; width:2.234ex; height:2.009ex;

values: vertical-align: -0.838ex; width:17.057ex; height:3.176ex;

values: vertical-align: -0.338ex; width:2.6ex; height:2.676ex;

values: vertical-align: -1.005ex; width:7.766ex; height:3.009ex;

attrs: columnalign rowspacing columnspacing displaystyle

values: vertical-align: -2.505ex; width:27.437ex; height:6.176ex;

values: vertical-align: -0.338ex; width:1.008ex; height:2.176ex;

values: vertical-align: -1.005ex; width:37.761ex; height:3.009ex;

values: vertical-align: -0.671ex; width:2.436ex; height:2.009ex;

values: vertical-align: -0.671ex; width:3.031ex; height:2.509ex;

values: vertical-align: -0.671ex; width:6.329ex; height:2.509ex;

values: vertical-align: -1.005ex; width:7.91ex; height:2.843ex;

values: vertical-align: -0.671ex; width:6.097ex; height:2.509ex;

values: vertical-align: -0.338ex; width:1.24ex; height:1.676ex;

values: vertical-align: -0.838ex; width:74.262ex; height:2.843ex;

values: right left right left right left right left right left right left

values: 0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em

values: vertical-align: -2.505ex; width:9.021ex; height:6.176ex;

values: vertical-align: -0.838ex; width:33.018ex; height:2.843ex;

values:
/wiki/Variance_decomposition
mw-redirect
Variance decomposition

values: -moz-column-width: 20em; -webkit-column-width: 20em; column-width: 20em;

values:
reflist mw-references-wrap
list-style-type: decimal;

values: A more detailed citation is required. (November 2012)

values:
nofollow
external text
//doi.org/10.1093%2Fbiomet%2F6.1.1

values: -moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;

values: Journal of the Royal Statistical Society

values:
nofollow
external text
//doi.org/10.2307%2F2984159

values:
nofollow
external text
//www.jstor.org/stable/2984159

values:
/wiki/PubMed_Identifier
mw-redirect
PubMed Identifier

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/19565683

values:
/wiki/David_A._Freedman_(statistician)
mw-redirect
David A. Freedman (statistician)

attrs: alt src title width height srcset data-file-width data-file-height

attrs: alt src width height srcset data-file-width data-file-height

values:
plainlinks metadata ambox ambox-style
presentation

values:
nofollow
external text
//doi.org/10.1093%2Fbiomet%2F40.3-4.318

values:
nofollow
external text
//www.jstor.org/stable/2333350

values:
nofollow
external text
//doi.org/10.1214%2Faoms%2F1177728786

values:
nofollow
external text
//doi.org/10.1214%2Faoms%2F1177728717

values:
/wiki/Nancy_M._Reid
mw-redirect
Nancy M. Reid

values:
/wiki/Shape_of_the_distribution
mw-redirect
Shape of the distribution

values: Pearson product-moment correlation coefficient

values: Kendall tau rank correlation coefficient

values: nowraplinks collapsible collapsed navbox-subgroup

values:
/wiki/Population_(statistics)
mw-redirect
Population (statistics)

values:
/wiki/Maximum_likelihood
mw-redirect
Maximum likelihood

values:
/wiki/Median-unbiased_estimator
mw-redirect
Median-unbiased estimator

values:
/wiki/Likelihood_interval
mw-redirect
Likelihood interval

values:
/wiki/Power_(statistics)
mw-redirect
Power (statistics)

values:
/wiki/Rank_statistics
mw-redirect
Rank statistics

values:
/wiki/Sample_median
mw-redirect
Sample median

values: Kruskal–Wallis one-way analysis of variance

values:
CorrelationRegression_analysis
font-size:114%;margin:0 4em
hlist hlist-separated

values:
/wiki/Regression_model_validation
mw-redirect
Regression model validation

values: Multivariate adaptive regression splines

values: nowraplinks collapsible uncollapsed navbox-subgroup

values:
/wiki/Multivariate_distribution
mw-redirect
Multivariate distribution

values:
/wiki/Trend_estimation
mw-redirect
Trend estimation

values: List of fields of application of statistics

values: nowraplinks hlist collapsible collapsed navbox-inner

values: navigation navbox Statistics padding:3px

values:
/wiki/Experimental_unit
mw-redirect
Experimental unit

values:
/wiki/Sample_size
mw-redirect
Sample size

values:
/wiki/Random_effect
mw-redirect
Random effect

values:
/wiki/Hierarchical_linear_modeling
mw-redirect
Hierarchical linear modeling

values:
/wiki/Hierarchical_Bayes_model
mw-redirect
Hierarchical Bayes model

values:
/wiki/Comparing_means
mw-redirect
Comparing means

values:
/wiki/Multiple_comparison
mw-redirect
Multiple comparison

values:
/wiki/Plackett-Burman_design
mw-redirect
Plackett-Burman design

values: Polynomial and rational function modeling

values:
noprint metadata navbox
navigation
Portals
font-weight:bold;padding:0.4em 2em

values:
/wiki/PRECEDE-PROCEED_model
mw-redirect
PRECEDE-PROCEED model

values: European Centre for Disease Prevention and Control

values: European Parliament Committee on the Environment, Public Health and Food Safety

values: Centers for Disease Control and Prevention

values: Category:City and county health departments in the United States

values:
/wiki/Joseph_Lister,_1st_Baron_Lister
mw-redirect
Joseph Lister, 1st Baron Lister

values:
/wiki/John_Snow_(physician)
mw-redirect
John Snow (physician)

values: nowraplinks hlist collapsible autocollapse navbox-inner

values: Category:Articles needing more detailed references

values: Category:Webarchive template wayback links

values: Category:All articles with unsourced statements

values: Category:Articles with unsourced statements from August 2011

values: Category:Articles with unsourced statements from October 2013

values: Category:All pages needing factual verification

values: Category:Wikipedia articles needing factual verification from December 2014

values: Category:Articles with unsourced statements from May 2011

values: Category:Wikipedia spam cleanup from November 2014

values: Category:Wikipedia further reading cleanup

values: Discussion about edits from this IP address [ctrl-alt-n]

values: A list of edits made from this IP address [ctrl-alt-y]

values: You are encouraged to create an account and log in; however, it is not mandatory

values: You're encouraged to log in; however, it's not mandatory. [ctrl-alt-o]

values:
/wiki/Analysis_of_variance
View the content page [ctrl-alt-c]
c

values: Discussion about the content page [ctrl-alt-t]

values: p-namespaces navigation vectorTabs p-namespaces-label

values:
p-variants
navigation
vectorMenu emptyPortlet
p-variants-label

values: Past revisions of this page [ctrl-alt-h]

values: p-views navigation vectorTabs p-views-label

values:
p-cactions
navigation
vectorMenu emptyPortlet
p-cactions-label

values:
mw-wiki-logo
/wiki/Main_Page
Visit the main page

values: A list of all special pages [ctrl-alt-q]

values: Permanent link to this revision of the page

values: Link to connected data repository item [ctrl-alt-g]

values: Printable version of this page [ctrl-alt-p]

values: portal navigation p-wikibase-otherprojects p-wikibase-otherprojects-label

values:
uls-settings-trigger
Language settings
0
button
true

values: تحليل التباين – Arabic

values: Дисперсионен анализ – Bulgarian

values: Ανάλυση διακύμανσης – Greek

values: تحلیل واریانس – Persian

values: અંતરનું વિશ્લેષણ – Gujarati

values: Դիսպերս վերլուծություն – Armenian

values: भिन्नता का विश्लेषण – Hindi

values: ניתוח שונות – Hebrew

values: Дисперсионный анализ – Russian

values: மாறும் அளவுப் பகுப்பாய்வு – Tamil

values: అంతర్భేధం యొక్క విశ్లేషణము – Telugu

values: Таҳлили варионс – Tajik

values: Дисперсійний аналіз – Ukrainian

values: /static/images/wikimedia-button-1.5x.png 1.5x, /static/images/wikimedia-button-2x.png 2x

values: /static/images/poweredby_mediawiki_132x47.png 1.5x, /static/images/poweredby_mediawiki_176x62.png 2x

values: mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Analysis_of_variance rootpage-Analysis_of_variance vector-nav-directionality skin-vector action-view


text:
Template:Largest Metropolitan Areas of Canada
From Wikipedia, the free encyclopedia
Jump to: navigation, search
Largest census metropolitan areas in Canada by population (2016 Census)
view
talk
edit
CMA
Province
Population
CMA
Province
Population
Toronto Ontario 5,928,040 London Ontario 494,069
Montreal Quebec 4,098,927 St. Catharines–Niagara Ontario 406,074
Vancouver British Columbia 2,463,431 Halifax Nova Scotia 403,390
Calgary Alberta 1,392,609 Oshawa Ontario 379,848
Ottawa–Gatineau Ontario–Quebec 1,323,783 Victoria British Columbia 367,770
Edmonton Alberta 1,321,426 Windsor Ontario 329,144
Quebec Quebec 800,296 Saskatoon Saskatchewan 295,095
Winnipeg Manitoba 778,489 Regina Saskatchewan 236,481
Hamilton Ontario 747,545 Sherbrooke Quebec 212,105
Kitchener–Cambridge–Waterloo Ontario 523,894 St. John's Newfoundland and Labrador 205,955
References
These references will appear in the article, but this list appears only on this page.

Retrieved from "https://en.wikipedia.org/w/index.php?title=Template:Largest_Metropolitan_Areas_of_Canada&oldid=822209552"
Categories:
Largest metropolitan areas of North America templates
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Template
Talk
Variants
Views
Read
Edit
View history
More
Search
Navigation
Main page
Contents
Featured content
Current events
Random article
Donate to Wikipedia
Wikipedia store
Interaction
Help
About Wikipedia
Community portal
Recent changes
Contact page
Tools
What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Wikidata item
Print/export
Download as PDF
Languages
မြန်မာဘာသာ
Türkçe
Tiếng Việt
中文
Edit links
This page was last edited on 25 January 2018, at 01:22.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.
Privacy policy
About Wikipedia
Disclaimers
Contact Wikipedia
Developers
Cookie statement
Mobile view
Enable previews

values: mw-page-base noprint

values: mw-head-base noprint

values: firstHeading firstHeading en

values: siteSub noprint

values: jump-to-nav mw-jump

values: List of census metropolitan areas and agglomerations in Canada

value: span

value: li

values: Template talk:Largest Metropolitan Areas of Canada

attrs: class href

value: ul

attrs: class style

values:
plainlinks hlist navbar
float:right; margin-right:0.75em

value: th

value: tr

values:
/wiki/Census_metropolitan_area
mw-redirect
Census metropolitan area

values: Category:Largest metropolitan areas of North America templates

values:
catlinks mw-normal-catlinks
catlinks mw-normal-catlinks
interface

values: Discussion about edits from this IP address [ctrl-alt-n]

values: A list of edits made from this IP address [ctrl-alt-y]

values: You are encouraged to create an account and log in; however, it is not mandatory

values: You're encouraged to log in; however, it's not mandatory. [ctrl-alt-o]

values:
/wiki/Main_Page
Visit the main page [ctrl-alt-z]
z

values: Featured content – the best of Wikipedia

values: Find background information on current events

values:
/wiki/Special:Random
Load a random article [ctrl-alt-x]
x

values: portal navigation p-navigation p-navigation-label

values: Guidance on how to use and edit Wikipedia

values: About the project, what you can do, where to find things

values: A list of recent changes in the wiki [ctrl-alt-r]

values: portal navigation p-interaction p-interaction-label

values: List of all English Wikipedia pages containing links to this page [ctrl-alt-j]

values: Recent changes in pages linked from this page [ctrl-alt-k]

values:
/wiki/Wikipedia:File_Upload_Wizard
Upload files [ctrl-alt-u]
u

values: Template:ကနေဒါနိုင်ငံ၏ အကြီးဆုံးသော မက်ထရိုပိုလစ်တန် ဒေသများ – Burmese

values: Şablon:Kanada'nın en büyük metropoliten alanları – Turkish

values: Bản mẫu:Largest Metropolitan Areas of Canada – Vietnamese

values:
suggestions
display: none; font-size: 13px;

values: mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-10 ns-subject page-Template_Largest_Metropolitan_Areas_of_Canada rootpage-Template_Largest_Metropolitan_Areas_of_Canada skin-vector action-view


text:
I Don't Ever Want to See You Again
From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
"I Don't Ever Want to See You Again"
Single by Uncle Sam
from the album Uncle Sam
B-side
"Can You Feel It"[1]
Released
October 21, 1997 (1997-10-21)[2]
Format
12", CD, cassette
Genre
R&B
Length
5:43 (album version)
4:12 (single edit)[1]
Label
Epic
Songwriter(s)
Nathan Morris[3]
Producer(s)
Nathan Morris[1]
Uncle Sam singles chronology
"I Don't Ever Want to See You Again"
(1997) "Can You Feel It"
(1997)
"I Don't Ever Want to See You Again"
(1997) "Can You Feel It"
(1997)
Music video
"I Don't Ever Want to See You Again" on YouTube
"I Don't Ever Want to See You Again" is a song performed by American contemporary R&B singer Uncle Sam. It is the closing track on his eponymous debut album and was issued as the album's lead single. The song was written[3] and produced[1] by Boyz II Men member Nathan Morris. Released in 1997,[2] it was Sam's only hit on the Billboard Hot 100, peaking at #6 in 1998.[4]
The single was certified platinum on April 6, 1998.[5]
Contents
1 Music video
2 Chart positions
2.1 Weekly charts
2.2 Year-end-charts
3 References
4 External links
Music video[edit]
This section needs expansion. You can help by adding to it. (February 2015)
The official music video for the song was directed by Christopher Erskin and Nathan Morris.[6]
Chart positions[edit]
Weekly charts[edit]
Chart (1998)[4]
Peak
position
US Billboard Hot 100 6
US Hot R&B/Hip-Hop Singles & Tracks (Billboard) 2
US Rhythmic Top 40 (Billboard) 3
US Top 40 Mainstream (Billboard) 28
Year-end-charts[edit]
Chart (1998)
Position
US Billboard Hot 100[7] 26
References[edit]
^ Jump up to: a b c d "Uncle Sam (4) - I Don't Ever Want To See You Again (CD) at Discogs". Discogs. Zink Media. Retrieved February 19, 2015.
^ Jump up to: a b "I Don't Ever Want to See You Again - Uncle Sam | Release Credits | AllMusic". AllMusic. All Media Network. Retrieved February 19, 2015.
^ Jump up to: a b "I Don't Ever Want to See You Again - Uncle Sam | Listen, Appearances, Song Review | AllMusic". AllMusic. All Media Network. Retrieved February 19, 2015.
^ Jump up to: a b "Uncle Sam | Awards | AllMusic". AllMusic. All Media Network. Retrieved February 19, 2015.
Jump up ^ "RIAA - Gold & Platinum Searchable Database - February 20, 2015". Recording Industry Association of America. Retrieved February 19, 2015.
Jump up ^ Garcia, Alex S. "mvdbase.com - Uncle Sam - "I don't ever want to see you again"". Music Video DataBase. Retrieved February 19, 2015.
Jump up ^ Pedro. "Longbored Surfer - 1998". Longbored Surfer. Retrieved February 19, 2015.
External links[edit]
"I Don't Ever Want to See You Again" at AllMusic
"I Don't Ever Want to See You Again" at Discogs (list of releases)
Lyrics of this song at MetroLyrics
Retrieved from "https://en.wikipedia.org/w/index.php?title=I_Don%27t_Ever_Want_to_See_You_Again&oldid=832705625"
Categories:
1997 singles
Debut singles
Epic Records singles
Music videos directed by Christopher Erskin
Song recordings produced by Nathan Morris
Songs written by Nathan Morris
Songs about infidelity
Uncle Sam (singer) songs
1997 songs
Hidden categories:
Articles which use infobox templates with no data rows
Music infoboxes with deprecated parameters
Articles with hAudio microformats
Articles to be expanded from February 2015
All articles to be expanded
Articles using small message boxes
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Article
Talk
Variants
Views
Read
Edit
View history
More
Search
Navigation
Main page
Contents
Featured content
Current events
Random article
Donate to Wikipedia
Wikipedia store
Interaction
Help
About Wikipedia
Community portal
Recent changes
Contact page
Tools
What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Wikidata item
Cite this page
Print/export
Create a book
Download as PDF
Printable version
Languages
Add links
This page was last edited on 27 March 2018, at 15:10 (UTC).
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.
Privacy policy
About Wikipedia
Disclaimers
Contact Wikipedia
Developers
Cookie statement
Mobile view
Enable previews

values: mw-page-base noprint

values: mw-head-base noprint

values: firstHeading firstHeading en

values: siteSub noprint

values: mw-jump-link #mw-head

values: mw-jump-link #p-search

attrs: colspan class style

value: img

attrs: href class

value: td

attrs: colspan style

values:
/wiki/12-inch_single
mw-redirect
12-inch single

values: width: 33%; text-align: center; vertical-align: top; padding: .2em .1em .2em 0;

values: width: 33%; text-align: center; vertical-align: top; padding:.2em .1em;

values: width: 33%; text-align: center; vertical-align: top; padding: .2em 0 .2em .1em;

values: background: transparent; width: 100%; min-width: 100%; border-collapse: collapse;

values: width:33%; padding:.2em .1em .2em 0; text-align: center;

values: width:33%; padding:.2em .1em; text-align: center;

values: width:33%; padding:.2em 0 .2em .1em; text-align: center;

values: List of 1990s one-hit wonders in the United States

values:
/wiki/RIAA_certification
mw-redirect
RIAA certification

values: checkbox button toctogglecheckbox toctogglecheckbox display:none

values: plainlinks metadata ambox mbox-small-left ambox-content

values:
nofollow
external text
http://longboredsurfer.com/charts/1998

values:
reflist mw-references-wrap
list-style-type: decimal;

values:
nofollow
external text
https://www.discogs.com/master/318249

values: Category:Music videos directed by Christopher Erskin

values: Category:Song recordings produced by Nathan Morris

values: Category:Articles which use infobox templates with no data rows

values: Category:Music infoboxes with deprecated parameters

values: Category:Articles with hAudio microformats

values: Category:Articles to be expanded from February 2015

values: Category:Articles using small message boxes

values: Discussion about edits from this IP address [ctrl-alt-n]

values: A list of edits made from this IP address [ctrl-alt-y]

values: You are encouraged to create an account and log in; however, it is not mandatory

values: You're encouraged to log in; however, it's not mandatory. [ctrl-alt-o]

values: Discussion about the content page [ctrl-alt-t]

values: p-namespaces navigation vectorTabs p-namespaces-label

values:
p-variants
navigation
vectorMenu emptyPortlet
p-variants-label

values: Past revisions of this page [ctrl-alt-h]

values: p-views navigation vectorTabs p-views-label

values:
p-cactions
navigation
vectorMenu emptyPortlet
p-cactions-label

values:
mw-wiki-logo
/wiki/Main_Page
Visit the main page

values:
/wiki/Main_Page
Visit the main page [ctrl-alt-z]
z

values: A list of all special pages [ctrl-alt-q]

values: Permanent link to this revision of the page

values: Link to connected data repository item [ctrl-alt-g]

values: Printable version of this page [ctrl-alt-p]

values: /static/images/wikimedia-button-1.5x.png 1.5x, /static/images/wikimedia-button-2x.png 2x

values: /static/images/poweredby_mediawiki_132x47.png 1.5x, /static/images/poweredby_mediawiki_176x62.png 2x

values: mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-I_Don_t_Ever_Want_to_See_You_Again rootpage-I_Don_t_Ever_Want_to_See_You_Again skin-vector action-view