text:
Convolutional neural network
From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
For other uses, see CNN (disambiguation).
Machine learning and
data mining
Problems[show]
Classification
Clustering
Regression
Anomaly detection
AutoML
Association rules
Reinforcement learning
Structured prediction
Feature engineering
Feature learning
Online learning
Semi-supervised learning
Unsupervised learning
Learning to rank
Grammar induction
Supervised learning
(classification • regression)
[show]
Decision trees
Ensembles (Bagging, Boosting, Random forest)
k-NN
Linear regression
Naive Bayes
Neural networks
Logistic regression
Perceptron
Relevance vector machine (RVM)
Support vector machine (SVM)
Clustering[show]
BIRCH
CURE
Hierarchical
k-means
Expectation–maximization (EM)

DBSCAN
OPTICS
Mean-shift
Dimensionality reduction[show]
Factor analysis
CCA
ICA
LDA
NMF
PCA
t-SNE
Structured prediction[show]
Graphical models (Bayes net, CRF, HMM)
Anomaly detection[show]
k-NN
Local outlier factor
Neural nets[show]
Autoencoder
Deep learning
Multilayer perceptron
RNN (LSTM, GRU)
Restricted Boltzmann machine
SOM
Convolutional neural network (U-Net)
Reinforcement learning[show]
Q-learning
SARSA
Temporal difference (TD)
Theory[show]
Bias-variance dilemma
Computational learning theory
Empirical risk minimization
Occam learning
PAC learning
Statistical learning
VC theory
Machine-learning venues[show]
NIPS
ICML
ML
JMLR
ArXiv:cs.LG
Glossary of artificial intelligence[show]
Glossary of artificial intelligence
Related articles[show]
List of datasets for machine-learning research
Outline of machine learning
Machine learning portal
v
t
e
In machine learning, a convolutional neural network (CNN, or ConvNet) is a class of deep, feed-forward artificial neural networks, most commonly applied to analyzing visual imagery.
CNNs use a variation of multilayer perceptrons designed to require minimal preprocessing.[1] They are also known as shift invariant or space invariant artificial neural networks (SIANN), based on their shared-weights architecture and translation invariance characteristics.[2][3]
Convolutional networks were inspired by biological processes[4] in that the connectivity pattern between neurons resembles the organization of the animal visual cortex. Individual cortical neurons respond to stimuli only in a restricted region of the visual field known as the receptive field. The receptive fields of different neurons partially overlap such that they cover the entire visual field.
CNNs use relatively little pre-processing compared to other image classification algorithms. This means that the network learns the filters that in traditional algorithms were hand-engineered. This independence from prior knowledge and human effort in feature design is a major advantage.
They have applications in image and video recognition, recommender systems[5] and natural language processing.[6]
Contents
1 Design
1.1 Convolutional
1.2 Pooling
1.3 Fully connected
1.4 Receptive field
1.5 Weights
2 History
2.1 Receptive fields in the visual cortex
2.2 Neocognitron
2.3 Trainable weights
2.3.1 LeNet-5
2.4 Shift-invariant neural network
2.5 Neural abstraction pyramid
2.6 GPU implementations
3 Distinguishing features
4 Building blocks
4.1 Convolutional layer
4.1.1 Local connectivity
4.1.2 Spatial arrangement
4.1.3 Parameter sharing
4.2 Pooling layer
4.3 ReLU layer
4.4 Fully connected layer
4.5 Loss layer
5 Choosing hyperparameters
5.1 Number of filters
5.2 Filter shape
5.3 Max pooling shape
6 Regularization methods
6.1 Empirical
6.1.1 Dropout
6.1.2 DropConnect
6.1.3 Stochastic pooling
6.1.4 Artificial data
6.2 Explicit
6.2.1 Early stopping
6.2.2 Number of parameters
6.2.3 Weight decay
6.2.4 Max norm constraints
7 Hierarchical coordinate frames
8 Applications
8.1 Image recognition
8.2 Video analysis
8.3 Natural language processing
8.4 Drug discovery
8.5 Checkers
8.6 Go
9 Fine-tuning
10 Extensions
10.1 Deep Q-networks
10.2 Deep belief networks
10.3 Time delay neural networks
11 Common libraries
12 Common APIs
13 Popular culture
14 See also
15 References
16 External links
Design[edit]
A CNN consists of an input and an output layer, as well as multiple hidden layers. The hidden layers of a CNN typically consist of convolutional layers, pooling layers, fully connected layers and normalization layers[citation needed].
Description of the process as a convolution in neural networks is by convention. Mathematically it is a cross-correlation rather than a convolution. This only has significance for the indices in the matrix, and thus which weights are placed at which index.
Convolutional[edit]
Convolutional layers apply a convolution operation to the input, passing the result to the next layer. The convolution emulates the response of an individual neuron to visual stimuli.[7]
Each convolutional neuron processes data only for its receptive field.
Although fully connected feedforward neural networks can be used to learn features as well as classify data, it is not practical to apply this architecture to images. A very high number of neurons would be necessary, even in a shallow (opposite of deep) architecture, due to the very large input sizes associated with images, where each pixel is a relevant variable. For instance, a fully connected layer for a (small) image of size 100 x 100 has 10000 weights for each neuron in the second layer. The convolution operation brings a solution to this problem as it reduces the number of free parameters, allowing the network to be deeper with fewer parameters.[8] For instance, regardless of image size, tiling regions of size 5 x 5, each with the same shared weights, requires only 25 learnable parameters. In this way, it resolves the vanishing or exploding gradients problem in training traditional multi-layer neural networks with many layers by using backpropagation[citation needed].
Pooling[edit]
Convolutional networks may include local or global pooling layers[clarification needed], which combine the outputs of neuron clusters at one layer into a single neuron in the next layer.[9][10] For example, max pooling uses the maximum value from each of a cluster of neurons at the prior layer.[11] Another example is average pooling, which uses the average value from each of a cluster of neurons at the prior layer[citation needed].
Fully connected[edit]
Fully connected layers connect every neuron in one layer to every neuron in another layer. It is in principle the same as the traditional multi-layer perceptron neural network (MLP).
Receptive field[edit]
In neural networks, each neuron receives input from some number locations in the previous layer. In a fully-connected layer, each neuron receives input from every element of the previous layer. In a convolutional layer, neurons receive input from only a restricted subarea of the previous layer. Typically the subarea is of a square shape (e.g., size 5 by 5). The input area of a neuron is called its receptive field. So, in a fully connected layer, the receptive field is the entire previous layer. In a convolutional layer, the receptive area is smaller than the entire previous layer.
Weights[edit]
Each neuron in a neural network computes an output value by applying some function to the input values coming from the receptive field in the previous layer. The function that is applied to the input values is specified by a vector of weights and a bias (typically real numbers). Learning in a neural network progresses by making incremental adjustments to the biases and weights. The vector of weights and the bias are called a filter and represents some feature of the input (e.g., a particular shape). A distinguishing feature of CNNs is that many neurons share the same filter. This reduces memory footprint because a single bias and a single vector of weights is used across all receptive fields sharing that filter, rather than each receptive field having its own bias and vector of weights. [1]
History[edit]
CNN design follows vision processing in living organisms[citation needed].
Receptive fields in the visual cortex[edit]
Work by Hubel and Wiesel in the 1950s and 1960s showed that cat and monkey visual cortexes contain neurons that individually respond to small regions of the visual field. Provided the eyes are not moving, the region of visual space within which visual stimuli affect the firing of a single neuron is known as its receptive field[citation needed]. Neighboring cells have similar and overlapping receptive fields[citation needed]. Receptive field size and location varies systematically across the cortex to form a complete map of visual space[citation needed]. The cortex in each hemisphere represents the contralateral visual field[citation needed].
Their 1968 paper[12] identified two basic visual cell types in the brain:
simple cells, whose output is maximized by straight edges having particular orientations within their receptive field
complex cells, which have larger receptive fields, whose output is insensitive to the exact position of the edges in the field.
Neocognitron[edit]
The neocognitron [13] was introduced in 1980.[11][14] The neocognitron does not require units located at multiple network positions to have the same trainable weights. This idea appears in 1986 in the book version of the original backpropagation paper.[15]:Figure 14 Neocognitrons were developed in 1988 for temporal signals.[clarification needed][16] Their design was improved in 1998,[17] generalized in 2003[18] and simplified in the same year.[19]
Trainable weights[edit]
A system to recognize hand-written zip code numbers[20] involved convolutions in which the kernel coefficients had been laboriously hand designed[21]
LeCun et al in 1989,[21] used back-propagation to learn the convolution kernel coefficients directly from images of hand-written numbers. Learning was thus fully automatic, performed better than manual coefficient design, and was suited to a broader range of image recognition problems and image types.
LeNet-5[edit]
LeNet-5, a pioneering 7-level convolutional network by LeCun et al. in 1998,[17] that classifies digits, was applied by several banks to recognise hand-written numbers on checks (cheques) digitized in 32x32 pixel images. The ability to process higher resolution images requires larger and more layers of convolutional neural networks, so this technique is constrained by the availability of computing resources.
Shift-invariant neural network[edit]
Similarly, a shift invariant neural network was proposed for image character recognition in 1988.[2][3] The architecture and training algorithm were modified in 1991[22] and applied for medical image processing[23] and automatic detection of breast cancer in mammograms.[24]
A different convolution-based design was proposed in 1988[25] for application to decomposition of one-dimensional electromyography convolved signals via de-convolution. This design was modified in 1989 to other de-convolution-based designs.[26][27]
Neural abstraction pyramid[edit]
The feed-forward architecture of convolutional neural networks was extended in the neural abstraction pyramid[28] by lateral and feedback connections. The resulting recurrent convolutional network allows for the flexible incorporation of contextual information to iteratively resolve local ambiguities. In contrast to previous models, image-like outputs at the highest resolution were generated.
GPU implementations[edit]
Following the 2005 paper that established the value of GPGPU for machine learning,[29] several publications described more efficient ways to train convolutional neural networks using GPUs.[30][31][32][33] In 2011, they were refined and implemented on a GPU, with impressive results.[9] In 2012, Ciresan et al. significantly improved on the best performance in the literature for multiple image databases, including the MNIST database, the NORB database, the HWDB1.0 dataset (Chinese characters), the CIFAR10 dataset (dataset of 60000 32x32 labeled RGB images),[11] and the ImageNet dataset.[34]
Distinguishing features[edit]
While traditional multilayer perceptron (MLP) models were successfully used for image recognition[example needed], due to the full connectivity between nodes they suffer from the curse of dimensionality, and thus do not scale well to higher resolution images.
CNN layers arranged in 3 dimensions
For example, in CIFAR-10, images are only of size 32x32x3 (32 wide, 32 high, 3 color channels), so a single fully connected neuron in a first hidden layer of a regular neural network would have 32*32*3 = 3,072 weights. A 200x200 image, however, would lead to neurons that have 200*200*3 = 120,000 weights.
Also, such network architecture does not take into account the spatial structure of data, treating input pixels which are far apart in the same way as pixels that are close together. Thus, full connectivity of neurons is wasteful for purposes such as image recognition that are dominated by spatially local input patterns.
Convolutional neural networks are biologically inspired variants of multilayer perceptrons that are designed to emulate the behavior of a visual cortex[citation needed]. These models mitigate the challenges posed by the MLP architecture by exploiting the strong spatially local correlation present in natural images. As opposed to MLPs, CNNs have the following distinguishing features:
3D volumes of neurons. The layers of a CNN have neurons arranged in 3 dimensions: width, height and depth. The neurons inside a layer are connected to only a small region of the layer before it, called a receptive field. Distinct types of layers, both locally and completely connected, are stacked to form a CNN architecture.
Local connectivity: following the concept of receptive fields, CNNs exploit spatial locality by enforcing a local connectivity pattern between neurons of adjacent layers. The architecture thus ensures that the learnt "filters" produce the strongest response to a spatially local input pattern. Stacking many such layers leads to non-linear filters that become increasingly global (i.e. responsive to a larger region of pixel space) so that the network first creates representations of small parts of the input, then from them assembles representations of larger areas.
Shared weights: In CNNs, each filter is replicated across the entire visual field. These replicated units share the same parameterization (weight vector and bias) and form a feature map. This means that all the neurons in a given convolutional layer respond to the same feature within their specific response field. Replicating units in this way allows for features to be detected regardless of their position in the visual field, thus constituting the property of translation invariance.
Together, these properties allow CNNs to achieve better generalization on vision problems. Weight sharing dramatically reduces the number of free parameters learned, thus lowering the memory requirements for running the network and allowing the training of larger, more powerful networks.
Building blocks[edit]
This section needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. (June 2017) (Learn how and when to remove this template message)
A CNN architecture is formed by a stack of distinct layers that transform the input volume into an output volume (e.g. holding the class scores) through a differentiable function. A few distinct types of layers are commonly used. We discuss them further below:
Neurons of a convolutional layer (blue), connected to their receptive field (red)
Convolutional layer[edit]
The convolutional layer is the core building block of a CNN. The layer's parameters consist of a set of learnable filters (or kernels), which have a small receptive field, but extend through the full depth of the input volume. During the forward pass, each filter is convolved across the width and height of the input volume, computing the dot product between the entries of the filter and the input and producing a 2-dimensional activation map of that filter. As a result, the network learns filters that activate when it detects some specific type of feature at some spatial position in the input.
Stacking the activation maps for all filters along the depth dimension forms the full output volume of the convolution layer. Every entry in the output volume can thus also be interpreted as an output of a neuron that looks at a small region in the input and shares parameters with neurons in the same activation map.
Local connectivity[edit]
When dealing with high-dimensional inputs such as images, it is impractical to connect neurons to all neurons in the previous volume because such a network architecture does not take the spatial structure of the data into account. Convolutional networks exploit spatially local correlation by enforcing a local connectivity pattern between neurons of adjacent layers: each neuron is connected to only a small region of the input volume. The extent of this connectivity is a hyperparameter called the receptive field of the neuron. The connections are local in space (along width and height), but always extend along the entire depth of the input volume. Such an architecture ensures that the learnt filters produce the strongest response to a spatially local input pattern.
Spatial arrangement[edit]
Three hyperparameters control the size of the output volume of the convolutional layer: the depth, stride and zero-padding.
The depth of the output volume controls the number of neurons in a layer that connect to the same region of the input volume. These neurons learn to activate for different features in the input. For example, if the first convolutional layer takes the raw image as input, then different neurons along the depth dimension may activate in the presence of various oriented edges, or blobs of color.
Stride controls how depth columns around the spatial dimensions (width and height) are allocated. When the stride is 1 then we move the filters one pixel at a time. This leads to heavily overlapping receptive fields between the columns, and also to large output volumes. When the stride is 2 (or rarely 3 or more) then the filters jump 2 pixels at a time as they slide around. The receptive fields overlap less and the resulting output volume has smaller spatial dimensions.[35]
Sometimes it is convenient to pad the input with zeros on the border of the input volume. The size of this padding is a third hyperparameter. Padding provides control of the output volume spatial size. In particular, sometimes it is desirable to exactly preserve the spatial size of the input volume.
The spatial size of the output volume can be computed as a function of the input volume size W {\displaystyle W} , the kernel field size of the Conv Layer neurons K {\displaystyle K} , the stride with which they are applied S {\displaystyle S} , and the amount of zero padding P {\displaystyle P} used on the border. The formula for calculating how many neurons "fit" in a given volume is given by ( W − K + 2 P ) / S + 1 {\displaystyle (W-K+2P)/S+1} . If this number is not an integer, then the strides are set incorrectly and the neurons cannot be tiled to fit across the input volume in a symmetric way. In general, setting zero padding to be P = ( K − 1 ) / 2 {\displaystyle P=(K-1)/2} when the stride is S = 1 {\displaystyle S=1} ensures that the input volume and output volume will have the same size spatially. Though it's generally not completely necessary to use up all of the neurons of the previous layer, for example, you may decide to use just a portion of padding.
Parameter sharing[edit]
A parameter sharing scheme is used in convolutional layers to control the number of free parameters. It relies on one reasonable assumption: That if a patch feature is useful to compute at some spatial position, then it should also be useful to compute at other positions. In other words, denoting a single 2-dimensional slice of depth as a depth slice, we constrain the neurons in each depth slice to use the same weights and bias.
Since all neurons in a single depth slice share the same parameters, then the forward pass in each depth slice of the CONV layer can be computed as a convolution of the neuron's weights with the input volume (hence the name: convolutional layer). Therefore, it is common to refer to the sets of weights as a filter (or a kernel), which is convolved with the input. The result of this convolution is an activation map, and the set of activation maps for each different filter are stacked together along the depth dimension to produce the output volume. Parameter sharing contributes to the translation invariance of the CNN architecture.
Sometimes the parameter sharing assumption may not make sense. This is especially the case when the input images to a CNN have some specific centered structure, in which we expect completely different features to be learned on different spatial locations. One practical example is when the input are faces that have been centered in the image: we might expect different eye-specific or hair-specific features to be learned in different parts of the image. In that case it is common to relax the parameter sharing scheme, and instead simply call the layer a locally connected layer.
Pooling layer[edit]
Max pooling with a 2x2 filter and stride = 2
Another important concept of CNNs is pooling, which is a form of non-linear down-sampling. There are several non-linear functions to implement pooling among which max pooling is the most common. It partitions the input image into a set of non-overlapping rectangles and, for each such sub-region, outputs the maximum. The intuition is that the exact location of a feature is less important than its rough location relative to other features. The pooling layer serves to progressively reduce the spatial size of the representation, to reduce the number of parameters and amount of computation in the network, and hence to also control overfitting. It is common to periodically insert a pooling layer between successive convolutional layers in a CNN architecture. The pooling operation provides another form of translation invariance.
The pooling layer operates independently on every depth slice of the input and resizes it spatially. The most common form is a pooling layer with filters of size 2x2 applied with a stride of 2 downsamples at every depth slice in the input by 2 along both width and height, discarding 75% of the activations. In this case, every max operation is over 4 numbers. The depth dimension remains unchanged.
In addition to max pooling, the pooling units can use other functions, such as average pooling or L2-norm pooling. Average pooling was often used historically but has recently fallen out of favor compared to max pooling, which works better in practice.[36]
Due to the aggressive reduction in the size of the representation, the trend is towards using smaller filters[37] or discarding the pooling layer altogether.[38]
RoI pooling to size 2x2. In this example region proposal (an input parameter) has size 7x5.
Region of Interest pooling (also known as RoI pooling) is a variant of max pooling, in which output size is fixed and input rectangle is a parameter.[39]
Pooling is an important component of convolutional neural networks for object detection based on Fast R-CNN[40] architecture.
ReLU layer[edit]
ReLU is the abbreviation of Rectified Linear Units. This layer applies the non-saturating activation function f ( x ) = max ( 0 , x ) {\displaystyle f(x)=\max(0,x)} . It increases the nonlinear properties of the decision function and of the overall network without affecting the receptive fields of the convolution layer.
Other functions are also used to increase nonlinearity, for example the saturating hyperbolic tangent f ( x ) = tanh  ( x ) {\displaystyle f(x)=\tanh(x)} , f ( x ) = | tanh  ( x ) | {\displaystyle f(x)=|\tanh(x)|} , and the sigmoid function f ( x ) = ( 1 + e − x ) − 1 {\displaystyle f(x)=(1+e^{-x})^{-1}} . ReLU is often preferred to other functions, because it trains the neural network several times faster[41] without a significant penalty to generalisation accuracy.
Fully connected layer[edit]
Finally, after several convolutional and max pooling layers, the high-level reasoning in the neural network is done via fully connected layers. Neurons in a fully connected layer have connections to all activations in the previous layer, as seen in regular neural networks. Their activations can hence be computed with a matrix multiplication followed by a bias offset.
Loss layer[edit]
The loss layer specifies how training penalizes the deviation between the predicted and true labels and is normally the final layer. Various loss functions appropriate for different tasks may be used there. Softmax loss is used for predicting a single class of K mutually exclusive classes. Sigmoid cross-entropy loss is used for predicting K independent probability values in [ 0 , 1 ] {\displaystyle [0,1]} . Euclidean loss is used for regressing to real-valued labels ( − ∞ , ∞ ) {\displaystyle (-\infty ,\infty )} .
Typical CNN architecture
Choosing hyperparameters[edit]
This section needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. (June 2017) (Learn how and when to remove this template message)
CNNs use more hyperparameters than a standard MLP. While the usual rules for learning rates and regularization constants still apply, the following should be kept in mind when optimising.
Number of filters[edit]
Since feature map size decreases with depth, layers near the input layer will tend to have fewer filters while higher layers can have more. To equalize computation at each layer, the feature x pixel position product is kept roughly constant across layers. Preserving more information about the input would require keeping the total number of activations (number of feature maps times number of pixel positions) non-decreasing from one layer to the next.
The number of feature maps directly controls capacity and depends on the number of available examples and task complexity.
Filter shape[edit]
Common filter shapes found in the literature vary greatly, and are usually chosen based on the dataset.
The challenge is thus to find the right level of granularity so as to create abstractions at the proper scale, given a particular dataset.
Max pooling shape[edit]
Typical values are 2x2. Very large input volumes may warrant 4x4 pooling in the lower-layers. However, choosing larger shapes will dramatically reduce the dimension of the signal, and may result in excess information loss. Often, non-overlapping pooling windows perform best.[36]
Regularization methods[edit]
Main article: Regularization (mathematics)
This section needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. (June 2017) (Learn how and when to remove this template message)
Regularization is a process of introducing additional information to solve an ill-posed problem or to prevent overfitting. CNNs use various types of regularization.
Empirical[edit]
Dropout[edit]
Because a fully connected layer occupies most of the parameters, it is prone to overfitting. One method to reduce overfitting is dropout.[42][43] At each training stage, individual nodes are either "dropped out" of the net with probability 1 − p {\displaystyle 1-p} or kept with probability p {\displaystyle p} , so that a reduced network is left; incoming and outgoing edges to a dropped-out node are also removed. Only the reduced network is trained on the data in that stage. The removed nodes are then reinserted into the network with their original weights.
In the training stages, the probability that a hidden node will be dropped is usually 0.5; for input nodes, this should be much lower, intuitively because information is directly lost when input nodes are ignored.
At testing time after training has finished, we would ideally like to find a sample average of all possible 2 n {\displaystyle 2^{n}} dropped-out networks; unfortunately this is unfeasible for large values of n {\displaystyle n} . However, we can find an approximation by using the full network with each node's output weighted by a factor of p {\displaystyle p} , so the expected value of the output of any node is the same as in the training stages. This is the biggest contribution of the dropout method: although it effectively generates 2 n {\displaystyle 2^{n}} neural nets, and as such allows for model combination, at test time only a single network needs to be tested.
By avoiding training all nodes on all training data, dropout decreases overfitting. The method also significantly improves training speed. This makes model combination practical, even for deep neural nets. The technique seems to reduce node interactions, leading them to learn more robust features that better generalize to new data.
DropConnect[edit]
DropConnect[44] is the generalization of dropout in which each connection, rather than each output unit, can be dropped with probability 1 − p {\displaystyle 1-p} . Each unit thus receives input from a random subset of units in the previous layer.
DropConnect is similar to dropout as it introduces dynamic sparsity within the model, but differs in that the sparsity is on the weights, rather than the output vectors of a layer. In other words, the fully connected layer with DropConnect becomes a sparsely connected layer in which the connections are chosen at random during the training stage.
Stochastic pooling[edit]
A major drawback to Dropout is that it does not have the same benefits for convolutional layers, where the neurons are not fully connected.
In stochastic pooling,[45] the conventional deterministic pooling operations are replaced with a stochastic procedure, where the activation within each pooling region is picked randomly according to a multinomial distribution, given by the activities within the pooling region. The approach is hyperparameter free and can be combined with other regularization approaches, such as dropout and data augmentation.
An alternate view of stochastic pooling is that it is equivalent to standard max pooling but with many copies of an input image, each having small local deformations. This is similar to explicit elastic deformations of the input images,[46] which delivers excellent MNIST performance. Using stochastic pooling in a multilayer model gives an exponential number of deformations since the selections in higher layers are independent of those below.
Artificial data[edit]
Since the degree of model overfitting is determined by both its power and the amount of training it receives, providing a convolutional network with more training examples can reduce overfitting. Since these networks are usually trained with all available data, one approach is to either generate new data from scratch (if possible) or perturb existing data to create new ones. For example, input images could be asymmetrically cropped by a few percent to create new examples with the same label as the original.[47]
Explicit[edit]
Early stopping[edit]
Main article: Early stopping
One of the simplest methods to prevent overfitting of a network is to simply stop the training before overfitting has had a chance to occur. It comes with the disadvantage that the learning process is halted.
Number of parameters[edit]
Another simple way to prevent overfitting is to limit the number of parameters, typically by limiting the number of hidden units in each layer or limiting network depth. For convolutional networks, the filter size also affects the number of parameters. Limiting the number of parameters restricts the predictive power of the network directly, reducing the complexity of the function that it can perform on the data, and thus limits the amount of overfitting. This is equivalent to a "zero norm".
Weight decay[edit]
A simple form of added regularizer is weight decay, which simply adds an additional error, proportional to the sum of weights (L1 norm) or squared magnitude (L2 norm) of the weight vector, to the error at each node. The level of acceptable model complexity can be reduced by increasing the proportionality constant, thus increasing the penalty for large weight vectors.
L2 regularization is the most common form of regularization. It can be implemented by penalizing the squared magnitude of all parameters directly in the objective. The L2 regularization has the intuitive interpretation of heavily penalizing peaky weight vectors and preferring diffuse weight vectors. Due to multiplicative interactions between weights and inputs this has the useful property of encouraging the network to use all of its inputs a little rather than some of its inputs a lot.
L1 regularization is another common form. It is possible to combine L1 with L2 regularization (this is called Elastic net regularization). The L1 regularization leads the weight vectors to become sparse during optimization. In other words, neurons with L1 regularization end up using only a sparse subset of their most important inputs and become nearly invariant to the noisy inputs.
Max norm constraints[edit]
Another form of regularization is to enforce an absolute upper bound on the magnitude of the weight vector for every neuron and use projected gradient descent to enforce the constraint. In practice, this corresponds to performing the parameter update as normal, and then enforcing the constraint by clamping the weight vector w → {\displaystyle {\vec {w}}} of every neuron to satisfy ‖ w → ‖ 2 < c {\displaystyle \|{\vec {w}}\|_{2}<c} . Typical values of c {\displaystyle c} are order of 3–4. Some papers report improvements[48] when using this form of regularization.
Hierarchical coordinate frames[edit]
Pooling loses the precise spatial relationships between high-level parts (such as nose and mouth in a face image). These relationships are needed for identity recognition. Overlapping the pools so that each feature occurs in multiple pools, helps retain the information. Translation alone cannot extrapolate the understanding of geometric relationships to a radically new viewpoint, such as a different orientation or scale. On the other hand, people are very good at extrapolating; after seeing a new shape once they can recognize it from a different viewpoint.[49]
Currently, the common way to deal with this problem is to train the network on transformed data in different orientations, scales, lighting, etc. so that the network can cope with these variations. This is computationally intensive for large data-sets. The alternative is to use a hierarchy of coordinate frames and to use a group of neurons to represent a conjunction of the shape of the feature and its pose relative to the retina. The pose relative to retina is the relationship between the coordinate frame of the retina and the intrinsic features' coordinate frame.[50]
Thus, one way of representing something is to embed the coordinate frame within it. Once this is done, large features can be recognized by using the consistency of the poses of their parts (e.g. nose and mouth poses make a consistent prediction of the pose of the whole face). Using this approach ensures that the higher level entity (e.g. face) is present when the lower level (e.g. nose and mouth) agree on its prediction of the pose. The vectors of neuronal activity that represent pose ("pose vectors") allow spatial transformations modeled as linear operations that make it easier for the network to learn the hierarchy of visual entities and generalize across viewpoints. This is similar to the way the human visual system imposes coordinate frames in order to represent shapes.[51]
Applications[edit]
Image recognition[edit]
CNNs are often used in image recognition systems. In 2012 an error rate of 0.23 percent on the MNIST database was reported.[11] Another paper on using CNN for image classification reported that the learning process was "surprisingly fast"; in the same paper, the best published results as of 2011 were achieved in the MNIST database and the NORB database.[9]
When applied to facial recognition, CNNs achieved a large decrease in error rate.[52] Another paper reported a 97.6 percent recognition rate on "5,600 still images of more than 10 subjects".[4] CNNs were used to assess video quality in an objective way after manual training; the resulting system had a very low root mean square error.[53]
The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object classification and detection, with millions of images and hundreds of object classes. In the ILSVRC 2014,[54] a large-scale visual recognition challenge, almost every highly ranked team used CNN as their basic framework. The winner GoogLeNet[55] (the foundation of DeepDream) increased the mean average precision of object detection to 0.439329, and reduced classification error to 0.06656, the best result to date. Its network applied more than 30 layers. That performance of convolutional neural networks on the ImageNet tests was close to that of humans.[56] The best algorithms still struggle with objects that are small or thin, such as a small ant on a stem of a flower or a person holding a quill in their hand. They also have trouble with images that have been distorted with filters, an increasingly common phenomenon with modern digital cameras. By contrast, those kinds of images rarely trouble humans. Humans, however, tend to have trouble with other issues. For example, they are not good at classifying objects into fine-grained categories such as the particular breed of dog or species of bird, whereas convolutional neural networks handle this.
In 2015 a many-layered CNN demonstrated the ability to spot faces from a wide range of angles, including upside down, even when partially occluded with competitive performance. The network trained on a database of 200,000 images that included faces at various angles and orientations and a further 20 million images without faces. They used batches of 128 images over 50,000 iterations.[57]
Video analysis[edit]
Compared to image data domains, there is relatively little work on applying CNNs to video classification. Video is more complex than images since it has another (temporal) dimension. However, some extensions of CNNs into the video domain have been explored. One approach is to treat space and time as equivalent dimensions of the input and perform convolutions in both time and space.[58][59] Another way is to fuse the features of two convolutional neural networks, one for the spatial and one for the temporal stream.[60][61][62] Unsupervised learning schemes for training spatio-temporal features have been introduced, based on Convolutional Gated Restricted Boltzmann Machines[63] and Independent Subspace Analysis.[64]
Natural language processing[edit]
CNNs have also explored natural language processing. CNN models are effective for various NLP problems and achieved excellent results in semantic parsing,[65] search query retrieval,[66] sentence modeling,[67] classification,[68] prediction[69] and other traditional NLP tasks.[70]
Drug discovery[edit]
CNNs have been used in drug discovery. Predicting the interaction between molecules and biological proteins can identify potential treatments. In 2015, Atomwise introduced AtomNet, the first deep learning neural network for structure-based rational drug design.[71] The system trains directly on 3-dimensional representations of chemical interactions. Similar to how image recognition networks learn to compose smaller, spatially proximate features into larger, complex structures,[72] AtomNet discovers chemical features, such as aromaticity, sp3 carbons and hydrogen bonding. Subsequently, AtomNet was used to predict novel candidate biomolecules for multiple disease targets, most notably treatments for the Ebola virus[73] and multiple sclerosis.[74]
Checkers[edit]
CNNs have been used in the game of checkers. From 1999–2001, Fogel and Chellapilla published papers showing how a convolutional neural network could learn to play checkers using co-evolution. The learning process did not use prior human professional games, but rather focused on a minimal set of information contained in the checkerboard: the location and type of pieces, and the piece differential[clarify]. Ultimately, the program (Blondie24) was tested on 165 games against players and ranked in the highest 0.4%.[75][76] It also earned a win against the program Chinook at its "expert" level of play.[77]
Go[edit]
CNNs have been used in computer Go. In December 2014, Clark and Storkey published a paper showing that a CNN trained by supervised learning from a database of human professional games could outperform GNU Go and win some games against Monte Carlo tree search Fuego 1.1 in a fraction of the time it took Fuego to play.[78] Later it was announced that a large 12-layer convolutional neural network had correctly predicted the professional move in 55% of positions, equalling the accuracy of a 6 dan human player. When the trained convolutional network was used directly to play games of Go, without any search, it beat the traditional search program GNU Go in 97% of games, and matched the performance of the Monte Carlo tree search program Fuego simulating ten thousand playouts (about a million positions) per move.[79]
A couple of CNNs for choosing moves to try ("policy network") and evaluating positions ("value network") driving MCTS were used by AlphaGo, the first to beat the best human player at the time.[80]
Fine-tuning[edit]
For many applications, little training data is available. Convolutional neural networks usually require a large amount of training data in order to avoid overfitting. A common technique is to train the network on a larger data set from a related domain. Once the network parameters have converged an additional training step is performed using the in-domain data to fine-tune the network weights. This allows convolutional networks to be successfully applied to problems with small training sets.[81]
Extensions[edit]
Deep Q-networks[edit]
A deep Q-network (DQN) is a type of deep learning model that combines a deep CNN with Q-learning, a form of reinforcement learning. Unlike earlier reinforcement learning agents, DQNs can learn directly from high-dimensional sensory inputs.
Preliminary results were presented in 2014, with an accompanying paper in February 2015.[82] The research described an application to Atari 2600 gaming. Other deep reinforcement learning models preceded it.[83]
Deep belief networks[edit]
Main article: Deep belief network
Convolutional deep belief networks (CDBN) have structure very similar to convolutional neural networks and are trained similarly to deep belief networks. Therefore, they exploit the 2D structure of images, like CNNs do, and make use of pre-training like deep belief networks. They provide a generic structure that can be used in many image and signal processing tasks. Benchmark results on standard image datasets like CIFAR[84] have been obtained using CDBNs.[85]
Time delay neural networks[edit]
A time delay neural network allows speech signals to be processed time-invariantly, analogous to the translation invariance offered by CNNs.[86] They were introduced in the early 1980s. The tiling of neuron outputs can cover timed stages.[53]
Common libraries[edit]
Caffe: A popular library for convolutional neural networks. Created by the Berkeley Vision and Learning Center (BVLC). It supports both CPU and GPU. Developed in C++, and has Python and MATLAB wrappers.
Deeplearning4j: Deep learning in Java and Scala on multi-GPU-enabled Spark. A general-purpose deep learning library for the JVM production stack running on a C++ scientific computing engine. Allows the creation of custom layers. Integrates with Hadoop and Kafka.
deeplearning-hs: Deep learning in Haskell, supports computations with CUDA.
MatConvNet: A convnet implementation in MATLAB.
MXNet: An open-source deep learning framework which is scalable, including support for multiple GPUs and CPUs in distribution. It supports interfaces in multiple languages (C++, Python, Julia, Matlab, JavaScript, Go, R, Scala, Perl, Wolfram Language).
neon: The fastest framework for convolutional neural networks and Deep Learning with support for GPU and CPU backends. The front-end is in Python, while the fast kernels are written in custom shader assembly. Created by Nervana Systems, which was acquired by Intel.
TensorFlow: Apache 2.0-licensed Theano-like library with support for CPU, GPU and Google's proprietary TPU,[87] mobile
TensorLayer (github): A deep learning and reinforcement learning library. It supports both CPU and GPU. Developed in Python.
Theano: The reference deep-learning library for Python with an API largely compatible with the popular NumPy library. Allows user to write symbolic mathematical expressions, then automatically generates their derivatives, saving the user from having to code gradients or backpropagation. These symbolic expressions are automatically compiled to CUDA code for a fast, on-the-GPU implementation.
Torch (www.torch.ch): A scientific computing framework with wide support for machine learning algorithms, written in C and lua. The main author is Ronan Collobert, and it is now used at Facebook AI Research and Twitter.
Dlib: ([1]): A toolkit for making real world machine learning and data analysis applications in C++.
Microsoft Cognitive Toolkit: A deep learning toolkit written by Microsoft with several unique features enhancing scalability over multiple nodes. It supports full-fledged interfaces for training in C++ and Python and with additional support for model inference in C# and Java.
Common APIs[edit]
roNNie.ai: The Lightweight AI - CNN framework to promote AI, and assist programmers and data scientists to simplify the difficulties of deep learning
Keras: A high level API written in Python for TensorFlow and Theano convolutional neural networks.[88]
Popular culture[edit]
Convolutional neural networks are mentioned in the 2017 novel Infinity Born.[89]
See also[edit]
Convolution
Deep learning
Natural-language processing
Neocognitron
Scale-invariant feature transform
Time delay neural network
Vision processing unit
References[edit]
^ Jump up to: a b LeCun, Yann. "LeNet-5, convolutional neural networks". Retrieved 16 November 2013.
^ Jump up to: a b Zhang, Wei (1988). "Shift-invariant pattern recognition neural network and its optical architecture". Proceedings of annual conference of the Japan Society of Applied Physics.
^ Jump up to: a b Zhang, Wei (1990). "Parallel distributed processing model with local space-invariant interconnections and its optical architecture". Applied Optics. 29 (32): 4790–7. Bibcode:1990ApOpt..29.4790Z. doi:10.1364/AO.29.004790. PMID 20577468.
^ Jump up to: a b Matusugu, Masakazu; Katsuhiko Mori; Yusuke Mitari; Yuji Kaneda (2003). "Subject independent facial expression recognition with robust face detection using a convolutional neural network" (PDF). Neural Networks. 16 (5): 555–559. doi:10.1016/S0893-6080(03)00115-1. Retrieved 17 November 2013.
Jump up ^ van den Oord, Aaron; Dieleman, Sander; Schrauwen, Benjamin (2013-01-01). Burges, C. J. C.; Bottou, L.; Welling, M.; Ghahramani, Z.; Weinberger, K. Q., eds. Deep content-based music recommendation (PDF). Curran Associates, Inc. pp. 2643–2651.
Jump up ^ Collobert, Ronan; Weston, Jason (2008-01-01). "A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning". Proceedings of the 25th International Conference on Machine Learning. ICML '08. New York, NY, USA: ACM: 160–167. doi:10.1145/1390156.1390177. ISBN 978-1-60558-205-4.
Jump up ^ "Convolutional Neural Networks (LeNet) – DeepLearning 0.1 documentation". DeepLearning 0.1. LISA Lab. Retrieved 31 August 2013.
Jump up ^ Habibi,, Aghdam, Hamed. Guide to convolutional neural networks : a practical application to traffic-sign detection and classification. Heravi, Elnaz Jahani,. Cham, Switzerland. ISBN 9783319575490. OCLC 987790957.
^ Jump up to: a b c Ciresan, Dan; Ueli Meier; Jonathan Masci; Luca M. Gambardella; Jurgen Schmidhuber (2011). "Flexible, High Performance Convolutional Neural Networks for Image Classification" (PDF). Proceedings of the Twenty-Second international joint conference on Artificial Intelligence-Volume Volume Two. 2: 1237–1242. Retrieved 17 November 2013.
Jump up ^ Krizhevsky, Alex. "ImageNet Classification with Deep Convolutional Neural Networks" (PDF). Retrieved 17 November 2013.
^ Jump up to: a b c d Ciresan, Dan; Meier, Ueli; Schmidhuber, Jürgen (June 2012). "Multi-column deep neural networks for image classification". 2012 IEEE Conference on Computer Vision and Pattern Recognition. New York, NY: Institute of Electrical and Electronics Engineers (IEEE): 3642–3649. arXiv:1202.2745  . doi:10.1109/CVPR.2012.6248110. ISBN 978-1-4673-1226-4. OCLC 812295155. Retrieved 2013-12-09.
Jump up ^ Hubel, D. H.; Wiesel, T. N. (1968-03-01). "Receptive fields and functional architecture of monkey striate cortex". The Journal of Physiology. 195 (1): 215–243. doi:10.1113/jphysiol.1968.sp008455. ISSN 0022-3751. PMC 1557912  . PMID 4966457.
Jump up ^ LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey (2015). "Deep learning". Nature. 521 (7553): 436–444. Bibcode:2015Natur.521..436L. doi:10.1038/nature14539. PMID 26017442.
Jump up ^ Fukushima, Kunihiko (1980). "Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position" (PDF). Biological Cybernetics. 36 (4): 193–202. doi:10.1007/BF00344251. PMID 7370364. Retrieved 16 November 2013.
Jump up ^ David E. Rumelhart; Geoffrey E. Hinton; Ronald J. Wiliams (1986). "Chapter 8 : Learning Internal Representations by ErrorPropagation". In Rumelhart, David E.; McClelland, James.L. Parallel Distributed Processing, Volume 1 (PDF). MIT Press. pp. 319–362. ISBN 9780262680530.
Jump up ^ Homma, Toshiteru; Les Atlas; Robert Marks II (1988). "An Artificial Neural Network for Spatio-Temporal Bipolar Patters: Application to Phoneme Classification" (PDF). Advances in Neural Information Processing Systems. 1: 31–40.
^ Jump up to: a b LeCun, Yann; Léon Bottou; Yoshua Bengio; Patrick Haffner (1998). "Gradient-based learning applied to document recognition" (PDF). Proceedings of the IEEE. 86 (11): 2278–2324. doi:10.1109/5.726791. Retrieved October 7, 2016.
Jump up ^ S. Behnke. Hierarchical Neural Networks for Image Interpretation, volume 2766 of Lecture Notes in Computer Science. Springer, 2003.
Jump up ^ Simard, Patrice, David Steinkraus, and John C. Platt. "Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis." In ICDAR, vol. 3, pp. 958–962. 2003.
Jump up ^ Denker, J S , Gardner, W R., Graf, H. P, Henderson, D, Howard, R E, Hubbard, W, Jackel, L D , BaIrd, H S, and Guyon (1989) Neural network recognizer for hand-written zip code digits, AT&T Bell Laboratories
^ Jump up to: a b Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, L. D. Jackel, Backpropagation Applied to Handwritten Zip Code Recognition; AT&T Bell Laboratories
Jump up ^ Zhang, Wei (1991). "Error Back Propagation with Minimum-Entropy Weights: A Technique for Better Generalization of 2-D Shift-Invariant NNs". Proceedings of the International Joint Conference on Neural Networks.
Jump up ^ Zhang, Wei (1991). "Image processing of human corneal endothelium based on a learning network". Applied Optics. 30 (29): 4211–7. Bibcode:1991ApOpt..30.4211Z. doi:10.1364/AO.30.004211. PMID 20706526.
Jump up ^ Zhang, Wei (1994). "Computerized detection of clustered microcalcifications in digital mammograms using a shift-invariant artificial neural network". Medical Physics. 21 (4): 517–24. Bibcode:1994MedPh..21..517Z. doi:10.1118/1.597177. PMID 8058017.
Jump up ^ Daniel Graupe, Ruey Wen Liu, George S Moschytz."Applications of neural networks to medical signal processing". In Proc. 27th IEEE Decision and Control Conf., pp. 343–347, 1988.
Jump up ^ Daniel Graupe, Boris Vern, G. Gruener, Aaron Field, and Qiu Huang. "Decomposition of surface EMG signals into single fiber action potentials by means of neural network". Proc. IEEE International Symp. on Circuits and Systems, pp. 1008–1011, 1989.
Jump up ^ Qiu Huang, Daniel Graupe, Yi Fang Huang, Ruey Wen Liu."Identification of firing patterns of neuronal signals." In Proc. 28th IEEE Decision and Control Conf., pp. 266–271, 1989.
Jump up ^ Behnke, Sven (2003). Hierarchical Neural Networks for Image Interpretation (PDF). Lecture Notes in Computer Science. 2766. Springer. doi:10.1007/b11963. ISBN 978-3-540-40722-5.
Jump up ^ Dave Steinkraus; Patrice Simard; Ian Buck (2005). "Using GPUs for Machine Learning Algorithms". 12th International Conference on Document Analysis and Recognition (ICDAR 2005). pp. 1115–1119.
Jump up ^ Kumar Chellapilla; Sid Puri; Patrice Simard (2006). "High Performance Convolutional Neural Networks for Document Processing". In Lorette, Guy. Tenth International Workshop on Frontiers in Handwriting Recognition. Suvisoft.
Jump up ^ Hinton, GE; Osindero, S; Teh, YW (Jul 2006). "A fast learning algorithm for deep belief nets". Neural computation. 18 (7): 1527–54. doi:10.1162/neco.2006.18.7.1527. PMID 16764513.
Jump up ^ Bengio, Yoshua; Lamblin, Pascal; Popovici, Dan; Larochelle, Hugo (2007). "Greedy Layer-Wise Training of Deep Networks". Advances in Neural Information Processing Systems: 153–160.
Jump up ^ Ranzato, MarcAurelio; Poultney, Christopher; Chopra, Sumit; LeCun, Yann (2007). "Efficient Learning of Sparse Representations with an Energy-Based Model" (PDF). Advances in Neural Information Processing Systems.
Jump up ^ 10. Deng, Jia, et al. "Imagenet: A large-scale hierarchical image database."Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009.
Jump up ^ "CS231n Convolutional Neural Networks for Visual Recognition". cs231n.github.io. Retrieved 2017-04-25.
^ Jump up to: a b Scherer, Dominik; Müller, Andreas C.; Behnke, Sven (2010). "Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition" (PDF). Artificial Neural Networks (ICANN), 20th International Conference on. Thessaloniki, Greece: Springer. pp. 92–101.
Jump up ^ Graham, Benjamin (2014-12-18). "Fractional Max-Pooling". arXiv:1412.6071  [cs.CV].
Jump up ^ Springenberg, Jost Tobias; Dosovitskiy, Alexey; Brox, Thomas; Riedmiller, Martin (2014-12-21). "Striving for Simplicity: The All Convolutional Net". arXiv:1412.6806  [cs.LG].
Jump up ^ Grel, Tomasz (2017-02-28). "Region of interest pooling explained". deepsense.io.
Jump up ^ Girshick, Ross (2015-09-27). "Fast R-CNN". arXiv:1504.08083  [cs.CV].
Jump up ^ Krizhevsky, A.; Sutskever, I.; Hinton, G. E. (2012). "Imagenet classification with deep convolutional neural networks" (PDF). Advances in Neural Information Processing Systems. 1: 1097–1105.
Jump up ^ Srivastava, Nitish; C. Geoffrey Hinton; Alex Krizhevsky; Ilya Sutskever; Ruslan Salakhutdinov (2014). "Dropout: A Simple Way to Prevent Neural Networks from overfitting" (PDF). Journal of Machine Learning Research. 15 (1): 1929–1958.
Jump up ^ Carlos E. Perez. "A Pattern Language for Deep Learning".
Jump up ^ "Regularization of Neural Networks using DropConnect | ICML 2013 | JMLR W&CP". jmlr.org. Retrieved 2015-12-17.
Jump up ^ Zeiler, Matthew D.; Fergus, Rob (2013-01-15). "Stochastic Pooling for Regularization of Deep Convolutional Neural Networks". arXiv:1301.3557  [cs.LG].
Jump up ^ "Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis – Microsoft Research". research.microsoft.com. Retrieved 2015-12-17.
Jump up ^ Hinton, Geoffrey E.; Srivastava, Nitish; Krizhevsky, Alex; Sutskever, Ilya; Salakhutdinov, Ruslan R. (2012). "Improving neural networks by preventing co-adaptation of feature detectors". arXiv:1207.0580  [cs.NE].
Jump up ^ "Dropout: A Simple Way to Prevent Neural Networks from Overfitting". jmlr.org. Retrieved 2015-12-17.
Jump up ^ Hinton, Geoffrey (1979). "Some demonstrations of the effects of structural descriptions in mental imagery". Cognitive Science. 3 (3): 231–250. doi:10.1016/s0364-0213(79)80008-7.
Jump up ^ Rock, Irvin. "The frame of reference." The legacy of Solomon Asch: Essays in cognition and social psychology (1990): 243–268.
Jump up ^ J. Hinton, Coursera lectures on Neural Networks, 2012, Url: https://www.coursera.org/learn/neural-networks
Jump up ^ Lawrence, Steve; C. Lee Giles; Ah Chung Tsoi; Andrew D. Back (1997). "Face Recognition: A Convolutional Neural Network Approach". Neural Networks, IEEE Transactions on. 8 (1): 98–113. CiteSeerX 10.1.1.92.5813  . doi:10.1109/72.554195.
^ Jump up to: a b Le Callet, Patrick; Christian Viard-Gaudin; Dominique Barba (2006). "A Convolutional Neural Network Approach for Objective Video Quality Assessment" (PDF). IEEE Transactions on Neural Networks. 17 (5): 1316–1327. doi:10.1109/TNN.2006.879766. PMID 17001990. Retrieved 17 November 2013.
Jump up ^ "ImageNet Large Scale Visual Recognition Competition 2014 (ILSVRC2014)". Retrieved 30 January 2016.
Jump up ^ Szegedy, Christian; Liu, Wei; Jia, Yangqing; Sermanet, Pierre; Reed, Scott; Anguelov, Dragomir; Erhan, Dumitru; Vanhoucke, Vincent; Rabinovich, Andrew (2014). "Going Deeper with Convolutions". Computing Research Repository. arXiv:1409.4842  . Bibcode:2014arXiv1409.4842S.
Jump up ^ Russakovsky, Olga; Deng, Jia; Su, Hao; Krause, Jonathan; Satheesh, Sanjeev; Ma, Sean; Huang, Zhiheng; Karpathy, Andrej; Khosla, Aditya; Bernstein, Michael; Berg, Alexander C.; Fei-Fei, Li (2014). "Image Net Large Scale Visual Recognition Challenge". arXiv:1409.0575  [cs.CV].
Jump up ^ "The Face Detection Algorithm Set To Revolutionize Image Search". Technology Review. February 16, 2015. Retrieved 27 October 2017.
Jump up ^ Baccouche, Moez; Mamalet, Franck; Wolf, Christian; Garcia, Christophe; Baskurt, Atilla (2011-11-16). "Sequential Deep Learning for Human Action Recognition". In Salah, Albert Ali; Lepri, Bruno. Human Behavior Unterstanding. Lecture Notes in Computer Science. 7065. Springer Berlin Heidelberg. pp. 29–39. doi:10.1007/978-3-642-25446-8_4. ISBN 978-3-642-25445-1.
Jump up ^ Ji, Shuiwang; Xu, Wei; Yang, Ming; Yu, Kai (2013-01-01). "3D Convolutional Neural Networks for Human Action Recognition". IEEE Transactions on Pattern Analysis and Machine Intelligence. 35 (1): 221–231. doi:10.1109/TPAMI.2012.59. ISSN 0162-8828. PMID 22392705.
Jump up ^ Huang, Jie; Zhou, Wengang; Zhang, Qilin; Li, Houqiang; Li, Weiping (2018). "Video-based Sign Language Recognition without Temporal Segmentation". arXiv:1801.10111  [cs.CV].
Jump up ^ Karpathy, Andrej, et al. "Large-scale video classification with convolutional neural networks." IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014.
Jump up ^ Simonyan, Karen; Zisserman, Andrew (2014). "Two-Stream Convolutional Networks for Action Recognition in Videos". arXiv:1406.2199  [cs.CV]. (2014).
Jump up ^ Taylor, Graham W.; Fergus, Rob; LeCun, Yann; Bregler, Christoph (2010-01-01). "Convolutional Learning of Spatio-temporal Features". Proceedings of the 11th European Conference on Computer Vision: Part VI. ECCV'10. Berlin, Heidelberg: Springer-Verlag: 140–153. ISBN 3-642-15566-9.
Jump up ^ Le, Q. V.; Zou, W. Y.; Yeung, S. Y.; Ng, A. Y. (2011-01-01). "Learning Hierarchical Invariant Spatio-temporal Features for Action Recognition with Independent Subspace Analysis". Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. CVPR '11. Washington, DC, USA: IEEE Computer Society: 3361–3368. doi:10.1109/CVPR.2011.5995496. ISBN 978-1-4577-0394-2.
Jump up ^ Grefenstette, Edward; Blunsom, Phil; de Freitas, Nando; Hermann, Karl Moritz (2014-04-29). "A Deep Architecture for Semantic Parsing". arXiv:1404.7296  [cs.CL].
Jump up ^ "Learning Semantic Representations Using Convolutional Neural Networks for Web Search – Microsoft Research". research.microsoft.com. Retrieved 2015-12-17.
Jump up ^ Kalchbrenner, Nal; Grefenstette, Edward; Blunsom, Phil (2014-04-08). "A Convolutional Neural Network for Modelling Sentences". arXiv:1404.2188  [cs.CL].
Jump up ^ Kim, Yoon (2014-08-25). "Convolutional Neural Networks for Sentence Classification". arXiv:1408.5882  [cs.CL].
Jump up ^ Collobert, Ronan, and Jason Weston. "A unified architecture for natural language processing: Deep neural networks with multitask learning."Proceedings of the 25th international conference on Machine learning. ACM, 2008.
Jump up ^ Collobert, Ronan; Weston, Jason; Bottou, Leon; Karlen, Michael; Kavukcuoglu, Koray; Kuksa, Pavel (2011-03-02). "Natural Language Processing (almost) from Scratch". arXiv:1103.0398  [cs.LG].
Jump up ^ Wallach, Izhar; Dzamba, Michael; Heifets, Abraham (2015-10-09). "AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery". arXiv:1510.02855  [cs.LG].
Jump up ^ Yosinski, Jason; Clune, Jeff; Nguyen, Anh; Fuchs, Thomas; Lipson, Hod (2015-06-22). "Understanding Neural Networks Through Deep Visualization". arXiv:1506.06579  [cs.CV].
Jump up ^ "Toronto startup has a faster way to discover effective medicines". The Globe and Mail. Retrieved 2015-11-09.
Jump up ^ "Startup Harnesses Supercomputers to Seek Cures". KQED Future of You. Retrieved 2015-11-09.
Jump up ^ Chellapilla, K; Fogel, DB (1999). "Evolving neural networks to play checkers without relying on expert knowledge". IEEE Trans Neural Netw. 10 (6): 1382–91. doi:10.1109/72.809083. PMID 18252639.
Jump up ^ http://ieeexplore.ieee.org/document/942536/
Jump up ^ Fogel, David (2001). Blondie24: Playing at the Edge of AI. San Francisco, CA: Morgan Kaufmann. ISBN 1558607838.
Jump up ^ Clark, Christopher; Storkey, Amos (2014). "Teaching Deep Convolutional Neural Networks to Play Go". arXiv:1412.3409  [cs.AI].
Jump up ^ Maddison, Chris J.; Huang, Aja; Sutskever, Ilya; Silver, David (2014). "Move Evaluation in Go Using Deep Convolutional Neural Networks". arXiv:1412.6564  [cs.LG].
Jump up ^ "AlphaGo – Google DeepMind". Retrieved 30 January 2016.
Jump up ^ Durjoy Sen Maitra; Ujjwal Bhattacharya; S.K. Parui, "CNN based common approach to handwritten character recognition of multiple scripts," in Document Analysis and Recognition (ICDAR), 2015 13th International Conference on, vol., no., pp.1021–1025, 23–26 Aug. 2015
Jump up ^ Mnih, Volodymyr; et al. (2015). "Human-level control through deep reinforcement learning". Nature. 518 (7540): 529–533. Bibcode:2015Natur.518..529M. doi:10.1038/nature14236. PMID 25719670.
Jump up ^ Sun, R.; Sessions, C. (June 2000). "Self-segmentation of sequences: automatic formation of hierarchies of sequential behaviors". IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics). 30 (3): 403–418. doi:10.1109/3477.846230. ISSN 1083-4419.
Jump up ^ "Convolutional Deep Belief Networks on CIFAR-10" (PDF).
Jump up ^ Lee, Honglak; Grosse, Roger; Ranganath, Rajesh; Ng, Andrew Y. (1 January 2009). "Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations". Proceedings of the 26th Annual International Conference on Machine Learning – ICML '09. ACM: 609–616. doi:10.1145/1553374.1553453. ISBN 9781605585161 – via ACM Digital Library.
Jump up ^ Alexander Waibel et al., Phoneme Recognition Using Time-Delay Neural Networks IEEE Transactions on Acoustics, Speech and Signal Processing, Volume 37, No. 3, pp. 328. - 339 March 1989.
Jump up ^ Cade Metz (May 18, 2016). "Google Built Its Very Own Chips to Power Its AI Bots". Wired.
Jump up ^ "Keras Documentation". keras.io.
Jump up ^ Richards, Douglas E. (2017-04-30). Infinity Born. Paragon Press. ISBN 1546406395.
External links[edit]
convnet-benchmarks — easy benchmarking of all public open-source implementations of convnets (with results)
Beginner's Guide to Convolutional Neural Nets — A gentle tutorial on how convolutional nets work
CS231n: Convolutional Neural Networks for Visual Recognition — Andrej Karpathy's Stanford CS class
An Intuitive Explanation of Convolutional Neural Networks — A beginner level introduction to what Convolutional Neural Networks are and how they work
Convolutional Neural Networks for Image Classification — Literature Survey
Retrieved from "https://en.wikipedia.org/w/index.php?title=Convolutional_neural_network&oldid=854252558"
Categories:
Artificial neural networks
Computer vision
Computational neuroscience
Hidden categories:
All articles with unsourced statements
Articles with unsourced statements from January 2018
Articles with unsourced statements from October 2017
Wikipedia articles needing clarification from September 2016
Articles needing additional references from June 2017
All articles needing additional references
All Wikipedia articles needing clarification
Wikipedia articles needing clarification from January 2018
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Article
Talk
Variants
Views
Read
Edit
View history
More
Search
Navigation
Main page
Contents
Featured content
Current events
Random article
Donate to Wikipedia
Wikipedia store
Interaction
Help
About Wikipedia
Community portal
Recent changes
Contact page
Tools
What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Wikidata item
Cite this page
Print/export
Create a book
Download as PDF
Printable version
Languages
العربية
Català
Deutsch
Español
فارسی
Français
한국어
Italiano
日本語
Português
Русский
Українська
中文
4 more
Edit links
This page was last edited on 10 August 2018, at 00:35 (UTC).
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.
Privacy policy
About Wikipedia
Disclaimers
Contact Wikipedia
Developers
Cookie statement
Mobile view
Enable previews

values: mw-page-base noprint

values: mw-head-base noprint

values: firstHeading firstHeading en

values: siteSub noprint

attrs: class href

values: mw-jump-link #mw-head

values: mw-jump-link #p-search

values:
/wiki/CNN_(disambiguation)
mw-disambig
CNN (disambiguation)

values:
note
hatnote navigation-not-searchable

value: th

value: tr

value: td

values: padding:0.2em 0 0.4em;padding:0.25em 0.25em 0.75em;

attrs: class id href

values: NavToggle NavToggle1 #

value: li

values: font-size: 105%; padding: 0.2em 0px 0.4em; text-align: center; display: none;

values:
/wiki/CURE_data_clustering_algorithm
mw-redirect
CURE data clustering algorithm

values:
/wiki/Canonical_correlation_analysis
mw-redirect
Canonical correlation analysis

values: T-distributed stochastic neighbor embedding

values:
/wiki/Bias-variance_dilemma
mw-redirect
Bias-variance dilemma

values: Conference on Neural Information Processing Systems

values: International Conference on Machine Learning

values:
nofollow
external text
https://arxiv.org/list/cs.LG/recent

values: List of datasets for machine-learning research

values: padding:0.3em 0.4em 0.3em;font-weight:bold;border-top: 1px solid #aaa; border-bottom: 1px solid #aaa;border-top:1px solid #aaa;border-bottom:1px solid #aaa;

values: float:right;clear:right;width:22.0em;margin:0 0 1.0em 1.0em;background:#f9f9f9;border:1px solid #aaa;padding:0.2em;border-spacing:0.4em 0;text-align:center;line-height:1.4em;font-size:88%

values:
/wiki/Translation_invariance
mw-redirect
Translation invariance

values:
/wiki/Mathematical_biology
mw-redirect
Mathematical biology

values:
/wiki/Cortical_neuron
mw-redirect
Cortical neuron

values:
/wiki/Image_classification
mw-redirect
Image classification

values: checkbox button toctogglecheckbox toctogglecheckbox display:none

values: This claim needs references to reliable sources. (January 2018)

values:
noprint Inline-Template Template-Fact
white-space:nowrap;

values: This claim needs references to reliable sources. (October 2017)

values: The text near this tag may need clarification or removal of jargon. (September 2016)

values:
noprint Inline-Template
margin-left:0.1em; white-space:nowrap;

values:
/wiki/Living_organisms
mw-redirect
Living organisms

values: Edit section: Receptive fields in the visual cortex

values:
/wiki/Simple_cells_(visual_cortex)
mw-redirect
Simple cells (visual cortex)

values:
/wiki/Complex_cells_(visual_cortex)
mw-redirect
Complex cells (visual cortex)

values: Edit section: Shift-invariant neural network

values: Edit section: Neural abstraction pyramid

values: Help:Introduction to referencing with Wiki Markup/1

values: plainlinks metadata ambox ambox-content ambox-Refimprove

values: vertical-align: -0.338ex; width:2.435ex; height:2.176ex;

values: vertical-align: -0.338ex; width:2.066ex; height:2.176ex;

values: vertical-align: -0.338ex; width:1.499ex; height:2.176ex;

values: vertical-align: -0.338ex; width:1.745ex; height:2.176ex;

values: vertical-align: -0.838ex; width:21.564ex; height:2.843ex;

values: vertical-align: -0.838ex; width:15.047ex; height:2.843ex;

values: vertical-align: -0.338ex; width:5.76ex; height:2.176ex;

values: vertical-align: -0.838ex; width:17.177ex; height:2.843ex;

values:
/wiki/Hyperbolic_tangent
mw-redirect
Hyperbolic tangent

values: vertical-align: -0.838ex; width:15.307ex; height:2.843ex;

values: vertical-align: -0.838ex; width:16.988ex; height:2.843ex;

values: vertical-align: -0.838ex; width:19.195ex; height:3.176ex;

values: vertical-align: -0.838ex; width:4.653ex; height:2.843ex;

values: vertical-align: -0.838ex; width:9.299ex; height:2.843ex;

values:
/wiki/Ill-posed_problem
mw-redirect
Ill-posed problem

values: vertical-align: -0.671ex; width:5.172ex; height:2.509ex;

values: vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;

values: vertical-align: -0.338ex; width:2.381ex; height:2.343ex;

values: vertical-align: -0.338ex; width:1.395ex; height:1.676ex;

values: vertical-align: -0.338ex; width:1.664ex; height:2.343ex;

values: vertical-align: -0.838ex; width:9.149ex; height:2.843ex;

values: vertical-align: -0.338ex; width:1.007ex; height:1.676ex;

values: Edit section: Hierarchical coordinate frames

values:
/wiki/Root_mean_square_error
mw-redirect
Root mean square error

values: ImageNet Large Scale Visual Recognition Challenge

values: Edit section: Natural language processing

values: padding-left:0.1em; padding-right:0.1em; color:#595959; border:1px solid #DDD;

values: This passage needs to be better explained. (January 2018)

values: Edit section: Time delay neural networks

values:
nofollow
http://deeplearning4j.org/
external text

values:
nofollow
http://www.vlfeat.org/matconvnet/
external text

values:
nofollow
external text
https://github.com/NervanaSystems/neon

values:
nofollow
external text
http://github.com/tensorlayer

values:
nofollow
http://www.torch.ch/
external text

values:
nofollow
external autonumber
http://dlib.net

values:
nofollow
external text
https://roNNie.ai

values:
/wiki/Natural-language_processing
mw-redirect
Natural-language processing

values:
nofollow
http://yann.lecun.com/exdb/lenet/
external text

values:
nofollow
external text
//doi.org/10.1364/AO.29.004790

values:
/wiki/PubMed_Identifier
mw-redirect
PubMed Identifier

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/20577468

values:
nofollow
external text
https://www.worldcat.org/oclc/987790957

values:
nofollow
external text
//www.worldcat.org/oclc/987790957

values: Institute of Electrical and Electronics Engineers

values:
nofollow
external text
//arxiv.org/abs/1202.2745

attrs: alt src title width height srcset data-file-width data-file-height

values:
nofollow
external text
//doi.org/10.1109/CVPR.2012.6248110

values:
nofollow
external text
//www.worldcat.org/oclc/812295155

values:
nofollow
external text
//www.worldcat.org/issn/0022-3751

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/4966457

values:
nofollow
external text
//doi.org/10.1038/nature14539

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/26017442

values:
nofollow
external text
//doi.org/10.1007/BF00344251

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/7370364

values:
nofollow
external text
//doi.org/10.1109/5.726791

values:
nofollow
external text
//doi.org/10.1364/AO.30.004211

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/20706526

values:
nofollow
external text
//doi.org/10.1118/1.597177

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/8058017

values:
nofollow
external text
//doi.org/10.1007/b11963

values:
nofollow
external text
//doi.org/10.1162/neco.2006.18.7.1527

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/16764513

values:
nofollow
external text
//arxiv.org/abs/1412.6071

values:
nofollow
external text
//arxiv.org/archive/cs.CV

values:
nofollow
external text
//arxiv.org/abs/1412.6806

values:
nofollow
external text
//arxiv.org/archive/cs.LG

values:
nofollow
external text
//arxiv.org/abs/1504.08083

values:
nofollow
external text
http://www.deeplearningpatterns.com

values:
nofollow
external text
//arxiv.org/abs/1301.3557

values:
nofollow
external text
//arxiv.org/abs/1207.0580

values:
nofollow
external text
//arxiv.org/archive/cs.NE

values:
nofollow
external text
//doi.org/10.1109/72.554195

values:
nofollow
external text
//doi.org/10.1109/TNN.2006.879766

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/17001990

values:
nofollow
external text
//arxiv.org/abs/1409.4842

values:
nofollow
external text
//arxiv.org/abs/1409.0575

values:
nofollow
external text
//doi.org/10.1109/TPAMI.2012.59

values:
nofollow
external text
//www.worldcat.org/issn/0162-8828

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/22392705

values:
nofollow
external text
//arxiv.org/abs/1801.10111

values:
nofollow
external text
//arxiv.org/abs/1406.2199

values:
nofollow
external text
//doi.org/10.1109/CVPR.2011.5995496

values:
nofollow
external text
//arxiv.org/abs/1404.7296

values:
nofollow
external text
//arxiv.org/archive/cs.CL

values:
nofollow
external text
//arxiv.org/abs/1404.2188

values:
nofollow
external text
//arxiv.org/abs/1408.5882

values:
nofollow
external text
//arxiv.org/abs/1103.0398

values:
nofollow
external text
//arxiv.org/abs/1510.02855

values:
nofollow
external text
//arxiv.org/abs/1506.06579

values:
nofollow
external text
//doi.org/10.1109/72.809083

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/18252639

values:
nofollow
external text
//arxiv.org/abs/1412.3409

values:
nofollow
external text
//arxiv.org/archive/cs.AI

values:
nofollow
external text
//arxiv.org/abs/1412.6564

values:
nofollow
external text
https://www.deepmind.com/alpha-go.html

values:
nofollow
external text
//doi.org/10.1038/nature14236

values:
nofollow
external text
//www.ncbi.nlm.nih.gov/pubmed/25719670

values:
nofollow
external text
//doi.org/10.1109/3477.846230

values:
nofollow
external text
//www.worldcat.org/issn/1083-4419

values: -moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;

values:
nofollow
https://cs231n.github.io/
external text

values: Category:All articles with unsourced statements

values: Category:Articles with unsourced statements from January 2018

values: Category:Articles with unsourced statements from October 2017

values: Category:Wikipedia articles needing clarification from September 2016

values: Category:Articles needing additional references from June 2017

values: Category:All articles needing additional references

values: Category:All Wikipedia articles needing clarification

values: Category:Wikipedia articles needing clarification from January 2018

values: Discussion about edits from this IP address [ctrl-alt-n]

values: A list of edits made from this IP address [ctrl-alt-y]

values: You are encouraged to create an account and log in; however, it is not mandatory

values: You're encouraged to log in; however, it's not mandatory. [ctrl-alt-o]

values:
/wiki/Convolutional_neural_network
View the content page [ctrl-alt-c]
c

values: portal navigation p-navigation p-navigation-label

values: Guidance on how to use and edit Wikipedia

values: About the project, what you can do, where to find things

values: A list of recent changes in the wiki [ctrl-alt-r]

values: portal navigation p-interaction p-interaction-label

values: List of all English Wikipedia pages containing links to this page [ctrl-alt-j]

values: Recent changes in pages linked from this page [ctrl-alt-k]

values:
/wiki/Wikipedia:File_Upload_Wizard
Upload files [ctrl-alt-u]
u

values: A list of all special pages [ctrl-alt-q]

values: Permanent link to this revision of the page

values: Link to connected data repository item [ctrl-alt-g]

values: Printable version of this page [ctrl-alt-p]

values: شبكة عصبونية التفافية – Arabic

values:
interlanguage-link interwiki-ca
display: none;

values: Redes neuronales convolucionales – Spanish

values: شبکه عصبی پیچشی – Persian

values:
interlanguage-link interwiki-fa
display: none;

values: 합성곱 신경망 – Korean

values:
interlanguage-link interwiki-it
display: none;

values: 畳み込みニューラルネットワーク – Japanese

values: Свёрточная нейронная сеть – Russian

values: Згорткова нейронна мережа – Ukrainian

values:
interlanguage-link interwiki-uk
display: none;

values: 卷积神经网络 – Chinese

values: All languages (initial selection from common choices by you and others)

values: mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Convolutional_neural_network rootpage-Convolutional_neural_network skin-vector action-view


text:
The Importance of Being Earnest (2002 film)
From Wikipedia, the free encyclopedia
Jump to: navigation, search
The Importance of Being Earnest
Theatrical release poster
Directed by
Oliver Parker
Produced by
Barnaby Thompson
David Brown
Uri Fruchtmann
Screenplay by
Oliver Parker
Based on
The Importance of Being Earnest
by Oscar Wilde
Starring
Rupert Everett
Colin Firth
Frances O'Connor
Reese Witherspoon
Judi Dench
Music by
Charlie Mole
Edited by
Guy Bensley
Production
company
Ealing Studios
Film Council
Fragile Films
Newmarket Capital Group
Distributed by
Miramax Films
Release date
17 May 2002 (2002-05-17) (US)
6 September 2002 (2002-09-06) (UK)
Running time
97 minutes
Country
United Kingdom
United States
Language
English
Budget
$15 million
Box office
$17,290,970
The Importance of Being Earnest is a 2002 British-American romantic comedy-drama film directed by Oliver Parker, based on Oscar Wilde's classic comedy of manners play The Importance of Being Earnest. The original music score is composed by Charlie Mole. The film grossed about $8.4 million in North America.[1]
Contents
[hide]
1 Cast
2 Production notes
3 Release
3.1 Critical reception
3.2 Box office performance
3.3 Awards and nominations
4 References
5 External links
Cast[edit]
Rupert Everett as Algernon "Algy" Moncrieff
Colin Firth as John "Jack" Worthing/Ernest
Frances O'Connor as Gwendolen Fairfax
Reese Witherspoon as Cecily Cardew
Judi Dench as Lady Bracknell
Tom Wilkinson as Dr Chasuble
Anna Massey as Miss Prism
Edward Fox as Lane
Patrick Godfrey as Merriman
Production notes[edit]
This section does not cite any sources. Please help improve this section by adding citations to reliable sources. Unsourced material may be challenged and removed. (September 2009) (Learn how and when to remove this template message)
Dame Judi Dench portrayed Lady Bracknell for the third time, having been cast in the 1982 National Theatre revival and the 1995 BBC Radio 4 adaptation.[2][3]
Actress Finty Williams, who plays Lady Bracknell as a young dancer, is the daughter of Dame Judi Dench, who plays the older Lady Bracknell.
The scenes where Rupert Everett slaps Colin Firth on his rear end and where Everett kisses Firth's cheek were ad libbed. Director Parker thought Firth's stunned reaction was so humorous he decided to leave it in.
The business with 'Ernest's' bill at the Savoy, and with the money collectors coming to Jack's country home, are taken from material Wilde cut from the play prior to its publication.
Though cut from the revised version, the gardener Molton can be seen in the background of many scenes.
The producers of the film paid £50,000 to use West Wycombe Park as Jack's home in the country.
Release[edit]
Critical reception[edit]
Roger Ebert of the Chicago Sun-Times awarded the film three out of four stars, saying the actors were well cast, and performed well.[4]
On review aggregate website Rotten Tomatoes, as of September 2012, the film holds a 58% approval rating, with the consensus "Oliver Parker's adaptation of Oscar Wilde's classic play is breezy entertainment, helped by an impressive cast, but it also suffers from some peculiar directorial choices that ultimately dampen the film's impact."[5]
Box office performance[edit]
The Importance of Being Earnest grossed $8,384,929 domestically and $8,906,041 internationally for a worldwide total of $17,290,970,[6] making the film a moderate box office success based on its $15 million budget.
Awards and nominations[edit]
This section does not cite any sources. Please help improve this section by adding citations to reliable sources. Unsourced material may be challenged and removed. (July 2012) (Learn how and when to remove this template message)
The film won the 2003 Italian National Syndicate of Film Journalists's Silver Ribbon award for Best Costume Design, the movie's costumes being designed by Maurizio Millenotti.
Reese Witherspoon was nominated for a Teen Choice Award (Choice Actress – Comedy) for her performance as Cecily.
References[edit]
Jump up ^ Films of 2002, by total gross
Jump up ^ "The Importance of Being Earnest", National Theatre, accessed 28 July 2013
Jump up ^ ISBN 1-85998-218-2
Jump up ^ Ebert, Roger. The Importance of Being Earnest 24 May 2002
Jump up ^ The Importance of Being Earnest at Rotten Tomatoes
Jump up ^ The Importance of Being Earnest at Box Office Mojo
External links[edit]
Official website
The Importance of Being Earnest on IMDb
The Importance of Being Earnest at AllMovie
The Importance of Being Earnest at Box Office Mojo
The Importance of Being Earnest at Rotten Tomatoes
Where Did They Film that? The Importance of Being Earnest
[show]
v
t
e
Oscar Wilde's The Importance of Being Earnest
Films
Al Compás de tu Mentira (1950)
The Importance of Being Earnest (1952)
The Importance of Being Earnest (2002)
The Importance of Being Earnest (2011)
Television
The Importance of Being Earnest (1957)
Other
Ernest in Love (musical)
The Importance of Being Earnest (opera)
[show]
v
t
e
Films directed by Oliver Parker
Othello (1995)
An Ideal Husband (1999)
The Importance of Being Earnest (2002)
Fade to Black (2006)
I Really Hate My Job (2007)
St Trinian's (2007)
Dorian Gray (2009)
St Trinian's 2: The Legend of Fritton's Gold (2009)
Johnny English Reborn (2011)
Dad's Army (2016)
Retrieved from "https://en.wikipedia.org/w/index.php?title=The_Importance_of_Being_Earnest_(2002_film)&oldid=805922139"
Categories:
2002 films
English-language films
2000s romantic comedy films
American films
American romantic comedy films
British films
British romantic comedy films
Films based on The Importance of Being Earnest
Films directed by Oliver Parker
Films set in England
Films set in the 19th century
Films shot in England
German-language films
Films set in the Victorian era
Films set in country houses
Hidden categories:
Use dmy dates from September 2013
Film articles using image size parameter
Pages using infobox film with unknown parameters
Articles needing additional references from September 2009
All articles needing additional references
Articles needing additional references from July 2012
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Article
Talk
Variants
Views
Read
Edit
View history
More
Search
Navigation
Main page
Contents
Featured content
Current events
Random article
Donate to Wikipedia
Wikipedia store
Interaction
Help
About Wikipedia
Community portal
Recent changes
Contact page
Tools
What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Wikidata item
Cite this page
Print/export
Create a book
Download as PDF
Printable version
Languages
Deutsch
Español
فارسی
Français
한국어
Italiano
עברית
Polski
Русский
Suomi
Svenska
中文
Edit links
This page was last edited on 18 October 2017, at 14:08.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.
Privacy policy
About Wikipedia
Disclaimers
Contact Wikipedia
Developers
Cookie statement
Mobile view
Enable previews

key: html

type: local

value: div

attrs: id class

values: mw-page-base noprint

values: mw-head-base noprint

value: a

value: i

attrs: id class lang

values: firstHeading firstHeading en

values: siteSub noprint

values: jump-to-nav mw-jump

attrs: colspan class style

value: img

attrs: alt src width height class srcset data-file-width data-file-height

attrs: href class

attrs: colspan style

attrs: scope style

attrs: href title

values:
/wiki/Miramax_Films
mw-redirect
Miramax Films

values:
/wiki/Tom_Wilkinson_(actor)
mw-redirect
Tom Wilkinson (actor)

attrs: alt src width height srcset data-file-width data-file-height

values: Help:Introduction to referencing with Wiki Markup/1

values: plainlinks metadata ambox ambox-content ambox-Unreferenced

values:
/wiki/Scene_(film)
mw-redirect
Scene (film)

values:
/wiki/Teen_Choice_Award
mw-redirect
Teen Choice Award

values:
/wiki/Ebert,_Roger
mw-redirect
Ebert, Roger

values:
reflist mw-references-wrap
list-style-type: decimal;

values:
nofollow
http://www.imdb.com/title/tt0278500/
external text

values:
nofollow
external text
https://www.allmovie.com/movie/v261812

values: Template:The Importance of Being Earnest

values: Template talk:The Importance of Being Earnest

values: The Importance of Being Earnest (1952 film)

values: The Importance of Being Earnest (2011 film)

values: The Importance of Being Earnest (1957 Australian TV play)

values: nowraplinks collapsible autocollapse navbox-inner

values: Category:Films based on The Importance of Being Earnest

values: Category:Films directed by Oliver Parker

values: Category:Use dmy dates from September 2013

values: Category:Film articles using image size parameter

values: Category:Pages using infobox film with unknown parameters

values: Category:Articles needing additional references from September 2009

values: Category:All articles needing additional references

values: Category:Articles needing additional references from July 2012

values: Discussion about edits from this IP address [ctrl-alt-n]

values: A list of edits made from this IP address [ctrl-alt-y]

values: You are encouraged to create an account and log in; however, it is not mandatory

values: You're encouraged to log in; however, it's not mandatory. [ctrl-alt-o]

values: Discussion about the content page [ctrl-alt-t]

values: p-namespaces navigation vectorTabs p-namespaces-label

values:
p-variants
navigation
vectorMenu emptyPortlet
p-variants-label

values: Past revisions of this page [ctrl-alt-h]

values: p-views navigation vectorTabs p-views-label

values:
p-cactions
navigation
vectorMenu emptyPortlet
p-cactions-label

values:
mw-wiki-logo
/wiki/Main_Page
Visit the main page

values:
/wiki/Main_Page
Visit the main page [ctrl-alt-z]
z

values: Featured content – the best of Wikipedia

values: Find background information on current events

values:
/wiki/Special:Random
Load a random article [ctrl-alt-x]
x

values: portal navigation p-navigation p-navigation-label

values: Guidance on how to use and edit Wikipedia

values: About the project, what you can do, where to find things

values:
uls-settings-trigger
Language settings
0
button
true

values: La importancia de llamarse Ernesto (2002) – Spanish

values: اهمیت جدی بودن (فیلم ۲۰۰۲) – Persian

values: L'Importance d'être Constant (film) – French

values: 진지함의 중요성 (영화) – Korean

values: L'importanza di chiamarsi Ernest – Italian

values: חשיבותה של רצינות (סרט, 2002) – Hebrew

values: Bądźmy poważni na serio (film 2002) – Polish

values: Как важно быть серьёзным (фильм, 2002) – Russian

values: /static/images/wikimedia-button-1.5x.png 1.5x, /static/images/wikimedia-button-2x.png 2x

values: /static/images/poweredby_mediawiki_132x47.png 1.5x, /static/images/poweredby_mediawiki_176x62.png 2x

values:
suggestions
display: none; font-size: 13px;

values: mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-The_Importance_of_Being_Earnest_2002_film rootpage-The_Importance_of_Being_Earnest_2002_film vector-experimental-print-styles vector-nav-directionality skin-vector action-view