Memes
Notable or relevant traits which have been found to be expressed in specific models.
-
alexnet-head
Classification head common in early convnets
-
attention
Position independent dependency
-
autoregression
Sequential modeling of a joint probability by using the tower rule for conditional probaiblities
-
convolution
Translation invariant linear maps
-
idiotropy
Anisotropy of an unusual, notable kind
-
local-response-normalization
Normalization over the out-dims / features of a convolution used in alexnets
-
multiscale
The processing or representation of data along multiple scales
-
relu
Rectified linear units
-
residual
Use of identity or simple skips to formulate a network as correcting an ideally small error
-
small-kernels
The use of consistently small convolution kernels
-
softmax-multinomial
Modeling the output distribution as a multinomial emitted by a softmax
-
transposed-convolution
Transpose of a strided convolution as a linear operator
-
vgg-rule
Preservation of time complexity as a rule in convnet layers
-
weight-decay
Augmented loss which punishes large weights