JP2002008033A - Pattern detection device and method, image processing device and method, and neural network device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently perform the recognition and detection of an intended pattern in a prescribed category independently of the intended size in input data. SOLUTION: This pattern detection device has a plurality of processing channels 1-n each performing a processing corresponding to a different resolution or scale lever to an inputted pattern, and a coupling processing circuit 1203 for coupling the outputs of the processing channels 1-n. Each of the processing channels 1-n is provided with a plurality of sub-sampling neuron circuits 1201 for detecting and outputting a plurality of characteristics to a characteristic integration layer in conformation to each point obtained by sub-sampling the input data by local averaging and a group coding neuron circuit 1202 for integrating the outputs by normalized liner coupling.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pattern detecting apparatus and method for performing pattern recognition, detection of a specific subject, and the like, a parallel processing device such as a neural network, an image processing apparatus and method, and a neural network circuit. .

[0002]

2. Description of the Related Art Conventionally, in the field of image recognition and voice recognition, a type in which a recognition processing algorithm specialized for a specific recognition target is sequentially calculated and executed as computer software, or a dedicated parallel image processor (SIM) is used.
D, MIMD machine, etc.).

[0003] Among image recognition algorithms,
There is a demand for a method whose performance does not depend on the position, size, and the like of the recognition target on the screen, and many methods have been proposed so far. For example, by performing so-called conformal mapping conversion as preprocessing, it is possible to perform recognition invariant to scale and rotation.

[0004] Specifically, by performing Log-Polar coordinate transformation on the logarithm of the distance from the center point of the recognition target in the image and the rotation angle, the size change and rotation of the same target can be converted into the coordinate system after the conversion. Is converted to a translation.
Thereafter, when a feature amount such as a correlation coefficient is calculated, the recognition target is detected as the same feature amount. The invariance of the detection characteristic with respect to the position is obtained by sequentially shifting the center point of the conversion with time and performing detection at each position.

Further, a given image is subjected to multi-scale representation for each local region, and
It has been pointed out that a similar size-invariant detection may be performed by performing mal mapping conversion (Wechsler, H. 199).
2, 'Multi-scale and Distributed Visual Representat
ions and Mappings for Invariant-Low-Level Percepti
on ', in Neural Networks for Perception, Vol. 1, W
echssler H. Ed.pp.462-476., Academic Press, Bosto
n).

Further, as an example of another method, in a type in which a feature amount relating to a similarity with a recognition target model is calculated and recognized, a method which enables recognition regardless of size is as follows.
Using the model data to be recognized as a template model,
Expressed on different scales, template matching with the input image (or its feature vector) is sparse to dense.
(coarse to fine) (Rosenfeld an
d Vanderburg, 1977, Coarse-fine template matching,
IEEE Trans. Systems, Man, and Cybernetics, vol. 2,
pp. 104-107), a method of mapping an input pattern to a unique image function space obtained by principal component analysis of model images of objects of different sizes and calculating a distance from the model in a feature space (Japanese Patent Laid-Open No. No. 8-153198; Murase, Nayer, 1995,
Image spotting of 3D objects by multi-resolution and eigenspace representation, Transactions of Information Processing Society of Japan, vol.36, pp.223
4-2243; Murase and Nayar, 1997, Detection of 3D ob
jects in cluttered scenes using hierarchical eigen
space, Pattern Recognition Letters, pp. 375-384),
The position of the matching area based on the distance image data to be recognized,
A method of performing collation after calculating and normalizing the size (Japanese Patent Laid-Open No. 5-108804), and a method of recognizing collation by sequentially switching multiple resolution data relating to a recognition target from a low resolution level to a high resolution level (JP-A-8-315141).

As a method of using a time-series input image, a method of generating a plurality of competing hypotheses regarding a recognition target from an image, accumulating them temporally, and inputting them to a category classifier such as ART2 by Carpenter et al. (Seiber
t, et al. 1992, Learning and recognizing 3D object
s from multiple views in a neural system, in Neura
l Networks for Perception, vol. 1 Human and Machin
e Perception (H. Wechsler Ed.) Academic Press, pp.4
27-444).

[0008] As a pattern recognition method using a neural network model inspired by the information processing mechanism of a living body, a method of obtaining a scale and position invariant representation of the center of an object using a dynamic routing network (Anderson, et al. 1995) ,
Routing Networks in VisualCortex, in Handbook of B
rain Theory and Neural Networks (M. Arbib, Ed.), M
IT Press, pp.823-826, Olhausen et al. 1995, A Mult
iscale Dynamic Routing Circuit for Forming Size- a
nd Position-Invariant Object Representations, J. C
omputational Neuroscience, vol.2 pp.45-62.) In this method, hierarchical representation (multi-resolution representation) of image data with a plurality of different resolutions is performed in advance,
Information at different resolutions is mapped to an object-centered representation by routing information through control neurons that have the ability to dynamically set connection weights.

On the other hand, as an attempt to more faithfully incorporate an information processing mechanism based on a biological neural network, a neural network model circuit has been proposed in which information is transmitted and represented by a pulse train corresponding to an action potential (Mur).
ray et al., 1991 Pulse-Stream VLSI Neural Networks
Mixing Analog and Digital Techniques, IEEE Trans.
on Neural Networks, vol.2, pp.193-204.
No. 62157, JP-A-7-334478, JP-A-8-153148
And Japanese Patent No. 2879767).

[0010]

As described above, in order to realize pattern recognition in which the recognition performance remains unchanged for recognition targets of different scales, a predetermined mapping conversion (conformal mapping conversion) conventionally performed. ) Has a problem that it is difficult to obtain a scale-invariant feature unless the center point of the transformation is set appropriately.

In the method of performing template matching,
When matching with the template model at different scales represented in advance, high recognition performance cannot be obtained unless the object in the input image substantially matches one of the scales, that is, a very large number of different scales. A template model was required, and there was a problem in practicality.

In the method disclosed in Japanese Patent Application Laid-Open No. 8-153198, which uses a parametric eigenspace obtained by performing principal component analysis on a target model image for a finite number of different sizes, a change in the size is determined on the parametric eigenspace. And can recognize objects of different sizes continuously, but the covariance matrix has a large number of dimensions (eg, 16,384 dimensions in Murase and Nayer (1995)), and the computational cost of eigenvectors is very high. There was a problem that was large. In order to get enough accuracy to respond to size changes,
Standard size 1.1, 1.2, 1.3, 1.4, 1.5 (= α)
Eigenvectors are calculated by preparing reference images having different sizes of about five stages, and the input image is further α- ¹ times, α
It was necessary to convert the data into a size such as ^-2 times, α- ³ times, etc., and the size of the memory space required for processing and the operation time were enormous.

According to the method disclosed in Japanese Patent Application Laid-Open No. 8-315141, matching is performed in order from low resolution to high resolution on multi-resolution expression data relating to a prepared object in advance, so that in order to perform scale-invariant recognition, the data must be prepared in advance. Since it is necessary to set the multiplexing level of the resolution to be sufficiently large, the processing efficiency is low. Further, it is suitable for obtaining rough information with a small amount of memory use, but is not suitable for highly accurate recognition and detection.

A method by a dynamic routing network (Anderson et al., 1995; Olshausen et al., 1995)
In this case, there is a need for a mechanism for dynamically setting a connection between neuron elements between predetermined scale levels by a local competition process between control neurons, and there has been a problem that a circuit configuration becomes complicated.

Further, since the method of generating competing hypotheses and inputting them to the category classifier (Seibert et al. 1992) is based on a time-series image, the scale-independent recognition is performed from a single still image in the first place. It is difficult.

In the configuration using the analog circuit elements,
It is generally known that the circuit configuration can be simplified (reduced in the number of elements), speed and power consumption can be reduced compared to the digital method, but on the other hand, the reliability of input / output characteristics due to variations in individual device characteristics, Noise resistance has not been a problem.

[0017]

According to the present invention, in order to solve the above-mentioned problems, an input means for inputting a pattern and a pattern input from the input means are provided to a pattern detection device. A plurality of processing means for performing processing corresponding to different resolutions or scale levels; and a multiplexing processing means for combining outputs of the plurality of processing means, wherein each of the plurality of processing means A plurality of feature detection elements for detecting and outputting a plurality of features, respectively, corresponding to each point obtained by sampling according to the above method, and the multiplexing processing means combines the outputs of the plurality of feature detection elements. It is characterized by the following.

According to another aspect, an input means for inputting a pattern to the pattern detection apparatus, and a plurality of features are detected for each of a plurality of resolutions or scale levels with respect to the pattern input from the input means. A plurality of signal processing elements coupled to each other by predetermined coupling means, and a predetermined part of the plurality of signal processing elements is input within a predetermined time range. And outputting a pulse signal at an output level corresponding to the arrival time pattern of the plurality of pulse signals.

Further, according to another aspect, the pattern detecting device has an input means for inputting a pattern, and a plurality of processing means for performing pattern processing on the pattern input from the input means. A plurality of processing means detects a plurality of features having different resolutions or scale levels, outputs a detection pulse signal for each feature, and converts the detection pulse signal to have a different phase in the time axis direction. Prepare.

According to another aspect, an input means for inputting a pattern and processing corresponding to different resolutions or scale levels are performed on the pattern input from the input means. A plurality of processing means, the processing means corresponding to each point obtained by sampling the input data by a predetermined method, a plurality of feature detection elements for detecting a plurality of features, respectively, Control means for controlling the resolution or scale level by combining a plurality of the feature detection element outputs of the processing means with different resolutions or scale levels.

Further, according to another aspect, the neural network device has a plurality of channels each performing processing at a different scale level or resolution using a group of neurons having a hierarchical structure, and the group of neurons includes a plurality of channels. It comprises a group of collectively coded neurons that combine outputs across channels.

According to another aspect, in the pattern detection method, an input step of inputting a pattern and processing corresponding to different resolutions or scale levels are performed on the pattern input in the input step. A plurality of processing steps, and a multiplexing processing step of combining processing results of the plurality of processing steps. In each of the plurality of processing steps, each point obtained by sampling the input data by a predetermined method. In response to the above, a plurality of feature detection elements respectively detect and output a plurality of features, and in the multiplexing processing step, the outputs of the plurality of feature detection elements are combined.

According to another aspect, in the pattern detection method, an input step of inputting a pattern, and a plurality of signal processing elements that are coupled to each other by a predetermined coupling mechanism with respect to the pattern input in the input step. A detection step of detecting a plurality of features for each of a plurality of resolutions or scale levels by a feature detection layer comprising: a plurality of predetermined portions of the plurality of signal processing elements input within a predetermined time range. The pulse signal is output at an output level according to the arrival time pattern of the pulse signal.

According to another aspect, the pattern detection method includes an input step of inputting a pattern, and a plurality of processing steps of performing pattern processing on the pattern input in the input step. In a plurality of processing steps, the feature detection element detects a plurality of features having different resolutions or scale levels, outputs a detection pulse signal for each of the temporally localized features, and outputs the detection pulse signal in the time axis direction. It is characterized in that conversion is performed so that the phases are different.

According to another aspect, in the pattern detection method, an input step of inputting a pattern and processing corresponding to different resolutions or scale levels are performed on the pattern input in the input step. A plurality of processing steps, wherein the processing step includes a plurality of feature detection elements respectively detecting a plurality of features corresponding to each point obtained by sampling the input data by a predetermined method. And a control step of controlling the resolution or scale level by combining a plurality of feature detection element outputs for performing the processing steps corresponding to different resolutions or scale levels.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS <First Embodiment> An embodiment of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a diagram showing the entire configuration of the pattern detection / recognition apparatus of the present embodiment. Here, the pattern information is processed by a What route and a Where route. Wh
The at path mainly deals with information related to recognition (detection) of an object or a geometric feature, and the Where path mainly deals with information on the position (arrangement) of the object or feature.

The What path is a so-called Convolutional network structure (LeCun, Y. and Bengio, Y., 1995, "Co
nvolutional Networks for Images Speech, and Time S
eries "in Handbook of Brain Theory and Neural Netw
orks (M. Arbib, Ed.), MIT Press, pp. 255-258). However, it differs from the conventional one in that the interlayer connection in the same path can form mutual connection (described later). The final output of the What path corresponds to a recognition result, that is, a recognized target category. The final output of the Where path indicates a location corresponding to the recognition result.

The data input layer 101 is a photoelectric conversion element such as a CMOS sensor or a CCD element for detecting and recognizing an image, and is a voice input sensor for detecting and recognizing voice. Also, the analysis result of the predetermined data analysis unit (for example, principal component analysis, vector quantization, etc.)
May be inputted. The data input layer 101 performs data input common to the two paths.

Hereinafter, the case of inputting an image will be described. The What path includes the feature detection layer 102 ((1, 0),
(1, 1), ..., (1, N)) and the feature integration layer 103 ((2, 0),
(2, 1), ..., (2, N)).

The first feature detection layer (1, 0) is Gabor wavel
By multi-resolution processing by et conversion or the like, local low-order features (which may include color component features in addition to geometric features) of the image pattern are transferred to each position on the entire screen (or a predetermined sampling point over the entire screen). At the same location at a plurality of scale levels or resolutions, the number of feature categories, and the types of feature amounts (for example, when a line segment in a predetermined direction is extracted as a geometric feature, the Of the receptive field according to the geometrical structure (slope of the line segment), and is composed of neuron elements that generate pulse trains according to the degree.

The feature detection layers (1,0), (1,1),...
The feature integration layers (2,0), (2,1),... Form a set of processing channels at a plurality of resolutions (or scale levels) as a whole. In each processing channel, processing at the same scale level (or resolution) proceeds, and detection and recognition from low-order features to high-order features are performed by hierarchical parallel processing.

In each layer, a plurality of circuit elements belonging to different processing channels exist in a predetermined arrangement. Here, the arrangement configuration of the processing channels is shown in FIG.
3 will be described as an example.

In the examples of FIGS. 12 and 13, the same processing channel is formed together with the subsequent layers if the scale levels (resolutions) are the same, but if the feature category (here, the directional component) is the same, FIG. 12 shows that the integrated cells are arranged at close positions even if the processing channels are different. If the scale levels (ie, the processing channels) are the same, the integrated cells are integrated at close positions even if the feature categories are different. The arrangement of cells is shown in FIG. The functional differences between the components will be described later. In the feature integration layer (2,0), each circuit element inputs a Gabor wavelet transform feature as a lower-order feature, which will be described later, from the cell output of the feature detection layer (1,0).

In general, in the feature detection layer (1, k) (k is a natural number), an output from a plurality of feature integrated cells forming the same channel in the preceding (2, k−1) layer is received. The feature detection cells are configured to belong to the channel.

The feature integration layer (2,0) on the What path is
It has a predetermined receptive field structure, is composed of neuron elements that generate pulse trains, and integrates a plurality of neuron element outputs in the same receptive field from the feature detection layer (1, 0) (subsampling by local averaging, etc. (Combination processing of processing results at the scale level). In addition, each receptive field of neurons in the feature integration layer has a common structure among neurons as far as the same feature category and the same scale level are concerned.

Each feature detection layer (1, 1), (1, 2),.
(1, N)) and each feature integration layer ((2,1), (2,2), ...,
(2, N)) have a predetermined receptive field structure acquired by learning, and the former ((1,1),
..) Detects a plurality of different features in each feature detection module, and the latter ((2, 1),...) Integrates detection results for a plurality of features from the preceding feature detection layer.
However, the former feature detection layer is connected (wired) to receive the cell element output of the preceding feature integration layer belonging to the same channel.
Have been. The feature integration layer performs two types of processing.

The first sub-sampling includes averaging the output from a local region (local receptive field of the feature integrated layer neuron) from the feature detection cell population of the same feature category and the same scale level. The second process, which is a process of combining processing results at different scale levels, is a linear combination (or non-linear combination) of outputs of a plurality of feature detection cell populations in the same feature category and over a plurality of different scale levels. I do.

The Where path has a feature position detection layer ((3,0),..., (3, k)), and a predetermined (not necessarily all) feature integration layer on the What path. It takes input and participates in the output of low, middle and high order feature locations. Hereinafter, a more detailed description of each layer of the Where path will be omitted.

As shown in FIG. 2A, a structure for connecting the neuron elements 201 between the layers includes a signal transmission unit 203 (wiring or delay line) corresponding to an axon or a dendrite of a nerve cell. And the synapse circuit S202. (A) of FIG.
Now, the configuration of the connection related to the output from the neuron group (n _i ) (input when viewed from the cell N) of the feature integrated (detected) cell that forms a receptive field for a certain detected (integrated) cell (N) is described below. Is shown. Signal transmission unit 203 indicated by bold line
Constitutes a common bus line, and pulse signals from a plurality of neurons are transmitted in time series on this signal transmission line. The same configuration is adopted when receiving an input from a cell (N) of an output destination. In this case, the input signal and the output signal may be divided and processed on the time axis with exactly the same configuration, or two systems for input (dendritic side) and output (axon side) may be used. The processing may be performed by providing the same configuration as that of FIG.

The synapse circuit S202 has an interlayer connection (a connection between a neuron on the feature detection layer 102 and a neuron on the feature integration layer 103, and each layer has a connection to a subsequent layer and a preceding layer. May be involved)
Some are involved in connections between neurons in the same layer. The latter is mainly used, if necessary, mainly for coupling a pacemaker neuron described later with a feature detection or feature integration neuron.

In the synapse circuit S202, the so-called excitatory coupling amplifies the pulse signal, and the inhibitory coupling conversely attenuates. When transmitting information using a pulse signal, amplification and attenuation can be realized by any of amplitude modulation, pulse width modulation, phase modulation, and frequency modulation of the pulse signal.

In this embodiment, the synapse circuit S20
2 is mainly used as a phase modulation element of the pulse, the amplification of the signal is converted as a substantial advance as a characteristic quantity of the pulse arrival time characteristic, and the attenuation is converted as a substantial delay. That is,
The synaptic connection gives the arrival position (phase) on the time axis peculiar to the feature of the output neuron as described later, and qualitatively the excitatory connection leads the phase of the arrival pulse with respect to a certain reference phase. In the case of inhibitory coupling, a delay is similarly given.

In FIG. 2A, each neuron element
n _j outputs a pulse signal (spike train) and uses a so-called integral-and-fire type neuron element as described later. As shown in FIG. 2 (C),
The synapse circuit and the neuron element may be combined to form a circuit block.

Each feature position detection layer 107 in the Where path
Receives the output of the What path feature integration layer 103, retains the positional relationship on the data input layer 101, and uses each point on the coarsely sampled grid points for recognition of the feature extraction results on the What path. Only neurons corresponding to useful components (pre-registered from recognition category patterns) respond by filtering or the like. For example, Wh
In the uppermost layer in the ere path, neurons corresponding to the category of the recognition target are arranged on a lattice, and represent at which position the corresponding target exists. Also, the neurons in the middle layer in the Where path receive sensitivity from the top down from the upper layer and respond only when a feature that can be arranged around the location of the corresponding recognition target is detected. Adjustment or the like can be performed.

The hierarchical feature detection in which the positional relationship is maintained is performed by W
When performing the here route, the receptive field structure is local (for example, elliptical) and the size gradually increases in the upper layer (or, from the middle layer to the upper layer, is larger than one pixel on the sensor surface). (The size is constant), the characteristic elements (graphic elements, graphic patterns)
The positional relationship between them can be such that each feature element (graphic element) is detected in each layer while preserving the positional relationship on the sensor surface to some extent.

As another output form of the Where path, a gaze area of a predetermined size on the data input layer is obtained from the saliency map of the feature obtained based on the output result of the feature detection layer (1, 0). It may be set and output the position and size of the area and the presence or absence of the recognition target category in the area. In still another embodiment, the size of the receptive field becomes larger in the upper layer hierarchically, and only the neuron corresponding to the detected symmetric category that outputs the maximum value is fired in the uppermost layer. A neural network may be used. In such a system, information on the arrangement relationship (spatial phase) in the data input layer is stored to some extent in the uppermost layer (and each intermediate layer).

Next, the neurons constituting each layer will be described. Each neuron element is a so-called integrate-and-
This is an extended model based on fire neurons.
The point where the result of linearly adding the input signal (pulse train corresponding to the action potential) spatiotemporally exceeds the threshold value is fired, and a pulse-like signal is output.
Same as ate-and-fire neurons.

FIG. 2B shows an example of a basic configuration representing the operation principle of a pulse generating circuit (CMOS circuit) as a neuron element, and a known circuit (IEEE Trans.
etworks Vol. 10, pp. 540). Here, it is configured to receive excitatory and inhibitory inputs as inputs.

Hereinafter, the operation principle of the pulse generation circuit will be described. Excitatory input side capacitor C1 and resistor R
The time constant of one circuit is smaller than the time constant of the capacitor C2 and the resistor R2.
T2 and T3 are shut off. Note that the resistance is actually
It is composed of a transistor as an active load.

The potential of the capacitor C1 increases,
Above that of C2 by the threshold of transistor T1, transistor T1 becomes active, further activating transistors T2 and T3. Transistors T2, T
Reference numeral 3 denotes a current mirror circuit, and the output of the circuit shown in FIG. 2B is output from the capacitor C1 by an output circuit (not shown). When the amount of charge stored in the capacitor C2 becomes maximum, the transistor T1 is turned off, and as a result, the transistors T2 and T3 are also turned off, so that the positive feedback becomes zero.

During the so-called refractory period, the capacitor C2 discharges, and the neuron does not respond unless the potential of the capacitor C1 is higher than the potential of the capacitor C2 and the difference does not exceed the threshold value of the transistor T1. Capacitor C1, C
A periodic pulse is output by repeating the alternate charging and discharging of No. 2 and its frequency is generally determined according to the level of the excitatory input. However, depending on the presence of the refractory period, it is possible to limit the maximum value or to output a constant frequency.

The potential of the capacitor, and hence the amount of charge storage, is
It is temporally controlled by a reference voltage control circuit (time window weight function generation circuit) 204. Reflecting this control characteristic,
This is weighted addition within a time window to be described later for the input pulse (see FIG. 7). This reference voltage control circuit 204 is based on an input timing from a pacemaker neuron to be described later (or an input of mutual coupling with a neuron in a subsequent layer) or another mechanism (a synchronous firing signal based on a weak interaction between neurons to be described later). , Reference voltage signal ((B) of FIG. 7)
Corresponding to the weight function of

Although there is a case where the input of the suppressiveness is not always necessary in the present embodiment, it is possible to prevent the divergence (saturation) of the output by making the input from the pacemaker neuron to the feature detection layer neuron to be suppressive. Can be.

In general, the relationship between the sum of the input signals and the output level (pulse phase, pulse frequency, pulse width, etc.) changes depending on the sensitivity characteristics of the neuron.
The sensitivity characteristic can be changed by a top-down input from an upper layer. Below, for convenience of explanation,
It is assumed that the circuit parameters are set so that the frequency of the pulse output according to the sum of the input signals rises steeply (thus, almost in binary in the frequency domain), and the output level (the timing of adding the phase modulation) is determined by the pulse phase modulation. Etc.) fluctuate.

As a pulse phase modulating section, a circuit as shown in FIG. 5 to be described later may be additionally used. As a result, the weight function within the time window is controlled by the reference voltage. As a result, the phase of the pulse output from this neuron changes, and this phase can be used as the output level of the neuron.

The time τ _w1 corresponding to the maximum value of the weight function as shown in FIG. 7B that gives the temporal integration characteristic (reception sensitivity characteristic) of the pulse subjected to the pulse phase modulation by the synaptic connection is generally The time is set earlier than the estimated arrival time τ _{s1 of the} pulse unique to the feature given by the synaptic connection. As a result, a pulse arriving earlier within a certain range from the estimated arrival time (in the example of FIG. 7B, a pulse that arrives too early is attenuated) is a pulse signal having a high output level in the neuron receiving it. Is integrated over time. The shape of the weight function is not limited to a symmetric shape such as Gaussian, but may be an asymmetric shape. It should be noted that the center of each weighting function in FIG. 7B is not the expected pulse arrival time for the above-described purpose.

The phase of the neuron output (before synapse) is based on the beginning of the time window as described later, and the delay (phase) from the reference time is the time when a reference pulse (by a pacemaker output or the like) is received. It has output characteristics determined by the amount of charge storage. The details of the circuit configuration for providing such output characteristics will not be described because they are not the focus of the present invention. The post-synaptic pulse phase is obtained by adding the pre-synaptic phase to the unique phase modulation amount given by the synapse.

Note that a known circuit configuration may be used in which an oscillation output is output with a predetermined timing delay when the sum of inputs obtained by using a window function or the like exceeds a threshold value.

The configuration of the neuron element is a neuron belonging to the feature detection layer 102 or the feature integration layer 103. If the firing pattern is controlled based on the output timing of a pacemaker neuron to be described later, a pulse from the pacemaker neuron is used. After receiving the output, the circuit configuration may be such that the neuron outputs a pulse with a phase delay according to the input level (simple or weighted sum of the above input) received from the receptive field of the preceding layer. . In this case, before the pulse signal is input from the pacemaker neuron, there is a transitional transition state in which the neurons output pulses at random phases with respect to each other according to the input level.

When a pacemaker neuron is not used as described later, between the neurons (the feature detection layer 10
Based on the synchronizing firing signal caused by the mutual coupling between (2 and the feature integration layer 103) and the network dynamics, the firing timing of the output pulse of the feature detecting neuron according to the input level as described above is controlled. The circuit configuration may be simple.

The neurons of the feature detection layer 102 have a receptive field structure corresponding to the feature category as described above, and input pulse signals (current values) from the neurons of the preceding layer (the input layer 101 or the feature integration layer 103). Or a potential), a non-decreasing and non-linear function that asymptotically saturates to a certain level, such as a sigmoid function, according to the sum when the weighted sum (described later) of the time window function exceeds a threshold value. That is, pulse output is performed at an output level that takes a so-called squashing function value (in this case, the output is given by a phase change, but may be a change based on frequency, amplitude, and pulse width).

The feature detection layer (1,0) detects a pattern structure (lower-order feature) having a predetermined spatial frequency in a local area of a certain size and having a vertical directional component. Assuming that there is a neuron N1, if a corresponding structure exists in the receptive field of the neuron N1 on the data input layer 101, a pulse is output with a phase corresponding to the saliency (contrast). Such a function can be realized by a Gabor filter. Hereinafter, a feature detection filter function performed by each neuron of the feature detection layer (1, 0) will be described.

In the feature detection layer (1, 0), multi-scale
It is assumed that Gabor wavelet transform represented by a filter set of multi-directional components is performed, and each neuron (or each group including a plurality of neurons) in a layer has a predetermined G value.
It has an abor filter function.

The feature detection layer 102 collectively groups a plurality of neuron groups each having a receptive field structure corresponding to a convolution operation kernel of a plurality of Gabor functions having a constant scale level (resolution) and different direction selectivity. Form one channel. At this time, as shown in FIG. 13, neuron groups forming the same channel have different direction selectivity, and neuron groups having the same size selectivity may be arranged at positions close to each other, as shown in FIG. The neuron groups belonging to the same feature category and belonging to different processing channels may be arranged close to each other.

This is because the arrangement configuration shown in each of the above-described drawings is easier to realize in terms of the circuit configuration, for the convenience of the joint processing described later in the collective encoding. FIG. 12, 1
The details of the circuit configuration of No. 3 will also be described later.

The Gabor wavelet is given by the following equation (1)
Has a shape obtained by modulating a sine wave having a constant directional component and a spatial frequency with a Gaussian function, as given by
It is specified by the index m of the scaling level and the index n of the direction component. As a wavelet, this set of filters has similar functional shapes and different main directions and sizes. This wavelet is that the function form is localized in the spatial frequency domain and the real space domain, the simultaneous uncertainty regarding the position and the spatial frequency is minimized, and it is the function that is most localized in the real space and the frequency space Known (J, G. Daugman (1985), Unce
rtainty relation for resolution in space, spatial
frequency, and orientation optimized by two-dimens
ional visual cortical filters, Journal of Optical
Society of America A, vol.2, pp. 1160-1169).

[0068]

[Outside 1]

Here, (x, y) is a position in the image, a is a scaling factor, θ _n is a directional component of the filter, and W
Is the fundamental spatial frequency, σ _x and σ _y are the x directions of the filter function,
It is a parameter that gives the size of the spread in the y direction. In the present embodiment, θ _n is 0 degree, 30 degrees, 60 degrees, 9 in six directions.
Taking values of 0 degrees, 120 degrees, and 150 degrees, a is set to 2 and m is given as an integer taking a value of 1 to 3.

The parameters σ _x ,
σ _y and a are desirably set such that there is no bias (sensitivity) to a specific spatial frequency and direction by appropriately and homogeneously overlapping each other in the Fourier domain. Therefore, for example, if the half-value levels for the amplitude maximum value after the Fourier transform are designed to be in contact with each other in the Fourier domain,

[0071]

[Outside 2] Becomes Here, U _H and U _L are the maximum and minimum values of the spatial frequency band covered by the wavelet transform, and M gives the number of scaling levels in that range.

Further, the structure of the receptive field of the feature detecting cell given by the equation (1) has scale selectivity and direction selectivity of a predetermined width determined by σ _x and σ _y . That is, since the Fourier transform of the equation (1) has a Gaussian function shape, a peak tuning (sensitivity) characteristic is given to a specific spatial frequency and direction. Gabor filters with different scale indices have different size selectivities, because the size (spread) of the Gabor filter kernel changes according to the scale index m. In collective coding described later, outputs from a plurality of feature detection cells whose sensitivity characteristics mainly overlap with each other with respect to size selectivity are integrated.

Each of the filters g _mn (x, y) and the input grayscale image
The Gabor wavelet transform is performed by performing a dimensional convolution operation. That is,

[0074]

[Outside 3]

Here, I is an input image, and Wmn is a Gabor wavelet transform coefficient. A set of W _mn (m = 1, 2, 3; n = 1,..., 6) is obtained at each point as a feature vector. ' ^* ' ^Indicates complex conjugate.

Each neuron in the feature detection layer (1, 0) is represented by g _mn
Has a receptive field structure corresponding to G _mn having the same scale index m has the same size receptive field, and the corresponding kernel g _mn size has a size corresponding to the scale index in operation. here,
30 × 30, 1 on the input image in order from the coarsest scale
The size was 5 × 15, 7 × 7.

Each neuron outputs a non-linear squashing function of a wavelet transform coefficient value obtained by performing a product-sum input of a distribution weighting coefficient and image data (here, a phase reference; where frequency, amplitude , A pulse width reference may be used). As a result,
As an output of the entire layer (1, 0), Gaborwa of Expression (4) is used.
This means that velet conversion has been performed.

Since the number of sampling points (the number of positions for performing feature detection and the like) for each scale level is the same, the feature expression at each point (location) extracted by the Gabor wavelet transform is Are schematically represented in a hierarchical structure as shown in FIG. 22 having different representative areas (the points on the input data representing the features). A multiple representation of the feature over a plurality of scale levels is obtained for each position (same position) in the image, which can be connected to a collective encoding process for corresponding to an arbitrary scale level described later. The points (circles) in FIG. 22 represent the same points on the input data at each scale level.

On the other hand, the subsequent feature detection layers ((1, 1), (1,
Each of the neurons 2),..., (1, N)) is different from the above-described detection layer and forms a receptive field structure for detecting a characteristic unique to the pattern to be recognized by a so-called Hebb learning rule or the like. The size of the local region where the feature detection is performed gradually becomes closer to the size of the entire recognition target in a later layer, and a medium-order or higher-order feature is geometrically detected. For example, in the case of performing face detection and recognition, the middle-order (or higher-order) features represent features at the level of graphic elements such as eyes, nose, and mouth that constitute the face.

In different processing channels, if the same hierarchical level (the complexity of detected features is the same level), the detected features differ in the same category, but are detected on different scales. It is to be. For example, “eye” as a second-order feature is detected as “eye” having a different size in a different processing channel. That is, detection is attempted in a plurality of processing channels with different scale level selectivity for a given size "eye" in the image. In addition, the feature detection layer neurons are supposed to suppress output (shunting inhibition) to stabilize output.
May be provided based on the output of the preceding layer.

Next, the feature integration layer 103 ((2,0), (2,
The neurons 1),... Will be described. As shown in FIG. 1, the feature detection layer 102 (for example, (1, 0)) to the feature integration layer 10
The connection to 3 (for example, (2,0)) is configured to receive an excitatory connection input from a neuron of the same feature element (type) in the preceding feature detection layer in the receptive field of the feature integration neuron, As described above, the neurons in the feature integration layer 103 are sub-averaged by local averaging for each feature category (calculation of the average value, calculation of the representative value, calculation of the maximum value of the input from the neurons forming the receptive field of the feature detection neuron, etc.). Sampling (sub-sampling neurons)
And those that combine outputs for features of the same category across different scales (processing channels) (collective coded neurons).

According to the former, a plurality of pulses of the same kind of feature are input, and they are integrated and averaged in a local area (receptive field) (or a representative value such as the maximum value in the receptive field). ), The position fluctuation of the feature,
Deformation can be reliably detected. For this reason, the receptive field structure of the feature integration layer neuron is uniform regardless of the feature category (for example, each is a rectangular region of a predetermined size, and the sensitivity or the weight coefficient is uniformly distributed therein). You may comprise so that it may become.

The latter, population coding
The mechanism will be described in detail. In collective coded neurons, outputs from multiple sub-sampling neurons at the same hierarchical level (with the same complexity of graphic features) but belonging to different processing channels with the same features and in the same feature integration layer By taking the normalized linear combination of For example, in the feature integration layer (2,0) receiving the output of the feature detection layer (1,0) that performs Gabor wavelet conversion,
Gabors belonging to different processing channels with equal direction selectivity
The outputs corresponding to the filter set {g _mn } (constant n, m = 1, 2,...) Are integrated by a linear combination or the like.

More specifically, if p _ij (t) has a direction component selectivity of i
The output of a sub-sampling neuron whose scale selectivity becomes j, and q _ij (t) is a population code (population code) having the same selectivity, represents the linear combination of the normalized output of the sub-sampling neuron. Expression (5) and Expression (6) representing a normalization method thereof are given. In addition,
Equations (5) and (6) represent the output state transitions of the sub-sampling neurons and the collective coding neurons as discrete time transitions for convenience of explanation.

[0085]

[Outside 4]

Here, w _{ij, ab} is a neuron (or a group of neurons) having a plurality of different selectivities (sensitivity characteristics).
From the subsampling neuron output of (feature category, i.e., the index of direction component selectivity is a and the index of scale level selectivity is b, the index of direction component selectivity is i and the index of scale level selectivity is j Coupling coefficients representing contributions (to the collective coding neurons). w _{ij, ab} is a filter function centered on directional component index i and scale level index j
(Selectivity), typically | i | and | jb | function shapes (w
_{ij, ab} = f (| ia |, | jb |)).

As will be described later, in the collective coding by linear combination via w _{ij, ab} , q _ij represents the feature category (direction component) and scale in consideration of the detection level of other selective neurons. The purpose is to give the probability of existence for the level. C is a normalization constant, λ,
β is a constant (typically β is 1-2, β
Is 2, there is an effect of compressing and reducing the contribution of the low-level noise component as compared with the case where β is 1.) C is p even if the sum of the collective codes for a feature category is almost zero.
It is a constant for preventing _ij from diverging. Note that q _ij (0) = p _ij (0) in the initial state when the system is started.

In correspondence with FIG. 12, in equations (5) and (6), addition is performed only for the scale level selectivity index. As a result, collective coded neurons have different scale levels (processing channels) for the same feature category
Will be output (proportion proportional to the existence probability) for each feature belonging to.

On the other hand, as in the case of FIG. 13, generally, addition is further performed on the directional component selectivity index, so that collective encoding is performed even on intermediate levels of a predetermined number of directional components. You can build a system to do. In this case, the parameter (formula (7) described later)
By appropriately setting β and w _{ij, lk in} (8), in the configuration shown in FIG. 13, each collective coded neuron has a feature existence probability (for each scale level and each feature category). Proportional amount) can be output.

As shown in equation (5), the collective code q _ij (t)
Is obtained by a normalized linear combination of the outputs of neurons with different scale-level sensitivity characteristics. When q _ij (t) that has reached the steady state is appropriately normalized (for example, normalized by the sum of q _ij ) so that the value is between 0 and 1, q _ij has a directional component. i gives the probability that the scale level corresponds to j.

Therefore, in order to explicitly obtain the scale level corresponding to the size of the target in the input data as a value,
A curve for fitting q _ij may be determined to estimate the maximum value, and the corresponding scale level may be determined. The scale level obtained in this way generally indicates an intermediate value of the preset scale level.

FIG. 23 is a diagram showing an example of collective encoding of the scale level. The horizontal axis represents the scale level, and the vertical axis represents the cell output. The output is equivalent to the pulse phase, and a neuron having a peak sensitivity at a specific scale has a lower output level for a feature having a size deviated from that scale as compared to a feature having a size corresponding to the specific scale. That is, a phase delay occurs.

The figure shows a sensitivity curve (so-called tuning curve) relating to the scale selectivity of each feature-detected cell, each cell output, and a collective code integrated output obtained by integrating them (moment related to the scale level of each cell output). , Ie, a linear sum). The position on the horizontal axis of the collective code integration output reflects the estimated value of the scale (size) of the recognition target.

In this embodiment, the scale level is not actually obtained explicitly, and the output from the feature integration layer to the feature detection layer is q _ij (or may be normalized q _ij ). That is, FIG.
In both cases 2 and 13, the feature integration layer 103 to the feature detection layer 1
The output to 02 is not the output from the sub-sampling neuron, but the output of the collective coding neuron, so that finally, like q _ij after normalization above,
It is collectively expressed as the detection probability of a specific object over a plurality of scale levels (resolutions).

In the circuit configuration of the feature integration layer 103 shown in FIG. 12, the sub-sampling neuron circuit 1201 first outputs the neuron output having the same size selectivity as each feature category among the neurons output from the preceding feature detection layer. Receives at the local receptive field of the subsampling neuron and performs local averaging. Each sub-sampling neuron output is sent to the joint processing circuit 1203. At this time, as described later, a pulse signal from each neuron is converted by a synapse circuit (not shown) into a predetermined phase amount (for example, when β in Expression (6) is 2, a square corresponding to the output level of the feature detection neuron). ) And propagated through the local common bus. However, the wiring between the neurons may be physically and independently wired without using a common bus.

The joint processing circuit performs processing corresponding to equations (5) and (6), and collectively encodes information having the same feature category but different size selectivity (across multiple processing channels).

Further, in FIG. 12, collective encoding is performed on the output of sub-sampling neurons having the same feature category (directional component selectivity). On the other hand, in the circuit configuration shown in FIG. The processing shown in the following Expressions (7) and (8) is performed by the joint processing circuit that performs over the whole of

[0098]

[Outside 5]

Next, a method of calculating the scale level of the recognition target will be described. As a result of the above processing, it is possible to calculate the average activity level scale level of the recognition target, based on (subsampling neuron output level) P _a of each processing channel. The specifically estimated scale level S is expressed, for example, as follows.

[0100]

[Outside 6]

[0102] Here, the average activity level of P _a processing channel a, S _a is the scale level of the processing channels a (or scale index), p _ia indicates the activity level of the sub-sampling neuron that belongs to the processing channel a . Also,
The use of P _a, in the processing of a subsequent layer can be used in activity level control of neurons belonging to each processing channel.

[0102] For example, it is possible to generate a signal for controlling the firing threshold of the neuron of the subsequent layer the P _a. In this case, for the maximum channel of Pa, the firing threshold of the succeeding layer is reduced (effectively the feature detection sensitivity is increased), and for the other channels, the firing threshold of the subsequent layer is increased (effectively the feature detection sensitivity is reduced). it makes it possible to activate only the maximum channel P _a (scale level), to achieve a less susceptible robust processing and low power consumption to noise. Further, performs threshold control of the subsequent layer in accordance with the value of P _a (relatively higher P _a higher channel, to lower the threshold)
This enables adaptive control of feature detection sensitivity based on the average channel activity level in the low-order feature extraction stage.

On the other hand, P _a for each channel in the (2,0) layer
Can be set such that signal amplification / attenuation (advance / delay of the pulse phase) according to the above is performed on the output from each collective encoding neuron. Figure 1
FIG. 5 shows a configuration of such a channel activity control circuit. This channel activity control circuit 1502 is similar to FIG.
The result obtained by processing the output of the collective encoding neuron by the average activity calculation circuit 1501 is set between the 13 collective encoding neurons and the feature detection layer which is the next layer.

In the final layer, the existence probability of a recognition target as a higher-order feature over a plurality of channels depends on the activity level of a neuron (ie, firing frequency, firing spike phase, etc.).
Is expressed as In the Where processing path (or when the position information of the detection / recognition target is also detected in the final layer), the existence probability of the target corresponding to the position (place) in the input data in the final layer ) Is detected as the activity level of each neuron.

The collective coding may be obtained by a linear combination without normalization. However, the collective coding may be easily affected by noise, and it is preferable to perform the normalization. The normalization shown in equations (6) and (8) is at the neural network level due to so-called shunting inhibition, and the linear combination as shown in equations (5) and (7) (Lateral connection).

FIG. 14 shows an example of the normalizing circuit when β is 2. This normalization circuit includes a sum-of-squares calculation circuit 1403 for calculating the sum of squares of the outputs of the feature detection cells n _ij belonging to different processing channels, and a shunt type suppression circuit 1404 for mainly normalizing Expression (6). , And a linear sum circuit 1405 for obtaining and outputting a linear sum of Expression (5).

In the square sum calculation circuit 1403, the interneuron (in) holds (pools) the square value of each feature detection cell.
ter-neuron) element 1406 is present and the interneuron 140
Each synapse coupling element 1402 that provides a coupling to 6 provides a pulse phase delay (or pulse width modulation, pulse frequency modulation) corresponding to the square of the output of the feature detection cell 1401.

The shunt type suppression circuit 1404 outputs a variable resistance element, a capacitor, and an output of the feature detection cell 1401 which are proportional to the reciprocal of a value obtained by multiplying the output of the interneuron 1406 by a predetermined coefficient (λ / C). Pulse phase modulation circuit giving a square
(Or a pulse width modulation circuit or a pulse frequency modulation circuit).

Next, a modification of the channel processing will be described. A configuration in which collective coding is performed for each processing channel as described above, and each processing channel output is transmitted to a subsequent layer (that is, a configuration in which the configuration of FIG. 12 or 13 is cascaded to the subsequent layer) ), To improve processing efficiency and reduce power consumption, the feature integration layer (2,
The output of the collective coded neuron may be propagated only to the feature detection cells (in the next layer) belonging to the same channel as the processing channel that gives the maximum response level in 0).

In this case, in addition to the configuration shown in FIGS. 12 and 13, a maximum input detection circuit, a so-called winner-take-off circuit, is used as a processing channel selection circuit that receives the output of the collective encoding neuron circuit and gives the maximum response level. All circuit (W
TA circuit) is set to exist between the output of the feature integration layer (2,0) and the next feature detection layer (1,1). The processing channel selection circuit may be set for each position of the feature integration layer, or may be set as a circuit for calculating the maximum response level for each processing channel for the entire input data regardless of the location, one for the layer. Is also good.

As the WTA circuit, for example, Japanese Patent Application Laid-Open No. 08-321
Known structures described in Japanese Patent No. 747, USP5059814, USP5146106 and others can be used. FIG. 16A schematically shows a configuration in which the output of only the processing channel exhibiting the maximum response of the feature integration layer is propagated to the next layer, the feature detection layer, by the WTA circuit in the feature integration layer. This is shown in FIG.
5 channel activity control circuit 1502 to gating circuit 16
Replaced with 02.

The gating circuit 1602 is shown in FIG.
And a channel selection circuit for transmitting the output of each neuron from the processing channel exhibiting the maximum average output level to the same channel in the next layer, as shown in FIG. 1604.

In the subsequent feature integration layer (2, k) (k is 1 or more), such a processing channel selection circuit is not necessarily required. For example, the output of the feature integration layer after higher-order feature detection is performed. The processing channel selection in the integrated layer of the low-order or medium-order features may be performed by feedback via the processing channel selection circuit. This concludes the description of the modification of the channel processing. Note that the flow of the sub-sampling, combination processing, and collective encoding as shown in FIGS. 12 and 13 is not limited to the configuration in which the flow is performed in the feature integration layer. For example, a separate layer for the combination processing and collective encoding is provided. Needless to say, this may be done.

Performing collective encoding on the output from the same type of feature category detection neuron at different scale levels has the following effects. That is,
The target size is different from the preset scale level,
It is possible to perform detection (or recognition) with high accuracy even at an intermediate level between them. In addition, by performing scale-multiplexed hierarchical parallel processing, even when objects of different sizes exist close to each other or overlap each other, it is possible to stably select a plurality of scale levels (or resolutions) appropriately. It becomes possible to recognize and detect the target. Further, a model corresponding to an enormous number of templates for handling an object of an arbitrary size is not required, and processing at a small number of scale levels is sufficient.

Further, in the configuration in which the processing channels are spatially divided up to the final layer as in the present embodiment, compared with the configuration in which the processing channels are developed on the time axis as described in Embodiments 2 and 3 described later. Collective encoding, such as joint processing between processing channels, can be performed without complicated timing control. Furthermore, since the differences in processing channels are output as firing characteristics of physically different neurons, recognition / detection processing can be performed in parallel in a short time on multiple scales, compared to the case where processing at each scale level is handled in chronological order. There is also an effect that can be performed.

Even when objects having substantially the same size exist close to each other or partially overlap each other, a plurality of partial features due to a local receptive field structure, a sub-sampling structure, and the like can be used. Needless to say, the recognition and detection performance of the object is maintained by the integrated detection mechanism.

Next, a method of pulse encoding and detecting a two-dimensional figure pattern will be described. FIG. 3 shows the feature integration layer 103.
FIG. 3 is a diagram schematically showing a state of propagation of a pulse signal from a to a feature detection layer 102 (for example, from a layer (2,0) to a layer (1,1) in FIG. 1). Each neuron n _i (n ₁
To n ₄ ) respectively correspond to different feature amounts (or feature elements), and the neuron n ′ _j on the feature detection layer 102 side is a higher-order feature obtained by combining the features in the same receptive field. (Graphic element).

The connection between each neuron includes
Between and neuron n_iFrom neuron n ' _jSynaptic connection to
(S_ij) Delay due to time delays etc.
Delay, and as a result, via the common bus line 301
The neuron n '_jPulse train P arriving at_iIs the feature integration layer 1
Learning as long as pulse output is made from each neuron in 03
The amount of delay in synaptic connections determined by
(In FIG. 3A,
P_Four, P_Three, P_Two, P₁In order of arrival).

FIG. 3B shows a case where the time window synchronization control is performed using a timing signal from a pacemaker neuron to be described later, and the feature integrated cells n ₁ , n ₂ , The timing of pulse propagation from n ₃ (representing different types of features) to a certain feature detection cell (n ' _j ) on layer number (1, k + 1) (which performs higher-order feature detection) Is shown.

FIG. 6 is a diagram showing a network configuration when the feature detection layer neuron has an input from a pacemaker neuron. In FIG. 6, pacemaker neurons 603 (n _p ) form feature receptive neurons 602 (n _j , _nk etc.) that form the same receptive field and detect different types of features.
, And form the same receptive field as those, and receive excitatory connections from the neurons 601 on the feature integration layer (or input layer). Then, at a predetermined timing (or frequency) determined by the sum of the inputs (or to control the state of the receptive field as a whole, such as an average activity level of the entire receptive field). Pulse output is performed to the feature detection neuron 602 and the feature integration neuron.

Each of the feature detecting neurons 602 is configured such that the time windows are phase-locked to each other using the input as a trigger signal. However, as described above, the phase is not locked before there is a pacemaker neuron input. Each neuron outputs a pulse with a random phase. In the feature detection neuron 602, a pacemaker neuron 603 is used.
Before the input from, the time window integration described later is not performed.
The same integration is performed using a pulse input from the pacemaker neuron 603 as a trigger.

Here, the time window is determined for each feature detection cell (n′i), and is common to each neuron in the feature integration layer forming the same receptive field for the cell and the pacemaker neuron 603, Gives the time range for time window integration.

The pacemaker neuron 603 at the layer number (1, k) (k is a natural number) outputs the pulse output to the layer number (2, k-1)
Each integrated cell and its pacemaker neuron 60
By outputting to the feature detection cell (layer number (1, k)) to which 3 belongs, a timing signal for generating a time window when the feature detection cell temporally adds inputs is given. The start time of this time window is a reference time for determining the arrival time of the pulse output from each integrated cell. That is, the pacemaker neuron 603 gives a pulse output time from the feature integration cell and a reference pulse for time window integration in the feature detection cell.

The aforementioned coupling circuit outputs a pulse corresponding to the collective coding level obtained by the formula (5) or (7) to each collective coding neuron, and outputs the characteristic of the layer number (2, k). The collective coded neurons as output cells (n ₁ , n ₂ , n ₃ ) of the integrated layer receive pulse input from the pacemaker neuron of the layer of the layer number (1, k + 1) and detect the characteristics of the preceding stage If the input from the layer or the sensor input layer (layer number (1, k)) causes the combined circuit output to be at a sufficient level (for example,
If the average number of input pulses in a certain time range or time window is larger than the threshold value or the pulse phase is advanced), the pulse output from the pacemaker at the time of the falling edge of the pulse is performed.

The above-described sub-sampling neurons are not controlled by any pacemaker neurons, and have an average (independent phase for each sub-sampling neuron) from the feature detection cells in the preceding (1, k) layer. The sub-sampling process is performed based on the output level within a time window having The pulse output timing control from the sub-sampling neuron to the connection processing circuit is also performed without passing through the pacemaker neuron, and the same applies to the pulse output from the connection processing circuit to the collective encoding neuron.

As described above, in the present embodiment, the feature-integrated cells (sub-sampling neurons, collective coding neurons, etc.) are controlled by the timing control from the pacemaker neuron on the feature detection layer of the preceding layer number (1, k). Is not configured to receive. This is because in a feature-integrated cell, the phase (frequency, pulse width, amplitude, etc.) determined by the input level within a certain time range (such as the temporal integration value of the input pulse) is not the arrival time pattern of the input pulse. This is because the pulse is output in the present embodiment, but the timing at which the time window is generated is not so important. Note that this is not intended to exclude a configuration in which the feature-integrated cells receive the timing control from the pacemaker neuron in the feature detection layer in the preceding layer, and it goes without saying that such a configuration is also possible.

Each pulse is given a predetermined amount of phase delay when passing through the synapse circuit, and further, reaches a feature detecting cell through a signal transmission line such as a common bus. The arrangement of the pulses on the time axis at this time is indicated by the pulses (P ₁ , P ₂ , P ₃ ) indicated by the dotted lines on the time axis of the feature detection cells.

Each pulse (P ₁ ,
P ₂ , P ₃ ) time window integration (usually, one integration; however, charge accumulation by multiple time window integrations or averaging of multiple time window integrations may be performed. If the result is larger than the threshold value, a pulse output (P _d ) is made based on the end time of the time window. The learning time window shown in FIG. 3B is referred to when a learning rule described later is executed.

[0129] Figure 4 is a diagram showing the structure of the synapse circuit S _i. Figure (A) in 4, the synapse circuit 202 (S _i), neuron n each subcircuit give a binding target synaptic to each neuron n _'j (phase delay) of the _i 401
Indicates that they are arranged in a matrix.
In this way, wiring from the synapse circuit to the connection destination neuron can be performed on the same line (local common bus 301) corresponding to each receptive field (wiring between neurons can be performed virtually). ), It is possible to reduce (eliminate) the wiring problem which has conventionally been a problem.

When a plurality of pulse inputs from the same receptive field are received, the neuron of the connection destination determines which neuron emitted each pulse from the arrival time of the pulse on the basis of the time window (the characteristic detection cell Corresponding to the feature to be detected,
It can be identified on the time axis by the phase delay inherent to the low-order features that constitute it.

As shown in FIG. 4B, each synapse coupling small circuit 401 includes a learning circuit 402 and a phase delay circuit 403. The learning circuit 402 adjusts the delay amount by changing the characteristics of the phase delay circuit 403, and also stores the characteristic value (or its control value) on the floating gate element or on a capacitor coupled to the floating gate element. It is something to memorize.

FIG. 5 is a diagram showing a detailed configuration of a synapse connection small circuit. The phase delay circuit 403 is a pulse phase modulation circuit. For example, as shown in FIG. 5A, monostable multivibrators 506 and 507, resistors 501 and 504, capacitors
503, 505 and a transistor 502 can be used. FIG. 5B shows a square wave P1 ([1] in FIG. 5B) input to the monostable multivibrator 506, and a square wave P2 (the same [2]) output from the monostable multivibrator 506. ), The respective timings of the square wave P3 (the same [3]) output from the monostable multivibrator 507.

The details of the operation mechanism of the phase delay circuit 403 are omitted, but the pulse width of P1 is determined by the time until the voltage of the capacitor 503 due to the charging current reaches a predetermined threshold, and the width of P2 Is determined by the time constant of the resistor 504 and the capacitor 505. When the pulse width of P2 is
If the falling point shifts later, the rising point of P3 also shifts by the same amount, but the pulse width of P3 does not change. As a result, only the phase of the input pulse is changed. Is modulated and output.

Refresh circuit using control voltage Ec as reference voltage
The pulse phase (delay amount) can be controlled by changing the learning circuit 402 that controls the amount of charge stored in the capacitor 508 that applies the connection weight to the capacitor 509. In order to maintain the connection weight for a long period of time, after the learning operation, as a charge of a floating gate element (not shown) added to the outside of the circuit of FIG. The coupling load may be stored. Other configurations designed to reduce the circuit scale (for example, see Japanese Patent Application Laid-Open No. 5-37317)
A publicly known circuit configuration such as that described in Japanese Patent Application Laid-Open No. H10-327054 can be used.

In the case where the network adopts a configuration of a covalent connection form of connection weights (particularly, a case where a plurality of synapse connections are represented by the same weight coefficient), the amount of delay at each synapse (described below). _Unlike the case of FIG. 3, P _{ij in} equation (9) may be uniform within the same receptive field.
In particular, the connection from the feature detection layer to the feature integration layer involves sub-sampling by local averaging of the output of the feature detection layer, which is the preceding layer, and the like, and therefore does not depend on the detection target (that is, the problem Regardless), it can be configured in this manner.

In this case, as shown in FIG. 4B, each small circuit of FIG. 4A is a single circuit S _{k, i} connected by the local common bus line 401. It becomes an economical circuit configuration. On the other hand, the feature integration layer 103 (or the sensor input layer 1
When the connection from (01) to the feature detection layer 102 is made like this, the feature detection neuron detects an event called simultaneous arrival (or almost simultaneous arrival) of pulses representing a plurality of different feature elements. is there.

When the connections have symmetry, the connections that give the same amount of load (phase delay) are represented by the same small synapse connection circuit, so that a considerable number of synapse connections are represented by a small number of circuits. It can be configured to be. Especially in the detection of geometric features, since the distribution of the connection load in the receptive field often has symmetry,
The number of synapse connection circuits can be reduced, and the circuit scale can be significantly reduced.

FIG. 5 shows an example of a learning circuit at the synapse that achieves simultaneous arrival of pulses or a predetermined amount of phase modulation.
What has a circuit element as shown in FIG. That is, the learning circuit 402 is connected to the pulse propagation time measurement circuit 510.
(Here, the propagation time is the time difference between the pulse output time at the pre-synapse of a neuron in a certain layer and the arrival time of the pulse at the output destination neuron on the next layer.
(B) shows the sum of the synapse delay and the time required for propagation), the time window generation circuit 511, and the pulse phase for adjusting the pulse phase modulation amount in the synapse so that the propagation time becomes a constant value. A modulation amount adjustment circuit 512 can be used.

As a propagation time measuring circuit, a clock pulse from a pacemaker neuron that forms the same local receptive field as described later is input, and within a predetermined time width (time window: see FIG. 3B). A configuration in which the propagation time is obtained based on the output of the clock pulse from the counter circuit is used. By setting the time window based on the firing time of the output destination neuron,
The extended Hebb's learning rule as shown below is applied.

Further, the learning circuit 402 may make the width of the time window narrower as the frequency with which objects of the same category are presented increases. In this way, the more familiar the pattern (ie, the number of times of presentation and the number of times of learning) are, the closer to the detection mode of simultaneous arrival of a plurality of pulses (coincidence detection). By doing so, the time required for feature detection can be reduced (the operation of instantaneous detection becomes possible) .However, it is possible to perform detailed comparative analysis of the spatial arrangement of feature elements, identification between similar patterns, etc. Becomes unsuitable.

The learning process of the delay amount is extended, for example, to the complex number domain, so that the neuron n _{i in the} feature detection layer is obtained.
Connection weight C _ij between the feature integration layer neuron n _j
Is given as C _ij = S _ij exp (iP _ij ) (11) Here, S _ij is the coupling strength, P _ij is the phase, and i before it is a pure imaginary number, and is a phase corresponding to the time delay of the pulse signal output from the neuron j to the neuron i at a predetermined frequency. Sij reflects the receptive field structure of neuron i and generally has a different structure depending on the object to be recognized and detected. This is formed separately by learning (supervised learning or self-organization) or is formed as a predetermined structure.

On the other hand, a learning rule for self-organization regarding the delay amount is as follows.

[0143]

[Outside 7] Given by However,

[0144]

[Outside 8] Is the time derivative of C, τ _ij is the above time delay (preset amount), and β (β1) is a constant.

[0145] Solving the above equation, C _ij converges to βexp (-2πiτ _ij), hence, P _ij converges to-tau _ij. An example of the application of the learning rule will be described with reference to the time window at the time of learning shown in FIG. 3 (B). The anterior neuron (n1, n2, n
The connection weight is updated according to equation (12) only when both 3) and the posterior neuron (feature detection cell) are firing within the time range of the learning time window. In addition,
In FIG. 3B, the feature detection cells are fired after the elapse of the time window, but may be fired before the elapse of the time window in FIG.

Further, even if the recognition / detection target is presented in one size at the time of learning, a plurality of scales (processing channels) are required.
Learning control can be performed. Specifically, at the time of learning, the channel activity control circuit of FIG. 15 is controlled so that the channel output having the highest average activity level among the outputs in the feature integration layer (2,0) is distributed to other channels. The learning as described above may be performed.

For example, for the same feature category,
Assuming that a learning rule is determined in advance so that the same pulse interval (arrival time pattern) is obtained between different processing channels, the output of the feature integrated layer neuron of the channel having the highest average activity level becomes Is distributed to the feature detection layer (next layer) neuron of the channel (or close to the scale level of the channel) according to the receptive field structure of the feature detection neuron and input (as the same pulse train pattern) do it. In this case,
All channels in subsequent layers will be activated at the same level. Therefore, it goes without saying that the same effect may be obtained by another method.

Thus, learning for a plurality of objects of different sizes (scale levels) can be performed in a single learning process without having to perform learning by changing the size of the recognition / detection target in various ways. It is possible to obtain the same effect as the above.

As the learning rule, another method may be used. Also, by introducing the principle of competitive learning, the pulses may arrive at a predetermined interval or more apart from each other (the difference in time delay becomes a predetermined value or more).

Hereinafter, processing mainly performed in the feature detection layer (at the time of learning and recognition) will be described.

In each of the feature detection layers 102, as described above, a pulse signal relating to a plurality of different features from the same receptive field is input into a processing channel set for each scale level, A total (load sum) calculation and threshold processing are performed. The pulse corresponding to each feature amount is determined by the delay amount (phase) determined in advance by learning.
Arrives at predetermined time intervals.

The learning control of the pulse arrival time pattern is not the main focus of the present application and will not be described in detail. For example, the more the characteristic elements constituting a certain figure pattern are the most remarkable features contributing to the detection of the figure. In order to distinguish each feature element at a pulse signal level between feature elements arriving earlier and having substantially equal saliency, competitive learning is introduced so that the feature elements arrive at a certain time interval from each other. Alternatively, a predetermined feature element (a feature element constituting a recognition target, which is considered to be particularly important:
For example, it may be designed to arrive at different time intervals between features having a large average curvature, features having high linearity, and the like.

The saliency corresponds to the reaction intensity (here, the pulse delay amount) of the detected cell of the characteristic element when the figure is detected. In this case, the neurons corresponding to each lower-order feature element in the same receptive field on the feature integration layer, which is the preceding layer, fire synchronously (pulse output) at a predetermined phase.
Will do.

In general, there is a connection to a feature detection neuron that detects the same higher-order feature at a different position but in the feature integration layer (in this case, the receptive field is different but the higher-order Having the same characteristics of the combination).
At this time, it is needless to say that synchronous firing occurs with these feature detection neurons. However, the output level (here, the phase is used as a reference, but the frequency, the amplitude, and the pulse width may be used as the reference) may be determined by summing (or averaging) contributions from a plurality of pacemaker neurons provided for each receptive field of the feature detection neuron. Etc.). In each neuron on the feature detection layer 102, the calculation of the spatiotemporally weighted sum of the input pulses (sum of weights) is performed only in a time window of a predetermined width for the pulse train that has arrived at the neuron. The mechanism for implementing the weighted addition within the time window is not limited to the neuron element circuit shown in FIG. 2, and it goes without saying that the mechanism may be implemented by another method.

This time window corresponds to the refractory period of the actual neuron.
(refractory period) It corresponds to the time zone to some extent. In other words, there is no output from the neuron regardless of any input during the refractory period (time range other than the time window), but the firing according to the input level occurs in the time window other than that time range. Is similar to the neuron.

The refractory period shown in FIG. 3B is a time period from immediately after the firing of the feature detection cells to the start of the next time window. Needless to say, the length of the refractory period and the width of the time window can be arbitrarily set, and the refractory period need not be shorter than the time window as shown in FIG. Even without using a pacemaker neuron, the start time of the time window is determined by the mechanism of synchronous firing between the neurons in the feature detection layer and the feature integration layer due to weak mutual coupling between neurons and predetermined coupling conditions (EMIzhikevich, 1999 'Weakly Pulse-Coupled
Oscillation, FM Interactions, Synchronization, an
d OscillatoryAssociative Memory 'IEEE Trans.on Ne
ural Networks, vol.10. pp.508-526.), these neurons are identical. It is known that the synchronous firing is generally caused by a mutual coupling and a pull-in phenomenon between neurons.

Therefore, also in the present embodiment, such an effect can be obtained without a pacemaker neuron by configuring so as to satisfy the weak mutual connection between neurons and a predetermined synaptic connection condition.

In the present embodiment, as schematically shown in FIG. 6, as the mechanism already described, for example, a pacemaker neuron (for a fixed frequency) receiving an input from the same receptive field for each feature detection layer neuron By inputting timing information (clock pulse) based on (pulse output), the above-described common start timing may be provided.

In the case of such a configuration, it is not necessary to perform the synchronization control of the time window (if it is necessary) over the entire network, and there is a fluctuation and fluctuation of the clock pulse as described above. Is also uniformly affected by the output from the local receptive field (the fluctuation of the position of the window function on the time axis is the same between neurons forming the same receptive field). The reliability of detection does not degrade. Since a highly reliable synchronous operation is enabled by such local circuit control, the tolerance of variation regarding circuit element parameters is also increased.

Hereinafter, for the sake of simplicity, a feature detecting neuron that detects a triangle as a feature will be described. The feature integration layer 103 at the preceding stage has L-shaped patterns (f ₁₁ , f ₁₂ ,...) Having various orientations as shown in FIG. 7C, and continuity (connectivity) with the L-shaped pattern. Those that respond to graphical features (feature elements) such as a combination pattern (f ₂₁ , f ₂₂ ,...) Of the line segments, and a combination (f ₃₁ ,. I do.

Also, in FIG.₄₁, f₄₂, f₄₃Has a different orientation
Feature that constitutes a triangle ₁₁, f₁₂, f₁₃Compatible with
The following shows the features to be performed. New layer connection by learning
The characteristic delay amount is set between the
In the sign detection neuron, the time window is divided
Sub time window (time slot) (w₁, w_Two,…)smell
Corresponding to the main and different features that make up the triangle.
The setting is made in advance so that Luz arrives.

For example, w ₁ , w ₂ ,
..., the w _n, as shown in FIG. 7 (A), pulses corresponding to the combination of a set of features as constituting a triangle as a whole arrive first. Here, the L-shaped pattern (f ₁₁ ,
f ₁₂ , f ₁₃ ) arrive within w ₁ , w ₂ , w ₃ respectively, and the feature element
pulse corresponding to _{(f 21, f 22, f} 23) , respectively w _1, w _2, w ₃
The amount of delay is set by learning so as to arrive within.

The pulses corresponding to the characteristic elements (f ₃₁ , f ₃₂ , f ₃₃ ) arrive in the same order. In the case of FIG. 7A, a pulse corresponding to one characteristic element arrives in one sub time window (time slot). The meaning of dividing into sub-time windows is that, by performing pulse detection (detection of characteristic elements) corresponding to different characteristic elements developed and expressed on the time axis in each sub-time window individually and reliably, those characteristics are obtained. The purpose of the present invention is to increase the changeability and adaptability of the processing mode, such as whether the conditions of all feature elements are to be detected or a certain percentage of features are to be detected. .

For example, in a situation where the recognition (detection) target is a face and the search (detection) of the eyes, which is a part of the face, is important (the eye pattern detection priority should be set high in the visual search). Case), by introducing feedback coupling from higher order feature detection layers,
It is possible to selectively enhance the reaction selectivity (detection sensitivity of a specific feature) corresponding to the feature element pattern constituting the eye. By doing so, it is possible to give higher importance to lower-order feature elements constituting higher-order feature elements (patterns) and detect them.

Further, if it is set in advance that a pulse arrives in a sub-time window earlier as important features,
By making the weighting function value in the sub-time window larger than the value in the other sub-time windows, a feature having a higher importance can be more easily detected. The importance (detection priority between features) can be obtained by learning or can be defined in advance.

Therefore, as long as the event of detecting a certain percentage of characteristic elements only needs to occur, the division into sub-time windows is almost meaningless, and may be performed in one time window.

Note that pulses corresponding to a plurality of (three) different characteristic elements may arrive and be added (see FIG. 7D). That is, one sub time window
(Time slot) includes a plurality of characteristic elements ((D) in FIG. 7),
Alternatively, it may be assumed that pulses corresponding to an arbitrary number of characteristic elements are input. In this case, in FIG. 7 (D), in the first sub-time window, pulses corresponding to the other feature elements f ₂₁ and f ₂₃ that support the detection of the apex portion f ₁₁ of the triangle arrive, and similarly, 2 In the second sub-time window, the vertex angle part f ₁₂
The pulses of other feature elements f ₂₂ and f ₃₁ have arrived that support the detection of.

The number of divisions into sub-time windows (time slots), the width of each sub-time window (time slot), the class of the feature, and the assignment of the pulse time interval corresponding to the feature are not limited to those described above. Needless to say, it can be changed. For example, a sub-time window corresponding to a feature element such as 'X' or '+' may be set in addition to the above-described feature element. Such feature elements can be said to be redundant (or unnecessary) in the detection of a figure of a triangle, but conversely, the detection accuracy of a figure pattern of a triangle can be increased by detecting that they do not exist.

Even when a deformation not represented by a combination of these characteristic elements is applied (for example, when rotation within a certain range is given), the output of the neuron of the characteristic integration layer representing the above-mentioned characteristic element is obtained. The pulse responds with a continuous phase delay (delay amount: a range in which the pulse arrives in a predetermined sub-time window (time slot)) according to the degree of deviation from the ideal pattern (so-called “gra”).
For this reason, the output is stabilized so that the allowable range for the deformation of the detected graphic feature becomes a certain level or more. For example, the triangle formed by the features corresponding to the triangle (Q1) which is formed by the features corresponding to the features f _11, f _12, f ₁₃ shown in (C) of FIG. _{7, f 41, f 42,} f 43 In (Q2), at least the directions should be different from each other.

In this case, detection (integration) corresponding to each feature
When cells are present, for a triangle (Q3) corresponding to an intermediate orientation of the two _{triangles, f 11, f 12, f} 13 corresponding to the detection (integration) cells and f _41, f _42, f ₄₃ The detection (cells) corresponding to the above are all lower than the maximum response output, and are directly at the output level according to the convolution operation value with the filter kernel as the receptive field structure determined according to the type of feature. If the vector quantity as an output from the cell is integrated as being unique to the intermediate figure, it is possible to detect an intermediate figure (when rotated) in the state of two triangles.

For example, qualitatively, the rotation angle is small,
F closer to Q1₁₁, F₁₂, F₁₃From cells corresponding to
The output is relatively large, and conversely, the closer it is to Q2, the more f₄₁,
f₄₂, F ₄₃The output from the cell corresponding to is increased.

Next, the calculation of the spatiotemporally weighted sum of input pulses (sum of weights) will be described. As shown in FIG. 7B, in each neuron, the weighted sum of the input pulses is calculated by a predetermined weighting function (for example, Gaussian) for each of the sub-time windows (time slots), and the sum of the weighted sums is a threshold value. Be compared. τ _j represents the center position of the weighting function of the sub time window j, and is based on the start time of the time window (elapsed time from the start time)
Expressed by The weight function is generally a function of a distance (shift on a time axis) from a predetermined center position (representing a pulse arrival time when a feature to be detected is detected).

Therefore, assuming that the peak position τ of the weight function of each sub-time window (time slot) of a neuron is a time delay after learning between neurons, a spatiotemporally weighted sum of input pulses (weighted sum) is performed. Neural networks are a kind of time-domain radial basis function network (Radial Bass Network).
is Function Network; hereinafter abbreviated as RBF). The time window F _Ti of the neuron n _i using the weight function of the Gaussian function is expressed as σ and the coefficient factor (equivalent to the synaptic connection weight value) of each sub-time window as b _ij .

[0174]

[Outside 9]

The weight function may take a negative value. For example, when a neuron of a certain feature detection layer is to finally detect a triangle, a feature (F _faulse ) that is clearly not a component of the figure pattern (for example, “X”, '+' Etc.)
Is detected, the detection output of the triangle is not finally made even if the contribution from other feature elements is large,
In the input sum calculation processing, the characteristic (F _faulse )
Can be given a weighting function that gives a negative contribution and a coupling from the feature detection (integrated) cells.

[0176] space sum when the input signal to the neuron n _i of the feature detection layer X _i (t) is

[0177]

[Outside 10] Can be expressed as Here, ε _j is the initial phase of the output pulse from the neuron n _j , and converges to 0 by synchronous firing with the neuron n _i , or changes the phase of the time window by the timing pulse input from the pacemaker neuron. When forcibly synchronizing to 0, ε _j may always be 0. When the pulse input of FIG. 7A and the weighted sum by the weight function shown in FIG. 7B are executed, a temporal transition of the weighted sum value as shown in FIG. 7E is obtained. The feature detection neuron outputs a pulse when the weighted sum reaches a threshold value (Vt).

[0178] The output pulse signal from the neuron n _i, as described above, with the spatio-temporal sum squashing nonlinear function to become an output level and time delay given by learning (the so-called total input sum) of the input signal (phase) , Output to the upper layer neuron (pulse output is fixed frequency (binary)
Then, a phase modulation amount serving as a squashing nonlinear function for the spatiotemporal sum of the input signal is added to a phase corresponding to a fixed delay amount determined by learning, and the resultant is output).

FIG. 8 is a flowchart showing the processing procedure of each layer described above. The flow of processing from low-order feature detection to high-order feature detection is summarized as shown in FIG.
First, in step S801, low-order feature detection (for example, calculation of a Gabor wavelet transform coefficient at each position) is performed. Next, in step S802, low-order feature integration processing for performing local averaging of those features and the like is performed. Further, steps S803 to 80
In step S4, detection and integration of secondary features are performed. In steps S805 to S806, detection and integration of high-order features are performed. Then, in step S807, the presence or absence of a recognition (detection) target or the detection position output is performed as the output of the final layer. Steps S803-804 and S805-806
Can be arbitrarily set or changed according to the task (such as a recognition target).

FIG. 9 is a flowchart showing the processing procedure of each feature detection neuron 602. First, step S9
In 01, a pulse corresponding to a plurality of feature categories is applied to the same receptive field 10 in the input layer 101 or the feature integration layer 103 as the preceding layer.
Step S9: receiving input from the neuron 601 forming 5
In step 02, a time window and a weighting function are generated based on a local synchronization signal input from the pacemaker neuron 603 (or obtained by interaction with a preceding layer neuron). The sum of the weights by the function is obtained, and it is determined in step S904 whether or not the threshold has been reached.
At 05, pulse output is performed. Although steps S902 and S903 are shown in time series, they are actually performed almost simultaneously.

The processing procedure of each feature integration neuron is as shown in the flowchart of FIG. That is, in step S1001, a pulse is input from the feature detection processing module 104 in the same category, which is a feature detection neuron that forms a local receptive field specific to the neuron. Input pulses are added in a time range other than the initial period.
In step S1003, a determination is made as to whether the total value of the input pulses (measured on the basis of potential, for example) has reached a threshold value. If the threshold value has been reached, in step S1004, a pulse is output at a phase corresponding to the total value. do.

Since the input pulse corresponds to the feature at each position in the spatial domain (or the spatial arrangement relationship of the feature element), it is possible to form a spatio-temporal RBF.

More specifically, each neuron output value is further weighted and added to obtain a sufficient number of predetermined feature element sets (feature detection cells) and a sufficient number of sub-time values. A spatio-temporal function of a pulse pattern corresponding to an arbitrary figure pattern can be expressed by calculating a weighted sum (load sum) in a window (time slot). If the category of recognition symmetry and its shape change are limited to some extent, necessary feature detection cells and sub-time windows
(Time slots) can be reduced.

In this embodiment, the common bus is a local bus line which is assigned to the same receptive field. However, the present invention is not limited to this. As in the case of the line, the pulse phase delay amount may be divided and set on the time axis. In addition, a common bus line may be used between adjacent receptive fields having a relatively large overlapping ratio.

It is to be noted that the processing (or the threshold processing) is performed so that the result of the weighted product-sum operation in each sub-time window (time slot) becomes a non-linear squashing function value without using the above-mentioned spatiotemporal RBF. ) And take the product of them. For example, a threshold processing result is obtained by a circuit configuration (not shown).
(Binary) may be obtained for each sub-time window and stored in the temporary storage unit, and the logical product of the threshold processing results sequentially obtained may be obtained in a time-series manner.

When the product is obtained by performing the threshold processing, it goes without saying that the tolerance of feature detection under pattern loss or low contrast conditions is reduced.

The above-described processing (detection of a graphic pattern by spatiotemporal RBF) can also be realized as an operation similar to the associative memory recall process. That is, even if a defect of a low-order (or medium-order) feature element to be detected in a certain local area (or entire area) occurs, some other feature elements are detected, and the sum value (expression (14) If)) exceeds the threshold value, it is possible to detect a medium-order (or higher-order) feature element (fire the corresponding neuron) in the entire spatiotemporal RBF network.

The configuration of the network need not be limited to that shown in FIG. 1, but may be MLP or any other configuration as long as it includes a layer for detecting a predetermined geometric characteristic element. Needless to say.

In this embodiment, Ga is used for low-order feature extraction.
Although the bor wavelet transform is used, it goes without saying that other multi-scale features (for example, a local autocorrelation coefficient obtained with a size proportional to the scale) may be used.

Next, FIG. 11 shows a case in which focusing on a specific subject, color correction of the specific subject, and exposure control are performed by mounting the pattern detection (recognition) device according to the configuration of the present embodiment in the imaging device. It will be described with reference to FIG. FIG. 11 is a diagram illustrating a configuration of an example in which the pattern detection (recognition) device according to the embodiment is used for an imaging device.

An imaging device 1101 shown in FIG. 11 includes an imaging optical system 1102 including a photographing lens and a drive control mechanism for zoom photographing,
D or CMOS image sensor 1103, imaging parameter measurement unit 1104, video signal processing circuit 1105, storage unit 1106, control signal generation unit 1107 that generates control signals such as control of imaging operation, control of imaging conditions, and viewfinders such as EVF And a recording medium 1110, and the above-described pattern detection device is provided as a subject detection (recognition) device 1111.

The image pickup apparatus 1101 performs detection of a face image of a person registered in advance (detection of the position and size of a person) from a captured video by a subject detection (recognition) apparatus 1111. When the position and size information of the person is input from the subject detection (recognition) device 1111 to the control signal generation unit 1107, the control signal generation unit 1107
Focus control for that person based on the output from 04,
A control signal for optimally controlling exposure condition control, white balance control, and the like is generated.

As a result of using the above-described pattern detection (recognition) device for an image pickup device, even if the size of the subject on the screen varies due to the difference in subject distance, the subject can be reliably detected (recognized). ), Realizing such a function with low power consumption and high speed (real time), and performing optimal control (AF, AE, etc.) of the detection of a person and the like based on the detection.

Note that, under the network configuration shown in FIG. 1, a synapse element that performs a pulse width (analog value) modulation operation is integrated with an integrated-and-fi as shown in the first embodiment.
Recognition of a figure pattern or the like may be performed by a network constituted by re-neurons. In this case, modulation by synaptic each pulse width and the pulse width of the postsynaptic the presynaptic signal, W _b, is given by W _a = S _ij W _b when the W _a. Here, S _ij has the same meaning as the coupling strength (Equation (9)) in the first embodiment. In order to increase the dynamic range of the modulation, it is necessary to make the basic pulse width of the pulse signal sufficiently smaller than the period (basic pulse interval).

The firing (pulse output) of the neuron occurs when the potential exceeds a predetermined threshold value due to accumulation of electric charges accompanying the inflow of a plurality of pulse currents representing predetermined characteristic elements.
In the present embodiment, weighted addition of arrival pulses for each sub time window is not particularly required, but integration is performed in a time window having a predetermined width. In this case, the feature element (figure pattern) to be detected depends only on the temporal sum of the signals input to the feature detection layer neurons (sum of the pulse current values). The width of the input pulse corresponds to the value of the weight function.

<Second Embodiment> In this embodiment, feature representations with different scale levels and collective encoding as described above are performed only for lower-order features, and scale-invariant by pulse phase modulation for each feature. Characteristic expression,
Medium- and high-order feature detection is performed in this scale-invariant feature expression domain.

That is, the structure up to the feature integration layer (2,0) has the same structure as that of the first embodiment, and the succeeding portion has a scale-invariant signal conversion related to lower-order features unique to the present embodiment, and a feature detection layer. From (1,1) onwards, feature detection and integration processing from middle to higher order for scale-invariant feature expressions are performed. With this configuration, it is possible to simplify the circuit configuration while maintaining scale-invariant recognition performance without using a configuration having a plurality of processing channels up to the middle and high orders as shown in the first embodiment. This leads to a reduction in size and a reduction in power consumption.

FIG. 18 is a diagram showing a network configuration used in the present embodiment. Specifically, the feature detection layer (1, k)
In (k is a natural number), Gabor wavelet extracted in (1,0) layer
Although higher-order features are detected than those obtained by conversion, etc., FIG.
As shown in FIG. 8, in the (1,1) layer and subsequent layers, differences in processing channels can be physically distinguished in terms of circuit configuration.

In FIG. 18, S _{k, m} is detected in the (1,0) layer, the scale level is m, representing the k-th feature category, and C _{k, m} is integrated in the (2,0) layer. , Represents the k-th feature category with a scale level of m. From the (1,1) layer, the feature level to be detected and integrated is not assigned a scale level index.

FIG. 20 is a diagram showing transition of a signal sequence.
As a precondition for obtaining a scale-invariant information expression by pulse phase modulation, a pulse phase conversion unit shown in FIG. 18 is used so that pulse signals belonging to different processing channels at different positions on the time axis and belonging to different processing channels are not mixed.
Phase conversion is performed once by 1701 (FIG. 20A), and a scale-invariant pulse signal train is obtained by a scale-invariant signal conversion unit 1702, and the pulse train from the feature integration layer (2,0) is converted into a feature detection layer. Arrival at (1,1) neuron.

More specifically, between processing channels of the same low-order feature category but of different scale levels (hereinafter, referred to as processing channels).
In the “same feature-different scale”, when reaching the feature detection cell, a learning rule such as the self-organization described above (self-organization of the phase delay amount) as described above is represented by a predetermined pulse interval pattern. ) May be set. For example, in the case of the same feature-different scale, the difference in the processing channel (lower scale property) in the low-order feature detection is due to the fact that the ratio of the input pulse intervals to the feature detection cells is the same as a result of the learning process. It is configured to be represented as a pattern with different scaling in the time axis direction or as a pattern in which the absolute value of the pulse interval has a different phase offset value according to the scale level.

That is, in the former case, the learning rule (Equation (12))
Is used (for simplicity, the index expression as a time delay from neuron j to neuron i in equation (12) is omitted), according to scale level index m and feature category index k, τ (k, m) = αη _k ρ _m + ξ _km (15), and in the latter case, τ is expressed as follows.

Τ (k, m) = η_k+ Αρ_m (16) where η_kIs the quantity specific to the feature category, ρ_mIs the scale
The level-specific quantity, α, is a positive constant. In the above formula
ρ_mRepresents the above-mentioned expansion and contraction rate in the time axis direction, and ξ_{k m}Is the phase
Represents offsets, both of which are specific to the scale level.
You. Equation (15) indicates that signals of different processing channels are
Phase offset to avoid mixing on the time axis
Ξ amount.

Next, the scale-invariant signal converter 1702 generates a time window weighting function for a time range corresponding to the specific channel (scale level) selected by the gating circuit (FIG. 20B), and performs detection. A conversion process to scale-invariant pulse train information relating to features to be performed is performed ((C) in FIG. 20). Normally, only a pulse train for a feature category detected at a specific scale level (processing channel) is extracted, the pulse train is duplicated, and a constant conversion may be given as necessary.

Specifically, a gating circuit as shown in FIG. 16 may be used to extract a pulse train of the processing channel having the highest average activity level. When the phase conversion by the pulse phase conversion unit 1701 is the former, the pulse phase conversion is such that the expansion / contraction ratio in the time axis direction is constant compared to the reference value, and in the latter case, a feature detection neuron that detects a certain graphic feature Is performed so that the phase offset amount of the arrival pattern of a plurality of pulses to the pulse becomes a constant value regardless of the pulse from any processing channel.

That is, in any case, as a result, as shown by τ = C ₁ η _k + C ₂ (17), conversion is performed so that the pulse phase depends only on the feature category. Where C ₁ is a positive constant, C ₂
Is a non-negative constant. Even when information is expressed by pulse width modulation, the same processing may be performed on the expansion or contraction or offset amount of the pulse width.

As described in the first embodiment, at the time of learning, even if the recognition (detection) target is presented in one size, it is internally converted into a plurality of scale levels to perform learning control. Good. In the case of the present embodiment, a pulse train pattern in a channel generated when a learning pattern on a fixed scale (size) for one channel is presented,
The channel activity control circuit of FIG. 15 may be converted to a different channel according to the formula (15) or (16), that is, converted to a different position on the time axis, copied, and transmitted to a subsequent layer (FIG. 20). (D)).

The conversion into the scale-invariant feature expression is not limited to Gabor wavelet conversion or low-order feature levels such as directional components, but is limited to medium-order features having a certain level of complexity. Alternatively, the processing may be performed in the same configuration as in the first embodiment. That is, the detection (integration) of features having a certain level of complexity or less is performed in physically independent (spatially arranged) processing channels. A scale-invariant expression may be obtained and processed as in the embodiment.

In this case, up to the middle-order features having a certain level of complexity or less, collective coding such as a joint process between processing channels is performed without complicated timing control. Output as firing characteristics of different neurons, it is possible to perform detection processing up to the middle-order feature in multiple scales in a short time and in parallel in comparison with the case of processing each scale level in time series. This has the effect.

[0210] As for the higher-order features, the above-described circuit is reduced in scale and power consumption is reduced. In this way, it is possible to determine at which characteristic level (complexity) the scale-invariant expression is obtained, taking into account the processing time, the complexity of timing control, the circuit scale, and the power consumption.

<Third Embodiment> In the present embodiment, in the feature detecting neurons belonging to different scale levels (processing channels), graphic features of the same category are used.
(For example, an L-shaped pattern), the learning rule is determined such that the interval of the arrival time of the pulse corresponding to the pulse signal (or the time pattern of the pulse arrival) differs depending on the scale level. Is performed by linear combination by weighted addition over

FIG. 19 is a diagram showing a network configuration used in this embodiment. As shown in the figure, in the present embodiment, (in the configuration shown in FIGS. 12 and 13, between the collectively encoded neuron output and the next layer, the feature detection layer)
Providing a pulse phase conversion unit 1701 and a collective encoding unit 1901 on the time axis, as a result, a signal obtained by expanding a pulse belonging to a different processing channel on the time axis of a predetermined channel is a feature detection layer (1, 1). Is input to

FIG. 17 shows a processing channel (scale level).
FIG. 10 is a diagram illustrating an example of a signal to the feature integration layer when patterns of different sizes are detected at the same location when the pattern is developed in the time axis direction. As shown in the figure, the difference between the processing channels (scale levels) is that the pulses arriving at the feature detection neuron are time-divided and distributed to different positions on the time axis (providing different phase offset amounts).

That is, the sub time windows w _1,1 , w _2,1 , w _{3,1 in FIG.}
Is a time window for the L-shaped pattern set detected at the scale level 1, and w1 _{, n} , w2 _{, n} , w3 _{, n} are the L-shaped patterns of the set of the same feature category corresponding to the scale level n. It is a signal pulse time window representing detection.

As in the second embodiment, the feature detection layer (1, 1)
In subsequent layers, the same circuit can perform multi-scale processing without providing a different circuit for each processing channel, resulting in an economical circuit configuration. That is, as shown in FIG. 19, in the (1,1) layer and subsequent layers, differences in processing channels can be physically distinguished in terms of circuit configuration. In the present embodiment, even after the (1,1) layer, for the signals of the scale levels (processing channels) arranged at different positions in time, the combination processing and the collective code corresponding to the equations (5) and (6) are performed. Is performed in the time domain.

The output from the feature detection layer (1, 1) to the feature integration layer (2, 1) (and the same applies to the subsequent layers) is output at each processing channel output (for each scale level). This is done in splits. That is, when temporally integrating the input signal of the characteristic detection cell, a pulse output for one processing channel is performed corresponding to the entire range of the sub time window forming one scale level. The collective coding performed in the feature integration layer is
This is performed by integrating the input pulse within the time window over each channel.

The processing for graphic feature signals of different scale levels (processing channels) performed by the collective coding neuron is performed in two stages as follows. First, by integrating the input pulse in a time window for each processing channel,
The same feature category is detected for each scale level, and the collective coding operation corresponding to equations (5) and (6) is performed by setting the integral value for each time window to p _ij. Do. By temporally integrating the processing channel outputs time-divided in this way, it is possible to detect (recognize) a graphic pattern of an arbitrary size.

As an example, FIG. 21 shows a case where the scale level is 1 to n and the feature category is 1 to 3, from the output of the feature detection cells on each channel (scale level). The transition of the signal sequence up to the output is shown in order from the top.

FIG. 21A shows a state in which the output from the feature detection cells to the feature integration layer is developed on the time axis for each scale and each feature. There are multiple outputs for each feature because there are multiple feature detection cells that simply detect the same feature (same scale), and each output has fluctuations due to Poisson noise or the like, or sensitivity characteristics related to feature category-scale level. This is to show that there is a variation with respect to At the scale level n, the lack of the pulse corresponding to the feature 2 simply indicates that the pulse was not detected. In the same scale, the phases of the other features 1 and 3 are delayed as compared with the other scale levels. This reflects the low detection level.

FIG. 21 (B) shows a weighting function when such a feature detection cell output is temporally integrated by the sub-sampling cells of the feature integration layer. FIG. 21C illustrates the resulting sub-sampled cell output. FIG.
(D) represents the time window weight function for each scale when integrating the sub-sampled cell outputs in the collectively encoded cells. (E) of FIG. 21 shows an output distribution on the time axis from the collectively encoded cells.

As an example showing the effect of collective coding,
Looking at the pulse positions before and after performing collective encoding for the feature 1 of the scale levels 1 and 2, the position on the time axis of each pulse is such that the scale level 1 has a phase lag (time delay amount Δp ₁₁ ), a scale Level 2 is modulated with a phase advance (time advance amount Δp ₁₂ ). This is a result of modulating the output of the sub-sampled cell based on the activity level of the entire channel and the like according to the equations (5) and (6).
This reflects that the scale level is between the preset scale levels 1 and 2. As subsequent processing, a channel activity control or gating function corresponding to FIGS. 15 and 16 of the first embodiment may be performed on the time axis.

More specifically, the channel activity control circuit or the gating circuit calculates the target channel activity (or selects a channel) from the time-divided channel data, and assigns the target channel activity to each channel (scale level). For each corresponding time window, the setting of the neuron threshold value of the subsequent layer and the control of the amplification / attenuation of the signal level according to the channel activity are performed (or the signal is passed through only the selected channel).

According to the embodiment described above, feature detection is performed based on a plurality of resolutions or scale levels, and collective encoding is performed on the detected features related to the plurality of scale levels, so that the size of the detection target can be arbitrarily determined. Even if it changes, deterioration of the detection performance can be avoided.

By detecting low-order or high-order characteristic elements (or graphic pattern elements) by converting them into pulse trains having different phases according to the resolution or scale level, a circuit for performing recognition processing on multiple scales can be realized. By configuring so that the same processing circuit can be used without providing physically different processing channels, the circuit scale and power consumption can be significantly reduced as compared with the case where such phase conversion is not performed. Was.

Further, by performing a threshold processing of a weighted weighted sum within a time window on a pulse signal as a feature detection signal, symmetric deformation (position fluctuation, rotation) to be detected (recognized) in a complicated and diverse background. In particular, a desired pattern can be reliably detected even if a loss in feature detection due to a change in size, illumination, noise, or the like occurs. This effect can be realized regardless of the specific network structure.

Finally, in the configuration shown in the present invention, each position in the time window of a pulse train temporally arranged on a single input line of a predetermined neuron corresponds to a characteristic of a predetermined pattern and a scale level. By doing so, the wiring problem between neuron elements can be reduced, and the scale and power consumption of a circuit for performing recognition and detection of a predetermined object by two-dimensional pattern processing can be reduced while maintaining the above-described reliability to a high degree. It can be suppressed much more than before.

[0227]

As described above, according to the present invention,
By performing feature detection with a plurality of resolutions or scale levels and performing multiplexing (or collective encoding) on the detected features related to the plurality of scale levels,
Even if the size of the detection target changes arbitrarily, it is possible to avoid the deterioration of the detection performance.

Also, by detecting characteristic elements by converting them into pulse trains having different phases according to the resolution or scale level, recognition processing on multiple scales can be performed by the same processing circuit. The effect that the circuit scale and power consumption can be made much smaller can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a network configuration according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a synapse section and a neuron element section.

FIG. 3 is a diagram showing how multiple pulses propagate from a feature integration layer or an input layer to a feature detection layer neuron in the first embodiment.

FIG. 4 is a diagram showing a configuration diagram of a synapse circuit.

FIG. 5 is a diagram showing a configuration of a synapse coupling small circuit and a configuration of a pulse phase delay circuit used in the first embodiment.

FIG. 6 is a diagram showing a network configuration in a case where an input from a pacemaker neuron is provided to a feature detection layer neuron.

FIG. 7 shows a configuration of a time window when processing a plurality of pulses corresponding to different feature elements input to the feature detection neuron,
It is a figure which shows the example of a weight function distribution, and the example of a characteristic element.

FIG. 8 is a flowchart showing a processing procedure of each layer.

FIG. 9 is a flowchart illustrating a processing procedure of each feature detection neuron.

FIG. 10 is a flowchart illustrating a processing procedure of each feature integration neuron.

FIG. 11 is a diagram illustrating a configuration of an example in which the pattern detection (recognition) device according to the embodiment is used for an imaging device.

FIG. 12 is a diagram showing a circuit configuration of a feature integration layer.

FIG. 13 is a diagram showing a circuit configuration of a feature integration layer.

FIG. 14 is a diagram illustrating a configuration of a normalization circuit.

FIG. 15 is a diagram showing a configuration of a channel activity control circuit.

FIG. 16 is a diagram illustrating a configuration of a gating circuit.

FIG. 17 is a diagram illustrating an example of a signal to the feature integration layer when a pattern having a different size is detected at the same location when a processing channel (scale level) is developed in the time axis direction.

FIG. 18 is a diagram illustrating a network configuration used in a second embodiment.

FIG. 19 is a diagram illustrating a network configuration used in a third embodiment.

FIG. 20 is a diagram illustrating transition of a signal sequence according to the second embodiment.

FIG. 21 is a diagram illustrating transition of a signal sequence according to the third embodiment.

FIG. 22 is a diagram schematically illustrating hierarchical expressions having different scale levels (resolutions).

FIG. 23 is a diagram illustrating an example of collective encoding at a scale level.

Claims (33)

[Claims] An input unit for inputting a pattern; a plurality of processing units each performing a process corresponding to a different resolution or scale level on the pattern input from the input unit; Multiplexing processing means for combining outputs, wherein each of the plurality of processing means corresponds to each point obtained by sampling the input data by a predetermined method,
A pattern detecting apparatus, comprising: a plurality of feature detecting elements for detecting and outputting a plurality of features, wherein the multiplexing processing means combines outputs of the plurality of feature detecting elements. 2. The method according to claim 1, wherein the plurality of processing units have a hierarchical network structure, and the multiplexing processing unit corresponds to a processing result of each of the processing units at a middle layer for a plurality of resolutions or scale levels. 2. The method according to claim 1, wherein a setting is made to select a resolution or a scale level or to perform a coupling process between processing outputs of a plurality of resolutions or scale levels based on a plurality of predetermined outputs among the detection element outputs. The pattern detection device according to the above. 3. The multiplexing processing means selects a resolution or a scale level indicating a maximum response output from processing results of the plurality of processing means in a predetermined intermediate layer corresponding to different resolutions or scale levels. The pattern detection device according to claim 2, wherein: 4. The resolution processing unit according to claim 1, wherein the multiplexing processing unit performs a combining process on a plurality of predetermined results among processing results of the plurality of processing units corresponding to different resolutions or scale levels in an intermediate layer. 3. The pattern detecting device according to claim 2, wherein a scale level is selected. 5. Each of the processing means includes a plurality of hierarchical processing layers, and the multiplexing processing means determines a resolution or a scale level by referring to intermediate processing results in processing layers of different hierarchical levels. The pattern detection device according to claim 1, wherein the pattern detection device is selected. 6. Each of the processing means includes a plurality of hierarchical processing layers, and the combining processing differs in a value obtained by combining processing results between different processing elements at the same hierarchical level of the processing means. 5. The pattern detecting apparatus according to claim 4, wherein a reference is made between resolutions or scale levels and an output relating to a predetermined resolution or scale level is performed. 7. The pattern according to claim 1, wherein said processing means has a feature detection layer for detecting each of the plurality of predetermined feature categories at different resolutions or scale levels. Detection device. 8. The pattern detecting apparatus according to claim 7, wherein said multiplexing processing means is provided for each characteristic detecting element of said characteristic detecting layer. 9. The pattern detection apparatus according to claim 7, wherein the feature detection layer locally performs spatial filtering on different spatial frequencies. 10. The pattern detection device according to claim 7, wherein the feature detection layer performs Gabor wavelet conversion. 11. Each feature detection element of the feature detection layer,
The pattern detecting apparatus according to claim 7, wherein a plurality of features at different resolutions or scale levels are detected in the same local area of the input data. 12. Each feature detection element of the feature detection layer,
8. The pattern detecting apparatus according to claim 7, wherein the same geometric feature category has selection characteristics centered on a plurality of different resolutions or scale levels, and the selection characteristics overlap each other. 13. The feature detection layer has a plurality of arithmetic elements having sensitivity characteristics approximated by basis functions having locally different direction selectivities at different resolutions or scale levels with respect to a feature category to be detected. The pattern detecting apparatus according to claim 7, wherein 14. An input unit for inputting a pattern, and a feature detection layer for detecting a plurality of features for each of a plurality of resolutions or scale levels with respect to the pattern input from the input unit; Has a plurality of signal processing elements that are coupled to each other by predetermined coupling means, and a predetermined part of the plurality of signal processing elements corresponds to an arrival time pattern of a plurality of pulse signals input within a predetermined time range. A pattern detection device for outputting a pulse signal at an output level. 15. The pattern detection apparatus according to claim 14, wherein the feature detection element outputs a signal at an output level according to the saliency of the features or the arrangement between the features. 16. A feature integration layer for integrating and outputting an output from the feature detection layer by a predetermined method, wherein the feature integration layer includes a plurality of feature integration elements connected to each other by a predetermined connection unit. The pattern detection device according to claim 14, wherein 17. The feature integration device includes a device for subsampling feature data for a local receptive field region, and a collective coding device for integrating the output of the subsampling device over a plurality of resolution or scale levels. 17. The pattern detecting device according to claim 16, comprising: 18. The pattern detection device according to claim 14, wherein the output level is a phase, frequency, amplitude, or pulse width of the pulse signal. 19. An input unit for inputting a pattern, and a plurality of processing units for performing pattern processing on the pattern input from the input unit, wherein the plurality of processing units have different resolutions or scale levels. A pattern detection device comprising: a feature detection element that detects a plurality of features, outputs a detection pulse signal for each feature, and converts the detection pulse signal so that the phase in the time axis direction is different. 20. The processing unit detects a plurality of features having different resolutions or scale levels only for lower-order features, and outputs a detected pulse signal output of each feature in a different phase in the time axis direction based on each resolution or scale level. 20. A feature detecting element for converting the phase of an output signal from the element into a scale-converted feature extracting element for obtaining a feature detecting signal invariable in resolution or scale level by converting a phase of an output signal from the element. 3. The pattern detection device according to 1. 21. An input means for inputting a pattern, and a plurality of processing means for performing processing corresponding to different resolutions or scale levels on the pattern input from the input means, wherein the processing means Corresponds to each point obtained by sampling the input data by a predetermined method, a plurality of feature detection elements for detecting a plurality of features, respectively, and a plurality of the processing means of different resolution or scale level from each other Control means for controlling the resolution or scale level by combining the outputs of the feature detection elements. 22. The pattern detecting apparatus according to claim 21, wherein said control means controls setting of a predetermined resolution or a scale level. 23. The pattern detection apparatus according to claim 21, wherein the control unit controls the activity of the feature detection element according to a resolution or a scale level. 24. The method according to claim 21, wherein the control means converts or duplicates a signal at a predetermined resolution or scale level to another resolution or scale level by a predetermined method during learning and distributes the signal. Pattern detection device. 25. An image processing apparatus, comprising: controlling a processing operation of an image to be processed based on a result of detecting a predetermined pattern from a pattern of the image to be processed by the pattern detection apparatus according to claim 1; apparatus. 26. A collective coding scheme having a plurality of channels, each processing at a different scale level or resolution using a hierarchically structured set of neurons, wherein the set of neurons combines outputs across the plurality of channels. A neural network device comprising a group of neurons. 27. The neural network device according to claim 26, wherein an optimal scale level or resolution is set based on a processing result of the group of collectively encoded neurons. 28. The neural network device according to claim 26, wherein the neuron group includes a sub-sampling neuron group that performs sub-sampling for each category. 29. An input step of inputting a pattern, a plurality of processing steps of performing processing corresponding to different resolutions or scale levels on the pattern input in the input step, and A multiplexing processing step of combining processing results, wherein in each of the plurality of processing steps, a plurality of feature detection elements are respectively provided corresponding to each point obtained by sampling the input data by a predetermined method. A pattern detection method comprising: detecting and outputting a plurality of features; and combining outputs of the plurality of feature detection elements in the multiplexing process. 30. An input step of inputting a pattern, and a plurality of resolutions or scales by a feature detection layer having a plurality of signal processing elements coupled to each other by a predetermined coupling mechanism with respect to the pattern input in the input step. A detection step of detecting a plurality of features for each level, wherein a predetermined part of the plurality of signal processing elements is at an output level corresponding to an arrival time pattern of a plurality of pulse signals input within a predetermined time range. A pattern detection method characterized by outputting a pulse signal. 31. An input step of inputting a pattern, and a plurality of processing steps of performing a pattern process on the pattern input in the input step, wherein the feature detecting element has a resolution in the plurality of processing steps. Or detecting a plurality of features having different scale levels, outputting detection pulse signals for each of the temporally localized features, and converting the detection pulse signals so as to have different phases in the time axis direction. Pattern detection method. 32. An inputting step of inputting a pattern, and a plurality of processing steps for performing processing corresponding to different resolutions or scale levels on the pattern input in the inputting step. However, corresponding to each point obtained by sampling the input data by a predetermined method, a detecting step in which a plurality of feature detecting elements respectively detect a plurality of features; and A control step of controlling a resolution or a scale level by combining outputs of a plurality of feature detection elements for performing a processing step. 33. Image processing characterized by controlling a processing operation of a processing target image based on a result of detecting a predetermined pattern from a pattern of the processing target image by the pattern detection method according to claim 29. Method.