JP2016538630A - A method for determining local discriminant colors for image feature detectors - Google Patents

Abstract

Techniques for multi-channel feature detection include computing devices that determine the filter response of each image channel of the multi-channel image with respect to one or more image filters. The computing device determines a local identification color vector based on the filter response, applies the filter response to the local identification color vector to generate an adaptive response, and determines the total response of the multi-channel image based on the adaptive response .

Description

  Computer vision utilizes a variety of image feature detectors to identify image features or “points of interest” within an image. The image feature detector can identify edges, corners, blobs (ie, regions of points of interest), and / or ridges of the image being analyzed, depending on the particular algorithm / detector. For example, the Canny algorithm and Sobel filter perform edge detection, the Harris detector performs corner detection, LoG (Laplacian of Gaussian), Gaussian Hessian determinant, and DoG (Difference of Gaussian detectors identify corners and blobs in the image. Feature detection systems often utilize a combination of algorithms and detectors to more accurately identify the features of the image being analyzed.

  Common feature detectors such as speeded up robust features (SURF), scale-invariant feature transform (SIFT), Canny, Harris, and Sobel are based on single channel images (ie, gray (Scale image) features are detected and expressed. Thus, a multi-channel image (ie, a color image) must be converted to a single channel image as a preliminary analysis step for feature detection, which can result in considerable loss of image information. For example, the image pixel values of a single channel grayscale image can be generated as a linear combination of corresponding pixel values in each of a plurality of channels of a multichannel image. Thus, contrast between multi-channel image pixels having different colors but having the same single channel gray scale representation is lost due to the gray scale conversion. Some algorithms use a perceptual-based color model (for example, CSIFT uses Kubelka-Munk theory to model the reflection spectrum of a colored body), but they are Use a global color for grayscale mapping, which results in loss of information.

The concepts described herein are illustrated by way of example and not limitation in the accompanying drawings. For simplicity and clarity of illustration, elements shown in the drawings are not necessarily drawn to scale. Reference labels have been repeated between drawings to indicate corresponding or similar elements where deemed appropriate.
FIG. 4 is a simplified block diagram of at least one embodiment of a computing device that performs multi-channel feature detection. FIG. 1 is a simplified block diagram of at least one embodiment of the computing device environment of FIG. FIG. 2 is a simplified flow diagram of at least one embodiment of a method for performing multi-channel feature detection on the computing device of FIG. FIG. 2 is a simplified flow diagram of at least one embodiment of a method for determining a local identification color vector on the computing device of FIG. Illustration of the captured image. FIG. 4 is a diagram of identified points of interest in a captured image based on the method for multi-channel feature detection of FIG. 3 and a SURF feature detector as an internal kernel.

  While the concepts of the present disclosure are amenable to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and are described in detail herein. However, the concept of the present disclosure is not intended to be limited to the particular forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives consistent with the present disclosure and the appended claims. Please understand that it is to cover.

  References herein such as “one embodiment”, “an embodiment”, “an exemplary embodiment”, and the like refer to any embodiment that the described embodiments may include particular features, structures, or characteristics. It is shown that the form may include such specific features, structures, or characteristics, or need not necessarily include such specific features, structures, or characteristics. Moreover, such phrases are not necessarily referring to the same embodiment. Further, where a particular feature, structure, or characteristic is described in the context of an embodiment, whether or not explicitly stated, such feature, structure, or It is assumed that implementing the characteristics is within the knowledge of those skilled in the art. Further, the items included in the list of the form “at least one of A, B, and C” are: (A); (B); (C); (A and B); (A and C); It should be understood that (B and C); or (A, B, and C) may be meant. Similarly, items listed in the form “at least one of A, B, or C” are (A); (B); (C); (A and B); (A and C); (B and C); or (A, B, and C).

  The disclosed embodiments can be implemented by hardware, firmware, software, or any combination thereof, as the case may be. The disclosed embodiments can also be carried by one or more machine-readable storage media (eg, computer-readable storage media) that can be read and executed by one or more processors. Or it can be implemented as instructions stored in such a medium. A machine-readable storage medium is any storage device, mechanism, or other physical structure that stores or transmits information in a form readable by a machine (eg, volatile memory, nonvolatile memory, media disk, or Other media devices).

  In the drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such specific arrangement and / or order is not required. Rather, in some embodiments, such features may be configured in a manner and / or order that differs from the manner and / or order shown in the illustrated drawings. Furthermore, the inclusion of structural or methodic features within a particular drawing does not imply that such features are required in all embodiments, and in some embodiments, such features It does not mean that such features are not included or that such features are not combined with other features.

という二次形式として表現され得る。オリジナルのＨａｒｒｉｓ内部カーネルの応答は、

という二次形式として表現され得、ここで、ｋはアルゴリズムパラメータである。さらに、Ｃａｎｎｙ内部カーネルの二乗応答（square response）は、

という二次形式として表現され得、ここで、Ｄ_ｘ及びＤ_ｙは一次空間微分フィルタ応答であり、ここでも、ｘ及びｙは画像の空間座標である。 Referring now to FIG. 1, a computing device 100 for multi-channel feature detection is configured to detect features of a multi-channel image (eg, points of interest such as corners, edges, blobs, etc.). To do so, the computing device 100 utilizes information from multiple image channels rather than a single channel image or a grayscale image (eg, a transformed image) when identifying image features. In an exemplary embodiment, computing device 100 may provide a local discriminating color (LDC) in which the response function of the internal kernel may be expressed as a linear form function or a quadratic form function. Configured to implement a low-complexity non-iterative algorithm for computing local differentiating color vectors. It should be understood that such a case covers a wide range of single channel feature detectors with an internal kernel that can be adapted for use with LDC vectors. For example, the second order spatial differential filter responses D _xx , D _xy , and D _yy can be calculated, where x and y are the spatial coordinates of the image. Accordingly, the response of the LoG internal kernel can be expressed in a primary form of (D _xx + D _yy ). The response of the SURF internal kernel is

Can be expressed as a secondary form. The response of the original Harris internal kernel is

Where k is an algorithm parameter. In addition, the square response of the Canny internal kernel is

Where D _x and D _y are first-order spatial differential filter responses, where x and y are the spatial coordinates of the image.

  The computing device 100 can be implemented as any type of computing device capable of multi-channel feature detection and capable of performing the functions described herein. For example, the computing device 100 may be a cellular phone, a smartphone, a tablet computer, a netbook, a notebook, an ultrabook, a laptop computer, a personal digital assistant, a mobile internet device, a desktop computer, a hybrid device, and / or It can be embodied as any other computing / communication device. As shown in FIG. 1, an exemplary computing device 100 includes a processor 110, an input / output (“I / O”) subsystem 112, memory 114, data storage 116, communication circuitry 118, and one or more peripherals. Device 120 is included. Further, the peripheral device 120 includes a camera 122 and a display 124. Of course, the computing device 100 may include other components or other components, such as components normally found in a typical computing device (eg, various input / output devices) in other embodiments. Further, in some embodiments, one or more of the illustrated components may be incorporated into another component or may be from a portion of another component. For example, the memory 114 or a portion of the memory 114 may be incorporated into the processor 110 in some embodiments.

  The processor 110 may be embodied as any type of processor that can perform the functions described herein. For example, the processor may be embodied as a single core processor, a multi-core processor, a digital signal processor, a microcontroller, or other processor or processing / control circuit. Similarly, the memory 114 can be embodied as any type of volatile memory, non-volatile memory, or data storage that can perform the functions described herein. During operation, memory 114 may store various software and data such as operating systems, applications, programs, libraries, and drivers that are used during operation of computing device 100. The memory 114 is communicatively connected to the processor 110 via the I / O subsystem 112. I / O subsystem 112 may be embodied as circuitry and / or components that facilitate input / output operations by processor 110, memory 114, and other components of computing device 100. For example, the I / O subsystem 112 may include a memory controller hub, input / output control hub, firmware device, communication link (ie, point-to-point link, bus link, wire, cable, light guide, printed circuit board trace, etc.), And / or can be implemented as or include other components and subsystems that facilitate input / output operations. In some embodiments, the I / O subsystem 112 can form part of a system-on-chip (SoC), and together with the processor 110, memory 114, and other components of the computing device 100, The integrated circuit chip can be incorporated.

  Data storage 116 may be any type of one or more configured for short-term or long-term storage of data, such as, for example, memory devices and circuits, memory cards, hard disk drives, solid state drives, or other data storage devices. It can be embodied as a device. Communication circuit 118 may be embodied as any communication circuit, communication device, or collection thereof that can enable communication between computing device 100 and other remote devices over a network (not shown). be able to. To do so, the communication circuit 118 may use any suitable communication technology (eg, wireless or wired communication) and associated protocols (eg, Ethernet®, Bluetooth®, Wi-Fi®). ), WiMAX (registered trademark), etc.), for example, depending on the type of network. The network may be embodied as any type of communication network that can facilitate communication between the computing device 100 and a remote device.

  Peripheral device 120 of computing device 100 may include any number of additional peripheral devices or interface devices. The specific devices included in peripheral device 120 may depend on, for example, the type and / or intended use of computing device 100. As described above, the peripheral device 120 includes the camera 122 and the display 124. The camera 122 may be embodied as any peripheral or integrated device suitable for capturing images, such as a still camera, video camera, webcam, or other device capable of capturing video and / or images. can do. The camera 122 can be used, for example, to capture a multi-channel image in which features are detected. The display 124 of the computing device 100 can be embodied as any one or more display screens that can display information to a viewer of the computing device 100. Display 124 may be embodied as any suitable display technology including, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, a cathode ray tube (CRT) display, a plasma display, and / or other display technologies. Or any such suitable display technology can be used. The display 124 can be used to display, for example, an image that shows the total response of the analyzed image. Although the camera 122 and the display 124 are shown in FIG. 1 as being integrated into the computing device 100, the camera 122 and / or the display 124 may be remote from the computing device 100 and compute in other embodiments. It should be understood that communication device 100 may be communicatively connected.

  With reference now to FIG. 2, in use, computing device 100 establishes an environment 200 for multi-channel feature detection. As described below, the computing device 100 determines the total image response of the analyzed multi-channel image based on the filter response and local identification color (LDC) vector of the individual image channels of the multi-channel image. The exemplary environment 200 of the computing device 100 includes an image capture module 202, an image analysis module 204, a display module 206, and a communication module 208. Further, the image analysis module 204 includes an image filtering module 210, a local identification color (LDC) module 212, and a response determination module 214. Each of the image capture module 202, the image analysis module 204, the display module 206, the communication module 208, the image filtering module 210, the local identification color module 212, and the response determination module 214 is as hardware, software, firmware, or a combination thereof. Can be embodied. Further, in some embodiments, one of these exemplary modules may form part of another module.

  The image capture module 202 controls the camera 122 to capture an image in the field of view of the camera 122 (eg, for multi-channel feature detection). Depending on the particular embodiment, the images may be captured as streaming video or as individual images / frames. In other embodiments, the image capture module 202 may obtain multi-channel images in another form for analysis and feature detection. For example, the multi-channel image may be received by the communication module 208 from a remote computing device (eg, in a cloud computing environment). It should be understood that the captured image can be embodied as any suitable multi-channel image. For example, the image can be a three-channel image such as an RGB (red-green-blue) image, an HSL (hue-saturation-luminance) image, or an HSV (hue-saturation-brightness) image. The multi-channel image feature detection described herein can be any type including a channel for a non-color space (eg, RGB-D (depth), infrared, temperature map, microwave map, or other image channel). It should be further understood that the present invention can be applied to multiple image channels.

  The image analysis module 204 acquires an image captured by the camera 122 from the image capture module 202. In an exemplary embodiment, the image analysis module 204 may include image extended space point coordinates and parameters (eg, for use by scale-space representations and / or scale-space detectors). Set. Further, as described in more detail below, the image analysis module 204 applies various filters to the image to be analyzed to determine an LDC vector for each image point of the image (or a subset thereof), and the image (or Determine the total response of each image point of that subset).

  The image filtering module 210 determines a filter response (ie, the result of applying the image filter to the image) for each image channel of the multi-channel image for one or more image filters. For example, an image filter may be applied to each pixel of the image in some embodiments. In doing so, it should be understood that the image filter can be applied, for example, using a “windowing” method in which the image filter is applied to a neighborhood of pixels (eg, the size of the image filter kernel). Image filters are generally applied to individual pixels of an image channel, but for purposes of simplicity and clarity herein, image filters are not the values of individual pixels, but the entire image channel or It can be described as applied to other entire structures. In embodiments where the multi-channel image includes three channels, the image filtering module 210 applies each image filter to each of the three channels to generate a corresponding filter response based on that image filter. It should be understood that the filter response for a particular image channel of a multi-channel image can be expressed as a vector that includes the corresponding response of the image channel to one or more image filters. Further, such a vector may be referred to as the “response vector” or “vector response” of the corresponding image channel. Further, in some embodiments, the particular image filter used needs to be a primary type or secondary type image filter. In other embodiments, the LDC vector may be applied to the pixels of the original image channel without previous filtering or with only a trivial / identity filter.

  The local identification color module 212 obtains a local identification color vector based on the filter response determined by the image filtering module 210. As described in detail below, the local identification color vector defines a weight for the linear combination of filter responses for the image channel and is extreme (ie, minimum or maximum depending on the particular embodiment). Calculated or determined as a vector that generates a total response. In the primary form, the local identification color module 212 determines a local identification color vector that is a collinear vector with the total response vector determined for each image channel. In the quadratic form, the local identification color vector is as an eigenvector (or normalized eigenvector) corresponding to the pole eigenvalue of a particular generated symmetric matrix (ie, the largest eigenvalue or the smallest eigenvalue depending on the particular embodiment). Desired. Thus, in the exemplary embodiment, the local identification color vector is not calculated as a result of an optimization algorithm (eg, minimizing or maximizing a cost function), but in closed form. Can be expressed.

  The response determination module 214 applies the local identification color vector to the image filter response generated by the image filtering module 210 to generate an adapted response, and based on the adaptive response, the total response of the multi-channel image. To decide. In the exemplary embodiment, the response determination module 214 applies the local identification color vector to the image filter response by separately calculating the inner product of the response vector and the local identification color vector for each image channel of the multi-channel image. To do. In addition, as will be described in more detail below, the response determination module 214 generates a scalar value based on the parameters and adaptive response of the particular filter and / or feature detection algorithm used, thereby generating a total multi-channel image. Determine the response.

  In the exemplary embodiment, response determination module 214 also suppresses the spatial non-extreme response of the multichannel image total response. That is, in some embodiments, the response determination module 214 removes non-interest points that can be identified based on a predetermined threshold from the total response. That is, an interest point can be identified as an image point having a local extreme response that is below or above a threshold, depending on the particular embodiment. Therefore, only the points of interest are left in the total response.

  Display module 206 is configured to render an image on display 124 for viewing by a user of computing device 100. For example, the display module 206 can display one or more captured / received images (see FIG. 5) and / or images (see FIG. 6) showing the total response of the images. Further, it should be understood that the display module 206 can render a visual display of the image at another stage of the feature detection process. For example, display module 206 can render a graphical and / or textual representation of the total response before suppressing individual filter responses, local identification color vectors, adaptive responses, and / or non-polar responses.

  The communication module 208 handles communication between the computing device 100 and a remote device over a network. As discussed above, the communication module 208 can receive multi-channel images from a remote computing device for analysis (eg, in a cloud computing environment or for offload execution purposes). Thus, in some embodiments, the communication module 208 can also send the result of the feature detection analysis (eg, total response) to the remote computing device.

  With reference now to FIG. 3, in use, computing device 100 may perform a method 300 for performing multi-channel feature detection. The example method 300 begins at block 302 in FIG. At block 302, the computing device 100 determines whether to perform multi-channel feature detection. If the computing device 100 determines to perform multi-channel feature detection, at block 304, the computing device 100 sets a coordinate system for the image expansion space point. That is, the computing device 100 may include, for example, additional parameters (eg, scale) and Cartesian coordinate systems (eg, the x-axis and y-axis are typically used) for use by a scale-spatial image feature detector. Set.

At block 306, the computing device 100 determines a filter response for each image channel based on one or more image filters (eg, Hessian determinant, Canny filter, Sobel filter, etc.). In doing so, at block 308, the computing device 100 generates a response vector for each image channel based on the filter response, as described above (ie, by applying image filters to the individual image channels). . For example, assume that the multi-channel image to be analyzed is a 3-channel RGB (red-green-blue) image, and the second order partial derivative of the Gaussian filter (ie, the component of the Hessian matrix) is used as the image filter. Thus, the image filter includes g _xx , g _yy , and g _xy that are second order partial differential coefficients for the corresponding image dimension. In such an embodiment, each of the image filters (ie, each of g _xx , g _yy , and g _xy ) is applied to the red channel to generate a response vector for the red image channel. As described above, the image filter can be applied to each pixel of the image. Thus, a response vector can be generated for each pixel of the image channel. Similarly, each of the image filters is applied to a blue image channel and a green image channel, and a response vector is generated for each of these channels. Each response vector can be converted to a scalar value. For example, the Hessian determinant can be determined by the quadratic form [g _xx g _yy g _xy ] B [g _xx g _yy g _xy ] ^T , where B is a predetermined matrix, g _xx , g _yy and g _xy are second-order partial differential coefficients of the Gaussian filter obtained with respect to the corresponding spatial coordinates x and / or y. Of course, other embodiments may use different numbers of image filters and / or analyze images with different numbers of channels. Thus, as a general case, assume there are n channels and p filters. In such a case, the computing device 100 generates n response vectors (ie, one for each channel) of size / length p (or more specifically, size p × 1). The component of this response vector is the image channel filter response for the corresponding image filter.

をそれぞれ算出することができる。ここで、画像チャネルｉ及びｊについてＡ＝｛ｑ_ｉｊ｝である。そのような実施形態において、ｑ_ｉｊは、ｉ番目の行及びｊ番目の列に位置する行列Ａの成分を表し、ｆは、ｆのインデックスに対応する画像チャネルに関する応答ベクトルであり、Ｔは、転置演算子であり（すなわち、ｆ^Ｔは、ｆの転置であり）、Ｂは、１以上の画像フィルタに基づく予め定められた行列である。例えば、ヘッセ行列に関して上述した実施形態において、行列Ｂは、

として定められ得、演繹的に（a priori）既知である。別の実施形態では、行列Ｂは、画像及び／又はフィルタパラメータに基づいて算出され得る。例示的な実施形態において、コンピューティングデバイス１００は、ブロック４０８において、３チャネル画像（例えば、ＲＧＢ画像）について、行列Ａを、

として算出する。もちろん、他の実施形態では、解析される画像は、より少ない又はより多い数のチャネルを含んでもよく、そのような実施形態においては、行列Ａは、それに応じたサイズとなる（例えば、解析される画像が４つのチャネルを含む実施形態では４×４行列、等）。すなわち、行列Ａは、ｎ×ｎ行列として具現化され、ここで、ｎは、画像チャネルの数である。 At block 310, the computing device 100 determines a local identification color vector (eg, a normalized LDC vector) based on the filter response of each image channel. That is, the computing device 100 uses the response vector for the image channel to generate a local identification color vector. To do so, the computing device 100 may perform a method 400 for determining a local identification color vector, as shown in FIG. The example method 400 begins at block 402. In block 402, the computing device 100 determines whether to generate a local identification color vector. When generating a local identification color vector, at block 404, the computing device 100 generates or otherwise determines a symmetric matrix A for the image. It should be understood that the computing device 100 can generate a symmetric quadratic matrix using any suitable technique, algorithm, and / or mechanism. For example, in one embodiment, the computing device 100 determines that the matrix A in block 406

Can be calculated respectively. Here, A = {q _ij } for image channels i and j. In such an embodiment, q _ij represents the components of matrix A located in the i th row and j th column, f is the response vector for the image channel corresponding to the index of f, and T is Is a transpose operator (ie, f ^T is a transpose of f), and B is a predetermined matrix based on one or more image filters. For example, in the embodiment described above with respect to the Hessian matrix, the matrix B is

And is known a priori. In another embodiment, the matrix B may be calculated based on the image and / or filter parameters. In the exemplary embodiment, computing device 100 at block 408 computes matrix A for a three-channel image (eg, an RGB image),

Calculate as Of course, in other embodiments, the image to be analyzed may include a smaller or greater number of channels, and in such embodiments, the matrix A is sized accordingly (eg, analyzed). In an embodiment where the image contains 4 channels, etc.). That is, matrix A is embodied as an n × n matrix, where n is the number of image channels.

  In block 410, the computing device 100 determines eigenvalues of the matrix A. It should be understood that the computing device 100 can use any suitable technology, algorithm, or mechanism to do so. For example, the computing device 100 can determine and use the characteristic equation of the matrix A when specifying the eigenvalues of the matrix A. At block 412, the computing device 100 identifies the eigenvector corresponding to the extreme eigenvalue of matrix A (ie, the maximum or minimum eigenvalue depending on the particular embodiment), and at block 414, the computing device 100 100 selects the identified eigenvector as a local identification color vector. In doing so, in some embodiments, the computing device 100 may generate a unit vector of the identified eigenvectors at block 416. That is, the computing device 100 can normalize the eigenvector to generate a unit vector corresponding to the eigenvector, and can select this unit vector as a local identification color vector.

Returning to FIG. 3, at block 312, the computing device 100 applies the image filter response to the generated / determined local identification color vector to generate a corresponding adaptive response. In doing so, at block 314, the computing device 100 calculates the dot product of the image filter response and the local identification color vector (eg, to generate one vector). For example, in the Hessian example described, the dot product of the local discriminant color vector and the vector containing the second partial differential coefficient is calculated for all channels of the image, since this vector transposition is multiplied by the local discriminant color vector. Is equivalent to Specifically, [g _xx g _xy g _yy ] ^T is multiplied with the local identification color vector for all channels of the image. At block 316, the computing device 100 generates a total response based on the adaptive response. In the exemplary embodiment, computing device 100 generates a scalar value based on the particular feature detection algorithm / filter and adaptive response used. For example, in embodiments where a Hessian matrix is used, the computing device 100 can use the characteristics and / or parameters of the Hessian matrix to generate a total response (eg, using a Hessian determinant). It should be appreciated that the computing device 100 can utilize any suitable technology, algorithm, and / or mechanism to do so.

  At block 318, the computing device 100 suppresses the spatial non-polar response of the total response in the expansion space. That is, at block 320, the computing device 100 may remove non-interest points from the total response. As described above, interest points and non-interest points can be identified based on a predetermined threshold. For example, in one embodiment, a spatial image point of the total response that has an intensity value or local polar response that exceeds a predetermined threshold is considered to be an “interest point”, whereas a predetermined threshold is Spatial image points of the total response with intensity values or local polar responses not exceeding are non-interest points. It should be understood that the computing device 100 can properly identify “interest points” using any feature detection algorithm (eg, SURF) that has a quadratic response function. As suggested above, depending on the particular algorithm, the points of interest may include corners, edges, blobs, and / or other image characteristics. Further, in some embodiments, generating a total response at block 316 includes suppressing a spatial non-polar response.

  As described above, the computing device 100 can generate and display an image showing the total response of the analyzed multi-channel image for viewing by the user. For example, a simplified analyzed image 500 is shown in FIG. 5 and is a simplified example output image 600 (which is based on multi-channel feature detection (by the SURF internal kernel) of image 500. Is generated by way of example). In the simplified output image 600, the identified points of interest / features are shown differently as hatched circles to show corresponding different colored circles. Of course, image 600 is a simplified version of the actual output image that would be generated using the techniques described herein, such actual output image being, for example, an original parsed It should be understood that points of interest / features can be identified using a greater or lesser number of circles having a wider range of different colors and sizes depending on the image being captured. Further, feature detection performed on images analyzed by computing device 100 to generate an output image as described herein differs from single channel grayscale feature detection in grayscale conversion. It should be understood that there is no inherent loss of information.

Examples Illustrative examples of the techniques disclosed herein are provided below. Embodiments of the technology may include any one or more and any combination of the examples described below.

  Example 1 is a computing device for multi-channel feature detection, an image filtering module that determines a filter response for each image channel of a multi-channel image for one or more image filters, and a local based on the filter response A local identification color module for determining an identification color vector; (i) applying the filter response to the local identification color vector to generate an adaptive response; (ii) based on the adaptive response, A computing device comprising: a response determining module for determining a response.

  Example 2 includes the subject matter of Example 1, and the one or more image filters include an identification filter.

  Example 3 includes the subject matter of any of Examples 1 and 2, wherein determining the local identification color vector includes a vector of total responses determined for each image channel based on the filter response of each image channel Including determining a vector that is collinear.

  Example 4 includes the subject matter of any of Examples 1-3, wherein determining the local identification color vector includes determining a symmetric matrix for the multi-channel image and a minimum eigenvalue of the symmetric matrix. Or identifying the eigenvector corresponding to the largest eigenvalue.

  Example 5 includes the subject matter of any of Examples 1-4, wherein determining the locally discriminating color vector includes determining a symmetric quadratic matrix for the multi-channel image and the quadratic form Identifying eigenvectors corresponding to the minimum or maximum eigenvalues of the matrix.

であり、ｑ_ｉｊは、ｉ番目の行及びｊ番目の列における行列Ａの成分であり、ｆは、画像チャネルのフィルタ応答に基づく、ｆのインデックスに対応する該画像チャネルに関する応答ベクトルであり、Ｔは、転置演算子であり、Ｂは、前記１以上の画像フィルタに基づく予め定められた行列である。 Example 6 includes the subject matter of any of Examples 1-5, and determining the quadratic matrix includes calculating a matrix A = {q _ij } for image channels i and j, where so,

Q _ij is the component of matrix A in the i th row and j th column, f is the response vector for the image channel corresponding to the index of f based on the filter response of the image channel, T is a transpose operator, and B is a predetermined matrix based on the one or more image filters.

  Example 7 includes the subject matter of any of Examples 1-6, and determining the local identification color vector includes normalizing the identified eigenvector to generate the local identification color vector. .

  Example 8 includes the subject matter of any of Examples 1-7, and determining the filter response of each image channel of the multi-channel image includes, for each pixel of the multi-channel image, each of the multi-channel image Determining the filter response of the image channel.

  Example 9 includes the subject matter of any of Examples 1-8, wherein determining the filter response of each image channel of the multi-channel image is based on the filter response of each image channel. Generating a response vector for.

  Example 10 includes the subject matter of any of Examples 1-9, wherein applying the filter response to the local identification color vector calculates an inner product of the local identification color vector and the filter response. Including.

  Example 11 includes the subject matter of any of Examples 1-10, and determining the local identification color vector includes determining a normalized local identification color vector based on the filter response.

  Example 12 includes the subject matter of any of Examples 1-11, wherein the response determination module further suppresses the spatial non-polar response of the total response of the multi-channel image.

  Example 13 includes the subject matter of any of Examples 1-12, wherein suppressing the spatial non-polar response includes removing non-interest points from the total response of the multi-channel image; Interest points are identified based on a predetermined threshold.

  Example 14 includes the subject matter of any of Examples 1-13, and further comprises a display module that displays an image showing the total response on a display of the computing device.

  Example 15 includes the subject matter of any of Examples 1-14, further comprising an image capture module that captures a captured image by a camera of the computing device, wherein the multi-channel image is the captured image.

  Example 16 includes the subject matter of any of Examples 1-15, wherein the one or more image filters include one or more of a first derivative image filter and a second derivative image filter.

  Example 17 is a method for performing multi-channel feature detection on a computing device, wherein the computing device determines a filter response for each image channel of a multi-channel image with respect to one or more image filters. Determining a local identification color vector based on the filter response with the computing device; applying the filter response to the local identification color vector with the computing device to generate an adaptive response; Determining a total response of the multi-channel image based on the adaptive response by the computing device.

  Example 18 includes the subject matter of Example 17, wherein the one or more image filters include an identification filter.

  Example 19 includes the subject matter of any of Examples 17 and 18, wherein determining the local identification color vector includes a vector of total responses determined for each image channel based on the filter response of each image channel Including determining a vector that is collinear.

  Example 20 includes the subject matter of any of Examples 17-19, wherein determining the local discriminating color vector is determining a symmetric matrix for the multi-channel image and a minimum eigenvalue of the symmetric matrix. Or identifying the eigenvector corresponding to the largest eigenvalue.

  Example 21 includes the subject matter of any of Examples 17-20, wherein determining the local identification color vector includes determining a symmetric quadratic matrix for the multi-channel image and the quadratic form Identifying eigenvectors corresponding to the minimum or maximum eigenvalues of the matrix.

であり、ｑ_ｉｊは、ｉ番目の行及びｊ番目の列における行列Ａの成分であり、ｆは、画像チャネルのフィルタ応答に基づく、ｆのインデックスに対応する該画像チャネルに関する応答ベクトルであり、Ｔは、転置演算子であり、Ｂは、前記１以上の画像フィルタに基づく予め定められた行列である。 Example 22 includes the subject of any of Examples 17-21, and determining the quadratic form matrix includes calculating a matrix A = {q _ij } for image channels i and j, wherein so,

Q _ij is the component of matrix A in the i th row and j th column, f is the response vector for the image channel corresponding to the index of f based on the filter response of the image channel, T is a transpose operator, and B is a predetermined matrix based on the one or more image filters.

  Example 23 includes the subject matter of any of Examples 17-22, and determining the local identification color vector includes normalizing the identified eigenvector to generate the local identification color vector. .

  Example 24 includes the subject matter of any of Examples 17-23, and determining the filter response of each image channel of the multi-channel image for each pixel of the multi-channel image Determining the filter response of the image channel.

  Example 25 includes the subject matter of any of Examples 17-24, wherein determining the filter response of each image channel of the multi-channel image is based on the filter response of each image channel. Generating a response vector for.

  Example 26 includes the subject matter of any of Examples 17-25, wherein applying the filter response to the local discriminating color vector calculates an inner product of the local discriminating color vector and the filter response. Including.

  Example 27 includes the subject matter of any of Examples 17-26, and determining the local identification color vector includes determining a normalized local identification color vector based on the filter response.

  Example 28 includes the subject matter of any of Examples 17-27, and further includes suppressing, by the computing device, a spatial non-polar response of the total response of the multi-channel image.

  Example 29 includes the subject matter of any of Examples 17-28, wherein suppressing the spatial non-polar response includes removing non-interest points from the total response of the multi-channel image, Interest points are identified based on a predetermined threshold.

  Example 30 includes the subject matter of any of Examples 17-29, and further includes displaying an image showing the total response on a display of the computing device.

  Example 31 includes the subject matter of any of Examples 17-30, further comprising capturing a captured image with a camera of the computing device, wherein the multi-channel image is the captured image.

  Example 32 includes the subject matter of any of Examples 17-31, wherein the one or more image filters include one or more of a first derivative image filter and a second derivative image filter.

  Example 33 is a computing device, a processor and a memory storing a plurality of instructions, wherein the plurality of instructions are executed on the computing device when executed by the processor. A computing device having a memory for performing any of the methods.

  Example 34 is one or more machine-readable storage media storing a plurality of instructions, wherein the plurality of instructions are transmitted to a computing device in response to execution of the plurality of instructions. Including one or more machine-readable storage media that cause any of the methods of 32 to be performed.

  Example 35 is a computing device for multi-channel feature detection, for one or more image filters, means for determining a filter response for each image channel of the multi-channel image, and a local identification color based on the filter response Means for determining a vector; means for applying the filter response to the local identification color vector to generate an adaptive response; and means for determining a total response of the multi-channel image based on the adaptive response. Including computing devices.

  Example 36 includes the subject matter of Example 35, and the one or more image filters include an identification filter.

  Example 37 includes the subject matter of any of Examples 35 and 36, wherein the means for determining the local identification color vector is a vector of total responses determined for each image channel based on the filter response of each image channel. And means for obtaining a vector that is collinear.

  Example 38 includes the subject matter of any of Examples 35-37, wherein the means for determining the local identification color vector includes means for determining a symmetric matrix for the multi-channel image, and a minimum of the symmetric matrix. Means for identifying an eigenvector corresponding to the eigenvalue or the maximum eigenvalue.

  Example 39 includes the subject matter of any of Examples 35-38, wherein the means for determining the local identification color vector includes means for determining a symmetric quadratic matrix for the multi-channel image; Means for identifying an eigenvector corresponding to a minimum or maximum eigenvalue of a matrix of the form.

であり、ｑ_ｉｊは、ｉ番目の行及びｊ番目の列における行列Ａの成分であり、ｆは、画像チャネルのフィルタ応答に基づく、ｆのインデックスに対応する該画像チャネルに関する応答ベクトルであり、Ｔは、転置演算子であり、Ｂは、前記１以上の画像フィルタに基づく予め定められた行列である。 Example 40 includes the subject matter of any of Examples 35-39, wherein the means for determining a matrix of the quadratic form includes calculating a matrix A = {q _ij } for image channels i and j; here,

Q _ij is the component of matrix A in the i th row and j th column, f is the response vector for the image channel corresponding to the index of f based on the filter response of the image channel, T is a transpose operator, and B is a predetermined matrix based on the one or more image filters.

  Example 41 includes the subject matter of any of Examples 35-40, wherein the means for determining the local identification color vector includes means for normalizing the identified eigenvector to generate the local identification color vector. Including.

  Example 42 includes the subject matter of any of Examples 35-41, wherein the means for determining the filter response of each image channel of the multi-channel image includes, for each pixel of the multi-channel image, the multi-channel image Means for determining the filter response of each image channel.

  Example 43 includes the subject matter of any of Examples 35-42, wherein the means for determining the filter response of each image channel of the multi-channel image is based on the filter response of each image channel. Means for generating a response vector for the channel.

  Example 44 includes the subject matter of any of Examples 35-43, wherein the means for applying the filter response to the local identification color vector to generate an adaptive response includes the local identification color vector and the filter response. Means for calculating the inner product of.

  Example 45 includes the subject matter of any of Examples 35-44, wherein the means for determining the local identification color vector includes means for determining a normalized local identification color vector based on the filter response.

  Example 46 includes the subject matter of any of Examples 35-45, further comprising means for suppressing a spatial non-polar response of the total response of the multi-channel image.

  Example 47 includes the subject matter of any of Examples 35-46, wherein the means for suppressing the spatial non-polar response includes means for removing non-interest points from the total response of the multi-channel image, Non-interest points are identified based on a predetermined threshold.

  Example 48 includes the subject matter of any of Examples 35-47, further comprising means for displaying an image showing the total response on the display of the computing device.

  Example 49 includes the subject matter of any of Examples 35-48, further comprising means for capturing a captured image by a camera of the computing device, wherein the multi-channel image is the captured image.

  Example 50 includes the subject matter of any of Examples 35-49, wherein the one or more image filters include one or more of a first derivative image filter and a second derivative image filter.

  Example 51 is a computing device for multi-channel feature detection, wherein a local identification color module that obtains a local identification color vector based on pixel values of each image channel of a multi-channel image, and (i) each image channel Computing comprising: applying the pixel values to the local identification color vector to generate an adaptive response; and (ii) determining a total response of the multi-channel image based on the adaptive response. Includes devices.

  Example 52 includes the subject of Example 51, wherein determining the local identification color vector determines a vector that is collinear with a vector of total responses determined for each image channel based on the pixel values of each image channel. Including that.

  Example 53 includes the subject matter of any of Examples 51 and 52, wherein determining the local discriminating color vector includes determining a symmetric matrix for the multi-channel image and a minimum eigenvalue of the symmetric matrix. Or identifying the eigenvector corresponding to the largest eigenvalue.

  Example 54 includes the subject matter of any of Examples 51-53, and determining the local identification color vector includes normalizing the identified eigenvector to generate the local identification color vector. .

  Example 55 includes the subject matter of any of examples 51-54, wherein the response determination module further suppresses a spatial non-polar response of the total response of the multi-channel image.

  Example 56 is a method for performing multi-channel feature detection on a computing device, wherein the computing device determines a local identification color vector based on pixel values for each image channel of the multi-channel image; Applying the pixel value of each image channel to the local identification color vector by the computing device to generate an adaptive response; and based on the adaptive response, the multi-channel image by the computing device. Determining a total response.

  Example 57 includes the subject of Example 56, wherein determining the local identification color vector determines a vector that is collinear with the total response vector determined for each image channel based on the pixel values of each image channel. Including that.

  Example 58 includes the subject matter of any of Examples 56 and 57, wherein determining the local identification color vector includes determining a symmetric matrix for the multi-channel image and a minimum eigenvalue of the symmetric matrix. Or identifying the eigenvector corresponding to the largest eigenvalue.

  Example 59 includes the subject matter of any of Examples 56-58, and determining the local identification color vector includes normalizing the identified eigenvector to generate the local identification color vector. .

  Example 60 includes the subject matter of any of Examples 56-59, and further includes suppressing, by the computing device, a spatial non-polar response of the total response of the multi-channel image.

  Example 61 is a computing device, a processor and a memory storing a plurality of instructions, wherein when the plurality of instructions are executed by the processor, the computing device is transferred to the computing device. A computing device having a memory for performing any of the methods.

  Example 62 is one or more machine-readable storage media storing a plurality of instructions, wherein the plurality of instructions are transmitted to a computing device in response to execution of the plurality of instructions. One or more machine-readable storage media are included that cause any of the methods of 60 to be performed.

  Example 63 includes a computing device for multi-channel feature detection comprising means for performing the method of any of Examples 56-60.

Claims (26)

A computing device for multi-channel feature detection,
An image filtering module that determines a filter response of each image channel of the multi-channel image with respect to one or more image filters;
A local identification color module for determining a local identification color vector based on the filter response;
(I) applying the filter response to the local identification color vector to generate an adaptive response; (ii) a response determination module for determining a total response of the multi-channel image based on the adaptive response;
A computing device equipped with.   The computing of claim 1, wherein determining the local identification color vector comprises determining a vector that is collinear with a vector of total responses determined for each image channel based on the filter response of each image channel. device. Obtaining the local identification color vector includes
Obtaining a symmetric quadratic matrix for the multichannel image;
Identifying the eigenvector corresponding to the minimum or maximum eigenvalue of the matrix of the quadratic form;
The computing device of claim 1, comprising: Determining the matrix of the quadratic form includes calculating a matrix A = {q _ij } for image channels i and j, where

Q _ij is the component of matrix A in the i th row and j th column, f is the response vector for the image channel corresponding to the index of f based on the filter response of the image channel, The computing device of claim 3, wherein T is a transpose operator and B is a predetermined matrix based on the one or more image filters.   The computing device of claim 3, wherein determining the local identification color vector includes normalizing the identified eigenvector to generate the local identification color vector.   The computer of claim 1, wherein determining the filter response of each image channel of the multi-channel image includes determining, for each pixel of the multi-channel image, a filter response of each image channel of the multi-channel image. Device.   The computing device of claim 1, wherein determining the filter response of each image channel of the multi-channel image includes generating a response vector for each image channel based on the filter response of each image channel. .   The computing device of claim 7, wherein applying the filter response to the local identification color vector includes calculating an inner product of the local identification color vector and the filter response.   The computing device of claim 1, wherein determining the local identification color vector includes determining a normalized local identification color vector based on the filter response.   The computing device of claim 1, wherein the response determination module further suppresses a spatial non-polar response of the total response of the multi-channel image.   The suppressing of the spatial non-polar response includes removing non-interest points from the total response of the multi-channel image, wherein the non-interest points are identified based on a predetermined threshold. 11. The computing device according to 10. The computing device according to claim 1, further comprising: a display module configured to display an image indicating the total response on a display of the computing device.   The computing device according to claim 1, wherein the one or more image filters include one or more of a first-order differential image filter and a second-order differential image filter. A method for performing multi-channel feature detection on a computing device, comprising:
Determining a filter response for each image channel of the multi-channel image with respect to one or more image filters by the computing device;
Determining, by the computing device, a local identification color vector based on the filter response;
Applying, by the computing device, the filter response to the local identification color vector to generate an adaptive response;
Determining, by the computing device, a total response of the multi-channel image based on the adaptive response;
Including methods.   The method of claim 14, wherein determining the local identification color vector comprises determining a vector that is collinear with a vector of total responses determined for each image channel based on the filter response of each image channel. Obtaining the local identification color vector includes
Determining a symmetric matrix for the multi-channel image;
Identifying the eigenvector corresponding to the minimum or maximum eigenvalue of the symmetric matrix;
15. The method of claim 14, comprising: Obtaining the local identification color vector includes
Obtaining a symmetric quadratic matrix for the multichannel image;
Identifying the eigenvector corresponding to the minimum or maximum eigenvalue of the matrix of the quadratic form;
15. The method of claim 14, comprising: Determining the matrix of the quadratic form includes calculating a matrix A = {q _ij } for image channels i and j, where

Q _ij is the component of matrix A in the i th row and j th column, f is the response vector for the image channel corresponding to the index of f based on the filter response of the image channel, 18. The method of claim 17, wherein T is a transpose operator and B is a predetermined matrix based on the one or more image filters.   The method of claim 17, wherein determining the local identification color vector includes normalizing the identified eigenvector to generate the local identification color vector.   The method of claim 14, wherein determining the filter response of each image channel of the multi-channel image includes determining, for each pixel of the multi-channel image, the filter response of each image channel of the multi-channel image. .   The method of claim 14, wherein applying the filter response to the local identification color vector comprises calculating an inner product of the local identification color vector and the filter response. 15. The method of claim 14, further comprising suppressing by the computing device a spatial non-polar response of the total response of the multi-channel image.   23. A program for causing a computing device to execute the method according to any one of claims 14 to 22. A computing device for multi-channel feature detection,
A local identification color module for determining a local identification color vector based on the pixel value of each image channel of the multi-channel image;
(I) applying the pixel values of each image channel to the local identification color vector to generate an adaptive response; and (ii) determining a total response of the multi-channel image based on the adaptive response. When,
A computing device equipped with. Obtaining the local identification color vector includes: (i) obtaining a symmetric matrix for the multi-channel image; and (ii) identifying an eigenvector corresponding to a minimum eigenvalue or a maximum eigenvalue of the symmetric matrix. Including,
25. The computing device of claim 24, wherein the response determination module further suppresses a spatial non-polar response of the total response of the multi-channel image.   A machine-readable storage medium storing the program according to claim 23.

Images