JP6006676B2 - Marker embedding device, marker detecting device, marker embedding method, marker detecting method, and program - Google Patents

Description

  The present invention provides a marker embedding device, a marker detection device, a marker embedding method for facilitating reverse conversion to an original still image or moving image even when projective transformation is performed on the still image or moving image. The present invention relates to a marker detection method and a program.

  In the distribution of content such as images and videos, other information is embedded in the content so that it is not perceived by the human eye for purposes such as content identification and management, content copyright protection and management, and related information provision. A method using a digital watermark technique is known. In particular, in the digital watermark technology for image / video content, a method based on minute change of the pixel value is common. As a specific method, for example, when embedding a digital watermark, based on embedding target component position information, spread spectrum is generated by independently changing the real and imaginary components of the complex matrix, and a watermark pattern is generated independently of the input image. When an embedded image is generated by adding an actual image pattern and a digital watermark is detected, a detection target sequence is generated based on detection target component position information, and offset information is extracted and detected. There is a digital watermarking method that performs spectral despreading after correcting a target sequence and detects a digital watermark embedded in a cut-out pixel block (for example, Patent Document 1).

  In addition, in digital watermarking, it is necessary that a digital signal representing digital content is resistant to various modifications received after embedding the digital watermark, that is, the digital watermark can be detected after being subjected to various modifications. . An important modification that should be particularly resistant to digital watermarking is geometric transformation. Resistance to geometric transformation is an indispensable requirement when a content such as a target image or video is re-photographed with a camera and a digital watermark embedded in the content is detected. This is because geometric transformation occurs in the content according to the difference in the distance and angle at which the content is shot.

  As a method of providing resistance to geometric transformation, the edge of the image in which the digital watermark is embedded is detected, and the positions of the four corners of the image are specified to perform reverse projection transformation, and the state before the transformation is determined. There is a method of restoring (Patent Document 2). With this method, a digital watermark can be detected from an image accompanied by deformation by projective transformation, but in order to stably detect the edge of the image, it is necessary to surround the image with a black frame or a white frame. Since this is perceived by human eyes, there is a problem that the design of the content itself is greatly impaired.

  On the other hand, when a digital watermark is embedded as a spatial repetitive pattern and is detected, a method of correcting using the autocorrelation of the pattern (Patent Document 3), a marker is embedded in a frequency space of an image invisible to humans, There is a method (Patent Document 4 and Patent Document 5) for correction using this. These are not perceived by the human eye, and can detect digital watermarks from images with geometric transformation, but can support affine transformations represented by rotation, scale transformation, etc., and support projection transformation There is a problem that cannot be done.

JP 2003-219148 A Japanese Patent No. 4020093 Japanese translation of PCT publication No. 2003-510931 Japanese Patent No. 3949679 JP 2010-232886 A

  As described above, in the inventions and techniques described in Patent Documents 1 to 5, the original image or video is obtained by reverse projection transformation on the image or video that has undergone projection transformation without impairing the design of the content. There is a problem that can not be converted to.

  The present invention has been made to solve the above problems, and its purpose is to use a marker used when converting an image or video subjected to projective transformation into the original image or video without losing the design of the content. Providing a marker embedding device, a marker detecting device, a marker embedding method, a marker detecting method, and a program for embedding an image in an image or video.

  One embodiment of the present invention includes an image input unit that inputs one or a plurality of continuous images, and a color component that is difficult to perceive in the image, and the color component has a concentric striped pattern. The distance between the center points of the markers is predetermined along a first side defining a predetermined rectangular area on the image of at least four markers having four common shading patterns on a straight line passing through the center point And a distance between the center points of the markers along the second side different from the first side defining the rectangular region is superimposed on a straight line at a position forming a cross ratio of And a marker superimposing unit that superimposes on a straight line at a position that forms a cross-ratio, and a first identification code is superimposed between the markers superimposed along the first side for the color component of the image. , The matrix superimposed along the second side An identification code superimposing unit that superimposes a second identification code corresponding to the first identification code between the colors, and the marker, the first identification code, and the second identification code are superimposed on the color component. And a marker embedding device comprising an image output unit for outputting the image.

One embodiment of the present invention is characterized in that the marker embedding device further includes an information superimposing unit that superimposes information on the image in the rectangular area.
According to another aspect of the present invention, in the marker embedding device described above, the identification code superimposing unit uses the same identification code for the first identification code and the second identification code. .
According to another aspect of the present invention, in the marker embedding device, the identification code superimposing unit can identify the vertical direction of the image with respect to the first identification code and the second identification code. It is characterized by using.

  According to another aspect of the present invention, an image input unit that inputs one or a plurality of continuous images and four points in which a predetermined light and shade pattern exists in a color component that is difficult to perceive are detected and detected. When the cross ratio calculated from the four points is a predetermined value, the marker having the concentric striped pattern and having the four common shade patterns on the straight line passing through the center point of the concentric circle is located. The marker detection unit to be determined and the marker detected by the marker detection unit include at least two marker sets in which the center points are aligned on a straight line and the ratio of the distances between the center points has a predetermined cross ratio. A region specifying unit that specifies a partial region in the image based on a pair of marker pairs to which identification codes superimposed on the color components between the specified pairs of markers are associated, and the region A marker detecting apparatus characterized by comprising a projective transformation correction unit for performing projective transformation to the partial area constant section is specific to a given rectangular region.

Further, one aspect of the present invention is the marker detection apparatus described above, further comprising an information detection unit that detects information related to the image from the partial region that is projectively transformed into a rectangular region by the projective transformation correction unit. To do. Further, according to one aspect of the present invention, in the marker detection device described above, the marker detection unit is a box filter determined according to the shading pattern of the marker, and using a plurality of size box filters. , Raster scanning is sequentially performed on each pixel of the color component of the image to detect a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel, and the detected pixels continue in the raster scan direction. The marker is detected by a cross ratio calculated based on a positional relationship between four pixels.
Further, according to one aspect of the present invention, in the above-described marker detection device, the marker detection device further includes an integrated image calculation unit that calculates an integrated image in the color component of the image, and the marker detection unit is based on the integrated image, The position of the marker in the image is detected.
Further, according to one aspect of the present invention, in the marker detection device described above, the region specifying unit uses a pair of markers with matching identification codes superimposed between the markers of the specified set of markers as a pair of markers. A quadrilateral region obtained by connecting the line segments defined in (1) is specified as the partial region.
Further, one aspect of the present invention is characterized in that the marker detection device further includes an information output unit that outputs information indicating a position in the image of the partial region specified by the region specifying unit.

  Further, according to one embodiment of the present invention, there is an image input step of inputting one or a plurality of continuous images, and the color components have a concentric striped pattern with respect to color components that are difficult to perceive. The distance between the center points of the markers along the first side that defines a predetermined rectangular area on the image with at least four markers having four common shading patterns on a straight line passing through the center point of the concentric circles Is superimposed on a straight line at a position where a predetermined cross ratio is formed, and the distance between the center points of the markers along the second side different from the first side defining the rectangular region is set to at least four other markers. A marker superimposing step for superimposing the predetermined cross ratio on a straight line, and a first identification code between the markers superimposed along the first side with respect to the color component of the image; Overlap and along the second side An identification code superimposing step of superimposing a second identification code corresponding to the first identification code between the superimposed markers, the marker, the first identification code, and the second identification code in the color component And an image output step of outputting the image on which is superimposed.

One aspect of the present invention is the marker embedding method described above, further including an information superimposing step of superimposing information on the image on the rectangular area.
According to another aspect of the present invention, in the marker embedding method described above, the identification code superimposing step uses the same identification code for the first identification code and the second identification code. .
Further, according to one aspect of the present invention, in the marker embedding method described above, in the identification code superimposing step, an identification code that can identify the vertical direction of the image with respect to the first identification code and the second identification code It is characterized by using.

  Further, according to one aspect of the present invention, an image input step of inputting one or a plurality of continuous images and four points where a predetermined shading pattern exists in a color component that is difficult to perceive are detected and detected. When the cross ratio calculated from the four points is a predetermined value, the marker having the concentric striped pattern and having the four common shade patterns on the straight line passing through the center point of the concentric circle is located. Then, a marker detection step to be determined and at least two marker pairs in which the center points are aligned on a straight line and the ratio of the distances between the center points forms a predetermined cross ratio from the markers detected in the marker detection step. A region that identifies a partial region in the image based on a pair of marker pairs to which identification codes superimposed in the color component between the identified pairs of markers correspond. And specifying step, a marker detection method characterized by having a projection transformation correction step of performing projective transformation to predetermined rectangular region specified partial area in the region specifying step.

Further, one aspect of the present invention is characterized in that the marker detection method further includes an information detection step of detecting information on the image from the partial region that is projectively transformed into a rectangular region in the projective transformation correction step. To do.
Further, according to one aspect of the present invention, in the marker detection method described above, in the marker detection step, a box filter defined according to the shading pattern of the marker and using a plurality of size box filters is used. , Raster scanning is sequentially performed on each pixel of the color component of the image to detect a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel, and the detected pixels continue in the raster scan direction. The marker is detected by a cross ratio calculated based on a positional relationship between four pixels.
Further, one aspect of the present invention further includes an integrated image calculation step of calculating an integrated image in the color component of the image in the marker detection method described above, and in the marker detection step, based on the integrated image, The position of the marker in the image is detected.
Further, according to one aspect of the present invention, in the marker detection method described above, in the region specifying step, a marker set in which the identification codes superimposed between the markers in the specified marker set match is used as a pair. A quadrilateral region obtained by connecting the line segments defined in (1) is specified as the partial region.
One embodiment of the present invention is characterized in that the marker detection method further includes an information output step of outputting information indicating a position in the image of the partial region specified in the region specifying step.

  One embodiment of the present invention is a program for causing a computer to execute each step in the marker embedding method described above or each step in the marker detection method described above.

  According to the present invention, a marker that is difficult to be perceived by human eyes is realized by superimposing a marker on a color component that is difficult to perceive in an image. Further, by calculating a cross ratio that is not affected by the projective transformation from four points detected by the shading pattern of the marker, the position of the marker on the image can be detected. By superimposing the marker on a straight line at a position where the distance between the center points has a predetermined cross ratio, the marker superimposed on the straight line along the first and second sides in the projection-transformed image is also obtained. An image that has been subjected to the projective transformation can be converted to the original image by obtaining a reverse projective transformation that can be detected with high accuracy and transforms the partial region defined by the first and second sides into a predetermined rectangular region. be able to.

It is a figure which shows an example of the marker used in this embodiment. It is a figure which shows the example of an image which the marker embedding apparatus of this embodiment superimposed the marker on the image. It is a figure which shows an example of the Box filter which a marker detection apparatus uses in this embodiment. It is a block diagram which shows the structure of the marker embedding apparatus 1 in this embodiment. It is a figure which shows an example of the characteristic shading pattern which the marker used in this embodiment has. It is a figure which shows the example of an image on which the marker and ID code were superimposed by the marker embedding apparatus 1 in this embodiment. It is a flowchart which shows the marker embedding process which the marker embedding apparatus 1 in this embodiment performs. It is a block diagram which shows the structure of the marker detection apparatus 2 in this embodiment. It is a figure which shows the outline | summary of the Box filter which is the Box filter which the marker detection part 22 uses in this Embodiment, and has a different magnitude | size. 5 is a diagram showing an outline of processing in a line segment ID code detection unit 23. FIG. It is a figure which shows the example from which the area | region which embeds digital watermark information in this embodiment differs. It is a flowchart which shows the marker detection process which the marker detection apparatus 2 put in this embodiment performs. It is a figure which shows an example of the area | region which superimposes digital watermark information in this embodiment.

  Hereinafter, a marker embedding device, a marker detection device, a marker embedding method, a marker detection method, and a program according to an embodiment of the present invention will be described with reference to the drawings.

  The marker embedding device and the marker detection device according to the present embodiment are illustrated in order to detect conversion performed on an image or video when converting the image or video subjected to projective transformation to the original image or video. A concentric shading pattern as shown in FIG. FIG. 1 is a diagram illustrating an example of a marker used in the present embodiment. As shown in the figure, the marker is formed of a concentric striped pattern, and −1, +2 from the center point of the concentric circle toward the outer circumference with reference to the color difference (or luminance, etc.) of the circle at the center of the marker. , -2, +2, and -1 are equally spaced shading patterns having color differences according to the ratio. The marker embedding device superimposes markers at four corners of a predetermined area on an image or video. The marker embedding device converts the image or video from the RGB color space to the YCbCr (or YPbPr) color space when embedding the marker in the image or video, or the Cb component that is most difficult to be perceived by humans in terms of visual characteristics, or its Next, a marker is embedded in the Cr component that is difficult to perceive. In the RGB color space, a marker may be embedded in a B component that is difficult for humans to perceive among RGB components. In addition, when an image or video is expressed in another color space, a marker is placed on the color component that is most difficult to be perceived by humans or the color component that is next difficult to be perceived. It may be embedded. Hereinafter, a case where a marker is embedded in the Cb component will be described.

FIG. 2 is a diagram illustrating an image example in which the marker embedding device according to the present embodiment superimposes a marker on an image. As shown in the figure, the marker embedding device embeds four markers in the image along each of the right side and the left side defining a predetermined digital watermark embedding region (rectangular region) on the image ( Superimpose). At this time, the marker embedding device places markers on both ends of the right side and both ends of the left side. That is, the marker embedding device embeds a marker at a position including the four corners of the digital watermark embedding area. Further, the marker embedding apparatus places the marker at a position where the center point of the marker to be embedded along the left side is on the same straight line and the ratio of the distances between the center points of the adjacent markers is 1: x: y. Deploy. The marker embedding device also arranges the markers to be embedded along the right side.
Thus, a rectangular area surrounded by eight markers on the image is an area in which the digital watermark is embedded. That is, a rectangular area sandwiched between two line segments in which four markers are arranged is an area in which a digital watermark is embedded.

  Hereinafter, the marker (FIG. 1) used in this embodiment will be further described. The marker needs to be able to detect the marker stably even if the marker shape changes due to geometric transformation such as projective transformation. Therefore, a cross ratio which is a projective transformation invariant is used. Specifically, as shown in FIG. 1, four points (point A, point B, point C, and point D) are defined on a straight line L passing through the center of the marker (the center point of a concentric circle). These four points (point A, point B, point C, and point D) are detected using a Box filter (box filter). Here, the position of each point on the marker is determined so that the four points (point A, point B, point C, and point D) satisfy the following expression (1).

In Formula (1), n is an arbitrary real number. When the expression (1) is satisfied, the cross ratio calculated from the positions of the four points (point A, point B, point C, and point D) is expressed by the following expression (2). This cross ratio is a constant value regardless of the projective transformation. For example, when n = 3, the cross ratio is ((3 × 5) / (4 ² ) = 15/16).

  Here, a Box filter for detecting a marker having a striped pattern as shown in FIG. 1 will be described. FIG. 3 is a diagram illustrating an example of a Box filter used by the marker detection device in the present embodiment. In the figure, the horizontal direction is the X direction and the vertical direction is the Y direction. The Box filter shown in the figure is a 9 pixel × 9 pixel Box filter, and “0”, “−1”, and “+2” are weighting factors for each pixel (region) divided by hatching. It is. That is, in the pixel of ((x, y); 1 ≦ x ≦ 9, 1 ≦ y ≦ 3) and ((x, y); 1 ≦ x ≦ 9, 7 ≦ y ≦ 9), the Cb component And multiply by 0. In the pixel of ((x, y); 1 ≦ x ≦ 3, 4 ≦ y ≦ 6) and ((x, y); 7 ≦ x ≦ 9, 4 ≦ y ≦ 6), the Cb component Is weighted by “−1”. In the pixel of ((x, y); 4 ≦ x ≦ 6, 4 ≦ y ≦ 6), the Cb component is weighted by “+2”.

  When a marker is searched by raster scanning for each pixel of the image using this Box filter, for example, a stripe of “+2” including a point A on the marker is ((x, y); 4 ≦ x ≦ 6, 4 ≦ y ≦ 6), and the fringe of “−1” on the outer peripheral side in contact with the stripe including the point A on the marker is ((x, y); 1 ≦ x ≦ 3, 4 ≦ y ≦ 6) Is located at ((x, y); 7 ≦ x ≦ 9, 4 ≦ y ≦ 6). The value calculated using the Box filter is larger than the surrounding pixels. When detecting a marker, a point (pixel) where the calculated value is larger than the surroundings is detected, and a cross ratio is calculated from four points adjacent to the raster scan direction among the detected points (pixels). calculate. When the calculated cross ratio matches the marker cross ratio or the difference from the marker cross ratio is within a predetermined range, the four points corresponding to the calculated cross ratio are four points (point A on the marker). , Point B, point C, and point D), the marker is detected.

  The extreme value of the value calculated sequentially based on the Cb component (the largest value or the smallest value in the vicinity) that is obtained sequentially by raster scanning the captured image using the Box filter as shown in FIG. ) Is detected. Points (pixels) corresponding to the detected extreme values and four extreme values continuous in the scanning direction are defined as a point A ′, a point B ′, a point C ′, and a point D ′. Then, the cross ratio is calculated by the following equation (3) from four points (point A ′, point B ′, point C ′, and point D ′) corresponding to the detected extreme value.

  When the cross ratio calculated using the equation (3) is equal to the cross ratio in the marker embedded in the image in advance, the four points (point A ′, point B ′, point C ′, and point D ′) are The point on the straight line passing through the center point of the marker and corresponding to the characteristic shading pattern of the marker. In this way, candidate points of the four feature points (point A, point B, point C, and point D) on the marker are detected from the photographed image using the Box filter, and the cross ratio of the detected candidate points is If it is equal to the cross ratio, the candidate point is determined to be four feature points (point A, point B, point C, and point D) on the marker. This makes it possible to detect a marker embedded in the original image in the captured image and specify its position.

  Further, the marker detection device selects four markers from the markers detected using the Box filter described above. The marker detection device determines whether or not the center point of the selected marker is located on one straight line, and the ratio of the distances between the center points of the markers is 1: x: y. The marker detection device detects two sets of four markers in which the center points are located on a straight line and the distance ratio between the center points satisfies 1: x: y. The marker detection apparatus performs inverse transformation (reverse projection transformation) of the projection transformation performed on the original image based on the detected positions of the two sets of markers (eight markers).

  Hereinafter, specific configuration examples of the marker embedding device and the marker detection device in the present embodiment will be described. FIG. 4 is a block diagram showing a configuration of the marker embedding device 1 in the present embodiment. The marker embedding device 1 has the above-described features at positions including four corners of a predetermined rectangular area on image content or video content with respect to input image content (still image) or video content (moving image). Embed a marker. Further, the marker embedding device 1 embeds an ID code between the markers and embeds a predetermined digital watermark in the rectangular area. The ID code (identification code) includes information for identifying a set of markers whose center points are located on a straight line. The marker embedding device 1 includes an image input unit 11, a marker superimposing unit 12, a line segment ID code superimposing unit 13, a digital watermark superimposing unit 14, and an embedded image output unit 15, as shown in FIG.

  The image input unit 11 inputs image content or video content prepared in advance. When video content is input, each frame image continuous in time series of video is input one by one. The input image content or video content is input as digital data.

  The marker superimposing unit 12 superimposes markers with appropriate sizes on eight positions including four corners of a predetermined rectangular area with respect to each frame image of the image or video input to the image input unit 11. The marker that the marker superimposing unit 12 superimposes on an image or a frame image is a marker having the shading pattern shown in FIG. Further, the size of the marker may be a predetermined size or may be determined according to the size of the digital watermark superimposed on the rectangular area. For example, as shown in FIG. 2, the marker superimposing unit 12 has four markers along the left edge of the image, the center points of which are located on a straight line, and the ratio of the distances between the center points is 1: x: y. A marker is superimposed on. Similarly, the marker superimposing unit 12 superimposes four markers along the right end. That is, the marker superimposing unit 12 superimposes one set of two markers on a rectangular area for each image or video frame image.

  The marker embedded in each frame image of the image or video by the marker superimposing unit 12 is a concentric pattern, and there are four characteristic shade patterns common to any straight line passing through the center of the concentric circle pattern. It has two configurations to do. Regarding the latter configuration, for example, in the marker shown in FIG. 1, there is a change in shading as shown in FIG. 5 around point A, point B, point C, and point D. FIG. 5 is a diagram showing an example of a characteristic shading pattern possessed by the marker used in the present embodiment. In the figure, the horizontal axis corresponds to a straight line passing through the center of the marker, and the vertical axis represents the light and shade when the center of the marker is used as a reference on the straight line. The marker shown in FIG. 1 has four characteristic shade patterns (point A, point B, point C, and point D) shown in FIG. For example, the point A is (−1, +2, −2), the point B is (−2, +2, −1), and the lightness is positive (+) with respect to the point (or region) of interest. Is given, and negative (-) shades are given to points (or regions) adjacent to the point (or region) of interest. The marker shown in FIG. 1 is an example, and may be a different shape from the marker shown in FIG. 1 as long as it is a marker composed of the light and shade pattern having the two configurations described above.

  Specifically, the marker superimposing unit 12 converts the input image from the RGB color space signal to the YCbCr color space signal as a superposition method for superimposing the marker on the image. The marker superimposing unit 12 superimposes markers at eight positions including four corners of a predetermined rectangular area on the image, and adds the gray value of the marker to the Cb component at the position (pixel) where the marker is superimposed. When the marker shown in FIG. 1 is used, the numerical value shown in FIG. 1 is a value added to the Cb component. Here, addition is performed when the value is a positive value (plus), and subtraction is performed when the value is a negative value (minus). In addition, in order to increase the intensity of light and shade by marker superimposition, a fixed value a set in advance as a parameter may be multiplied as a coefficient to the light and dark value of the marker and then added to the Cb component of the image. That is, a value obtained by multiplying the gray value of the marker by a fixed value a may be added to the Cb component of the image. In addition, as a geometrical transformation for image content and video content, when rotation of ± 45 degrees or more is applied, the four corners of the rectangular area are set so that the correspondence of the four corner markers can be correctly taken when detecting the markers. Invert the sign of one of the markers to be placed in to add. Hereinafter, as in the marker superposition example shown in FIG. 2, a case will be described in which superpositions obtained by reversing the sign of the lower right marker are applied.

  The line segment ID code superimposing unit 13 further superimposes an ID code on each frame image of an image or video between markers whose ratio of the distance between the center points superimposed by the marker superimposing unit 12 is 1: x: y. To do. The ID code includes information for identifying the marker. The ID code is, for example, a binary value composed of “1” and “0”. The line segment ID code superimposing unit 13 adds a predetermined value to the value of the Cb component in the region (pixel) corresponding to “1”, and sets the predetermined value from the value of the Cb component in the region corresponding to “0”. By subtracting, the ID code is superimposed on each frame image of the image or video. At this time, the ID code is superimposed on a line segment connecting the center points of the markers whose ratio of the distance between the center points is 1: x: y.

  In addition, reading of the ID code is performed, for example, by calculating the difference from the area adjacent to the direction in which the ID code is arranged for each area (pixel) where the ID code is superimposed, and based on the calculated difference value. This is done by determining the superimposed value (“0” or “1”). In addition, the same ID code is used between the two sets of markers to indicate correspondence between the markers. Further, it is desirable to use different ID codes for different images and videos. In particular, when it is known in advance that a plurality of images or videos are arranged at spatially close positions and are displayed, an ID code different from the neighboring images and videos is assigned.

  FIG. 6 is a diagram illustrating an image example in which a marker and an ID code are superimposed by the marker embedding device 1 according to the present embodiment. Here, the markers are arranged along the right side and the left side of the rectangular area indicating the digital watermark embedding area, and are arranged so that the ratio of the center point distances is 1: x: y. Among the markers, two ID codes are superimposed between markers corresponding to a distance ratio of 1 and between markers corresponding to a distance ratio of y. The two ID codes superimposed between the set of markers arranged at the right end and the two ID codes superimposed between the set of markers arranged at the left end are the same ID code as described above. It has become. In addition, although the figure shows an example in which two ID codes are arranged between the markers, one ID code may be arranged between the markers, or three or more ID codes may be arranged. .

  The digital watermark superimposing unit 14 superimposes digital watermark information as information about the image in a rectangular area surrounded by the eight markers superimposed on the image by the marker superimposing unit 12. The superimposition method follows a digital watermark method to be superimposed. Various schemes such as the scheme disclosed in Patent Document 1 have been proposed for the digital watermark scheme, and any scheme may be employed. In addition, the digital watermark information superimposed by the digital watermark superimposing unit 14 may not be resistant to geometric transformation itself.

  The embedded image output unit 15 converts the image processed in the marker superimposing unit 12, the line segment ID code superimposing unit 13, and the digital watermark superimposing unit 14 into an “embedding” in which a marker, an ID code, and a digital watermark are superimposed. Output as a “completed image”. When video content is input to the image input unit 11, the embedded image is output as a single video content by connecting it to a series of video sequences as a frame. The output destination of the embedded image output unit 15 is, for example, an HDD, an electronic recording medium such as a DVD-ROM, or a display device. When the output destination is an electronic recording medium, the embedded image is recorded on the electronic recording medium, and when the output destination is a display device, the embedded image is displayed on the display device.

FIG. 7 is a flowchart showing marker embedding processing performed by the marker embedding device 1 according to the present embodiment. When the marker embedding process is started in the marker embedding device 1, an image (or each frame image of a video) as an object to be embedded with the marker and the digital watermark is input to the image input unit 11 (step S11).
The marker superimposing unit 12 has a center point on the same straight line in a predetermined rectangular area as shown in FIG. 2 with respect to the Cb component of the image input to the image input unit 11, and an interval between the center points. Are superimposed on the left end and the right end at a position satisfying a predetermined cross ratio (1: x: y). The marker superimposing unit 12 outputs an image in which eight markers are superimposed to the line segment ID code superimposing unit 13 (step S12).

The line segment ID code superimposing unit 13 superimposes the ID code between the markers in the image output from the marker superimposing unit 12, and outputs the image on which the ID code is superimposed to the digital watermark superimposing unit 14 (step S13).
The digital watermark superimposing unit 14 superimposes the digital watermark information on the rectangular region surrounded by the eight markers superimposed on the image in the image output from the line segment ID code superimposing unit 13 and superimposes the digital watermark information. The processed image is output to the embedded image output unit 15 (step S14).

The embedded image output unit 15 outputs the image input from the digital watermark superimposing unit 14 to an external device (step S15).
The embedded image output unit 15 determines whether or not the process of superimposing the marker, the ID code, and the digital watermark information is completed for all the images (step S16), and the process is performed for all the images. If completed (step S16: YES), the marker embedding process is terminated. On the other hand, when the processing has not been completed for all the images (step S16: NO), the embedded image output unit 15 returns the processing to step S11 and repeats the processing from step S11 to step S15.

  As described above, in the marker embedding device 1, the image content or video content input to the image input unit 11 includes the marker superimposing unit 12, the line segment ID code superimposing unit 13, the digital watermark superimposing unit 14, and the embedded image. A series of processing is performed in the output unit 15. As a result, the image content or video content includes four markers (eight in total) on the left side and the right side in a predetermined rectangular area in the content, an ID code between the markers, and an electronic code. The watermark information is superimposed.

  FIG. 8 is a block diagram showing a configuration of the marker detection device 2 in the present embodiment. The marker detection device 2 detects a predetermined marker for the input image content or video content. The marker detection device 2 performs determination using a cross ratio and an ID code for the detected marker. The marker detection device 2 detects digital watermark information superimposed on image content or video content based on a marker that satisfies the determination, and outputs the detected digital watermark information. As shown in the figure, the marker detection device 2 includes an image input unit 21, a marker detection unit 22, a line segment ID code detection unit 23, a projective transformation correction unit 24, a digital watermark detection unit 25, and a detection result output unit 26. It has.

  The image input unit 21 processes image data or video content in which a digital watermark to be detected is captured by a camera or the like, and image content or video content in which the digital watermark to be detected is embedded (for example, , Projective transformation) is input. When the embedded digital watermark requires a plurality of images that are continuous in time series at the time of detection, the required number of images are input to the image input unit 21 in time series in a time series. The

  The marker detection unit 22 identifies the position where the marker is superimposed on the image input to the image input unit 21. A specific method will be described in the case where the marker embedded in the image is the marker shown in FIG. The marker detection unit 22 converts the input image from the RGB color space signal to the YCbCr color space signal, and extracts the Cb component of the image. The marker detection unit 22 performs a raster scan from the upper left to the lower right by repeatedly scanning the Cb component image in the horizontal direction for each of a plurality of Box filters prepared in advance. Use a filter. Here, a case will be described in which three different size Box filters are prepared in advance as shown in FIG.

  FIG. 9 is a diagram showing an outline of a Box filter used by the marker detection unit 22 in the present embodiment and having different sizes. As the Box filter, for example, a Box filter of 3 pixels × 3 pixels, 5 pixels × 5 pixels, 7 pixels × 7 pixels, and 15 pixels × 15 pixels is prepared in advance. Note that the size of each Box filter and the number of Box filters may be determined in advance according to the target content and the size of the marker superimposed on the content. The marker detection unit 22 calculates a value for each pixel in a raster scan using each Box filter.

  Here, on the three-dimensional scale space composed of the scan direction, the direction perpendicular to the scan direction, and the size direction of the Box filter, an area adjacent to each direction is defined as 26 neighborhoods. The marker detection unit 22 compares the calculated value of the current scan position (pixel) with the calculated value near 26, and detects the calculated value of the scan position as a maximum value when the calculated value of the scan position becomes the maximum value. . The marker detection unit 22 calculates a cross ratio by using points corresponding to the four maximum values continuous in the scanning direction as point A ′, point B ′, point C ′, and point D ′ using Expression (3). . The marker detection unit 22 compares the cross ratio calculated using Expression (3) with the cross ratio calculated using Expression (2) determined for the marker. It determines with existing in the position defined by (point A ', point B', point C ', and point D'), and memorize | stores the center point of a marker. The marker detection unit 22 performs processing for detecting the center point of the marker using the Box filter on the entire image. Note that the local maximum value may be detected by comparing the calculated values in the vicinity of 8 on the two-dimensional scale space formed by either the scanning direction or the direction perpendicular to the scanning direction and the size direction of the Box filter.

  Moreover, the marker detection part 22 is performed as follows about the marker corresponding to the lower right of a rectangular area. Note that the marker corresponding to the lower right corner of the rectangular area has a tone pattern opposite to that of the other three markers, so the value calculated by the Box filter described above is near the feature point of the marker. Smaller than this pixel. The marker detection unit 22 compares the calculated value of the current scan position (pixel) with the calculated value near 26, and detects the calculated value of the scan position as a minimum value when the calculated value of the scan position is minimized. . The marker detection unit 22 uses points (3) as points A ′, B ′, C ′, and D ′ as points A ′, B ′, C ′, and D ′ that correspond to the four consecutive minimum values (minimum values) in the scanning direction. Calculate the ratio. The marker detection unit 22 compares the cross ratio calculated using Expression (3) with the cross ratio calculated using Expression (2) determined for the marker. It determines with existing in the position defined by (point A ', point B', point C ', and point D'), and memorize | stores the center point of a marker.

  The line segment ID code detection unit 23 searches all the sets of four markers located on the same straight line with respect to the marker (center point) detected by the marker detection unit 22. The line segment ID code detection unit 23 calculates the cross ratio of the distances between the center points for each set of four detected markers, and the calculated cross ratio is a predetermined cross ratio (1: x: y). It is determined whether or not. When the calculated cross ratio matches the following expression (4), it is determined that the set of four markers is a set of markers arranged on the left side or the right side of the rectangular area where the digital watermark information is superimposed. When the calculated cross ratio is not (1: x: y), it is determined that the set of four markers is not a set of markers indicating a rectangular area on which digital watermark information is superimposed.

In this determination, when the difference between the calculated cross ratio and the value of Expression (4) is within a predetermined range, it may be determined as a set of markers indicating a rectangular area on which digital watermark information is superimposed.
The line segment ID code detection unit 23 reads the ID code superimposed by the line segment ID code superimposing unit 13 between the markers based on the positional relationship of the four markers in each of the four marker sets indicating the rectangular area. At this time, the line segment ID code detection unit 23 uses the fact that the position coordinates of the superimposed marker are known to read back the position of the ID code embedded on the straight line. The line segment ID code detection unit 23 determines that two sets having the same ID code read for each set of four markers are markers indicating the left end and the right end of the area where the digital watermark information is superimposed. If there are two other sets that match with an ID code different from the above two sets, it is determined that a plurality of pieces of digital watermark information are superimposed on the image.

  The line segment ID code detection unit 23 determines that the area in the quadrilateral obtained by connecting both ends of each line segment indicated by the two sets of detected markers is a partial area corresponding to the rectangular area on which the digital watermark information is superimposed. . Moreover, since the marker located in the lower right among the markers indicating the rectangular area has already been identified in the marker detection unit 22, the line segment ID code detection unit 23 identifies the markers located in the other three corners. . That is, the line segment ID code detection unit 23 specifies markers arranged at the four corners among markers indicating the partial area corresponding to the rectangular area.

FIG. 10 is a diagram showing an outline of processing in the line segment ID code detection unit 23. A set of four markers that are located on the same straight line and satisfy a predetermined cross ratio (formula (4)) with respect to the center point of the marker detected by the marker detection unit 22 as one line segment. To summarize. At this time, the cross ratio is a cross ratio of the distance between the center points of the markers embedded in the marker embedding apparatus 1.
The line segment ID code detection unit 23 reads the ID code arranged on the line segment, connects two ends of the two line segments having the matching ID code as a pair, and detects a quadrangle. The line segment ID code detection unit 23 identifies the detected quadrilateral as a partial region in which digital watermark information is embedded.
In this way, the line segment ID code detection unit 23 can specify all the positions of the eight markers arranged so as to surround the rectangular area in which the digital watermark information is embedded.

  Note that the rectangular area in which the digital watermark information is embedded may be determined so as not to include the area where the marker is superimposed, as shown in FIG. FIG. 11 is a diagram illustrating different examples of areas in which digital watermark information is embedded (superimposed) in the present embodiment. In this case, the marker embedding device 1 superimposes digital watermark information on a rectangular area that does not overlap with the marker. In this case, the marker embedding device 1 embeds the marker in the image along each of the left side and the right side of the rectangular area.

  Note that, even if the embedded marker has a shape other than that shown in FIG. 1, a filter that can extract the “common characteristic grayscale pattern” that is the marker configuration in the present embodiment is used. Thus, the marker position can be detected in the same manner. Further, when only 7 or less marker positions can be detected, the marker detection unit 22 determines that the digital watermark is not embedded or the marker detection has failed, and does not perform the subsequent processing. In addition, when the line ID code detection unit 23 cannot detect two sets (a group at the left end and a group at the right end) that match the ID code, the digital watermark is not embedded or the marker detection fails. It is determined that it has been performed, and the subsequent processing is not performed.

  The projective transformation correction unit 24 performs reverse projection transformation on the input image based on the markers (center points) located at the vertices of the quadrilateral identified by the line segment ID code detection unit 23. At this time, the projective transformation correction unit 24 performs reverse projection transformation so that the positions of the markers corresponding to the vertices of the quadrilateral are the positions of the four corners of the rectangular region in which the marker embedding device 1 has embedded the marker. At this time, the projective transformation correction unit 24 performs reverse projective transformation by applying the same technique as that described in Patent Document 2, for example.

  The digital watermark detection unit 25 is an electronic device in which the digital watermark superimposing unit 14 of the marker embedding device 1 superimposes the image on the image from the rectangular areas where the markers on the image obtained by the reverse projection conversion by the projection conversion correction unit 24 are arranged at the four corners. Detect watermark information. At this time, when the embedded digital watermark information requires a plurality of images in time series, the marker detection unit 22, the line segment ID code detection unit 23, and the projective transformation correction unit 24 require the required number of images. The digital watermark is detected after obtaining the necessary number of images corrected by the reverse projection transformation. The digital watermark detection unit 25 outputs the detected digital watermark information to the detection result output unit 26.

  The detection result output unit 26 outputs information output from the digital watermark detection unit 25, that is, digital watermark information indicated by the digital watermark, to the outside as digital data.

FIG. 12 is a flowchart showing marker detection processing performed by the marker detection device 2 according to the present embodiment. When the marker detection process is started in the marker detection device 2, an image (or each frame image of a video) as a target for detecting the marker and digital watermark information is input to the image input unit 21 (step S21).
The marker detection unit 22 detects the center point of the marker using the Box filter in the Cb component image of the image input to the image input unit 21 (step S22).

The line segment ID code detection unit 23 detects a line segment ID code based on the marker detected by the marker detection unit 22 and identifies an area in which digital watermark information is superimposed from a pair of line segments formed by the four markers. (Step S23).
The line segment ID code detection unit 23 determines whether or not the area on which the digital watermark information is superimposed has been detected (step S24). If the area has not been detected (step S24: NO), the marker detection process is terminated. .

On the other hand, if it can be detected (step S24: YES), the projective transformation correction unit 24 matches the region specified by the line segment ID code detection unit 23 with a rectangular region having a predetermined size for the input image. The inverse projection transformation is performed (step S25).
When the line segment ID code detection unit 23 detects a plurality of regions on which digital watermark information is superimposed, the projective transformation correction unit 24 performs reverse projection transformation for each detected region.

The digital watermark detection unit 25 detects digital watermark information from the rectangular area indicated by the marker in the image obtained by the reverse projection conversion by the projection conversion correction unit 24 (step S26).
When the line segment ID code detection unit 23 detects a plurality of areas where the digital watermark information is superimposed, the digital watermark detection unit 25 detects the digital watermark information for each detected area.

The digital watermark detection unit 25 determines whether or not the detected digital watermark information is correct (step S27). If the digital watermark information is not correct (step S27: NO), the process returns to step S21, and the process from step S21 to step S27 is performed. Let the process repeat.
On the other hand, when the detected digital watermark information is correct (step S27: YES), the digital watermark detection unit 25 outputs the detected digital watermark information to the detection result output unit 26. The digital watermark information output from the detection result output unit 26 and the digital watermark detection unit 25 is output to the outside (step S28), and the marker detection process is terminated.

As described above, in the marker detection device 2, the image content or video content input to the image input unit 21 includes the marker detection unit 22, the line segment ID code detection unit 23, the projective transformation correction unit 24, and the digital watermark detection unit 25. The detection result output unit 26 performs a series of processes. Thereby, the marker superimposed on the image content or the video content is detected, the ID code arranged between the detected markers is detected, and based on the cross ratio of the distance between the markers (center points) and the ID code By identifying a region surrounded by the identified marker and performing reverse projection transformation on the region, even digital watermark information that is not resistant to projection transformation can be acquired.
At this time, the line segment ID code detection unit 23 is positioned on the same straight line with respect to the marker detected by the marker detection unit 22, and the ratio of the distance between the markers (center points) is determined in advance. It is determined whether or not it is a marker that identifies a region where digital watermark information is superimposed, on the condition that the ratio matches the ratio (equation (4)). Thereby, even when the marker detection unit 22 detects the marker by mistake, it is possible to detect the area where the digital watermark information is superimposed while suppressing the influence of the marker, and the digital watermark detection unit 25 can detect the digital watermark information. The accuracy of detecting can be improved.

By using the marker embedding device 1 and the marker detection device 2 described above, a marker capable of detecting a cross ratio is embedded in a Cb component that is most difficult to be perceived by humans in visual characteristics, and the marker is detected by a Box filter. It becomes possible.
By defining four common characteristic grayscale patterns on a straight line passing through the center of the marker, the characteristic grayscale patterns can be detected by filtering. Also, the cross ratio determined by the four points on the straight line passing through the center of the marker is the same as the cross ratio calculated from the four points in the marker after the projective transformation, even if the marker is deformed by the projective transformation. To do. By detecting the marker using this cross ratio, it is possible to detect the marker from the image subjected to the projective transformation.

In the filter processing, since the raster scan using the Box filter corresponding to the characteristic gray pattern of the marker is performed, the calculation is performed when the characteristic gray pattern area overlaps with the Box filter area. The value obtained is larger than the neighboring value. Thereby, specification of the position of a marker can be made easy. By performing reverse projection conversion based on the identified marker position, it is possible to restore the image content and video content embedded with information such as digital watermarks to the state before geometric conversion.
As a result, reverse projection conversion is applied to images that have undergone projective transformation that occurs when image content or video content embedded with information such as digital watermarks is captured by a camera, etc. Information can be acquired.

  In addition, with respect to the marker detected from the image, whether the center point of the marker is on a straight line, whether the cross ratio of the distance between the center points of the marker satisfies a predetermined cross ratio (Equation (4)), Whether or not it is a marker indicating a rectangular area in which digital watermark information is embedded is determined using a condition that the embedded ID code corresponds. Thereby, the detection accuracy of the marker which shows a rectangular area can be improved. In addition, since a marker indicating a rectangular area can be identified by an ID code, even when a plurality of digital watermarks are embedded in an image, the rectangular area in which each digital watermark is embedded can be detected individually. it can.

  In the present embodiment, four markers are arranged so that the distance between the center points satisfies a predetermined cross ratio not only along the four corners of the rectangular area but also along one side of the rectangular area. Thereby, the distortion which arose by projective transformation with respect to ID code arrange | positioned between markers can be specified, and the precision which reads ID code is improved. As a result, even when a plurality of digital watermarks are embedded in an image, each rectangular region in which each digital watermark is embedded can be detected with high accuracy.

  In the present embodiment, as shown in FIG. 2, a marker is superimposed at a position including the four corners of an image to be embedded with digital watermark information, and most of the region of the image is handled as a region on which digital watermark information is superimposed. Was illustrated. However, the present invention is not limited to this, and a specific rectangular area in the target image may be used as an area for superimposing digital watermark information. In this case, markers may be superimposed at eight locations including four corners in a specific rectangular area in the image. For example, assuming that two persons appear in the target image, it is also possible to superimpose different digital watermark information by superimposing a marker on each rectangular area corresponding to the face of each person. By doing so, it becomes possible to detect information corresponding to each person of one piece of image content. Furthermore, the user may be guided to other content (for example, a Web information site) based on the detected information.

  Further, in the present embodiment, the case where the area surrounded by the marker is treated as the area on which the digital watermark information is superimposed is illustrated. However, for example, as shown in FIG. 13, a part of the rectangular area specified by the marker May be an area where digital watermark information is superimposed. FIG. 13 is a diagram illustrating an example of a region in which digital watermark information is superimposed in the present embodiment. In the example shown in the figure, the range of the center 50% in the rectangular area specified by the eight markers is used as an area for superimposing digital watermark information. Note that the digital watermark information may be superimposed by designating an arbitrary area within the area specified by the marker.

  Further, the ID code superimposed by the line segment ID code superimposing unit 13 may be provided with redundancy. For example, an ID code in which a predetermined pattern is repeated is used. When the line segment ID code detection unit 23 detects an error in the ID code (for example, when a repetition of a predetermined pattern is not detected), the line marker ID code detection unit 23 excludes four markers (center points) corresponding to the ID code from the processing target. You may make it do. Thereby, it is possible to improve the accuracy of detecting the region where the digital watermark information is superimposed.

Further, a code indicating the vertical direction may be embedded in a predetermined bit of the ID code. For example, in the ID code, the bit code placed at the top position is set to “1” and the bit code placed at the bottom position is set to “0”, so that the ID code is read and The direction of the direction can be specified. In this case, it is possible to specify all the markers at the four corners without arranging a marker in which the sign of the shading is inverted at the lower right corner or the like.
Moreover, although the case where the set of markers is identified using the same ID code for the ID code superimposed between the leftmost marker and the ID code superimposed between the rightmost marker has been illustrated, Any ID code can be used as long as the correspondence can be detected. For example, an XOR (exclusive OR) operation is performed for each bit on the left end ID code and the right end ID code, and when all the bits are “1” as the operation result, the corresponding ID code (marker You may make it determine with it being a group.

  Further, in the present embodiment, the configuration in which the marker detection device 2 detects the digital watermark information after the reverse projection conversion has been described, but information other than the digital watermark information may be detected. For example, in a similar application using image recognition (for example, an application that provides information for accessing a Web information site related to an object photographed by a camera) in addition to the purpose of specifying an embedded area of digital watermark information. It can be used as preprocessing for image recognition. In this case, the marker detection device 2 may include an information output unit that outputs information indicating the position in the image of the partial area specified by the line segment ID code detection unit 23 based on the marker and the ID code. However, the object including the area where the information is embedded is a planar object, and is limited to the case where the marker can be embedded in time by printing or the like. Of course, when such use is assumed, the digital watermark superimposing unit 14 in the marker embedding device 1 and the projective transformation correcting unit 24 and the digital watermark detecting unit 25 in the marker detecting device 2 are unnecessary, and the detection result output unit 26 is Alternatively, the position information of the marker on the embedded image may be output. The recognition module can improve recognition accuracy by cutting out an object and correcting geometric transformation based on the output marker position information. Further, the recognition accuracy may be improved by calculating the relative spatial positional relationship between the object and the camera based on the marker position information and using it together with the recognition result.

In this embodiment, the case where the cross ratios of the four markers that specify the partial areas corresponding to the rectangular areas are the same has been described. However, the cross ratios of the four markers may be different. In this case, even when a rotation of ± 45 degrees or more is applied, the original position of each marker can be grasped, so that it is not necessary to invert the density of one marker as shown in FIG.
Further, in the present embodiment, the case where the markers are arranged at the left end and the right end including the four corners of the area where the digital watermark information is superimposed is illustrated, but the markers are arranged at the upper end and the lower end including the four corners of the area where the digital watermark information is superimposed. May be arranged.
Moreover, although the case where four markers are arranged along one side of the rectangular area has been illustrated in the present embodiment, five or more markers may be arranged along one side of the rectangular area.

  Further, in the raster scan using the Box filter described in the above embodiment, by calculating an integral image of the target image before performing the raster scan, the calculation amount by filtering is greatly increased. It can be reduced and the processing speed can be increased (reference: Yasushi Yagi, Hideo Saito, “Computer Vision Advanced Guide 2”, Adcom Media Corporation, p. 46, June 2010). Specifically, an integrated image calculation unit that calculates a total value of Cb components for each region multiplied by each weighting coefficient from the integrated image is provided between the image input unit 21 and the marker detection unit 22. The marker detection unit 22 calculates the sum of values obtained by multiplying the total value of the Cb components of each region obtained from the integrated image by a weighting coefficient. As a result, the amount of computation that increases in accordance with the size of the Box filter can be suppressed, and the amount of computation required for marker detection processing can be reduced.

  You may make it implement | achieve the marker embedding apparatus 1 (FIG. 4) and the marker detection apparatus 2 (FIG. 8) in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” is a program that dynamically holds a program for a short time, like a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The present invention can also be applied to a use in which reverse projection transformation for returning an image obtained by performing projection transformation to an original image is indispensable.

DESCRIPTION OF SYMBOLS 1 ... Marker embedding apparatus 2 ... Marker detection apparatus 11, 21 ... Image input part 12 ... Marker superimposition part 13 ... Line segment ID code superimposition part (identification code superimposition part)
14 ... Digital watermark superimposing unit (information superimposing unit)
DESCRIPTION OF SYMBOLS 15 ... Embedded image output part 22 ... Marker detection part 23 ... Line segment ID code detection part (area | region specific part)
24 ... Projection conversion correction unit 25 ... Digital watermark detection unit (information detection unit)
26. Detection result output unit

Claims (21)

An image input unit for inputting one or more continuous images;
For the color component that is difficult to perceive in the image, at least four markers having a concentric stripe pattern of the color component and four common gray patterns on a straight line passing through the center point of the concentric circle. A first line defining a predetermined rectangular area on the image is superimposed on a straight line at a position where a distance between the center points of the markers forms a predetermined cross ratio, and at least four other markers are A marker superimposing unit that superimposes on a straight line at a position where the distance between the center points of the markers forms a predetermined cross ratio along a second side different from the first side defining a rectangular region;
For the color component of the image, a first identification code is superimposed between the markers superimposed along the first side, and the marker is superimposed along the second side. An identification code superimposing unit for superimposing a second identification code corresponding to the first identification code;
A marker embedding device comprising: an image output unit that outputs the image in which the marker, the first identification code, and the second identification code are superimposed on the color component. The marker embedding device according to claim 1,
The marker embedding device, further comprising: an information superimposing unit that superimposes information on the image on the rectangular area. In the marker embedding device according to claim 1 or 2,
The identification code superimposing unit is
The marker embedding device, wherein the same identification code is used for the first identification code and the second identification code. In the marker embedding device according to any one of claims 1 to 3,
The identification code superimposing unit is
The marker embedding device, wherein an identification code capable of identifying the vertical direction of the image with respect to the first identification code and the second identification code is used. An image input unit for inputting one or more continuous images;
When four points where a predetermined shading pattern exists in the color component that is difficult to perceive in the image are detected and the cross ratio calculated from the detected four points is a predetermined value, a concentric striped pattern A marker detection unit that determines that markers having four common shade patterns are located on a straight line passing through the center point of the concentric circles,
From the markers detected by the marker detection unit, specify at least two marker pairs in which the center points are aligned on a straight line and the ratio of the distances between the center points has a predetermined cross ratio. An area specifying unit for specifying a partial area in the image, based on a pair of marker pairs to which the identification code superimposed in the color component between the markers corresponds;
A marker detection device, comprising: a projective transformation correction unit that performs projective transformation in which the partial region specified by the region specifying unit is a predetermined rectangular region. In the marker detection apparatus according to claim 5,
A marker detection apparatus, further comprising: an information detection unit that detects information related to the image from the partial region that is projectively transformed into a rectangular region by the projective transformation correction unit. In the marker detection apparatus according to claim 5 or 6,
The marker detection unit
A raster filter is sequentially applied to each pixel of the color component of the image using a plurality of size box filters, which are box filters determined according to the shading pattern of the marker, A pixel corresponding to a maximum value or a minimum value in the vicinity is detected, and the marker is detected by a cross ratio calculated based on a positional relationship of four pixels that are consecutive in the raster scan direction among the detected pixels. Marker detection device. In the marker detection apparatus according to any one of claims 5 to 7,
Further comprising an integral image calculation unit for calculating an integral image in the color component of the image;
The marker detection unit
A marker detection device that detects a position of the marker in the image based on the integrated image. In the marker detection apparatus according to any one of claims 5 to 8,
The region specifying unit includes:
A quadrilateral area obtained by connecting a line segment defined by a marker set is identified as the partial area by using a pair of markers with matching identification codes superimposed between the markers of the specified marker set. A marker detection device characterized by the above. In the marker detection apparatus according to any one of claims 5 to 9,
The marker detection apparatus further comprising: an information output unit that outputs information indicating a position of the partial region specified by the region specifying unit in the image. An image input step for inputting one or more continuous images;
For the color component that is difficult to perceive in the image, at least four markers having a concentric stripe pattern of the color component and four common gray patterns on a straight line passing through the center point of the concentric circle. A first line defining a predetermined rectangular area on the image is superimposed on a straight line at a position where a distance between the center points of the markers forms a predetermined cross ratio, and at least four other markers are A marker superimposing step of superimposing on a straight line at a position where the distance between the center points of the markers forms a predetermined cross ratio along a second side different from the first side defining a rectangular region;
For the color component of the image, a first identification code is superimposed between the markers superimposed along the first side, and the marker is superimposed along the second side. An identification code superimposing step of superimposing a second identification code corresponding to the first identification code;
An image output step of outputting the image in which the marker, the first identification code, and the second identification code are superimposed on the color component. The marker embedding method according to claim 11,
An information superimposing step of superimposing information on the image on the rectangular area. In the marker embedding method according to any one of claims 11 and 12,
In the identification code superimposing step,
The marker embedding method, wherein the same identification code is used for the first identification code and the second identification code. The marker embedding method according to any one of claims 11 to 13,
In the identification code superimposing step,
The marker embedding method characterized by using an identification code that can identify the vertical direction of the image with respect to the first identification code and the second identification code. An image input step for inputting one or more continuous images;
When four points where a predetermined shading pattern exists in the color component that is difficult to perceive in the image are detected and the cross ratio calculated from the detected four points is a predetermined value, a concentric striped pattern And a marker detection step for determining that the markers having the four common shade patterns are located on a straight line passing through the center point of the concentric circles,
From the markers detected in the marker detection step, specify at least two marker pairs in which the center points are aligned on a straight line and the ratio of the distances between the center points has a predetermined cross ratio. An area specifying step for specifying a partial area in the image based on a pair of marker pairs corresponding to the identification code superimposed in the color component between the markers;
A marker detection method comprising: a projective transformation correction step for performing a projective transformation in which the partial region identified in the region identifying step is a predetermined rectangular region. The marker detection method according to claim 15, wherein
A marker detection method, further comprising: an information detection step of detecting information related to the image from the partial region that has been projectively transformed into a rectangular region in the projective transformation correction step. The marker detection method according to any one of claims 15 and 16,
In the marker detection step,
A raster filter is sequentially applied to each pixel of the color component of the image using a plurality of size box filters, which are box filters determined according to the shading pattern of the marker, A pixel corresponding to a maximum value or a minimum value in the vicinity is detected, and the marker is detected by a cross ratio calculated based on a positional relationship of four pixels that are consecutive in the raster scan direction among the detected pixels. Marker detection method. The marker detection method according to any one of claims 15 to 17,
An integrated image calculating step of calculating an integrated image in the color component of the image;
In the marker detection step,
A marker detection method, comprising: detecting a position of the marker in the image based on the integrated image. In the marker detection method according to any one of claims 15 to 18,
In the region specifying step,
A quadrilateral area obtained by connecting a line segment defined by a marker set is identified as the partial area by using a pair of markers with matching identification codes superimposed between the markers of the specified marker set. The marker detection method characterized by this. The marker detection method according to any one of claims 15 to 19,
A marker detection method, further comprising: an information output step of outputting information indicating a position of the partial region specified in the region specifying step in the image.   The computer executes each step in the marker embedding method according to any one of claims 11 to 14, or each step in the marker detection method according to any one of claims 15 to 20. Program for.

Images