JP6880867B2 - Image processing equipment, image processing methods and programs - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method and a program.

A method of extracting feature points of a closed figure that forms an area such as a rectangle or a circle, a combination of open figures that do not form an area by itself such as a straight line, or a figure composed of a single body is known.

A feature point is a point that becomes an end point of each straight line when the outline of the target figure is expressed by a plurality of straight lines more simply. One of the merits of extracting feature points is the reduction of the required memory capacity. For example, by extracting feature points from a figure that draws an arc as shown in FIG. 15 (a) and connecting those feature points, the portion that draws the initial arc as shown in FIG. 15 (b) is simplified. It is possible to obtain a modified figure and reduce the required memory capacity.

The DP method (Douglas-Peucker algorithm) (Non-Patent Document 1) is known as a method for further simplifying a figure after extracting feature points. In the DP method, the distance between the straight line drawn from the two feature points of interest and the feature point between the two feature points and the farthest from the straight line is compared with the threshold value, and the distance is equal to or less than the threshold value. This is a method to simplify the figure by recursively performing the process of thinning out the feature points and connecting the remaining feature points. The method of thinning out by the DP method will be explained in detail in a later paragraph.

As an example of application of the DP method to an MFP (Multi Function Printer), a medium such as a business card placed on a platen is scanned by an image reader, the outline of the medium is simplified by the DP method, and then rotation conversion is performed to perform contour. There is a technique to store what is cut out along the line as image data. Normally, when cutting out a rectangle from scanning data read by the MFP, the coordinates of the four corners of the medium included in the scanning data are extracted, rotation conversion is performed on the medium area, and the closed contour line of the medium is read as an image. A general image data format is a cutout shape that matches the x-axis and y-axis when either the main scanning direction or the sub-scanning direction of the reading direction of the device is the x-axis and the other is the y-axis. Cut out into a rectangular shape so that Normally, since a medium such as a business card has a rectangular shape, the coordinates of the vertices located at the four corners are extracted as feature points by the DP method.

If the internal angles of the medium to be extracted are all right angles, the coordinates of the vertices at the four corners of the medium can be extracted by, for example, the Harris algorithm, without using the DP method. However, if the corners of the medium are rounded or bent, the Harris algorithm will extract a plurality of feature points at that location. When a plurality of feature points are extracted at at least one corner, as a result, five or more feature points are extracted in the entire medium, and the region is not recognized as an area to be cut out. The reason why the region where five or more feature points are extracted is not recognized as the region to be cut out is that the angles between adjacent contour lines are not always right angles, so the contour lines that match the x-axis and y-axis cannot be determined. This is because it is not possible to perform rotation conversion with high accuracy.

Therefore, by applying the DP method to a medium with rounded corners or a medium with bent corners, the contour is simplified, and by extracting four feature points that replace the vertices, it is recognized as an area to be cut out. Will be possible.

However, the user's intention when scanning with the MFP is not to scan a simplified version of the medium, but to scan the entire medium so that it does not fall out. When four feature points are extracted as the vertices of a medium from a medium having rounded corners by the DP method, one point on the rounded contour line is extracted as a feature point for the rounded part. To. If even one point on the contour line of the rounded corner is included in the four feature points, the area of the medium protrudes from the area connecting the four feature points and includes the entire original area of the medium. However, the area cannot be extracted.
An object of the present invention is to cut out image data having a shape close to a rectangle as image data of a medium.

In order to solve the above-mentioned problems, the image processing apparatus of the present invention is represented by an edge extraction unit that extracts edge pixels included in the image data obtained by reading a region including a medium, and a set of edge pixels. Among the edges, the contour line extraction unit that extracts the closed contour line corresponding to the contour of the medium, the first feature point extraction unit that extracts the first feature point on the closed contour line, and the first feature on the closed contour line. The contour line division part that divides the closed contour line into multiple contour lines by deleting the edge pixels located around the points, and the second feature point that is the intersection on the extension line of each divided contour line, respectively. A second feature point extraction unit to be extracted and a cutout unit for cutting out a region obtained by connecting each of the second feature points are provided.

According to the present invention, it is possible to cut out image data having a shape close to a rectangle as image data of a medium.

It is a figure which shows an example of the flow of the DP method with respect to the open figure which concerns on this embodiment. It is a figure which shows an example of the flow of the DP method with respect to the closed figure which concerns on this embodiment. It is a figure which shows an example of application example of the DP method to the open figure which concerns on this embodiment. It is a figure which shows an example of the hardware composition of the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the functional structure of the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the functional structure of the feature point extraction part which concerns on this embodiment. It is a figure explaining the feature point which concerns on this embodiment. It is a figure explaining the optimality of the feature point which concerns on this embodiment. It is a figure explaining an example of the method of dividing the closed contour line of the medium which concerns on this embodiment. It is a figure explaining the optimum feature point which concerns on this Embodiment. It is a figure explaining the optimality of the optimum feature point which concerns on this embodiment. It is a figure which shows an example of the functional structure of the post-processing part which concerns on this embodiment. It is a figure which shows an example of the operation screen which concerns on this embodiment. It is a figure which shows an example of a series of processing flows which concerns on this embodiment. It is a figure which shows an example which extracts a feature point and simplifies a figure.

The DP method will be described in detail using a flowchart. The DP method is a process applied to a figure whose feature points on the contour line have been extracted, and the following flow will be described on the assumption that the feature points have already been extracted. Further, the flow described in the present embodiment is an example of thinning out the feature points using the DP method, and the thinning out of the feature points using the DP method is not limited to the flow in the present embodiment.

FIG. 1 is an example of a flow when thinning out feature points by the DP method for an open figure. The open figure in the present embodiment is a figure having two or more end points, and in FIG. 1, a figure having two end points is given as an example. In S101, a straight line connecting both end points of the open figure is drawn. This is because the DP method has the property of recursively processing feature points located between two feature points, so if both end points are not selected, it is more end than the two feature points. This is because the DP method cannot be applied to the feature points located in the part. Since the feature points at both end points of the open figure are not thinned out, they are the feature points after thinning out. In the DP method, the feature points after the thinning are sequentially determined, and the final figure after the thinning is the connection of the feature points after the thinning.

In S102, the feature point located between the line diagrams connecting the two selected feature points and the farthest from the straight line connecting the two feature points is extracted. In S103, the distance between the feature point extracted in S102 and the straight line connecting the two feature points is calculated and compared with a predetermined threshold value. If the distance between the feature points extracted in S102 and the straight line connecting the two feature points is larger than the threshold value, the process proceeds to S104, and the extracted feature points are determined as one of the feature points after thinning out. In S105, one of the feature points newly extracted in S104 and one of the two original feature points is newly selected as two feature points, a straight line connecting them is drawn, and the process returns to S102. At this time, the straight line connecting the original two feature points is deleted. The thinning of the feature points by the DP method depends on the threshold value, and if the threshold value is set large, the thinning becomes easy to be performed, and it becomes easy to obtain a more simplified figure. If the threshold value is set small, thinning is less likely to occur, and it becomes easier to obtain a figure having a shape closer to the original figure. If the distance between the feature point extracted in S102 and the straight line connecting the two feature points is smaller than the threshold value, the process proceeds to S106, and the end flag is given to the section between the two feature points. The feature points located in the section to which the end flag is given are not set as the feature points after the thinning out thereafter, and are determined as the thinned out section when the end flag is given.

After the end flag is given in S106, the process proceeds to S107, and it is determined whether or not the end flag is given to all the sections. When the end flag is given to all the sections (S107, Yes), the flow ends. If there is a section to which the end flag is not given (S107, No), the process proceeds to S108. In S108, a straight line connecting both end points of the section to which the end flag is not given is drawn, and the process returns to S102. The above process is repeated, and the flow is terminated when it is determined in S107 that the end flag is given to all the sections. The feature points and straight lines extracted after the end of the flow become open figures after thinning out by the DP method.

FIG. 2 is an example of a flow when thinning out feature points by the DP method for a closed figure. The closed figure in the present embodiment is a figure having no end points. The basic flow is the same as in the case of the open figure, but differs from the case of the open figure in that the figure is divided at the beginning. In S201, the closed figure is divided into two open figures. As an example of the division method, two points located at a certain distance apart, such as the feature point at the upper left end of the figure and the feature point at the lower right end, are extracted. After dividing into two open figures in S201, one of them is selected and the process proceeds to S202. Since the flow from S202 to S209 is the same as the flow from S101 to S108, a duplicate description will be omitted here.

When the end flag is given to all the sections in S208 (S208, Yes), the process proceeds to S210, and it is determined whether or not the end flag is given to all the open figures. When the end flag is not given to all the open figures (S210, No), the process returns to S202, and the flow of S202 to S209 is applied to the open figures to which the end flag is not given. If the end flag is given to all open figures (S210, Yes), the process proceeds to S211. In S211 a process of connecting the end points when divided in S201 is performed. The feature points and straight lines extracted after the end of the flow become closed figures after thinning out by the DP method.

An example in which the flow of FIG. 1 is applied will be described with reference to the drawings. In the following, which flow in FIG. 1 corresponds to is described in parentheses. FIG. 3A shows an open figure represented by a solid line and feature points a to g before applying the DP method. The feature points a to g before applying the DP method are obtained by, for example, the Harris algorithm. The DP method is applied to this open figure according to the flow of FIG. First, the feature point a and the feature point g, which are the end points, are connected by a straight line (S101), and the feature point e, which is between the two feature points and is the farthest from the connected straight line, is extracted (S102). Next, as shown in FIG. 3B, the distance α1 between the straight line ag and the feature point e is compared with the threshold value (S103), and if the distance α1 is larger than the threshold value, the feature point e is thinned out. It is determined as a feature point (feature point remaining after thinning out) (S104). Here, it is assumed that the distance α1 is larger than the threshold value (S103, Yes), and the feature point e is set as one of the feature points after thinning out.

Next, as shown in FIG. 3C, the extracted feature point e and the feature point g located at one end of the previous straight line ag are selected, the straight line eg is drawn (S105), and the feature point e and the feature point are drawn. The distance α2 between the feature point f, which is between g and is farthest from the straight line egg, and the straight line egg is compared with the threshold value. Here, it is assumed that the distance α2 is smaller than the threshold value (S103, No), and the end flag is given to the interval between the feature point e and the feature point g (S106). The feature point f located in the section to which the end flag is given is deleted. Next, it is determined whether or not the end flag is given to all the sections (S107). Here, since the end flag is not given to the section from the feature point a to the feature point e (S107, No), as shown in FIG. 3D, it is the end point of the section to which the end flag is not given. A straight line ae connecting the feature point a and the feature point e is drawn (S108), and the section between the feature point a and the feature point e is similarly thinned by the DP method. Finally, as shown in FIG. 3 (e), a figure obtained by thinning out the feature points c and the feature points f is obtained. The above is the explanation of the method of thinning out the feature points by the DP method.

FIG. 4 is a diagram illustrating an example of the hardware configuration of the image processing apparatus 100 according to the present embodiment. In the image processing device 100 of the present embodiment, the CPU 101, the RAM 102, the ROM 103, the image reading device 104 (reading unit), the storage device 105, the external I / F 106, and the input / output I / F 107 are connected via the bus 108, respectively. ..

The CPU 101 is an arithmetic processing device in the image processing device 100, and controls the operation of the entire image processing device 100 by executing a control program. The RAM 102 is a volatile storage medium for reading and writing information at high speed, and functions as a work area when the CPU 101 executes a control program. The ROM 103 is a read-only non-volatile storage medium in which the control program is stored.

The image reading device 104 converts an image signal obtained by reading an object set on the platen with a line sensor into R (red), G (green), and B (blue) image data (RGB data). Output. The RGB data represents the values of R, G, and B of each pixel as integer values from 0 to 255, respectively. The size of the image reader 104 to be read is determined in the default state. For example, even when a business card size medium smaller than the default reading size is set on the platen, the input / output I / F 107 Unless otherwise specified, the image reader 104 reads the medium at the default reading size. The default read size is generally set to A4 size. The image reading device 104 reads the pressure plate on the portion where nothing is placed on the platen, and the pixel corresponding to the pressure plate portion is given an RGB value representing white.

The storage device 105 is, for example, an HDD (Hard Disk Drive), which is a large-capacity non-volatile storage medium capable of reading and writing information, and stores a control program and image data. The storage device 105 may be a storage medium of another type such as an SSD (Solid State Drive).

The external I / F 106 is a communication interface for connecting to a network such as a LAN, and is connected to an external terminal such as a PC (Personal Computer) or a smartphone. The external I / F 106 has a receiving function and a transmitting function, receives information sent from an external terminal, and transmits image data stored in the storage device 105.
The input / output I / F 107 is an interface that inputs instructions and setting information to the image processing device 100 from the outside and outputs information indicating the state of the image processing device 100 to the outside. A mouse, keyboard, touch panel display, etc. are connected to the input / output I / F 107.

The CPU 101 controls the target device by reading the corresponding control program from the ROM 103, developing it in the RAM 102, and executing it based on the information input by the user via the input / output I / F 107. The CPU 101 can also receive information from an external terminal such as a PC or a smartphone through the external I / F 106, read the corresponding control program from the ROM 103, and control the target device. Further, some functions of the CPU 101 may be executed by using a circuit such as an ASIC (Application Specific integrated Circuit) instead. Further, the CPU 101 may be composed of a plurality of hardware (multi-CPU).

FIG. 5 is a diagram illustrating an example of a functional configuration of the image processing device 100 according to the present embodiment. The image processing device 100 includes an image processing unit 201 that processes RGB data input from the image reading device 104 and outputs cutout image data. The image processing unit 201 executes processing of RGB data according to a command from the CPU 101. The image processing unit 201 includes an image conversion unit 202, an edge extraction unit 203, a contour line extraction unit 204, a feature point extraction unit 205, and a post-processing unit 206, which will be described later.

The image conversion unit 202 converts the RGB data read by the image reading device 104 into C (cyan), M (magenta), Y (yellow), and K (black) image data (CMYK data). The CMYK data represents the amount of color material used in each pixel when the pixel value represented by RGB data is expressed by printing, and the amount of color material used for each CMYK used in the target pixel is an integer from 0 to 255. Represented by. The conversion to CMYK data is performed using the conversion table stored in the storage device 105. CMYK values corresponding to RGB values are input to the conversion table, and these values are updated by periodic calibration.

The edge extraction unit 203 extracts edges included in the CMYK image data by a filtering process. As the filter coefficient used for the filtering process, in addition to the default coefficient normally used, different coefficients corresponding to the mode to be used are prepared. The edge extraction unit 203 performs filtering processing on the attention pixel and its peripheral pixels, and adds information indicating that the attention pixel is an edge pixel to the attention pixel determined to form an edge. The edge pixels extracted by the edge extraction unit 203 include pixels corresponding to the boundary between characters and figures included in the medium and the background portion, as well as pixels at the boundary between the medium and the pressure plate. In the present embodiment, an edge is an aggregate of pixels composed of continuous edge pixels.

The contour line extraction unit 204 extracts the outermost edge of the edges extracted by the edge extraction unit 203 as a closed contour line of the medium by using the method described in Non-Patent Document 2. When a plurality of media are placed on the platen, the closed contour line of each medium is extracted. If there are a plurality of areas surrounded by closed contour lines, the subsequent processing is performed for each area.

The feature point extraction unit 205 extracts feature points corresponding to the corners of the medium from the closed contour lines extracted by the contour line extraction unit 204. In the present embodiment, a rectangle or a medium close to a rectangle is targeted for detection, and the feature point extraction unit 205 extracts four feature points that are optimal positions for the corners of the medium. The extraction in the present embodiment is a process of specifying the position of the point to be extracted, and an example thereof is a process of storing the position coordinates of the point to be extracted in the RAM 102 or the storage device 105. The feature points that are the optimum positions are located so that the entire area of the medium is included in the quadrangular area obtained when the four feature points are connected, and the area of the quadrangular area is the minimum. It means a feature point located at the coordinates. The feature points extracted by the feature point extraction unit 205 are used for the processing in the post-processing unit 206 described later.

The image processing device 100 in the present embodiment aims to extract a rectangle or a medium close to a rectangle, and the image processing device 100 cuts out the entire medium, so that the cut-out shape is a general image data. Rotation conversion and contour extraction are performed so that the shape becomes a rectangle. Cutting out the medium in the present embodiment means that the image data of the region to be cut out is selectively output from the image data of the size output by the image reading device 104. Since the medium in which the number of feature points extracted by the feature point extraction unit 205 is four is generally rectangular such as a business card or a medium close to a rectangle, the post-processing unit 206 is a detection target. It is determined that the medium is a certain medium, and processing is performed on a rectangle obtained by connecting four feature points. The medium in which the number of feature points extracted by the feature point extraction unit 205 is 5 points or more or 3 points or less is different from a medium having a generally rectangular shape such as a business card or a medium having a rectangular shape. The processing unit 206 determines that the medium is not the detection target. The feature points are extracted by the DP method, and are extracted as the vertices when the target figure is simply expressed. In the present embodiment, a medium having rounded corners or a medium having bent corners, which is close to a rectangle, is also targeted for extraction, and four feature points are extracted from these media by thinning out the feature points by the DP method. Will be done.

FIG. 6 is a diagram illustrating an example of the functional configuration of the feature point extraction unit 205. After the feature point extraction unit 205 extracts a temporary feature point with respect to the closed contour line by the first feature point extraction unit 207, the feature point extraction unit 205 becomes the optimum position by the contour line division unit 208 and the second feature point extraction unit 209. Extract feature points of points. Each process will be described in detail.

The first feature point extraction unit 207 applies the DP method to the closed contour line extracted by the contour line extraction unit 204, and extracts feature points for simplifying the contour of the medium. FIG. 7A is an example of a diagram showing feature points R1 extracted by the first feature point extraction unit 207 when a rectangular medium is read, and FIG. 7B is an example of a figure when a medium with rounded corners is read. It is an example of the figure which shows the feature point R1 extracted by the 1st feature point extraction unit 207. The figure shows one of the corners of the medium enlarged, and one region surrounded by a square is a pixel extracted as an edge pixel by the edge extraction unit. Among the edge pixels, the pixel shown by the diagonal line is the feature point R1 extracted by the first feature point extraction unit 207. In the present embodiment, the closed contour line is treated as an aggregate of edge pixels, and feature points are extracted by applying the DP method to the aggregate of points composed of edge pixels. By using the points that are edge pixels as they are as feature points in this way, it is possible to save the trouble of redetermining the points corresponding to the feature points.

Here, the optimumity of the feature points after being thinned out by the first feature point extraction unit 207 will be described.
FIG. 8A shows the feature points R01 to R04 after the DP method was applied to the rectangular medium, and the region X connecting them, and FIG. 8B shows the medium having rounded corners. The feature points R11 to R14 after the DP method is applied, and the region X connecting them are shown. The shaded area in the figure indicates the difference area between the medium area and the area connecting the feature points. The dotted line part in the figure shows the outline of the medium, and the solid line part shows the outline when the feature points are connected.
In the case of a rectangular medium, it can be said that the optimum feature points are extracted because the area X includes the entire medium area and the area is the smallest.
On the other hand, in the case of a medium having rounded corners, since the feature points after being thinned out by the DP method are points on the edge, the medium region does not fit in the region X, and the optimum feature points are extracted. I can't say that there is. In short, when the region to be cut out is determined based on the feature points determined only by the DP method, the information of the difference region is lost for the medium having rounded corners. Further, since the region X is not rectangular, the x-axis and the x-axis when one of the main scanning direction and the sub-scanning direction of the reading direction in the image reading device 104 in the post-processing unit 206 is the x-axis and the other is the y-axis It is not possible to accurately rotate and convert the image along the y-axis.

Therefore, the contour line dividing unit 208 and the second feature point extraction unit 209 are in the case of a medium having rounded corners, with the four feature points thinned out by the first feature point extraction unit 207 as temporary feature points. Extract four new feature points so that the extracted feature points are optimal.

The contour line dividing unit 208 deletes the edge pixels and the temporary feature points located around the temporary feature points, and divides the closed contour line into four contour lines S1 to S4. The periphery of the tentative feature point in the present embodiment indicates a range of a predetermined length from the feature point or a region of n × n pixels centered on the feature point. Further, the division of the closed contour line in the present embodiment means a state in which each of the four contour lines is recognized as an individual one.
FIG. 9 is a diagram illustrating an example of a method of dividing the closed contour line of the medium when the corners are rounded by the contour line dividing portion 208.
The contour line dividing unit 208 deletes the edge pixels forming the closed contour line located around the temporary feature points extracted by the first feature point extracting unit 207. Since the edge pixels around the temporary feature point are deleted, the edge pixels corresponding to the rounded corners are deleted. In the figure, the edge pixels without color are the edge pixels to be deleted, the edge pixels painted by the halftone dot pattern are the edge pixels remaining after the deletion, and the set of these edge pixels is a divided contour line. Corresponds to. In the figure, the contour line on the upper side corresponds to S1 and the contour line on the right side corresponds to S2, and although not shown in FIG. 9, the contour line on the lower side is S3 and the contour line on the left side is S4. Further, in the figure, the edge pixels included in the area of 17 × 17 pixels centered on the temporary feature point are deleted, but the area to be deleted can be appropriately adjusted. For example, by enlarging the area to be deleted, the edge pixels located at the round corners can be reliably deleted, but when the medium to be read is small, the pixels other than the edge pixels located at the corners are also deleted. Therefore, there is a possibility that the closed contour line cannot be accurately divided into four. On the contrary, by reducing the area to be deleted, the closed contour line can be accurately divided into four even in a small medium, but the edge pixels located in the rounded corners cannot be completely deleted. there is a possibility. Adjustments to values other than the default values are made by the user through the I / O I / F 107. The deletion of the edge pixel in the present embodiment means that the information indicating that the edge pixel is deleted is deleted so that the edge pixel is not recognized as the edge pixel. Further, additional information may be added to the edge pixels to be deleted so that the edge pixels to be deleted in the subsequent processing are not recognized as edge pixels in the subsequent processing.

The second feature point extraction unit 209 derives the linear equations of the four divided contour lines S1 to S4, respectively, and determines the coordinates of the intersection R1'on the extension lines of the contour lines S1 and S2 as shown in FIG. Derived. In the figure, the intersection point R1'is represented by hatching of vertical lines, which corresponds to the optimum feature point in the present embodiment. The linear equation is derived by randomly selecting some of the edge pixels included in one of the divided edges and using the least squares method for the coordinates of those edge pixels. The four intersections R1'to R4' on the extension lines of the four divided contour lines S1 to S4 are the optimum four feature points. Since the linear equation has some error, the contour lines S1 to S4 do not necessarily intersect in the vertical direction, but may intersect in a substantially vertical direction.

FIG. 11 is a diagram for explaining the optimumity of the feature points R1 ′ to R4 ′ extracted by the second feature point extraction unit 209 with respect to the medium having rounded corners. In the figure, the area surrounded by the dotted line indicates the medium area, and the area surrounded by the solid line indicates the area X'enclosed by the feature points R1'to R4'. This region includes the entire medium region, and the area of the region is the minimum or a state close to the minimum. Since the linear equation has some errors, the feature points R1'to R4'are considered to be the optimum feature points even if the medium region slightly protrudes from the region X'. As described above, the feature point extraction unit 205 in the present embodiment sets the feature points obtained by thinning out by the DP method as temporary feature points, and extracts the optimum feature points from the temporary feature points. Based on the coordinates of the optimum feature points R1'to R4' extracted by the feature point extraction unit 205, the post-processing unit 206 performs region complementation, rotation conversion, and cutting out.

FIG. 12 is a diagram illustrating an example of the functional configuration of the post-processing unit 206. The post-processing unit 206 performs processing on a quadrangle obtained by connecting four feature points of a medium in which the number of feature points to be extracted is four. First, the determination unit 210 determines the medium to be processed, the area complement unit 211 complements the pixels in the region X'with respect to a specific region, and the skew correction unit 212 performs rotation conversion by affine transformation. A region including the medium region corrected by the cutout unit 213 is cut out and output as cutout image data. Complementation in the present embodiment means replacing the pixel value of the target pixel with another pixel value. Further, the complement in the present embodiment includes the case where the pixel value before replacement and the pixel value after replacement are the same. Each process will be described in detail.

The determination unit 210 extracts a medium having four feature points. The area complementing unit 211 is a difference area that targets the medium extracted by the determination unit 210 and is within a rectangular area obtained by connecting the optimum feature points extracted by the feature point extraction unit 205 and does not overlap with the medium area. Area completion is performed for. Examples of the method of complementing the difference region by the region complement unit 211 include a method of applying the pixel values of the pixels located in the medium region and located around the difference region to the pixels included in the difference region, and the difference. An example is a method in which a difference region is set as a transparent region by applying a pixel value indicating that the pixel is a transparent region to the pixels included in the region.
When complementing the difference area as a transparent area, a predetermined pixel value is allocated to the pixels included in the difference area. Complementing the pixel values by the transparent region may be performed by selecting one from a plurality of pixel values stored in advance. In addition, it may not be interpolated. The method of interpolating the difference area by the area complementing unit 211 may be set in the ROM 103 in advance, or the user may select an arbitrary method.
FIG. 13 is an example of an operation screen that allows the user to select a method for complementing the difference area. The area complementing unit 211 allows the user to select a method of complementing the difference region. The user can perform a predetermined process on the difference region by selecting a desired process.

The skew correction unit 212 performs rotational conversion by affine transformation on the region X'obtained by connecting the four feature points extracted by the feature point extraction unit 205, and performs rotation conversion by affine transformation, and the main scanning direction and the sub scanning direction in the image reading device 104. When one of the two sides is the x-axis and the other is the y-axis, the two sides of the rectangle are matched with the x-axis and the y-axis. However, since the angles of the four vertices of the area surrounded by connecting the four feature points are expected to deviate slightly from the right angle, the two sides do not always match, and they match within a predetermined error range. Anything that does is fine. That is, when one of the two adjacent sides of the rectangular image data output by the image reading device 104 is the x-axis and the other is the y-axis, the skew correction unit 212 has two adjacent sides of the region X', respectively. Rotational conversion is performed on the region X'so that it is substantially parallel to the x-axis and the y-axis.

The cutout unit 213 cuts out the area X'from the image data that has been rotationally converted by the skew correction unit 212, and outputs it as cutout image data. If there are multiple cutout areas, all of them are output as separate cutout image data. The output destination is an external terminal or the like connected through the storage device 105 or the external I / F 106.

As described above, the quadrangular region whose rotation is converted by the skew correction unit 212 has four corners at right angles or substantially right angles, and the medium region is not partially deleted by the cutout unit 213 after the rotation conversion. Further, since the optimum feature points are extracted by the second feature point extraction unit 209, the medium region is not partially deleted by the cutout unit 213 when the closed contour line is simplified by the DP method.

FIG. 14 is a diagram illustrating an example of a series of processing flows in the present embodiment. In S301, the edge extraction unit 203 extracts the edge pixels included in the image data acquired by the image reading device 104. In S302, the contour line extraction unit 204 extracts a set of edge pixels corresponding to the contour of the medium from the edge pixels extracted in S301 as a closed contour line. In S303, the first feature point extraction unit 207 extracts the feature points of the target closed contour line. In S304, the contour line dividing unit 208 divides the closed contour line into a plurality of contour lines. In S305, the second feature point extraction unit 209 extracts the optimum feature points based on the contour lines divided in S304. In S306, the region complementing unit complements the difference region of the region surrounded by the straight line connecting the medium region and the optimum feature point with a predetermined pixel. In S307, the skew correction unit 212 performs rotation conversion on the region surrounded by the straight line connecting the optimum feature points, and the cutting unit 213 cuts out the region after the rotation conversion is performed, and the cutout image data is obtained. Output. In S308, it is determined whether or not the cutting portion 213 has cut out the region composed of all the closed contour line sets, and if the cutting is performed (S308, Yes), the flow is terminated. When the cutting is not performed (S308, No), the process returns to S303, and the flow of S303 to S307 is performed on the closed contour line set that has not been cut out.

As described above, according to the image processing device 100 according to the present embodiment, the edge extraction unit 203 extracts edge pixels from the image data acquired by the image reading device 104, and the contour line extraction unit 204 extracts the acquired edge pixels. Among them, a set of edge pixels corresponding to the contour of the medium is extracted as a closed contour line, the first feature point extraction unit 207 extracts a temporary feature point of the closed contour line, and the contour line dividing section 208 extracts the closed contour line. It is divided into a plurality of contour lines, the second feature point extraction unit 209 extracts the optimum feature points based on the divided contour lines, the area complementing unit 211 complements the area, and the skew correction unit 212 performs skew correction. By doing so, even a medium having rounded corners or a medium having bent corners can be stored as image data without damaging the original medium area.

Although the specific embodiment of the present invention has been described above, the above-described embodiment shows an example of the present invention. The present invention is not limited to the above-described embodiment, and can be embodied by making various modifications and changes without departing from the gist of the present invention at the implementation stage.

A program that realizes each function by being executed by each device of the above embodiment is a file in an installable format or an executable format, such as a CD-ROM, a CD-R, a memory card, a DVD (Digital Versatile Disk), or a file. It is stored and provided in a computer-readable storage medium such as a flexible disk (FD).
Further, the above program may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. Further, the above program may be provided or distributed via a network such as the Internet. Further, the above program may be provided by incorporating it into a ROM or the like in advance.

[Summary of Examples of Embodiments of the Present Invention, Actions, and Effects]
The image processing apparatus 100 according to the first embodiment of the present invention is represented by an edge extraction unit 203 that extracts edge pixels included in image data obtained by reading a region including a medium, and a set of edge pixels. Of the edges, the contour line extraction unit 204 that extracts the closed contour line corresponding to the contour of the medium, the first feature point extraction unit 207 that extracts the first feature point on the closed contour line, and the first on the closed contour line. The edge pixel located around the feature point of is deleted, and the contour line dividing portion 208 that divides the closed contour line into a plurality of contour lines and the second feature that is an intersection point on the extension line of each divided contour line. A second feature point extraction unit 209 for extracting points, and a cutout section 213 for cutting out a region obtained by connecting the second feature points to each other are provided.
Here, a typical example of a medium is a rectangular business card.
According to this embodiment, it is possible to cut out image data having a shape close to a rectangle as image data of the medium even if the medium has rounded corners or a medium in which a part of the corners is bent. That is, it is unlikely that the sides other than the corners of the rectangular medium are significantly inclined even if the corners are deformed. Therefore, the plurality of contour lines obtained by deleting the edge pixels located around the first feature point on the closed contour line represent the contour lines extending along the sides of the medium excluding the deformed corners. As a result, the area surrounded by the straight line including each contour line after division has a shape close to a rectangle. According to this aspect, rectangular image data can be obtained as the image data of the region.

In the image processing apparatus 100 according to the second embodiment of the present invention, the first feature point extraction unit 207 extracts the first feature point by thinning out a set of edge pixels constituting the closed contour line. To do.
By thinning out a set of edge pixels, an appropriate point representing a corner of the medium can be mechanically determined.

The image processing apparatus 100 according to the third embodiment of the present invention is a pixel complementing unit (a pixel complementing unit) that complements pixels in a difference region that is inside a region obtained by connecting each of the second feature points and does not overlap with the media region. The area complementing unit 211) is further provided.
Here, complementation means replacing the pixel values of the target pixels with the same or different pixel values. The pixel value includes a pixel value indicating a transparent region. In the case of a medium having rounded corners, the reading unit (image reading device 104) cannot acquire the image data of the medium for the difference region. In this aspect, the pixels in the difference region are complemented with pixel values suitable for the usage mode.

In the image processing apparatus 100 according to the fourth embodiment of the present invention, the pixel complementing unit (region complementing unit 211) uses the pixel values related to the pixels in the region of the medium located around the difference region to obtain the difference. Complements the pixel values of the pixels in the area.
According to this aspect, as the cutout image data output by the cutout unit 213, it is possible to obtain image data that does not give a sense of discomfort in the hue of the difference region.

The image processing apparatus 100 according to the fifth embodiment of the present invention further includes a rotation conversion unit (skew correction unit 212) that performs rotation conversion on a region obtained by connecting the second feature points to each other. The output unit 213 cuts out a region that has been rotationally converted by the rotation conversion unit.
Since the region obtained by connecting the second feature points to each other is a substantially rectangular region, the medium region is not partially deleted by the rotation transformation. According to this aspect, it is possible to cut out the medium according to the user's intention without partially deleting the medium area.

The image processing method according to the sixth embodiment of the present invention includes an edge extraction step (S301) for acquiring image data obtained by reading a region including a medium and extracting edge pixels included in the image data, and an edge. A contour line extraction step (S302) for extracting a contour line corresponding to the contour of the medium among the edges represented by a set of pixels, and a first feature point extraction step (S303) for extracting the first feature point on the contour line. ), The contour line dividing step (S304) for dividing the closed contour line into a plurality of contour lines by deleting the edge pixels located around the first feature point on the contour line, and each of the divided contour lines. A second feature point extraction step (S305) for extracting each of the second feature points that are intersections on the extension line, and a cutout step (S307) for cutting out a region obtained by connecting each of the second feature points. Has.
This aspect exhibits the same actions and effects as those of the first embodiment.

The image processing apparatus 100 according to the seventh embodiment of the present invention is a program for causing a computer to execute the image processing method according to the sixth embodiment.
This aspect exhibits the same actions and effects as those of the first embodiment.

100 Image processing device 101 CPU 102 RAM 103 ROM 104 Image reading device (reading unit) 105 Storage device 106 External I / F 107 Input I / F 108 Bus 201 Image processing unit 202 Image conversion unit 203 Edge extraction unit 204 Contour line extraction unit 205 Feature point extraction section 206 Post-processing section 207 First special vertex extraction section 208 Contour line division section 209 Second feature point extraction section 210 Judgment section 211 Area complement section 212 Skew correction section 213 Cutout section

Shin-Ting Wu; Adler C. G. da Silva; Mercedes R. G. Marquez, The Douglas-peucker algorithm, J. Braz. Comp. Soc. Vol.9 no.3 Campinas Apr. 2004 Satoshi Suzuki; Keiichi Abe, Topological structural analysis of digitized binary images by border following, Computer Vision Graphics and Image Processing 30 (1): 32-46 March 1985

Claims (7)

An edge extraction unit that extracts edge pixels included in the image data obtained by reading an area including a medium, and an edge extraction unit.
A contour line extraction unit that extracts a closed contour line corresponding to the contour of the medium among the edges represented by the set of edge pixels, and a contour line extraction unit.
A first feature point extraction unit that extracts the first feature point on the closed contour line,
A contour line dividing portion that divides the closed contour line into a plurality of contour lines by deleting edge pixels located around the first feature point on the closed contour line.
A second feature point extraction unit that extracts each of the second feature points that are intersections on the extension lines of the divided contour lines, and a second feature point extraction unit.
A cutout portion that cuts out a region obtained by connecting each of the second feature points and
An image processing device comprising. The image processing apparatus according to claim 1, wherein the first feature point extraction unit extracts the first feature point by thinning out a set of edge pixels constituting the closed contour line. The image processing apparatus according to claim 1 or 2, further comprising a pixel complementing portion that complements pixels in a difference region that is inside a region obtained by connecting the second feature points and does not overlap with the region of the medium. The image processing apparatus according to claim 3, wherein the pixel complementing unit complements the pixel values of the pixels in the difference region by using the pixel values of the pixels in the region of the medium located around the difference region. A rotation conversion unit that performs rotation conversion for a region obtained by connecting the second feature points to each other is further provided.
The image processing apparatus according to claim 1 to 4, wherein the cutout unit cuts out a region converted by rotation by the rotation conversion unit. An edge extraction step of acquiring image data obtained by reading an area including a medium and extracting edge pixels included in the image data, and an edge extraction step.
A contour line extraction step for extracting a closed contour line corresponding to the contour of the medium among the edges represented by the set of edge pixels, and
The first feature point extraction step for extracting the first feature point on the closed contour line and
Remove the edge pixels located around the first feature point of the closed contour, a contour dividing step of dividing said closed contour to a plurality of contour lines,
A second feature point extraction step for extracting each of the second feature points that are intersections on the extension lines of the divided contour lines, and a second feature point extraction step.
A cutting step for cutting out a region obtained by connecting each of the second feature points, and
Image processing method having. A program for causing a computer to execute the image processing method according to claim 6.

Images