JP5626939B2 - Sensing processing device for moving ball, sensing processing method, and virtual golf simulation device using the same - Google Patents

Description

  The present invention relates to a sensing processing device for a moving ball, a sensing processing method, and a virtual golf simulation device using the sensing processing method. More specifically, the present invention acquires and analyzes an image of a moving ball such as a golf ball. The present invention relates to a sensing processing device, a sensing processing method for a moving ball, and a virtual golf simulation device using the same so that the physical information can be calculated.

  Recently, active developments have been made on various devices that allow popular sports competitions such as baseball, soccer, basketball, and golf to be enjoyed in the form of interactive sports games through simulations indoors or at specific locations. Has been made.

  In such an interactive sports game, physical information about the moving ball, that is, the movement of the ball is used so that a simulation for sports using a ball such as a baseball ball, a soccer ball, a basketball, and a golf ball is performed. Research and development on various sensing systems for accurate sensing is very active.

  For example, various sensing systems such as a sensing system using an infrared sensor, a sensing system using a laser sensor, a sensing system using an acoustic sensor, and a sensing system using a camera sensor have appeared.

  The present invention accurately determines the coordinates of the center point for the ball image on the acquired image at the sub-pixel level, even though the image for the moving ball is acquired by a low-resolution low-speed camera device and strobe device. Providing a sensing processing device for a moving ball, a sensing processing method, and a virtual golf simulation device using the same, which can realize high sensing processing capability and sensing accuracy at a low cost. Is to do.

A sensing processing apparatus according to an embodiment of the present invention is a sensing processing apparatus that acquires and analyzes an image of a ball moving under a predetermined illumination. The sensing processing apparatus extracts a ball image from the acquired image and extracts the ball image. Illumination direction estimating means for extracting a point having the highest luminance value of the ball image and estimating a direction of illumination with respect to the moving ball based on the point having the highest luminance value ; and along the estimated illumination direction By fitting to the shape of the ball on the basis of the side receiving the illumination of the ball image, the fitting is performed so that the shade region on the opposite side of the point having the highest luminance value on the ball image is included in the fitting. And image processing means for extracting the coordinates of the center point of the image.

  A sensing processing device according to another embodiment of the present invention is a sensing processing device that acquires and analyzes an image of a ball moving under a predetermined illumination, and is set in advance at a specific position of the ball image or A semicircle fitting means for forming a semicircle curve of the diameter measured through the ball image, and fitting a semicircle opposite to the semicircle curve based on the diameter; a center of the semicircle fitted ball image; Coordinate extracting means for extracting the coordinates of the point;

Meanwhile, a sensing processing method according to an embodiment of the present invention is a sensing processing method for acquiring and analyzing an image of a ball moving under a predetermined illumination, extracting a ball image from the acquired image, and Extracting a point having the highest luminance value of the extracted ball image and estimating a direction of illumination with respect to the moving ball based on the point having the highest luminance value ; and along the estimated illumination direction Fitting the shape of the ball with respect to the side receiving the illumination of the ball image so that the shade region on the opposite side of the point having the highest luminance value on the ball image is included in the fitting ; Extracting the coordinates of the center point of the fitted image.

Meanwhile, a virtual golf simulation apparatus according to an embodiment of the present invention includes a camera unit that acquires a multiple exposure image of a golf ball that is hit by a golfer and moves, and a sensor unit that includes a strobe device; golf from the acquired image An illumination direction for extracting a ball image, extracting a point having the highest luminance value of the extracted ball image, and estimating a direction of the strobe illumination with respect to the golf ball based on the point having the highest luminance value By fitting to the shape of the golf ball on the basis of the estimated illumination direction and the side receiving the golf ball image illumination along the estimated illumination direction, the opposite side of the point having the highest luminance value on the ball image wherein the fitting so as to shade area is included in the fitting A sensing processing unit including image processing means for extracting the coordinates of the center point of the image; by calculating a change in the coordinates of the center point, calculating physical information about the moving golf ball and simulating the trajectory of the golf ball And a simulator.

  The sensing processing device, the sensing processing method, and the virtual golf simulation apparatus using the sensing processing device for the moving ball according to the present invention are obtained even though the image of the moving ball is acquired by a low-resolution low-speed camera device and a strobe device. By enabling the coordinates of the center point for the ball image on the acquired image to be accurately obtained at the sub-pixel level, it is possible to realize high sensing processing capability and sensing accuracy at a low cost. There is.

It is a block diagram which shows the sensing processing apparatus or virtual golf simulation apparatus which concerns on one Example of this invention. It is a figure which shows an example of the screen golf system to which the sensing processing apparatus or virtual golf simulation apparatus shown by FIG. 1 was applied. It is a figure which shows the operation signal system of the camera apparatus which acquires the image processed with the sensing processing apparatus which concerns on one Example of this invention, and a strobe device. (A) And (b) is a figure which respectively shows the image pattern of the ball | bowl acquired by the system shown by FIG. FIG. 5A is a diagram showing an example of an actually acquired image of the image of the pattern shown in FIG. 4A, and FIG. 5B is an image shown in FIG. It is a figure which shows the image by which the pre-processing with respect to was carried out. FIG. 6 is an enlarged view of the image shown in FIG. 5B and a plurality of frames superimposed on each other, and is a diagram illustrating a process of extracting a ball image. FIG. 7A shows a ball image, FIG. 7B shows an example of incorrect fitting for a ball image, and FIG. 7C shows the principle of ball image fitting. It is a figure which shows the example of the fitting of the ball image by the fitting means of invention. FIG. 8A is an image in a state in which the background is removed, and FIG. 8B is an image showing the area A in FIG. 8A in an enlarged manner. FIG. 9A is an image obtained by performing low-level image processing on the image shown in FIG. 8A, and FIG. 9B shows the region B in FIG. 9A. This is an enlarged image. 10A is an image obtained by performing high-level image processing on the image shown in FIG. 8A, and FIG. 9B shows the region C in FIG. 9A. This is an enlarged image. FIG. 11A is an image in a state in which the maximum bright spot is extracted using the image shown in FIG. 10B, and FIG. 11B shows the image shown in FIG. It is an image with the maximum bright spot applied. 12A is a diagram showing a process of performing a line scan centering on the maximum bright point with respect to the ball image, FIG. 12B is an image showing a result of the line scan, and FIG. 12C is a diagram of FIG. This is an image obtained by applying an average filter to the line scan result shown in FIG. It is a figure for demonstrating the estimation of the illumination direction with respect to a ball image. It is a figure which shows the axis | shaft of the illumination direction estimated by the illumination direction estimation means of the sensing processing apparatus which concerns on this invention on a ball image. It is a figure shown about the process of measuring the diameter with respect to the ball image shown by FIG.9 (b). It is a figure which shows the fitting process with respect to a ball image in consideration of the estimated illumination direction. (A) And (b) is a figure which shows the fitting process with respect to a ball image in consideration of the estimated illumination direction. FIG. 18 is an image in a state where a center point is extracted for a ball image fitted by the fitting process shown in FIGS. 17 (a) and 17 (b). It is a flowchart which shows each Example regarding the sensing processing method which concerns on this invention. It is a flowchart which shows each Example regarding the sensing processing method which concerns on this invention. FIG. 21 is a flowchart showing a more specific process of fitting the ball image shown in FIGS. 19 and 20. FIG.

  Embodiments relating to a sensing processing apparatus, a sensing processing method, and a virtual golf simulation apparatus using the sensing processing apparatus for a moving ball according to the present invention will be described more specifically with reference to the drawings.

  The sensing processing apparatus for a moving ball according to the present invention is applied to all systems in which sensing is performed by acquiring and analyzing an image related to a moving state of a moving ball in sports using a ball such as golf. For example, the present invention can be applied to a so-called screen golf system to which a virtual golf simulation apparatus is applied.

  1 and 2 show a schematic configuration of a sensing processing apparatus for a moving ball according to the present invention and a virtual golf simulation apparatus using the sensing processing apparatus.

  First, a sensing processing device for a moving ball and a virtual golf simulation device using the same according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2.

  As shown in FIG. 1, the present invention processes an image of a moving ball by processing an image acquired by a sensor unit including a camera device 310, 320, a strobe device 330, and a signal generation unit 210. The sensing processing device 220 according to an embodiment of the present invention relates to a sensing processing device 220 that extracts the coordinates of the center point. Etc. are preferably configured.

  First, the sensor unit will be described. The camera devices 310 and 320 are provided to acquire images of a plurality of frames with respect to a state in which the ball moves in the movement direction from the initial position of the ball.

  Although FIG. 1 shows a case where two camera apparatuses 310 and 320 are provided, the present invention is not limited to this, and all cases including one or more camera apparatuses can be included.

  The strobe device 330 is an illumination device using an LED or the like, and is used as a photographing light source for the camera device. The strobe device 330 is used for strobe illumination at a predetermined time interval (a plurality of illumination flashes are activated at a predetermined time interval). A multiple exposure image is acquired by the camera devices 310 and 320.

  That is, a multiple exposure image that appears by shooting the ball by the number of flashes by the strobe device 330 is obtained in an image of one frame photographed by the camera device.

  Specific matters regarding the operation of the camera devices 310 and 320 and the strobe device 330 will be described later.

  On the other hand, a trigger signal for operating the camera devices 310 and 320 and the strobe device 330 is generated by the signal generator 210, and the multiple exposure images by the camera devices 310 and 320 and the strobe device 330 are the present invention. Is processed by the sensing processing device 220 and transmitted to the simulator 100.

  The illumination direction estimating means 250 of the sensing processing device 220 according to an embodiment of the present invention is implemented to extract the maximum bright point of the ball image and estimate the illumination direction of the strobe illumination passing through the maximum bright point. The The illumination direction estimating unit 250 includes a luminance analyzing unit (not shown), a scanning unit (not shown), and the like, which will be described in detail later.

  On the other hand, the image processing unit 260 performs fitting by fitting the ball image into the shape of the ball with reference to the outline on the side receiving the illumination of the ball image along the illumination direction estimated by the illumination direction estimating unit 250. The coordinates of the center point of the captured image are extracted.

  That is, the image processing unit 260 forms a semicircular curve having a diameter set in advance or measured through the ball image at a specific position of the ball image, and sets the semicircle on the opposite side of the semicircular curve to the semicircle curve. It is preferable to include a semicircle fitting means 270 for fitting based on the diameter, and a coordinate extraction means 280 for extracting the coordinates of the center point of the semicircle-fitted ball image. Specific matters regarding this will be described later.

  Here, when a value measured through a ball image is used as the diameter of the semicircular curve, the sensing processing device preferably further includes a diameter measuring unit for measuring the diameter of the ball image.

  Meanwhile, the pre-processing unit 230 is implemented to perform processing such as removing a background image other than the image, noise, and the like through a predetermined process from the multiple exposure image acquired by the sensor unit.

  The image extracting unit 240 is implemented to extract a ball image from the image preprocessed by the preprocessing unit 230 (more specifically, an image estimated as a ball, that is, a ball image candidate is extracted). The ball image extracted by the image extraction unit 240 is used to estimate the illumination direction on the ball image by the illumination direction estimation unit 250 and to the ball image by the image processing unit 260. Fitting is performed.

  Meanwhile, the simulator 100 preferably includes a control unit M, a database 110, a video processing unit 120, a video output unit 130, and the like.

  The control unit M receives transmission of coordinate information of a moving ball acquired by image processing by the sensing processing device 220, converts the coordinate information into three-dimensional coordinate information, and converts the movement of the moving ball in a three-dimensional space. By calculating the change in coordinates, predetermined physical information for the motion trajectory simulation of the ball is calculated and transmitted to the video processing unit 120.

  At this time, the predetermined data for the simulation of the motion trajectory of the ball is extracted from the database 110 and used, and the simulation video processing regarding the motion trajectory of the ball in the video processing unit 120 is the video stored in the database 110. This can be done by extracting data.

  Here, a converting means for converting the coordinate information of the moving ball transmitted from the sensing processing device 220 into the three-dimensional coordinate information can be provided separately from the control unit M.

  The preprocessing unit 230, the image extraction unit 240, the illumination direction estimation unit 250, the image processing unit 260, and the like are embodied as one controller provided to perform the function of each unit in terms of hardware. Alternatively, the corresponding function may be implemented by a separate controller for each means. Moreover, in terms of software, the functions may be embodied as a single program, or the corresponding functions may be embodied by separate programs for each means.

  FIG. 2 shows an example of a screen golf system to which a sensing processing device having the above-described device configuration or a virtual golf simulation device is applied.

  As shown in FIG. 2, a golf tee on which a golf ball 10 is placed on one side of a golf booth B and a ground (preferably including at least one of a fairway mat, a rough mat, and a bunker mat) are provided. A hitting mat 30 and a swing plate 20 provided for a golfer to perform a golf swing are provided.

  In front of the golf booth B, a screen 40 on which a virtual golf simulation video realized by the video output unit 130 is displayed is provided, and camera devices 310 and 320 and a strobe device 330 are provided on the ceiling, respectively.

  FIG. 2 shows the case where the camera devices 310 and 320 are respectively provided on the ceiling and the wall. However, the present invention is not limited to this, and it is possible to effectively acquire an image relating to the motion state of the golf ball 10 and It can be installed at any position as long as it is a preset position that does not interfere with the golf ball hit by the golfer and does not collide with the golf ball.

  FIG. 2 shows the case where the strobe device 330 is attached to the ceiling and installed to provide strobe lighting in a direction substantially perpendicular to the striking point, but is not limited thereto. Any location that can provide effective strobe lighting is possible.

  In the system as described above, when the golf player hits the golf ball 10 on the hitting mat 30 toward the screen 40 with the swing plate 20, as shown in FIG. 2, a predetermined area where the hit is made is photographed. The camera devices 310 and 320 respectively acquire images of a plurality of frames. At this time, the strobe device 330 applies a plurality of flashes per frame, and acquires a multiple exposure image of the moving golf ball.

  FIG. 3 shows a trigger signal generation system of the camera device and strobe device by the signal generator 210 (see FIG. 1).

  As shown in FIG. 3, the trigger signal for the camera device is made at tc time intervals. That is, a trigger signal is generated at tc time intervals per frame. At this time, the exposure time per frame is te time (in this case, it is preferable to satisfy tc> te), and the data for the image acquired during the tc-te time is the sensing processing device. It is preferable to transmit to 220 (see FIG. 1).

  That is, as shown in FIG. 3, the camera device is exposed during the trigger signal generation time interval per frame of the camera device, and data for the acquired multiple exposure image is transmitted to the sensing processing device.

Then, strobe illumination by the strobe device is generated a plurality of times during the exposure time (te time) of the camera device, and FIG. 3 shows a case where the strobe illumination trigger signal is generated three times at a time interval of ts ₁ . .

That is, during the exposure time (te time) of the camera device, the strobe illumination occurs three times at equal time intervals of ts ₁ hour. At this time, it is preferable that the signal generator 210 (see FIG. 1) synchronizes the camera device and the strobe device so that the first trigger signal is generated simultaneously.

Also, as shown in FIG. 3, the time from the last strobe illumination trigger signal to the first strobe illumination trigger signal of the next frame among the three strobe illuminations is set to an interval of ts ₂ hours. It is preferable to do. The ts ₁ and ts ₂ time intervals may be set to be the same, but as shown in FIG. 3, it is preferable to set the ts ₁ hour and ts ₂ time to be different from each other, and ts ₂ hours. Can be set to be longer than ts ₁ hour.

  As described above, since the intervals between the trigger signals of the camera device and the strobe device are fixed, the interval between the ball images on the acquired multiple exposure image must be constant.

  Therefore, when processing a multiple exposure image with a sensing processing device, an accurate ball image is obtained from the ball image existing on the multiple exposure image and various noises, and the above-described feature related to the fixed interval of the trigger signal is provided. Can be effectively separated based on

  Also, since the strobe illumination interval for each frame of the camera device is fixed, images of various patterns can be generated in the multiple exposure image depending on the motion speed of the ball.

  FIG. 4 shows a case where an image of a moving ball is acquired by hitting a ball (golf ball) with a golf club, and is obtained by a camera device and a strobe device using the signal generation system shown in FIG. Two patterns of the exposed image are shown in (a) and (b), respectively.

  In the image (I1) shown in FIG. 4A, when the golf club images C1, C2, C3 and the ball images 11a, 11b, 11c are acquired by multiple exposure, the ball images 11a, 11b, 11c are mutually connected. This is the case where they are shown separated at a predetermined interval.

  Also, the image 12 shown in FIG. 4B shows a case where the balls are shown as an image region 12 having a predetermined size by overlapping each other.

  That is, the image shown in FIG. 4A is a case where the image is formed at a predetermined distance when each strobe illumination is triggered because the ball moves at high speed. The image shown in FIG. 4 (b) is a case where strobe illuminations are triggered and superimposed on each other before the ball travels far because the ball moves at a low speed.

  As shown in FIG. 4 (a), the sensing processing apparatus and sensing processing method according to the present invention allows multiple exposure images by moving a ball at a high speed of a predetermined speed or more by a fixed period of strobe illumination. When the balls are shown separated from each other on the upper side, the coordinates of the center point of each ball are accurately obtained. According to the sensing processing method of the present invention, as shown in FIG. If the ball is shown in an overlapping manner as shown in FIG.

  In addition, as described above, the image regarding the excluded superimposed ball is processed by a separate process, which deviates from the scope of the present invention, and a specific description thereof will be omitted.

  Hereinafter, processing of the ball images separated from each other as shown in FIG. 4A by the sensing processing apparatus for a moving ball according to the present invention will be described with reference to FIGS.

  FIG. 5 (a) shows the original multi-exposure image containing images of balls separated from each other, acquired using a camera device operating at a low speed of 75 fps with a resolution of 320 × 240 and a strobe device of 330 Hz. It is an image.

  With respect to the original image as shown in FIG. 5A, the pre-processing means 230 (see FIG. 1) removes an image that has been stopped through subtraction or the like, that is, a background image, and Gaussian. A predetermined pre-processing process such as blurring (Gaussian Blur) is performed. FIG. 5B shows an image after the preprocessing process.

  In this way, it becomes easier to extract the ball image in a state where the background and noise on the multiple exposure image are appropriately removed by the preprocessing means. This is illustrated in FIG.

  The image shown in FIG. 6 is an image of a plurality of frames taken by a camera device superimposed on a single image, and three ball images appearing by three strobe illuminations are shown as one image. When configuring an image set, the S0 image set is an image for the 0th frame, the S1 image set is an image for the first frame, and the S2 image set is an image for the second frame.

  Here, the image extraction unit 240 (see FIG. 1) can extract an image estimated as a ball image through a contour check, a check window, and the like, while excluding an image estimated as not a ball. It is possible to process in such a manner that only the image that is finally estimated as a ball is left.

  That is, since the ball has a certain diameter, the contour for each image on the multiple exposure image is checked in consideration of this, and the checked contour is too large to be considered a ball. If it is too small, this can be excluded.

  Then, as shown in FIG. 6, the check window W is formed so that the corresponding image 11 is included in accordance with the size of each image 11 on the multiple exposure image, and the horizontal length of the check window W is formed. And the aspect ratio of the length in the vertical direction.

  Since the ball is circular, its aspect ratio must be approximately constant. Therefore, if the aspect ratio of the check window W for each image 11 on the multiple exposure image is too large and has a wide width or is too small and the width is too narrow, this is regarded as not a ball. Can be excluded.

  The image 11 estimated as a ball on the multiple exposure image can be extracted by the method exemplified as described above.

  On the other hand, as described above, the image 11 extracted as a ball image is fitted to the shape of the ball, as shown in FIG. 7, so that the coordinates of the center point can be extracted. .

  7A shows an example of a ball image acquired under strobe illumination, FIG. 7B shows an erroneous fitting result for the ball, and FIG. 7C shows the present invention. The result of fitting the ball correctly by the semicircle fitting means 270 (see FIG. 1) of the image processing means will be shown.

  As shown in FIG. 7A, since the image 11 of the ball is photographed in a state of receiving the strobe illumination, an area R1 that receives the illumination is clearly shown on one side of the ball image 11, and A shade is generated on the opposite side of the region R1 that receives the illumination, and a shade region R2 that blocks a part of the ball image 11 is present.

  Such a shade phenomenon may appear a little depending on the angle of illumination, and more than half of the ball image may be blocked by the shade phenomenon.

  When the ball image acquired under the strobe illumination is shaded, as shown in FIG. 7B, when the ball image is fitted in the shape of the ball, the center point thereof is distorted. A large error occurs in the information on the coordinates of the center point, and as a result, the accuracy of the motion simulation for the ball is greatly reduced.

  Therefore, it is preferable that the fitting for the ball image acquired under the strobe illumination is performed in consideration of the shade area appearing in the ball image.

  As an example of fitting in consideration of the shade area for the ball image, the direction of the strobe illumination with respect to the ball is estimated, and the fitting is performed based on the illumination direction, so that the shade area becomes a fitted curve. It can be included.

  FIG. 7C shows that the ball image is fitted in consideration of the illumination direction as described above. As shown in FIG. 7 (c), the fitting curve FC is formed in the region R1 that receives the illumination of the ball image 11, so that the fitting is performed so that the shade region R2 is included in the fitting curve FC. I can confirm.

  By fitting the ball image in the above manner, the center point CP can be accurately extracted.

  In order to extract the exact center point for the ball image as described above, the background and noise are appropriately removed from the first acquired multiple exposure image, and preprocessing is performed so that only the ball image remains as much as possible. (Preliminary processing) needs to be performed. For illumination direction estimation and ball image fitting, prior to appropriate Gaussian blur for the image, threshold processing for specific pixel values, and normalization for image pixel values, etc. A process is required.

  Also, due to pre-processing to extract specific information from the image, a situation can occur where the ball image is damaged. For this reason, it is preferable to generate an image for each of a plurality of steps and extract specific information from the image of each step.

  The images shown in FIGS. 8 to 10 respectively show the case where the image is generated in a plurality of steps by the pre-processing process of the image by the pre-processing means 230 (see FIG. 1).

  FIG. 8A shows an image in a state where the background is removed by subtraction or the like from the first multiple exposure image acquired. FIG. 8B is an enlarged view of the area A shown in FIG. 8A, and is an image shown for the original ball image 11-0.

  As shown in FIG. 8B, since the ball has a large number of dimples, the influence of illumination is irregular. Therefore, the distribution of pixel values of the original ball image 11-0 is very irregular. Also, it can be seen that there are a large number of so-called 'Salt and Pepper noise' having irregular pixel values in one or more small pixel units in the vicinity.

  In such an image, it is very difficult to extract various kinds of information for accurate center point extraction, and there is a high possibility that an error will occur in the extracted information.

  Therefore, it is necessary to remove all surrounding noise and appropriately adjust the irregular pixel value distribution of the original ball image 11-0.

  In FIG. 9A, an appropriate level of Gaussian blurring and threshold processing is performed on the image shown in FIG. 8A so as not to significantly damage the shape of the original ball image. FIG. 9B is an enlarged view of the region B shown in FIG. 9A, and shows the first processing. This is shown for image 11-1.

  In FIG. 10A, the image shown in FIG. 8A is subjected to higher level Gaussian blurring processing and threshold processing than the first processing image 11-1, and a ball image is obtained. FIG. 10B shows an image of a state in which normalization processing is performed on the upper pixel value. FIG. 10B is an enlarged view of the C region shown in FIG. A processing image 11-2 is shown.

  In the second processed image 11-2, the shape of the original ball image 11-0 is greatly damaged. However, the influence of the irregular pixel value distribution due to the dimple of the ball is removed, and the portion that is illuminated is clear. It can be seen that it is divided into

  Therefore, it is preferable to estimate the illumination direction as shown in FIG. 11 using the image shown in FIG. 10, that is, the second processed image 11-2.

  In the second processed image 11-2, the pixel values are appropriately distributed and the pixel values are clearly divided, so that the illumination direction can be easily estimated by finding an area where the pixel values are high. I have to. Here, since the image is in gray scale, the pixel value has information on the luminance value.

  That is, in the second processed image 11-2 shown in FIG. 11A, it can be seen that the illumination is concentrated in the R3 region having relatively high luminance, and the center of the R3 region is the center point in the illumination direction. Can be considered. That is, the point with the highest luminance value (Brighttest Point) is the point through which the axis in the illumination direction passes. In FIG. 11 (a), a point with the highest luminance value, which is the center point of the R3 region, that is, the maximum bright point P1 is displayed.

  If all the pixels on the ball image 11-2 of the second processed image have one pixel having the highest luminance value, that pixel can be designated as the maximum bright point P1. However, if the place with the highest luminance value is an irregularly shaped region composed of a plurality of pixels, the center of mass of the corresponding region is extracted from a predetermined process, and the corresponding point is A bright point P1 can be designated.

  On the other hand, the direction of the illumination is estimated by selecting the maximum bright point P1 on the ball image as the first point and selecting the other one point as the second point to form a line connecting the first point and the second point. And can be done by setting the line as the axis of illumination direction. The first point may be extracted by a luminance analysis unit of an illumination direction estimation unit 250 (see FIG. 1), and the second point may be extracted by a scanning unit of the illumination direction estimation unit 250 (see FIG. 1).

  The luminance analysis means can extract the first point described above by finding the center point of the R3 region on the ball image shown in FIG. 11A, that is, the maximum bright point P1.

  In addition, the scanning unit can extract the second point through the processes shown in FIGS. 11B, 12A, and 12B, respectively. First, as shown in FIG. 11B, the maximum bright point P1 shown in FIG. 11A is applied to the ball image 11-1 of the first processing image.

  This is because the coordinate value of the maximum bright point P1 on the ball image 11-2 of the second processed image is applied to the ball image 11-1 of the first processed image, or the ball image 11-2 of the second processed image is By superimposing on the ball image 11-1 of the 1 processing image, the maximum bright point P1 can be displayed on the ball image 11-1 of the 1st processing image.

  As described above, a 360 ° line scan is performed as shown in FIG. 12A around the maximum bright point P1 on the ball image 11-1 of the first processed image.

  FIG. 12B shows a line scan result for the ball image, as shown in FIG.

  As shown in FIG. 12B, the outline L1 by the line scan with respect to the ball image 11-1 of the first processed image has a very irregular shape.

  Therefore, as shown in FIG. 12C, it is preferable to adjust the contour line to be a fitted contour line L2 by using an averaging filter.

  As shown in FIG. 12C, it can be seen from the result of the line scan on the ball image that the point having the shortest distance Dmin and the point having the longest distance Dmax are clearly shown.

  Here, in the result of the line scan, it is preferable to select the point P2 that forms the shortest distance Dmin as the second point. The reason will be described with reference to FIG.

  As shown in FIG. 13, the ball 11 shown by the moving ball being photographed under strobe illumination has a portion 11-1 photographed by receiving the illumination light and a portion R <b> 2 that is not shaded. Even in the portion 11-1 taken on the image, there is a portion R 1 that appears a little brighter due to the influence of illumination, and there is a portion R 3 that has the highest pixel luminance value among the portions R 1 that appear brighter due to the illumination.

  It can be seen that the strobe illumination direction is the brightest portion on the ball image, that is, the maximum bright point P1 having the highest luminance value, which is the center of the axis AX of the illumination direction.

  Also, as shown in FIG. 13, the axis AX in the illumination direction must be a point on the contour line that forms the shortest distance from the maximum bright point P1 to the contour of the ball image 11-1, that is, the second point P2. Can be estimated.

  Therefore, the point closest to the maximum bright point P1 among the points on the outline of the ball image 11-1 can be designated as the second point P2.

  The second point P2 can be determined as the point P2 on the contour line when the shortest distance Dmin is formed by the line scanning process shown in FIGS.

  Therefore, a line connecting the first point P1 which is the maximum bright point and the second point P2 obtained by the above-described method can be set as the axis AX in the illumination direction.

  FIG. 14 shows the first point P1 and the second point P2 on the ball image 11-1 in the state shown in FIG. 12B, and the illumination direction axis AX connecting the two points. It is an image which shows the extracted state.

  On the other hand, as shown in FIG. 14, after extracting the axis AX of the illumination direction on the ball image, the image processing means 260 (see FIG. 1) causes the ball image as shown in FIGS. Fitting and extraction of center point coordinates.

  Here, in order to improve the accuracy of the fitting with respect to the ball image, it is preferable to perform the fitting using the first processing image 11-1 rather than the fitting using the second processing image 11-2.

  This is because the first processed image 11-1 can extract the center point more accurately at the time of fitting because the shape of the ball image is not greatly damaged.

  On the other hand, for fitting to a ball image, a sensing processing apparatus according to an embodiment of the present invention includes a diameter measuring unit to measure the diameter of the ball image.

  The diameter value for fitting the ball image may be a value set in advance with respect to the value of the diameter of the ball. You may perform fitting by the value of a diameter.

  However, since the ball image can be damaged to some extent due to the effects of shaded parts on the initially acquired image, resolution limitations, and multiple stages of pre-processing, Rather than fitting using a set diameter value, it is preferable to measure the diameter value with a ball image and use it for fitting in order to extract the center point more accurately.

  FIG. 15 shows a process of measuring the diameter value in the ball image 11-1 by the diameter measuring means.

  First, in order to easily measure the diameter, as shown in FIG. 15, the illumination direction axis AX is vertical, the illumination area on the ball image is on the upper side, and the shade area is on the lower side. It is preferable to be on the side.

  Here, as shown in FIG. 15, by performing a line scan along the illumination direction axis AX through the horizontal line L, the longest section among the sections in which the change in the pixel value changes beyond the set value is represented by the diameter De. Can be extracted as

  As described above, when the measurement of the diameter is completed by the diameter measuring means, the fitting of the ball image is performed by the semicircular fitting means 270 (see FIG. 1), and FIGS. Fitting is performed by the process as shown in FIG.

  When the semicircle fitting means is illuminated and a shade area exists on the ball image, the fitting is performed so that the shade area is included, and the coordinates of the center point can be accurately extracted.

  More specifically, the semicircular fitting means includes a pixel check means for checking a pixel value along a direction estimated as an illumination direction in the ball image, and the checked pixel value is equal to or greater than a preset set value. It is preferable to include a fitting processing means for forming the semicircular curve at a position where the semicircular curve is changed and fitting the opposite semicircle.

  Here, as shown in FIG. 14, when the axis AX of the illumination direction based on the estimated illumination direction is inclined at an arbitrary angle, the semicircular fitting can be performed more easily. As shown in FIG. 16, it is preferable to rotate the entire image so that the axis AX in the illumination direction is vertical.

  For the semicircle fitting, first, as shown in FIG. 16, a semicircle curve FC-1 is prepared based on the diameter De measured by the diameter measuring means.

  Then, as shown in FIG. 16, the semicircular curve FC-1 is moved toward the ball image 11 along the axis AX of the illumination direction in a vertical state.

  At this time, the pixel check means checks the pixel value (that is, the luminance value) of the image on the semicircular curve FC-1. More preferably, the sum of the luminance values of the image on the semicircular curve FC-1 is calculated along the axis AX in the illumination direction so as to sense the change.

  While the semicircular curve FC-1 moves along the axis AX in the illumination direction, the luminance value or the sum of the luminance values of the images on the semicircular curve is calculated, and the luminance is reached when reaching the outer contour side of the upper end of the ball image. Value or the sum of luminance values on a semi-circular curve increases rapidly.

  In this way, the position where the luminance value checked along the axis AX in the illumination direction or the sum of the luminance values on the semicircular curve starts to change abruptly can be determined as the outline of the ball image 11, and the fitting The processing means arranges the semicircular curve FC-1 at the position.

  That is, as shown in FIG. 17A, the semicircular curve FC-1 is placed on the outline of the region R1 that receives the illumination of the ball image 11.

  Then, as shown in FIG. 17B, the fitting processing means includes the shade region R2 of the ball image 11 by fitting the opposite semicircle FC-2 with the measured diameter De. Accurate fitting can be performed.

  When the image fitting is performed through the above-described process, the coordinate extracting unit 280 (see FIG. 1), as shown in FIG. 18, the center point CP of the ball image 11 through the fitting curve FC for each ball image 11. Can be accurately extracted.

  Therefore, it is possible to accurately extract the coordinates of the center point with respect to the ball in spite of using the low-quality image acquired by the low-speed camera device and the strobe device with low resolution.

  The coordinates of the center point of each ball extracted in this way are converted into three-dimensional coordinates by converting means, and based on this, physical information on the motion trajectory of the simulated ball is calculated.

  On the other hand, with reference to FIGS. 19 and 20, each embodiment of the sensing processing method for a moving ball according to the present invention will be described.

  As shown in FIG. 19, first, a trigger signal is applied by the signal generation unit, and the camera device and the strobe device respectively acquire multiple exposure images regarding the motion state of the ball at a predetermined cycle (S10).

  Various image preprocessing steps are performed to remove the background image and various noises from the acquired multiple exposure image (S20).

  A ball image is extracted from the preprocessed image (S30) (Ball image is extracted, but noise may be extracted together. More strictly speaking, it should be regarded as extracting a ball image candidate. The pixel value of the extracted ball image, that is, the luminance value is checked (S40), and the first point having the highest luminance value is extracted (S50).

  A 360 ° line scan is performed on the image around the extracted first point (S60), and an average filter is applied to the contour line of the ball image in the extracted line scan result. The outline is made smooth (S70).

  Of the lines connecting the first point and the outline of the ball image, a point that meets the outline of the line having the shortest distance is extracted as a second point (S80).

  By forming a line passing through the first point and the second point, the corresponding line can be set as an axis in the illumination direction (S90).

  However, the angle between the illumination direction axis set by the method described above and the vertical line or the horizontal line may deviate from the set range.

  That is, in the above, in order to extract the second point, it is assumed that the axis of the illumination direction passes through the point that forms the shortest distance between the maximum bright point and the outline, but the angle of the illumination direction is a vertical line. If the angle becomes larger than a certain range, the above assumption may not be satisfied.

  In such a case, it is necessary to adjust the angle of the axis of the illumination direction through a predetermined process.

  Therefore, it is determined whether or not the angle formed by the illumination direction axis is included in the setting range (S100). If the angle is included in the setting range, the corresponding illumination direction axis becomes a preferred axis. To S80, the illumination direction estimated in the process of S80 is corrected, and the corrected axis is set as the axis of the illumination direction (S110).

  As described above, when the axis of the illumination direction is finally determined, the ball image is fitted in consideration of the illumination direction estimated as described above (S120), and the center point of the fitted image is extracted (S120). S130).

  On the other hand, the flowchart of the sensing processing method according to the embodiment shown in FIG. 20 extracts the center point of the ball by using the images generated for each of the preprocessing steps described in FIGS. 8 to 11 separately. It shows the process.

  The sensing processing method according to the present embodiment is basically the same as the flowchart according to the embodiment shown in FIG. 19, in which an image is generated separately for each image preprocessing step, and steps such as illumination direction estimation are performed from this. Since there is a difference in the point where the process proceeds, the description regarding the overlapping part is omitted, and the part with the difference will be described intensively.

  A pre-processing process for the acquired multiple exposure image proceeds (S20), and a first processed image and a second processed image are generated according to the level of image pre-processing (S21, S22).

  Here, items related to the first processing image are shown in FIG. 9, and items related to the second processing image are shown in FIG.

  A ball image is extracted from the second processed image (S30), a pixel value of the extracted ball image, that is, a luminance value is checked (S40), and a first point having the highest luminance value is extracted (S50).

  Meanwhile, the extracted first point is applied to the first processed image, and a 360 ° line scan is performed on the ball image of the first processed image around the first point (S60). In the scan result, an averaging filter is applied to the outline of the ball image (S70), and the outline of the line that forms the shortest distance among the lines connecting the first point and the outline of the ball image. The point that meets the line is extracted as the second point (S80), and can be set as the axis of the illumination direction connecting the two points (S90).

  Since the subsequent flow is the same as that shown in FIG. 19, a detailed description thereof will be omitted.

  On the other hand, FIG. 21 is a flowchart showing in more detail the step S120 shown in FIGS. 19 and 20, that is, the step of fitting the ball image in consideration of the illumination direction, as shown in FIG. For fitting, it is preferable to first measure the diameter of the ball image (S123).

  Here, the diameter of the ball image can be measured by the method illustrated in FIG. 12 in a state where the ball image is rotated so that the axis of the illumination direction is vertical.

  On the other hand, a semicircular curve is generated based on the measured diameter (S124), the pixel value (luminance value) is checked along the axis of the illumination direction, and the semicircular curve is encircled by the portion of the ball image that is illuminated. (S125).

  A ball image is fitted by forming a remaining semicircle on the opposite side of the semicircle curve based on the measured diameter (S126), and each of the ball images fitted by the above-described method is used. The coordinates of the center point are extracted (S130).

By calculating the change in the coordinates of the center point of the extracted ball image, the physical characteristic information of the moving ball can be extracted.
[Item 2]
The illumination direction estimating means is configured to estimate a direction of illumination with respect to the moving ball by extracting a maximum bright point of the image of the ball,
The sensing processing device according to claim 1, wherein the image processing means is configured to fit on the ball image including a shade region opposite to the maximum bright point.
[Item 4]
The luminance analysis means includes
The first point may be calculated by checking a pixel having the highest luminance value among all the pixels on the ball image and calculating a centroid for the checked at least one pixel. Item 4. The sensing processing device according to item 3.
[Item 5]
The scanning means includes
The sensing processing device according to claim 3, wherein the second point is extracted by performing a 360-degree line scan around the first point of the ball image.
[Item 7]
The image processing means includes
A semicircular curve having a preset diameter is formed at a specific position of the ball image with reference to the illumination direction estimated by the illumination direction estimating means, and a semicircle on the opposite side of the semicircle curve is set to the diameter. Item 4. The sensing processing device according to Item 1, further comprising semicircular fitting means for fitting based on the fitting.
[Item 10]
Diameter measuring means for analyzing the ball image and measuring the diameter of the ball image;
The sensing processing device according to item 8, wherein the semicircular fitting means is configured to perform semicircular fitting based on the measured diameter.
[Item 11]
The semicircular fitting means includes
Pixel checking means for checking pixel values along a direction estimated as a direction of illumination received by the moving ball in the ball image;
Fitting processing means for forming the semicircle curve at a position where the checked pixel value changes to a preset value or more and fitting the opposite semicircle, 8. The sensing processing device according to 8.
[Item 16]
Estimating the direction of the illumination comprises:
Determining whether an angle formed by the axis of the estimated illumination direction deviates from a set angle range;
The sensing processing method according to claim 13, further comprising a step of correcting the estimated illumination direction when the set angle range is deviated.
[Item 18]
The step of forming the semicircular curve includes:
Analyzing the ball image and extracting a shade region on the ball image generated by illumination;
The sensing processing method according to claim 17, further comprising: forming the semicircular curve on the opposite side of the shade region on the ball image.
[Item 22]
Measuring the diameter of the ball image comprises:
Determining whether the length estimated as the diameter in the step of estimating the diameter of the ball image and a preset error of the diameter of the ball are included in the setting range;
The sensing processing method according to claim 21, further comprising a step of setting the estimated length as a measured diameter when the error is included in a setting range.
[Item 23]
Measuring the diameter of the ball image comprises:
Analyzing the degree of shade by the illumination in the ball image, and correcting the length estimated as the diameter in the step of estimating the diameter of the ball image according to the analysis result for the degree of shade. The sensing processing method according to item 21.

Claims (14)

In a sensing processing device that acquires and analyzes an image of a ball moving under predetermined lighting,
A ball image is extracted from the acquired image, a point having the highest luminance value of the extracted ball image is extracted, and a direction of illumination with respect to the moving ball is determined based on the point having the highest luminance value. A lighting direction estimating means for estimating;
By fitting to the shape of the ball with reference to the side receiving the illumination of the ball image along the estimated illumination direction, a shade region on the opposite side of the point having the highest luminance value on the ball image is included in the fitting. Image processing means for extracting the coordinates of the center point of the fitted image so as to be included . The illumination direction estimating means includes
Luminance analysis means for extracting a first point which is a point having the highest luminance value among all the pixels on the ball image;
Scanning means for extracting a second point that is a point that forms the shortest distance from the first point to the outline of the ball image;
The sensing processing apparatus according to claim 1, wherein a line connecting the first point and the second point is configured as an axis of the illumination direction. Preprocessing is performed so as to remove background and noise from the acquired image, and a first processing image and a second processing image on which higher level preprocessing than the first processing image is performed according to the degree of preprocessing. Further comprising pre-processing means each configured to generate,
The illumination direction estimating means includes
Out of all the pixels on the second processed image, luminance analysis means for extracting a first point which is the highest luminance value, and a position corresponding to the first point on the first processed image. Scanning means for scanning the shortest distance to the outline of the first processed image and extracting a second point that is an outline point on the corresponding distance;
The sensing processing device according to claim 1, wherein a line connecting the first point and the second point is set as an axis of the illumination direction. The specific position of the ball image, to form a semi-circular curve of the measured diameter through preset or the ball image, semicircle fitting based semicircle opposite to said diameter to the semicircular curve Fitting means;
The sensing processing apparatus according to claim 1, further comprising: a coordinate extracting unit that extracts coordinates of a center point of the semicircle-fitted ball image. The semicircular fitting means includes
The semicircular curve is formed in an outline on the maximum bright point side of the ball image, and fitting is performed including a shade region on the ball image by fitting the semicircle on the opposite side. The sensing processing device according to claim 4. In a sensing processing method for acquiring and analyzing an image of a ball moving under a predetermined illumination,
A ball image is extracted from the acquired image, a point having the highest luminance value of the extracted ball image is extracted, and a direction of illumination with respect to the moving ball is determined based on the point having the highest luminance value. Estimating, and
By fitting to the shape of the ball along the estimated illumination direction with the side receiving the illumination of the ball image as a reference, a shade region on the opposite side of the point having the highest luminance value on the ball image is included in the fitting. Steps to be included ,
Extracting the coordinates of the center point of the fitted image. Estimating the direction of the illumination comprises:
Extracting a first point that is a point having the highest luminance value among all the pixels on the ball image;
Extracting a second point which is a point having the shortest distance from the first point to the outline of the ball image;
The sensing processing method according to claim 6 , further comprising: setting a line connecting the first point and the second point as an axis of an illumination direction with respect to the ball image. Extracting the first point comprises:
Checking the pixel having the highest luminance value among all the pixels on the ball image;
The sensing processing method according to claim 7, further comprising: calculating the first point by calculating a centroid for the checked at least one pixel. Extracting the second point comprises:
Performing a 360 ° line scan around the first point with respect to the ball image;
The method according to claim 7, further comprising: extracting, as the second point, a point corresponding to the shortest distance connecting the first point and the outline of the ball image. . The fitting step includes
Forming a semi-circular curve of a diameter set in advance or measured through the ball image at a specific position of the ball image with respect to the estimated illumination direction;
The method according to claim 6, further comprising: fitting a semicircle on the opposite side to the semicircular curve based on the diameter. The step of forming the semicircular curve includes:
Extracting an axis of illumination direction based on the estimated illumination direction;
Rotating the ball image so that the illumination direction axis is vertical;
Checking the sum of pixel values on the semi-circular curve while moving the semi-circular curve along the illumination direction axis to the ball image;
The sensing processing method according to claim 10, further comprising: positioning the semicircular curve at a position where a sum of the pixel values to be checked changes to a set value or more. The step of forming the semicircular curve includes:
Analyzing the ball image to measure a diameter of the ball image, and forming a semicircular curve based on the measured diameter;
The fitting step includes
11. The sensing processing method according to claim 10, wherein a semicircle opposite to the formed semicircular curve is fitted based on the measured diameter. Measuring the diameter of the ball image comprises:
Extracting an axis of illumination direction based on the estimated illumination direction;
Rotating the ball image so that the illumination direction axis is vertical;
Performing a horizontal line scan on the ball image along the vertically rotated illumination direction axis;
Estimating the length between the two points of the horizontal line when the length connecting the two points meeting the outline of the ball image is the longest as the diameter of the ball image. The sensing processing method according to claim 12, wherein the sensing processing method is characterized. A sensor unit including a camera device and a strobe lighting device for acquiring a multiple exposure image of a golf ball that is struck and moved by a golfer;
An image of a golf ball is extracted from the acquired image, a point with the highest luminance value of the extracted ball image is extracted, and the strobe illumination for the golf ball is based on the point with the highest luminance value illuminating direction estimating means for estimating the direction of the device, and by fitting to the shape of the golf ball relative to the side receiving the illumination of said estimated the golf ball image along the illumination direction, the brightest on the ball image A sensing processing unit including image processing means for extracting a coordinate of a center point of the fitted image so that a shade region opposite to a point having a high value is included in the fitting;
A virtual golf simulation apparatus comprising: a simulator that calculates physical information about a moving golf ball by simulating a change in coordinates of the center point and simulates a trajectory of the golf ball.