GB2357207A - Ball tracking and trajectory prediction - Google Patents

Abstract

A video processing system uses at two cameras to determine the 3D position of a ball in a series of frames. The future trajectory of the ball is calculated. The system uses at least four cameras so as to allow some redundancy in the system. The established trajectory is used to establish the likely occurrence of an event, principally for lbw determination in cricket. The location of the ball is preferably established by an algorithm that does not rely upon the colour of the ball. A statistics generator may be used to record data relating to the ball track to be used in the decision making process. The field of view of each camera may be minimised so as to maximise the resolution of the ball. The system may be automatic or manual, have active/idle modes and produce a virtual replay of the ball's trajectory.

Description

2357207 1 Improvements in or relating to video processor systems.

This invention relates to video processor systems suitable for tracking a ball during ball games and more especially but not exclusively it relates to such systems for use in cricket.

Due to the increasing professionalism of sport in general and ball games in particular, and the availability of appropriate technology, there is increasing interest in applying electronics technology to provide information of interest to spectators and to minimise the risk of human error affecting results. In tennis for example, service speed is now quite commonly measured and displayed to spectators, and electronic sensors are used to aid Umpires/Linesmen when making net-cord decisions and to determine whether services are within the area prescribed.

However, although such systems as presently applied to tennis for net-cord and service decisions might be further developed to include video tracking to determine where a tennis ball pitches at all times during a game, video processing to track a ball for general ball game applications and for cricket applications in particular, is much more difficult because of potential problems which arise.

In connection with cricket these potential problems include the following: the distance between a video camera positioned beyond the boundary and the wicket is variable depending on the wicket position and the particular cricket ground; the ball must be identified in 3D space with high accuracy throughout its entire flight path; in order to achieve the accuracy required to assist umpiring decisions, the field of view of video cameras used needs

2 to be as small as possible; players may come between the ball and a camera so as to obscure the ball; the image of the ball may be confused with other objects of similar size such as for example a flying bird; the colour of the ball, which may vary, might adversely affect ball sensing; the leg stump position, which is critical for LBW (Leg Before Wicket) decisions and which is different for left and right handed batsmen must be identified and the point where a ball hits a batsman also needs to be identified; a prediction of where a ball would have passed the stumps must be made; and processing which takes the forgoing requirements into account must be done in real time.

It is an object of the present invention to provide a video processing system for use in cricket which will operate to satisfy the forgoing requirements to a degree acceptable to cricket control authorities.

According to the present invention, a video processing system for use in ball games played within a predetermined area on a pitch or the like, comprises at least four video cameras arranged in spaced apart relationship at fixed positions around the said area, a video processor and ball tracker to which signals from the cameras are fed, a data store for data which models the said area and includes data representative of characteristic features positioned thereon for use in performance of the game, and a store for data appertaining to rules and/or key events of the ball game played, the video processor being operative to; (a) identify in each frame, from each camera, groups of pixels corresponding to the image of a ball; (b) compute for each frame the 3D position of an image thus identified using ball image data from at least two different cameras; (c) predict a ball flight-path 3 from the said 31) ball position as computed in successive frames; and, (d) map the predicted flight-path on the modelled area so as to identify any interaction with one or more of the said characteristic features, which interaction signifies the occurrence of a key event or a rule infringement.

The system may be used in connection with cricket wherein the said area is a cricket pitch, the said characteristic features include the position in 31) of the stumps, and the position of the wickets/creases and the said key events/rules include an LBW definition.

The said key events may also include the stumps/bails being struck by the ball for example.

Thus it will be appreciated that the system may be used in cricket to determine whether or not a player is out LBW for example.

When applied to cricket, six or more cameras may be used which are positioned around a cricket ground and positioned high in the cricket stands so that a camera is less likely to be obscured by fielders on the pitch.

In order to provide additional data, a statistics generator may be included which records and stores data appertaining to ball track, ball size, and ball velocity for example which may be used in decision making processes in the processor and/or to provide information concerning ball delivery of interest to TV cricket commentators for example.

Each camera's field of view may be minimised so that the size of the ball within the field is as large as possible without occasions occurring when the ball moves outside the field of view.

4 One embodiment of the invention will now be described by way of example only with reference to the accompanying drawings, wherein corresponding parts of the various Figures bear the same numerical designations and in which; Figure 1. is a generally schematic diagram of a video processor system for use in cricket; Figure l a. is a block schematic diagram (shown inset in Figure 1) of a CPU which forms a part of the system as shown in Figure 1; Figure 2 is a somewhat schematic block diagram of a six camera system which shows the most suitable positions for the cameras and; Figure 3 is a flow diagram which shows the processing steps performed by the CPU as shown in Figure la.

Referring now to the drawings, a video processing system as shown in Figure 1, for use in cricket comprises cameras 1 to 6, as shown in Figure 2, disposed in spaced apart relationship around the boundary 7, of a cricket field 8, so as to view a pitch 9, which has a two sets of stumps 10 and bails 11 which define wickets 12 spaced 22 yds apart, (only one of which is shown in detail in Figure 1).

Video signals from the cameras 1 to 6 are fed to a CPU 13, which conveniently forms a part of a PC (Personal Computer) which also includes a monitor 14. As shown in Figure l a, the CPU comprises a video processor and track generator 15, a store 16, in which data appertaining to the pitch and wicket positions is stored, a store 17, for data appertaining to cricket law/rules relating to LBW for example, and a statistics generator/store 18 for historically based data which may be drawn upon to facilitate current processing in the CPU 13, and /or to provide data of interest to TV commentators for example.

In operation of the system, video signals from the cameras I to 6, are processed as indicated by steps 20 to 26 shown in Figure 3. After camera calibration, step 20, and start processing, step 21, in which the stores 16, and 17, are loaded, signals from the cameras 1 to 6, are initially fed to the video processor and track generator 15, of the CPU 13, wherein the pixels of each frame from each camera are examined to perform a ball recognition function, step 22, in dependence upon algorithm processing, step 23, so as to identify pixel groups which correspond in size and shape to the image of a cricket ball 19, as shown schematically in Figure 1.

Having identified the 3D position, step 24, of the ball 19, a flight path or tracking function, step 25, is performed and a track is predicted, step 26, from the data received. This predicted track, is mapped on to a model of the pitch which is contained in the store 16, together with characteristic features of the pitch such as the positions of the wickets 12.

By calculating a predicted track (just ahead of the actual track) which is continuously corrected as necessary, so that it corresponds very closely to the actual track, temporary loss of the image for short periods, due to obstructions for example, can be tolerated and do not interfere with the integrity of the system.

Having mapped the ball track on to the pitch/wickets, any interactions therebetween which correspond to a law infringement or key event, such as an LBW event for example, as defined in accordance with data in the store 17, are identified and an appropriate indication is provided on the monitor 14.

6 In to order provide statistical data which might be made available to TV commentators, the statistics generator 18, is arranged to store data appertaining the previous ball tracks. This could include data relating to ball velocity, line and length, and swing, as will herein after be explained and this data may also be used selectively to facilitate tracking by restricting pixel examination to those regions of the display through which the ball is most likely to travel thereby to speedup processing.

It will be appreciated that before using the system a setting-up procedure will be required. Once the position and field of view has been set for each camera, anchor points can be established in the field of view which may include the cricket stumps as the main anchor points and the establishment of a reference line extending between the middle stump at each end of the cricket wicket which can be used for fine calibration of the video images.

To save computational expense, the cameras 1 to 6, may be arranged to operate in active or in idle mode. The active mode is initiated by a camera recognising either automatically from player position and bowler movement, or by the injection of a manually produced signal, that a bowler is about to bowl, the active mode being arranged to remain active for a set period of time. This mode may be manually overridden. It may be arranged that all cameras are triggered into active mode at the same time and operate synchronously.

Front-on cameras are used to automatically identify if a batsman is left or right handed, as this is important for LBW decisions and this decision may be manually overridden.

7 As already explained, for each frame of each camera, the cricket ball is identified in the video image and this is done by pixel analysis from a knowledge of the shape and size of the ball and involves image processing techniques which are so well known to those skilled in the art that detailed description herein is believed to be unnecessary.

It is important to note that colour is not used to aid identification since cricket balls may be of different colours.

As already mentioned, the accuracy with which the ball can be identified is improved by using knowledge of where the ball was in previous frames, and the direction in which the ball is travelling, which may be determined by statistical analysis in the statistics generator 18. The position of the sun may also be used to ensure that the ball and the ball's shadow are not confused. This identification process provides X and Y co-ordinates of the ball for each image frame of each camera. The process will return 'not found' if it is not sure that a ball can be identified. The use of six cameras enables some redundancy should any one camera be obscured at any one time.

Given the X and Y co-ordinates of the ball from the various cameras, an algorithm is used to establish the ball position in 3 D.

It will be appreciated that provided data from at least two cameras is available the position of the ball can be established in 3D.

When a 3 D position of the ball has been calculated for each camera frame, this information is compiled to generate a predicted 31) track of the ball. Of particular interest within this track are key points such as a point where the ball pitches, or a point where the ball hits a batsman which may be identified by 8 mapping the predicted track onto a stored map of the pitch as hereinbefore explained. It will be appreciated that one or other of these points may or may not occur and that side-on cameras are best placed to identify the exact point at which the ball hits a batsman.

Given knowledge of a path which the ball has followed up to a point of contact with the batsman, the position w here the ball struck the batsman and other information such as the speed of the ball, and gravity parameters. This information may be used to predict where the ball would have passed by the stumps and a probabilistic prediction may be made as to whether the ball would have hit the stumps. The distance between the ball pitching and hitting the batsman, and the distance between the point of contact and the stumps both affect this probabilistic prediction.

Although the invention may find application in various ball games such as baseball for example, it is especially suitable for application to cricket and in this connection may be used in a variety of ways as will now be described.

Firstly, the credibility of Umpires is greatly reduced if TV replays show that they have made a mistake. TV replays are however too slow to be used to help an Umpire make an LBW decision. The present system however is able to generate information in real time which could be made available to an Umpire to help him make a correct LBW decision for example.

Three vitaf pieces of information could for example be made available to the Urnire, namely, whether the ball pitched outside leg stump, whether the ball hit the batsman in line with the stumps, and whether the probability of the ball going on to hit the stumps is high e.g. above a given probability threshold.

9 As already suggested, the system may be used to provide TV commentators with statistical data. Use of statistics in sports television coverage has increased greatly over the past 5 years.

Additional statistics could be presented by dividing the cricket wicket into a number of sectors, the percentage/number of deliveries pitching in each sector being presented together with the average number of runs scored from balls pitching in each sector. Viewers may also be interested in the average amount of swing, seam or spin a bowler is able to generate as well as delivery velocity.

This system also lends itself to the provision of virtual replays since ball flight path data is available and thus virtual replays may be produced as seen from any angle. Of particular interest would be replays from the batsman's perspective.

The system may additionally find application in Strategy Planning. Test and county cricket teams already employ video analysts to find strengths and weaknesses in the opposition. This system will greatly facilitate, and add additional value, to an analyst's role. Bowlers will be much better able to decide from which end to bowl, and the most effective delivery for each batsman.

The system may also find application in Cricket Coaching.

The database of information gathered from a video processing system set up at a practice session can greatly help players to learn new skills. Players can experiment with changes to their action, different grips and reverse swing and immediately analyse the effect a change has made to the delivery of the ball.

Additionally the system may be used for training Wor testing cricket Umpires. Because of cost considerations, a video processing system may be employed only at Test level cricket to aid Umpires make correct LBW decisions, Club and even County Umpires might not be afforded this aid. Currently Umpires are selected for matches based on reports given about them from previous matches. The video processing database provides an excellent way to train Umpires to make more correct LBW decisions. It also could provide information to help determine which Umpires should be selected for the more important matches.

Claims (8)

CLAIMS.

1. A video processing system for use in ball games played within a predetermined area on a pitch or the like, comprises at least four video cameras arranged in spaced apart relationship at fixed positions around the said area, a video processor and ball tracker to which signals from the cameras are fed, a data store for data which models the said area and includes data representative of characteristic features positioned thereon for use in performance of the game, and a store for data appertaining to rules and/or key events of the ball game played, the video processor being operative to; (a) identify in each frame, from each camera, groups of pixels corresponding to the image of a ball; (b) compute for each frame the 3D position of an image thus identified using ball image data from at least two different cameras; (c) predict a ball flight-path from the said 3D ball position as computed in successive frames; and, (d) map the predicted flight-path on the modelled area so as to identify any interaction with one or more of the said characteristic features, which interaction signifies the occurrence of a key event or a rule infringement.

2. A system as claimed in Claim 1, use for in connection with cricket, wherein the said area is a cricket pitch, the said characteristic features include the position in 3D of the stumps, and the position of the wickets and the said key event/rules include an LBW definition.

3. A system as claimed in Claim 1, wherein at least six cameras are used.

12

4. A system as claimed in any preceding claim, including a statistics generator which records and stores data appertaining to ball track, which is used in decision making processes in the processor.

5. A system as claimed in any preceding claim, the field of view of each camera is minimised so that the size of the ball within the field is as large as possible without occasions occurring when the ball moves outside the field of view.

6. A system as claimed in any preceding claim, arranged to operate either in an active mode or in an idle mode in dependence upon the requirement for system operation as determined either automatically or manually.

7. A system as claimed in any preceding claim, wherein a stored ball track is used to facilitate the provision of a virtual replay of a clip featuring the track as viewed from a predetermined perspective.

8. A system as hereinbefore described with reference to the accompanying drawings.