WO2013136458A1 - Image correction processing device - Google Patents
Image correction processing device Download PDFInfo
- Publication number
- WO2013136458A1 WO2013136458A1 PCT/JP2012/056497 JP2012056497W WO2013136458A1 WO 2013136458 A1 WO2013136458 A1 WO 2013136458A1 JP 2012056497 W JP2012056497 W JP 2012056497W WO 2013136458 A1 WO2013136458 A1 WO 2013136458A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel data
- correction
- line
- image
- correction target
- Prior art date
Links
- 238000012545 processing Methods 0.000 title claims abstract description 68
- 238000003702 image correction Methods 0.000 title claims abstract description 23
- 238000012937 correction Methods 0.000 claims abstract description 78
- 239000000872 buffer Substances 0.000 claims abstract description 32
- 238000003384 imaging method Methods 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 description 4
- 238000005096 rolling process Methods 0.000 description 4
- 238000003705 background correction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/68—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects
Definitions
- the present invention relates to an image correction processing apparatus that corrects pixel data for one frame (one screen) of an image sensor.
- a problem to be solved by the present invention is to provide an image correction processing apparatus capable of realizing cost reduction and speeding up of image correction processing.
- the present invention provides an image sensor that captures an imaging target, and a plurality of pixels that sequentially read out pixel data for each scanning line of the image sensor and temporarily store pixel data for a plurality of lines.
- a line buffer and a predetermined line of pixel data for a plurality of lines stored in the plurality of line buffers is set as a correction target line, and the correction is performed using pixel data of at least one line adjacent to the correction target line.
- a pixel data correction unit that corrects pixel data of the target line and transfers the corrected pixel data to the image processing circuit, and shifts the scan line to obtain pixel data for a plurality of lines including the next correction target line.
- Processing for storing in the plurality of line buffers, and correcting the pixel data of the correction target line by the pixel data correction unit to perform the image processing Wherein by repeating the process of transferring to the road by sequentially corrects the pixel data of each line of one frame of the image pickup device is obtained by the successively transferred to the image processing circuit.
- a distortion correction amount table in which the relationship between the distortion correction amount and the pixel coordinates is set is stored in the storage unit, and the pixel data correction unit is configured to store the periphery of each pixel of the correction target line.
- each pixel data of the correction target line may be subjected to distortion correction by an interpolation correction method. As a result, the processing for correcting image distortion can be speeded up.
- the image correction processing apparatus of the present invention can be applied to various apparatuses equipped with an image processing function for processing an image captured by an image sensor.
- an image processing function for processing an image captured by an image sensor.
- an industrial machine such as a component mounter, a component mounter or the like It is possible to achieve both cost reduction and high speed processing of an image processing function of an industrial machine.
- FIG. 1 is a block diagram showing an image correction processing apparatus according to an embodiment of the present invention.
- FIG. 2 is a flowchart showing a flow of image correction processing when exposure is performed by a rolling shutter system.
- the camera unit 12 incorporating the image pickup device 11 is replaceably attached to an industrial machine such as a component mounter.
- the camera unit 12 is provided with a non-volatile memory 13 such as a flash memory, ROM, PROM, EPROM, or EEPROM, which is a non-volatile storage means.
- the nonvolatile memory 13 stores one or more image correction parameters among image distortion correction parameters, shading correction parameters, contrast correction parameters, pixel defect correction parameters, and the like.
- the image processing device 14 of an industrial machine such as a component mounter includes an FPGA (Field Programmable Gate Array) 15 that is a programmable logic device, a frame buffer, and the like.
- An image processing circuit 16 including the image processing circuit 16 is provided.
- the nonvolatile memory 13 in which the image correction parameters are stored is provided in the camera unit 12, but this may be provided in the image processing device 14.
- the FPGA 15 includes a plurality of line buffers 17 that sequentially read out pixel data for each scanning line of the image sensor 11 and temporarily store pixel data for a plurality of lines, a pixel data correction processing unit 18 (pixel data correction unit), and the like. Is provided.
- the pixel data correction processing unit 18 sets an intermediate line (n) of the pixel data for a plurality of lines stored in the plurality of line buffers 17 as a correction target line, and is adjacent to both upper and lower sides of the correction target line (n).
- the data is corrected, and the corrected pixel data is transferred to the image processing circuit 16.
- the nonvolatile memory 13 when correcting image distortion, it is stored in advance in the nonvolatile memory 13 as a distortion correction amount table in which the relationship between the distortion correction amount, which is an image distortion correction parameter, and pixel coordinates is set, and a pixel data correction processing unit 18, an interpolation correction method is applied to each pixel data of the correction target line (n) using a plurality of pixel data around each pixel of the correction target line (n) and the distortion correction amount retrieved from the distortion correction amount table. Therefore, the distortion may be corrected.
- the interpolation correction method for example, any one of bilinear interpolation, bicubic interpolation, nearest neighbor interpolation, and the like may be used.
- the FPGA 15 shifts the scanning line to store the pixel data of the plurality of lines (n ⁇ 3) to (n + 3) including the next correction target line in the plurality of line buffers 17 and corrects the pixel data by the pixel data correction processing unit 18.
- the pixel data of each line for one frame of the image sensor 11 is sequentially corrected and the image The data is sequentially transferred to the processing circuit 16.
- the FPGA 15 is provided with a buffer (memory) that temporarily stores the corrected pixel data transferred from the pixel data correction processing unit 18 for a predetermined number of lines, and the buffer is provided with a corrected number of lines after the correction. Each time pixel data is stored, corrected pixel data for a predetermined number of lines may be transferred from the buffer to the image processing circuit 16. In this way, high-speed correction processing can be performed while reducing the number of interruptions to the main CPU of the image processing apparatus 14.
- the electronic shutter of the image sensor 11 may be a rolling shutter that sequentially releases the shutter for each scanning line, or a global shutter that simultaneously releases the shutter for one frame.
- FIG. 2 is a flowchart showing the flow of image correction processing when exposure is performed by the rolling shutter method.
- the exposure is performed by sequentially releasing the shutter for each scanning line by the rolling shutter method, and the pixel data of the scanning line is stored in, for example, the line buffer 17 of the first column (step 101), and the line of the first column at the previous processing is stored.
- the pixel data of the line stored in the buffer 17 is shifted to the line buffer 17 in the second column.
- the pixel data of the line stored in the line buffer 17 in the “N” column during the previous processing is shifted to the line buffer 17 in the “N + 1” column.
- the pixel data of the line stored in the line buffer 17 in the last column at the previous processing is discarded.
- a total of six lines (n ⁇ 3), (n-3) adjacent to the upper and lower sides of the correction target line (n) stored in the intermediate line buffer 17 ( n-2), (n-1), (n + 1), (n + 2), and (n + 3) are used to correct the pixel data of the correction target line (n) by interpolation correction (step 102).
- the corrected pixel data is transferred to the image processing circuit 16 (step 103).
- the pixel data of each line for one frame of the image sensor 11 is sequentially corrected and sequentially transferred to the image processing circuit 16.
- the exposure need only be performed once at the beginning.
- the memory necessary for the image correction processing is stored. The amount can be greatly reduced, the circuit scale of the memory can be reduced, and the cost can be reduced.
- the processing for storing the pixel data for a plurality of lines in the plurality of line buffers 17 and the processing for correcting the pixel data of the correction target line can be performed in parallel, the time required for the image correction processing can be shortened, and image correction The processing speed can be increased.
- the number of line buffers 17 is not limited to 7 columns, and may be 3 columns, 5 columns, and the like. Further, a plurality of correction target lines may be set to correct pixel data of a plurality of correction target lines.
- the present invention is not limited to industrial machines such as component mounting machines, machine tools, printing machines, and inspection machines, and can be applied to various apparatuses equipped with an image processing function for processing an image captured by an image sensor.
- industrial machines such as component mounting machines, machine tools, printing machines, and inspection machines
- image processing function for processing an image captured by an image sensor.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
An image correction processing device is provided in which an FPGA (15) comprises: a plurality of line buffers (17) that sequentially read the pixel data of scan lines from an imaging element (11) and temporarily store the pixel data for a plurality of lines; and a pixel data correction processing unit (18). The pixel data correction processing unit (18): designates a line (n) as a correction target, said line (n) being in the middle of the pixel data for the plurality of lines stored in the plurality of line buffers (17); uses the pixel data for the lines that are adjacent to the correction target line (n) to correct the pixel data for the correction target line (n); and transfers the post-correction pixel data to an image processing circuit (16). This type of processing is performed repeatedly by shifting the scan lines one by one so that the pixel data for each line of a frame from the imaging element (11) is corrected and sent sequentially to the image processing circuit (16).
Description
本発明は、撮像素子の１フレーム分（１画面分）の画素データを補正する画像補正処理装置に関する発明である。
The present invention relates to an image correction processing apparatus that corrects pixel data for one frame (one screen) of an image sensor.
従来より、撮像素子で撮像する画像の品質を向上させるために、画像歪み補正、シェーディング補正、画素欠陥補正等の様々な画像補正機能を搭載した画像処理装置が実用化されている。特に、部品実装機に搭載した画像処理装置では、精密な位置決め精度が要求されるため、画像の歪みが大きな問題となる。従来の画像歪み補正技術は、予め、ドットマークが所定ピッチでマトリックス状に表示された画像歪み補正治具を撮像素子で撮像してドットマーク毎に歪み量を測定して歪み量のテーブルデータを作成して記憶しておき、撮像素子の１フレーム分の画素データを全てフレームバッファに記憶し終えてから、歪み量のテーブルデータを用いて画像の歪みを補正するようにしている（特許文献１，２参照）。
Conventionally, image processing apparatuses equipped with various image correction functions such as image distortion correction, shading correction, and pixel defect correction have been put into practical use in order to improve the quality of an image captured by an image sensor. In particular, in an image processing apparatus mounted on a component mounter, since precise positioning accuracy is required, image distortion becomes a serious problem. In the conventional image distortion correction technology, an image distortion correction jig in which dot marks are displayed in a matrix at a predetermined pitch is imaged with an image sensor, and the distortion amount is measured for each dot mark to obtain table data of the distortion amount. It is created and stored, and after all the pixel data for one frame of the image sensor has been stored in the frame buffer, the distortion of the image is corrected using the distortion amount table data (Patent Document 1). , 2).
しかし、上記従来の構成では、撮像素子の１フレーム分の画素データを全てフレームバッファに記憶し終えてから、画像補正処理を行うようにしているため、画素データを記憶するバッファの回路規模が大きくなり、コスト高になると共に、画像補正処理の高速化にも限界がある。
However, in the above-described conventional configuration, image correction processing is performed after all the pixel data for one frame of the image sensor has been stored in the frame buffer, so the circuit scale of the buffer for storing the pixel data is large. As a result, the cost increases and the speed of image correction processing is limited.
そこで、本発明が解決しようとする課題は、低コスト化と画像補正処理の高速化を実現できる画像補正処理装置を提供することである。
Therefore, a problem to be solved by the present invention is to provide an image correction processing apparatus capable of realizing cost reduction and speeding up of image correction processing.
上記課題を解決するために、本発明は、撮像対象物を撮像する撮像素子と、前記撮像素子の走査ライン毎に画素データを順次読み出して複数ライン分の画素データを一時的に記憶する複数のラインバッファと、前記複数のラインバッファに記憶された複数ライン分の画素データのうちの所定のラインを補正対象ラインとし、該補正対象ラインに隣接する少なくとも１つのラインの画素データを用いて該補正対象ラインの画素データを補正して、補正後の画素データを画像処理回路へ転送する画素データ補正手段とを備え、走査ラインをシフトさせて次の補正対象ラインを含む複数ライン分の画素データを前記複数のラインバッファに記憶する処理と、前記画素データ補正手段により前記補正対象ラインの画素データを補正して前記画像処理回路へ転送する処理とを繰り返すことで前記撮像素子の１フレーム分の各ラインの画素データを順次補正して該画像処理回路へ順次転送するようにしたものである。
In order to solve the above-described problems, the present invention provides an image sensor that captures an imaging target, and a plurality of pixels that sequentially read out pixel data for each scanning line of the image sensor and temporarily store pixel data for a plurality of lines. A line buffer and a predetermined line of pixel data for a plurality of lines stored in the plurality of line buffers is set as a correction target line, and the correction is performed using pixel data of at least one line adjacent to the correction target line. A pixel data correction unit that corrects pixel data of the target line and transfers the corrected pixel data to the image processing circuit, and shifts the scan line to obtain pixel data for a plurality of lines including the next correction target line. Processing for storing in the plurality of line buffers, and correcting the pixel data of the correction target line by the pixel data correction unit to perform the image processing Wherein by repeating the process of transferring to the road by sequentially corrects the pixel data of each line of one frame of the image pickup device is obtained by the successively transferred to the image processing circuit.
この構成では、補正対象ラインの画素データを補正するのに必要な複数ライン分の画素データを複数のラインバッファに記憶するだけであるため、画像補正処理に必要なメモリ量を大幅に削減でき、メモリの回路規模を小さくできて、低コスト化を実現できる。しかも、複数ライン分の画素データを複数のラインバッファに記憶する処理と、補正対象ライ
ンの画素データを補正する処理とを並列処理できるため、画像補正処理に要する時間を短縮できて、画像補正処理の高速化を実現できる。 In this configuration, since only the pixel data for a plurality of lines necessary to correct the pixel data of the correction target line is stored in a plurality of line buffers, the amount of memory required for the image correction process can be greatly reduced, The memory circuit scale can be reduced and the cost can be reduced. Moreover, since the processing for storing pixel data for a plurality of lines in a plurality of line buffers and the processing for correcting the pixel data of the correction target line can be performed in parallel, the time required for the image correction processing can be shortened, and the image correction processing Can be speeded up.
ンの画素データを補正する処理とを並列処理できるため、画像補正処理に要する時間を短縮できて、画像補正処理の高速化を実現できる。 In this configuration, since only the pixel data for a plurality of lines necessary to correct the pixel data of the correction target line is stored in a plurality of line buffers, the amount of memory required for the image correction process can be greatly reduced, The memory circuit scale can be reduced and the cost can be reduced. Moreover, since the processing for storing pixel data for a plurality of lines in a plurality of line buffers and the processing for correcting the pixel data of the correction target line can be performed in parallel, the time required for the image correction processing can be shortened, and the image correction processing Can be speeded up.
更に、画像歪みを補正する場合は、歪み補正量と画素座標との関係を設定した歪み補正量テーブルを記憶手段に記憶しておき、画素データ補正手段において、補正対象ラインの各画素の周辺の複数の画素データと前記歪み補正量テーブルから検索した歪み補正量とを用いて該補正対象ラインの各画素データを補間補正法により歪み補正するようにすれば良い。これにより、画像歪みを補正する処理を高速化できる。
Further, when correcting image distortion, a distortion correction amount table in which the relationship between the distortion correction amount and the pixel coordinates is set is stored in the storage unit, and the pixel data correction unit is configured to store the periphery of each pixel of the correction target line. By using a plurality of pixel data and a distortion correction amount retrieved from the distortion correction amount table, each pixel data of the correction target line may be subjected to distortion correction by an interpolation correction method. As a result, the processing for correcting image distortion can be speeded up.
本発明の画像補正処理装置は、撮像素子で撮像した画像を処理する画像処理機能を搭載した様々な装置に適用でき、例えば、部品実装機等の産業用機械に適用すれば、部品実装機等の産業用機械の画像処理機能の低コスト化と高速処理化とを両立させることができる。
The image correction processing apparatus of the present invention can be applied to various apparatuses equipped with an image processing function for processing an image captured by an image sensor. For example, when applied to an industrial machine such as a component mounter, a component mounter or the like It is possible to achieve both cost reduction and high speed processing of an image processing function of an industrial machine.
以下、本発明を実施するための形態を部品実装機、工作機、印刷機、検査機等の産業用機械の画像処理装置に適用して具体化した一実施例を図面を用いて説明する。
Hereinafter, an embodiment in which a mode for carrying out the present invention is applied to an image processing apparatus of an industrial machine such as a component mounting machine, a machine tool, a printing machine, and an inspection machine will be described with reference to the drawings.
図１に示すように、撮像素子１１を内蔵したカメラユニット１２は、部品実装機等の産業用機械に交換可能に取り付けられる。このカメラユニット１２には、不揮発性の記憶手段であるフラッシュメモリ、ＲＯＭ、ＰＲＯＭ、ＥＰＲＯＭ、ＥＥＰＲＯＭ等の不揮発性メモリ１３が取り付けられている。この不揮発性メモリ１３には、画像歪み補正パラメータ、シェーディング補正パラメータ、コントラスト補正パラメータ、画素欠陥補正パラメータ等のうちの１つ以上の画像補正パラメータが記憶されている。
As shown in FIG. 1, the camera unit 12 incorporating the image pickup device 11 is replaceably attached to an industrial machine such as a component mounter. The camera unit 12 is provided with a non-volatile memory 13 such as a flash memory, ROM, PROM, EPROM, or EEPROM, which is a non-volatile storage means. The nonvolatile memory 13 stores one or more image correction parameters among image distortion correction parameters, shading correction parameters, contrast correction parameters, pixel defect correction parameters, and the like.
一方、部品実装機等の産業用機械の画像処理装置１４には、メインＣＰＵ（図示せず）の他に、プログラム可能なロジックデバイスであるＦＰＧＡ（Field Programmable Gate Array ）１５と、フレームバッファ等を含む画像処理回路１６等が設けられている。尚、図１の構成例では、画像補正パラメータが記憶された不揮発性メモリ１３をカメラユニット１２に設けたが、これを画像処理装置１４に設けた構成としても良い。
On the other hand, in addition to the main CPU (not shown), the image processing device 14 of an industrial machine such as a component mounter includes an FPGA (Field Programmable Gate Array) 15 that is a programmable logic device, a frame buffer, and the like. An image processing circuit 16 including the image processing circuit 16 is provided. In the configuration example of FIG. 1, the nonvolatile memory 13 in which the image correction parameters are stored is provided in the camera unit 12, but this may be provided in the image processing device 14.
ＦＰＧＡ１５には、撮像素子１１の走査ライン毎に画素データを順次読み出して複数ライン分の画素データを一時的に記憶する複数のラインバッファ１７と、画素データ補正処理部１８（画素データ補正手段）とが設けられている。画素データ補正処理部１８は、複数のラインバッファ１７に記憶された複数ライン分の画素データのうちの中間のライン（ｎ）を補正対象ラインとし、該補正対象ライン（ｎ）の上下両側に隣接する合計６本のライン（ｎ－３），（ｎ－２），（ｎ－１），（ｎ＋１），（ｎ＋２），（ｎ＋３）の画素データを用いて該補正対象ライン（ｎ）の画素データを補正して、補正後の画素データを画像処理回路１６へ転送する。
The FPGA 15 includes a plurality of line buffers 17 that sequentially read out pixel data for each scanning line of the image sensor 11 and temporarily store pixel data for a plurality of lines, a pixel data correction processing unit 18 (pixel data correction unit), and the like. Is provided. The pixel data correction processing unit 18 sets an intermediate line (n) of the pixel data for a plurality of lines stored in the plurality of line buffers 17 as a correction target line, and is adjacent to both upper and lower sides of the correction target line (n). Pixels of the correction target line (n) using pixel data of a total of six lines (n-3), (n-2), (n-1), (n + 1), (n + 2), (n + 3) The data is corrected, and the corrected pixel data is transferred to the image processing circuit 16.
例えば、画像歪みを補正する場合は、予め、画像歪み補正パラメータである歪み補正量と画素座標との関係を設定した歪み補正量テーブルとして不揮発性メモリ１３に記憶しておき、画素データ補正処理部１８において、補正対象ライン（ｎ）の各画素の周辺の複数
の画素データと前記歪み補正量テーブルから検索した歪み補正量とを用いて該補正対象ライン（ｎ）の各画素データを補間補正法により歪み補正すれば良い。補間補正法としては、例えば、バイリニア補間、バイキュービック補間、最近傍補間等のいずれかを用いれば良い。 For example, when correcting image distortion, it is stored in advance in thenonvolatile memory 13 as a distortion correction amount table in which the relationship between the distortion correction amount, which is an image distortion correction parameter, and pixel coordinates is set, and a pixel data correction processing unit 18, an interpolation correction method is applied to each pixel data of the correction target line (n) using a plurality of pixel data around each pixel of the correction target line (n) and the distortion correction amount retrieved from the distortion correction amount table. Therefore, the distortion may be corrected. As the interpolation correction method, for example, any one of bilinear interpolation, bicubic interpolation, nearest neighbor interpolation, and the like may be used.
の画素データと前記歪み補正量テーブルから検索した歪み補正量とを用いて該補正対象ライン（ｎ）の各画素データを補間補正法により歪み補正すれば良い。補間補正法としては、例えば、バイリニア補間、バイキュービック補間、最近傍補間等のいずれかを用いれば良い。 For example, when correcting image distortion, it is stored in advance in the
ＦＰＧＡ１５は、走査ラインをシフトさせて次の補正対象ラインを含む複数ライン（ｎ－３）～（ｎ＋３）の画素データを複数のラインバッファ１７に記憶する処理と、画素データ補正処理部１８により補正対象ライン（ｎ）の画素データを補間補正法により補正して画像処理回路１６へ転送する処理とを繰り返すことで、撮像素子１１の１フレーム分の各ラインの画素データを順次補正して該画像処理回路１６へ順次転送する。
The FPGA 15 shifts the scanning line to store the pixel data of the plurality of lines (n−3) to (n + 3) including the next correction target line in the plurality of line buffers 17 and corrects the pixel data by the pixel data correction processing unit 18. By repeating the process of correcting the pixel data of the target line (n) by the interpolation correction method and transferring it to the image processing circuit 16, the pixel data of each line for one frame of the image sensor 11 is sequentially corrected and the image The data is sequentially transferred to the processing circuit 16.
尚、ＦＰＧＡ１５に、画素データ補正処理部１８から転送される補正後の画素データを所定ライン数分だけ一時的に記憶するバッファ（メモリ）を設けて、該バッファに所定ライン数分の補正後の画素データを記憶し終える毎に、該バッファから所定ライン数分の補正後の画素データを画像処理回路１６へ転送するようにしても良い。このようにすれば、画像処理装置１４のメインＣＰＵへの割り込み回数を削減しつつ高速な補正処理を行うことができる。
The FPGA 15 is provided with a buffer (memory) that temporarily stores the corrected pixel data transferred from the pixel data correction processing unit 18 for a predetermined number of lines, and the buffer is provided with a corrected number of lines after the correction. Each time pixel data is stored, corrected pixel data for a predetermined number of lines may be transferred from the buffer to the image processing circuit 16. In this way, high-speed correction processing can be performed while reducing the number of interruptions to the main CPU of the image processing apparatus 14.
撮像素子１１の電子シャッタは、走査ライン毎に順次シャッタを切るローリングシャッタを用いても良いし、１フレーム同時にシャッタを切るグローバルシャッタを用いても良い。
The electronic shutter of the image sensor 11 may be a rolling shutter that sequentially releases the shutter for each scanning line, or a global shutter that simultaneously releases the shutter for one frame.
図２は、ローリングシャッタ方式で露光する場合の画像補正処理の流れを示すフローチャートである。ローリングシャッタ方式で走査ライン毎に順次シャッタを切って露光して、該走査ラインの画素データを、例えば、１列目のラインバッファ１７に記憶し（ステップ１０１）、前回処理時に１列目のラインバッファ１７に記憶されていたラインの画素データを２列目のラインバッファ１７にシフトさせる。前回処理時に「Ｎ」列目のラインバッファ１７に記憶されていたラインの画素データは、「Ｎ＋１」列目のラインバッファ１７にシフトされる。前回処理時に最終列のラインバッファ１７に記憶されていたラインの画素データは、廃棄される。
FIG. 2 is a flowchart showing the flow of image correction processing when exposure is performed by the rolling shutter method. The exposure is performed by sequentially releasing the shutter for each scanning line by the rolling shutter method, and the pixel data of the scanning line is stored in, for example, the line buffer 17 of the first column (step 101), and the line of the first column at the previous processing is stored. The pixel data of the line stored in the buffer 17 is shifted to the line buffer 17 in the second column. The pixel data of the line stored in the line buffer 17 in the “N” column during the previous processing is shifted to the line buffer 17 in the “N + 1” column. The pixel data of the line stored in the line buffer 17 in the last column at the previous processing is discarded.
走査ラインの画素データを１列目のラインバッファ１７に取り込む毎に、中間のラインバッファ１７に記憶された補正対象ライン（ｎ）の上下両側に隣接する合計６つのライン（ｎ－３），（ｎ－２），（ｎ－１），（ｎ＋１），（ｎ＋２），（ｎ＋３）の画素データを用いて該補正対象ライン（ｎ）の画素データを補間補正により補正して（ステップ１０２）、補正後の画素データを画像処理回路１６へ転送する（ステップ１０３）。
Every time the pixel data of the scanning line is taken into the line buffer 17 of the first column, a total of six lines (n−3), (n-3) adjacent to the upper and lower sides of the correction target line (n) stored in the intermediate line buffer 17 ( n-2), (n-1), (n + 1), (n + 2), and (n + 3) are used to correct the pixel data of the correction target line (n) by interpolation correction (step 102). The corrected pixel data is transferred to the image processing circuit 16 (step 103).
以上の処理を走査ラインを１ラインずつシフトさせて繰り返すことで、撮像素子１１の１フレーム分の各ラインの画素データを順次補正して該画像処理回路１６へ順次転送する。
By repeating the above processing by shifting the scanning line by one line, the pixel data of each line for one frame of the image sensor 11 is sequentially corrected and sequentially transferred to the image processing circuit 16.
尚、グローバルシャッタ方式で露光する場合は、露光は最初に１回行うだけで良い。
以上説明した本実施例によれば、補正対象ラインの画素データを補正するのに必要な複数ライン分の画素データを複数のラインバッファ１７に記憶するだけであるため、画像補正処理に必要なメモリ量を大幅に削減でき、メモリの回路規模を小さくできて、低コスト化を実現できる。しかも、複数ライン分の画素データを複数のラインバッファ１７に記憶する処理と、補正対象ラインの画素データを補正する処理とを並列処理できるため、画像補正処理に要する時間を短縮できて、画像補正処理の高速化を実現できる。 When exposure is performed using the global shutter method, the exposure need only be performed once at the beginning.
According to the present embodiment described above, since the pixel data for a plurality of lines necessary for correcting the pixel data of the correction target line is only stored in the plurality ofline buffers 17, the memory necessary for the image correction processing is stored. The amount can be greatly reduced, the circuit scale of the memory can be reduced, and the cost can be reduced. In addition, since the processing for storing the pixel data for a plurality of lines in the plurality of line buffers 17 and the processing for correcting the pixel data of the correction target line can be performed in parallel, the time required for the image correction processing can be shortened, and image correction The processing speed can be increased.
以上説明した本実施例によれば、補正対象ラインの画素データを補正するのに必要な複数ライン分の画素データを複数のラインバッファ１７に記憶するだけであるため、画像補正処理に必要なメモリ量を大幅に削減でき、メモリの回路規模を小さくできて、低コスト化を実現できる。しかも、複数ライン分の画素データを複数のラインバッファ１７に記憶する処理と、補正対象ラインの画素データを補正する処理とを並列処理できるため、画像補正処理に要する時間を短縮できて、画像補正処理の高速化を実現できる。 When exposure is performed using the global shutter method, the exposure need only be performed once at the beginning.
According to the present embodiment described above, since the pixel data for a plurality of lines necessary for correcting the pixel data of the correction target line is only stored in the plurality of
尚、ラインバッファ１７の数は７列に限定されず、例えば３列、５列等であっても良い
。また、補正対象ラインを複数設定して、複数の補正対象ラインの画素データを補正するようにしても良い。 The number ofline buffers 17 is not limited to 7 columns, and may be 3 columns, 5 columns, and the like. Further, a plurality of correction target lines may be set to correct pixel data of a plurality of correction target lines.
。また、補正対象ラインを複数設定して、複数の補正対象ラインの画素データを補正するようにしても良い。 The number of
その他、本発明は、部品実装機、工作機、印刷機、検査機等の産業用機械に限定されず、撮像素子で撮像した画像を処理する画像処理機能を搭載した様々な装置に適用できる等、要旨を逸脱しない範囲内で種々変更して実施できる。
In addition, the present invention is not limited to industrial machines such as component mounting machines, machine tools, printing machines, and inspection machines, and can be applied to various apparatuses equipped with an image processing function for processing an image captured by an image sensor. Various modifications can be made without departing from the scope of the invention.
１１…撮像素子、１２…カメラユニット、１３…不揮発性メモリ（記憶手段）、１４…画像処理装置、１５…ＦＰＧＡ、１６…画像処理回路、１７…ラインバッファ、１８…画素データ補正処理部（画素データ補正手段）
DESCRIPTION OF SYMBOLS 11 ... Image sensor, 12 ... Camera unit, 13 ... Non-volatile memory (memory | storage means), 14 ... Image processing apparatus, 15 ... FPGA, 16 ... Image processing circuit, 17 ... Line buffer, 18 ... Pixel data correction process part (pixel) Data correction means)
Claims (3)
- 撮像対象物を撮像する撮像素子と、
前記撮像素子の走査ライン毎に画素データを順次読み出して複数ライン分の画素データを一時的に記憶する複数のラインバッファと、
前記複数のラインバッファに記憶された複数ライン分の画素データのうちの所定のラインを補正対象ラインとし、該補正対象ラインに隣接する少なくとも１つのラインの画素データを用いて該補正対象ラインの画素データを補正して、補正後の画素データを画像処理回路へ転送する画素データ補正手段とを備え、
走査ラインをシフトさせて次の補正対象ラインを含む複数ライン分の画素データを前記複数のラインバッファに記憶する処理と、前記画素データ補正手段により前記補正対象ラインの画素データを補正して前記画像処理回路へ転送する処理とを繰り返すことで前記撮像素子の１フレーム分の各ラインの画素データを順次補正して該画像処理回路へ順次転送することを特徴とする画像補正処理装置。 An image sensor for imaging an imaging object;
A plurality of line buffers for sequentially reading out pixel data for each scanning line of the image sensor and temporarily storing pixel data for a plurality of lines;
A predetermined line of pixel data for a plurality of lines stored in the plurality of line buffers is set as a correction target line, and pixels of the correction target line are used by using pixel data of at least one line adjacent to the correction target line. Pixel data correction means for correcting the data and transferring the corrected pixel data to the image processing circuit,
A process of shifting a scanning line and storing pixel data for a plurality of lines including the next correction target line in the plurality of line buffers, and correcting the pixel data of the correction target line by the pixel data correction unit, and the image An image correction processing apparatus characterized by repeating the process of transferring to a processing circuit to sequentially correct pixel data of each line for one frame of the image sensor and sequentially transferring the pixel data to the image processing circuit. - 画像歪みを補正するための歪み補正量と画素座標との関係を設定した歪み補正量テーブルを記憶した記憶手段を備え、
前記画素データ補正手段は、前記補正対象ラインの各画素の周辺の複数の画素データと
前記歪み補正量テーブルから検索した歪み補正量とを用いて該補正対象ラインの各画素データを補間補正法により歪み補正することを特徴とする請求項１に記載の画像補正処理装置。 A storage unit that stores a distortion correction amount table in which a relationship between a distortion correction amount and pixel coordinates for correcting image distortion is stored;
The pixel data correction means uses a plurality of pixel data around each pixel of the correction target line and the distortion correction amount retrieved from the distortion correction amount table to interpolate each pixel data of the correction target line by an interpolation correction method. The image correction processing apparatus according to claim 1, wherein distortion correction is performed. - 前記撮像素子は、部品実装機に搭載されていることを特徴とする請求項１に記載の画像補正処理装置。 The image correction processing apparatus according to claim 1, wherein the image sensor is mounted on a component mounter.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/JP2012/056497 WO2013136458A1 (en) | 2012-03-14 | 2012-03-14 | Image correction processing device |
| JP2014504548A JP6146819B2 (en) | 2012-03-14 | 2012-03-14 | Image correction processing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/JP2012/056497 WO2013136458A1 (en) | 2012-03-14 | 2012-03-14 | Image correction processing device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2013136458A1 true WO2013136458A1 (en) | 2013-09-19 |
Family
ID=49160427
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2012/056497 WO2013136458A1 (en) | 2012-03-14 | 2012-03-14 | Image correction processing device |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP6146819B2 (en) |
| WO (1) | WO2013136458A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2021060590A (en) * | 2015-05-04 | 2021-04-15 | グーグル エルエルシーＧｏｏｇｌｅ ＬＬＣ | Pass-through display of captured imagery |
| US11076092B2 (en) | 2018-09-18 | 2021-07-27 | Kabushiki Kaisha Toshiba | Image processing apparatus, image processing method, and image processing program |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH11154228A (en) * | 1997-11-25 | 1999-06-08 | Fuji Xerox Co Ltd | Image processor and its method |
| JP2001101396A (en) * | 1999-09-30 | 2001-04-13 | Toshiba Corp | Processor and method for correcting image distortion and medium with program performing image distortion correction processing stored therein |
| JP2005044098A (en) * | 2003-07-28 | 2005-02-17 | Olympus Corp | Image processor and image processing method |
| JP2011180084A (en) * | 2010-03-03 | 2011-09-15 | Fuji Mach Mfg Co Ltd | Picked-up image processor of component mounting machine |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH11154288A (en) * | 1997-11-19 | 1999-06-08 | Matsushita Electric Works Ltd | Fire sensor and fitting method therefor |
| JP3986877B2 (en) * | 2002-04-26 | 2007-10-03 | 株式会社リコー | Image processing device |
| JP3897247B2 (en) * | 2002-05-16 | 2007-03-22 | 富士フイルム株式会社 | Optical distortion correction method and correction apparatus |
| JP4919836B2 (en) * | 2006-03-01 | 2012-04-18 | パナソニック株式会社 | Image processing apparatus, image pickup apparatus, and image distortion correction method for correcting image distortion |
| JP5299383B2 (en) * | 2010-08-20 | 2013-09-25 | 株式会社Ｊｖｃケンウッド | Image correction apparatus and image correction method |
-
2012
- 2012-03-14 WO PCT/JP2012/056497 patent/WO2013136458A1/en active Application Filing
- 2012-03-14 JP JP2014504548A patent/JP6146819B2/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH11154228A (en) * | 1997-11-25 | 1999-06-08 | Fuji Xerox Co Ltd | Image processor and its method |
| JP2001101396A (en) * | 1999-09-30 | 2001-04-13 | Toshiba Corp | Processor and method for correcting image distortion and medium with program performing image distortion correction processing stored therein |
| JP2005044098A (en) * | 2003-07-28 | 2005-02-17 | Olympus Corp | Image processor and image processing method |
| JP2011180084A (en) * | 2010-03-03 | 2011-09-15 | Fuji Mach Mfg Co Ltd | Picked-up image processor of component mounting machine |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2021060590A (en) * | 2015-05-04 | 2021-04-15 | グーグル エルエルシーＧｏｏｇｌｅ ＬＬＣ | Pass-through display of captured imagery |
| US11076092B2 (en) | 2018-09-18 | 2021-07-27 | Kabushiki Kaisha Toshiba | Image processing apparatus, image processing method, and image processing program |
Also Published As
| Publication number | Publication date |
|---|---|
| JPWO2013136458A1 (en) | 2015-08-03 |
| JP6146819B2 (en) | 2017-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI514872B (en) | Image processing system and method using multiple imagers for providing extended view | |
| EP1993277A3 (en) | Image pickup apparatus and method of correcting captured image data | |
| JP6141084B2 (en) | Imaging device | |
| EP2068280A3 (en) | Image processing apparatus, image processing method, image processing program and position detecting apparatus as well as mobile object having the same | |
| EP2302906A3 (en) | Image processing apparatus and method, recording medium, and program | |
| JP2008072565A (en) | Imaging device and defective pixel correction method | |
| EP2373048B1 (en) | Method for detecting and correcting bad pixels in image sensor | |
| TWI638149B (en) | Luminance measurement method, luminance measurement device, and image quality adjustment technique using the method and device | |
| JP6146819B2 (en) | Image correction processing device | |
| US7990581B2 (en) | Image processing apparatus, control method for controlling the image processing apparatus, and computer-readable storage medium | |
| JP5121870B2 (en) | Image processing method and image processing apparatus | |
| US20120242784A1 (en) | Image processing method, camera module, and photographing method | |
| JP4771901B2 (en) | Imaging apparatus, image distortion correction method, and program | |
| JP2006058945A (en) | Method and device for correcting rolling shutter image | |
| JP2008118570A5 (en) | ||
| CN113824935A (en) | Time delay integration method, device and equipment of pseudo color line scanning camera | |
| JP4521858B2 (en) | Image processing device | |
| JP2010212788A (en) | Image capturing apparatus, defective pixel correction method, and electronic information apparatus | |
| JP5676233B2 (en) | Image processing apparatus, image processing method, and imaging apparatus | |
| US8730370B2 (en) | Solid-state imaging device having image pixel position correction to reduce shape distortion | |
| JP2014033415A (en) | Image processor, image processing method and imaging apparatus | |
| JP2005210164A5 (en) | ||
| JP2005352703A (en) | Image processing apparatus | |
| JP2017098775A (en) | Imaging device | |
| JP2012168372A5 (en) | Image processing apparatus and image processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12871019 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 2014504548 Country of ref document: JP Kind code of ref document: A |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 12871019 Country of ref document: EP Kind code of ref document: A1 |