JP3570198B2 - Image processing method and apparatus - Google Patents

Description

【００３７】
なお（１）式において、ｔｈ_Ｓ ，ｔｈ_Ｆ はあらかじめ定められたしきい値であって、ｔｈ_Ｓ には十分に大きな輝度値が、ｔｈ_Ｆ には十分に小さな輝度値が、それぞれ設定される。
例えば、入力画像の各画素の輝度値が８ビット構成のデータをとる場合、ｔｈ_Ｓ は２４０付近に、ｔｈ_Ｆ は１０付近に設定される。
【００３８】
前記図３（１）（２）に示した各入力画像Ｉ_Ｓ ，Ｉ_Ｆ の場合、黒い車輌Ｃ_Ｂ に対応する画素位置については（１）−１式が適用され、合成画像の輝度値Ｇ（ｉ，ｊ）として、遅いシャッタ速度Ｓ_Ｓ による入力画像Ｉ_Ｓ 側の輝度値ｇ_Ｓ （ｉ，ｊ）が採用される。また白い車輌Ｃ_Ｗ に対応する画素位置については（１）−２式が適用され、合成画像の輝度値Ｇ（ｉ，ｊ）として、速いシャッタ速度Ｓ_Ｆ による入力画像Ｉ_Ｆ 側の輝度値ｇ_Ｆ （ｉ，ｊ）が採用される。その他の車輌の存在しない画素位置においては（１）−３式が適用され、各輝度値ｇ_Ｓ （ｉ，ｊ），ｇ_Ｆ （ｉ，ｊ）の平均値をもって合成画像の輝度値Ｇ（ｉ，ｊ）が求められることになる。
なお画像の合成方法は、上記に限らず、合成画像のすべての画素の輝度値Ｇ（ｉ，ｊ）とも、画素ｇ_Ｓ （ｉ，ｊ）とｇ_Ｆ （ｉ，ｊ）との平均値により表してもよい。
【００３９】
図４は、各入力画像Ｉ_Ｓ ，Ｉ_Ｆ を上記（１）−１〜（１）−３を適用して合成処理した結果を示すもので、各車輌Ｃ_Ｂ ，Ｃ_Ｗ の部分に、その車輌を精度よく捉えた画像側のデータを採用することにより、各車輌Ｃ_Ｂ ，Ｃ_Ｗ を鮮明に捉えた画像が出現している。
【００４０】
特徴抽出部１２は、この合成画像上にエッジ抽出用フィルタを走査するなどして、車輌の輪郭を表すエッジを抽出する。物体検出部１３は、内部メモリに記憶された画像上の各駐車枠の位置に基づき、各駐車枠内のエッジ部分の面積を算出するなどして、車輌が存在するか否かを判別する。この判別結果は、各駐車枠の識別データ（ラベルや位置データなど）に対応づけられ、外部に出力される。
【００４１】
（２）第２実施例
図５は、観測装置の第２の構成を示す。
この実施例の制御装置２も、第１の実施例と同様の構成の画像入力部６により、各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ により得られた画像Ｉ_Ｓ ，Ｉ_Ｆ を個別に入力する。また認識処理部７として、２組の特徴抽出部１２Ｓ， １２Ｆ，特徴統合部１４，物体検出部１３の各構成を具備しており、各特徴抽出部１２Ｓ，１２Ｆにより、各入力画像Ｉ_Ｓ ，Ｉ_Ｆ の画像上のエッジが個別に抽出される。
【００４２】
図６（１）（２）は、前記図３（１）（２）の入力画像Ｉ _Ｓ ，Ｉ _Ｆ に対するエッジ抽出処理により生成されたエッジ画像Ｅ_Ｓ，Ｅ_Ｆを示す。なおここでは、各エッジの抽出位置を明確にするために、入力画像上の駐車枠のエッジがすべて抽出されたように図示している。
【００４３】
エッジ画像Ｅ_Ｓ 上では、２個の車輌Ｃ_Ｂ ，Ｃ_Ｗ の駐車位置Ａ，Ｂのうち、画像上に鮮明に現れた車輌Ｃ_Ｂ の駐車位置Ａのみでエッジが抽出される（図中の駐車位置Ｂに細線の点線で示されたエッジは、仮想のものである）。反対にエッジ画像Ｅ_Ｆ 上では、車輌Ｃ_Ｗ の駐車位置Ｂでのみエッジが抽出される。
【００４４】
図５に戻って、特徴統合部１４は、各入力画像Ｉ_Ｓ ，Ｉ_Ｆ についてのエッジ抽出結果を統合して、１枚のエッジ画像を生成する。この統合処理は、つぎの（２）式に示すように、各エッジ画像Ｅ_Ｓ ，Ｅ_Ｆ の対応する画素毎に各画素値ｂ_Ｓ （ｉ，ｊ），ｂ_Ｆ （ｉ，ｊ）の論理和（（２）式中「｜」で示す）を求めて、統合後のエッジ画像の画素Ｂ（ｉ，ｊ）の画素値とするものである。すなわち各画素値Ｂ（ｉ，ｊ）により形成されるエッジ画像は、両方のエッジ画像Ｅ_Ｓ ，Ｅ_Ｆ を重ね合わせたもので、統合処理により、すべてのエッジ抽出結果が採用されることになる。
【００４５】
【数２】

【００４６】
図７は、前記図６（１）（２）の各エッジ抽出結果を上記（２）式を用いて統合した結果を示すもので、上記した論理和演算により、いずれの車輌Ｃ_Ｂ ，Ｃ_Ｗ の駐車位置Ａ，Ｂにも、エッジが精度よく出現している。
【００４７】
物体検出部１３は、この統合されたエッジ画像上で、前記第１の実施例と同様、各駐車枠毎に車輌の有無を判定し、その判定結果を出力するもので、両方の車輌Ｃ_Ｂ ，Ｃ_Ｗ をともに精度よく検出することができる。
【００４８】
（３）第３実施例
図８は、観測装置の第３の構成を示す。この実施例の制御装置２は、前記第１，２の実施例と同様の画像入力部６により、２種類のシャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ により撮像された画像Ｉ_Ｓ ，Ｉ_Ｆ を個別に入力した後、各入力画像Ｉ_Ｓ ，Ｉ_Ｆ を個別に処理して各画像上の車輌を検出し、さらに各検出結果を統合して最終的な判定を行うものである。
【００４９】
認識処理部７は、各入力画像毎に特徴抽出部１２Ｓ，１２Ｆおよび物体検出部１３Ｓ，１３Ｆを具備するほか、検出結果統合部１５を具備する。各特徴抽出部１２Ｓ，１２Ｆは、第２の実施例と同様に、それぞれ入力画像Ｉ_Ｓ ，Ｉ_Ｆ に対するエッジ抽出処理を実施して、その結果を示すエッジ画像Ｅ_Ｓ ，Ｅ_Ｆ を生成する。各物体検出部１３Ｓ，１３Ｆは、対応するエッジ画像Ｅ_Ｓ ，Ｅ_Ｆ 上で、各駐車枠内に車輌が存在するか否かを判定するもので、各判定結果は、車輌枠を示すラベルに対応づけられた形で検出結果統合部１５へと出力される。検出結果統合部１５は、駐車枠毎の判定結果の論理和をとることにより、各エッジ画像上の車輌検出結果を統合し、この統合結果を最終的な車輌検出結果として出力する。
【００５０】
図９は、前記図６（１）（２）のエッジ画像Ｅ_Ｓ ，Ｅ_Ｆ を用いて、各駐車枠毎に車輌の有無を判定した結果であって、図中の左端欄に、各駐車枠に付与されたラベル番号を表すとともに、各ラベル番号に対応づけて、各エッジ画像Ｅ_Ｓ ，Ｅ_Ｆ 毎の車輌の有無の判定結果と最終的な統合結果とが示してある。なお図中のデータ「１」は「車輌あり」の判定結果を、データ「０」は「車輌なし」の判定結果を、それぞれ表すものとする。
【００５１】
図示例は、前記画像上の各駐車枠に対し、左から右方向に順にラベル番号を付してあり、ラベル番号「１」の駐車枠（前記駐車位置Ａに相当する）にある車輌Ｃ_Ｂ はエッジ画像Ｅ_Ｓ 上で検出され、またラベル番号「８」の駐車枠（前記駐車位置Ｂに相当する）にある車輌Ｃ_Ｗ はエッジ画像Ｅ_Ｆ 上で検出されている。
よって両検出結果を統合した結果、「１」「８」いずれの駐車枠にも、車輌が存在するものと判定される。
【００５２】
（４）実施例１〜３の変形例 ＜１＞
上記した３つの実施例は、いずれも２とおりのシャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ による画像を用いて車輌の検出処理を行うようにしているが、これに限らず、３とおり以上のシャッタ速度による画像を生成して車輌検出処理を行うようにしてもよい。
【００５３】
図１０〜１２は、それぞれ第１〜第３の各実施例を発展させた構成を示すもので、画像入力部６には、Ｎとおり（Ｎ≧３）のシャッタ速度により得られた各画像を記憶するためのＮ個の画像メモリ１０Ｍ_１ 〜１０Ｍ_Ｎ が配備される。
【００５４】
図１０の実施例では、認識処理部７の画像統合部１１により、各入力画像間の対応する画素毎に輝度平均値が算出されて各画像が統合された後、特徴抽出部１２により、この統合画像上のエッジ抽出処理が実施される。さらに物体検出部１３により、前記エッジ抽出結果を用いた各車輌の検出処理が行われ、その結果が出力される。
【００５５】
図１１の実施例の認識処理部７は、各入力画像毎に特徴抽出部１２Ａ_１ 〜１２Ａ_Ｎ を具備しており、入力画像毎にエッジ抽出処理が行われる。特徴統合部１４は、前記エッジ抽出処理により得られたエッジ画像について、前記第２の実施例と同様、対応する画素間の論理和演算を行って、各エッジ画像を統合する。この後、前記図１０の実施例と同様、物体検出部１３による車輌の検出処理が行われる。
【００５６】
図１２の実施例の認識処理部７は、各入力画像毎に特徴抽出部１２Ａ_１ 〜１２Ａ_Ｎ および物体検出部１３Ａ_１ 〜１３Ａ_Ｎ を具備しており、これら構成により各入力画像毎に個別に車輌検出処理が行われる。この後、前記第３の実施例と同様、検出結果統合部１５により各検出結果が統合されて、最終的な検出結果を示すデータが生成され、外部に出力される。
【００５７】
このように複数とおりのシャッタ速度による画像を用いて認識処理を行うようにすれば、カメラ１のダイナミックレンジをより広く設定でき、さまざまな色彩の車輌が駐車している場合にも、各車輌を精度良く検出することが可能となる。
【００５８】
（５）実施例４
前記第１〜３の各実施例では、１台のカメラ１をシャッタ速度を変更して連続的に撮像を行うようにしている。これに対し、第４の実施例では、図１３に示すように、異なるシャッタ速度が設定された２台のカメラ１Ｓ，１Ｆを配備し、各カメラ１Ｓ，１Ｆを同時に動作させる。
【００５９】
図１４は、上記図１３の観測装置の構成を示すもので、制御装置２の画像入力部６には、各カメラ１Ｓ，１Ｆ毎のＡ／Ｄ変換回路８Ｓ，８Ｆおよび画像メモリ１０Ｓ，１０Ｆが配備される。
なおこの実施例の認識処理部７には、前記第１〜３のいずれの実施例の構成を導入してもよいので、認識処理部７に関する詳細な構成の図示および説明は省略する。
【００６０】
ただし認識処理部７に第１または第２の実施例の構成を導入する場合、各カメラ１Ｓ，１Ｆは、画像の統合処理に支障が生じないように、光軸を平行にした状態で出来るだけ接近させて配備する必要がある。さらに画像の統合処理時には、カメラ１Ｓ，１Ｆ間の視差によるずれを補正するために、いずれかの画像を、各カメラの光軸間の距離に対応する画素数分だけ、視差の生じる方向（図示例の場合は縦方向）にシフトさせる必要がある。
【００６１】
（６）第５実施例
この実施例は、図１の設置例と同様、１台のカメラによる画像を取り込んで車輌検出処理を実施するものであるが、この実施例のカメラには、図１５に示すように、２枚のＣＣＤ撮像素子１７Ｓ，１７Ｆを具備するカメラ１Ｘが用いられる。各ＣＣＤ撮像素子１７Ｓ，１７Ｆには、それぞれ前記シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ に相当する電荷蓄積時間が設定されており、カメラレンズ１６を通過した光が各ＣＣＤ素子１７Ｓ，１７Ｆに同時に与えられて、この光像を異なるシャッタ速度でとらえた２枚の画像Ｉ_Ｓ ，Ｉ_Ｆ が生成される。
【００６２】
制御装置２側の画像入力部６は、各ＣＣＤ撮像素子１７Ｓ，１７Ｆからの画像データを同時に受けることができるように、入力画像毎のＡ／Ｄ変換回路８Ｓ，８Ｆおよび画像メモリ１０Ｓ，１０Ｆとを具備する。
なお認識処理部７には、前記第４の実施例と同様、第１〜３のいずれの構成を導入しても良いので、ここでは図示および詳細な説明は省略する。
【００６３】
この実施例の構成によれば、同一の観測領域について、各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ による画像を同時に取り込んで処理することができるので、各入力画像間の時間のずれをなくせる上、前記第４の実施例のように画像の統合処理時にいずれか一方の画像をシフト処理する必要もない。よって車輌の検出精度を向上させると同時に、処理時間を短縮させることができる。
【００６４】
（７）第６実施例
この実施例も、上記第５の実施例と同様の２個のＣＣＤ撮像素子１７Ｓ，１７Ｆを具備するカメラ１Ｘにより、各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ による画像を同時に生成するものである。
図１６は、この第６の実施例の構成を示すもので、制御装置２は、先の実施例と同様の構成の画像入力部６を具備するほか、画像統合部１１，モニタ１８を含んでいる。
【００６５】
カメラ１Ｘにより生成された各画像は、画像入力部６を介して画像統合部１１に与えられ、前記第１の実施例と同様の方法で統合される。モニタ１８は、この統合処理により生成される画像を表示するためのもので、これにより駐車エリア４内に明るさの異なる車輌が存在する場合にも、各車輌を１つのモニタ画面上で観測することが可能となる。
【００６６】
なお前記モニタ１８を、駐車エリア４から離れた場所に設置し、制御装置２の本体からモニタ１８に、統合処理された画像データを通信回線を介して送信することも可能である。このようにすれば、直接駐車エリア４を監視できない地域でも、駐車エリア４の状況を正確に把握することができる。
【００６７】
またこの実施例の構成に、前記した第１〜３のいずれかの認識処理部７の構成を加え、前記モニタ１８に、統合処理された画像とともに、各駐車枠毎の車輌の有無判定の結果を表示することも可能である。
【００６８】
（８）第７実施例
図１７は、観測装置の第７の構成を示す。この実施例は、前記第１〜３のいずれかの構成に光強度計１９を付加してカメラ１の観測領域内の明るさを検出し、その明るさの度合いにより、カメラ１の各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ を可変設定するようにしている。
【００６９】
図１８は、シャッタ速度の設定例を示す。
前記光強度計１９は、観測領域からの光を受光してその光強度を検出するもので、ここではあらかじめ各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ に３段階の設定値（ｓ１〜ｓ３，ｆ１〜ｆ３）を設けて、シャッタ速度制御装置５のメモリに記憶させておき、前記光強度の検出値λを所定のしきい値ｔｈ１，ｔｈ２（ｔｈ１＞ｔｈ２）と比較することにより、各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ にいずれかの設定値を採用するようにしている。
【００７０】
このように観測領域内の明るさの度合いによりカメラ１のシャッタ速度を制御するので、カメラ１により大きなダイナミックレンジを設定することができ、観測領域内の明るさが変動しても、安定した車輌検出を行うことができる。
【００７１】
（９）第８実施例
図１９は、観測装置の第８の構成を示す。この実施例も、前記第７の実施例と同様、前記第１〜３のいずれかの構成に、カメラの各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ を可変設定する構成を付加したものであるが、ここでは観測領域内の明るさに代えて、各入力画像の明るさにより、シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ の設定値を変動するようにしている。
【００７２】
制御装置２には、第１〜３の各構成と同様の画像入力部６，これら３つの構成のいずれかと同構成の認識処理部７のほか、２個の輝度平均算出部２０Ｓ，２０Ｆが配備される。これら輝度平均算出部２０Ｓ，２０Ｆは、画像メモリ１０Ｓ，１０Ｆに記憶された入力画像Ｉ_Ｓ ，Ｉ_Ｆ について、各構成画素の輝度値の平均を算出し、シャッタ速度制御装置５に出力する。シャッタ速度制御装置５は、各算出値を個々に取り込んで、各シャッタ速度Ｓ_ｓ ，Ｓ_Ｆ の設定値を決定し、つぎの撮像時には、この決定値に基づき、カメラ１のシャッタ速度を制御する。
【００７３】
なお処理開始時点では、各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ には所定の初期値が設定され、以後の数サイクルの撮像動作により最適値に調整される。
【００７４】
図２０は、シャッタ速度を決定するための手順を示す。なおここでは、遅い方のシャッタ速度Ｓ_Ｓ を決定する手順を示すが、速い方のシャッタ速度Ｓ_Ｆ も同様の手順で決定される。
【００７５】
最初のステップ１（図中「ＳＴ１」で示す）で、輝度平均算出部２０Ｓは、画像メモリ１０Ｓより入力画像Ｉ_Ｓ の画像データを読み込んで、平均輝度値ＡＶ_Ｓ を算出する。つぎのステップ２，３で、シャッタ速度制御装置５は、この算出値ＡＶ_Ｓ を平均輝度値の最適値Ｍ_Ｓ と比較し、その比較結果を用いて、シャッタ速度Ｓ_Ｓ の現在値を維持するか、変更するかを決定する。
【００７６】
前記平均輝度値の最適値Ｍ_Ｓ は、あらかじめ内部のメモリに登録されているもので、前記平均輝度値の算出値ＡＶ_Ｓ と最適値Ｍ_Ｓ との差が、所定のしきい値Δ_ｓ２からΔ_ｓ１の間にある場合（Δ_ｓ２＜Δ_Ｓ１）は、ステップ２，３がいずれも「ＮＯ」となり、シャッタ速度Ｓ_Ｓ の現在値が維持される（ステップ６）。
【００７７】
入力画像Ｉ_Ｓ が明るすぎる場合には、前記平均輝度値の算出値ＡＶ_Ｓ と最適値Ｍ_Ｓ との間にしきい値Δ_Ｓ１を上回る差が生じる。この場合はステップ２が「ＹＥＳ」となってステップ４へと移行し、シャッタ速度Ｓ_Ｓ の設定値が１／２に更新される。
【００７８】
逆に、入力画像Ｉ_Ｓ の明るさが落ちた場合には、前記平均輝度値の算出値ＡＶ_Ｓ と最適値Ｍ_Ｓ との差がしきい値Δ_ｓ２を下回る。この場合には、ステップ３が「ＹＥＳ」となってステップ５へと移行し、、シャッタ速度Ｓ_Ｓ の設定値は２倍に更新される。
【００７９】
なお上記ステップ４，５におけるシャッタ速度Ｓ_Ｓ の更新は、図示例に限らず、例えば現在の設定値に所定の度数分の値を加算（または減算）するようにしてもよい。
【００８０】
このように、画像入力毎に、つぎの段階でのシャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ を、それぞれ対応する入力画像Ｉ_Ｓ ，Ｉ_Ｆ を用いて決定することにより、実際に得られた画像上の明るさに基づき、各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ を適切に調整することができる。
なお図１５，１６に示した第５，６の実施例にも、上記第７，８の実施例のシャッタ速度の調整にかかる構成を導入することが可能である。
【００８１】
（１０）第９実施例
上記した各実施例の構成は、いずれも所定の明るさの下での観測が可能な昼間に処理を実施することを前提とするもので、夜間など、観測領域全体が暗くなる条件下では、速いシャッタ速度により得られた画像上には、いずれの車輌の画像も表れない虞がある。
【００８２】
図２１に示す第９の構成は、複数とおりのシャッタ速度による画像を合成して認識処理を行うものであるが、夜間時にはこの処理を休止して、１枚の画像を用いた通常の認識処理を実施するように、構成される。
【００８３】
この夜間時における処理の切換えは、制御装置２内部のタイマ２１の計時時刻に基づき行われるもので、タイマ２１の計時時刻が所定の期間内にあるときは、カメラ１には、最も遅いシャッタ速度のみがセットされる。また図中、２２は、第２以下の画像メモリ１０Ｍ_２ 〜１０Ｍ_Ｎ と認識処理部７との接続をオンオフするための切換部であって、前記カメラ１に単一のシャッタ速度がセットされると同時に、この切換部２２はオフにセットされる。また同時に、認識処理部７の画像統合部１１の機能が休止し、特徴抽出部１２および物体検出部１３により、画像メモリ１０Ｍ_１ からの画像データのみを用いた認識処理が行われる。
【００８４】
なお図示例の制御装置２は、認識処理部７に前記図１０の構成を採用しているが、他の構成をとる場合も、同様に、昼間と夜間とで処理の切換えを行うように設定することができる。
【００８５】
このように夜間時に処理を切り換えることにより、夜間時の無用な演算処理を削減することができる。またこの無用な演算処理により検出結果に誤差が生じる虞もなくなり、車輌検出の精度を安定させることができる。
【００８６】
（１１）第１０実施例
図２２は、第１０番目の構成を用いる場合の観測装置の設置例を示す。この観測装置は、２台のカメラ１Ａ，１Ｂからの画像を用いた３次元計測処理により、各駐車枠毎の車輌の有無を判定するもので、各カメラ１Ａ，１Ｂは、駐車エリア４に向けて、光軸を平行かつ撮像面を同一面上に位置させた状態で、縦並びに配備される。
【００８７】
この実施例も、前記した第１〜８の各実施例と同様、明るさの異なる車輌Ｃ_Ｂ ，Ｃ_Ｗ をともに精度良く検出できるように、各カメラ１Ａ，１Ｂに複数とおりのシャッタ速度を設定して連続的に撮像させている。また制御装置２においては、各カメラ１Ａ，１Ｂより同じシャッタ速度で得られた画像の組毎に、車輌を示す特徴点の抽出から特徴点の対応づけまでの処理を行った後、各対応づけ結果を統合して３次元計測処理を実施するようにしている。
【００８８】
図２３は、前記観測装置の構成を示す。
シャッタ速度制御装置５は、前記第１〜３の各実施例と同様、固定された２とおりのシャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ を交互に設定するもので、各カメラ１Ａ，１Ｂは、このシャッタ速度制御装置５による制御を受けて、各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ による画像を、それぞれ同時に生成する。
【００８９】
制御装置２は、前記した各実施例と同様、画像入力部６と認識処理部７とを備えて成る。画像入力部６は、各カメラ毎に、Ａ／Ｄ変換回路８Ａ，８Ｂおよび２組の画像メモリ（１０Ａ_Ｓ ，１０Ａ_Ｆ ）（１０Ｂ_Ｓ ，１０Ｂ_Ｆ ）を具備する。各組の一方の画像メモリ１０Ａ_Ｓ ，１０Ｂ_Ｓ には、それぞれ各カメラ１Ａ，１Ｂからの遅い方のシャッタ速度Ｓ_Ｓ による画像Ｕ_Ｓ ，Ｌ_Ｓ が格納される。また他方の画像メモリ１０Ａ_Ｆ ，１０Ｂ_Ｆ には、それぞれ各カメラからの速い方のシャッタ速度Ｓ_Ｆ による画像Ｕ_Ｆ ，Ｌ_Ｆ が格納される。
【００９０】
認識処理部７は、２組の特徴抽出部１２Ｂ_Ｓ ，１２Ｂ_Ｆ ，２組の対応づけ処理部２３Ｓ，２３Ｆを含むとともに、特徴統合部２４，３次元計測部２５，物体検出部２６などの各構成を具備する。特徴抽出部１２Ｂ_Ｓ ，１２Ｂ_Ｆ は、それぞれ下側のカメラ１Ｂにより得られた各画像Ｌ_Ｓ ，Ｌ_Ｆ について、画像上のエッジを抽出するためのもので、特徴抽出部１２Ｂ_Ｓ による画像Ｌ_Ｓ に対するエッジ抽出結果は第１の対応づけ処理部２３Ｓに、特徴抽出部１２Ｂ_Ｆ による画像Ｌ_Ｆ に対するエッジ抽出結果は第２の対応づけ処理部２３Ｆに、それぞれ個別に与えられる。
【００９１】
第１の対応づけ処理部２３Ｓは、各カメラ１Ａ，１Ｂより遅い方のシャッタ速度Ｓ_Ｓ で得られた入力画像Ｕ_ｓ ，Ｌ_ｓ について、それぞれ空間内の同じ物点を表す特徴点を特定して両者を対応づけるためのものである。
【００９２】
この対応づけ処理は、具体的には、前記特徴抽出部１２Ｂ_Ｓ により画像Ｌ_Ｓ 上で抽出された各エッジ構成点Ｐについて、それぞれ画像Ｕ_Ｓ 上での対応点Ｑを抽出することにより行われる。
【００９３】
図２４は、前記点Ｐへの対応点Ｑを抽出する具体的な方法を示す。
まず対応づけ処理部２３Ｓは、前記エッジ抽出処理により生成されたエッジ画像ＥＬ_Ｓ 上の所定の特徴点Ｐ（ｘ，ｙ_Ｌ ）について、原画像Ｌ_Ｓ 上の点Ｐの位置に、この点Ｐを中心点とする所定大きさのウィンドウＷ_Ｌ を設定する。ついで対応づけ処理部は、画像Ｕ_Ｓ 上の前記点ＰのエピポーララインＥＰ上に前記ウィンドウＷ_Ｌ と同じ大きさのウィンドウＷ_Ｕ を設定して走査し、各走査位置毎に、つぎの（３）式を実行して、各ウィンドウＷ_Ｌ ，Ｗ_Ｕ 内の画像データの相違度ＤＦを算出する。
【００９４】
なお（３）式において、ｇ_Ｌ（ｘ，ｙ）はウィンドウＷ_Ｌ内の所定の画素の輝度値を、またｇ_Ｕ（ｘ，ｙ）はウィンドウＷ_Ｕ内の所定の画素の輝度値を、それぞれ示す。またSZは各ウィンドウＷ_Ｌ，Ｗ_Ｕのサイズを示す。さらにｍ，ｎは各ウィンドウ内の画素を特定するための変数であって、０から前記サイズ SZの範囲内で変動する。
【００９５】
【数３】

【００９６】
対応づけ処理部２３Ｓは、各走査位置において求められた相違度ＤＦを比較し、この相違度ＤＦが最も小さくなる時点でのウィンドウＷ_Ｕ の中心点（ｘ，ｙ_Ｕ ）を、前記点Ｐの対応点Ｑとして特定する。
【００９７】
なお前記したように、各カメラ１Ａ，１Ｂは縦並び、かつ光軸を平行にして配備されているので、エピポーララインＥＰはｘ軸に垂直になり、ウィンドウＷ_U を容易に設定でき、また相違度ＤＦの演算も簡単になる。またエピポーララインＥＰ上の各走査位置で相違度ＤＦを算出する方法に代えて、画像Ｕ_S 上でもエッジを抽出し、このうち前記エピポーララインＥＰ上に位置するエッジ構成点を、特徴点Ｐの対応候補点として設定し、これら対応候補点についてのみ相違度ＤＦを演算するようにしてもよい。
【００９８】
図２３に戻って、第２の対応づけ処理部２３Ｆは、各カメラ１Ａ，１Ｂより速い方のシャッタ速度Ｓ_Ｆ により得られた各入力画像Ｕ_Ｆ ，Ｌ_Ｆ について、それぞれ上記と同様の対応づけ処理を行うもので、各対応づけ処理部２３Ｓ，２３Ｆによる対応づけ結果は、特徴統合部２４において統合処理される。
【００９９】
この統合処理は、各対応づけ処理部２３Ｓ，２３Ｆにより対応づけられた特徴点の組をすべて採用するもので、対応する各特徴点の組毎の座標データは、３次元計測部２５に与えられる。３次元計測部２５は、与えられた特徴点の組毎に、各特徴点の座標（ｘ，ｙ_ｕ ）（ｘ，ｙ_Ｌ ）を三角測量の原理にあてはめることにより、各特徴点に対応する３次元座標を算出する。
【０１００】
こうして各特徴点の組毎に対応する３次元座標が算出され、物体検出部２６へと与えられる。物体検出部２６は、各駐車枠位置における特徴点の数およびその高さデータから、各駐車枠内に車輌が存在するか否かを判別する。
【０１０１】
なおこの第１０の実施例では、説明を簡単にするために、各カメラのシャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ を固定値にしたが、前記第７，８の実施例のように、観測領域の明るさや入力画像の明るさに応じて各シャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ を可変設定してもよい。またシャッタ速度Ｓ_Ｓ ，Ｓ_Ｆ は１種類に限らず、図１０〜１２に示したように、複数とおりのシャッタ速度を設定し、各シャッタ速度毎に画像間での特徴点の対応づけを行った後、各対応づけ結果を統合するようにしてもよい。さらに前記第９の実施例に示したように、夜間時には各カメラを遅いシャッタ速度のみで撮像させ、１組の画像のみで３次元計測処理を行うように構成してもよい。
【０１０２】
（１２）その他の実施例
上記した各実施例は、いずれも、カメラにより、シャッタ速度の異なる複数枚の画像を生成するようにしたが、シャッタ速度のみならず、カメラの絞りや、カメラからの出力電圧のゲインを調整することによっても、明るさの異なる対象物の画像を個別に生成することが可能である。
【０１０３】
またここでは、駐車場用の観測装置における各構成例を説明したが、観測対象は車輌に限らず、他の物体であってもよく、また観測領域は室内であってもよい。さらにこの実施例の観測装置は、各駐車枠毎の車輌の有無を認識するだけのものであるが、これに限らず、車輌の大きさ、形状などを認識するようにしてもよく、カラー画像生成用のカメラを用いて、各車輌の車体色を認識するようにしてもよい。
【０１０４】
さらに個々に示した駐車場用の観測装置は、静止した車輌を認識対象とするが、道路上を走行する車輌などの移動体を認識対象とする場合にも、この発明を適用することができる。ただしこの場合、各シャッタ速度毎の画像を連続的に生成すると、画像間に対象物の移動分のずれが生じるので、前記第４または第５の実施例の構成を導入するのが望ましい。
【０１０５】
【発明の効果】
請求項１，２および請求項４，５の発明によれば、観測領域内に車体色の異なる複数の車輌が存在しても、各車輌をそれぞれその車体色に対応する露光量による画像上に出現させることができ、各種色彩の車輌を精度良く認識することができる。
【０１０８】
請求項６，７の発明では、各画像を供給する撮像手段に対し、観測領域内もしくは入力画像の明るさを用いて露光量を調整するので、周囲環境の変化により観測領域内の明るさが変動しても、各画像により車輌を鮮明にとらえることができ、安定した認識処理を行うことができる。
【０１０９】
請求項３および９の発明では、観測領域を複数個の撮像手段により撮像して、３次元計測処理を行う際に、各撮像手段により同じ露光量で生成された画像の組毎に画像上の特徴点の対応づけを行った後に、各組毎の対応づけ結果を統合して３次元計測処理を実施するので、観測領域内に明るさの異なる対象物がある場合にも、各対象物の３次元形状や空間位置を、精度良く認識することができる。
【０１１０】
請求項８および１０の発明では、タイマの計時時刻により、１日のうちの所定の期間内は、特定の露光量による画像のみを用いた認識処理を行うので、夜間など明度の高い対象物が検出されることのない時間帯の無用な処理を回避して、装置コストを削減できる。また必要外の処理による誤計測を防止することも可能となる。
【図面の簡単な説明】
【図１】この発明が適用された観測装置の設置例を示す斜視図である。
【図２】観測装置の第１の構成を示すブロック図である。
【図３】入力画像の一例を示す説明図である。
【図４】図３の入力画像の合成画像を示す説明図である。
【図５】観測装置の第２の構成を示すブロック図である。
【図６】入力画像のエッジ抽出処理により得られたエッジ画像を示す説明図である。
【図７】図６のエッジ画像を統合した結果を示す説明図である。
【図８】観測装置の第３の構成を示すブロック図である。
【図９】駐車枠ごとの車輌の有無の判別結果を示す説明図である。
【図１０】観測装置の第１の構成を変形した例を示すブロック図である。
【図１１】観測装置の第２の構成を変形した例を示すブロック図である。
【図１２】観測装置の第３の構成を変形した例を示すブロック図である。
【図１３】第４の構成の観測装置の設置例を示す斜視図である。
【図１４】観測装置の第４の構成を示すブロック図である。
【図１５】観測装置の第５の構成を示すブロック図である。
【図１６】観測装置の第６の構成を示すブロック図である。
【図１７】観測装置の第７の構成を示すブロック図である。
【図１８】図１７の観測装置におけるシャッタ速度の設定例を示す説明図である。
【図１９】観測装置の第８の構成を示すブロック図である。
【図２０】図１９の観測装置におけるシャッタ速度の設定手順を示すフローチャートである。
【図２１】観測装置の第９の構成を示すブロック図である。
【図２２】第１０の構成の観測装置の設置例を示す斜視図である。
【図２３】観測装置の第１０の構成を示すブロック図である。
【図２４】特徴点の対応づけ処理の具体例を示す説明図である。
【符号の説明】
１，１Ｓ，１Ｆ，１Ｘ，１Ａ，１Ｂ カメラ
２ 制御装置
５ シャッタ速度制御装置
６ 画像入力部
７ 認識処理部
１１ 画像統合部
１２，１２Ｓ，１２Ｆ，１２Ｂ_Ｓ ，１２Ｂ_Ｆ 特徴抽出部
１３，１３Ｓ，１３Ｆ，２６ 物体検出部
１５ 検出結果統合部
１９ 光強度計
２３Ｓ，２３Ｆ 対応づけ処理部
２４ 特徴統合部
２５ ３次元計測部[0001]
[Industrial applications]
The present invention processes an image obtained by imaging a predetermined observation area by a single or a plurality of imaging means, and processes the image in the observation area.The present invention relates to an image processing method and apparatus for recognizing a vehicle. Further, according to the present invention, a three-dimensional measurement process using images obtained by a plurality of imaging means is performed for an object within a predetermined observation area.The present invention relates to an image processing method and apparatus for performing various recognition processes such as presence / absence, size, position, and shape.
[0002]
[Prior art]
2. Description of the Related Art In recent years, an observation device using an image processing method has been developed in order to observe an object in a predetermined observation area such as a parking state of a vehicle in a parking lot or a traveling state of a vehicle on a road. In this type of observation device, a CCD camera (hereinafter simply referred to as a “camera”) is provided at a position above an observation target area, and images from the camera are continuously taken into a control device, and each input image is displayed. The feature of the object is extracted. When the object is a moving object such as a vehicle, the moving direction and speed of the object are measured based on the time-series data of the feature extraction result.
[0003]
Further, there has been proposed an apparatus in which a plurality of cameras are arranged in a predetermined positional relationship with respect to an observation area, and an image from each camera is taken into a control device to perform a three-dimensional measurement process of an object. .
[0004]
[Problems to be solved by the invention]
However, in this type of observation processing, large variations occur in the brightness in the observation area due to the effects of lighting conditions, etc. Brightness may vary greatly. Under such a situation, there arises a problem that the brightness of all the objects in the observation area cannot be within the dynamic range of the camera, and some objects cannot be recognized.
[0005]
The present invention has been made in view of the above problems,It is an object of the present invention to recognize a vehicle with high accuracy even when a plurality of vehicles having different brightness due to a difference in body color exist in an observation area, and to greatly improve recognition accuracy.
In addition, the present invention is also applicable to a case where a plurality of objects having different brightnesses exist in an observation area in recognition processing by three-dimensional measurement.It is an object of the present invention to accurately recognize each object and to greatly improve the recognition accuracy.
[0006]
[Means for Solving the Problems]
Claims 1 and 2The present invention relates to an image processing method for recognizing a vehicle. In the image processing method according to the first aspect of the present invention, the same observation region is respectively imaged by a plurality of exposure amounts corresponding to the type of vehicle body color of a vehicle, and a plurality of images generated by the imaging are captured.For each image, an edge representing the outline of the vehicle is extracted, and the edge extraction result for each image is integrated, and the integration result is obtained.Is used to recognize the vehicle in the observation area.
[0008]
Claim 2In the image processing method of the invention, the same observation region is respectively imaged by a plurality of exposure amounts according to the type of the vehicle body color of the vehicle, and the outline of the vehicle is respectively formed for each of the plurality of images generated by the imaging. Executing a process of extracting an edge to be represented, and a process of recognizing a vehicle in the observation region in association with the position using the extraction result, wherein the recognition position of the vehicle is included in the recognition result of each image. Differences between the respective vehicles are regarded as recognition results for individual vehicles, and the recognition results are integrated to finally recognize vehicles in the observation area.
[0009]
Claim 3The image processing method according to the invention is characterized in that a plurality of images with different exposure amounts are generated by a plurality of imaging units arranged toward a predetermined observation area, and the images generated by each imaging unit with the same exposure amount. For each set, after associating the feature points on each image between the images, integrating the association results for each set, and performing a three-dimensional measurement process using the integrated association results, The method is characterized in that an object in the observation area is recognized based on the measurement result.
[0011]
Claim 4The image processing apparatus according to the invention, image input means for individually inputting a plurality of images obtained by imaging the same observation region with a plurality of exposure amounts according to the type of vehicle body color of the vehicle, Edge extracting means for extracting an edge representing the outline of the vehicle from each of the input images; integrating means for integrating the edge extraction results for each of the input images; and the observation using the edge extraction results integrated by the integrating means. Recognizing means for recognizing vehicles in the area.
[0012]
Claim 5The image processing apparatus according to the invention, image input means for individually inputting a plurality of images obtained by imaging the same observation region with a plurality of exposure amounts according to the type of vehicle body color of the vehicle, Edge extraction means for extracting an edge representing the outline of the vehicle from each of the input images; and recognition means for recognizing the vehicle in the observation area in association with its position using the edge extraction result for each of the input images. Of the recognition results for each of the input images, those in which the recognition positions of the vehicles differ between the images are regarded as the recognition results for the individual vehicles, and the respective recognition results are integrated, and the final recognition of the vehicles in the observation region is performed. A recognition result integrating means for generating data indicating the recognition result.
[0013]
Claim 6The image processing apparatus according toClaim 4 or 5In the configuration, the exposure amount corresponding to each image is adjusted using the brightness detection result for the brightness detection unit that detects the brightness in the observation area and the imaging unit that supplies each input image. Exposure amount adjusting means is added.
[0014]
Claim 7The image processing apparatus according toClaim 4 or 5The image brightness detection means for detecting the brightness of each image input by the image input means; and the imaging means for supplying each input image, the exposure amount corresponding to each image is detected by the brightness detection. Exposure amount adjusting means for adjusting using the result is added.
[0015]
Claim 8The image processing apparatus according toClaim 4 or 5A timer for measuring the time is added to the configuration of the above. Based on the time measured by the timer, only one image with a specific exposure amount is input for a predetermined period of a day, and a vehicle in the observation area is input. Is configured to be recognized.
[0016]
Claim 9The image processing apparatus according to the invention is configured such that, from a plurality of imaging units arranged toward a predetermined observation area, an image input unit for inputting a plurality of images with different exposure amounts, and the same exposure amount by each imaging unit. For each set of generated input images, associating means for associating feature points on each image between the images, integrating means for integrating the associating result for each of the sets, A three-dimensional measuring means for performing the used three-dimensional measuring process, and a recognizing means for recognizing an object in the observation area using the three-dimensional measurement result are provided.
[0017]
Claim 10The image processing apparatus according toClaim 9In addition to the configuration of the above, a timer for measuring the time is further added, and based on the time measured by the timer, only one image with a specific exposure amount is input from each imaging unit for a predetermined period of one day. An object in the observation area is recognized by a three-dimensional measurement process using features corresponding to the input images.
[0018]
[Action]
By imaging the same observation area with different exposures, even if there are objects with different brightness in the observation area, each object can appear on the image with one of the exposures It becomes.
[0020]
Claims 1 and 4According to the present invention, edges representing the outline of the vehicle are extracted from a plurality of images with a plurality of exposure amounts corresponding to the types of vehicle body colors of the vehicle, and the edge extraction results are integrated. Therefore, even if a plurality of vehicles having different vehicle colors exist in the observation area, it is possible to obtain the edges on the images corresponding to the vehicle colors, and each vehicle can be obtained by using the integrated result of the edge extraction processing. Can be accurately recognized.
[0021]
Claims 2 and 5According to the invention, for a plurality of images with a plurality of exposure amounts according to the type of vehicle body color of a vehicle, an edge representing a contour of the vehicle is extracted, and a vehicle in an observation area is extracted using the edge extraction result. A recognition process is executed in association with the position. Therefore, even if a plurality of vehicles having different vehicle colors exist in the observation area, the vehicles can be accurately recognized on the images corresponding to the respective vehicle colors. Furthermore, when integrating the recognition results for each image, the recognition results in which the recognition positions of the vehicles differ between the images are regarded as the recognition results for the individual vehicles, so that all the vehicles in the observation area must be accurately recognized. Can be.
[0022]
Claims 6 and 7According to the invention, the exposure amount is adjusted using the brightness of the observation area or the input image with respect to the imaging unit that supplies each image, so even if the brightness in the observation area fluctuates due to a change in the surrounding environment, An image that clearly captures the characteristics of the vehicle can be generated.
[0023]
Claims 3 and 9According to the invention, when an observation area is imaged by a plurality of image pickup means and a three-dimensional measurement process is performed, a feature point on an image is associated with each set of images generated with the same exposure amount by each image pickup means. Is performed and the association results for each set are integrated, so that feature points relating to object points of all objects in the observation area can be associated. Therefore, by performing the three-dimensional measurement process using the integrated association result, the three-dimensional shape and the spatial position of each object can be accurately recognized.
[0024]
Claims 8 and 10According to the invention, the recognition process using only the image with the specific exposure amount is performed within a predetermined period of the day based on the time measured by the timer, so that a high-brightness object such as at night may be detected. It is possible to avoid unnecessary processing in a time zone that is not used.
[0025]
【Example】
FIG. 1 shows an installation example of a parking lot observation device to which the present invention is applied. This observation device is for recognizing whether or not a vehicle is parked in each of the parking areas 4 divided into a plurality of parking frames, and is installed near the parking area 4. The camera 1 and the control device 2 are fixedly supported on the support 3.
[0026]
The camera 1 is attached to the upper position of the column 3 via the support rod 3A, and images the parking area 4 from an obliquely upper position to generate an image including the entire area. This image is taken into the control device 2, and any of the recognition processes described later is performed, and it is recognized whether or not a vehicle exists in each parking frame. This recognition result is transmitted to an external device such as a management center by a transmission unit (not shown) inside the device.
[0027]
In the figure, C_B, C_WIs a parked vehicle, and vehicle C_BIs of a color with low reflectance such as black. In contrast, vehicle C_WIs a highly reflective color such as white, and under conditions where the entire imaging area is brightly illuminated, such as in fine weather, each vehicle C_B, C_WA noticeable difference appears in the upper brightness.
[0028]
In this observation device, the vehicle C_B, C_WAs described above, even when vehicles having significantly different brightness on the vehicle body are parked together, the camera 1 continuously captures images at two different shutter speeds so that each vehicle can be accurately recognized. In addition to performing the setting, the control device 2 performs the recognition process by integrally processing the images for each shutter speed.
Hereinafter, a specific configuration example of the observation device of FIG. 1 and a recognition processing method in each configuration example will be sequentially described.
[0029]
(1) First embodiment
FIG. 2 shows a configuration of the first observation processing device, which includes a shutter speed control device 5 in addition to the camera 1 and the control device 2 described above.
The shutter speed control device 5 is incorporated in the housing of the camera 1 or in the control device 2. The shutter speed control device 5 adjusts the charge accumulation time of the CCD image pickup device in accordance with the imaging timing of the camera 1, so that the shutter speed control device 5 has two types. Shutter speed S_S, S_F(S_S> S_F) Are set alternately.
[0030]
The control device 2 includes an image input unit 6 for capturing an image from the camera 1 and a recognition processing unit 7 for processing the captured image (the same applies to the following embodiments).
[0031]
An image input unit 6 includes an A / D conversion circuit 8 for digitally converting analog image data from the camera 1, two image memories 10S and 10F for storing image data for each shutter speed, and switching. And the like. The switching unit 9 is for switching the connection destination of the A / D conversion circuit 8 in accordance with the imaging operation of the camera 1, and has a lower shutter speed S_SI obtained by_SIndicates that the faster shutter speed S is stored in the first image memory 10S._FI obtained by_FAre respectively stored in the second image memory 10F.
[0032]
FIGS. 3A and 3B show the state shown in FIG._S, S_FImage I obtained by imaging_S, I_FIs shown.
Shutter speed S_SIs the vehicle C_BShutter speed (for example, 1/30 second) suitable for capturing a dark color such as the body color of the vehicle, and an image I generated by this shutter speed._SThen, as shown in FIG._BImages appear clearly. However, white C of body color_WWith respect to the vehicle, the brightness of the corresponding position on the image is saturated, and the vehicle C_WIs difficult to recognize.
[0033]
The other shutter speed S_FIs a short shutter speed (for example, 1/120 second) suitable for capturing strong reflected light, and this shutter speed S_FImage I by_FIn the above, as shown in FIG._WA clear image of appears. However, this image I_FVehicle C on_BThe part against is blurred black, and vehicle C_BIs difficult to recognize.
[0034]
Returning to FIG. 2, the recognition processing unit 7 of this embodiment includes an image integration unit 11, a feature extraction unit 12, an object detection unit 13, and the like. Each configuration of the recognition processing unit 7 is specifically realized by providing an algorithm for performing each process described below to a CPU of a computer (the same applies to the following embodiments).
[0035]
In this embodiment, the two types of shutter speeds S_S, S_FEach input image I by_S, I_FAre combined into one image, and the presence or absence of a vehicle is recognized for each vehicle frame using the characteristics of the combined image.
The image integration unit 11 is for performing a synthesizing process of each input image._S, I_F, The luminance value g of the pixel located at the i-th row and the j-th column, respectively._S(I, j), g_FBy applying (i, j) to the following principles (1) -1 to (1) -3, the luminance value G (i, j) of the corresponding pixel of the composite image is determined.
[0036]
(Equation 1)

[0037]
Note that in equation (1), th_S, Th_FIs a predetermined threshold value, th_SHas a sufficiently large luminance value,_F, A sufficiently small luminance value is set.
For example, when the luminance value of each pixel of the input image takes 8-bit data, th_SIs near 240, th_FIs set to around 10.
[0038]
Each of the input images I shown in FIGS._S, I_FIn the case of, black vehicle C_B(1) -1 is applied to the pixel position corresponding to, and the slow shutter speed S as the luminance value G (i, j) of the composite image_SInput image I by_SSide luminance value g_S(I, j) is adopted. Also white vehicle C_W(1) -2 is applied to the pixel position corresponding to, and the fast shutter speed S as the luminance value G (i, j) of the composite image_FInput image I by_FSide luminance value g_F(I, j) is adopted. Equation (1) -3 is applied to pixel positions where other vehicles do not exist, and each luminance value g_S(I, j), g_FThe luminance value G (i, j) of the composite image is obtained using the average value of (i, j).
Note that the method of synthesizing the image is not limited to the method described above._S(I, j) and g_FIt may be represented by an average value with (i, j).
[0039]
FIG. 4 shows each input image I_S, I_FShows the result of the synthesis processing by applying the above (1) -1 to (1) -3._B, C_WBy using the data on the image side that accurately captures the vehicle in the portion of_B, C_WAn image that clearly captures the image has appeared.
[0040]
The feature extraction unit 12 extracts an edge representing the outline of the vehicle by scanning an edge extraction filter on the composite image. The object detection unit 13 determines whether or not a vehicle exists, for example, by calculating the area of an edge portion in each parking frame based on the position of each parking frame on the image stored in the internal memory. This determination result is associated with the identification data (label, position data, and the like) of each parking frame and output to the outside.
[0041]
(2) Second embodiment
FIG. 5 shows a second configuration of the observation device.
The control device 2 of this embodiment also uses the image input unit 6 having the same configuration as that of the first embodiment to control each shutter speed S_S, S_FI obtained by_S, I_FAre entered individually. Further, the recognition processing unit 7 includes the respective configurations of two sets of feature extraction units 12S and 12F, a feature integration unit 14, and an object detection unit 13. Each of the input images I and_S, I_FAre extracted individually.
[0042]
FIGS. 6 (1) and (2) correspond to FIGS. 3 (1) and (2).Input image I _S , I _F Image E generated by edge extraction processing for_S, E_FIs shown. Here, in order to clarify the extraction position of each edge, it is illustrated that all edges of the parking frame on the input image are extracted.
[0043]
Edge image E_SAbove, two vehicles C_B, C_WOf the parking positions A and B of the vehicle C clearly appearing on the image_BThe edge is extracted only at the parking position A (the edge indicated by the thin dotted line at the parking position B in the figure is virtual). Conversely, the edge image E_FAbove, vehicle C_WThe edge is extracted only at the parking position B of.
[0044]
Returning to FIG. 5, the feature integration unit 14_S, I_FAre integrated to generate one edge image. This integration process is performed as shown in the following equation (2)._S, E_FPixel value b for each corresponding pixel of_S(I, j), b_FThe logical sum of (i, j) (indicated by “|” in the equation (2)) is obtained, and is set as the pixel value of the pixel B (i, j) of the edge image after integration. That is, the edge image formed by each pixel value B (i, j) is the_S, E_FAre superimposed, and all the edge extraction results are adopted by the integration processing.
[0045]
(Equation 2)

[0046]
FIG. 7 shows a result obtained by integrating the respective edge extraction results of FIGS. 6 (1) and (2) by using the above equation (2)._B, C_WThe edges also appear with high precision at the parking positions A and B.
[0047]
The object detection unit 13 determines the presence or absence of a vehicle for each parking frame on the integrated edge image as in the first embodiment, and outputs the determination result._B, C_WCan be accurately detected.
[0048]
(3) Third embodiment
FIG. 8 shows a third configuration of the observation device. The control device 2 of this embodiment uses the same image input unit 6 as in the first and second embodiments to control two types of shutter speeds S._S, S_FI taken by_S, I_FAre individually input, and then each input image I_S, I_FAre individually processed to detect a vehicle on each image, and the respective detection results are integrated to make a final determination.
[0049]
The recognition processing unit 7 includes a feature extraction unit 12S, 12F and an object detection unit 13S, 13F for each input image, and also includes a detection result integration unit 15. Each of the feature extraction units 12S and 12F respectively controls the input image I_S, I_FIs performed, and an edge image E indicating the result is obtained._S, E_FGenerate Each of the object detection units 13S and 13F outputs the corresponding edge image E_S, E_FIn the above, it is determined whether or not a vehicle is present in each parking frame. Each determination result is output to the detection result integration unit 15 in a form associated with a label indicating the vehicle frame. The detection result integration unit 15 integrates the vehicle detection results on each edge image by taking the logical sum of the determination results for each parking frame, and outputs this integration result as the final vehicle detection result.
[0050]
FIG. 9 shows the edge images E of FIGS. 6 (1) and (2)._S, E_FIs the result of determining the presence or absence of a vehicle for each parking frame, using the label number given to each parking frame in the leftmost column in the figure, associating each label number with each edge. Image E_S, E_FThe determination result of the presence or absence of each vehicle and the final integration result are shown. It should be noted that data “1” in the drawing represents the determination result of “vehicles”, and data “0” represents the determination result of “no vehicles”.
[0051]
In the illustrated example, each parking frame on the image is provided with a label number in order from left to right, and a vehicle C in a parking frame with a label number “1” (corresponding to the parking position A)._BIs the edge image E_SVehicle C detected in the above and in the parking frame (corresponding to the parking position B) with the label number “8”_WIs the edge image E_FHas been detected above.
Therefore, as a result of integrating the two detection results, it is determined that the vehicle exists in any of the “1” and “8” parking frames.
[0052]
(4) Modifications of Embodiments 1 to 3 <1>
In each of the three embodiments described above, two shutter speeds S are used._S, S_FIs used to perform the vehicle detection process, but the invention is not limited to this. The vehicle detection process may be performed by generating images with three or more shutter speeds.
[0053]
10 to 12 show configurations obtained by developing the first to third embodiments, respectively. The image input unit 6 stores images obtained at N different (N ≧ 3) shutter speeds. N image memories 10M for storing₁-10M_NIs deployed.
[0054]
In the embodiment of FIG. 10, after the image integration unit 11 of the recognition processing unit 7 calculates the average luminance value for each corresponding pixel between the input images and integrates the images, the feature extraction unit 12 Edge extraction processing on the integrated image is performed. Further, the object detection unit 13 performs detection processing of each vehicle using the edge extraction result, and outputs the result.
[0055]
The recognition processing unit 7 in the embodiment of FIG. 11 includes a feature extraction unit 12A for each input image.₁~ 12A_N, And an edge extraction process is performed for each input image. The feature integration unit 14 integrates each edge image by performing a logical sum operation between corresponding pixels on the edge image obtained by the edge extraction processing, as in the second embodiment. Thereafter, similarly to the embodiment of FIG. 10, the vehicle detection process is performed by the object detection unit 13.
[0056]
The recognition processing unit 7 in the embodiment of FIG. 12 includes a feature extraction unit 12A for each input image.₁~ 12A_NAnd object detection unit 13A₁~ 13A_NWith these configurations, the vehicle detection process is individually performed for each input image. Thereafter, similarly to the third embodiment, the detection result integrating unit 15 integrates the respective detection results, generates data indicating the final detection result, and outputs the data to the outside.
[0057]
If the recognition process is performed using images at a plurality of shutter speeds as described above, the dynamic range of the camera 1 can be set wider, and even if vehicles of various colors are parked, each vehicle can be controlled. It is possible to detect with high accuracy.
[0058]
(5) Example 4
In each of the first to third embodiments, the shutter speed of one camera 1 is changed to continuously capture images. On the other hand, in the fourth embodiment, as shown in FIG. 13, two cameras 1S and 1F with different shutter speeds are provided, and the cameras 1S and 1F are operated simultaneously.
[0059]
FIG. 14 shows the configuration of the observation device shown in FIG. 13. The image input unit 6 of the control device 2 includes A / D conversion circuits 8S and 8F and image memories 10S and 10F for each of the cameras 1S and 1F. Be deployed.
Since the configuration of any of the first to third embodiments may be introduced into the recognition processing unit 7 of this embodiment, illustration and description of the detailed configuration of the recognition processing unit 7 will be omitted.
[0060]
However, when the configuration of the first or second embodiment is introduced into the recognition processing unit 7, the cameras 1S and 1F are set as much as possible with their optical axes parallel so as not to hinder image integration processing. They need to be deployed close together. Further, at the time of image integration processing, in order to correct the displacement due to the parallax between the cameras 1S and 1F, any one of the images is displaced by the number of pixels corresponding to the distance between the optical axes of the cameras (FIG. It is necessary to shift in the vertical direction in the example shown.
[0061]
(6) Fifth embodiment
In this embodiment, similar to the installation example in FIG. 1, the vehicle detection process is performed by capturing an image by one camera. As shown in FIG. A camera 1X having the CCD image pickup devices 17S and 17F is used. The shutter speed S is applied to each of the CCD image sensors 17S and 17F._S, S_FIs set, and the light that has passed through the camera lens 16 is simultaneously given to each of the CCD elements 17S and 17F, and two images I that capture this light image at different shutter speeds are set._S, I_FIs generated.
[0062]
The image input unit 6 of the control device 2 is provided with an A / D conversion circuit 8S, 8F and an image memory 10S, 10F for each input image so as to simultaneously receive image data from each of the CCD image pickup devices 17S, 17F. Is provided.
Note that any of the first to third configurations may be introduced into the recognition processing unit 7 as in the fourth embodiment, so that illustration and detailed description are omitted here.
[0063]
According to the configuration of this embodiment, for the same observation area, each shutter speed S_S, S_FImages can be simultaneously captured and processed, so that there is no need to shift the time between input images, and it is necessary to shift one of the images during image integration processing as in the fourth embodiment. Nor. Therefore, it is possible to improve the detection accuracy of the vehicle and shorten the processing time.
[0064]
(7) Sixth embodiment
Also in this embodiment, each shutter speed S is controlled by a camera 1X having the same two CCD image pickup devices 17S and 17F as in the fifth embodiment._S, S_FAre simultaneously generated.
FIG. 16 shows a configuration of the sixth embodiment. The control device 2 includes an image input unit 6 having the same configuration as that of the previous embodiment, and further includes an image integration unit 11 and a monitor 18. I have.
[0065]
Each image generated by the camera 1X is provided to the image integration unit 11 via the image input unit 6, and integrated by the same method as in the first embodiment. The monitor 18 is for displaying an image generated by the integration process, so that even when vehicles having different brightnesses exist in the parking area 4, each vehicle is observed on one monitor screen. It becomes possible.
[0066]
The monitor 18 can be installed at a location away from the parking area 4 and the integrated image data can be transmitted from the main body of the control device 2 to the monitor 18 via a communication line. In this way, even in an area where the parking area 4 cannot be directly monitored, the situation of the parking area 4 can be accurately grasped.
[0067]
In addition to the configuration of this embodiment, the configuration of any one of the above-described first to third recognition processing units 7 is added, and the result of the presence / absence determination of the vehicle for each parking frame is displayed on the monitor 18 together with the integrated image. Can also be displayed.
[0068]
(8) Seventh embodiment
FIG. 17 shows a seventh configuration of the observation device. In this embodiment, a light intensity meter 19 is added to any one of the first to third configurations to detect the brightness in the observation area of the camera 1, and each shutter speed of the camera 1 is determined according to the degree of the brightness. S_S, S_FIs variably set.
[0069]
FIG. 18 shows an example of setting the shutter speed.
The light intensity meter 19 receives light from the observation area and detects the light intensity._S, S_FAre provided in the memory of the shutter speed control device 5, and the detected value λ of the light intensity is set to a predetermined threshold value th1, th2 (th1). > Th2), each shutter speed S_S, S_F, One of the set values is adopted.
[0070]
As described above, since the shutter speed of the camera 1 is controlled based on the degree of brightness in the observation area, a larger dynamic range can be set for the camera 1, and even if the brightness in the observation area fluctuates, a stable vehicle can be obtained. Detection can be performed.
[0071]
(9) Eighth embodiment
FIG. 19 shows an eighth configuration of the observation device. In this embodiment, as in the seventh embodiment, each of the shutter speeds S_S, S_FIs added, but here, the shutter speed S is determined by the brightness of each input image instead of the brightness in the observation area._S, S_FIs set to fluctuate.
[0072]
The control device 2 includes an image input unit 6 similar to each of the first to third configurations, a recognition processing unit 7 having the same configuration as one of these three configurations, and two luminance average calculation units 20S and 20F. Is done. These luminance average calculation units 20S and 20F are adapted to input image I stored in image memories 10S and 10F._S, I_F, An average of the luminance values of the respective constituent pixels is calculated and output to the shutter speed control device 5. The shutter speed control device 5 individually fetches each calculated value and sets each shutter speed S_s, S_FIs determined, and the shutter speed of the camera 1 is controlled based on the determined value at the time of the next imaging.
[0073]
At the start of processing, each shutter speed S_S, S_FIs set to a predetermined initial value, and is adjusted to an optimum value by the imaging operation in the next several cycles.
[0074]
FIG. 20 shows a procedure for determining the shutter speed. Here, the slower shutter speed S_SIs determined, the faster shutter speed S_FIs determined in a similar procedure.
[0075]
In the first step 1 (indicated by “ST1” in the figure), the luminance average calculation unit 20S sends the input image I_SIs read, and the average luminance value AV_SIs calculated. In the next steps 2 and 3, the shutter speed control device 5 sets the calculated value AV_SIs the optimum value M of the average luminance value._S, And using the comparison result, the shutter speed S_SDetermine whether to keep or change the current value of
[0076]
The optimum value M of the average luminance value_SIs a previously registered value in the internal memory, and the calculated average brightness value AV_SAnd the optimal value M_SIs a predetermined threshold Δ_s2From Δ_s1(Δ_s2<Δ_S1) Is “NO” in steps 2 and 3 and the shutter speed S_SIs maintained (step 6).
[0077]
Input image I_SIs too bright, the calculated average brightness value AV_SAnd the optimal value M_SAnd the threshold Δ_S1Is generated. In this case, step 2 becomes "YES" and the process shifts to step 4, where the shutter speed S_SIs updated to １ ／.
[0078]
Conversely, the input image I_SWhen the brightness of the average brightness value decreases, the calculated value AV of the average brightness value_SAnd the optimal value M_SIs the threshold Δ_s2Below. In this case, step 3 becomes “YES” and the process proceeds to step 5, where the shutter speed S_SIs updated twice.
[0079]
Note that the shutter speed S in steps 4 and 5 above is used._SIs not limited to the illustrated example, and for example, a value of a predetermined frequency may be added (or subtracted) to the current set value.
[0080]
Thus, the shutter speed S at the next stage is_S, S_FTo the corresponding input image I_S, I_F, The respective shutter speeds S based on the actually obtained brightness on the image._S, S_FCan be adjusted appropriately.
It should be noted that in the fifth and sixth embodiments shown in FIGS. 15 and 16, the configuration for adjusting the shutter speed of the seventh and eighth embodiments can be introduced.
[0081]
(10) Ninth embodiment
The configuration of each of the above-described embodiments is based on the premise that processing is performed during the day when observation is possible under a predetermined brightness, and under conditions where the entire observation area is dark, such as at night, There is a possibility that no image of any vehicle appears on an image obtained at a high shutter speed.
[0082]
The ninth configuration shown in FIG. 21 performs recognition processing by synthesizing images at a plurality of shutter speeds. At nighttime, this processing is suspended and normal recognition processing using one image is performed. Is implemented.
[0083]
The switching of the processing during the night time is performed based on the time counted by the timer 21 inside the control device 2. When the time counted by the timer 21 is within a predetermined period, the camera 1 has the slowest shutter speed. Only is set. In the figure, reference numeral 22 denotes a second or lower image memory 10M.₂-10M_NAnd a switching unit for turning on and off the connection between the camera and the recognition processing unit 7. When a single shutter speed is set in the camera 1, the switching unit 22 is set to off. At the same time, the function of the image integration unit 11 of the recognition processing unit 7 is suspended, and the feature extraction unit 12 and the object detection unit 13 cause the image memory 10M₁A recognition process using only the image data from is performed.
[0084]
In the illustrated example, the control device 2 employs the configuration shown in FIG. 10 for the recognition processing unit 7. However, in the case of adopting another configuration, similarly, it is set so that the process is switched between daytime and nighttime. can do.
[0085]
By switching the processing at night, unnecessary calculation processing at night can be reduced. In addition, there is no possibility that an error occurs in the detection result due to the unnecessary calculation processing, and the accuracy of vehicle detection can be stabilized.
[0086]
(11) Tenth embodiment
FIG. 22 shows an example of installation of an observation device when the tenth configuration is used. This observation device determines the presence or absence of a vehicle in each parking frame by a three-dimensional measurement process using images from two cameras 1A and 1B. Thus, they are arranged vertically with the optical axes parallel and the imaging planes positioned on the same plane.
[0087]
In this embodiment, similarly to the first to eighth embodiments, the vehicle C having different brightness is used._B, C_WAre set to a plurality of shutter speeds in each of the cameras 1A and 1B so as to continuously detect images. In the control device 2, for each set of images obtained from each of the cameras 1A and 1B at the same shutter speed, processing from extraction of a feature point indicating a vehicle to correspondence of the feature points is performed. The results are integrated to perform a three-dimensional measurement process.
[0088]
FIG. 23 shows a configuration of the observation device.
The shutter speed control device 5 has two fixed shutter speeds S, as in the first to third embodiments._S, S_FAre alternately set. Each of the cameras 1A and 1B is controlled by the shutter speed control device 5 so that each of the shutter speeds S_S, S_FAre simultaneously generated.
[0089]
The control device 2 includes an image input unit 6 and a recognition processing unit 7 as in the above embodiments. The image input unit 6 includes A / D conversion circuits 8A and 8B and two sets of image memories (10A) for each camera._S, 10A_F) (10B_S, 10B_F). One image memory 10A of each set_S, 10B_SShows the slower shutter speed S from each of the cameras 1A and 1B._SImage U by_S, L_SIs stored. The other image memory 10A_F, 10B_FIs the faster shutter speed S from each camera._FImage U by_F, L_FIs stored.
[0090]
The recognition processing unit 7 includes two sets of feature extraction units 12B._S, 12B_F, Two sets of association processing units 23S and 23F, and also includes respective components such as a feature integration unit 24, a three-dimensional measurement unit 25, and an object detection unit 26. Feature extraction unit 12B_S, 12B_FAre images L obtained by the lower camera 1B, respectively._S, L_FIs for extracting an edge on an image, and the feature extraction unit 12B_SImage L by_SIs extracted by the first association processing unit 23S and the feature extraction unit 12B_FImage L by_FAre individually given to the second association processing unit 23F.
[0091]
The first association processing unit 23S is a shutter speed S that is slower than each of the cameras 1A and 1B._SInput image U obtained in_s, L_sAre for identifying feature points representing the same object point in space and associating them with each other.
[0092]
This association processing is specifically performed by the feature extraction unit 12B._SImage L_SFor each of the edge constituent points P extracted above, an image U_SThis is performed by extracting the corresponding point Q above.
[0093]
FIG. 24 shows a specific method of extracting a corresponding point Q to the point P.
First, the associating processing unit 23S outputs the edge image EL generated by the edge extraction processing._SThe predetermined feature point P (x, y_L), The original image L_SAt a position of the upper point P, a window W of a predetermined size centered on the point P_LSet. Next, the associating processing section sets the image U_SThe window W is placed on the epipolar line EP at the point P above._LWindow W of the same size as_UIs set and scanning is performed, and the following equation (3) is executed for each scanning position, and each window W_L, W_UDF of the image data in the image data is calculated.
[0094]
In the equation (3), g_L(X, y) is the window W_LThe luminance value of a given pixel within_U(X, y) is the window W_UThe luminance value of a predetermined pixel in each is shown. SZ is each window W_L, W_UIndicates the size of Further, m and n are variables for specifying a pixel in each window, and0 to the above size SZFluctuates within the range.
[0095]
(Equation 3)

[0096]
The associating processing unit 23S compares the dissimilarity DF obtained at each scanning position, and determines the window W at the time when the dissimilarity DF becomes the smallest._UCenter point (x, y_U) Is specified as the corresponding point Q of the point P.
[0097]
As mentioned above, each camera 1A, 1BVertically aligned and the optical axisSince they are arranged in parallel, the epipolar line EP is perpendicular to the x-axis and the window W_U Can be easily set, and the calculation of the difference DF can be simplified. Further, instead of calculating the difference DF at each scanning position on the epipolar line EP, the image U_S Edges may also be extracted above, and among these, edge constituent points located on the epipolar line EP may be set as corresponding candidate points of the feature point P, and the difference DF may be calculated only for these corresponding candidate points. .
[0098]
Returning to FIG. 23, the second association processing unit 23F outputs a shutter speed S faster than each of the cameras 1A and 1B._FInput image U obtained by_F, L_FAre performed in the same manner as described above, and the results of the association performed by the association processing units 23S and 23F are integrated by the feature integration unit 24.
[0099]
This integration process employs all the sets of feature points associated by the respective association processing units 23S and 23F, and the coordinate data for each corresponding feature point pair is provided to the three-dimensional measurement unit 25. . The three-dimensional measuring unit 25 calculates the coordinates (x, y) of each feature point for each given set of feature points._u) (X, y_L) Is applied to the principle of triangulation to calculate three-dimensional coordinates corresponding to each feature point.
[0100]
In this way, three-dimensional coordinates corresponding to each set of feature points are calculated and provided to the object detection unit 26. The object detection unit 26 determines whether or not a vehicle exists in each parking frame from the number of feature points at each parking frame position and its height data.
[0101]
In the tenth embodiment, for the sake of simplicity, the shutter speed S_S, S_FIs a fixed value, but as in the seventh and eighth embodiments, each shutter speed S depends on the brightness of the observation area and the brightness of the input image._S, S_FMay be variably set. Also, the shutter speed S_S, S_FIs not limited to one type, and as shown in FIGS. 10 to 12, a plurality of shutter speeds are set, feature points are associated between images for each shutter speed, and each association result is displayed. You may make it unify. Further, as shown in the ninth embodiment, at night, each camera may be imaged only at a slow shutter speed, and the three-dimensional measurement processing may be performed using only one set of images.
[0102]
(12) Other embodiments
In each of the above-described embodiments, the camera generates a plurality of images having different shutter speeds. However, not only the shutter speed but also the aperture of the camera and the gain of the output voltage from the camera are adjusted. In this way, it is also possible to individually generate images of objects having different brightness.
[0103]
Further, here, the respective configuration examples in the parking lot observation device have been described, but the observation target is not limited to the vehicle, but may be another object, and the observation region may be indoors. Further, the observation device of this embodiment is only for recognizing the presence or absence of a vehicle for each parking frame, but is not limited thereto, and may be configured to recognize the size, shape, etc. of the vehicle. The vehicle color of each vehicle may be recognized using the generation camera.
[0104]
Further, the parking lot observation device individually shown targets a stationary vehicle as a recognition target, but the present invention can be applied to a mobile object such as a vehicle running on a road as a recognition target. . However, in this case, if the images for each shutter speed are continuously generated, a shift of the movement of the object occurs between the images. Therefore, it is desirable to introduce the configuration of the fourth or fifth embodiment.
[0105]
【The invention's effect】
Claims 1 and 2 and Claims 4 and 5According to the invention, even if a plurality of vehicles having different vehicle colors exist in the observation area, each vehicle can be caused to appear on an image with an exposure amount corresponding to the vehicle color, and vehicles of various colors can be displayed. It can be recognized with high accuracy.
[0108]
Claims 6 and 7According to the invention, the exposure amount is adjusted using the brightness of the observation area or the input image with respect to the imaging unit that supplies each image, so even if the brightness in the observation area fluctuates due to a change in the surrounding environment, The vehicle can be clearly recognized by each image, and stable recognition processing can be performed.
[0109]
Claims 3 and 9According to the invention, when an observation area is imaged by a plurality of image pickup means and a three-dimensional measurement process is performed, a feature point on an image is associated with each set of images generated with the same exposure amount by each image pickup means. After performing the above, the three-dimensional measurement processing is performed by integrating the association results of each group. Therefore, even when there are objects having different brightnesses in the observation area, the three-dimensional shape and space of each object are obtained. The position can be accurately recognized.
[0110]
Claims 8 and 10According to the invention, the recognition process using only the image with the specific exposure amount is performed within a predetermined period of the day based on the time measured by the timer, so that a high-brightness object such as at night may be detected. It is possible to reduce unnecessary equipment costs by avoiding unnecessary processing during non-period time zones. In addition, erroneous measurement due to unnecessary processing can be prevented.
[Brief description of the drawings]
FIG. 1 is a perspective view showing an installation example of an observation device to which the present invention is applied.
FIG. 2 is a block diagram showing a first configuration of the observation device.
FIG. 3 is an explanatory diagram illustrating an example of an input image.
FIG. 4 is an explanatory diagram showing a composite image of the input image of FIG. 3;
FIG. 5 is a block diagram showing a second configuration of the observation device.
FIG. 6 is an explanatory diagram illustrating an edge image obtained by performing edge extraction processing on an input image.
FIG. 7 is an explanatory diagram showing a result of integrating the edge images of FIG. 6;
FIG. 8 is a block diagram showing a third configuration of the observation device.
FIG. 9 is an explanatory diagram showing a determination result of the presence or absence of a vehicle for each parking frame.
FIG. 10 is a block diagram illustrating a modified example of the first configuration of the observation device.
FIG. 11 is a block diagram showing a modified example of the second configuration of the observation device.
FIG. 12 is a block diagram illustrating a modified example of the third configuration of the observation device.
FIG. 13 is a perspective view showing an installation example of an observation device having a fourth configuration.
FIG. 14 is a block diagram showing a fourth configuration of the observation device.
FIG. 15 is a block diagram showing a fifth configuration of the observation device.
FIG. 16 is a block diagram showing a sixth configuration of the observation device.
FIG. 17 is a block diagram showing a seventh configuration of the observation device.
FIG. 18 is an explanatory diagram showing an example of setting a shutter speed in the observation device of FIG. 17;
FIG. 19 is a block diagram showing an eighth configuration of the observation device.
20 is a flowchart showing a procedure for setting a shutter speed in the observation device of FIG. 19;
FIG. 21 is a block diagram showing a ninth configuration of the observation device.
FIG. 22 is a perspective view showing an installation example of an observation device having a tenth configuration.
FIG. 23 is a block diagram showing a tenth configuration of the observation device.
FIG. 24 is an explanatory diagram showing a specific example of a feature point association process;
[Explanation of symbols]
1,1S, 1F, 1X, 1A, 1B camera
2 Control device
5 Shutter speed control device
6 Image input section
7 Recognition processing unit
11 Image Integration Unit
12, 12S, 12F, 12B_S, 12B_F  Feature extraction unit
13, 13S, 13F, 26 Object detector
15 Detection result integration unit
19 Light intensity meter
23S, 23F Correlation processing unit
24 Feature Integration Unit
25 3D measurement unit

Claims (10)

An image processing method for recognizing a vehicle,
Each of the same observation areas is imaged with a plurality of exposure amounts corresponding to the type of vehicle body color,
For each of the plurality of images generated by the imaging, each edge representing the contour of the vehicle is extracted,
An image processing method comprising: integrating a feature point extraction result for each image; and recognizing a vehicle in the observation area using the integration result. An image processing method for recognizing a vehicle,
Each of the same observation areas is imaged with a plurality of exposure amounts corresponding to the type of vehicle body color,
For each of the plurality of images generated by the imaging, a process of extracting an edge representing a contour of the vehicle, and a process of recognizing a vehicle in the observation area in association with the position using the extraction result. Run,
Among the recognition results for each image, those in which the recognition position of the vehicle is different between the images are regarded as the recognition results for the individual vehicles, and the respective recognition results are integrated to finally recognize the vehicle in the observation area. An image processing method characterized by the following. A plurality of images with different exposure amounts are generated by a plurality of imaging units arranged toward a predetermined observation area. Each image set generated by each imaging unit with the same exposure amount has a plurality of images. After associating the feature points between the images, the association results for each set are integrated,
An image processing method, comprising: performing a three-dimensional measurement process using the integrated association result, and recognizing an object in the observation region based on the measurement result. An image processing device for recognizing a vehicle,
Image input means for individually inputting a plurality of images obtained by imaging the same observation region with a plurality of exposure amounts according to the type of vehicle body color,
Edge extraction means for extracting an edge representing the outline of the vehicle from each of the input images,
Integrating means for integrating the edge extraction result for each input image,
An image processing apparatus comprising: recognition means for recognizing a vehicle in the observation area using the edge extraction result integrated by the integration means. An image processing device for recognizing a vehicle,
Image input means for individually inputting a plurality of images obtained by imaging the same observation region with a plurality of exposure amounts according to the type of vehicle body color,
Edge extraction means for extracting an edge representing the outline of the vehicle from each of the input images,
For each of the input images, a recognition unit that recognizes a vehicle in the observation area in association with the position using the edge extraction result,
Among the recognition results for each of the input images, the recognition position of the vehicle that differs between the images is regarded as the recognition result for each individual vehicle, and the respective recognition results are integrated, and the final recognition of the vehicle in the observation area is performed. An image processing apparatus comprising: a recognition result integrating unit that generates data indicating a result. The image processing apparatus according to claim 4 , wherein
Further, an exposure amount for adjusting the exposure amount corresponding to each image using the brightness detection result with respect to a brightness detection unit that detects the brightness in the observation region and an imaging unit that supplies each input image. An image processing apparatus comprising an adjusting unit. The image processing apparatus according to claim 4 , wherein
Further, image brightness detection means for detecting the brightness of each image input by the image input means, and imaging means for supplying each input image, the exposure amount corresponding to each image, the brightness detection result An image processing apparatus comprising: an exposure amount adjusting unit that adjusts using the same. The image processing apparatus according to claim 4 , wherein
Further, a timer for measuring the time is provided. Based on the time measured by the timer, during a predetermined period of the day, only one image with a specific exposure is input to recognize a vehicle in the observation area. Image processing device configured as described above. From a plurality of imaging means arranged toward a predetermined observation area, an image input means for inputting a plurality of images with different exposure amounts,
For each set of input images generated with the same exposure amount by each imaging unit, associating means for associating feature points on each image between the images,
Integrating means for integrating the association result of each set,
Three-dimensional measurement means for performing three-dimensional measurement processing using the integrated correspondence result;
An image processing apparatus comprising: recognition means for recognizing an object in the observation area using the three-dimensional measurement result. The image processing device according to claim 9 ,
Further, a timer for measuring the time is provided, and based on the time measured by the timer, only one image with a specific exposure amount is input from each of the imaging means for a predetermined period of the day. An image processing apparatus configured to recognize an object in the observation area by three-dimensional measurement processing using corresponding feature points.

Images