JP4188632B2 - Range image integration method and range image integration apparatus - Google Patents

Description

を満たすＲを求める。そこで、｛ｅ_ｉ｝，｛ｆ_ｉ｝からなる相関行列Ｋを以下のように定義する。
【００５９】
ここでは、上付きＴは転置行列を示している。
【００６０】
【数２】

ここで、相関行列Ｋの特異値分解によって下記式が得られる。ここで、行列Ｖ，Ｕは正規直交行列である。
【００６１】
【数３】

すると、回転パラメータＲは以下のように求めることができる。
【００６２】
【数４】

上記のように求められた、回転パラメータＲを使用して下記の処理を行う。
【００６３】
３） 視点Ａの全ての平面グループの法線ベクトルｎ_ｉに対して前記求めた回転パラメータ（回転行列）を掛ける。
Ｒｎ_ｉ＝ｍ_ｊ …（７）
４） 視点Ｂにおいて、上記ｍ_ｊと等しい法線ベクトルを持つ平面グループが存在すれば、法線ベクトルｎ_ｉを持つ平面グループと法線ベクトルｍ_ｊを持つ平面グループの類似度をＳＵＭに加算する。
【００６４】
もし、ｍ_ｊと等しい平面グループが視点Ｂになければ、視点Ａの法線ベクトルｎ_ｉを持つ平面グループしか存在しない場合の類似度をＳＵＭに加算する。
５） ３×３の回転パラメータ（回転行列）の逆行列Ｒ^−１を求める。
【００６５】
６） 次に視点Ｂの平面グループの法線ベクトルｍ_ｊに対して回転パラメータ（回転行列）を掛ける。
逆行列Ｒ^−１ｍ_ｊ＝ｎ_ｉ …（８）
７） 次に視点Ａにおいて、ｎ_ｉと等しい法線ベクトルを持つ平面グループが存在すれば、法線ベクトルｎ_ｉを持つ平面グループと法線ベクトルｍ_ｊを持つ平面グループの類似度をＳＵＭに加算する。
【００６６】
もし、ｎ_ｉと等しい平面グループが視点Ａになければ、視点Ｂの法線ベクトルｍ_ｊを持つ平面グループしか存在しない場合の類似度をＳＵＭに加算する。
上記のようにして、平面グループ間の類似度演算を行う。
【００６７】
なお、上記説明では、視点Ａ、視点Ｂで説明したが、視点Ａ、視点Ｃにおいても、同様に処理する。
（Ｓ５０）
次のＳ５０では、今回算出した類似度の和（ＳＵＭの値）と過去に算出した最大の類似度の和（過去に算出したＳＵＭの値）と比較して、今回の類似度の和（今回値）が、過去の最大の類似度の和よりも大きいか否かを判定する。今回値の方が大きければ、Ｓ６０に移行し、そうでなければ、Ｓ７０に移行する。
【００６８】
（Ｓ６０）
Ｓ６０では、今回値を最大の類似度の和として更新するとともに、その対応する平面グループと、シフト量を記憶手段に格納する。
【００６９】
（Ｓ７０）
Ｓ７０では、全ての平面グループの組合せを探索したかを判定し、全ての平面グループの探索が終了していない場合には、Ｓ１０に戻る。全ての平面グループの探索が終了した場合には、Ｓ８０に移行する。
【００７０】
（変換パラメータ推定部１７）
変換パラメータ推定部１７は、図１０のＳ８０の処理を行う。
すなわち、Ｓ８０では、前記記憶手段に記憶させた対応する平面グループを基に変換パラメータとしての回転パラメータ及び並進パラメータを推定する。
【００７１】
ここでは、精度良く、回転パラメータ（回転行列）を推定するため、３つの平面グループ以外の互いに対応する平面グループがないかを調べる必要がある。この場合、対応関係があったとしても、全体の誤差を大きくするような、「外れ平面グループ」を持つ組合せは、削除する必要がある。
【００７２】
そこで、Ｓ８０では、下記の手順で、外れ平面グループを検出し、回転パラメータを求める際に、この外れ平面グループを含む、平面グループの組合せを除外する。
【００７３】
（外れ平面グループの検出及び除外）
１） 平面グループ間の角度差が互いに等しい平面グループ同士の法線ベクトルｎp11とｍp21、ｎp12とｍp22、ｎp13とｍp23との内積を用いて、誤差εを下記式（９）で求める。なお、ｎp11は平面グループＰ11の法線ベクトルの意味であり、他の法線ベクトルｎ，ｍも同様に平面グループの符号に対応して付している。
【００７４】
ε＝(ｎp11・ｍp21-1)²+(ｎp12・ｍp22-1)²+(ｎp13・ｍp23-1)²
…（９）
２） 誤差εに対して、Ｍｅｄｉａｎ（メディアン）をとり、
ε≧（Ｍｅｄｉａｎ × ２．０）
となる平面グループを外れ平面グループとする。
【００７５】
上記の条件で、外れ平面グループを含む平面グループの組合せを排除する。
そして、上記のように、外れ平面グループの組合せを排除して、Ｓ４０と同様の方法により回転パラメータを推定する。
【００７６】
並進パラメータの推定は下記の通りに行う。
まず、対応する平面グループに関する３つの法線ベクトルｎ１，ｎ２，ｎ３とそれぞれのシフト量ｄ１，ｄ２，ｄ３から、それぞれの法線ベクトルと垂直で、かつ原点からそれぞれのシフト量だけ移動した３つの平面を得る。
【００７７】
なお、
ｎ１＝（ａ１，ｂ１，ｃ１）
ｎ２＝（ａ２，ｂ２，ｃ２）
ｎ３＝（ａ３，ｂ３，ｃ３）
とする。
【００７８】
それらの３つの平面の交点（図１３参照）は、１つの視点に対する他の１つの視点の平行移動量Ｔとなる。ここで、並進パラメータはＲＴとなる。平面グループマッチング部１６により既に求めた回転パラメータ（回転行列）と平行移動量Ｔを基に、１つの視点の座標系を下記式（１０）を使用して他の１つの視点の座標系に変換する。
【００７９】
ここでは、説明の便宜上、視点Ｂの座標系Ｐｂを視点Ａの座標系Ｐａに変換する場合を説明する。
Ｐａ＝Ｒ（Ｐｂ＋Ｔ） …（１０）
平行移動量Ｔ（３つの平面の交点）の推定は、下記の通りに行われる。
【００８０】
１） ３つの平面方程式を求める。
これは、対応する３つの法線ベクトルｎ１＝（ａ１，ｂ１，ｃ１），ｎ２＝（ａ２，ｂ２，ｃ２），ｎ３＝（ａ３，ｂ３，ｃ３）とそれぞれのシフト量ｄ１，ｄ２，ｄ３から下記（１１）式の方程式を得る。
【００８１】
【数５】

２） ３つの平面方程式から平行移動量Ｔを得る。
【００８２】
上記式（１１）の交点（平行移動量Ｔ）は以下のように求める。
但し、下記式のＵ^１はＵの逆行列を示している。
【００８３】
【数６】

本実施形態では、変換パラメータ推定部１７は、さらに、視点Ｃの座標系Ｐｃを視点Ａの座標系Ｐａに変換するが、前記と同様に処理して、回転パラメータ及び並進パラメータを推定する。
【００８４】
（統合部１８）
統合部１８は、図１０のＳ９０の処理を行う。
すなわち、Ｓ９０では、統合部１８はそれぞれ求められた回転パラメータ及び並進パラメータを基に、各視点におけるパノラマレンジデータを、１つの座標系に変換する。
【００８５】
本実施形態では、視点Ｂ及び視点Ｃのパノラマレンジデータを視点Ａの座標系に変換する（図３参照）。
この変換した結果、複数の視点から得られたパノラマレンジデータの座標系を１の視点の座標系に統合することができる。
【００８６】
上記実施形態によれば、下記の効果を有する。
（１） 本実施形態では、視点Ａ、視点Ｂ、視点Ｃの複数の視点から周囲環境３６０°（所定範囲）を撮像して得られたパノラマレンジデータ（距離画像）に基づき各視点毎に法線ベクトル（局所法線ベクトル）を求めるようにした。次に、法線ベクトルのヒストグラムを生成し、同法線ベクトルのうち、同じ向きを持つもの同士を平面グループとして抽出した。
【００８７】
そして、得られた平面グループの法線ベクトルに基づき、法線軸に対して、視点からの距離に応じて投影したグローバル統計量を求めた。
さらに、視点Ａ、視点Ｂ、視点Ｃのそれぞれの視点間における、平面グループ間の角度差と、対応する平面グループ同士のグローバル統計量の類似度に基づいて、複数視点間の最も類似度の高い平面グループの対応を求めた。
【００８８】
又、マッチングした平面グループ同士間の回転パラメータ及び並進パラメータ（変換パラメータ）を推定し、求めた回転パラメータ及び並進パラメータに基づいて、視点Ａ、視点Ｂ、視点Ｃの各視点の距離画像の座標系を１つの座標系に統合するようにした。
【００８９】
この結果、本実施形態の距離画像の統合方法によれば、従来の距離画像の統合のために設置していたマーカ等の道具が不要である効果を奏する。
又、各視点毎に、平面の法線ベクトルを求めるようにして、従来手法と異なり対応点を設置することがないため、複数の距離画像のデータ間に適当な対応点を持たない点が多数含まれていた場合があっても、これらの点の影響を受けることがなく、ロバスト性が高い効果がある。
【００９０】
又、曲率等を対応点としないため、ノイズを含んだ距離画像から正しく特徴点を抽出することが困難となることはなく、この点からもロバスト性が高い効果がある。すなわち、本実施形態によれば、多視点間の変換パラメータ（回転パラメータと並進パラメータ）を効率良くロバストに推定できる効果がある。
【００９１】
（２） 本実施形態の距離画像統合装置１０は、視点Ａ、視点Ｂ、視点Ｃの複数の視点から周囲環境３６０°（所定範囲）を撮像して得られたパノラマレンジデータ（距離画像）に基づき各視点毎に法線ベクトル（局所法線ベクトル）を求める法線ベクトル算出部１２（法線ベクトル算出手段）を備えるようにした。又、法線ベクトルのヒストグラムを生成するヒストグラム生成部１３（ヒストグラム生成手段）と、同法線ベクトルのうち、同じ向きを持つもの同士を平面グループとして抽出する平面グループ抽出部１４（平面グループ抽出手段）を備えた。そして、得られた平面グループの法線ベクトルに基づき、法線軸に対して、視点からの距離に応じて投影したグローバル統計量を生成するグローバル統計量生成部１５（グローバル統計量生成手段）を備えた。
【００９２】
さらに、視点Ａ、視点Ｂ、視点Ｃのそれぞれの視点間における、平面グループ間の角度差と、対応する平面グループ同士のグローバル統計量の類似度に基づいて、複数視点間の最も類似度の高い平面グループの対応を求める平面グループマッチング部１６（平面グループマッチング手段）を備えた。又、マッチングした平面グループ同士間の回転パラメータ及び並進パラメータ（変換パラメータ）を推定する変換パラメータ推定部１７（変換パラメータ推定手段）を備えた。
【００９３】
さらに、求めた回転パラメータ及び並進パラメータに基づいて、視点Ａ、視点Ｂ、視点Ｃの各視点の距離画像の座標系を１つの座標系に統合する統合部１８（統合手段）とを備えた。
【００９４】
この結果、本実施形態では、装置として上記（１）と同様の効果を奏する。
（３） 本実施形態では、グローバル統計量生成部１５はロバスト性を考慮し、平面グループの法線ベクトルと、ある程度角度が類似している平面も用いて、グローバル特徴空間を生成するようにした。
【００９５】
この結果、ロバスト性を高くすることができる。
（４） 本実施形態では、回転パラメータ（回転行列）を安定的に求めるために、対応付けられた平面グループ間においては、６０°以上の角度差を持つ平面グループを探索するようにした。
【００９６】
この結果、回転パラメータ（回転行列）を安定的に求めることができる。
なお、本発明の実施形態は上記実施の形態に限定されるものではなく、発明の趣旨を逸脱しない範囲で、適宜に変更して次のように実施することもできる。
【００９７】
（１） 前記実施形態では、周囲環境３６０°の範囲のレンジデータ、すなわち、パノラマレンジデータを使用したが、所定範囲は３６０°に限定するものではなく、各視点からそれぞれ、共通の撮像対象を含むようにして、レンジデータ測定装置にて撮像するものであれば、３６０°よりも少ない範囲でもよい。
【００９８】
（２） 前記実施形態では、視点は、３つの視点としたが、視点の数は、２つ以上であればよい。
（３） 前記実施形態では、平面グループマッチング部１６はＳ８０において、「外れ平面グループの検出及び除外」を行ったが、この処理を省略してもよい。この場合、誤差が出る場合があるが、本発明の効果を奏することができる。
【００９９】
又、こうすると、対応する平面グループ同士を３個のみ求め、この平面グループに係る３方向の法線ベクトルから回転パラメータを推定することになるが、それ以外の対応する平面グループを用いれば、より精度良く回転パラメータを推定することができる。
【０１００】
（４） 前記実施形態では、グローバル統計量生成部１５はロバスト性を考慮し、平面グループの法線ベクトルと、ある程度角度が類似している平面も用いて、グローバル特徴空間を生成するようにした。すなわち、グローバル統計量生成部１５は、対象とする平面グループとほぼ同じ向きから４５°の角度をなす平面までを対象として、グローバル特徴空間を生成するようにした。しかし、４５°に限定するものではなく、他の角度にしてもよい。
【０１０１】
（５） 前記実施形態では、各視点毎に、データ入力部１１、法線ベクトル算出部１２、ヒストグラム生成部１３、平面グループ抽出部１４、及びグローバル統計量生成部１５を設けた。これに代えて、データ入力部１１のみ、各視点に応じて複数個設け、データ入力部１１、法線ベクトル算出部１２、ヒストグラム生成部１３、平面グループ抽出部１４、及びグローバル統計量生成部１５は単一のものとしてもよい。この場合、各視点に配置されたデータ入力部１１は図１に点線で示すように、１つの法線ベクトル算出部１２に出力する。そして、各視点毎に、法線ベクトル算出部１２、ヒストグラム生成部１３、平面グループ抽出部１４、及びグローバル統計量生成部１５にて処理してもよい。
【０１０２】
この場合、法線ベクトル算出部１２、ヒストグラム生成部１３、平面グループ抽出部１４、及びグローバル統計量生成部１５は、各部共通の１つのコンピュータにて構成されている。
【０１０３】
又、平面グループマッチング部１６、変換パラメータ推定部１７、統合部１８は、各部共通の１つのコンピュータにて構成されている。
この場合、法線ベクトル算出部１２、ヒストグラム生成部１３、及び平面グループ抽出部１４を構成するコンピュータと、平面グループマッチング部１６、変換パラメータ推定部１７、及び統合部１８を構成するコンピュータとは、別体でも、共通でもよい。
【０１０４】
次に、上記した実施形態から把握できる請求項に記載した発明以外の技術的思想について、それらの効果とともに以下に記載する。
（１） 請求項１において、複数視点間における、平面グループ間の角度差と、対応する平面グループ同士のグローバル統計量の類似度に基づいて複数視点間の最も類似度の高い平面グループ同士を求めるとともに、前記マッチングした平面グループ同士間の変換パラメータを推定する際に、対応付けられた平面グループ間においては、６０°以上の角度差を持つ平面グループを探索するようにしたことを特徴とする距離画像の統合方法。こうすると、回転パラメータ（回転行列）を安定的に求めることができる。
【０１０５】
（２） 請求項２において、平面グループマッチング手段は、複数視点間における、平面グループ間の角度差と、対応する平面グループ同士のグローバル統計量の類似度に基づいて複数視点間の最も類似度の高い平面グループの対応を求めるとともに、前記マッチングした平面グループ同士間の変換パラメータを推定する際に、対応付けられた平面グループ間においては、６０°以上の角度差を持つ平面グループを探索するようにしたことを特徴とする距離画像統合装置。こうすると、回転パラメータ（回転行列）を安定的に求めることができる。
【０１０６】
（３） 請求項２において、平面グループマッチング手段は、複数視点間の最も類似度の高い、対応する平面グループ同士を３個のみ求め、この平面グループに係る３方向の法線ベクトルから回転パラメータを推定することを特徴とする距離画像統合装置。
【０１０７】
なお、対応する平面グループ同士を３個以上用いた方が精度良く回転パラメータを求めることができる。
【０１０８】
【発明の効果】
以上詳述したように、請求項１及び請求項２に記載の発明は、従来の距離画像の統合のために設置していたマーカ等の道具が不要であり、多視点間の変換パラメータ（回転パラメータと並進パラメータ）を効率良くロバストに推定できる効果を奏する。
【図面の簡単な説明】
【図１】距離画像統合装置１０の全体図。
【図２】（ａ）〜（ｃ）は、同一の部屋ＣＨ内において、視点Ａ，Ｂ，Ｃのそれぞれの位置を示す説明図。
【図３】部屋ＣＨ内において、視点Ａ，Ｂ，Ｃのそれぞれの位置を示す説明図。
【図４】７×７の場合の注目点を示すための説明図。
【図５】（ａ）は法線ベクトル投票空間を示す斜視図、（ｂ）は、同じく法線ベクトル投票空間の正面図。
【図６】（ａ）は法線ベクトル投票空間を示す斜視図、（ｂ）は、同じく法線ベクトル投票空間の正面図。
【図７】（ａ）はヒストグラム空間（θxz（ｎ），φy（ｎ））の説明図、（ｂ）は、ヒストグラム空間（θyz（ｎ），φx（ｎ））の説明図。
【図８】視点Ａ，Ｂ，Ｃで、共通の平面５０，５１を見た場合の距離、角度の相違を示すための説明図。
【図９】３次元モデルとグローバル特徴空間の説明図。
【図１０】平面グループマッチング部１６〜統合部１８が実行するフローチャート。
【図１１】２つの視点間にある平面グループの対応例を示す説明図。
【図１２】同じく２つの視点間にある平面グループの対応例を示す説明図。
【図１３】３法線ベクトルとそのシフト量に基づく３平面の説明図。
【符号の説明】
１０…距離画像統合装置
１１…データ入力部
１２…法線ベクトル算出部（法線ベクトル算出手段）
１３…ヒストグラム生成部（ヒストグラム生成手段）
１４…平面グループ抽出部（平面グループ抽出手段）
１５…グローバル統計量生成部（グローバル統計量生成手段）
１６…平面グループマッチング部（平面グループマッチング手段）
１７…変換パラメータ推定部（変換パラメータ推定手段）
１８…統合部（統合手段）
Ｌ…法線軸
Ａ，Ｂ，Ｃ…視点[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a distance image integration method and a distance image integration device.
[0002]
[Prior art]
In recent years, in the fields of object-pair recognition and virtual reality, it is important to measure the three-dimensional shape of an object or scene and model the shape. On the other hand, with the emergence of a highly accurate and relatively inexpensive rangefinder, it is easy to acquire range data, that is, a distance image. In the measurement from a single viewpoint by the range finder, there are parts where data cannot be obtained, such as the part hidden behind the backside or unevenness, so in order to obtain the three-dimensional shape of the object or the entire scene, from multiple viewpoints The range data obtained by measurement must be integrated.
[0003]
In the integration of range images obtained from a plurality of viewpoints, a method has been proposed in which corresponding features are found between range images and integrated based on the corresponding points. As features used here, for example, information such as edges and points obtained from markers whose shapes are known in advance (for example, shapes such as squares, circles, etc.), and motion-invariant information such as curvature are used as corresponding points. There is an example to do.
[0004]
[Problems to be solved by the invention]
However, if there are many points that do not have appropriate corresponding points among the data of a plurality of distance images, they may not be integrated correctly due to the influence of these points and lack robustness. Further, when the curvature or the like is used as a feature point, it may be difficult to correctly extract the feature point from a distance image including noise.
[0005]
An object of the present invention is that a distance image that can efficiently and robustly estimate conversion parameters (rotation parameters and translation parameters) between multiple viewpoints is unnecessary, without using tools such as markers that have been installed for integration of conventional distance images. And a distance image integration apparatus.
[0006]
[Means for Solving the Problems]
In order to solve the above problem, the invention according to claim 1 determines a local normal vector for each viewpoint based on a distance image obtained by imaging a predetermined range from a plurality of viewpoints. A histogram is generated, and the local normal vectors having the same orientation are extracted as a plane group. Based on the local normal vector of the obtained plane group, the normal axis is viewed from the viewpoint. Obtain the area histogram (hereinafter referred to as global statistic) of the local plane patch projected according to the distance, and based on the angle difference between the plane groups between multiple viewpoints and the similarity of the global statistics between the corresponding plane groups In addition to finding the correspondence of the plane groups with the highest similarity between multiple viewpoints, the conversion parameters between the matched plane groups were estimated and found. Based on the conversion parameters, it is how to integrate distance image for integrating the coordinate system of the range image at each viewpoint in one coordinate system as to the subject matter.
[0007]
The invention according to claim 2 is a normal vector calculation means for obtaining a local normal vector for each viewpoint based on a distance image obtained by imaging a predetermined range from a plurality of viewpoints, and generates a histogram of the local normal vector. Based on the local normal vector of the obtained plane group based on the local normal vector of the obtained plane group On the other hand, a global statistic generation means that generates an area histogram (hereinafter referred to as global statistic) of the local plane patch projected according to the distance from the viewpoint, and the angle difference between plane groups between multiple viewpoints The plane group with the highest similarity between multiple viewpoints based on the similarity of the global statistics between the plane groups A group matching unit, a conversion parameter estimation unit that estimates a conversion parameter between matched plane groups, and an integration unit that integrates the coordinate system of the distance image of each viewpoint into one coordinate system based on the obtained conversion parameter; A gist of a range image integrating device characterized by comprising:
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is an overall view of the distance image integration device 10.
[0009]
The distance image integration device 10 includes a data input unit 11, a normal vector calculation unit 12, a histogram generation unit 13, a plane group extraction unit 14, a global statistic generation unit 15, a plane group matching unit 16, a conversion parameter estimation unit 17, an integration The unit 18 is configured.
[0010]
The normal vector calculation unit 12 corresponds to normal vector calculation means, and the histogram generation unit 13 corresponds to histogram generation means. The plane group extraction unit 14 corresponds to plane group extraction means, and the global statistic generation unit 15 corresponds to global statistic generation means. The plane group matching unit 16 corresponds to a plane group matching unit. The conversion parameter estimation unit 17 corresponds to conversion parameter estimation means. The integration unit 18 corresponds to integration means.
[0011]
In the present embodiment, the data input unit 11, the normal vector calculation unit 12, the histogram generation unit 13, the plane group extraction unit 14, and the global statistic generation unit 15 are provided as one set, and three sets are provided. The normal vector calculation unit 12, the histogram generation unit 13, the plane group extraction unit 14, and the global statistic generation unit 15 in each set are configured by one computer. That is, three computers are provided corresponding to the three groups. Further, the plane group matching unit 16, the conversion parameter estimation unit 17, and the integration unit 18 are configured by one computer different from the computer provided for each set.
[0012]
In the following description, the data input unit 11, the normal vector calculation unit 12, the histogram generation unit 13, the plane group extraction unit 14, and the global statistic generation unit 15 are described as one set, and the other sets Since these functions in the same manner, description thereof is omitted.
[0013]
(Data input unit 11)
As the data input unit 11, a range data measuring device (range finder) is used, and the range data of an object is transmitted from a predetermined viewpoint (referred to as viewpoint A, viewpoint B, viewpoint C according to each set) to the surrounding environment 360 °. 3D data, that is, panorama range data. The obtained range data is expressed in a viewpoint coordinate system with the range data measuring device as the origin. The panorama range data corresponds to a distance image.
[0014]
2A to 2C are explanatory diagrams showing the positions of the viewpoints A, B, and C in the same room CH. FIG. 3 is an explanatory diagram showing the positions of the viewpoints A, B, and C in the room CH.
[0015]
(Normal vector calculation unit 12)
The panorama range data is omnidirectional three-dimensional data with a large amount of three-dimensional information, unlike the three-dimensional data obtained by a general three-dimensional measuring instrument. Hereinafter, the panorama range data is simply referred to as range data.
[0016]
Since points on the same plane have similar local normals, a histogram space based on the local normal vectors is generated, and a peak value is detected to extract a plane group.
[0017]
Therefore, each set of normal vector calculation units 12 calculates a unit normal vector (hereinafter simply referred to as a normal vector) based on the panorama range data obtained by the data input unit 11. The normal vector corresponds to a local normal vector.
[0018]
The calculation of the normal vector will be described in detail. In this embodiment, the normal vector calculation unit 12 first applies a plane to the range data by a least square approximation using the following plane equation within a 7 × 7 local region centered on the point of interest. Do. FIG. 4 shows attention points in the case of 7 × 7. The present invention is not limited to 7 × 7.
[0019]
pX + qY + rZ + s = 0 (1)
(P, q, r, s are parameters)
p²+ Q²+ R²+ S²= 1 (2)
Under the constraints of
Σ (pX_i+ QY_i+ RZ_i+ S)²
The parameter of the plane is estimated by minimizing.
[0020]
A unit normal vector is obtained based on the obtained plane parameters, and the sign of the unit normal vector is determined so that the direction faces outward.
(Histogram generator 13)
Although the normal vectors obtained by the normal vector calculation unit 12 are distributed on a spherical surface, the spherical coordinate system is difficult to handle, and the histogram generation unit 13 is shown in FIGS. 5 (a) and 6 (a). The normal vector histogram voted on the spherical surface is developed in the orthogonal coordinate space of the two ring-shaped regions, and this is used as the feature space of the normal vector.
[0021]
5A and 6A are perspective views of a normal vector voting space. FIG. 5B and FIG. 6B similarly show front views of the normal vector voting space, and the hatched area shows a ring-shaped area.
[0022]
Furthermore, in order to prevent the size of the histogram space from changing due to the distance or angle from the range data measurement device (range finder) even in the same size region, the histogram generation unit 13 displays the histogram as a local area of the target point. Obtained from the actual area occupied by the region.
[0023]
Note that, in the obtained two histogram spaces, overlapping portions are generated, and one of these overlapping portions is used in the plane group extraction unit 14 described later.
[0024]
5 (a) and 5 (b) and θyz (n) in FIGS. 6 (a) and 6 (b) are obtained when the normal vector n is projected onto the xz plane and the yz plane, respectively. Azimuth. Φy (n) is the angle between the xz plane and the normal vector n, and φx (n) is the angle between the yz plane and the normal vector n.
[0025]
In order to maintain the uniformity of the histogram space, the histogram generation unit 13 multiplies the obtained normal vector histogram by the scale function 1 / sin (φy) or the function 1 / sin (φx) along the φ axis. .
[0026]
FIG. 7A shows an example of the histogram space (θxz (n), φy (n)) in the ring-shaped region obtained by the histogram generator 13.
FIG. 7B shows an example of the histogram space (θyz (n), φx (n)) in the ring-shaped region obtained by the histogram generator 13.
[0027]
In the histogram spaces of FIGS. 7A and 7B, plane groups having similar orientations form one distribution.
(Plane group extraction unit 14)
In order to extract a plane group from the obtained histogram space, the plane group extraction unit 14 detects a peak value existing in the histogram space and extracts the distribution to obtain a plane group. That is, the plane group extraction unit 14 detects a plurality of peaks in order from the largest peak value based on the distribution of the two histograms in FIGS. 7A and 7B. Each of these peak values is a plane group.
[0028]
In the present embodiment, each plane group extraction unit 14 extracts a plurality of, for example, 10 plane groups.
(Global statistics generation unit 15)
Here, the operation of the global statistic generation unit 15 will be described.
[0029]
In one plane group having the same normal vector orientation, each plane included in the plane group is distributed at various locations in an actual three-dimensional space. Therefore, the distribution feature is used as a global feature of the plane group and is used for calculating the similarity of the plane group.
[0030]
In the present embodiment, the global statistic generation unit 15 generates a global feature space using a plane that is somewhat similar in angle to a normal vector of a plane group in consideration of robustness. That is, the global statistic generation unit 15 generates a global feature space for a plane from an almost same direction as a target plane group to a plane that forms an angle of 45 °. The present invention is not limited to an angle of 45 °.
[0031]
When generating the global feature space using the number of pixels that can be seen from the viewpoint position, the global feature space can be used in the same size region on the 3D space depending on the distance and angle from the range data measurement device (range finder). The occupying size is uneven.
[0032]
FIG. 8 shows the difference in distance and angle when the common planes 50 and 51 are viewed at the viewpoints A, B, and C. FIG. Moreover, the arrow in FIG. 8 has shown the direction of the normal vector regarding each plane 50,51.
[0033]
Therefore, in the global statistic generation unit 15, as shown in FIG. 9, the local projection is performed according to the distance α between the point α that projects the sensor center (viewpoint) on the normal axis L and the point that projects the plane onto the normal axis L. The area histogram of the planar patch is the global feature space. The area histogram of the local planar patch corresponds to a global statistic. FIG. 9 is an explanatory diagram of a three-dimensional model and a global feature space.
[0034]
The plane is a plane existing in the room CH, and the normal axis L is the axis of the normal vector of the extracted plane group.
In this case, the horizontal axis of the area histogram shown in FIG. 9 indicates the distance α between the point α where the sensor center (viewpoint) is projected on the normal axis L and the point β where the plane is projected on the normal axis L, and the vertical axis indicates the area and Become. FIG. 9 shows an example of the three-dimensional model (a) and the obtained global feature space distribution (b). In FIG. 9, six planes exist in the three-dimensional space, but the plane (I) and the plane (VI) having a large area appear as large values in the global feature space. When there is a normal vector and an angle as in the plane (II) or the plane (V), the global feature space appears as a wide distribution.
[0035]
The global statistic generation unit 15 performs the above-described processing on the plane groups obtained by the A to C plane group extraction units 14.
(Planar group matching unit 16)
The plane group matching unit 16 to the integration unit 18 integrate multi-viewpoint panorama range data using the plane group obtained by the plane group extraction unit 14 and the global feature space distribution obtained by the global statistic generation unit 15. I do.
[0036]
Here, an outline of the processing of the plane group matching unit 16 will be described.
In the following description, the case where the coordinate system of the viewpoint B is converted to the coordinate system of the viewpoint A will be described for the sake of convenience unless otherwise specified. However, the coordinate system of the viewpoint C may be converted to the coordinate system of the viewpoint A. Since it is the same, description is abbreviate | omitted.
[0037]
The plane group matching unit 16 searches for matching between a plurality of plane groups in which the angle difference between plane groups in one viewpoint and the angle difference between plane groups in another one viewpoint are equal to each other (FIG. 10). S30). Thereafter, the similarity between the corresponding plane groups is obtained using the global feature space distribution in the plane group, and matching between the plane groups with the highest similarity is searched (see S50 in FIG. 10).
[0038]
In the global feature space distribution, if there is the same plane group among the plane groups obtained from different viewpoints, the global feature space distributions of the plane groups are similar. Therefore, the plane group matching unit 16 uses the similarity of the area histogram of the global feature space distribution, that is, the similarity of the global statistics, as the similarity between the two plane groups at each viewpoint.
[0039]
Further, the similarity of the plane group is calculated for each plane group between the viewpoints, that is, between the points A and B, between the points A and C, and between the points B and C.
Specifically, the plane group similarity is calculated by the plane group matching unit 16 by shifting one area histogram in the horizontal axis direction (see the distance β shown in FIG. 9), and the other area histogram and SSD ( It is to calculate using the residual sum of squares).
[0040]
Further, the plane group matching unit 16 obtains the shift amount and the similarity when the similarity is the highest from the two area histograms. The similarity is also obtained when only one area histogram exists (see S40 in FIG. 10). These similarities are used when determining the same plane group between two viewpoints.
[0041]
Hereinafter, the process performed by the plane group matching unit 16 will be described in detail.
In the present embodiment, the plane group matching unit 16 performs matching under the following restrictions (1) and (2).
[0042]
(1) The relationship of the angle difference between the plane groups in one viewpoint is also established between the plane groups in the other viewpoint.
(2) If the correspondence between the three plane groups is obtained between the two viewpoints, the rotation parameter (rotation matrix) can be obtained.
[0043]
In the flowchart of FIG. 10, S <b> 10 to S <b> 70 are steps processed by the plane group matching unit 16 (hereinafter referred to as “S”).
(S10)
In S10, the plane group extracting unit 14 extracts ten plane groups from each viewpoint, and therefore, three plane groups are selected from each viewpoint. That is, a combination of plane groups between viewpoints is generated. Here, between the viewpoints means between the viewpoints A and B, between the viewpoints A and C, and between the viewpoints B and C in this embodiment.
[0044]
(S20)
In S20, the angle difference between the plane groups is calculated at each viewpoint. FIG. 12 shows the plane groups P11, P12. P13 and plane groups P21, P22, and P23 are shown. The angle difference between the plane groups at each viewpoint is θa1, θa2, θa3, and θb1, θb2, θb3 as shown in FIG.
[0045]
(S30)
In S30, the angle difference between the plane groups between the respective viewpoints is 60 ° or more (first condition), and the angle difference between the plane groups in one viewpoint is the angle between the corresponding plane groups in the other viewpoint. It is determined whether or not the difference is equal (second condition).
[0046]
In the example of FIG. 12, plane groups P11, P12. It is determined whether the angle differences θa1, θa2, θa3 between P13 and the angle differences θb1, θb2, θb3 of the plane groups P21, P22, P23 are 60 ° or more. Further, it is determined whether or not the angular differences θa1, θa2, and θa3 are equal to the angular differences θb1, θb2, and θb3.
[0047]
The reason for determining whether or not the angle difference between the plane groups between the respective viewpoints is 60 ° or more is as follows.
In order to stably determine the rotation parameter (rotation matrix), it is necessary to search for a certain angle difference between the associated plane groups. In general, it is known that three surfaces are required to form a minimum closed space, and in this case, the angle between the surfaces is 60 ° on average. Therefore, in this embodiment, a plane group having an angle difference of 60 ° or more is searched.
[0048]
If the combination of plane groups satisfies both the first condition and the second condition in S30, the process proceeds to S40. If the combination is a plane group that does not satisfy either the first condition or the second condition, the process returns to S10 to generate the next combination of plane groups.
[0049]
(S40)
In S40, similarity calculation between plane groups is performed.
The similarity calculation between plane groups includes calculation of the similarity between plane groups between different viewpoints and calculation of the sum of similarities.
[0050]
First, in the calculation process of the similarity between plane groups between different viewpoints, as described above, one area histogram is shifted in the horizontal axis direction (see the distance β shown in FIG. 9), and the other area histogram. And SSD (residual sum of squares).
[0051]
The degree of similarity between the plane groups obtained by the calculation is stored in a storage means (not shown), and is read out when calculating the sum of the degrees of similarity described later.
Next, the calculation of the sum of similarities will be described.
[0052]
The calculation of the sum of similarities, that is, the calculation of the sum of similarities of the global feature space distribution (global statistic) is performed according to the following procedure.
1) First, SUM = 0.
[0053]
2) A 3 × 3 rotation parameter (rotation matrix) R is obtained from the three normal vectors of the corresponding plane groups P11 and P21, P12 and P22, P13 and P23.
The rotation parameter corresponds to a conversion parameter.
[0054]
(Rotation parameter calculation)
Here, how to obtain the rotation parameter (rotation matrix) will be described.
In the “rotation parameter calculation”, the plane groups at the two viewpoints are described by distinguishing them by e and f for convenience of explanation.
[0055]
FIG. 11 shows an example of correspondence between plane groups between two viewpoints.
Corresponding plane group (e₁, F₁), (E₂, F₂), (E₂, F₂), ..., (e_i, F_i) To estimate the rotation parameter R (3 × 3). (I = 1, 2, 3 ... N)
In addition, the method of calculating | requiring this rotation parameter is well-known in the following reference.
[0056]
<References>
Kenichi Kanatani: "Analysis of 3-D Rotation Fitting". IEEE Trans. PAMI, Vol. 16, No. 5, pp543-549, 1994
The estimation method of the rotation parameter R is performed as follows.
[0057]
First, two sets {e with corresponding N plane groups {e_i}, {F_i} (I = 1, 2, 3... N)
[0058]
[Expression 1]

Find R that satisfies So, {e_i}, {F_i} Is defined as follows.
[0059]
Here, the superscript T indicates a transposed matrix.
[0060]
[Expression 2]

Here, the following equation is obtained by singular value decomposition of the correlation matrix K. Here, the matrices V and U are orthonormal matrices.
[0061]
[Equation 3]

Then, the rotation parameter R can be obtained as follows.
[0062]
[Expression 4]

The following processing is performed using the rotation parameter R obtained as described above.
[0063]
3) Normal vector n of all plane groups of viewpoint A_iIs multiplied by the obtained rotation parameter (rotation matrix).
Rn_i= M_j          ... (7)
4) In view point B, m above_jIf there is a plane group with a normal vector equal to_iPlane group with normal vector m_jThe similarity of the plane group having is added to SUM.
[0064]
If m_jIf no plane group equal to is in view B, normal vector n of view A_iThe similarity in the case where only the plane group having exists is added to SUM.
5) Inverse matrix R of 3 × 3 rotation parameters (rotation matrix)^-1Ask for.
[0065]
6) Next, the normal vector m of the plane group of viewpoint B_jIs multiplied by a rotation parameter (rotation matrix).
Inverse matrix R^-1m_j= N_i    (8)
7) Next, at viewpoint A, n_iIf there is a plane group with a normal vector equal to_iPlane group with normal vector m_jThe similarity of the plane group having is added to SUM.
[0066]
If n_iIf there is no plane group equal to the viewpoint A, the normal vector m of the viewpoint B_jThe similarity in the case where only the plane group having exists is added to SUM.
As described above, similarity calculation between plane groups is performed.
[0067]
In the above description, the viewpoint A and the viewpoint B are described, but the same processing is performed for the viewpoint A and the viewpoint C.
(S50)
In the next S50, the sum of the similarities calculated this time (SUM value) is compared with the sum of the maximum similarities calculated in the past (SUM value calculated in the past). It is determined whether or not (value) is greater than the sum of past maximum similarities. If the current value is larger, the process proceeds to S60, and if not, the process proceeds to S70.
[0068]
(S60)
In S60, the current value is updated as the maximum sum of similarities, and the corresponding plane group and the shift amount are stored in the storage means.
[0069]
(S70)
In S70, it is determined whether all plane group combinations have been searched. If the search for all plane groups has not been completed, the process returns to S10. When the search for all plane groups is completed, the process proceeds to S80.
[0070]
(Conversion parameter estimation unit 17)
The conversion parameter estimation unit 17 performs the process of S80 of FIG.
That is, in S80, the rotation parameter and the translation parameter as the conversion parameters are estimated based on the corresponding plane group stored in the storage means.
[0071]
Here, in order to estimate the rotation parameter (rotation matrix) with high accuracy, it is necessary to check whether there is a plane group corresponding to each other other than the three plane groups. In this case, even if there is a correspondence relationship, it is necessary to delete a combination having an “outlier plane group” that increases the overall error.
[0072]
Therefore, in S80, in the following procedure, the outlier plane group is detected, and when obtaining the rotation parameter, combinations of the plane groups including the outlier plane group are excluded.
[0073]
(Detection and exclusion of out-of-plane groups)
1) Using the inner product of the normal vectors np11 and mp21, np12 and mp22, np13 and mp23 of the plane groups having the same angle difference between the plane groups, the error ε is obtained by the following equation (9). Note that np11 means the normal vector of the plane group P11, and the other normal vectors n and m are similarly assigned corresponding to the codes of the plane group.
[0074]
ε = (np11 ・ mp21-1)²+ (np12 ・ mp22-1)²+ (np13 ・ mp23-1)²
... (9)
2) Take the median for the error ε,
ε ≧ (Median × 2.0)
The plane group that becomes is the out-of-plane group.
[0075]
Under the above conditions, combinations of plane groups including outlier plane groups are excluded.
Then, as described above, the rotation parameters are estimated by the same method as in S40, excluding combinations of outlier plane groups.
[0076]
The translation parameter is estimated as follows.
First, three normal vectors n1, n2, and n3 and their respective shift amounts d1, d2, and d3 related to the corresponding plane group are perpendicular to the respective normal vectors and moved from the origin by the respective shift amounts. Get a plane.
[0077]
In addition,
n1 = (a1, b1, c1)
n2 = (a2, b2, c2)
n3 = (a3, b3, c3)
And
[0078]
The intersection of these three planes (see FIG. 13) is the parallel movement amount T of one other viewpoint with respect to one viewpoint. Here, the translation parameter is RT. Based on the rotation parameter (rotation matrix) already obtained by the plane group matching unit 16 and the translation amount T, the coordinate system of one viewpoint is converted into the coordinate system of one other viewpoint using the following equation (10). To do.
[0079]
Here, for convenience of explanation, a case where the coordinate system Pb of the viewpoint B is converted into the coordinate system Pa of the viewpoint A will be described.
Pa = R (Pb + T) (10)
The estimation of the translation amount T (intersection of three planes) is performed as follows.
[0080]
1) Find three plane equations.
This is based on the corresponding three normal vectors n1 = (a1, b1, c1), n2 = (a2, b2, c2), n3 = (a3, b3, c3) and the respective shift amounts d1, d2, d3. The following equation (11) is obtained.
[0081]
[Equation 5]

2) A translation amount T is obtained from the three plane equations.
[0082]
The intersection (parallel movement amount T) of the above equation (11) is obtained as follows.
However, U in the following formula¹Indicates an inverse matrix of U.
[0083]
[Formula 6]

In the present embodiment, the conversion parameter estimation unit 17 further converts the coordinate system Pc of the viewpoint C into the coordinate system Pa of the viewpoint A, and performs the same processing as described above to estimate the rotation parameter and the translation parameter.
[0084]
(Integration unit 18)
The integration unit 18 performs the process of S90 in FIG.
That is, in S90, the integration unit 18 converts the panorama range data at each viewpoint into one coordinate system based on the obtained rotation parameter and translation parameter.
[0085]
In the present embodiment, the panorama range data of the viewpoints B and C is converted into the coordinate system of the viewpoint A (see FIG. 3).
As a result of this conversion, the coordinate system of panorama range data obtained from a plurality of viewpoints can be integrated into the coordinate system of one viewpoint.
[0086]
According to the embodiment, the following effects are obtained.
(1) In this embodiment, a method is used for each viewpoint based on panorama range data (distance image) obtained by imaging the surrounding environment 360 ° (predetermined range) from a plurality of viewpoints A, B, and C. A line vector (local normal vector) was obtained. Next, a histogram of normal vectors was generated, and the same normal vectors having the same orientation were extracted as a plane group.
[0087]
And based on the normal vector of the obtained plane group, the global statistic projected on the normal axis according to the distance from the viewpoint was obtained.
Furthermore, based on the angle difference between plane groups between viewpoints A, B, and C, and the similarity of global statistics between corresponding plane groups, the highest similarity between multiple viewpoints The correspondence of the plane group was sought.
[0088]
In addition, a rotation parameter and a translation parameter (conversion parameter) between the matched plane groups are estimated, and based on the obtained rotation parameter and translation parameter, a coordinate system of distance images of viewpoints A, B, and C is obtained. Are integrated into one coordinate system.
[0089]
As a result, according to the distance image integration method of the present embodiment, there is an effect that a tool such as a marker installed for the integration of the conventional distance image is unnecessary.
In addition, unlike the conventional method, a plane normal vector is obtained for each viewpoint, and no corresponding points are set. Therefore, there are many points that do not have appropriate corresponding points between data of a plurality of distance images. Even if it is included, it is not affected by these points and has an effect of high robustness.
[0090]
In addition, since the curvature or the like is not used as a corresponding point, it is not difficult to correctly extract a feature point from a distance image including noise, which is also effective in terms of robustness. That is, according to the present embodiment, there is an effect that conversion parameters (rotation parameters and translation parameters) between multiple viewpoints can be estimated efficiently and robustly.
[0091]
(2) The distance image integration device 10 of the present embodiment uses panorama range data (distance image) obtained by imaging the surrounding environment 360 ° (predetermined range) from a plurality of viewpoints A, B, and C. Based on this, a normal vector calculation unit 12 (normal vector calculation means) for obtaining a normal vector (local normal vector) for each viewpoint is provided. Also, a histogram generation unit 13 (histogram generation unit) that generates a histogram of normal vectors, and a plane group extraction unit 14 (plane group extraction unit) that extracts the same normal vectors having the same direction as plane groups. ). Then, based on the obtained normal vector of the plane group, a global statistic generating unit 15 (global statistic generating means) that generates a global statistic projected according to the distance from the viewpoint with respect to the normal axis is provided. It was.
[0092]
Furthermore, based on the angle difference between plane groups between viewpoints A, B, and C, and the similarity of global statistics between corresponding plane groups, the highest similarity between multiple viewpoints A plane group matching unit 16 (plane group matching means) for obtaining correspondence between plane groups is provided. Moreover, the conversion parameter estimation part 17 (conversion parameter estimation means) which estimates the rotation parameter and translation parameter (conversion parameter) between the matched plane groups was provided.
[0093]
Furthermore, an integration unit 18 (integration unit) that integrates the coordinate systems of the distance images of the viewpoints A, B, and C into one coordinate system based on the obtained rotation parameter and translation parameter is provided.
[0094]
As a result, in this embodiment, the apparatus has the same effect as the above (1).
(3) In this embodiment, the global statistic generation unit 15 generates a global feature space using a plane group normal vector and a plane similar in angle to a certain degree in consideration of robustness. .
[0095]
As a result, robustness can be increased.
(4) In the present embodiment, in order to stably obtain the rotation parameter (rotation matrix), a plane group having an angle difference of 60 ° or more is searched between the associated plane groups.
[0096]
As a result, the rotation parameter (rotation matrix) can be obtained stably.
The embodiment of the present invention is not limited to the above-described embodiment, and can be implemented as follows with appropriate modifications without departing from the spirit of the invention.
[0097]
(1) In the above embodiment, range data in the range of the surrounding environment 360 °, that is, panorama range data is used. However, the predetermined range is not limited to 360 °, and a common imaging target is selected from each viewpoint. As long as the range data measuring device captures an image, the range may be less than 360 °.
[0098]
(2) In the embodiment, three viewpoints are used, but the number of viewpoints may be two or more.
(3) In the above embodiment, the plane group matching unit 16 performs “detection and exclusion of outlier plane groups” in S80, but this process may be omitted. In this case, an error may occur, but the effect of the present invention can be achieved.
[0099]
In addition, in this case, only three corresponding plane groups are obtained, and rotation parameters are estimated from normal vectors in three directions related to the plane group. However, if other corresponding plane groups are used, The rotation parameter can be estimated with high accuracy.
[0100]
(4) In the embodiment described above, the global statistic generation unit 15 generates a global feature space using a plane that is somewhat similar in angle to the normal vector of the plane group in consideration of robustness. . In other words, the global statistic generation unit 15 generates a global feature space from a plane almost at the same angle as a target plane group to a plane that forms an angle of 45 °. However, the angle is not limited to 45 °, and other angles may be used.
[0101]
(5) In the above embodiment, the data input unit 11, the normal vector calculation unit 12, the histogram generation unit 13, the plane group extraction unit 14, and the global statistic generation unit 15 are provided for each viewpoint. Instead, only the data input unit 11 is provided in accordance with each viewpoint, and the data input unit 11, the normal vector calculation unit 12, the histogram generation unit 13, the plane group extraction unit 14, and the global statistic generation unit 15 are provided. May be single. In this case, the data input unit 11 arranged at each viewpoint outputs to one normal vector calculation unit 12 as indicated by a dotted line in FIG. Then, the normal vector calculation unit 12, the histogram generation unit 13, the plane group extraction unit 14, and the global statistic generation unit 15 may perform processing for each viewpoint.
[0102]
In this case, the normal vector calculation unit 12, the histogram generation unit 13, the plane group extraction unit 14, and the global statistic generation unit 15 are configured by one common computer.
[0103]
Further, the plane group matching unit 16, the conversion parameter estimation unit 17, and the integration unit 18 are configured by a single computer common to the respective units.
In this case, the computer that constitutes the normal vector calculation unit 12, the histogram generation unit 13, and the plane group extraction unit 14, and the computer that constitutes the plane group matching unit 16, the transformation parameter estimation unit 17, and the integration unit 18 are: It may be separate or common.
[0104]
Next, technical ideas other than the invention described in the claims that can be grasped from the above-described embodiment will be described below together with their effects.
(1) In claim 1, plane groups having the highest similarity between a plurality of viewpoints are obtained based on the angular difference between the plane groups between the viewpoints and the similarity of the global statistics between the corresponding plane groups. In addition, when estimating the conversion parameters between the matched plane groups, a distance between the associated plane groups is searched for a plane group having an angle difference of 60 ° or more. Image integration method. In this way, the rotation parameter (rotation matrix) can be obtained stably.
[0105]
(2) In claim 2, the plane group matching means is configured to calculate the highest similarity between a plurality of viewpoints based on the angle difference between the plane groups and the similarity of the global statistics between the corresponding plane groups. While obtaining correspondence between high plane groups, and estimating the conversion parameters between the matched plane groups, search for plane groups having an angle difference of 60 ° or more between the associated plane groups. A distance image integration device characterized by that. In this way, the rotation parameter (rotation matrix) can be obtained stably.
[0106]
(3) In claim 2, the plane group matching means obtains only three corresponding plane groups having the highest similarity between a plurality of viewpoints, and calculates a rotation parameter from a normal vector in three directions related to the plane group. A range image integration device characterized by estimating.
[0107]
Note that the rotation parameter can be obtained with higher accuracy by using three or more corresponding plane groups.
[0108]
【The invention's effect】
As described above in detail, the inventions described in claims 1 and 2 do not require a tool such as a marker that has been installed for the integration of conventional distance images, and convert parameters between multiple viewpoints (rotation). Parameter and translation parameter) can be efficiently and robustly estimated.
[Brief description of the drawings]
FIG. 1 is an overall view of a distance image integration device 10;
FIGS. 2A to 2C are explanatory views showing respective positions of viewpoints A, B, and C in the same room CH.
FIG. 3 is an explanatory diagram showing the positions of viewpoints A, B, and C in a room CH.
FIG. 4 is an explanatory diagram for showing a point of interest in the case of 7 × 7.
FIG. 5A is a perspective view showing a normal vector voting space, and FIG. 5B is a front view of the normal vector voting space.
6A is a perspective view showing a normal vector voting space, and FIG. 6B is a front view of the normal vector voting space. FIG.
7A is an explanatory diagram of a histogram space (θxz (n), φy (n)), and FIG. 7B is an explanatory diagram of a histogram space (θyz (n), φx (n)).
FIG. 8 is an explanatory diagram for showing a difference in distance and angle when the common planes 50 and 51 are viewed at the viewpoints A, B, and C;
FIG. 9 is an explanatory diagram of a three-dimensional model and a global feature space.
FIG. 10 is a flowchart executed by the plane group matching unit 16 to the integration unit 18;
FIG. 11 is an explanatory diagram showing a correspondence example of a plane group between two viewpoints.
FIG. 12 is an explanatory diagram showing an example of correspondence between plane groups that are similarly between two viewpoints.
FIG. 13 is an explanatory diagram of three planes based on three normal vectors and their shift amounts.
[Explanation of symbols]
10. Distance image integration device
11. Data input part
12: Normal vector calculation unit (normal vector calculation means)
13 ... Histogram generator (histogram generator)
14 ... Plane group extraction unit (plane group extraction means)
15 ... Global statistic generator (global statistic generator)
16 ... Plane group matching section (plane group matching means)
17 ... Conversion parameter estimation unit (conversion parameter estimation means)
18 ... Integration unit (integration means)
L ... Normal axis
A, B, C ... Viewpoint

Claims (2)

A local normal vector is obtained for each viewpoint based on a distance image obtained by imaging a predetermined range from a plurality of viewpoints,
Generate a histogram of the local normal vector,
Of the local normal vectors, those with the same orientation are extracted as plane groups,
Based on the local normal vector of the obtained plane group, an area histogram (hereinafter referred to as global statistics) of the local plane patch projected according to the distance from the viewpoint with respect to the normal axis,
Find the correspondence of the plane group with the highest similarity between multiple viewpoints based on the difference in angle between the plane groups between the multiple viewpoints and the similarity of the global statistics between the corresponding plane groups. Estimate the conversion parameters between
A distance image integration method comprising: integrating a coordinate system of a distance image of each viewpoint into one coordinate system based on the obtained conversion parameter. Normal vector calculation means for obtaining a local normal vector for each viewpoint based on a distance image obtained by imaging a predetermined range from a plurality of viewpoints;
Histogram generating means for generating a histogram of the local normal vector;
A plane group extracting means for extracting the same normal vectors as those having the same direction as a plane group;
A global statistic that generates an area histogram (hereinafter referred to as a global statistic) of a local plane patch projected according to the distance from the viewpoint with respect to the normal axis, based on the local normal vector of the obtained plane group Generating means;
A plane group matching means for obtaining a correspondence of the plane group having the highest similarity between the plurality of viewpoints based on the angle difference between the plane groups between the plurality of viewpoints and the similarity of the global statistics between the corresponding plane groups,
A conversion parameter estimation means for estimating a conversion parameter between matched plane groups;
A distance image integration apparatus comprising: an integration unit that integrates a coordinate system of distance images of each viewpoint into one coordinate system based on the obtained conversion parameters.

Images