JP4019204B2 - Image processing apparatus, image processing method, and medium on which image processing control program is recorded - Google Patents

Description

【００６１】
とするとともに、
Ｋｉ＝Ｗｉ／ＳＰ …（４）
とすると、
【００６２】
【数２】

【００６３】
として求められる。
【００６４】
このようにして再評価された輝度分布のヒストグラムを得たら、この特徴量から画像処理の強度を求める。すなわち、コントラストを拡大するための幅を決定する。拡大幅を決定するにあたり、輝度分布の両端を求めることを考える。写真画像の輝度分布は図１４に示すように概ね山形に表れる。むろん、その位置、形状についてはさまざまである。輝度分布の幅はこの両端をどこに決めるかによって決定されるが、単に裾野が延びて分布数が「０」となる点を両端とすることはできない。裾野部分では分布数が「０」付近で変移する場合があるし、統計的に見れば限りなく「０」に近づきながら推移していくからである。
【００６５】
このため、分布範囲において最も輝度の大きい側と小さい側からある分布割合だけ内側に経た部分を分布の両端とする。本実施形態においては、同図に示すように、この分布割合を０．５％に設定している。むろん、この割合については、適宜、変更することが可能である。このように、ある分布割合だけ上端と下端をカットすることにより、ノイズなどに起因して生じている白点や黒点を無視することもできる。すなわち、このような処理をしなければ一点でも白点や黒点があればそれが輝度分布の両端となってしまうので、２５５階調の輝度値であれば、多くの場合において最下端は階調「０」であるし、最上端は階調「２５５」となってしまうが、上端部分から０．５％の画素数だけ内側に入った部分を端部とすることにより、このようなことが無くなる。
【００６６】
実際の処理では再評価して得られたヒストグラムに基づいて画素数に対する０．５％を演算し、再現可能な輝度分布における上端の輝度値及び下端の輝度値から順番に内側に向かいながらそれぞれの分布数を累積し、０．５％の値となった輝度値を求める。以後、この上端側をｙｍａｘと呼び、下端側をｙｍｉｎと呼ぶ。また、本実施形態においては、コントラストの拡大とともに明度の修正も行なうこととしており、そのために必要なメジアンｙｍｅｄも上記再評価されたヒストグラムに基づいて決定する。以上の処理をステップＳ１３０にて行なう。
【００６７】
再現可能な輝度の範囲を「０」〜「２５５」としたときに、変換前の輝度ｙと輝度の分布範囲の最大値ｙｍａｘと最小値ｙｍｉｎから変換先の輝度Ｙを次式に基づいて求める。
【００６８】
Ｙ＝ａｙ＋ｂ …（６）
ただし
ａ＝２５５／（ｙmax−ｙmin） …（７）
ｂ＝−ａ・ｙminあるいは２５５−ａ・ｙmax …（８）
また、上記変換式にてＹ＜０ならばＹ＝０とし、Ｙ＞２５５ならばＹ＝２５５とする。ここにおける、ａは傾きであり、ｂはオフセットといえる。この変換式によれば、図１５に示すように、あるせまい幅を持った輝度分布を再現可能な範囲まで広げることができる。ただし、再現可能な範囲を最大限に利用して輝度分布の拡大を図った場合、ハイライト部分が白く抜けてしまったり、ハイシャドウ部分が黒くつぶれてしまうことが起こる。これを防止するため本実施形態においては、再現可能な範囲を制限している。すなわち、再現可能な範囲の上端と下端に拡大しない範囲として輝度値で「５」だけ残している。この結果、変換式のパラメータは次式のようになる。
【００６９】
ａ＝２４５／（ｙmax−ｙmin） …（９）
ｂ＝５−ａ・ｙminあるいは２５０−ａ・ｙmax …（１０）
そして、この場合にはｙ＜ｙminと、ｙ＞ｙmaxの範囲においては変換を行わないようにする。
【００７０】
ただし、このままの拡大率（ａに対応）を適用してしまうと、非常に大きな拡大率が得られる場合も生じてしまう。例えば、夕方のような薄暮の状態では最も明るい部分から暗い部分までのコントラストの幅が狭くて当然であるのに、この画像についてコントラストを大きく拡大しようとする結果、昼間の画像のように変換されてしまいかねない。このような変換は希望されないので、拡大率には制限を設けておき、ａが１．５（〜２）以上とはならないように制限する。これにより、薄暮は薄暮なりに表現されるようになる。なお、この場合は輝度分布の中心位置がなるべく変化しないような処理を行っておく。ステップＳ１４０では、このようにして拡大率ａ及び傾きｂを求める処理を実行する。
【００７１】
ところで、輝度の変換時に、毎回、上記変換式（Ｙ＝ａｙ＋ｂ）を実行するのは非合理的である。というのは、輝度ｙの取りうる範囲が「０」〜「２５５」でしかあり得ないため、予め輝度ｙが取りうる全ての値に対応して変換後の輝度Ｙを求めておくことも可能である。従って、図１６に示すようなテーブルとして記憶しておく。
【００７２】
このような輝度の範囲の拡大によってコントラストを強調するだけでなく、合わせて明るさを調整することも極めて有効であるため、画像の明るさを判断して補正のためのパラメータも生成する。
【００７３】
例えば、図１７にて実線で示すように輝度分布の山が全体的に暗い側に寄っている場合には破線で示すように全体的に明るい側に山を移動させると良いし、逆に、図１８にて実線で示すように輝度分布の山が全体的に明るい側に寄っている場合には破線で示すように全体的に暗い側に山を移動させると良い。
【００７４】
各種の実験を行った結果、本実施形態においては、輝度分布におけるメジアンｙｍｅｄを求め、同メジアンｙｍｅｄが「８５」未満である場合に暗い画像と判断して以下のγ値に対応するγ補正で明るくする。
【００７５】
γ＝ｙｍｅｄ／８５ …（１１）
あるいは、
γ＝（ｙｍｅｄ／８５）**（１／２） …（１２）
とする。
【００７６】
この場合、γ＜０．７となっても、γ＝０．７とする。このような限界を設けておかないと夜の画像が昼間のようになってしまうからである。なお、明るくしすぎると全体的に白っぽい画像になってコントラストが弱い画像になりやすいため、彩度を合わせて強調するなどの処理が好適である。
【００７７】
一方、メジアンｙｍｅｄが「１２８」より大きい場合に明るい画像と判断して以下のγ値に対応するγ補正で暗くする。
【００７８】
γ＝ｙｍｅｄ／１２８ …（１３）
あるいは、
γ＝（ｙｍｅｄ／１２８）**（１／２） …（１４）
とする。この場合、γ＞１．３となっても、γ＝１．３として暗くなり過ぎないように限界を設けておく。
【００７９】
なお、このγ補正は変換前の輝度分布に対して行っても良いし、変換後の輝度分布に対して行っても良い。γ補正をした場合における対応関係を図１９に示しており、γ＜１であれば上方に膨らむカーブとなり、γ＞１であれば下方に膨らむカーブとなる。むろん、かかるγ補正の結果も図１６に示すテーブル内に反映させておけばよく、テーブルデータに対して同補正を行っておく。ステップＳ１５０ではこのような変換テーブルを生成する処理を実行する。
【００８０】
この後、（６）式に基づく変換を行うが、同式の変換式は、ＲＧＢの成分値との対応関係においても当てはめることができ、変換前の成分値（Ｒ0 ，Ｇ0 ，Ｂ0 ）に対して変換後の成分値（Ｒ，Ｇ，Ｂ）は、
Ｒ＝ａ・Ｒ0 ＋ｂ …（１５）
Ｇ＝ａ・Ｇ0 ＋ｂ …（１６）
Ｂ＝ａ・Ｂ0 ＋ｂ …（１７）
として求めることもできる。ここで、輝度ｙ，Ｙが階調「０」〜階調「２５５」であるのに対応してＲＧＢの各成分値（Ｒ0 ，Ｇ0 ，Ｂ0 ），（Ｒ,Ｇ,Ｂ ）も同じ範囲となっており、上述した輝度ｙ，Ｙの変換テーブルをそのまま利用すればよいといえる。
【００８１】
従って、ステップＳ１６０では全画素の画像データ（Ｒ0 ，Ｇ0 ，Ｂ0 ）について（１５）〜（１７）式に対応する変換テーブルを参照し、変換後の画像データ（Ｒ,Ｇ,Ｂ ）を得るという処理を繰り返すことになる。
【００８２】
むろん、これらのステップＳ１３０〜ステップＳ１６０を実行するハードウェア構成とソフトウェアとによって処理内容決定手段ないし処理手段を構成することになる。なお、本実施形態においては、画像処理としてコントラストの拡大処理や明度の修正処理を実行しているが、他の画像強調処理などにおいても全く同様に適用可能であることはいうまでもない。
【００８３】
以上の処理では、画像の中での位置に応じた重み付けによって特徴量が再評価されるものであった。しかしながら、重み付けの基準はこれに限られるものではなく、各種の態様が可能である。図２０はその一例として画像の変化度合いから本来の被写体を検出し、重み付けを変化させる場合のフローチャートを示している。
【００８４】
ステップＳ２１０は、上述したステップＳ１１０に代わるものであり、均等に間引いた画素について輝度を集計する。しかし、輝度だけの集計ではなく、次に示すようなエッジ量も集計する。
【００８５】
入力画像のうちの背景は濃淡変化が緩やかであるといえるし、本来の被写体はシャープであるがゆえに輝度の変化が激しいといえる。従って、図２１に示すようにある一つの画素を注目画素としてその周縁の画素との濃度差を求めることとし、この濃度差をその注目画素のエッジ量とする。この濃度差はフィルタを適用して演算することができ、図２２（ａ）〜（ｆ）はこのようなフィルタの数例を示しており、注目画素と周縁の八画素について各画素の輝度を重み付け加算する際の重み付け係数を示している。ここにおいて、同図（ａ）の場合は九画素についての重み付け加算であるから、各画素でのエッジ量を求めるためには九回の乗算と八回の加算が必要になる。この演算量は画像が大きくなってくるにつれて無視できなくなるため、同図（ｂ）（ｃ）では五回の乗算と四回の加算で済ませ、同図（ｄ）（ｅ）では三回の乗算と二回の加算で済ませ、同図（ｆ）では二回の乗算と一回の加算で済ませるようにしている。
【００８６】
これらの例では注目画素に対してこれを一重に取り囲む画素とだけ比較しているが、いわゆるアンシャープマスクを使用してより広範囲な画素のデータを使用して注目画素でのシャープさを求めることは可能である。ただし、本実施形態におけるエッジ量は、あくまでもブロック毎の重み付けを評価するためのものであるから演算量を少なくしたこれらの例のフィルタのものでも十分である。
【００８７】
エッジ量の集計は各ブロック毎に各画素で求めたエッジ量を集計していっても良いし、このエッジ量の絶対値が所定のしきい値よりも大きい場合にエッジ画素と判定し、各ブロック毎のエッジ画素総数を集計していくようにしても良い。各ブロックでのエッジ量をＥＲｉ（ｉ＝１〜１５）とすると、その総量ＳＥは、
【００８８】
【数３】

【００８９】
となるから、重み付け係数ＫＥｉ自体は、
ＫＥｉ＝ＥＲｉ／ＳＥ …（１９）
で表される。従って、この場合の重み付けして再評価された輝度分布ＤＹは、
【００９０】
【数４】

【００９１】
として求められる。また、各ブロックでのエッジ画素総数をＥＮｉ（ｉ＝１〜１５）とすると、その総量ＳＥは、
【００９２】
【数５】

【００９３】
となるから、重み付け係数ＫＥｉ自体は、
ＫＥｉ＝ＥＮｉ／ＳＥ …（２２）
で表され、（２０）式を利用して再評価された輝度分布ＤＹを得ることができる。いずれの場合においても、ステップＳ２２０では（２０）式の演算式に基づいて輝度分布ＤＹを再評価する。
【００９４】
この例においては、エッジ画素と判定された画素の輝度をサンプリングして利用するというわけではなく、エッジ量やエッジ画素総数をブロックの重み付け係数の決定に使用しているだけである。すなわち、特定の性質（エッジ）を持つ画素のみの特徴量を集計しているわけではなく、そのブロックにおいて偏らない平均的な特徴量を得ることができる。
【００９５】
なお、このようにして輝度分布を再評価したら上述したステップＳ１３０〜Ｓ１６０の処理を経てコントラストを拡大するとともに明度を修正すればよい。
【００９６】
さらに、本来の被写体が人物像であることが多いことを考慮すると、肌色の画素が多いブロックに重きをおいて再評価するといったことも可能である。図２３はこのような特定の色に注目してブロックの重み付け係数を決定するフローチャートを示している。
【００９７】
ステップＳ１１０に対応するステップＳ３１０では、同様の間引き処理で輝度を集計するとともに、各画素の色度に基づいて肌色らしき画素であるかを判定し、肌色画素数の集計を行う。色度については各画素についてのｘ−ｙ色度を計算する。いま、対象画素のＲＧＢ表色系におけるＲＧＢ階調データが（Ｒ，Ｇ，Ｂ）であるとするときに、
ｒ＝Ｒ／（Ｒ＋Ｇ＋Ｂ） …（２３）
ｇ＝Ｇ／（Ｒ＋Ｇ＋Ｂ） …（２４）
とおくとすると、ＸＹＺ表色系における色度座標ｘ，ｙとの間には、
ｘ＝（１．１３０２＋１．６３８７ｒ＋０．６２１５ｇ）
／（６．７８４６−３．０１５７ｒ−０．３８５７ｇ）…（２５）
ｙ＝（０．０６０１＋０．９３９９ｒ＋４．５３０６ｇ）
／（６．７８４６−３．０１５７ｒ−０．３８５７ｇ）…（２６）
なる対応関係が成立する。ここにおいて、色度は明るさに左右されることなく色の刺激値の絶対的な割合を表すものであるから、色度からその画素がどのような対象物かを判断することができるといえる。肌色の場合は、
０．３５＜ｘ＜０．４０ …（２７）
０．３３＜ｙ＜０．３６ …（２８）
というような範囲に含まれているから、各画素の色度を求めたときにこの範囲内であればその画素は人間の肌を示す画素と考え、ブロック内の肌色画素数を一つ増加する。
【００９８】
このようにして肌色画素数が得られたら、次のステップＳ３２０では上述したエッジ画素数の場合と同様にして重み付け係数を決定し、輝度分布ＤＹを再評価する。すなわち、各ブロックでの肌色画素数をＣＮｉ（ｉ＝１〜１５）とすると、その総量ＳＣは、
【００９９】
【数６】

【０１００】
となるから、重み付けＫＣｉ自体は、
ＫＣｉ＝ＣＮｉ／ＳＣ …（３０）
で表され、この場合の重み付けして再評価された輝度分布ＤＹは、
【０１０１】
【数７】

【０１０２】
として求められる。この例においても、肌色の画素と判定された画素の輝度をサンプリングして利用するというわけではなく、肌色画素の総数をブロックの重み付け係数の決定に使用しているだけである。従って、そのブロックにおいて偏らない平均的な特徴量を得ることができる。この場合も、このようにして輝度分布を再評価したら上述したステップＳ１３０〜Ｓ１６０の処理を経てコントラストを拡大するとともに明度を修正すればよい。図２４に示す写真の場合、中央付近に女の子が写っており、顔、手足の画素で肌色画素と判断されることになる。むろん、他の色についての色度を求めて画素数を集計するようにしても良い。
【０１０３】
ところで、これまでは重み付け係数を一つの要因によって決定していたが、それぞれの要因の重要度を加味して重複して適用することもできる。個々の要因ｊ（１：画像の中での位置、２：エッジ量、３：肌色画素数）について各ブロックＢｉ（ｉ＝１〜１５）の重み付け係数をＴｊｉとした場合、要因毎の各ブロックに分配した重み付けＴｊｉは仮の重み付けとなり、
【０１０４】
【数８】

【０１０５】
とするとともに、
Ｋｊｉ＝Ｔｊｉ／Ｓｊ …（３３）
とすると、ブロックＢｉにおける真の重み付け係数Ｋｉは、
【０１０６】
【数９】

【０１０７】
で表される。ここにおいて、Ａｊは各要因毎の重要度を表す係数であり、総数が１となる範囲で適宜決定する。一例として、肌色重視とするならば、Ａ１＝０．２、Ａ２＝０．２、Ａ３＝０．６といった設定などが可能である。
【０１０８】
次に、上記構成からなる本実施形態の動作を説明する。最初に、先の実施形態に沿って説明する。
【０１０９】
写真画像をスキャナ１１で読み込み、プリンタ３１にて印刷する場合を想定する。すると、まず、コンピュータ２１にてオペレーティングシステム２１ａが稼働しているもとで、画像処理アプリケーション２１ｄを起動させ、スキャナ１１に対して写真の読み取りを開始させる。読み取られた画像データが同オペレーティングシステム２１ａを介して画像処理アプリケーション２１ｄに取り込まれたら、ステップＳ１１０にて間引きしながら各画素の輝度を集計する。集計された輝度分布ｄＹｉは、ステップＳ１２０にて図１２あるいは図１３に示すブロック毎の位置に対応して決定される重み付けに基づいて再評価し、再評価された輝度分布ＤＹに基づいてステップＳ１３０ではｙｍａｘ，ｙｍｉｎ，ｙｍｅｄを求める。
【０１１０】
次なるステップＳ１４０では、（９）式あるいは（１０）式に基づいて強調パラメータである傾きａとオフセットｂとを算出するとともに、（１１）式〜（１４）式に基づいて明度修正に要するγ補正のγ値を求め、ステップＳ１５０では図６に示す変換テーブルを作成する。そして、最後に、ステップＳ１６０では全画素についての画像データを同変換テーブルを参照して変換する。
【０１１１】
図１２に示す重み付けを使用する場合には、中央に近いブロックほど重み付けが重いので、集計された輝度分布も中央に近いものほど大きく評価される。例えば、夜間にフラッシュを使用して人物像を撮影したとする。人物像についての輝度分布はフラッシュの効果もあって図２５（ａ）に示すように概ね良好な輝度分布が得られたとしても、人物の周囲は暗く、同図（ｂ）に示すような暗い側に偏った輝度分布が得られる。この場合、単純に平均化すれば同図（ｃ）に示すように全体的に暗い側に偏った輝度分布が得られる。従って、このままコントラストや明度を修正すれば暗い画像を無理やり明るくしてしまい、良好な画像は得られない。
【０１１２】
しかしながら、図１２に示すようにして中央のブロックに重み付けを重くするようにすると、図２５（ｄ）に示すように画像の中央部分で得られた輝度分布の影響を強く受けた輝度分布ＤＹが得られる。従って、かかる輝度分布に基づいて決定される画像処理の強度は過度にコントラストを拡大したり明度を修正したりするようなものではなくなる。
【０１１３】
逆に、逆光の状態で人物を撮影すると、顔が暗く、背景が明るくなってしまうが、画像全体で見れば良好な輝度分布ともなりかねない。しかしながら、このような場合でも図１２に示すようにして暗い顔が写っている中央のブロックでの輝度分布に重きをおいて評価することにより、暗い輝度分布が反映され、コントラストを拡大したり画像を明るくしたりする画像処理を実行することができるようになる。
【０１１４】
以上の処理により、スキャナ１１を介して読み込まれた写真の画像データは自動的に最適な強度で画像処理を施され、ディスプレイ３２に表示された後、プリンタ３１にて印刷される。
【０１１５】
このように、画像処理の中枢をなすコンピュータ２１はステップＳ１１０にて均等に画素を選択しながら特徴量である輝度の分布を領域毎に集計した後、ステップＳ１２０では各領域毎に決められた重み付けで再評価することにより、均等にサンプリングを行いながらも本来の被写体の輝度分布の影響を強く受けた輝度分布を得ることができ、ステップＳ１３０〜Ｓ１５０にてかかる輝度分布に基づいて画像処理の強度などを決定した後、ステップＳ１６０で画像データを変換するため、処理を軽くしつつ最適な強度で画像処理を実行することができる。
【図面の簡単な説明】
【図１】本発明の一実施形態にかかる画像処理装置を適用した画像処理システムのブロック図である。
【図２】同画像処理装置の具体的ハードウェアのブロック図である。
【図３】本発明の画像処理装置の他の適用例を示す概略ブロック図である。
【図４】本発明の画像処理装置の他の適用例を示す概略ブロック図である。
【図５】本発明の画像処理装置における画像処理を示すフローチャートである。
【図６】処理対象画素を移動させていく状態を示す図である。
【図７】サンプリング周期を示す図である。
【図８】サンプリング画素数を示す図である。
【図９】変換元の画像とサンプリングされる画素の関係を示す図である。
【図１０】画像を領域分割したブロックの配置を示す図である。
【図１１】各ブロックでの輝度分布の例を示す図である。
【図１２】ブロック毎の重み付けの一例を示す図である。
【図１３】ブロック毎の重み付けの他の一例を示す図である。
【図１４】輝度分布の端部処理と端部処理にて得られる端部を示す図である。
【図１５】輝度分布の拡大と再現可能な輝度の範囲を示す図である。
【図１６】輝度分布を拡大する際の変換テーブルを示す図である。
【図１７】γ補正で明るくする概念を示す図である。
【図１８】γ補正で暗くする概念を示す図である。
【図１９】γ補正で変更される輝度の対応関係を示す図である。
【図２０】エッジ量に基づいて特徴量を再評価する場合のフローチャートである。
【図２１】エッジ量を判断するための注目画素と周縁画素との関係を示す図である。
【図２２】エッジ量を算出するためのフィルタの例を示す図である。
【図２３】所定の色の画素数に基づいて特徴量を再評価する場合のフローチャートである。
【図２４】写真画像の一例を示す図である。
【図２５】夜間に撮影した写真画像の輝度分布を示す図である。
【符号の説明】
１０…画像入力装置
２０…画像処理装置
２１…コンピュータ
２１ａ…オペレーティングシステム
２１ｂ…プリンタドライバ
２１ｃ…ディスプレイドライバ
２１ｄ…画像処理アプリケーション
３０…画像出力装置[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus, an image processing method, and a medium on which an image processing control program is recorded that automatically executes optimum image processing on real image data such as a digital photographic image.
[0002]
[Prior art]
Various image processing is performed on the digital image data. For example, image processing such as increasing the contrast, correcting the color tone, or correcting the brightness. Such image processing is normally executable by a microcomputer, and an operator confirms an image on a monitor and selects necessary image processing, or determines image processing parameters.
[0003]
[Problems to be solved by the invention]
In recent years, various image processing techniques have been proposed and are actually effective. However, humans must still be involved when it comes to how much processing is performed. This is because it has not been possible to determine what is important in the digital image data to be subjected to image processing.
[0004]
For example, when image processing for correcting brightness is considered, it is assumed that automatic processing is performed in which bright correction is performed when the average of the entire screen is dark, and dark correction is performed when the average is bright. Here, it is assumed that there is actual image data of a human image taken at night. Although the background is almost completely dark, it is assumed that the person was able to photograph well. When this real image data is automatically corrected, the background is completely dark, so the average of the entire image becomes dark and the image is corrected brightly, resulting in a daytime image.
[0005]
In this case, if a person is involved, the fact that the background is dark is considered without giving too much weight, and only the portion of the person image is focused. If the person image is dark, correction is made brighter, and conversely, correction is made to darken if the person image is too bright due to an effect such as flash.
[0006]
As described above, in the conventional image processing, the importance according to each part in the photographed image data cannot be determined, and thus there is a problem that a human must be involved.
[0007]
On the other hand, even if the importance of an image can be determined by some method, it is an operation for determining in units of pixels. Therefore, if the importance is changed in real time, the amount of calculation increases.
[0008]
The present invention has been made in view of the above problems, and image processing capable of automatically executing optimum image processing in consideration of importance relatively easily in actual image data such as a digital photographic image. An object is to provide an apparatus, an image processing method, and a medium on which an image processing control program is recorded.
[0009]
[Means for solving the problems and their functions and effects]
  In order to achieve the above object, an image processing apparatus of the present invention provides:
  An image processing apparatus that performs predetermined image processing on a real image composed of a plurality of pixels,
  The image data of the pixels that make up the live-action image,From the whole live-action image, While decimating evenly according to predetermined standards,Input means to input evenly;
  While thinning out the entire live-action image aboveBased on the input image data,Histogram showing luminance distribution for each pixelRepresents the entire live-action imageFeature quantity extraction means for extracting as feature quantities;
  The above representative of the whole live-action imageFor featuresPre-set for each region that divides the image subject to image processing into a plurality of predetermined criteriaRe-evaluation, which is an evaluation using weighting factors, is performed.Find the re-evaluated features for the entire live-action imageA feature re-evaluation means;
  Processing content determination means for determining image processing including processing for correcting at least one of brightness and contrast of an image based on the reevaluated feature amount;
  Processing means for performing image processing with the determined content on the entire image data of the real image;
  Be equippede,
The feature quantity re-evaluation means sets a weighting coefficient that is weighted more than the weighting coefficient set in the other area for the area to be evaluated with a larger weight than the other area in the re-evaluation of the feature quantity. , Perform the above re-evaluation
Is the gist.
[0010]
  In the above invention, the input means inputs the image data of the pixels constituting the photographed image evenly from the entire photographed image, and the feature quantity extracting means determines the image processing intensity and the like based on the inputted image data. A histogram indicating a luminance distribution for each pixel necessary for determining the contents of image processing is extracted as a feature amount. Then, the feature quantity re-evaluation means performs a re-evaluation that is an evaluation using a predetermined weighting on the feature quantity extracted by the feature quantity extraction means, and based on the feature quantity obtained by such re-evaluation, The processing content determination means determines the intensity of image processing including processing for correcting at least one of brightness and contrast of the image, and the processing means performs image processing with the determined content. This is performed on the entire image data of the real image.
[0011]
  That is, at the extraction stageWhole live action imageSince the predetermined weighting is performed after the extraction, the resulting feature amount is different from that obtained uniformly over the entire image.
[0012]
In addition,Pixel featureofExtractionMake up a live-action imageExtraction may be performed for all pixels, or even if not all pixels are extracted. As an example of the latter,The input means constitutes the live-action imagePixels that are selected by thinning evenly with a predetermined standard for all pixelsThe feature amount extracting means is based on the image data input by the thinning.Above featuresofExtractionAs a means to doConstitutionCan think.
[0013]
  the aboveTo the configuration ofIn this case, the number of pixels to be processed is reduced by evenly thinning out all the pixels according to a predetermined standard, and the feature amount is extracted for the reduced pixels.
[0014]
In this case, the uniform thinning includes those selected by thinning out at a fixed period, and those selected at random and thinned out.
[0015]
  As described above, the feature amount re-evaluation unit performs re-evaluation on the extracted feature amount, which is an evaluation using a plurality of weighting coefficients prepared in advance according to the position of the image. Corresponding to the feature amount being in units of pixels, weighting in units of pixels may be performed, or weighting may be performed for each appropriate group. As an example of the latter, a weight storage unit that stores in advance a weight set in accordance with the position of the region in the image for each of a plurality of regions that divide the image to be subjected to the image processing on a predetermined basis, The input means is means for inputting the image data from each of the plurality of regions, and the feature amount extraction means is configured to input the feature amount based on the image data input from each of the plurality of regions. A configuration may be considered in which extraction is performed for each region, and the feature amount re-evaluation unit is a unit that performs re-evaluation of the extracted feature amount using the weight stored for each region.
[0016]
  the aboveDepartureLightThen,Based on the premise of the weighting of the area unit that divides the image by a predetermined standardVolume extractionThe extraction means extracts features in units of such areas, and featuresRe-quantityEvaluation means for each areaMemoryWeightingUsingRe-evaluate the feature value for each region.
[0017]
Such division of the area may be always constant or may be changed for each image. In this case, the division method may be changed according to the content of the image.
[0018]
Various weighting methods can be employed as long as the reevaluation is not limited to simple sampling.
[0019]
  As an example,The weight storage unit may be a unit that stores in advance a weight set in accordance with the position of each pixel with respect to an image for each of the plurality of regions. In particular,A configuration in which the weighting is changed in a correspondence relationship determined by the position of each pixel with respect to the image.Can think.
[0020]
When considering the composition of a photograph, the person image is often located at the center. Therefore, if feature quantities are extracted uniformly from the entire image and then weighted by increasing the weight of the feature quantity in the central portion, the feature quantities extracted from the pixels of the human image are greatly evaluated as a result.
[0021]
  the aboveTo the configuration ofFor example, if it is decided to increase the weight of the central part of the image and reduce the weight of the surrounding area,Re-quantityEvaluation means,The position of each pixel with respect to the image is determined, and re-evaluation is performed using a weighting that varies depending on the position.
[0022]
  As another example of the weighting method, PaintingObtain the degree of image change, and increase the weighting in areas where the degree of image change is largeThere is a technique. Specifically, based on the input image data, an area specifying unit that specifies an area with a large degree of image change in the photographed image, and an area specified by the area specifying unit includes other areas other than the area. A weight assigning means for assigning a weight that is heavier than the area, and the feature amount re-evaluating means is a means for re-evaluating the feature amount using at least the weight assigned by the weight assigning means. Can think. In this case, a pixel number calculation means for obtaining the number of pixels in which the change degree of the image data with the adjacent pixels is equal to or greater than a predetermined value among the pixels constituting the photographed image based on the input image data is provided. In consideration of the number of pixels obtained by the pixel number calculation means, the area specifying means may specify an area having a large degree of image change in the photographed image.
[0023]
  the aboveTo the configuration ofIn the featureRe-quantityThe degree of image change before the evaluation means re-evaluatesButAskingIsThe The degree of change in the image can be said to be the sharpness of the image, but the degree of change is large because the contour portion is clearer as the image is in focus. On the other hand, if the image is not in focus, the image gradually changes in the outline portion of the image, and the degree of change becomes small. In the case of a photograph or the like, the focused part is the original subject and the unfocused part is considered equivalent to the background. For this reason, a place where the degree of change in the image is large is considered an original subject,Re-quantityEvaluation means,By evaluating with such weighting being heavy at such a portion where the degree of change of the image is large, the same result as that obtained by extracting many feature amounts is obtained.
[0024]
  As yet another exampleCalculating means for obtaining the number of pixels within a predetermined color range among the pixels in the photographed image, identifying means for identifying a region having a large number of pixels obtained by the computing means in the photographed image, and the identification And an assigning unit that assigns a weight that is heavier than other regions other than the region to the area specified by the means, and the feature amount re-evaluation unit uses at least the weight assigned by the assigning unit. Re-evaluateIt is good as well. Specifically, the chromaticity of each pixel is obtained, the number of pixels within the chromaticity range of the target from which the same chromaticity is to be extracted is obtained, and the weighting is increased in a portion where the number of pixels is large.Can think.
[0025]
PictureIn image processing, an object may be specified by a specific color. For example, in the case of a person, it may be determined as a target by searching for a skin color portion.In addition,Normal color image dataInSince the element of brightness is also included, specify the skin colorButdifficultIn some cases. On the other handIn the above configuration, each pixelChromaticity(Represents the absolute proportion of color stimulus values and is not affected by brightnessStuff). Therefore, if it is in the range of chromaticity that the skin color can take, it can be determined as a pixel of a human image. Of course, in addition to the skin color, the same can be said of the range that the green of trees can take or the blue of the blue sky.
[0026]
  In this background, featuresRe-quantityEvaluation means,The chromaticity calculated for each pixel isPredetermined colorWhen the number of pixels falls within the range, the number of pixels is counted.regionIs determined to be a target, and the weight is increased, and the same result as that obtained by extracting many feature values from the target is obtained.
[0027]
  The weighting method as described above is not necessarily an alternative, but as a suitable example in the case of overlapping application,Means for setting each weighting coefficient for a plurality of elements, and means for determining a final weighting coefficient by adding each weighting coefficient for each of the plurality of elements by weighting according to importance. The feature amount re-evaluation means may re-evaluate the feature amount using the final weighting coefficient. In particular,A configuration in which temporary weighting factors are obtained individually based on a plurality of elements, and further, these are added with weighting according to importance and applied as a final weighting factorCan think.
[0028]
  the aboveTo the configuration ofIn, DoubleTemporary weighting factor individually based on number elementsButAskingIsThe Further, these are added by weighting according to importance, and the feature amount extracted as the final weighting coefficient is reevaluated. Therefore, even if a large weight is given at the stage evaluated by one weighting method, if the importance of the weighting method is low, a large weight may not be given as a result. In addition, there is a case in which the final weighting is higher when the weighting is generally evaluated to be higher than the average with respect to the weighting method having a large difference.
[0029]
  As described above, the method of performing the entire weighted image evenly in the extraction stage and performing the predetermined weighting after the extraction is not necessarily limited to a substantial apparatus. As an example, the image processing method of the present invention includes: A method of performing predetermined image processing on a live-action image composed of a plurality of pixels,
  The image data of the pixels constituting the live-action image is selectively input by thinning evenly on a predetermined basis with respect to all the pixels constituting the live-action image,
  While thinning out the entire live-action image aboveBased on the input image data,Histogram showing luminance distribution for each pixelRepresents the entire live-action imageExtract as feature quantity,
For each region that divides the image subject to image processing into a plurality of predetermined criteria, a weighting coefficient set in accordance with the position of the region in the image is stored in advance.
  The above representative of the whole live-action imageFor featuresMultiple aboveRe-evaluation, which is an evaluation using a weighting coefficient,Find the re-evaluated features for the whole live-action image,
  Based on the reevaluated feature amount, image processing including processing for correcting at least one of brightness and contrast of the image is determined,
  The image processing with the determined contents is performed on the entire image data of the photographed image.And
For the area to be evaluated with a greater weight than the other areas in the reevaluation of the feature amount, a weighting coefficient weighted larger than the weighting coefficient set in the other area is stored, and the stored weighting Perform reevaluation using coefficientsthing
  Is the gist.
[0030]
That is, it is not necessarily limited to a substantial apparatus, and there is no difference that the method is also effective.
[0031]
By the way, as described above, an image processing apparatus that re-evaluates feature amounts by weighting and performs image processing may exist alone or may be used in a state of being incorporated in a certain device. Includes various embodiments. Further, it can be changed as appropriate, for example, by hardware or by software.
[0032]
In the case of software for controlling an image processing apparatus as an embodiment of the idea of the invention, it naturally exists on a recording medium on which such software is recorded, and it must be used.
[0033]
  As an example, the recording medium of the present invention is a recording medium in which a program corresponding to predetermined image processing performed on a real image composed of a plurality of pixels is recorded so as to be readable by a computer.
  A function for selectively inputting image data of pixels constituting the live-action image by thinning evenly on a predetermined basis with respect to all pixels constituting the live-action image;
  While thinning out the entire live-action image aboveBased on the input image data,Histogram showing luminance distribution for each pixelRepresents the entire live-action imageA function to extract as a feature amount;
  The above representative of the whole live-action imageFor featuresFor each area that divides the image subject to image processing into a plurality of predetermined criteriaPrepared in advanceHeavyRe-evaluation, which is an evaluation using the found coefficient, is performed.The feature value is re-evaluated with respect to the entire photographed image.Function and
  A function of determining image processing including processing for correcting at least one of brightness and contrast of an image based on the reevaluated feature amount;
  A function of performing image processing with the determined contents on the entire image data of the real image;
  Record the program that realizes
The function for obtaining the reevaluated feature value is a weighting coefficient that is weighted more than the weighting coefficient set in the other area for the area to be evaluated with a larger weight than the other area in the reevaluation of the feature value. The function to perform the above reevaluation by setting
  Is the gist.
[0034]
Of course, the recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium that will be developed in the future. In addition, the duplication stages such as the primary duplication product and the secondary duplication product are equivalent without any question. In addition, even when the communication method is used as a supply method, the present invention is not changed, and the same applies to the case where data is written on a semiconductor chip.
[0035]
Further, even when a part is software and a part is realized by hardware, there is nothing completely different in the idea of the invention, and a part is stored on a recording medium, and it is appropriately changed as necessary. It may be in the form of being read.
[0036]
  As explained above,The present inventionImage processing equipmentIsExtraction of pixel feature values for entire live-action imageEvenly acrossBecause a predetermined weighting is performed after extraction,Without increasing the amount of processing,And,There is no need to perform irrelevant evaluations as if they were extracted evenly, and automaticallyIn the evaluationPerform image processingNaPossibleBecome.
[0037]
  AlsoThe pixels that make up a live-action imageThinningIf image data is input, the image processingThe amount of processing can be reduced.
[0038]
TerritoryChange the weight for each areaIf configured, the image processingCalculation is relatively easy.
[0039]
  In addition, weightingTheDetermined by pixel positionIf the configuration is used, the image processingCalculation is relatively easy.
[0040]
  In addition, a weighting that is heavier than other regions other than the region is given to a region where the degree of change of the image in the photographed image is large, and the feature amount is reevaluated using this weighting.If,PiecesAccurately distinguishes different targets depending on the image, and featuresobtainbe able to.For example, the weight used in the reevaluation may be changed according to the sharpness of the image.
[0041]
  further,For parts with a large number of pixels that fall within the predetermined color range, the weight is re-evaluated with a heavy weight, so the colorTherefore, it is possible to select a specific target.obtainbe able to.
[0042]
  further,When re-evaluating the feature amount, using a final weighting coefficient determined by adding each weighting coefficient for a plurality of elements with weighting according to importance,A more suitable feature amount can be evaluated by appropriately combining a plurality of weighting methods.
[0043]
  further,Image processing method of the present inventionAccording toWhen processing imagesIt is possible to provide an image processing method that can perform image processing with an optimal evaluation with a small amount of calculation,Recording medium of the present inventionAccording to this, it is possible to provide a medium on which an image processing control program capable of obtaining the same effect is recorded.
[0044]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0045]
FIG. 1 is a block diagram illustrating an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied. FIG. 2 is a schematic block diagram illustrating a specific hardware configuration example.
[0046]
  In FIG. 1, an image input apparatus 10 outputs real image data representing a photograph or the like as a pixel in a dot matrix form to the image processing apparatus 20, and the image processing apparatus 20 determines the degree of enhancement of image processing through a predetermined process. Then, image processing is executed. The image processing apparatus 20 outputs the image processed image data to the image output apparatus 30, and the image output apparatus 30 outputs the image-processed image with a dot matrix pixel. Here, the image data output by the image processing apparatus 20 is:Based on the image data evenly input from the whole live-action image,After extracting features,This featureWith a predetermined weightEvaluation (hereinafter,reassessmentCalled)The image processing is performed with the degree of enhancement determined according to the reevaluated feature amount. Therefore, the image processing apparatus 20Input means for uniformly inputting image data of pixels constituting the photographed image from the entire photographed image, and pixel data based on the input image dataFeature valueExtractFeaturesVolume extractionAnd means for re-evaluating the extracted feature quantity with a predetermined weightRe-quantityEvaluation means and re-evaluated featuresOn the basis of theDegree of emphasisProcessing content determination means for determining the content of the image processing, etc., and the determined contentImage processingFor all image dataProcessing means.
[0047]
A specific example of the image input apparatus 10 corresponds to the scanner 11, the digital still camera 12, or the video camera 14 in FIG. 2, and specific examples of the image processing apparatus 20 include a computer 21, a hard disk 22, a keyboard 23, and a CD-ROM drive 24. A computer system including a floppy disk drive 25 and a modem 26 is applicable, and a specific example of the image output device 30 is a printer 31, a display 32, or the like. In the case of the present embodiment, since an object is found as image processing and appropriate image processing is performed, actual image data such as a photograph is preferable as the image data. The modem 26 is connected to a public communication line, connected to an external network via the public communication line, and can be installed by downloading software and data.
[0048]
In the present embodiment, the scanner 11 and the digital still camera 12 as the image input device 10 output RGB (green, blue, red) gradation data as image data, and the printer 31 as the image output device 30 has a floor. As the tone data, CMY (cyan, magenta, yellow) or CMYK binary data obtained by adding black to this is required as input, and the display 32 requires RGB gradation data as input. On the other hand, an operating system 21a operates in the computer 21, and a printer driver 21b and a display driver 21c corresponding to the printer 31 and the display 32 are incorporated. The image processing application 21d is controlled by the operating system 21a to execute processing, and executes predetermined image processing in cooperation with the printer driver 21b and the display driver 21c as necessary. Therefore, the specific role of the computer 21 as the image processing apparatus 20 is to create RGB gradation data that has been subjected to optimal image processing by inputting RGB gradation data, and displays the display 32 via the display driver 21c. And is converted into binary data of CMY (or CMYK) via the printer driver 21b and printed on the printer 31.
[0049]
As described above, in this embodiment, a computer system is incorporated between image input / output devices to perform image processing. However, such a computer system is not necessarily required, and image data is not necessarily processed. Any system that performs various types of image processing may be used. For example, as shown in FIG. 3, a digital still camera 12a incorporates an image processing apparatus that determines an object and performs image processing, and displays the converted image data on the display 32a or prints it on the printer 31a. There may be. As shown in FIG. 4, in the printer 31b that inputs and prints image data without going through a computer system, the image data inputted through the scanner 11b, the digital still camera 12b, the modem 26b, or the like is automatically used. It is also possible to configure so that an object is determined and image processing is performed.
[0050]
Specifically, the above-described object determination and accompanying image processing are performed in the computer 21 by an image processing program corresponding to the flowchart shown in FIG. In the flowchart shown in the figure, image processing for adjusting the contrast of an image is performed. In step S110, luminance is extracted as a feature amount while thinning out pixels uniformly from the entire image in step S110, and then in step S120. The feature amount is reevaluated by performing predetermined weighting, and image processing for adjusting the luminance is performed in steps S130 to S160.
[0051]
In step S110, as shown in FIG. 6, the brightness of each pixel is obtained for dot matrix image data in the vertical and horizontal directions, and a histogram is generated. In this case, although it can be said that it is accurate if it is performed for all the pixels, the total result is weighted and re-evaluated as will be described later, so it is not necessarily accurate. Therefore, it is possible to reduce the processing amount and increase the speed by thinning out pixels from which luminance is extracted to a certain error range. According to the statistical error, the error with respect to the number of samples N can be expressed as 1 / (N ** (1/2)). However, ** represents a power. Therefore, in order to perform processing with an error of about 1%, N = 10000.
[0052]
Here, the bitmap screen shown in FIG. 6 has the number of pixels of (width) × (height), and the sampling period ratio is
ratio = min (width, height) / A + 1 (1)
And Here, min (width, height) is the smaller of width and height, and A is a constant. Further, the sampling period ratio here indicates how many pixels are sampled, and the pixels marked with ◯ in FIG. 7 indicate the case where the sampling period ratio = 2. That is, one pixel is sampled every two pixels in the vertical and horizontal directions, and every other pixel is sampled. The number of sampling pixels in one line when A = 200 is as shown in FIG.
[0053]
As can be seen from the figure, except for the case where the sampling period ratio = 1 in which sampling is not performed, when there is a width of 200 pixels or more, the number of samples is at least 100 pixels. Therefore, in the case of 200 pixels or more in the vertical direction and the horizontal direction, (100 pixels) × (100 pixels) = (10000 pixels) is secured, and the error can be reduced to 1% or less.
[0054]
Here, the reason for using min (width, height) as a reference is as follows. For example, as shown in FIG. 9A, when width >> height is determined with the longer width as shown in FIG. 9A, the sampling period ratio is determined as shown in FIG. 9B. In addition, in the vertical direction, only two lines of the upper end and the lower end may be extracted. However, if the sampling period ratio is determined based on the smaller one as min (width, height), thinning is performed so as to include the intermediate portion in the smaller vertical direction as shown in FIG. Will be able to. That is, sampling with a predetermined number of extractions is possible.
[0055]
In this way, it is brightness that is extracted by thinning out pixels. As described above, in the present embodiment, the computer 21 handles RGB gradation data, and does not have a luminance value directly. Although it is possible to perform color conversion to the Luv color space in order to obtain the luminance, it is not a good idea because of problems such as the amount of computation. For this reason, the following conversion formula for directly obtaining the luminance from the RGB used in the case of a television or the like is used.
[0056]
Y = 0.30R + 0.59G + 0.11B (2)
Further, the luminance histogram is not aggregated for one image, but the input image is separately aggregated into 15 blocks of 3 horizontal blocks and 5 vertical blocks as shown in FIG. In the present embodiment, such 15 blocks are used, but the block dividing method is arbitrary. In particular, a printer driver or the like receives image data for each block from an application. However, area division for weighting may be performed using this block.
[0057]
The aggregation for each block is for the purpose of reducing the processing amount. In the sense of performing weighting and re-evaluation in step S120, it is not always necessary to add up for each block, and it is also possible to add up as a histogram in consideration of weighting for each selected pixel. Further, even if each block is used, it means that the weighting of the totaling result is changed, and it is possible to totalize with one histogram using the weighting according to the block. FIG. 11 is a diagram illustrating an example of the luminance distribution of the block Bi.
[0058]
When counting is performed for each block, weighting for each region is performed and re-evaluated in step S120. 12 and 13 show examples in which weighting is given to each block. If a general photographic image is assumed, the composition is usually such that the original subject enters the central portion. In this sense, the feature value should be evaluated with emphasis on the image at the center of the image data. On the other hand, when taking an example of taking a commemorative photo in front of a building as another composition, the human figure is taken in the lower part of the center. This is because the ground level is usually located at the bottom of the image. Therefore, in this case, it can be said that the feature amount should be evaluated with a weight placed below the center of the image. The example of FIG. 12 corresponds to the former, and FIG. 13 corresponds to the latter.
[0059]
The weight of each block is set to Wi (i = 1 to 15), the luminance distribution weighted and re-evaluated is set to DY,
[0060]
[Expression 1]

[0061]
And
Ki = Wi / SP (4)
Then,
[0062]
[Expression 2]

[0063]
As required.
[0064]
When the brightness distribution histogram reevaluated in this way is obtained, the intensity of image processing is obtained from this feature amount. That is, the width for increasing the contrast is determined. In determining the enlargement width, consider obtaining both ends of the luminance distribution. The luminance distribution of a photographic image appears generally in a mountain shape as shown in FIG. Of course, there are various positions and shapes. The width of the luminance distribution is determined by where the both ends are determined. However, the point where the base is simply extended and the number of distributions becomes “0” cannot be the both ends. This is because the number of distributions may change around “0” in the base portion, and if it is statistically viewed, it changes while approaching “0” as much as possible.
[0065]
For this reason, in the distribution range, portions that have passed a certain distribution ratio from the side with the highest luminance and the side with the lowest luminance are defined as both ends of the distribution. In the present embodiment, as shown in the figure, this distribution ratio is set to 0.5%. Of course, this ratio can be appropriately changed. In this way, by cutting the upper end and the lower end by a certain distribution ratio, it is possible to ignore white spots and black spots caused by noise or the like. In other words, if such a process is not performed, if there is even a single point of white or black, it will be at both ends of the luminance distribution. Although it is “0” and the uppermost end is gradation “255”, such a thing can be achieved by setting a portion that is inward by 0.5% of the number of pixels from the upper end portion as an end portion. Disappear.
[0066]
In actual processing, 0.5% of the number of pixels is calculated on the basis of the histogram obtained by re-evaluation, and each of the luminance values at the top and bottom in the reproducible luminance distribution is sequentially turned inward. The number of distributions is accumulated to obtain a luminance value that is 0.5%. Hereinafter, this upper end side is called ymax, and the lower end side is called ymin. Further, in the present embodiment, the brightness is corrected together with the contrast expansion, and the median ymed necessary for this is determined based on the reevaluated histogram. The above process is performed in step S130.
[0067]
When the reproducible luminance range is “0” to “255”, the conversion destination luminance Y is obtained from the luminance y before conversion and the maximum value ymax and minimum value ymin of the luminance distribution range based on the following equation. .
[0068]
Y = ay + b (6)
However,
a = 255 / (ymax−ymin) (7)
b = −a · ymin or 255−a · ymax (8)
In the above conversion equation, if Y <0, Y = 0, and if Y> 255, Y = 255. Here, a is an inclination and b can be said to be an offset. According to this conversion equation, as shown in FIG. 15, the luminance distribution having a certain narrow width can be expanded to a reproducible range. However, when the luminance distribution is expanded by making the maximum use of the reproducible range, the highlight portion may be white and the high shadow portion may be black. In order to prevent this, the reproducible range is limited in this embodiment. That is, the luminance value “5” is left as a range that does not expand to the upper end and the lower end of the reproducible range. As a result, the parameters of the conversion equation are as follows:
[0069]
a = 245 / (ymax−ymin) (9)
b = 5-a · ymin or 250-a · ymax (10)
In this case, conversion is not performed in the range of y <ymin and y> ymax.
[0070]
However, if the same enlargement ratio (corresponding to a) is applied, a very large enlargement ratio may be obtained. For example, in the twilight state in the evening, it is natural that the contrast range from the brightest part to the dark part is narrow, but as a result of trying to enlarge the contrast greatly for this image, it is converted like a daytime image. It can be. Since such conversion is not desired, a restriction is set on the enlargement ratio so that a is not 1.5 (˜2) or more. As a result, twilight is expressed as twilight. In this case, processing is performed so that the center position of the luminance distribution does not change as much as possible. In step S140, processing for obtaining the enlargement ratio a and the slope b is executed in this way.
[0071]
By the way, it is unreasonable to execute the conversion equation (Y = ay + b) every time the luminance is converted. This is because the range that the luminance y can take is only “0” to “255”, and it is possible to obtain the luminance Y after conversion corresponding to all the values that the luminance y can take in advance. It is. Therefore, it is stored as a table as shown in FIG.
[0072]
It is extremely effective to not only enhance the contrast by expanding the brightness range but also adjust the brightness together. Therefore, the brightness of the image is judged and a parameter for correction is also generated.
[0073]
  For example, as shown by the solid line in FIG.BreakAs shown by the line, it is better to move the mountain to the bright side as a whole, and conversely, as shown by the solid line in FIG. It is better to move the mountain to the dark side as shown.
[0074]
As a result of various experiments, in the present embodiment, the median ymed in the luminance distribution is obtained, and when the median ymed is less than “85”, the image is determined to be a dark image and γ correction corresponding to the following γ values is performed. Brighten.
[0075]
γ = ymed / 85 (11)
Or
γ = (ymed / 85) ** (1/2) (12)
And
[0076]
In this case, even if γ <0.7, γ = 0.7. This is because if such a limit is not set, the night image becomes like the daytime. Note that if the image is too bright, the overall image becomes whitish and the contrast tends to be low, so that processing such as enhancement of saturation is suitable.
[0077]
On the other hand, when the median ymed is larger than “128”, the image is judged to be a bright image and darkened by γ correction corresponding to the following γ values.
[0078]
γ = ymed / 128 (13)
Or
γ = (ymed / 128) ** (1/2) (14)
And In this case, even if γ> 1.3, a limit is set so that γ = 1.3 so as not to be too dark.
[0079]
This γ correction may be performed on the luminance distribution before conversion or on the luminance distribution after conversion. FIG. 19 shows the correspondence when γ correction is performed. When γ <1, the curve swells upward, and when γ> 1, the curve swells downward. Of course, the result of such γ correction may be reflected in the table shown in FIG. 16, and the same correction is performed on the table data. In step S150, processing for generating such a conversion table is executed.
[0080]
Thereafter, the conversion based on the expression (6) is performed. The conversion expression of the expression can also be applied in the correspondence relationship with the RGB component values, and for the component values (R0, G0, B0) before the conversion. The component values (R, G, B) after conversion are
R = a · R0 + b (15)
G = a · G0 + b (16)
B = a · B0 + b (17)
Can also be sought. Here, corresponding to the luminance y and Y ranging from gradation “0” to gradation “255”, the RGB component values (R0, G0, B0), (R, G, B) are also in the same range. Therefore, it can be said that the luminance y and Y conversion tables described above may be used as they are.
[0081]
Therefore, in step S160, the image data (R, G, B) after conversion is obtained by referring to the conversion table corresponding to the equations (15) to (17) for the image data (R0, G0, B0) of all pixels. The process will be repeated.
[0082]
  Of course, depending on the hardware configuration and software that execute these steps S130 to S160.Processing content decision means orProcessing means will be constituted. In the present embodiment, contrast enlargement processing and brightness correction processing are executed as image processing. Needless to say, the present invention can be applied to other image enhancement processing in the same manner.
[0083]
In the above processing, the feature amount is re-evaluated by weighting according to the position in the image. However, the weighting criterion is not limited to this, and various modes are possible. FIG. 20 shows a flowchart in the case where the original subject is detected from the degree of change of the image and the weighting is changed as an example.
[0084]
Step S210 is an alternative to step S110 described above, and the luminance is totalized for pixels that are thinned out evenly. However, not only the luminance but also the following edge amounts are totaled.
[0085]
It can be said that the background of the input image undergoes a gradual change in shading, and that the original subject is sharp, so that the change in luminance is severe. Therefore, as shown in FIG. 21, one pixel is regarded as a pixel of interest, and a density difference with a peripheral pixel is obtained, and this density difference is set as an edge amount of the pixel of interest. This density difference can be calculated by applying a filter, and FIGS. 22A to 22F show several examples of such a filter. The brightness of each pixel is calculated for the target pixel and the eight peripheral pixels. A weighting coefficient for weighted addition is shown. Here, in the case of FIG. 9A, since weighted addition is performed for nine pixels, nine multiplications and eight additions are required to obtain the edge amount at each pixel. Since this amount of computation cannot be ignored as the image becomes larger, it is sufficient to perform five multiplications and four additions in FIGS. 5B and 5C, and three multiplications in FIGS. In FIG. 8F, only two multiplications and one addition are required.
[0086]
In these examples, the pixel of interest is compared only with the pixel that surrounds it, but the sharpness at the pixel of interest is determined using a wider range of pixel data using a so-called unsharp mask. Is possible. However, since the edge amount in the present embodiment is only for evaluating the weighting for each block, the filter of these examples with a reduced amount of calculation is sufficient.
[0087]
The total amount of edges may be totaled for each pixel for each block, and when the absolute value of this edge amount is larger than a predetermined threshold, it is determined as an edge pixel, You may make it total the total number of edge pixels for every block. If the edge amount in each block is ERi (i = 1 to 15), the total amount SE is
[0088]
[Equation 3]

[0089]
Therefore, the weighting coefficient KEi itself is
KEi = ERi / SE (19)
It is represented by Therefore, the luminance distribution DY re-evaluated by weighting in this case is
[0090]
[Expression 4]

[0091]
As required. If the total number of edge pixels in each block is ENi (i = 1 to 15), the total amount SE is
[0092]
[Equation 5]

[0093]
Therefore, the weighting coefficient KEi itself is
KEi = ENi / SE (22)
The luminance distribution DY that is reevaluated by using the equation (20) can be obtained. In any case, in step S220, the luminance distribution DY is re-evaluated based on the arithmetic expression (20).
[0094]
In this example, the luminance of the pixel determined as the edge pixel is not sampled and used, but the edge amount and the total number of edge pixels are only used for determining the weighting coefficient of the block. That is, the feature amounts of only the pixels having a specific property (edge) are not aggregated, and an average feature amount that is not biased in the block can be obtained.
[0095]
If the luminance distribution is re-evaluated in this way, the contrast may be increased and the lightness may be corrected through the processes in steps S130 to S160 described above.
[0096]
Furthermore, considering that the original subject is often a human image, it is possible to re-evaluate with emphasis on blocks with many skin-colored pixels. FIG. 23 shows a flowchart for determining a weighting coefficient of a block by paying attention to such a specific color.
[0097]
In step S310 corresponding to step S110, the luminance is totaled by the same thinning process, and it is determined based on the chromaticity of each pixel whether the pixel looks like a skin color, and the number of skin color pixels is totaled. For chromaticity, the xy chromaticity for each pixel is calculated. Now, when the RGB gradation data in the RGB color system of the target pixel is (R, G, B),
r = R / (R + G + B) (23)
g = G / (R + G + B) (24)
Then, between the chromaticity coordinates x and y in the XYZ color system,
x = (1.1302 + 1.6387r + 0.6215 g)
/(6.78446-3.0157r-0.3857g)...(25)
y = (0.0601 + 0.9399r + 4.5306 g)
/(6.78446-3.0157r-0.3857g)...(26)
The corresponding relationship is established. Here, the chromaticity represents the absolute ratio of the stimulus value of the color without being influenced by the brightness, so it can be said that it is possible to determine what kind of object the pixel is based on the chromaticity. . For skin tone,
0.35 <x <0.40 (27)
0.33 <y <0.36 (28)
Therefore, if the chromaticity of each pixel is obtained, the pixel is considered to be a pixel indicating human skin, and the number of skin color pixels in the block is increased by one. .
[0098]
When the number of skin color pixels is obtained in this way, in the next step S320, the weighting coefficient is determined in the same manner as in the case of the number of edge pixels described above, and the luminance distribution DY is reevaluated. That is, if the number of skin color pixels in each block is CNi (i = 1 to 15), the total amount SC is
[0099]
[Formula 6]

[0100]
Therefore, the weighting KCi itself is
KCi = CNi / SC (30)
In this case, the weighted distribution DY re-evaluated is
[0101]
[Expression 7]

[0102]
As required. In this example as well, the luminance of pixels determined to be skin color pixels is not sampled and used, but the total number of skin color pixels is only used to determine the weighting coefficient of the block. Therefore, it is possible to obtain an average feature amount that is not biased in the block. Also in this case, if the luminance distribution is re-evaluated in this way, the contrast may be enlarged and the lightness may be corrected through the processes of steps S130 to S160 described above. In the case of the photograph shown in FIG. 24, a girl appears in the vicinity of the center, and the face and limb pixels are determined to be skin color pixels. Of course, the number of pixels may be totaled by obtaining chromaticities for other colors.
[0103]
By the way, the weighting coefficient has been determined by one factor so far, but it can be applied in duplicate with the importance of each factor taken into account. When the weighting coefficient of each block Bi (i = 1 to 15) is Tji for each factor j (1: position in the image, 2: edge amount, 3: skin color pixel number), each block for each factor The weight Tji distributed to is a temporary weight,
[0104]
[Equation 8]

[0105]
And
Kji = Tji / Sj (33)
Then, the true weighting coefficient Ki in the block Bi is
[0106]
[Equation 9]

[0107]
It is represented by Here, Aj is a coefficient representing the importance for each factor, and is appropriately determined within a range where the total number is 1. As an example, if emphasis is placed on skin color, settings such as A1 = 0.2, A2 = 0.2, and A3 = 0.6 are possible.
[0108]
Next, the operation of the present embodiment configured as described above will be described. First, it demonstrates along previous embodiment.
[0109]
Assume that a photographic image is read by the scanner 11 and printed by the printer 31. Then, first, while the operating system 21a is operating on the computer 21, the image processing application 21d is started, and the scanner 11 is started to read a photograph. When the read image data is taken into the image processing application 21d via the operating system 21a, the luminance of each pixel is totaled while thinning out in step S110. The aggregated luminance distribution dYi is re-evaluated based on the weight determined corresponding to the position for each block shown in FIG. 12 or FIG. 13 in step S120, and step S130 based on the re-evaluated luminance distribution DY. Then, ymax, ymin, and ymed are obtained.
[0110]
In the next step S140, the slope a and the offset b, which are enhancement parameters, are calculated based on the formula (9) or the formula (10), and γ required for the brightness correction based on the formulas (11) to (14). A correction γ value is obtained, and a conversion table shown in FIG. 6 is created in step S150. Finally, in step S160, the image data for all pixels is converted with reference to the conversion table.
[0111]
In the case of using the weighting shown in FIG. 12, since the weighting is heavier as the block is closer to the center, the aggregated luminance distribution is also more greatly evaluated as being closer to the center. For example, assume that a person image is photographed using a flash at night. The luminance distribution of the person image has a flash effect, and even if a generally good luminance distribution is obtained as shown in FIG. 25A, the periphery of the person is dark and dark as shown in FIG. A luminance distribution biased to the side is obtained. In this case, if it is simply averaged, a luminance distribution biased toward the dark side as a whole can be obtained as shown in FIG. Therefore, if the contrast and brightness are corrected as they are, a dark image is forcibly brightened and a good image cannot be obtained.
[0112]
However, when the weight is increased in the central block as shown in FIG. 12, the luminance distribution DY strongly influenced by the luminance distribution obtained in the central portion of the image is obtained as shown in FIG. can get. Therefore, the intensity of the image processing determined based on the luminance distribution does not excessively increase the contrast or correct the brightness.
[0113]
Conversely, when a person is photographed in a backlight condition, the face becomes dark and the background becomes bright. However, if the entire image is viewed, the luminance distribution may be good. However, even in such a case, as shown in FIG. 12, the evaluation is performed with emphasis on the luminance distribution in the central block in which the dark face is reflected, so that the dark luminance distribution is reflected and the contrast is increased. It becomes possible to execute image processing for making the image brighter.
[0114]
Through the above processing, the image data of the photograph read through the scanner 11 is automatically subjected to image processing with the optimum intensity, displayed on the display 32, and then printed by the printer 31.
[0115]
As described above, the computer 21, which is the center of the image processing, aggregates the luminance distribution as the feature amount for each region while selecting pixels uniformly in step S110, and then, in step S120, the weight determined for each region. By re-evaluating in step S130, it is possible to obtain a luminance distribution that is strongly influenced by the luminance distribution of the original subject while uniformly sampling, and in steps S130 to S150, based on the luminance distribution, the intensity of image processing Since the image data is converted in step S160, the image processing can be executed with the optimum intensity while reducing the processing.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied.
FIG. 2 is a block diagram of specific hardware of the image processing apparatus.
FIG. 3 is a schematic block diagram illustrating another application example of the image processing apparatus of the present invention.
FIG. 4 is a schematic block diagram illustrating another application example of the image processing apparatus of the present invention.
FIG. 5 is a flowchart showing image processing in the image processing apparatus of the present invention.
FIG. 6 is a diagram illustrating a state in which a processing target pixel is moved.
FIG. 7 is a diagram showing a sampling period.
FIG. 8 is a diagram illustrating the number of sampling pixels.
FIG. 9 is a diagram illustrating a relationship between a conversion source image and sampled pixels.
FIG. 10 is a diagram illustrating an arrangement of blocks obtained by dividing an image into regions.
FIG. 11 is a diagram illustrating an example of a luminance distribution in each block.
FIG. 12 is a diagram illustrating an example of weighting for each block.
FIG. 13 is a diagram illustrating another example of weighting for each block.
FIG. 14 is a diagram showing edge processing of luminance distribution and edges obtained by the edge processing.
FIG. 15 is a diagram illustrating an expansion of a luminance distribution and a reproducible luminance range.
FIG. 16 is a diagram illustrating a conversion table when a luminance distribution is enlarged.
FIG. 17 is a diagram showing a concept of brightening by γ correction.
FIG. 18 is a diagram showing a concept of darkening by γ correction.
FIG. 19 is a diagram illustrating a correspondence relationship of luminance changed by γ correction.
FIG. 20 is a flowchart for re-evaluating a feature amount based on an edge amount.
FIG. 21 is a diagram illustrating a relationship between a target pixel and a peripheral pixel for determining an edge amount.
FIG. 22 is a diagram illustrating an example of a filter for calculating an edge amount.
FIG. 23 is a flowchart for re-evaluating feature amounts based on the number of pixels of a predetermined color.
FIG. 24 is a diagram illustrating an example of a photographic image.
FIG. 25 is a diagram showing a luminance distribution of a photographic image taken at night.
[Explanation of symbols]
10. Image input device
20 Image processing apparatus
21 ... Computer
21a ... Operating system
21b ... Printer driver
21c ... Display driver
21d: Image processing application
30. Image output device

Claims (3)

An image processing apparatus that performs predetermined image processing on a real image composed of a plurality of pixels,
The image data of the pixels constituting the photographed image, from the entire said real image, an input unit while uniformly thinned by a predetermined reference, entering evenly,
Based on the image data inputted while thinning the whole of the photographed image, and feature extraction means for extracting a histogram showing the distribution of brightness of each pixel as the feature representing the entire the photographed image,
Re-evaluation, which is an evaluation using a weighting coefficient set in advance for each region that divides the image to be subjected to the image processing into a plurality of parts based on a predetermined reference, with respect to the feature amount representing the entire photographed image. row stomach, a feature amount revaluation unit for obtaining the feature amount reevaluated for the entire the photographed image,
Processing content determination means for determining image processing including processing for correcting at least one of brightness and contrast of an image based on the reevaluated feature amount;
The image processing according to the content the determined, e Bei and processing means for implementing the entire image data of the photographed image,
The feature amount re-evaluation means sets a weighting factor that is weighted more than the weighting factor set in the other region for the region to be evaluated with a larger weight than the other region in the re-evaluation of the feature amount. , Re-evaluate
Images processing device. A method for performing predetermined image processing on a live-action image composed of a plurality of pixels,
The image data of the pixels constituting the live-action image is selectively input by thinning evenly on a predetermined basis with respect to all the pixels constituting the live-action image,
Based on the image data inputted while thinning the whole of the photographed image, and extracts a histogram indicating a distribution of luminance of each pixel as the feature representing the entire the photographed image,
For each region that divides the image subject to image processing into a plurality of predetermined criteria, a weighting coefficient set in accordance with the position of the region in the image is stored in advance.
Re-evaluation, which is an evaluation using the plurality of weighting factors, is performed on the feature amount representing the entire photographed image , and the reevaluated feature amount for the entire photographed image is obtained.
Based on the reevaluated feature amount, image processing including processing for correcting at least one of brightness and contrast of the image is determined,
The image processing with the determined content is performed on the entire image data of the real image ,
For the area to be evaluated with a greater weight than the other areas in the reevaluation of the feature amount, a weighting coefficient that is weighted more than the weighting coefficient set in the other area is stored, and the stored weighting An image processing method for performing reevaluation using a coefficient . A recording medium in which a program corresponding to predetermined image processing performed on a real image composed of a plurality of pixels is recorded so as to be readable by a computer,
A function for selectively inputting image data of pixels constituting the live-action image by thinning evenly on a predetermined basis with respect to all pixels constituting the live-action image;
Based on the image data inputted while thinning the whole of the photographed image, a function of extracting a histogram showing the distribution of brightness of each pixel as the feature representing the entire the photographed image,
With respect to the feature quantity that represents the entire above photographed image is the assessment using weighted with coefficients prepared in advance for each of the regions for dividing the image to be the image processing into a plurality at a predetermined reference again evaluation rows that have and a function of obtaining the feature amount is reevaluated for the entire the photographed image,
A function for determining image processing including processing for correcting at least one of brightness and contrast of an image based on the reevaluated feature amount;
A program for realizing, by a computer, a function for executing the image processing with the determined contents on the entire image data of the photographed image,
The function for obtaining the re-evaluated feature value is a weighting coefficient that is weighted more than the weighting coefficient set in the other area for the area to be evaluated with a larger weight than the other area in the re-evaluation of the feature value. A recording medium which is a function for performing the above-described reevaluation .

Images