CN101034440A

CN101034440A - Identification method for spherical fruit and vegetables

Info

Publication number: CN101034440A
Application number: CN 200710066695
Authority: CN
Inventors: 古辉; 芦亚亚
Original assignee: Zhejiang University of Technology ZJUT
Current assignee: Zhejiang University of Technology ZJUT
Priority date: 2007-01-12
Filing date: 2007-01-12
Publication date: 2007-09-12
Anticipated expiration: 2027-01-12
Also published as: CN100463001C

Abstract

The invention is a recognition method of spherical fruit and vegetable category, transforming image obtained to the 2 r-g-b color model and LCD color model, according to classifier principle, discriminant F1, F2, F3, F4 were constructed separately under the two coordinate system, getting separate beeline omitted from the leaves and branches, the input image is divided into a small piece of the same size; orderly select two small plots B1 and B2, calculating the four direction of the gray co-occurrence matrix and two eigenvalues: entropy ENT and energy ASM, and the mean ENT and ASM; two adjacent plots of B1 and B2, such as discriminants F1, F2 , F3 and F4 is greater than zero, or the difference between ENT and ASM of B1 and B2 is less than the threshold T set, retaining B1, recognized as fruit; if the texture margin is greater than the threshold T and discriminants F1, F2, F3 and F4 is less than zero, discarding the entire plot B1; orderly take the next another plot as B2, repeating the steps until all plots have been disposed of. The invention can effectively recognize spherical fruits and vegetables, is simple to calculate with identify of high accuracy.

Description

A method for identifying spherical fruits and vegetables

(一)技术领域(1) Technical field

本发明涉及一种类球形果蔬的识别方法。The invention relates to a method for identifying spherical fruits and vegetables.

(二)背景技术(2) Background technology

水果与蔬菜的采收是一项劳动密集型的工作。果蔬采收的季节性强，工作环境艰苦，劳动强度大。现阶段的采收基本依靠人力，采收的效率比较低，人力成本在整个果蔬生产成本中占了相当大的比例。因此，实现果蔬采收的自动化具有实际应用价值。Harvesting fruit and vegetables is labor intensive. The harvesting of fruits and vegetables is seasonal, the working environment is difficult, and the labor intensity is high. Harvesting at this stage basically relies on manpower, and the efficiency of harvesting is relatively low, and labor costs account for a considerable proportion of the entire fruit and vegetable production costs. Therefore, the automation of fruit and vegetable harvesting has practical application value.

目前为止，国内还没有一台真正投入生产使用的果蔬采摘机器人，主要原因之一是视觉系统技术难题尚未完全解决。通常，采摘机器人视觉系统的工作方式：首先获取果蔬的数字化图像，然后再通过相关图像处理算法识别并确定图像中果蔬的位置。传感器是机器视觉系统最重要的部件，主要包括图像传感器和距离传感器等。图像传感器可以是CCD黑白相机、彩色摄像机或者立体摄像机，一般安装于机械臂或末端执行器上。距离传感器有激光测距、超声、无线和红外传感器等。So far, there is no domestic fruit and vegetable picking robot that has actually been put into production. One of the main reasons is that the technical problems of the vision system have not been completely resolved. Usually, the working method of the vision system of the picking robot is: first obtain the digital image of the fruit and vegetable, and then identify and determine the position of the fruit and vegetable in the image through the relevant image processing algorithm. Sensors are the most important components of machine vision systems, mainly including image sensors and distance sensors. The image sensor can be a CCD black-and-white camera, a color camera or a stereo camera, and is generally installed on a robotic arm or an end effector. Distance sensors include laser ranging, ultrasonic, wireless and infrared sensors.

较早期的果蔬采摘机器人视觉系统大多是基于二维系统，当叶子和果蔬的颜色对比明显时，这种二维系统可以成功地将果蔬从叶子中检测出来；然而，若有多个果蔬重叠或者其颜色与背景色近似时，则很难进行识别。这时，一般是根据果蔬表面不同的光谱反射特性来检测果蔬，检测到果蔬后，利用三维视觉传感器再对果蔬进行精确识别，这是最常用的光谱反射率法，但在自然光照情况下，由于图像中存在噪声等各种干扰信息，这种方法往往并不理想。有研究人员利用果蔬的形状来识别和定位果实，采用形状定位方式，一般要求目标具有完整的边界条件，若物体被遮挡，就很难做到。又有人提出一种称为Hough变换的新方法，它的特点是不需要整个轮廓信息，根据目标果蔬形状的曲率信息来定位果蔬的中心，但是该方法非常耗时。总之，由于环境的高度复杂性，目前的采摘机器人视觉系统在规则环境下，如已知光照背景下效果比较好，但在自然环境条件，如自然光照下则不是很成功。所以，概括起来，当前识别果蔬目标的方法，主要还是通过颜色空间模型的变换，利用颜色特征参数来提取果蔬目标，能够解决的识别问题非常有限。Most of the early vision systems for fruit and vegetable picking robots were based on two-dimensional systems. When the color contrast between leaves and fruits and vegetables is obvious, this two-dimensional system can successfully detect fruits and vegetables from leaves; however, if there are multiple fruits and vegetables overlapping or When its color is similar to the background color, it is difficult to identify. At this time, the fruits and vegetables are generally detected according to the different spectral reflectance characteristics of the surface of the fruits and vegetables. After the fruits and vegetables are detected, the three-dimensional vision sensor is used to accurately identify the fruits and vegetables. This is the most commonly used spectral reflectance method, but under natural light conditions, This method is often not ideal due to various interference information such as noise in the image. Some researchers use the shape of fruits and vegetables to identify and locate fruits. The shape location method generally requires the target to have complete boundary conditions. If the object is blocked, it is difficult to do so. Someone proposed a new method called Hough transform, which is characterized in that it does not need the entire contour information, and locates the center of the fruit and vegetable according to the curvature information of the target fruit and vegetable shape, but this method is very time-consuming. In conclusion, due to the high complexity of the environment, current picking robot vision systems work well in regular environments, such as known lighting backgrounds, but are not very successful in natural environmental conditions, such as natural lighting. Therefore, to sum up, the current methods for identifying fruit and vegetable targets mainly use color feature parameters to extract fruit and vegetable targets through the transformation of the color space model, and the identification problems that can be solved are very limited.

(三)发明内容(3) Contents of the invention

为了克服已有的类球形果蔬识别方法不能有效识别类球形果蔬、计算复杂、识别精度低的不足，本发明提供一种能够有效识别类球形果蔬、计算简单、识别精度高的一种类球形果蔬的识别方法。In order to overcome the shortcomings of the existing spherical-shaped fruit and vegetable identification methods that cannot effectively identify spherical-shaped fruits and vegetables, the calculation is complicated, and the recognition accuracy is low, the present invention provides a spherical-shaped fruit and vegetable that can effectively identify spherical-shaped fruits and vegetables, and has simple calculation and high recognition accuracy. recognition methods.

本发明解决其技术问题所采用的技术方案是：The technical solution adopted by the present invention to solve its technical problems is:

一种类球形果蔬的识别方法，该方法包括以下步骤：A method for identifying spherical fruits and vegetables, the method comprising the following steps:

(1)、获取自然场景下的果蔬图像；(1) Obtain images of fruits and vegetables in natural scenes;

(2)、将获取的图像同时变换到2r-g-b颜色模型和LCD颜色模型，对2r-g-b颜色模型建立2r-g二维坐标系，LCD颜色模型建立Y-Cr二维坐标系；(2), the acquired image is simultaneously converted to the 2r-g-b color model and the LCD color model, the 2r-g two-dimensional coordinate system is established for the 2r-g-b color model, and the Y-Cr two-dimensional coordinate system is established for the LCD color model;

(3)、根据分类器原理，对Y-C_r二维坐标系下的特征属性Y，C_r分别构造判别式F₁、F₂，设定特征属性值Y和C_r的果实目标的平均向量m_fruit、树叶的平均向量m_leaf、枝干的平均向量m_branch，其算式为：(1)、(2)：(3) According to the principle of the classifier, construct discriminant formulas F ₁ and F ₂ respectively for the characteristic attributes Y and C _r in the YC _r two-dimensional coordinate system, and set the average vector m of the fruit target of the characteristic attribute values Y and C _r _fruit , the average vector m _leaf of the leaves, and the average vector m _{branch of} the branches, the formulas are: (1), (2):

F₁＝[Y，C_r]^T(m_fruit-m_leaf)-1/2[(m_fruit ^Tm_fruit)-(m_leaf ^Tm_leaf)] (1)F ₁ =[Y, C _r ] ^T (m _fruit -m _leaf )-1/2[(m _fruit ^T m _fruit )-(m _leaf ^T m _leaf )] (1)

F₂＝[Y，C_r]^T(m_fruit-m_branch)-1/2[(m_fruit ^Tm_fruit)-(m_branch ^Tm_lbranch)] (2)F ₂ =[Y, C _r ] ^T (m _fruit -m _branch )-1/2[(m _fruit ^T m _fruit )-(m _branch ^T m _lbranch )] (2)

对2r-g二维坐标系的特征属性2r和g分别构造判别式F₃、F₄，设定特征属性值2r和g的果实目标的平均向量m′f_ruit、树叶的平均向量m′_leaf、枝干的平均向量m′_branch，其算式为：(3)、(4)：Construct discriminant formulas F ₃ and F ₄ for the characteristic attributes 2r and g of the 2r-g two-dimensional coordinate system respectively, and set the average vector m′fruit of the fruit target and the average vector m′ _leaf of the _leaf of the characteristic attribute values 2r and g , the average vector m′ _branch of the branches, the formulas are: (3), (4):

F₃＝[2r，g]^T(m′_fruit-m′_leaf)-1/2[(m′_fruit ^Tm′_fruit)-(m′_leaf ^Tm′_leaf)] (3)F ₃ =[2r, g] ^T (m′ _fruit -m′ _leaf )-1/2[(m′ _fruit ^T m′ _fruit )-(m′ _leaf ^T m′ _leaf )] (3)

F₄＝[2r，g]^T(m′_fruit-m′_branch)-1/2[(m′_fruit ^Tm′_fruit)-(m′_branch ^Tm′_lbranchz)] (4)F ₄ =[2r, g] ^T (m′ _fruit -m′ _branch )-1/2[(m′ _fruit ^T m′ _fruit )-(m′ _branch ^T m′ _lbranchz )] (4)

根据判别式得到省去叶子和枝干的分离直线，并把输入图像分成大小相等的小块，每块大小为L×L，L为奇数；According to the discriminant formula, the separating straight line omitting the leaves and branches is obtained, and the input image is divided into small blocks of equal size, each block is L×L, and L is an odd number;

(4)、顺序选择两个小块B₁和B₂，分别计算其4个方向的灰度共生矩阵，设图像的一个区域大小为N_c×N_r像素，并设灰度级为G＝0，1，……，N_q-1，共生矩阵P_(d，q)是一个大小为N_q×N_q的方阵，包括所有间距为d，方向为q，且灰度级为a和b的像素对出现的频率，P_(d，q)中的元素表示为P_{(a，b|d，q)}，在区域内任选两个像素(k，l)和(m，n)，其中k，m＝1，2，……，N_c；n＝1，2，……，N_r；(4) Select two small blocks B ₁ and B ₂ sequentially, and calculate the gray level co-occurrence matrix in the four directions respectively, set the size of an area of the image as N _c × N _r pixels, and set the gray level as G= 0, 1, ..., N _q -1, the co-occurrence matrix P _{(d, q)} is a square matrix of size N _q ×N _q , including all spacing d, direction q, and gray levels a and The frequency of occurrence of the pixel pair of b, the element in P _{(d, q)} is expressed as P _{(a, b|d, q)} , and two pixels (k, l) and (m, n) are selected in the area, where k, m=1, 2, ..., N _c ; n = 1, 2, ..., N _r ;

计算和各个灰度共生矩阵的两个特征值：熵ENT和能量ASM，其算式分别为(5)、(6)：Calculation and two eigenvalues of each gray level co-occurrence matrix: entropy ENT and energy ASM, the formulas are (5), (6):

$ENT ENT = = - - {Σ Σ}_{i i = = 00}^{{N N}_{q q} - - 11} {Σ Σ}_{j j = = 00}^{{N N}_{q q} - - 11} P P ((a a,, b b)) log log P P ((a a,, b b)) - - - - - - ((55))$

$ASM ASM = = {Σ Σ}_{i i = = 00}^{{N N}_{q q} - - 11} {Σ Σ}_{j j = = 00}^{{N N}_{q q} - - 11} {((P P ((a a,, b b))))}^{22} - - - - - - ((66))$

其中，a，b分别表示像素的灰度级，p(a，b)表示灰度共生矩阵；并计算平均4个方向的灰度共生矩阵的特征值得到平均特征值 ENT和ASM；Among them, a, b represent the gray level of the pixel respectively, p(a, b) represents the gray level co-occurrence matrix; and calculate the average eigenvalue of the gray level co-occurrence matrix in 4 directions to obtain the average eigenvalue ENT and ASM;

(5)、对于相邻的两个小块B₁和B₂，如判别式F₁、F₂、F₃和F₄大于0或者B₁和B₂的 ENT和 ASM之差小于设定的阈值T，则保留B₁，该小块确认为果实；如纹理差值大于阈值T且判别式F₁、F₂、F₃和F₄小于0，则丢弃整个小块B₁；并将B₂中相关参数赋给B₁，然后将B₂的参数置为空，顺序往后取另一个小块作为B₂，重复所述步骤，直到所有小块处理完毕。(5) For two adjacent small blocks B ₁ and B ₂ , if the discriminants F ₁ , F ₂ , F ₃ and F ₄ are greater than 0 or the difference between ENT and ASM of B ₁ and B ₂ is less than the set threshold T, then keep B ₁ , and the small block is confirmed as a fruit; if the texture difference is greater than the threshold T and the discriminants F ₁ , F ₂ , F ₃ and F ₄ are less than 0, discard the entire small block B ₁ ; and set B Assign the relevant parameters in ₂ to B ₁ , then set the parameters of B ₂ to be empty, take another small block as B ₂ in sequence, and repeat the steps until all the small blocks are processed.

作为优选的一种方案：在所述(5)中，对于最后一个小块，如果判别式F₁、F₂、F₃和F₄大于0，可以默认为果实直接保留，否则认为是背景，直接丢弃。As a preferred solution: in the above (5), for the last small block, if the discriminant formulas F ₁ , F ₂ , F ₃ and F ₄ are greater than 0, the fruit can be retained by default, otherwise it is considered as the background, Just discard.

本发明的技术构思为：由于果实在颜色空间具有聚类性，本方案借鉴平均距离分类器原理对自然场景下的果蔬图像进行分类识别。The technical idea of the present invention is: since the fruit has clustering property in the color space, this scheme uses the principle of the average distance classifier to classify and recognize the fruit and vegetable images in the natural scene.

平均距离分类器通过分析不同的果蔬图像数字特征，把图像数据归类。分类算法包括训练阶段和测试阶段。在训练阶段，区分特定图像的特征属性，基于这些属性，产生描述特定类的唯一方法，即产生训练集，构造判别式。一般根据特征属性m和n，利用最小距离分类器或者平均距离分类器或者其他分类器可产生判别式：F(m，n)＝am+bn+c。a，b，c为任意常数，只要满足F(m，n)＝0即可。在测试阶段，利用这些特征空间，即这些判别式，每个判别式可将图像分成两个部分，达到识别图像的目的。The average distance classifier classifies the image data by analyzing the digital features of different fruit and vegetable images. Classification algorithms include a training phase and a testing phase. In the training phase, the characteristic attributes of specific images are distinguished, and based on these attributes, a unique method to describe a specific class is generated, that is, a training set is generated and a discriminant is constructed. Generally, according to the feature attributes m and n, the minimum distance classifier or the average distance classifier or other classifiers can be used to generate a discriminant formula: F(m, n)=am+bn+c. a, b, c are arbitrary constants, as long as F(m, n)=0 is satisfied. In the test phase, using these feature spaces, that is, these discriminants, each discriminant can divide the image into two parts to achieve the purpose of image recognition.

针对果实目标、叶子和枝干在每个颜色空间设计分类器，为每个颜色空间分别构造3个判别式实现分离：F_(fruit/leaf)，简写为F₁，用来分离果实和叶子；F_{(fruit/branch)}，简写为F₂，用来分离果实和枝干；F_{(leaf /branch)}，简写为F₃，用来分离叶子和枝干。由于叶子和枝干被视作背景，故只需构造F₁和F₂两个判别式。Design classifiers in each color space for fruit targets, leaves and branches, and construct three discriminants for each color space to achieve separation: F _(fruit/leaf) , abbreviated as F ₁ , is used to separate fruits and leaves; F _{(fruit/branch)} , abbreviated as F ₂ , is used to separate fruit and branches; F _{(leaf /branch)} , abbreviated as F ₃ , is used to separate leaves and branches. Since the leaves and branches are regarded as the background, only two discriminant formulas, F ₁ and F ₂ , need to be constructed.

融合2r-g-b颜色模型和LCD(luminance and color difference)颜色模型的特点，得到一种新的颜色模型LNM(LCD color model combinedwith Normalized-RGB color model)。该模型避免了LCD颜色模型对光的敏感度，同时避免了2r-g-b颜色模型无法识别红色分量值弱的果实的弊端，LNM与RGB颜色模型的对应关系如下式所示：Combining the characteristics of the 2r-g-b color model and the LCD (luminance and color difference) color model, a new color model LNM (LCD color model combined with Normalized-RGB color model) is obtained. This model avoids the sensitivity of the LCD color model to light, and at the same time avoids the disadvantage that the 2r-g-b color model cannot identify fruits with weak red component values. The corresponding relationship between LNM and RGB color models is shown in the following formula:

$\{\begin{matrix} Y Y = = 0.299 0.299 R R + + 0.587 0.587 G G + + 0.114 0.114 B B \\ {C C}_{r r} = = R R - - Y Y \\ 22 r r = = 22 R R / / ((R R + + G G + + B B)) \\ g g = = G G / / ((R R + + G G + + B B)) \end{matrix}$

因为融合了两种颜色空间，在实现识别的过程中，我们共构建4个判别式。Because of the fusion of two color spaces, we constructed four discriminants in the process of recognition.

同时，采用灰度共生矩阵提取图像纹理特征。首先给出共生矩阵的概念：设图像的一个区域大小为N_c×N_r像素，并设灰度级为G＝0，1，……，N_q-1。那么共生矩阵P_(d，q)是一个大小为N_q×N_q的方阵，包括了所有间距为d，方向为q，且灰度级为a和b的像素对出现的频率。P_(d，q)中的元素可表示为P_{(a，b|d，q)}。在区域内任选两个像素(k，l)和(m，n)，其中k，m＝1，2，……，N_c；n＝1，2，……，N_r。则：At the same time, the gray level co-occurrence matrix is used to extract the image texture features. Firstly, the concept of co-occurrence matrix is given: the size of a region of the image is N _c ×N _r pixels, and the gray level is G=0, 1, ..., N _q -1. Then the co-occurrence matrix P _{(d, q)} is a square matrix with a size of N _q ×N _q , including the frequency of occurrence of all pixel pairs with a distance of d, a direction of q, and gray levels of a and b. The elements in P _{(d, q)} can be expressed as P _{(a, b|d, q)} . Two pixels (k, l) and (m, n) are randomly selected in the region, where k, m=1, 2, ..., N _c ; n = 1, 2, ..., N _r . but:

P(a，b|d，q)＝∑[(k，l)，(m，n)]P(a,b|d,q)=∑[(k,l),(m,n)]

其中，如果where, if

|k-m|＝q，|l-n|＝d，g(k，l)＝a，g(m，n)＝b|k-m|=q, |l-n|=d, g(k,l)=a, g(m,n)=b

成立，那么established, then

[(k，l)，(m，n)]＝1[(k,l),(m,n)]=1

否则otherwise

[(k，l)，(m，n)]＝0[(k,l),(m,n)]=0

其中函数g(k，l)代表(k，l)处像素的灰度级。另外，方向q可以取值为0°、45°、90°、135°。所以对于一个给定的距离d，可以对应四个灰度共生矩阵：where the function g(k, l) represents the gray level of the pixel at (k, l). In addition, the direction q may take values of 0°, 45°, 90°, and 135°. So for a given distance d, it can correspond to four gray-level co-occurrence matrices:

p(a，b|d，0°)＝∑[(k，l)，(m，n)]，k-m＝0，|l-n|＝d，(k，l)＝a，(m，n)＝bp(a, b|d, 0°)=∑[(k, l), (m, n)], k-m=0, |l-n|=d, (k, l)=a, (m, n) =b

p(a，b|d，45°)＝∑[(k，l)，(m，n)]，k-m＝d，l-n＝-d，(k，l)＝a，(m，n)＝bp(a, b|d, 45°)=∑[(k, l), (m, n)], k-m=d, l-n=-d, (k, l)=a, (m, n)= b

或k-m＝-d，l-n＝d，(k，l)＝a，(m，n)＝bOr k-m=-d, l-n=d, (k,l)=a, (m,n)=b

p(a，b|d，90°)＝∑[(k，l)，(m，n)]，|k-m|＝d，l-n＝0，(k，l)＝a，(m，n)＝bp(a,b|d,90°)=∑[(k,l),(m,n)],|k-m|=d,l-n=0,(k,l)=a,(m,n) =b

p(a，b|d，135°)＝∑[(k，l)，(m，n)]，k-m＝d，l-n＝d，(k，l)＝a，(m，n)＝bp(a,b|d,135°)=∑[(k,l),(m,n)], k-m=d, l-n=d, (k,l)=a, (m,n)=b

分别采样多种果蔬的果实和叶子，采用灰度共生矩阵提取图像特征，得到可以区分果实和叶子的纹理特征值熵和能量，其表达式分别为(5)、(6)：The fruits and leaves of various fruits and vegetables are sampled separately, and the image features are extracted by using the gray level co-occurrence matrix, and the texture feature value entropy and energy that can distinguish the fruits and leaves are obtained. The expressions are (5), (6):

其中，a，b分别表示像素的灰度级，p(a，b)表示灰度共生矩阵。Among them, a, b represent the gray level of the pixel respectively, and p(a, b) represents the gray level co-occurrence matrix.

然后利用基于子块的区域生长法较好地解决了利用单一颜色参数出现欠分割的缺点。Then, the sub-block-based region growing method is used to solve the shortcoming of under-segmentation when using a single color parameter.

本发明的有益效果主要表现在：能够有效识别类球形果蔬、计算简单、识别精度高。The beneficial effects of the present invention are mainly manifested in that the spherical fruit and vegetable can be effectively identified, the calculation is simple, and the identification accuracy is high.

(四)附图说明(4) Description of drawings

图1是本发明一个流程示意图；Fig. 1 is a schematic flow chart of the present invention;

图2是分类器示意图；Fig. 2 is a schematic diagram of a classifier;

图3是2r-g-b分类器示意图；Figure 3 is a schematic diagram of a 2r-g-b classifier;

图4是LCD分类器示意图；Fig. 4 is a schematic diagram of an LCD classifier;

图5是果实和背景的属性分布图；Fig. 5 is the attribute distribution figure of fruit and background;

图6是根据判别式得出的分割直线图。Fig. 6 is a segmentation line diagram obtained according to the discriminant formula.

(五)具体实施方式(5) Specific implementation methods

下面结合附图对本发明作进一步描述。The present invention will be further described below in conjunction with the accompanying drawings.

参照图1～图6，一种类球形果蔬的识别方法，该方法包括以下步骤：(1)、获取自然场景下的果蔬图像；(2)、将获取的图像同时变换到2r-g-b颜色模型和LCD颜色模型，对2r-g-b颜色模型建立2r-g二维坐标系，LCD颜色模型建立Y-C_r二维坐标系；(3)、根据分类器原理，对Y-C_r二维坐标系下的特征属性Y，C_r分别构造判别式F₁、F₂，设定特征属性值Y和C_r的果实目标的平均向量m_fruit、树叶的平均向量m_leaf、枝干的平均向量m_branch，其算式为：(1)、(2)：With reference to Fig. 1～Fig. 6, a kind of identification method of spherical fruit and vegetable, this method comprises the following steps: (1), obtain the fruit and vegetable image under the natural scene; (2), transform the obtained image into 2r-gb color model and LCD color model, establish a 2r-g two-dimensional coordinate system for the 2r-gb color model, and establish a YC _r two-dimensional coordinate system for the LCD color model; (3), according to the principle of the classifier, the characteristic attributes under the YC _r two-dimensional coordinate system Y and C _r respectively construct the discriminant formulas F ₁ and F ₂ , and set the average vector m _fruit of the fruit target, the average vector m _leaf of the leaves, and the average vector m _{branch of the branches} of the characteristic attribute values Y and C _r . The formula is : (1), (2):

对2r-g二维坐标系的特征属性2r和g分别构造判别式F₃、F₄，设定特征属性值2r和g的果实目标的平均向量m′_fruit、树叶的平均向量m′_leaf、枝干的平均向量m′_branch，其算式为：(3)、(4)：Construct discriminant formulas F ₃ and F ₄ for the characteristic attributes 2r and g of the 2r-g two-dimensional coordinate system respectively, and set the average vector m′ _fruit of the fruit target and the average vector m′ _leaf of the leaves of the characteristic attribute values 2r and g The average vector m′ _branch of the branch, its calculation formula is: (3), (4):

F₃＝[2r，g]^T(m′_fruit-m′_leaf)-1/2[(m′_fruit ^Tm′f_ruit)-(m′_leaf ^Tm′_leaf)] (3)F ₃ =[2r, g] ^T (m' _fruit -m' _leaf )-1/2[(m' _fruit ^T m'f _fruit )-(m' _leaf ^T m' _leaf )] (3)

F₄＝[2r，g]T(m′_fruit-m′_branch)-1/2[(m′_fruit ^Tm′_fruit)-(m′_branch ^Tm′_lbranch)] (4)F ₄ =[2r, g]T(m' _fruit -m' _branch )-1/2[(m' _fruit ^T m' _fruit )-(m' _branch ^T m' _lbranch )] (4)

在所述(5)中，对于最后一个小块，如果判别式F₁、F₂、F₃和F₄大于0，可以默认为果实直接保留，否则认为是背景，直接丢弃。In (5), for the last small block, if the discriminant formulas F ₁ , F ₂ , F _{3 ,} and F ₄ are greater than 0, it can be regarded as the fruit by default and kept directly, otherwise it is regarded as the background and discarded directly.

本实施例中的具体步骤为：Concrete steps in the present embodiment are:

(1)、通过数码相机或者摄像头获取自然场景下的果蔬图像。(1) Obtain images of fruits and vegetables in natural scenes through a digital camera or video camera.

(2)、将获取的图像同时变换到2r-g-b颜色模型和LCD颜色模型。(2) Transform the acquired image to the 2r-g-b color model and the LCD color model simultaneously.

(3)、根据分类器原理，对2r-g-b模型的2r-g二维坐标系下的特征属性2r，g和LCD的Y-C_r二维坐标系下的特征属性Y，C_r分别构造判别式，产生4个判别式F₁，F₂，F₃，F₄。同时，把输入图像分成大小相等的小块，每块大小为L×L，L为奇数。(3) According to the principle of the classifier, construct discriminant formulas for the characteristic attributes 2r, g under the 2r-g two-dimensional coordinate system of the 2r-gb model and the characteristic attributes Y and C _r under the YC _r two-dimensional coordinate system of the LCD , produce four discriminant formulas F ₁ , F ₂ , F ₃ , F ₄ . At the same time, the input image is divided into small blocks of equal size, each block size is L×L, and L is an odd number.

举例说明：for example:

假设每个训练类由一个平均向量表示：Assume each training class is represented by a mean vector:

$m_{j} = 1 / N_{j} \underset{x &Element; w_{j}}{Σ} x$ for j＝1，2，…，M $m_{j} = 1 / N_{j} \underset{x &Element; w_{j}}{Σ} x$ for j=1, 2, ..., M

这里N_i是类W_j的训练模式向量的数量。假设有基于特征属性值Y和C_r的果实目标、树叶和枝干，假定将特征属性描述在二维特征空间中，如图5所示。并且可以得到平均值为：m_fruit＝[94.14，114.26]^T，m_leaf＝[77.02，218.45]^T，m_branch＝[52.88，33.67]^T，如图5中用#表示。Here N _i is the number of training pattern vectors for class W _j . Suppose there are fruit targets, leaves and branches based on the characteristic attribute values Y and _Cr , and the characteristic attributes are assumed to be described in a two-dimensional feature space, as shown in Figure 5. And the average value can be obtained as: m _fruit =[94.14, 114.26] ^T , m _leaf =[77.02, 218.45] ^T , m _branch =[52.88, 33.67] ^T , represented by # in Fig. 5 .

基于这一点，可以通过判定最接近m_j的原型，分配任意给定的模式x给类。如采用欧几里德距离作为测量相似性距离，原型距离为：Based on this, any given schema x can be assigned to a class by determining the prototype closest to _mj . If Euclidean distance is used as the measurement similarity distance, the prototype distance is:

F_j(x)＝‖x-m_j‖ for j＝1，2，…，MF _j (x) = ‖ xm _j ‖ for j = 1, 2, ..., M

这相当于：This is equivalent to:

F_j(x)＝x^Tm_j-1/2(m_j ^Tm_j) for j＝1，2，…，MF _j (x)=x ^T m _j -1/2(m _j ^T m _j ) for j=1, 2,..., M

在本例中，简单假设每个得到的fruit的平均值为[a，b]，leaf的平均值为[c，d]，branch的平均值为[e，f]。那么可以得到In this example, simply assume that the average value of each obtained fruit is [a, b], the average value of leaf is [c, d], and the average value of branch is [e, f]. then you can get

$\{\begin{matrix} {F f}_{fruit fruit} = = aY aY + + bCr bCr - - 11 / / 22 (({a a}^{22} + + {b b}^{22})) \\ {F f}_{leaf leaf} = = cY cY + + dCr dCr - - 11 / / 22 (({c c}^{22} + + {d d}^{22})) \\ {F f}_{branch branch} = = eY eY + + fCr f - - 11 / / 22 (({e e}^{22} + + {f f}^{22})) \end{matrix}$

根据这个方法，可以计算得到以下决策函数：According to this method, the following decision function can be calculated:

$\{\begin{matrix} {F f}_{fruit fruit} = = 94.14 94.14 Y Y + + 114.26 114.26 Cr Cr - - 10958.84 10958.84 \\ {F f}_{leaf leaf} = = 77.02 77.02 Y Y + + 218.45 218.45 Cr Cr - - 26826.24 26826.24 \\ {F f}_{branch branch} = = 52.88 52.88 Y Y + + 33.67 33.67 Cr Cr - - 1964.98 1964.98 \end{matrix}$

最后，根据这些决策函数，可得分离类w_i和w_j的判别式满足：Finally, according to these decision functions, the discriminant for separating classes w _i and w _j satisfies:

F_i(x)-F_j(x)＝0F _i (x) - F _j (x) = 0

即：Right now:

$\{\begin{matrix} {F f}_{fruit fruit / / leaf leaf} = = ((a a - - c c)) Y Y + + ((b b - - d d)) Cr Cr + + 11 / / 22 (({a a}^{22} + + {b b}^{22} - - {c c}^{22} - - {d d}^{22})) = = 00 \\ {F f}_{fruit fruit / / branch branch} = = ((a a - - e e)) Y Y + + ((b b - - f f)) Cr Cr + + 11 / / 22 (({a a}^{22} + + {b b}^{22} - - {e e}^{22} - - {f f}^{22})) = = 00 \\ {F f}_{leaf leaf / / branch branch} = = ((c c - - e e)) Y Y + + ((d d - - f f)) Cr Cr + + 11 / / 22 (({c c}^{22} + + {d d}^{22} - - {e e}^{22} - - {f f}^{22})) = = 00 \end{matrix}$

在这个例中，得到判别式F₁，F₂，F₃如下：In this example, the discriminant formulas F ₁ , F ₂ , and F ₃ are obtained as follows:

$\{\begin{matrix} {F f}_{fruit fruit / / leaf leaf} = = 17.12 17.12 Y Y - - 104.19 104.19 Cr Cr + + 15867.4 15867.4 = = 00 \\ {F f}_{fruit fruit / / branch branch} = = 41.26 41.26 Y Y + + 80.59 80.59 Cr Cr + + 8993.86 8993.86 = = 00 \\ {F f}_{leaf leaf / / branch branch} = = 24.14 24.14 Y Y + + 184.78 184.78 Cr Cr + + 24861.26 24861.26 = = 00 \end{matrix}$

根据这些判别式，可以得出分离直线，省去叶子和枝干的分离直线，如图6所示。According to these discriminants, the separation straight line can be obtained, omitting the separation straight line of leaves and branches, as shown in Figure 6.

如果采用的原型距离不一样得到的判别式也是不一样的，如可以采用标准化的欧式距离和马哈朗诺必斯距离等。If different prototype distances are used, the obtained discriminants are also different. For example, standardized Euclidean distance and Mahalanobis distance can be used.

(4)、顺序选择两个小块B₁和B₂，分别计算其4个方向的灰度共生矩阵和各个灰度共生矩阵的两个特征值：：熵ENT和能量ASM，其算式分别为(5)、(6)：(4) Select two small blocks B ₁ and B ₂ in sequence, respectively calculate the gray level co-occurrence matrix in four directions and the two eigenvalues of each gray level co-occurrence matrix: entropy ENT and energy ASM, the calculation formulas are respectively (5), (6):

其中，a，b分别表示像素的灰度级，p(a，b)表示灰度共生矩阵；Among them, a, b represent the gray level of the pixel respectively, and p(a, b) represents the gray level co-occurrence matrix;

(5)、平均4个方向的灰度共生矩阵的特征值得到平均特征值 ENT和 ASM作为判别的纹理特征。(5), average the eigenvalues of the gray level co-occurrence matrix in 4 directions to obtain the average eigenvalue ENT and ASM as discriminative texture features.

(6)、对于相邻的两个小块B₁和B₂，根据颜色模型构造的判别式F₁，F₂，F₃，F₄和纹理特征值进行判断。如果判别式都大于0或者B₁和B₂的 ENT和 ASM之差小于设定的阈值T，则保留B₁，将其压入一个堆栈。同时将B₂中相关参数赋给B₁，然后将B₂的参数置为空。顺序往后取另一个小块作为B₂，重复上述步骤。如果纹理差值大于阈值T并且判别式F₁，F₂，F₃，F₄都小于0，则丢弃整个小块B₁。直到所有的小块处理完毕。(6) For two adjacent small blocks B ₁ and B ₂ , judge according to the discriminant formulas F ₁ , F ₂ , F ₃ , F ₄ constructed by the color model and the texture feature values. If the discriminants are both greater than 0 or the difference between ENT and ASM of B ₁ and B ₂ is smaller than the set threshold T, keep B ₁ and push it into a stack. At the same time, assign the relevant parameters in B ₂ to B ₁ , and then set the parameters of B ₂ to empty. Sequentially take another small block as B ₂ and repeat the above steps. If the texture difference is greater than the threshold T and the discriminants F ₁ , F ₂ , F ₃ , and F ₄ are all less than 0, then discard the entire small block B ₁ . until all small blocks are processed.

Claims

1, the recognition methods of a kind spheral fruit, this method may further comprise the steps:

(1), obtains fruits and vegetables image under the natural scene;

(2), the image that obtains is transformed to 2r-g-b color model and LCD color model simultaneously, the 2r-g-b color model is set up the 2r-g two-dimensional coordinate system, the LCD color model is set up Y-C _rTwo-dimensional coordinate system;

(3), according to the sorter principle, to Y-C _rCharacteristic attribute Y under the two-dimensional coordinate system, C _rConstruct discriminant F respectively ₁, F ₂, set characteristic attribute value Y and C _rThe average vector m of fruit target _Fruit, leaf average vector m _Leaf, limb average vector m _Branch, its formula is: (1), (2):

F ₁＝[Y，C _r] ^T(m _fruit-m _leaf)-1/2[(m _fruit ^Tm _fruit)-(m _leaf ^Tm _leaf)] (1)

F ₂＝[Y，C _r] ^T(m _fruit-m _branch)-1/2[(m _fruit ^Tm _fruit)-(m _branch ^Tm _lbranch)] (2)

Characteristic attribute 2r and g to the 2r-g two-dimensional coordinate system construct discriminant F respectively ₃, F ₄, the average vector m ' of the fruit target of setting characteristic attribute value 2r and g _Fruit, leaf average vector m ' _Leaf, limb average vector m ' _Branch, its formula is: (3), (4):

F ₃＝[2r，g] ^T(m′ _fruit-m′ _leaf)-1/2[(m′ _fruit ^Tm′ _fruit)-(m′ _leaf ^Tm′ _leaf)] (3)

F ₄＝[2r，g] ^T(m′ _fruit-m′ _branch)-1/2[(m′ _fruit ^Tm′ _fruit)-m′ _branch ^Tm′ _lbranch)] (4)

Obtain saving the straight line that separates of leaf and limb according to discriminant, and input picture is divided into equal-sized fritter, every block size is L * L, and L is an odd number;

(4), two fritter B of select progressively ₁And B ₂, calculate the gray level co-occurrence matrixes of its 4 directions respectively, an area size of establishing image is N _c* N _rPixel, and to establish gray level be G=0,1 ..., N _q-1, co-occurrence matrix P _{(d, q)}Be that a size is N _q* N _qSquare formation, comprise that all spacings are d, direction is q, and gray level is the frequency of pixel to occurring of a and b, P _{(d, q)}In element representation be P _{(a, b|d, q)}, optional two pixels in the zone (k, l) with (m, n), k wherein, m=1,2 ..., N _cN=1,2 ..., N _r

Calculate two eigenwerts with each gray level co-occurrence matrixes: entropy ENT and energy ASM, its formula are respectively (5), (6):

ENT = - Σ_{i = 0}^{N_{q} - 1} Σ_{j = 0}^{N_{q} - 1} P (a, b) \log P (a, b) . . . (5)

ASM = {Σ_{i = 0}^{N_{q} - 1} Σ_{j = 0}^{N_{q} - 1} (P (a, b))}^{2} . . . (6)

Wherein, a, b is the gray level of remarked pixel respectively, p (a, b) expression gray level co-occurrence matrixes; And the eigenwert of calculating the gray level co-occurrence matrixes of 4 directions obtains mean eigenvalue ENT and ASM;

(5), for two adjacent fritter B ₁And B ₂, as discriminant F ₁, F ₂, F ₃And F ₄Greater than 0 or B ₁And B ₂ENT and the difference of ASM less than preset threshold T, then keep B ₁, this fritter is confirmed as fruit; As the texture difference greater than threshold value T and discriminant F ₁, F ₂, F ₃And F ₄Less than 0, then abandon whole fritter B ₁And with B ₂Middle correlation parameter is composed to B ₁, then with B ₂Parameter be changed to sky, the order get another fritter as B backward ₂, repeating said steps disposes up to all fritters.

2, the recognition methods of a kind spheral fruit as claimed in claim 1 is characterized in that: in described (5), for last fritter, if discriminant F ₁, F ₂, F ₃And F ₄Greater than 0, can be defaulted as fruit and directly keep, otherwise think background, directly abandon.