CN117197062B - A method and system for measuring leaf nitrogen content based on RGB images - Google Patents

Abstract

The present invention provides a leaf nitrogen content measurement method and system based on RGB images, which belongs to the field of image processing technology, including: taking crop canopy photos with a camera; producing image data sets for white balance correction and exposure correction based on the taken photos; constructing a convolutional neural network for image white balance and exposure correction, and performing correction tests on the photos taken by the camera; training a neural network model for leaf nitrogen content estimation based on the corrected crop canopy images, and applying it to crops of different shooting angles, different shooting equipment and different varieties. Compared with traditional methods, the present invention has low implementation cost, simple operation, no special requirements for users and equipment, can complete measurement tasks under various weather conditions, and has strong universality.

Description

A method and system for measuring leaf nitrogen content based on RGB images

Technical Field

The present invention relates to the technical field of image processing, and in particular to a method and system for measuring the nitrogen content of leaves based on RGB images.

Background Art

Nitrogen is one of the indispensable nutrients in the growth and development of crops, and has a very important influence on cell growth, division and new cell formation. The application of nitrogen fertilizer can effectively increase crop yields, but excessive nitrogen fertilizer application can also cause serious environmental pollution and even reduce production. Therefore, accurate estimation of crop nitrogen status is very necessary for nitrogen management in precision agriculture.

Traditional destructive sampling methods collect plant leaves in the field and then perform chemical analysis in the laboratory, which is time-consuming and laborious. Among non-destructive methods, single photon avalanche diodes (SPADs) have a very small measurement range and are inefficient in practical applications; equipment such as hyperspectral/multispectral instruments are usually very expensive, which also limits their application in agricultural production. The popularity of smartphones has made it easy to obtain images of field crops. Existing studies have shown that computer vision methods can be used to estimate the nitrogen status of crops. However, current research on monitoring crop nitrogen using camera devices such as smartphones is still insufficient, and most of them ignore the impact of natural light or incorrect camera settings during shooting on crop nitrogen estimation results. Although the characteristics of the crop have not changed, different digital cameras (or the same camera with different settings) will produce inconsistent colors when acquiring images; the colors of images taken at different times using the same camera may also be different.

Therefore, if a single computer vision technology is used, it will be difficult to adapt to the nitrogen estimation task in complex field environments.

Summary of the invention

The present invention provides a leaf nitrogen content measurement method and system based on RGB images, which are used to solve the defect that the single visual measurement technology used in the prior art cannot accurately measure the leaf nitrogen content.

In a first aspect, the present invention provides a method for measuring leaf nitrogen content based on RGB images, comprising:

Collect photos of crop canopy samples;

Adjusting the white balance mode of the crop canopy sample photo to obtain a white balance correction data set, and constructing an image white balance correction neural network based on the white balance correction data set;

Adjusting the exposure mode of the crop canopy sample photo to obtain an exposure correction data set, and constructing an image exposure correction neural network based on the correction data set;

Correcting the crop canopy sample photo by integrating the image white balance correction neural network and the image exposure correction neural network to obtain a corrected data set;

Constructing an initial neural network model for measuring leaf nitrogen content, obtaining leaf nitrogen labels, and training the initial neural network model for measuring leaf nitrogen content based on the corrected data set and the leaf nitrogen labels to obtain a leaf nitrogen content measurement model;

The canopy photo of the crop to be measured is input into the leaf nitrogen content measurement model to obtain the leaf nitrogen content measurement result.

According to a leaf nitrogen content measurement method based on RGB images provided by the present invention, a crop canopy sample photo is collected, comprising:

Use a smartphone camera to photograph crop canopies at any time during the day, collect photos from any three different preset angles, and save the photos in a preset file format;

The white balance setting and exposure setting of the photographed photo are adjusted using image processing software to obtain a true value image.

According to a leaf nitrogen content measurement method based on RGB images provided by the present invention, the white balance mode of the crop canopy sample photo is adjusted to obtain a white balance correction data set, and an image white balance correction neural network is constructed from the white balance correction data set, including:

Adjusting the white balance mode of the true value image to obtain a plurality of color temperature deviation images including different color temperatures, and combining the plurality of color temperature deviation images and the true value image to form the white balance correction data set;

Based on the white balance correction data set, a white balance correction full convolutional neural network is constructed, and the L1 loss function and the Adam optimizer are used to train and converge the white balance correction full convolutional neural network to obtain the image white balance correction neural network.

According to a leaf nitrogen content measurement method based on RGB images provided by the present invention, the exposure mode of the crop canopy sample photo is adjusted to obtain an exposure correction data set, and an image exposure correction neural network is constructed based on the correction data set, including:

Adjusting the exposure mode of the true value image to obtain a plurality of exposure deviation images including different exposure values, and forming the exposure correction data set with the plurality of exposure deviation images and the true value image;

Based on the exposure correction data set, an exposure correction full convolutional neural network is constructed, and the exposure correction full convolutional neural network is trained and converged using an L1 loss function and an Adam optimizer to obtain the image exposure correction neural network.

According to a leaf nitrogen content measurement method based on RGB images provided by the present invention, the image white balance correction neural network and the image exposure correction neural network are integrated to correct the crop canopy sample photo to obtain a corrected data set, including:

Inputting the crop canopy sample photo into the image white balance correction neural network to obtain a white balance corrected data set;

The crop canopy sample photos are input into the image exposure correction neural network to obtain an exposure-corrected data set.

According to a leaf nitrogen content measurement method based on RGB images provided by the present invention, an initial model of a leaf nitrogen content measurement neural network is constructed, leaf nitrogen labels are obtained, and the initial model of the leaf nitrogen content measurement neural network is trained based on the corrected data set and the leaf nitrogen labels. Before obtaining the leaf nitrogen content measurement model, the method further includes:

Cropping the color card part in the corrected data set to obtain a cropped data set that only includes crop canopy information images;

Data enhancement is performed on the cropped data set to obtain an enhanced data set.

According to a leaf nitrogen content measurement method based on RGB images provided by the present invention, an initial model of a leaf nitrogen content measurement neural network is constructed, a leaf nitrogen label is obtained, and the initial model of the leaf nitrogen content measurement neural network is trained based on the corrected data set and the leaf nitrogen label to obtain a leaf nitrogen content measurement model, including:

A preset convolutional neural network is used to construct an initial neural network model for measuring leaf nitrogen content;

Obtaining the leaf nitrogen label by experimentally measuring the crop canopy sample photo;

The mean square error (MSE) loss function and the Adam optimizer are used to train and converge the initial model of the leaf nitrogen content measurement neural network to obtain the leaf nitrogen content measurement model.

A leaf nitrogen content measurement method based on RGB images provided by the present invention also includes:

Inputting crop canopy images of different shooting angles, different shooting equipment and different crop varieties into the leaf nitrogen content measurement model to obtain measurement results of different leaf nitrogen contents;

The nitrogen content measurement results of the different leaves are evaluated and compared using different preset accuracy evaluation indicators.

In a second aspect, the present invention further provides a leaf nitrogen content measurement system based on RGB images, comprising:

A collection module for collecting sample photos of crop canopies;

A white balance data processing module, used for adjusting the white balance mode of the crop canopy sample photo to obtain a white balance correction data set, and constructing an image white balance correction neural network based on the white balance correction data set;

An exposure data processing module, used for adjusting the exposure mode of the crop canopy sample photo to obtain an exposure correction data set, and constructing an image exposure correction neural network based on the correction data set;

A correction module, used to correct the crop canopy sample photo by integrating the image white balance correction neural network and the image exposure correction neural network to obtain a corrected data set;

A training module is used to construct an initial model of a neural network for measuring leaf nitrogen content, obtain leaf nitrogen labels, and train the initial model of the neural network for measuring leaf nitrogen content based on the corrected data set and the leaf nitrogen labels to obtain a leaf nitrogen content measurement model;

The measurement module is used to input the canopy photo of the crop to be measured into the leaf nitrogen content measurement model to obtain the leaf nitrogen content measurement result.

In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a leaf nitrogen content measurement method based on RGB images as described above is implemented.

In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for measuring leaf nitrogen content based on RGB images.

The leaf nitrogen content measurement method and system based on RGB images provided by the present invention can well complete the white balance and exposure correction of camera photos by adopting a fully convolutional neural network with an attention mechanism, effectively reducing the impact of complex light changes. Based on the images after white balance and exposure correction, a convolutional neural network for estimating the nitrogen content of crop leaves is trained to improve the accuracy of crop nitrogen estimation. A series of computer vision algorithms based on this architecture solve various image quality problems caused by light changes in complex outdoor environments and user shooting setting errors, and successfully complete the task of estimating the nitrogen content of crop leaves.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic diagram of a flow chart of a method for measuring nitrogen content in leaves based on RGB images provided by the present invention;

FIG2 is a second flow chart of the method for measuring nitrogen content in leaves based on RGB images provided by the present invention;

FIG3 is an example diagram of a data set for white balance correction provided by the present invention;

FIG4 is an example diagram of a data set for exposure correction provided by the present invention;

FIG5 is a diagram showing the correction effect of the white balance correction neural network provided by the present invention on pictures with different color temperature settings;

FIG6 is a diagram showing the correction effect of the exposure correction neural network provided by the present invention on pictures with different EV value settings;

FIG7 is a comparison chart of the accuracy of different convolutional neural networks provided by the present invention in estimating leaf nitrogen content using RGB images;

FIG8 is a schematic diagram of the structure of a leaf nitrogen content measurement system based on RGB images provided by the present invention;

FIG. 9 is a schematic diagram of the structure of an electronic device provided by the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

FIG. 1 is a flow chart of a method for measuring nitrogen content in leaves based on RGB images provided by an embodiment of the present invention. As shown in FIG. 1 , the method comprises:

Step 100: Collecting crop canopy sample photos;

Step 200: adjusting the white balance mode of the crop canopy sample photo to obtain a white balance correction data set, and constructing an image white balance correction neural network based on the white balance correction data set;

Step 300: adjusting the exposure mode of the crop canopy sample photo to obtain an exposure correction data set, and constructing an image exposure correction neural network based on the correction data set;

Step 400: Correcting the crop canopy sample photo by integrating the image white balance correction neural network and the image exposure correction neural network to obtain a corrected data set;

Step 500: constructing an initial neural network model for measuring leaf nitrogen content, obtaining leaf nitrogen labels, and training the initial neural network model for measuring leaf nitrogen content based on the corrected data set and the leaf nitrogen labels to obtain a leaf nitrogen content measurement model;

Step 600: Input the crop canopy photo to be measured into the leaf nitrogen content measurement model to obtain the leaf nitrogen content measurement result.

The embodiment of the present invention first uses a camera to take a canopy photo of a crop, and then renders the obtained canopy photo to obtain images in different white balance modes, and constructs an image white balance correction neural network. It also renders the obtained canopy photo to obtain images in different exposure modes, and constructs an image exposure correction neural network. The above-mentioned image white balance correction neural network and image exposure correction neural network are combined to correct the crop canopy photo taken by the camera. The corrected crop canopy image is used to train a neural network model for estimating leaf nitrogen content, and its application effect on crops at different shooting angles, different shooting equipment, and different varieties is verified.

Specifically, as shown in Figure 2, a camera is used to take a photo of the crop canopy to obtain a true image. For the true image, a white balance correction data set and an exposure correction data set are respectively produced. Then, a white balance correction neural network is constructed from the white balance correction data set, and an exposure correction neural network is constructed from the exposure correction data set. Image white balance correction and image exposure correction are performed respectively to obtain the corrected image. The leaf nitrogen content estimation neural network is trained with the obtained leaf nitrogen label to obtain a leaf nitrogen content measurement model. Finally, the effect is evaluated from three aspects: shooting angle, shooting mobile phone, and crop variety.

The present invention uses a fully convolutional neural network with an attention mechanism to effectively complete the white balance and exposure correction of camera photos, effectively reducing the impact of complex light changes. Based on the images after white balance and exposure correction, a convolutional neural network for estimating the nitrogen content of crop leaves is trained to improve the accuracy of crop nitrogen estimation. A series of computer vision algorithms based on this architecture solve various image quality problems caused by light changes in complex outdoor environments and user shooting setting errors, and successfully complete the task of estimating the nitrogen content of crop leaves.

Based on the above embodiment, collecting crop canopy sample photos includes:

Use a smartphone camera to photograph crop canopies at any time during the day, collect photos from any three different preset angles, and save the photos in a preset file format;

The white balance setting and exposure setting of the photographed photo are adjusted using image processing software to obtain a true value image.

Specifically, the embodiment of the present invention uses a camera of a smart phone to take photos of crop canopies, and the photos can be taken at any time during the day. The crop canopy is the main subject of the photo, a color card is included in the screen, and photos are taken at three angles. The angles between the camera and the horizontal direction are approximately 0°, 30°, and 60°, and the final image is saved in RAW file format. The RAW file is opened in Photoshop software, and the white balance and exposure settings of the image are adjusted using the color card to obtain a true value image, that is, a photo with correct white balance and exposure settings.

Based on the above embodiment, the white balance mode of the crop canopy sample photo is adjusted to obtain a white balance correction data set, and an image white balance correction neural network is constructed from the white balance correction data set, including:

Adjusting the white balance mode of the true value image to obtain a plurality of color temperature deviation images including different color temperatures, and combining the plurality of color temperature deviation images and the true value image to form the white balance correction data set;

Based on the white balance correction data set, a white balance correction full convolutional neural network is constructed, and the L1 loss function and the Adam optimizer are used to train and converge the white balance correction full convolutional neural network to obtain the image white balance correction neural network.

Specifically, based on the true value image, different white balance settings are performed on the image through Photoshop, so that an image with incorrect white balance can be rendered. By adjusting the white balance mode of the image, pictures at different color temperatures can be obtained as a data set of incorrect white balance settings, which together with the true value pictures constitute a data set of white balance correction, that is, images with color temperatures of 2850K, 3800K, 5500K, 6500K and 7500K and images with correct white balance settings, as shown in Figure 3, where the color card part has been removed.

Based on the white balance correction dataset, a fully convolutional neural network for white balance correction is constructed. During the network training process, the loss function uses L1 loss, the training optimizer uses the Adam algorithm, the initial learning rate is 0.001, and the training process is 500 epochs.

Based on the above embodiment, the exposure mode of the crop canopy sample photo is adjusted to obtain an exposure correction data set, and an image exposure correction neural network is constructed from the correction data set, including:

Adjusting the exposure mode of the true value image to obtain a plurality of exposure deviation images including different exposure values, and forming the exposure correction data set with the plurality of exposure deviation images and the true value image;

Based on the exposure correction data set, an exposure correction full convolutional neural network is constructed, and the exposure correction full convolutional neural network is trained and converged using an L1 loss function and an Adam optimizer to obtain the image exposure correction neural network.

Specifically, based on the true value image, different exposure settings are set for the image through Photoshop, so that an image with incorrect exposure can be rendered. By adjusting the exposure values (EV) of the image, images under different exposure modes can be obtained as a data set of incorrect exposure settings. Together with the true value image, the exposure correction data set consists of images with EV values of -1.5, -1, +1, +1.5 and correct exposure settings, as shown in Figure 4, where the color card part has been removed.

Based on the exposure correction dataset, a fully convolutional neural network for exposure correction is constructed. During the network training process, the loss function uses L1 loss, the training optimizer uses the Adam algorithm, the initial learning rate is 0.001, and the training process is 500 epochs.

Based on the above embodiment, the image white balance correction neural network and the image exposure correction neural network are integrated to correct the crop canopy sample photos to obtain a corrected data set, including:

Inputting the crop canopy sample photo into the image white balance correction neural network to obtain a white balance corrected data set;

The crop canopy sample photos are input into the image exposure correction neural network to obtain an exposure-corrected data set.

Specifically, the canopy photos of crops taken by the camera are input into the constructed white balance correction neural network to obtain the image output with the correct white balance setting. As shown in Figure 5, after white balance correction, the color difference ΔE of the original image is significantly reduced. The color difference ΔE of the 2850K color temperature image is reduced from 13.63 to 1.64, the color difference ΔE of the 3800K image is reduced from 7.31 to 1.15, the color difference ΔE of the 5500K image is reduced from 2.00 to 0.90, the color difference ΔE of the 6500K image is reduced from 4.20 to 1.30, and the color difference ΔE of the 7500K image is reduced from 5.01 to 1.51. Overall, there is no obvious difference between the corrected image and the true value image with the correct white balance setting.

The canopy photos of crops taken by the camera are input into the constructed exposure correction neural network to obtain the image output with the correct exposure setting. As shown in Figure 6, after exposure correction, the Peak Signal-to-Noise Ratio (PSNR) of the original image is significantly improved. The PSNR of the image with an EV value of -1.5 increases from 11.75 to 27.19, the PSNR of the image with an EV value of -1.0 increases from 14.62 to 26.79, the PSNR of the image with an EV value of +1.0 increases from 14.53 to 24.86, and the PSNR of the image with an EV value of +1.5 increases from 11.25 to 27.17. Overall, there is no significant difference between the corrected image and the true value image with the correct exposure setting.

Based on the above embodiment, an initial model of a neural network for measuring leaf nitrogen content is constructed, a leaf nitrogen label is obtained, and the initial model of a neural network for measuring leaf nitrogen content is trained based on the corrected data set and the leaf nitrogen label. Before obtaining the leaf nitrogen content measurement model, the method further includes:

Cropping the color card part in the corrected data set to obtain a cropped data set that only includes crop canopy information images;

Data enhancement is performed on the cropped data set to obtain an enhanced data set.

Specifically, using the obtained crop canopy image dataset, the color card part in the image is first cropped out to obtain an image with only crop canopy information, and then data enhancement is performed, including random cropping, random rotation, and mirroring operations to expand the dataset.

Based on the above embodiment, an initial model of a neural network for measuring leaf nitrogen content is constructed, leaf nitrogen labels are obtained, and the initial model of a neural network for measuring leaf nitrogen content is trained based on the corrected data set and the leaf nitrogen labels to obtain a leaf nitrogen content measurement model, including:

A preset convolutional neural network is used to construct an initial neural network model for measuring leaf nitrogen content;

Obtaining the leaf nitrogen label by experimentally measuring the crop canopy sample photo;

The mean square error (MSE) loss function and the Adam optimizer are used to train and converge the initial model of the leaf nitrogen content measurement neural network to obtain the leaf nitrogen content measurement model.

Specifically, the model used in the embodiment of the present invention is a convolutional neural network, including but not limited to AlexNet, Inception_v3, DenseNet161, MobilenetV3, ResNet50 and ResNet50_CBAM (ResNet50 with attention mechanism). The input feature of the model is the RGB value of the image after data enhancement, and the label is the nitrogen content of the crop leaves measured by laboratory analysis. During the network training process, the loss function uses the mean square error (MSE) loss, the training optimizer uses the Adam algorithm, the initial learning rate is 0.001, and the training process is 500 epochs.

Based on the above embodiment, it also includes:

Inputting crop canopy images of different shooting angles, different shooting equipment and different crop varieties into the leaf nitrogen content measurement model to obtain measurement results of different leaf nitrogen contents;

The nitrogen content measurement results of the different leaves are evaluated and compared using different preset accuracy evaluation indicators.

Specifically, the trained neural network is used to estimate the nitrogen content of target crop leaves. The crop canopy images of different shooting angles, different shooting equipment and different varieties are input into the network, and the corresponding leaf nitrogen content is output. It is compared with the leaf nitrogen content observed in the laboratory, and the accuracy evaluation index R2, root mean square error (RMSE) and relative root mean square error (RRMSE) are calculated. The results are shown in Table 1 and Figure 7:

Table 1

Algorithms R2 RMSE RRMSE AlexNet 0.73 0.49 0.17 Inception_v3 0.81 0.41 0.14 DenseNet161 0.85 0.37 0.12 MobilenetV3 0.84 0.39 0.13 ResNet50 0.85 0.38 0.13 ResNet50_CBAM 0.82 0.40 0.14

As can be seen from Table 1 and Figure 7, the shallower network AlexNet (8 layers) performs poorly in estimating the nitrogen content of crop leaves, with an R2 of 0.73, RMSE of 0.49, and RRMSE of 0.17 on the test set; as the number of neural network layers increases, the estimation accuracy of leaf nitrogen content has been significantly improved, with R2 between 0.81 and 0.85, among which Inception_v3 (46 layers) has an R2 of 0.81, ResNet50 (50 layers) has an R2 of 0.85, and DenseNet161 (161 layers) has an R2 of 0.85. At this time, as the number of network layers continues to increase, the performance of the model no longer continues to improve. ResNet50_CBAM adds spatial and channel attention mechanisms to ResNet50, but does not improve the performance of the model, but rather decreases it a little. MobilenetV3 uses network structure search NAS, redesigns some time-consuming architectures, adopts shortcut connections and SE attention mechanisms, reduces model parameters, and achieves high accuracy (R2 is 0.84, RMSE is 0.39, and RRMSE is 0.13) while greatly improving the computational efficiency of the model.

The leaf nitrogen content measurement system based on RGB images provided by the present invention is described below. The leaf nitrogen content measurement system based on RGB images described below and the leaf nitrogen content measurement method based on RGB images described above can be referenced to each other.

FIG8 is a schematic diagram of the structure of a leaf nitrogen content measurement system based on RGB images provided by an embodiment of the present invention. As shown in FIG8 , the system comprises: an acquisition module 81, a white balance data processing module 82, an exposure data processing module 83, a correction module 84, a training module 85 and a measurement module 86, wherein:

The acquisition module 81 is used to acquire crop canopy sample photos; the white balance data processing module 82 is used to adjust the white balance mode of the crop canopy sample photos to obtain a white balance correction data set, and construct an image white balance correction neural network from the white balance correction data set; the exposure data processing module 83 is used to adjust the exposure mode of the crop canopy sample photos to obtain an exposure correction data set, and construct an image exposure correction neural network from the correction data set; the correction module 84 is used to correct the crop canopy sample photos by integrating the image white balance correction neural network and the image exposure correction neural network to obtain a corrected data set; the training module 85 is used to construct an initial model of a leaf nitrogen content measurement neural network, obtain a leaf nitrogen label, and train the initial model of the leaf nitrogen content measurement neural network based on the corrected data set and the leaf nitrogen label to obtain a leaf nitrogen content measurement model; the measurement module 86 is used to input the crop canopy photos to be measured into the leaf nitrogen content measurement model to obtain leaf nitrogen content measurement results.

Figure 9 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 9, the electronic device may include: a processor (processor) 910, a communication interface (Communications Interface) 920, a memory (memory) 930 and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call the logic instructions in the memory 930 to execute the leaf nitrogen content measurement method based on RGB images, which includes: collecting crop canopy sample photos; adjusting the white balance mode of the crop canopy sample photos to obtain a white balance correction data set, and constructing an image white balance correction neural network from the white balance correction data set; adjusting the exposure mode of the crop canopy sample photos to obtain an exposure correction data set, and constructing an image exposure correction neural network from the correction data set; correcting the crop canopy sample photos by combining the image white balance correction neural network and the image exposure correction neural network to obtain a corrected data set; constructing an initial model of the leaf nitrogen content measurement neural network, obtaining a leaf nitrogen label, and training the initial model of the leaf nitrogen content measurement neural network based on the corrected data set and the leaf nitrogen label to obtain a leaf nitrogen content measurement model; inputting the crop canopy photos to be measured into the leaf nitrogen content measurement model to obtain leaf nitrogen content measurement results.

In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the leaf nitrogen content measurement method based on RGB images provided by the above-mentioned methods, the method comprising: collecting crop canopy sample photos; adjusting the white balance mode of the crop canopy sample photos to obtain a white balance correction data set, and constructing an image white balance correction neural network from the white balance correction data set; adjusting the exposure mode of the crop canopy sample photos to obtain an exposure correction data set, and constructing an image exposure correction neural network from the correction data set; correcting the crop canopy sample photos by combining the image white balance correction neural network and the image exposure correction neural network to obtain a corrected data set; constructing an initial model of a leaf nitrogen content measurement neural network, obtaining a leaf nitrogen label, and training the initial model of the leaf nitrogen content measurement neural network based on the corrected data set and the leaf nitrogen label to obtain a leaf nitrogen content measurement model; inputting the crop canopy photos to be measured into the leaf nitrogen content measurement model to obtain leaf nitrogen content measurement results.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (7)

1. A method for measuring leaf nitrogen content based on RGB images, comprising: Collect photos of crop canopy samples; Adjusting the white balance mode of the crop canopy sample photo to obtain a white balance correction data set, and constructing an image white balance correction neural network based on the white balance correction data set; Adjusting the exposure mode of the crop canopy sample photo to obtain an exposure correction data set, and constructing an image exposure correction neural network based on the correction data set; Correcting the crop canopy sample photo by integrating the image white balance correction neural network and the image exposure correction neural network to obtain a corrected data set; Constructing an initial neural network model for measuring leaf nitrogen content, obtaining leaf nitrogen labels, and training the initial neural network model for measuring leaf nitrogen content based on the corrected data set and the leaf nitrogen labels to obtain a leaf nitrogen content measurement model; Inputting a photo of the crop canopy to be measured into the leaf nitrogen content measurement model to obtain a leaf nitrogen content measurement result; Collect sample photographs of crop canopies, including: Use a smartphone camera to photograph crop canopies at any time during the day, collect photos from any three different preset angles, and save the photos in a preset file format; Using image processing software to adjust the white balance setting and exposure setting of the photograph to obtain a true value image; Before constructing an initial neural network model for measuring leaf nitrogen content, obtaining leaf nitrogen labels, and training the initial neural network model for measuring leaf nitrogen content based on the corrected data set and the leaf nitrogen labels to obtain the leaf nitrogen content measurement model, the method further includes: Cropping the color card part in the corrected data set to obtain a cropped data set that only includes crop canopy information images; Performing data enhancement on the cropped data set to obtain an enhanced data set; Constructing an initial model of a neural network for measuring leaf nitrogen content, obtaining leaf nitrogen labels, and training the initial model of a neural network for measuring leaf nitrogen content based on the corrected data set and the leaf nitrogen labels to obtain a leaf nitrogen content measurement model, including: A preset convolutional neural network is used to construct an initial neural network model for measuring leaf nitrogen content; Obtaining the leaf nitrogen label by experimentally measuring the crop canopy sample photo; The mean square error (MSE) loss function and the Adam optimizer are used to train and converge the initial model of the leaf nitrogen content measurement neural network to obtain the leaf nitrogen content measurement model. 2. The leaf nitrogen content measurement method based on RGB images according to claim 1, characterized in that the white balance mode of the crop canopy sample photo is adjusted to obtain a white balance correction data set, and an image white balance correction neural network is constructed from the white balance correction data set, comprising: Adjusting the white balance mode of the true value image to obtain a plurality of color temperature deviation images including different color temperatures, and combining the plurality of color temperature deviation images and the true value image to form the white balance correction data set; Based on the white balance correction data set, a white balance correction full convolutional neural network is constructed, and the L1 loss function and the Adam optimizer are used to train and converge the white balance correction full convolutional neural network to obtain the image white balance correction neural network. 3. The leaf nitrogen content measurement method based on RGB images according to claim 1 is characterized in that the exposure mode of the crop canopy sample photo is adjusted to obtain an exposure correction data set, and an image exposure correction neural network is constructed based on the correction data set, comprising: Adjusting the exposure mode of the true value image to obtain a plurality of exposure deviation images including different exposure values, and forming the exposure correction data set with the plurality of exposure deviation images and the true value image; Based on the exposure correction data set, an exposure correction full convolutional neural network is constructed, and the exposure correction full convolutional neural network is trained and converged using an L1 loss function and an Adam optimizer to obtain the image exposure correction neural network. 4. The leaf nitrogen content measurement method based on RGB images according to claim 1 is characterized in that the image white balance correction neural network and the image exposure correction neural network are used to correct the crop canopy sample photo to obtain a corrected data set, including: Inputting the crop canopy sample photo into the image white balance correction neural network to obtain a white balance corrected data set; The crop canopy sample photos are input into the image exposure correction neural network to obtain an exposure-corrected data set. 5. The method for measuring leaf nitrogen content based on RGB images according to claim 1, further comprising: Inputting crop canopy images of different shooting angles, different shooting equipment and different crop varieties into the leaf nitrogen content measurement model to obtain measurement results of different leaf nitrogen contents; The nitrogen content measurement results of the different leaves are evaluated and compared using different preset accuracy evaluation indicators. 6. A leaf nitrogen content measurement system based on RGB images, comprising: A collection module for collecting sample photos of crop canopies; A white balance data processing module, used for adjusting the white balance mode of the crop canopy sample photo to obtain a white balance correction data set, and constructing an image white balance correction neural network based on the white balance correction data set; An exposure data processing module, used for adjusting the exposure mode of the crop canopy sample photo to obtain an exposure correction data set, and constructing an image exposure correction neural network based on the correction data set; A correction module, used to correct the crop canopy sample photo by integrating the image white balance correction neural network and the image exposure correction neural network to obtain a corrected data set; A training module is used to construct an initial model of a neural network for measuring leaf nitrogen content, obtain leaf nitrogen labels, and train the initial model of the neural network for measuring leaf nitrogen content based on the corrected data set and the leaf nitrogen labels to obtain a leaf nitrogen content measurement model; A measurement module, used for inputting a canopy photo of a crop to be measured into the leaf nitrogen content measurement model to obtain a leaf nitrogen content measurement result; The acquisition module is specifically used for: Use a smartphone camera to photograph crop canopies at any time during the day, collect photos from any three different preset angles, and save the photos in a preset file format; Using image processing software to adjust the white balance setting and exposure setting of the photograph to obtain a true value image; The training module is specifically used for: Cropping the color card part in the corrected data set to obtain a cropped data set that only includes crop canopy information images; Performing data enhancement on the cropped data set to obtain an enhanced data set; A preset convolutional neural network is used to construct an initial neural network model for measuring leaf nitrogen content; Obtaining the leaf nitrogen label by experimentally measuring the crop canopy sample photo; The mean square error (MSE) loss function and the Adam optimizer are used to train and converge the initial model of the leaf nitrogen content measurement neural network to obtain the leaf nitrogen content measurement model. 7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for measuring nitrogen content in leaves based on RGB images as described in any one of claims 1 to 5 is implemented.