In-Season Cotton Yield Prediction with Scale-Aware Convolutional Neural Network Models and Unmanned Aerial Vehicle RGB Imagery
<p>The experimental cotton field.</p> "> Figure 2
<p>To predict the cotton yield with CNN models at the grid level, the authors first split the large scale of UAV cotton image into smaller scales with ArcGIS Pro, which created 5376 images for each sampling date.</p> "> Figure 3
<p>The implementation of the CNN models relied on the TensorFlow 2.0 framework [<a href="#B27-sensors-24-02432" class="html-bibr">27</a>] and KerasTuner [<a href="#B25-sensors-24-02432" class="html-bibr">25</a>]. An illustration depicting the architecture of the CNN model is provided in this figure.</p> "> Figure 4
<p>The violin plot illustrates the distribution of row cotton yield data across four different irrigation treatments. Each violin represents the probability density of yields within a specific treatment group, with wider sections indicating higher density regions.</p> "> Figure 5
<p>The regression analysis of blue reflectance and grid cotton yield.</p> "> Figure 6
<p>The violin plot illustrates the distribution of grid cotton yield data across four different irrigation treatments. Each violin represents the probability density of yields within a specific treatment group, with wider sections indicating higher density regions.</p> "> Figure 7
<p>The training and testing performance of the CNN models with the cotton row image dataset. (<b>a</b>) Training performance at the cotton row level; (<b>b</b>) testing performance at the cotton row level.</p> "> Figure 8
<p>The <math display="inline"><semantics> <msup> <mi mathvariant="normal">R</mi> <mn>2</mn> </msup> </semantics></math> of the CNN models for cotton yield prediction at the cotton row level. (<b>a</b>) The proposed CNN model; (<b>b</b>) the AlexNet model; (<b>c</b>) the CNN-3D model; (<b>d</b>) the ResNet model; (<b>e</b>) the CNN-LSTM model.</p> "> Figure 9
<p>The training and testing performance of the CNN models with the cotton grid image dataset. (<b>a</b>) Training performance at the cotton grid level; (<b>b</b>) testing performance at the cotton grid level.</p> "> Figure 10
<p>The <math display="inline"><semantics> <msup> <mi mathvariant="normal">R</mi> <mn>2</mn> </msup> </semantics></math> of the CNN models for cotton yield prediction at the cotton grid level. (<b>a</b>) The proposed CNN model; (<b>b</b>) the AlexNet model; (<b>c</b>) the CNN-3D model; (<b>d</b>) the ResNet model; (<b>e</b>) the CNN-LSTM model.</p> ">
Abstract
:1. Introduction
2. Materials and Methods
2.1. The Study Site and Yield Data Collection
2.2. Description of the UAV and RGB Image Processing
2.3. Convolutional Neural Networks
2.3.1. A Customized Convolutional Neural Network
2.3.2. Convolutional Neural Networks with Long Short-Term Memory (CNN-LSTM) Networks
2.3.3. Residual Network (ResNet)
2.3.4. Three-Dimensional Convolutional Neural Networks (CNN-3Ds)
2.3.5. AlexNet
2.4. CNN Regression Models’ Evaluation Metrics
3. Results and Discussion
3.1. Exploratory Cotton Yield Analysis
3.2. The Performance of CNN Models at the Row Level
3.3. The Performance of CNN Models at the Grid Level
4. Conclusions
5. Research Reproducibility
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
ARS | Agricultural Research Service |
CMOS | Complementary metal–oxide–semiconductor |
CNN | Convolutional neural network |
CONUS | Continental United States |
CSRL | Cropping Systems Research Laboratory |
DFNN | Deep fully connected neural network |
EDA | Exploratory data analysis |
Crop evapotranspiration | |
EXGI | Excess green index |
LST | Land surface temperature |
LSTM | Long short-term memory |
MAE | Mean absolute error |
MAPE | Mean absolute percentage error |
MDPI | Multidisciplinary Digital Publishing Institute |
ML | Machine learning |
MLP | Multi-Layer Perceptron |
MODIS | Moderate Resolution Imaging Spectroradiometer |
MP | Megapixels |
NDVI | Normalized difference vegetation index |
RFR | Random forest regression |
RGB | Red, green, and blue |
ResNet | Residual network |
RMSE | Root-mean-square error |
RNN | Recurrent neural network |
SGD | Stochastic gradient descent |
SR | Surface reflectance |
SVR | Support vector regression |
THP | Texas High Plains |
UAV | Unmanned Aerial Vehicle |
US | United States |
USDA | United States Department of Agriculture |
References
- Johnson, J.D.; Kiawu, J.; MacDonald, S.; Meyer, L.A.; Skelly, C. The World and United States Cotton Outlook; Technical Report; United States Department of Agriculture: Washington, DC, USA, 2013. [Google Scholar]
- Adhikari, P.; Gowda, P.; Marek, G.; Brauer, D.; Kisekka, I.; Northup, B.; Rocateli, A. Calibration and validation of CSM-CROPGRO-Cotton model using lysimeter data in the Texas High Plains. J. Contemp. Water Res. Educ. 2017, 162, 61–78. [Google Scholar] [CrossRef]
- Adhikari, P.; Ale, S.; Bordovsky, J.P.; Thorp, K.R.; Modala, N.R.; Rajan, N.; Barnes, E.M. Simulating future climate change impacts on seed cotton yield in the Texas High Plains using the CSM-CROPGRO-Cotton model. Agric. Water Manag. 2016, 164, 317–330. [Google Scholar] [CrossRef]
- Ale, S.; Himanshu, S.K.; Mauget, S.A.; Hudson, D.; Goebel, T.S.; Liu, B.; Baumhardt, R.L.; Bordovsky, J.P.; Brauer, D.K.; Lascano, R.J.; et al. Simulated dryland cotton yield response to selected scenario factors associated with soil health. Front. Sustain. Food Syst. 2021, 4, 617509. [Google Scholar] [CrossRef]
- Bordovsky, J.P.; Mustian, J.T.; Ritchie, G.L.; Lewis, K.L. Cotton irrigation timing with variable seasonal irrigation capacities in the Texas South Plains. Appl. Eng. Agric. 2015, 31, 883–897. [Google Scholar]
- Panek, E.; Gozdowski, D. Analysis of relationship between cereal yield and NDVI for selected regions of Central Europe based on MODIS satellite data. Remote Sens. Appl. Soc. Environ. 2020, 17, 100286. [Google Scholar] [CrossRef]
- Kamir, E.; Waldner, F.; Hochman, Z. Estimating wheat yields in Australia using climate records, satellite image time series and machine learning methods. ISPRS J. Photogramm. Remote Sens. 2020, 160, 124–135. [Google Scholar] [CrossRef]
- Ashapure, A.; Jung, J.; Chang, A.; Oh, S.; Yeom, J.; Maeda, M.; Maeda, A.; Dube, N.; Landivar, J.; Hague, S.; et al. Developing a machine learning based cotton yield estimation framework using multi-temporal UAS data. ISPRS J. Photogramm. Remote. Sens. 2020, 169, 180–194. [Google Scholar] [CrossRef]
- Niu, H.; Wang, D.; Ehsani, R.; Chen, Y. Scale-aware pomegranate yield prediction using UAV imagery and machine learning. J. ASABE 2023, 66, 1331–1340. [Google Scholar] [CrossRef]
- Zhao, T.; Koumis, A.; Niu, H.; Wang, D.; Chen, Y. Onion irrigation treatment inference using a low-cost hyperspectral scanner. In Proceedings of the Multispectral, Hyperspectral, and Ultraspectral Remote Sensing Technology, Techniques and Applications VII. International Society for Optics and Photonics, Honolulu, HI, USA, 24–26 September 2018. [Google Scholar]
- Niu, H.; Zhao, T.; Wang, D.; Chen, Y. A UAV resolution and waveband aware path planning for onion irrigation treatments inference. In Proceedings of the 2019 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA, 11–14 June 2019; pp. 808–812. [Google Scholar]
- Zhao, T.; Chen, Y.; Ray, A.; Doll, D. Quantifying almond water stress using unmanned aerial vehicles (UAVs): Correlation of stem water potential and higher order moments of non-normalized canopy distribution. In Proceedings of the ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Cleveland, OH, USA, 6–9 August 2017. [Google Scholar]
- Zhang, L.; Zhang, H.; Niu, Y.; Han, W. Mapping maize water stress based on UAV multispectral remote sensing. Remote Sens. 2019, 11, 605. [Google Scholar] [CrossRef]
- Zhao, T.; Niu, H.; Anderson, A.; Chen, Y.; Viers, J. A detailed study on accuracy of uncooled thermal cameras by exploring the data collection workflow. In Proceedings of the Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping III, Orlando, FL, USA, 3–7 April 2022. [Google Scholar]
- Khanal, S.; Fulton, J.; Shearer, S. An overview of current and potential applications of thermal remote sensing in precision agriculture. Comput. Electron. Agric. 2017, 139, 22–32. [Google Scholar] [CrossRef]
- Awad, M.; Khanna, R.; Awad, M.; Khanna, R. Support vector regression. Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers; Springer: Berlin/Heidelberg, Germany, 2015; pp. 67–80. [Google Scholar]
- Segal, M.R. Machine Learning Benchmarks and Random Forest Regression; Kluwer Academic Publishers: Amsterdam, The Netherlands, 2003. [Google Scholar]
- Feng, A.; Zhou, J.; Vories, E.D.; Sudduth, K.A.; Zhang, M. Yield estimation in cotton using UAV-based multi-sensor imagery. Biosyst. Eng. 2020, 193, 101–114. [Google Scholar] [CrossRef]
- Da Silva, E.E.; Baio, F.H.R.; Teodoro, L.P.R.; da Silva, C.A., Jr.; Borges, R.S.; Teodoro, P.E. UAV-multispectral and vegetation indices in soybean grain yield prediction based on in situ observation. Remote Sens. Appl. Soc. Environ. 2020, 18, 100318. [Google Scholar] [CrossRef]
- Khaki, S.; Wang, L.; Archontoulis, S.V. A CNN-RNN framework for crop yield prediction. Front. Plant Sci. 2020, 10, 1750. [Google Scholar] [CrossRef] [PubMed]
- Sun, J.; Di, L.; Sun, Z.; Shen, Y.; Lai, Z. County-level soybean yield prediction using deep CNN-LSTM model. Sensors 2019, 19, 4363. [Google Scholar] [CrossRef] [PubMed]
- Shahhosseini, M.; Hu, G.; Khaki, S.; Archontoulis, S.V. Corn yield prediction with ensemble CNN-DNN. Front. Plant Sci. 2021, 12, 709008. [Google Scholar] [CrossRef] [PubMed]
- Chandel, N.S.; Chakraborty, S.K.; Rajwade, Y.A.; Dubey, K.; Tiwari, M.K.; Jat, D. Identifying crop water stress using deep learning models. Neural Comput. Appl. 2021, 33, 5353–5367. [Google Scholar] [CrossRef]
- Li, R.; Wang, R.; Zhang, J.; Xie, C.; Liu, L.; Wang, F.; Chen, H.; Chen, T.; Hu, H.; Jia, X.; et al. An effective data augmentation strategy for CNN-based pest localization and recognition in the field. IEEE Access 2019, 7, 160274–160283. [Google Scholar] [CrossRef]
- O’Malley, T.; Bursztein, E.; Long, J.; Chollet, F.; Jin, H.; Invernizzi, L. KerasTuner. 2019. Available online: https://github.com/keras-team/keras-tuner (accessed on 30 October 2019).
- Niu, H.; Landivar, J.; Duffield, N. Classification of cotton water stress using convolutional neural networks and UAV-based RGB imagery. Adv. Mod. Agric. 2024, 5, 1. [Google Scholar]
- Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. arXiv 2015, arXiv:1603.04467. [Google Scholar]
- Xing, D.; Wang, Y.; Sun, P.; Huang, H.; Lin, E. A CNN-LSTM-att hybrid model for classification and evaluation of growth status under drought and heat stress in Chinese fir (Cunninghamia lanceolata). Plant Methods 2023, 19, 66. [Google Scholar] [CrossRef]
- Wang, J.; Si, H.; Gao, Z.; Shi, L. Winter wheat yield prediction using an LSTM model from MODIS LAI products. Agriculture 2022, 12, 1707. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 770–778. [Google Scholar]
- Noshiri, N.; Beck, M.A.; Bidinosti, C.P.; Henry, C.J. A comprehensive review of 3D convolutional neural network-based classification techniques of diseased and defective crops using non-UAV-based hyperspectral images. arXiv 2023, arXiv:2306.09418. [Google Scholar] [CrossRef]
- Ji, S.; Zhang, C.; Xu, A.; Shi, Y.; Duan, Y. 3D convolutional neural networks for crop classification with multi-temporal remote sensing images. Remote Sens. 2018, 10, 75. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012. [Google Scholar]
- Mia, M.S.; Tanabe, R.; Habibi, L.N.; Hashimoto, N.; Homma, K.; Maki, M.; Matsui, T.; Tanaka, T.S. Multimodal deep learning for rice yield prediction using UAV-based multispectral imagery and weather data. Remote Sens. 2023, 15, 2511. [Google Scholar] [CrossRef]
Data | Treatment | Mean (lb) | Std (lb) | Count |
---|---|---|---|---|
Grid yield | Rainfed | 0.36 | 0.14 | 1344 |
Percent deficit | 0.61 | 0.17 | 1344 | |
Time delay | 0.72 | 0.16 | 1344 | |
Fully irrigated | 1.02 | 0.12 | 1344 | |
Row yield | Rainfed | 19.92 | 2.66 | 24 |
Percent deficit | 34.26 | 5.72 | 24 | |
Time delay | 38.57 | 2.89 | 24 | |
Fully irrigated | 58.68 | 9.31 | 24 |
CNN Models | MAE (lb) | MAPE (%) | |
---|---|---|---|
AlexNet | 4.84 | 14.4 | 0.84 |
ResNet | 5.44 | 13.53 | 0.80 |
CNN-3D | 5.25 | 12.08 | 0.76 |
CNN-LSTM | 11.97 | 35.07 | −0.03 |
Proposed CNN | 3.08 | 7.76 | 0.93 |
CNN Models | MAE (lb) | MAPE (%) | |
---|---|---|---|
AlexNet | 0.05 | 10.08 | 0.96 |
ResNet | 0.09 | 15.61 | 0.84 |
CNN-3D | 0.09 | 14.17 | 0.85 |
CNN-LSTM | 0.05 | 10.46 | 0.95 |
Proposed CNN | 0.05 | 10.00 | 0.95 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Niu, H.; Peddagudreddygari, J.R.; Bhandari, M.; Landivar, J.A.; Bednarz, C.W.; Duffield, N. In-Season Cotton Yield Prediction with Scale-Aware Convolutional Neural Network Models and Unmanned Aerial Vehicle RGB Imagery. Sensors 2024, 24, 2432. https://doi.org/10.3390/s24082432
Niu H, Peddagudreddygari JR, Bhandari M, Landivar JA, Bednarz CW, Duffield N. In-Season Cotton Yield Prediction with Scale-Aware Convolutional Neural Network Models and Unmanned Aerial Vehicle RGB Imagery. Sensors. 2024; 24(8):2432. https://doi.org/10.3390/s24082432
Chicago/Turabian StyleNiu, Haoyu, Janvita Reddy Peddagudreddygari, Mahendra Bhandari, Juan A. Landivar, Craig W. Bednarz, and Nick Duffield. 2024. "In-Season Cotton Yield Prediction with Scale-Aware Convolutional Neural Network Models and Unmanned Aerial Vehicle RGB Imagery" Sensors 24, no. 8: 2432. https://doi.org/10.3390/s24082432
APA StyleNiu, H., Peddagudreddygari, J. R., Bhandari, M., Landivar, J. A., Bednarz, C. W., & Duffield, N. (2024). In-Season Cotton Yield Prediction with Scale-Aware Convolutional Neural Network Models and Unmanned Aerial Vehicle RGB Imagery. Sensors, 24(8), 2432. https://doi.org/10.3390/s24082432