WO2020191483A1

WO2020191483A1 - Guided harvesting system for mushrooms

Info

Publication number: WO2020191483A1
Application number: PCT/CA2020/050359
Authority: WO
Inventors: Mohamed KASHKOUSH; Gideon AVIGAD
Original assignee: Vineland Research And Innovation Centre
Priority date: 2019-03-22
Filing date: 2020-03-18
Publication date: 2020-10-01

Abstract

A system for guided harvesting of mushrooms includes at least one shelving unit having a plurality of shelves, a plurality of mobile equipment carts movably mounted on the plurality of shelves and one or more computer processors. The carts are equipped with image data acquisition subsystems to acquire growing data of the mushrooms and non-invasive mushroom tagging subsystems to permit tagging of harvestable mushrooms. The one or more computer processors are programmed to identify the harvestable mushrooms from a synthesis of the growing data acquired by the image data acquisition subsystems on the plurality of carts and to operate the tagging subsystems to tag the harvestable mushrooms based on the harvestable mushrooms so identified.

Description

GUIDED HARVESTING SYSTEM FOR MUSHROOMS

Cross-reference to Related Applications

This application claims the benefit of United States Provisional Patent Application USSN 62/822,1 19 filed March 22, 2019, the entire contents of which is herein incorporated by reference.

Field

This application relates to a method and system for harvesting mushrooms. Background

Mushroom ( Agaricus bisporus) is a very fast-growing year-round crop. The harvesting cycle of mushrooms is three to five days with four to six harvesting rounds taking place daily. Harvesting of mushrooms for the fresh market is generally done manually, where harvesters should be able to select and pick mushrooms that should be harvested at each harvesting round.

The selection process is significantly more complex than just picking large-sized mushrooms. Mushrooms tend to form dense clusters. Dense clusters, if not properly relaxed on time (i.e., thinned), may lead to damaged or misshapen mushrooms that are sold as a second grade, at a discounted price. Furthermore, dense clusters will stress the mushrooms and will not allow for their healthy growth. On the other hand, having no clusters at all will certainly compromise yield. In addition to geometrical and clustering considerations, high quality mushrooms for the fresh market must be harvested at the right maturity/ripeness level. Two mushrooms with the same size could have different maturity levels, where one of the mushrooms should be harvested to avoid quality deterioration, while the other should be left to grow into a bigger size, to maximize yield. Accordingly, proper selection of growing mushrooms for harvesting is critical for yield and quality. Furthermore, it is estimated that 50 percent of a mushroom farm’s operational cost is associated with labor cost and that is mainly around harvesting. Thus, any improvement in harvesting labor productivity would mean significant savings on the production cost.

The main problems/challenges with current practices for mushroom harvesting include: a. Wasted time and cost for training: Much training and practice is required for human harvesters to become skilled in selecting mushrooms for harvesting. It takes a new harvester at least three months to be considered as fully trained. b. Wasted time in selecting mushrooms for harvesting: Proper selection of mushrooms for harvesting takes a considerable percentage of the harvesting time. c. Compromised quality due to contact-based ripeness assessment: Harvesters frequently touch and feel mushrooms in order to assess ripeness (based on firmness). Frequent touching of growing mushrooms results in increased chances of disease and also causes bruises/discoloration, which reduces mushroom quality. d. Compromised yield and quality due to lack of experience/skills: Quality of harvested mushrooms is compromised due to lack of experience/skills of some harvesters. e. Compromised harvesting yield and quality due to inconsistency of decision, even by well-trained harvesters: Inconsistency in decision making by well-trained harvesters happens due to many reasons such as: difficulty of contactless assessment, labor fatigue, and limitations on time allocated for assessment/harvesting. f. Compromised harvesting yield and quality due to limitation on field of view: To maximize space utilization, mushrooms are typically grown and harvested in multi-shelve structures with tight distances in between. Harvesters may accordingly experience a limited field of view (eye sight) during harvesting, especially at the middle of shelves.

There are a few known methods that use cameras for locating and measuring cap diameters of mushrooms in a mushroom bed or tray before harvesting the mushrooms with a mechanical picker. These methods are described in NL 86/00887, US 5,058,368, US 5,471 ,827, US 8,033,087 and US 2005/0268587. However, such technologies are not able to guide or assist harvesting labor (i.e. no guided harvesting functionality) or send data wirelessly on a regular basis to a computer server for further processing and analysis.

The mushroom harvesting process could be greatly improved and optimized through the use of advanced computer algorithms (Decision Support Systems) to assist/replace humans in making harvesting decisions, while the actual picking of mushrooms could still be done by harvesting labor. Computer-based Decision Support Systems can be more accurate than humans in determining mushrooms sizes and quality through data acquired by advanced vision and sensing devices. The computer-based Decision Support Systems can also use mathematical optimization and simulations models/algorithms in making selections that can maximize ultimate yield and quality of harvested mushrooms. Decision Support Systems for mushroom harvesting (e.g. US 9,730,394 issued August 15, 2017, the entire contents of which is herein incorporated by reference) are known. The system described in US 9,730,394 comprises a vision system, a computer-based Decision Support System and a robotic picker, which is designed for automated harvesting in a fixed robotic station. The system does not support in-shelf data acquisition or guided harvesting functionalities. However, there is no existing system or method that has been developed to facilitate the actual deployment and the use of such existing Decision Support Systems, or even future ones, to guide harvesting labor in a typical mushroom farm setting.

Mobichamp™ has demonstrated a system to acquire mushrooms images, make harvesting decision and then highlight selected mushrooms for harvesting. That system is a stationary system that require mushroom trays to be brought to the imaging/projection station mounted to the ceiling. Only one station is used and a mechanical conveyance system is utilized in bringing trays to the harvesting station and return them back to the growing room. So, such a system is not capable of carrying out the main functionalities of the developed system which is to be able to acquire data, communicate with a computer server, and convey computer based harvesting decisions to harvesting labor in a typical metal shelving growing system used by modern farms. Furthermore, the Mobichamp™ system does not use 3-D, thermal or hyperspectral vision capabilities.

The majority of the modern mushroom farms, especially the larger ones, use a standard multilayer metal shelving structure known as“Dutch shelves”. In order to make the best utilization of the farm space, the distance between the shelves is very tight. Shelf spacing may be in a range of about 25 cm to about 40 cm, for example about 33 cm. A computer-based Decision Support System will need continuous data on the growing mushrooms and will also need a practical means/method to deliver (convey or inform) the decisions made by the computer to the harvesting labor. Such tight shelving space, makes it very challenging to have a proper vision system in such a tight space and there is no existing methods or implementations in that regard, especially for 3-D and thermal vision capabilities. There are also no existing methods or implementations that could be used within such a tight space to direct or guide laborers, using light or other means, to the mushrooms that need to be picked.

There remains a need for a method and/or system for guiding harvesters to the mushrooms that require harvesting when the mushrooms are growing in a mushroom bed or tray situated in a multilayer metal shelving structure having tightly spaced shelves.

Summary

In one aspect, there is provided a system for guided harvesting of mushrooms comprising: at least one shelving unit comprising a plurality of shelves vertically spaced apart by a distance sufficient to permit full growth of mushrooms growing on the shelves; a plurality of mobile equipment carts movably mounted on the plurality of shelves, the plurality of carts equipped with image data acquisition subsystems and non-invasive mushroom tagging subsystems, the plurality of carts translatable along the plurality of shelves to permit the image data acquisition subsystems to acquire growing data of the mushrooms growing on the plurality of shelves without removing the mushrooms from the shelving unit and to permit the tagging subsystems to tag harvestable mushrooms for harvesting by a harvester; and, one or more computer processors configured to receive the growing data from the image data acquisition subsystems and to send instructions to the tagging subsystems, at least one of the one or more computer processors programmed to identify the harvestable mushrooms from a synthesis of the growing data acquired by the image data acquisition subsystems and to operate the tagging subsystems to tag the harvestable mushrooms based on the harvestable mushrooms so identified.

In another aspect, there is provided a system for guided harvesting of mushrooms comprising: at least one shelving unit comprising a plurality of shelves vertically spaced apart by a distance sufficient to permit full growth of mushrooms growing on the shelves; a plurality of mobile equipment carts movably mounted on the plurality of shelves, the plurality of carts equipped with image data acquisition subsystems or non-invasive mushroom tagging subsystems, the plurality of carts translatable along the plurality of shelves to permit the image data acquisition subsystems to acquire growing data of the mushrooms growing on the plurality of shelves or to permit the non-invasive mushroom tagging subsystems to tag harvestable mushrooms on the plurality of shelves for harvesting by a harvester without removing the mushrooms from the shelving unit; and, one or more computer processors programmed to identify the harvestable mushrooms from a synthesis of growing data acquired on the mushrooms and to inform the harvester of the harvestable mushrooms so identified.

In another aspect, there is provided a method of guided harvesting of mushrooms comprising: growing mushrooms in at least one shelving unit comprising a plurality of shelves vertically spaced apart by a distance sufficient to permit full growth of the mushrooms growing on the shelves; operating a plurality of mobile equipment carts movably mounted on the plurality of shelves, the plurality of carts equipped with image data acquisition subsystems and non-invasive mushroom tagging subsystems, the plurality of carts operated to translate along the plurality of shelves; acquiring growing data of the mushrooms growing on the plurality of shelves without removing the mushrooms from the shelving unit by operating the image data acquisition subsystems while the carts translate along the plurality of shelves; identifying the harvestable mushrooms from the growing data by using one or more computer processors programmed to receive the growing data from the image data acquisition subsystems and programmed to decide from a synthesis of the growing data which of the growing mushrooms are harvestable mushrooms; tagging the harvestable mushrooms for manual harvesting by a non-robotic harvester by using the one or more computer processors to send instructions to the tagging subsystems to operate the tagging subsystems to tag the harvestable mushrooms; and, harvesting the tagged mushrooms.

The present system and method advantageously provide one or more of the following benefits. The use of a computerized decision-making system lessens the need to have the harvesters making decisions on harvesting. Therefore, no, or very little, training for that activity is required and the harvesting labor does not have to waste any time in decision making. The data needed to make decisions is captured in a contactless approach, therefore, no contact-based assessments occur thereby reducing loss of quality due to contact-based ripeness assessments. The decisions made by the computer are at least as good as a human expert, minimizing the chances of compromising harvesting decisions due to lack of experience or skills of some harvesters. Decisions made by the computer are consistent and are not influenced by state of mind. By relying on mobile carts moving in-shelf and equipped with image data acquisition subsystems that have top views of the growing mushrooms, the guided harvesting system is able to clearly see mushrooms in the tight distances between the shelves. By centrally synthesizing all of the data collected from the mobile equipment carts in an entire farm, a computer-based Decision Support System (DSS) is able to use data from all of the mobile equipment carts to make decisions about which mushrooms to harvest in any one cart, thereby increasing efficiency and effectiveness of mushroom harvesting across the entire farm.

The guided harvesting system of the present invention facilitates actual implementation of computer-based decision making for harvesting in a modern mushroom farm setting (e.g. a Dutch shelving installation). The present system can operate under the limiting constraints of the multi-layered Dutch shelving structure to guide harvesting labor to the mushrooms that need to be picked.

Further features will be described or will become apparent in the course of the following detailed description. It should be understood that each feature described herein may be utilized in any combination with any one or more of the other described features, and that each feature does not necessarily rely on the presence of another feature except where evident to one of skill in the art.

Brief Description of the Drawings

For clearer understanding, preferred embodiments will now be described in detail by way of example, with reference to the accompanying drawings, in which:

Fig. 1 depicts a schematic diagram of a guided harvesting system showing harvesters, Dutch shelving units and mobile equipment carts mounted on the Dutch shelving units in different growing rooms communicating with a central computer processing unit; Fig. 2 depicts a schematic diagram for operating the mobile equipment carts in the guided harvesting system of Fig. 1 ;

Fig. 3 depicts a Dutch shelving unit with a mobile equipment cart mounted thereon;

Fig. 4A depicts a perspective view of one embodiment of a mobile equipment cart for use in the guided harvesting system;

Fig. 4B depicts a top view of the mobile equipment cart of Fig. 4A;

Fig. 4C depicts an end view of the mobile equipment cart of Fig. 4A;

Fig. 4D depicts a side view of the mobile equipment cart of Fig. 4A; Fig. 5A depicts a top view of a shelf in the Dutch shelving unit of Fig. 3 with a mobile equipment cart mounted thereon;

Fig. 5B depicts a side view of Fig. 5A;

Fig. 5C depicts an end view of Fig. 5A; Fig. 5D depicts a magnified view of detail A in Fig. 5A;

Fig. 6A depicts a top broken view of the shelf of Fig. 4A illustrating a driver end (left) and an idler end (right) of a wire and pulley drive mechanism for the mobile equipment cart on the Dutch shelving unit;

Fig. 6B depicts a side view of Fig. 6A; Fig. 6C depicts a bottom view of Fig. 6A;

Fig. 6D depicts an end view of Fig. 6A looking toward the driver side;

Fig. 6E depicts a magnified view of detail B in Fig. 6A;

Fig. 6F depicts a magnified view of detail C in Fig. 6C;

Fig. 6G depicts a magnified view of detail D in Fig. 6D; Fig. 7A depicts a perspective view the mobile equipment cart showing how the cart is mounted on a rail attached to the shelving unit;

Fig. 7B depicts a side view of the mobile equipment cart of Fig. 7A;

Fig. 7C depicts a magnified view of detail E in Fig. 7B;

Fig. 8A depicts a cross-sectional end view of the mobile equipment cart of Fig. 5A showing how the cart is mounted on a wire rope for driving the cart;

Fig. 8B depicts a view of section A-A from Fig. 8A; and,

Fig. 8C depicts a magnified view of detail F in Fig. 8A.

Detailed Description

With reference to Fig. 1 , in one embodiment, a system 1 for guided harvesting of mushrooms facilitates deployment and use of a computer-based harvesting Decision Support System (DSS) in a modern mushroom farm having a plurality of growing rooms 2 (only one labeled of three depicted) and a plurality of Dutch shelving units 5 (only one labeled) in the growing rooms 2 in which mushrooms grow. The system 1 further comprises a plurality of mobile equipment carts 10 (only one labeled) movably mounted on shelves of the plurality of shelving units 5, the plurality of mobile equipment carts operated by a plurality of harvesting personnel 90 (only one labeled). While the system is illustrated in the context of manual harvesting with harvesting personnel 90, harvesting may instead be accomplished robotically or by a hybrid of manual and robotic harvesting. Therefore, the harvester could be a person, a robot or both. Data from the plurality of mobile equipment carts 10 are transmitted wirelessly to a central computer server 100 where the DSS processes the data to decide which of the growing mushrooms are ready to be harvested. Harvesting decisions are transmitted wirelessly back to the plurality of mobile equipment carts 10 from the central computer server 100 to inform the plurality of harvesting personnel 90 which mushrooms to harvest.

With reference to Fig. 2, in one embodiment, each of the mobile equipment carts 10 in the guided harvesting system 1 comprises an image data acquisition subsystem 20 and a non-invasive mushroom tagging subsystem 30. The image data acquisition subsystem 20 collects image data 21 from growing mushrooms 22 in a mushroom growing bed 24, and such growing data collected by the image data acquisition subsystem 20, as well as cart data acquired by sensors 40 to acquire cart data, are transmitted to a computer processing unit 50 mounted on the cart 10, which wirelessly transmits signals 81 carrying the data to a central computer processing unit (central CPU) 80. The central CPU 80 receives data from a plurality of the mobile equipment carts 10, preferably all of the mobile equipment carts 10, in the guided harvesting system 1 , and synthesizes the collective data using a Decision Support System (DSS) programmed into the central CPU 80 to make decisions regarding which mushrooms to harvest in the next harvesting pass. In synthesizing the collective data, the DSS preferably uses data from all of the mobile equipment carts 10 to assist in making a harvesting decision about the mushrooms growing on one of the shelves of one of the shelving units 5. As part of the decision-making process, the DSS may predict from the growing data how the mushrooms on each of the plurality of shelves will look in a subsequent harvesting pass. After making the harvesting decision, the central CPU 80 wirelessly transmits signals 82 to each of the computer processing units 50 on each of the carts 10 to instruct the non- invasive mushroom tagging subsystems 30 on each cart to tag the mushrooms to be harvested. The non-invasive mushroom tagging subsystem 30 transmits tags 23 to the selected mushrooms, so that the harvesting personnel know which mushrooms to harvest. While data to and from the central CPU 80 is conveniently transmitted wirelessly, the central CPU 80 may instead be hard-wired to the computer processing unit 50 mounted on the cart 10. Further, while the system 1 preferably comprises sensors to acquire cart data, such sensors to acquire cart data may be omitted in some embodiments.

The image data acquisition subsystem may comprise one or more image data acquisition devices. Image acquisition devices may be, for example, a 2-D camera, a 3-D camera (e.g. a laser scanner), a thermal camera, a short wave infra-red (SWIR) camera, a stereo vision camera, a time-of-flight (TOF) sensor or any combination thereof. In one embodiment, 3-D cameras can be used provide an accurate and high resolution 3-D map of the growing mushrooms in a very tight working distance, while thermal and hyperspectral cameras mounted on the cart can be used to provide additional non-visual characteristics of the growing mushrooms.

The non-invasive mushroom tagging subsystem may comprise any device that can identify a mushroom to the harvesting personnel without causing damage to the mushroom. In some embodiments, the non-invasive mushroom tagging subsystem 30 may be based on visual and/or auditory indicia. Visual indicia are preferred. Visual indicia may be transmitted using a projection device, for example a projector that can place light dots (e.g. LED light dots) on the mushrooms that the DSS has decided to harvest. In one embodiment, the tagging subsystem may be an ultra-short throw projector. The tagging subsystem may be calibrated to the image data acquisition subsystem. For example, an ultra-short throw projector may be calibrated to a 3-D camera, where the projector is able to project on multiple planes depending on the height of the mushrooms.

The mobile equipment cart is preferably designed to work effectively within the space constraints of the shelving units. The mobile equipment cart may be driven on the shelving unit by any suitable propulsion system. In one embodiment, an external electric motor (e.g. a DC motor) may be used to drive the cart using a closed-loop cable system. In other embodiments, the driving mechanism may be a pneumatic system or a system for self-propelling the cart. While the mobile equipment cart preferably moves in between the shelves of the shelving units and above the crop, the cart may instead move along but outside of the shelving unit, for example on tracks or a mobile robot moving on the floor of the installation. If the mobile equipment cart were to move outside of the shelving unit, the image data acquisition subsystem could be inserted in between the shelves and taken thereout if studs of the shelving unit need to be avoided. Sensors to acquire a variety of cart data may be used, and the cart data synthesized with the growing data by the DSS to make harvesting decisions. Some examples of cart data include location of the cart, cart identification number, cart speed, status of devices on board the cart (e.g. battery, cameras, projectors, processors, electronic cards and various sensors) and any combination thereof. Localization sensors may be used to accurately determine locations of each cart. Such localization sensors may comprise one or more of laser sensors, rotary encoders and the like. Localization could be also realized using the image data acquisition subsystem based on analyzing indicia on the shelves. Various other cart sensors may include, for example inertial measurement unit (IMU) sensors.

With reference to Fig. 3, in one embodiment, each of the shelving units 5 comprise a plurality of vertically spaced-apart shelves 6 on which mushroom growing beds (not shown) are supported. Fig. 2 shows three shelves 6 shown on one shelving unit 5 but any number of shelves may be used. The shelving units 5 in Dutch shelving systems are typically about 30 m long. Above each of the shelves 6 mounted on the shelving unit 5 are translatable cart mounts 7 mounted on rails 8, the translatable cart mounts 7 operatively connected to driving mechanisms 9. The cart mounts 7 in Fig. 3 are shown disconnected from the carts 10. The rails 8 extend longitudinally along the length of the shelving unit 5 so that the driving mechanisms 9 can drive the cart mounts 7 and thus the mobile equipment carts 10 longitudinally along the shelving unit 5 above the shelves 6. A first distance di, which is the distance between the shelves 6 and the rails 8 is typically tight, though large enough to accommodate full growth of the mushrooms while permitting space for manual harvesting to occur. With the presence of the mobile equipment cart 10 above the shelf 6, the amount of space above the shelf 6 is reduced to a second distance d₂ between the shelf 6 and the mobile equipment cart 10, which is still large enough to permit full growth of the mushrooms but may be too tight to permit harvesting the mushroom directly underneath the mobile equipment cart 10. Further, time is required to capture the data, transmit the data, process the data, transmit instructions back to the tagging subsystem and tag the mushroom to be harvested. For these reasons, the mobile equipment cart 10 may precede the harvesting personnel 90 as the cart 10 and harvesting personnel 90 move longitudinally along the shelving unit 5 in a harvesting pass. The harvesting personnel 90 must therefore wait before starting to harvest, which could cause a loss of patience by the harvesting personnel 90 and/or a loss of valuable harvesting time. Alternatively, the tagging subsystem could tag mushrooms in one mushroom bed after the cart moves to the next mushroom bed, any time lag thereby being caused by moving the cart, assuming that data collection and processing ends by the time the cart moves. Or, the image data acquisition subsystem could acquire growing data from the mushroom bed next in line from the one being harvested, which means that the growing data acquired in the previous harvesting pass should be used for decision making.

In a possible variation, the image data acquisition system and the non-invasive mushroom tagging subsystem may be physically decoupled so that the two subsystems move separately. Such an arrangement could improve efficiency, provide for lighter carts and/or permit the image data acquisition system to acquire data from a larger section of the shelving unit, making control and monitoring simpler.

In one embodiment, there may be one mobile equipment cart 10 per harvesting personnel 90, in which case the harvesting personnel 90 may carry the cart 10 from shelf to shelf and mount the cart 10 on the cart mount 7 above the desired shelf 6 before operating the cart 10. In another embodiment, there may be one cart 10 per shelving unit 5 and a lift system (not shown) may be used to lift and lower the cart 10 between shelves 6. In yet another embodiment, there may be one cart 10 per shelf 6, in which case moving carts 10 between shelves 6 is not necessary. In a fully robotic harvesting operation, one cart 10 per shelf 6 is particularly desirable.

With reference to Fig. 4A, Fig. 4B, Fig. 4C and Fig. 4D, one embodiment of the mobile equipment cart 10 is illustrated. The mobile equipment cart 10 comprises a frame 11 having a slim profile to maximize space between the cart 10 and the shelf below when the cart 10 is mounted on the rail above the shelf. A plurality of cameras 12, including at least one 3-D camera, at least one thermal camera and at least one short wave infra-red (SWIR) camera are mounted to an underside of the frame 11 and oriented to provide sufficient field of view so that images of all of the mushrooms growing in the growing beds resting on the shelf can be obtained as the cart 10 travels along the rail. An ultra-short throw LED projector 13 is mounted at one end edge of the cart 10, the projector 13 calibrated to the cameras 12 and able to project on multiple planes depending on the height of the mushrooms. A human/machine interface (HMI), for example a touch screen monitor 14, is mounted along one side edge of the frame 11. Alternatively or additionally, a computer tablet could be used to communicate and control cart operation wirelessly or wired. The monitor 14, projector 13 and cameras 12 are hard-wired together and to a battery housed in a battery compartment 15 mounted at an opposite end edge of the cart 10 from the projector 13. The battery compartment 15, monitor 14 and projector 13 are mounted at the edges of the frame 11 to minimize the overall height of the cart 10 to maintain a slim profile for the cart 10. The mobile equipment cart 10 is shown connected to a linear bearing subassembly 16, the linear bearing subassembly 16 being a part of the cart mount 7. The cart 10 is connected to the linear bearing subassembly 16 by a pair of cart locking carriages 17 connected to a pair of locking carriage housings 18, which are discussed more fully in connection with how the cart 10 is mounted on the rail and driving mechanism as shown in Fig. 7A to Fig. 8C.

The mobile cart may further comprise environmental sensors to acquire environmental data related to growing conditions for the mushrooms. The growing conditions may comprise, for example, one or more of ambient air temperature, relative humidity, carbon dioxide level, air flow and growing medium surface temperature. Proper monitoring and control of growing conditions is important to a productive operation, and is currently done in the growing room as a whole. The ability to measure mushroom growing conditions at a localized level by equipping the mobile cart with environmental sensors permits acquiring environmental data every time the mushrooms are scanned, thereby discerning any localized issues. Any issues discovered can then be mitigated immediately, which would ultimately contribute to the improved yield and quality of the crop. With reference to Fig. 4A, Fig. 4B, Fig. 4C and Fig. 4D, the mobile equipment cart 10 comprises an air flow sensor box 111 with an air flow sensor probe 222 mounted to the underside of the frame 11 and oriented to permit measuring rate of air flow past the mobile cart 10. The mobile equipment cart 10 also comprises a measurement unit 333 mounted to the underside of the frame 11 , the measurement unit 333 comprising a temperature sensor, a relative humidity sensor and a carbon dioxide sensor for measuring temperature, humidity level and carbon dioxide level in the air around the mobile cart 10. The air flow sensor box 111 and the measurement unit 333 are conveniently clustered together on the frame 11.

With reference to Fig. 5A, Fig. 5B, Fig. 5C and Fig. 5D, the Dutch shelving unit 5 of Fig. 3 with the mobile equipment cart 10 of Fig. 4A mounted thereon. The shelving unit 5 has mounted thereon a pair of laterally spaced-apart longitudinally extending rails 8. The rails 8 comprise aluminum extrusions shaped to be fitted in the linear bearing subassembly 16 of the cart mount 7 so that the cart mount 7, and therefore the cart 10, can slide on the rails 8. The cart 10 may be mounted on either of the rails 8, and must be dismounted from one rail and moved and mounted to the other rail by the harvesting personnel when it is desired to do so. More details on how the cart 10 is mounted on the rail 8 are provided below in connection with Fig. 7A, Fig. 7B and Fig. 7C. The rails 8 are secured to the shelving unit 5 by rail mounting brackets 61 where the rails 8 cross laterally extending elongated support members 62 of the shelving unit 5. The drive mechanism 9 comprises a loop of wire 71 looped through four corner pulleys 72 arranged in a horizontal plane with respect to the ground such that the loop of wire 71 generally forms a rectangular loop with a first length of the wire 71 parallel to and in the same vertical plane as one of the rails 8 and a second length of the wire 71 parallel to and in the same vertical plane as the other of the rails 8. Two of the four corner pulleys 72 are at a driver end 73 of the drive mechanism 9; and, the other two of the four corner pulleys 72 are at an idler end 74 of the drive mechanism 9. At the driver end 73 of the drive mechanism 9, the loop of wire 71 passes over a drive pulley 75 (see Fig. 6F), which is driven by a reversible variable speed motor 79 to move the wire 71 thereby moving the mobile equipment cart 10 connected to the wire 71. Direction of motion X-X of the cart 10 is shown in Fig. 5A. More details of the connection of the cart 10 to the wire 71 are provided below in connection with Fig. 8A, Fig. 8B and Fig. 8C. More details of the drive mechanism 9 are provided below in connection with Fig. 6A, Fig. 6B, Fig. 6C, Fig. 6D, Fig. 6E, Fig. 6F and Fig. 6G.

With reference to Fig. 6A, Fig. 6B, Fig. 6C, Fig. 6D, Fig. 6E, Fig. 6F and Fig. 6G, the drive mechanism 9 is shown in more detail. At the driver end 73, the drive mechanism 9 comprises a first mounting plate 77a mounted on the rails 8 to which the motor 79, two of the corner pulleys 72, the drive pulley 75 and two tensioner pulleys 76 are mounted. At the idler end 74, the drive mechanism 9 comprises a second mounting plate 77b mounted on the rails 8 to which the other two corner pulleys 72 are mounted. The wire 71 is a continuous loop threaded around the corner pulleys 72, the drive pulley 75 and the tensioner pulleys 76. The motor 79 is operatively connected to the drive pulley 75 through a gearbox 78 to drive pulley 75. The motor 79 can drive the drive pulley 75 in either direction at various speeds to control the direction and speed of motion of the cart 10. A timing belt 70 between the drive pulley 75 and one of the tensioner pulleys 76 is employed to ensure synchronicity of movement of the wire 71 along both the first and second lengths.

With reference to Fig. 7A, Fig. 7B and Fig. 7C, the mobile equipment cart 10 is shown mounted on the rail 8, the rail 8 being attached to the shelving unit. The rail 8 comprises an aluminum extrusion shaped to be fitted in the linear bearing subassembly 16 of the cart mount 7 so that the cart mount 7, and therefore the cart 10, can slide on the rails 8. It should be noted that for clarity, Fig. 7A shows only the cart 10 and the rail 8, while Fig. 7B and Fig. 7C show more of the details that hold the cart 10 to the rail 8. The linear bearing subassembly 16 slides on the rail 8 when the cart 10 translates along the shelf of the shelving unit. The linear bearing subassembly 16 is fixedly attached to the frame 11 of the cart 10 through T-shaped feet 19 that are fitted within T-shaped grooves in the cart locking carriages 17. The cart locking carriages 17 are bolted to the locking carriage housings 18, which are in turn bolted to the frame 11 of the cart 10.

To move the cart 10 along the rail 8, the cart mount 7 is attached to the wire 71 of the drive mechanism 9 by a split collar 68, which clamps the wire 71. The split collar 68 is attached to a collar bracket 69, the collar bracket 69 being bolted to the linear bearing subassembly 16, as best seen in Fig. 8A, Fig. 8B and Fig. 8C. Detaching the cart 10 from the linear bearing subassembly 16 leaves the linear bearing subassembly 16 attached to the wire 71.

The Decision Support System (DSS) is a computer program programmed into the one or more computer processors. The DSS comprises a mathematical optimization model developed to make decisions as to which mushrooms to harvest during each harvesting pass. The mathematical optimization model together with a ripeness assessment model form the fundamental elements of the DSS.

Preferably, the DSS is programmed into a single central computer to facilitate synthesizing all of the data acquired from all of the carts. Data acquired from the carts include growing data, and may also include cart data and/or the environmental data. Growing data is associated with features of the mushrooms, while cart data is associated with features of the cart itself and environmental data is associated with the growing conditions around the bed. Growing data may be acquired over time, and may include, for example, one or more of the following mushroom features: mushroom location in the bed, size, height, shape (roundness), color, temperature and/or reflectance response (e.g. intensity) at one or more specific hyperspectral shortwave infrared (SWIR) wavelengths. The growing data not only may include the absolute values of each of the above features at a given time, but may also include temporal features, which are the rate of change of one or more of the above features over time. Cart data may also be acquired over time, and may include, for example, one or more of the following cart features: cart location, battery status, working status of the data acquisition subsystem and/or working status of the tagging subsystem. Environmental data may also be acquired over time, and may include, for example, one or more of ambient air temperature, relative humidity, carbon dioxide level, air flow and growing medium surface temperature.

Preferably, the one or more computer processors comprises a central processing unit configured to receive the cart data from all of the carts, receive the growing data from all of the image data acquisition subsystems, receive the environmental data from the environmental sensors and send instructions to all of the tagging subsystems. Preferably, data is received by and transmitted from the one or more computer processors wirelessly, although a hard-wired system could be employed. The DSS programmed into the one or more computer processors may synthesize the growing data using data from all of the mobile equipment carts to identify the harvestable mushrooms growing on one of the shelves of one of the shelving units. The one or more computer processors may therefore be programmed to predict from the growing data how the mushrooms on each of the plurality of shelves will look in a subsequent harvesting pass. The DSS may also include all of the cart data from all of the carts in the synthesis.

In one embodiment, the optimization model developed may be an integer programming (IP) model. An IP model is a system of mathematical equations and inequalities that are formulated in terms of a predefined set of integer variables. When those equations and inequalities are solved together, they provide the optimal values for those variables. The main challenge for building such models is to formulate the right equations and inequalities that properly represent the objective to be optimized and the constraints to be satisfied. Furthermore, in order for an IP model to be practical, these equations need to be formulated in a way that will allow for them to be solved in a reasonable amount of time.

In the embodiment of the integer programming (IP) model that was formulated, the harvesting decision to be taken for each mushroom in the model is represented with a binary (0-1) variable, where a value of "0" means a "pick" decision and a value of "1 " means a "stay" decision. Upon solving the IP model, mushrooms that have a value of 0 receive a pick decision and are harvested, and others are left in the bed to be further assessed in the following harvesting pass. The inputs for the optimization model primarily comprise the sizes, heights, ripeness values (from color, reflectance and/or reflectance response), shapes (roundness) and temporal data for each mushroom.

The logic behind the developed IP model is to seek the best balance between two specific objectives. One objective is to leave more mushrooms for the next pass and the other objective is to reduce the expected clustering between mushrooms within the next pass, which is predicted based on growth slopes. That balance is controlled through a clustering coefficient, which is a parameter that determines to what extent clustering reduction is important. Several experiments were conducted in order to find the best value for this clustering coefficient value and to further make it dependent on other inputs, such as time and ripeness level. The model input parameters are extracted from the image data acquisition system using various image processing and data extraction algorithms. Models may be developed that include any one or more of the following input parameters. n

Total number of mushrooms (in a given area or section). r_i

Expected radius of mushroom / in the next harvesting pass. pick_max

Maximum number of mushrooms to be harvested in the next harvesting pass. r_min

Minimum radius to be harvested (nothing to be harvested less than this radius). r_max

Maximum radius to be harvested (nothing to be left to grow bigger than that radius). ripe_i

A binary (0-1) element valued at 1 if mushroom / is expected to be ripe by the next harvesting pass, according to the ripeness assessment model.

area_i

Surface area of each mushroom i. ovlp_ij

Penalty or reward value for the contact/overlap between mushroom / and mushroom j. This is a separate pre-processing algorithm that considers many factors including overlap area (to calculate clustering coefficient), relative heights, ripeness, etc.

The decision variables (output) of the model may be as follows: x_i A binary (0-1) variable that takes the value 1 if mushroom / should stay (and not be harvested) in the next pass.

xx_ij An auxiliary binary decision variable (a linearization variable) that takes the value 1 if both mushrooms / and j are to stay in the next harvesting pass.

Minimize Loss (loss of potential mushroom growth)

(1)

Subject to

The objective function (Eq. 1) is the total sum of the overlap factor between mushrooms minus the total area of the mushrooms. The clustering coefficient is the main parameter that determines the importance of the overlap factor in the objective function and to what degree it should be minimized. Constraints 1 to 3 (Eq. 2 to Eq. 4) are linearization constraints. The objective function is originally a nonlinear quadratic function (not shown here). The linear form of the objective function shown here (Eq. 1) is only possible through the use of constraints 1 to 3. Constraint 4 (Eq. 5) is to ensure that the quantity (count) of mushrooms to be harvested is less than or maximum allowed quantity (count). Constraint 5 (Eq. 6) is to ensure that no mushrooms with a radius that is bigger than the maximum radius (an input parameter) are to be left without harvesting. Constraint 6 (Eq. 7) is to ensure that no mushrooms with a radius that is less than the minimum radius are to be harvested. Constraint 7 (Eq. 8) is to ensure that all mushrooms that are ripe (according to the ripeness assessment model) are to be harvested.

A simpler algorithm could be developed to assign a quality or worth value to each mushroom (based on its clustering condition) and then pick the ones with the least values. However, a problem with such an approach is that once one mushroom is to be considered for picking, then the assigned values to those other mushrooms that are in contact with that mushroom are in fact no longer applicable. However, the power of mathematical optimization is that it considers (in an indirect way) all the possible scenarios before it comes with the ones that need to be harvested for a minimal total value of the objective function. The way the solution methods for those models work makes it possible to find the optimal values for each decision variable (the optimal scenario) without the full enumeration of all possible cases.

The developed model is the core of the optimization part of the DSS. Algorithms generally known to one skilled in the art may be used to preprocess data and prepare it for the model. Other generally known algorithms can be used to calculate the estimated sizes, heights and locations of mushrooms at the time of harvest. The DSS could be operated relying only on the mathematical optimization model; however, combining the mathematical optimization model with ripeness assessment in the DSS improves performance of the DSS. Further refinements to the IP model to enhance performance and reliability can be accomplished by conducting experiments to tune the value for the overlap coefficient as well as other parameters that are used in calculating the penalty and reward values. A computer-based mushroom growth simulation model is one way to tune/optimize values for these parameters.

The IP model described above may be adapted to provide the DSS with the capability to optimize the harvesting process for an entire mushroom growing farm. To adapt the model into a global (farm-wide) DSS, the model may include further data concerning required market orders for a predefined time frame, and what each order consists of in terms of amounts, sizes and grades of mushrooms. A preprocessing algorithm may be included to keep track of all the mushrooms that have been harvested (amounts, sizes and grades) towards each order and the mushrooms that could be harvested from the farm in the predefined time frame using statistical estimation methods. Based on these data, the preprocessing model may obtain the ratio between sizes and the ratio between grades that need to be enforced in a future harvesting pass. Enforcing these ratios may be facilitated by augmenting the objective function with an additional term that represent the mismatch between the harvested sizes/ grades and the required ones as per the received orders. Minimizing the mismatch optimizes the decision making.

The novel features will become apparent to those of skill in the art upon examination of the description. It should be understood, however, that the scope of the claims should not be limited by the embodiments, but should be given the broadest interpretation consistent with the wording of the claims and the specification as a whole.

Claims

Claims:

1. A system for guided harvesting of mushrooms comprising: at least one shelving unit comprising a plurality of shelves vertically spaced apart by a distance sufficient to permit full growth of mushrooms growing on the shelves; a plurality of mobile equipment carts movably mounted on the plurality of shelves, the plurality of carts equipped with image data acquisition subsystems and non-invasive mushroom tagging subsystems, the plurality of carts translatable along the plurality of shelves to permit the image data acquisition subsystems to acquire growing data of the mushrooms growing on the plurality of shelves without removing the mushrooms from the shelving unit and to permit the tagging subsystems to tag harvestable mushrooms for harvesting by a harvester; and, one or more computer processors configured to receive the growing data from the image data acquisition subsystems and to send instructions to the tagging subsystems, at least one of the one or more computer processors programmed to identify the harvestable mushrooms from a synthesis of the growing data acquired by the image data acquisition subsystems and to operate the tagging subsystems to tag the harvestable mushrooms based on the harvestable mushrooms so identified.

2. The system of claim 1 , wherein the image data acquisition subsystems comprise at least one of a 2-D camera, a 3-D camera, a thermal camera and a shortwave infrared (SWIR) camera.

3. The system of claim 1 or claim 2, wherein the non-invasive mushroom tagging subsystems comprise projection systems that tag the harvestable mushrooms with visible light.

4. The system of any one of claims 1 to 3, wherein the at least one shelving unit comprises a plurality of rails on which the mobile equipment carts are removably mountable, the carts translatable along the rails when mounted thereon.

5. The system of any one of claims 1 to 4, wherein the at least one shelving unit comprises a plurality of shelving units.

6. The system of any one of claims 1 to 5, wherein the growing data comprises a feature of the growing mushrooms, the feature comprising location in the bed, size, height, shape, color, temperature, reflectance response at specific hyperspectral (SWIR) wavelength or any combination thereof.

7. The system of claim 6, wherein the growing data comprises one or more temporal features of the growing mushrooms, the one or more temporal features comprising a rate of change of one or more of the size, height, shape, color, temperature, reflectance response at specific hyperspectral (SWIR) wavelength or any combination thereof.

8. The system of any one of claims 1 to 7, wherein the plurality of mobile equipment carts comprises a plurality of sensors to acquire cart data, and wherein the one or more computer processors comprises a central processing unit configured to receive the cart data from all of the carts, receive the growing data from all of the image data acquisition subsystems and send instructions to all of the tagging subsystems.

9. The system of claim 8, wherein the cart data comprises at least one of cart location, cart identification number, cart speed, status of devices on board the cart or any combination thereof.

10. The system of claim 8 or claim 9, wherein the central processing unit receives the cart data and growing data and sends the instructions by wireless signals transmitted between the central processing unit and the sensors, image data acquisition subsystems and tagging subsystems.

1 1. The system of any one of claims 1 to 10, wherein the plurality of mobile equipment carts further comprises a plurality of environmental sensors to acquire environmental data related to growing conditions for the mushrooms, the growing conditions comprising one or more of ambient air temperature, relative humidity, carbon dioxide level, air flow and growing medium surface temperature.

12. The system of any one of claims 1 to 1 1 , wherein the harvester is a person or a robot.

13. A method of guided harvesting of mushrooms comprising: growing mushrooms in at least one shelving unit comprising a plurality of shelves vertically spaced apart by a distance sufficient to permit full growth of the mushrooms growing on the shelves; operating a plurality of mobile equipment carts movably mounted on the plurality of shelves, the plurality of carts equipped with image data acquisition subsystems and non-invasive mushroom tagging subsystems, the plurality of carts operated to translate along the plurality of shelves; acquiring growing data of the mushrooms growing on the plurality of shelves without removing the mushrooms from the shelving unit by operating the image data acquisition subsystems while the carts translate along the plurality of shelves; identifying the harvestable mushrooms from the growing data by using one or more computer processors programmed to receive the growing data from the image data acquisition subsystems and programmed to decide from a synthesis of the growing data which of the growing mushrooms are harvestable mushrooms; tagging the harvestable mushrooms for manual harvesting by a non-robotic harvester by using the one or more computer processors to send instructions to the tagging subsystems to operate the tagging subsystems to tag the harvestable mushrooms; and, harvesting the tagged mushrooms.

14. The method of claim 13, wherein the synthesis of the growing data comprises using data from all of the mobile equipment carts to identify the harvestable mushrooms growing on one of the shelves of one of the shelving units.

15. The method of claim 13 or claim 14, wherein the growing data comprises a feature of the growing mushrooms, the feature comprising location in the bed, size, height, shape, color, temperature, reflectance response at specific hyperspectral (SWIR) wavelength or any combination thereof.

16. The method of claim 15, wherein the growing data comprises one or more temporal features of the growing mushrooms, the one or more temporal features comprising a rate of change of one or more of the size, height, shape, color, temperature, reflectance response at specific hyperspectral (SWIR) wavelength or any combination thereof.

17. The method of any one of claims 13 to 16, further comprising acquiring cart data about the plurality of mobile equipment carts.

18. The method of claim 17, wherein the cart data comprises at least one cart location, cart identification number, cart speed, status of devices on board the cart or any combination thereof.

19. The method of claim 17 or claim 18, wherein the one or more computer processors comprises a central processing unit configured to receive the cart data from all of the carts, receive the growing data from all of the image data acquisition subsystems and send instructions to all of the tagging subsystems.

20. The method of claim 19, wherein the central processing unit receives the cart data and growing data and sends the instructions by wireless signals.

21. The method of any one of claims 13 to 20, wherein the one or more computer processors are programmed to predict from the growing data how the mushrooms on each of the plurality of shelves will look in a subsequent harvesting pass.

22. The method of any one of claims 13 to 21 , further comprising acquiring environmental data related to growing conditions for the mushrooms from a plurality of environmental sensors equipped on the plurality of mobile carts, the growing conditions comprising one or more of ambient air temperature, relative humidity, carbon dioxide level, air flow and growing medium surface temperature.

23. A system for guided harvesting of mushrooms comprising: at least one shelving unit comprising a plurality of shelves vertically spaced apart by a distance sufficient to permit full growth of mushrooms growing on the shelves; a plurality of mobile equipment carts movably mounted on the plurality of shelves, the plurality of carts equipped with image data acquisition subsystems or non-invasive mushroom tagging subsystems, the plurality of carts translatable along the plurality of shelves to permit the image data acquisition subsystems to acquire growing data of the mushrooms growing on the plurality of shelves or to permit the non-invasive mushroom tagging subsystems to tag harvestable mushrooms on the plurality of shelves for harvesting by a harvester without removing the mushrooms from the shelving unit; and, one or more computer processors programmed to identify the harvestable mushrooms from a synthesis of growing data acquired on the mushrooms and to inform the harvester of the harvestable mushrooms so identified.