EP2894972B1 - Method and system for creating a map of a barn - Google Patents
Method and system for creating a map of a barn Download PDFInfo
- Publication number
- EP2894972B1 EP2894972B1 EP13756719.4A EP13756719A EP2894972B1 EP 2894972 B1 EP2894972 B1 EP 2894972B1 EP 13756719 A EP13756719 A EP 13756719A EP 2894972 B1 EP2894972 B1 EP 2894972B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- animal
- barn
- type
- station
- criteria
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 35
- 241001465754 Metazoa Species 0.000 claims description 230
- 230000035622 drinking Effects 0.000 claims description 18
- 230000007937 eating Effects 0.000 claims description 10
- 230000022676 rumination Effects 0.000 claims description 10
- 208000015212 rumination disease Diseases 0.000 claims description 10
- 238000000926 separation method Methods 0.000 claims description 10
- 238000004458 analytical method Methods 0.000 claims description 8
- 230000001419 dependent effect Effects 0.000 claims description 7
- 238000006073 displacement reaction Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 4
- 230000036962 time dependent Effects 0.000 claims description 2
- 241000283690 Bos taurus Species 0.000 description 38
- 238000004590 computer program Methods 0.000 description 8
- 238000013459 approach Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000009304 pastoral farming Methods 0.000 description 4
- 241000282849 Ruminantia Species 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 235000013365 dairy product Nutrition 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000008267 milk Substances 0.000 description 2
- 235000013336 milk Nutrition 0.000 description 2
- 210000004080 milk Anatomy 0.000 description 2
- 241000283707 Capra Species 0.000 description 1
- 241001494479 Pecora Species 0.000 description 1
- 241001272996 Polyphylla fullo Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 210000003608 fece Anatomy 0.000 description 1
- FKLFBQCQQYDUAM-UHFFFAOYSA-N fenpiclonil Chemical compound ClC1=CC=CC(C=2C(=CNC=2)C#N)=C1Cl FKLFBQCQQYDUAM-UHFFFAOYSA-N 0.000 description 1
- 244000144980 herd Species 0.000 description 1
- 239000010871 livestock manure Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 239000008399 tap water Substances 0.000 description 1
- 235000020679 tap water Nutrition 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
- A01K29/00—Other apparatus for animal husbandry
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
- A01K29/00—Other apparatus for animal husbandry
- A01K29/005—Monitoring or measuring activity, e.g. detecting heat or mating
Definitions
- the present invention relates to a method of automatically creating a digital map of an animal barn, wherein the animal barn comprises a plurality of types of barn elements, a plurality of animals in said barn, and an animal locating system that is configured to determine a position of at least one of said animals in said barn.
- the invention further relates to a system configured to automatically create a digital wap of an animal barn with a plurality of types of barn elements. (See e.g. WO2011/055004 A1 )
- a digital map of a barn i.e. a walled space in which animals are free to walk.
- a farmer desires to know the whereabouts of a certain individual animal, such as to give the animal a particular treatment or to fetch the animal for some reason, such a map can be helpful to him, especially if available in digital form.
- to learn about animal behaviour it is deemed of interest to know where individual animals are during their stay in the barn.
- a more general advantage lies in the possibility to use the map in farm or animal management.
- patent document WO2011/055005 generally deals with monitoring the path of an animal or human in the home by means of a locating system.
- a method is disclosed for drawing a digital map.
- a person provided with a device by which his position can be monitored walks around through the home.
- the person's position is registered, e.g. after pressing a button in the monitoring device.
- some computer that is not further described generates a floor plan.
- a human user may also enter furniture and the like in the map.
- This known method, and device has a drawback in that it requires a significant amount of human work.
- a method of automatically creating a digital map of an animal barn wherein the animal barn comprises a plurality of types of barn elements, a plurality of animals in said barn, and an animal locating system that is configured to determine a position of at least one of said animals in said barn, the method comprising the steps of automatically collecting position data with the animal locating system about the position of said at least one animal in said barn as a function of time, providing, for at least a plurality of types of barn element, a first set of criteria for correlating said position data to said type of barn element and a second set of criteria for determining the position of said type of barn element, automatically applying the sets of criteria to the position data, to thereby derive the type and position of at least a first barn element of a first type, and automatically creating the map with at least said first barn element and the type and position thereof.
- the invention is based at least in part on the insight that it is possible to provide criteria to determine an element type from animal position data, more in particular to provide criteria for automatically determining the element type from a plurality of element types, on the basis of position data.
- criteria for automatically determining the element type from a plurality of element types, on the basis of position data.
- This will greatly help e.g. a farmer in obtaining a map of his barn.
- Such a map may use for managing the animals in that barn. For example, it may help him when looking for an animal to be treated. Indicating the animal on the map will shorten the time needed for him to find the animal.
- the farmer only after having such a map in the first place may the farmer obtain a real insight in the behaviour of the animals in the barn.
- the barn elements may be of many types, such as a cubicle, a feed fence, a milking robot, a gates, a walking path, or any other type of barn element that has some interaction with the animal, from which interaction the type of element and its position may be determined with a reasonable, predetermined certainty.
- light switches, tap water connections and the like are not types of barn element to be determined here.
- the invention relates to using the position data of non-human animals. It is noted that it is known to use human work, in the form of a person walking around with a location sensor, for making a map of a room, building, office or the like. However, this known method requires a lot of human work, while the present invention is aimed at preventing, or at least significantly reducing, that human work. Thereto, it uses the animals in the barn themselves. Note furthermore that it suffices if the positions in time of at least one animal are determined, as data from one animal would in principle suffice. However, using the position data from more animals, or even all animals in the barn, will likely lead to more data and thus a quicker and/or more reliable determination of type and position of barn elements.
- animals have habits that prevent them from using e.g. certain cubicles or parts of a feed fence, which would thus remain "invisible” or undetected. It should also be realised that it is sometimes rather expensive to collect position data from all (or at least many) animals, as this may sometimes require tagging the animals with dedicated tags.
- the position data about the position of said at least one animal in said barn are collected as a function of time. This collecting takes place during a predetermined period of time, or until the position data are determined to be sufficient, i.e. fulfil a predetermined sufficiency criterion.
- the data are collected during two or three days, or any other period of time in which the herd will likely have exhibited its normal behaviour.
- the predetermined period of time may be dependent on the number of animals for which data are collected, as a higher number will likely generate more data per unit time.
- the position data may be collected until a sufficiency criterion has been fulfilled.
- this may relate to the likelihood with which a barn element has been determined from the data, such as the determination with a 95% likelihood that a feed fence is present at a certain position.
- it would not make much sense to keep on collecting data which would require resources unnecessarily.
- the specific data collected enable a quick determination, while in other cases more data are required.
- the predetermined period of time and/or the sufficiency criterion may be dependent on the type of barn element. For example, in the straightforward case that a certain barn element has its own ID (identification) station, determining its position in the barn requires not more than one animal position datum. Other elements may require a large number of data before a reliable determination can be obtained.
- the first set of criteria need not comprise more than one criterion.
- a simple example would be that only barn elements with an ID station are determined, such as a milking station, a concentrate feeding station and a selection gate. Then, the only criterion would relate to e.g. determining a position at the time of the ID determining, and coupling the position with the type of ID station.
- the criteria for determining the type of barn element and/or the criteria for determining its position in the barn are based on the animal position data.
- the method need not succeed in actually determining the position of more than one barn element, as sometimes the data will be too ambiguous or sparse.
- the invention relies on the possibility to automatically determine a type of barn element from a plurality of possible barn elements, based on animal position data.
- criteria it is relatively arbitrary what exact criteria are used. Although examples of criteria will be given below, these may vary widely. But the insight is the possibility to devise such criteria, and to use them to select an element type, and its position, from a plurality of element types automatically, and finally to draw the map automatically.
- the method further comprises outputting the map to an output device.
- the output devices may be of any known kind, such as visual output means, e.g. a computer screen, in particular a desktop or laptop computer, but preferably also personal mobile devices such as a smart phone or the like. It may also be physical output means, e.g. a printer or plotter or the like. It is not necessary, though, to output the map. It is also possible to just create the map in e.g. a computer memory and further use it in software, such as for example analysing animal behaviour or the use of devices in the barn.
- the step of applying the sets of criteria to the position data to thereby create the map comprises deriving the type and position of at least a second barn element of a second type.
- Such embodiments use the invention advantageously in that they allow to discern between different types of barn elements. This special insight is helpful in letting the animals do the work. Note that there is a conceptual difference between, on the one hand, determining whether or not a barn element of one certain type is present or not, and if yes, determining its position, and on the other hand determining which type of barn element relates to the collected position data, assigning the correct type to (part of) the data, and determining the position connected to the determined barn element.
- This insight has been implemented in the first set of criteria, i.e. the set that is used to determine the type of barn element. Again, the insight is that it now becomes feasible to do so automatically. Examples of such criteria will be given.
- the type of barn elements is selected from a cubicle, a feed fence/device, a gate, a walking path, a drinking trough/device, an animal body brush, a waiting area, a separation area, and an animal ID station.
- an animal ID station is any station, or device, at which an animal is automatically identified, i.e. its ID established.
- such an animal ID station is or comprises a milking robot or an individual feeding station.
- any other desired type may also be included, as long as it is possible and feasible to define one or more criteria for discerning such barn element from the other types. It is the combination of defining the barn element types and the criteria for discerning them that lies at the heart of the invention.
- the first set comprises an animal ID station criterion for determining the element type from animal position data.
- this comprises assigning the type of animal ID station to the animal position of an animal.
- this information is already sent to the managament system, often if not always at least for any device at that animal ID station, such as a milking robot or a feed dispenser or the like.
- the ID station is provided there in order to determine the animal's ID for setting individual values of the relevant device. Then, the type of ID station is certainly known, and can easily be coupled to the animal position of the identified animal.
- the animal's position at the time right before and/or after the identification may already be a sufficiently reliable position determination for the animal ID station. It is possible to increase the accuracy by taking the average of a plurality of determined animal positions for a corresponding plurality of animal identifications, as the animals will likely move towards or from the animal ID station in a more or less random fashion. Even more accurate determinations become possible if the animal's position is determined when said animal is identified at said animal ID station.
- the animal position of an animal is actively determined upon identification thereof in an animal ID station.
- the second set comprises assigning the animal position to the animal ID station position.
- the first set comprises at least one movement criterion that is based on at least a movement of an animal, wherein the movement is determined as a time-dependent displacement, in particular in a straight line between two or more consecutively determined positions of said animal. It provides advantages if not only position data are used but also information on movement of animals. Such movement may be determined by looking at consecutive animal positions. Analysing such a path may provide useful clues for distinguishing various barn elements from one another, as will be exemplified below. For example, it is possible thereby to determine a path followed by the animal, or an area in which many consecutive position are located, as a kind of favourite location.
- speed may be determined, and in a first approximation as the linear displacement between two consecutive animal positions, divided by the time therebetween. More refined models, including acceleration and deceleration or the like, may be used to determine speed more accurately. As an example, if an animal walks from a first position to a second position in a more or less straight line and with a more or less constant speed, the presence of barn elements other than a walking area or path between the first and second position is not very likely. If the same holds for more animals when measured (approximately) between those two positions, it may be safe to assume that there is indeed a walking area or path between the two positions. In any case, using not only positions per se, but analysing the positions in time, including speed as derived therefrom, expands the gambit of possible criteria for use in distinguishing barn element types.
- said movement criterion comprises a cubicle criterion.
- a cubicle criterion is used to distinguish a cubicle in the position data collected. Cubicles are relevant in animal management because they are an indication for the time that an animal spends ruminating, which is in turn an indication for the future milk production. Therefore, such a cubicle criterion is important to have. However, it is noted that a cubicle is a barn element poor in features, and thus relatively difficult to characterise.
- said cubicle criterion comprises at least one of: determining that an animal position remains at a prospect position within a predetermined margin during at least a predetermined first period of time, wherein said margin is at most a half length of said animal, and determining that said animal movement comprises a movement in a first direction, followed by a subsequent animal movement in a second direction substantially perpendicular to said first direction, followed by an animal position at a substantially constant prospect position during at least a predetermined first period of time, followed by a animal movement substantially opposite the second direction, followed by a animal movement substantially perpendicular to the second direction, and wherein the second set comprises assigning a cubicle position to said prospect position.
- said predetermine d first period of time is animal type dependent, but for cows, a period of 30 minutes, more in particular 60 minutes, is a safe threshold for distinguishing rumination, when lying down, from other activities.
- said period of time is at least one hour. Note that the basic insight used here is that a dairy animal will spend periods of time of at least a predetermined length almost exclusively for ruminating, in particular if cubicles are available. For almost any other activity, she will change her position much quicker.
- an animal's position, the prospect position remains substantially the same, such as in particular within half the animal's length or within a certain predetermined area, for at least a predetermined period of time, and then assigning a cubicle position to said prospect position.
- Some of the embodiments mentioned above further take into account that cubicles are almost always provided in a side-by-side row, with a walking alley in front of such a row. Then, a dairy animal will almost always walk along such a path, and then enter a cubicle. This will be done by making a right-angle turn and lying down during a period of time. Then, for leaving the cubicle, the animal will have to stand up, walk backwards, unless the cubicle is very wide, make another right-angle turn and walk along the path.
- a feed fence where animals may stand for a while, but where the animals may choose to walk directly from one position to another, without having to go backwards and then via a walking alley or the like.
- a group of milk boxes also side by side, should be distinguished. The latter however, will have ID stations in most cases.
- the method further comprises detecting a sound selected from eating sounds, rumination sounds and drinking sounds, by means of a sound sensor worn by at least one of the animals, wherein at least one of the criteria is dependent on the detection of at least one of said sounds.
- at least one criterion may comprise distinguishing the type of action performed by the animal, such as drinking when a drinking noise is detected, and feeding when a feeding noise is detected, and ruminating when rumination sounds are detected.
- the at least one criterion may then also comprise distinguishing between a feeding station/feed fence or the like, a drinking station/trough, and a cubicle, respectively, and of course assigning the relevant measured animal position to the distinguished station/device.
- Devices for determining sounds from animals and assigning these sounds to certain activities are for example disclosed in WO00/13393 and WO02/07644 , and marketed by e.g. Lely Industries as the QWES-HR tag.
- a more elaborate criterion, or set of criteria, for a feed fence may comprise the movement of an animal in a first direction, a substantially right angle turn, followed by a displacement over a short distance, followed by a stay during a period of time that is usually associated with eating of the animal. This period may be determined when the above mentioned eating sounds sensor or arrangement is provided. The period could then be followed by a moving away over a short distance, which however need not be backwards, but could comprise a simple quarter turn or the like, for easy distinguishing from a cubicle. Then, the animal could continue its path in the barn in any direction.
- the various positions are more or less randomly distributed, and there may be traffic between the various positions along a border of the collection of these positions, unlike cubicle positions.
- the position data are used to assign a collected animal position to a barn element, or at least derive a position of the barn element from the collected animal position data, with the help of the second set of criteria.
- the second set comprises assigning to the position of a barn element a position that is based on at least one of the animal positions that were used in positively assigning an element type to said barn element. This is one of the easiest ways to determine a barn element position from the animal position data.
- the position of an animal ID station can be determined by assigning an (average) animal position upon identifying that animal at the animal ID station.
- positions and/or movements of animals may also be used in a different way.
- the way in which animals approach the barn element For example a milking station's position is taken as the (average) position of the animal upon identification at the milking station's ID station.
- the milking station or milking robot or milking box
- the box has a certain spatial extent which is considerable.
- the invention also relates to a system configured to automatically create a digital map of an animal barn with a plurality of types of barn elements, according to the method of the invention, the system comprising an animal location system, configured to determine position data of at least one animal in said barn, a computer system configured to: automatically collect said position data as a function of time, characterized in that the computer system is further configured to: store a first set of criteria for correlating said position data to said type of barn element and a second set of criteria for determining the position of said type of barn element, apply the sets of criteria to the position data, to thereby derive the type and position of at least a first barn element of a first type, and automatically create the map with at least said first barn element and the type and position thereof.
- the system may be a stand alone system, or be integrated with an animal management system or the like. Furthermore, it is remarked here that the advantages as discussed for the method according to the invention largely hold as well for the system, and need not be repeated here in all detail.
- the system further comprises an output device, wherein the computer system is configured to output the map to the output device.
- the output device often is a screen, of a computer, cell phone or the like, or a printer, and so on.
- the output device may be operatively connected to said computer system and configured to display said map, e.g. for direct use by a user such as a veterinarian.
- the animal location system may comprise at least one position tag worn by said at least one animals and a receiver device arranged to detect wirelessly the position of said tag in the barn.
- a combination of animal worn tags and a receiver device serves as a kind of GPS in the barn, as all animals with such a tag can be positioned. It could also be a true GPS system, such as disclosed in US2012182145 , or any other known animal locator system.
- the tag may be, or may be combined, with the animal ID tag, that is used for identification at e.g. a milking station.
- the animal location system may also advantageously comprise at least one autonomous vehicle that either comprises a position tag or comprises a dedicated positioning system arranged to communicate with the animal location system.
- the position information obtained by locating the vehicle can then simply be used for mapping.
- a vehicle could be a manure displacer, such as the Lely Discovery (TM), a feed displacer, such as the Lely Juno (TM) or any other such vehicle.
- TM Lely Discovery
- TM Lely Juno
- a vehicle can be arranged for following a wall, such as by simply keeping a constant distance to an object. In this way, it is even possible to map sidewalls or other large wall-like objects as barn elements, which are somewhat difficult to track by the animals, as they will often have no need to follow or stay close to a wall along a greater part of its length.
- the system further comprises an arrangement operatively coupled to the computer system and provided with a sound sensor for detecting animal sound, comprising at least one of eating sounds, rumination sounds and drinking sounds.
- a sound sensor for detecting animal sound, comprising at least one of eating sounds, rumination sounds and drinking sounds.
- a sound sensor for detecting animal sound, comprising at least one of eating sounds, rumination sounds and drinking sounds.
- TM Lely QWES-HR
- Such a sound sensor is advantageously provided on a neck tag, in particular on said position tag. There it is positioned suitable for determining the noises with the highest accuracy.
- At least one of said sound sensor, said arrangement and said computer system may comprise a sound analysis device for sound analysis, configured to analyse said detected animal sound.
- a barn for keeping animals and comprising a plurality of barn elements of a plurality of types, and comprising a system according to the invention.
- a barn for keeping animals and comprising a plurality of barn elements of a plurality of types, and comprising a system according to the invention.
- it is easy to digitise its lay-out, as the animals themselves will provide the necessary information, which is then processed by the system into a digital map.
- a new map is equally easily created, by having the animals redo the work. It is also interesting to check how the animals get used to a new barn lay-out.
- a computer program comprising computer program instructions adapted to perform the method of the invention when run on a computer. If there is already a barn with an animal location system, such a computer program may be run for drawing the map, according to the present invention. In this way, a lot of work can be saved, and the program can be used for existing barns. It is noted that the program may be accesible via a network such as the Internet.
- a computer readable medium having stored thereon computer program readable instructions adapted to perform the method of the invention when run on a computer. Such a computer readable medium may come in the form of a (hard) disc, a USB-stick, a CD or DVD, or other memory device. It is noted that in this computer program, as well as in the underlying method, one embodiment comprises automatically collecting animal position data, and then performing one or more times the steps of
- the position data are used step-by-step to determine the types and positions of as many (different) barn elements as possible.
- Figure 1 is a diagrammatic top view of a barn 1, with a system according to the invention.
- the barn 1 has sidewalls 2 enclosing the barn space.
- an animal location system shown here as a number (four) of antennae 3 for receiving signals from animals and the like in the barn.
- the antennae 3 are operatively connected to a control system (not shown here). Their number is of course arbitrary, and dependent on type of traveln system, size and shape of the barn, presence of interefering elements, and so on.
- barn elements such as rows 4a, 4b and 4c of cubicles 5, a walking path 6, a free area 7, a milking station 8, a cow brush 17, a drinking trough 18, a feed fence 20 and an exit gate 22.
- First of all the walls 2 may either be a given, or may be determined as the outer bounds of any position ever collected, if the barn is completely closed.
- the rows of cubicles 5 are used by ruminants to ruminate quietly. This means that ruminants 27, from here on taken to be cows, although any other type of ruminant such as sheep or goats would do as well, are present. These cows 27 will lie down in the cubicles 5 after entering same from the walking path 6 or the free area 7. Their position will then remain constant for quite some time, and by taking a predetermined time threshold to be for example 1 hour, a stay of more than the time threshold will be a criterion for assigning a cubicle to a position.
- the walking path 6 will predominantly show cow traffic such as from cow 26, in that there will be a large part of positions that can be detected to be part of walking, i.e. a regular displacement with a substantially constant speed and largely all in the same direction.
- cows are relatively efficient walkers, that will not often stray unnecessarily.
- the free area 7 will show more different directions, as there are more different destinations to be reached from the free area 7 e.g. by the cow 23.
- she wears a neck collar 24 with a tag 25. It is this tag 25 that is detected by the antennae 3 of the animal location system.
- the tag 25 may also comprise an arrangement/sound sensor for detecting rumination, eating and/or drinking sounds.
- the milking station 8 with a milking box 9, a milking robot 10 and an animal identification (ID) device 11.
- the milking box is accessible along a guide 12, via entrance gate 13, and exit gate 14.
- Reference numeral 15 indicates a selection gate for guiding animals, such as cows 29 and 30, that need to be separated into a separation area 16, from which animals can only leave when the farmer lets them out.
- the milking station shows a queue of cows 28 waiting to be milked, with a slowly changing position, towards a position in an animal ID station.
- the ID device 11 will identify a cow, last time cow 29. When the cow is identified, there is a coupling between the type of barn element, which is clearly the ID station 11 belonging to the milking station 8, and a specific cow 29 with its position.
- cow brush 17 is a device that may be approached by a cow 31 from various directions, is then used during a period of time of, say, several minutes, and then left in any desired direction. This may reflect in the "flocking" of cow positions with various approach and leaving routes.
- the drinking trough 18, with a tap 19, is used by cow 22.
- the sound sensor in the tag may detect drinking sounds. If these are detected, the position can be determined actively, or later on there may be made a connection in the data set between the time of detection of the drinking sounds and a position of said cow 22 at or around such time.
- the feed fence 20 is a special barn element, in that it forms a barrier for the cows, in this case cows 33, although it is still inside the barn, with a feed area 21 behind it, only accessible to the farmer.
- cows approach the feed fence from one side only, in a less orderly fashion than the rows of cubicles.
- a gate 22 that can sway open in the direction of the arrows, for letting cow 3 pass to a grazing area behind the gate. This can be detected by a cow position signal dissappearing from the barn/system, and lateron reappearing.
- an animal ID station for identification of the animal, for example to see if grazing is allowed for this animal.
- any other barn element such as the cow brush, the drinking trough, the feed fence, and even the cubicles. In such a case, identification becomes very quick and very precise.
- identification becomes very quick and very precise.
- Such complex and expensive system is however not necessary with the present invention.
- FIG. 2 there is provided a diagram with animal positions during a predetermined period of time. Let us assume for the moment that this period of time either fulfils the criterion of a predetermined measuring time, and or that it has been established that all barn elements have been distinguished with sufficient certainty or likelihood.
- a possible parameter for determining this is the density of animal position data, or more precisely the difference in data density between parts of the total area measured. Generally speaking, if a certain area element has a certain data density (a certain number of positions per area), and that density is at least a certain factor higher than the density in the vicinity of that area element, there is a good chance that the data and the one or more criteria supplied suffice to determine the barn element type with sufficient reliability. Other ways to determine this may apply.
- clusters More of these clusters have been indicated by 44 and 45. For clarity, any clusters between 44 and 45 have not been indicated, but may be assumed present. It is remarked that not every cubicle need be taken by a cow during the measuring time, due to preferences and habits. Nevertheless, detecting a row of clusters like 40, 41, 42, 44 and 45 is definitely a clear clue for a row of cubicles.
- At 47 there has been indicated a number of cow positions for different cows, that have been detected at an animal ID station belonging to the milking station. Thereby, it is easy to determine that the (average of the) positions 47 can be assigned to the ID station of that milking station.
- the positions 47 may have been determined on cue, from a signal from the ID station, or as a position determined nearest in time to the time of identification.
- numeral 46 there has been indicated a kind of waiting area or queue. The average distance between consecutive positions of the same animal, or in other words the speed of those animals, in the vicinity of 46 is low, indicating waiting.
- the area covered by those positions indicates the separation area.
- an indication from the management system, or at least the milking robot controller that indicates when a cow is to be separated. It is thus also possible to determine with high reliability the separation area by measuring the range of positions for cows that have been separated as determined by the milking robot controller.
- a long stretched cloud of positions indicates a lively visiting at numerous positions, with movements directly between such positions possible.
- the time spent can be measured, as well as the approaches and the leaving. All this, together with the long and straight shape of the cloud 52, indicates that this is the feed fence. If sound sensors are provided, eating sounds may be detected, which further increase the reliability of the detection.
- Figure 3 shows a flow diagram of a computer program and method of the present invention.
- the first step is a start-up step 100.
- the sets of criteria may be entered, together with the dimensions of the barn, if desired, and other start data. These could also be as few as a label for the barn for which a map is to be drawn.
- the animal position data is collected, by having the location system determine lots of animal positions as a function of time.
- the criteria are applied to the position data, as explained in all of the above. Here, it is tried to find correlations in the animal position data to enable a barn element to be distinguished.
- step 120 it is checked if a barn element can actually be distinguished. If no, then more position data are needed, and will be collected at step 140, after which the criteria are applied anew. If yes, the map is updated with the distinguished barn element. It is then checked if the program is ready, such as "Are all barn elements located?". If ready, the map is output, ending the program. If not, steps 120 through 160 are repeated.
Landscapes
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Animal Husbandry (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biophysics (AREA)
- Housing For Livestock And Birds (AREA)
Description
- The present invention relates to a method of automatically creating a digital map of an animal barn, wherein the animal barn comprises a plurality of types of barn elements, a plurality of animals in said barn, and an animal locating system that is configured to determine a position of at least one of said animals in said barn. The invention further relates to a system configured to automatically create a digital wap of an animal barn with a plurality of types of barn elements. (See e.g.
WO2011/055004 A1 ) - In the practice of animal husbandry, it is advantageous to have available a digital map of a barn, i.e. a walled space in which animals are free to walk. E.g. if a farmer desires to know the whereabouts of a certain individual animal, such as to give the animal a particular treatment or to fetch the animal for some reason, such a map can be helpful to him, especially if available in digital form. Also, to learn about animal behaviour, it is deemed of interest to know where individual animals are during their stay in the barn. A more general advantage lies in the possibility to use the map in farm or animal management.
- To obtain such a map, in the art there have been disclosures of systems to generate a map based on position measurements. However, these all require human input. For example, a most straightforward solution, which is however at the same time very labour-intensive, is to draw a digital map on a computer, by means of a drawing program and a paper map. There have been devised ways to try and automate this. For example, patent document
WO2011/055005 generally deals with monitoring the path of an animal or human in the home by means of a locating system. In the document, a method is disclosed for drawing a digital map. Thereto a person provided with a device by which his position can be monitored walks around through the home. In each corner of each room, the person's position is registered, e.g. after pressing a button in the monitoring device. Then, on the basis of the registered positions, some computer that is not further described generates a floor plan. Lateron, a human user may also enter furniture and the like in the map. - This known method, and device, has a drawback in that it requires a significant amount of human work.
- It is therefore an object of the present invention to provide a method of drawing such map with minimal human intervention.
- This object is achieved with a method according to claim 1, in particular a method of automatically creating a digital map of an animal barn, wherein the animal barn comprises a plurality of types of barn elements, a plurality of animals in said barn, and an animal locating system that is configured to determine a position of at least one of said animals in said barn, the method comprising the steps of automatically collecting position data with the animal locating system about the position of said at least one animal in said barn as a function of time, providing, for at least a plurality of types of barn element, a first set of criteria for correlating said position data to said type of barn element and a second set of criteria for determining the position of said type of barn element, automatically applying the sets of criteria to the position data, to thereby derive the type and position of at least a first barn element of a first type, and automatically creating the map with at least said first barn element and the type and position thereof.
- The invention is based at least in part on the insight that it is possible to provide criteria to determine an element type from animal position data, more in particular to provide criteria for automatically determining the element type from a plurality of element types, on the basis of position data. Thereby, it becomes feasible possible to automatically position that element in a (virtual) barn, and subsequently draw a map based on the data collected and processed. This will greatly help e.g. a farmer in obtaining a map of his barn. Such a map he may use for managing the animals in that barn. For example, it may help him when looking for an animal to be treated. Indicating the animal on the map will shorten the time needed for him to find the animal. Furthermore, only after having such a map in the first place may the farmer obtain a real insight in the behaviour of the animals in the barn.
- Note that even when the map is not complete, in that one or more barn elements have not been found or derived from the data, the farmer will still have a good starting point for the map. This will still save him a lot of work. Of course, a big advantage herein is that the animals do the work for the farmer, simply by following their daily routine. In the present invention, the barn elements may be of many types, such as a cubicle, a feed fence, a milking robot, a gates, a walking path, or any other type of barn element that has some interaction with the animal, from which interaction the type of element and its position may be determined with a reasonable, predetermined certainty. Obviously, light switches, tap water connections and the like are not types of barn element to be determined here.
- Furthermore, it will be clear that the invention relates to using the position data of non-human animals. It is noted that it is known to use human work, in the form of a person walking around with a location sensor, for making a map of a room, building, office or the like. However, this known method requires a lot of human work, while the present invention is aimed at preventing, or at least significantly reducing, that human work. Thereto, it uses the animals in the barn themselves. Note furthermore that it suffices if the positions in time of at least one animal are determined, as data from one animal would in principle suffice. However, using the position data from more animals, or even all animals in the barn, will likely lead to more data and thus a quicker and/or more reliable determination of type and position of barn elements. Furthermore, in some cases animals have habits that prevent them from using e.g. certain cubicles or parts of a feed fence, which would thus remain "invisible" or undetected. It should also be realised that it is sometimes rather expensive to collect position data from all (or at least many) animals, as this may sometimes require tagging the animals with dedicated tags.
- In the present method, the position data about the position of said at least one animal in said barn are collected as a function of time. This collecting takes place during a predetermined period of time, or until the position data are determined to be sufficient, i.e. fulfil a predetermined sufficiency criterion. For example, the data are collected during two or three days, or any other period of time in which the herd will likely have exhibited its normal behaviour. In many cases, the predetermined period of time may be dependent on the number of animals for which data are collected, as a higher number will likely generate more data per unit time. Alternatively or additionally, the position data may be collected until a sufficiency criterion has been fulfilled. For example, this may relate to the likelihood with which a barn element has been determined from the data, such as the determination with a 95% likelihood that a feed fence is present at a certain position. In such cases, it would not make much sense to keep on collecting data, which would require resources unnecessarily. Furthermore, in some cases it may happen that, by some coincidence in the animals' behaviour, the specific data collected enable a quick determination, while in other cases more data are required. Moreover, the predetermined period of time and/or the sufficiency criterion may be dependent on the type of barn element. For example, in the straightforward case that a certain barn element has its own ID (identification) station, determining its position in the barn requires not more than one animal position datum. Other elements may require a large number of data before a reliable determination can be obtained.
- Another remark is that automatically drawing the map per se is deemed well-known to the skilled person. Thereto, there will be provided some computer program in the computer system that places the barn elements that have been derived from the animal position data, together with the positions of those barn elements, in their relative positions, i.e. in a map. This map will now become available for further use, although it is not necessary to output it, as it may also be used for further processing for example animal data. This will elucidated further below.
- It is also noted that the first set of criteria need not comprise more than one criterion. A simple example would be that only barn elements with an ID station are determined, such as a milking station, a concentrate feeding station and a selection gate. Then, the only criterion would relate to e.g. determining a position at the time of the ID determining, and coupling the position with the type of ID station. Of course, in many other cases there may be more than one criterion for determining what type of barn element is positioned where. Herein, the criteria for determining the type of barn element and/or the criteria for determining its position in the barn are based on the animal position data. Finally, it is noted that the method need not succeed in actually determining the position of more than one barn element, as sometimes the data will be too ambiguous or sparse. However, it is to be noted that the invention relies on the possibility to automatically determine a type of barn element from a plurality of possible barn elements, based on animal position data. Herein, it is relatively arbitrary what exact criteria are used. Although examples of criteria will be given below, these may vary widely. But the insight is the possibility to devise such criteria, and to use them to select an element type, and its position, from a plurality of element types automatically, and finally to draw the map automatically. Advantageous and special embodiments will be described in the dependent claims, as well as in the description that follows.
- In embodiments, the method further comprises outputting the map to an output device. In this way the farmer, veterinarian or other user can check positions and the like on the output device, in order to use the displayed (positional) information for work. The output devices may be of any known kind, such as visual output means, e.g. a computer screen, in particular a desktop or laptop computer, but preferably also personal mobile devices such as a smart phone or the like. It may also be physical output means, e.g. a printer or plotter or the like. It is not necessary, though, to output the map. It is also possible to just create the map in e.g. a computer memory and further use it in software, such as for example analysing animal behaviour or the use of devices in the barn. In such cases, outputting the map would not always serve a purpose and could be left out. In providing the map, all of its outline and elements may be drawn to scale. However, it is also possible to draw the outer dimensions of the barn as well as all positions of barn elements in the barn to scale, while the elements themselves are shown with exagerated dimensions, for more clarity. Other ways to show the map are not excluded, such as using pictograms.
- In particular, the step of applying the sets of criteria to the position data to thereby create the map comprises deriving the type and position of at least a second barn element of a second type. Such embodiments use the invention advantageously in that they allow to discern between different types of barn elements. This special insight is helpful in letting the animals do the work. Note that there is a conceptual difference between, on the one hand, determining whether or not a barn element of one certain type is present or not, and if yes, determining its position, and on the other hand determining which type of barn element relates to the collected position data, assigning the correct type to (part of) the data, and determining the position connected to the determined barn element. This insight has been implemented in the first set of criteria, i.e. the set that is used to determine the type of barn element. Again, the insight is that it now becomes feasible to do so automatically. Examples of such criteria will be given.
- In certain embodiments, the type of barn elements is selected from a cubicle, a feed fence/device, a gate, a walking path, a drinking trough/device, an animal body brush, a waiting area, a separation area, and an animal ID station. Such an animal ID station is any station, or device, at which an animal is automatically identified, i.e. its ID established. In particular, such an animal ID station is or comprises a milking robot or an individual feeding station. With the above types of barn elements, it is possible to cover relevant types of barn element, with respect to animal management, with some of the most relevant being the milking station(s), the feed fence/device and the cubicles, as they relate directly to the most important managerial aspect, viz. milking. In addition to the above mentioned types of barn elements, any other desired type may also be included, as long as it is possible and feasible to define one or more criteria for discerning such barn element from the other types. It is the combination of defining the barn element types and the criteria for discerning them that lies at the heart of the invention.
- In certain embodiments, the first set comprises an animal ID station criterion for determining the element type from animal position data. In particular this comprises assigning the type of animal ID station to the animal position of an animal. When an animal is identified at an animal ID station, this information is already sent to the managament system, often if not always at least for any device at that animal ID station, such as a milking robot or a feed dispenser or the like. After all, the ID station is provided there in order to determine the animal's ID for setting individual values of the relevant device. Then, the type of ID station is certainly known, and can easily be coupled to the animal position of the identified animal. If that animal's position is determined frequently enough, such as every ten seconds or the like, the animal's position at the time right before and/or after the identification may already be a sufficiently reliable position determination for the animal ID station. It is possible to increase the accuracy by taking the average of a plurality of determined animal positions for a corresponding plurality of animal identifications, as the animals will likely move towards or from the animal ID station in a more or less random fashion. Even more accurate determinations become possible if the animal's position is determined when said animal is identified at said animal ID station. Herein, one could say that the animal position of an animal is actively determined upon identification thereof in an animal ID station. It follows that, in embodiments, the second set comprises assigning the animal position to the animal ID station position.
- In embodiments, the first set comprises at least one movement criterion that is based on at least a movement of an animal, wherein the movement is determined as a time-dependent displacement, in particular in a straight line between two or more consecutively determined positions of said animal. It provides advantages if not only position data are used but also information on movement of animals. Such movement may be determined by looking at consecutive animal positions. Analysing such a path may provide useful clues for distinguishing various barn elements from one another, as will be exemplified below. For example, it is possible thereby to determine a path followed by the animal, or an area in which many consecutive position are located, as a kind of favourite location. Also, speed may be determined, and in a first approximation as the linear displacement between two consecutive animal positions, divided by the time therebetween. More refined models, including acceleration and deceleration or the like, may be used to determine speed more accurately. As an example, if an animal walks from a first position to a second position in a more or less straight line and with a more or less constant speed, the presence of barn elements other than a walking area or path between the first and second position is not very likely. If the same holds for more animals when measured (approximately) between those two positions, it may be safe to assume that there is indeed a walking area or path between the two positions. In any case, using not only positions per se, but analysing the positions in time, including speed as derived therefrom, expands the gambit of possible criteria for use in distinguishing barn element types.
- In exemplary embodiments, said movement criterion comprises a cubicle criterion. Herein, a cubicle criterion is used to distinguish a cubicle in the position data collected. Cubicles are relevant in animal management because they are an indication for the time that an animal spends ruminating, which is in turn an indication for the future milk production. Therefore, such a cubicle criterion is important to have. However, it is noted that a cubicle is a barn element poor in features, and thus relatively difficult to characterise. Nevertheless, in exemplary modiments, said cubicle criterion comprises at least one of: determining that an animal position remains at a prospect position within a predetermined margin during at least a predetermined first period of time, wherein said margin is at most a half length of said animal, and determining that said animal movement comprises a movement in a first direction, followed by a subsequent animal movement in a second direction substantially perpendicular to said first direction, followed by an animal position at a substantially constant prospect position during at least a predetermined first period of time, followed by a animal movement substantially opposite the second direction, followed by a animal movement substantially perpendicular to the second direction,
and wherein the second set comprises assigning a cubicle position to said prospect position. In particular, said predetermine d first period of time is animal type dependent, but for cows, a period of 30 minutes, more in particular 60 minutes, is a safe threshold for distinguishing rumination, when lying down, from other activities. In particular, said period of time is at least one hour. Note that the basic insight used here is that a dairy animal will spend periods of time of at least a predetermined length almost exclusively for ruminating, in particular if cubicles are available. For almost any other activity, she will change her position much quicker. Therefore, it would in principle suffice to determine that an animal's position, the prospect position, remains substantially the same, such as in particular within half the animal's length or within a certain predetermined area, for at least a predetermined period of time, and then assigning a cubicle position to said prospect position. Some of the embodiments mentioned above further take into account that cubicles are almost always provided in a side-by-side row, with a walking alley in front of such a row. Then, a dairy animal will almost always walk along such a path, and then enter a cubicle. This will be done by making a right-angle turn and lying down during a period of time. Then, for leaving the cubicle, the animal will have to stand up, walk backwards, unless the cubicle is very wide, make another right-angle turn and walk along the path. - The above embodiments show that it is feasible to define criteria for distinguishing a cubicle from other barn elements. Nevertheless, other criteria may be used, as alternatives or additions. As a very general remark here, it is possible to enter boundary conditions, such as the total dimensions of the barn, the number of animal ID stations and the number of cubicles, although these are, in all cases shown, possible to determine from animal position data alone. However, having such data beforehand allows to determine cubicles in another way, such as by determining one or more groups of positions/areas in which animals are present individually, that is one at most, during prolonged periods of time, while no traffic is possible between such positions/areas directly. This is a clear difference to a walking area or e.g. a feed fence, where animals may stand for a while, but where the animals may choose to walk directly from one position to another, without having to go backwards and then via a walking alley or the like. It is to be noted, however, that for example a group of milk boxes, also side by side, should be distinguished. The latter however, will have ID stations in most cases.
- In embodiments, the method further comprises detecting a sound selected from eating sounds, rumination sounds and drinking sounds, by means of a sound sensor worn by at least one of the animals, wherein at least one of the criteria is dependent on the detection of at least one of said sounds. Then, at least one criterion may comprise distinguishing the type of action performed by the animal, such as drinking when a drinking noise is detected, and feeding when a feeding noise is detected, and ruminating when rumination sounds are detected. The at least one criterion may then also comprise distinguishing between a feeding station/feed fence or the like, a drinking station/trough, and a cubicle, respectively, and of course assigning the relevant measured animal position to the distinguished station/device. Devices for determining sounds from animals and assigning these sounds to certain activities are for example disclosed in
WO00/13393 WO02/07644 - A more elaborate criterion, or set of criteria, for a feed fence may comprise the movement of an animal in a first direction, a substantially right angle turn, followed by a displacement over a short distance, followed by a stay during a period of time that is usually associated with eating of the animal. This period may be determined when the above mentioned eating sounds sensor or arrangement is provided. The period could then be followed by a moving away over a short distance, which however need not be backwards, but could comprise a simple quarter turn or the like, for easy distinguishing from a cubicle. Then, the animal could continue its path in the barn in any direction. Herein, the various positions are more or less randomly distributed, and there may be traffic between the various positions along a border of the collection of these positions, unlike cubicle positions.
- In all of the above, many examples have been given how to discern various types of elements in a barn from animal position data. The skilled person is thus enabled to derive the position of all the barn elements for which he can design suitable selection criteria, which criteria are then put into the automated method according to the invention. It is important to point out that the very thought that it is possible and feasible to automatically distinguish barn elements from animal position data which lies at the heart of the invention. And even if not all barn elements may be located and positioned on the map, there is still provided a way to save a lot of work by automatically performing this arduous task for a lot of barn elements.
- As already indicated in the previous part of the description, the position data are used to assign a collected animal position to a barn element, or at least derive a position of the barn element from the collected animal position data, with the help of the second set of criteria. In embodiments, the second set comprises assigning to the position of a barn element a position that is based on at least one of the animal positions that were used in positively assigning an element type to said barn element. This is one of the easiest ways to determine a barn element position from the animal position data. For example, as has already been indicated in the above, the position of an animal ID station can be determined by assigning an (average) animal position upon identifying that animal at the animal ID station. It is also possible to use a plurality of the animal positions that were used in positively assigning an element type to said barn element. For example, taking an average of the animal positions upon their respective identification at said animal ID station gives a more accurate position of that animal ID station. In addition, and importantly, it is to be realised that most barn elements have a certain size and spatial extent. Therefore, although the position of a barn element can be approximated by a single point, or animal position or the like, it may be more instructive to know the spatial extent of a barn element, up to a certain extent. This may include dimension, directions et cetera.
- Additionally or alternatively, positions and/or movements of animals, or more generally the animal position data, may also be used in a different way. In order to determine the orientation and spatial extent, it is also possible to use the way in which animals approach the barn element. For example a milking station's position is taken as the (average) position of the animal upon identification at the milking station's ID station. However, the milking station (or milking robot or milking box) has a certain spatial extent which is considerable. By realising that there may be queues of waiting animals at the entrance gate, but not normally any waiting animals after the exit gate, one can use data where animals queue up in the vicinity of the milking station as the position of the entrance gate, and position data where animals only walk away from the milking station as the exit gate. With this information, the box may be oriented and positioned more accurately. Similar considerations may hold for other barn elements, and will be contemplatable for the skilled person.
- The invention also relates to a system configured to automatically create a digital map of an animal barn with a plurality of types of barn elements, according to the method of the invention, the system comprising an animal location system, configured to determine position data of at least one animal in said barn, a computer system configured to: automatically collect said position data as a function of time, characterized in that the computer system is further configured to: store a first set of criteria for correlating said position data to said type of barn element and a second set of criteria for determining the position of said type of barn element, apply the sets of criteria to the position data, to thereby derive the type and position of at least a first barn element of a first type, and automatically create the map with at least said first barn element and the type and position thereof. The system may be a stand alone system, or be integrated with an animal management system or the like. Furthermore, it is remarked here that the advantages as discussed for the method according to the invention largely hold as well for the system, and need not be repeated here in all detail.
- In embodiments, the system further comprises an output device, wherein the computer system is configured to output the map to the output device. As mentioned before, such output device often is a screen, of a computer, cell phone or the like, or a printer, and so on. Advantageously, the output device may be operatively connected to said computer system and configured to display said map, e.g. for direct use by a user such as a veterinarian.
- The animal location system may comprise at least one position tag worn by said at least one animals and a receiver device arranged to detect wirelessly the position of said tag in the barn. Such a combination of animal worn tags and a receiver device serves as a kind of GPS in the barn, as all animals with such a tag can be positioned. It could also be a true GPS system, such as disclosed in
US2012182145 , or any other known animal locator system. The tag may be, or may be combined, with the animal ID tag, that is used for identification at e.g. a milking station. - The animal location system may also advantageously comprise at least one autonomous vehicle that either comprises a position tag or comprises a dedicated positioning system arranged to communicate with the animal location system. The position information obtained by locating the vehicle can then simply be used for mapping. Such a vehicle could be a manure displacer, such as the Lely Discovery (TM), a feed displacer, such as the Lely Juno (TM) or any other such vehicle. Advantageously, such a vehicle can be arranged for following a wall, such as by simply keeping a constant distance to an object. In this way, it is even possible to map sidewalls or other large wall-like objects as barn elements, which are somewhat difficult to track by the animals, as they will often have no need to follow or stay close to a wall along a greater part of its length.
- In embodiments, the system further comprises an arrangement operatively coupled to the computer system and provided with a sound sensor for detecting animal sound, comprising at least one of eating sounds, rumination sounds and drinking sounds. As mentioned before, such sound sensor and arrangement may be helpful in determining the barn element, by providing additional information about the type of activity performed by the animal at its location. An example of such a sound sensor is the rumination sound sensor of the Lely QWES-HR (TM). Such a sound sensor is advantageously provided on a neck tag, in particular on said position tag. There it is positioned suitable for determining the noises with the highest accuracy.
- Although it is not particularly limiting where the analysis of the sound is performed, at least one of said sound sensor, said arrangement and said computer system may comprise a sound analysis device for sound analysis, configured to analyse said detected animal sound.
- Herein is also disclosed a barn for keeping animals and comprising a plurality of barn elements of a plurality of types, and comprising a system according to the invention. In such a barn, it is easy to digitise its lay-out, as the animals themselves will provide the necessary information, which is then processed by the system into a digital map. Furthermore, when rearranging the barn's lay-out, a new map is equally easily created, by having the animals redo the work. It is also interesting to check how the animals get used to a new barn lay-out.
- Herein is also disclosed a computer program comprising computer program instructions adapted to perform the method of the invention when run on a computer. If there is already a barn with an animal location system, such a computer program may be run for drawing the map, according to the present invention. In this way, a lot of work can be saved, and the program can be used for existing barns. It is noted that the program may be accesible via a network such as the Internet. Herein is disclosed a computer readable medium having stored thereon computer program readable instructions adapted to perform the method of the invention when run on a computer. Such a computer readable medium may come in the form of a (hard) disc, a USB-stick, a CD or DVD, or other memory device. It is noted that in this computer program, as well as in the underlying method, one embodiment comprises automatically collecting animal position data, and then performing one or more times the steps of
- applying one or more criteria for correlating said position data to a first type of barn element,
- if successful, determine the presence of a barn element of said first type of barn element
- applying one or more criteria for determining the position of said first type of barn element
- automatically updating the map with at least said barn element and said first type and the position thereof.
- In this way, the position data are used step-by-step to determine the types and positions of as many (different) barn elements as possible.
- The invention will now be explained further with reference to non-limiting embodiments, as shown and described in the drawings, in which
-
Figure 1 is a diagrammatic top view of a barn, with a system according to the invention; -
Figure 2 is a diagram with animal positions during a predetermined period of time; and -
Figure 3 shows a flow diagram of a computer program and method of the present invention. -
Figure 1 is a diagrammatic top view of a barn 1, with a system according to the invention. - The barn 1 has sidewalls 2 enclosing the barn space. In the barn, there is provided an animal location system, shown here as a number (four) of
antennae 3 for receiving signals from animals and the like in the barn. Theantennae 3 are operatively connected to a control system (not shown here). Their number is of course arbitrary, and dependent on type of locatin system, size and shape of the barn, presence of interefering elements, and so on. - In the barn are also provided a number of barn elements, such as
rows walking path 6, afree area 7, a milkingstation 8, a cow brush 17, a drinking trough 18, afeed fence 20 and anexit gate 22. These will now be discussed in more detail. - First of all the
walls 2 may either be a given, or may be determined as the outer bounds of any position ever collected, if the barn is completely closed. The rows of cubicles 5 are used by ruminants to ruminate quietly. This means thatruminants 27, from here on taken to be cows, although any other type of ruminant such as sheep or goats would do as well, are present. Thesecows 27 will lie down in the cubicles 5 after entering same from the walkingpath 6 or thefree area 7. Their position will then remain constant for quite some time, and by taking a predetermined time threshold to be for example 1 hour, a stay of more than the time threshold will be a criterion for assigning a cubicle to a position. The walkingpath 6 will predominantly show cow traffic such as fromcow 26, in that there will be a large part of positions that can be detected to be part of walking, i.e. a regular displacement with a substantially constant speed and largely all in the same direction. Note herein that cows are relatively efficient walkers, that will not often stray unnecessarily. Slightly in contrast, thefree area 7 will show more different directions, as there are more different destinations to be reached from thefree area 7 e.g. by thecow 23. Note that only for thiscow 23, there is shown that she wears aneck collar 24 with atag 25. It is thistag 25 that is detected by theantennae 3 of the animal location system. Thetag 25 may also comprise an arrangement/sound sensor for detecting rumination, eating and/or drinking sounds. - One of these is the milking
station 8, with amilking box 9, a milkingrobot 10 and an animal identification (ID)device 11. The milking box is accessible along aguide 12, via entrance gate 13, and exit gate 14. Reference numeral 15 indicates a selection gate for guiding animals, such ascows 29 and 30, that need to be separated into aseparation area 16, from which animals can only leave when the farmer lets them out. The milking station shows a queue of cows 28 waiting to be milked, with a slowly changing position, towards a position in an animal ID station. There, theID device 11 will identify a cow, last time cow 29. When the cow is identified, there is a coupling between the type of barn element, which is clearly theID station 11 belonging to the milkingstation 8, and a specific cow 29 with its position. If that position is actively determined as soon as, or at least very close to the moment when, cow 29 is identified byID station 11, the position of that station, and thereby of the milkingstation 8 is known. From the positions of queue cows 28, a position of the entrance gate 13 may be derived, and thus by implication also the position of exit gate 14. If, directly after identification inID station 11, the position of the animal is tied to a certainsmall area 16 close to the milkingstation 8, thatarea 16 must be the separation area, with a separation gate 15 inbetween the milking station and the separation area. Note that the separation area is very often very close to the milking robot, although some corridor of any length might also lead to the separation area. - Furthermore, the cow brush 17 is a device that may be approached by a
cow 31 from various directions, is then used during a period of time of, say, several minutes, and then left in any desired direction. This may reflect in the "flocking" of cow positions with various approach and leaving routes. - The drinking trough 18, with a tap 19, is used by
cow 22. The sound sensor in the tag may detect drinking sounds. If these are detected, the position can be determined actively, or later on there may be made a connection in the data set between the time of detection of the drinking sounds and a position of saidcow 22 at or around such time. - The
feed fence 20 is a special barn element, in that it forms a barrier for the cows, in this case cows 33, although it is still inside the barn, with afeed area 21 behind it, only accessible to the farmer. Typical behaviour is that cows approach the feed fence from one side only, in a less orderly fashion than the rows of cubicles. There may be a similar approach along a walking path past thefeed fence 20, followed by a right angle turn. It is also possible that there is a more straight line approach. Furthermore, there may be a moving from one position at thefeed fence 20 to another in a straight line, without going backwards for a certain distance, as with cubicles 5. Certainly not least, it is possible to couple the cow position with the detectin of eating sounds, by means of the sound sensor in the tag. - Interestingly, if grazing is part of the animal management, there may be provided a
gate 22, that can sway open in the direction of the arrows, for lettingcow 3 pass to a grazing area behind the gate. This can be detected by a cow position signal dissappearing from the barn/system, and lateron reappearing. Note that, in general, it is possible for the present invention for there to be provided an animal ID station for identification of the animal, for example to see if grazing is allowed for this animal. The same may hold true for any other barn element, such as the cow brush, the drinking trough, the feed fence, and even the cubicles. In such a case, identification becomes very quick and very precise. Such complex and expensive system is however not necessary with the present invention. - In connection with
Figure 2 , it will now be explained how the various types of barn element may be distinguished from animal position data. These are given in the form of a diagram in which all animal positions during this measuring time period have been indicated, mostly as crosses, sometimes with different labels. - In
Figure 2 , there is provided a diagram with animal positions during a predetermined period of time. Let us assume for the moment that this period of time either fulfils the criterion of a predetermined measuring time, and or that it has been established that all barn elements have been distinguished with sufficient certainty or likelihood. A possible parameter for determining this is the density of animal position data, or more precisely the difference in data density between parts of the total area measured. Generally speaking, if a certain area element has a certain data density (a certain number of positions per area), and that density is at least a certain factor higher than the density in the vicinity of that area element, there is a good chance that the data and the one or more criteria supplied suffice to determine the barn element type with sufficient reliability. Other ways to determine this may apply. - To begin with the analysis of the animal position data, there are a number of sets of closely-spaced
positions groups clusters - More of these clusters have been indicated by 44 and 45. For clarity, any clusters between 44 and 45 have not been indicated, but may be assumed present. It is remarked that not every cubicle need be taken by a cow during the measuring time, due to preferences and habits. Nevertheless, detecting a row of clusters like 40, 41, 42, 44 and 45 is definitely a clear clue for a row of cubicles.
- At 47, there has been indicated a number of cow positions for different cows, that have been detected at an animal ID station belonging to the milking station. Thereby, it is easy to determine that the (average of the) positions 47 can be assigned to the ID station of that milking station. The
positions 47 may have been determined on cue, from a signal from the ID station, or as a position determined nearest in time to the time of identification.
Furthermore, withnumeral 46, there has been indicated a kind of waiting area or queue. The average distance between consecutive positions of the same animal, or in other words the speed of those animals, in the vicinity of 46 is low, indicating waiting. Contrarily, on the opposite side with respect to thecluster 47, there is a dispersion of positions. This indicates that the entrance gate must be at the 46 side, while thus the exit gate is at the opposite side. - At 49, there is indicated an area with a number of positions of only a few cows (here only one) that are no longer present in any other part of the barn, and which therefore seem separated. Thus, the area covered by those positions indicates the separation area. Of course, it is also possible to use an indication from the management system, or at least the milking robot controller, that indicates when a cow is to be separated. It is thus also possible to determine with high reliability the separation area by measuring the range of positions for cows that have been separated as determined by the milking robot controller.
- At 50, there is present a point cloud of positions, that have been approached from all sides, and can be left in all directions. The cloud indicates that the cows spend a certain amount of time at that position, but not above, say, a threshold for cubicles. No eating, rumination or drinking sounds are detected. Thus, it is assumed that this must be the cow brush area.
- At 51, there is also a cloud of positions, but here drinking sounds have been detected. Thus, this is assigned the drinking trough.
- At the positions indicated by 52, a long stretched cloud of positions indicates a lively visiting at numerous positions, with movements directly between such positions possible. The time spent can be measured, as well as the approaches and the leaving. All this, together with the long and straight shape of the
cloud 52, indicates that this is the feed fence. If sound sensors are provided, eating sounds may be detected, which further increase the reliability of the detection. - At 53, there is indicated a cloud of positions that lie outside the boundaries of the barn, either as entered beforehand or determined as absolute outer bounds for most of the area, assuming straight sidewalls. Furthermore, the determination of position stops there for longer periods of time, because the cow is out of reach of the antennae/the location system. Then it is assumed that there is a gate at 53 to a grazing area.
- In the above, many different barn elements have been indicated, as well as ways to distinguish these from animal position information. In case another barn element type is introduced, it will almost always be possible to connect animal behaviour, in particular their positions or movements, to that barn element, and distinguish it in the total position data. note furthermore that a lot of animal positions measured have not been indicated, to make the picture a lot clearer. In practice, there will be lots of positions that are only indicative of say
free area 7, if at all. -
Figure 3 shows a flow diagram of a computer program and method of the present invention. - The first step is a start-up
step 100. Here, for example the sets of criteria may be entered, together with the dimensions of the barn, if desired, and other start data. These could also be as few as a label for the barn for which a map is to be drawn. - Then, at 110 the animal position data is collected, by having the location system determine lots of animal positions as a function of time.
- In the next step at 120, the criteria are applied to the position data, as explained in all of the above. Here, it is tried to find correlations in the animal position data to enable a barn element to be distinguished.
- In the subsequent step, it is checked if a barn element can actually be distinguished. If no, then more position data are needed, and will be collected at
step 140, after which the criteria are applied anew. If yes, the map is updated with the distinguished barn element. It is then checked if the program is ready, such as "Are all barn elements located?". If ready, the map is output, ending the program. If not, steps 120 through 160 are repeated. - Of course, this is only a rough outline of the total program, and most of the work will be in designing the criteria and writing code to execute the criteria. Nevertheless, the program will be able to save a lot of work for the user, most often a farmer.
- Although the above embodiments have been described in detail, it is to be understood that these embodiments have been given for explanatory purposes only, and are not deemed to be limiting for the scope. Instead, the scope of the invention is determined by the claims as appended.
Claims (15)
- Method of automatically creating a digital map of an animal barn (1), wherein the animal barn comprises a plurality of types of barn elements (4a, 4b, 4c, 5, 6, 7, 8, 17, 18, 20, 22), a plurality of animals in said barn, and an animal locating system that is configured to determine a position of at least one of said animals in said barn, the method comprising the steps of- automatically collecting position data with the animal locating system about the position of said at least one animal in said barn as a function of time- providing, for at least a plurality of types of barn elements, a first set of criteria for correlating said position data to said type of barn element and a second set of criteria for determining the position of said type of barn element- automatically applying the sets of criteria to the position data, to thereby derive the type and position of at least a first barn element of a first type,- automatically creating the map with at least said first barn element and the type and position thereof.
- Method according to claim 1, further comprising outputting the map to an output device.
- Method according to any one of the preceding claims, wherein the step of applying the sets of criteria to the position data to thereby create the map comprises deriving the type and position of at least a second barn element of a second type.
- Method according to any one of the preceding claims, wherein the type of barn elements is selected from a cubicle (5), a feed fence/device (20), a gate (22), a walking path (6), a drinking trough/device (18), an animal body brush (17), a waiting area, a separation area (16), and an animal ID station (11), and in particular a milking robot (10) or an individual feeding station.
- Method according to any one of the preceding claims, wherein the first set comprises an animal ID station criterion for determining the element type from animal position data, in particular comprising assigning the type of animal ID station (11) to the animal position of an animal, as determined when said animal is identified at said animal ID station, in particular wherein the second set comprises assigning the animal position to the animal ID station position.
- Method according to claim 5, wherein the barn comprises an animal ID station (11), the method comprising the step of actively determining the animal position of an animal upon identification thereof in an animal ID station.
- The method according to any one of the preceding claims, wherein the first set comprises at least one movement criterion that is based on at least a movement of an animal, wherein the movement is determined as a time-dependent displacement, in particular in a straight line between two or more consecutively determined positions of said animal.
- The method of claim 7, wherein said movement criterion comprises a cubicle criterion.
- The method of claim 8, wherein said cubicle criterion comprises at least one of:- determining that an animal position remains at a prospect position within a predetermined margin during at least a predetermined period of time, wherein said margin is at most a half length of said animal, and in particular wherein said period of time is at least one hour, and- determining that said animal movement comprises a movement in a first direction,followed by a subsequent animal movement in a second direction substantially perpendicular to said first direction, followed by an animal position at a substantially constant prospect position during at least a predetermined first period of time, followed by a animal movement substantially opposite the second direction, followed by a animal movement substantially perpendicular to the second direction,
and wherein the second set comprises assigning a cubicle position to said prospect position. - Method according to any one of the preceding claims, further comprising detecting a sound selected from eating sounds, rumination sounds and drinking sounds, by means of a sound sensor worn by at least one of the animals, wherein at least one of the criteria is dependent on the detection of at least one of said sounds.
- Method according to any one of the preceding claims, wherein the second set comprises assigning to the position of a barn element a position that is based on at least one of the animal positions that were used in positively assigning an element type to said barn element.
- System configured to automatically create a digital map of an animal barn (1) with a plurality of types of barn elements (4a, 4b, 4c, 5, 6, 7, 8, 17, 18, 20, 22), according to the method of any one of the preceding claims, the system comprising- an animal location system, configured to determine position data of at least one animal in said barn- a computer system configured to:- automatically collect said position data as a function of time, characterized in that the computer system is further configured to:- store a first set of criteria for correlating said position data to said type of barn element and a second set of criteria for determining the position of said type of barn element,- apply the sets of criteria to the position data, to thereby derive the type and position of at least a first barn element of a first type,- automatically create the map with at least said first barn element and the type and position thereof.
- System according to claim 12, further comprising an output device, wherein the computer system is configured to output the map to the output device.
- System according to claim 12 or 13, wherein the animal location system comprises at least one position tag (25) worn by said at least one animals and a receiver device (3) arranged to detect wirelessly the position of said tag in the barn.
- System according to any one of claims 12-14, further comprising an arrangement operatively coupled to the computer system and provided with a sound sensor for detecting animal sound, in particular a sound sensor provided on neck tag (25), more in particular on said position tag, comprising at least one of eating sounds, rumination sounds and drinking sounds, and wherein at least one of said sound sensor, said arrangement and said computer system comprises a sound analysis device for sound analysis, configured to analyse said detected animal sound.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
NL2009456A NL2009456C2 (en) | 2012-09-13 | 2012-09-13 | Method and system for creating a map of a barn. |
PCT/NL2013/050578 WO2014042519A1 (en) | 2012-09-13 | 2013-08-05 | Method and system for creating a map of a barn |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2894972A1 EP2894972A1 (en) | 2015-07-22 |
EP2894972B1 true EP2894972B1 (en) | 2016-11-02 |
Family
ID=47427402
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13756719.4A Active EP2894972B1 (en) | 2012-09-13 | 2013-08-05 | Method and system for creating a map of a barn |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP2894972B1 (en) |
DK (1) | DK2894972T3 (en) |
NL (1) | NL2009456C2 (en) |
WO (1) | WO2014042519A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6052930B2 (en) * | 2012-03-28 | 2016-12-27 | デラヴァル ホルディング アーベー | Systems and methods for care-related farm decision support |
JPWO2017141521A1 (en) | 2016-02-16 | 2018-12-06 | ソニー株式会社 | Information processing apparatus, information processing method, and program |
CA3222296A1 (en) * | 2021-07-01 | 2023-01-05 | Peter BAHLENBERG | System for determining a time distribution of an animal into zones of a barn |
WO2023244158A1 (en) | 2022-06-17 | 2023-12-21 | Delaval Holding Ab | Categorizing system, computer-implemented method, computer program and non-volatile data carrier |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6901369B2 (en) * | 1992-11-09 | 2005-05-31 | Lextron, Inc. | Satellite based global positioning system for feedlot computer network and method |
IL125940A (en) | 1998-08-26 | 2002-05-23 | Bar Shalom Avshalom | Device, method and system for automatic identification of sound patterns made by animals |
IL137381A (en) | 2000-07-19 | 2005-06-19 | Avshalom Bar Shalom | Method and system for monitoring ruminant animals, particularly dairy cattle |
FI122052B (en) * | 2009-11-06 | 2011-08-15 | Domuset Oy | A method and arrangement for tracking the path of a pet pet at home |
US20110113148A1 (en) | 2009-11-09 | 2011-05-12 | Nokia Corporation | Method and apparatus for providing a meeting point and routes for participants to a proposed meeting |
US8823513B2 (en) | 2011-01-18 | 2014-09-02 | Radio Systems Corporation | Systems and methods to track movement of animals |
-
2012
- 2012-09-13 NL NL2009456A patent/NL2009456C2/en not_active IP Right Cessation
-
2013
- 2013-08-05 EP EP13756719.4A patent/EP2894972B1/en active Active
- 2013-08-05 DK DK13756719.4T patent/DK2894972T3/en active
- 2013-08-05 WO PCT/NL2013/050578 patent/WO2014042519A1/en active Application Filing
Also Published As
Publication number | Publication date |
---|---|
EP2894972A1 (en) | 2015-07-22 |
NL2009456C2 (en) | 2014-03-18 |
DK2894972T3 (en) | 2017-02-06 |
WO2014042519A1 (en) | 2014-03-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2943146B1 (en) | Method and system for generating an oestrus attention signal for a cattle animal | |
US8488844B2 (en) | Device and a method for providing information about animals when walking through an animal passage | |
NL1038445C2 (en) | System and method for automatically determining animal position and animal activity. | |
US9439389B2 (en) | System and method for milking stall assignment using real-time location | |
EP2894972B1 (en) | Method and system for creating a map of a barn | |
AU2013240637B2 (en) | System and method for grooming-related farm decision support | |
US20220221325A1 (en) | Weight determination of an animal based on 3d imaging | |
US20210212292A1 (en) | Holding area and method executed in a holding area | |
Chapa et al. | Use of a real-time location system to detect cows in distinct functional areas within a barn | |
US10278358B2 (en) | System and method for managing dairy animals | |
US20240292811A1 (en) | System for determining a time distribution of an animal into zones of a barn | |
NL2016113B1 (en) | Method and device for detecting and registering grazing of cattle. | |
KR102403814B1 (en) | Milking information recording system by object | |
US20240315209A1 (en) | System for determining a walked distance of an animal within a barn | |
JP2023168245A (en) | Pseudo-grooming execution individual discrimination system, grooming information collection system, classification model generation system, dataset generation method and pseudo-grooming execution individual discrimination method | |
WO2023031759A1 (en) | Animal husbandry system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20150413 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAX | Request for extension of the european patent (deleted) | ||
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: LELY PATENT N.V. |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20160606 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 840834 Country of ref document: AT Kind code of ref document: T Effective date: 20161115 Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602013013552 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: FP |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20170201 |
|
REG | Reference to a national code |
Ref country code: SE Ref legal event code: TRGR |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170202 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170203 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170302 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170302 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602013013552 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 5 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170202 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20170803 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170831 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170831 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170805 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170805 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170805 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20130805 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: UEP Ref document number: 840834 Country of ref document: AT Kind code of ref document: T Effective date: 20161102 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DK Payment date: 20190828 Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161102 |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: EBP Effective date: 20200831 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200831 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: AT Payment date: 20210721 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20210827 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20220826 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: SE Payment date: 20220827 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20220825 Year of fee payment: 10 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 840834 Country of ref document: AT Kind code of ref document: T Effective date: 20220805 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20220805 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220805 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230505 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R084 Ref document number: 602013013552 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220805 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MM Effective date: 20230901 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230901 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230806 Ref country code: NL Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230901 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230831 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240828 Year of fee payment: 12 |