EP3835017A1 - Scheibenklassifizierungssystem und -verfahren für eine aufschnittmaschine - Google Patents

Scheibenklassifizierungssystem und -verfahren für eine aufschnittmaschine Download PDF

Info

Publication number: EP3835017A1
Authority: EP; European Patent Office
Prior art keywords: images; slices; area; region; food
Prior art date: 2019-12-13
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.): Granted

Application number

EP19383114.6A

Other languages

English (en)

French (fr)

Other versions

EP3835017B1 (de

Inventor

Emilio DE LA RED BELLVIS

Iván CALLEN BARCELONA

Daniel ARTAL GONZÁLEZ

Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)

Inndeo Proyectos Industriales SL

Original Assignee

Inndeo Proyectos Industriales SL

Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)

2019-12-13

Filing date

2019-12-13

Publication date

2021-06-16

2019-12-13 Application filed by Inndeo Proyectos Industriales SL filed Critical Inndeo Proyectos Industriales SL

2019-12-13 Priority to ES19383114T priority Critical patent/ES2956779T3/es

2019-12-13 Priority to EP19383114.6A priority patent/EP3835017B1/de

2021-06-16 Publication of EP3835017A1 publication Critical patent/EP3835017A1/de

2023-06-21 Application granted granted Critical

2023-06-21 Publication of EP3835017B1 publication Critical patent/EP3835017B1/de

Status Active legal-status Critical Current

2039-12-13 Anticipated expiration legal-status Critical

Images

Classifications

- B—PERFORMING OPERATIONS; TRANSPORTING
- B26—HAND CUTTING TOOLS; CUTTING; SEVERING
- B26D—CUTTING; DETAILS COMMON TO MACHINES FOR PERFORATING, PUNCHING, CUTTING-OUT, STAMPING-OUT OR SEVERING
- B26D5/00—Arrangements for operating and controlling machines or devices for cutting, cutting-out, stamping-out, punching, perforating, or severing by means other than cutting
- B26D5/007—Control means comprising cameras, vision or image processing systems
- B—PERFORMING OPERATIONS; TRANSPORTING
- B26—HAND CUTTING TOOLS; CUTTING; SEVERING
- B26D—CUTTING; DETAILS COMMON TO MACHINES FOR PERFORATING, PUNCHING, CUTTING-OUT, STAMPING-OUT OR SEVERING
- B26D7/00—Details of apparatus for cutting, cutting-out, stamping-out, punching, perforating, or severing by means other than cutting
- B26D7/27—Means for performing other operations combined with cutting
- B26D7/32—Means for performing other operations combined with cutting for conveying or stacking cut product
- B—PERFORMING OPERATIONS; TRANSPORTING
- B26—HAND CUTTING TOOLS; CUTTING; SEVERING
- B26D—CUTTING; DETAILS COMMON TO MACHINES FOR PERFORATING, PUNCHING, CUTTING-OUT, STAMPING-OUT OR SEVERING
- B26D7/00—Details of apparatus for cutting, cutting-out, stamping-out, punching, perforating, or severing by means other than cutting
- B26D7/27—Means for performing other operations combined with cutting
- B26D7/32—Means for performing other operations combined with cutting for conveying or stacking cut product
- B26D7/325—Means for performing other operations combined with cutting for conveying or stacking cut product stacking the cut product individually separated by separator elements
- B—PERFORMING OPERATIONS; TRANSPORTING
- B26—HAND CUTTING TOOLS; CUTTING; SEVERING
- B26D—CUTTING; DETAILS COMMON TO MACHINES FOR PERFORATING, PUNCHING, CUTTING-OUT, STAMPING-OUT OR SEVERING
- B26D2210/00—Machines or methods used for cutting special materials
- B26D2210/02—Machines or methods used for cutting special materials for cutting food products, e.g. food slicers

Definitions

the present invention belongs to the technical field of food processing.
this invention relates to a system and a method for a food slicer machine, which are capable of classifying the slices of food by capturing and processing digital matrix images, taken at high speed during the process of cutting the food into slices. Thanks to said image classification, the system and the method of the invention enable possible quality failures to be detected in the slices obtained, prior to the packaging thereof.
system and the method according to the present invention enable the slices of food to be classified according to different preconfigurable quality parameters.
These machines are intended to slice foods (for example and without limitation: cheeses, sausages, or ham) which initially have an elongated shape, usually a shape similar to a bar, in order to obtain slices thereof.
Each set of slices is grouped vertically in a portion in the shape of a stack of products, which is subsequently carried by a conveyor belt.
Said belt can tilt in different positions in order to direct them to different lines.
One of the positions usually enables distinguishing the set of slices if it has an excess or lack of weight, rejecting the portion of slices.
a plastic sheet can also be arranged, also known as “interleaver”, intended to separate said pair of slices and prevent them from sticking together.
Cutting devices of this type are disclosed, for example, in US 5 129 298A or in DE 199 06 021 A1 .
the slices obtained by cutting the foods "in bars” do not meet the quality requirements required for them to be sold.
the most common defects that slices usually have in these types of foods are: the presence of contaminants (for example, blood, rust or bones in the ham), the deformation of some region of the product (for example, excess subcutaneous or central fat in the ham), the lack of interleaver (or breakage thereof), or breakage of the slices.
the slicer machines currently used on an industrial scale work at high cutting speeds, in the order of 900-2000 cuts/minute. This fact makes it difficult for computer vision equipment to be able to take at least one image of each of the slices created, and this in turn significantly complicates the possibility to classify said images in a time frame compatible with the processing speed of the food.
the present invention aims to solve the difficulties and disadvantages of the prior art, indicated above.
a first object of the invention relates to a slice classification system for a food slicer machine, said slicer machine comprising:
the present invention enables, in spite of the high cutting speeds of the commonly used slicer machines, all the slices making up the formed stacks of slices to be analysed before they are packaged. This represents a significant improvement in the detection of possible defects, with respect to the prior art.
the image capture means exclusively take images of a viewing area of the camera included inside the cutting region, inside of which there is a final end of the food which will be cut into a slice. Also, inside the viewing area, one or several control regions are established, which enable the position of the cutting blade and one or several regions of interest to be detected, which enable the quality of the slicing to be examined.
the processing means when analysing said control regions, preferably by means of computer vision algorithms, are able to select the specific images (or frames) from the complete sequence of images wherein the regions of interest can be processed, since they are not hidden (either by the cutting blade, or by an interleaver).
the images thereby selected are precisely called “preferred images” and each one shows one of the slices obtained by the slicer machine.
the hue of the pixels of said control area is compared with at least one predetermined chromatic hue, corresponding to the colour of the cutting blade and/or the colour of the separating sheet (or interleaver). Since both the cutting blade and the interleaver have a colour that is notably different from the colour of the food, thanks to said comparison it is possible to determine the presence of the cutting blade and/or the interleaver and choose as preferred images those wherein none of these elements are present.
each preferred image is only carried out in those image portions corresponding to the region or regions of interest, reducing the load of computing operations and the volume of data used by the processing means.
This enables the preferred images to be processed quickly and enables the slices shown in said images to be classified (as acceptable slices or slices with defects) and enables decisions to be made online (accepting or rejecting the stack wherein said slice is located), in other words, making decisions before the stack of slices reaches the classification means.
the present invention enables deciding whether a stack of slices meets the quality requirements and therefore continues to the packaging line, or, on the other hand, if the presence of a defect is detected, ordering the rejection thereof by sending a corresponding instruction to the actuation means of the classification means.
the slice classification system according to the present invention is independent of the cutting machine, being able to be coupled to any type of new or already-existing slicer machine. To do so, the shape and size of the image capture means can be adapted to the geometry of the slicer machine.
At least one control region is preferably located in the highest portion, or in the lowest portion, of the viewing area (meaning, in the vicinity of the upper edge or the lower edge of the viewing area, that which is determined by the size of the slice with a margin on the perimeter thereof of at least two centimetres). These specific positions are preferred because they can more easily detect the position of the cutting blade with respect to the slice, as well as the presence and position of the interleaver.
the processing means preferably processes the region (or regions) of interest of the selected images, also called preferred images, transforming them into a predetermined colour model.
the processing means quantify the pixels of at least one predetermined hue which are present in the region of interest of the preferred images, grouping them into portions with a predetermined hue and analysing the outline and surface of said portions. Subsequently, the processing means compare the outline and surface of said portions with a predetermined hue, comparing them with the reference patterns. This enables the processing means to classify said regions of interest, determining by means of simple comparisons if they meet the necessary requirements. For example, if a feature of the food can be inscribed with a circumference having a certain diameter, or if a possible defect can be circumscribed in a circle having a certain diameter or if a percentage of a certain hue of the pixels contained in a region is exceeded.
the reference patterns can be, for example and without limitation, reference patterns for shape, size and hue.
the present invention it is also possible to determine whether or not the slice has been cut, since the slicer machine stops cutting between one portion of the stack of slices and another. This makes it possible to count the number of slices in each stack, which in turn enables the initial and final cut of the bar of food to be determined, since a smaller number of slices is cut than that of the stack in both cases.
the present invention also enables certain regions to be discriminated according to the shape thereof, such that areas that are not of interest can be removed from the comparison in order to determine whether or not a product has the quality necessary.
the image capture means comprise a digital camera with a speed for recording sequences of images that is greater than 120 frames per second.
Said digital camera is preferably far from the cutting area in order to avoid splashing due to waste projected from the cutting, although the lens of said camera focuses on the viewing area.
the image capture means preferably comprise an elongated lighting device, arranged parallel to a cutting edge and provided with a semi-diffusion filter. Said semi-diffusion filter reduces the appearance of glare in the slices being cut.
the image capture means are also preferably provided with a protective casing equipped with an elastomeric sealing gasket, fastened by means of mechanical fasteners. These mechanical fasteners enable the easy opening thereof for maintenance operations and in turn guarantee the seal during the cleaning operations with pressurised hot water.
the image capture means may be provided with a communication cable connected to the processing means by which they transmit the sequence of digital images.
the processing means may be provided with a computer.
Said computer is preferably located in a control cabinet, far from the cutting area in order to avoid splashing.
the classification instructions can be control signals from a PLC (Programmable Logic Controller).
the actuation means may comprise at least one pneumatic solenoid valve which can be actuated by means of control signals from a PCL.
the slice classification system according to the present invention is preferably provided with a monitor configured to show images of the sequence of images captured by the image capture means.
the images shown may be images of the defects detected.
the slice classification system according to the present invention is preferably provided with a touch interface.
Said touch interface may preferably enable, among other functions, parameters to be set for the operation of the processing means and/or it may enable the historical record of classified stacks to be consulted.
a second aspect of the invention relates to a method for classifying slices of food cut by a slicer machine, said slicer machine comprising:
any technical feature explained in relation to a slice classification system according to the first aspect of the present invention is also fully applicable to a slice classification method according to the second aspect of the present invention.
Figure 1 shows a schematic representation of a cutting device according to the invention.
the device for cutting food (10), such as, for example, sausage, ham or cheese, has a product feeder mechanism with a product seat, which is open towards a cutting region (13).
the product (10) to be cut is located in the food feeder mechanism on the seat surface and in contact with a tightening roller (24). Said food (10) can be led to the cutting region (13) with the help of an advancing mechanism (not shown), which acts on the rear end of the product.
slices (11) of product (10) are separated in a rapid sequence with the help of a cutting blade (20), for example, a rotating blade, which are stacked in a portion of a stack of slices (25) as the conveyor belt (16) advances.
a cutting blade (20) for example, a rotating blade
the separated product slices (25) are withdrawn separately or in portions by means of an endless belt (16) outside the cutting region (13). Between the withdrawal mechanism (16) and the cutting edge (14), there is a withdrawal belt (15), responsible for rejecting the initial and final ends of the bar of product, which are led to a tank.
means (1) for capturing the images at a high speed have been arranged above the cutting edge (14).
Said capture means (1) are used to obtain images of the viewing area (22) focused on the cutting area (13) of the product with the aim of being able to determine whether the slices have any quality defects.
the images captured at high speed by the means (1) are transmitted by a communication cable to processing means, which in this embodiment of the invention include a computer (8) located in the control cabinet (18), which processes the images received by means of computer vision software which uses the image classification method claimed in this patent, determining whether the slice (11) has a quality defect, communicating, if affirmative, a signal to actuation means, which in this embodiment of the invention include a PLC (9) in order to remove the portions (26) of product actuating once it reaches the classification means, which in this embodiment of the invention, include a tilting classifying belt (17).
processing means which in this embodiment of the invention include a computer (8) located in the control cabinet (18), which processes the images received by means of computer vision software which uses the image classification method claimed in this patent, determining whether the slice (11) has a quality defect, communicating, if affirmative, a signal to actuation means, which in this embodiment of the invention include a PLC (9) in order to remove the portions (26) of product actuating once it reaches the classification means
a monitor (19) is further provided wherein a control panel is displayed on which the images that are being captured by the image capture means (1) are displayed, in addition to counters which show operating parameters of the equipment, as well as quality failures detected in the stacks (25) of slices.
Figure 2 shows a schematic representation of the viewing area of the device.
the viewing area (22) of the camera is contained in the cutting region (13), and inside of it the slice (11), the lower cutting edge (14) and the tightening roller (24) are located (not visible in Fig. 2 ).
the processing means are processed in the processing means by computer vision algorithms with the aim of detecting the position of the cutting blade (20) and the interleaver with respect to the slice (11) and being able to determine the precise moment at which to process the region of interest (23) by means of computer vision algorithms with which a classification is performed based on the quality thereof.
Figures 3A-3C show sequences of images of slicing wherein they illustrate, among other aspects of the invention, how the cut or uncut state of a slice is detected by means of the processing of the different control regions.
Fig. 3A shows a sequence of images taken during the process of cutting a slice of food.
the space shown in said images corresponds to the viewing area (22).
the pixels corresponding to four different control areas designated with the numerical reference (21) are processed.
the images of the areas of said control areas (21) are shown in green, the presence of the interleaver is evaluated. It is possible to detect the presence of the interleaver because, since it is translucent, if said interleaver is present it modifies the colour of the region. If the occupation of the blade is not detected, according to a method similar to that for detecting the interleaver, in the entirety of the lower cutting region, a slice is subsequently counted.
Fig. 3B show a sequence of steps of the cutting blade wherein the slice is not cut.
the pixels corresponding to four different control areas designated with the numerical reference (21) are processed.
the two upper control areas (21) are activated, the images of the areas of said control areas (21) are shown in green, the presence of the interleaver is evaluated. If said interleaver is not subsequently detected, this may be due to a quality error or to the fact that a slice is not going to be cut.
the processing means of the system of the invention determine that a slice has not been cut.
the images in Fig. 3C show an example of processing the region (23) of interest, once the frame of interest of the captured sequence of images has been selected.
the processing means identify the presence of central fat and subcutaneous fat (shown in red and blue in the images) in order to detect if they are within predefined quality values.
Figures 4 and 5 show an embodiment of the slice classification system coupled to a slicer machine from the state of the art.
Figure 4 shows in two isometric views the different elements of the slice classification system according to the invention already described in Figures 1 and 2 , with the aim of facilitating an understanding of the diagrams.
Figure 5 shows two views from different angles of the image capture means (1).
the image capture means (1) are provided with a protective casing (4) provided with an inner cavity (5) on the inside of which a high speed camera (2) and lighting means (3) are housed.
the casing (4) also has an elastomeric sealing gasket (7), fastened by means of mechanical joints (6) which enable it to be opened for maintenance operations and which in turn guarantees the seal during the cleaning operations with pressurised hot water.

Landscapes

Engineering & Computer Science (AREA)
Life Sciences & Earth Sciences (AREA)
Forests & Forestry (AREA)
Mechanical Engineering (AREA)
Computer Vision & Pattern Recognition (AREA)
Investigating Or Analysing Materials By Optical Means (AREA)
Dairy Products (AREA)

EP19383114.6A 2019-12-13 2019-12-13 Scheibenklassifizierungssystem und -verfahren für eine aufschnittmaschine Active EP3835017B1 (de)

Priority Applications (2)

Application Number	Priority Date	Filing Date	Title
ES19383114T ES2956779T3 (es)	2019-12-13	2019-12-13	Sistema y método de clasificación de secciones, para una máquina rebanadora de alimentos
EP19383114.6A EP3835017B1 (de)	2019-12-13	2019-12-13	Scheibenklassifizierungssystem und -verfahren für eine aufschnittmaschine

Applications Claiming Priority (1)

Application Number	Priority Date	Filing Date	Title
EP19383114.6A EP3835017B1 (de)	2019-12-13	2019-12-13	Scheibenklassifizierungssystem und -verfahren für eine aufschnittmaschine

Publications (2)

Publication Number	Publication Date
EP3835017A1 true EP3835017A1 (de)	2021-06-16
EP3835017B1 EP3835017B1 (de)	2023-06-21

Family

ID=69174290

Family Applications (1)

Application Number	Title	Priority Date	Filing Date
EP19383114.6A Active EP3835017B1 (de)	2019-12-13	2019-12-13	Scheibenklassifizierungssystem und -verfahren für eine aufschnittmaschine

Country Status (2)

Country	Link
EP (1)	EP3835017B1 (de)
ES (1)	ES2956779T3 (de)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
EP0449512A1 (de) *	1990-03-27	1991-10-02	Thurne Engineering Co Ltd	Schneidevorrichtung
US5129298A (en)	1990-06-29	1992-07-14	Societe Des Viandes Bretagne Anjou-Soviba La Noelle Ancenis (Loire Atlantique) France	Automatic machine for slicing non-rigid products, such as foodstuffs such as meats
DE19906021A1 (de)	1999-02-16	2000-08-17	Holger Wente	Verfahren und Vorrichtung zum Abtrennen scheibenförmiger Körper von einem Ursprungskörper
WO2004000512A1 (en) *	2002-06-25	2003-12-31	Formax, Inc.	Optical grading system for slicer apparatus

2019
- 2019-12-13 ES ES19383114T patent/ES2956779T3/es active Active
- 2019-12-13 EP EP19383114.6A patent/EP3835017B1/de active Active

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
EP0449512A1 (de) *	1990-03-27	1991-10-02	Thurne Engineering Co Ltd	Schneidevorrichtung
US5129298A (en)	1990-06-29	1992-07-14	Societe Des Viandes Bretagne Anjou-Soviba La Noelle Ancenis (Loire Atlantique) France	Automatic machine for slicing non-rigid products, such as foodstuffs such as meats
DE19906021A1 (de)	1999-02-16	2000-08-17	Holger Wente	Verfahren und Vorrichtung zum Abtrennen scheibenförmiger Körper von einem Ursprungskörper
WO2004000512A1 (en) *	2002-06-25	2003-12-31	Formax, Inc.	Optical grading system for slicer apparatus

Also Published As

Publication number	Publication date
EP3835017B1 (de)	2023-06-21
ES2956779T3 (es)	2023-12-27

Publication	Publication Date	Title
EP1534478B1 (de)	2017-03-01	Optisches Klassifizierungssystem und entsprechendes Verfahren für Schneidevorrichtung
EP2923305B1 (de)	2020-04-22	Verfahren zum rillen und zur qualitätskontrolle von nahrungsmitteln in einer dynamischen fertigungsstrasse
CN109661176B (zh)	2022-04-12	获取和分析数据的装置和加工产品的系统和方法
EP2029330B1 (de)	2014-04-30	Aufschneidemaschine mit GESTROBTE BELEUCHTUNG VERWENDENDEM OPTISCHEM SYSTEM sowie Verfahren zum Tasten einer Strirnseite eines Produktes
EP1915240B1 (de)	2014-04-30	Zweifarben-bilderkennungssystem
US5267168A (en)	1993-11-30	Apparatus for and method of controlling slicing machine
EP3352574B1 (de)	2021-03-31	Sensorgeführtes automatisiertes verfahren und system zur verarbeitung von krustentieren
JP4579332B2 (ja)	2010-11-10	加工ラインに沿って搬送される魚、鶏肉又は他の肉製品を加工する方法及び装置
US20080212842A1 (en)	2008-09-04	Two Colour Vision System
US20220118483A1 (en)	2022-04-21	A food processing device
EP3835017B1 (de)	2023-06-21	Scheibenklassifizierungssystem und -verfahren für eine aufschnittmaschine
JP2023551471A (ja)	2023-12-08	加工施設での食品追跡方法および食品を加工するためのシステム
EP3550982A1 (de)	2019-10-16	Verfahren zum bearbeiten eines lebensmittelobjekts
GB2362710A (en)	2001-11-28	Scanning meat samples for bones where sample container has a data storage element read by control means
CN113329851B (zh)	2022-09-27	用于加工饮料封盖的设备及方法
CN111842216A (zh)	2020-10-30	一种基于机器视觉的农产品质量检测自动化分拣管理系统
CN117686424B (zh)	2024-04-26	基于机器视觉的肉类碎骨智能检测系统及方法
JP3140585U (ja)	2008-04-03	にんじんの抽苔検出機能を有する裁断装置
US20230005127A1 (en)	2023-01-05	Method and device for detecting containers which have fallen over and/or are damaged in a container mass flow
US20240284922A1 (en)	2024-08-29	Vision-based quality control and audit system and method of auditing, for carcass processing facility
CN115564720A (zh)	2023-01-03	一种基于三维扫描和机器视觉的带钢缺陷检测方法及系统
EP2610781B1 (de)	2015-08-05	Zylindererkennungssystem
WO2014136133A1 (en)	2014-09-12	Cuts control device in a visual control machine and automatic selection
Davies et al.	1994	An application of machine vision for automating the manufacture of cork stoppers

Legal Events

Date	Code	Title	Description
2021-05-14	PUAI	Public reference made under article 153(3) epc to a published international application that has entered the european phase	Free format text: ORIGINAL CODE: 0009012
2021-05-14	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED
2021-06-16	AK	Designated contracting states	Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
2021-12-17	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE
2022-01-19	17P	Request for examination filed	Effective date: 20211215
2022-01-19	RBV	Designated contracting states (corrected)	Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
2023-02-05	GRAP	Despatch of communication of intention to grant a patent	Free format text: ORIGINAL CODE: EPIDOSNIGR1
2023-02-05	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: GRANT OF PATENT IS INTENDED
2023-03-08	INTG	Intention to grant announced	Effective date: 20230206
2023-05-12	GRAS	Grant fee paid	Free format text: ORIGINAL CODE: EPIDOSNIGR3
2023-05-19	GRAA	(expected) grant	Free format text: ORIGINAL CODE: 0009210
2023-05-19	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: THE PATENT HAS BEEN GRANTED
2023-06-21	AK	Designated contracting states	Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
2023-06-30	REG	Reference to a national code	Ref country code: CH Ref legal event code: EP
2023-07-06	REG	Reference to a national code	Ref country code: DE Ref legal event code: R096 Ref document number: 602019031304 Country of ref document: DE
2023-07-15	REG	Reference to a national code	Ref country code: AT Ref legal event code: REF Ref document number: 1580592 Country of ref document: AT Kind code of ref document: T Effective date: 20230715
2023-07-19	REG	Reference to a national code	Ref country code: IE Ref legal event code: FG4D
2023-10-10	REG	Reference to a national code	Ref country code: LT Ref legal event code: MG9D
2023-10-25	REG	Reference to a national code	Ref country code: NL Ref legal event code: MP Effective date: 20230621
2023-10-31	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230921
2023-11-15	REG	Reference to a national code	Ref country code: AT Ref legal event code: MK05 Ref document number: 1580592 Country of ref document: AT Kind code of ref document: T Effective date: 20230621
2023-11-30	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230922
2023-12-27	REG	Reference to a national code	Ref country code: ES Ref legal event code: FG2A Ref document number: 2956779 Country of ref document: ES Kind code of ref document: T3 Effective date: 20231227
2023-12-29	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621
2024-01-16	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621
2024-01-23	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231021
2024-01-31	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231023 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231021 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621
2024-02-29	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621
2024-03-22	REG	Reference to a national code	Ref country code: DE Ref legal event code: R097 Ref document number: 602019031304 Country of ref document: DE
2024-04-16	PGFP	Annual fee paid to national office [announced via postgrant information from national office to epo]	Ref country code: ES Payment date: 20240102 Year of fee payment: 5
2024-04-27	PLBE	No opposition filed within time limit	Free format text: ORIGINAL CODE: 0009261
2024-04-27	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT
2024-04-30	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621
2024-04-30	PGFP	Annual fee paid to national office [announced via postgrant information from national office to epo]	Ref country code: DE Payment date: 20231221 Year of fee payment: 5
2024-05-03	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621
2024-05-29	26N	No opposition filed	Effective date: 20240322
2024-05-31	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621
2024-07-31	REG	Reference to a national code	Ref country code: CH Ref legal event code: PL
2024-08-23	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231213
2024-08-27	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621
2024-08-28	GBPC	Gb: european patent ceased through non-payment of renewal fee	Effective date: 20231213
2024-08-29	REG	Reference to a national code	Ref country code: BE Ref legal event code: MM Effective date: 20231231
2024-08-30	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230621 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231213
2024-10-09	REG	Reference to a national code	Ref country code: IE Ref legal event code: MM4A
2024-10-10	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231213
2024-10-14	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231213
2024-10-15	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231231
2024-10-16	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231231
2024-10-21	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231231
2024-10-31	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231213 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231213 Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231231 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231231 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231231