[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2023110069A1 - Data processing apparatus and method implementing a software lockstep - Google Patents

Data processing apparatus and method implementing a software lockstep Download PDF

Info

Publication number
WO2023110069A1
WO2023110069A1 PCT/EP2021/085828 EP2021085828W WO2023110069A1 WO 2023110069 A1 WO2023110069 A1 WO 2023110069A1 EP 2021085828 W EP2021085828 W EP 2021085828W WO 2023110069 A1 WO2023110069 A1 WO 2023110069A1
Authority
WO
WIPO (PCT)
Prior art keywords
program code
data processing
processing apparatus
instance
lockstep
Prior art date
Application number
PCT/EP2021/085828
Other languages
French (fr)
Inventor
Shiqing FAN
Original Assignee
Huawei Technologies Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co., Ltd. filed Critical Huawei Technologies Co., Ltd.
Priority to PCT/EP2021/085828 priority Critical patent/WO2023110069A1/en
Priority to CN202180105000.6A priority patent/CN118401924A/en
Priority to EP21839151.4A priority patent/EP4427134A1/en
Publication of WO2023110069A1 publication Critical patent/WO2023110069A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/1629Error detection by comparing the output of redundant processing systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/1629Error detection by comparing the output of redundant processing systems
    • G06F11/1641Error detection by comparing the output of redundant processing systems where the comparison is not performed by the redundant processing components

Definitions

  • the present disclosure relates to data processing systems. More specifically, the present disclosure relates to a data processing apparatus and method implementing a software lockstep.
  • Lockstep systems are generally fault tolerant computer systems, i.e. data processing systems that run the same set of operations defined by an application on two different hardware and/or software resources, such as a processor at the same time in parallel.
  • a lockstep system may implement a hardware lockstep or a software lockstep.
  • a data processing system implementing a hardware lockstep generally requires special hardware support, with limited flexibility, where each CPU instruction needs to be compared.
  • the execution performance of a typical dual core hardware lockstep is highly dependent on one or more of the following parameters: the application size; the time required for both processors to execute the application in hand-shake; the number of checkpoints performed, where the time to each checkpoint depends of the amount of data saved and the setup used; and, in case of errors, the time to recover the system, for instance, by means of a rollback operation.
  • the execution overhead is also very high mainly because each instruction is duplicated and compared, which involves a huge amount of memory access. This is not avoidable due to the nature of hardware lockstep architecture.
  • a conventional hardware lockstep system is described, for instance, in US20180089059A1.
  • a data processing system implementing a software lockstep step does not have hardware dependency, but normally requires additional synchronization, and comparison logic as a separate software module, which may result in performance degradation, but provides more flexibility.
  • a data processing apparatus configured to monitor the execution of a program code defining at least a portion of a software application using a software lockstep scheme.
  • the data processing apparatus comprises is configured to: lift the program code based on a lockstep configuration and a plurality of software lockstep program code templates; execute a first instance of the lifted program code; execute a second instance of the lifted program code substantially in parallel with the execution of the first instance of the lifted program code; and detect an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance with the second instance of the lifted program code.
  • the data processing apparatus provides a software lockstep with a comparable performance and powerful functionality as a hardware lockstep, while retaining the flexibility of a software lockstep, such as being easily adjustable for dynamic or different application scenarios.
  • the data processing apparatus is further configured to generate the one or more comparison rules based on the lockstep configuration and the plurality of software lockstep program code templates.
  • the data processing apparatus further comprises a memory configured to store the plurality of software lockstep program code templates.
  • the data processing apparatus is configured to store a first output generated by the execution of the first instance of the lifted program code and a second output generated by the execution of the second instance of the lifted program code in the shared memory.
  • the data processing apparatus further comprises a communication interface configured to transmit a first output generated by the execution of the first instance of the lifted program code and a second output generated by the execution of the second instance of the lifted program code to a remote network node.
  • the data processing apparatus comprises a first processor configured to execute the first instance of the lifted program code and a second processor configured to execute the second instance of the lifted program code substantially in parallel with the execution of the first instance of the lifted program code.
  • the data processing apparatus for lifting the program code based on the lockstep configuration and the plurality of software lockstep program code templates the data processing apparatus is configured to parse the program code and to insert, based on the lockstep configuration, one or more of the plurality of software lockstep program code templates into the program code.
  • the data processing apparatus is configured to parse the program code as source code or byte code.
  • the program code comprises one or more functional blocks and each functional block comprises one or more instructions
  • the data processing apparatus is configured to detect an inconsistency of the execution of the program code on the level of the one or more functional blocks or on the level of the one or more instructions.
  • the plurality of software lockstep program code templates comprises at least one of: a control jump, a register copy, an input sync, and a send output.
  • the lockstep configuration comprises information about a least one of: a redundancy level between the execution of the first instance and the second instance of the lifted program code, one or more input/output parameters, a redundancy scope, a comparison request, and one or more binding parameters.
  • a method for monitoring the execution of a program code defining at least a portion of a software application using a software lockstep scheme, wherein the method comprises: lifting the program code based on a lockstep configuration and a plurality of software lockstep program code templates; executing a first instance of the lifted program code; executing a second instance of the lifted program code substantially in parallel with the execution of the first instance of the lifted program code; and detecting an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance with the second instance of the lifted program code.
  • the method according to the second aspect of the present disclosure can be performed by the data processing apparatus according to the first aspect of the present disclosure.
  • further features of the method according to the second aspect of the present disclosure result directly from the functionality of the data processing apparatus according to the first aspect of the present disclosure as well as its different implementation forms described above and below.
  • a computer program product comprising a computer- readable storage medium for storing program code which causes a computer or a processor to perform the method according to the second aspect, when the program code is executed by the computer or the processor.
  • Fig. 1 shows a schematic diagram illustrating a data processing apparatus according to an embodiment
  • FIGS. 2 and 3 show schematic diagrams illustrating in more detail aspects of a data processing apparatus according to an embodiment
  • Fig. 4 shows a schematic diagram illustrating an exemplary embodiment for the redundant execution of an application by the data processing apparatus according to an embodiment
  • Fig. 5 shows a schematic diagram illustrating a further exemplary embodiment for the redundant execution of an application by the data processing apparatus according to an embodiment
  • Fig. 6 shows a schematic diagram illustrating a further exemplary embodiment for the redundant execution of an application by the data processing apparatus according to an embodiment
  • Fig. 7 shows a flow diagram illustrating a data processing method according to an embodiment.
  • a disclosure in connection with a described method may also hold true for a corresponding device or system configured to perform the method and vice versa.
  • a corresponding device may include one or a plurality of units, e.g. functional units, to perform the described one or plurality of method steps (e.g. one unit performing the one or plurality of steps, or a plurality of units each performing one or more of the plurality of steps), even if such one or more units are not explicitly described or illustrated in the figures.
  • a specific apparatus is described based on one or a plurality of units, e.g.
  • a corresponding method may include one step to perform the functionality of the one or plurality of units (e.g. one step performing the functionality of the one or plurality of units, or a plurality of steps each performing the functionality of one or more of the plurality of units), even if such one or plurality of steps are not explicitly described or illustrated in the figures. Further, it is understood that the features of the various exemplary embodiments and/or aspects described herein may be combined with each other, unless specifically noted otherwise.
  • Figure 1 shows a schematic diagram of a data processing apparatus 100 according to an embodiment.
  • the data processing apparatus 100 is a multi-core or multi-processor computing system 100. More specifically, in the embodiment shown in figure 1, the data processing apparatus 100 comprises at least two processors or cores 110a, b configured to access a shared memory 130 via a communication channel 140, in particular a communication bus 140. As illustrated in figure 1 , the data processing apparatus 100 may further comprise a communication interface 120 configured to receive data from and/or transmit data to a remote server.
  • the data processing apparatus 100 illustrated in figure 1 is configured to monitor the execution of a program code defining at least a portion of a software application using a software lockstep scheme.
  • the data processing apparatus 100 is configured to lift the program code based on a lockstep configuration and a plurality of.
  • this first lockstep stage may be provided by a redundancy lifting engine 250 of the data processing apparatus 100.
  • the data processing apparatus 100 is configured to execute a first instance 111a of the lifted program code, for instance, on the first processor 110a illustrated in figure 1 and to redundantly execute a second instance 111 b of the lifted program code, for instance, on the second processor 110b illustrated in figure 1 substantially in parallel with the execution of the first instance 111a of the lifted program code.
  • the data processing apparatus 100 is configured to detect an inconsistency of the execution of the program code by comparing based on one or more comparison rules (defined, for instance, by a voter executor 290 described in more detail in the context of figure 2 below) the execution of the first instance 111a with the second instance 111b of the lifted program code.
  • the first lockstep stage may be provided by the redundancy lifting engine 250 of the data processing apparatus 100 illustrated in figure 2.
  • the main elements illustrated in figure 2 are the lockstep configuration (referred to as "redundant configuration" in figure 2) 210, a redundant controller 260 and the plurality of software lockstep program code templates 270.
  • the application i.e. program code
  • the redundant controller 260 regenerates, i.e. lifts the application code with software lockstep control logic, i.e. additional assembly code will be added at the necessary control points of the application code for obtaining the lifted program code 111.
  • the redundant controller 260 generates the voter executor 290 containing the more comparison rules.
  • the data processing apparatus 100 may implement the voter executor 290 as a service or a separate process.
  • the lifted application code 111 may interoperate with the voter executor 290, for instance, for input/output synchronization, execution comparison, error report and the like.
  • the lockstep, i.e. redundant configuration 210 used by the data processing apparatus 100 for generating the lifted application code 111 may comprise the following information for defining details of the software lockstep.
  • the lockstep, i.e. redundant configuration 210 may comprise redundant level information 211 for defining the behaviour of the software lockstep on an instruction level and/or a code block level.
  • the lockstep, i.e. redundant configuration 210 may comprise redundant scope information 213 for defining which portion(s) of the program code should be run with the software lockstep (i.e. only the selected portions of the program will be lifted and checked by the redundant controller 260).
  • redundant configuration 210 may comprise input/output information 212 defining one or more initial values that are provided into a selected code block as input, as well as one or more variables or register values that are used for comparing the two instances 111a,b of the program code 111.
  • the lockstep, i.e. redundant configuration 210 may comprise compare request information 214 defining how to compare the outputs of the first and the second instance 111 a,b of the program code 111.
  • the compare request information 214 may prompt the data processing apparatus 100 to simply check whether the outputs of the first and the second instance 111a, b of the program code 111 are equal or not.
  • the compare request information 214 may include a mathematical equation for generating the comparison results based on the outputs of the first and the second instance 111 a,b of the program code 111.
  • the lockstep, i.e. redundant configuration 210 may comprise voter and datatype binding information 215, which may comprise a glue code for combining the compare request information 214 with a basic voter operations (OP) template 275 of the software lockstep templates 270 described in the following in more detail.
  • the basic voter OP template 275 may define basic operations, such as an addition, reduction, multiplication operation and the like.
  • the lifting engine 250 uses the compare request information 214 and the datatypes defined by the voter and datatype binding information 215 for apply this information to the basic voter OP template 275, in order to generate the final comparison code which may have the form of a mathematical equation.
  • the software lockstep templates 270 comprise pre-defined program code segments for the lifted application code 111 that may be configured to synchronize the execution of the first instance 111a and the second instance 111 b of the lifted program code 111, copy data from and to one or more registers or the memory 130, as well as the one or more comparison rules for comparing the outputs form the redundant execution of the two instances 111a, b of the lifted application code 111.
  • the redundant controller 260 is configured to integrate the software lockstep templates 270 into the application code 201 a, b for generating the lifted application code 111 in accordance with the lockstep configuration 210.
  • the software lockstep templates 270 may comprise a control jump template 271, which may be a segment of code that will redirect the execution of a program to another code location, e.g. jump to a different part of the application context, or call an external function from a linked library.
  • the software lockstep templates 270 may comprise a register copy template 272 comprising a segment of code to copy one or more register values to a further entity, such as the shared memory 130.
  • the register copy template 272 may depend on the specific hardware of the data processing apparatus 100, because different hardware may have different calls, functions, instructions to perform such operations.
  • the software lockstep templates 270 may comprise an input sync template 273 which may comprise a segment of code to synchronize the input of the controlled program code 111 , or to make sure the redundancies are starting at the same time.
  • the input sync template 273 may synchronize a start time of the first and second instance 111a,b of the program code 111.
  • the input sync template 273 may make use of a time synchronization protocol, e.g. GTP, to synchronize the start at the same timestamp (this usually does not require any control or management from the voter side).
  • the input sync template 273 may be configured to communicate with the controlled voter 290 for comparison or error report (as shown in figure 6, where there is an additional connection between an input sync 401 and the communication channel 140.
  • the input sync template 273 may be implementation specific and may be implemented inside of the lifted code or the voter code.
  • the software lockstep templates 270 may comprise a send output template 274 which may comprise a segment of code for sending the output value to the controlled voter 290.
  • the send output template 274 may depend on the type of communication channel 140 and/or communication protocol used. In an embodiment, the send output template 274 may be implementation specific and may be implemented inside of the lifted code or the voter code.
  • the redundant lifting engine 250 in particular the redundant controller 260 provides the functionality for lifting the original application code 201 a, b based on the lockstep configuration 210 into the lifted application code 111 and for generating the one or more comparison rules defined by the voter 290.
  • the redundant lifting engine 250 in particular the redundant controller 260 may use compiler features to parse the application source code 201a, or use the intermediate byte code 201b to analyse and identify the code structure of the application 111 , e.g. function names and parameters that are configured by the lockstep configuration to be run under software lockstep control.
  • the redundant controller 260 further combines the lockstep configuration 210 with the software lockstep templates 270 to generate the lifted program code 111, e.g. as software lockstep binary code.
  • the additional software lockstep control code provided by one or more of the lockstep templates 270 is injected into the application code 201a, b for interacting with the generated software lockstep voter executor 290.
  • Figures 4 to 6 show further details of exemplary embodiments for the redundant execution of an application by the data processing apparatus 100 according to an embodiment.
  • Figure 4 illustrates in more detail the interaction between the additional software lockstep code elements (illustrated as grey boxes in figure 4) and the original application code elements (illustrated as white boxes in figure 4) of the first instance 111a and the second instance 111 b of the lifted application code 111.
  • the uncontrolled application code 201 a, b executes normally, before it goes into the controlled code by an additional control jump inserted in the original code.
  • An input sync call 401 may provide the timing synchronization, which may ensure that the redundant applications 111a,b start the controlled coded in an acceptable deviation of time.
  • the data processing apparatus 100 may comprise a time synchronization module, if the redundant applications 111a,b are running on two different machines or systems on a chip, which are based on different time resources, e.g. clocks. It also provides input, such as random numbers that will make sure the calculation in the controlled code will generate comparable output (different random numbers may generate results with a huge gap for comparison). This is because, some program codes 111, such as image processing algorithms, require a random number as input. However, in order to process the two instances of the program code 111 redundantly, it has to be made sure that the input random number at each control jump (such as illustrated in figure 6) is identical for each instance 111a,b. For example, in figure 6 the "in” parameter of the function block "Func_bloc" may contain the random number, but this must be synchronized for both instances 111 a,b, because otherwise the calculation will generate different or incomparable outputs.
  • the output(s) of the lifted application code instances 111a, b may be register states, function call output variables, or any other states and variables used in the respective code segment, which may be defined by the redundant configuration 210.
  • a control jump at the end of the lifted application code instances 111a, b may collect the outputs of the redundant application code and forward them to the voter 290 for further validation and comparison.
  • the communication between an output sync control 403 and the voter 290 may depend on the implementation of the communication channel 140. For lower performance overhead and fast reaction from the voter 290 a shared memory 130 may be used.
  • the communication interface 120 may be used, for instance, using UDP or another communication protocol.
  • the voter 290 comprises an output receiver 291 for receiving as input the data provided by the output sync control with a timestamp.
  • a voter algorithm executor 295 of the voter 290 compares the results using the one or more comparison rules defined by the lockstep configuration 210.
  • an error reporter 293 of the voter 290 may generate one or more error messages for the other components of the data processing apparatus 100, e.g. error handling or monitoring messages.
  • Figure 5 shows further details of a further exemplary embodiment for the redundant execution of an application 201a, b on an instruction level by the data processing apparatus 100 according to an embodiment.
  • the lifted program code 111a,b comprises a register push instruction, so that the state of each instruction execution may be stored in the shared memory 130.
  • the synchronization of the output may be achieved by using preserved result queues, which may be automatically compared by the voter 290 for checking the results.
  • the register values may be saved with an instruction signature, so that the voter 290 is able to know from which instruction they were generated. As the voter 290 may only check whether the results are equal or not, in this exemplary embodiment there may be no voter algorithm execution component 295 (as in the embodiment of figure 4).
  • Figure 6 shows further details of a further exemplary embodiment for the redundant execution of an application 201 a, b on a functional level by the data processing apparatus 100 according to an embodiment.
  • the lifted code 111a,b is a code block, i.e. a function call, and the return value of the function call is compared by the voter 290.
  • the output synchronization 403 uses the configured communication channel 140 to send the output to the voter 290, which operates as already described above in the context of the embodiment of figure 4.
  • Figure 7 is a flow diagram illustrating a method 700 for monitoring the execution of a program code defining at least a portion of a software application using a software lockstep scheme.
  • the method 700 comprises a step 701 of lifting the program code based on a lockstep configuration and a plurality of software lockstep program code templates.
  • the method 700 further comprises a step 703 of executing a first instance 111a of the lifted program code 111 and a step 705 executing a second instance 111 b of the lifted program code 111 substantially in parallel with the execution of the first instance 111a of the lifted program code 111.
  • the method 700 comprises a step 707 of detecting an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance 111a with the second instance 111 b of the lifted program code 111.
  • the method 700 can be performed by the data processing apparatus 100 according to an embodiment. Thus, further features of the method 700 result directly from the functionality of the data processing apparatus 100 as well as the different embodiments thereof described above and below.
  • Embodiments of the data processing apparatus 100 disclosed herein provide one or more of the following advantages: comparable performance with a hardware lockstep with greater flexibility; dynamic adjustment on the redundant execution; whitebox, greybox and/or blackbox support; high speed comparison of register values; high speed data transfer through shared memory 130; configurable level and scope of redundancy for the user application; configurable voter algorithms and data bindings; low performance overhead; control instrumentation can be certified; and/or flexible deployment on the same SoC or over a communication network.
  • the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • functional units in the embodiments of the disclosure may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Hardware Redundancy (AREA)

Abstract

A data processing apparatus (100) is configured to monitor the execution of a program code defining at least a portion of a software application using a software lockstep scheme. To this end, the data processing apparatus (100) is configured to: lift the program code based on a lockstep configuration and a plurality of software lockstep program code templates; execute a first instance (111a) of the lifted program code; execute a second instance (111b) of the lifted program code substantially in parallel with the execution of the first instance (111a) of the lifted program code; and detect an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance (111a) with the second instance (111b) of the lifted program code.

Description

Data processing apparatus and method implementing a software lockstep
TECHNICAL FIELD
The present disclosure relates to data processing systems. More specifically, the present disclosure relates to a data processing apparatus and method implementing a software lockstep.
BACKGROUND
Lockstep systems are generally fault tolerant computer systems, i.e. data processing systems that run the same set of operations defined by an application on two different hardware and/or software resources, such as a processor at the same time in parallel. A lockstep system may implement a hardware lockstep or a software lockstep.
A data processing system implementing a hardware lockstep generally requires special hardware support, with limited flexibility, where each CPU instruction needs to be compared. The execution performance of a typical dual core hardware lockstep is highly dependent on one or more of the following parameters: the application size; the time required for both processors to execute the application in hand-shake; the number of checkpoints performed, where the time to each checkpoint depends of the amount of data saved and the setup used; and, in case of errors, the time to recover the system, for instance, by means of a rollback operation. The execution overhead is also very high mainly because each instruction is duplicated and compared, which involves a huge amount of memory access. This is not avoidable due to the nature of hardware lockstep architecture. A conventional hardware lockstep system is described, for instance, in US20180089059A1.
On the other hand, a data processing system implementing a software lockstep step does not have hardware dependency, but normally requires additional synchronization, and comparison logic as a separate software module, which may result in performance degradation, but provides more flexibility.
SUMMARY
It is an objective of the present disclosure to provide an improved data processing apparatus and a data processing method implementing a software lockstep. The foregoing and other objectives are achieved by the subject matter of the independent claims. Further implementation forms are apparent from the dependent claims, the description and the figures.
According to a first aspect a data processing apparatus configured to monitor the execution of a program code defining at least a portion of a software application using a software lockstep scheme is provided. The data processing apparatus comprises is configured to: lift the program code based on a lockstep configuration and a plurality of software lockstep program code templates; execute a first instance of the lifted program code; execute a second instance of the lifted program code substantially in parallel with the execution of the first instance of the lifted program code; and detect an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance with the second instance of the lifted program code.
The data processing apparatus according to the first aspect provides a software lockstep with a comparable performance and powerful functionality as a hardware lockstep, while retaining the flexibility of a software lockstep, such as being easily adjustable for dynamic or different application scenarios.
In a further possible implementation form, the data processing apparatus is further configured to generate the one or more comparison rules based on the lockstep configuration and the plurality of software lockstep program code templates.
In a further possible implementation form, the data processing apparatus further comprises a memory configured to store the plurality of software lockstep program code templates.
In a further possible implementation form, the data processing apparatus is configured to store a first output generated by the execution of the first instance of the lifted program code and a second output generated by the execution of the second instance of the lifted program code in the shared memory.
In a further possible implementation form, the data processing apparatus further comprises a communication interface configured to transmit a first output generated by the execution of the first instance of the lifted program code and a second output generated by the execution of the second instance of the lifted program code to a remote network node. In a further possible implementation form, the data processing apparatus comprises a first processor configured to execute the first instance of the lifted program code and a second processor configured to execute the second instance of the lifted program code substantially in parallel with the execution of the first instance of the lifted program code.
In a further possible implementation form, for lifting the program code based on the lockstep configuration and the plurality of software lockstep program code templates the data processing apparatus is configured to parse the program code and to insert, based on the lockstep configuration, one or more of the plurality of software lockstep program code templates into the program code.
In a further possible implementation form, the data processing apparatus is configured to parse the program code as source code or byte code.
In a further possible implementation form, the program code comprises one or more functional blocks and each functional block comprises one or more instructions, wherein the data processing apparatus is configured to detect an inconsistency of the execution of the program code on the level of the one or more functional blocks or on the level of the one or more instructions.
In a further possible implementation form, the plurality of software lockstep program code templates comprises at least one of: a control jump, a register copy, an input sync, and a send output.
In a further possible implementation form, the lockstep configuration comprises information about a least one of: a redundancy level between the execution of the first instance and the second instance of the lifted program code, one or more input/output parameters, a redundancy scope, a comparison request, and one or more binding parameters.
According to a second aspect a method is provided for monitoring the execution of a program code defining at least a portion of a software application using a software lockstep scheme, wherein the method comprises: lifting the program code based on a lockstep configuration and a plurality of software lockstep program code templates; executing a first instance of the lifted program code; executing a second instance of the lifted program code substantially in parallel with the execution of the first instance of the lifted program code; and detecting an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance with the second instance of the lifted program code.
The method according to the second aspect of the present disclosure can be performed by the data processing apparatus according to the first aspect of the present disclosure. Thus, further features of the method according to the second aspect of the present disclosure result directly from the functionality of the data processing apparatus according to the first aspect of the present disclosure as well as its different implementation forms described above and below.
According to a third aspect a computer program product is provided, comprising a computer- readable storage medium for storing program code which causes a computer or a processor to perform the method according to the second aspect, when the program code is executed by the computer or the processor.
Details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims.
BRIEF DESCRIPTION OF THE DRAWINGS
In the following, embodiments of the present disclosure are described in more detail with reference to the attached figures and drawings, in which:
Fig. 1 shows a schematic diagram illustrating a data processing apparatus according to an embodiment;
Figs. 2 and 3 show schematic diagrams illustrating in more detail aspects of a data processing apparatus according to an embodiment;
Fig. 4 shows a schematic diagram illustrating an exemplary embodiment for the redundant execution of an application by the data processing apparatus according to an embodiment; Fig. 5 shows a schematic diagram illustrating a further exemplary embodiment for the redundant execution of an application by the data processing apparatus according to an embodiment; Fig. 6 shows a schematic diagram illustrating a further exemplary embodiment for the redundant execution of an application by the data processing apparatus according to an embodiment; and
Fig. 7 shows a flow diagram illustrating a data processing method according to an embodiment.
In the following, identical reference signs refer to identical or at least functionally equivalent features.
DETAILED DESCRIPTION OF THE EMBODIMENTS
In the following description, reference is made to the accompanying figures, which form part of the disclosure, and which show, by way of illustration, specific aspects of embodiments of the present disclosure or specific aspects in which embodiments of the present disclosure may be used. It is understood that embodiments of the present disclosure may be used in other aspects and comprise structural or logical changes not depicted in the figures. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims.
For instance, it is to be understood that a disclosure in connection with a described method may also hold true for a corresponding device or system configured to perform the method and vice versa. For example, if one or a plurality of specific method steps are described, a corresponding device may include one or a plurality of units, e.g. functional units, to perform the described one or plurality of method steps (e.g. one unit performing the one or plurality of steps, or a plurality of units each performing one or more of the plurality of steps), even if such one or more units are not explicitly described or illustrated in the figures. On the other hand, for example, if a specific apparatus is described based on one or a plurality of units, e.g. functional units, a corresponding method may include one step to perform the functionality of the one or plurality of units (e.g. one step performing the functionality of the one or plurality of units, or a plurality of steps each performing the functionality of one or more of the plurality of units), even if such one or plurality of steps are not explicitly described or illustrated in the figures. Further, it is understood that the features of the various exemplary embodiments and/or aspects described herein may be combined with each other, unless specifically noted otherwise.
Figure 1 shows a schematic diagram of a data processing apparatus 100 according to an embodiment. In the embodiment shown in figure 1, the data processing apparatus 100 is a multi-core or multi-processor computing system 100. More specifically, in the embodiment shown in figure 1, the data processing apparatus 100 comprises at least two processors or cores 110a, b configured to access a shared memory 130 via a communication channel 140, in particular a communication bus 140. As illustrated in figure 1 , the data processing apparatus 100 may further comprise a communication interface 120 configured to receive data from and/or transmit data to a remote server.
As will be described in more detail below, the data processing apparatus 100 illustrated in figure 1 is configured to monitor the execution of a program code defining at least a portion of a software application using a software lockstep scheme. To this end, in a first lockstep stage, the data processing apparatus 100 is configured to lift the program code based on a lockstep configuration and a plurality of. As will be described in more detail under further reference to figure 2 below, this first lockstep stage may be provided by a redundancy lifting engine 250 of the data processing apparatus 100.
In a second lockstep stage, the data processing apparatus 100 is configured to execute a first instance 111a of the lifted program code, for instance, on the first processor 110a illustrated in figure 1 and to redundantly execute a second instance 111 b of the lifted program code, for instance, on the second processor 110b illustrated in figure 1 substantially in parallel with the execution of the first instance 111a of the lifted program code. In a third lockstep stage, the data processing apparatus 100 is configured to detect an inconsistency of the execution of the program code by comparing based on one or more comparison rules (defined, for instance, by a voter executor 290 described in more detail in the context of figure 2 below) the execution of the first instance 111a with the second instance 111b of the lifted program code.
As already mentioned above, the first lockstep stage may be provided by the redundancy lifting engine 250 of the data processing apparatus 100 illustrated in figure 2. The main elements illustrated in figure 2 are the lockstep configuration (referred to as "redundant configuration" in figure 2) 210, a redundant controller 260 and the plurality of software lockstep program code templates 270.
In the embodiment shown in figure 2, the application, i.e. program code, which may be provided as application source code 201a and/or application bytecode 201b, is parsed by a compiler 261 and/or a coder interpreter 262 of the redundant controller 260. Based on the lockstep, i.e. redundant configuration 210, the redundant controller 260 regenerates, i.e. lifts the application code with software lockstep control logic, i.e. additional assembly code will be added at the necessary control points of the application code for obtaining the lifted program code 111. Furthermore, the redundant controller 260 generates the voter executor 290 containing the more comparison rules. In an embodiment, the data processing apparatus 100 may implement the voter executor 290 as a service or a separate process. Based on voter and datatype bindings 215 defined by the lockstep configuration 210, the lifted application code 111 may interoperate with the voter executor 290, for instance, for input/output synchronization, execution comparison, error report and the like.
As illustrated in figure 2, the lockstep, i.e. redundant configuration 210 used by the data processing apparatus 100 for generating the lifted application code 111 may comprise the following information for defining details of the software lockstep. In an embodiment, the lockstep, i.e. redundant configuration 210 may comprise redundant level information 211 for defining the behaviour of the software lockstep on an instruction level and/or a code block level. In an embodiment, the lockstep, i.e. redundant configuration 210 may comprise redundant scope information 213 for defining which portion(s) of the program code should be run with the software lockstep (i.e. only the selected portions of the program will be lifted and checked by the redundant controller 260). In an embodiment, the lockstep, i.e. redundant configuration 210 may comprise input/output information 212 defining one or more initial values that are provided into a selected code block as input, as well as one or more variables or register values that are used for comparing the two instances 111a,b of the program code 111. In an embodiment, the lockstep, i.e. redundant configuration 210 may comprise compare request information 214 defining how to compare the outputs of the first and the second instance 111 a,b of the program code 111. In an embodiment, the compare request information 214 may prompt the data processing apparatus 100 to simply check whether the outputs of the first and the second instance 111a, b of the program code 111 are equal or not. In a further embodiment, the compare request information 214 may include a mathematical equation for generating the comparison results based on the outputs of the first and the second instance 111 a,b of the program code 111. In an embodiment, as already mentioned above, the lockstep, i.e. redundant configuration 210 may comprise voter and datatype binding information 215, which may comprise a glue code for combining the compare request information 214 with a basic voter operations (OP) template 275 of the software lockstep templates 270 described in the following in more detail. In an embodiment, the basic voter OP template 275 may define basic operations, such as an addition, reduction, multiplication operation and the like. In an embodiment, the lifting engine 250 uses the compare request information 214 and the datatypes defined by the voter and datatype binding information 215 for apply this information to the basic voter OP template 275, in order to generate the final comparison code which may have the form of a mathematical equation. The software lockstep templates 270 comprise pre-defined program code segments for the lifted application code 111 that may be configured to synchronize the execution of the first instance 111a and the second instance 111 b of the lifted program code 111, copy data from and to one or more registers or the memory 130, as well as the one or more comparison rules for comparing the outputs form the redundant execution of the two instances 111a, b of the lifted application code 111. The redundant controller 260 is configured to integrate the software lockstep templates 270 into the application code 201 a, b for generating the lifted application code 111 in accordance with the lockstep configuration 210.
More specifically, as illustrated in figure 2, the software lockstep templates 270 may comprise a control jump template 271, which may be a segment of code that will redirect the execution of a program to another code location, e.g. jump to a different part of the application context, or call an external function from a linked library. In an embodiment, the software lockstep templates 270 may comprise a register copy template 272 comprising a segment of code to copy one or more register values to a further entity, such as the shared memory 130. The register copy template 272 may depend on the specific hardware of the data processing apparatus 100, because different hardware may have different calls, functions, instructions to perform such operations.
In an embodiment, the software lockstep templates 270 may comprise an input sync template 273 which may comprise a segment of code to synchronize the input of the controlled program code 111 , or to make sure the redundancies are starting at the same time. For instance, the input sync template 273 may synchronize a start time of the first and second instance 111a,b of the program code 111. In an embodiment, the input sync template 273 may make use of a time synchronization protocol, e.g. GTP, to synchronize the start at the same timestamp (this usually does not require any control or management from the voter side). In an embodiment, the input sync template 273 may be configured to communicate with the controlled voter 290 for comparison or error report (as shown in figure 6, where there is an additional connection between an input sync 401 and the communication channel 140. In an embodiment, the input sync template 273 may be implementation specific and may be implemented inside of the lifted code or the voter code.
In an embodiment, the software lockstep templates 270 may comprise a send output template 274 which may comprise a segment of code for sending the output value to the controlled voter 290. The send output template 274 may depend on the type of communication channel 140 and/or communication protocol used. In an embodiment, the send output template 274 may be implementation specific and may be implemented inside of the lifted code or the voter code.
As already described above and further illustrated in figure 3, the redundant lifting engine 250, in particular the redundant controller 260 provides the functionality for lifting the original application code 201 a, b based on the lockstep configuration 210 into the lifted application code 111 and for generating the one or more comparison rules defined by the voter 290. As already described above, the redundant lifting engine 250, in particular the redundant controller 260 may use compiler features to parse the application source code 201a, or use the intermediate byte code 201b to analyse and identify the code structure of the application 111 , e.g. function names and parameters that are configured by the lockstep configuration to be run under software lockstep control. As already described above, the redundant controller 260 further combines the lockstep configuration 210 with the software lockstep templates 270 to generate the lifted program code 111, e.g. as software lockstep binary code. As already described above, the additional software lockstep control code provided by one or more of the lockstep templates 270 is injected into the application code 201a, b for interacting with the generated software lockstep voter executor 290.
Figures 4 to 6 show further details of exemplary embodiments for the redundant execution of an application by the data processing apparatus 100 according to an embodiment. Figure 4 illustrates in more detail the interaction between the additional software lockstep code elements (illustrated as grey boxes in figure 4) and the original application code elements (illustrated as white boxes in figure 4) of the first instance 111a and the second instance 111 b of the lifted application code 111. As can be taken from figure 4, the uncontrolled application code 201 a, b executes normally, before it goes into the controlled code by an additional control jump inserted in the original code. An input sync call 401 may provide the timing synchronization, which may ensure that the redundant applications 111a,b start the controlled coded in an acceptable deviation of time. In an embodiment, the data processing apparatus 100 may comprise a time synchronization module, if the redundant applications 111a,b are running on two different machines or systems on a chip, which are based on different time resources, e.g. clocks. It also provides input, such as random numbers that will make sure the calculation in the controlled code will generate comparable output (different random numbers may generate results with a huge gap for comparison). This is because, some program codes 111, such as image processing algorithms, require a random number as input. However, in order to process the two instances of the program code 111 redundantly, it has to be made sure that the input random number at each control jump (such as illustrated in figure 6) is identical for each instance 111a,b. For example, in figure 6 the "in" parameter of the function block "Func_bloc" may contain the random number, but this must be synchronized for both instances 111 a,b, because otherwise the calculation will generate different or incomparable outputs.
In an embodiment, the output(s) of the lifted application code instances 111a, b may be register states, function call output variables, or any other states and variables used in the respective code segment, which may be defined by the redundant configuration 210. A control jump at the end of the lifted application code instances 111a, b may collect the outputs of the redundant application code and forward them to the voter 290 for further validation and comparison. In an embodiment, the communication between an output sync control 403 and the voter 290 may depend on the implementation of the communication channel 140. For lower performance overhead and fast reaction from the voter 290 a shared memory 130 may be used. In a further embodiment, the communication interface 120 may be used, for instance, using UDP or another communication protocol.
As illustrated in figure 4, the voter 290 comprises an output receiver 291 for receiving as input the data provided by the output sync control with a timestamp. A voter algorithm executor 295 of the voter 290 compares the results using the one or more comparison rules defined by the lockstep configuration 210. In case of any mismatch of the comparison result provided by the voter algorithm executor of the voter 290, an error reporter 293 of the voter 290 may generate one or more error messages for the other components of the data processing apparatus 100, e.g. error handling or monitoring messages.
Figure 5 shows further details of a further exemplary embodiment for the redundant execution of an application 201a, b on an instruction level by the data processing apparatus 100 according to an embodiment. In the example shown in figure 5, the lifted program code 111a,b comprises a register push instruction, so that the state of each instruction execution may be stored in the shared memory 130. In the exemplary embodiment shown in figure 5, the synchronization of the output may be achieved by using preserved result queues, which may be automatically compared by the voter 290 for checking the results. The register values may be saved with an instruction signature, so that the voter 290 is able to know from which instruction they were generated. As the voter 290 may only check whether the results are equal or not, in this exemplary embodiment there may be no voter algorithm execution component 295 (as in the embodiment of figure 4).
Figure 6 shows further details of a further exemplary embodiment for the redundant execution of an application 201 a, b on a functional level by the data processing apparatus 100 according to an embodiment. In the embodiment shown in figure 6 the lifted code 111a,b is a code block, i.e. a function call, and the return value of the function call is compared by the voter 290. The output synchronization 403 uses the configured communication channel 140 to send the output to the voter 290, which operates as already described above in the context of the embodiment of figure 4.
Figure 7 is a flow diagram illustrating a method 700 for monitoring the execution of a program code defining at least a portion of a software application using a software lockstep scheme. The method 700 comprises a step 701 of lifting the program code based on a lockstep configuration and a plurality of software lockstep program code templates. The method 700 further comprises a step 703 of executing a first instance 111a of the lifted program code 111 and a step 705 executing a second instance 111 b of the lifted program code 111 substantially in parallel with the execution of the first instance 111a of the lifted program code 111. Moreover, the method 700 comprises a step 707 of detecting an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance 111a with the second instance 111 b of the lifted program code 111.
The method 700 can be performed by the data processing apparatus 100 according to an embodiment. Thus, further features of the method 700 result directly from the functionality of the data processing apparatus 100 as well as the different embodiments thereof described above and below.
Embodiments of the data processing apparatus 100 disclosed herein provide one or more of the following advantages: comparable performance with a hardware lockstep with greater flexibility; dynamic adjustment on the redundant execution; whitebox, greybox and/or blackbox support; high speed comparison of register values; high speed data transfer through shared memory 130; configurable level and scope of redundancy for the user application; configurable voter algorithms and data bindings; low performance overhead; control instrumentation can be certified; and/or flexible deployment on the same SoC or over a communication network.
The person skilled in the art will understand that the "blocks" ("units") of the various figures (method and apparatus) represent or describe functionalities of embodiments of the present disclosure (rather than necessarily individual "units" in hardware or software) and thus describe equally functions or features of apparatus embodiments as well as method embodiments (unit = step). In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the described embodiment of an apparatus is merely exemplary. For example, the unit division is merely logical function division and may be another division in an actual implementation. For example, a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented by using some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
In addition, functional units in the embodiments of the disclosure may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.

Claims

1. A data processing apparatus (100) configured to monitor the execution of a program code defining at least a portion of a software application using a software lockstep scheme, wherein the data processing apparatus (100) is configured to: lift the program code based on a lockstep configuration and a plurality of software lockstep program code templates; execute a first instance (111a) of the lifted program code; execute a second instance (111b) of the lifted program code substantially in parallel with the execution of the first instance (111a) of the lifted program code; and detect an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance (111a) with the second instance (111b) of the lifted program code.
2. The data processing apparatus (100) of claim 1, wherein the data processing apparatus (100) is further configured to generate the one or more comparison rules based on the lockstep configuration and the plurality of software lockstep program code templates.
3. The data processing apparatus (100) of claim 1 or 2, wherein the data processing apparatus (100) comprises a memory (130) configured to store the plurality of software lockstep program code templates.
4. The data processing apparatus (100) of claim 3, wherein the data processing apparatus (100) is configured to store a first output generated by the execution of the first instance of the lifted program code and a second output generated by the execution of the second instance of the lifted program code in the memory (130).
5. The data processing apparatus (100) of any one of claims 1 to 3, wherein the data processing apparatus comprises a communication interface (120) configured to transmit a first output generated by the execution of the first instance (111a) of the lifted program code and a second output generated by the execution of the second instance (111b) of the lifted program code (111) to a remote network node.
6. The data processing apparatus (100) of any one of the preceding claims, wherein the data processing apparatus (100) comprises a first processor (110a) configured to execute the first instance (111a) of the lifted program code (111) and a second processor (110b) configured to execute the second instance (111b) of the lifted program code (111) substantially in parallel with the execution of the first instance (111a) of the lifted program code (111).
7. The data processing apparatus (100) of any one of the preceding claims, wherein for lifting the program code based on the lockstep configuration and the plurality of software lockstep program code templates the data processing apparatus (100) is configured to parse the program code and to insert, based on the lockstep configuration, one or more of the plurality of software lockstep program code templates into the program code.
8. The data processing apparatus (100) of claim 6, wherein the data processing apparatus (100) is configured to parse the program code as source code or byte code.
9. The data processing apparatus (100) of any one of the preceding claims, wherein the program code comprises one or more functional blocks and wherein each functional block comprises one or more instructions, wherein the data processing apparatus (100) is configured to detect an inconsistency of the execution of the program code on the level of the one or more functional blocks and/or on the level of the one or more instructions.
10. The data processing apparatus (100) of any one of the preceding claims, wherein the plurality of software lockstep program code templates comprise at least one of: a control jump, a register copy, an input sync, and a send output.
11. The data processing apparatus (100) of any one of the preceding claims, wherein the lockstep configuration comprises information about a least one of: a redundancy level, one or more input/output parameters, a redundancy scope, a comparison request, and one or more binding parameters.
12. A method (700) for monitoring the execution of a program code defining at least a portion of a software application using a software lockstep scheme, wherein the method (700) comprises: lifting (701) the program code based on a lockstep configuration and a plurality of software lockstep program code templates; executing (703) a first instance (111a) of the lifted program code (111); executing (705) a second instance (111b) of the lifted program code (111) substantially in parallel with the execution of the first instance (111a) of the lifted program code (111); and detecting (707) an inconsistency of the execution of the program code by comparing based on one or more comparison rules the execution of the first instance (111a) with the second instance (111b) of the lifted program code (111).
13. A computer program product comprising a computer-readable storage medium for storing program code which causes a computer or a processor to perform the method (700) of claim 12 when the program code is executed by the computer or the processor.
15
PCT/EP2021/085828 2021-12-15 2021-12-15 Data processing apparatus and method implementing a software lockstep WO2023110069A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/EP2021/085828 WO2023110069A1 (en) 2021-12-15 2021-12-15 Data processing apparatus and method implementing a software lockstep
CN202180105000.6A CN118401924A (en) 2021-12-15 2021-12-15 Data processing device and method for realizing software lock step
EP21839151.4A EP4427134A1 (en) 2021-12-15 2021-12-15 Data processing apparatus and method implementing a software lockstep

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2021/085828 WO2023110069A1 (en) 2021-12-15 2021-12-15 Data processing apparatus and method implementing a software lockstep

Publications (1)

Publication Number Publication Date
WO2023110069A1 true WO2023110069A1 (en) 2023-06-22

Family

ID=79270098

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2021/085828 WO2023110069A1 (en) 2021-12-15 2021-12-15 Data processing apparatus and method implementing a software lockstep

Country Status (3)

Country Link
EP (1) EP4427134A1 (en)
CN (1) CN118401924A (en)
WO (1) WO2023110069A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180089059A1 (en) 2016-09-29 2018-03-29 2236008 Ontario Inc. Non-coupled software lockstep
US20200192742A1 (en) * 2017-10-05 2020-06-18 Arm Limited Error recovery for intra-core lockstep mode
US11068360B2 (en) * 2019-05-31 2021-07-20 Huawei Technologies Co., Ltd. Error recovery method and apparatus based on a lockup mechanism

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180089059A1 (en) 2016-09-29 2018-03-29 2236008 Ontario Inc. Non-coupled software lockstep
US20200192742A1 (en) * 2017-10-05 2020-06-18 Arm Limited Error recovery for intra-core lockstep mode
US11068360B2 (en) * 2019-05-31 2021-07-20 Huawei Technologies Co., Ltd. Error recovery method and apparatus based on a lockup mechanism

Also Published As

Publication number Publication date
EP4427134A1 (en) 2024-09-11
CN118401924A (en) 2024-07-26

Similar Documents

Publication Publication Date Title
US7797457B2 (en) Leaderless byzantine consensus
EP1236110B1 (en) A method for isolating a fault from error messages
US9477576B2 (en) Using application state data and additional code to resolve deadlocks
CN109508295B (en) Block chain consensus algorithm testing method and device, calculating device and storage medium
US11334451B2 (en) Method and apparatus for redundant data processing in which there is no checking for determining whether respective transformations are linked to a correct processor core
CN108804109B (en) Industrial deployment and control method based on multi-path functional equivalent module redundancy arbitration
CN103455393A (en) Fault tolerant system design method based on process redundancy
CN115408371A (en) Dynamic redundancy deployment method and device for redis database
CN113064755B (en) Data recovery method, device, equipment, medium and program product
CN115129110A (en) Method and device for controlling a driving function
Evrard et al. Automatic distributed code generation from formal models of asynchronous concurrent processes
CN114327484A (en) Multi-architecture supporting K8S integration and deployment method, system and storage medium
EP4427134A1 (en) Data processing apparatus and method implementing a software lockstep
JP7512529B2 (en) Data Processing Network for Data Processing
CN117499412A (en) Cluster optimization processing method based on high-availability link and related equipment thereof
CN114327673B (en) Task starting method and device, electronic equipment and storage medium
Tarafdar et al. Software fault tolerance of concurrent programs using controlled re-execution
EP3944071B1 (en) Computer system and method for validation of parallelized computer programs
US11909821B2 (en) Method for processing application programs in a distributed automation system
Di Francia Rosso et al. Ocftl: An mpi implementation-independent fault tolerance library for task-based applications
CN114115046A (en) Industrial safety controller, control method, electronic device, and storage medium
CN114443356A (en) Container cluster processing method and system
US20190026198A1 (en) Method and device for configuring an execution means and for detecting a state of operation thereof
EP3367242B1 (en) Method of error detection in a microcontroller unit
Pedersen et al. An analysis of quorum-based abstractions: A case study using Gorums to implement raft

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21839151

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2021839151

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2021839151

Country of ref document: EP

Effective date: 20240606

WWE Wipo information: entry into national phase

Ref document number: 202180105000.6

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE