KR20230120850A - Deep-learning compiler for supporting heterogeneous computing platform and method thereof - Google Patents

Abstract

Disclosed are a deep learning compile system supporting a heterogeneous computer platform and an operating method of a deep learning compiler supporting the heterogeneous computer platform. The deep learning compiler supporting the heterogeneous computing platform according to an embodiment of the present invention comprises: a memory wherein at least one program is recorded; and a processor that executes the program, wherein the program may perform a step of converting a deep neural net model into a graph connected according to data flow of operators, and a step of converting the deep neural net model into a code by considering execution performance measurement results of the operators in at least two or more heterogeneous accelerators included in a target platform and an execution order of the operators divided and assigned to the heterogeneous accelerators. Therefore, the present invention is capable of mapping the accelerator wherein the relevant graph will execute.

Description

Deep learning compiler supporting heterogeneous computing platform and its method

The described embodiment relates to a technology for automatically converting a deep neural network into a program for a hardware platform including a plurality of heterogeneous accelerators.

Various deep learning platforms such as Pytorch, TensorFlow, and Caffe that support the convenient creation and execution of deep neural networks are widely used in various fields. However, these deep learning platforms are difficult to install and run on resource-constrained embedded systems.

Accordingly, deep learning compilers that convert deep neural networks into machine codes for hardware platforms or compilable source codes (hereinafter referred to as 'code') without installing a deep learning platform have been proposed. Deep learning compilers represented by TVM, Glow, and XLA convert deep neural net models created in deep learning platforms into optimized codes that can operate quickly in the architecture of the target HW.

These existing deep learning compilers basically generate codes for target CPUs (X86, ARM, etc.) and can additionally convert specific operators of neural networks into codes that can be operated on GPUs or NPUs. For example, XLA can generate GPU or TPU code, and TVM can generate NPU (Neural Processing Unit) code called VTA.

To this end, deep learning compilers convert the input deep neural network model into a computation graph, and based on the converted computation graph, perform graph-level optimization such as fusing/elimination and operator sinking. After execution, executable code is generated on the target HW.

However, existing deep learning compilers can specify only one type of target HW. That is, you can generate code that runs neural net operators on only one accelerator. There are also deep learning compilers that provide a function of offloading the CPU to use if the designated accelerator cannot execute a specific operator, but basically only one target HW can be specified.

However, in the case of embedded systems such as the latest mobile devices, various heterogeneous accelerators such as CPU, GPU, and NPU are all embedded. All of these heterogeneous accelerators have different hardware characteristics, so their execution speed and power consumption differ depending on the type of neural network operator. Therefore, a technology capable of minimizing the execution speed or power consumption of the neural net by utilizing all the different characteristics of each of the available heterogeneous accelerators in the system is required.

An object of the described embodiment is to provide a deep learning compilation technology capable of automatically generating high-performance deep neural network execution codes operating on a heterogeneous computing platform incorporating a plurality of heterogeneous accelerators.

The disclosed embodiment is to provide a technology for converting a deep neural network into a code that can be executed in parallel on a heterogeneous accelerator by dividing a deep neural network based on a graph so as to minimize execution speed or power consumption by utilizing all the different characteristics of each of a plurality of heterogeneous accelerators. It has a purpose.

The heterogeneous computing platform support deep learning compiler according to the embodiment includes a memory in which at least one program is recorded and a processor that executes the program, and the program converts the deep neural net model into a graph connected according to the data flow of operators. and converting the deep neural net model into code by considering the execution performance measurement results of operators in at least two or more heterogeneous accelerators included in the target platform and the execution order of operators divided and allocated to the heterogeneous accelerators. can

In this case, the execution performance measurement result may be obtained through measurement of execution performance of operators in at least two or more heterogeneous accelerators included in the target platform based on the graph, or may be input from the outside.

At this time, when the program performs execution performance measurement, generating a unit performance specification file as an execution performance result of the unit operator for each accelerator measured based on the input target platform accelerator specification information; and generating a fusing operator performance specification file as a result of execution performance of operator groups for each accelerator measured based on the input fusing operator specification information.

In this case, the execution order may be determined through scheduling in consideration of execution performance measurement results or may be input from the outside.

At this time, when the program performs scheduling, matching the accelerator having the maximum execution performance to the operator based on the execution performance measurement result; generating a partition by partitioning the graph while traversing the operators in order; and determining an execution order of the created partitions.

At this time, before performing scheduling, the program further performs a step of optimizing the transformed graph independently of hardware, but for optimization, removing unnecessary operators, merging continuously generated transpose operators, and transposing At least one of moving the (Transpose) operation to the bottom of the graph and merging consecutively occurring Concat operators can be performed.

At this time, in the case of performing scheduling, the program further performs the step of dividing the graph into a plurality of branches including at least one operator based on the measurement results of the execution performance of operators in two or more heterogeneous accelerators, , The matching step may include matching an accelerator having a maximum execution performance for each of at least one operator included in each of a plurality of branches; Searching for an accelerator having maximum execution performance for an operator group included in each of a plurality of branches, based on a fusing operator performance specification file; and re-matching an accelerator matched to the individual operator with the searched accelerator when the execution performance of the operator group is better than that of the individual operator.

At this time, in the case of performing scheduling, the program further performs a step of distinguishing whether a plurality of branches are executed sequentially or in parallel, but in the matching step, the same accelerator is matched to the branches executed in parallel. In this case, if performance is improved by using parallelism, a step of reallocating different accelerators may be further performed.

At this time, in the step of creating a partition, the program traverses the operators in the branch in order, and if the current operator matches the previous operator and the same accelerator, it determines the same partition, and the current operator has the same accelerator as the previous operator. If not matched, a step of creating and adding a new partition may be further performed.

At this time, in the step of determining the execution order, the program determines the sequential execution order while traversing the breadth-first search method, and determines the partitions included in the same parallel branch set and to which different accelerators are assigned. It can be determined by parallel partitioning.

At this time, the sequential execution order and parallel partition information of the determined partitions may be stored as one of a partition schedule plan expressing execution information of partitions in a form independent of a specific platform and a partition schedule code that is a code compilable on a specific platform. .

The deep learning compilation method supporting heterogeneous computing platforms according to the embodiment includes the steps of converting a deep neural net model into a graph connected according to the data flow of operators, and the execution performance of operators in at least two or more heterogeneous accelerators included in a target platform. A step of converting the deep neural network model into code may be performed in consideration of the measurement result and the execution order of operators divided and assigned to heterogeneous accelerators.

In this case, the execution performance measurement result may be obtained through measurement of execution performance of operators in at least two or more heterogeneous accelerators included in the target platform based on the graph, or may be input from the outside.

At this time, when performing execution performance measurement, generating a unit performance specification file based on the execution performance result of the unit operator for each accelerator measured based on the input target platform accelerator specification information; and generating a fusing operator performance specification file as a result of execution performance of operator groups for each accelerator measured based on the input fusing operator specification information.

In this case, the execution order may be determined through scheduling in consideration of execution performance measurement results or may be input from the outside.

At this time, in the case of performing scheduling, matching an accelerator having maximum execution performance to an operator based on an execution performance measurement result; generating a partition by partitioning the graph while traversing the operators in order; and determining an execution order of the created partitions.

At this time, before performing scheduling, a step of optimizing the transformed graph independently of hardware is further performed, but for optimization, unnecessary operators are removed, consecutively generated transpose operators are merged, and transpose At least one of moving the operation to the bottom of the graph and merging consecutively occurring Concat operators can be performed.

At this time, in the case of performing scheduling, further performing the step of separating the graph into a plurality of branches including at least one operator based on the measurement results of the execution performance of operators in two or more heterogeneous accelerators, but matching The step may include matching an accelerator having a maximum execution performance for each of at least one operator included in each of a plurality of branches; Searching for an accelerator having maximum execution performance for an operator group included in each of a plurality of branches, based on a fusing operator performance specification file; and re-matching an accelerator matched to the individual operator with the searched accelerator when the execution performance of the operator group is better than that of the individual operator.

At this time, in the case of performing scheduling, a step of distinguishing whether sequential or parallel execution of a plurality of branches is further performed, but the matching step is, when the same accelerator is matched to branches executed in parallel, parallelism If performance is improved by using , a step of reallocating different accelerators may be further performed.

At this time, in the step of creating partitions, while traversing the operators in the branch in order, if the current operator matches the same accelerator as the previous operator, it is determined as the same partition, and the current operator does not match the same accelerator as the previous operator. If not, a step of creating and adding a new partition may be further performed.

At this time, in the step of determining the execution order, the sequential execution order is determined by traversing in a breadth-first search method, and partitions included in the same parallel branch set and to which different accelerators are assigned are designated as parallel partitions. can decide

At this time, the sequential execution order and parallel partition information of the determined partitions may be stored as one of a partition schedule plan expressing execution information of partitions in a form independent of a specific platform and a partition schedule code that is a code compilable on a specific platform. .

A graph division and schedule generation method for heterogeneous computing platform-supported deep learning compilation according to an embodiment is based on a graph connecting a deep neural net model according to data flow of operators in at least two heterogeneous accelerators included in a target platform. Measuring the execution performance of the operators, matching the accelerator with the maximum execution performance to the operator based on the execution performance measurement result, generating partitions by traversing the operators in order and dividing the graph, and executing the created partitions You can perform steps to determine the order.

At this time, the step of measuring the execution performance is the step of generating a unit performance specification file as a result of the execution performance of the unit operator for each accelerator measured based on the input target platform accelerator specification information and the input fusing operator specification information. A step of generating a fusing operator performance specification file may be performed as a result of the measured execution performance of the operator group for each accelerator.

A method for dividing a graph and generating a schedule for deep learning compilation supporting heterogeneous computing platforms according to an embodiment converts a graph into a plurality of branches including at least one operator, based on results of measurement of execution performance of operators in two or more heterogeneous accelerators. Further comprising the step of separating into , wherein the matching step comprises: matching an accelerator having a maximum execution performance for each of the at least one operator included in each of a plurality of branches; and an operator group included in each of the plurality of branches. Searching for an accelerator having the maximum execution performance based on the fusing operator performance specification file, and if the execution performance in the operator group is better than the execution performance of an individual operator, the accelerator matched to the individual operator is replayed as the searched accelerator. Matching steps can be performed.

The graph division and schedule generation method for deep learning compilation supporting heterogeneous computing platforms according to an embodiment further includes determining whether a plurality of branches are executed sequentially or in parallel, and the matching step is executed in parallel. If the same accelerator is matched to the branches and performance is improved by using parallelism, a step of reallocating different accelerators may be further performed.

According to the described embodiment, a deep neural network model can be automatically converted into a high-performance deep neural network execution code that operates on a heterogeneous computing platform incorporating a plurality of heterogeneous accelerators.

According to the described embodiment, a deep neural network may be segmented based on a graph to minimize execution speed or power consumption by utilizing all of the different characteristics of each of a plurality of heterogeneous accelerators, and an accelerator on which the corresponding graph will be executed may be mapped.

According to the described embodiment, since graph partitioning is performed based on performance results measured by actually executing neural net operators on an accelerator built into a target platform, it is possible to accurately find an optimal partition in terms of execution time or power consumption.

According to the described embodiment, since a performance result automatically generated using a profiler is used, it is possible to quickly find an optimum partition consuming the minimum execution time or the minimum power at once.

1 is a configuration diagram schematically illustrating a deep learning compiler supporting heterogeneous computer platforms according to an embodiment.
2 is a diagram showing an example of execution time of a code generated by a conventional deep learning compiler.
3 is a diagram illustrating an example of execution time of a code generated by a deep learning compiler according to an embodiment.
4 is an internal block diagram of a deep learning compiler supporting heterogeneous computer platforms according to an embodiment.
5 is a detailed internal block configuration diagram of the schedule generating unit shown in FIG. 4 according to an embodiment.
6 is a flowchart for explaining the operation of the schedule generating unit shown in FIG. 5 according to an embodiment.
7 is an internal block diagram of a deep learning compiler supporting heterogeneous computer platforms according to another embodiment.
8 is a flowchart for explaining in detail an execution performance measurement operation for each graph-based accelerator according to an embodiment.
9 is an exemplary diagram in which a graph is divided into branches.
10 is a flowchart for explaining in detail a step of matching an operator and an accelerator of a graph according to an embodiment.
11 is an exemplary diagram of a parallel branch set including multiple branches.
12 is an exemplary view of graph division according to an embodiment.
13 is a diagram showing a configuration of a computer system according to an embodiment.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

Although "first" or "second" is used to describe various elements, these elements are not limited by the above terms. Such terms may only be used to distinguish one component from another. Therefore, the first component mentioned below may also be the second component within the technical spirit of the present invention.

Terms used in this specification are for describing embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" or "comprising" implies that a stated component or step does not preclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used herein may be interpreted as meanings commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

Hereinafter, with reference to FIGS. 1 to 13 , a heterogeneous platform supporting deep learning compilation system and method according to an embodiment will be described in detail.

1 is a configuration diagram for schematically explaining a deep learning compilation system supporting heterogeneous computer platforms according to an embodiment, and FIG. 2 is a diagram showing an example of execution time of a code generated by a conventional deep learning compiler, and FIG. It is a diagram showing an example of the execution time of the code generated by the deep learning compiler according to the example.

Referring to FIG. 1 , a deep learning compiler (hereinafter referred to as 'compiler') 100 supporting heterogeneous computer platforms according to an embodiment converts an input deep neural network model to a plurality of heterogeneous accelerators 10-1, ..., 10-N) into an execution code optimized for a heterogeneous computing platform (hereinafter referred to as 'target platform') 10 including.

Here, the deep neural net model may be created by learning in advance on a deep learning platform such as TensorFlow, Pytorch, and Caffe, and may be composed of a plurality of operators.

In this case, the accelerator may be a processing unit that performs a deep neural network operation, such as a central processing unit (CPU), a graphics processing unit (GPU), and a neural processing unit (NPU).

The compiler 100 according to the embodiment converts the input deep neural net model into code that effectively runs on the target platform 10 . The compiler 100 first uses a plurality of heterogeneous accelerators 10-1, ..., 10-N included in the target platform 10 to generate code with optimal performance in the target platform 10. Perform accelerator performance profiling to measure the performance of

At this time, accelerator performance profiling means the task of measuring the time and power consumed in the execution of deep neural net operators (or groups of operators) in each accelerator.

Next, the compiler 100 determines which accelerator to execute by partitioning the deep neural network model based on the profiling result. At this time, deep neural net partitions that can be performed using multiple accelerators in parallel are also found.

Finally, the compiler 100 generates code with high parallelism that can produce optimal performance in the target platform based on this information.

2 and 3 show the execution time aspects of codes generated by existing deep learning compilers and codes generated by the compiler 100 according to the embodiment, assuming that the target platform 10 includes one CPU and one NPU. A performance comparison example is shown.

The graphs shown in FIGS. 2 and 3 are computational graphs (hereinafter referred to as 'graphs') internally generated by deep learning compilers to execute an input deep neural network model.

Each node in the graph is an operator, and edges represent data flow between operators. Also, the number written next to the operator is the operator execution time. Dotted circles mark the partitions of the graph that will run on the accelerator. Here, a partition means a set of consecutive operators to which the same accelerator is assigned.

In the diagram shown on the right side of FIG. 2, operators enclosed by solid lines represent operators that are executed faster than when each is executed in series when executed in VTA, and the number written next to the circle of the solid line is the execution time.

Figure 2 shows the execution time when the entire graph is executed on CPU and NPU without partitioning, and Figure 3 shows the execution time when the graph is divided into a plurality of partitions and executed based on the results of accelerator performance profiling. .

Existing deep learning compilers allow users to designate only one type of accelerator, so as shown in FIG. 2, the same accelerator is assigned to all operators in the graph, and then code executable in the accelerator is generated. Thus, the total execution time of the graph is the sum of the times it takes to continuously execute the operators on the user-selected accelerator.

Comparing FIGS. 2 and 3 , if the graph is divided and executed based on accelerator information that allows calculation and calculation groups to operate quickly, it is possible to execute the graph faster than when the entire graph is executed in a single accelerator. In particular, since partitions that do not depend on data can be divided into different accelerators and executed at the same time, if the graph is partitioned and executed, an additional speed-up effect through parallel execution can be obtained.

4 is an internal block diagram of a deep learning compiler supporting heterogeneous computer platforms according to an embodiment;

Referring to FIG. 4 , the compiler 100 according to the embodiment transforms the input deep neural net model into high-performance code that utilizes the plurality of heterogeneous accelerators built into the target platform 10 to the maximum based on user input information for setting the compiler. convert

Here, user input information for compiler setting may include target platform accelerator specifications and fusing operator specifications.

Among them, the target platform accelerator specification specifies the properties of each of the heterogeneous accelerators 10-1, ..., 10-N built into the target platform 10, an example of which is the following <Table 1>. Same as

Referring to <Table 1>, accelerator properties may include an accelerator name (backendName), an identifier (ID), and types of unsupported operators (nonSupportedNodes).

The compiler 100 may perform profiling, partitioning, and accelerator allocation for each partition by referring to the content described in the target platform accelerator specification.

In addition, the fusing operator group specification indicates information about a specific operator set that provides higher performance when each operation is sequentially performed than when each operation is individually performed for each type of accelerator. This is because which set of operators is efficient for which accelerator may vary depending on the HW architecture of the accelerator and the implementation of the compiler's code generation module. An example is shown in <Table 2> below.

Heterogeneous computing platform support The deep learning compiler 100, in the form of <Table 2>, if an operator group identical to the operator group (fusing operator group) specified by the user exists in the graph, the corresponding operator group is generated as a code and all Perform profiling to measure performance on the accelerator. However, if the fusing operator group specification is not given, the performance is measured for all consecutive operators in the graph. The number of operators that can be executed consecutively uses the value set internally by the compiler.

Referring back to FIG. 4 , the compiler 100 includes a deep neural network model loader 110, a graph-based hardware independent optimization unit (hereinafter referred to as an 'optimization unit') 120 and a hardware dependent optimization and accelerator code generation unit ( (hereinafter referred to as 'code generation unit') 140 may be included. Depending on the embodiment, a plurality of heterogeneous accelerators (10-1, ..., 10-1, ..., 10-N) to generate high-performance code.

The schedule generation unit 130 performs partitioning of the HW independent optimization-completed graph output from the optimization unit 120 . When the partitioning is completed, a partition list, a schedule plan indicating the execution order of the partitions included in the partition list, and a schedule code, which is code for executing partitions according to the schedule plan, are generated.

Among them, the partition list is transmitted to the code generator 140 and converted into a partition code that can operate in the plurality of heterogeneous accelerators 10-1, ..., 10-N. The generated partition code and schedule code are compiled together and converted into executable code that performs deep neural network inference.

Then, each component shown in FIG. 4 will be looked at in detail.

The deep neural net model loader 110 converts the input deep neural net model into an internal representation in the form of a graph. As shown in FIGS. 2 and 3 , the graph has a form in which nodes 21 representing operators of the deep neural network model are connected by edges 22 representing the flow of data between operators.

The optimizer 120 optimizes the graph generated by the deep neural network model loader 110 not dependent on a specific accelerator.

In this case, hardware-independent optimization may include, for example, removing unnecessary operators, merging continuously occurring transpose operators, moving transpose operations to the bottom of the graph, and merging consecutively occurring concat operators.

The schedule generator 130 divides the operators according to the results of measuring the execution performance of the operators in the at least two or more heterogeneous accelerators 10-1, ..., 10-N included in the target platform 10, and divides the operators into heterogeneous accelerators. Allocate accelerators and schedule execution order.

In other words, the execution performance (execution time and power consumption) of individual operators and operator groups of the neural net is measured on the accelerator specified in the target platform accelerator specification using a profiler, and the input graph is drawn in the fastest or smallest way based on the measured results. Match operators and accelerators to run on full power.

Graph partitioning is performed based on this matched information, and a partition list in which partitions are sequentially connected is created. In this case, the partition refers to a subgraph to which an accelerator is assigned.

In addition, by analyzing the graph topology, information on partitions that can be executed in parallel is extracted and partition schedule plans and codes are created. In addition, each partition included in the partition list is transmitted to the code generator 140 to generate a partition code optimized for the accelerator.

A detailed description of the schedule generating unit 130 will be described later with reference to FIG. 5 .

The code generator 140 converts the deep neural network model into code optimized for an accelerator assigned to each operator and an execution order based on the result of the graph-based schedule by the schedule generator 130 . That is, the partition received as input is converted into code that operates with optimal performance considering the accelerator architecture.

5 is a detailed internal block diagram of a schedule generator according to an embodiment, and FIG. 6 is a flowchart for explaining the operation of a schedule generator according to an embodiment.

Referring to FIG. 5 , the schedule generator 130 according to the embodiment includes a profiler 131, a branch analyzer 132, an accelerator allocator 133, a graph divider 134, and a partition scheduler 135. can include

The profiler 131 uses a plurality of heterogeneous accelerators 10-1, ..., 10-N included in the target platform 10 to generate execution code that produces optimal performance in the target platform 10. ) performs accelerator performance profiling to measure each performance (S210).

At this time, the accelerator performance profiling measures the time and power consumed by each of the plurality of heterogeneous accelerators 10-1, ..., 10-N to execute an operator or operator group constituting the deep neural network model, To create a performance specification file.

At this time, the accelerator performance profiling may be performed through the profiler 150 located outside the compiler as shown in FIG. 7, and receives the performance specification file generated through the external profiler 150 as input. can be used

8 is a flowchart for explaining in detail an execution performance measurement operation for each graph-based accelerator according to an embodiment.

Referring to FIG. 8, the step of measuring the execution performance of each graph-based accelerator (S210) is largely a step of generating a unit operator performance specification file for each accelerator through profiling (S211 to S213) and a fusing operator group corresponding to the partition. It may include generating a performance specification file (S214 to S216).

That is, the profiler 131 creates partitions including only one operator by dividing the graph into individual operator units (S211).

Then, the profiler 131 generates code executable in the accelerator specified in the target platform accelerator specification from the subgraphs generated by the hardware-dependent optimization module 120 (S212).

Then, the profiler 131 generates a unit operator performance specification file by recording the average execution time and power consumption of operators while repeatedly executing the execution code generated on each accelerator. As an example, the unit operator performance specification file is shown in <Table 3> below.

Referring to Table 3, the unit operator performance specification file includes an operator identifier (ID), execution time, and power consumption.

In this case, the operator ID is generated using at least one of the operator type, input data type, input array size, and used parameter value. In this case, operators with the same ID are regarded as the same operator.

Next, the profiler 131 traverses the graph, finds consecutive operators specified in the fusing operation specification, and creates independent partitions (S214).

Then, the profiler 131 generates code executable in the accelerator specified in the target platform accelerator specification from the partitions generated by the hardware-dependent optimization module 120 (S215).

Then, the profiler 131 creates a fusing operator performance specification file by specifying and recording the execution time and power consumption while executing the execution code generated on each accelerator. As an example, a fusing operator performance specification file is shown in <Table 4> below.

As shown in <Table 4>, the fusing operator group performance specification file includes the operator group ID, execution time, and power consumption specified in the fusing operator specification. At this time, the ID of the operator group is a value obtained by connecting the IDs of each operator included in the operator group with '-'.

Referring back to FIGS. 5 and 6 , the branch analyzer 132 divides the input graph into a plurality of branches based on the execution performance of each accelerator measured in S210 (S220). At this time, branches that should be executed sequentially and branches that may be executed in parallel are extracted from the graph.

9 is an exemplary diagram in which a graph is divided into branches.

Referring to FIG. 9, when the graph is G(V, E), V is a set of nodes of G and E is a set of edges. Each node contains information about a single operator.

In this case, a branch may be defined as a subgraph composed of only consecutive nodes and not including a path branching from G(V,E). For example, four branches are extracted from G shown in FIG. 8: branch 1, branch 2-1, branch 2-2 and branch 3.

In this case, a parallel branch may be defined as two branches sharing an input node and an output node. For example, branches 2-1 and 2-2 shown in FIG. 8 may be extracted as parallel branches.

Referring back to FIGS. 5 and 6 , the accelerator determining unit 130 allocates an accelerator for each node to optimize performance for each branch (S230).

At this time, the accelerator decision is to match the operator and accelerator that can produce the best performance.

In this case, the type of performance may be a minimum execution time or a minimum power consumption, but this is only an embodiment and the present invention is not limited thereto. That is, the type of performance can be input from the user or defined internally by the compiler.

10 is a flowchart for explaining in detail a step of matching an operator and an accelerator of a graph according to an embodiment.

Referring to FIG. 10 , the accelerator allocator 133 loads the execution speed and power consumption of individual operators and operator groups from the unit operator performance specification file and the fusing operator performance specification file (S231).

Then, the accelerator allocator 133 traverses the individual operators included in each branch to find and match the accelerator that executes each operator the fastest or consumes the least power (S232).

In addition, the accelerator allocation unit 133 searches for an operator group included in the fusing operation specification among operators included in each branch and finds an accelerator having the minimum execution time and power consumption of the corresponding group (S233).

The accelerator allocator 133, if the execution time of the operator group in the accelerator found in S233 is better than the performance of executing individual nodes on the accelerators found in S232, all the operators included in the group are transferred to the accelerator found in S233. Matching (S234).

Then, when the same accelerator is matched to the parallel branches, the accelerator allocation unit 133 reallocates different accelerators when it is determined that performance is improved by using parallelism (S235).

11 is an exemplary diagram of a parallel branch set including multiple branches.

Referring to FIG. 11 , the accelerator allocator 131 may allocate an accelerator that can execute branches b2 and b3 belonging to the same parallel branch group the fastest. If the criterion for allocating an accelerator is power consumption, an accelerator consuming the least amount of power may be allocated.

Referring back to FIGS. 5 and 6 , the graph division unit 134 divides the graph into several operator groups (S240). That is, the graph partitioning unit 134 traverses the operators in the branch in order, and if the current operator matches the previous operator with the same accelerator, the graph division unit 134 groups them into the same partition, otherwise creates and adds a new partition. At this time, the first operator is unconditionally added to the new partition.

12 is an exemplary view of graph division according to an embodiment.

Referring to FIG. 12, the graph on the left shows partitions (p1-p4) created based on branch information and accelerator matching information for each operator.

Referring back to FIGS. 5 and 6 , the partition scheduler 135 determines the partition execution order to minimize the execution time of the input deep neural net based on the branch analysis information in S240 (S250).

That is, the partition scheduler 135 determines a sequential execution order while traversing the partitions in a breath-first manner. For example, referring to FIG. 12 , the sequential execution order determined by the partition scheduler may be p1->p2->p3->p4.

Then, the partition scheduler 135 finds a partition included in the same parallel branch set and to which different accelerators are assigned, and determines it as a parallel partition. For example, referring to FIG. 12, p2 and p3 may be parallel partitions.

Meanwhile, two embodiments are possible for the sequential execution order of the determined partition and the storage form of parallel partition information.

According to one embodiment, it is a partition schedule plan that expresses execution information of partitions in a form that is not dependent on a specific platform.

Here, the partition schedule plan may include the total number of partitions, the names of accelerators to execute each partition, the names of nodes included in the partitions, and information on parallel partitions. And, except for parallel partitions, partitions are executed in the order specified in the partition name field (partitionNames).

According to another embodiment, the partition schedule code is in the form of code compilable on the platform.

Here, the partition schedule code is a C language code that executes a partition specified in the partition schedule plan.

In addition, if the partition code and the partition schedule code are compiled together in the target platform, they become executable codes that can be executed on the platform.

The generated execution code may be target platform-dependent machine code that has the same input/output type as that of the deep neural network model input by the compiler and performs the same inference operation.

In addition, although not shown in the drawing, the execution sequence according to an embodiment of the present invention may apply information input from the outside without performing a separate scheduling task.

13 is a diagram showing a configuration of a computer system according to an embodiment.

The heterogeneous computer platform supporting deep learning compiler 100 according to the embodiment may be implemented in a computer system 1000 such as a computer-readable recording medium.

Computer system 1000 may include one or more processors 1010, memory 1030, user interface input devices 1040, user interface output devices 1050, and storage 1060 that communicate with each other via a bus 1020. can In addition, computer system 1000 may further include a network interface 1070 coupled to network 1080 . The processor 1010 may be a central processing unit or a semiconductor device that executes programs or processing instructions stored in the memory 1030 or the storage 1060 . The memory 1030 and the storage 1060 may be storage media including at least one of volatile media, nonvolatile media, removable media, non-removable media, communication media, and information delivery media. For example, memory 1030 may include ROM 1031 or RAM 1032 .

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art can implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand that there is Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting.

100: heterogeneous computer platform support deep learning compiler
110: Deep Neural Net Model Loader
120: graph-based hardware independent optimization unit
130: graph division and schedule generation unit 131: profiler
132: branch analysis unit 133: accelerator assignment
134: graph division unit 135: partition scheduler
140: hardware dependent optimization and accelerator code generation unit

Claims (20)

a memory in which at least one program is recorded; and
A processor that executes a program;
program,
converting the deep neural network model into a graph connected according to the data flow of operators; and
A heterogeneous computing platform that performs a step of converting a deep neural network model into code by considering the execution performance measurement results of operators in at least two heterogeneous accelerators included in the target platform and the execution order of operators divided and allocated to the heterogeneous accelerators. Support deep learning compiler. The method of claim 1, wherein the execution performance measurement result is
A deep learning compiler supporting heterogeneous computing platforms, obtained through measurement of execution performance of operators in at least two heterogeneous accelerators included in a target platform based on a graph or input from the outside. The method of claim 2, wherein the program, when performing performance measurement,
generating a unit performance specification file as a result of execution performance of unit operators for each accelerator measured based on input target platform accelerator specification information; and
A deep learning compiler supporting heterogeneous computing platforms, including generating a fusing operator performance specification file as a result of execution performance of operator groups for each accelerator measured based on input fusing operator specification information. 4. The method of claim 3, wherein the order of execution is
Deep learning compiler supporting heterogeneous computing platforms, determined through scheduling considering execution performance measurement results or input from the outside. The method of claim 4, wherein the program, when performing scheduling,
matching an accelerator with maximum execution performance to an operator based on an execution performance measurement result;
generating a partition by partitioning the graph while traversing the operators in order; and
A deep learning compiler supporting heterogeneous computing platforms, which performs a step of determining the execution order of the generated partitions. The method of claim 4, wherein the program,
Before performing scheduling, further performing the step of optimizing the transformed graph independently of hardware,
For optimization, heterogeneous computing platform support deep, which performs at least one of: removing unnecessary operators, merging consecutively occurring Transpose operators, moving Transpose operations to the bottom of the graph, and merging consecutively occurring Concat operators. running compiler. The method of claim 5, wherein the program,
When performing scheduling,
Further performing the step of separating the graph into a plurality of branches including at least one operator based on the measurement results of the execution performance of the operators in the two or more heterogeneous accelerators,
The matching step is
matching an accelerator having a maximum execution performance for each of at least one operator included in each of a plurality of branches;
Searching for an accelerator having maximum execution performance for an operator group included in each of a plurality of branches, based on a fusing operator performance specification file; and
If the execution performance of the operator group is better than the execution performance of individual operators, a deep learning compiler supporting heterogeneous computing platforms further performs a step of re-matching the accelerator matched to the individual operator with the found accelerator. The method of claim 5, wherein the program,
When performing scheduling,
Further performing the step of distinguishing whether the plurality of branches are executed sequentially or in parallel,
The matching step is
A deep learning compiler supporting heterogeneous computing platforms, which further performs a step of reallocating different accelerators when performance is improved by using parallelism when the same accelerator is matched to branches executed in parallel. The method of claim 5, wherein the program,
At the stage of creating a partition,
While traversing the operators in the branch in order, if the current operator matches the previous operator and the same accelerator, it is determined as the same partition,
A deep learning compiler supporting heterogeneous computing platforms that performs additional steps of creating and adding new partitions when the current operator does not match the same accelerator as the previous operator. The method of claim 8, wherein the program,
At the stage of determining the execution order,
determine the sequential order of execution by traversing in a breadth-first manner;
A deep learning compiler supporting heterogeneous computing platforms that determines partitions that are included in the same set of parallel branches and have different accelerators assigned to them as parallel partitions. The method of claim 10, wherein the sequential execution order and parallel partition information of the determined partition are:
A deep learning compiler supporting heterogeneous computing platforms that is stored as one of a partition schedule plan that expresses execution information of partitions in a form independent of a specific platform and a partition schedule code that is a code form that can be compiled on a specific platform. converting the deep neural network model into a graph connected according to the data flow of operators; and
A heterogeneous computing platform that performs a step of converting a deep neural network model into code by considering the execution performance measurement results of operators in at least two heterogeneous accelerators included in the target platform and the execution order of operators divided and allocated to the heterogeneous accelerators. Support deep learning compilation method. According to claim 12,
The execution performance measurement result is,
Obtained through measuring the execution performance of operators in at least two or more heterogeneous accelerators included in the target platform based on the graph or input from the outside,
When performing performance measurements,
generating a unit performance specification file as a result of execution performance of unit operators for each accelerator measured based on input target platform accelerator specification information; and
A deep learning compilation method supporting heterogeneous computing platforms, which performs a step of generating a fusing operator performance specification file as a result of the execution performance of an operator group for each accelerator measured based on input fusing operator specification information. According to claim 13,
The order of execution is
It is determined through scheduling considering the execution performance measurement result or input from the outside,
When performing scheduling,
matching an accelerator with maximum execution performance to an operator based on an execution performance measurement result;
generating a partition by partitioning the graph while traversing the operators in order; and
A deep learning compilation method supporting heterogeneous computing platforms, which performs a step of determining an execution order of generated partitions. According to claim 14,
Before performing scheduling, further performing the step of optimizing the transformed graph independently of hardware,
For optimization, heterogeneous computing platform support deep, which performs at least one of: removing unnecessary operators, merging consecutively occurring Transpose operators, moving Transpose operations to the bottom of the graph, and merging consecutively occurring Concat operators. Running compile method. According to claim 14,
When performing scheduling,
Further performing the step of separating the graph into a plurality of branches including at least one operator based on the measurement results of the execution performance of the operators in the two or more heterogeneous accelerators,
The matching step is
matching an accelerator having a maximum execution performance for each of at least one operator included in each of a plurality of branches;
Searching for an accelerator having maximum execution performance for an operator group included in each of a plurality of branches, based on a fusing operator performance specification file; and
If the execution performance in the operator group is better than the execution performance of the individual operator, a deep learning compilation method supporting heterogeneous computing platforms further performing a step of re-matching the accelerator matched to the individual operator with the found accelerator. measuring execution performance of operators in at least two or more heterogeneous accelerators included in a target platform based on a graph in which a deep neural network model is connected according to data flows of operators;
matching an accelerator with maximum execution performance to an operator based on an execution performance measurement result;
generating a partition by partitioning the graph while traversing the operators in order; and
A graph partitioning and schedule generation method for deep learning compilation supporting heterogeneous computing platforms, which performs a step of determining an execution order of generated partitions. 18. The method of claim 17, wherein measuring execution performance comprises:
generating a unit performance specification file as a result of execution performance of unit operators for each accelerator measured based on input target platform accelerator specification information; and
A graph division and schedule generation method for deep learning compilation supporting heterogeneous computing platforms, which performs steps of generating a fusing operator performance specification file as a result of the execution performance of operator groups for each accelerator measured based on input fusing operator specification information. According to claim 18,
Separating the graph into a plurality of branches including at least one operator based on the measurement results of the execution performance of the operators in the two or more heterogeneous accelerators,
The matching step is
matching an accelerator having a maximum execution performance for each of at least one operator included in each of a plurality of branches;
Searching for an accelerator having maximum execution performance for an operator group included in each of a plurality of branches, based on a fusing operator performance specification file; and
If the execution performance of an operator group is better than that of an individual operator, a step of re-matching an accelerator matched to the individual operator with a searched accelerator is performed, and graph division and schedule generation for deep learning compilation supporting heterogeneous computing platforms are performed. method. According to claim 19,
Further comprising the step of distinguishing whether the plurality of branches are executed sequentially or in parallel,
The matching step is
When the same accelerator is matched to branches executed in parallel, if performance is improved by using parallelism, a step of reallocating different accelerators is further performed, graph division and schedule for deep learning compilation supporting heterogeneous computing platforms How to create.

Images