JP7579972B2 - Scaling Neural Architectures for Hardware Accelerators - Google Patents

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Patent Application No. 17,175,029, filed February 12, 2021, which claims the benefit under 35 U.S.C. § 119(e) of U.S. Patent Application No. 63/137,926, filed January 15, 2021, the disclosure of which is incorporated herein by reference.

Background A neural network is a machine learning model that includes one or more layers of nonlinear operations to predict an output for a received input. In addition to an input layer and an output layer, some neural networks include one or more hidden layers. The output of each hidden layer can be input to another hidden layer or to an output layer of the neural network. Each layer of a neural network can generate each output from a received input depending on the value of one or more model parameters of that layer. The model parameters can be weights or biases determined by a training algorithm to cause the neural network to generate an accurate output.

Overview Systems implemented according to aspects of the present invention can reduce the latency of neural network architectures by exploring candidate neural network architectures according to each candidate's computational requirements (e.g., FLOPS), computational intensity, and execution efficiency. As described herein, computational requirements, computational intensity, and execution efficiency have been found to be the underlying causes of neural network latency on a target computing resource, as opposed to computational requirements alone impacting latency, including inference latency. Aspects of the present disclosure enable techniques for performing neural architecture exploration and scaling through latency-aware composite scaling and by expanding the space of neural network candidates to be explored based on this observed latency-to-computational, computational intensity, and execution efficiency.

Furthermore, the system can perform compound scaling to uniformly scale multiple parameters of a neural network according to multiple objectives. This can improve the performance of the scaled neural network over approaches where a single objective is considered or where the neural network scaling parameters are explored separately. Using latency-aware compound scaling, a family of neural network architectures can be quickly constructed that are scaled according to different values from an initial scaled neural network architecture and that may suit different use cases.

According to an aspect of the present disclosure, a computer-implemented method includes a method for determining an architecture of a neural network. The method includes one or more processors receiving training data corresponding to a neural network task and information specifying a target computing resource, the one or more processors performing a neural architecture search of a search space using the training data to identify an architecture of a base neural network according to a plurality of first objectives, and the one or more processors identifying a plurality of scaling parameter values for scaling the base neural network according to the information specifying the target computing resource and a plurality of scaling parameters of the base neural network. The identifying includes iteratively selecting a plurality of candidate scaling parameter values and determining a performance evaluation metric of the scaled base neural network according to the plurality of candidate scaling parameter values, the performance evaluation metric being determined according to a plurality of second objectives including a latency objective. The method may further include one or more processors generating an architecture of the scaled neural network using the architecture of the base neural network scaled according to the plurality of scaling parameter values.

These and other embodiments may each include one or more of the following features, either alone or in combination, as appropriate.

The first objectives for performing the neural architecture search may be the same as the second objectives for identifying the scaling parameter values.

The multiple first objectives and multiple second objectives may include an accuracy rate objective corresponding to the accuracy rate of the output of the basic neural network.

The performance metric may correspond at least in part to a metric of latency between the basic neural network accepting an input and generating an output when the basic neural network is scaled in response to multiple candidate scaling parameter values and deployed on a target computing resource.

The latency objective may correspond to the minimum latency between the base neural network accepting an input and producing an output when the base neural network is deployed on a target computing resource.

The search space may include candidate neural network layers, where each candidate neural network layer is configured to perform one or more operations. The search space may include candidate neural network layers that include different activation functions.

The architecture of the basic neural network may include multiple candidate components, each component having multiple neural network layers. The search space may include multiple candidate neural network layer components, including a first candidate network layer component that includes a first activation function and a second candidate network layer component that includes a second activation function that is different from the first activation function.

The information specifying the target computing resource may specify one or more hardware accelerators, and the method further includes executing the scaled neural network on the one or more hardware accelerators to perform the neural network task.

The target computing resource may include a first target computing resource, and the plurality of scaling parameter values are a plurality of first scaling parameter values, and the method may further include the one or more processors receiving information specifying a second target computing resource different from the first target computing resource, and identifying a plurality of second scaling parameter values for scaling the basic neural network in response to the information specifying the second target computing resource, and the plurality of second scaling parameter values are different from the plurality of first scaling parameter values.

The plurality of scaling parameter values are a plurality of first scaling parameter values, and the method further includes generating a scaled neural network architecture from the base neural network architecture scaled with a plurality of second scaling parameter values, the second scaling parameter values being generated as a function of the plurality of first scaling parameter values and one or more compound coefficients that uniformly modify the value of each of the first scaling parameter values.

The base neural network may be a convolutional neural network, and the multiple scaling parameters may include one or more of a depth of the base neural network, a width of the base neural network, and a resolution of the input of the base neural network.

According to another aspect, a method for determining an architecture of a neural network includes one or more processors accepting information specifying a target computing resource, one or more processors receiving data specifying an architecture of a base neural network, and one or more processors identifying a plurality of scaling parameter values for scaling the base neural network in response to the information specifying the target computing resource and a plurality of scaling parameters of the base neural network. The identifying includes iteratively performing a selection of a plurality of scaling parameter value candidates and a determination of a performance evaluation metric of the scaled base neural network in response to the plurality of scaling parameter value candidates, the performance evaluation metric being determined in response to a plurality of objectives including a latency objective, and the method further includes one or more processors generating an architecture of the scaled neural network using the architecture of the base neural network scaled in response to the plurality of scaling parameter values.

The multiple objectives may be multiple secondary objectives, and receiving data specifying an architecture of the base neural network may include one or more processors receiving training data corresponding to a neural network task, and one or more processors performing a neural architecture search of a search space using the training data to identify an architecture of the base neural network in accordance with the multiple first objectives.

Other embodiments include computer systems, devices, and computer programs stored on one or more computer storage devices, each configured to perform the operations of the methods.

FIG. 1 is a block diagram illustrating a family of scaled neural network architectures for deployment in data centers housing hardware accelerators on which the deployed neural networks run. FIG. 1 is a flow diagram of an example process for generating a scaled neural network architecture for execution on a target computing resource. 1 is an exemplary process of latency-aware complex scaling of a basic neural network architecture. FIG. 1 is a block diagram of a NAS-LACS (Neural Architecture Search-Latency Aware Composite Scaling) system according to an aspect of the present disclosure. FIG. 1 is a block diagram of an exemplary environment for implementing a NAS-LACS system.

DETAILED DESCRIPTION Overview The technology described herein generally relates to scaling neural networks for execution on different target computing resources, such as different hardware accelerators. The neural networks may be scaled according to a number of different performance objectives. These performance objectives may include the distinct objectives of minimizing processing time (referred to herein as latency) and maximizing the accuracy rate of the neural network when scaled to execute on the target computing resources.

Generally, a NAS (Neural Architecture Search) system may be deployed to select a neural network architecture from a given search space of candidate architectures according to one or more objectives. One common objective is the accuracy rate of the neural network. Generally, systems implementing NAS techniques favor networks that have a high accuracy rate after training over networks with a low accuracy rate. After a base neural network is selected following the NAS, the base neural network may be scaled according to one or more scaling parameters. Scaling may include searching for one or more scaling parameter values for scaling the base neural network, for example, by searching for scaling parameters in a coefficient search space composed of numbers. Scaling may include increasing or decreasing the number of layers the neural network has or the size of each layer to make better use of the computational and/or memory resources available for deploying the neural network.

A commonly held belief in neural architecture exploration and scaling is that the computational requirements of a neural network, measured for example in FLOPS (floating point operations per second), required to process an input through the network, is proportional to the latency between sending the input to the network and receiving the output. That is, it is believed that neural networks with low computational requirements (low FLOPS) will generate output faster than networks with high computational requirements (high FLOPS) because fewer operations are performed overall. Thus, many NAS systems choose neural networks with low computational requirements. However, it has become clear that the relationship between computational requirements and latency is not linear, as other characteristics of neural networks, such as computational intensity, parallelism, and execution efficiency, can affect the overall latency of a neural network.

The technology described herein enables LACS (Latency Aware Compound Scaling) and the expansion of the neural network candidate search space from which neural networks are selected. In the context of expanding the search space, operations and architectures may be included in the search space that are "hardware accelerator friendly" in that their addition provides higher computational intensity, execution efficiency, and parallelism suitable for deployment on various types of hardware accelerators. Such operations may include space-to-depth operations, space-to-batch operations, fused convolutional structures, and component-wise search activation functions.

Latency-aware composite scaling of neural networks can improve the scaling of neural networks over traditional approaches that do not perform optimizations according to latency. Instead, LACS may be used to identify scaled parameter values for a scaled neural network that are accurate and operate with low latency on the target computing resources.

This technology also enables multi-objective scaling that shares the objective of neural architecture exploration using NAS or similar techniques. The scaled neural network is identifiable according to the same objective used in exploring the base neural network. As a result, rather than treating each stage of base architecture exploration and scaling as a task with a separate objective, the scaled neural network can be optimized for performance in these two stages.

LACS can be integrated with existing NAS systems because, at a minimum, the same objectives can be used for both searching and scaling to build an end-to-end system for determining scaled neural network architectures. Furthermore, it can identify a family of scaled neural network architectures faster than searching-without-scaling approaches, which search for neural network architectures but do not scale them for deployment on target computing resources.

The technology described herein may enable improved neural networks over those identified by traditional search-and-scaling techniques that do not use LACS. In addition, families of neural networks with different trade-offs between objectives, such as model accuracy and inference latency, may be rapidly generated for application to various use cases. The technology may also identify neural networks for performing specific tasks faster, but the identified neural networks may perform with improved accuracy over neural networks identified using other techniques. This is at least because the neural networks identified as a result of performing the search and scaling described herein consider characteristics such as computational intensity and execution efficiency that may affect latency, rather than simply considering the computational requirements of the network. In this way, the identified neural networks can operate faster during inference without sacrificing the accuracy of the network.

This technology may further enable a generally applicable framework for quickly migrating existing neural networks to improved computing resource environments. For example, the LACS and NAS described herein may be applied when migrating the execution of an existing neural network selected for a data center with specific hardware to a data center using different hardware. In this regard, a family of neural networks to perform the tasks of the existing neural network may be quickly identified and deployed on the new data center hardware. This application method may be particularly useful in areas of rapid deployment that require state-of-the-art hardware to perform efficiently, such as networks performing tasks in computer vision or other image processing tasks.

FIG. 1 is a block diagram of a family 103 of scaled neural network architectures 104A-104N for deployment in a data center 115 housing a hardware accelerator 116 on which the deployed neural network runs. The hardware accelerator 116 can be any type of processor, such as a CPU, GPU, FPGA, or an ASIC, such as a TPU. According to aspects of the present disclosure, the family 103 of scaled neural network architectures can be generated from a base neural network architecture 101.

The architecture of a neural network refers to the characteristics that identify the neural network. For example, the architecture may include characteristics of the different neural network layers that make up a network, how these layers process inputs, how these layers interact with each other, etc. For example, a ConvNet (convolutional neural network) architecture may specify discrete convolutional layers that receive input image data, then a pooling layer, then a fully connected layer that generates output according to the neural network task, such as classifying the content of the input image data. The neural network architecture may also specify the type of operations that are performed within each layer. For example, a ConvNet architecture may specify the use of a ReLU activation function in the fully connected layer of the network.

Using the NAS, a base neural network architecture 101 can be identified according to a set of objectives. From a search space of candidate neural network architectures, the NAS can be used to identify a base neural network architecture 101 according to a set of objectives. As described in more detail herein, the search space of candidate neural network architectures can be expanded to include different network components, different operations, and different layers from which a base network that satisfies the objectives can be identified.

The set of objectives used to identify the base neural network architecture 101 can also be applied to identify scaling parameter values for each of the neural networks 104A-104N in the family 103. The base neural network architecture 101 and the scaled neural network architectures 104A-104N can be characterized by a number of parameters that are scaled to different degrees in the scaled neural network architectures 104A-104N. In FIG. 1, the neural networks 101, 104A are shown to have three scaling parameters: D, which indicates the number of layers in the neural network; W, which indicates the width of the neural network layer or the number of neurons in the neural network layer; and R, which indicates the size of the inputs processed by the neural network at a given layer.

As described in more detail herein, a system configured to perform LACS can search the coefficient search space 108 to identify multiple sets of scaling parameter values. Each scaling parameter value is a coefficient in the coefficient search space and can be, for example, a set of positive real numbers. Each network candidate 107A-107N is scaled from the base neural network 101 according to the coefficient value candidates identified as part of the search in the coefficient search space 108. The system can apply any of a variety of search techniques to identify the coefficient candidates, such as a Pareto frontier search or a grid search. For each network candidate 107A-107N, the system can evaluate a performance metric of the network candidate in performing a neural network task. The performance metric can be based on multiple objectives, including a latency objective that measures the latency of the network candidate between accepting an input and generating a corresponding output as part of performing a neural network task.

After the scaling parameter value search is performed on the coefficient search space 108, the system may receive a scaled neural network architecture 109. The scaled neural network architecture 109 is scaled from the base neural network 101 using the scaling parameter values that result in the maximum performance evaluation metric of the network candidates 107A-107N identified during the search in the coefficient search space.

From the scaled neural network architecture 109, the system can generate a family 103 of scaled neural network architectures 104A-104N. The family 103 can be generated by scaling the scaled neural network architecture 109 according to different values. Each scaling parameter value of the scaled neural network architecture 109 can be scaled uniformly to generate other scaled neural network architectures in the family 103. For example, each scaling parameter value of the scaled neural network architecture 109 can be scaled by increasing each scaling parameter value by a factor of two. The scaled neural network architecture 109 can be scaled for different values - or "compound coefficients" - that are applied uniformly to each scaling parameter value of the scaled neural network architecture 109. In some implementations, the scaled neural network architecture 109 is scaled in a different manner, such as by scaling each scaling parameter value separately, to generate the scaled neural network architectures in the family 103.

By scaling the scaled neural network architecture 109 to different values, different neural network architectures can be quickly generated to perform tasks for various use cases. The different use cases can be specified as different tradeoffs between objectives used to identify the scaled neural network architecture 109. For example, one scaled neural network architecture can be identified as meeting a higher accuracy threshold at the expense of a higher latency during execution. Another scaled neural network architecture can be identified as meeting a lower accuracy threshold, but can run with low latency on the hardware accelerator 116. Another scaled neural network architecture can be identified as balancing the tradeoff between accuracy and latency on the hardware accelerator 116.

As an example, to perform a computer vision task such as object recognition, as part of an application continuously accepting video or image data and the task of identifying a particular class of objects in the received data, a neural network architecture would need to generate output in real-time or near real-time. For this exemplary task, the tolerance for accuracy may be low, so a neural network architecture scaled with an appropriate tradeoff between low latency and low accuracy may be deployed to perform the task.

As another example, a neural network architecture may be tasked with classifying all objects in a scene received from image or video data. In this example, a scaled neural network may be deployed that provides high accuracy at the expense of latency, where latency in performing this exemplary task is not considered as important as performing the task accurately. In other examples, scaled neural network architectures may be deployed that balance tradeoffs between accuracy, latency, and other objectives. No particular tradeoffs are identified or desired here when performing the neural network task.

The scaled neural network architecture 104N may be scaled using different scaling parameter values than those used to scale the scaled neural network architecture 109 to obtain the scaled neural network 104A, and may represent a different use case, e.g., a use case in which accuracy is more desirable than latency during inference.

The LACS and NAS techniques described herein may generate a family 103 of hardware accelerators 116 and may receive additional training data and information specifying multiple different computing resources, such as different types of hardware accelerators. In addition to generating a family 103 of hardware accelerators 116, the system may explore the base neural network architecture and generate a family of scaled neural network architectures for other hardware accelerators. For example, for GPUs and TPUs, the system may generate separate model families optimized for accuracy vs. latency tradeoffs for GPUs and TPUs, respectively. In some implementations, the system may generate multiple scaled families from the same base neural network architecture.

2 is a flow diagram of an exemplary process 200 for generating a scaled neural network architecture for execution on a target computing resource. The exemplary process 200 may be performed on a system comprised of one or more processors in one or more locations. For example, the NAS-LACS (Neural Architecture Search-Latency Aware Composite Scaling) system described herein may perform the process 200.

As shown in block 210, the system receives training data corresponding to a neural network task. A neural network task is a machine learning task that may be performed by a neural network. The scaled neural network may be configured to accept any type of data input and generate an output for performing the neural network task. By way of example, the output may be any type of score, classification, or regression output based on the input. Correspondingly, the neural network task may be a scoring task, a classification task, and/or a regression task to predict an output for a given input. These tasks may correspond to a variety of applications in processing images, video, text, audio, or other types of data.

The received training data may be in any format suitable for training a neural network according to one of a variety of learning techniques. Learning techniques for training a neural network may include supervised, unsupervised, and semi-supervised learning techniques. For example, the training data may include a number of training examples that the neural network may accept as inputs. The training examples may be labeled with known outputs that correspond to outputs that are to be generated by a neural network properly trained to perform a particular neural network task. For example, if the neural network task is a classification task, the training examples may be images labeled with one or more classes that categorize the objects depicted in the images.

As shown in block 220, the system accepts information specifying a target computing resource. The target computing resource data may specify characteristics of the computing resource on which at least a portion of the neural network may be deployed. The computing resource may be housed in one or more data centers or other physical locations hosting various types of hardware devices. The hardware types may include central processing units (CPUs), graphics processing units (GPUs), edge or mobile computing devices, field programmable gate arrays (FPGAs), and various types of application-specific circuits (ASICs).

Some devices may be configured for hardware acceleration, including devices configured to efficiently perform certain types of operations. These hardware accelerators, including GPUs and Tensor Processing Units (TPUs), may perform specialized functions of hardware acceleration, such as configurations for performing operations commonly associated with running machine learning models, such as matrix multiplication. Also, by way of example, these specialized functions may include matrix multiply-accumulate units available in different types of GPUs and matrix multiplication units available in TPUs.

The target computing resource data may include data about one or more target computing resource sets. The target computing resource set refers to a collection of computing devices on which one wishes to deploy the neural network. The information specifying the target computing resource set may refer to the type and quantity of hardware accelerators or other computing devices included in the target set. The target set may include devices of the same or different types. For example, the target computing resource set may specify the hardware characteristics and quantity of a particular type of hardware accelerator, including processing power, throughput, and memory capacity. As described herein, the system may generate a family of scaled neural network architectures for each device specified in the target computing resource set.

In addition, the target computing resource data may specify different sets of target computing resources, e.g., reflecting different possible configurations of computing resources housed in a data center. From this training and the target computing resource data, the system may generate a family of neural network architectures. Each architecture may be generated from a base neural network identified by the system.

As shown in block 230, the system may use the training data to perform a neural architecture search on the search space to identify a base neural network architecture. The system may use any of a variety of NAS techniques, such as techniques based on reinforcement learning, evolutionary search, or differentiable search. In some implementations, the system may directly accept data specifying the base neural network architecture, for example, without accepting training data and performing NAS as described herein. For example, the initial neural network architecture candidates may be predefined based on an analysis of types of neural network architectures suitable for performing a particular neural network task. As another example, the initial neural network architecture candidates may be randomly selected from a search space of neural network candidates. As another example, at least some characteristics of the initial neural network candidates may be selected based on similar characteristics of previously trained neural networks, such as, for example, a similar or the same number of layers, weight values, etc.

The search space refers to the candidate neural networks or a portion of candidate neural networks that may be selected as part of the basic neural network architecture. The portion of the candidate neural network architecture may refer to the components of the neural network. The architecture of the neural network may be defined according to the components of the neural network. In a neural network, each component includes one or more neural network layers. The characteristics of the neural network layers can be defined in the component level architecture, which means that the architecture can define specific operations in the component, so that each neural network in the component performs the same operations defined for the component. Also, the components may be defined in the architecture by the number of layers they have.

As part of performing the NAS, the system may iteratively identify candidate neural networks, obtain performance metrics corresponding to multiple objectives, and evaluate the candidate neural networks according to each of these performance metrics. As part of obtaining performance metrics, such as accuracy and latency metrics, for the candidate neural networks, the system may train the candidate neural networks using the received training data. Once trained, the system may evaluate the candidate neural network architectures, determine their performance metrics, and compare these performance metrics according to the currently optimal candidate.

The system may perform this search process iteratively by selecting candidate neural networks, training the networks, and comparing their performance metrics until a stopping metric is reached. The stopping metric may be a predetermined minimum threshold of performance that the current candidate network meets. Additionally or alternatively, the stopping metric may be a maximum number of search iterations, or a maximum period of time allotted to perform the search. The stopping metric may be a condition under which the performance of the neural network converges, e.g., the performance of a subsequent iteration differs from the performance of a previous iteration by less than a threshold.

In the context of optimizing various performance metrics of a neural network, such as accuracy and latency, the stopping metric may specify a predetermined threshold range that is "optimal." For example, the optimal latency threshold range may be a threshold range from a theoretical or measured minimum latency that the target computing resource achieves. The theoretical or measured minimum latency may be based on physical characteristics of the computing resource, such as the minimum time required for a component of the computing resource to physically read and process incoming data. In some implementations, the latency is held to a minimum, e.g., as close to zero delay as physically possible, and is not based on a measured or calculated target latency from the target computing resource.

The system may be configured to use machine learning models or other techniques to select the next candidate neural network architecture, where the selection may be based at least in part on learned characteristics of different candidate neural networks that are likely to perform well given the objectives of a particular neural network task.

In some examples, the system may use a multi-objective reward mechanism to identify the underlying neural network architecture as follows:

To measure the accuracy of the candidate neural network, the system may train the candidate neural network to perform the neural network task using the training set. The system may split the training data into a training set and a validation set, for example, by an 80/20 split. For example, the system may apply supervised learning techniques to calculate the error between the output generated by the candidate neural network and the ground truth labels of the training examples processed by the network. The system may utilize any of a variety of loss or error functions appropriate to the type of task for which the neural network is being trained, such as cross entropy error for classification tasks and mean squared error for regression tasks. The gradient of the error as the weights of the candidate neural network are changed may be calculated, for example, using a backpropagation algorithm, and the weights of the neural network may be updated. The system may train the candidate neural network until a stopping metric is met, such as the number of iterations for training, a maximum period, convergence, or a minimum threshold for accuracy.

(1) In addition to other performance metrics including the computational intensity of the candidate base neural network when deployed on the target computing resource and/or the execution efficiency of the candidate base neural network on the target computing resource, the system may generate performance metrics of accuracy and latency of the candidate neural network architecture on the target computing resource. In some implementations, in addition to accuracy and latency, the performance evaluation index of the candidate base neural network is based at least in part on the computational intensity and/or execution efficiency.

Latency, computational intensity, and execution efficiency can be defined as follows:

The system can explore base neural network architectures and improve the latency of the final neural network by simultaneously exploring multiple candidate neural network architectures with improved computational intensity, execution efficiency, and computational requirements, rather than only searching for and exploring networks with low computational complexity. The system can be configured to operate in this manner to reduce the overall latency of the final base neural network architecture.

In addition, particularly when the target computing resource is a data center hardware accelerator, the system can expand the architecture candidate search space from which to select a base neural network architecture, broadening the variety of available neural network candidates that are likely to operate accurately with low inference latency on the target computing resource.

Expanding the search space as described can increase the number of candidate neural network architectures that are more suitable for data center accelerator deployment, thereby enabling identification of basic neural network architectures that would not have been candidates in a search space that was not expanded in accordance with aspects of the present disclosure. In examples where the target computing resources specify hardware accelerators such as GPUs and TPUs, the search space can be expanded with candidate architectures or portions of architectures, such as components or operations that improve computational intensity, parallelism, and/or execution efficiency.

In one exemplary extension method, the search space can be extended to include neural network architecture components having layers that implement one of many different types of activation functions. For TPUs and GPUs, it has been found that activation functions such as ReLU or swish are typically low in computational intensity and typically constrained by memory on these types of hardware accelerators. Because executing activation functions in neural networks is generally constrained by the amount of memory available on the target computing resource, executing these functions can have a significant negative impact on the performance of the inference speed of the end-to-end network.

One exemplary extension of the search space relative to activation functions is to introduce activation functions fused with associated discrete convolutions into the search space. Activation functions are generally element-wise operations and operate on the per-unit basis of hardware accelerators configured for vector operations, so that these activation functions can be performed in parallel with discrete convolutions. Discrete convolutions are typically matrix-based operations that operate on the matrix units of the hardware accelerator. These fused activation function-convolution operations can be selected by the system as candidate neural network components as part of the basic neural network architecture search described herein. Any of a variety of activation functions can be utilized, including Swish, ReLU (Rectified Linear Unit), Sigmaid, Tanh, and Softmax.

Various components including layers of fused activation function-convolution operations can be added to the search space, which may vary depending on the type of activation function used. For example, one component of an activation function-convolution layer may include a ReLU activation function, while another component may include a Swish activation function. Because different hardware accelerators may operate more efficiently with different activation functions, it has been found that expanding the search space to include fused activation function-convolutions of multiple types of activation functions can further improve identifying the underlying neural network architecture that is most suitable for performing the neural network task of interest.

In addition to the components with different activation functions described herein, other fused convolutional structures can be used to extend the search space and further enrich it with convolutions of different shapes, types, and sizes. Different convolutional structures can be components added as part of a candidate neural network architecture and can include extension layers of 1x1 convolutions, depth-wise convolutions, projection layers of 1x1 convolutions, and other operations such as activation functions, batch normalization functions, and/or skip connections.

Thus, the search space of the NAS can be expanded in other ways to include operations that can exploit the parallelism available on the hardware accelerator. The search space can include one or more operations for fusing a depthwise convolution with adjacent 1x1 convolutions and for shaping the input of the neural network. For example, the input to the candidate neural network can be a tensor. A tensor is a data structure that can represent multiple values according to different ranks. For example, a first-order tensor can be a vector, a second-order tensor can be a matrix, a third-order matrix can be a three-dimensional matrix, etc. Fusing depthwise convolutions can be beneficial because depthwise operations are generally low-intensity operations, and fusing them with adjacent convolutions can increase the computation intensity to near the maximum capability of the hardware accelerator.

The search space may also include operations that reshape an input tensor by moving elements of the tensor to various locations in memory on the target computing resource. Additionally or alternatively, the operations may replicate elements to various locations in memory.

In some implementations, the system is configured to receive the base neural network for scaling directly and not perform a NAS or other search to identify the base neural network. In some implementations, multiple devices individually perform at least a portion of process 200 by identifying the base neural network on one device and scaling the base neural network on another device as described herein.

As shown in block 240 of FIG. 2, the system may identify multiple scaling parameter values for scaling the base neural network in response to the information specifying the target computing resource and multiple scaling parameters. The system may use latency-aware composite scaling as described herein to use the accuracy rate and latency of the scaled neural network candidates to search for scaling parameter values for the base neural network. For example, the system may apply the latency-aware composite scaling process 300 in step 240 of FIG. 2 to identify multiple scaling parameter values.

In general, scaling techniques are applied in conjunction with NAS to identify neural networks that are scaled for deployment on target computing resources. Model scaling can be used in conjunction with NAS to more efficiently explore families of neural networks that support different use cases. Under the scaling approach, different techniques can be used to explore different values for scaling parameters such as the depth, width, and resolution of the neural network. Scaling can be done by exploring values for each scaling parameter separately or by exploring a uniform set of values for tuning multiple scaling parameters together. The former may be referred to as simple scaling, and the latter may be referred to as composite scaling.

Scaling techniques that use accuracy as the only objective do not result in a scaled neural network that adequately accounts for the performance/speed impacts when deployed on dedicated hardware such as data center accelerators. As explained in further detail in Figure 3, LACS may use both accuracy and latency objectives, which may be shared as the same objective as the objective used to identify the base neural network architecture.

Figure 3 is an example process 300 for latency-aware composite scaling of a base neural network architecture. The example process 300 may be performed on a system or device comprised of one or more processors in one or more locations. For example, the process 300 may be performed on a NAS-LACS system as described herein. For example, the system may perform the process 300 as part of step 240 to identify multiple scaling parameter values for scaling the base neural network.

In a composite scaling approach, scaling coefficients for each scaling parameter are searched together. The system may apply any of a variety of search techniques to identify tuples of coefficients, such as Pareto frontier search or grid search. The system may search the tuples of coefficients according to the same objectives used to identify the basic neural network architecture described herein with reference to FIG. 2. In addition, the multiple objectives may include both accuracy and latency, which may be expressed as follows:

As part of determining the performance metrics, the system may further train and tune the candidate scaled neural network architecture using the received training data. The system may determine a performance metric for the trained scaled neural network and determine whether the performance metric meets a performance threshold according to block 330. The performance metric and the performance threshold may be a composite of multiple performance metrics and multiple performance thresholds, respectively. For example, the system may determine one performance metric from both the accuracy rate and inference latency metrics of the scaled neural network, or may determine separate performance metrics for different objectives and compare each metric to a corresponding performance threshold.

If the performance metric meets the performance threshold, process 300 ends. Otherwise, the process continues and the system selects new scaling parameter value candidates according to block 310. For example, the system may select a new tuple of scaling coefficients from the coefficient search space based at least in part on the previously selected tuple candidates and their corresponding performance metric. In some implementations, the system searches multiple tuples of coefficients and performs a finer search according to multiple objectives in the neighborhood of each of the tuple candidates.

The system may implement any of a variety of techniques to iteratively explore the coefficient candidate space, such as using grid search, reinforcement learning, evolutionary search, etc. As previously described, the system may continue to explore scaling parameter values until a stopping metric, such as convergence or number of iterations, is reached.

In some implementations, the system may be tuned in response to one or more controller parameter values. The controller parameter values may be manually tuned, learned by machine learning techniques, or a combination of these. The controller parameters may affect the relative effect of each objective on the overall performance metric of the candidate tuple. In some examples, particular values or relationships between values in the candidate tuple may be favored or disfavored based on learned characteristics of ideal scaling coefficients reflected at least in part in the controller parameter values.

Pursuant to block 340, the system generates one or more scaling parameter value groups from the selected scaling parameter value candidates in response to one or more objective trade-offs. The objective trade-offs may represent different thresholds for each objective, such as accuracy and latency, that may be satisfied by different scaled neural networks. For example, one objective trade-off has a high threshold for network accuracy but a low threshold for inference latency (i.e., high latency for a network with high accuracy). As another example, the objective trade-off has a low threshold for network accuracy but a high threshold for inference latency (i.e., low latency for a network with low accuracy). As another example, the objective trade-off is a balance between accuracy and latency performance.

For each objective trade-off, the system may identify a group of scaling parameter values that the system can use to scale the base neural network architecture to satisfy the objective trade-off. That is, the system may iterate between the selection shown in block 310, the determination of the performance evaluation index shown in block 320, and the determination of whether the performance evaluation index satisfies the performance threshold shown in block 330 (except that the performance threshold is determined by the objective trade-off). In some implementations, rather than searching for tuples of the base neural network architecture, the system may search for candidate scaling coefficient tuples of the base neural network architecture scaled according to the selected candidate scaling parameter values that first satisfy the performance evaluation indexes of the multiple objectives according to block 330.

Returning to FIG. 2, as shown in block 250, the NAS-LACS system may generate one or more architectures of scaled neural networks using the architecture of the base neural network scaled according to multiple scaling parameter values. The scaled neural network architecture may be a family of neural networks generated from the base neural network architecture and various scaling parameter values.

If the information specifying the target computing resources includes multiple target computing resources, e.g., one or more sets of multiple hardware accelerators of different types, the system may repeat process 200 and process 300 for each hardware accelerator to generate a family of scaled neural network architectures, each corresponding to a target set. For each hardware accelerator, the system may generate a family of scaled neural network architectures according to various objective trade-offs between latency and accuracy, or between latency, accuracy, and other objectives (including, in particular, computational intensity and execution efficiency as described with reference to (3)).

In some implementations, the system may generate a family of multiple scaled neural network architectures from the same base neural network architecture. This approach may be useful in situations where various target devices share similar hardware characteristics, allowing faster identification of corresponding scaled families for each device, since at least one search of base neural network architectures, e.g., as shown in the process of FIG. 2, is performed only once.

4 is a block diagram of a NAS-LACS (Neural Architecture Search-Latency Aware Composite Scaling) system 400 according to an aspect of the disclosure. System 400 is configured to receive training data 401 for performing a neural network task and target computing resource data 402 specifying a target computing resource. As described herein with reference to FIGS. 1-3, system 400 may be configured to implement techniques for generating a family of scaled neural network architectures.

System 400 may be configured to receive input data in response to a user interface. For example, system 400 may receive data as part of a call to an API (application program interface) that exposes system 400. As described herein with reference to FIG. 5, system 400 may be implemented on one or more computing devices. For example, input to system 400 may be provided through a storage medium, including remote storage, connected to one or more computing devices over a network, or input may be provided through a user interface on a client computing device coupled to system 400.

The system 400 may be configured to output a scaled neural network architecture 409, such as a family of scaled neural network architectures. The scaled neural network architecture 409 may be sent as an output for display, for example on a user display, and visualized, if desired, according to the shape and size of each neural network layer defined in the architecture. In some implementations, the system 400 may be configured to provide the scaled neural network architecture 409 as a set of computer-readable instructions, such as one or more computer programs. The set of computer-readable instructions may be executed by a target computing resource to implement the scaled neural network architecture 409.

A computer program may be written in any type of programming language according to any programming paradigm, such as, for example, declarative, procedural, assembly, object-oriented, data-oriented, functional, or imperative. A computer program may be written to perform one or more different functions and to operate within a computing environment, such as on a physical device, on a virtual machine, or across multiple devices. A computer program also implements the functionality described herein, such as the functionality performed by a system, engine, module, or model.

In some implementations, system 400 is configured to transfer data for scaled neural network architecture 409 to one or more other devices configured to convert the architecture into an executable program written in a computer programming language (optionally as part of a framework for generating a machine learning model). System 400 may also be configured to send data corresponding to scaled neural network architecture 409 to a storage device for storage and subsequent retrieval.

The system 400 may include a NAS engine 405. The NAS engine 405 and other components of the system 400 may be implemented as one or more computer programs, specially configured electronic circuits, or any combination thereof. The NAS engine 405 may be configured to receive the training data 401 and the target computing resource data 402 and generate a base neural network architecture 407. The base neural network architecture 407 may be transmitted to the LACS engine 415. The NAS engine 405 may implement any of the various techniques of neural architecture search described herein with reference to FIGS. 1-3. The system may be configured according to aspects of the present disclosure to perform NAS using multiple objectives, including inference latency and accuracy rate of candidate neural networks when executed on the target computing resource. As part of determining performance metrics available to the NAS engine 405 for searching base neural network architectures, the system 400 may include a performance measurement engine 410.

The performance measurement engine 410 may be configured to receive the architecture of the candidate base neural network and generate performance metrics according to the objectives used to perform the NAS by the NAS engine 405. The performance metrics may provide an overall performance evaluation indicator of the candidate neural network according to multiple objectives. To determine the accuracy rate of the candidate base neural network, the performance measurement engine 410 may run the candidate base neural network on the validation set, for example, by obtaining a set of training examples for validation by retaining a portion of the training data 401.

To measure latency, the performance measurement engine 410 may communicate with a computing resource that corresponds to the target computing resource specified by the data 402. For example, if the data 402 for the target computing resource specifies a TPU as the target resource, the performance measurement engine 410 may send the base neural network candidates for execution on the corresponding TPU. The TPU may be housed in a data center in communication (e.g., over a network, which will be described in further detail with reference to FIG. 5) with one or more processors implementing the system 400.

The performance measurement engine 410 may receive latency information indicating the latency between the target computing resource accepting the input and generating the output. The latency information may be measured directly on-site on the target computing resource and sent to the performance measurement engine 410, or may be measured by the performance measurement engine 410 itself. If the performance measurement engine 410 measures latency, the engine 410 may be configured to compensate for latency not caused by processing of the base neural network candidate, e.g., network latency to communicate with the target computing resource. As another example, the performance measurement engine 410 may estimate the latency of processing the input through the base neural network candidate based on previous measurements of the target computing resource and the hardware characteristics of the target computing resource.

The performance measurement engine 410 may generate performance metrics of other characteristics of candidate neural network architectures, such as computational intensity and execution efficiency. As described herein with reference to Figures 1-3, inference latency may be determined as a function of FLOPS (computational requirements), execution efficiency, and computational intensity, and in some implementations, the system 400 explores and directly or indirectly scales neural networks based on these additional characteristics.

Once the performance metrics are generated, the performance measurement engine 410 may send the metrics to the NAS engine 405, which may then repeat a new search for new base neural network architecture candidates, as described herein with reference to FIG. 2, until a stopping metric is reached.

In some examples, the NAS engine 405 is tuned according to one or more controller parameters to adjust how the NAS engine 405 selects the next base neural network architecture candidate. The controller parameters can be manually tuned according to the desired characteristics of the neural network for a particular neural network task. In some examples, the controller parameters can be learned by various machine learning techniques, and the NAS engine 405 can implement one or more machine learning models trained to select the base neural network architecture according to multiple objectives, such as latency and accuracy. For example, the NAS engine 405 can implement a recurrent neural network trained to use features of previous base neural network candidates and multiple objectives to predict base network candidates that are more likely to meet these objectives. The neural network can be trained using a training dataset related to the neural network task, and the training data and performance metrics labeled to indicate the final base neural architecture selected given the data of the target computing resource.

The LACS engine 415 may be configured to perform latency-aware composite scaling as described according to aspects of the present disclosure. The LACS engine 415 is configured to receive data 407 from the NAS engine 405 specifying the base neural network architecture. Like the NAS engine 405, the LACS engine 415 may communicate with the performance measurement engine 410 to obtain performance metrics of the scaled neural network architecture candidates. As described herein with reference to Figures 1-3, the LACS engine 415 may hold a search space in memory of different tuples of scaling coefficients and may be configured to scale the final scaled architecture to quickly obtain a family of scaled neural network architectures. In some implementations, the LACS engine 415 is configured to perform other forms of scaling, e.g., simple scaling, but using multiple objectives including latency employed by the NAS engine 405.

5 is a block diagram of an exemplary environment 500 for implementing a NAS-LACS system 400. The system 400 may be implemented on one or more devices having one or more processors in one or more locations, such as in a server computing device 515. The client computing device 512 and the server computing device 515 may be communicatively coupled to one or more storage devices 530 over a network 560. The storage device(s) 530 may be a combination of volatile and non-volatile memory and may be in the same physical location as the computing devices 512, 515 or in a different physical location. For example, the storage device(s) 530 may include any type of non-transitory computer-readable medium capable of storing information, such as hard drives, solid-state drives, tape drives, optical storage devices, memory cards, ROM, RAM, DVDs, CD-ROMs, writable memory, and read-only memory.

The server computing device 515 may include one or more processors 513 and a memory 514. The memory 514 may store information accessible to the processor(s) 513, including instructions 521 that may be executed by the processor(s) 513. The memory 514 may also include data 523 that the processor(s) 513 may retrieve, manipulate, or store. The memory 514 may be any type of non-transitory computer-readable medium capable of storing information accessible to the processor(s) 513, such as volatile and non-volatile memory. The processor(s) 513 may include one or more central processing units (CPUs), one or more graphic processing units (GPUs), one or more field programmable gate arrays (FPGAs), and/or one or more application-specific integrated circuits (ASICs), such as a tensor processing unit (TPU).

Instructions 521 may include one or more instructions that, when executed by processor(s) 513, cause the one or more processors to perform the operations defined by the instructions. Instructions 521 may be stored in object code format for direct processing by processor(s) 513, or in other formats, including interpretable scripts or collections of separate source code modules that are interpreted or pre-compiled upon request. Instructions 521 may include instructions for implementing system 400 consistent with aspects of the present disclosure. System 400 may be executed using processor(s) 513 and/or other processors located remotely from server computing device 515.

Data 523 may be retrieved, stored, or modified by processor(s) 513 according to instructions 521. Data 523 may be stored in computer registers, in a relational or non-relational database as a table with multiple different fields and records, or as a JSON, YAML, proto, or XML document. Data 523 may also be formatted in a computer readable format, such as, but not limited to, binary values, ASCII, or Unicode. Data 523 may also include information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memory, such as other network locations, or information used by a function to calculate the relevant data.

The client computing device 512, like the server computing device 515, may also include one or more processors 516, memory 517, instructions 518, and data 519. The client computing device 512 may also include a user output 526 and a user input 524. The user input 524 may include any suitable mechanism or technology for accepting input from a user, such as a keyboard, a mouse, a mechanical actuator, a soft actuator, a touch screen, a microphone, and a sensor.

The server computing device 515 may be configured to transmit data to the client computing device 512, which may be configured to display at least a portion of the received data on a display implemented as part of the user output 526. The user output 526 may also be used to display an interface between the client computing device 512 and the server computing device 515. Alternatively or additionally, the user output 526 may include one or more speakers, transducers or other audio outputs, a haptic interface or other haptic feedback that provides non-visual and non-audible information to a platform user of the client computing device 512.

5 illustrates processors 513, 516 and memories 514, 517 within computing devices 515, 512, the components described herein, including processors 513, 516 and memories 514, 517, may each operate in different physical locations and may include multiple processors and multiple memories that are not within the same computing device. For example, some of the instructions 521, 518 and data 523, 519 may be stored on a removable SD card and the rest may be stored in a read-only computer chip. Some or all of the instructions and data may be stored in a location physically separate from but accessible to the processors 513, 516. Similarly, the processors 513, 516 may include a collection of processors that may operate simultaneously and/or sequentially. The computing devices 515, 512 may each include one or more internal clocks that provide timing information. The timing information may be used to time operations and programs executed by the computing devices 515, 512.

The server computing device 515 may be connected by a network 560 to a data center 550 in which the hardware accelerators 551A-551N are housed. The data center 550 may be one of multiple data centers or other facilities in which various types of computing devices, such as hardware accelerators, are located. As described herein, the computing resources housed in the data center 550 may be designated as part of the target computing resources for deploying the scaled neural network architecture.

The server computing device 515 may be configured to accept requests to process data from the client computing devices 512 on computing resources in the data center 550. For example, the environment 500 may be part of a computing platform configured to provide various services to users through various user interfaces and/or APIs exposing platform services. One or more of the services may be a machine learning framework or a set of tools for generating neural networks or other machine learning models in response to a specified task and training data. The client computing devices 512 may receive and transmit data specifying target computing resources to be assigned to execute a neural network trained to perform a particular neural network task. According to aspects of the disclosure described herein with reference to FIGS. 1-4, the NAS-LACS system 400 may receive data specifying the target computing resources and the training data and, in response, generate a family of scaled neural network architectures for deployment on the target computing resources.

As another example of a service that a platform implementing environment 500 may provide, server computing device 515 may maintain a family of different scaled neural network architectures according to different target computing resources that may be available in data center 550. For example, server computing device 515 may maintain different families for deploying neural networks on different types of TPUs and/or GPUs housed in data center 550 or otherwise available for processing.

The devices 512, 515 and the data center 550 can communicate directly or indirectly over the network 560. For example, using network sockets, the client computing device 512 can connect to services running in the data center 550 through the Internet Protocol. The devices 515, 512 can set up listening sockets that can accept initiating connections to send and receive information. The network 560 itself can include a variety of configurations and protocols, including the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local network, and a private network using one or more company-proprietary communication protocols. The network 560 can support a variety of short-range and long-range connections. The short-range and long-range connections may be made at various bandwidths, such as 2.402 GHz to 2.480 GHz (commonly associated with the Bluetooth® standard), 2.4 GHz and 5 GHz (commonly associated with the Wi-Fi® communication protocol), or using various communication standards, such as the LTE® standard for wireless broadband communications. Additionally or alternatively, the network 560 may support wired connections between the devices 512, 515 and the data center 550, including wired connections over various types of Ethernet® connections.

Although one server computing device 515, one client computing device 512, and one data center 550 are shown in FIG. 5, it should be understood that aspects of the disclosure can be implemented in various configurations and quantities of computing devices, such as in paradigms for serial or parallel processing, or on a distributed network of multiple devices. In some implementations, aspects of the disclosure can be performed on a single device connected to multiple hardware accelerators configured to process neural networks, or any combination thereof.

As described herein, aspects of the present disclosure enable the generation of scaled neural network architectures from a base neural network according to a multi-objective approach. Examples of neural network tasks are:

By way of example, the input to a neural network may be in the form of an image or video. As part of processing the given input, the neural network may be configured to extract, identify, and generate features, e.g., as part of a computer vision task. A neural network trained to perform this type of neural network task may be trained to generate an output classification from a set of possible classifications. Additionally or alternatively, the neural network may be trained to output a score that corresponds to a hypothesis that an object identified in an image or video is likely to belong to a particular class.

As another example, the input to a neural network may be a data file corresponding to a particular format, e.g., formatted metadata obtained from other types of data, such as metadata for an HTML file, a word processing document, or an image file. The neural network task in this context may be to classify, score, or otherwise predict characteristics about the received input. For example, a neural network may be trained to predict the likelihood that a received input contains text related to a particular subject. A neural network may also be trained to generate text predictions as part of performing a particular task, e.g., as part of a tool for auto-completion of text in a document while the document is being composed. For example, a neural network may be trained to predict the translation of text in an input document into a target language while a message is being composed.

Other types of input documents can be data related to the characteristics of a network of interconnected devices. These input documents can include activity logs and records regarding the access privileges that allow various computing devices to access various sources of potentially sensitive data. Neural networks can be trained to process these and other types of documents to predict current or future security breaches in the network. For example, a neural network can be trained to predict intrusions into a network by malicious actors.

As another example, the input to the neural network may be audio input, including streaming audio, pre-recorded audio, and audio as part of a video or other source or media. In the context of audio, the neural network task may include speech recognition, including separating audio from other identified audio sources and/or enhancing characteristics of identified audio to make it easier to hear. For example, a neural network may be trained to predict accurate translations of input audio into a target language in real time as part of a translation tool.

In addition to data inputs, including the various types of data described herein, neural networks can also be trained to process features corresponding to a given input. A feature is a value, e.g., a numeric or explicit value, that is related to a characteristic of the input. For example, in the context of an image, an image's features may relate to the RGB values for each pixel in the image. A neural network task in an image/video context may be to classify the content of the image or video, e.g., for the presence of various people, places, or objects. Neural networks can be trained to extract and select relevant features that are processed to generate an output for a given input, and can also be trained to generate new features based on learned relationships between various characteristics of the input data.

Aspects of the present disclosure may be implemented as digital circuitry, a computer-readable storage medium, one or more computer programs, or as a combination of one or more of these. The computer-readable storage medium may be a non-transitory computer-readable storage medium, for example, as one or more instructions executable by a processor(s) and stored on a tangible storage device.

In this specification, the phrase "configured to" is used in various contexts in connection with a computer system, hardware, or part of a computer program. When a system is said to be configured to perform one or more operations, this means that the system has installed thereon appropriate software, firmware, and/or hardware that, when in operation, causes the system to perform one or more operations. When hardware is said to be configured to perform one or more operations, this means that the hardware comprises one or more circuits that, when in operation, accepts inputs and, in response to the inputs, generates outputs corresponding to one or more operations. When a computer program is said to be configured to perform one or more operations, this means that the computer program includes one or more program instructions that, when executed by one or more computers, cause the one or more computers to perform one or more operations.

Although the operations illustrated in the figures and recited in the claims are shown in a particular order, it should be understood that the operations may be performed in an order different from that shown, that some operations may be omitted, may be performed more than once, and/or may be performed in parallel with other operations. Furthermore, separation of various system components configured to perform various operations should not be understood as requiring separation of these components. The components, modules, programs, and engines described may be integrated into a system or may be part of multiple systems.

Unless otherwise expressly stated, most of the above other examples are not mutually exclusive. However, they may be implemented in various combinations to achieve unique advantages. These and other variations and combinations of the above features may be utilized without departing from the subject matter of the invention as set forth in the appended claims, and therefore the description of the above embodiments should be taken as examples, rather than as limitations on the subject matter of the invention as set forth in the appended claims. In addition, the provision of examples described herein, and the provision of clauses expressed with words such as "such as," "including," and the like, should not be construed as limiting the subject matter of the appended claims to specific examples. Rather, these examples are merely illustrative of one of many possible embodiments. Furthermore, the same reference numbers in different drawings may identify the same or similar elements.

Claims (20)

1. A computer-implemented method for determining an architecture of a neural network, comprising:
receiving, by one or more processors, information specifying a target computing resource;
receiving, by the one or more processors, data specifying an architecture of a base neural network;
and the one or more processors identifying a plurality of scaling parameter values for scaling the basic neural network in response to information specifying the target computing resource and a plurality of scaling parameters of the basic neural network, the identifying comprising:
selecting a plurality of candidate scaling parameter values;
and determining a performance evaluation metric of the base neural network scaled in response to the plurality of scaling parameter value candidates, the performance evaluation metric being determined in response to a plurality of objectives including a latency objective, the method comprising:
The method further includes the one or more processors generating a scaled neural network architecture using the base neural network architecture scaled according to the plurality of scaling parameter values. the plurality of objectives being a plurality of second objectives;
Receiving the data specifying the architecture of the base neural network comprises:
receiving, by one or more processors, training data corresponding to a neural network task;
2. The method of claim 1, further comprising: the one or more processors performing a neural architecture search of a search space using the training data to identify an architecture for the base neural network according to a plurality of first objectives. the search space includes candidate neural network layers, each candidate neural network layer configured to perform one or more operations;
The method of claim 2 , wherein the search space includes candidate neural network layers that include different activation functions. The basic neural network architecture includes a plurality of candidate components, each component having a plurality of neural network layers;
4. The method of claim 3, wherein the search space includes a plurality of candidate neural network layer components, the candidate neural network layer components including a first candidate network layer component including a first activation function and a second candidate network layer component including a second activation function different from the first activation function. The method of claim 2, wherein the first objectives for performing the neural architecture search are the same as the second objectives for identifying the scaling parameter values. The method of claim 2, wherein the plurality of first objectives and the plurality of second objectives include an accuracy rate objective corresponding to an accuracy rate of the output of the basic neural network when trained using the training data. The method of any one of claims 1 to 6, wherein the performance evaluation metric corresponds at least in part to an evaluation metric of the latency between the basic neural network accepting an input and generating an output when the basic neural network is scaled in response to the plurality of scaling parameter value candidates and deployed on the target computing resource. The method of any one of claims 1 to 7, wherein the latency objective corresponds to a minimum latency between the basic neural network accepting an input and generating an output when the basic neural network is deployed on the target computing resource. The information specifying the target computing resource specifies one or more hardware accelerators;
3. The method of claim 2 , further comprising executing the scaled neural network on the one or more hardware accelerators to perform the neural network task. the target computing resource is a first target computing resource, and the plurality of scaling parameter values are a plurality of first scaling parameter values;
The method comprises:
receiving, by the one or more processors, information specifying a second target computing resource different from the first target computing resource;
10. The method of claim 9, further comprising: identifying a second plurality of scaling parameter values for scaling the basic neural network in response to information specifying the second target computing resource, the second plurality of scaling parameter values being different from the first plurality of scaling parameter values. the plurality of scaling parameter values being a plurality of first scaling parameter values;
10. The method of claim 1, further comprising generating a scaled neural network architecture from the base neural network architecture scaled with a plurality of second scaling parameter values, the second scaling parameter values being generated as a function of the plurality of first scaling parameter values and one or more compound coefficients that uniformly modify a value of each of the first scaling parameter values. The method according to any one of claims 1 to 9, wherein the basic neural network is a convolutional neural network, and the plurality of scaling parameters include one or more of the depth of the basic neural network, the width of the basic neural network, and the resolution of the input of the basic neural network. 1. A system comprising:
one or more processors;
and one or more memory devices coupled to the one or more processors storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for determining an architecture of a neural network, the operations including:
accepting information specifying a target computing resource;
receiving data specifying an architecture of a base neural network;
and identifying a plurality of scaling parameter values for scaling the basic neural network in response to information specifying the target computing resource and a plurality of scaling parameters of the basic neural network, the identifying act comprising:
selecting a plurality of candidate scaling parameter values;
and determining a performance evaluation metric of the base neural network scaled in response to the plurality of scaling parameter value candidates, the performance evaluation metric being determined in response to a plurality of objectives including a latency objective, the operations further comprising:
The system includes an operation of generating a scaled neural network architecture using the base neural network architecture scaled according to the plurality of scaling parameter values. the plurality of objectives being a plurality of second objectives;
The act of receiving the data specifying the architecture of the base neural network comprises:
receiving training data corresponding to a neural network task;
and performing a neural architecture search of a search space using the training data to identify an architecture for the base neural network according to a plurality of first objectives. the search space includes candidate neural network layers, each candidate neural network layer configured to perform one or more operations;
The system of claim 14 , wherein the search space includes candidate neural network layers that include different activation functions. The system of claim 14, wherein the first objectives for performing the neural architecture search are the same as the second objectives for identifying the scaling parameter values. 15. The system of claim 14, wherein the plurality of first objectives and the plurality of second objectives include a accuracy objective corresponding to an accuracy rate of an output of the basic neural network when trained with the training data. The system of any one of claims 13 to 17, wherein the performance evaluation metric corresponds at least in part to an evaluation metric of the latency between the basic neural network accepting an input and generating an output when the basic neural network is scaled in response to the plurality of scaling parameter value candidates and deployed on the target computing resource. The system of any one of claims 13 to 18, wherein the latency objective corresponds to a minimum latency between the basic neural network accepting an input and generating an output when the basic neural network is deployed on the target computing resource. A program that causes one or more processors to execute the method according to any one of claims 1 to 12.

Images