WO2024214878A1 - Electronic device for prefetching data by predicting memory address and method for operating same - Google Patents

Abstract

An electronic device for prefetching data by predicting a memory address, and a method for operating same, are illustrated. A method for operating an electronic device according to an embodiment of the present disclosure may comprise the steps of: sampling prior demand memory addresses included in a prior demand memory request; generating a prefetch memory request on the basis of a stride between the sampled prior demand memory addresses; transmitting, to a preloader, a prefetch memory response according to the prefetch memory request; and storing, in a preloader cache, prefetch data included in the prefetch memory response and a prefetch memory address corresponding to the prefetch data.

Description

Electronic device for prefetching data by predicting memory address and method of operation thereof

An electronic device and method of operating the same for prefetching data by predicting a memory address are described.

This application is the result of a study conducted with the support of the National IT Industry Promotion Agency (NIPA) with funding from the government (Ministry of Science and ICT) in 2023 (Project No.: RS-2022-00155966, Project Name: Artificial Intelligence Convergence Innovation Human Resources Development (Ewha Womans University), National Project Unique Number: 1711179344).

In addition, this application is the result of research conducted with the support of the Information and Communication Planning and Evaluation Institute with funding from the government (Ministry of Science and ICT) in 2023 (Project No.: 2021-0-02068, Project Name: Artificial Intelligence Innovation Hub Research and Development, National Project Unique Number: 1711193622).

In addition, this application is the result of research conducted with the support of the National Research Foundation of Korea with funding from the government (Ministry of Science and ICT) in 2023 (Project Number: 2021R1G1A1092196, Project Name: Fine Power Management Techniques for Massively Parallel Processors, National Project Unique Number: 1711189100).

The processor can execute applications such as machine learning and blockchain. When executing an application on the processor, a pipeline stall may occur because the operation by the next instruction is executed after the operation by one instruction is completed. The processor can perform better as the pipeline stall is reduced. Therefore, research is actively being conducted to reduce this pipeline stall.

The present disclosure can reduce pipeline stalls by preloading memory required for the next operation into a preloader cache using the memory address used in the operation.

The present disclosure can reduce pipeline stalls by prefetching memory required for the next operation into the preloader cache, thereby eliminating the need to load data into the L1 cache again even if the data is fetched from the L1 cache.

An operating method of an electronic device according to one embodiment of the present disclosure may include the steps of sampling prior demand memory addresses included in a prior demand memory request, generating a prefetch memory request based on a stride between the sampled prior demand memory addresses, transmitting a prefetch memory response according to the prefetch memory request to a preloader, and storing prefetch data included in the prefetch memory response and a prefetch memory address corresponding to the prefetch data in a preloader cache.

The step of generating the above prefetch memory request can predict the prefetch memory address based on the step, and generate the prefetch memory request including the prefetch memory address.

The stride includes one or a combination of two or more of an intra-warp pattern stride and an inter-warp pattern stride, and the step of generating the prefetch memory request can determine the prefetch memory request by determining a priority of the intra-warp pattern stride higher than that of the inter-warp pattern stride.

The step of generating the above prefetch memory request may include generating the prefetch memory request based on the intra-warp pattern stride when the stride includes both an intra-warp pattern stride and an inter-warp pattern stride and the prefetch memory addresses based on each of the intra-warp pattern stride and the inter-warp pattern stride are different.

The method may further include a step of transmitting a demand memory request for a next operation from a load/store unit (LDST unit) to the preloader, and a step of transmitting data required for the next operation to the LDST unit based on a demand memory address included in the demand memory request and the prefetch memory address.

The step of transmitting data required for the above-described next operation to the LDST unit may include transmitting the prefetch data corresponding to the prefetch memory address from the preloader cache to the LDST unit when the requested memory address and the prefetch memory address match.

The step of transmitting data required for the above-described next operation to the LDST unit may include, when the requested memory address and the prefetch memory address are different, transmitting the requested memory request to a lower cache, and transmitting the data from the lower cache that received the requested memory request to the LDST unit.

The above preloader cache may include a plurality of entries, and each of the plurality of entries may include a status bit indicating a status of the entry.

The method may further include updating the status bits depending on whether the prefetch memory request has been generated, whether the prefetch memory response has reached the preloader before the requested memory request is generated, and whether the requested memory address and the prefetch memory address match.

The step of storing in the above preloader cache may generate a plurality of sub-prefetch memory requests and store the prefetch memory address based on the plurality of sub-prefetch memory requests.

According to one embodiment of the present disclosure, a method of operating an electronic device may include the steps of generating a prefetch memory request based on a stride of sampled previous request memory addresses, storing prefetch data included in a prefetch memory response which is a response to the prefetch memory request and a prefetch memory address corresponding to the prefetch data in a preloader cache, determining whether a request memory address included in a request memory request for a next operation generated from an LDST unit and the prefetch memory address stored in the preloader cache match, and if the request memory address and the prefetch memory address match, transmitting the prefetch data to the LDST unit, and if the request memory address and the prefetch memory address do not match, forwarding the request memory request to a lower cache of the preloader cache.

An electronic device according to one embodiment of the present invention includes a processor configured to sample prior demand memory addresses included in a prior demand memory request, generate a prefetch memory request based on a stride between the sampled prior demand memory addresses, transmit a prefetch memory response according to the prefetch memory request to a preloader, and store prefetch data included in the prefetch memory response and a prefetch memory address corresponding to the prefetch data in a preloader cache, wherein the processor includes the preloader that generates the prefetch memory request, and the preloader may include the preloader cache that stores the prefetch memory address and the prefetch data.

The processor can predict the prefetch memory address based on the stride and generate the prefetch memory request including the prefetch memory address.

The above stride includes one or a combination of two or more of an intra-warp pattern stride and an inter-warp pattern stride, and the processor can determine the prefetch memory request by determining a priority of the intra-warp pattern stride higher than that of the inter-warp pattern stride.

The processor may generate the prefetch memory request based on the intra-warp pattern stride, when the stride includes both an intra-warp pattern stride and an inter-warp pattern stride, and the prefetch memory addresses based on each of the intra-warp pattern stride and the inter-warp pattern stride are different.

The above processor can transmit a demand memory request for the next operation from the LDST unit to the preloader, and transmit data required for the next operation to the LDST unit based on a demand memory address included in the demand memory request and the prefetch memory address.

The processor can transmit the prefetch data corresponding to the prefetch memory address from the preloader cache to the LDST unit when the requested memory address and the prefetch memory address match.

The processor can, if the requested memory address and the prefetch memory address are different, transfer the requested memory request to a lower cache and transmit the data from the lower cache that received the requested memory request to the LDST unit.

The above preloader cache may include status bits indicating the status of entries included in the preloader cache.

The processor can generate a plurality of sub-prefetch memory requests and merge the plurality of sub-prefetch memory requests to generate the prefetch memory address.

According to one embodiment of the present disclosure, pipeline stalls can be reduced by preloading memory required for the next operation into a preloader cache.

According to one embodiment of the present disclosure, by prefetching memory required for the next operation into the preloader cache, even if data is fetched from the L1 cache, there is no need to load the data into the L1 cache again.

FIG. 1 is a diagram illustrating an electronic device according to one embodiment of the present disclosure.

FIG. 2 is a diagram illustrating a GPU according to one embodiment of the present disclosure.

Figures 3a and 3b are drawings for explaining the stride.

Figures 4a to 4d are diagrams for explaining multiple prefetching scenarios.

FIG. 5 is a flow chart illustrating the operation of an electronic device including a preloader according to one embodiment of the present disclosure.

FIG. 6 is a flow chart for explaining the operation of a GPU according to one embodiment of the present disclosure.

FIGS. 7A to 7C are drawings for explaining the operation of a preloader according to one embodiment of the present disclosure.

FIG. 8 is a diagram for explaining a status update of an entry according to one embodiment of the present disclosure.

FIG. 9 is a flowchart for explaining the operation of an electronic device according to one embodiment of the present disclosure.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. The same reference numerals presented in each drawing represent the same components.

The embodiments described below may be modified in various ways. The embodiments described below are not intended to be limiting in terms of the embodiments, and should be understood to include all modifications, equivalents, or alternatives thereto.

Although the terms first or second may be used to describe various components, these terms should be understood only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

The terminology used in the examples is only used to describe specific embodiments and is not intended to be limiting of the embodiments. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among the phrases, or all possible combinations thereof. It should be understood that the terms "comprises" or "has" in this specification specify that a feature, number, step, operation, component, part or combination thereof described in the specification is present, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms defined in commonly used dictionaries, such as those defined in common usage dictionaries, should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless expressly defined in this application.

In addition, when describing with reference to the attached drawings, the same components will be given the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted. When describing an embodiment, if it is determined that a specific description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

FIG. 1 is a diagram illustrating an electronic device according to one embodiment of the present disclosure.

Referring to FIG. 1, an electronic device (100) is illustrated. The electronic device (100) may include a processor (110). Only components related to embodiments of the present disclosure are illustrated in the electronic device (100) illustrated in FIG. 1. Accordingly, it is apparent to a person skilled in the art that the electronic device (100) may further include other general components in addition to the components illustrated in FIG. 5.

The processor (110) may perform a role of performing overall functions for controlling the electronic device (100). The processor (110) may control the electronic device (100) overall by executing programs and/or commands stored in a memory (not shown). The processor (110) may be implemented as a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), and a neural processing unit (NPU) provided in the electronic device (100), but is not limited thereto.

The processor (110) may include a preloader (111). The preloader (111) may include a cache (not shown). The preloader (111) is a type of prefetcher, but will be referred to as a preloader (111) to distinguish it from a prefetcher that does not include a cache (not shown). The cache (not shown) included in the preloader (111) may be referred to as a preloader cache. The preloader (111) may calculate a stride using a memory address already generated in a previous operation. The preloader (111) may predict a next memory address using this stride. The preloader (111) may generate a prefetch memory request to preload data stored in the predicted next memory address into the preloader cache.

Hereinafter, for convenience of explanation, the present disclosure will be explained using a GPU, which is an example of a processor (110). However, it is obvious to those skilled in the art that the following explanation can be applied not only to a GPU but also to a CPU, TPU, and NPU when a preloader is included.

FIG. 2 is a diagram illustrating a GPU according to one embodiment of the present disclosure.

Referring to FIG. 2, a GPU (200) according to an embodiment of the present disclosure is illustrated. The GPU (200) may include a thread block scheduler, a plurality of streaming multiprocessors (210-1, 210-2, 210-N), an inter connection, a shared L2 cache (220), and a global memory (230).

Each of a plurality of streaming multiprocessors (210-1, 210-2, 210-N) (hereinafter, SM) can execute thousands of threads simultaneously. Each SM can include hundreds of cores called streaming processors (hereinafter, SPs) (213). The SPs (213) can perform arithmetic operations. Threads scheduled to the SMs can be divided into a plurality of warps. The plurality of warps can be scheduled to the SPs (213) by a warp scheduler (211) according to various scheduling policies.

The GPU (200) may have a memory hierarchy similar to that of the CPU. The SM may include a large-sized register file (212). The SM may include an L1 cache (216) and a scratchpad memory (217) as lower layers of the register file (212). Finally, the GPU (200) may include a shared L2 cache (220) and a global memory (230) shared by all scheduled threads.

Data may need to be loaded into the register file (212) in order to be used by an instruction. Therefore, memory latency can be reduced through prefetching, which loads data in advance into the immediate lower cache of the register file (212).

The SM may include a preloader (215). The preloader (215) may perform prefetching. The preloader (215) may be a type of prefetcher that performs prefetching. However, the preloader (215) may further include a preloader cache that stores a prefetch memory address and prefetch data therein.

The preloader (215) can prefetch data from the lower cache, the L1 cache (216), the shared L2 cache (220), or the global memory (230). Hereinafter, data prefetched from the L1 cache (216), the shared L2 cache (220), or the global memory (230) will be referred to as prefetch data. In addition, the address of the memory where the prefetch data is stored will be referred to as a prefetch memory address. The preloader (215) can store the prefetch data in the preloader cache. The preloader cache can also be referred to as an L0 cache in the sense that it is higher than the L1 cache. In other words, the L0 cache can be used to store the prefetch data. When the preloader (215) receives a demand memory request from the LDST unit (load/store unit), it can compare the demand memory address included in the demand memory request with the prefetch memory address included in the preloader cache. Depending on the comparison result, the preloader (215) can pass the prefetch data to the LDST unit or pass the required memory request to the lower cache.

Below, we will explain how the preloader (215) and the prefetcher predict the prefetch memory address.

Figures 3a and 3b are diagrams for explaining the stride. In Figures 3a and 3b, we will explain a method for determining the stride and prefetching data into the L1 cache based on a prefetcher that does not include a preloader cache, rather than a preloader that includes the preloader cache described above.

However, it is obvious to those skilled in the art that the method described below can also be applied to a preloader including a preloader cache. In this case, when the preloader performs the method below, the prefetch data can be stored in the preloader cache rather than the L1 cache, as described in FIG. 6, FIG. 7a, FIG. 7b, and FIG. 7c.

Referring to FIGS. 3A and 3B, a method for determining a stride for predicting a prefetch memory address is illustrated. Specifically, FIG. 3A illustrates a method for determining an intra-warp pattern stride and prefetching data. FIG. 3B illustrates a method for determining an inter-warp pattern stride and prefetching data.

In FIGS. 3A and 3B, block (301) may represent a previous request memory address according to a previous request memory request. Block (303) may represent a prefetch memory address included in a prefetch memory request. Block (305) may represent a request memory address included in a request memory request for a next operation.

The previous requested memory address according to the previous requested memory request may mean a memory address where data used in the previous operation is stored. The prefetch memory address included in the prefetch memory request may mean a memory address where data predicted by the prefetcher to be used in the next operation is stored. The requested memory address included in the requested memory request for the next operation may mean a memory address where data required for the next operation requested by the LDST unit is stored.

Intra-warp pattern stride may be an algorithm that predicts a prefetch memory address by using the stride between request memory requests generated from the same warp. The prefetcher may sample previous request memory addresses included in previous request memory requests generated from the same warp. The prefetcher may determine a prefetch memory address based on the stride between the sampled previous request memory addresses. The prefetcher may load and store prefetch data, which is data stored in the determined prefetch memory address, into an L1 cache. That is, the prefetcher may perform prefetching on the prefetch data. Thereafter, when there is a request memory request for the next operation, the prefetcher may compare the request memory address included in the request memory request with the prefetch memory address stored in the L1 cache. If the request memory address and the prefetch memory address match, the prefetcher may transfer the prefetch data from the L1 cache to the LDST unit.

For example, referring to FIG. 3a, the prefetcher can sample previous request memory addresses included in a previous request memory request generated from the same warp #00. The prefetcher can calculate a stride between the previous request memory addresses 0x0100 and 0x0200 (① in FIG. 3a). At this time, the stride can be 0x0100, which is a difference between 0x0100 and 0x0200. The prefetcher can use the stride to predict a prefetch memory address where prefetch data required for the next operation is stored as 0x0300 (② in FIG. 3a). The prefetcher can load and store the prefetch data located at 0x0300 in the L1 cache. When a request memory request with a request memory address of 0x0300 is generated from warp #00 for the next operation, the LDST unit can transmit the request memory request to the L1 cache. At this time, the prefetch data stored in the L1 cache and corresponding to the prefetch memory address 0x0300 can be transmitted to the LDST unit. On the other hand, if the requested memory address and the prefetch memory address are different, the requested memory request can be forwarded to a lower cache (e.g., L2 cache).

Inter-warp pattern stride can be an algorithm that predicts prefetch memory addresses by using strides between request memory requests generated from different warps. The prefetcher can sample previous request memory addresses included in previous request memory requests generated from different warps. The step of determining the stride by using the sampled previous request memory addresses is the same as the intra-warp pattern stride, so the description will be omitted.

For example, referring to FIG. 3b, the prefetcher can sample previous request memory addresses 0x0100 and 0x0200 from different warp #00 and warp #01, respectively. The prefetcher can determine a stride as 0x0100 using the previous request memory addresses 0x0100 and 0x0200 (① in FIG. 3b). The prefetcher can predict a prefetch memory address where prefetch data required by warp #02 for the next operation is located as 0x0300 using the stride (② in FIG. 3b). The prefetcher can load and store the prefetch data located at 0x0300 into the L1 cache. When a request memory request with a request memory address of 0x0300 is generated from warp #02 for the next operation, the LDST unit can transmit the request memory request to the L1 cache. At this time, the prefetch data stored in the L1 cache and corresponding to the prefetch memory address 0x0300 can be transmitted to the LDST unit. On the other hand, if the requested memory address and the prefetch memory address are different, the requested memory request can be forwarded to a lower cache (e.g., L2 cache).

Below, we will describe scenarios that can occur when prefetching data into the L1 cache.

Figures 4a to 4d are diagrams for explaining multiple prefetching scenarios.

Referring to FIGS. 4A to 4D, the GPU (400) may include a streaming multiprocessor (410), an interconnection (420), an L2 cache (430), and a global memory (440). The streaming multiprocessor (410) may include a register file (411), an LDST unit (413), a prefetcher (415), and an L1 cache (417).

Referring to FIG. 4A, in ①, the prefetcher (415) can predict a prefetch memory address, which is an address of prefetch data to be used for the next operation. The prefetcher (415) can generate a prefetch memory request including the prefetch memory address, and transmit the prefetch memory request to the L1 cache. The prefetch memory request can check whether prefetch data corresponding to the prefetcher memory address is loaded in the L1 cache. If the prefetch data is not loaded, a cache miss may occur. If a cache miss occurs, the prefetch memory request may be transmitted to the L2 cache (430), which is a lower cache, and/or the global memory (440) through the interconnection (420).

② In case that prefetch data corresponding to the prefetcher memory address exists in the L2 cache (430) and/or global memory (440), a prefetch memory response according to the prefetch memory request may be generated. The prefetch memory response may be transmitted to the L1 cache (417) via the interconnection (420). The prefetch memory response may include prefetch data corresponding to the prefetcher memory address. The prefetch data and the prefetch memory address corresponding to the prefetch data may be stored in the L1 cache (417).

③, the LDST unit (413) may generate a request memory request to load data stored in the L1 cache (417) into the register file (411). When the request memory request is transmitted to the L1 cache (417), the request memory address included in the request memory request may be compared with a prefetch memory address stored in the L1 cache (417). If the request memory address and the prefetch memory address match, a cache hit may occur.

④ In case of a cache hit, prefetch data may be transferred from the L1 cache (417) to the register file (411) through the LDST unit (413). If a cache hit occurs, the requested memory request may not be transferred to a lower cache of the L1 cache (417). If a cache miss occurs, the requested memory request may be transferred to the L2 cache (430), which is a lower cache of the L1 cache (417), and/or the global memory (440).

That is, after a prefetch memory request occurs, a demand memory request occurs so that prefetch data can be loaded into the L1 cache (417) in advance. At this time, the demand memory request does not need to be transferred to a lower cache of the L1 cache (417), so that memory delay is reduced and high performance improvement can be obtained.

Referring to FIG. 4b, in ①, the prefetcher (415) can predict a prefetch memory address, which is an address of prefetch data to be used for the next operation. The prefetcher (415) can generate a prefetch memory request including the prefetch memory address, and transmit the prefetch memory request to the L1 cache. At this time, a cache hit may occur because the prefetch data is already loaded into the L1 cache (417). The prefetch memory request may be deleted because a cache hit occurs.

②, the LDST unit (413) can generate a request memory request to load data stored in the L1 cache into the register file (411). When the request memory request is transmitted to the L1 cache (417), the request memory address included in the request memory request can be compared with a prefetch memory address stored in the L1 cache (417). If the request memory address and the prefetch memory address match, a cache hit can occur.

③ In case a cache hit occurs, prefetch data can be transferred from the L1 cache (417) to the register file (411) through the LDST unit (413).

However, since data has already been loaded into the L1 cache (417) in Fig. 4b, there is no effect due to prefetching.

Referring to Fig. 4c, steps ① and ② will be omitted as described in Fig. 4a.

A GPU may be structured to execute thousands of threads simultaneously and share an L1 cache (417). Therefore, the size of the L1 cache (417) may be limited.

Therefore, in ③, the prefetch data stored in the L1 cache (417) can be eviction through steps ① and ②. For example, before the required memory request reaches the L1 cache (417), the prefetch data can be eviction by a required memory request of another warp (or thread), and the data required by the other warp (or thread) can be written.

④ In the request memory request, the request memory request is generated, but it may be after the prefetch data has already been extracted. Therefore, even if the request memory request is delivered to the L1 cache (417), a cache miss may occur because the prefetch data has already been extracted. Therefore, the request memory request may be delivered to the L2 cache (430) or the global memory (440) through the interconnection (420).

⑤ In response to a demand memory request, data can be transferred from the L2 cache (430) or global memory (440) to the register file (411) through the LDST unit (413).

In other words, there may be a problem that hundreds of cycles are required to load the data again from the L2 cache (430) or global memory (440) because the prefetch data is extracted even though it has been normally loaded into the L1 cache (417). Accordingly, the load on the L1 cache (417), interconnection (420), L2 cache (430), and global memory (440) may increase. Ultimately, the memory delay may increase rapidly, which may reduce the performance of the GPU.

Referring to FIG. 4d, in ①, the prefetcher (415) predicts the prefetch memory address where the prefetch data required for the next operation is located, but the prediction may be incorrect. That is, the prefetch data may be loaded into the L1 cache (417) through steps ① and ② described in FIG. 4a, but if the prediction is incorrect, the prefetch data may unnecessarily occupy only the space of the L1 cache (417).

③ In step ③, the LDST unit (413) generates a memory request to load data stored in the L1 cache (417) into the register file (411), but a cache miss may occur due to an error in the prediction of the prefetch memory address. The steps after the cache miss occurs will be omitted as described above in Fig. 4c.

As a result, even if the prediction is wrong, there may be a problem where the memory latency increases sharply and the GPU performance decreases just like when the prefetch data is extracted. However, since the application running on the GPU generally has a constant stride, the cases where the prediction is wrong can be limited.

Below, we will describe a method to solve the above-described problem by using a preloader that includes a separate preloader cache.

FIG. 5 is a flow chart illustrating the operation of an electronic device including a preloader according to one embodiment of the present disclosure.

In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Steps (501) to (509) may be performed by the processor, but the embodiment is not limited thereto, and may be performed by the preloader.

At step (501), the processor may sample previous requested memory addresses included in the previous requested memory request.

The processor may sample previous request memory addresses included in previous request memory requests of the same warp. Alternatively, the processor may sample previous request memory addresses included in previous request memory requests of a different warp.

At step (502), the processor may generate a prefetch memory request based on the stride between sampled previous requested memory addresses.

The processor can determine an intra-warp pattern stride based on a stride between previous requested memory addresses of the same warp. The processor can determine an inter-warp pattern stride based on a stride between previous requested memory addresses of different warps. The processor can predict a prefetch memory address where data required for a next operation is stored based on the intra-warp pattern stride and/or the inter-warp pattern stride. The prefetch memory request can include the predicted prefetch memory address.

As a result of the processor determining the stride, the stride may include both the intra-warp pattern stride and the inter-warp pattern stride. At this time, the priority of the intra-warp pattern stride may be higher than that of the inter-warp pattern stride. Therefore, if as a result of the processor determining the stride, the stride includes both the intra-warp pattern stride and the inter-warp pattern stride, and the intra-warp pattern stride and the inter-warp pattern stride are different, the processor may generate a prefetch memory request based on the intra-warp pattern stride.

In step (503), a prefetch memory response according to a prefetch memory request can be transmitted to the preloader.

The processor can forward the prefetch memory request to a sub-cache of the preloader cache included in the preloader. The sub-cache of the preloader cache can be an L1 cache, an L2 cache, and a global memory. If prefetch data is stored in the sub-cache that received the prefetch memory request, a prefetch memory response can be generated. The processor can transmit the generated prefetch memory response to the preloader. The prefetch memory response can include the prefetch data. For example, if the prefetch data exists in the L2 cache, the prefetch memory response can be generated in the L2 cache. The prefetch memory response can be transmitted from the L2 cache to the preloader, and the prefetch memory response can include the prefetch data.

In step (504), the processor can store prefetch data included in the prefetch memory request and a prefetch memory address corresponding to the prefetch data in a preloader cache.

The processor may store the prefetch memory address to check whether the prefetch memory address is correctly predicted when a demand memory request is received by the preloader. The demand memory request may include the demand memory address. The demand memory address may mean an address where data required by an instruction for the next operation is stored. Therefore, if the prefetch data is correctly loaded into the preloader cache, the prefetch memory address and the demand memory address may match.

At step (505), the processor may forward a request for memory for the next operation generated in the LDST unit to the preloader.

At step (506), the processor can check whether the requested memory address matches the prefetch memory address stored in the preloader cache.

In step (507), if the requested memory address does not match the prefetch memory address stored in the preloader cache, the processor may forward the requested memory request to a lower cache.

In other words, the processor can forward the requested memory request to a sub-cache of the cache included in the preloader, where the sub-cache can include the L1 cache, the L2 cache, and the global memory.

At step (508), the processor can transfer data stored in the lower cache to the LDST unit.

If the lower cache that received the requested memory request contains data corresponding to the requested memory address, the processor may transmit a requested memory response to the LDST unit. The requested memory response may contain data corresponding to the requested memory address.

In step (509), if the requested memory address matches the prefetch memory address stored in the preloader cache, the processor can transfer the prefetch data stored in the preloader cache to the LDST unit.

In other words, the processor can transfer prefetch data stored in the preloader cache to the LDST unit in response to a demand memory request.

Ultimately, data predicted to be needed for the next operation can be stored in the preloader cache rather than the L1 cache, thereby preventing data extraction as in Fig. 4c.

Below, the operation of a processor including the above-described preloader will be explained using drawings.

FIG. 6 is a flow chart for explaining the operation of a GPU according to one embodiment of the present disclosure.

Referring to FIG. 6, the GPU (600) may include a streaming multiprocessor (610), an interconnection (620), an L2 cache (630), and a global memory (640). The streaming multiprocessor (610) may include a register file (611), an LDST unit (613), a preloader (615), and an L1 cache (617). The preloader (615) may include a cache (not shown), which is an internal storage device. Hereinafter, the cache (not shown) included in the preloader (615) will be referred to as a preloader cache.

The preloader (615) can sample previous request memory addresses included in a previous request memory request. The preloader (615) can determine a stride using the sampled previous request memory addresses. The preloader (615) can determine whether the stride is an inter-warp pattern stride and/or an intra-warp pattern stride using the sampled previous request memory addresses. The preloader (615) can predict a prefetch memory address, which is an address where data to be used for a next operation is stored, based on the determined stride. In other words, the preloader (615) can predict a request memory address to be included in a request memory request to be generated by the LDST unit.

Thereafter, the preloader (615) may generate a prefetch memory request including a prefetch memory address. The prefetch memory request may be forwarded to the L1 cache (617). If prefetch data corresponding to the prefetch memory address exists in the L1 cache, a cache hit may occur. If a cache hit occurs, a prefetch memory response according to the prefetch memory request may be transmitted from the L1 cache (617) to the preloader (615). At this time, the prefetch memory response may include prefetch data. The preloader (615) may receive the prefetch memory response and store the prefetch data in the preloader cache.

On the other hand, if there is no prefetch data corresponding to the prefetch memory address in the L1 cache (617), a cache miss may occur. At this time, the prefetch memory request may be transmitted to a lower cache through the interconnection (620). If there is prefetch data corresponding to the prefetch memory address in the lower cache that received the prefetch memory request, a cache hit may occur and a prefetch memory response may be generated in the lower cache. At this time, the prefetch memory response may be transmitted to the preloader (615). The preloader (615) may store the prefetch data included in the prefetch memory response in the preloader cache.

For example, if prefetch data does not exist in the L2 cache, a cache miss occurs and a prefetch memory request may be forwarded to the global memory (640). If prefetch data exists in the global memory (640), a prefetch memory response may be generated. The generated prefetch memory response may be transmitted to the preloader (615).

Next, the LDST unit (613) may generate a demand memory request to load data required for the next operation into the register file (611). The demand memory request may include a demand memory address where the data required for the next operation is stored.

The demand memory request can be forwarded to the preloader (615). If the prefetch data address of the prefetch data stored in the preloader cache is compared with the demand memory address of the demand memory request and they are the same, the prefetch data can be loaded into the register file (611) through the LDST unit (613).

Ultimately, prefetch data can be loaded into the register file (611) even if the required memory request does not access the L1 cache (617) or lower caches. In other words, required data can be loaded into the register file (611) with minimal memory delay.

Below, the internal operation of the preloader (615) will be explained.

FIGS. 7A to 7C are drawings for explaining the operation of a preloader according to one embodiment of the present disclosure.

Referring to FIGS. 7A to 7C, an LDST unit (710), a preloader (720), and an L1 cache (730) are illustrated. The preloader (720) may include a prefetch address generator (721) and a preloader cache (723). The preloader cache (723) may be an internal storage device of the preloader (720). The preloader (720) may be connected to the LDST unit (710). Accordingly, all required memory requests generated from the LDST unit (710) may pass through the preloader (720). Recent GPUs may utilize a sector L1 cache that divides a 128-byte sized cache line into four 32-byte sized sectors. The sector L1 cache may load data at the sector level instead of loading the entire 128-byte sized cache line. The L1 cache (730) may be a sector L1 cache.

The preloader cache (723) may include one entry per warp to store the status of the entry, the prefetch memory address, and the prefetch data. Therefore, the preloader cache (723) may include a total of 64 entries. In other words, the preloader cache (723) may have a total of 64 rows. Each of the entries may be indexed by a warp ID. For example, the first entry may be indexed by warp #00, the second entry may be indexed by warp #01, and so on. Each of the entries may include 4 status bits indicating the status of the entry, 106 bits indicating the prefetch memory address, and 1024 bits indicating the prefetch data.

The preloader (720) can generate multiple sub-prefetch memory requests to prefetch sector-sized data. For convenience of explanation, it will be assumed below that a prefetch memory request consists of four sub-prefetch memory requests. Therefore, it will be apparent to those skilled in the art that the present disclosure is not limited to the case where there are four prefetch memory requests. In this case, each of the four sub-prefetch memory requests can access data of sectors located in four different cache lines.

Figure 7a is a diagram for explaining a method of generating a prefetch memory request. In the following, ① to ⑦ are numbers only for convenience of explanation and do not mean that the operations are performed sequentially.

① In the prefetch address generator (721), a prefetch memory request can be generated. The prefetch memory request includes a prefetch memory address and can be delivered to the L1 cache (730). The prefetch address generator (721) can generate up to four sub-prefetch memory requests for each warp. Each sub-prefetch memory request can be a request for one sector delivered to the L1 cache. In other words, the prefetch memory request can be composed of four sub-prefetch memory requests. Each of the sub-prefetch memory requests can include an address. The prefetch memory address can be composed of addresses included in the four sub-prefetch memory requests.

② In order to check whether the requested memory address included in the requested memory request matches the prefetch memory address included in the prefetch memory request, the preloader (720) may store the prefetch memory address in the preloader cache (723). At this time, rather than storing the entire prefetch memory address, only a part of the addresses of the four sub-prefetch memory requests constituting the prefetch memory address may be stored.

That is, in ③, rather than storing all addresses of the four sub-prefetch memory requests, the offsets of the addresses included in the first sub-prefetch memory request and the addresses included in the other sub-prefetch memory requests can be stored. The offset is limited to 20 bits, but only 14 bits, excluding the least significant bit (LSB) 6 bits, can be stored as a sector offset.

Therefore, in ⑥, the 64-bit address included in the first sub-prefetch memory request and the 14-bit offset of each of the other sub-prefetch memory requests can be combined.

Finally, in ⑦, the prefetch memory address can be converted to 106 bits and stored in the preloader cache (723).

④ In order to store the 106-bit converted address described above in the preloader cache (723), the preloader (720) can use the warp ID as an index. Since the preloader (720) can generate one prefetch memory request for one warp, the preloader cache (723) can have 64 entries to store the state of the entry, the converted address, and the prefetch data. Since each of the entries is indexed by the warp ID, the preloader (720) can use the warp ID as an index to store the converted address.

⑤ In case a prefetch memory request is forwarded to the L1 cache (730), a status bit indicating the status of the entry may be updated. The status bit may be updated to indicate that a prefetch memory request has been generated. The method by which the status of the entry is updated will be further described in FIG. 8.

Figure 7b is a diagram for explaining when a request for memory is transmitted from the LDST unit (710) to the preloader (720). ① to ⑥ are numbers only for convenience of explanation and do not mean that the operations are performed sequentially.

A demand memory request can be composed of four demand sub-memory requests. For convenience of explanation, it will be assumed below that a demand memory request is composed of four demand sub-memory requests. However, it will be apparent to those skilled in the art that the present disclosure is not limited to the case where there are four demand sub-memory requests. Each of the demand sub-memory requests can include a demand address. A demand memory request can include a demand memory address. The demand memory address can be composed of demand addresses of the demand sub-memory requests.

① In case a demand memory request arrives at the preloader (720), the preloader (720) can select an entry from the preloader cache (723) using the warp ID. The demand memory request may be for the next operation of a specific warp. Therefore, the preloader (720) can select an entry from the preloader cache (723) using the warp ID of this specific warp.

② In the request sub-memory requests, the offsets of the request sub-memory requests can be calculated in the same manner as the prefetch sub-memory address in Fig. 7a. That is, the offsets in the request addresses of the request sub-memory requests other than the first request sub-memory request are limited to 20 bits, but only 14 bits excluding the LSB 6 bits can be calculated as the offset.

③ In the first request sub-memory request address 64 bits and the offset 14 bits of each of the other request sub-memory requests can be combined.

④ In the prefetch memory address stored in the entry of the preloader cache (723), the converted address and the address combined in ③ can be compared. In other words, the prefetch memory address and the requested memory address can be compared.

⑤ In the case where the status bit is updated based on the comparison result and the status of the selected entry, the request memory request can be transmitted to the L1 cache (730) based on the comparison result and the selected entry.

If the comparison result addresses are the same and the status of the entry is such that a prefetch request has already been generated, the requested memory request may be deleted and not forwarded to the L1 cache (730). In other words, if the comparison result addresses are the same and the status of the entry is such that a prefetch request has already been generated, this may mean that the prefetch memory address stored in the preloader cache (723) was accurately predicted.

Therefore, in this case ⑥, the prefetch data from the preloader cache (723) may be transferred to the LDST unit (710) through a writeback operation, and there may be no need to transfer the required memory request to the L1 cache (730).

However, if the addresses of the comparison results in ⑤ are different, the write back operation is not performed and the required memory request may be transferred to the Ll cache (730).

After the above-described operation, the status of the entry can be updated. The method by which the status of the entry is updated is described in Fig. 8.

Figure 7c is a diagram illustrating a method for managing prefetch memory responses.

A prefetch memory request is forwarded to a lower cache of the preloader cache (723), and in response, a prefetch memory response may be forwarded from the lower cache to the preloader (720). The prefetch memory response may include prefetch data. At this time, the prefetch data may be stored in the preloader cache (723) or bypassed to the LDST unit (710).

Specifically, in ①, the preloader (720) can use the warp ID to select an entry that has detailed information of a prefetch memory request. The preloader (720) can use the warp ID to select an entry indexed by the warp ID from the preloader cache (723).

② In ②, it is possible to determine whether the status of the entry is a state in which a requested memory request has already been created and whether the prefetch memory address stored in the entry and the requested memory address included in the requested memory request match.

③ In case the status of the entry is that the requested memory request has already been generated and the prefetch memory address stored in the entry and the requested memory address included in the requested memory request match, the prefetch memory response can be written back to the LDST unit using a demultiplexer. That is, if the requested memory request has already been generated, the prefetch data can be bypassed to the LDST unit without being stored in the preloader cache (723).

④ In case the status of the entry is not a state in which a requested memory request has already been generated or the prefetch memory address stored in the entry and the requested memory address included in the requested memory request do not match, the prefetch data included in the prefetch memory response may be stored in the preloader cache (723). At this time, the stored prefetch data may be transmitted to the LDST unit according to the method described in Fig. 7b when a requested memory request is generated by the LDST unit later.

Below, we will explain how the status of an entry is updated depending on the status of the preloader (720).

FIG. 8 is a diagram for explaining a status update of an entry according to one embodiment of the present disclosure.

Each of the entries included in the preloader cache can have five states individually. The state of the entry can be expressed by a state bit. The state bit can include multiple bits. The lower two bits of the state bits can represent the state of the entry. The upper bits of the state bits excluding the lower two bits can be used to manage multiple sub-prefetch memory requests. The number of the upper bits excluding the lower two bits of the multiple bits can be based on the number of multiple sub-prefetch memory requests. In other words, when the number of upper bits excluding the lower two bits is N, 2 ^N sub-prefetch memory requests can be managed. The upper bits excluding the lower two bits can be updated according to the number of multiple sub-prefetch memory responses corresponding to the multiple sub-prefetch memory requests that have arrived at the preloader.

In the following, for the convenience of explanation, it will be explained assuming that the status bit is 4 bits. In this case, the lower 2 bits can indicate the status of the entry, and the upper 2 bits can be used to manage 2 ² sub-prefetch memory requests. However, it is obvious to those skilled in the art that the explanation below is not limited to the case where the status bit is 4 bits.

The state of an entry can always start in the Invalid state, and in the Invalid state, the state bits can be represented as [0 0 0 0].

① In case a prefetch memory request is generated and a 106-bit address is stored in the entry, the state of the entry can be updated from the Invalid state to the Pref. Gen. state. The Pref. Gen. state can be expressed as [0 0 0 1].

The Pref. Gen. state may be a state indicating that a prefetch memory request has been generated.

The state of an entry can be updated to one of three states from Pref. Gen. state.

② In ②, if a demand memory request is generated and the prefetch data address stored in the entry matches the demand memory address included in the demand memory request, the state of the entry can be updated from Pref. Gen. state to Dem. Gen. state. The Dem. Gen. state can be expressed as [# # 1 1].

Here, the upper two bits can be updated according to the number of sub-prefetch memory responses corresponding to the sub-prefetch memory requests. For example, if only two demand sub-memory responses arrive at the preloader in response to four demand sub-memory requests, the upper two bits can be represented as [1 0].

③ In case a demand memory request is generated and the prefetch data address stored in the entry does not match the demand memory address included in the demand memory request, the state of the entry can be updated from the Pref. Gen. state to the Miss Match state. The Miss Match state can be expressed as [# # 1 0].

④ In case a prefetch memory response arrives according to a prefetch memory request before a demand memory request is generated, the state of the entry can be updated from the Pref. Gen. state to the Pre. Data Ack. state. The Pre. Data Ack. state can be expressed as [# # 0 1].

At this time, the upper two bits of the Dem. Gen. status, Miss Match status, and Pre. Data Ack. status may indicate the number of sub-prefetch memory responses that arrived at the preloader in response to multiple sub-prefetch memory requests.

A Miss Match condition may indicate that the prefetch memory address and the requested memory address do not match.

Therefore, in ⑤, when all sub-prefetch memory responses for the sub-prefetch memory request arrive, the state of the entry can be updated from the Miss Match state to the Invalid state again. In addition, the required memory request in the Miss Match state can be forwarded to the lower cache to load the data.

The Dem. Gen. state can indicate a state where the prefetch memory address matches the requested memory address. At this time, the requested memory address can be removed so as not to be delivered to the L1 cache.

⑥ In case all sub-prefetch memory responses according to sub-prefetch memory requests arrive at the preloader, the status of the entry can be updated from the Dem. Gen. status to the Invalid status.

Pre. Data Ack. state can indicate that a sub-prefetch memory response has arrived at the preloader for a sub-prefetch memory request before the requested memory request is generated. Accordingly, data loaded into the preloader can be stored in the preloader cache (i.e., entry).

⑦ In the request memory request is generated, and if the requested memory address matches the prefetch memory address stored in the preloader cache, the prefetch data stored in the preloader cache can be transferred to the LDST unit. Then, the state of the entry can be updated from the Pre. Data Ack. state to the Dem. Gen. state.

If in ⑧, the requested memory address does not match the prefetch memory address stored in the preloader cache, the state of the entry can be updated from the Pre. Data Ack. state to the Miss Match state. Then, the requested memory request can be propagated to the lower cache.

FIG. 9 is a flowchart for explaining the operation of an electronic device according to one embodiment of the present disclosure.

In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Steps (901) to (907) may be performed by the processor, but the embodiment is not limited thereto, and may be performed by the preloader.

At step (901), the processor may generate a prefetch memory request based on the stride of sampled previous requested memory addresses.

In step (903), the processor may store prefetch data included in a prefetch memory response, which is a response to a prefetch memory request, and a prefetch memory address corresponding to the prefetch data in a preloader cache.

At step (905), the processor can determine whether a request memory address included in a request memory request for the next operation generated from the LDST unit and a prefetch memory address stored in the preloader cache match.

In step (907), the processor may transmit prefetch data to the LDST unit if the requested memory address and the prefetch memory address match, and may forward the requested memory request to a lower cache of the preloader cache if the requested memory address and the prefetch memory address do not match.

Meanwhile, the method according to the present invention can be written as a program that can be executed on a computer and can be implemented in various recording media such as a magnetic storage medium, an optical reading medium, and a digital storage medium.

The implementations of the various technologies described herein may be implemented as digital electronic circuitry, or as computer hardware, firmware, software, or combinations thereof. The implementations may be implemented as a computer program product, i.e., a computer program tangibly embodied in an information carrier, e.g., a machine-readable storage medium (computer-readable medium) or a radio signal, for processing by the operation of a data processing device, e.g., a programmable processor, a computer, or multiple computers, or for controlling the operation thereof. A computer program, such as the computer program(s) described above, may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program may be deployed to be processed on one computer or multiple computers at one site, or distributed across multiple sites and interconnected by a communications network.

Processors suitable for processing a computer program include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. Elements of the computer may include at least one processor for executing instructions and one or more memory devices for storing instructions and data. Typically, the computer may include, or be coupled to receive data from, transmit data to, or both, one or more mass storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include, by way of example, semiconductor memory devices, magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs (Compact Disk Read Only Memory) and DVDs (Digital Video Disk), magneto-optical media such as floptical disks, ROMs (Read Only Memory), RAMs (Random Access Memory), flash memory, EPROMs (Erasable Programmable ROM), EEPROMs (Electrically Erasable Programmable ROM), etc. The processor and memory may be supplemented by, or included in, special purpose logic circuitry.

Additionally, the computer-readable medium can be any available medium that can be accessed by a computer, and can include both computer storage media and transmission media.

While this specification contains details of a number of specific implementations, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be unique to particular embodiments of particular inventions. Certain features described herein in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Furthermore, although features may operate in a particular combination and may initially be described as being claimed as such, one or more features from a claimed combination may in some cases be excised from that combination, and the claimed combination may be modified into a subcombination or variation of a subcombination.

Likewise, although operations are depicted in the drawings in a particular order, this should not be understood to imply that the operations must be performed in the particular order illustrated or in any sequential order to achieve a desirable result, or that all of the illustrated operations must be performed. In certain cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various device components of the embodiments described above should not be understood to require such separation in all embodiments, and it should be understood that the program components and devices described may generally be integrated together in a single software product or packaged into multiple software products.

Meanwhile, the embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to help understanding, and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art to which the present invention pertains that other modified examples based on the technical idea of the present invention can be implemented in addition to the embodiments disclosed herein.

Claims (20)

In a method of operating an electronic device, A step of sampling previous requested memory addresses included in a previous requested memory request; A step of generating a prefetch memory request based on the step between the sampled previous requested memory addresses; A step of transmitting a prefetch memory response according to the above prefetch memory request to the preloader; and A step of storing prefetch data included in the above prefetch memory response and a prefetch memory address corresponding to the above prefetch data in a preloader cache. A method of operating an electronic device comprising: In the first paragraph, The step of generating the above prefetch memory request is: An operating method for predicting the prefetch memory address based on the step and generating the prefetch memory request including the prefetch memory address. In the second paragraph, The above stride is, Including one or more combinations of intra-warp pattern strides and inter-warp pattern strides, The step of generating the above prefetch memory request is: An operating method for determining the prefetch memory request by determining the priority of the intra-warp pattern stride to be higher than the inter-warp pattern stride. In the second paragraph, The step of generating the above prefetch memory request is: An operating method, wherein when the stride includes both an intra-warp pattern stride and an inter-warp pattern stride, and the prefetch memory addresses based on each of the intra-warp pattern stride and the inter-warp pattern stride are different, generating the prefetch memory request based on the intra-warp pattern stride. In the first paragraph, A step of transmitting a request for memory for the next operation from the LDST unit to the preloader; and A step of transmitting data required for the next operation to the LDST unit based on the requested memory address included in the requested memory request and the prefetch memory address. A method of operation, further comprising: In paragraph 5, The step of transmitting the data required for the above-mentioned next operation to the LDST unit is: An operating method for transmitting the prefetch data corresponding to the prefetch memory address from the preloader cache to the LDST unit when the requested memory address and the prefetch memory address match. In paragraph 5, The step of transmitting the data required for the above-mentioned next operation to the LDST unit is: An operating method for transmitting the requested memory request to a lower cache when the requested memory address and the prefetch memory address are different, and transmitting the data from the lower cache that received the requested memory request to the LDST unit. In the first paragraph, The above preloader cache is, Contains multiple entries, A method of operation, wherein each of said plurality of entries includes a status bit indicating the status of the entry. In Article 8, A method of operation, further comprising the step of updating the status bits depending on whether the prefetch memory request has been generated, whether the prefetch memory response has reached the preloader before the requested memory request is generated, and whether the requested memory address and the prefetch memory address match. In the first paragraph, The step of saving to the above preloader cache is: An operational method comprising: generating a plurality of sub-prefetch memory requests and storing the prefetch memory address based on the plurality of sub-prefetch memory requests. In a method of operating an electronic device, A step of generating a prefetch memory request based on the stride of sampled previous requested memory addresses; A step of storing prefetch data included in a prefetch memory response, which is a response to the prefetch memory request, and a prefetch memory address corresponding to the prefetch data in a preloader cache; A step of determining whether a request memory address included in a request memory request for the next operation generated from the LDST unit and the prefetch memory address stored in the preloader cache match; and If the requested memory address and the prefetch memory address match, the step of transmitting the prefetch data to the LDST unit, and if the requested memory address and the prefetch memory address do not match, the step of transmitting the requested memory request to a lower cache of the preloader cache. A method of operating an electronic device, comprising: A processor for sampling previous request memory addresses included in a previous request memory request, generating a prefetch memory request based on a step between the sampled previous request memory addresses, transmitting a prefetch memory response according to the prefetch memory request to a preloader, and storing prefetch data included in the prefetch memory response and a prefetch memory address corresponding to the prefetch data in a preloader cache, The above processor, Including the preloader that generates the above pre-fetch memory request, The above preloader, The preloader cache storing the above prefetch memory address and the above prefetch data. An electronic device comprising: In Article 12, The above processor, An electronic device predicting the prefetch memory address based on the step and generating the prefetch memory request including the prefetch memory address. In Article 13, The above stride is, Including one or more combinations of intra-warp pattern strides and inter-warp pattern strides, The above processor, An electronic device that determines the prefetch memory request by determining the priority of the intra-warp pattern stride to be higher than the inter-warp pattern stride. In Article 13, The above processor, An electronic device, wherein the prefetch memory request is generated based on the intra-warp pattern stride, when the stride includes both an intra-warp pattern stride and an inter-warp pattern stride, and the prefetch memory addresses based on each of the intra-warp pattern stride and the inter-warp pattern stride are different. In Article 12, The above processor, An electronic device that transmits a request for a memory for a next operation from an LDST unit to the preloader, and transmits data required for the next operation to the LDST unit based on a request memory address included in the request memory request and the prefetch memory address. In Article 16, The above processor, An electronic device that transmits the prefetch data corresponding to the prefetch memory address from the preloader cache to the LDST unit when the requested memory address and the prefetch memory address match. In Article 16, The above processor, An electronic device that, when the requested memory address and the prefetch memory address are different, transmits the requested memory request to a lower cache, and transmits the data from the lower cache that received the requested memory request to the LDST unit. In Article 12, The above preloader cache is, Contains multiple entries, An electronic device, wherein each of said plurality of entries includes a status bit indicating the status of the entry. In Article 12, The above processor, An electronic device that generates a plurality of sub-prefetch memory requests and generates the prefetch memory address based on the plurality of sub-prefetch memory requests.

Images