KR20220078566A - memory-based processor - Google Patents

Abstract

In some embodiments, an integrated circuit may include a substrate and a memory array disposed on the substrate, the memory array including a plurality of discrete memory banks. The integrated circuit may also include a processing array disposed on the substrate, the processing array including a plurality of processor subunits, each processor subunit of the plurality of processor subunits comprising one of the plurality of discrete memory banks. Associated with one or more discrete memory banks. The integrated circuit may also include a controller configured to implement at least one security measure for operation of the integrated circuit and to take one or more solutions when the at least one security measure is triggered.

Description

memory based processor

The present disclosure relates to devices that enable memory intensive operation. In particular, the present disclosure relates to a hardware chip comprising processing elements coupled to dedicated memory banks. The present disclosure also relates to devices for improving power efficiency and speed of memory chips. In particular, the present disclosure relates to systems and methods with partial or no refreshes on memory chips. The present disclosure also relates to memory chips of selectable capacities and dual port capabilities on the memory chips.

This application is entitled to U.S. Provisional Application No. 62/886,328, filed on August 13, 2019, U.S. Provisional Application No. 62/907,659, filed on September 29, 2019, and U.S. Provisional Application No., filed February 7, 2020 62/971,912, and U.S. Provisional Application Serial No. 62/983,174, filed on February 28, 2020, the contents of which are incorporated herein by reference.

As processor speed and memory capacity continue to rise, a significant limit to effective processing speed is the von Neumann bottleneck. The von Neumann bottleneck is caused by the throughput limit of the existing computer architecture. In particular, the transfer of data from memory to the processor is often a bottleneck compared to the actual computation performed by the processor. Accordingly, in memory-intensive processing, the number of clock cycles for reading and writing to the memory significantly increases. As clock cycles are spent reading and writing to memory and cannot be utilized to perform operations on data, the result is a loss in effective processing speed. Also, the computational bandwidth of a processor is generally greater than the bandwidth of the bus that the processor uses to access memory.

This bottleneck is more pronounced in neural networks and other machine learning algorithms, in memory-intensive processes such as database building, search indexing, and query processing, and other tasks that involve more read and write operations than data processing operations.

In addition, the rapid growth in the size and granularity of available digital data created opportunities for the construction of machine learning algorithms and enabled new technologies. However, this has also created cumbersome problems in the world of databases and parallel operations. For example, the rise of social media and the Internet of Things is creating digital data at an unprecedented rate. This new data can be used to create algorithms for a variety of purposes, from new advertising methods to more sophisticated industrial process control methods. However, storing, processing, analyzing, and handling the new data has not been easy.

New data sources can be huge, on the scale of petabytes to zettabytes. In addition, the growth rate of these data sources may exceed their data processing capabilities. Therefore, data scientists have adopted parallel data processing methods to solve these problems. In an effort to improve computational power and handle huge amounts of data, scientists have attempted to develop systems and methods capable of parallel intensive computation. However, existing systems and methods have not been able to keep up with the requirements of data processing because their methods are often limited by the need for additional resources for data management, integration of separated data, and analysis of separated data.

To enable handling of large data sets, engineers and scientists are now trying to improve the hardware used to analyze data. For example, by introducing memory and processing functions within a single substrate fabricated in a technology that is more suited to memory operation than arithmetic calculations, a new semiconductor processor or chip (such as the processor or chip described herein) can be specifically designed for data-intensive tasks. can be With integrated circuits specifically designed for data-intensive tasks, new data processing requirements can be met. Nevertheless, this approach to solving the data processing problem of large data sets requires solving new problems in chip design and manufacturing. For example, if a new chip designed for data-intensive tasks is manufactured with manufacturing techniques and architectures used in conventional chips, the new chip will suffer from reduced performance and/or poor yield. In addition, if the new chip is designed to operate with the existing data handling method, the new chip's ability to process parallel operations is limited by the existing method, so the performance of the new chip will be degraded.

The present disclosure relates to devices that enable memory intensive operation. In particular, the present disclosure relates to a hardware chip comprising processing elements coupled to dedicated memory banks. The present disclosure also relates to devices for improving power efficiency and speed of memory chips. In particular, the present disclosure relates to systems and methods with partial or no refreshes on memory chips. The present disclosure also relates to memory chips of selectable capacities and dual port capabilities on the memory chips.

In some embodiments, an integrated circuit may include a substrate and a memory array disposed on the substrate, the memory array including a plurality of discrete memory banks. The integrated circuit may also include a processing array disposed on the substrate, the processing array including a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks. The integrated circuit may also include a controller configured to implement at least one security measure for operation of the integrated circuit and to take one or more solutions when the at least one security measure is triggered.

Disclosed embodiments may also include a method of protecting an integrated circuit from counterfeiting, the method comprising: implementing at least one security measure for operation of the integrated circuit using a controller associated with the integrated circuit; taking one or more solutions when at least one security measure is triggered, the integrated circuit comprising: a substrate; a memory array disposed on the substrate and comprising a plurality of discrete memory banks; and a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks.

Disclosed embodiments may include an integrated circuit comprising: a substrate; a memory array disposed on the substrate and comprising a plurality of discrete memory banks; a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks; and a controller configured to implement at least one security measure for operation of the integrated circuit, wherein the at least one security measure comprises copying the program code to at least two different memory portions.

In some embodiments, a distributed processor memory chip is provided that includes a substrate, a memory array disposed on the substrate, a processing array disposed on the substrate, a first communication port, and a second communication port. The memory array may include a plurality of discrete memory banks. The processing array may include a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks. The first communication port may be configured to establish a communication connection between the distributed processor memory chip and an external entity other than another distributed processor memory chip. The second communication port may be configured to establish a communication connection between the distributed processor memory chip and the first additional distributed processor memory chip.

In some embodiments, a method of transferring data between a first distributed processor memory chip and a second distributed processor memory chip includes: a first processor subunit of a plurality of processor subunits disposed on the first distributed processor memory chip. Determining, using a controller associated with at least one of the first distributed processor memory chip and the second distributed processor memory chip, whether data is ready to be transmitted to a second processor subunit included in the second distributed processor memory chip to do; and after determining that the first processor subunit is ready to transmit the data to the second processor subunit, using a clock enable signal controlled by the controller, from the first processor subunit to the second initiating the transmission of the data to the processor subunit.

In some embodiments, the memory unit comprises: a memory array including a plurality of memory banks; at least one controller configured to control at least one aspect of a read operation for the plurality of memory banks; and at least one zero value detection logic configured to detect a multi-bit zero value stored at a specific address of the plurality of memory banks, wherein the at least one controller and the at least one zero value detection logic include the at least one and send a zero-value indicator to one or more circuits external to the memory unit in response to detecting a zero-value by the zero-value detection logic of the memory unit.

Some embodiments may include a method of detecting a zero value at a specific address of a plurality of discrete memory banks, the method comprising: receiving from circuitry external to the memory unit a request to read data stored at addresses of the plurality of discrete memory banks to do; activating a zero value detection logic in response to the received request so that the controller detects a zero value at the received address; and in response to detecting the zero value by the zero value detection logic, the controller sending a zero value indicator to the circuit.

Some embodiments may include a non-transitory computer-readable medium storing a series of instructions executable by a controller of the memory unit to cause the memory unit to detect a value of zero at a particular address in a plurality of discrete memory banks, The method includes: receiving a read request for data stored in addresses of a plurality of discrete memory banks from circuitry external to the memory unit; activating a zero value detection logic in response to the received request so that the controller detects a zero value at the received address; and in response to detecting the zero value by the zero value detection logic, the controller sending a zero value indicator to the circuit.

In some embodiments, the memory unit comprises: one or more memory banks; bank controller; and an address generator, wherein the address generator provides a current address of a current row to be accessed in an associated memory bank to the bank controller, determines a predicted address of a next row to be accessed in the associated memory bank, and and provide the predicted address to the bank controller before a read operation on the current row associated with the current address is completed.

In some embodiments, a memory unit may include one or more memory banks, each memory bank of the one or more memory banks a first row controller configured to control a plurality of rows, a first subset of the plurality of rows , a single data input for receiving data to be stored in the plurality of rows, and a single data output for providing data retrieved from the plurality of rows.

In some embodiments, a distributed processor memory chip comprises: a substrate; a memory array disposed on the substrate and comprising a plurality of discrete memory banks; a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with a corresponding dedicated one of the plurality of discrete memory banks; a first plurality of buses each coupling one of said plurality of processor subunits to a corresponding dedicated memory bank; and a second plurality of buses respectively connecting one of the plurality of processor subunits to the other one of the plurality of processor subunits. At least one of the memory banks may include at least one DRAM memory mat disposed on the substrate. At least one of the processor subunits may include one or more logic elements associated with the at least one memory mat. The at least one memory mat and the one or more logic elements may be configured to serve as a cache for one or more of the plurality of processor subunits.

In some embodiments, a method of executing at least one instruction of a distributed processor memory chip includes: retrieving one or more data values from a memory array of the distributed processor memory chip; storing the one or more data values in a register formed in a memory mat of the distributed processor memory chip; and accessing, according to at least one instruction executed by a processor element, the one or more data values stored in the register; the memory array includes a plurality of discrete memory banks disposed on the substrate; the processor element is a processor subunit of a plurality of processor subunits included in a processing array disposed on the substrate, each processor subunit associated with a corresponding dedicated discrete memory bank of the plurality of discrete memory banks; The resistor is provided by a memory mat disposed on the substrate.

Some embodiments include a substrate; a processing unit disposed on the substrate; and a memory unit disposed on the substrate, wherein the memory unit is configured to store data to be accessed by the processing unit, and the processing unit includes a memory mat configured to serve as a cache for the processing unit.

Processing systems are expected to process increasingly large amounts of information at very high speeds. For example, fifth-generation (5G) mobile Internet networks are expected to receive vast amounts of information streams and process these information streams at higher rates.

The processing system may include one or more buffers and processors. There may be delays in processing operations applied by the processor, which may require extensive buffers. A large buffer can be costly and/or space intensive.

Passing vast amounts of information from the buffer to the processor may require a high-bandwidth connector and/or a high-bandwidth bus between the buffer and the processor, which may also increase the cost and space of the processing system.

Accordingly, there is an increasing demand for providing an efficient processing system.

A separate server contains multiple subsystems, each with a unique role. For example, a separate server may include one or more switching subsystems, one or more computing subsystems, and one or more storage subsystems.

The one or more computing subsystems and the one or more storage subsystems are coupled to each other via the one or more switching subsystems.

A computing subsystem may include multiple computing units.

The switching subsystem may include multiple switching units.

A storage subsystem may include multiple storage units.

The bottleneck of these separate servers is the bandwidth required to pass information between subsystems.

This is especially true when performing distributed computations necessary to share units of information among all (or a significant portion) of computing units in different computing subsystems.

There are N computing units participating in the sharing, where N is a very large integer (eg at least 1024), each of the N computing units transmitting (or from) an information unit to (or from) every other computing unit. or receive), it is necessary to perform about N information unit transmission processes. This large number of transfer processes is time- and energy-consuming and dramatically limits the throughput of separate servers.

Accordingly, there is an increasing demand for an efficient separate server and a method for efficiently performing distributed processing.

A database contains many entries containing multiple fields. Database processing includes one or more filtering parameters (eg, identity and values of one or more related fields) and one or more operation parameters capable of determining the kind of operation to be performed, a variable or integer to be used when applying the operation, etc. It was common to include the execution of one or more queries containing

For example, a database query may request to perform a statistical operation (operation parameter) on all records in the database in which a specific field has a value within a predetermined range (filtering parameter). As another example, a database query may request deletion of records (operation parameters) that have certain fields that are less than a threshold (a filtering parameter).

Large databases are typically stored on storage devices. In order to respond to a query, the database is transferred to a memory unit, typically in turn by database segment.

The entry in the database is transferred from the memory unit to a processor not belonging to the same integrated circuit as the memory unit. The entry is then processed by the processor.

For each database segment of the database stored in the memory unit, the processing includes: (i) selecting a record of the database segment; (ii) transferring the write from the memory unit to the processor; (iii) the processor filtering the records to determine whether the records are relevant; (iv) performing one or more additional operations (sums, application of other mathematical and/or statistical operations) on the related records.

The filtering step stops after all the records have been sent to the processor and the processor determines which records are relevant.

If the relevant entry in the database is not stored in the processor, it is necessary to send this relevant record to the processor for further processing (applying the operation following the processing) after the filtering step.

When multiple processing operations follow a single filtering, the results of each operation may be transmitted to the memory unit and then transmitted back to the processor.

This process is bandwidth and time consuming.

Accordingly, there is an increasing demand for an efficient method for performing database processing.

Word embedding is an umbrella term for a collection of language modeling and feature learning approaches in natural language processing (NLP) in which words or phrases in a vocabulary are mapped to vectors of elements. Conceptually, a mathematical mapping is involved from a space with many dimensions per word to a continuous vector space with many fewer dimensions (see www.wikipedia.org).

Methods for generating such mappings include neural networks, dimensionality reduction for word co-occurrence matrices, descriptive knowledge-based methods, and explicit representations of the context dimension in which words appear.

Word and phrase embeddings have been shown to increase performance in NLP tasks such as syntactic parsing and sentiment analysis when used as the basis for input representation.

A sentence may be divided into words or phrases, and each section may be expressed as a vector. A sentence can be represented as a matrix containing all vectors representing words or phrases in the sentence.

Vocabularies that map words to vectors may be stored in a memory unit (eg, DRAM) that can be accessed using a word or phrase (or an index representing that word).

The access may be a random access which reduces the throughput of the DRAM. Also, accesses can saturate the DRAM, especially if a large number of accesses are put into the DRAM.

The words in a sentence are usually very random. Accessing the DRAM memory that stores the mapping generally results in performance degradation of random access, even when utilizing DRAM bursts. This is because typically during a burst only one of a small fraction of DRAM memory bank entries will store entries that are relevant to a particular sentence.

Accordingly, the throughput of DRAM memory is low and not continuous.

Each word or phrase in a sentence is pulled from DRAM memory under the control of a host computer external to the DRAM memory's integrated circuit, and based on knowledge of the location of that word, it must control the retrieval of each vector representing each word or section. do. This is a time-consuming and resource-intensive operation.

Data centers and other computer systems are expected to process and exchange increasing amounts of information at very high rates.

The exchange of ever-increasing amounts of data can become a bottleneck for data centers and other computer systems, causing only a fraction of the capabilities of these data centers and other computer systems to be utilized.

96A shows an example of a prior art database 12010 and a prior art motherboard 12011. The database may include multiple servers, and each server may include a multi-server motherboard (“CPU + Memory + Network”). Each server motherboard 12011 receives traffic and includes a CPU 12012 (such as INTEL's XEON) connected to a memory unit 12013 (RAM) and a multiple database accelerator (DB accelerator) 12014 .

The DB accelerator is optional, and the DB acceleration operation may be performed by the CPU 12012 .

All traffic flows through the CPU, which can be coupled to the DB accelerator via a relatively limited bandwidth link such as PCIe.

A large amount of resources is dedicated to sending units of information between multiple motherboards.

Accordingly, there is an increasing demand for the provision of efficient data centers and other computer systems.

Artificial intelligence (AI) applications, such as neural networks, are growing dramatically in size. To cope with the increasing size of neural networks, multiple servers, each acting as an AI acceleration server (including server motherboard), are used to perform neural network processing tasks such as learning. An example of a system including multiple AI acceleration servers placed in different racks is shown in FIG. 1 .

In a typical training session, a very large number of images are processed simultaneously, giving a vast number of values such as loss. A huge number of values are passed between different AI-accelerated servers, resulting in an unusual amount of traffic. For example, some neural network layers may be computed across multiple GPUs located on different AI acceleration servers, and may require aggregation across the network, which will require a lot of bandwidth.

Carrying an unusual amount of traffic requires ultra-high bandwidth, which may not be feasible or cost-effective.

97A shows a system 12050 that includes subsystems. Here, each subsystem is a switch ( 12051), and the CPU 12054 is connected to multiple AI accelerators 12057 (eg, graphic processing unit, AI chip (AI ASIC), FPGA, etc.) (via a PCIe bus). The NICs are coupled to each other (eg, by one or more switches) by the network (eg, using Ethernet, UDP links, etc.), and these NICs may be capable of delivering the ultra-high bandwidth required by the system.

Accordingly, the demand for an efficient AI computing system is increasing.

According to other disclosed embodiments, a non-transitory computer-readable storage medium may store program instructions that are executed by at least one processing device and perform one or more of the methods described herein.

The above general description and the following detailed description are illustrative only and not limiting on the scope of the claims of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various embodiments of the disclosure.
1 schematically shows a central processing unit (CPU).
2 schematically illustrates a graphic processing unit (GPU).
3A schematically illustrates one embodiment of an exemplary hardware chip in accordance with the disclosed embodiment.
3B schematically illustrates another embodiment of an exemplary hardware chip in accordance with the disclosed embodiment.
4 schematically illustrates general instructions executed by an exemplary hardware chip in accordance with disclosed embodiments.
5 schematically illustrates special instructions executed by an exemplary hardware chip in accordance with the disclosed embodiment.
6 schematically illustrates a processing group for use in an example hardware chip in accordance with the disclosed embodiment.
7A schematically illustrates a rectangular array of processing groups in accordance with a disclosed embodiment.
7B schematically illustrates an elliptical array of processing groups in accordance with a disclosed embodiment.
7C schematically illustrates an array of hardware chips in accordance with a disclosed embodiment.
7D schematically illustrates another array of hardware chips in accordance with the disclosed embodiment.
8 is a flow chart illustrating an exemplary method for compiling a set of instructions for execution on an exemplary hardware chip in accordance with the disclosed embodiment.
Fig. 9 schematically shows a memory bank.
Fig. 10 schematically shows a memory bank.
11 schematically illustrates one embodiment of an exemplary memory bank with subbank control in accordance with the disclosed embodiment.
12 schematically illustrates another embodiment of an exemplary memory bank with subbank control in accordance with the disclosed embodiments.
13 is a block diagram of an exemplary memory chip according to the disclosed embodiment.
14 is a schematic diagram of an exemplary redundant logic block set in accordance with the disclosed embodiment.
15 is a schematic diagram of an exemplary logic block in accordance with the disclosed embodiment.
16 is a block diagram of an exemplary logic block to which a bus is connected according to the disclosed embodiment.
17 is a schematic diagram of exemplary logic blocks connected in series in accordance with the disclosed embodiment.
18 is a schematic diagram of exemplary logic blocks connected in a two-dimensional array in accordance with disclosed embodiments.
19 is a block diagram of an exemplary logic block that is compositely connected according to the disclosed embodiment.
20 is an exemplary flow diagram illustrating a redundant block activation process in accordance with the disclosed embodiment.
21 is an exemplary flow diagram illustrating an address assignment process in accordance with the disclosed embodiment.
22 is a schematic diagram of an exemplary processing apparatus according to the disclosed embodiment.
23 is a block diagram of an exemplary processing apparatus according to the disclosed embodiment.
24 includes an exemplary memory configuration diagram in accordance with disclosed embodiments.
25 is an exemplary flow chart illustrating a memory setup process in accordance with the disclosed embodiment.
26 is an exemplary flow diagram illustrating a memory read process in accordance with the disclosed embodiment.
27 is an exemplary flowchart illustrating an execution process in accordance with the disclosed embodiment.
28 illustrates an exemplary memory chip including a refresh controller according to the present disclosure.
29A illustrates an exemplary refresh controller in accordance with the present disclosure.
29B illustrates another exemplary refresh controller according to the present disclosure.
30 is an exemplary flowchart of a process executed by a refresh controller according to the present disclosure;
31 is an exemplary flowchart of a process implemented by a compiler in accordance with the present disclosure.
32 is another exemplary flowchart of a process implemented by a compiler according to the present disclosure.
33 illustrates an exemplary refresh controller configured by a stored pattern according to the present disclosure.
34 is an exemplary flowchart of a process implemented by software in a refresh controller according to the present disclosure;
35A illustrates an exemplary wafer including a die according to the present disclosure.
35B illustrates an exemplary memory chip coupled to an input/output bus in accordance with the present disclosure.
35C illustrates an exemplary wafer comprising memory chips arranged in rows and coupled to an input-output bus in accordance with the present disclosure.
35D illustrates two memory chips forming a group and coupled to an input-output bus in accordance with the present disclosure.
35E illustrates an exemplary wafer including dies arranged in a hexagonal grid and coupled to an input-output bus in accordance with the present disclosure.
36A-36D illustrate various possible configurations of a memory chip coupled to an input/output bus according to the present disclosure.
37 illustrates an exemplary grouping of dies sharing glue logic in accordance with the present disclosure.
38A-38B illustrate exemplary cuts of a wafer in accordance with the present disclosure.
38C illustrates an exemplary arrangement of dies on a wafer and arrangement of an input-output bus in accordance with the present disclosure.
39 illustrates an exemplary memory chip on a wafer including interconnected processor subunits in accordance with the present disclosure.
40 is an exemplary flowchart of a process for laying out a group of memory chips from a wafer in accordance with the present disclosure.
41A is another exemplary flow diagram of a process for laying out a group of memory chips from a wafer in accordance with the present disclosure.
41B-41C are exemplary flowcharts of a process for determining a cut pattern for cutting one or more groups of memory chips from a wafer in accordance with the present disclosure.
42 shows an example of circuitry in a memory chip that provides dual port access along a column in accordance with the present disclosure.
43 shows an example of circuitry in a memory chip that provides dual port access along a row in accordance with the present disclosure.
44 shows an example of circuitry in a memory chip that provides dual port access along rows and columns in accordance with the present disclosure.
45A illustrates dual reads utilizing duplicated memory arrays or mats.
45B illustrates dual writes utilizing duplicated memory arrays or mats.
46 shows an example of circuitry in a memory chip including a switching element for dual port access along a row in accordance with the present disclosure.
47A is an exemplary flow diagram of a process for providing dual port access on a single port memory array or mat in accordance with the present disclosure.
47B is an exemplary flow diagram of another process for providing dual port access on a single port memory array or mat in accordance with the present disclosure.
48 illustrates another example of circuitry in a memory chip that provides dual port access along rows and columns in accordance with the present disclosure.
49 shows an example of a switching element for dual port access in a memory mat according to the present disclosure.
50 illustrates an exemplary integrated circuit including a reduction unit configured to access a partial word in accordance with the present disclosure.
FIG. 51 illustrates a memory bank for utilizing a reduction unit as described with respect to FIG. 50 .
52 illustrates a memory bank utilizing a reduction unit integrated with PIM logic in accordance with the present disclosure.
53 illustrates a memory bank utilizing PIM logic in accordance with the present disclosure to activate a switch to access a partial word.
54A illustrates a memory bank including a partitioned column multiplex disabling partial word access in accordance with the present disclosure.
54B is an exemplary flowchart of a process for partial word access in memory according to the present disclosure.
55 illustrates a conventional memory chip including multiple memory mats.
56 illustrates an exemplary memory chip including an enable circuit for reducing power consumption during opening of a line in accordance with the present disclosure.
57 depicts another exemplary memory chip including an activation circuit for reducing power consumption during opening of a line in accordance with the present disclosure.
58 illustrates another exemplary memory chip including an activation circuit for reducing power consumption during opening of a line in accordance with the present disclosure.
59 illustrates a further exemplary memory chip including an activation circuit for reducing power consumption during opening of a line in accordance with the present disclosure.
60 illustrates an exemplary memory chip including global wordlines and local wordlines to reduce power consumption during opening of a line in accordance with the present disclosure.
61 illustrates another exemplary memory chip including a global wordline and a local wordline for reducing power consumption during opening of a line in accordance with the present disclosure.
62 is an exemplary flowchart of a process for sequential opening of a line within a memory in accordance with the present disclosure.
63 shows a conventional tester for a memory chip.
64 shows another conventional tester for a memory chip.
65 illustrates an example of testing a memory chip using a logic element on the same substrate as a memory according to the present disclosure.
66 illustrates another example of testing a memory chip using a logic element on the same substrate as a memory according to the present disclosure.
67 illustrates another example of testing a memory chip using a logic element on the same substrate as a memory according to the present disclosure.
68 illustrates an additional example of testing a memory chip using a logic element on the same substrate as a memory according to the present disclosure.
69 illustrates another additional example of testing a memory chip using a logic element on the same substrate as a memory according to the present disclosure.
70 is an exemplary flowchart of a process for testing a memory chip in accordance with the present disclosure.
71 is an exemplary flowchart of another process for testing a memory chip in accordance with the present disclosure.
72A illustrates an integrated circuit including a memory array and a processing array in accordance with embodiments of the present disclosure.
72B illustrates a memory area inside an integrated circuit according to embodiments of the present disclosure.
73A illustrates an integrated circuit in which a controller is exemplarily configured according to embodiments of the present disclosure.
73B illustrates a configuration for concurrently executing a replication model according to embodiments of the present disclosure.
74A illustrates an integrated circuit in which a controller is another exemplary configuration according to embodiments of the present disclosure.
74B is a flowchart of a method of protecting an integrated circuit according to embodiments of the present disclosure;
74C illustrates a detection element positioned within a chip according to embodiments of the present disclosure.
75A illustrates a scalable processor memory system including a plurality of distributed processor memory chips according to embodiments of the present disclosure.
75B illustrates a scalable processor memory system including a plurality of distributed processor memory chips according to embodiments of the present disclosure.
75C illustrates a scalable processor memory system including a plurality of distributed processor memory chips according to embodiments of the present disclosure.
75D illustrates a dual port distributed processor memory chip according to embodiments of the present disclosure.
75E exemplarily illustrates timing according to embodiments of the present disclosure.
76 illustrates a processor memory chip having an integrated controller and an interface module and configuring a scalable processor memory system according to embodiments of the present disclosure.
77 is a flowchart illustrating data transmission between processor memory chips in the scalable processor memory system illustrated in FIG. 75A according to embodiments of the present disclosure.
78A illustrates a system for detecting a zero value stored in one or more specific addresses of a plurality of memory banks implemented in a memory chip at a chip level according to embodiments of the present disclosure.
78B illustrates a memory chip that detects a 0 value stored in one or more specific addresses of a plurality of memory banks at a memory bank level according to embodiments of the present disclosure.
79 illustrates a memory bank for detecting a zero value stored in one or more specific addresses of a plurality of memory mats at a memory mat level according to embodiments of the present disclosure;
80 is a flowchart of an exemplary method for detecting a zero value at a specific address of a plurality of discrete memory banks in accordance with embodiments of the present disclosure.
81A illustrates a system for activating a next row associated with a memory bank based on next row prediction according to embodiments of the present disclosure.
81B illustrates another embodiment of the system of FIG. 81A in accordance with embodiments of the present disclosure.
81C illustrates a first subbank row controller and a second subbank row controller of each memory subbank according to embodiments of the present disclosure.
81D illustrates an embodiment for next row prediction according to embodiments of the present disclosure.
81E illustrates an embodiment of a memory bank according to embodiments of the present disclosure.
81F illustrates another embodiment of a memory bank according to embodiments of the present disclosure.
82 illustrates a dual control memory bank for reducing a memory row activation penalty according to embodiments of the present disclosure.
83A shows a first example of accessing and activating a row of a memory bank.
83B shows a second example of accessing and activating a row of a memory bank.
83C shows a third example of accessing and activating a row of a memory bank.
84 shows a conventional CPU/register file and external memory architecture.
85A illustrates an exemplary distributed processor memory chip including a memory mat serving as a register file in accordance with one embodiment.
85B illustrates an exemplary distributed processor memory chip including a memory mat configured to serve as a register file in accordance with another embodiment.
85C illustrates an exemplary device including a memory mat serving as a register file according to another embodiment.
86 depicts a flow diagram of an exemplary method of executing at least one instruction in a distributed processor memory chip in accordance with disclosed embodiments.
87A shows an example of a separate server.
87B shows an example of distributed processing.
87C shows an example of a memory/processing unit.
87D shows an example of a memory/processing unit.
87E shows an example of a memory/processing unit.
87F shows an example of an integrated circuit including a memory/processing unit and one or more communication modules.
87G illustrates an example of an integrated circuit including a memory/processing unit and one or more communication modules.
87H is an example of a method.
87I is an example of a method.
88A is an example of a method.
88B is an example of a method.
88C is an example of a method.
89A is an example of a memory/processing unit and vocabulary.
89B is an example of a memory/processing unit.
89C is an example of a memory/processing unit.
89D is an example of a memory/processing unit.
89E is an example of a memory/processing unit.
89F is an example of a memory/processing unit.
89G is an example of a memory/processing unit.
89H is an example of a memory/processing unit.
90A is an example of a system.
90B is an example of a system.
90C is an example of a system.
90D is an example of a system.
90E is an example of a system.
90F is an example of a method.
91A is an example of a memory and filtering system, a storage device, and a CPU.
91B is an example of a memory and processing system, a storage device, and a CPU.
92A is an example of a memory and processing system, a storage device, and a CPU.
92B is an example of a memory/processing unit.
92C is an example of a memory and filtering system, a storage device, and a CPU.
92D is an example of a memory and processing system, a storage device, and a CPU.
92E is an example of a memory and processing system, a storage device, and a CPU.
92F is an example of a method.
92G is an example of a method.
92H is an example of a method.
92I is an example of a method.
92J is an example of a method.
92K is an example of a method.
93A is a cross-sectional view of an example of a hybrid integrated circuit;
93B is a cross-sectional view of an example of a hybrid integrated circuit.
93C is a cross-sectional view of an example of a hybrid integrated circuit.
93D is a cross-sectional view of an example of a hybrid integrated circuit.
93E is a plan view of an example of a hybrid integrated circuit.
93F is a plan view of an example of a hybrid integrated circuit.
93G is a plan view of an example of a hybrid integrated circuit.
93H is a cross-sectional view of an example of a hybrid integrated circuit.
93I is a cross-sectional view of an example of a hybrid integrated circuit.
93J is an example of a method.
94A is an example of a storage system, one or more devices, and a computer system.
94B is an example of a storage system, one or more devices, and a computer system.
94C is an example of one or more devices and computer systems.
94D is an example of one or more devices and computer systems.
94E is an example of a database acceleration integrated circuit.
94F is an example of a database acceleration integrated circuit.
94G is an example of a database acceleration integrated circuit.
94H is an example of a database acceleration unit.
94I is an example of a group of blade and database acceleration integrated circuits.
94J is an example of groups of database acceleration integrated circuits.
94K is an example of groups of database acceleration integrated circuits.
94L is an example of groups of database acceleration integrated circuits.
94M is an example of groups of database acceleration integrated circuits.
94N is an example of a system.
94o is an example of a system.
94p is an example of a method.
95A is an example of a method.
95B is an example of a method.
95C is an example of a method.
96A is an example of a prior art system.
96B is an example of a system.
96C is an example of a database accelerator board.
96D is an example of a portion of a system.
97A is an example of a prior art system.
97B is an example of a system.
97C is an example of an AI network interface card.

The following detailed description refers to the accompanying drawings. For convenience, the same reference numbers are used for the same or similar elements in the drawings and description. While several exemplary embodiments are described, various changes, modifications, implementations, and the like are possible. For example, components illustrated in the drawings may be substituted, or added, or changed, and methods included in the description may be changed by substituting, changing the order, deleting, or adding steps. Accordingly, the following detailed description is not limited to the disclosed embodiments and examples, and the scope of the correct claims is defined by the appended claims.

processor architecture

Throughout this disclosure, the term 'hardware chip' refers to a semiconductor wafer (such as silicon) on which one or more circuit elements (eg, transistors, capacitors, resistors, etc.) are formed. The circuit element may form a processing element or a memory element. The 'processing element' refers to one or more circuit elements that simultaneously perform at least one logical function (eg, an arithmetic function, a logic gate, other Boolean operation, etc.). A processing element may be a general-purpose processing element (eg, a plurality of configurable transistors) or a dedicated processing element (eg, a specific logic gate or plurality of circuit elements designed to perform a particular logic function). A 'memory element' refers to one or more circuit elements that can be utilized to store data. A 'memory element' may also be referred to as a 'memory cell'. The memory device may be dynamic (ie, requiring an electrical refresh to retain data storage), static (ie, retaining data for a period of time after power is turned off), or non-volatile memory.

The processing elements may be bonded together to form a processor subunit. Accordingly, the 'processor subunit' may include a minimum set of processing elements capable of executing at least one task or instruction (eg, an operation or instruction of a processor instruction set). For example, a subunit may include one or more general purpose processing elements configured to together execute instructions, one or more general purpose processing elements configured to be paired with one or more dedicated processing elements to execute instructions in a complementary manner, and the like. The processor subunits may be disposed in an array on a substrate (eg, a wafer). An 'array' may include a rectangular shape, but an array arrangement of sub-units may be formed in any other shape on a substrate.

The memory elements may be bonded to each other to form a memory bank. For example, a memory bank may include one or more lines of memory elements connected along at least one line (or other conductive connection). Also, the memory elements may be connected along at least one additional line in a different direction. For example, the memory device may be disposed along a wordline and a bitline as described below. Although a memory bank may include lines, any other arrangement may be utilized to place devices within a substrate to form the bank on the substrate. Also, one or more banks may be electrically coupled to at least one memory controller to form a memory array. A memory array may include a rectangular arrangement of banks, although the arrangement of banks within the array may be formed in any other shape on the substrate.

Throughout this disclosure, a 'bus' refers to any communication connection between devices on a substrate. For example, a line or line (forming an electrical connection), optical fiber (forming an optical connection), or any other connection that communicates between components may be referred to as a 'bus'.

In conventional processors, general-purpose logic circuitry and shared memory are paired. The shared memory may store not only a set of instructions to be executed by the logic circuit, but also data to be utilized for the execution of the instruction set and data obtained as a result of the execution of the instruction set. As described below, some existing processors utilize a caching system to reduce read latency from shared memory, but the existing caching system maintains a shared state. Existing processors include central processing units (CPUs), graphic processing units (GPUs), and various application-specific integrated circuits (ASICs). 1 shows an example of a CPU, and FIG. 2 shows an example of a GPU.

1 , the CPU 100 may include a processing unit 110 including one or more processor subunits such as a processor subunit 120a and a processor subunit 120b. Although not shown in FIG. 1 , each processor subunit may include a plurality of processing elements. In addition, the processing unit 110 may include one or more stages of an on-chip cache (on-chip cache). These cache elements are mostly formed on the same semiconductor die as the processor 110 rather than being connected to the processor subunits 120a and 120b and the processor subunits 120a and 120b through one or more buses formed in the substrate including the cache elements. do. Direct placement on the same die rather than connecting via a bus is common for both first-level (L1) and second-level (L2) caches of traditional processors. Alternatively, in older processors, the L2 cache was shared between the processor subunits and the processor subunits using a back-side bus between the L2 caches. The rear bus is generally larger than the front-side bus, as described below. Accordingly, the cache must be shared with all processor subunits on the die, such that the cache 130 is formed on the same die as the processor subunits 120a, 120b or via one or more backside buses to the processor subunits 120a, 120b. may be communicatively coupled to In both busless embodiments (eg, the cache is formed directly on the die) and embodiments that utilize a backside bus, the cache is shared between the processor subunits of the CPU.

In addition, the processing unit 110 communicates with the shared memory 140a and the shared memory 140b. For example, the memories 140a and 140b may represent memory banks of shared dynamic random access memory (DRAM). Although two banks are shown in the figure, most conventional memory chips include 8 to 16 memory banks. Accordingly, the processor subunits 120a and 120b may store data to be operated by the processor subunits 120a and 120b by utilizing the shared memories 140a and 140b. However, as a result of this configuration, when the clock speed of the processing unit 110 exceeds the data transfer rate of the bus, the bus between the memories 140a and 140b and the processing unit 110 becomes a bottleneck. This is common in conventional processors, which results in lower throughput than the throughput specifications based on clock speed and number of transistors.

As shown in Figure 2, a similar problem exists with the GPU. GPU 200 includes one or more processor subunits (eg, subunits 220a, 220b, 220c, 220d, 220e, 220f, 220g, 220h, 220i, 220j, 220k, 220l, 220m, 220n, 220o, 220p). It may include a processing unit 210 that includes. In addition, the processing unit 210 may include one or more stages of on-chip cache and/or register files. Such a cache element is generally formed on the same semiconductor die as the processing unit 210 . Indeed, in the example of Figure 2, cache 210 is formed on the same die as processing unit 210 and shared among all processor subunits, whereas caches 230a, 230b, 230c, 230d are each formed on a subset of processors ( It is formed on a subset) and is dedicated to it.

In addition, the processing unit 210 communicates with the shared memories 250a, 250b, 250c, and 250d. For example, the memories 250a, 250b, 250c, and 250d may represent a memory bank of a shared DRAM. Accordingly, the processor subunit of the processing unit 210 may store data subsequently operated by the processor subunit by utilizing the shared memories 250a, 250b, 250c, and 250d. However, similar to the description of the CPU above, as a result of this arrangement, the bus between the memories 250a, 250b, 250c, and 250d and the processing unit 210 becomes a bottleneck.

Overview of Disclosed Hardware Chips

3A schematically illustrates one embodiment of an exemplary hardware chip 300 . The hardware chip 300 may include a distributed processor designed to alleviate bottlenecks for CPUs, GPUs, and other existing processors described above. A distributed processor may include a plurality of processor subunits spatially distributed on a single substrate. Further, as described above, in the distributed processor of the present disclosure, the corresponding memory banks are also spatially distributed on the substrate. In some embodiments, a distributed processor is associated with an instruction set, and each subunit of the processor subunits of the distributed processor may be responsible for performing one or more tasks included in the instruction set.

As shown in FIG. 3A , the hardware chip 300 may include a plurality of processor subunits, such as logic and control subunits 320a, 320b, 320c, 320d, 320e, 320f, 320g, 320h. As further shown in FIG. 3A , each processor subunit may include a dedicated memory instance. For example, logic and control subunit 320a is operatively coupled to dedicated memory instance 330a, logic and control subunit 320b is operatively coupled to dedicated memory instance 330b, and logic and control subunit 330b. Unit 320c is operatively coupled to dedicated memory instance 330c, logic and control subunit 320d is operatively coupled to dedicated memory instance 330d, and logic and control subunit 320e is dedicated Logic and control subunit 320f is operatively coupled to memory instance 330e, logic and control subunit 320f is operatively coupled to dedicated memory instance 330f, and logic and control subunit 320g is operatively coupled to dedicated memory instance 330g. operatively coupled, and the logic and control subunit 320h is operatively coupled to the dedicated memory instance 330h.

Although each memory instance is shown as being a single memory bank in FIG. 3A , the hardware chip 300 may include two or more memory banks with dedicated memory instances for processor subunits on the hardware chip 300 . Also, although in Figure 3A each processor subunit is shown as including both logic elements and controls for each dedicated memory bank, the hardware chip 300 provides, at least in part, control separate from the logic elements for the memory banks. can also be used Also, as shown in FIG. 3A , two or more processor subunits and corresponding memory banks may be combined into processing groups 310a, 310b, 310c, 310d, for example. The 'processing group' may indicate a spatial division on a substrate on which the hardware chip 300 is formed. Accordingly, the processing group may further include additional controls for the memory banks of the processing group, such as, for example, controls 340a, 340b, 340c, 340d. Additionally or alternatively, the 'processing group' may indicate a logical bundle for compiling code to be executed on the hardware chip 300 . Accordingly, a compiler for the hardware chip 300 (described below) may separate the overall instruction set between processing groups on the hardware chip 300 .

In addition, the host 350 may provide commands, data, and other inputs to the hardware chip 300 and read outputs from the hardware chip 300 . Accordingly, the entire set of instructions may be executed on a single die, eg, a die hosting the hardware chip 300 . In practice, out-of-die communication may be all about loading instructions into the hardware chip 300 , sending inputs to the hardware chip 300 , and reading the output from the hardware chip 300 . Accordingly, since the processor subunit of the hardware chip 300 communicates with the dedicated memory bank of the hardware chip 300 , all calculations and memory operations can be performed on the die (on the hardware chip 300 ).

3B schematically illustrates an embodiment of another exemplary hardware chip 300'. Although shown as an alternative to the hardware chip 300 , the architecture shown in FIG. 3B may be at least partially merged with the architecture shown in FIG. 3A .

As shown in FIG. 3B , the hardware chip 300 ′ may include a plurality of processor subunits such as processor subunits 350a , 350b , 350c and 350d . As further shown in FIG. 3B , each processor subunit may include a plurality of dedicated memory instances. For example, processor subunit 350a is coupled to dedicated memory instances 330a, 330b, processor subunit 350b is coupled to dedicated memory instances 330c, 330d, and processor subunit 350c is coupled to dedicated memory instances. coupled to 330e and 330f, and processor subunit 350d coupled to dedicated memory instances 330g and 330h. Also, as shown in FIG. 3B , the processor subunits and corresponding memory banks may be combined into processing groups 310a, 310b, 310c, and 310d, for example. As described above, the 'processing group' may indicate a logical bundle for spatial division on a substrate on which the hardware chip 300' is formed and/or for compiling code to be executed on the hardware chip 300'.

As further shown in FIG. 3B , the processor subunits may communicate with each other via a bus. For example, as shown in FIG. 3B , processor subunit 350a communicates with processor subunit 350b via bus 360a, communicates with processor subunit 350c via bus 360c, and , may communicate with the processor subunit 350d through the bus 360f. Similarly, processor subunit 350b communicates with processor subunit 350a via bus 360a (described above), and communicates with processor subunit 350c via bus 360e (described above). , may communicate with the processor subunit 350d through the bus 360d. In addition, processor subunit 350c communicates with processor subunit 350a via bus 360c (described above), and communicates with processor subunit 350b via bus 360e (described above). , may communicate with the processor subunit 350d through the bus 360b. Accordingly, the processor subunit 350d communicates with the processor subunit 350a via the bus 360f (described above), and communicates with the processor subunit 350b via the bus 360d (described above). ), may communicate with the processor subunit 350c via the bus 360b (described above). Those skilled in the art will appreciate that fewer buses than those shown in FIG. 3B may be used. For example, the bus 360e may be removed so that communication between the processor subunit 350b and the processor subunit 350c is via the processor subunit 350a and/or the processor subunit 350d. Similarly, the bus 360f may be eliminated so that communication between the processor subunit 350a and the processor subunit 350d is via the processor subunit 350b or the processor subunit 350c.

Additionally, those skilled in the art will appreciate that architectures other than those shown in FIGS. 3A and 3B may be utilized. For example, an array of processing groups may be disposed on a substrate, each comprising a single processor subunit and a memory instance. The processor subunit may additionally or alternatively form part of a controller for a corresponding dedicated memory bank, part of a controller for a corresponding dedicated memory mat, and the like.

In accordance with the architecture described above, the hardware chips 300, 300' can provide significantly increased efficiency over conventional architectures for memory intensive tasks. For example, database operations and artificial intelligence algorithms (eg, neural networks) are examples of memory-intensive tasks for which conventional architectures are less efficient than hardware chips 300 and 300'. Accordingly, the hardware chips 300 and 300' may be referred to as a database accelerator processor and/or an artificial intelligence accelerator processor.

Configuration of the disclosed hardware chip

The hardware chip architecture described above may be configured for the execution of code. For example, each processor subunit may execute code (defining a set of instructions) separately from other processor subunits within a hardware chip. Accordingly, rather than relying on the operating system to manage multithreading or utilizing multitasking (concurrency rather than parallelism), the hardware chip of the present disclosure allows the processor subunits to operate completely in parallel.

In addition to the fully parallel implementation described above, at least some of the instructions assigned to each processor subunit may overlap. For example, a plurality of processor subunits disposed on a distributed processor may execute redundant instructions, such as an implementation of an operating system or other management software, while non-parallel tasks may be performed within the context of an operating system or other management software. You can run non-overlapping commands.

4 illustrates an exemplary process 400 for executing general instructions with processing group 410 . For example, processing group 410 may include some of the hardware chips of the present disclosure (hardware chip 300 , hardware chip 300 ′, etc.).

As shown in FIG. 4 , the command may be sent to the processor subunit 430 paired with the dedicated memory instance 420 . An external host (eg, host 350 ) may send commands to processing group 410 for execution. Alternatively, the host 350 may send and store the instruction set including the instruction set in the memory instance 420 so that the processor subunit 430 can retrieve the instruction from the memory instance 420 and execute the fetched instruction. . Accordingly, the instruction may be executed by processing element 440 , which is a generic processing element that may be configured to execute the fetched instruction. In addition, processing group 410 may include control 460 for memory instance 420 . As shown in FIG. 4 , control 460 may perform reads and/or writes to memory instance 420 as required by processing element 440 when executing the received command. After execution of the command, the processing group 410 may output the result of the command to an external host or another processing group on the same hardware chip.

In some embodiments, as shown in FIG. 4 , the processor subunit 430 may further include an address generator 450 . The 'address generator' may include a plurality of processing elements configured to determine the addresses of one or more memory banks for reading and writing, and also perform operations on data located at the determined addresses (eg, addition, subtraction, multiplication, etc.) ) can be done. For example, the address generator 450 may determine an address for reading or writing to the memory. In an embodiment, when the read value is no longer needed, the address generator 450 may improve efficiency by overwriting the read value with a new value determined based on a command. Additionally or alternatively, address generator 450 may select an available address to store the results of instruction execution. This makes it possible to schedule the reading of results for a later clock cycle, which is more convenient for the external host. In another example, the address generator 450 may determine which addresses to read and write to during multicycle calculations, such as vector or matrix multiply-accumulate calculations. Accordingly, the address generator 450 may maintain or calculate memory addresses for reading data and writing the intermediate results of multicycle calculations, so that the processor subunit 430 may continue to process these memory addresses without the need to store them. have.

5 depicts an exemplary process 500 for executing special instructions with processing group 510 . For example, processing group 510 may include a portion of a hardware chip of the present disclosure (hardware chip 300 , hardware chip 300 ′, etc.).

As shown in FIG. 5 , a special instruction (eg, multiply-accumulate instruction) may be sent to the processing element 530 paired with the dedicated memory instance 520 . An external host (eg, host 350 ) may transmit a command to processing element 530 for execution. Accordingly, commands may be executed on specific signals from the host by processing element 530 , which is a special processing element configured to execute specific commands (including received commands). Alternatively, processing element 530 can retrieve instructions from memory instance 520 for execution. Thus, in the example of FIG. 5 , processing element 530 is a MAC circuit configured to execute multiply-accumulate (MAC) commands received from an external host or fetched from memory instance 520 . After executing the command, the processing group 510 may output the result of the command to an external host or another processing group of the same hardware chip. Although only a single command and a single result are shown in the figure, multiple commands may be received, retrieved, and executed, and multiple results may be merged on processing group 510 prior to output.

Although the MAC circuit is shown in FIG. 5, additional or alternative special circuits may be included in the processing group. For example, a MAX-read instruction (outputs the maximum value of a vector), a MAX0-read instruction (a function called a rectifier that outputs the entire vector but MAX to zero) may be implemented.

Although the general processing group 410 of FIG. 4 and the special processing group 510 of FIG. 5 are shown as separate, they may be merged. For example, a generic processor subunit may be combined with one or more specialized processor subunits to form a processor subunit. Accordingly, a generic processor subunit may be utilized for all instructions that are not executable by one or more specialized processor subunits.

Those skilled in the art will appreciate that neural network implementations and other memory-intensive tasks may be handled by specialized logic circuitry. For example, database queries, packet inspection, string comparison, and other functions may be more efficient if implemented by the hardware chips described herein.

Memory-based architecture for distributed processing

On a hardware chip according to the present disclosure, a dedicated bus may transfer data between processor subunits on the chip and/or between a processor subunit and a corresponding dedicated memory bank. Using a dedicated bus can reduce the cost of arbitration because conflicting demands are either impossible or can be easily circumvented using software rather than hardware.

6 schematically illustrates a processing group 600 . Processing group 600 may be for use in a hardware chip, such as hardware chip 300 , hardware chip 300 ′, or the like. The processor subunit 610 may be coupled to the memory 620 via a bus 630 . The memory 620 may include a RAM element that stores data and code for execution by the processor subunit 610 . In some embodiments, memory 620 may be an N-way memory (where N is a number greater than or equal to 1 indicating the number of segments in interleaved memory 620 ). Since processor subunit 610 is coupled to memory 620 dedicated to processor subunit 610 via bus 630, N may be kept to a relatively low number without loss of execution performance. This means an improvement over the existing multi-way register file or cache, where small N results in poor execution performance and high N results in large area and power loss.

The size of the memory 620 , the number of N of N-ways, and the width of the bus 630 depending on the size of data associated with the task, etc. to suit the requirements of the task and application implementation of the system using the processing group 600 . This can be adjusted. The memory element 620 is, for example, a volatile memory (RAM, DRAM, SRAM, phase-change RAM or PRAM), a ferromagnetic RAM (magnetoresistive RAM or MRAM), a resistive RAM (resistive RAM or ReRAM), etc. ) or non-volatile memory (flash memory or ROM), one or more types of memory known in the art. According to some embodiments, a portion of the memory device 620 may include a first memory type, and another portion may include a different memory type. For example, the code region of the memory device 620 may include a ROM device, and the data region of the memory device 620 may include a DRAM device. As another example of this division, the weight of the neural network may be stored in flash memory, while data for calculation may be stored in DRAM.

The processor subunit 610 includes a processing element 640 that may include a processor. The processor may or may not be pipelined, as will be understood by one of ordinary skill in the art, and a custom Reduced Instruction Set Computing (RISC) implemented on any commercially available integrated circuit (eg, ARM, ARC, RISC-V, etc.) known in the art. ) element or other processing schema. The processing element 640 may include a controller including an Arithmetic Logic Unit (ALU) or other controller in some embodiments.

According to some embodiments, the processing element 640 that executes received or stored code may include a general processing element, so that it is flexible and possible to perform a wide variety of processing operations. When comparing the power consumed during the performance of a specific operation, the non-dedicated circuit consumes more power than the specific operation dedicated circuit. Therefore, when performing certain complex arithmetic calculations, the processing element 640 consumes more power than dedicated hardware and may be less efficient. Thus, in accordance with some embodiments, the controller of processing element 640 may be designed to perform a specific operation (eg, a summation or 'move' operation).

In one example, certain operations may be performed by one or more accelerators 650 . Each accelerator can be programmed exclusively to perform a specific computation (eg, multiplication, floating point vector operation, etc.). By using the accelerator, the average power consumed per computation per processor subunit can be reduced, and future computational throughput is increased. The accelerator 650 may be selected according to an application the system is designed to implement (eg, execution of a neural network, execution of a database query, etc.). The accelerator 650 may be set by the processing element 640 and may work in concert with the processing element 640 to reduce power consumption and accelerate computation. Like a smart direct memory access (DMA) peripheral, the accelerator may additionally transfer data between the memory of the processing group 600 and the MUX/DEMUX/input/output ports (eg, MUX 650 , DEMUX 660 ). or alternatively may be used.

The accelerator 650 may be configured to perform various functions. For example, some accelerators can be configured to perform 16-bit floating-point calculations or 8-bit integer calculations often used in neural networks. Another example of an accelerator function is a 32-bit floating-point calculation often used in the training phase of neural networks. Another example of an accelerator function is query processing, such as used in a database. In some embodiments, accelerator 650 may include specialized processing elements to perform these functions and/or may be set according to configuration data stored in memory element 620 to enable modification.

Accelerator 650 may additionally or alternatively implement a time shift in a configurable written list of memory shifts of data to/from memory 620 or to/from other accelerators and/or inputs/outputs. Accordingly, as further described below, any data movement within a hardware chip using processing group 600 may utilize software synchronization rather than hardware synchronization. For example, an accelerator in one processing group (e.g. 600) can send data from input to accelerator every 10th cycle, and then output data in the next cycle, causing information to move from the processing group's memory to another processing group. have.

6 , in some embodiments, processing group 600 includes at least one input multiplexer (MUX) 660 coupled to an input port and at least one output demultiplexer (DEMUX) 670 coupled to an output port. ) may be further included. This MUX/DEMUX is controlled by a control signal (not shown) from one of the processing elements 640 and/or accelerator 650 , and current instructions performed by the processing element 640 and/or accelerator 650 . It may be determined according to an operation executed by one of the accelerators. In some cases, processing group 600 may be required to transmit data from an input port to an output port (according to a predetermined instruction from a code memory). Accordingly, each of the MUXs/DEMUXs is connected to the processing element 640 and the accelerator 650, as well as one or more of the input MUXs (eg, MUX 660) via one or more buses to the output DEMUXs (eg, DEMUX ( 670)).

The processing group 600 of FIG. 6 may be arranged to form a distributed processor such as that shown in FIG. 7A . A processing group may be disposed on the substrate 710 to form an array. In some embodiments, the substrate 710 may include a semiconductor substrate such as silicon. Additionally or alternatively, the substrate 710 may include a circuit board, such as a flexible circuit board.

As shown in FIG. 7A , substrate 710 may include a plurality of processing groups disposed thereon, such as processing group 600 . Accordingly, the substrate 710 includes a memory array including a plurality of banks, such as banks 720a, 720b, 720c, 720d, 720e, 720f, 720g, 720h. The substrate 710 also includes a processing array that may include a plurality of processor subunits, such as subunits 730a, 730b, 730c, 730d, 730e, 730f, 730g, 730h.

Also, as discussed above, each processing group may include a processor subunit and one or more corresponding memory banks dedicated to the processor subunit. Accordingly, as shown in Fig. 7A, each subunit is associated with a corresponding dedicated memory bank. For example, processor subunit 730a is associated with memory bank 720a , processor subunit 730b is associated with memory bank 720b , and processor subunit 730c is associated with memory bank 720c processor subunit 730d is associated with memory bank 720d, processor subunit 730e is associated with memory bank 720e, processor subunit 730f is associated with memory bank 720f, Processor subunit 730g is associated with memory bank 720g, and processor subunit 730h is associated with memory bank 720h.

To allow each processor subunit to communicate with a corresponding dedicated memory bank, the substrate 710 may include a first plurality of buses connecting one of the processor subunits to a corresponding dedicated memory bank. Accordingly, bus 740a connects processor subunit 730a to memory bank 720a, bus 740b connects processor subunit 730b to memory bank 720b, and bus 740c The processor subunit 730c connects to the memory bank 720c, the bus 740d connects the processor subunit 730d to the memory bank 720d, and the bus 740e connects the processor subunit 730e to the memory. to bank 720e, bus 740f connects processor subunit 730f to memory bank 720f, bus 740g connects processor subunit 730g to memory bank 720g, Bus 740h connects processor subunit 730h to memory bank 720h. Further, to allow each processor subunit to communicate with the other processor subunits, the substrate 710 may include a second plurality of buses connecting one of the processor subunits to the other of the processor subunits. In the example of Figure 7a, bus 750a connects processor subunit 730a to processor subunit 750e, bus 750b connects processor subunit 730a to processor subunit 750b, Bus 750c connects processor subunit 730b to processor subunit 750f, bus 750d connects processor subunit 730b to processor subunit 750c, and bus 750e connects processor subunit 750c The subunit 730c connects to the processor subunit 750g, the bus 750f connects the processor subunit 730c to the processor subunit 750d, and the bus 750g connects the processor subunit 730d to the processor subunit 750d. The bus 750h connects the processor subunit 730h to the processor subunit 750g, and the bus 750i connects the processor subunit 730g to the processor subunit 750g. and the bus 750j connects the processor subunit 730f to the processor subunit 750e.

Accordingly, in the exemplary arrangement shown in FIG. 7A , the plurality of logical processor subunits are arranged in at least one row and at least one column. A second plurality of buses connects each processor subunit to at least one adjacent processor subunit in the same row and to at least one adjacent processor subunit in the same column. 7A may be referred to as a 'partial tile connection'.

The arrangement shown in FIG. 7A may be modified to form a 'full tile connection'. A full tile connection includes an additional bus connecting the diagonal processor subunits. For example, the second plurality of buses may be connected between the processor subunit 730a and the processor subunit 730f, between the processor subunit 730b and the processor subunit 730e, and between the processor subunit 730b and the processor subunit 730b. between 730g, between processor subunit 730c and processor subunit 730f, between processor subunit 730c and processor subunit 730h, and between processor subunit 730d and processor subunit 730g may contain additional buses.

Full tile concatenation can be used for convolution computations, utilizing data and results stored in nearby processor subunits. For example, during convolutional image processing, each processor subunit may receive an image tile (eg, a pixel or group of pixels). In order to compute the convolution result, each processor subunit may obtain data from eight adjacent processor subunits, each receiving a corresponding tile. In a partial tile connection, data from a diagonally adjacent processor subunit may pass through another adjacent processor subunit coupled to that processor subunit. Accordingly, the distributed processor on the chip may be an artificial intelligence accelerator processor.

In a specific example of convolutional computation, an N x M image may be partitioned across a plurality of processor subunits. Each processor subunit may perform convolution with an A x B filter on the corresponding tile. To perform filtering on one or more pixels on a boundary between tiles, each processor subunit may request data from a neighboring processor subunit that has tiles that contain pixels on the same boundary. Accordingly, the generated code for each processor subunit configures that subunit to compute a convolution and pull data from one of the second plurality of buses whenever it needs data from an adjacent subunit. Corresponding commands for outputting data to the second plurality of buses are provided to the corresponding subunits to ensure proper timing of necessary data transfers.

The partial tile connection of FIG. 7A may be modified to be an N-partial tile connection. In this modification, the second plurality of buses route each processor subunit within a threshold distance of the corresponding processor subunit in the four directions (ie, up, down, left, right direction) through which the bus of FIG. 7A passes (eg, n processor subunits). It can further link to a processor subunit in the unit). A similar modification is made to the full tile connection (i.e., resulting in an N-full tile connection) so that a second plurality of buses connect each processor subunit to the corresponding processor subunit in 4 directions and 2 diagonal directions through which the bus of Figure 7a passes. Further connections may be made to processor subunits within a threshold distance (eg, within n processor subunits).

Other arrangements are possible. For example, in the arrangement shown in FIG. 7B , bus 750a connects processor subunit 730a to processor subunit 730d, and bus 750b connects processor subunit 730a to processor subunit 730d. 730b), the bus 750c connects the processor subunit 730b to the processor subunit 730c, and the bus 750d connects the processor subunit 730c to the processor subunit 730d. Accordingly, in the example shown in FIG. 7B , the plurality of processor subunits are arranged in a star pattern. A second plurality of buses connects each processor subunit to at least one adjacent processor subunit within a star pattern.

Additional arrangements (not shown) are also possible. For example, a neighbor connection arrangement may be used such that a plurality of processor subunits are arranged in one or more lines (similar to the arrangement shown in FIG. 7A ). In the neighbor connection arrangement, the second plurality of buses connect each processor subunit to a processor subunit on the left side of the line, a processor subunit on the right side of the line, a processor subunit on both left and right sides of the same line, and the like.

In another example, an N-linear connection arrangement may be used. In an N-linearly coupled arrangement, a second plurality of buses connects each processor subunit to a processor subunit within a threshold distance of that processor subunit (eg, within n processor subunits). The N-linear connection arrangement may be used with a line array (see above), a rectangular array (shown in FIG. 7A ), an elliptical array (shown in FIG. 7B ), or other geometric arrays.

In another example, an N-log connected arrangement may be used. In an N-log connected arrangement, a second plurality of buses connects each processor subunit to a processor subunit that is within a power-of-two threshold distance (eg, within 2 ⁿ processor subunits) of that processor subunit. The N-log connected arrangement may be used with a line array (see above), a rectangular array (shown in FIG. 7A ), an elliptical array (shown in FIG. 7B ), or other geometric arrays.

Any of the above-described connection schemes can be merged and used on the same hardware chip. For example, a full tile connection may be used in one area, and a partial tile connection may be used in another area. As another example, an N-linear connection arrangement may be used in one area, and an N-complete tile connection arrangement may be used in another area.

Alternatively or in addition to a dedicated bus between processor subunits of a memory chip, one or more shared buses may be used to interconnect all processor subunits (or a subset of all processor subunits) of a distributed processor. Conflicts on the shared bus can be avoided by utilizing code executed by the processor subunit to adjust the timing of data transfers on the shared bus, as described below. Additionally or alternatively to a shared bus, a configurable bus may be used to dynamically connect processor subunits to form groups of processor subunits that are connected by separate buses from each other. For example, the configurable bus may include a transistor or other mechanism that may be controlled by the processor subunit to transfer data to the selected processor subunit.

7A and 7B , a plurality of processor subunits of a processing array are spatially distributed among a plurality of discrete memory banks of the memory array. In another alternative embodiment (not shown), the plurality of processor subunits may be clustered in one or more regions of the substrate, and the plurality of memory banks may be clustered in one or more different regions of the substrate. In some embodiments, a combination of spatial distribution and clustering (not shown) may be used. For example, one region of the substrate may include a cluster of processor subunits, another region of the substrate may include a cluster of memory banks, and another region of the substrate may include a processing array distributed among the memory banks. .

Those skilled in the art will appreciate that forming an array of processing groups 600 on a substrate is not an exclusive embodiment. For example, each processor subunit may be associated with at least two dedicated memory banks. Accordingly, processing groups 310a , 310b , 310c , and 310d of FIG. 3B may be used in place of or in conjunction with processing group 600 to form a processing array and a memory array. Other processing groups including three, four, or more dedicated memory banks (not shown) may also be used.

Each processor subunit of the plurality of processor subunits may be configured to separately execute software code associated with a particular application for other processor subunits included in the plurality of processor subunits. For example, as described below, a plurality of sub-series of instructions may be grouped into machine code and provided to each processor subunit for execution.

In some embodiments, each dedicated memory bank includes at least one DRAM. Alternatively, the memory bank may include a combination of memory types such as SRAM, DRAM, flash memory, and the like.

In conventional processors, data sharing between processor subunits is generally performed with shared memory. Shared memory is usually implemented by a bus that requires a large portion of the chip area and/or is managed by additional hardware (eg, an arbiter). Buses are bottlenecks, as discussed earlier. In addition, shared memory, which may be external to the chip, usually includes cache coherency mechanisms and more complex caches (eg, L1 cache, L2 cache, shared DRAM) to provide correct and updated data to the processor subunits. . As will be described further below, the dedicated bus shown in FIGS. 7A and 7B enables a hardware chip that does not require hardware management (eg, arbiter). In addition, the use of dedicated memory such as that shown in Figures 7a and 7b eliminates the need for complex caching layers and coherency mechanisms.

Instead, it utilizes code executed individually by each processor subunit to dynamically A bus is provided on which timing is performed. This eliminates the need for most or all of the bus management hardware used in the past. In addition, complex caching mechanisms can be replaced with direct transfers over these buses, resulting in reduced latency during memory reads and writes.

memory-based processing array

7A and 7B , the memory chips of the present disclosure may operate individually. Alternatively, the memory chips of the present disclosure may be operative in one or more additional integrated circuits, such as memory devices (eg, one or more banks of DRAM), system-on-chip, field-programmable gate arrays (FPGAs), or other processing and/or memory chips. can be connected to In such embodiments, the tasks of a set of instructions executed by the architecture may be partitioned (eg, by a compiler, as described below) between a processor subunit of a memory chip and a processor subunit of an additional integrated circuit. have. For example, the additional integrated circuit may include a host (eg, host 350 of FIG. 3A ) that inputs commands and/or data to and receives output from the memory chip.

In order to interconnect the memory chip of the present disclosure and one or more additional integrated circuits, the memory chip may include a memory interface, such as a memory interface conforming to a Joint Electron Device Engineering Council (JEDEC) standard or a revised standard thereof. The one or more additional integrated circuits may then be coupled to a memory interface. Accordingly, when one or more additional integrated circuits are coupled to the plurality of memory chips of the present disclosure, data may be shared between the memory chips through the one or more additional integrated circuits. Additionally or alternatively, the one or more additional integrated circuits may include a bus for coupling with a bus of the memory chip of the present disclosure to cause the one or more additional integrated circuits to execute code in cooperation with the memory chip of the present disclosure. In such embodiments, one or more additional integrated circuits may additionally support distributed processing even if on a different substrate than the memory chip of the present disclosure.

Also, the memory chips of the present disclosure may be arranged in an array to form an array of a distributed processor. For example, as shown in FIG. 7C , one or more buses may connect the memory chip 770a to the additional memory chip 770b. In the example of FIG. 7C , memory chip 770a includes a processor subunit in which one or more corresponding memory banks are dedicated to each processor subunit. For example, processor subunit 730a is associated with memory bank 720a, processor subunit 730b is associated with memory bank 720b, and processor subunit 730e is associated with memory bank 720c. and the processor subunit 730f is associated with the memory bank 720d. A bus connects each processor subunit to a corresponding memory bank. Accordingly, bus 740a connects processor subunit 730a to memory bank 720a, bus 740b connects processor subunit 730b to memory bank 720b, and bus 740c is Processor subunit 730e is coupled to memory bank 720c, and bus 740d connects processor subunit 730f to memory bank 720d. Bus 750a also connects processor subunit 730a to processor subunit 750a, bus 750b connects processor subunit 730a to processor subunit 750b, and bus 750c connects the processor subunit 730b to the processor subunit 750f, and a bus 750d connects the processor subunit 730e to the processor subunit 750f. As described above, other arrangements of the memory chip 770a may be utilized.

Likewise, the memory chip 770b includes a processor subunit in which one or more corresponding memory banks are dedicated to each processor subunit. For example, processor subunit 730c is associated with memory bank 720e, processor subunit 730d is associated with memory bank 720f, and processor subunit 730g is associated with memory bank 720g. and the processor subunit 730h is associated with the memory bank 720h. A bus connects each processor subunit to a corresponding memory bank. Accordingly, bus 740e connects processor subunit 730c to memory bank 720e, bus 740f connects processor subunit 730d to memory bank 720f, and bus 740g The processor subunit 730g connects to the memory bank 720g, and a bus 740h connects the processor subunit 730h to the memory bank 720h. Bus 750g also connects processor subunit 730c to processor subunit 750g, bus 750h connects processor subunit 730d to processor subunit 750h, and bus 750i connects the processor subunit 730c to the processor subunit 750d, and a bus 750j connects the processor subunit 730g to the processor subunit 750h. As described above, other arrangements of the memory chip 770b may also be utilized.

The processor subunits of the memory chips 770a and 770b may be connected to each other using one or more buses. Accordingly, in the example of FIG. 7C , the bus 750e may connect the processor subunit 730b of the memory chip 770a and the processor subunit 730c of the memory chip 770b to each other, and the bus 750f is The processor subunit 730f of the memory chip 770a and the processor subunit 730c of the memory chip 770b may be connected to each other. For example, bus 750e can serve as an input bus to memory chip 770b (and thus an output bus of memory chip 770a), and bus 750f is an input bus to memory chip 770a. (thus, the output bus of the memory chip 770b), or vice versa. Alternatively, the buses 750e and 750f may both serve as bidirectional buses between the memory chips 770a and 770b.

Buses 750e and 750f may include direct circuit or interleaved over high-speed connections to reduce pins used in the inter-chip interface between memory chip 770a and integrated circuit 770b. . Additionally, any of the previously described connection configurations used for the memory chip may be used to connect the memory chip to one or more additional integrated circuits. For example, the memory chips 770a and 770b may be connected using a full tile connection or a partial tile connection, rather than using only two buses as shown in FIG. 7C .

Accordingly, although illustrated using buses 750e and 750f, architecture 760 may include fewer or more buses. For example, a single bus may be used between processor subunits 730a and 730b or between processor subunits 730f and 730c. Alternatively, an additional bus may be used between the processor subunits 730b and 730d or between the processor subunits 730f and 730d, and the like.

Also, although shown using a single memory chip and additional integrated circuits, multiple memory chips may be connected as described above. For example, as shown in FIG. 7C , the memory chips 770a, 770b, 770c, and 770d are connected in an array. Each memory chip includes a processor subunit and a dedicated memory bank, similar to the memory chip described above. Accordingly, descriptions of these components are not repeated herein.

In the example of FIG. 7C , the memory chips 770a, 770b, 770c, and 770d are connected in a loop. Accordingly, the bus 750a connects the memory chips 770a and 770d, the bus 750c connects the memory chips 770a and 770b, and the bus 750e connects the memory chips 770b and 770c. , the bus 750g connects the memory chips 770c and 770d. The memory chips 770a, 770b, 770c, and 770d may be connected in full tile connections, partial tile connections, or other connection configurations, although the example of FIG. 7C has fewer pins between memory chips 770a, 770b, 770c, 770d. make the connection possible.

relatively large memory

Embodiments of the present disclosure may use a dedicated memory having a relatively large size compared to a shared memory of an existing processor. By using dedicated memory rather than shared memory, work can proceed without reducing efficiency due to increased memory. As a result, memory-intensive tasks such as neural network processing and database queries can be performed more efficiently than those performed in a conventional processor in which efficiency improvement due to an increase in shared memory is reduced due to a von Neumann bottleneck.

For example, in the distributed processor of the present disclosure, a memory array disposed on a substrate of the distributed processor may include a plurality of discrete memory banks. Each discrete memory bank may include a processing array disposed on a substrate and having a capacity of at least one megabyte and including a plurality of processor subunits. As described above, each processor subunit of the processor subunit may be associated with a corresponding dedicated discrete memory bank among a plurality of discrete memory banks. In some embodiments, the plurality of processor subunits may be spatially distributed among a plurality of discrete memory banks within the memory array. By using at least 1 megabyte of dedicated memory rather than using a few megabytes of shared cache for large CPUs or GPUs, the distributed processor of the present disclosure is not possible in conventional systems due to von Neumann bottlenecks of CPU and GPU. gain efficiency.

Different memories may be used as dedicated memories. For example, each dedicated memory bank may include at least one DRAM bank. Alternatively, each dedicated memory bank may include at least one SRAM bank. In other embodiments, different types of memory may be merged on a single hardware chip.

As discussed above, each dedicated memory can be at least 1 megabyte. Accordingly, each dedicated memory bank may have the same size, or at least two memory banks of the plurality of memory banks may have different sizes.

Further, as described above, the distributed processor includes a plurality of first plurality of buses each connecting one processor subunit of the plurality of processor subunits to a corresponding dedicated memory bank and one processor subunit of each of the plurality of processor subunits. and a second plurality of buses for connecting to other processor subunits of the processor subunits.

Synchronization with software

As described above, the hardware chip of the present disclosure may manage data transmission by utilizing software rather than hardware. In particular, since the timing of transfers on the bus, reading and writing of memory, and computation of the processor subunit are set by the subseries of instructions executed by the processor subunit, the hardware chip of the present disclosure executes code on the bus collision can be avoided. Accordingly, the hardware chip of the present disclosure includes a hardware mechanism (eg, a network controller in the chip, a packet parser and packet transferor between processor subunits), a bus arbiter, and an arbitration conventionally used for management of data transmission. multiple buses to avoid) can be avoided.

If the hardware chip of the present disclosure transmits data in a conventional manner, then extensive MUX or bus arbitration controlled by an arbiter would be required to connect the N processor subunits to the bus. On the other hand, as described above, the embodiment of the present disclosure may use a bus that is only a line, an optical cable, etc. between the processor subunits, and the processor subunits may execute codes individually to prevent collisions on the bus. Accordingly, embodiments of the present disclosure can conserve space on the substrate, as well as reduce material cost and efficiency degradation (due to power and time consumption by intervention). Compared to other architectures that use first-in-first-out (FIFO) controls and/or mailboxes, the efficiency and space advantages are greater.

Also, as discussed above, each processor subunit may include one or more accelerators in addition to one or more processing elements. In some embodiments, the accelerator rather than the processing element may read and write to the bus. In such embodiments, additional efficiencies may be ensured by having the accelerator transmit data during the same cycle as the processing element performs one or more calculations. However, this embodiment requires additional material for the accelerator. For example, transistors may be additionally required for the manufacture of accelerators.

The code may also address internal operations, including timing and delay, of the processor subunit (eg, processing elements and/or accelerators forming part of the processor subunit). For example, a compiler (discussed below) can perform pre-processing to cope with timing and delays when generating a subseries of instructions that control data transfer.

In one example, the plurality of processor subunits may be assigned the task of calculating a neural network layer comprising a plurality of neurons fully connected to a greater plurality of neurons of a previous layer. Assuming that the data of the previous layer is evenly spread among the plurality of processor subunits, one way to perform the above calculation is to configure each processor subunit to transmit the data of the previous layer to the main bus in turn, Each processor subunit will then multiply this data by the weight of the corresponding neuron implemented by the processor subunit. Since each processor subunit computes one or more neurons, each processor subunit will transmit data from the previous layer as many as the number of neurons. Accordingly, the code of each processor subunit is not the same as the code of the other processor subunits, since the processor subunits will transmit at different times.

In some embodiments, the distributed processor is disposed on a substrate (eg, a semiconductor substrate such as silicon and/or a circuit substrate such as a flexible circuit board) on which a memory array including a plurality of discrete memory banks is disposed and is shown in FIG. 7A . It may include a processing array including a plurality of processor subunits such as shown in FIG. 7B or the like. As described above, each processor subunit may be associated with a corresponding dedicated discrete memory bank among a plurality of discrete memory banks. In addition, as illustrated in FIGS. 7A and 7B , the distributed processor may further include a plurality of buses each connecting one of the plurality of processor subunits to at least another one of the plurality of processor subunits.

As described above, the plurality of buses can be controlled by software. Accordingly, the plurality of buses may lack timing hardware logic elements such that data transfers between processor subunits and over corresponding buses of the plurality of buses may not be controlled by the timing hardware logic elements. In one example, the plurality of buses may not have bus arbiters such that data transfers between processor subunits and over corresponding buses of the plurality of buses may not be controlled by the bus arbiters.

In some embodiments, as shown in FIGS. 7A and 7B , etc., the distributed processor may further include a second plurality of buses coupling the plurality of processor subunits to corresponding dedicated memory banks. Similar to the plurality of buses described above, the second plurality of buses may lack timing hardware logic elements such that data transfers between the processor subunits and corresponding dedicated memory banks may not be controlled by the timing hardware logic elements. In one example, the second plurality of buses may not have a bus arbiter such that data transfer between the processor subunit and the corresponding dedicated memory bank may not be controlled by the bus arbiter.

In the present disclosure, the expression 'no' does not necessarily mean that timing hardware logic elements (eg, bus arbiters, arbitration structures, FIFO controllers, mailboxes, etc.) are absolutely absent. These elements may still be included in hardware chips that are described as 'free of these elements'. Rather, the expression 'no' refers to the function of the hardware chip. That is, a hardware chip 'without' a timing hardware logic element controls the timing of data transfers of the hardware chip without using a timing hardware logic element. For example, although the hardware chip includes timing hardware logic elements as a secondary precaution for protection from crashes due to errors in the executable code, the hardware chip provides subunits of instructions that control data transfer between processor subunits of the hardware chip. Execute the code containing the series.

As described above, the plurality of buses may include at least one of lines and optical fibers between corresponding processor subunits of the plurality of processor subunits. Thus, in one example, a distributed processor without timing hardware logic elements may include only line or fiber optics without bus arbiters, arbitration structures, FIFO controllers, mailboxes, and the like.

In some embodiments, the plurality of processor subunits are configured to transmit data over at least one of the plurality of buses in accordance with code executed by the plurality of processor subunits. Accordingly, as described above, the compiler can organize a subseries of instructions, each containing code executed by a single processor subunit. The subseries of instructions can instruct the processor subunit when to send data to one of the buses and when to get data from the bus. When subseries are executed cooperatively across distributed processors, the timing of transfers between processor subunits may be governed by the transmit and retrieve instructions contained in the subseries. Accordingly, the code controls the timing of data transfers over at least one of the plurality of buses. A compiler may generate code to be executed by a single processor subunit. In addition, the compiler may generate code to be executed by a group of processor subunits. In some cases, the compiler may treat all processor subunits as if they were one superprocessor (eg, a distributed processor), and the compiler could generate code to be executed by the superprocessor/distributed processor so defined.

As described above and illustrated in FIGS. 7A and 7B , a plurality of processor subunits may be spatially distributed among a plurality of discrete memory banks in a memory array. Alternatively, the plurality of processor subunits may be clustered in one or more regions of the substrate, and the plurality of memory banks may be clustered in one or more different regions of the substrate. In some embodiments, as described above, a combination of spatial distribution and clustering may be used.

In some embodiments, the distributed processor may include a substrate (eg, a semiconductor substrate such as silicon and/or a circuit substrate such as a flexible circuit board) on which a memory array including a plurality of discrete memory banks is disposed. A processing array may also include a plurality of processor subunits disposed on the substrate, such as those shown in FIGS. 7A and 7B and the like. As described above, each processor subunit may be associated with a corresponding dedicated discrete memory bank among a plurality of discrete memory banks. Also, as shown in FIGS. 7A and 7B , the distributed processor may further include a plurality of buses each coupling one of the plurality of processor subunits to a corresponding dedicated discrete memory bank of the plurality of discrete memory banks.

As described above, the plurality of buses can be controlled by software. Accordingly, the plurality of buses lack timing hardware logic elements such that data transfer between the processor subunit and corresponding dedicated discrete memory banks of the plurality of discrete memory banks and data transfers over corresponding buses of the plurality of buses are not implemented with timing hardware logic elements. may not be controlled by In one example, the plurality of buses may not have bus arbiters such that data transfers between processor subunits and over corresponding buses of the plurality of buses may not be controlled by the bus arbiters.

In some embodiments, as shown in FIGS. 7A and 7B , etc., the distributed processor may further include a second plurality of buses connecting one of the plurality of processor subunits to at least another one of the plurality of processor subunits. . Similar to the plurality of buses described above, the second plurality of buses may lack timing hardware logic elements such that data transfers between the processor subunits and corresponding dedicated memory banks may not be controlled by the timing hardware logic elements. In one example, the second plurality of buses may not have a bus arbiter such that data transfer between the processor subunit and the corresponding dedicated memory bank may not be controlled by the bus arbiter.

In some embodiments, a distributed processor may use a combination of software timing elements and hardware timing elements. For example, the distributed processor may include a substrate (eg, a semiconductor substrate such as silicon and/or a circuit substrate such as a flexible circuit board) on which a memory array including a plurality of discrete memory banks is disposed. A processing array may also include a plurality of processor subunits disposed on the substrate, such as those shown in FIGS. 7A and 7B and the like. As described above, each processor subunit may be associated with a corresponding dedicated discrete memory bank among a plurality of discrete memory banks. In addition, as illustrated in FIGS. 7A and 7B , the distributed processor may further include a plurality of buses each connecting one of the plurality of processor subunits to at least another one of the plurality of processor subunits. Also, as described above, the plurality of processor subunits may be configured to execute software that controls the timing of data transfers over the plurality of buses to avoid conflicting data transfers on at least one of the plurality of buses. In this example, software may control the timing of the data transfer, but the transfer itself may be controlled, at least in part, by one or more hardware elements.

In such embodiments, the distributed processor may further include a second plurality of buses coupling one of the plurality of processor subunits to a corresponding dedicated memory bank. Similar to the plurality of buses described above, the plurality of processor subunits are configured to execute software that controls the timing of data transfers over the second plurality of buses to avoid conflicting data transfers on at least one of the second plurality of buses. can be configured. In this example, as described above, software may control the timing of the data transfer, but the transfer itself may be controlled, at least in part, by one or more hardware elements.

splitting the code

As described above, the hardware chip of the present disclosure may execute codes in parallel in all of the processor subunits included on the substrate forming the hardware chip. In addition, the hardware chip of the present disclosure may perform multitasking. For example, the hardware chip of the present disclosure may perform area multitasking. That is, one group of processor subunits of the hardware chip may execute one task (eg, audio processing), and another group of processor subunits of the hardware chip may execute another task (eg, image processing). In another example, a hardware chip of the present disclosure may perform timing multitasking. That is, the one or more processor subunits of the hardware chip may execute one task during the first period of time and execute another task during the second period of time. A combination of area multitasking and timing multitasking is also used so that during a first period of time one task is assigned to a processor subunit of a first group and a task is assigned to a processor subunit of a second group during a first period of time after another task is assigned to a processor subunit of a second group. A third task may be assigned to the processor subunits included in the first group and the second group during the two time period.

To organize the machine code for execution on a memory chip of the present disclosure, the machine code may be partitioned among processor subunits of the memory chip. For example, a processor on a memory chip may include a substrate and a plurality of processor subunits disposed on the substrate. The memory chip may further include a corresponding plurality of memory banks disposed on the substrate, wherein each of the plurality of processor subunits has at least one dedicated memory that is not shared with any processor subunit of the plurality of processor subunits. It can be connected to a bank. Each processor subunit on the memory chip may be configured to execute a series of instructions independently of other processor subunits. Each set of instructions sets one or more general processing elements of the processor subunit according to the code defining the set of instructions and/or sets one or more special processing elements of the processor subunit according to a sequence provided in the code defining the series of instructions ( eg, by activating one or more accelerators).

Accordingly, each set of instructions may define a set of tasks to be performed by a single processor subunit. A single task may include instructions within an instruction set defined by the architecture of one or more processing elements within a processor subunit. For example, a processor subunit may contain specific registers, and a single operation may push data into a register, pull data from a register, perform arithmetic operations on data within a register, and register It is possible to perform logical operations on data within the range. In addition, the processor subunit is a 0-operand processor subunit (also referred to as a 'stack machine'), a one-operand processor subunit (also referred to as an accumulator machine), a two-operand It may be configured for any number of operands, such as a processor subunit (such as RISC), a three-operator processor subunit (such as a complex instruction set computer (CISC)), or the like. In another example, the processor subunit may include one or more accelerators, and a single task may activate the accelerators to perform specific functions, such as MAC functions, MAX functions, MAX-0 functions, and the like.

The series of commands may further include operations for reading and writing from a dedicated memory bank of the memory chip. For example, the operation may include writing one data to a memory bank dedicated to the processor subunit executing the operation, reading one data from a memory bank dedicated to the processor subunit executing the operation, and the like. In some embodiments, reads and writes may be performed by the processor subunit in cooperation with the controller of the memory bank. For example, the processor subunit may transmit a control signal to perform a read or write to the controller to execute a read or write operation. In some embodiments, the control signal may include a specific address to use for reading and writing. Alternatively, the processor subunit may leave the memory controller to select an address that can be used for reading and writing.

Additionally or alternatively, reads and writes may be performed by one or more accelerators in cooperation with the controller of the memory bank. For example, the accelerator may generate a control signal for the memory controller similarly to the processor subunit described above for generating the control signal.

In the above-described embodiment, the address generator may be used to instruct reads and writes to specific addresses in the memory bank. For example, the address generator may include a processing element configured to generate a memory address for reading and writing. The address generator may be configured to generate addresses to improve efficiency, such as by writing the results of later calculations to the same address as those of previous calculations that are no longer needed. Accordingly, the address generator may generate control signals for the memory controller in response to or in cooperation with the processor subunit in response to instructions from the processor subunit (eg, from one or more accelerators or processing elements included in the processor subunit). have. Additionally or alternatively, the address generator may generate an address based on some setting or register, for example, creating a nested loop structure that repeats for a particular address in memory in a particular pattern.

In some embodiments, each set of instructions may include a set of machine code defining a corresponding set of actions. Accordingly, the set of operations can be compressed into machine code comprising a set of instructions. In some embodiments, as described below with reference to FIG. 8 , the set of tasks is defined by a compiler configured to distribute a higher-level set of tasks among the plurality of logic circuits into a plurality of sets of tasks. can be For example, a compiler may create a plurality of sets of tasks based on a high-level set of tasks, so that a processor subunit, each executing a corresponding set of tasks together, may perform a function as described by the high-level set of tasks. can

As described below, a high-level set of tasks may include a set of instructions in a human-readable programming language. Correspondingly, the set of tasks for each processor subunit may include a lower-level set of tasks, each comprising a set of instructions in machine code.

As described above with reference to FIGS. 7A and 7B , the memory chip may further include a plurality of buses respectively connecting one of the plurality of processor subunits to at least another one of the plurality of processor subunits. Also, as described above, data transfers on multiple buses can be controlled using software. Accordingly, data transmission over at least one of the plurality of buses may be predetermined by a series of instructions included in a processor subunit coupled to at least one of the plurality of buses. Thus, one of the tasks included in the set of commands may include outputting data to one of the buses or retrieving data from one of the buses. These tasks may be executed by processing elements of the processor subunit or one or more accelerators included in the processor subunit. In the latter embodiment, the processor subunit may perform calculations or send control signals to the corresponding memory bank in a cycle, such as a cycle in which the accelerator fetches data from or places data on one of the buses.

In one example, the series of instructions included in a processor subunit coupled to at least one of the plurality of buses performs a transfer operation comprising instructions for causing the processor subunit coupled to at least one of the plurality of buses to write data to at least one of the plurality of buses. may include Additionally or alternatively, the series of instructions included in the processor subunit coupled to at least one of the plurality of buses includes instructions to cause the processor subunit coupled to at least one of the plurality of buses to read data from at least one of the plurality of buses. It may include a receiving operation to

Additionally or alternatively to distributing code between processor subunits, data may be partitioned between memory banks of memory chips. For example, as described above, the distributed processor on the memory chip may include a plurality of processor subunits disposed on the memory chip and a plurality of memory banks disposed on the memory chip. Each memory bank of the plurality of memory banks may be configured to store data separate from data stored in other memory banks of the plurality of memory banks, wherein each processor subunit of the plurality of processor subunits comprises at least one of the plurality of memory banks. It can be connected to a dedicated memory bank of For example, each processor subunit may access one or more memory controllers of one or more corresponding memory banks dedicated to that processor subunit, and no other processor subunit may access such corresponding one or more memory controllers. can Accordingly, since there is no memory controller that can be shared between the memory banks, the data stored in each memory bank may be separate from the memory stored in the other memory banks.

In some embodiments, as described below with reference to FIG. 8 , data stored in each of the plurality of memory banks may be defined by a compiler configured to distribute the data among the plurality of memory banks. Further, the compiler may be configured to distribute data defined in a high-level series of tasks across a plurality of memory banks using a plurality of low-level tasks distributed to corresponding processor subunits.

As described further below, a high-level set of tasks may include a set of instructions in a human-readable programming language. Correspondingly, the sequence of operations for each processor subunit may include a low-level sequence of operations each comprising a set of instructions in machine code.

As previously described with reference to FIGS. 7A and 7B , the memory chip may further include a plurality of buses each coupling one of the plurality of processor subunits to a corresponding dedicated memory bank of one or more of the plurality of memory banks. Also, as described above, data transfer on multiple buses can be controlled utilizing software. Accordingly, data transfer via a specific one of the plurality of buses may be controlled by a corresponding processor subunit connected to the specific one of the plurality of buses. Thus, one of the tasks included in the sequence of commands may include outputting data to one of the buses or retrieving data from one of the buses. As described above, these tasks may be executed by (i) processing elements in the processor subunit or (ii) one or more accelerators included in the processor subunit. In the latter embodiment, the processor subunit is configured in the same cycle in which the accelerator fetches data from one of the buses coupled to one or more corresponding dedicated memory banks or places data onto one of the buses coupled to one or more corresponding dedicated memory banks. A bus may be used to perform calculations or to connect processor subunits to other processor subunits.

Thus, in one example, a series of instructions included in a processor subunit coupled to at least one of the plurality of buses may include a transmit operation. The transfer operation may include instructions that cause a processor subunit coupled to at least one of the plurality of buses to write data to at least one of the plurality of buses to be stored in one or more corresponding dedicated memory banks. Additionally or alternatively, the series of instructions included in the processor subunit coupled to at least one of the plurality of buses may include a receive operation. The receive operation may include instructions to cause a processor subunit coupled to at least one of the plurality of buses to read from at least one of the plurality of buses data to be stored in one or more corresponding dedicated memory banks. Accordingly, the transmit and receive operations of these embodiments may include control signals transmitted along at least one of the plurality of buses to one or more memory controllers of one or more corresponding dedicated memory banks. In addition, transmitting and receiving tasks may be performed by one portion of a processor subunit (e.g., one or more other accelerators in a processor subunit) concurrently with computations or other operations performed by other portions of the processor subunit (eg, one or more other accelerators in a processor subunit). can be executed by An example of such concurrent execution may include MAC-relay instructions in which receive, multiply, and transmit are executed together.

In addition to distributing data between memory banks, specific portions of data may be duplicated in different memory banks. For example, as described above, the distributed processor on the memory chip may include a plurality of processor subunits disposed on the memory chip and a plurality of memory banks disposed on the memory chip. Each of the plurality of processor subunits may be coupled to at least one dedicated memory bank of the plurality of memory banks, each memory bank of the plurality of memory banks storing data separate from data stored in other memory banks of the plurality of memory banks can be configured to Also, at least a portion of the data stored in one specific memory bank of the plurality of memory banks may include a copy of the data stored in at least another memory bank of the plurality of memory banks. For example, numbers, strings, or other types of data used in a series of instructions may be stored in multiple memory banks dedicated to different processor subunits rather than being transferred from one memory bank to another processor subunit on a memory chip. can

In one example, parallel string matching may utilize data replication described above. For example, a plurality of character strings may be compared to the same character string. The conventional processor will sequentially compare each string of a plurality of strings with the same string. In the hardware chip of the present disclosure, the same character string may be duplicated throughout the memory bank, allowing the processor subunit to parallelly compare individual character strings of a plurality of character strings with the duplicated character string.

In some embodiments, as described below with reference to FIG. 8 , some data copied to a particular memory bank of one of the plurality of memory banks and at least another one of the plurality of memory banks is configured to duplicate data in the memory bank. Defined by the compiler. Further, the compiler may be configured to replicate at least some data using a plurality of low-level tasks distributed among corresponding processor subunits.

Replicating data can be useful for reusing the same part of data for other computations. By duplicating this portion of data, these calculations can be distributed and executed in parallel across processor subunits in a memory bank, while each processor subunit (not pushing and pulling this portion of data through the bus connecting the processor subunits) ) can store this portion of data in a dedicated memory bank and access the stored portion in the dedicated memory bank. In one example, some data copied to one specific memory bank of the plurality of memory banks and at least one other memory bank of the plurality of memory banks may include weights of the neural network. In this example, each node of the neural network may be defined by at least one processor subunit of a plurality of processor subunits. For example, each node may include machine code executed by at least one processor subunit defining the node. In this example, duplicating the weights allows each processor subunit to execute machine code while accessing only one or more dedicated memory banks (without performing data transfers with other processor subunits) to act at least in part on the corresponding node. . Since the timing of reads and writes to the dedicated memory is independent of other processor subunits, data transfer between processor subunits requires timing synchronization (using software as described previously), thus replicating memory between processor subunits. If you don't have to transfer the data, the overall execution can be more efficient.

As previously described with reference to FIGS. 7A and 7B , the memory chip may include a plurality of buses each coupling one of the plurality of processor subunits to one or more corresponding dedicated memory banks of the plurality of memory banks. Also, as described above, data transfer on multiple buses can be controlled using software. Thereby, data transfer over a specific bus of a plurality of buses can be controlled by a corresponding processor subunit connected to a specific bus of the plurality of buses. Thus, one of the tasks included in the sequence of commands may include outputting data to one of the buses or fetching data from one of the buses. As described above, these tasks may be executed by (i) processing elements in the processor subunit or (ii) one or more accelerators included in the processor subunit. As further described above, such operations may include transmit operations and/or receive operations comprising control signals transmitted along at least one of the plurality of buses to one or more memory controllers of one or more corresponding dedicated memory banks.

8 shows a flow diagram of a method 800 of compiling a sequence of instructions for execution on an exemplary memory chip of the present disclosure, such as those shown in FIGS. 7A and 7B . Method 800 may be implemented by a general purpose or dedicated, existing processor.

Method 800 may be executed as part of a computer program forming a compiler. In the present disclosure, a 'compiler' refers to a low-level language (eg, assembly code, object code, Any computer program that translates into machine code, etc.). Compilers can enable humans to program a set of instructions in a human readable language, which are later translated into a machine executable language.

In step 810 , the processor may assign tasks associated with a set of instructions to different subunits of the processor subunit. For example, a series of instructions may be divided into small groups to be executed in parallel in processor subunits. In one example, a neural network may be partitioned into nodes, and one or more nodes may be assigned to different processor subunits. In this example, each subgroup may include a plurality of nodes connected by different hierarchies. Accordingly, the processor subunit may implement a node from a first layer of the neural network, a node from a second layer connected to a node from the first layer implemented by the same processor subunit, and the like. By allocating nodes based on each connection, data transfer between processor subunits can be reduced, and as a result, efficiency can be improved as described above.

As described above with reference to FIGS. 7A and 7B , the processor subunits may be spatially distributed in a plurality of memory banks disposed on the memory chip. Accordingly, the assignment of tasks can be partitioned not only logically but also at least partially spatially.

In step 820 , the processor may create a task to transfer data between pairs of processor subunits of memory chips that are each connected by a bus. For example, as described above, data transfer may be controlled utilizing software. Accordingly, the processor subunit may be configured to push and pull data on the bus at synchronized times. Thus, the created jobs may include jobs to perform these synchronized pushes and pulls of data.

As described above, step 820 may include preprocessing to cope with internal operations including timing and delay of the processor subunit. For example, the processor may determine the known time and delay of the processor subunits (e.g., the time it takes to push data to and from the bus, the time it takes to pull data from the bus, and the time it takes to delay, etc.) can be used. Accordingly, a data transfer comprising at least one push by one or more processor subunits and at least one pull by one or more processor subunits does not incur delays due to timing differences between processor subunits, delays in processor subunits, etc. and can occur at the same time.

In step 830, the processor may classify the assigned and generated tasks into a plurality of groups of subseries instructions. For example, a subseries instruction may include a series of tasks each to be executed by a single processor subunit. Accordingly, each of the plurality of groups of subseries instructions may correspond to a different subunit of the plurality of processor subunits. Accordingly, as a result of steps 810, 820, and 830, a series of instructions may be divided into a plurality of groups of subseries instructions. As described above, step 820 may cause all data transmissions between different groups to be synchronized.

In step 840 , the processor may generate machine code corresponding to each of the plurality of groups of subseries instructions. For example, high-level code representing subseries instructions may be converted into low-level code, such as machine code, executable by a corresponding processor subunit.

In step 850, the processor may assign the generated machine code corresponding to each of the subseries instructions of the plurality of groups to the corresponding processor subunits of the plurality of processor subunits according to the partitioning. For example, the processor may attach an identifier of the processor subunit corresponding to each subseries instruction. Thus, when subseries commands are uploaded to the memory chip for execution (eg, by host 350 in FIG. 3A ), each subseries may set up the appropriate processor subunit.

In some embodiments, assigning tasks associated with a set of instructions to different processor subunits of a processor subunit may depend, at least in part, on spatial proximity between two or more processor subunits on a memory chip. For example, as described above, reducing the number of data transfers between processor subunits can improve efficiency. Accordingly, the processor can minimize data transfers that move data through two or more processor subunits. Thus, the processor uses one known layout of memory chips to allocate subseries to processor subunits in a way that maximizes contiguous transfers (at least locally) and minimizes transfers to non-neighboring processor subunits (at least locally). It can be used by merging with the above optimization algorithms (eg, greedy algorithm).

Method 800 may include further optimizations to the memory chip of the present disclosure. For example, the processor may classify data associated with a set of instructions based on the segmentation and allocate the data to a memory bank according to the segmentation. Accordingly, the memory banks may hold data used for subseries instructions assigned to each processor subunit that is dedicated to each memory bank.

In some embodiments, categorizing the data may include determining at least a portion of the data to be copied to two or more memory banks. For example, as described above, some data may be used in one or more subseries instructions. This data may be copied to memory banks dedicated to a plurality of processor subunits assigned different subseries instructions. Such optimization may further reduce data transfer between processor subunits.

The output of the method 800 may be an input to a memory chip of the present disclosure for execution. For example, the memory chip may include a plurality of processor subunits each coupled to at least one dedicated memory bank and a corresponding plurality of memory banks, wherein the processor subunits of the memory chip are generated by the method 800 . It may be configured to execute machine code. As described with reference to FIG. 3A , the host 350 may input and execute the machine code generated by the method 800 into the processor subunit.

Subbanks and subcontrollers

In conventional memory banks, the controller is provided at the bank level. Each bank includes a plurality of mats that are typically arranged in a rectangular shape, but may be arranged in any other geometry. Each mat also contains a plurality of memory cells that are typically arranged in a rectangular shape, but may be arranged in any other geometry. Each cell can store a single bit of data (depending on whether the cell is maintained at high or low voltage, etc.).

Examples of such a conventional architecture are shown in FIGS. 9 and 10 . 9 , at the bank level, a plurality of mats (eg, mats 930-1, 930-2, 940-1, 940-2) may form a bank 900 . In the existing rectangular structure, the bank 900 may be controlled by a global wordline (eg, wordline 950 ) and a global bitline (eg, bitline 960 ). Accordingly, the row decoder 910 may select the correct word line based on an input control signal (eg, a read request from an address, a write request from an address, etc.), and a global sense amplifier, 920 (and/or a global column decoder not shown in FIG. 9 ) may select the correct bit line based on the control signal. Amplifier 920 may also amplify the voltage level in the selected bank during a read operation. Although the figure is shown using a row decoder for initial selection and performing the amplification along the columns, the bank may additionally or alternatively use a column decoder to make the initial selection and perform the amplification along the rows. can

10 shows an example of the mat 1000 . For example, mat 1000 may form part of a memory bank, such as bank 900 of FIG. 9 . As shown in FIG. 10 , a plurality of cells (eg, 1030 - 1 , 1030 - 2 , and 1030 - 3 ) may form the mat 1000 . Each cell may contain capacitors, transistors, or other circuitry that stores at least one bit of data. For example, a cell may include a capacitor that is charged to represent a '1' and discharged to represent a '0', or a flip-flop comprising a first state representing a '1' and a second state representing a '0'. ) may be included. A conventional mat may include, for example, 512 bits x 512 bits. In embodiments where the mat 1000 forms part of an MRAM, ReRAM, or the like, the cell may include a transistor, resistor, capacitor, or other mechanism that separates ions or parts of a material that stores at least one bit of data. . For example, the cell may include electrolyte ions, chalcogenide glass, etc., in which the first state represents '1' and the second state represents '0'.

As further shown in FIG. 10 , in a conventional rectangular structure, the mat 1000 may be controlled across a local wordline (eg, wordline 1040 ) and a local bitline (eg, bitline 1050 ). have. Accordingly, a wordline driver (eg, wordline drivers 1020-1, 1020-2, . Reading, writing, or refresh may be performed based on a control signal (eg, a read request from an address, a write request from an address, a refresh signal, etc.) from a related controller. In addition, a local sense amplifier (eg, local amplifiers 1010-1, 1010-2, . . . , 1010-x) and/or a local column decoder (not shown in FIG. 10) controls the selected bit line to read; Write or refresh can be performed. The local sense amplifier may also amplify the voltage level of the selected cell during a read operation. Although the figure is shown using wordline drivers for initial selection and performing amplification along the columns, Matt may use bitline drivers for initial selection and performing amplification along the rows.

As described above, a large number of mats are duplicated to form memory banks. Memory banks may be grouped together to form memory chips. For example, a memory chip may include 8 to 32 memory banks. Accordingly, only 8 to 32 processor subunits are formed as a result of pairing the processor subunit and the memory bank on the existing memory chip. Accordingly, an embodiment of the present disclosure may include a memory chip having an additional sub-bank scheme. Such a memory chip of the present disclosure may thus include a processor subunit having a memory subbank used as a dedicated memory bank paired with the processor subunit. Accordingly, a larger number of sub-processors may be used, and parallel execution and performance of in-memory computing may be improved.

In some embodiments of the present disclosure, the global row decoder and global sense amplifier of the bank 900 may be replaced by a sub-bank controller. Accordingly, the controller of the memory bank may send the control signal to the appropriate subbank controller without sending the control signal to the global row decoder and global sense amplifier of the memory bank. The direction in which the control signals are sent can be dynamically controlled or built into hardware (eg, via one or more logic gates). In some embodiments, fuses may be used to instruct the controller of each subbank or mat to pass or block control signals to that subbank or mat. Thus, in such an embodiment, a bad subbank may be deactivated using a fuse.

In one example of such an embodiment, the memory chip may include a plurality of memory banks, each memory bank including a bank controller and a plurality of memory subbanks, each memory subbank being read into a respective location on the memory subbank It may include a sub-bank row decoder and a sub-bank column decoder to enable writing. Each subbank may include a plurality of memory mats, and each memory mat may include a plurality of memory cells and may include internally local row decoders, column decoders, and/or local sense amplifiers. The subbank row decoder and the subbank column decoder may process read and write requests from the bank controller or read and write requests from the subbank processor subunit used for in-memory calculation on the subbank memory to be described below. Additionally, each memory subbank will handle read requests and write requests from the bank controller and/or whether to forward these requests to the next level (e.g. the level of the row decoder and column decoder on the mat) or to block these requests; For example, it may further include a controller configured to determine whether to allow the internal processing element or processor subunit to access the memory. In some embodiments, the bank controller may be synchronized to a system clock. However, the subbank controller may not be synchronized to the system clock.

As described above, when the sub-bank is used, a larger number of processor sub-units can be included in the memory chip than when the processor sub-unit is paired with the memory bank of the existing chip. Accordingly, each subbank may further include a processor subunit using the subbank as a dedicated memory. As discussed above, a processor subunit may include a RISC, CISC, or other general purpose processor subunit and/or may include one or more accelerators. In addition, the processor subunit may include an address generator as described above. In all the above-described embodiments, each processor subunit may be configured to access a subbank dedicated to the processor subunit by using the row decoder and column decoder of the subbank without using the bank controller. The processor subunits associated with the sub-banks also handle memory mats (including the decoder and memory redundancy mechanisms described below) and/or higher-level (eg, bank-level or memory-level) read and write requests in response to transfer and It can be determined whether or not

In some embodiments, the sub-bank controller may further include a register for storing the state of the sub-bank. Accordingly, when the subbank controller receives a control signal from the memory controller while the register indicates that the subbank is in use, the subbank controller may output an error. In an embodiment where each subbank further includes a processor subunit, if the processor subunit of the subbank accesses the memory in conflict with an external request from the memory controller, the register may indicate an error.

11 shows an example of another embodiment of a memory bank using a subbank controller. In the example of FIG. 11 , bank 1100 has a plurality of memory subbanks (eg, subbanks) having a row decoder 1110 , a bank decoder 1120 , and a subbank controller (eg, controllers 1130a , 1130b , 1130c ). banks 1170a, 1170b, 1170c). The subbank controller may include an address resolver (eg, resolvers 1140a, 1140b, 1140c) capable of determining whether to transmit a request to one or more subbanks under the control of the subbank controller.

The sub-bank controller may further include one or more logic circuits (eg, circuits 1150a, 1150b, 1150c). For example, a logic circuit including one or more processing elements may be configured such that one or more operations, such as refreshing the cells of the subbank, emptying the cells of the subbank, etc., are performed outside of the bank 1100 without processing the request. can do. Alternatively, the logic circuit may cause all subbanks controlled by the subbank controller, including the processor subunits as described above, to be the corresponding dedicated memories of the processor subunits. 11, the corresponding dedicated memory of logic 1150a is subbank 1170a, the corresponding dedicated memory of logic 1150b is subbank 1170b, and the corresponding dedicated memory of logic 1150c is It is a subbank 1170c. In all of the previously described embodiments, the logic circuit has buses that connect to the subbanks, eg, buses 1131a, 1131b, 1131c. As shown in Figure 11, each of the subbank controllers has a subbank row decoder and a subbank that enable reads and writes to locations on the memory bank by a processing element or processor subunit, or by a high-level memory controller that issues commands. It may include a plurality of decoders such as a column decoder. For example, the sub-bank controller 1130a includes decoders 1160a, 1160b, and 1160c, the sub-bank controller 1130b includes decoders 1160d, 1160e, and 1160f, and the sub-bank controller 1130c is a decoder (1160g, 1160h, 1160i). At the request of the bank row decoder 111 , the sub-bank controller may select a word line using a decoder included in the sub-bank controller. The system described herein allows the processing elements or processor subunits of a subbank to access memory without interfering with other banks and even other subbanks, so that each subbank processor subunit can run parallel with other subbank subunits. can be used for memory calculations.

Also, each subbank may include a plurality of memory mats each including a plurality of memory cells. For example, subbank 1170a includes mats 1190a-1, 1190a-2, ..., 1190a-x, and subbank 1170b includes mats 1190b-1, 1190b-2, . . . , 1190b-x), and subbank 1170c includes mats 1190c-1, 1190c-2, . . ., 1190c-x. Also, as shown in FIG. 11 , each subbank may include at least one decoder. For example, subbank 1170a includes a decoder 1180a , subbank 1170b includes a decoder 1180b , and subbank 1170c includes a decoder 1180c . Accordingly, the bank column decoder 1120 may select a global bit line (eg, 1121a or 1121b) based on an external request, while the subbank selected by the bank row decoder 1110 utilizes the column decoder to select this sub A local bit line (eg, 1181a or 1181b) can be selected based on a local request of the logic circuit for which the bank is dedicated. Accordingly, each processor subunit may be configured to access the dedicated subbank of the processor subunit by utilizing the row decoder and column decoder of the subbank without using the bank row decoder and the bank column decoder. Thus, each processor subunit can access the corresponding subbank without interfering with the other subbanks. In addition, when the request to the sub-bank is made outside the processor sub-unit, the sub-bank decoder may reflect the accessed data to the bank decoder. Alternatively, in embodiments where there is only a single row of memory banks in each subbank, the local bitline may be the bitline of the mat rather than the bitline of the subbank.

A combination of the embodiment using the subbank row decoder and the subbank column decoder of the embodiment shown in FIG. 11 may also be used. For example, the bank row decoder can be removed but the bank column decoder can be retained and the local bitline can be used.

12 illustrates an example embodiment of a memory subbank 1200 including a plurality of mats. For example, the sub-bank 1200 may represent a part of the sub-bank 1100 of FIG. 11 or a memory bank implemented differently. In the example of FIG. 12 , subbank 1200 includes a plurality of mats (eg, 1240a, 1240b). Also, each mat may include a plurality of cells. For example, mat 1240a includes cells 1260a-1, 1260a-2, ..., 1260a-x, and mat 1240b includes cells 1260b-1, 1260b-2, ..., 1260a-x. 1260b-x).

Each mat may be assigned a range of addresses to be assigned to the mat's memory cells. These addresses may be set at manufacturing time so that mats can be moved around and defective mats are deactivated and not used (eg, with one or more fuses, as described below).

The subbank 1200 receives read and write requests from the memory controller 1210 . Although not shown in FIG. 12 , a request from the memory controller 1210 may be filtered through the controller of the subbank 1200 and sent to the appropriate mat of the subbank 1200 for address resolution. Alternatively, at least a portion (eg, the high bit) of the address of the request of the memory controller 1210 is sent to all the mats (eg, 1240a, 1240b) of the subbank 1200 so that the assigned address range of the mats is in the command. It is possible to process requests associated with an address and the entire address only if it contains a specified address. Similar to the subbank instructions above, the mat determination can be dynamically controlled or built into hardware. In some embodiments, a fuse may be used to determine the address range for each mat, and may also disable bad mats by assigning an illegal address range. The mat may additionally or alternatively be deactivated by other common methods or fuse connections.

In all of the above-described embodiments, each mat of a subbank may include a row decoder (eg, 1230a, 1230b) for selecting wordlines from the mat. In some embodiments, each mat may further include a fuse and a comparator (eg, 1220a, 1220b). As discussed above, a comparator may cause each mat to decide whether to process an incoming request, and a fuse may cause each mat to be deactivated if it is bad. Alternatively, the row decoders of banks and/or subbanks may be used in place of the row decoders of each mat.

In addition, in all the above-described embodiments, the column decoders (eg, 1250a and 1250b) included in the corresponding mat may select the local bitlines (eg, 1251 and 1253). The local bitline may be coupled to the global bitline of the memory bank. In embodiments where a subbank has its own local bitline, the cell's local bitline may further be coupled to the subbank's local bitline. Accordingly, the data of the selected cell passes through the column decoder (and/or sense amplifier) of the cell and then through the column decoder (and/or sense amplifier) of the subbank (including the subbank column decoder and/or sense amplifier). in an embodiment) can then be read through the bank's column decoder (and/or sense amplifier).

The mat 1200 may be duplicated and arranged in an array to form a memory bank (or memory subbank). For example, a memory chip of the present disclosure may include a plurality of memory banks, each memory bank having a plurality of memory subbanks, each memory subbank handling reads and writes to respective locations on the memory subbanks. There is a subbank controller for Further, each memory subbank may include a plurality of memory mats, each memory mat having a plurality of memory cells, a mat row decoder and a mat column decoder (shown in FIG. 12 ). The mat row decoder and mat column decoder can handle read and write requests from the subbank controller. For example, the mat decoder receives all requests and decides whether to process the requests based on each mat's known address range (e.g. utilizing a comparator), or based on mat selection by the subbank (or bank) controller. Thus, only requests within a known address range can be received.

Controller data transfer

The memory chip of the present disclosure may also utilize a memory controller (or a subbank controller or a mat controller) to share data in addition to sharing data by utilizing the processor subunit. For example, the memory chip of the present disclosure may include a plurality of memory banks (eg, SRAM bank, DRAM bank, etc.) each including a bank controller, a row decoder, and a column decoder to enable reading and writing to each location of the memory bank. and a plurality of buses connecting each controller of the plurality of bank controllers to at least one other controller of the plurality of bank controllers.

In some embodiments, the plurality of buses may be accessed without interfering with data transfers on the main bus of a memory bank coupled to one or more processor subunits. Accordingly, the memory bank (or subbank) may transmit or receive data to or from the corresponding processor subunit in the same clock cycle as data transmission or reception with another memory bank (or subbank). In embodiments where each controller is coupled to a plurality of other controllers, the controller may be configured to select one other controller of the plurality of other controllers for transmission or reception of data. In some embodiments, each controller may be coupled to at least one neighboring controller (eg, pairs of spatially adjacent controllers may be coupled to each other).

Redundant logic in memory circuits

The present disclosure relates to a memory chip having primary logic portions for on-chip data processing. The memory chip may include redundant logic portions capable of increasing the chip manufacturing yield by replacing the defective primary logic portion. Thus, the chip may include on-chip elements that enable the setting of the logic blocks of the memory chip based on individual tests of the logic unit. Since the memory chip is more susceptible to manufacturing errors as the area dedicated to the logic unit is larger, this feature of the chip can improve the yield. For example, DRAM memory chips with large redundant logic may be subject to manufacturing issues that reduce yield. However, implementing redundant logic can result in improved yield and reliability because it allows the producer or user of a DRAM memory chip to turn the entire logic on or off while maintaining a high degree of parallelism. In the present disclosure, an example of a specific memory type (eg, DRAM) may be given for convenience of description of the disclosed embodiment. Even so, it will be appreciated that the present disclosure is not limited to a particular memory type. Rather, memory types such as DRAM, flash memory, SRAM, ReRAM, PRAM, MRAM, ROM, or any other memory may be used with the disclosed embodiments, although few of the memory types exemplified in some portions of this disclosure.

13 is a block diagram of an exemplary memory chip 1300 according to the present disclosure. The memory chip 1300 may be implemented as a DRAM memory chip. Memory chip 1300 may also be implemented with any type of volatile or non-volatile memory, such as flash memory, SRAM, ReRAM, PRAM, and/or MRAM. The memory chip 1300 includes an address manager 1302 , a memory array 1304 including a plurality of memory banks 1304(a,a)-1304(z,z), memory logic 1306 , and business logic 1308 . ), and a substrate 13010 on which the redundant business logic 1310 is disposed. The memory logic 1306 and the business logic 1308 may constitute a primary logic block, while the redundant business logic 1310 may constitute a redundant block. Also, the memory chip 1300 may include a setting switch that may include a deactivation switch 1312 and an enable switch 1314 . A deactivation switch 1312 and an enable switch 1314 may also be disposed on the substrate 1301 . In this application, memory logic 1306 , business logic 1308 , and redundant business logic 1310 may also be collectively referred to as a 'logic block'.

Address manager 1302 may include row decoders and column decoders or other types of memory aids. Alternatively or additionally, address manager 1302 may include a microcontroller or processing unit.

In some embodiments, as shown in FIG. 13 , the memory chip 1300 may include a single memory array 1304 , in which a plurality of memory blocks may be disposed on a substrate 1301 in a two-dimensional array. However, in other embodiments, the memory chip 1300 may include multiple memory arrays 1304 , and each memory array 1304 may arrange memory blocks in different configurations. For example, memory blocks (ie, memory banks) of at least one of the memory arrays may be arranged in a radial distribution to enable routing from the address manager 1302 or memory logic 1306 to the memory blocks.

The business logic 1308 may be used for in-memory computation of applications that are independent of the logic used to manage the memory itself. For example, business logic 1308 may implement functions pertaining to AI, such as floating, integer, or MAC operations used as activation functions. In addition, the business logic 1308 may implement database-related functions such as min, max, sort, count, and the like. The memory logic 1306 may perform operations related to memory management including read, write, and refresh operations. Accordingly, business logic may be added to one or more of the group of bank level, mat level, and mat level. The business logic 1308 may have one or more address outputs and one or more data inputs/outputs. For example, business logic 1308 can address address manager 1302 by row/column line. However, in some embodiments, logic blocks may additionally or alternatively be addressed via data inputs/outputs.

Redundant business logic 1310 may be a duplicate of business logic 1308 . Additionally, redundant business logic 1310 may be coupled to a deactivation switch 1312 and/or an enable switch 1314 which may include a small fuse/anti-fuse, and may be coupled to one of the instances (eg, by default). It can be used for logic to deactivate or activate a connected instance) and activate one of the other logic blocks (eg, an instance that is disconnected by default). In some embodiments, as described below with reference to FIG. 15 , the redundancy of blocks may be confined within a logic block, such as business logic 1308 .

In some embodiments, the logic blocks of the memory chip 1300 may be coupled to a subset of the memory array 1304 by a dedicated bus. For example, the set of memory logic 1306 , business logic 1308 , and redundant business logic 1310 may include memory blocks of memory array 1304 (ie, 1304 (a,a) through 1304 (a,z) ) can be connected to the first row of A dedicated bus may allow the associated logic block to quickly access data from a memory block without the need to establish a communication line through an address manager 1302 or the like.

Each of the plurality of primary logic blocks may be coupled to at least one of the plurality of memory banks 1304 . Also, a redundant block, such as redundant business block 1310 , may be coupled to at least one of memory instances 1304(a,a)-(z,z). The redundant block may duplicate at least one of a plurality of primary logic blocks, such as memory logic 1306 or business logic 1308 . The deactivation switch 1312 may be connected to at least one of the plurality of primary logic blocks, and the enable switch 1314 may be connected to at least one of the plurality of redundant blocks.

In this embodiment, if a fault associated with one of the plurality of primary logic blocks (memory logic 1306 and/or business logic 1308) is detected, the deactivation switch 1312 disables one of the plurality of primary logic blocks. It can be configured to disable. At the same time, the activation switch 1314 may be configured to activate one of a plurality of redundant blocks, such as the redundant logic block 1310 , which duplicates one of the plurality of primary logic blocks.

In addition, the enable switch 1314 and the deactivation switch 1312, which may be collectively referred to as a 'setting switch', may include an external input for setting the state of the switch. For example, the activation switch 1314 may be set such that an activation signal of an external input causes a switch closed state, and the deactivation switch 1312 may be set such that an activation signal of an external input causes a switch open state. have. In some embodiments, all setting switches of the memory chip 1300 are disabled by default, and may be activated after the result of the check indicates that the relevant logic block is functioning and a signal is given to the external input. Alternatively, in some cases, all setting switches of the memory chip 1300 are in an activated state by default, and may be deactivated after the result of the check indicates that the relevant logic block is not functioning and a deactivation signal is given to an external input.

Regardless of whether the setup switch is initially enabled or disabled, if a fault associated with the associated logic block is detected, the setup switch may disable the associated logic block. If a configuration switch is initially activated, the state of the configuration switch may be changed to deactivated to deactivate the associated logic block. If the configuration switch is initially disabled, the state of the configuration switch may remain in the inactive state to disable the associated logic block. For example, an operability check may result in a specific logic block not working or not working within specific specifications. In this case, the logic block can be deactivated by not activating the corresponding setting switch.

In some embodiments, a setting switch may be coupled to more than one logic block and may be configured to select between different logic blocks. For example, a configuration switch may be coupled to both business logic 1308 and redundant logic block 1310 . A setup switch may activate the redundant logic block 1310 while deactivating the business logic 1308 .

Alternatively or additionally, at least one of the plurality of primary logic blocks (memory logic 1306 and/or business logic 1308 ) is connected to a subset of the plurality of memory banks or memory instances 1304 over a first dedicated connection. can be connected Thereafter, at least one of the plurality of redundant blocks (eg, redundant business block 1310 ) replicating at least one of the plurality of primary logic blocks is the same as a second dedicated connection, or a subset of the plurality of memory banks or instances 1304 . can be connected to

Also, memory logic 1306 may have different functions and capabilities than business logic 1308 . For example, memory logic 1306 may be designed to enable read and write operations to memory bank 1304 , while business logic 1308 may be designed to perform in-memory calculations. Thus, if business logic 1308 includes a first business logic block and business logic 1308 includes a second business logic block (eg, redundant business logic 1310), the defective business block 1308 It is possible to disconnect and reconnect to redundant business logic 1310 without loss of capability.

In some embodiments, setting switches (including disable switch 1312 and enable switch 1314) may be implemented as fuses, antifuses, or programmable devices (including disposable programmable devices), or other forms of non-volatile memory. have.

14 is a schematic diagram of an exemplary redundant logic block set 1400 in accordance with a disclosed embodiment. In some embodiments, the redundant logic block set 1400 may be disposed on the substrate 1301 . The redundant logic block set 1400 may include at least one of a business logic 1308 and a redundant business logic 1310 connected to the switches 1312 and 1314, respectively. Additionally, business logic 1308 and redundant business logic 1310 may be coupled to address bus 1402 and data bus 1404 .

In some embodiments, as shown in FIG. 14 , switches 1312 and 1314 may couple logic blocks to clock nodes. In this way, the setup switch can connect or disconnect a logic block with a clock signal and, in turn, enable or disable the logic block. However, in other embodiments, switches 1312 and 1314 may connect logic blocks to other nodes to enable or disable them. For example, a configuration switch may connect the logic block to a voltage supply node (eg, VCC) or a ground node (eg, GND) or a clock signal. In this way, the logic block can be enabled or disabled by the set switch because the set switch can make an open circuit or power down the logic block.

In some embodiments, as shown in FIG. 14 , address bus 1402 and data bus 1404 may be on opposite sides of a logic block connected in parallel to each other on their respective buses. As such, routing of different on-chip elements may be enabled by the logic block set 1400 .

In some embodiments, each of the plurality of disable switches 1312 couples at least one of the plurality of primary logic blocks with a clock node, and each of the plurality of enable switches 1314 clocks at least one of the plurality of redundant blocks. In conjunction with a node, it allows the clock to be connected/disconnected with a simple enable/disable mechanism.

Redundant business logic 1310 of redundant logic block set 1400 allows designers to select blocks worth replicating based on area and routing. For example, a chip designer may choose to duplicate a block with a larger area, as larger blocks are more prone to errors. Thus, the chip designer may decide to duplicate a large area logic block. On the other hand, designers may prefer to replicate small-area blocks because small-area blocks can be easily replicated without significant loss of space. In addition, by utilizing the configuration of FIG. 14 , a designer may easily select a copy of a logic block according to statistics of errors per area.

15 is a block diagram of an exemplary logic block 1500 in accordance with the disclosed embodiment. The logic block may be a business logic block 1308 and/or a redundant business logic block 1310 . However, in other embodiments, the example logic blocks may be memory logic 1306 or other elements of the memory chip 1300 .

Logic block 1500 presents another embodiment where logic redundancy is used within a small processor pipeline. The logic block 1500 may include a register 1508 , a fetch circuit 1504 , a decoder 1506 , and a write-back circuit 1518 . Also, the logic block 1500 may include a calculation unit 1510 and a replication calculation unit 1512 . However, in other embodiments, the logic block 1500 does not include a controller pipeline, but may include other elements including sporadic processing elements with the required business logic.

The calculation unit 1510 and the duplicate calculation unit 1512 may include digital circuits capable of performing digital calculations. For example, the calculation unit 1510 and the replication calculation unit 1512 may include an arithmetic logic unit (ALU) that performs arithmetic and bitwise operations on binary numbers. Alternatively, the calculation unit 1510 and the duplicate calculation unit 1512 may include a floating-point unit (FPU) that performs an operation on a floating-point number. Also, in some embodiments, the calculation unit 1510 and the replication calculation unit 1512 may implement database-related functions such as min, max, count, and compare.

In some embodiments, as shown in FIG. 15 , the calculator 1510 and the replication calculator 1512 may be connected to the switching circuits 1514 and 1516 . The switching circuit may activate or deactivate the calculation unit when activated.

In the logic block 1500 , the duplicate calculation unit 1512 may be a duplicate of the calculation unit 1510 . Also, in some embodiments, register 1508 , fetch circuit 1504 , decoder 1506 , and writeback circuit 1518 (collectively referred to as 'local logic') are larger than calculator 1510 . can be small Because a larger device is more likely to cause manufacturing problems, a designer may decide to duplicate a larger element (e.g., calculation unit 1510) rather than a smaller element (e.g., local logic unit) . However, depending on the yield and error rate record, the designer may choose to duplicate the local logic (or entire block) in addition to or as an alternative to larger elements. For example, the calculation unit 1510 may be larger in size than the register 1508 , the fetch circuit 1504 , the decoder 1506 , and the writeback circuit 1518 , and thus the error probability may be high. The designer may choose to duplicate the calculator 1510 or the entire block rather than other elements of the logic block 1500 .

The logic block 1500 may include a plurality of local setting switches respectively connected to at least one of the calculation unit 1510 and the replication calculation unit 1512 . The local setting switch may be set to deactivate the calculation unit 1510 and activate the copy calculation unit 1512 when a defect is detected in the calculation unit 1510 .

16 is a block diagram of an exemplary logic block connected by a bus in accordance with the disclosed embodiment. In some embodiments, logic blocks 1602 (eg, memory logic 1306 , business logic 1308 , or redundant business logic 1310 ) may be separate from each other, coupled via a bus, and externally specific It can be activated by addressing For example, the memory chip 1300 may include several logic blocks each having an ID number. However, in other embodiments, logic block 1602 may represent a larger element including one or more of memory logic 1306 , business logic 1308 , and redundant business logic 1310 .

In some embodiments, each of the logic blocks 1602 may overlap other logic blocks 1602 . This full redundancy, where every block can act as a primary block or a redundant block, allows designers to disconnect defective elements while maintaining the overall functionality of the chip, improving manufacturing yield. For example, since all duplicate blocks can be connected to the same address and data bus, a designer may have the ability to maintain similar computational power while disabling error-prone logical regions. For example, the initial number of logic blocks 1602 may be greater than the target capability. Thereafter, even if some logic blocks 1602 are deactivated, the target capability may not be affected.

Buses coupled to the logic block may include an address bus 1614 , a command line 1616 , and a data line 1618 . As shown in FIG. 16 , each of the logic blocks may be separately connected to each line of the bus. However, in certain embodiments, logic blocks 1602 may be connected in an organized structure to enable routing. For example, each line of the bus may be connected to a multiplexer that routes the line to another logic block 1602 .

In some embodiments, each of the logic blocks is fused, such as a fused ID 1604 , in order to enable external access without knowing the internal structure of the chip that may be changed due to activation/deactivation of a device. ID may be included. Fused ID 1604 may include an array of switches (eg, fuses) that may determine an ID and couple to management circuitry. For example, the fused ID 1604 may be coupled to the address manager 1302 . Alternatively, the fused ID 1604 may be coupled to a higher-level memory address unit. In such an embodiment, the fused ID 1604 may be set to a specific address. For example, fused ID 1604 may include a programmable non-volatile device that determines a final ID based on a command received from management circuitry.

The distributed processor on the memory chip may be designed with the configuration shown in FIG. 16 . Inspection procedures executed as BISTs during chip wakeup or factory inspection phase include execution ID numbers in blocks of a plurality of primary logic blocks (memory logic 1306 and business logic 1308) that pass the inspection protocol. can be assigned The inspection procedure may also assign illegal ID numbers to blocks of the plurality of primary logic blocks that fail the inspection protocol. The inspection procedure may also assign an execution ID number to blocks of the plurality of redundant blocks (redundant logic block 1310 ) that pass the inspection protocol. Because the redundant block replaces the failing primary logic block, the block of the plurality of redundant blocks assigned an execution ID number may be greater than or equal to the block of the plurality of primary logic blocks assigned an illegal ID number, thus The block is deactivated. In addition, each of the plurality of primary logic blocks and each of the plurality of redundant blocks may include at least one fused ID 1604 . Also, as shown in FIG. 16 , a bus connecting the logic block 1602 may include a command line, a data line, and an address line.

However, in other embodiments, all logic blocks 1602 coupled to the bus will start in an inactive state without an ID number. Checking one by one, each good block will be given an execution ID, and the non-working logic block will have an illegal ID remaining and will be deactivated. In this way, redundant logic blocks can improve manufacturing yield by replacing blocks known to be defective during inspection.

Address bus 1614 may couple management circuitry to each of a plurality of memory banks, each of a plurality of primary logic blocks, and each of a plurality of redundant blocks. Due to this coupling, upon detecting a fault associated with the primary logic block, the management circuit may assign an invalid address to one of the plurality of primary logic blocks and a valid address to one of the plurality of redundant blocks.

For example, as shown in FIG. 16A ), an illegal ID (eg, address 0xFFF) may be set in all logic blocks 1602(a)-(c). After the check, logic blocks 1602(a) and 1602(c) are confirmed to function properly and logic block 1602(b) is identified as having a malfunction. In FIG. 16A ), non-shaded logic blocks may represent logic blocks that have passed the function check, and shaded logic blocks may represent logic blocks that have failed the function check. Thereafter, the inspection procedure changes the illegal ID for a properly functioning logic block to a legal ID, and maintains the illegal ID for the malfunctioning logic block as it is. For example, in FIG. 16A), addresses for logic blocks 1602(a) and 1602(c) are changed from 0xFFF to 0x001 and 0x002, respectively. On the other hand, the address for logic block 1602(b) remains at the illegal address 0xFFF. In some embodiments, the ID is changed by programming the corresponding fused ID 1604 .

If the test result of the logic block 1602 is different, the setting may be different. For example, as shown in FIG. 16B ), the address manager 1302 may initially assign an illegal ID (ie, 0xFFF ) to all logic blocks 1602 . However, as a result of inspection, it may appear that both logic blocks 1602(a) and 1602(b) function normally. In this case, since the memory chip 1300 may have only two logic blocks, it may not be necessary to check the logic block 1602(c). Accordingly, in order to minimize the resources required for inspection, the logic block may be inspected only according to the minimum quantity of functional logic blocks required by the product definition of the memory chip 1300 and other blocks may not be inspected. In FIG. 16B ), non-shaded logic blocks represent logic blocks that have passed the functional check, and shaded logic blocks represent unchecked logic blocks.

In these embodiments, the production tester (external or built-in, automatic or manual) or the controller running the BIST at startup changes the illegal ID to the run ID for the logic block that has been found to function through inspection and the logic that is not checked. Blocks may not change illegal IDs. For example, in FIG. 16B), the addresses of logic blocks 1602(a) and 1602(b) are changed from 0xFFF to 0x001 and 0x002, respectively. On the other hand, the unchecked address for the logic block 1602(c) maintains the illegal address 0xFFF.

17 is a schematic diagram of exemplary devices 1702 and 1712 connected in series in accordance with disclosed embodiments. 17 may show an entire system or a chip. Alternatively, FIG. 17 may show a block inside a chip including other functional blocks.

Devices 1702 , 1712 may represent an entire device including a plurality of logic blocks, such as memory logic 1306 and/or business logic 1308 . In such embodiments, elements 1702 and 1712 may also include elements necessary to perform operations, such as address manager 1302 . However, in other embodiments, elements 1702 , 1712 may be representative of logic elements such as business logic 1308 or redundant business logic 1310 .

17 presents an embodiment in which devices 1702 and 1712 may need to communicate with each other. In this case, elements 1702 and 1712 may be connected in series. However, non-functional devices can prevent connectivity between logic blocks. Thus, the connection between devices may include a bypass option in case the device needs to be deactivated due to a fault. The bypass option may also be part of the bypassed device itself.

In Figure 17, devices can be connected in series (eg, 1702(a)-(c)) and failing devices (eg, 1702(b)) can be bypassed in case of failure. The device may further be connected in parallel with the switching circuit. For example, in some embodiments, as shown in FIG. 17 , devices 1702 , 1712 may be coupled with switching circuits 1722 , 1728 . In the example shown in Fig. 17, element 1702(b) is defective. For example, device 1702(b) fails the circuit function test. Accordingly, device 1702(b) may be deactivated utilizing an enable switch 1314 (not shown in FIG. 17) or the like and/or switching circuitry 1722(b) is activated to activate device 1702(b). It can bypass and maintain connectivity between logic blocks.

Accordingly, when a plurality of primary elements are connected in series, each of the plurality of elements may be connected in parallel with a parallel switch. When a defect associated with one of the plurality of elements is detected, a parallel switch connected with one of the plurality of elements may be activated to connect two elements of the plurality of elements to each other.

In another embodiment, as shown in FIG. 17 , the switching circuit 1728 may include one or more sampling points that delay cycles to maintain synchronization between different lines of the device. When a device is inactive, breaking the connection between adjacent logic blocks can cause synchronization errors with other calculations. For example, if an operation requires data from both lines A and B, and lines A and B are each performed by a separate series of devices, then disabling one device will result in lines A and B. A synchronization between the two will occur, which will require additional data management. To prevent desynchronization, the sample circuit 1730 may simulate a delay caused by the deactivated element 1712(b). However, in some embodiments, the parallel switch may include an antifuse instead of the sampling circuit 1730 .

18 is a block diagram of exemplary devices connected in a two-dimensional array according to the disclosed embodiment. 18 may represent an entire system or a chip. Alternatively, FIG. 18 may show a block inside a chip including other functional blocks.

Device 1806 may be representative of an autonomous unit comprising a plurality of logic blocks, such as memory logic 1306 and/or business logic 1308 . However, in other embodiments, element 1806 may be representative of a logic element such as business logic 1308 . For convenience of description, the description of FIG. 18 may refer to the element (eg, the memory chip 1300 ) previously described with reference to FIG. 13 .

As shown in FIG. 18 , the device 1806 (which may include or represent one or more of memory logic 1306 , business logic 1308 , and redundant business logic 1310 ) includes a switching box 1808 . and a two-dimensional array connected to each other through a connection box 1810 . Also, to control the configuration of the two-dimensional array, the two-dimensional array may include an I/O block 1804 around it.

The connection box 1810 may be a programmable and reconfigurable device that may correspond to a signal input from the I/O block 1804 . For example, the connection box may include a plurality of input pins from the device 1806 and may also be connected to the switching box 1808 . Alternatively, connection box 1810 may include a group of switches connecting pins of a programmable logic cell with routing tracks, and switching box 1808 may include a group of switches connecting pins of a programmable logic cell with different tracks. have.

In some embodiments, the connection box 1810 and the switching box 1808 may be implemented as setting switches, such as the switches 1312 and 1314 . In this embodiment, the connection box 1810 and the switching box 1808 can be set up by a BIST or production tester running on chip startup.

In some embodiments, connection box 1810 and switching box 1808 may be established after circuit function testing for device 1806 . In such an embodiment, the I/O block 1804 may be used to send a test signal to the device 1806 . According to the test result, the I/O block 1804 deactivates the device 1806 that fails the test protocol and activates the device 1806 that passes the test protocol, the connection box 1810 and the switching box 1808 It can transmit a programming signal that sets

In this embodiment, the plurality of primary logic blocks and the plurality of redundant blocks may be arranged on the substrate in a two-dimensional grid. Accordingly, each element of the plurality of primary elements 1806 and each block of the plurality of redundant blocks, such as the redundant business logic 1310 , may be connected to each other by a switching box 1808 , and the input block is each line of the two-dimensional grid and can be placed on the perimeter of each column.

19 is a block diagram of an exemplary device that is compositely connected according to the disclosed embodiment. 19 may show the entire system. Alternatively, FIG. 19 may show a block inside a chip including other functional blocks.

The composite connection of FIG. 19 includes elements 1902(a)-(f) and setting switches 1904(a)-(h). Device 1902 may be representative of an autonomous device comprising a plurality of logic blocks, such as memory logic 1306 and/or business logic 1308 . However, in other embodiments, element 1902 may be representative of a logic element such as memory logic 1306 , business logic 1308 , or redundant business logic 1310 . The setting switch 1904 may include one or more of a deactivation switch 1312 and an enable switch 1314 .

As shown in FIG. 19 , the composite connection may include elements 1902 on two sides. For example, a composite connection may include two separate substrates spaced apart along the z-axis. Alternatively or additionally, the device 1902 may be disposed on two sides of the substrate. For example, for the purpose of reducing the area of the memory chip 1300 , the substrate 1301 may be disposed on two overlapping surfaces and may be connected to a three-dimensionally disposed setting switch 1904 . The setting switch may include a deactivation switch 1312 and/or an enable switch 1314 .

The first side of the substrate may include a 'main' element 1902 . This block can be enabled by default. In such an embodiment, the second side may include a 'redundant' element 1902 . This device can be disabled by default.

In some embodiments, setting switch 1904 may include an antifuse. Thus, after inspection of the element 1902, by switching a particular antifuse to an 'always-on' state and deactivating the selected element 1902, blocks can be connected to tiles of functional elements, even if they are on different sides. . In the example shown in FIG. 19, one 1902(e) of the 'main' element is faulty. In FIG. 19 , a block having a problem with a function or not inspected may be displayed as shaded, and a block that has also been tested and not having a problem with a function may be displayed in a non-shaded state. Accordingly, the setting switch 1904 may be set such that one (eg, 1902(f)) of the logic blocks on different sides is activated. Thus, even if one of the main logic blocks is defective, the memory chip still functions by replacing it with a spare logic element.

Also in FIG. 19 , one 1902(c) of the elements 1902 on the second side has not been tested and has not been activated because the main logic block is functioning. For example, in Fig. 19, the main elements 1902(a) and 1902(d) both passed the functional test. Thus, device 1902(c) has not been tested or activated. Thus, Fig. 19 illustrates the ability to specifically select a logic block to be activated according to a test result.

In some embodiments, as shown in FIG. 19 , not all devices 1902 on the first side have corresponding redundant or redundant blocks. However, in another embodiment, when all devices are both primary and redundant, all devices may be redundant with respect to each other, thereby providing perfect redundancy. In addition, although some are implemented as a radial network (star network) shown in FIG. 19, parallel connection, series connection, and/or implementation of connecting different elements in parallel or series with a setting switch is also possible.

20 is an exemplary flow diagram illustrating a redundant block activation process 2000 in accordance with a disclosed embodiment. Activation process 2000 may be performed for memory chip 1300 , particularly a DRAM memory chip. In some embodiments, the process 200 includes testing at least one circuit function for each block of a plurality of logic blocks disposed on a substrate of a memory chip, in the plurality of primary logic blocks based on a result of the testing. identifying a defective logic block; checking at least one circuit function for at least one redundant or additional logic block disposed on a substrate of a memory chip; applying an external signal to a disable switch to determine the at least one defective It may include the steps of deactivating the logic block with the , and activating the at least one redundant block by applying an external signal to the activation switch. Here, the activation switch may be connected to at least one redundant block and may be disposed on a substrate of the memory chip. Each step of the process 2000 is further described in the description of FIG. 20 below.

Process 2000 may include examining a plurality of logical blocks, such as business block 1308 and a plurality of redundant blocks (eg, redundant business block 1310) (step 2002). The inspection may be performed prior to packaging using a probing station for on-wafer inspection, or the like. However, process 2000 may be performed after packaging.

The test of step 2002 may include applying a finite sequence of test signals to all logic blocks of the memory chip 1300 or a subset of the logic blocks of the memory chip 1300 . The check signal may request a calculation that is expected to output a 0 or 1. In another embodiment, the test signal may request to read a specific address of a memory bank or write to a specific memory bank.

A check scheme may be implemented to check the response of the logic block under the iterative process of step 2002 . For example, after sending a command to write data to a memory bank, the logic block can be checked by verifying the integrity of the written data. In other embodiments, the data may be reversed and the algorithm checked iteratively.

In another embodiment, the inspection of step 2002 may include executing the model of the logic block to generate a target memory image based on the set of inspection instructions. Thereafter, the same sequence of commands may be executed in a logic block of the memory chip, and the result may be written. The remaining memory images of the simulation can also be compared with images obtained from the inspection, and discrepancies can be flagged as failing.

Alternatively, at step 2002, the testing may include shadow modeling, where a diagnosis is generated but not necessarily predictive of an outcome. Rather, inspections utilizing shadow modeling can be run in parallel on the memory chip and simulation. For example, when a logic block in a memory chip completes an instruction or task, it can signal the simulation to execute the same instruction. When the logic block of the memory chip completes the instruction, it compares the architectural state of the two models. If there is any discrepancy here, it is flagged as failing.

In some embodiments, step 2002 may examine all logic blocks (including each of memory logic 1306 , business logic 1308 , and redundant business logic 1310 , etc.). However, in other embodiments, only a subset of the logic blocks may be tested in different orders of inspection. For example, in the first test, only the memory logic 1306 and its associated blocks may be checked. In the secondary inspection, only the business logic 1308 and its associated blocks may be inspected. In the third inspection, the logic block associated with the redundant business logic 1310 may be inspected according to the results of the previous two inspections.

Process 2000 may proceed to step 2004 . In step 2004, a defective logic block is identified, and a defective redundant block may also be identified. For example, a logic block that fails the check in step 2002 may be identified as a defective block in step 2004 . However, in other embodiments, only certain defective blocks may be initially identified. For example, in some embodiments, only logic blocks associated with business logic 1308 may be identified, and a defective redundant logic block may only be identified if the redundant logic block should replace the defective logic block. have. Also, identifying the defective logic block may include writing identification information of the identified defective logic block to a memory bank or non-volatile memory.

In step 2006, the faulty logic block may be deactivated. For example, a faulty logic block can be deactivated using a setup circuit by disconnecting the faulty logic block from a clock, ground, and/or power node. Alternatively, a faulty logic block can be deactivated by setting the connection box in a layout that avoids the logic block. Also, in another embodiment, a defective logic block may be deactivated by receiving an illegal address from the address manager 1302 .

At step 2008, a redundant block that duplicates the defective logic block may be identified. In order to maintain the capability of the memory chip even if some logic blocks are defective, in step 2008, the defective logic block may be duplicated and a usable redundant block may be identified. For example, if it is determined that the logic block that performs vector multiplication is defective, in step 2008, the address manager 1302 or the on-chip controller may identify an usable redundant logic block that also performs vector multiplication.

In step 2010, the redundant block identified in step 2008 may be activated. Contrary to the deactivation operation of step 2006, in step 2010, the identified redundant block may be activated by coupling the identified redundant block to a clock, ground, and/or power node. Alternatively, the identified redundant block may be activated by setting a connection box in a configuration that connects the identified redundant block. Further, in another embodiment, by receiving an execution address at check procedure execution time, the identified redundant block may be activated.

21 is an exemplary flow diagram illustrating an address assignment process 2100 in accordance with the disclosed embodiment. The address assignment process 2100 may be performed for the memory chip 1300 , particularly a DRAM memory chip. As described with reference to FIG. 16 , in some embodiments, a logic block of the memory chip 1300 may be coupled to a data bus and may include an address ID. Process 2100 provides an addressing method for deactivating a faulty logic block and activating a logic block that passes inspection. While the steps of process 2100 are described as being performed by a BIST or production tester running on chip startup, other elements of memory chip 1300 and/or external devices may also perform one or more steps of process 2100 . can

In step 2102, the tester can deactivate all logic blocks and redundant blocks by assigning an illegal ID to each logic block at the chip level.

At step 2104, the tester may execute the logic block's inspection protocol. For example, the tester may execute the test method described in step 2002 on one or more of the logic blocks of the memory chip 1300 .

In step 2106, according to the result of the check in step 2104, the tester may determine whether the logic block is defective. If the logic block is not defective (NO at step 2106), the address manager may assign an execution ID to the checked logic block at step 2108. If the logic block is faulty (YES at step 2106), the address manager 1302 may leave an illegal ID in the faulty logic block at step 2110.

In step 2112, address manager 1302 may select a redundant logic block that duplicates the defective logic block. In some embodiments, the redundant logic block that duplicates the defective logic block may have the same elements and connections as the defective logic block. However, in other embodiments, the redundant logic block may include other elements and/or connections than the defective logic block, but capable of performing equivalent operations. For example, if the faulty logic block is designed to perform vector multiplication, the selected redundant logic block may also be able to perform vector multiplication, even if it is not of the same architecture as the faulty logic block.

In step 2114, the address manager 1302 may examine the redundant block. For example, the tester may apply the inspection scheme applied in step 2104 to the identified redundant blocks.

In operation 2116 , based on the result of the inspection in operation 2114 , the tester may determine whether the redundant block is defective. If, at step 2118, the redundant block is not defective ('no' at step 2116), the tester may assign a run ID to the identified redundant block. In some embodiments, process 2100 may return to step 2104 after step 2118 to form an iterative loop that examines all logic blocks of the memory chip.

If the tester determines that the redundant block is defective (YES in step 2116), in step 2120 the tester may determine whether additional redundant blocks are available. For example, a tester can query a memory bank for information about available redundant logic blocks. If there is a redundant logic block available (YES at step 2120), the tester can return to step 2112 to identify a new redundant logic block that duplicates the faulty logic block. If no redundant logic block is available (No at step 2120), then at step 2122, the tester may generate an error signal. The error signal may include information of a defective logic block and a defective redundant block.

combined memory bank

Embodiments disclosed herein also include a distributed high-performance processor. The processor may include a memory controller that accesses the memory bank and the processing unit. The processor may be configured such that the data is quickly passed to the processing unit for calculation. For example, if a processing unit requires two instances of data to perform an operation, the memory controller may be configured such that communication lines can separately provide access to information from both data instances. The disclosed memory architecture seeks to minimize the hardware requirements associated with complex cache memory and complex register file schema. Typically, a processor chip includes a cache scheme that allows the core to work directly with registers. However, cache operation requires significant die area and consumes additional power. The disclosed memory architecture avoids the use of a cache scheme by adding logical elements to the memory.

The disclosed architecture also allows for strategic (or optimized) placement of data within memory banks. Even when a memory bank contains only a single port and has high latency, the disclosed memory architecture can enable high performance and avoid memory bottlenecks by strategically placing data in different blocks of the memory bank. For the purpose of providing a continuous stream of data to the processing unit, a compilation optimization step may determine how data should be stored in a memory bank for a particular or general operation. Thereafter, the memory controller that accesses the processing unit and the memory bank may be configured to allow access to a specific processing unit when data is needed to perform an operation.

The setting of the memory chip may be performed by a processing unit (eg, a setting manager) or an external interface. Settings may also be recorded by a compiler or other software tool. In addition, the configuration of the memory controller may be based on a port usable for the memory bank and a data structure of the memory bank. Accordingly, the disclosed architecture may provide a continuous flow of data from different memory banks or simultaneous information to the processing unit. This allows in-memory computational tasks to be expedited, avoiding latency bottlenecks or cache memory requirements.

In addition, data stored in the memory chip may be arranged based on a compilation optimization step. Compilation may allow the processor to build processing routines that efficiently allocate work to processing units with no memory latency. Compilation is performed by the compiler and can be sent to the host connected to the external interface of the board. In general, the result of high latency and/or a small number of ports for a particular access pattern can be a data bottleneck for the processing units that need the data. However, the disclosed compilation may place data in memory banks such that the processing unit can continue to receive data even with adverse memory types.

Also, in some embodiments, the configuration manager may signal the necessary processing unit based on the calculation required for the job. Different processing units or logic blocks on a chip may have specialized hardware or architectures for different tasks. Thus, depending on the task to be performed, a processing unit or group of processing units may be selected to perform the corresponding operation. The on-board memory controller may send or allow access to data depending on the selection of the processor subunit to improve the data transfer rate. For example, based on compilation optimizations and memory architecture, a processing unit may be allowed access to a memory bank when required to perform a task.

In addition, the chip architecture may include on-chip elements that enable the transfer of data by reducing the time required for data access of the memory bank. Accordingly, the present disclosure describes a chip architecture for a high-performance processor capable of performing a specific or general operation by utilizing a simple memory instance along with a compilation optimization step. A memory instance may have a low number of ports and/or high latency, such as those used in DRAM devices or other memory oriented technologies, but the disclosed architecture enables a continuous (or near continuous) flow of data from the memory bank to the processing unit. Thus, these shortcomings can be overcome.

In the present application, simultaneous communication may refer to communication within a clock cycle. Alternatively, simultaneous communication may mean transmitting information within a predetermined time. For example, simultaneous communication may refer to communication within a few nanoseconds (billionths of a second).

22 is a schematic diagram of an exemplary processing apparatus according to the disclosed embodiment. 22A illustrates a first embodiment of the processing device 2200 in which the memory controller 2210 connects the first memory block 2202 and the second memory block 2204 by using a multiplexer. The memory controller 2210 may also connect at least the configuration manager 2212 , the logic block 2214 , and the plurality of accelerators 2216(a)-(n). 22B), the memory controller 2210 utilizes a bus connecting the memory controller 2210 and at least the configuration manager 2212, the logic block 2214, and the plurality of accelerators 2216(a)-(n)). Thus, a second embodiment of the processing unit 2200 connecting the memory blocks 2202 and 2204 is shown. Also, the host 2230 may be located outside and may be connected to the processing device 2200 through an external interface or the like.

Memory blocks 2202 and 2204 may include DRAM mats or mat groups, DRAM banks, MRAM\PRAM\RERAM\SRAM units, flash mats, or other memory technologies. Memory blocks 2202 and 2204 may alternatively include non-volatile memory, flash memory devices, resistive random access memory (ReRAM) devices, or magnetoresistive random access memory (MRAM) devices.

The memory blocks 2202 and 2204 may additionally include a plurality of memory cells arranged in rows and columns between a plurality of word lines (not shown) and a plurality of bit lines (not shown). A gate of each row of memory cells may be coupled to each of a plurality of wordlines. Each column of memory cells may be connected to each of a plurality of bitlines.

In other embodiments, memory regions (including memory blocks 2202 and 2204) may be constructed from a single memory instance. In the present application, the term 'memory instance' may be used interchangeably with 'memory block'. The properties of a memory instance (or block) can be poor. For example, there may be only one port in memory and the random access latency may be large. Alternatively or additionally, the memory may become inaccessible during thermal and line changes and may face data access issues related to capacitive charging and/or circuit setup, and the like. However, the architecture presented in FIG. 22 enables parallel processing of the memory device by allowing a dedicated connection between the memory instance and the processing unit and arranging the data in a specific way that takes into account the characteristics of the block.

In some device architectures, a memory instance may have multiple ports, allowing parallel operation. However, in such an embodiment, the chip may achieve performance improvements if the data is compiled and organized according to the chip architecture. For example, the compiler can improve the access efficiency of the memory area by configuring the data arrangement by providing a command so that it can be easily accessed even using a single port memory.

Also, memory blocks 2202 and 2204 may be of multiple types for a single chip of memory. For example, memory blocks 2202 and 2204 may be eFlash and eDRAM. Also, a memory block may include DRAM with instances of ROM.

The memory controller 2210 may include a logic circuit that processes memory access and transmits the result to the remaining modules. For example, the memory controller 2210 may include a selection device such as an address manager and a multiplexer to transfer data between the memory block and the processing unit and allow access to the memory block. Alternatively, the memory controller 2210 includes a double data rate (DDR) memory controller used for driving a DDR SDRAM in which data is transferred on a rising edge and a falling edge of a memory clock of the system. can do.

Also, the memory controller 2210 may constitute a dual channel memory controller. When the dual-channel memory is introduced, control of parallel access by the memory controller 2210 may be possible. Parallel access lines can be configured to enable synchronization of data of equal length to each other when multiple lines are used together. Alternatively or additionally, parallel access lines may allow access of multiple memory ports of a memory bank.

In some embodiments, processing device 2200 may include one or more multiplexers that may be coupled to a processing unit. The processing unit may include a configuration manager 2212 , a logic block 2214 , and an accelerator 2216 that may be directly coupled to the multiplexer. Also, the memory controller 2210 may include at least one data input from the plurality of memory banks or blocks 2202 and at least one data output connected to each of the plurality of processing units. With this configuration, the memory controller 2210 simultaneously receives data from the memory banks or blocks 2202 and 2204 through two data inputs, and simultaneously transmits the data received through at least one selected processing unit through two data outputs. can be transmitted However, in some embodiments, the at least one data input and the at least one data output may be implemented on a single port allowing only read or write operations. In such an embodiment, a single port may be implemented as a data bus containing data, address, and command lines.

The memory controller 2210 may be connected to each of the plurality of memory blocks 2202 and 2204 , and may also be connected to a processing unit through a selection switch or the like. Also, a processing unit on a substrate including the setting manager 2212 , the logic block 2214 , and the accelerator 2216 may be individually connected to the memory controller 2210 . In some embodiments, settings manager 2212 may receive an indication of a task to be performed, and correspondingly memory controller 2210 , accelerator 2216 , and/or logic according to settings stored in memory or provided externally. Block 2214 may be established. Alternatively, the memory controller 2210 may be configured by an external interface. The operation may require at least one calculation that may be used to select at least one processing unit from among the plurality of processing units. Alternatively or additionally, the selection may be made based at least in part on the ability of the selected processing unit to perform the at least one calculation. In response, the memory controller 2210 may allow access to the memory bank or transfer data between the at least one selected processor and the at least two memory banks, utilizing a dedicated bus and/or in pipelined memory access. have.

In some embodiments, the first memory block 2202 of the at least two memory blocks may be disposed on a first side of the plurality of processing units, and the second memory block 2204 of the at least two memory blocks may be disposed on the first side of the plurality of processing units. It may be disposed on the second side of the plurality of processing units opposite to each other. Also, the selected processing unit to perform the task, e.g., accelerator 2216(n), is connected to the second memory bank 2204 during the clock cycle in which the communication line is opened to the first memory bank or first memory block 2202. can be set to access. Alternatively, the selected processing unit may be configured to transmit data to the second memory block 2204 during a clock cycle in which the communication line is opened to the first memory block 2202 .

In some embodiments, as shown in FIG. 22 , the memory controller 2210 may be implemented as an individual element. However, in another embodiment, the memory controller 2210 may be embedded in the memory area or disposed along the accelerators 2216(a)-(n).

The processing region of the processing device 2200 may include a configuration manager 2212 , a logic block 2214 , and accelerators 2216(a)-(n). The accelerator 2216 may include a plurality of processing circuits with predefined functions and may be defined by a particular application. For example, an accelerator may be a vector MAC unit or a Direct Memory Access (DMA) unit that handles memory movement between modules. The accelerator 2216 may also calculate its own address and request or write data to the memory controller 2210 . For example, the settings manager 2212 can signal at least one of the accelerators 2216 that it can access the memory bank. Thereafter, the accelerator 2216 may set the memory controller 2210 to transmit data or allow access. The accelerator 2216 may also include at least one arithmetic logic unit, at least one vector processing logic unit, at least one string comparison logic unit, at least one register, and at least one direct memory access (DMA). .

The settings manager 2212 may include digital processing circuitry that configures the accelerator 2216 and directs the execution of tasks. For example, the setting manager 2212 may be connected to each of the memory controller 2210 and the plurality of accelerators 2216 . The settings manager 2212 may have a dedicated memory to store the settings of the accelerator 2216 . The setting manager 2212 may obtain commands and settings through the memory controller 2210 using the memory bank. Alternatively, the setting manager 2212 may be programmed through an external interface. In some embodiments, the configuration manager 2212 may be implemented with an on-chip reduced instruction set computer (RISC) or an on-chip composite CPU with its own cache scheme. In some embodiments, the configuration manager 2212 may be omitted, and the accelerator may be configured through an external interface.

The processing device 2200 may also include an external interface (not shown). The external interface allows access to memory from a higher level, such as a memory bank controller that receives commands from the external host 2230 or the on-chip main processor, or allows access to the memory from the external host 2230 or the on-chip main processor. An external interface may allow programming of the configuration manager 2212 and accelerator 2216 by writing settings or code to memory via the memory controller 2210 to be later used by the configuration manager 2212 or components 2214, 2216. can However, the external interface may also directly program the processing unit without being transmitted through the memory controller 2210 . When the configuration manager 2212 is a microcontroller, the configuration manager 2212 may allow loading of code from the main memory into the controller local memory via an external interface. The memory controller 2210 may be set to stop the operation in response to receiving a request from the external interface.

The external interface may include a plurality of connectors associated with logic circuitry that provides an adhesive-free interface to various elements on the processing device. External interfaces include data I/O inputs for reading data and outputs for writing data, external address outputs, external CE0 chip select pins, active-low chip selectors, byte enable pins, and wait counterparts on memory cycles. a pin for , a write enable pin, an output enable pin, and a read-write enable pin. Thus, the external interface has the requested inputs and outputs that control the process and obtain information from the processing device. For example, an external device may be compliant with the JEDEC DDR standard. Alternatively or additionally, the external device may be compliant with SPI\OSPI or other standards such as UART.

In some embodiments, the external interface may be disposed on a chip substrate and may be connected to the external host 2230 . The external host may secure access to the memory blocks 2202 and 2204 , the memory controller 2210 , and the processing unit through the external interface. Additionally or alternatively, the external host 2230 may read and write to the memory, and sends a signal to the configuration manager 2212 through the read and write commands to perform an operation such as starting and/or stopping a process. can do it Also, the external host 2230 may directly configure the accelerator 2216 . In some embodiments, the external host 2230 may directly perform read/write operations on the memory blocks 2202 and 2204 .

In some embodiments, settings manager 2212 and accelerator 2216 may be configured to associate device regions with memory regions using a direct bus depending on the target task. For example, a subset of accelerators 2216 may associate with a memory instance 2204 if the subset of accelerators has the capability to perform the calculations necessary to execute a task. By performing this separation, the bandwidth BW required for the memory blocks 2202 and 2204 can be secured to the dedicated accelerator. In addition, since connecting a memory instance to the memory controller 2210 allows quick access to data in different memories even with high row latency, this configuration with a dedicated bus converts large-capacity memory into smaller instances or blocks. It can make sharing possible. To parallelize the connections, the memory controller 2210 may be coupled to each of the memory instances with data, address, and/or control buses.

The inclusion of the memory controller 2210 as described above may eliminate the need for a cache scheme or complex register file in the processing unit. Although the system capability may be enhanced by adding a cache scheme, the architecture of the processing unit 2200 allows the designer to add sufficient memory blocks or instances based on the processing operation and thus manage the instances without the cache scheme. For example, due to the architecture of the processing unit 2200, a cache scheme may be obviated by eliminating pipelined memory accesses. In a pipelined memory access, the processing unit may receive a continuous flow of data in every cycle, and certain data lines may be opened (or activated) while other data lines may receive or transmit data. Due to the continuous data flow utilizing individual communication lines, execution speed can be improved and delays due to line changes can be minimized.

In addition, the disclosed architecture of FIG. 22 enables a pipelined memory access that organizes data in a small number of memory blocks and reduces power loss due to line changes. For example, in some embodiments, the compiler may send and receive data organization or data organization method of a memory bank with the host 2230 and enable access to data during a given operation. Thereafter, the configuration manager 2212 may define which memory bank (and in some cases, which port of the memory bank) can be accessed by the accelerator. Synchronization between data locations in memory banks and data access methods improves computational work by feeding data to the accelerator with minimal delay. For example, in embodiments where configuration manager 2212 includes a RISC\CPU, the method may be implemented in offline software, after which configuration manager 2212 may be programmed to execute the method. Methods can be written in any language executable by a RISC\CPU computer and run on any platform. The input of the method may include the data itself, along with the settings and memory access patterns of the memory behind the memory controller. In addition, the method may be implemented in a language or machine language specific to the embodiment, and may simply be a set of binary numbers or characters.

As described above, in some embodiments, the compiler may provide instructions to the host 2230 to organize data into memory blocks 2202 and 2204 for pipelined memory accesses. Pipelined memory access generally involves receiving a plurality of addresses of a plurality of memory banks or memory blocks 2202, 2204, accessing the plurality of memory banks using individual data lines according to the received addresses, a plurality of supplies data to at least one of the plurality of processing units via a first communication line from a first address in a first memory bank of a memory bank of opening a communication line, and supplying data from the second address to at least one of the plurality of processing units via the second communication line and providing a third communication to a third address in the first memory bank of the first communication line within a second clock cycle It may include the step of opening a line. In some embodiments, pipelined memory access may be implemented with two memory blocks connected to a single port. In this embodiment, the memory controller 2210 may hide the memory block behind a single port, but may send data to the processor in a pipelined memory access approach.

In some embodiments, the compiler may run on host 2230 before executing the job. In such an embodiment, since the compiler may already know the setting of the data flow, the compiler may be able to determine the setting of the data flow based on the architecture of the memory device.

In another embodiment, if the settings of the memory blocks 2204 and 2202 are not known at offline time, the pipelined method may be executed on the host 2230, where the host 2230 writes the data to the memory blocks before starting the computation. can be placed For example, the host 2230 may write data directly to the memory blocks 2204 and 2202 . In this embodiment, the processing unit such as the setting manager 2212 and the memory controller 2210 may not know information about necessary hardware until runtime. Thereafter, it may be necessary to delay selection of accelerator 2216 until execution of the task begins. In this situation, the processing unit or memory controller 2210 may randomly select the accelerator 2216 and generate an inspection data access pattern that may be modified as the task is executed.

However, if the task is known in advance, the compiler places data and instructions in the memory bank so that the host 2230 establishes a signal connection that minimizes the access delay to a processing unit such as the configuration manager 2212 . For example, in some cases, accelerator 2216 may require n words simultaneously. However, each memory instance can only support fetching m words at a time. Here, both m and n are integers, and m　<　n. Thus, the compiler can place the necessary data in different memory instances or blocks to enable data access. Also, in order to prevent line drop delay, the host may divide data into different lines of different memory instances when there are multiple memory instances in the processing device 2200 . Splitting data makes it possible to access the next line of data in the next instance while continuing to use the data in the current instance.

For example, accelerator 2216(a) may be configured to multiply two vectors. Each vector may be stored in a separate memory block, such as memory blocks 2202 and 2204, and each vector may contain multiple words. Thus, to complete a task requiring multiplication by accelerator 2216(a), it may be necessary to access two memory blocks and fetch multiple words. However, in some embodiments, the memory block allows access to only one word per clock cycle. For example, a memory block may have a single port. In this case, in order to speed up data transfer during operation, the compiler may arrange the words constituting the vector in different memory blocks to enable parallel and/or simultaneous reading of the words. In this situation, the compiler can store the word in a block of memory with a dedicated line. For example, if each vector contains two words and the memory controller has direct access to four memory blocks, the compiler places the data in four memory blocks, and each memory block transfers words and handles data transfer. You can do it quickly. Also, in an embodiment in which the memory controller 2210 may have more than a single connection to each memory block, the compiler may instruct the configuration manager 2212 (or other processing unit) to access a specific port. In this way, the processing unit 2200 can continuously provide data to the processing unit by performing pipelined memory accesses, simultaneously loading words on some lines and transferring data on other lines.

23 is a block diagram of an exemplary processing device 2300 in accordance with a disclosed embodiment. The configuration diagram shows a MAC unit 2302, a configuration manager 2304 (equivalent to or similar to the configuration manager 2212), a memory controller 2306 (equivalent to or similar to the memory controller 2210), and a plurality of memory blocks 2308 (a). A simplified processing unit 2300 showing a single accelerator of the form )-(d)) is shown.

In some embodiments, the MAC unit 2302 may be a specific accelerator to handle a specific task. For example, the processing device 2300 may be given a 2D-convolution operation. In this case, the configuration manager 2304 may signal an accelerator with the appropriate hardware to perform the calculations associated with this task. For example, the MAC unit 2302 has four internal incrementing counters (logical adders and registers to manage the four loops required to compute the convolution) and a multiply accumulate unit) may exist. The settings manager 2304 may signal the MAC unit 2302 to process the incoming data and execute a task. The settings manager 2304 may send an indication to the MAC unit 2302 to execute the task. In this situation, the MAC unit 2302 may iterate on the computed address, multiply the number, and accumulate in an internal register.

In some embodiments, configuration manager 2304 may configure accelerators, while memory controller 2306 utilizes a dedicated bus to allow access to block 2308 and MAC unit 2302 . However, in another embodiment, the memory controller 2306 may directly configure the accelerator based on a command received from the configuration manager 2304 or an external interface. Alternatively or additionally, the configuration manager 2304 can preload some settings and have the accelerator run repeatedly on different addresses with different sizes. In such an embodiment, settings manager 2304 may include a cache memory that stores instructions before they are sent to at least one of a plurality of processing units, such as accelerator 2216 . However, in other embodiments, the settings manager 2304 may not include a cache.

In some embodiments, settings manager 2304 or memory controller 2306 may receive addresses that need to be accessed for operations. The settings manager 2304 or the memory controller 2306 may check the registers to determine whether this address already exists in a line loaded into one of the memory blocks 2308 . If this address already exists, the memory controller 2306 may read the word from the memory block 2308 and forward it to the MAC unit 2302 . If the loaded line does not have this address, the settings manager 2304 can request the memory controller 2306 to load this line and signal the MAC unit 2302 to wait until it gets this address.

In some embodiments, as shown in FIG. 23 , the memory controller 2306 may include two inputs from two separate addresses. However, if more than one address needs to be accessed simultaneously, and these addresses are in a single memory block (eg, only in memory block 2308(a)), then memory controller 2306 or configuration manager 2304 is an exception. can be placed Alternatively, the setting manager 2304 may output an invalid data signal when two addresses are accessible only through a single line. In other embodiments, the unit may defer process execution until it is able to fetch all the necessary data. As a result, overall performance may be degraded. However, the compiler can find settings and data placements that can avoid delays.

In some embodiments, the compiler may configure the configuration manager 2304, memory controller 2306, and accelerator 2302 to correspond to situations where multiple addresses must be accessed from a single memory block, but there is only one port in the memory block. You can create settings or commands for 2300 . For example, the compiler may relocate data in the memory block 2308 so that the processing unit can access multiple lines of the memory block 2308 .

Also, the memory controller 2306 may act on more than one input simultaneously. For example, the memory controller 2306 may allow access to one of the memory blocks 2308 through one port and may allow the supply of data while receiving a request from the other memory block at the other input. Thus, as a result of this operation, the accelerator 2216 may be subjected to an exemplary 2D-convolution operation that receives data from a dedicated line of communication with the corresponding memory block.

Additionally or alternatively, the memory controller 2306 or logic block may maintain a refresh counter for each memory block and process the refresh of all lines. The presence of such a counter allows the memory controller 2306 to insert a refresh cycle between dead access times from the device.

Also, it may be possible for the memory controller 2306 to perform a pipelined memory access to receive an address before supplying data and set it to open a line to the memory block. Pipelined memory access can provide data to processing without interruption or delayed clock cycles. For example, the memory controller 2306 or one of the logic blocks may access data on the right line of FIG. 23 , while data may be transferred on the left line. This method will be described in detail with reference to FIG. 26 .

In response to the required data, processing device 2300 may utilize a multiplexer and/or other switching device to select which device will perform a given task. For example, the configuration manager 2304 may configure the multiplexer so that at least two data lines run to the MAC unit 2302 . This can speed up operations that require data from multiple addresses, such as 2D-convolution, because vectors or words that require multiplication during convolution can reach the processing unit simultaneously in a single clock. Due to this data transmission method, a processing unit such as the accelerator 2216 can quickly output a result.

In some embodiments, the settings manager 2304 may be capable of setting processes to run based on the priority of the task. For example, the settings manager 2304 can be set to complete the running process without interruption. In this case, the settings manager 2304 can provide the accelerator 2216 with a set of commands or tasks to execute without interruption, and only switch the multiplexer when the task is complete. However, in other embodiments, the configuration manager 2304 may abort the task and re-establish data routing when receiving a priority task, such as a request from an external interface. However, if there are enough memory blocks 2308, the memory controller 2306 can be set up to route data or allow access to processing units with dedicated lines that do not have to change until the operation is complete. Also, in some embodiments, all devices may be connected to the entrance of configuration manager 2304 by a bus, and devices may manage access between devices and the bus (eg, utilize the same logic as a multiplexer). Accordingly, the memory controller 2306 may be directly connected to a plurality of memory instances or memory blocks.

Alternatively, the memory controller 2306 may be directly connected to a memory sub-instance. In some embodiments, each memory instance or block may be constructed from sub-instances (eg, DRAM may be constructed from mats in which individual data lines are arranged into multiple sub-blocks). Instances may also include DRAM mats, DRAMs, banks, flash mats, SRAM mats, or other types of memory. Thereafter, the memory controller 2306 may include a dedicated line to the address sub-instance to directly minimize the delay during pipelined memory access.

In some embodiments, the memory controller 2306 also has the logic required for a particular memory instance (row/column decoder, refresh logic, etc.), and the memory block 2308 can handle its own logic. Thus, the memory block 2308 can obtain an address and generate commands for output/write data.

24 shows an exemplary memory configuration diagram in accordance with the disclosed embodiment. In some embodiments, a compiler that generates code or settings for the processing unit 2200 may perform a method of pre-locating data into each memory block to establish loading from the memory blocks 2202 and 2204 . For example, the compiler can pre-place data so that each word needed for an operation is correlated to a line of memory instance or memory block. However, for tasks that require more memory blocks than the available memory blocks of the processing unit 2200, the compiler may implement a method of fitting data into one or more memory locations in each memory block. The compiler can also store the data as a sequence and evaluate the delay of each memory block to avoid missing line delays. In some embodiments, the host may be part of a processing unit such as the configuration manager 2212 , but in other embodiments, the compiler host may be connected to the processing unit 2200 through an external interface. In such an embodiment, the host may execute compilation functions such as those described for the compiler.

In some embodiments, the settings manager 2212 may be a CPU or microcontroller (uC). In such an embodiment, the settings manager 2212 may need to access the memory and fetch the commands placed in the memory. Certain compilers can generate and place code in memory in such a way that successive instructions are stored on the same memory line and in multiple memory banks, allowing pipelined memory access for fetched instructions. In such an embodiment, the configuration manager 2212 and the memory controller 2210 may be able to avoid row latencies in linear execution by enabling pipelined memory accesses.

In the previous case method of linear execution of programs, the compiler recognized and placed the instructions, resulting in pipelined memory execution. However, other software architectures may be more complex and may require the compiler to recognize the instructions and act accordingly. For example, if a task requires loops and branches, the compiler can place all the loop code inside a single line so that a single line repeats without delaying line-opening. Accordingly, the memory controller 2210 may not need to change the line during execution.

In some embodiments, the settings manager 2212 may include internal caching or small memory. Internal caching can handle branches and loops by storing instructions executed by configuration manager 2212 . For example, instructions in the internal caching memory may include instructions to set up an accelerator to access a block of memory.

25 is an exemplary flow chart illustrating a memory setup process 2500 in accordance with the disclosed embodiment. For convenience of description of the memory setup process 2500, reference may be made to the components illustrated in FIG. 22 described above. In some embodiments, process 2500 may be executed by a compiler that provides instructions to a connected host through an external interface. In other embodiments, process 2500 may be executed by a component of processing device 2200 , such as settings manager 2212 .

In general, process 2500 includes determining the number of words required simultaneously to perform an operation, determining the number of words that can be accessed simultaneously from each of a plurality of memory banks, and the number of words required concurrently being accessed simultaneously. partitioning among multiple memory banks the number of words needed at the same time if it is greater than the number of words that can be. Also, dividing the number of words required at the same time may include executing a word of a cyclic structure and sequentially allocating one word per memory bank.

More specifically, process 2500 may begin with step 2502 where a compiler may receive a job specification. This specification may include required computational and/or priority levels.

At step 2504 , the compiler may identify an accelerator or group of accelerators that may perform a task. Alternatively, the compiler may generate a command so that a processing unit such as the configuration manager 2212 can identify an accelerator to perform an operation. For example, utilizing the required calculation, the configuration manager can identify accelerators that can perform tasks in the group of accelerators 2216 .

In step 2506, the compiler can determine the number of words that must be accessed concurrently to execute the operation. For example, multiplication of two vectors requires access to at least two vectors, so the compiler can determine that vector words must be accessed simultaneously to perform the operation.

At step 2508 , the compiler may determine the number of cycles required to execute the task. For example, if an operation requires a convolution operation of four by-products, the compiler may determine that at least 4 cycles will be required to perform the operation.

In step 2510, the compiler may place words that need to be accessed concurrently into different memory banks. Accordingly, the memory controller 2210 may be configured to open a line to different memory instances and access a required memory block within a clock cycle without requiring data stored in the cache.

At step 2512 , the compiler may place sequentially accessed words into the same memory bank. For example, if four cycles of operation are required, the compiler can generate instructions to write the necessary words in sequential cycles to a single memory block to avoid line changes between different memory blocks during execution.

In step 2514 , the compiler may generate an instruction to program a processing unit such as the configuration manager 2212 . Commands can specify conditions that operate a switching device (eg, a multiplexer) or establish a data bus. With this command, the configuration manager 2212 may configure the memory controller 2210 to route data from the memory block to the processing unit using a dedicated communication line according to the task or to allow access to the memory block.

26 is an exemplary flow diagram illustrating a memory read process 2600 in accordance with the disclosed embodiment. For convenience of description of the memory read process 2600 , reference may be made to the components illustrated in FIG. 22 described above. In some embodiments, as described below, process 2600 may be performed by memory controller 2210 . However, in other embodiments, process 2600 may be implemented by other components of processing device 2200 , such as settings manager 2212 .

In step 2602 , the memory controller 2210 , the configuration manager 2212 , or other processing unit may receive a request to route data to or allow access to the memory bank. This request can specify an address and a block of memory.

In some embodiments, the request may be received over a data bus specifying a read command on line 2218 and an address on line 2220. In another embodiment, the request may be received through a demultiplexer connected to the memory controller 2210 .

In step 2604, the configuration manager 2212, host, or other processing unit may query the internal registers. The internal registers may contain information regarding the opened line into the memory bank, the opened address, the opened memory block, and/or the next operation. Based on the information in the internal register, it may be determined whether there is an open line to the memory bank and/or whether the memory block has received the request in step 2602 . Alternatively or additionally, the memory controller 2210 may directly query the internal registers.

If the internal register indicates that no memory bank is loaded on the opened line (i.e., no in step 2606), process 2600 proceeds to step 2616 where the line may be loaded into the memory bank associated with the received address. have. In addition, a processing unit such as the memory controller 2210 or the setting manager 2212 may send a delay signal to a component requesting information from the memory address in operation 2616 . For example, when the accelerator 2216 requests memory information located in a memory block that is already in use, the memory controller 210 may send a delay signal to the accelerator in operation 2618 . In operation 2620 , the configuration manager 2212 or the memory controller 2210 may update an internal register to indicate that a line is opened to a new memory bank or a new memory block.

If the internal register indicates that the open line is loaded with a memory bank (ie, 'Yes' at step 2606 ), the process 2600 may proceed to step 2608 . In step 2608, it may be determined whether the line loaded into the memory bank is being used for another address. If the line is being used at a different address (ie 'yes' in step 2608), it may indicate that there are two instances in a single block, and thus cannot be accessed simultaneously. Accordingly, an error or exemption signal may be transmitted from the memory address to the component requesting information in step 2616 . However, if the line is not being used for another address (i.e., no at step 2608), then the line can be opened to the address and fetch data from the target memory bank and proceed to step 2614 to request information from the memory address. You can send data to the component.

Through the process 2600 , the processing unit 2200 may establish a direct connection between the processing unit and a memory block or memory instance containing information necessary to perform a task. This organization of data makes it possible not only to read information from vectors constructed in different memory instances, but also to simultaneously retrieve information from different memory blocks when a device requests a plurality of such addresses.

27 is an exemplary flow diagram illustrating an execution process 2700 in accordance with a disclosed embodiment. For convenience in describing the execution process 2700 , reference may be made to the components illustrated in FIG. 22 described above.

At step 2702 , a local unit such as a compiler or configuration manager 2212 may receive a request for work to be performed. Operations can involve single operations (eg, multiplication) or complex operations (eg, convolution between matrices). Tasks can also represent required calculations.

In step 2704, the compiler or configuration manager 2212 may determine the number of words simultaneously required to perform the operation. For example, the configuration manager or compiler may determine that two words are needed simultaneously to perform multiplication between vectors. As another example, in the 2D convolution operation, the setting manager 2212 may determine that the word of 'n' times 'm', where 'n' and 'm' are matrix dimensions, is required for convolution between matrices. have. In step 2704, the settings manager 2212 may also determine the number of cycles required to perform the task.

In step 2706, according to the determination of step 2704, the compiler may write words to be accessed simultaneously to a plurality of memory banks disposed on the substrate. For example, if the number of words that can be accessed simultaneously from a plurality of memory banks is less than the number of words needed at the same time, the compiler can place the data in multiple memory banks so that the different words needed can be accessed within a clock. Also, if setup manager 2212 or the compiler determines that multiple cycles are required to perform an operation, the compiler writes the necessary words in sequential cycles to a single memory bank of multiple memory banks to prevent line changes between memory banks. can do.

In operation 2708 , the memory controller 2210 reads at least one first word from a first memory bank of a plurality of memory banks or blocks by using the first memory line or allows access to the at least one first word can be set.

In step 2170 , a processing unit, such as one of the accelerators 2216 , may process the job utilizing the at least one first word.

In operation 2712 , the memory controller 2210 may be set to open a second memory line in the second memory bank. For example, using a task-based and pipelined memory approach, the memory controller 2210 may be configured to open a second memory line to the second memory block in which the information required for the task is written in step 2706 . . In some embodiments, the second memory line may be opened when the operation of step 2170 is about to be completed. For example, if 100 clocks are required for the operation, the second memory line may be opened at the 90th clock.

In some embodiments, steps 2708 through 2712 may be executed within one line access cycle.

In operation 2714 , the memory controller 2210 may be configured to allow access to data of at least one second word of the second memory bank by utilizing the second memory line opened in operation 2710 .

In step 2176 , a processing unit such as one of the accelerators 2216 may utilize the at least one second word to process the task.

In operation 2718 , the memory controller 2710 may be set to open a second memory line in the first memory bank. For example, using a task-based and pipelined memory approach, the memory controller 2210 may be configured to open a second memory line to a first memory block. In some embodiments, the second memory line to the first block may be opened when the operation of step 2176 is about to complete.

In some embodiments, steps 2714 through 2718 may be executed within one line access cycle.

In step 2720, the memory controller utilizes the second memory line of the first bank or the first line of the third bank and proceeds to another memory bank to read at least one third word from the first memory bank of the plurality of memory banks Alternatively, access to at least one third word may be allowed.

Partial Respect

Some memory chips, such as DRAM chips, utilize refresh to prevent loss of stored data (eg, utilization of capacitance) due to voltage decay on the chip's capacitors or other electrical components. For example, in DRAM, each cell must be refreshed from time to time (depending on a particular process or design) to restore charge to the capacitor so that data is not lost or corrupted. As the memory capacity of DRAM chips increases, a significant amount of time is required to refresh the memory. As long as the memory of a particular line is being refreshed, the bank containing the line being refreshed is inaccessible. This can lead to poor performance. In addition, the power associated with the refresh process is also significant. Previous efforts have been made to reduce the rate at which refreshes are performed to reduce the adverse effects associated with memory refreshes, but most of these efforts have focused only on the physical layer of DRAM.

A refresh operation is similar to reading and writing a row in memory. Utilizing this principle and focusing on access patterns to the memory, embodiments of the present disclosure include modifications to the memory chip and software and hardware methods to use less power to refresh and reduce the time it takes for the memory to be refreshed. For example, to outline, some implementations may use hardware and/or software to track line access timing and skip recently accessed rows within a refresh cycle (eg, based on a timing threshold). . In another example, some embodiments may rely on software executed by the refresh controller of the memory chip to allocate reads and writes such that accesses to the memory are not random. Accordingly, the software can more precisely control refreshes, avoiding wasted refresh cycles and/or lines. These methods can be used alone or with a compiler that encodes instructions to the refresh controller along with the machine code for the processor to ensure that accesses to memory are also not randomized. Utilizing any and all combinations of these methods and configurations detailed below, the disclosed embodiments may reduce the time it takes for a memory unit to be refreshed, thereby reducing memory refresh power requirements and/or improving system performance.

28 illustrates an exemplary memory chip 2800 including a refresh controller 2803 in accordance with the present disclosure. For example, the memory chip 2800 may include a plurality of memory banks (eg, a memory bank 2801a) on a substrate. In the example of Figure 28, the substrate includes four memory banks, each memory bank having four lines. A line represents any other collection of memory cells in the memory chip 2800, such as a wordline in one or more memory banks of the memory chip 2800, or some or all of a row along a memory bank or group of memory banks. can mean

In other embodiments, the substrate may include any number of memory banks, and each memory bank may include any number of lines. Some memory banks may include the same number of lines (eg, for FIG. 28 ), while other memory banks may include a different number of lines. As shown in FIG. 28 , the memory chip 2800 may include a controller 2805 to receive input to the memory chip 2800 and transmit an output from the memory chip 2800 (see 'code division' above). described in).

In some embodiments, the plurality of memory banks may include dynamic random access memory (DRAM). However, the plurality of memory banks may include any volatile memory that stores data requiring periodic refresh.

As will be described in greater detail below, the disclosed embodiments may utilize a counter or resistor-capacitor circuit to perform the timing of the refresh cycle. For example, if a counter or timer reaches its target value after counting since the last full refresh cycle, another counter can be utilized to iterate through all rows. Embodiments of the present disclosure may additionally track access to segments of the memory chip 2800 and reduce refresh power required. For example, although not shown in FIG. 28 , the memory chip 2800 may further include data storage configured to store access information indicative of an access operation for one or more segments of a plurality of memory banks. For example, one or more segments may include any portion of any line, column, or any other group of memory cells within the memory chip 2800 . In one particular example, the one or more segments may include at least one row of memory structures in a plurality of memory banks. The refresh controller 2803 may be configured to perform a refresh operation of one or more segments based at least in part on the stored access information.

For example, data storage may include one or more registers, SRAM cells, etc. associated with a segment of memory chip 2800 (eg, a line, column, or any other group of memory cells within memory chip 2800 ). have. Additionally, the data storage may be configured to store a bit indicating whether the associated segment has been accessed in one or more previous cycles. A 'bit' may include any arbitrary data structure that stores at least one bit, such as a register, SRAM cell, non-volatile memory, and the like. A bit can also be set by setting a corresponding switch (or a switching element such as a transistor) in the data structure to ON (which may be equivalent to '1' or 'true'). Additionally, or alternatively, a bit modifies any other property in the data structure to write a '1' (or any other value indicating the setting of the bit) to the data structure (e.g., the floating gate of a flash memory). charging, modifying the state of one or more flip-flops in the SRAM, etc.). When it is determined that the bit is set as part of the refresh operation of the memory controller, the refresh controller 2803 may skip the refresh cycle of the associated segment and empty the register associated with the portion.

In another example, data storage is one or more non-volatile memory (eg, flash memory, etc.) associated with a segment of memory chip 2800 (eg, a line, column, or any other group of memory cells within memory chip 2800 ). ) may be included. The non-volatile memory may be configured to store a bit indicating whether the associated segment has been accessed in one or more previous cycles.

Some embodiments, additionally or alternatively, a timestamp register on each row or group of rows (or other segment of memory chip 2800 ) for which the line has the last tick within the current refresh cycle accessed. can be added. This means that on each row access, the refresh controller can update the row timestamp register. Thus, the next time a refresh occurs (e.g., at the end of a refresh cycle), the refresh controller compares the stored timestamp, and the associated segment is within a certain threshold within a certain time period (e.g., within a certain threshold as previously applied to the stored timestamp). ), the refresh controller can skip to the next segment. This prevents the system from using refresh power on recently accessed segments. In addition, the refresh controller can keep track of access as each segment is accessed or refreshed in the next cycle.

Accordingly, in another example, data storage may include one or more registers or non-volatile memory associated with a segment of memory chip 2800 (eg, a line, column, or any other group of memory cells within memory chip 2800 ). may include Rather than using bits to indicate whether an associated segment was accessed, a register or non-volatile memory may be configured to store a timestamp or other information indicating the most recent access of the associated segment. In this example, the refresh controller 2803 determines that the amount of time between the current time and the timestamp stored in the associated register or memory (eg, the time from the timer as described in FIGS. 29A and 29B below) is a predetermined threshold. Based on whether a value (eg, 8ms, 16ms, 32ms, 64ms, etc.) is exceeded, it is possible to determine whether to refresh or access the associated segment.

Accordingly, the predetermined threshold may include an amount of time for a refresh cycle that causes the associated segment to be refreshed (if not accessed) at least once per refresh cycle. Alternatively, the predetermined threshold may include an amount of time less than the time required for the refresh cycle (eg, to ensure that any required refresh or access signals reach the associated segment before the refresh cycle is complete). ). For example, the predetermined time could be 7 ms for a memory chip with a refresh period of 8 ms, such that if the segment is not accessed within 7 ms, the refresh controller will send a refresh or access signal arriving at the segment before the end of the 8 ms refresh period. can be In some embodiments, the predetermined threshold may depend on the size of the associated segment. For example, the predetermined threshold value may become smaller as the segment of the memory chip 2800 is smaller.

Although described above with reference to a memory chip, the refresh controller of the present disclosure may also be used in distributed processor architectures such as those described above and throughout this disclosure. An example of such an architecture is shown in FIG. 7A . In such an embodiment, the same substrate as the memory chip 2800 may include a plurality of processing groups disposed thereon, as shown in FIG. 7A . As described above with reference to FIG. 3A , a 'processing group' may mean two or more processor subunits and a memory bank on a substrate corresponding to the subunits. A processing group may represent a logical grouping for purposes of spatial distribution on a substrate and/or compilation of code for execution on a memory chip 2800 . Accordingly, the substrate may include a memory array including a plurality of banks, such as the bank 2801a shown in FIG. 28 and other banks. The substrate may also include a processing array that may include a plurality of processor subunits (eg, subunits 730a, 730b, 730c, 730d, 730e, 730f, 730g, 730h shown in FIG. 7A ).

As further described above with reference to FIG. 7A , each processing group may include a processor subunit and a corresponding one or more memory banks dedicated to the processor subunit. Further, to enable each processor subunit to communicate with a corresponding dedicated memory bank, the substrate may include a first plurality of buses connecting one of the processor subunits to a corresponding dedicated memory bank.

In such an embodiment, as shown in FIG. 7A , the substrate may include each processor subunit with at least one other processor subunit (eg, an adjacent subunit in the same row, an adjacent processor subunit in the same column, or any other processor subunit on the substrate). and a second plurality of buses for connecting to other processor subunits). As described above in the 'Synchronization with software' section, the first and/or second plurality of buses such that data transfer between the processor subunits and over corresponding buses of the plurality of buses is not controlled by the timing hardware logic element. may have no timing hardware.

In embodiments in which the same substrate as memory chip 2800 may include a plurality of processing groups (eg, as shown in FIG. 7A ) disposed thereon, the processor subunit may include an address generator (eg, as shown in FIG. 4 ). (450)) may be further included. Additionally, each processing group may include a processor subunit and a corresponding one or more memory banks dedicated to the processor subunit. Accordingly, each address generator may be associated with a corresponding dedicated memory bank of the plurality of memory banks. The substrate may also include a plurality of buses each coupling one of the plurality of address generators to a corresponding dedicated memory bank.

29A illustrates an exemplary refresh controller 2900 in accordance with the present disclosure. The refresh controller 2900 may be included in a memory chip of the present disclosure, such as the memory chip 2800 of FIG. 28 . 29A , the refresh controller 2900 may include a timer 2901, which may include an on-chip oscillator or any other timing circuitry for the refresh controller 2900. . In the configuration shown in FIG. 29A , the timer 2901 may trigger a refresh cycle periodically (eg, every 8 ms, 16 ms, 32 ms, 64 ms, etc.). A refresh cycle may utilize a row counter 2903 to cycle through all rows of the corresponding memory chip and a summer 2907 with an active bit 2905 to generate a refresh signal for each row. 29A, bit 2905 may be set to 1 ('true') to ensure that each row is refreshed during one cycle.

In an embodiment of the present disclosure, the refresh controller 2900 may include data storage. As previously described, data storage may include one or more registers or non-volatile memory associated with a segment of memory chip 2800 (eg, a line, column, or any other group of memory cells within memory chip 2800 ). can A register or non-volatile memory may be configured to store a timestamp or other information indicating the most recent access of the associated segment.

The refresh controller 2900 may skip the refresh of the segment of the memory chip 2800 by using the stored information. For example, when stored information indicates that a certain segment has been refreshed during one or more previous refresh cycles, the refresh controller 2900 may skip the corresponding segment in the current refresh cycle. In another example, when a difference between a stored timestamp for a certain segment and the current time is less than a threshold value, the refresh controller 2900 may skip the corresponding segment in the current refresh cycle. The refresh controller 2900 may also keep track of accesses and refreshes of segments of the memory chip 2800 through multiple refresh cycles. For example, the refresh controller 2900 may update the stored timestamp by utilizing the timer 2901 . In such an embodiment, the refresh controller 2900 may be configured to utilize the output of the timer to empty the access information stored in the data storage after a threshold time interval. For example, in an embodiment where the data storage stores the timestamp of the most recent access or refresh to the associated segment, the refresh controller 2900 may store a new timestamp whenever an access command or refresh signal is sent to the segment. can be stored in If the data storage stores bits instead of timestamps, timer 2901 may be configured to free bits set longer than a threshold time period. For example, in an embodiment where the data storage stores a bit indicating that the associated segment was accessed in one or more previous cycles, the refresh controller 2900 may indicate that the timer 2901 is more critical than the timer 2901 set in the associated bit (eg, 1). Each time you trigger a new refresh cycle after a number of cycles (eg, 1, 2, etc.), a bit in the data storage can be freed (eg, set to 0).

The refresh controller 2900 may cooperate with other hardware of the memory chip 2800 to track access of segments of the memory chip 2800 . For example, a memory chip uses a sense amplifier to perform a read operation (eg, as shown in FIGS. 9 and 10 ). The sense amplifier senses low-power signals from segments of the memory chip 2800 that store data in one or more memory cells, such that the data can be interpreted by logic, such as an external CPU or GPU or integrated processor subunit, as previously described. It may include a plurality of transistors configured to amplify a small voltage swing to a high voltage level. Although not shown in FIG. 29A , the refresh controller 2900 may also be in communication with a sense amplifier configured to access one or more segments and change the state of at least one bit register. For example, when the sense amplifier accesses more than one segment, it may set (eg, set to 1) a bit indicating that the associated segment was accessed in a previous cycle. In embodiments where the data storage stores the timestamp of the most recent access or refresh to the associated segment, when the sense amplifier accesses one or more segments, a register, memory, or other element including data storage from timer 2901 can trigger the logging of timestamps.

In all the above-described embodiments, the refresh controller 2900 may be integrated with a memory controller for a plurality of memory banks. For example, similar to the embodiment shown in FIG. 3A , the refresh controller 2900 may be included in a logic and control subunit associated with a memory bank or other segment of the memory chip 2800 .

29B illustrates another exemplary refresh controller 2900' according to the present disclosure. The refresh controller 2900 ′ may be included in a memory chip of the present disclosure, such as the memory chip 2800 of FIG. 28 . Similar to refresh controller 2900 , refresh controller 2900 ′ includes a timer 2901 , a row counter 2903 , an active bit 2905 , and a summer 2907 . Additionally, the refresh controller 2900 ′ may include a data storage 2909 . 29B , data storage 2909 may include one or more registers associated with a segment of memory chip 2800 (eg, a line, column, or any other group of memory cells within memory chip 2800 ) or may include non-volatile memory, and may be configured to change (eg, by a sense amplifier or other element of refresh controller 2900', as described above) in response to one or more segments with which a state within the data storage is associated. have. Accordingly, the refresh controller 2900' may be configured to skip the refresh of one or more segments based on the state in the data storage. For example, if the state associated with the segment is active (eg, by turning on a switch to set it to 1, changing the property to store a '1', etc.), the refresh controller 2900' skips the refresh cycle for the associated segment. You can run and empty the state associated with that part. The state may be stored in at least one bit registers or any other memory structure configured to store at least one bit of data.

To ensure that segments of the memory chip must be refreshed or accessed during each refresh cycle, the refresh controller 2900' may reset or clear the state to trigger a refresh signal during the next refresh cycle. In some embodiments, after skipping a segment, refresh controller 2900' may flush the associated state to ensure that the segment is refreshed on the next refresh cycle. In another embodiment, the refresh controller 2900' may be configured to reset the state in the data storage after a threshold time interval. For example, when the refresh controller 2900' exceeds a threshold time after the timer 2901 has set the associated state (eg, turning on a switch to set it to 1, changing the property to store '1', etc.) The state of the data storage can be cleared (eg, set to 0) every time. In some embodiments, the refresh controller 2900' may use a threshold number of refresh cycles (eg, 1, 2, etc.) or a threshold number of clock cycles (eg, 2, 4, etc.) instead of a threshold time.

In another embodiment, the status includes a timestamp of the most recent refresh or access of the associated segment, such that the amount of time between the timestamp and the current time (eg, the time from timer 2901 of FIGS. 29A and 29B ) is When a predetermined threshold (e.g., 8ms, 16ms, 32ms, 64ms, etc.) is exceeded, the refresh controller 2900' sends an access command or refresh signal to the associated segment and updates the timestamp associated with that segment (e.g., A timer 2901 may be used). Additionally or alternatively, the refresh controller 2900' may be configured to skip refresh operations on one or more segments of the plurality of memory banks when the refresh time indicator indicates a last refresh time within a predetermined time threshold. have. In such embodiments, after skipping a refresh operation for one or more segments, the refresh controller 2900' may be configured to change the stored refresh time indicator associated with the one or more segments to cause the one or more segments to be refreshed during the next cycle of operation. can For example, as described above, the refresh controller 2900 ′ may update the stored refresh time indicator using the timer 2901 .

Accordingly, the data storage may include a timestamp register configured to store a refresh time indicator indicating a time at which one or more segments of the plurality of memory banks were last refreshed. Also, the refresh controller 2900 ′ may use the output of the timer to empty the access information stored in the data storage after a threshold time interval.

In all of the embodiments described above, accessing one or more segments may include a write operation associated with the one or more segments. Additionally or alternatively, accessing the one or more segments may include a read operation associated with the one or more segments.

Further, as shown in FIG. 29B , the refresh controller 2900 ′ includes a summer 2907 and a row counter 2903 configured to aid in the updating of the data storage 2909 based, at least in part, on a state within the data storage. may include Data storage 2909 may include a bit table associated with a plurality of memory banks. For example, a bit table may include an array of switches (or switching elements, such as transistors) or registers (eg, SRAM, etc.) configured to have bits for an associated segment. Additionally or alternatively, data storage 2909 may store timestamps associated with the plurality of memory banks.

Additionally, the refresh controller 2900' may include a refresh gate 2911 configured to control whether a refresh to one or more segments will occur based on corresponding values stored in the bit table. For example, refresh gate 2911 invalidates the refresh signal from row counter 2903 if the corresponding state of data storage 2909 indicates that the associated segment has been refreshed or accessed during one or more previous clock cycles. It may include a logic gate (eg, an 'and' gate) configured to In another embodiment, the refresh gate 2911 triggers a refresh signal from the row counter 2903 when the corresponding timestamp from the data storage 2909 indicates that the associated segment has been refreshed or accessed within a predetermined threshold time value. may include a microprocessor or other circuitry configured to override the

30 is an exemplary flow diagram of a process 3000 for a partial refresh of a memory chip (eg, memory chip 2800 of FIG. 28 ). Process 3000 may be executed by a refresh controller according to the present disclosure, such as refresh controller 2900 of FIG. 29A or refresh controller 2900' of FIG. 29B.

In operation 3010, the refresh controller may access information indicating an access operation for one or more segments of the plurality of memory banks. For example, as previously described with reference to FIGS. 29A and 29B , the refresh controller may be configured to configure a segment of memory chip 2800 (eg, a line, column, or any other group of memory cells within memory chip 2800 ). and data storage configured to store a timestamp or other information associated with the associated segment indicative of a most recent access of the associated segment.

In operation 3020 , the refresh controller may generate a refresh and/or access command based at least in part on the accessed information. For example, as previously described with reference to FIGS. 29A and 29B , the refresh controller may be configured to indicate that the information accessed is the last refresh or access time within a predetermined time threshold and/or the last refresh that occurred during one or more previous clock cycles or When indicating access, refresh operations on one or more segments of the plurality of memory banks may be skipped. Additionally or alternatively, the refresh controller is configured to determine whether a most recent refresh or access time indicated by the accessed information exceeds a predetermined threshold and/or if the most recent refresh or access did not occur during one or more previous clock cycles. In some cases, it may generate a command to refresh or access the associated segment.

At step 3030, the refresh controller may change the stored refresh time indicator associated with the one or more segments to cause the one or more segments to be refreshed during the next cycle of operation. For example, after skipping a refresh operation for one or more segments, the refresh controller may change the information indicative of an access operation for one or more segments to cause the one or more segments to be refreshed during the next clock cycle. Accordingly, the refresh controller can flush (eg set to zero) the state for the segment after skipping the refresh cycle. Additionally or alternatively, the refresh controller may set (eg, set to 1) the state for segments being refreshed and/or accessed during the current cycle. In embodiments in which the information indicative of an access operation for one or more segments includes timestamps, the refresh controller may update any stored timestamps associated with the segments being refreshed and/or accessed during the current cycle.

Method 3000 may further include additional steps. For example, in addition to or alternatively to step 3030, the sense amplifier may access one or more segments and change information associated with the one or more segments. Additionally or alternatively, the sense amplifier may signal to the refresh controller when an access occurs, causing the refresh controller to update information associated with one or more segments. As previously described, the sense amplifier senses a low-power signal from a segment of a memory chip that stores data in one or more memory cells, where the data is to be interpreted by logic, such as an external CPU or GPU or integrated processor subunit, as previously described. It may include a plurality of transistors configured to amplify a small voltage swing to a high voltage level. In this example, whenever the sense amplifier accesses one or more segments, it may set a bit associated with the segment (eg, set to 1) to indicate that the associated segment was accessed in a previous cycle. In embodiments in which the information indicative of an access operation for one or more segments includes a timestamp, whenever the sense amplifier accesses one or more segments, it triggers a write of the timestamp from a timer in the refresh controller to data storage to be associated with the segment. Any arbitrary stored timestamp can be updated.

31 is an exemplary flow diagram of a process 3100 for determining a refresh for a memory chip (eg, memory chip 2800 of FIG. 28 ). Process 3100 may be implemented within a compiler in accordance with the present disclosure. As described above, a 'compiler' is a high-level language (eg, procedural languages such as C, FORTRAN, BASIC, etc.; object-oriented languages such as Java, C++, Pascal, Python, etc.) and low-level languages (eg assembly code, object code). , machine code, etc.) Compilers can enable humans to program a set of instructions in a human-readable language, which are later translated into a machine-executable language. A compiler may include software instructions that are executed by one or more processors.

At step 3110, the one or more processors may receive high-level computer code. For example, high-level computer code may be encoded in one or more files on a memory (eg, non-volatile memory such as a hard disk drive, volatile memory such as DRAM, etc.) or received over a network (eg, the Internet, etc.). Additionally or alternatively, high-level computer code may be received from a user (eg, utilizing an input device such as a keyboard).

At step 3120 , the one or more processors may identify a plurality of memory segments distributed across a plurality of memory banks associated with a memory chip to be accessed by the high-level computer code. For example, one or more processors may have access to data structures defining a plurality of memory banks and corresponding structures in a memory chip. The one or more processors may access data structures from memory (eg, non-volatile memory, such as a hard disk drive, etc., volatile memory, such as DRAM, etc.) or receive data structures over a network (eg, the Internet, etc.). In such embodiments, data structures may be included in one or more libraries that may be accessed by the compiler, allowing the compiler to generate instructions for a particular memory chip to be accessed.

At step 3130, the one or more processors may access high-level computer code to identify a plurality of memory read commands to occur in a plurality of memory access cycles. For example, the one or more processors may identify each operation within the high-level computer code that requires one or more read instructions from and/or one or more write instructions to the memory. These instructions may include variable initialization, variable reallocation, logical operation of a variable, input-output operation, and the like.

At step 3140 , the one or more processors may cause distribution of data associated with the plurality of memory access instructions through each segment of the plurality of memory segments such that each segment of the plurality of memory segments is accessed during each cycle of the plurality of memory access cycles. . For example, one or more processors may identify a memory segment from a data structure that defines the structure of a memory chip and then assign variables from high-level code to the various segments of the memory segment so that each memory segment undergoes each refresh cycle (a certain number of clock cycles). may be accessed (e.g., via write or read) at least once during In this example, the one or more processors may cause each line of high-level code to allocate a variable from a line of high-level code such that each memory segment is accessed (eg, via a write or read) at least once during a specified number of clock cycles. You have access to information representing the number of clock cycles.

In another example, the one or more processors may first generate machine code or other low-level code from the high-level code. One or more processors then assign variables from low-level code to the various segments of the memory segment so that each memory segment is accessed (e.g., writes or reads) at least once during each refresh cycle (which may contain a certain number of clock cycles). through) can be done. In this example, each line of low-level code may require a single clock cycle.

In all of the above embodiments, the one or more processors may further allocate logical operations or other instructions that use temporary outputs to the various segments of the memory. The result of this temporary output may still be read and/or write commands, allowing the allocated memory segment to still be accessed during refresh cycles, even if the named variable has not yet been assigned to that memory segment.

Method 3100 may further include additional steps. For example, in embodiments where variables are assigned prior to compilation, one or more processors may generate machine code or other low-level code from high-level code. Additionally, the one or more processors may transmit the compiled code for execution by the memory chips and corresponding logic circuits. The logic circuit may include conventional circuitry such as GPU or CPU, or it may include processing groups on the same substrate as a memory chip as shown in FIG. 7A . Accordingly, as described above, the substrate may include a memory array including a plurality of banks, such as the bank 2801a shown in FIG. 28 and other banks. The substrate may also include a processing array including a plurality of processor subunits (eg, subunits 730a , 730b , 730c , 730d , 730e , 730f , 730g , 730h illustrated in FIG. 7A ).

32 is another exemplary flow diagram of a process 3200 for determining a refresh for a memory chip (eg, memory chip 2800 in FIG. 28 ). Process 3200 may be implemented within a compiler in accordance with the present disclosure. Process 3200 may be executed by one or more processors executing software instructions, including compilers. Process 3200 may be performed separately or in conjunction with process 3100 of FIG. 31 .

At step 3210, similar to step 3110, one or more processors may receive high-level computer code. At step 3220, similar to step 3120, the one or more processors may identify a plurality of memory segments distributed across a plurality of memory banks associated with a memory chip to be accessed by high-level computer code.

In step 3230, the one or more processors may evaluate the high-level computer code to identify a plurality of memory read instructions each associating one or more of the plurality of memory segments. For example, one or more processors may identify each operation in the high-level computer code that requires one or more read instructions from and/or one or more write instructions to memory. These instructions may include variable initialization, variable reallocation, logical operation of a variable, input-output operation, and the like.

In some embodiments, one or more processors may utilize logic circuitry and multiple memory segments to simulate execution of high-level code. For example, the simulation may include line-by-line step-through of high-level code, similar to a debugger or other instruction set simulator (ISS). Similar to a debugger that maintains an internal variable representing a register of a processor, the simulation may further maintain an internal variable representing the address of a plurality of memory segments.

At step 3240 , the one or more processors may track, for each memory segment of the plurality of memory segments, an amount of time accumulated since the last access to the memory segment based on the analysis of the memory access instruction. For example, utilizing the simulation described above, the one or more processors may determine the length of time between each access (eg, read or write) to one or more addresses within each segment of the plurality of memory segments. The length of time can be measured in absolute time, clock cycles, or refresh cycles (eg, determined by the known refresh rate of the memory chip).

In step 3250, in response to determining that the amount of time since last access to any particular memory segment will exceed a predetermined threshold, the one or more processors issue a memory refresh instruction configured to cause an access to the particular memory segment; At least one of the memory access instructions may be introduced into high-level computer code. For example, the one or more processors may include a refresh command for execution by a refresh controller (eg, refresh controller 2900 of FIG. 29A or refresh controller 2900' of FIG. 29B ). In embodiments where the logic circuitry is not embedded on the same substrate as the memory chip, the one or more processors may generate refresh commands for transmission to the memory chip separately from the low-level code for transmission to the logic circuitry.

Additionally or alternatively, one or more processors may include access instructions for execution by a memory controller (which may be separate or integral to the refresh controller). The access instruction may include a dummy instruction configured to trigger a read operation on the memory segment but prevent the logic circuit from performing further operations on the variable that has been read or written to the memory segment.

In some embodiments, the compiler may include a combination of the steps of process 3100 and process 3200 . For example, the compiler may run the simulation described above after allocating the variables according to step 3140 to add any additional memory refresh instructions or memory access instructions according to step 3250 . With this combination, the compiler is able to spread variables across as many memory segments as possible and generate refresh or access instructions for any arbitrary memory segment that cannot be accessed within a predetermined threshold amount of time. In another example combination, the compiler may simulate code according to step 3230 and assign a variable according to step 3140 based on any and all segments that the simulation indicates will not be approached within a predetermined threshold amount of time. In some embodiments, this combination may further include step 3250, such that, even after the allocation according to step 3140 is complete, the compiler may generate a refresh or access instruction for a memory segment that cannot be accessed within a predetermined threshold amount of time. .

With the refresh controller of the present disclosure, software executed by a logic circuit (a processing group on the same substrate as a conventional logic circuit such as a CPU and a GPU or a memory chip as shown in Fig. 7A) is automatically executed by the refresh controller You can disable refreshes and control them through software running instead. Accordingly, some embodiments of the present disclosure may provide software having a known memory chip access pattern (eg, when a compiler accesses a data structure defining a plurality of memory banks or a corresponding structure of a memory chip) ). In such an embodiment, the post-compiling optimizer can disable automatic refreshes and manually set refresh controls only for segments of the memory chip that have not been accessed within a threshold amount of time. Thus, similar to step 3250 described above, but after compilation, the post-compile optimizer may generate a refresh instruction that ensures that each memory segment is necessarily accessed or refreshed for a predetermined threshold amount of time.

Another example of reducing refresh cycles is accessing predefined patterns to the memory chip. For example, where software executed by logic circuitry can control access patterns to a memory chip, some embodiments can create access patterns for refresh beyond conventional linear line refreshes. For example, if, at the judgment of the controller, software executed by the logic circuit periodically accesses every two rows of the memory, the refresh controller of the present disclosure refreshes every two lines in order to increase the speed of the memory chip and reduce power consumption. You can use an access pattern that doesn't.

An example of such a refresh controller is shown in FIG. 33 . 33 illustrates an exemplary refresh controller 3300 configured by a stored pattern according to the present disclosure. The refresh controller 3300 may be included in the memory chip of the present disclosure. For example, a plurality of memory banks and a plurality of memory segments may be included in each of a plurality of memory banks, such as the memory chip 2800 of FIG. 28 .

The refresh controller 3300 includes a timer 3301 (similar to timer 2901 in FIGS. 29A and 29B), a row counter 3303 (similar to row counter 2903 in FIGS. 29A and 29B), and a summer ( 3305) (similar to summer 2907 of FIGS. 29A and 29B). Also, the refresh controller 3300 includes a data storage 3307 . Unlike data storage 2909 of FIG. 29B , data storage 3307 may include at least one memory refresh pattern to be implemented in refresh of a plurality of memory segments included in each of a plurality of memory banks. For example, as shown in FIG. 33 , data storage 3307 divides segments of a memory bank into rows and/or columns, such as Li (eg, L1, L2, L3, L4 in the example of FIG. 33) and Hi ( Yes, it may include H1, H2, H3, H4) in the example of FIG. 33 . In addition, each segment has an Inci variable (e.g., Inc1 in the example of Figure 33, Inc2, Inc3, Inc4). Thus, as shown in Fig. 33, the refresh pattern is used in software to identify a range of a plurality of memory segments within a particular memory bank that will be refreshed during a refresh cycle and a range of a plurality of memory segments of a particular memory bank that will not be refreshed during a refresh cycle. It may include a table including a plurality of memory segment identifiers allocated by

Thus, data storage 3307 is a refresh that software executed by logic circuits (either conventional logic circuits such as CPUs and GPUs or processing groups on the same substrate as a memory chip as shown in FIG. 7A) may choose to use. You can define patterns. A memory refresh pattern is configured utilizing software to identify which memory segments among a plurality of memory segments in a particular memory bank will be refreshed during a refresh cycle and which memory segments among a plurality of memory segments in a particular memory bank will not be refreshed during a refresh cycle. It may be possible. Accordingly, the refresh controller 3300 may refresh some or all rows in the defined segment that are not accessed during the current cycle according to Inci. The refresh controller 3300 may skip another row of a defined segment set to be accessed during the current cycle.

In an embodiment where the data storage 3307 of the refresh controller 3300 includes a plurality of memory refresh patterns, each memory refresh pattern includes a different refresh pattern for refreshing the plurality of memory segments included in each of the plurality of memory banks. can indicate The memory refresh pattern may be selectable for use on a plurality of memory segments. Accordingly, the refresh controller 3300 may be configured to enable selection of a memory refresh pattern to be implemented from among a plurality of memory refresh patterns during a specific refresh cycle. For example, software executed by a logic circuit (a processing group on the same substrate as a conventional logic circuit such as a CPU and a GPU or a memory chip as shown in Figure 7A) may have different memory refresh patterns to be used during one or more different refresh cycles. can be selected. Alternatively, software executed by the logic circuit may select one memory refresh pattern to use during some or all of the different refresh cycles.

The memory refresh pattern may be encoded utilizing one or more variables stored in data storage 3307 . For example, in embodiments in which a plurality of memory segments are arranged in rows, each memory segment identifier may be configured to identify a specific location within the row of memory at which a memory refresh should begin or end. For example, in addition to Li and Hi, one or more additional variables may define which part of the row defined by Li and Hi is within a segment.

34 is an exemplary flow diagram of a process 3400 for determining a refresh for a memory chip (eg, memory chip 2800 of FIG. 28 ). Process 3100 may be implemented by software in a refresh controller according to the present disclosure (eg, refresh controller 3300 of FIG. 33 ).

In operation 3410 , the refresh controller may store at least one memory refresh pattern to be performed for refresh of a plurality of memory segments included in each of the plurality of memory banks. For example, as previously described with reference to FIG. 33, the refresh pattern defines a range of a plurality of memory segments within a particular memory bank to be refreshed during a refresh cycle and a range of a plurality of memory segments in a particular memory bank that will not be refreshed during a refresh cycle. and a table comprising a plurality of memory segment identifiers assigned by software to identify them.

In some embodiments, the at least one refresh pattern may be encoded on a refresh controller at manufacturing time (eg, onto a ROM associated with or at least accessible by the refresh controller). Accordingly, the refresh controller may access at least one refresh pattern but not store this refresh pattern.

In steps 3420 and 3430, the refresh controller utilizes software to determine which memory segment among the plurality of memory segments within a particular memory bank will be refreshed during the refresh cycle and which memory segment from among the plurality of memory segments of the particular memory bank during the refresh cycle is refreshed. It can be identified whether or not For example, as previously described with reference to FIG. 33 , software executed by a logic circuit (a processing group on the same substrate as a conventional logic circuit such as a CPU and a GPU or a memory chip as shown in FIG. 7A ) may be at least One memory refresh pattern can be selected. Also, the refresh controller may access the selected at least one memory refresh pattern and generate a corresponding refresh signal during each refresh cycle. The refresh controller may refresh some or all parts within the defined segment that are not accessed during the current refresh cycle according to at least one refresh pattern, and skip other parts of the defined segment set to be accessed during the current cycle.

In step 3440, the refresh controller may generate a corresponding refresh command. For example, as shown in FIG. 33 , summer 3305 includes logic circuitry configured to invalidate refresh signals for particular segments not to be refreshed according to at least one memory refresh pattern in data storage 3307 . can do. Additionally or alternatively, a microprocessor (not shown in FIG. 33 ) may generate a specific refresh signal based on which segments are to be refreshed according to at least one memory refresh pattern in data storage 3307 .

Method 3400 may further include additional steps. For example, an implementation configured such that at least one memory refresh pattern changes every 1, 2, or any other number of refresh cycles (eg, from L1, H1, Inc1 to L2, H2, Inc2 as shown in FIG. 33 ). In an example, the refresh controller may access different portions of the data storage for subsequent determination of the refresh signal according to steps 3430 and 3440 . Similarly, software executed by logic circuits (processing groups on the same substrate as conventional logic circuits such as CPUs and GPUs or memory chips as shown in FIG. When the storage selects a new memory refresh pattern, the refresh controller may access different portions of the data storage for subsequent determination of the refresh signal according to steps 3430 and 3440 .

Optional sized memory chips

When designing a memory chip and targeting a specific capacity of memory, changes in increasing or decreasing memory capacity may require redesigning the product and the entire mask set. Often, product design and market research occur simultaneously, and in some cases, product design is completed before market research is possible. Therefore, there may be a gap between product design and actual market demand. The present disclosure provides a method of flexibly providing a memory chip having a memory capacity corresponding to market demand. The design method involves designing a die with appropriate interconnect circuitry so that memory chips, which may include more than one die, are selected from the wafer to provide the opportunity to produce memory chips of variable size memory capacity from a single wafer. can be cut with

The present disclosure relates to a system and method for manufacturing a memory chip by cutting a memory chip from a wafer. The method can be utilized to produce memory chips of selective sizes from wafers. An exemplary embodiment of a wafer 3501 including a die 3503 is shown in FIG. 35A . The wafer 3501 is formed of a semiconductor material (eg, silicon (Si), silicon germanium (SiGe), silicon on insulator (SOI), gallium nitride (GaN), aluminum nitride (AlN), aluminum gallium nitride (AlGaN), boron nitride) BN), gallium arsenide (GaAs), gallium aluminum arsenide (AlGaAs), indium nitride (InN), and combinations thereof). Die 3503 may include any suitable circuit element (eg, transistor, capacitor, resistor, etc.) that may include any suitable semiconductor, dielectric, or metallic component. Die 3503 may be formed of a semiconductor material that may be the same as or different from that of wafer 3501 . In addition to die 3503 , wafer 3501 may include other structures and/or circuitry. In some embodiments, one or more coupling circuits may be provided and may couple one or more of the dies to each other. In an exemplary embodiment, such a coupling circuit may include a bus shared by two or more dies 3503 . Additionally, the coupling circuitry may include one or more logic circuits designed to control circuitry associated with die 3503 and/or to provide information to and retrieve information from die 3503 . In some cases, the coupling circuitry may include memory access management logic. This logic may translate a logical memory address into a physical address associated with die 3503 . Here, the term 'manufacturing' may collectively refer to all steps of constructing the disclosed waiter, die, and/or chip. For example, fabrication may refer to the simultaneous layout and formation of the various dies (and any and all other circuitry) contained on the wafer. Fabrication may also mean cutting a memory chip of an optional size from a wafer to contain one die or multiple dies as the case may be. Of course, the term manufacturing is not limited to these examples and may include other aspects associated with the creation of some or all of the disclosed memory chips and intermediate structures thereof.

A die 3503 or a group of dies may be used for manufacturing a memory chip. The memory chip may include a distributed processor as described elsewhere in this disclosure. As shown in FIG. 35B , die 3503 may include a substrate 3507 and a memory array disposed on the substrate. The memory array may include one or more memory units, such as memory banks 3511A-3511D, etc. designed to store data. In various embodiments, a memory bank may include semiconductor-based circuit elements such as transistors, capacitors, and the like. In one exemplary embodiment, a memory bank may include multiple rows and columns of storage devices. In some cases, the capacity of such a memory bank may be 1 megabyte or more. The memory bank may include DRAM or SRAM.

Die 3503 may further include a processing array disposed on a substrate, and the processing array may include a plurality of processor subunits 3515A-3515D as shown in FIG. 35B . As described above, each memory bank may include dedicated processor subunits connected by dedicated buses. For example, processor subunit 3515A is associated with memory bank 3511A via a bus or connection 3512 . Here, it will be understood that various connections are possible between the memory banks 3511A-3511D and the processor subunits 3515A-3515D, and only some of the connections are shown in FIG. 35B . In one exemplary embodiment, the processor subunit may perform read/write operations on the associated memory bank, and may further perform a refresh operation or other suitable operation on the memory stored in the various memory banks.

Die 3503 may include a first group of buses configured to couple processor subunits with corresponding memory banks. An exemplary bus may include a set of wires or conductors that connect electrical components and allow transfer of data and addresses between each memory bank and its associated processor subunit. In an exemplary embodiment, connection 3512 may serve as a dedicated bus for coupling processor subunit 3515A to memory bank 3511A. Die 3503 may include a group of such buses, each bus connecting the processor subunits to a corresponding dedicated memory bank. In addition, die 3503 may include different groups of buses, each bus capable of interconnecting processor subunits (eg, subunits 3515A-3515D). For example, such a bus may include connections 3516A-3516D. In various embodiments, data for memory banks 3511A- 3511D may be communicated via input-output bus 3530 . In one exemplary embodiment, the input-output bus 3530 may carry data-related information and instruction-related information for controlling the operation of the memory unit of the die 3503 . The data information includes data for storage in a memory bank, data read from the memory bank, processing results from one or more processor subunits based on operations performed on the data stored in the corresponding memory bank, instruction-related information, various codes, and the like. may include

In various cases, the data and commands sent by the input-output bus 3530 may be controlled by an input-output (IO) controller 35231 . In one exemplary embodiment, the IO controller 3521 may control the flow of data between the bus 3530 and the processor subunits 3515A-3515D. The IO controller 3521 may determine from which subunit of the processor subunits 3515A-3515D the information is to be retrieved. In various embodiments, the IO controller 3521 may include a fuse 3554 configured to disable the IO controller 3521 . The fuse 3554 may be used when merging multiple dies together to form a larger memory chip (also referred to as a multi-die memory chip relative to a single die memory chip that includes only a single die). Thereafter, the multi-die memory chip may use one of the IO controllers of one of the die units forming the multi-die memory chip, and use a fuse corresponding to the other IO controller to disable the other IO controllers associated with the other die unit. .

Each memory chip or processor die or group of dies may include a distributed processor associated with a corresponding memory bank. In some embodiments, such distributed processors may be arranged in a processing array disposed on the same substrate as a plurality of memory banks. Further, the processing array may include one or more logic units each including an address generator (also referred to as an address generating unit (AGU)). In some cases, the address generator may be part of at least one processor subunit. The address generator may generate the necessary memory addresses to fetch data from one or more memory banks associated with the memory chip. Address generation calculations may include addition, subtraction, modulo operations, or integer arithmetic operations such as bit shifts. The address generator may be configured to operate on several operands at once. In addition, multiple address generators can perform one or more address calculation operations at the same time. In various embodiments, an address generator may be associated with a corresponding memory bank. The address generator may be coupled to the corresponding memory bank via a corresponding bus line.

In various embodiments, different regions of wafer 3501 may be selectively cut to form selectable sized memory chips from wafer 3501 . As described, a wafer may include a group of dies 3503, which group is a group of two or more dies (eg, 2, 3, 4, 5, 10, or more dies) included on the wafer. may include As detailed below, in some cases, a single memory chip may be formed by cutting a portion of a wafer that contains only one die of a group of dies. In this case, the resulting memory chip may include a memory chip associated with one die. On the other hand, in other cases, a memory chip of a selectable size may be formed to include one or more dies. Such a memory chip may be formed by cutting a region of a wafer containing two or more dies of a group of dies included on the wafer. In this case, the die provides a multi-die memory chip with coupling circuitry that couples the die together. Some circuit elements may also be additionally coupled between the chips, such as clock elements, data buses, or any suitable logic circuitry.

In some cases, at least one controller associated with the die group may be configured to control operation of the die group with a single memory chip (eg, multiple memory unit memory chips). The controller may include one or more circuitry to manage data flow to and from the memory chip. The memory controller may be part of a memory chip or part of a separate chip that is not directly associated with the memory chip. In one exemplary embodiment, the controller may be configured to enable read and write requests or other commands associated with the distributed processor of the memory chip, and any other aspect of the memory chip (eg, refresh of the memory chip, distributed processor interaction with, etc.). In some cases, the controller may be part of die 3503 , and in other cases, the controller may be laid out adjacent to die 3503 . In various embodiments, the controller may also include at least one memory controller of at least one memory unit included on the memory chip. In some cases, the protocol used to access information on the memory chip may be independent of the replication logic and memory units (eg, memory banks) that may exist on the memory chip. Protocols can be configured to have different IDs or address ranges for proper access of data on the memory chip. Examples of chips with this protocol include a Joint Electron Device Engineering Council (JEDEC) double data rate (DDR) controller with different address ranges in different memory banks, SPI (serial) with different IDs in different memory units (e.g. memory banks). There may be a chip having a peripheral interface) or the like.

In various embodiments, multiple regions may be cut from the wafer, wherein the various regions may include one or more dies. In some cases, each separate region may be used in the formation of a multi-die memory chip. In some cases, two or more regions may be of the same shape, and the same number of dies may be coupled to the coupling circuit in the same manner. Alternatively, in one exemplary embodiment, a first group of regions may be utilized in the formation of a first type of memory chip, and a second group of regions may be utilized in the formation of a second type of memory chip. For example, a wafer 3501 as shown in FIG. 35C may include a region 3505 that may include a single die, and a second region 3504 may include two distinct groups. have. When region 3505 is cut from wafer 3501, a single die memory chip is provided. When region 3504 is cut from wafer 3501, a multi-die memory chip is provided. The groups shown in FIG. 35C are exemplary only, and various other regions and die groups may be cut from the wafer 3501 .

In various embodiments, dies may be formed on wafer 3501 , such that the dies may be arranged along one or more rows of wafers, for example as shown in FIG. 35C . The dies may share an input-output bus 3530 corresponding to one or more rows. In one exemplary embodiment, groups of dies may be cut from wafer 3501 using various cut shapes, where a shared input-output when cutting a group of dies that may be utilized in the formation of memory chips. At least a portion of the bus 3530 may be excluded (eg, only a portion of the input-output bus 3530 may be included as part of a memory chip formed including a group of dies).

As previously described, when multiple dies (eg, dies 3506A and 3506B shown in FIG. 35C ) are used to form memory chip 3517 , one IO controller corresponding to one of the dies is activated and the die may be configured to control data flow to all processor subunits of 3506A and 3506B. For example, FIG. 35D shows a memory chip 3517 including memory banks 3511A-3511H, processor subunits 3515A-3515H, IO controllers 3521A, 3521B, and fuses 3554A, 3554B. Dies 3506A and 3506B are shown combined for Here, memory chip 3517 corresponds to region 3517 of wafer 3501 prior to removal of the memory chip from the wafer. That is, in the present disclosure, the regions 3504, 3505, 3517, etc. of the wafer 3501 once cut from the wafer 3501 become memory chips 3504, 3505, 3517, and the like. In addition, a fuse may also refer to a deactivation element here. In one exemplary embodiment, fuse 3554B may be used to disable IO controller 3521B, which IO controller 3521A sends and receives data to and from processor subunits 3515A-3515H, thereby causing all memory banks 3511A- 3511H). In one exemplary embodiment, the IO controller 3521A may be coupled with the various processor subunits utilizing any and all suitable connections. In some embodiments, as described further below, the processor subunits 3515A-3515H may be interconnected, and the IO controller 3521A forms the processing logic of the memory chip 3517 processor subunit 3515A- 3515H).

In one exemplary embodiment, IO controllers, such as controllers 3521A and 3521B and corresponding fuses 3554A and 3554B, form wafer 3501 with memory banks 3511A-3511H and processor subunits 3515A-3515H. ) can be formed on the In various embodiments, when forming memory chip 3517 , dies 3506A and 3506B form memory chip 3517 that functions as a single chip and is controlled by a single input-output controller (eg, controller 3521A). One of the fuses (eg, fuse 3554B) may be deactivated to be configured to form. In one exemplary embodiment, activating the fuse may include applying a current to trigger the fuse. In various embodiments, when more than one die is used to form a memory chip, all IO controllers except one IO controller may be disabled via a corresponding fuse.

In various embodiments, as shown in FIG. 35C , multiple dies are formed on wafer 3501 with a set of input-output buses and/or control buses. An exemplary input-output bus 3530 is shown in FIG. 35C . In one exemplary embodiment, one of the input-output buses (eg, input-output bus 3530 ) may be coupled to multiple dies. 35C shows one exemplary embodiment of an input-output bus 3530 passing by dies 3506A and 3506B. The configurations of the dies 3506A and 3506B and the input-output bus 3530 as shown in FIG. 35C are exemplary only, and various other configurations may be used. For example, FIG. 35E shows a die 3540 formed on a wafer 3501 and arranged in a hexagonal formation. In one exemplary embodiment, a memory chip 3532 comprising four dies 3540 may be cut from the wafer 3501 . In one exemplary embodiment, the memory chip 3532 may include a portion of an input-output bus 3530 connected to the four dies by suitable bus lines (eg, line 3533 shown in FIG. 35E). . To pass information to the appropriate memory unit of memory chip 3532 , memory chip 3532 may include input/output controllers 3542A and 3542B disposed at junctions to input-output bus 3530 . Controllers 3542A and 3542B may select a branch of bus 3530 for receiving command data via input-output bus 3530 and transferring information to an appropriate memory unit. For example, if the command data includes read/write information in a memory unit associated with die 3546, controller 3542A receives the command request and transfers the data to the branch of bus 3530 shown in FIG. 35D. 3531A), while controller 3542B may receive the command request and send data to branch 3531B. 35E shows that various cuts in different areas can be made by expressing cut lines with broken lines.

In one exemplary embodiment, a group of die and interconnect circuitry may be designed to be included in a memory chip 3506 such as that shown in FIG. 36A. Such embodiments may include (for in-memory processing) a processor subunit that may be configured to communicate with the company. For example, each die to be included in the memory chip 3506 may include various memory units such as memory banks 3511A-3511D, processor subunits 3515A-3515D, and IO controllers 3521 and 3522 . IO controllers 3521 and 3522 may be connected in parallel with input-output bus 3530 . The IO controller 3521 may have a bus 3554 , and the IO controller 3522 may have a bus 3555 . In one exemplary embodiment, processor subunits 3515A - 3515D may be coupled via bus 3613 , for example. In some cases, one of the IO controllers can be disabled using a corresponding fuse. For example, IO controller 3522 may be disabled utilizing fuse 3555 , IO controller 3521 may be deactivated via bus 3613 and memory bank 3511A via processor subunits 3515A-3515D connected to each other via bus 3613 . -3511D) can control the data flow.

The configuration of the memory unit as shown in FIG. 36A is merely an example, and various other configurations may be formed by cutting different regions of the wafer 3501 . For example, FIG. 36B shows a configuration having three regions 3601-3603 including a memory unit and coupled to an input-output bus 3530 . In one exemplary embodiment, regions 3601-3603 are coupled to input-output buses 3530 utilizing IO control modules 3521-3523 which can be disabled by corresponding fuses 3554-3556. Another example of an embodiment of arranging regions comprising memory units is shown in FIG. 36C , where three regions 3601 , 3602 , 3603 utilize bus lines 3611 , 3612 , 3613 to input-output bus 3530 . ) is connected to 36D depicts another exemplary embodiment of a memory chip 3506A-3506D coupled to input-output buses 3530A and 3530B via an IO controller 3521-3524. In one exemplary embodiment, the IO controller may be deactivated utilizing a corresponding fuse as shown in FIG. 36D .

FIG. 37 illustrates various groups of dies 3503 , such as group 3713 and group 3715 , which may include one or more dies 3503 . In one exemplary embodiment, in addition to forming die 3503 on wafer 3501 , wafer 3501 may also include logic circuitry 3711 called glue logic 3711 . Since the glue logic 3711 occupies a portion of the space on the wafer 3501 , the number of dies manufactured per wafer 3501 may be reduced compared to the case where the glue logic 3711 is manufactured without the glue logic 3711 . However, the presence of glue logic 3711 allows multiple dies to be configured to function as a single memory chip. For example, glue logic can connect multiple dies without changing the configuration and designating regions inside the die for circuitry used only to connect dies to each other. In various embodiments, the glue logic 3711 may provide an interface with other memory controllers to allow the multi-die memory chip to function as a single memory chip. Glue logic 3711 may be cut together into groups of dies shown as group 3713, for example. Alternatively, if only one die for the memory chip, for example group 3715, is needed, the glue logic may not be cut. For example, glue logic can be selectively removed when not needed to enable cooperation between different dies. In FIG. 37 , various cuts in different regions, such as, for example, broken-line regions, can be made as shown. In various embodiments, as shown in FIG. 37 , one glue logic element 3711 may be laid out on the wafer every two dies 3506 . In some cases, one glue logic element 3711 may be used for any number of dies 3506 forming a group of dies. Glue logic 3711 may be configured to connect to all dies in a die group. In various embodiments, a die connected to glue logic 3711 may be configured to form a multi-die memory chip, and if not connected to glue logic 3711 may be configured to form a separate single die memory chip. . In various embodiments, dies connected to glue logic 3711 and designed to function together may be cut from wafer 3501 in groups and may include glue logic 3711 as indicated by group 3713 or the like. . Dies not connected to glue logic 3711 may be cut from wafer 3501 without including glue logic 3711 as indicated by groups 3715 or the like to form a single die memory chip.

In some embodiments, during fabrication of multi-die memory chips on wafer 3501, one or more cut features (eg, shapes that form groups 3713 and 3715) are cut to create a desired set of multi-die memory chips. can be decided. In some cases, as shown by group 3715 , the cut shape may not include glue logic 3711 .

In various embodiments, the glue logic 3711 may be a controller for controlling multiple memory units of a multi-die memory chip. In some cases, the glue logic 3711 may include parameters that may be modified by various other controllers. For example, the combination circuit of a multi-die memory chip may include circuitry for configuring parameters of glue logic 3711 or parameters of a memory controller (e.g., processor subunits 3515A-3515D shown in FIG. 35B). . The glue logic 3711 may be configured to perform various tasks. For example, the glue logic 3711 may be configured to determine which die should be addressed. In some cases, glue logic 3711 may be used to synchronize multiple memory units. In various embodiments, the glue logic 3711 may be configured to control various memory units such that the memory units operate as a single chip. In some cases, an amplifier may be added between the input-output bus (eg, bus 3530 shown in FIG. 35C ) and processor subunits 3515A- 3515D to amplify the data signal from bus 3530 .

In various embodiments, cutting complex shapes in wafer 3501 may be technically difficult/expensive, and simpler cutting methods may be employed if dies are aligned on wafer 3501 . 38A shows die 3506 aligned to form a rectangular grid. In an exemplary embodiment, a vertical cut 3803 and a cross cut 3801 may be performed on the entire wafer 3501 to separate groups of cut dies. In one exemplary embodiment, longitudinal and transverse cuts 3803 and 3801 may create a group comprising a selected number of dies. For example, as a result of cut 3803 and cut 3801, a region containing a single die (eg, 3811A), a region containing two dies (eg, 3811B), and a region containing four dies (eg, 3811C) may be formed. The areas formed by cut 3803 and cut 3801 are exemplary only, and any other suitable area may be formed. In various embodiments, depending on the alignment of the die, various cuts may be made. For example, as shown in FIG. 38B , when the dies are arranged in a triangular grid, cutting lines such as lines 3802 , 3804 , and 3806 may be utilized to form a multi-die memory chip. For example, some regions may include 6 dies, 5 dies, 3 dies, 2 dies, 1 die, or any other suitable number of dies.

38C illustrates that bus lines 3530 are arranged in a triangular grid, with die 3503 aligned in the center of a triangle formed by intersecting bus lines 3530 . Die 3503 may be connected to all neighboring buslines via bus line 3820 . By severing a region that includes two or more adjacent dies (eg, region 3822 shown in FIG. 38C ), at least one bus line (eg, line 3824 ) is left in region 3822 , and region 3822 ), a bus line 3824 may be utilized to provide data and commands to a multi-die memory chip formed by utilizing .

39 illustrates that various connections may be made between processor subunits 3515A-3515P to allow groups of memory units to function as a single memory chip. For example, group 3901 of various memory units may include a connection 3905 between processor subunit 3515B and processor subunit 3515E. Connection 3905 may be used as a bus line for the transfer of data and commands from subunit 3515B to subunit 3515E, which may be used to control memory bank 3511E. In various embodiments, connections between processor subunits may be implemented during die formation on wafer 3501 . In some cases, additional connections may be made during packaging of memory chips formed from multiple dies.

As shown in FIG. 39, processor subunits 3515A-3515P may be coupled to each other utilizing various buses (eg, connection 3905). Connection 3905 does not have a timing hardware logic element so that data transfer between processor subunits and data transfer over connection 3905 may not be controlled by the timing hardware logic element. In various embodiments, the buses connecting the processor subunits 3515A-3515P may be laid out on the wafer 3501 prior to fabricating the various circuits on the wafer 3501 .

In various embodiments, processor subunits (eg, subunits 3515A-3515P) may be interconnected. For example, subunits 3515A-3515P may be connected by a suitable bus (eg, connection 3905). Connection 3905 may connect one or more of subunits 3515A-3515P with another one or more of subunits 3515A-3515P. In one exemplary embodiment, connected subunits may be on the same die (eg, 3515A and 3515B), and in other cases, connected subunits may be on different die (eg, 3515B and 3515E). case). Connection 3905 may include a dedicated bus for connecting subunits and may be configured to efficiently transfer data between subunits 3515A-3515P.

Various aspects of the present disclosure relate to methods for producing selectable sized memory chips from wafers. In one exemplary embodiment, memory chips of selectable sizes may be formed from one or more dies. As discussed above, the dies may be arranged along one or more rows, for example as shown in FIG. 35C . In some cases, at least one shared input-output bus corresponding to one or more rows may be laid out on the wafer 3501 . For example, the bus 3530 may be laid out as shown in FIG. 35C . In various embodiments, bus 3530 may be electrically coupled to memory units of at least two dies, and the connected dies may be used to form a multi-die memory chip. In one exemplary embodiment, one or more controllers (eg, input-output controllers 3521 , 3522 shown in FIG. 35B ) may be configured to control memory units of at least two dies used in the formation of a multi-die memory chip. can In various embodiments, a die with memory units connected by bus 3530 may have a shared input-output bus (eg, the bus shown in FIG. 35B ) that transmits information to at least one controller (eg, controllers 3521 , 3522 ). The controller may be configured to control the memory units of the connected die together to function as a single chip, cut from the wafer together with at least one corresponding portion of ( 3530 ).

In some cases, a memory unit located on the wafer 3501 may be inspected prior to manufacturing a memory chip by cutting a region of the wafer 3501 . The check may be performed using at least one shared input-output bus (eg, bus 3530 shown in FIG. 35C ). A memory chip may be formed from a group of dies containing the memory unit if the memory unit passes the test. Memory units that fail the test may be discarded and not used for filing the memory chip.

40 depicts an exemplary process 4000 for constructing a memory chip from a group of dies. At step 4011 of process 4000 , a die may be laid out on semiconductor wafer 3501 . In step 4015, a die may be fabricated on wafer 3501 utilizing any suitable manner. For example, a die may be fabricated by etching the wafer 3501 , depositing various dielectric, metal, or semiconductor layers, further etching the deposited layers, and the like. For example, multiple layers may be deposited and etched. In various embodiments, the layer may be n-type doped or p-type doped utilizing any and all suitable doping elements. For example, the semiconductor layer may be n-type doped with phosphorus (P) and p-type doped with boron (B). Dies 3503 as shown in FIG. 35A may be separated from each other by a space that may be used to cut die 3503 from wafer 3501 . For example, the dies 3503 may be spaced apart from each other by a spatial region, wherein the width of the spatial region may be selected to enable wafer cutting in the spatial region.

At step 4017 , the die 3503 may be cut from the wafer 3501 utilizing any suitable manner. In one exemplary embodiment, die 3503 may be cut utilizing a laser. In one exemplary embodiment, wafer 3501 may be first scribed and then mechanically cut. Alternatively, a mechanical dicing saw may be used. In some cases, a stealth dicing process may be used. During the cutting process, the wafer 3501 may be mounted on a dicing tape for holding the die after the die is cut. In various embodiments, large cuts may be made, as shown by cuts 3801 and 3803 of FIG. 38A or cuts 3802, 3804, and 3806 of FIG. 38B. Once the dies 3503 are cut individually or in groups, such as group 3504 in FIG. 35C , the dies 3503 may be packaged. Packaging of the die includes forming contacts on the die 3503 , depositing a protective layer on the contacts, attaching a thermal management device (eg, a heatsink), and encapsulating the die 3503 . . In various embodiments, an appropriate configuration of contacts and buses may be used, depending on the number of dies selected to form the memory chip. In one exemplary embodiment, some of the contacts between the dies forming the memory chip may be formed during the memory chip packaging process.

41A depicts an exemplary process 4100 for manufacturing a memory chip including multiple dies. Step 4011 of process 4100 may be the same as step 4011 of process 4000 . In step 4111 , the glue logic 3711 as shown in FIG. 37 may be laid out on the wafer 3501 . The glue logic 3711 may be any suitable logic for controlling the operation of the die 3506 as shown in FIG. 37 . As described above, the presence of glue logic 3711 allows multiple dies to function as a single memory chip. The glue logic 3711 may provide an interface with another memory controller so that a memory chip formed from a plurality of dies may function as a single memory chip.

At step 4113 of process 4100 , a bus (eg, an input-output bus and a control bus) may be laid out on wafer 3501 . The bus may be laid out to connect with various logic circuits, such as various die and glue logic 3711 . In some cases, a bus may connect memory units. For example, a bus may be configured to connect processor subunits of different dies. At step 4115, the die, glue logic, and bus may be fabricated utilizing any suitable approach. For example, the logic element may be fabricated by etching the wafer 3501 , depositing various dielectric, metal, or semiconductor layers, further etching the deposited layers, and the like. The bus may be fabricated utilizing, for example, metal deposition.

At step 4140 , the cut shape may be utilized to cut a group of dies connected to a single glue logic 3711 as shown in FIG. 37 . The cut shape may be determined utilizing the memory requirements for a memory chip comprising multiple dies 3503 . For example, FIG. 41B shows process 4101 , which may be a variant of process 4100 , where there may be steps 4117 and 4119 prior to step 4140 of process 4100 . At step 4117, the wafer 3501 cutting system may receive a command describing the requirements for the memory chip. For example, a requirement may include the formation of a memory chip comprising four dies 3503 . In some cases, at step 4119 the program software may determine a periodic pattern for the group of dies and the glue logic 3711 . For example, the periodic pattern may include two glue logic 3711 elements and a pattern in which two dies are connected for every one glue logic 3711 in four dies 3503 . Alternatively, in step 4119 , the pattern may be provided by a designer of the memory chip.

In some cases, the pattern may be selected to maximize the yield of memory chips from wafer 3501 . In one exemplary embodiment, the memory units of die 3503 are inspected to identify the die with the bad memory units (referred to as 'bad dies'), and based on the location of the bad dies, memory units that passed the inspection are identified. The group of dies 3503 it contains can be identified and an appropriate cut pattern can be determined. For example, when a large number of dies 3503 are defective at the edge of the wafer 3501 , a cutting pattern excluding the dies at the edge of the wafer 3501 may be determined. Other steps in process 4101 such as steps 4011 , 4111 , 4113 , 4115 , and 4140 may be identical to the same numbered steps in process 4100 .

41C depicts an example process 4102 that may be a variant of process 4101 . Step 4011 , step 4111 , step 4113 , step 4115 , and step 4140 of process 4102 may be identical to the same numbered steps of process 4101 , and step 4131 of process 4102 is equivalent to step 4101 of process 4101 . Step 4117 may be substituted, and step 4133 of process 4102 may be substituted for step 4119 of process 4101 . At step 4131 , the wafer 3501 cleaving system may receive commands describing requirements for the first set of memory chips and the second set of memory chips. For example, a requirement may include forming a first set of memory chips with memory chips comprising four dies 3503 and forming a second set of memory chips with memory chips comprising two dies 3503 . may include In some cases, more than one set of memory chips may have to be formed on wafer 3501 . For example, the third set of memory chips may include memory chips including only one die 3503 . In some cases, at step 4133 , the program software may determine a periodic pattern of glue logic 3711 and groups of dies to form memory chips for each set of memory chips. For example, the first set of memory chips may include a memory chip including two glue logic 3711 and a pattern in which two dies are connected for every one glue logic 3711 in four dies 3503 . . In various embodiments, the glue logic 3711 for the same memory chip may be connected to each other to function as a single glue logic. For example, in the manufacturing process of the glue logic 3711 , a bus line connecting the glue logic 3711 to each other may be formed.

The second set of memory chips may include memory chips with one glue logic 3711 and two dies 3503 having a pattern in which the die 3503 is connected to the glue logic 3711 . In some cases, if a second set of memory chips is selected, and the second set of memory chips includes memory chips comprising a single die 3503 , the glue logic 3711 may not be needed.

Dual port function

When designing a memory chip or memory instance within a chip, one important characteristic is the number of words that can be accessed simultaneously during a single clock cycle. The more addresses (eg, addresses along rows and columns, also called bits or bitlines) that can be accessed simultaneously for reading and/or writing, the faster the memory chip. Efforts have been made to develop memory with multi-directional ports that allow simultaneous access to multiple addresses, such as building register files, caches, or shared memory, but most instances are large in size and support multi-address access. use a memory mat that However, DRAM chips typically include a single bit line and a single row line coupled to each capacitor of each memory cell. Accordingly, embodiments of the present disclosure seek to provide multi-port access on an existing DRAM chip without changing this conventional single-port memory structure of a DRAM array.

In embodiments of the present disclosure, the clock speed of the memory instance or chip may be twice as fast as the speed of the logic circuit using the memory. Thus, any arbitrary logic circuit that uses a memory may 'correspond' to the memory and its components. Accordingly, embodiments of the present disclosure can read or write to two addresses with a memory array clock cycle, which is equivalent to a single processing clock cycle for a logic circuit. The logic circuitry may include circuitry such as a controller, accelerator, GPU, or CPU, or it may include a processing group on the same substrate as the memory chip as shown in FIG. 7A . As described above with reference to FIG. 3A , a 'processing group' may refer to two or more processor subunits and corresponding memory banks on a substrate. Groups may represent logical bundles for spatial distribution on a substrate and/or compilation of code for execution on memory chip 2800 . Accordingly, as described above with reference to FIG. 7A , the substrate with the memory chip may include a memory array having a plurality of banks, such as the bank 2801a shown in FIG. 28 and other banks. The substrate may also include a processing array that may include a plurality of processor subunits (eg, subunits 730a, 730b, 730c, 730d, 730e, 730f, 730g, 730h shown in FIG. 7A ).

Accordingly, in an embodiment of the present disclosure, in order to process two addresses for each logic cycle and provide two results to the logic as if a single port memory array were a two port memory chip, the array in each of two successive memory cycles data can be read from Through additional clocking, the memory chips of the present disclosure may allow a single port array to function as a two port memory instance, a three port memory instance, a four port memory instance, or any other multi-port memory instance.

42 illustrates an exemplary circuit 4200 that provides dual port access along the row of memory chips in which the circuit 4200 according to the present disclosure is used. 42 uses one memory array 4201 with two column multiplexers ('mux') 4205a and 4205b to access 2 words of the same row during the same clock cycle for the logic circuit. can For example, during a memory clock cycle, RowAddrA is used in row decoder 4203 and ColAddrA is used in multiplexer 4205a to buffer data from memory with addresses (RowAddrA, ColAddrA). During the same memory clock cycle, ColAddrB is used in multiplexer 4205b to buffer data from the memory with addresses (RowAddrA, ColAddrB). Accordingly, circuit 4200 may enable dual port access to data (eg, DataA and DataB) stored on memory cells at two different addresses along the same row or wordline. Thus, two addresses can share a row so that the row decoder 4203 activates the same wordline for both reads. Also, embodiments such as the example shown in FIG. 42 may use a column multiplexer so that two addresses can be accessed during the same memory clock cycle.

Similarly, FIG. 43 depicts an exemplary circuit 4300 that provides dual port access along the row of memory chips in which the circuit 4300 according to the present disclosure is used. 43 shows that the row decoder 4303 uses one memory array 4301 coupled to a multiplexer ('mux') to access two words on the same column during the same clock cycle for the logic circuit. can For example, in the first cycle of 2 memory clock cycles, RowAddrA is used in row decoder 4303 and ColAddrA is used in column multiplexer 4305 to buffer data from memory cells with addresses (RowAddrA, ColAddrA). It can be (eg, with the 'Buffered Word' buffer of FIG. 43). In the second cycle of two memory clock cycles, RowAddrB is used in the row decoder 4303 and ColAddrA is used in the column multiplexer 4305 to buffer data from the memory cell with addresses (RowAddrB, ColAddrA). Accordingly, circuitry 4300 may enable dual port access to data (eg, DataA and DataB) stored on memory cells at two different addresses along the same column or bitline. Thus, two addresses may share a row such that a column decoder (which may be separate or merged with one or more column multiplexers, as shown in FIG. 43) activates the same bitline for both reads. Since the row decoder 4303 may require one memory clock cycle to activate each word line, embodiments such as the example shown in FIG. 43 may use two memory clock cycles. Accordingly, a memory chip using the circuit 4300 can function as a dual-port memory if the clock speed is at least twice as fast as the corresponding logic circuit.

Accordingly, as described above, Figure 43 can read DataA and DataB during two memory clock cycles faster than the clock cycle for the corresponding logic circuit. For example, a row decoder (eg, row decoder 4303 in FIG. 43 ) and a column decoder (which may be separate or merged with one or more column multiplexers, as shown in FIG. 43 ) may have corresponding logic generating two addresses. It may be configured to be clocked at a speed at least twice as fast as the speed of the circuit. For example, a clock circuit (not shown in FIG. 43 ) for circuit 4300 may clock circuit 4300 at a speed that is at least twice as fast as the speed of a corresponding logic circuit generating two addresses.

42 and 43 may be used separately or together. Accordingly, a circuit (eg, circuit 4200 or circuit 4300) that provides dual port functionality on a single port memory array or mat may include a plurality of memory banks arranged along at least one row and at least one column. may include The plurality of memory banks is illustrated as a memory array 4201 in FIG. 42 and a memory array 4301 in FIG. 43 . This embodiment may also use at least one row multiplexer (shown in Figure 43) or at least one column multiplexer (shown in Figure 42) configured to receive two addresses for reading or writing during a single clock cycle. In addition, this embodiment separates the row decoder (eg, the row decoder 4203 of FIG. 42 and the row decoder 4303 of FIG. 43 ) and the column decoder (as shown in FIGS. 42 and 43 , one or more column multiplexers) can be combined or merged) to read or write to the two addresses. For example, the row decoder and the column decoder read a first address of two addresses from at least one row multiplexer or at least one column multiplexer during a first cycle and decode wordlines and bitlines corresponding to the first addresses. can In addition, the row decoder and the column decoder may read a second address of two addresses from at least one row multiplexer or at least one column multiplexer during a second cycle and decode a word line and a bit line corresponding to the second address. . Each of these reads can be accomplished by activating a word line corresponding to an address using a row decoder and activating a bit line on an activated word line corresponding to this address using a column decoder, respectively.

Although the read has been described above, the embodiments of FIGS. 42 and 43 may include a write command regardless of whether they are implemented individually or together. For example, during the first cycle, the row decoder and the column decoder may write first data read from the at least one row multiplexer or the at least one column multiplexer to a first address of the two addresses. Also, during the second cycle, the row decoder and the column decoder may write second data read from the at least one row multiplexer or the at least one column multiplexer to a second address of the two addresses.

The example of FIG. 42 shows this process when the first and second addresses share a wordline address, and the example of FIG. 43 shows this process when the first and second addresses share a column address. As described below with reference to FIG. 47 , the same process may be implemented when the first and second addresses do not share a wordline address or a column address.

Accordingly, while the above examples provide dual port access along at least one of rows and columns, additional embodiments may provide dual port access along both rows and columns. 44 illustrates an exemplary circuit 4400 that provides dual port access along both rows and columns of a memory chip in which circuit 4400 according to the present disclosure is used. Accordingly, the circuit 4400 may represent a combination of the circuit 4200 of FIG. 42 and the circuit 4300 of FIG. 43 .

44 may use one memory array 4401 with a row decoder 4403 coupled with a multiplexer (“mux”) to access two rows during the same clock cycle for the logic circuit. . 44 may also use the memory array 4401 with a column decoder (or multiplexer) 4405 coupled with a multiplexer (“mux”) to access two columns during the same clock cycle. For example, in the first cycle of 2 memory clock cycles, RowAddrA is used in row decoder 4403 and ColAddrA is used in column multiplexer 4405 to buffer data from memory cells with addresses (RowAddrA, ColAddrA). It can be (eg, with the 'Buffered Word' buffer of FIG. 44). In the second cycle of two memory clock cycles, RowAddrB is used in row decoder 4403 and ColAddrA can be used in column multiplexer 4405 to buffer data from the memory cell with addresses (RowAddrB, ColAddrA). Accordingly, circuitry 4400 may enable dual port access to data (eg, DataA and DataB) stored on memory cells at two different addresses. Since the row decoder 4403 may require one memory clock cycle to activate each word line, embodiments such as the example shown in FIG. 44 may use an additional buffer. Accordingly, a memory chip using circuit 4400 can function as a dual-port memory if the clock speed is at least twice as fast as the corresponding logic circuit.

Although not shown in FIG. 44 , circuit 4400 may further include additional circuitry of FIG. 46 (discussed below) along rows or wordlines and/or similar additional circuitry along columns or bitlines. Accordingly, the circuit 4400 activates a corresponding circuit (eg, by opening one or more switching elements, such as one or more of the switching elements 4613a, 4613b of FIG. 46 ) to activate the de-energized portion comprising the address. This can be done (eg, by connecting a voltage to a de-energized part or passing a current through it). Accordingly, a circuit may be configured to provide a voltage to a memory cell identified by an address when an element of the circuit (eg, a line, etc.) includes a location identified by an address and/or when an element (eg, a switching element) of the circuit is identified by an address. / or may be 'corresponding' in the case of controlling the flow of current. Thereafter, the circuit 4400 may decode the corresponding word line and the bit line using the row decoder 4403 and the column multiplexer 4405 , and may read or write data to an address located in the activated single power portion.

As further shown in Figure 44, circuit 4400 includes at least one row multiplexer (shown as separate from row decoder 4403, but configured to fetch two addresses for read and write during a single clock cycle). may be included) and/or at least one column multiplexer (shown as separate from column multiplexer 4405 but may be included therein) may further be used. Accordingly, in the present embodiments, reading and writing to two addresses can be performed using a row decoder (eg, the row decoder 4403 ) and a column decoder (which may be separated from or combined with the column multiplexer 4405 ). For example, a row decoder and a column decoder may obtain a first address of two addresses from at least one row multiplexer or at least one column multiplexer during a memory clock cycle and decode a wordline and a bitline corresponding to the first address. can Further, the row decoder and the column decoder may obtain a second address of the two addresses from the at least one row multiplexer or the at least one column multiplexer during the same memory cycle and decode the word line and the bit line corresponding to the second address. .

45A and 45B illustrate a conventional replication scheme for providing dual port functionality on a single port memory array or mat. As shown in FIG. 45A , dual port reads can be provided by keeping duplicate copies of data synchronized across a memory array or mat. Accordingly, the read may be performed on both copies of the memory instance as shown in FIG. 45A . Also, as shown in FIG. 45B , dual port writes can be provided by duplicating all writes across the memory array or mat. For example, a memory chip may require that all logic circuits using the memory chip send a copy of a write command, one for each duplicate copy of the data. Or, in some embodiments, as shown in FIG. 45A , by having additional circuitry automatically by the additional circuitry to create duplicate copies of the data written across the memory array or mat to keep the copies synchronized. can send a single write command duplicated by the logic circuitry using the memory instance. 42, 43, and 44 use a multiplexer to access two bitlines in a single memory clock cycle (see, eg, FIG. 42) and/or to speed the clock speed of the memory faster than the corresponding logic circuit and by providing an additional multiplexer to process additional addresses instead of duplicating all data into memory (eg, see FIG. 43 ), redundancy in this existing replication scheme can be reduced.

In addition to the clock rate increase and/or additional multiplexers described above, embodiments of the present disclosure may use circuitry to de-energize bitlines and/or wordlines at any point within the memory array. In such embodiments, multiple simultaneous accesses to the array are possible as long as the row decoder and column decoder access different locations that are not coupled to the same part of the single circuit. For example, a single circuit may allow the row decoder and column decoder to access different addresses without electrical interference, so that locations with different wordlines and bitlines can be accessed simultaneously. The granularity of the single power region in the memory may be considered in comparison with the additional parts required by the single power circuit in the design process of the memory chip.

An architecture for implementing this concurrent approach is shown in FIG. 46 . Specifically, FIG. 46 shows an exemplary circuit 4600 for providing dual port functionality to a single port memory array or mat. 46 , the circuit 4600 may include a plurality of memory mats (eg, 4609a, 4609b, etc.) arranged along at least one row and at least one column. The layout of circuit 4600 may further include a plurality of wordlines, such as wordlines 4611a and 4611b corresponding to rows, and bitlines 4615a, 4615b, corresponding to columns.

The example of Figure 46 includes 12 memory mats each with 2 lines and 8 columns. In other embodiments, the substrate may include any number of memory mats, and each memory mat may include any number of lines and any number of columns. Some memory mats may include the same number of lines and columns (example in FIG. 46), while other memory mats may include different numbers of lines and/or columns.

Although not shown in FIG. 46, circuit 4600 includes at least one row multiplexer (row decoder 4601a) configured to receive, during a single clock cycle, two (or three or any arbitrary plurality) of addresses for reading and writing. and/or separate or integrated with 4601b) or at least one column multiplexer (eg, column multiplexers 4603a and/or 4603b). Further, the present embodiments use a row decoder (eg, which may be separate or integral with a row decoder 4601a and/or 4601b and a column multiplexer 4603a and/or 4603b) to two (or more) addresses can read or write to. For example, a row decoder and a column decoder may read a first address of two addresses from at least one row multiplexer or at least one column multiplexer during a memory clock cycle and decode a word line or bit line corresponding to the first address. can In addition, the row decoder and the column decoder may read a second address of the two addresses from the at least one row multiplexer or the at least one column multiplexer and decode a word line or a bit line corresponding to the second address during the same memory cycle. . As discussed above, as long as the two addresses are in different locations that are not coupled to the same part of the power circuit (eg, a switching element such as 4613a, 4613b, etc.), accesses can occur during the same memory clock cycle. Further, the circuit 4600 may simultaneously access the first two addresses during a first memory clock cycle and then concurrently access the second two addresses during a second memory clock cycle. In such an embodiment, a memory chip using circuit 4600 may function as a four-port memory if the clock speed is at least twice as fast as the corresponding logic circuit.

46 further includes at least one row circuit and at least one column circuit configured to function as a switch. For example, a corresponding switching element, such as 4613a, 4613b, etc., may include a transistor or a transistor configured to allow or disengage current and/or to connect or disengage voltage from a wordline or bitline coupled to a switching element such as 4613a, 4613b, etc. It may include any and all other electrical elements. Accordingly, a corresponding switching element may divide circuit 4600 into a de-energized portion. Although shown as including a single row and 16 columns of each row, the de-energized portions within circuit 4600 may include different levels of granularity depending on the design of circuit 4600 .

Circuit 4600 may activate corresponding circuits of at least one row circuit and at least one column circuit using a controller (eg, row control 4607) to activate corresponding de-energized regions during the address operations described above. can For example, circuit 4600 may send one or more control signals to a nearby corresponding one of the switching elements (eg, 4613a, 4613b, etc.). In embodiments where the switching elements 4613a, 4613b, etc. include a transistor, the control signal may include a voltage to open the transistor.

Depending on the de-energized region containing the address, one or more of the switching elements may be activated by the circuit 4600 . For example, to reach an address in the memory mat 4609b of FIG. 46 , a switching element allowing access to the memory mat 4609a must be opened along with the switching element allowing access to the memory mat 4609b. The row control 4607 may determine activation of the switching element according to the specific address in order to read the specific address in the circuit 4600 .

46 shows an example of circuit 4600 used to isolate wordlines of a memory array (eg, including memory mats 4609a, 4609b, etc.). However, other embodiments may use similar circuitry (eg, a switching element that separates the memory chip 4600 into disconnected regions) to isolate the bitlines of the memory array. Accordingly, the architecture of circuit 4600 may be used for a dual-column approach as shown in FIG. 42 or 44 as well as for a dual-row approach as shown in FIG. 43 or 44 .

A process for multi-cycle access to a memory array or metro is shown in FIG. 47A. Specifically, FIG. 47A is an exemplary flow diagram of a process 4700 for providing dual port access on a single port memory array or mat (eg, using circuit 4300 of FIG. 43 or circuit 4400 of FIG. 44 ). . Process 4700 includes a row decoder 4303 or 4304 and a column decoder of FIG. 43 or 44 (which may be separate or integral with one or more column multiplexers, such as column multiplexer 4305 or 4405 of FIG. 43 or 44 ); The same may be implemented using a row decoder and a column decoder according to the present disclosure.

At step 4710, during a first memory clock cycle, the circuitry may decode the wordline and bitline corresponding to the first address of the two addresses using the at least one row multiplexer and the at least one column multiplexer. For example, the at least one row decoder may activate a word line, and the at least one column multiplexer may amplify a voltage from the memory cell along the activated word line and corresponding to a first address. The amplified voltage may be provided to a logic circuit utilizing a memory chip comprising the circuit or may be buffered according to step 4720 described below. The logic circuit may include circuitry such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as shown in FIG. 7A .

Although described above as a read operation, the method 4700 may similarly process a write operation. For example, the at least one row decoder may activate a word line, and the at least one column multiplexer may write new data to the memory cell by applying a voltage to the memory cell along the activated word line and corresponding to the first address. . In some embodiments, the circuitry may provide an acknowledgment of a write to a logic circuit utilizing a memory chip comprising the circuitry or may buffer acknowledgment according to step 4720 below.

In operation 4720, the circuit may buffer the read data of the first address. For example, as shown in FIGS. 43 and 44 , due to the buffer, the circuit can read the second address among the two addresses (described in step 4730 below) and output the results of both reads together. can The buffer may include registers, SRAM, non-volatile memory, or any other data storage device.

In step 4730, during a second memory clock cycle, the circuitry may use the at least one row multiplexer and the at least one column multiplexer to decode the wordline and the bitline corresponding to the second address of the two addresses. For example, the at least one row decoder may activate a word line, and the at least one column multiplexer may amplify a voltage from the memory cell along the activated word line and corresponding to the second address. The amplified voltage may be provided to a logic circuit utilizing a memory chip comprising the circuit, either individually or with a buffered voltage, for example in step 4720 . The logic circuit may include circuitry such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as shown in FIG. 7A .

Although described above as a read operation, the method 4700 may similarly process a write operation. For example, the at least one row decoder may activate a word line, and the at least one column multiplexer may write new data to the memory cell by applying a voltage to the memory cell along the activated word line and corresponding to the second address. . In some embodiments, the circuit may provide confirmation of a write to a logic circuit utilizing a memory chip that contains the circuit, either individually or with a buffered voltage at step 4720 or the like.

In operation 4740, the circuit may output the read data of the second address together with the buffered first address. For example, as shown in FIGS. 43 and 44 , the circuit may output the results of both readings (eg, steps 4710 and 4730 ) together. The circuit may output a result to a logic circuit utilizing a memory chip including the circuit. The logic circuit may include circuitry such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as shown in FIG. 7A .

Although multiple cycles have been described above, as shown in FIG. 42 , when two addresses share a wordline, the method 4700 allows single-cycle access to the two addresses as well. For example, since a multi-column multiplexer can decode different bitlines on the same wordline during the same memory clock cycle, steps 4710 and 4730 can be performed during the same memory clock cycle. In such an embodiment, step 4720 of buffering may be skipped.

A process for concurrent access (eg, utilizing circuit 4600 described above) is shown in FIG. 47B . Accordingly, although the steps are shown sequentially, the steps of FIG. 47B may be performed during the same memory clock cycle, and at least some steps (eg, steps 4760 and 4780 or steps 4770 and 4790) may be executed concurrently. Specifically, FIG. 47B is an exemplary flow diagram of a process 4750 for providing dual port access on a single port memory array or mat (eg, utilizing circuit 4200 of FIG. 42 or circuit 4600 of FIG. 46 ). . Process 4750 separates or separates from one or more column multiplexers, such as row decoders 4203 or 4601a and 4601b and column decoders of FIG. 42 or 46 (column multiplexers 4205a and 4205b or 4603a and 4603b of FIG. 42 or 46). may be integrally implemented) using a row decoder and a column decoder according to the present disclosure.

In step 4760, the circuit may activate, during the memory clock cycle, corresponding circuits of the at least one row circuit and the at least one column circuit based on the first address of the two addresses. For example, the circuit may send one or more control signals to a nearby corresponding element of the switching element comprising at least one row circuit and at least one column circuit. Accordingly, the circuit can access the corresponding disconnection area including the first address of the two addresses.

At step 4770, during a memory clock cycle, the circuitry may utilize the at least one row multiplexer and the at least one column multiplexer to decode the wordline and bitline corresponding to the first address. For example, the at least one row decoder may activate a word line, and the at least one column multiplexer may amplify a voltage from the memory cell along the activated word line and corresponding to a first address. The amplified voltage may be provided to a logic circuit utilizing a memory chip including the circuit. For example, as described above, the logic circuit may include a circuit such as a GPU or a CPU, or may include a processing group on the same substrate as the memory chip, for example, as shown in FIG. 7A .

Although the above has been described as a read operation, the method 4750 may similarly process a write operation. For example, the at least one row decoder may activate a word line, and the at least one column multiplexer may write new data to the memory cell by applying a voltage to the memory cell along the activated word line and corresponding to the first address. . In some embodiments, the circuit may provide confirmation of a write to a logic circuit that utilizes a memory chip that includes the circuit.

In step 4780, during the same cycle, the circuit may activate corresponding circuits of the at least one row circuit and the at least one column circuit based on the second address of the two addresses. For example, the circuit may send one or more control signals to a nearby corresponding element of the switching element comprising at least one row circuit and at least one column circuit. Accordingly, the circuit can access the corresponding disconnection area including the second address of the two addresses.

In step 4790, during the same cycle, the circuit may decode the wordline and the bitline corresponding to the second address using the at least one row multiplexer and the at least one column multiplexer. For example, the at least one row decoder may activate a word line, and the at least one column multiplexer may amplify a voltage from the memory cell along the activated word line and corresponding to the second address. The amplified voltage may be provided to a logic circuit utilizing a memory chip including the circuit. For example, as described above, the logic circuit may include a conventional circuit such as a GPU or CPU, or may include a processing group on the same substrate as the memory chip as shown in FIG. 7A .

Although described above as a read operation, the method 4750 may similarly process a write operation. For example, the at least one row decoder may activate a word line, and the at least one column multiplexer may write new data to the memory cell by applying a voltage to the memory cell along the activated word line and corresponding to the second address. . In some embodiments, the circuit may provide confirmation of a write to a logic circuit that utilizes a memory chip comprising the circuit.

Although described above with reference to a single cycle, if the two addresses are in a de-energized region that shares a wordline and a bitline (or shares a switching element in at least one row circuit and at least one column circuit), the method ( 4750) can enable multi-cycle access with two addresses. For example, steps 4760 and 4770 may be performed during a first memory clock cycle during which the first row decoder and the first column multiplexer may decode the wordline and bitline corresponding to the first address, and step 4780 and Operation 4790 may be performed during a second memory clock cycle in which the second row decoder and the second column multiplexer may decode the word line and the bit line corresponding to the second address.

Another example of an architecture with dual port access along both rows and columns is shown in FIG. 48 . Specifically, FIG. 48 shows an exemplary circuit 4800 utilizing a multi-row decoder in conjunction with a multi-column multiplexer to provide dual port access along both rows and columns. 48, row decoder 4801a can access a first wordline, and column multiplexer 4803a can decode data from one or more memory cells along the first wordline, while row decoder 4801b ) may access the second wordline, and the column multiplexer 4803b may decode data from one or more memory cells along the second wordline.

As described with reference to FIG. 47B , these accesses may be concurrent during one memory clock cycle. Accordingly, similar to the architecture of FIG. 46 , the architecture of FIG. 48 (including the memory mat described in FIG. 49 below) may allow multiple addresses to be accessed in the same clock cycle. For example, the architecture of FIG. 48 includes any arbitrary number of row decoders and any arbitrary number of column multiplexers so that a number of addresses corresponding to the number of row decoders and column multiplexers are all accessed within a single memory clock cycle. can do.

In other embodiments, this approach may be sequential along two memory clock cycles. By making the clock speed of the memory chip 4800 faster than the corresponding logic circuit, two memory clock cycles can be equivalent to one clock cycle for the logic circuit using the memory. For example, as described above, the logic circuit may include a conventional circuit such as a GPU or CPU, or may include a processing group on the same substrate as the memory chip as shown in FIG. 7A .

Other embodiments may enable concurrent access. For example, as described with reference to FIG. 42, a multi-column decoder (which may include column multiplexers such as 4803a and 4803b shown in FIG. 48) writes multiple bitlines along the same wordline during a single memory clock cycle. can read Additionally or alternatively, as described with reference to FIG. 46 , circuit 4800 may include additional circuitry to allow this approach to be concurrent. For example, the row decoder 4801a accesses the first wordline and the column multiplexer 4803a accesses the memory cells along the first wordline during the same memory clock cycle as the row decoder 4801b accesses the second wordline. may decode the data from, and the column multiplexer 4803b may decode the data from the memory cell along the second wordline.

The architecture of FIG. 48 may be used with a variant memory mat forming a memory bank as shown in FIG. 49 . 49, each memory cell (shown as a capacitor similar to DRAM, but may include a plurality of transistors arranged in a manner similar to SRAM or any other memory cell) has two wordlines and two bitlines accessed do. Accordingly, the memory mat 4900 of FIG. 49 enables simultaneous access of two different bits or even access of the same bit by two different logic circuits. However, the embodiment of Figure 49 utilizes a variant of the memory mat rather than implementing a dual port solution on a standard DRAM memory mat that is wired for single port access as in the above embodiment.

Although described above with two ports, any of the embodiments described above can be extended to more than two ports. For example, the embodiments of Figures 42, 46, 48, and 49 may each include an additional column or row multiplexer to provide access to each additional column or row during a single clock cycle. For another example, the embodiments of FIGS. 43 and 44 may include additional row decoders and/or column multiplexers to provide access to additional rows or columns, respectively, during a single clock cycle.

Variable word length access in memory

In the above and below, the term 'coupled' may be used in a sense including direct connection, indirect connection, electrical connection, and the like.

In addition, terms such as 'first' and 'second' are used only to distinguish between components or method steps having the same or similar names or names, and do not necessarily indicate spatial or temporal order.

Generally, a memory chip includes a memory bank. The memory bank may be coupled to a row decoder and a column decoder configured to select a particular word (or other fixed-size data unit) to be read or written. Each memory bank may include memory cells for storing data units, sense amplifiers for amplifying voltages from memory cells selected by the row decoder and column decoder, and any other suitable circuitry.

Each memory bank usually has a specific I/O width. For example, an I/O width may include words.

Some processes executed by logic circuitry using memory chips may benefit from using very long words, while some other processes may only need a portion of a word.

Indeed, when an in-memory computing unit (eg, a processor subunit disposed on the same substrate as a memory chip, as shown and described in FIG. 7A ) performs a memory access operation that requires only a subset of words. is common

To reduce the delay associated with accessing an entire word when only a portion of a word is used, embodiments of the present disclosure provide a method and system for retrieving only one or more portions of a word, thereby reducing data loss associated with transmission of an unnecessary portion of a word. and may reduce power consumption of the memory device.

In addition, embodiments of the present disclosure include a memory chip and other components that access a memory chip that reads and writes only a part of a word (eg, separate like a CPU or GPU, or a processor subunit illustrated and described in FIG. 7A ) Power consumption of the interaction between the memory chip and the logic circuit included on the same substrate) can also be reduced.

A memory access command (eg, a command from a logic circuit using the memory) may include an address in the memory. For example, the address may include a row address and a column address or may be converted into a row address and a column address by a memory controller of a memory.

In many volatile memories, such as DRAM, the row address is sent to a row decoder (e.g., either directly by logic circuitry or utilizing a memory controller), which activates an entire row (also called a wordline) and contains it in that row. All bitlines are loaded.

The column address identifies the bitline on the active row that is transferred to the next level circuit and outside the memory bank containing the bitline. For example, the next level circuit may include the I/O bus of the memory chip. In embodiments utilizing in-memory processing, the next level circuitry may include a processor subunit (eg, shown in FIG. 7A ) of a memory chip.

Accordingly, the memory chip described below is shown in one or more of FIGS. 3A, 3B, 4 to 6, 7A to 7D, 11 to 13, 16 to 19, 22, and 23 . It may be included in a memory chip as shown or may include a memory chip.

The memory chip may be manufactured by a first manufacturing process optimized for memory cells rather than logic cells. For example, a memory cell manufactured by a first manufacturing process may have a smaller critical dimension (eg, 2x, 3x, 4x, 5x, 6x, 7x) than a logic circuit manufactured by the first manufacturing process. , 8x, 9x, 10x, etc.). For example, the first manufacturing process may include an analog manufacturing process, a DRAM manufacturing process, or the like.

Such a memory chip may include an integrated circuit that may include a memory unit. The memory unit may include a memory cell, an output port, and a read circuit. In some embodiments, the memory unit may further include a processing unit such as the processor subunit described above.

For example, the read circuit may include a first group of memory read paths and reduction units for outputting up to a first number of bits through the output port. The output port may be connected to an off-chip logic circuit (eg, accelerator, CPU, GPU, etc.) or an on-chip processor subunit as described above.

In some embodiments, the processing unit may include, be part of, be different from, or include a reduction unit, a reduction unit.

An in-memory read path may include any circuit and/or link included in an integrated circuit (eg, included in a memory unit) or configured to read and/or write to a memory cell. For example, the in-memory read path may include a sense amplifier, a conductor coupled to the memory cell, a multiplexer, and the like.

The processing unit may be configured to send a read request for reading the second number of bits from the memory unit to the memory unit. Additionally or alternatively, the read request may originate from an off-chip logic circuit (eg, accelerator, CPU, GPU, etc.).

The reduction unit may be configured to assist in reducing power consumption associated with access requests (eg, utilizing one or more of the partial word accesses described herein).

The reduction unit may be configured to control the memory read path during a read operation triggered by the read request according to the first number of bits and the second number of bits. For example, a control signal from the reduction unit may influence memory consumption of the read path, thereby reducing energy consumption of the memory read path unrelated to the second number of bits requested. For example, the reduction unit may be configured to control the irrelevant memory read path when the second number is less than the first number.

As previously described, an integrated circuit may be illustrated in one or more of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, and 23 . It may be included in or include a memory chip as such.

The irrelevant in-memory read path may be associated with an irrelevant bit of the first number of bits, such as bits of the first number of bits that are not included in the second number of bits.

50 is a read circuit 5040 comprising memory cells 5001-5008 of a memory cell array 5050, an output port 5020 comprising bits 5021-5028, and a memory read path 5011-5018; and an exemplary integrated circuit 5000 including a reduction unit 5030 .

If the second number of bits are read utilizing a corresponding memory read path, the unrelated bits of the first number of bits may correspond to bits that should not be read (eg, bits that are not included in the second number of bits). .

During a read operation, reduction unit 5030 may be configured to activate a memory read path corresponding to the second number of bits such that the activated memory read path may be configured to pass the second number of bits. In such an embodiment, only memory read paths corresponding to the second number of bits may be active.

During a read operation, reduction unit 5030 may be configured to close at least a portion of each irrelevant memory read path. For example, an irrelevant memory read path may correspond to an irrelevant bit of the first number of bits.

Here, instead of closing at least a portion of the irrelevant memory path, the reduction unit 5030 may prevent the irrelevant path from being activated.

Additionally or alternatively, during a read operation, reduction unit 5030 may be configured to maintain the unrelated memory read path in a low power mode. For example, the low power mode may include a mode in which an unrelated memory path is supplied with a voltage or current lower than a normal operating voltage or current.

The reduction unit 5030 may be further configured to control the bitlines of the irrelevant memory read path.

Accordingly, the reduction unit 5030 may be configured to load the bitlines of the relevant memory read paths and maintain the bitlines of the irrelevant memory read paths in a low power mode. For example, only the bitlines of the relevant memory read path can be loaded.

Additionally or alternatively, reduction unit 5030 may be configured to load the bitlines of the relevant memory read path while keeping the unrelated memory read path in an inactive state.

In some embodiments, reduction unit 5030 may be configured to utilize a portion of the memory read path that is relevant during a read operation and maintain a portion of each memory read path that is not relevant (different from the bitline) in a low power mode.

As described above, the memory chip may use a sense amplifier to amplify a voltage from a memory cell included therein. Accordingly, reduction unit 5030 may be configured to utilize a portion of the relevant memory read path and maintain the sense amplifier associated with at least a portion of the unrelated memory read path in a low power mode during a read operation.

In such an embodiment, reduction unit 5030 may be configured to utilize a portion of the relevant memory read path during a read operation and to maintain one or more sense amplifiers associated with both the unrelated memory read path in a low power mode.

Additionally or alternatively, reduction unit 5030 utilizes a portion of the relevant memory read path during a read operation and follows (eg, spatially and/or temporally) independent of one or more sense amplifiers associated with the unrelated memory read path. It can be configured to keep part of a memory read path in a low power mode.

In any of the embodiments described above, the memory unit may include a column multiplexer (not shown).

In such an embodiment, a reduction unit 5030 may be coupled between the column multiplexer and the output port.

Additionally or alternatively, the reduction unit 5030 may be built into the column multiplexer.

Additionally or alternatively, a reduction unit 5030 may be coupled between the memory cell and the column multiplexer.

The reduction unit 5030 may include a reduction subunit that may be independently controllable. For example, different reduction subunits may be associated with different memory unit columns.

Although the above has been described with reference to the read operation and the read circuit, the above embodiments may be similarly applied to the write operation and the write circuit.

For example, an integrated circuit according to the present disclosure may include a memory unit including a memory cell, an output port, and a write circuit. In some embodiments, the memory unit may further include a processing unit such as the processor subunit described above. The write circuit may include a first group of memory write paths and a reduction unit for outputting up to a first number of bits through the output port. The processing unit may be configured to send a write request for writing the second number of bits from the memory unit to the memory unit. Additionally or alternatively, the write request may originate from an off-chip logic circuit (eg, accelerator, CPU, GPU, etc.). The reduction unit 5030 may be configured to control a memory write path during a write operation triggered by a write request, based on the first number of bits and the second number of bits.

Figure 51 shows a memory bank 5100 comprising an array of memory cells 5111 addressed utilizing row and column addresses (e.g., from an on-chip processor subunit or off-chip logic circuitry such as an accelerator, CPU, GPU, etc.) did it As shown in Figure 51, the memory cells are connected by bitlines (vertical) and wordlines (horizontally - mostly depicted for simplicity). In addition, a row address may be supplied to the row decoder 5112 (eg, from an on-chip processor subunit, an off-chip logic circuit, or a memory controller not shown in FIG. 51 ), and a column address may be supplied to the column multiplexer 5113 . may be (eg, from an on-chip processor subunit, off-chip logic circuitry, or a memory controller not shown in FIG. 51 ), and column multiplexer 5113 receives output from up to the entire line and word via output bus 5115 . You can print up to . In FIG. 51 , the output bus 5115 of the column multiplexer 5113 is coupled to the main I/O bus 5114 . In another embodiment, output bus 5115 may be coupled to a processor subunit (eg, shown in FIG. 7A ) of a memory chip that transmits row and column addresses. For the sake of brevity, partitioning a memory bank into memory mats is not shown.

52 shows a memory bank 5101 . In FIG. 52 , a memory bank is shown that also includes processing in memory (PIM) logic 5116 whose input is coupled to an output bus 5115 . The PIM logic 5116 may generate an address (eg, including a row address and a column address) and output it through the PIM address bus 5118 to access the memory bank. The PIM logic 5116 is an example of a reduction unit (eg, 5030 ) that includes a processing unit. PIM logic 5116 may control other circuitry not shown in FIG. 52 that aids in power reduction. PIM logic 5116 may also include a memory path of a memory unit including memory bank 5101 .

As discussed above, the word length (eg, the number of bitlines selected to be transmitted at one time) can be large in some cases.

In this case, each word for read and/or write may be associated with a memory path that may consume power at various stages of a read and/or write operation, such as in the example below.

a. Bitline Loading—To prevent the bitline from being loaded with the required value (either from the capacitor on the bitline in a read cycle or with a new value to be written to the capacitor in a write cycle), disable the sense amplifier located at the end of the memory array and retain the data It is necessary to ensure that the capacitor is not discharged or charged (otherwise the stored data will be destroyed).

b. data from the sense amplifier, through a column multiplexer that selects the bitlines, to the rest of the chip (either to an I/O bus that transfers data into or out of the chip or to a processor subunit on the same board as the memory, such as a processor subunit). using, with built-in logic) to move data.

To enable power saving, the integrated circuit of the present disclosure may send a deactivation signal to one or more sense amplifiers for the irrelevant portion of the word after determining at row activation time that some portion of the word is irrelevant.

53 shows a memory unit 5102 including an array of memory cells 5111 , a row decoder 5112 , a column multiplexer 5113 coupled to an output bus 5115 , and PIM logic 5116 .

Memory unit 5102 also includes a switch 5201 that activates or deactivates the path of bits to column multiplexer 5113 . The switch 5201 may include an analog switch, a transistor configured to function as a switch, or any other circuitry configured to control the supply of voltage and/or the flow of current as part of the memory unit 5102 . A sense amplifier (not shown) may be located at the end of the memory cell array, eg, before (spatially and/or temporally) the switch 5201 .

The switch 5201 may be controlled by an enable signal transmitted from the PIM logic 5116 over the bus 5117 . The switch may be configured to disconnect the sense amplifier (not shown) of the memory unit 5102 and thus not discharge or charge the disconnected bit line from the sense amplifier when disconnected.

The switch 5201 and the PIM logic 5116 may constitute a reduction unit (eg, 5030 ).

In another example, PIM logic 5116 may send an enable signal to a sense amplifier instead of sending it to switch 5201 (eg, if the sense amplifier has an enable input).

The bitline may additionally or alternatively be disconnected at another point, eg at a point other than at the end of the bitline and after the sense amplifier. For example, the bitline may be disconnected prior to entering the array 5111 .

In such an embodiment, power may also be saved in transferring data from the sense amplifier and transmission hardware (eg, output bus 5115).

Other embodiments (those that save less power but are easier to implement) focus on reducing the power of the column multiplexer 5113 and the transmission loss from the column multiplexer 5113 to the next level circuit. For example, as described above, the next level circuit may include an I/O bus (eg, bus 5115) of the memory chip. In embodiments that utilize in-memory processing, the next level circuitry may additionally or alternatively include a processor subunit (eg, PIM logic 5116 ) of a memory chip.

FIG. 54A shows column multiplexer 5113 divided into segments 5202 . Each segment 5202 of column multiplexer 5113 may be individually activated or deactivated by activation and/or deactivation signals sent from PIM logic 5116 over bus 5119 . A column multiplexer 5113 may also be supplied by an address column bus 5118 .

54A may provide better control than other portions of the output from column multiplexer 5113.

Here, the control of different memory paths may have different bit resolutions. For example, bit resolution may range from single bit resolution to multi-bit resolution. The former may be more effective at reducing power, and the latter may be simpler to implement and require less control signals.

54B illustrates an example method 5130 . For example, method 5130 may be implemented utilizing any of the memory units described above with reference to FIG. 50, 51, 52, 53, or 54A.

Method 5130 may include steps 5132 and 5134 .

In operation 5132 , the processing unit (eg, the PIM logic 5116 ) of the integrated circuit may transmit an access request for reading the second number of bits from the memory unit to the memory unit of the integrated circuit. The memory unit may be configured through a memory cell (eg, memory cells of array 5111 ), an output port (eg, output bus 5115 ), and a reduction unit (eg, reduction unit 5030 ) and a first number of output ports. and read/write circuitry that may include a first group of memory read/write paths for input and/or output to bits.

The access request may include a read request and/or a write request.

The memory input/output path may include a memory read path, a memory write path, and/or a path used for both reading and writing.

In step 5134, it may respond to the access request.

For example, in step 5134 , based on the first number of bits and the second number of bits, during an access operation triggered by an access request, a reduction unit (eg, reduction unit 5030 ) is configured to execute a memory read/write path can control

Further, in step 5134, any one of and/or any combination of one or more of the following may be performed. Any of the actions listed below may be executed while responding to an access request, but may also be executed before and/or after responding to an access request.

Accordingly, step 5134 may include at least one of the following.

a. controlling the irrelevant memory read path if the second number is less than the first number, wherein the irrelevant memory read path is associated with bits of the first number of bits that are not included in the second number of bits;

b. activating an associated memory read path during a read operation, wherein the associated memory read path is configured to pass a second number of bits;

c. closing at least a portion of each of the irrelevant memory read paths during a read operation;

d. maintaining an unrelated memory read path in a low power mode during a read operation;

e. controlling a bit line of an irrelevant memory read path;

f. loading the bitlines of the relevant memory read paths and maintaining the bitlines of the unrelated memory read paths in a low power mode;

g. loading the bitlines of the relevant memory read paths while keeping the bitlines of the unrelated memory read paths inactive;

h. utilizing portions of the relevant memory read path and maintaining a portion of each of the unrelated memory read paths in a low power mode during a read operation, wherein the portion is different from a bitline;

i. utilizing portions of the relevant memory read path and maintaining the sense amplifier for at least some unrelated memory read paths in a low power mode during a read operation;

j. utilizing portions of the relevant memory read path during a read operation and maintaining at least the sense amplifier of the unrelated memory read path in a low power mode; and

k. Utilizing the portions of the relevant memory read path during the read operation and maintaining the portions of the irrelevant memory read path after the sense amplifier of the irrelevant memory read path in a low power mode.

The low power mode or idle mode may include a mode in which power consumption of the memory access path is lower than power consumption when the memory access path is used for an access operation. In some embodiments, the low power mode may include closing the memory access path. The low power mode may additionally or alternatively include not activating the memory access path.

Here, the relevance or irrelevance of the memory access path may need to be known before the wordline is opened for power reduction to occur during the bitline phase. Power reductions occurring in other locations (eg, column multiplexers) may enable determination of the relevance or irrelevance of each access's memory access path.

High-speed, low-power activation and fast-access memory

DRAM and other types of memory (SRAM, Flash, etc.) are often constructed from memory banks, which are typically organized with row and column access schemes in mind.

FIG. 55 shows an example of a memory chip 5140 that includes multiple memory mats and associative logic (eg, row and column decoders shown as RD and COL in FIG. 55, respectively). In the example of Figure 55, mats are grouped into banks and wordlines and bitlines run through. The memory bank and associative logic are denoted 5141 , 5142 , 5143 , 5144 , 5145 , and 5146 in FIG. 55 and share at least one bus 5147 .

The memory chip 5140 may be configured as shown in one or more of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, and 23 . It may be included in a memory chip or may include a memory chip.

For example, DRAM has a lot of overhead associated with the activation of a new row (ie preparing a new line for access). Once a line is activated (or open a line), the data in that row becomes available for much faster access. In DRAM, this approach can happen in a random fashion.

Two issues associated with the activation of a new line are power and time.

c. Power rises due to the rapid flow of current caused by having to access all capacitors on the line at once and load the line (for example, if you open a line with only a few memory banks, the power can reach several amps) .

d. The time delay problem is mostly related to the time taken to load a column (bitline) after loading a row (wordline).

Some embodiments of the present disclosure may include a system and method for reducing peak power consumption during activation of a line and reducing activation time of a line. Some embodiments may, to some extent, reduce this power and time cost at the expense of a completely random approach.

For example, in one embodiment, the memory unit does not activate the first group of memory cells included in the first memory mat, the second memory mat, and the second group included in the second memory mat. and an activation unit configured to activate the memory cell of the . Both the first group of memory cells and the second group of memory cells may belong to a single row of the memory unit.

Alternatively, the activation unit may be configured to activate the second group of memory cells included in the second memory mat without activating the first group of memory cells.

In some embodiments, the activation unit may be configured to activate the second group of memory cells after activation of the first group of memory cells.

For example, the activator may be configured to activate the second group of memory cells following expiration of a delay period initiated after activation of the first group of memory cells is completed.

Additionally or alternatively, the activator may be configured to activate the second group of memory cells based on a value of a signal generated on a first wordline coupled to the first group of memory cells.

In any of the embodiments described above, the activator may include an intermediate circuit disposed between the first wordline segment and the second wordline segment. A first wordline segment may be coupled to a first memory cell, and a second wordline segment may be coupled to a second memory cell. Examples of the intermediate circuit include, but are not limited to, switches, flip-flops, buffers, inverters, and the like, some of which are illustrated in FIGS. 56 to 61 .

In some embodiments, the second memory cell may be coupled to a second wordline segment. In such an embodiment, the second wordline segment may be coupled to a bypass wordline path through at least the first memory mat. An example of such a bypass path is shown in FIG. 61 .

The activation unit may include a control unit configured to control supply of voltage (and/or flow of current) to the first group of memory cells and the second group of memory cells based on an activation signal from a wordline associated with the single row. .

In another exemplary embodiment, the memory unit supplies an activation signal to the first memory mat, the second memory mat, and the first group of memory cells of the first memory mat, and at least when activation of the first group of memory cells is complete. and an activation unit configured to delay supply of an activation signal to the memory cells of the second group of the second memory mat until . The first group of memory cells and the second group of memory cells may belong to a single row of memory units.

For example, the activation unit may include a delay unit configured to delay the supply of the activation signal.

Additionally or alternatively, the activator may comprise a comparator configured to receive the activation signal at the input and control the delay part based on at least one characteristic of the activation signal.

In another exemplary embodiment, the memory mat comprises a first memory mat, a second memory mat, and a first memory cell of the first memory mat during an initial activation period in which the first memory cell is activated to a second memory of the second memory mat. an isolation portion configured to isolate from the cell and couple the first memory cell to the second memory cell after an initial activation period. The first and second memory cells may belong to a single row of memory units.

In the examples below, no modifications to the memory mat itself are necessary. In some examples, embodiments may rely on minor modifications to the memory bank.

The figures described below show a mechanism for dividing a word line into several shorter portions by shortening a word signal added to a memory bank.

In the following drawings, various memory bank configurations are omitted for the sake of brevity.

56-61 show multiple memory mats 5150(1), 5150(2), 5150(3), 5150(4), 5150(5), 5150(6), 5151(1) grouped into different groups. , 5151(2), 5151(3), 5151(4), 5151(5), 5151(6), 5152(1), 5152(2), 5152(3), 5152(4), 5152(5) , and 5152(6) and portions of the memory bank including row decoders (5140(1), 5140(2), 5140(3), 5140(4), 5140(5), and 5149(6), respectively) indicated by ) is shown.

The memory mats arranged in rows may include different groups.

56 to 59 and 61 show nine groups of memory mats, each group including a pair of memory mats. Any number of groups may be used, each containing any number of memory mats.

Memory mats 5150(1), 5150(2), 5150(3), 5150(4), 5150(5), and 5150(6) are arranged in rows, share multiple memory lines, and are grouped into three groups. classified as That is, the first upper group includes memory mats denoted 5150(1) and 5150(2), the second upper group includes memory mats denoted 5150(3) and 5150(4), and the third upper group includes memory mats denoted 5150(3) and 5150(4). includes memory mats labeled 5150(5) and 5150(6).

Similarly, memory mats 5151(1), 5151(2), 5151(3), 5151(4), 5151(5), and 5151(6) are arranged in rows and share multiple memory lines, 3 classified into groups of dogs. That is, the first intermediate group includes the memory mats labeled 5151(1) and 5151(2), the second intermediate group includes the memory mats labeled 5151(3) and 5151(4), and the third intermediate group includes the memory mats labeled 5151(3) and 5151(4). includes memory mats labeled 5151(5) and 5151(6).

In addition, memory mats 5152(1), 5152(2), 5152(3), 5152(4), 5152(5), and 5152(6) are arranged in rows, share multiple memory lines, and classified into groups. That is, the first subgroup includes the memory mats labeled 5152(1) and 5152(2), the second subgroup includes the memory mats labeled 5152(3) and 5152(4), and the third subgroup includes the memory mats labeled 5152(3) and 5152(4). includes memory mats labeled 5152(5) and 5152(6). Any number of memory mats can be arranged in rows, share memory lines, and grouped into any number of groups.

For example, the number of memory mats in each group may be 1, 2, or more.

As described above, the activation circuit may be configured to activate one group of memory mats without activating another group of memory mats that share the same memory line. Alternatively, the line addresses may be coupled at least to different memory line segments that are the same.

56 to 61 show different examples of the activation circuit. In some embodiments, at least a portion of activation circuitry (eg, intermediate circuitry) may be positioned between groups of memory mats to allow one group of memory mats to be activated while other groups of memory mats in the same row not activated. have.

Figure 56 shows an intermediate circuit, such as delay or isolation circuitry 5153(1) - 5153(3), positioned between different lines of a first upper group of memory mats and a second upper group of memory mats.

Figure 56 also shows intermediate circuitry, such as delay or isolation circuitry 5154(1) - 5154(3), located between different lines of the second upper group of memory mats and the third upper group of memory mats. Additionally, some delay or isolation circuitry is placed between groups formed from intermediate groups of memory mats. Also, some delay or isolation circuitry is placed between groups formed from subgroups of memory mats.

The delay or defer circuit may delay or stop the wordline signal from the row decoder 5112 from propagating along other groups of rows.

57 shows an intermediate circuit, such as a delay or isolation circuit, comprising flip-flops 5155(1) - 5155(3) and 5156(1)-5156(3).

When an activation signal is injected into a wordline, the first group of mats (according to the wordline) are activated while the other groups on the wordline remain inactive. Another group can be active on the next clock cycle. For example, a second group of another group may be activated at the next clock cycle, and a third group of the other group may be activated after another clock cycle.

A flip-flop may include a D-type flip-flop or any other type of flip-flop. The clock supplied to the D-type flip-flop is omitted from the drawing for simplicity.

Thus, access to the first group can use power to only charge a portion of the wordline associated with the first group, which is faster and requires less current than charging the entire wordline.

More than one flip-flop may be used between groups of memory mats, thereby increasing the delay between open portions. Additionally or alternatively, embodiments may slow the clock to increase delay.

Also, the groups being activated may still contain groups from previously used line values. For example, the method may allow activating a new line segment while still accessing the data of the previous line, thereby reducing the penalty associated with activation of a new line.

Accordingly, some embodiments may have a first group activated, and may cause other groups of previously activated lines to remain activated with a signal of a bit line that does not interfere with each other.

Additionally, some embodiments may include switches and control signals. The control signal may be controlled by the bank controller or may be controlled by adding flip-flops between the control signals (eg, creating the same timing effect as the mechanism described above).

Figure 58 is a switch (such as 5157(1) - 5157(3) and 5158(1)-5158(3)) and shows an intermediate circuit, such as a delay or isolation circuit, positioned between one group and the other. A set of switches located between groups can be controlled by dedicated control signals. 58, the control signal is transmitted by row control 5160(1) and may be delayed by one or more consecutive delays between different sets of switches (eg, 5160(2) and 5160(3)). .

59 is a series of inverter gates or buffers (eg, 159(1) - 5159(3) and 5159'1 (0 - 5159'(3)) and intermediate such as delay or isolation circuitry located between groups of memory mats; circuit is shown.

Instead of a switch, a buffer can be used between groups of memory mats. The buffer can prevent a voltage drop across the wordline between the switch and the switch, an effect that sometimes occurs when using a single transistor structure.

Other embodiments may provide very low activation power and time while allowing more random access by utilizing the added area to the memory bank.

An example of using wordlines (eg, 5152(1) - 5152(8)) located close to a memory mat is shown in FIG. 60 . These wordlines may or may not pass through the memory mat and may couple to wordlines within the memory mat through an intermediate circuit such as a switch (eg, 5157(1) - 5157(8)). Switches can control which memory mats are active and allow the memory controller to only activate the relevant portion of the line at each point in time. Unlike the embodiment using continuous activation of line portions described above, the example of FIG. 60 may provide stronger control.

Enable signals such as low part enable signals 5170 ( 1 ) and 5170 ( 2 ) may originate from logic such as a memory controller (not shown).

Figure 61 shows that global wordline 5180 passes through the memory mat and forms a bypass path for wordline signals that may not need to be routed out of the memory mat. Accordingly, the embodiment shown in FIG. 61 can reduce the area of the memory bank with a slight loss in memory density.

61 , the global wordline may pass through the memory mat without interruption and may not be coupled to a memory cell. A local wordline segment may be controlled by one of the switches and coupled to a memory cell within the memory mat.

If a group of memory mats provide significant partitioning of wordlines, the memory bank can virtually support completely random access.

Another embodiment that also saves some wiring and logic and reduces the propagation speed of the enable signal along the wordlines is to avoid using a dedicated enable signal and a dedicated line to carry the enable signal, but with other buffering or other buffering between memory mats. Use isolation circuits and/or switches

For example, a comparator may be used to control a switch or other buffering or isolation circuit. The comparator may activate a switch or other buffering or isolation circuitry when the level of the signal on the wordline segment monitored by the comparator reaches a certain level. For example, a certain level may indicate that the previous wordline segment was fully loaded.

62 illustrates a method 5190 for operation of a memory unit. For example, method 5190 may be implemented using one or more of the memory banks described above with reference to FIGS. 56-61 .

Method 5190 may include steps 5192 and 5194 .

Step 5192 may include, by the activation unit, activating the first group of memory cells included in the first memory mat of the memory unit without activating the second group of memory cells included in the second memory mat of the memory unit . Both the first group of memory cells and the second group of memory cells may belong to a single row of the memory unit.

Operation 5194 may include an activator activating the second group of memory cells after operation 5192 .

Step 5194 is performed during activation of the first group of memory cells, after full activation of the first group of memory cells, and after expiration of a delay period initiated after activation of the first group of memory cells is completed, the first group of memory cells It can be executed after it has been deactivated, etc.

The delay period may be fixed or adjustable. For example, the length of the delay period may be based on an expected access pattern of the memory unit or may be set irrespective of the expected access pattern. The delay period may range from one thousandths of a second to one second or more.

In some embodiments, step 5194 may be initiated based on a value of a signal generated on a first wordline segment coupled to a first group of memory cells. For example, when the value of the signal exceeds the first threshold value, it may indicate that the memory cells of the first group are fully activated.

One of steps 5192 and 5194 may use an intermediate circuit (eg, an intermediate circuit of an activation unit) disposed between the first wordline segment and the second wordline segment. A first wordline segment may be coupled to a first memory cell, and a second wordline segment may be coupled to a second memory cell.

Examples of intermediate circuits are shown in FIGS. 56-61.

Steps 5192 and 5194 may further include controlling, by the controller, application of an activation signal from a word line associated with a single row to the memory cells of the first group and the memory cells of the second group.

Inspection logic in memory utilizing vector parallelism and memory to reduce inspection time

Some embodiments of the present disclosure may increase the speed of testing by using an in-chip testing unit.

In general, the inspection of a memory chip requires a considerable inspection time. Reducing inspection time can reduce production costs and increase product reliability by allowing more inspections.

63 and 64 show an inspector 5200 and a chip (or wafer of chips) 5210 . The tester 5200 may include software for managing the test. After executing a different sequence of data for the entirety of the memory 5210 , the checker 5200 may read the sequence again to identify a place where the bad bit of the memory 5210 is located. Once recognized, the checker 5200 can generate an instruction to repair the bit, and if the problem is resolved, the checker 5200 can assert that the memory 5210 passes the test. Otherwise, some chips may be flagged as defective.

The tester 5200 may read the data again after recording the test sequence and compare it with an expected result.

64 shows an inspection system with an inspector 5200 and an entire wafer 5202 of chips (eg, 5210) being inspected in parallel. For example, the tester 5200 may be connected to each of the chips by a bus of wiring.

As shown in FIG. 64 , the checker 5200 needs to read and write several times to all of the memory chips, and data must pass through the external chip interface.

It may also be beneficial to test both the logic and memory banks of the integrated circuit using programmable configuration information, etc. that may be provided utilizing normal I/O operations.

Also, it may be advantageous for inspection to have an inspection unit within the integrated circuit.

The inspection unit may belong to the integrated circuit and may analyze the inspection result and detect failures in logic (eg, the processor subunit illustrated and described in FIG. 7A ) and/or memory (eg, an entire plurality of memory banks).

Memory checkers are usually very simple and exchange test vectors with integrated circuits according to a simple form. For example, there may be a write vector containing a pair of addresses of a memory entry to be written and a value to be written to the memory entry. There may also be a read vector containing the address of the memory entry to be read. At least a portion of the address of the write vector may be the same as at least a portion of the address of the read vector. At least some other addresses in the write vector may be different from at least some other addresses in the read vector. Once programmed, the memory checker may also receive an expected result vector which may contain the address of the memory entry to be read and an expected value to be read. The memory checker can compare the expected value to the value it reads.

According to one embodiment, the logic (eg, processor subunit) of an integrated circuit (with or without memory of the integrated circuit) may be tested by the memory tester using the same protocol/format. for example. Part of the value of the write vector may be an instruction to be executed by the logic of the integrated circuit (and may include computation and/or memory access). The memory checker may be programmed with a budget result vector and a read vector which may contain memory entry addresses (at least some of which store the expected values of the calculations). Accordingly, the memory checker can be used to check not only logic but also memory. Memory checkers are generally much simpler and less expensive than logic checkers, and the presented method allows a simple memory checker to be utilized to perform complex logic checks.

In some embodiments, the logic within the memory utilizes only vectors (or other data structures) and uses more complex mechanisms commonly used for logic checking (e.g., communicating with the controller via an interface, etc. to instruct the logic which circuit to test). Enables checking of logic in memory without being utilized.

Instead of using the check unit, the memory controller may be configured to receive a command to access a memory entry included in the configuration information, execute the access command, and output a result.

One or more of the integrated circuits shown in FIGS. 65-69 may perform inspection without an inspection unit or even with an inspection unit incapable of performing the inspection.

Embodiments of the present disclosure may include a method and system for shortening and improving test time by utilizing parallelism of memory and internal chip bandwidth.

The method and system allow the memory chip to test itself (rather than for the inspector to run the test, read the test results, and analyze the results), store the results, and ultimately cause the tester to read the results (and , reprogramming the memory chip, if necessary, to activate the redundancy mechanism, etc.). Testing may include testing of memory or testing of memory banks and logic (in the case of computational memory with functional logic to be tested as previously described with reference to FIG. 7A).

In one embodiment, the method may include reading and writing data within the chip such that external bandwidth does not limit the inspection.

In embodiments where the memory chip includes processor subunits, each processor subunit may be programmed with a test code or configuration.

In embodiments where the memory chip includes a processor subunit that is not capable of executing check code, or does not have a processor subunit but includes a memory controller, the memory controller reads and writes patterns (eg, programmed externally to the controller) for further analysis It may be configured to indicate the location of the bad (eg, write a value to a memory entry, read the entry, and receive a value different from the written value).

Here, the inspection of the memory may require inspection of a vast number of bits, for example, inspecting each bit of the memory and verifying that the inspected bits function. Also, the memory test may be repeated from time to time under different voltage and temperature conditions.

For some faults, more than one redundancy mechanism may be activated (eg by programming flash or OTP or burning a fuse). It may also be necessary to inspect the logic and analog circuits of the memory chip (eg, controllers, regulators, I/Os).

In one embodiment, an integrated circuit may include a substrate, a memory array disposed on the substrate, a processing array disposed on the substrate, and an interface disposed on the substrate.

The integrated circuit described herein may be a circuit as shown in one or more of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, and 23 . It may be included in a memory chip or may include a memory chip.

65 to 69 show various integrated circuits 5210 and testers 5200 .

The integrated circuit includes a memory bank 5212, a chip interface 5211 (e.g., a bus 5213 and I/O controller 5214 shared by the memory bank), and logic (hereinafter referred to as “logic”) 5215 is shown to include. 66 shows a fuse interface 5216 and a bus 5217 coupled to a different memory bank than the fuse interface.

65-70 also illustrate the various stages of the inspection process as follows.

a. writing the check sequence 5221 (FIGS. 65, 67, 68, 69);

b. reading the test result ( 5222 ) ( FIGS. 67 , 68 , and 69 );

c. writing the expected result sequence 5223 (FIG. 65);

d. reading a bad address to be corrected (5224) (FIG. 66); and

e. Programming the fuse 5225 (FIG. 66).

Each memory bank may be coupled to its own logic 5215 and/or controlled by logic 5215 . However, as previously described, any arbitrary assignment to logic 5215 may be provided. Accordingly, the number of logic units 5215 may be different from the number of memory banks, and the logic units may control one or more memory banks or portions of one memory bank, and the like.

The logic unit 5215 may include one or more check units. Further, reference numeral 65 shows a check unit (TU) 5218 in the logic unit 5215 . A TU may be included in all or part of the logic unit 5215 . Here, the inspection unit may be separate from the logic unit or may be integrated with the logic unit.

65 also shows an inspection pattern generator (denoted GEN) 5219 within the TU 5218 .

The inspection pattern generator may be included in all or part of the inspection unit. For simplicity of illustration, the inspection pattern generator and the inspection unit are not shown in FIGS. 66 to 70 , but may be included in these embodiments.

A memory array may include multiple memory banks. In addition, the processing array may include a plurality of inspection units. The plurality of test units may be configured to test multiple memory banks and provide test results. The interface may be configured to output information indicative of the test result to a device external to the integrated circuit.

The plurality of inspection units may include at least one inspection pattern generator configured to generate at least one inspection pattern for use in one or more inspections of multiple memory banks. In some embodiments, as described above, each of the plurality of inspection units may include an inspection pattern generator configured to generate an inspection pattern to be used by a specific inspection unit of the plurality of inspection units to test at least one of the multiple memory banks. As described above, FIG. 65 shows an inspection pattern generator (GEN) 5219 in the inspection unit. One or more or all of the logic may include a check pattern generator.

The at least one inspection pattern generator may be configured to receive a command from the interface to generate the at least one inspection pattern. The test pattern may include memory entries to be accessed (eg, read and/or written) and/or values to be written to the entries during the test.

The interface may be configured to receive configuration information including commands for generating at least one test pattern from an external device that may be external to the integrated circuit.

The at least one test pattern generator may be configured to read configuration information including a command for generating the at least one test pattern from the memory array.

In some embodiments, the configuration information may include a vector.

The interface may be configured to receive configuration information comprising a command, which may be at least one test pattern, from a device that may be external to the integrated circuit.

For example, the at least one test pattern may include a memory array entry to be accessed during a test of the memory array.

The at least one test pattern may further include input data to be written to the memory array accessed during the test of the memory array.

Additionally or alternatively, the at least one check pattern may further include an expected value of input data to be written to a memory array entry accessed during testing of the memory array and output data to be read from a memory array entry accessed during testing of the memory array. can

In some embodiments, the plurality of inspection units may be configured to obtain a test command from the memory array that, when executed by the plurality of inspection units, causes the plurality of inspection units to inspect the memory array.

For example, the inspection command may be included in the configuration information.

The configuration information may include expected results of memory array tests.

Additionally or alternatively, the configuration information may include values of output data to be read from memory array entries accessed during inspection of the memory array.

Additionally or alternatively, the configuration information may include a vector.

In some embodiments, the plurality of inspection units may be configured to obtain a test command from the memory array that, when executed by the plurality of inspection units, causes the plurality of inspection units to inspect the memory array and to inspect the processing array.

For example, the inspection command may be included in the configuration information.

The configuration information may include a vector.

Additionally or alternatively, the configuration information may include expected results of memory array tests and processing array tests.

In some embodiments, as described above, the plurality of inspection units may not have an inspection pattern generator for generating inspection patterns used during inspection of multiple memory banks.

In such an embodiment, at least two test units of the plurality of test units may be configured to test at least two memory banks of a multiple memory bank in parallel.

Alternatively, at least two test sections of the plurality of test sections may be configured to test at least two memory banks of the multiple memory bank in series.

In some embodiments, the information indicative of the test result may include an identifier of a bad memory array entry.

In some embodiments, the interface may be configured to fetch partial test results obtained by the plurality of test circuits multiple times during testing of the memory array.

In some embodiments, the integrated circuit may include an error correction unit configured to correct at least one error detected during inspection of the memory array. For example, the error correction unit may be configured to correct memory errors utilizing any suitable method, such as disabling some memory words and replacing these words with duplicate words.

In any of the above embodiments, the integrated circuit may be a memory chip.

For example, an integrated circuit may include a distributed processor, wherein the processing array may include a plurality of subunits of the distributed processor as shown in FIG. 7A .

In such embodiments, each of the processor subunits may be associated with a corresponding dedicated memory bank of multiple memory banks.

In any of the above-described embodiments, the information indicating the test result may indicate the state of at least one memory bank. The state of a memory bank may be provided at one or more granularities per memory word, per group of entries, or per entire memory bank.

65 to 66 show four stages of the inspection machine inspection step.

In a first step, the tester writes 5221 the test sequence, and the logic of the memory bank writes the data to the memory. The logic may be complex enough to receive commands from the tester and generate its own test sequence (discussed below).

In the second step, the checker writes the expected result to the checked memory ( 5223 ), and the logic unit compares the expected result with data read from the memory bank and stores a list of errors. The step of recording expected results may be simplified if the logic is complex enough to itself generate a sequence of expected results (discussed below).

In a third step, the checker reads 5224 the bad address from the logic.

In a fourth step, the checker may act 5225 based on the result and correct the error. For example, you could connect to a particular interface and program a fuse in memory, but utilize any other mechanism that allows you to program an error correction mechanism in memory.

In such an embodiment, the memory checker may utilize the vector to check the memory.

For example, each vector can be constructed from an input series and an output series.

The input series may contain pairs of addresses and data to be written to memory (in many embodiments, this series may be modeled as a formula that allows a program, such as a program executed by logic, to be created on demand) have).

In some embodiments, the inspection pattern generator may generate such a vector.

Here, a vector is an exemplary data structure, but other embodiments may use other data structures. The data structure may conform to other inspection data structures generated by a tester located external to the integrated circuit.

The output series may contain data and address pairs containing expected data to be read from memory (in some embodiments, the series may be generated by a program at runtime, e.g., by a logic unit) have).

Memory checking usually involves running a list of vectors that reads data according to an output series and compares it to expected data after each vector writes data to memory according to the input series.

In case of a mismatch, the memory may be classified as bad, or if there is a mechanism for redundancy in the memory, the redundancy mechanism may be activated, causing the vector to be checked again on the activated redundancy mechanism.

In embodiments where the memory includes a processor subunit (described with reference to FIG. 7A ) or includes multiple memory controllers, the full check may be handled by the logic of the memory bank. Accordingly, the memory controller or the processor subunit may perform the check.

The memory controller may be programmed from the tester, and the results of the test may be stored in the memory controller itself for later reading by the tester.

To configure and check the operation of the logic, the checker may configure the logic to be a memory access and verify that the check result can be read by the memory access.

For example, the input vector may contain the programming sequence for the logic, and the output vector may contain the expected result of this check. For example, if logic, such as a processor subunit, includes a multiplier or adder configured to perform calculations on two addresses in memory, the input vector is a set of instructions that write data to memory and a summer May contain a set of instructions to write to the /multiplier logic. As long as the summer/multiplier result can be read into an output vector, the result can be sent to the checker.

The checking may further include loading the logic configuration from the memory and causing the logic output to be transferred to the memory.

In embodiments where the logic loads its configuration from memory (eg, where the logic is a memory controller), the logic may execute code from the memory itself.

Accordingly, the input vector can include a program for the logic section, and the program itself can check the various circuits of the logic section.

Thus, inspection may not be limited to receiving vectors in a format used by an external inspector.

When the instruction loaded into the logic instructs the logic to write the results to the memory bank, the checker can read these results and compare them to the expected output series.

For example, a vector written to memory is a check program for logic or a check program (e.g. the check assumes that the memory is valid, but even if it is not, the written check program will not work, and the check fails , which is an acceptable result since the chip is not valid anyway) and/or how logic executes code and writes the result to memory. All checks of the logic part can be performed via memory (writing the logic check inputs to memory and writing the test results), so the checker can perform simple vector checks with the input sequence and the expected output sequence.

Logic constructs and results can be accessed with read and/or write commands.

68 shows a checker 5200 that transmits a write check sequence 5221 that is a vector.

Portions of the vector include test code 5232 divided between memory banks 5212 coupled to logic 5215 of the processing array.

Each logic 5215 may execute code 5232 stored in an associated memory bank, which execution may access one or more memory banks, perform calculations, and execute results (eg, test results 5231 ). storing in memory bank 5212 .

The test results may be returned by the tester 5200 (eg, read the results 5222).

As such, the logic 5215 may be controlled by commands received by the I/O controller 5214 .

In Figure 68, I/O controller 5214 is logically coupled with the memory bank. In another embodiment, logic may be coupled between the I/O controller 5214 and the memory bank.

70 illustrates a method 5300 of examining a memory bank. For example, method 5300 may be implemented utilizing one or more of the memory banks previously described with reference to FIGS. 65-69 .

Method 5300 may include steps 5302 , 5310 , and 5320 . At step 5302, a request may be received to examine a memory bank of the integrated circuit. An integrated circuit may include a substrate, a memory array disposed on the substrate and including a memory bank, a processing array disposed on the substrate, and an interface disposed on the substrate. The processing array may include a plurality of inspection units as described above.

In some embodiments, the request may include configuration information, one or more vectors, instructions, and the like.

In such embodiments, the configuration information may include expected results of a memory array test, commands, data, values of output data to be read from memory array entries accessed during test of the memory array, test patterns, and the like.

The check pattern is derived from (i) memory array entries to be accessed during testing of the memory array, (ii) input data to be written to the memory array accessed during testing of the memory array, and (iii) memory array entries accessed during testing of the memory array. at least one of the expected values of the output data to be read.

Step 5302 may include at least one of the following steps and/or may be followed by at least one of the following steps.

a. receiving, by the at least one inspection pattern generator, a command for generating the at least one inspection pattern from the interface;

b. receiving, by the interface, configuration information including a command for generating at least one inspection pattern from an external device external to the integrated circuit;

c. at least one test pattern generator reading configuration information including a command for generating at least one test pattern from the memory array;

d. receiving, by the interface, configuration information including a command that is at least one test pattern from an external device external to the integrated circuit;

e. retrieving, by the plurality of inspection units, a test command from the memory array, which, when executed by the plurality of inspection units, causes the plurality of inspection units to inspect the memory array; and

f. receiving a test command from the memory array that, when executed by the plurality of test units, causes the plurality of test units to test the memory array and test the processing array.

Step 5310 may be performed after step 5302. In operation 5310 , a plurality of inspection units may inspect multiple memory banks in response to the request and provide inspection results.

The method 5300 may further include the interface receiving the partial test results obtained by the plurality of test units multiple times during the test of the memory array.

Step 5310 may include at least one of the following steps and/or may be followed by at least one of the following steps.

a. generating, by one or more inspection pattern generators (eg, an inspection pattern generator included in one, some, or all of the plurality of inspection units), an inspection pattern to be used by the inspection unit to inspect at least one of the one or more multiple memory banks;

b. testing, by at least two test units of the plurality of test units, at least two memory banks of a multi-memory bank in parallel;

c. testing, by at least two test units of the plurality of test units, at least two memory banks of a multi-memory bank in series;

d. writing a value to the memory entry, reading the memory entry, and comparing the results; and

e. Correcting, by the error correction unit, at least one error detected during inspection of the memory array.

After step 5310, step 5320 may be performed. In operation 5320, the interface may output information indicating the test result to the outside of the integrated circuit.

The information indicating the inspection result may include an identifier of a bad memory array entry. This saves time by not sending read data for each memory entry.

Additionally or alternatively, the information indicative of the test result may indicate the status of at least one memory bank.

Accordingly, in some embodiments, the information indicative of the test result may be much smaller than the total size of data units written to or read from the memory bank during the test, and may be transmitted from the tester examining the memory without the aid of the test unit. It can be much smaller than the input data that is present.

The tested integrated circuit may include a memory chip and/or a distributed processor as shown in one or more of the figures described above. For example, the integrated circuit described herein may be shown in one or more of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, and 23 . It may be included in a memory chip as shown or may include a memory chip.

71 shows an example of a method 5350 of examining a memory bank of an integrated circuit. For example, method 5350 may be implemented utilizing one or more of the memory banks previously described with reference to FIGS. 65-69 .

Method 5350 may include steps 5352 , 5355 , and 5358 . In step 5352, the interface of the integrated circuit may receive configuration information comprising a command. An integrated circuit including an interface may also include a substrate, a memory array including a memory bank and disposed on the substrate, a processing array disposed on the substrate, and an interface disposed on the substrate.

The configuration information may include expected results of a memory array test, commands, data, values of output data to be read from memory array entries accessed during test of the memory array, test patterns, and the like.

Additionally or alternatively, the configuration information may include an instruction, an address of a memory entry into which to write the instruction, input data, and may also include an address of a memory entry for receiving an output value calculated during execution of the instruction.

The inspection pattern is derived from (i) memory array entries to be accessed during inspection of the memory array, (ii) input data to be written to the memory array accessed during inspection of the memory array, and (iii) memory array entries accessed during inspection of the memory array. at least one of the expected values of the output data to be read.

After step 5352, step 5355 may be performed. At step 5355, the processing array can execute instructions by accessing the memory array, performing computational operations, and providing results.

Step 5358 may be performed after step 5355. In step 5358, the interface may output information indicating the result to the outside of the integrated circuit.

Cybersecurity and counterfeit detection methods

Memory chips and/or processors may be targeted by malicious users and may be the target of various types of cyberattacks. In some cases, such an attack may be an attempt to alter data and/or code stored in one or more memory resources. Cyberattacks can be particularly problematic for learning neural networks or other types of artificial intelligence (AI) models that rely on significant amounts of data stored in memory. If stored data is manipulated or even obscured, such manipulation can be detrimental. For example, an autonomous vehicle system that relies on data-intensive AI models to identify other vehicles or pedestrians, etc. may misinterpret the surroundings of the host vehicle if the data it relies on becomes corrupted or unclear. As a result, accidents may occur. As AI models are applied in a wider range of technologies, cyberattacks on the data associated with these models pose a significant risk of disruption.

In other instances, a cyber attack may involve one or more actors falsifying or attempting to falsify operating parameters associated with a processor or other type of integrated circuit based logic element. For example, processors are often designed to operate within specific operating specifications. Cyberattacks, including counterfeiting, may attempt to exceed designed operating specifications (e.g. clock speed, bandwidth specifications, temperature limits, operating speed, etc.) by changing one or more of the operating parameters of a processor, memory unit, or other circuitry. have. Such counterfeiting can lead to failure of the target hardware.

Existing methods for defense against cyber attacks may include computer programs (eg, anti-virus software, anti-malware software) running at the processor level. Other approaches may include the use of a software-based firewall associated with a router or other hardware. While these methods are countermeasures against cyberattacks using software programs executed outside of the memory unit, additional or alternative methods for efficiently protecting data stored in the memory unit are still needed, and the accuracy and availability of such data This is especially true when it is absolutely necessary for the operation of memory intensive applications such as neural networks. Embodiments of the present disclosure provide various integrated circuit designs including a memory that is resistant to cyber attacks on the memory.

After fetching sensitive information and commands to the integrated circuit in a secure way (e.g. during the boot process when the interface out of the chip/integrated circuit is not yet active), keep such sensitive information and commands within the integrated circuit without exposing it outside the integrated circuit and completing operations within the integrated circuit can improve the security of sensitive information and instructions. CPUs and other types of processing units are vulnerable to cyber attacks, especially when such CPUs and processing units operate in conjunction with external memory. Disclosed embodiments comprising a distributed processor subunit disposed on a memory chip of a memory array comprising a plurality of memory banks may be less susceptible to cyberattacks and counterfeiting (eg, as processing takes place within the memory chip). . Any and every combination of the disclosed safety measures described in greater detail below may further reduce the vulnerability of the disclosed embodiments to cyber attacks and/or counterfeiting.

72A illustrates an integrated circuit 7200 including a memory array and a processing array in accordance with embodiments of the present disclosure. For example, integrated circuit 7200 may include any and all distributed processor-on-a-memory chip architecture (and functionality) described above and generally described herein. The memory array and the processing array may be formed on the same substrate, and in some embodiments of the present disclosure, the integrated circuit 7200 may constitute a memory chip. For example, as described above, the integrated circuit 7200 may include a memory chip including a plurality of memory banks and a plurality of processor subunits spatially distributed on the memory chip, wherein the plurality of memory banks include: each associated with one or more dedicated subunits of the plurality of processor subunits. In some cases, each processor subunit may be dedicated to one or more memory banks.

In some embodiments, as shown in FIG. 72A , the memory array may include a plurality of discrete memory banks 7210_1 , 7210_2 , ... 7210_J1 , 7210_Jn. According to embodiments of the present disclosure, the memory array 7210 may include, for example, volatile memory (RAM, DRAM, SRAM, phase change RAM (PRAM), magnetoresistive RAM (MRAM), resistive RAM (ReRAM), etc.) or non-volatile memory may include one or more types of memory including (flash memory or ROM). According to some embodiments of the present disclosure, the memory banks 7210_1 to 7210_Jn may include a plurality of MOS memory structures.

As mentioned above, the processing array may include a plurality of processor subunits 7220_1 to 7220_K. In some embodiments, the processor subunits 7220_1 - 7220_K may each be associated with one or more discrete memory banks among the plurality of discrete memory banks 7210_1 - 7210_Jn. 72A, each processor subunit is shown associating with two discrete memory banks 7210; however, each processor subunit may be associated with any and any number of dedicated discrete memory banks it would be natural Or, conversely, each memory bank may be associated with any and all number of processor subunits. According to embodiments of the present disclosure, the number of discrete memory banks included in the memory array of the integrated circuit 7200 is equal to, less than, or equal to, compared to the number of processor subunits included in the processing array of the integrated circuit 7200 . , can be many.

The integrated circuit 7200 may further include a plurality of first buses 7260 according to (and described above) embodiments of the present disclosure. Each bus 7260 may connect a processor subunit 7220_k to a corresponding dedicated memory bank 7210_j. According to some embodiments of the present disclosure, the integrated circuit 7200 may further include a plurality of second buses 7261 . Each bus 7261 may connect a processor subunit 7220_k to another processor subunit 7220_k+1. 72A , a plurality of processor subunits 7220_1 to 7220_K may be connected to each other through a bus 7261 . Although a plurality of processor subunits 7220_1 through 7220_K are shown connected in series via a bus 7261 in FIG. 72A to form a loop, processor subunit 7220 may be connected in any and any other manner. will be taken for granted. For example, in some cases, a particular processor subunit may not be coupled to another processor subunit via bus 7261 . In other cases, a particular processor subunit may be coupled to only one other processor subunit, and in still other cases, a particular processor subunit may be coupled to two or more other processor subunits via one or more buses 7261 (eg, serially). connected, paralleled, branched, etc.). Here, the described embodiments of the integrated circuit 7200 are merely examples. In some cases, integrated circuit 7200 may have different internal components and connections, and in other cases, one or more internal components and described connections may be omitted (eg, depending on the needs of a particular application).

Referring back to FIG. 72A , the integrated circuit 7200 may include one or more structures for implementing at least one security measure for the integrated circuit 7200 . In some cases, such structures may be configured to detect cyber hoaxes that manipulate or obscure (or attempt to manipulate or obscure) data stored in one or more memory banks. In other instances, such a structure may include falsification of operating parameters associated with integrated circuit 7200 or one or more hardware elements that directly or indirectly affect one or more operations associated with integrated circuit 7200 (within integrated circuit 7200 ). hardware elements included or external to the integrated circuit 7200).

In some cases, controller 7240 may be included in integrated circuit 7200 . The controller 7240 may be coupled to, for example, one or more of the processor subunits 7220_1 ... 7220_k via one or more buses 7250 . The controller 7240 may also be connected to one or more of the memory banks 7210_1 ... 7210_Jn. Although one controller 7240 is shown in FIG. 72A , it will be appreciated that the controller 7240 may include multiple processor elements and/or logic circuits. In the disclosed embodiments, the controller 7240 may be configured to implement at least one security measure for at least one operation of the integrated circuit 7200 . Further, in the disclosed embodiments, the controller 7240 may be configured to take (or trigger) one or more solutions when at least one security action is triggered.

According to some embodiments of the present disclosure, the at least one security measure may include a process implemented by the controller to block access to a particular aspect of the integrated circuit 7200 . Blocking access involves having the controller prevent access (for read and/or write purposes) to specific areas of memory from outside the chip. Access control may be applied to address resolution, determination of a portion of a memory bank, determination of a memory bank, and the like. In some cases, one or more physical locations of memory associated with integrated circuit 7200 (eg, one or more memory banks of integrated circuit 7200 or any portion of one or more memory banks) may be blocked. In some embodiments, the controller 7240 may block access to certain portions of the integrated circuit 7200 associated with the execution of the artificial intelligence model (or other type of software-based system). For example, in some embodiments, the controller 7240 may block access to the weights of the neural network model stored in the memory associated with the integrated circuit 7200 . Here, a software program (ie, model) may include three components: input data to the program, code data of the program, and output data from program execution. These components can also be applied to neural network models. In the course of operation of such a model, input data may be generated and supplied to the model, and executing the model may generate output data for reading. However, program codes and data values associated with model execution using the received input data may be fixed.

As used herein, the term 'blocking' may refer to, for example, an operation of a controller that does not allow a read or write operation to a specific area of a memory initiated from the outside of a chip/integrated circuit. A controller through which the I/O of a chip/integrated circuit can pass can block an entire bank of memory, but can use any range of memory addresses within a memory bank, e.g. You can also block up to a range of memory addresses (or any address range in between).

Since memory locations associated with receiving input data and storing output data are associated with change values and interactions with components external to integrated circuit 7200 (components that provide input data or receive output data), in some cases these Blocking access to memory locations may not be practical. On the other hand, blocking access to memory locations associated with model code and fixed data values can be effective against certain types of cyberattacks. Accordingly, in some embodiments, memory associated with program code and data values (eg, memory that is not used to write/receive input data and read/provide output data) may be blocked as a security measure. Access restrictions may include blocking certain memory locations from changing certain program code and/or data values (eg, values associated with the execution of a model based on received input data). Also, a memory area associated with intermediate data (eg, data generated during the execution of a model) may be blocked from external access. Accordingly, various computational logic, whether on or external to integrated circuit 7200 , may provide data to, or receive data from, a memory location associated with receiving input data or retrieving generated output data. On the other hand, such computational logic would not have the ability to access or modify memory locations that store program code and data values associated with program execution based on received input data.

In addition to blocking memory locations on the integrated circuit 7200 to provide security measures, other security measures may be implemented by restricting access to specific computational logic elements (and memory regions accessed) configured to execute code associated with a specific program or model. can In some cases, such access restrictions may include computational logic (and associated memory regions) located on integrated circuit 7200 (eg, computational memory (eg, memory having computational capabilities, such as a distributed processor on a memory chip of the present disclosure), etc.). can be done for arithmetic logic (and associated memory regions) associated with any arbitrary execution of code stored in the blocked memory portion of integrated circuit 7200 or associated with any arbitrary access to data values stored in the blocked memory portion of integrated circuit 7200 ) may also be blocked/restricted regardless of whether the corresponding operation logic is located on the integrated circuit 7200 . Restricting access to the computational logic responsible for the execution of the program/model can further ensure that protection from being manipulated, obscured, and the like, of code and data values associated with operations according to received input data is maintained.

The security measures implemented by the controller, including blocking or restricting access to hardware-based regions associated with particular portions of the memory array of integrated circuit 7200, may be in any and any suitable manner. In some embodiments, such blocking may be accomplished by adding or providing a command to the controller 7240 configured to cause the controller 7240 to block a particular portion of memory. In some embodiments, the hardware-based memory portion to be blocked may be designated by a specific memory address (eg, an address associated with any and all memory elements of memory banks 7210_1 ... 7210_J2, etc.). In some embodiments, a blocked region of memory may remain frozen during program or model execution. In other cases, the blocked area may be configurable. That is, in some cases, the controller 7240 may be provided with instructions such that the blocked region may be changed during execution of the program or model. For example, at a specific time, a specific memory location may be added to a blocked area of memory. Alternatively, at a specific time, a specific memory location (eg, a previously blocked memory location) may be excluded from a blocked area of memory.

Blocking of a particular memory location may be accomplished in any and all manner. In some cases, writes of blocked memory locations (eg, files, dataspaces that store and identify blocked memory addresses; data structures, etc.) may be accessible by the controller 7240 . In some cases, the controller 7240 maintains a database of blocked addresses that is used to control access to specific memory locations. In other cases, there is a table or set of one or more registers that are configurable until blocked and may contain a fixed predetermined value that identifies a memory location to block (e.g. a memory location where memory access from outside the chip is restricted). It could be in the controller. For example, when a memory access is requested, the controller 7240 may compare a memory address associated with the corresponding memory access request to a blocked memory address. When it is determined that the memory address associated with the memory access request is in the list of blocked memory addresses, the memory access request (regardless of whether a read or write operation is performed) may be rejected.

As described above, the at least one security measure may include blocking access to specific memory portions of the memory array 7210 that are not used for receiving input data or providing access to generated output data. In some cases, portions of the memory within the blocked region may be adjusted. For example, a blocked memory portion may be unblocked, and an unblocked memory portion may be blocked. Any and all suitable methods may be utilized to unblock the blocked memory portion. For example, implementing security measures may require a password to unblock one or more portions of a blocked memory area.

The implemented security measures may be triggered when an action contrary to the implemented security measures is detected. For example, an attempt to access a blocked portion of memory (whether a read or write request) can trigger a security measure. In addition, if a password that does not match a predetermined password is input (eg, an attempt to unblock a blocked memory portion), security measures may be triggered. In some cases, a security action may be triggered if the correct password is not entered within an allowed threshold number of password entry attempts (eg 1, 2, 3, etc.).

A portion of the memory may be blocked at any time. For example, in some cases, portions of memory may be blocked at various times during program execution. In other cases, portions of memory may be blocked at the beginning and/or prior to program/model execution. For example, the memory address to be blocked may be determined and identified with programming of the program/model code or with the creation and storage of data to be accessed by the program/model. Accordingly, the vulnerability to the memory array 7210 attack may be reduced or eliminated during or after the program/model execution is started, after data to be used by the program/model is generated and stored, and the like.

Blocked memory may be unblocked by any suitable method or at any suitable time. As described above, the blocked memory portion can be unblocked after receipt of the correct password. In other cases, blocked memory may be unblocked by restarting (either by command or by power cycling) or erasing the entire memory array 7210 . Additionally or alternatively, one or more memory portions may be unblocked by executing a sequence of unblock commands.

According to embodiments of the present disclosure, as also described above, the controller 7240 may be configured to control traffic to and from the integrated circuit 7200 , in particular from a source external to the integrated circuit 7200 . For example, as shown in FIG. 72A , traffic between a component external to the integrated circuit 7200 and a component internal to the integrated circuit 7200 (eg, a memory array 7210 or a processor subunit 7220) is controlled by the controller 7240 . Such traffic may pass through the controller 7240 or one or more buses controlled or monitored by the controller 7240 (eg, 7250, 7260, or 7261).

According to some embodiments of the present disclosure, the integrated circuit 7200 transmits immutable data (eg, fixed data such as model weights, coefficients, etc.) or a specific instruction (eg, a code identifying a portion of memory to be blocked) during the booting process. can receive Here, the immutable data may refer to data that is continuously fixed during execution of a program or model and remains unchanged until a subsequent booting process. During program execution, integrated circuit 7200 may interact with immutable data, which may include input data to be processed and/or output data generated by processing associated with integrated circuit 7200 . As previously discussed, access to memory array 7210 or processing array 7220 may be restricted during program or model execution. For example, access to particular portions of memory array 7210 or access to particular processor subunits associated with processing or interaction with input data to be written or processing or interaction with generated output data to be read is restricted. can be During program or model execution, portions of memory containing immutable data may be blocked and thereby rendered inaccessible. In some embodiments, the immutable data and/or instructions associated with the portion of memory to be blocked may be included in any and all suitable data structures. For example, such data and/or commands may be made available to the controller 7240 via one or more configuration files accessible during or after the boot sequence.

Referring back to FIG. 72A , the integrated circuit 7200 may further include a communication port 7230 . 72A , a controller 7240 may be coupled between a communication port 7230 and a bus 7250 shared between processing subunits 7220_1 - 7220_K. In some embodiments, communication port 7230 may be coupled directly or indirectly to a host computer 7270 associated with host memory 7280 , which may include non-volatile memory or the like. In some embodiments, host computer 7270 stores mutable data 7281 (eg, input data to be used during execution of a program or model), immutable data 7282 , and/or instructions 7283 to associated host memory. (7280). Mutable data 7281 , non-modifiable data 7282 , and commands 7283 may be uploaded from the host computer 7270 to the controller 7240 via the communication port 7230 during the boot procedure.

72B illustrates a memory area inside an integrated circuit according to embodiments of the present disclosure. As shown, FIG. 72B depicts an example of a data structure contained in host memory 7280 .

73A illustrates another example of an integrated circuit according to embodiments of the present disclosure. As shown in FIG. 73A , the controller 7240 may include a cyberattack detector 7241 and a response module 7242 . In some embodiments of the present disclosure, the controller 7240 may be configured to store or access the access control rule 7243 . According to some embodiments of the present disclosure, the access control rule 7243 may be included in a configuration file accessible to the controller 7240 . In some embodiments, access control rules 7243 may be uploaded to controller 7240 during the boot procedure. Access control rules 7243 may include information indicative of access rules associated with any and all mutable data 7281 , immutable data 7282 , commands 7283 , and their corresponding memory locations. As described above, the access control rule 7243 or the configuration file may include information identifying a specific memory address in the memory array 7210 . In some embodiments, the controller 7240 may be configured to provide a blocking mechanism and/or function to block various addresses of the memory array 7210, eg, addresses for storing commands or immutable data.

The controller 7240 may be configured to enforce the access control rules 7243 , eg, to prevent unauthorized entities from changing immutable data or commands. In some embodiments, reading of immutable data or commands may be prohibited according to access control rule 7243 . According to some embodiments of the present disclosure, the controller 7240 may be configured to determine whether there is an attempt to access at least a portion of a specific command or immutable data. The controller 7240 (eg, including the cyber attack detector 7241 ) compares the memory address associated with the access request to the memory address for immutable data and instructions to determine whether an unauthorized access attempt was made to one or more blocked memory locations. can be detected. Thereby, for example, the cyber attack detector 7241 of the controller 7240 can determine whether a suspected cyber attack is occurring, e.g., changing one or more commands or altering the immutable data associated with one or more blocked memory portions, or and may be configured to determine whether there is a request to disambiguate. The response module 7242 may be configured to determine how to respond to a detected cyber attack and/or how to implement a response. For example, in some cases, in response to a detected attack on data or commands in one or more blocked memory locations, the response module 7242 of the controller 7240 may perform one or more actions, such as a memory access operation associated with the detected attack. Implement or cause a response to be implemented, which may include stopping, etc. Response to a detected attack may also include aborting one or more operations associated with execution of a program or model, outputting a warning or other indicator of an attack attempt, asserting an instruction line to the host, or erasing entire memory, and the like.

In addition to blocking the memory portion, other methods of protecting against cyber attacks may be implemented to provide the described security measures associated with the integrated circuit 7200 . For example, in some embodiments, the controller 7240 may be configured to duplicate a program or model within a processor subunit and a different memory location associated with the integrated circuit 7200 . Thereby, the program/model and the copy of the program/model can be executed separately from each other, and the results of the individual program/model execution can be compared. For example, the program/model may be copied to two different memory banks 7210 and executed on different processor subunits 7220 within the integrated circuit 7200 . In other embodiments, the program/model may be replicated on two different integrated circuits 7200 . In either case, the results of the program/model runs can be compared to determine whether differences exist between duplicate program/model runs. A detected difference in execution results (eg, intermediate execution results, final execution results, etc.) may be indicative of the presence of a cyberattack that has altered one or more aspects of the program/model or data associated therewith. In some embodiments, different memory bank 7210 and processor subunit 7220 may be configured to execute two replicated models based on the same input data. In some embodiments, intermediate results may be compared while running two replicated models based on the same input data, and if there is a discrepancy between two intermediate results in the same step, execution may be halted as a potential solution. have. When processor subunits of the same integrated circuit run two replicated models, the integrated circuit can also compare the results. This can be done without informing entities external to the integrated circuit about the execution of the two replicated models. That is, the entity outside the chip is unaware that the replica models are running in parallel on the integrated circuit.

73B illustrates a configuration for concurrently executing a replication model according to embodiments of the present disclosure. Although a single program/model clone is described as an example for detecting the likelihood of a cyber attack, any and any number of clones (eg 1, 2, 3, or more) may be used to detect the likelihood of a cyber attack. . As the number of clones and the execution of individual programs/models increases, the level of confidence in detecting cyberattacks may also increase. As the number of clones increases, the probability of success of a cyber attack also decreases, as it may be more difficult for an attacker to influence multiple program/model clones. The number of program or model clones can be determined at runtime, further increasing the difficulty for a cyber attacker to successfully influence the program or model execution.

In some embodiments, replicated models may differ from each other in one or more aspects. In this example, the codes associated with the two programs/models may be different from each other, but all of these programs/models may be designed to produce the same output result. At least in this way, the two programs/models can be considered to be replicas of each other. For example, two neural network models may have different order of neurons in a layer relative to each other. However, despite these changes in the model code, the same output result can be obtained. Cloning a program/model in this way may make it more difficult for a cyber attacker to identify such an effective copy of the program or model to compromise, and as a result, the clone model/program provides redundancy that minimizes the impact of a cyber attack. cyberattack detection as well as improving cyberattack detection (e.g. by highlighting counterfeiting or unauthorized access where a cyberattacker changes one program/model or its data but does not change a copy of the program/model correspondingly) .

In many cases, clone programs/models (especially including clone programs/models that exhibit code differences) construct soft values (e.g. nearly identical output values) that are not exact and fixed values, although their outputs are not exactly identical. can be designed to In such embodiments, output results from two or more effective program/model replicas are compared (e.g., utilizing a dedicated module or host processor) to determine if the difference between the output results (intermediate result or final result) is within a predetermined range. can determine whether A difference in the output soft value that does not exceed a predetermined threshold or range may be considered as evidence that forgery, unauthorized access, or the like has not occurred. On the other hand, when the difference between the output soft values exceeds a predetermined threshold or range, the difference may be considered as evidence that there has been a cyber attack in the form of forgery or unauthorized access to the memory. In such a case, cloning program/model security measures may be triggered and one or more remedies (eg, halt execution of the program or model, halt one or more operations of integrated circuit 7200 , operate in safe mode limiting functionality, etc.) ) can be taken.

Security measures associated with integrated circuit 7200 may also include quantitative analysis of data associated with execution of a program or model. For example, in some embodiments, the controller 7240 performs one or more checksum/hash/cyclic redundancy check (CRC)/parity ( parity) value. The calculated value(s) may be compared to a predetermined value(s). If there is a difference between the compared values, the difference may be interpreted as evidence of forgery in data stored in at least a portion of the memory array 7210 . In some embodiments, a checksum/hash/CRC/parity value may be calculated for all memory locations associated with the memory array 7210 to identify a change in data. In this example, the entire memory (or memory bank) in question can be read by the host computer 7270 or a processor associated with the integrated circuit 7200, or the like, for calculation of checksum/hash/CRC/parity values. In other cases, checksum/hash/CRC/parity values may be calculated for a given subset of memory locations associated with memory array 7210 to identify changes in data associated with the subset of memory locations. In some embodiments, the controller 7240 may be configured to calculate a checksum/hash/CRC/parity value (eg, associated with a memory access pattern) associated with a given data path, the computed values being either relative to each other or to a given data path. It can be compared to the value of to determine whether there has been a forgery or other form of cyberattack.

Integrated circuit 7200 may include one or more predetermined values (eg, expected checksum/hash/CRC/parity values, intermediate output results, or final output results) at locations within or accessible to integrated circuit 7200 . Security against cyberattacks can be further strengthened by protecting the expected difference value, the expected difference range associated with a specific value, etc.). For example, in some embodiments, one or more predetermined values may be stored in registers of memory array 7210 and utilized to evaluate intermediate or final output values, checksums, etc. during or after each run of the model (e.g., : by the controller 7240 of the integrated circuit 7200). In some cases, register values may be updated utilizing a “save final result data” instruction to directly compute a given value, and the computed value may be stored in a register or other memory location. Thereby, a valid output value can be utilized to update a predetermined value that is utilized for comparison after each program or model run or partial run. Such methods may increase the difficulty of cyber attackers' attempts to alter or falsify one or more predetermined reference values designed to expose cyber-attack activity.

In operation, a CRC calculator may be used to track memory accesses. For example, such calculation circuitry may be located at the memory bank level, in a processor subunit, or in a controller, each of which may be configured to accumulate with a CRC calculator for each memory access.

74A illustrates another embodiment of an integrated circuit 7200 . 74A , the controller 7240 may include a spoof detector 7245 and a response module 7246 . Similar to other disclosed embodiments, spoof detector 7245 may be configured to detect evidence of a potential spoof attempt. According to some embodiments of the present disclosure, for example, a security measure associated with integrated circuit 7200 and implemented by controller 7240 may include comparison of an actual program/model operating pattern with a predefined/allowed operating pattern. can A security action may be triggered when the actual program/model operating pattern differs from a predetermined/allowed operating pattern in one or more aspects. When a security action is triggered, the response module 7246 of the controller 7240 may be configured to respond by implementing one or more remedies.

74C illustrates a detection element that may be located at various points within a chip according to an example of the present disclosure. The detection of cyber attacks and counterfeiting as described above may be performed by utilizing detection elements located at various points within the chip as shown in the example of FIG. 74C . For example, particular code may be associated with an expected number of processing events within a particular period of time. The detector shown in FIG. 74C is capable of counting the number of events (monitored by the event counter) that the system experiences during a particular period of time (monitored by the time counter). If the number of events exceeds a predetermined threshold (eg, the number of events expected during a predetermined period of time), this may indicate a forgery. 74C, such detectors may be included at multiple points in the system to monitor various types of events.

More specifically, in some embodiments, the controller 7240 may be configured to store or access the budget program/model operating pattern 7244 . For example, in some cases, the operating pattern may be depicted as a graph 7247 representing an allowed load pattern per hour and a forbidden or illegal pattern per hour. Counterfeiting attempts may cause memory array 7210 or processing array 7220 to operate outside of specific operating specifications. This may cause the memory array 7210 or processing array 7220 to heat up or not function properly, and may cause data or code associated with the memory array 7210 or processing array 7220 to change. As a result of these changes, an operating pattern outside the permitted operating pattern may result, as shown in graph 7247 .

According to some embodiments of the present disclosure, the controller 7240 may be configured to monitor an operating pattern associated with the memory array 7210 or the processing array 7220 . The operation pattern may be associated with the number of access requests, the type of access request, the time of the access request, and the like. The controller 7240 may be further configured to detect a spoof attack when the operation pattern is different from the allowable operation pattern.

Here, the disclosed embodiments may be utilized for protection against non-malicious errors in operation in addition to protection against cyber attacks. For example, the disclosed embodiments may also be effective in protecting a system such as integrated circuit 7200 against errors due to environmental factors such as temperature or voltage changes or levels, particularly when the level is outside the operating specifications of the integrated circuit 7200. have.

Any and all suitable remedies may be implemented in response to the detection of a suspected cyber hoax (eg in response to a triggered security action). For example, a solution may include stopping one or more operations associated with program/model execution, operating one or more components associated with integrated circuit 7200 in a safe mode, blocking additional input or access to one or more components of integrated circuit 7200, etc. may include

74B depicts a flow diagram of a method 7450 of protecting an integrated circuit against counterfeiting in accordance with the disclosed exemplary embodiments. For example, in step 7452, a controller associated with the integrated circuit may be used to implement at least one security measure for the operation of the integrated circuit. At step 7454 , if at least one security action is triggered, one or more solutions may be taken. The integrated circuit may include a substrate, a memory array disposed on the substrate and comprising a plurality of discrete memory banks, and a processing array disposed on the substrate and comprising a plurality of processor subunits, each of the plurality of processor subunits comprising: It may be associated with one or more discrete memory banks of the plurality of discrete memory banks.

In some embodiments, the disclosed security measures may be implemented in multiple memory chips, and at least one or more of the disclosed security mechanisms may be implemented for each memory chip/integrated circuit. In some cases, each memory chip/integrated circuit may implement the same security measures, while in other cases different memory chips/integrated circuits may implement different security measures (eg, different security measures may may be more suitable for the specific type of behavior involved). In some embodiments, one or more security measures may be implemented by a particular controller of the integrated circuit. For example, a particular integrated circuit may implement any and all number or type of security measures disclosed. Further, a particular integrated circuit controller may be configured to implement a number of different solutions in response to the triggered security action.

Here, two or more of the above security mechanisms may be combined to enhance security against a cyber attack or a forgery attack. Also, the security measures may be implemented on different integrated circuits, and these integrated circuits may cooperate with each other to implement the security measures. For example, model replication may be performed in one memory chip or both in different memory chips. In such an example, results from one memory chip or results from two or more memory chips may be compared to detect the possibility of a cyber attack or spoof attack. In some embodiments, replication security measures applied to all multiple integrated circuits may include one or more of the disclosed access blocking mechanism, hash protection mechanism, model replication, program/model execution pattern analysis, and any and all combinations thereof or other disclosed embodiments. may include

Multi-port processor subunit in DRAM

As previously described, embodiments disclosed herein may include a distributed processor memory chip comprising an array of processor subunits and an array of memory banks, wherein each of the processor subunits is dedicated to at least one of the array of memory banks. can be As discussed below, a distributed processor memory chip can serve as the basis for a scalable system. That is, in some cases, a distributed processor memory chip may include one or more communication ports configured to transfer data from one distributed processor memory chip to another distributed processor memory chip. Thereby, any and any desired number of distributed processor memory chips may be interconnected (eg, in series, parallel, loop, or any combination thereof) to form a scalable array of distributed processor memory chips. Such an array can efficiently perform memory-intensive operations and provide a flexible solution for scaling computational resources associated with performing memory-intensive operations. Because distributed processor memory chips may include clocks with different timing patterns, embodiments disclosed herein include features that accurately control data transfer between distributed processor memory chips even with clock timing differences. Such embodiments may enable efficient data sharing between different distributed processor memory chips.

75A illustrates a scalable processor memory system including a plurality of distributed processor memory chips according to embodiments of the present disclosure. According to embodiments of the present disclosure, a scalable processor memory system includes a plurality of distributed processor memory chips, such as a first distributed processor memory chip 7500 , a second distributed processor memory chip 7500 ′, and a third distributed processor memory chip 7500 ″. The first distributed processor memory chip 7500, the second distributed processor memory chip 7500', and the third distributed processor memory chip 7500" are each any may include any and all configurations and/or features associated with all embodiments of

In some embodiments, the first distributed processor memory chip 7500, the second distributed processor memory chip 7500', and the third distributed processor memory chip 7500" are each integrated circuit 7200 shown in FIG. 72A. 75A, the first distributed processor memory chip 7500 may include a memory array 7510, a processing array 7520, and a controller 7540. Memory Array 7510 , processing array 7520 , and controller 7540 may be configured similarly to memory array 7210 , processing array 7220 , and controller 7240 of FIG. 72A .

According to embodiments of the present disclosure, the first distributed processor memory chip 7500 may include a first communication port 7530 . In some embodiments, first communication port 7530 may be configured to communicate with one or more external entities. For example, the communication port 7530 may be configured to establish a communication connection between the distributed processor memory chip 7500 and an external entity other than the distributed processor memory chip, such as the distributed processor memory chips 7500', 7500". For example, the communication port 7530 may be coupled directly or indirectly to a host computer, or any and any other computing device, communication module, etc. as shown in Figure 72 .

According to embodiments of the present disclosure, the first distributed processor memory chip 7500 may further include one or more additional communication ports configured to communicate with other distributed processor memory chips (eg, 7500', 7500"). Some In embodiments, the one or more additional communication ports may include a second communication port 7531 and a third communication port 7532 as shown in Figure 75A. The second communication port 7531 may be a second distributed may be configured to communicate with the processor memory chip 7500' and establish a communication connection between the first distributed processor memory chip 7500 and the second distributed processor memory chip 7500' Similarly, the third communication port 7532 ) may be configured to communicate with the third distributed processor memory chip 7500" and establish a communication connection between the first distributed processor memory chip 7500 and the third distributed processor memory chip 7500". Some embodiments In certain instances, the first distributed processor memory chip 7500 (and any and all memory chips disclosed herein) may be any and any suitable number (eg, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 50, 100, 1000, etc.) may include a plurality of communication ports including communication ports.

In some embodiments, the first communication port, the second communication port, and the third communication port are associated with a corresponding bus. The corresponding bus may be a bus common to each of the first communication port, the second communication port, and the third communication port. In some embodiments, a corresponding bus associated with each of the first communication port, the second communication port, and the third communication port are all coupled to the plurality of discrete memory banks. In some embodiments, the first communication port is connected to at least one of a main bus inside the memory chip and at least one processor subunit included inside the memory chip. In some embodiments, the second communication port is connected to at least one of a main bus inside the memory chip and at least one processor subunit included within the memory chip.

Although the disclosed configuration of the distributed processor memory chip has been described with respect to the first distributed processor memory chip 7500 , the second distributed processor memory chip 7500 ′ and the third distributed processor memory chip 7500 ″ are also described with respect to the first distributed processor memory chip. It may be configured similarly to 7500. For example, the second distributed processor memory chip 7500' may also include a memory array 7510', a processing array 7520', a controller 7540', and/or a port. It may include a plurality of communication ports, such as 7530', 7531', 7532'. Similarly, the third distributed processor memory chip 7500" also includes a memory array 7510", a processing array 7520", a controller 7540", and/or a plurality of communication ports, such as ports 7530", 7531", 7532". In some embodiments, the second communication port 7531 ′ and the third communication port 7532 ′ of the second distributed processor memory chip 7500 ′ are connected to the third distributed processor memory chip 7500 ″ and the first distributed processor Each may be configured to communicate with the memory chip 7500. Similarly, the second communication port 7531 ″ and the third communication port 7532″ of the third distributed processor memory chip 7500 ″ may be configured to communicate with the first distributed processor memory chip 7500 ″. It may be configured to communicate with the processor memory chip 7500 and the second distributed processor memory chip 7500 ′, respectively. Scaling of a computing system based on the disclosed distributed processor memory chip may be facilitated due to configuration similarity between distributed processor memory chips. Furthermore, the disclosed configuration of communication ports associated with each distributed processor memory chip may allow flexible placement of arrays of distributed processor memory chips (eg, serial, parallel, loop, star, mesh, etc. connections).

According to embodiments of the present disclosure, distributed processor memory chips such as the first to third distributed processor memory chips 7500 , 7500 ′ and 7500 ″ may communicate with each other via the bus 7533 . Some embodiments According to , the bus 7533 may connect two communication ports of two different distributed processor memory chips, for example, the second communication port 7531 of the first processor memory chip 7500 may be connected to the bus 7533 may be connected to the third communication port 7532' of the second processor memory chip 7500'. According to embodiments of the present disclosure, the first to third distributed processor memory chips 7500, 7500', and 7500 ) may communicate with an external entity (eg, a host computer) via a bus such as bus 7534 . For example, the first communication port 7530 of the first distributed processor memory chip 7500 may be coupled to one or more external entities via a bus 7534 . Distributed processor memory chips may be interconnected in a variety of ways. In some cases, distributed processor memory chips may exhibit serial connectivity in which each distributed processor memory chip is coupled to a pair of adjacent distributed processor memory chips. In other cases, distributed processor memory chips may exhibit a higher degree of connectivity in which at least one distributed processor memory chip is coupled to two or more other distributed processor memory chips. In some cases, all distributed processor memory chips in a plurality of memory chips may be plurally coupled to all other distributed processor memory chips.

75A, bus 7533 (or any other bus associated with the embodiment of FIG. 75A) may be unidirectional. While bus 7533 is shown in FIG. 75A as unidirectional and with a specific data transfer flow (indicated by arrows in FIG. 75A), bus 7533 (or any other bus in FIG. 75A) is implemented as a bidirectional bus. can be According to some embodiments of the present disclosure, a bus connected between two distributed processor memory chips may be configured to have a faster communication speed than a bus connected between the distributed processor memory chip and an external entity. In some embodiments, the communication between the distributed processor memory chip and an external entity is generated by the execution of the neural network model for a limited amount of time, eg, during preparation for execution (loading of program code, input data, weight data, etc. from the host computer). This may occur during the cycle of outputting the output to the host computer. During execution of one or more programs associated with the distributed processor memory chips 7500, 7500', 7500" (eg, during memory intensive operations associated with artificial intelligence applications, etc.), communications between the distributed processor memory chips 7500, 7500', 7500", etc. In some embodiments, communication between a distributed processor memory chip and an external entity may occur less frequently than communication between two processor memory chips.According to communication requirements and embodiments, the distributed processor The bus between the memory chip and the external entity may be configured such that a communication speed is equal to, faster, or slower than a communication speed of the bus between the distributed processor memory chips.

In some embodiments, as depicted in FIG. 75A , a plurality of distributed processor memory chips, such as first through third distributed processor memory chips 7500 , 7500 ′, 7500″ may be configured to communicate with each other. The capability may facilitate assembly of a scalable distributed processor memory chip system. For example, the memory arrays 7510, 7510', 7510 of the first through third distributed processor memory chips 7500, 7500', 7500". ") and processing arrays 7520, 7520', 7520" may be considered to effectively belong to a single distributed processor memory chip when connected by a communication channel (eg, the bus shown in FIG. 75A).

According to embodiments of the present disclosure, communication between a plurality of distributed processor memory chips and/or communication between a distributed processor memory chip and one or more external entities may be managed in any and any suitable manner. In some embodiments, such communication may be managed by a processing resource, such as processing array 7520 of distributed processor memory chip 7500 . In some other embodiments, such as to offload processing resources provided by an array of distributed processors from computational load due to communication management, a controller, such as controllers 7540, 7540', 7540" of a distributed processor memory chip, is be configured to manage communication between the processor memory chips and/or communication between the distributed processor memory chip(s) and one or more external entities, for example, first to third distributed processor memory chips 7500, 7500' , 7500") may be configured to manage communications of corresponding distributed processor memory chips to other distributed processor memory chips associated with other distributed processor memory chips. Some implementations In examples, the controller 7540, 7540', 7540" may be configured to control such communication via a corresponding communication port, such as ports 7531, 7531', 7531", 7532, 7532', 7532", etc. .

Controllers 7540, 7540', 7540" may also be configured to manage communications between distributed processor memory chips while taking into account time differences that may exist between them. For example, distributed processor memory chips (e.g., : 7500) may be supplied by an internal clock which may be different (eg different timing pattern) from the clocks of other distributed processor memory chips (eg 7500', 7500"). Accordingly, in some embodiments, the controller 7540 implements one or more strategies to account for different timing patterns between the distributed processor memory chips and to account for possible time variability between the distributed processor memory chips for communication between the distributed processor memory chips. can be configured to manage

For example, in some embodiments, the controller 7540 of the first distributed processor memory chip 7500 is configured to transfer data from the first distributed processor memory chip 7500 to the second distributed processor memory chip 7500 ′ under certain conditions. may be configured to enable data transfer. In some cases, the controller 7540 may suspend data transmission when one or more processor subunits of the first distributed processor memory chip 7500 are not ready to transmit data. Alternatively or additionally, the controller 7540 may suspend data transmission when the receiving processor subunit of the second distributed processor memory chip 7500' is not ready to receive the data. In some cases, the controller 7540 indicates that a transmitting processor subunit (eg, a subunit inside chip 7500) is ready to transmit data and a receiving processor subunit (eg, a subunit inside chip 7500'). After confirming that it is all ready to receive this data, data transmission from the transmitting processor subunit to the receiving processor subunit may be initiated. In other embodiments, the controller 7540 may be the transmitting processor subunit, particularly if data may be buffered in the controller (eg, 7540 or 7540') until the receiving processor subunit is ready to receive the transmitted data. Data transmission can be initiated only based on whether this data is ready to be transmitted.

According to embodiments of the present disclosure, the controller 7540 may be configured to determine whether one or more other timing constraints have been met to enable data transmission. These timing constraints may include the time difference between the transmit time of the transmitting processor subunit and the receive time of the receiving processor subunit, the data access request being processed by an external entity (eg, a host computer), the memory associated with the transmitting processor subunit and the receiving processor subunit. It may be related to a refresh operation performed on a resource (eg, a memory array).

75E exemplarily illustrates timing according to embodiments of the present disclosure. 75E shows the following example.

In some embodiments, controller 7540 and other controllers associated with the distributed processor memory chip may be configured to utilize a clock enable signal to manage data transfer between the chips. For example, the processing array 7520 may be supplied by a clock. In some embodiments, whether one or more processor subunits respond to the supplied clock signal may be controlled by the controller 7540 or the like utilizing a clock enable signal (labeled “CE” in FIG. 7A ). Each processor subunit (eg, 7520_1 to 7520_K) may execute a program code, and the program code may include a communication instruction. According to some embodiments of the present disclosure, the controller 7540 may control the timing of a communication command by controlling a clock enable signal to the processor subunits 7520_1 to 7520_K. For example, if the transmitting processor subunit (eg, the processor subunit of the first distributed processor memory chip 7500) is programmed to transmit data at a specific cycle (eg, the 1000th clock cycle) and the receiving processor subunit (eg, the processor subunit of the first distributed processor memory chip 7500) : When the processor subunit of the second distributed processor memory chip 7500 ′) is programmed to receive data at a specific cycle (eg, the 1000th clock cycle), the controller 7540 of the first distributed processor memory chip 7500 . ) and the controller 7540' of the second distributed processor memory chip 7500' may not allow data transmission until both the transmitting processor subunit and the receiving processor subunit are ready to perform data transmission. For example, the controller 7540 is configured to provide a specific clock enable signal (eg, logic low) that prevents the transmit processor subunit from transmitting data in response to the received clock signal to the transmit processor subunit by supplying it from the transmit processor subunit. data transmission can be 'waited'. Certain clock enable signals can 'freeze' all or part of a distributed processor memory chip. On the other hand, the controller 7540 supplies the transmit processor subunit with an opposite clock enable signal (eg, logic high) that causes the transmit processor subunit to correspond to the received clock signal to the transmit processor subunit, so that the transmit processor subunit initiates data transmission. may cause it to A similar operation such as whether the subunit receives or does not receive in the reception processor of the chip 7500' may be controlled by using the clock enable signal provided by the controller 7540'.

In some embodiments, the clock enable signal may be sent to all processor sub-units (eg, 7520_1 to 7520_K) in the processor memory chip (eg, 7500). The clock enable signal may generally have the effect of causing the processor subunit to either respond to or ignore each clock signal. For example, in some cases, when the clock enable signal is high (according to the convention of a particular application), the processor subunit may correspond to the clock signal and execute one or more instructions according to the clock signal timing. . On the other hand, when the clock signal is low, the processor subunit may be prevented from responding to the clock signal and thus not executing the instruction in response to the clock timing. That is, when the clock enable signal is low, the processor subunit may ignore the received clock signal.

Referring back to FIG. 75A, any controller 7540, 7540', 7540" utilizes a clock enable signal to cause one or more processor subunits in that array to either respond or not respond to a received clock signal, thereby utilizing each distributed It may be configured to control the operation of the processor memory chip. In some embodiments, the controllers 7540, 7540', 7540" selectively control code execution, eg, whether such code relates to data transfer operations and their timing. When including data transfer operations and their timing, they may be configured to proceed. In some embodiments, the controller 7540, 7540', 7540" utilizes a clock enable signal to communicate via any communication port 7531, 7531', 7531", 7532, 7532', 7532" to two different It may be configured to control the timing of data transfers between the distributed processor memory chips. In some embodiments, the controllers 7540, 7540', 7540" utilize a clock enable signal to any communication port 7531, 7531. ', 7531", 7532, 7532', 7532") may be configured to control the time of data reception between two different distributed processor memory chips.

In some embodiments, the data transfer timing between two different distributed processor memory chips may be set based on a compilation optimization step. Compilation allows building processing routines in which tasks can be efficiently assigned to processing subunits without being affected by delays in transmission over the bus connected between two different distributed processor memory chips. Compilation may be performed by a compiler in the host computer or transmitted to the host computer. Normally, delays in transmission over the bus between two different distributed processor memory chips can create a data bottleneck for the processing subunits that need the data. The disclosed compilation can schedule data transfers in such a way that the processing unit can continuously receive data even with undesirable transfer delays over the bus.

While the embodiment of Figure 75A includes three ports per distributed processor memory chip 7500', 7500'', and 7500'', a distributed processor memory chip in accordance with the disclosed embodiment may include any and any number of ports. can For example, in some cases, a distributed processor memory chip may include a greater number or fewer ports. In the embodiment of FIG. 75B , each distributed processor memory chip (eg, 7500A-7500I) may be configured with multiple ports. These ports may be substantially the same as or different from each other. In the illustrated example, each distributed processor memory chip includes five ports, including one host communication port 7570 and four chip ports 7572 . Host communication port 7570 is configured for communication (via bus 7534) between any of the distributed processor memory chips in the array as shown in FIG. 75B and a host computer located remotely to the array of distributed processor memory chips, etc. can be Chip port 7572 may be configured to enable communication between distributed processor memory chips via bus 7535 .

Any and all number of distributed processor memory chips may be interconnected. In the example shown in FIG. 75B , including four chip ports per distributed processor memory chip may enable an array in which each distributed processor memory chip is coupled to two or more other distributed processor memory chips, and in some cases, a particular chip may have 4 It can be connected to multiple different distributed processor memory chips. The inclusion of more chip ports in a distributed processor memory chip can further expand the interconnectivity between the distributed processor memory chips.

Also, although FIG. 75B shows that there are two different types of communication ports 7570 and 7572 on the distributed processor memory chips 7500A-7500I, in some cases there is a single type of communication port on each distributed processor memory chip. may be included. In other cases, two or more types of communication ports may be included in one or more distributed processor memory chips. In the example of Figure 75C, each distributed processor memory chip 7500A'-7500C' includes two (or more) communication ports 7570 of the same type. In this embodiment, communication port 7570 may be configured to enable communication with an external entity, such as a host computer, via bus 7534 , and also via bus 7535 for distributed processor memory chips (eg, 7500B). ' and 7500C').

In some embodiments, ports provided on one or more distributed processor memory chips may be used to provide access to one or more hosts. For example, in the embodiment shown in Figure 75D, the distributed processor memory chip includes two or more ports 7570. The port 7570 may constitute a host port, a chip port, or a combination of a host port and a chip port. In the illustrated embodiment, the two ports 7570, 7570' are distributed processor memory chips via buses 7534, 7534' to two different hosts (eg, host computers or computational elements, or other types of logic). access to (7500A) may be provided. Such an embodiment may provide access to the distributed processor memory chip 7500A to two (or more) different host computers. However, in other embodiments, such as when a host entity requires additional bandwidth or parallel access to one or more of the processor subunits/memory banks of the distributed processor memory chip 7500A, the bus 7534, 7534' can all be connected to the same host entity.

In some cases, one or more controllers 7540 , 7540 ′ may be used to control access to the distributed processor subunit/memory bank of the distributed processor memory chip 7500A, as shown in FIG. 75D . In other cases, a single controller may be used to handle communications from one or more external host entities.

Additionally, one or more buses within the distributed processor memory chip 7500A may enable parallel access to the distributed processor subunits/memory banks of the distributed processor memory chip 7500A. For example, the distributed processor memory chip 7500A has a first bus 7580 and a second bus 7580 that enable parallel access to the distributed processor subunits 7520_1 to 7520_6 and corresponding dedicated memory banks 7510_1 to 7510_6, etc. ') may be included. This configuration may enable simultaneous access to two different locations within the distributed processor memory chip 7500A. In addition, in cases where all ports are not used at the same time, it is possible to share hardware resources (eg, a common bus and/or a common controller) within the distributed processor memory chip 7500A, and to configure multiplexed IO with such hardware. have.

In some embodiments, some of the computational units (eg, processor subunits 7520_1 to 7520_6 ) may be coupled to an additional port 7570 ′ or controller, while other computational units may not be coupled. However, data from computational units not connected to additional ports 7570' may enter through the internal grid of connections to computational units connected to ports 7570'. This allows communication to be performed simultaneously on both ports 7570 and 7570' without adding an additional bus.

Although the communication port (eg, 7530 to 7532) and the controller (eg, 7540) are shown as separate elements from each other, according to embodiments of the present disclosure, the communication port and the controller (or any other component) are integrated It will be natural that it can be implemented as a unit. 76 illustrates a distributed processor memory chip 7600 having an integrated controller and an interface module according to embodiments of the present disclosure. As shown in FIG. 76 , the processor memory chip 7600 is implemented with an integrated controller and interface module 7547 configured to perform the functions of the controller 7540 and communication ports 7530 , 7531 , 7532 of FIG. 75 . can be As shown in Figure 76, the controller and interface module 7547 communicates via interfaces 7548_1 to 7548_N similar to communication ports (eg, 7530, 7531, 7532) to external entities, such as one or more distributed processor memory chips, and the like. It may be configured to communicate with different entities. The controller and interface module 7547 may also be configured to control communications between the distributed processor memory chips or between the distributed processor memory chip 7600 and an external entity, such as a host computer. In some embodiments, the controller and interface module 7547 may include communication interfaces 7548_1 through 7548_N configured for parallel communication with one or more other distributed processor memory chips and for parallel communication with an external entity such as a host computer, communication module, etc. have.

77 is a flowchart illustrating data transfer between distributed processor memory chips in the scalable processor memory system shown in FIG. 75A according to embodiments of the present disclosure. For illustration purposes, the flow of data transfer will be described with reference to FIG. 75A and it will be assumed that data is transferred from the first processor memory chip 7500 to the second processor memory chip 7500 ′.

In step S7710, a data transmission request may be received. However, here, as described above, in some embodiments, a data transmission request may not be necessary. For example, in some cases, the timing of data transfer may be predetermined (eg, by specific software code). In this case, data transmission may proceed without a separate data transmission request. Step S7710 may be performed, for example, by the controller 7540 . In some embodiments, the data transfer request may include a data transfer request from one processor subunit of the first distributed processor memory chip 7500 to another processor subunit of the second distributed processor memory chip 7500 ′. have.

In step S7720, a data transmission time may be determined. As described, the data transmission timing may be predetermined and may depend on the execution order of a particular software program. Step S7720 may be performed, for example, by the controller 7540 or the like. In some embodiments, the timing of data transmission is determined by considering (1) whether the transmitting processor subunit is ready to transmit data and/or (2) whether the receiving processor subunit is ready to receive data. can be According to embodiments of the present disclosure, it may also be considered whether one or more other timing constraints have been fulfilled to enable such data transmission. One or more of these timing constraints may be a time difference between a transmission time from the transmitting processor subunit and a reception time of the receiving processor subunit, requesting access to the processed data from an external entity (eg a host computer), sending or receiving it. It may relate to a refresh operation performed on a memory resource (eg, a memory array) associated with the processor subunit, and the like. According to embodiments of the present disclosure, the processing subunit may be supplied by a clock. In some embodiments, the clock supplied to the processing subunit may be controlled using a clock enable signal or the like. According to some embodiments of the present disclosure, the controller 7540 may control the timing of the communication command by controlling the clock enable signal to the processor subunits 7520_1 to 7520_K.

In step S7703, data transmission may be performed based on the data transmission timing determined in step S7720. Step S7730 may be managed by the controller 7540 or the like. For example, the transmitting processor subunit of the first distributed processor memory chip 7500 may transmit data to the receiving processor subunit of the second distributed processor memory chip 75000' according to the data transmission timing determined in step S7720.

The disclosed architecture can be utilized for a variety of applications. For example, in some cases, the architecture may facilitate data such as weights or neuron values or partial neural values associated with a neural network (especially a large neural network) to be shared between different distributed processor memory chips. Also, certain operations, such as SUM, AVG, etc., may require data from multiple different distributed processor memory chips. In such a case, the disclosed architecture may facilitate sharing of such data among multiple different distributed processor memory chips. Furthermore, the disclosed architecture may facilitate the sharing of records between distributed processor memory chips to support concatenated operations of queries, and the like.

In addition, although the described embodiments relate to a distributed processor memory chip, the same principle and method may be applied to a general memory chip that does not include a distributed processor subunit or the like. For example, in some cases, multiple memory chips may be coupled together into a multi-port memory chip to form an array of memory chips without the need for an array of processor subunits. In another embodiment, multiple memory chips may be coupled together to form an array of interconnected memories to provide the host with effectively one large memory comprised of multiple memory chips.

The internal connection of the port may be a connection to a main bus or a connection to one of the internal processor subunits included in the processing array.

In-memory zero value detection

Some embodiments of the present disclosure relate to a memory unit for detecting a zero value stored in one or more specific addresses of a plurality of memory banks. The disclosed zero value detection function of the memory unit may be useful for reducing power consumption of a computing system, and additionally or alternatively, may reduce the processing time required to retrieve a zero value from memory. This function may also eliminate the need for retrieving a zero value from memory on systems where the data read actually has many zero values (e.g. a zero value multiplied by any other value to get zero) and the arithmetic circuitry may be one of the operands. Of particular relevance to computational operations, such as operations such as multiplication/addition/subtraction, that can take advantage of the fact that is 0 and computes the result more time- or energy-efficiently. In this case, detection of the presence of a zero value can be utilized in lieu of memory access and retrieval of the zero value from memory.

In this section, the disclosed embodiments are described with respect to the read function. It should be understood, however, that the disclosed architectures and schemes are equally applicable to zero-value write operations and certain other predetermined non-zero value operations in which other values are likely to appear more frequently.

In the disclosed embodiments, instead of fetching a zero value from memory, when such a value is detected at a specific address, the memory unit is configured with one or more circuitry external to the memory unit (eg, one or more processors located external to the memory unit, CPU etc.) to emit a zero-value indicator. A zero-value is a multi-bit zero-valued zero (eg, zero-value byte, zero-valued word, multi-bit zero-value that is less than one byte, greater than one byte, etc.). Since the zero-value indicator is a 1-bit signal indicating a zero value stored in the memory, transmitting the 1-bit zero-value indicating signal is more advantageous than transmitting n-bit data stored in the memory. A transmitted zero indication can reduce the energy consumption for transmission by 1/n, and is associated with computation of inputs by weights of neurons, convolution, application of kernels to input data, and learning neural networks, artificial intelligence. It can speed up calculations, such as when multiplication operations are involved in calculations and other types of calculations in a wide array. To provide this functionality, the disclosed memory unit detects the presence of a zero value in a specific location in the memory, prevents retrieval of a zero value (eg, via a read command), and provides a zero-value indicator to circuitry external to the memory unit. It may instead include one or more zero-value detection logic to be transmitted (eg, utilizing one or more control lines of memory, one or more buses associated with the memory unit, etc.). Zero value detection may be performed at the memory mat level, bank level, subbank level, chip level, and the like.

Here, although the disclosed embodiments are described with respect to passing a zero-value indicator to a location external to the memory chip, the disclosed embodiments and functions may be significantly advantageous when processing is performed inside the memory chip. For example, in embodiments such as the distributed processor memory chip disclosed herein, processing may be performed on data in various memory banks by corresponding processing subunits. In many cases, such as in the execution of data analysis or neural networks where the associated data may contain many zeros, the disclosed scheme may increase the speed and/or reduce power consumption of processing associated with processing performed by processor subunits of a distributed processor memory chip. can bring

78A illustrates a system 7800 for detecting a zero value stored in one or more specific addresses of a plurality of memory banks implemented in a memory chip 7810 at a chip level according to embodiments of the present disclosure. The system 7800 may include a memory chip 7810 and a host 7820 . The memory chip 7810 may include a plurality of controllers, and each controller may include a dedicated memory bank. For example, the control may be operatively coupled to a dedicated memory bank.

For example, in some cases involving a distributed processor memory chip of the present disclosure that includes processor subunits spatially distributed between an array of memory banks, processing within the memory chip may involve memory accesses (whether read or written). have. Even in the case of processing inside a memory chip, the disclosed method of detecting a zero value associated with a read or write command may cause the internal processor unit or subunit to abandon the transmission of the actual zero value. Instead, in response to detection of a zero-value and transmission of a zero-value indicator (eg, to one or more internal processing subunits), the distributed processor memory chip may conserve energy that may have been expended in transmitting zero-valued data within the memory chip. have.

In another example, the memory chip 7810 and the host 7820 may include input/output (IO) that enables communication between the memory chip 7810 and the host 7820 , respectively. Each IO may be coupled to a zero value indicator line 7830A and a bus 7840A. The zero-value indicator line 7830A may transmit a zero-value indicator from the memory chip 7810 to the host 7820, where the zero-value indicator detects a zero value stored at a specific address in the memory bank requested by the host 7820. Accordingly, the 1-bit signal generated by the memory chip 7810 may be included. When the host 7820 receives the zero-value indicator through the zero-value indicator line 7830A, it may perform one or more predefined actions associated with the zero-value indicator. For example, if host 7820 has requested memory chip 7810 to retrieve an operand for multiplication, host 7820 from a received zero-value indicator that one of the operands is zero (without receiving an actual memory value) ), the host 7820 can calculate the multiplication more efficiently. The host 7820 can also provide commands, data, and other inputs to the memory chip 7810 and read outputs from the memory chip 7810 over the bus 7840 . Upon receiving the communication from the host 7820 , the memory chip 7810 may retrieve data associated with the received communication and transmit the retrieved data to the host 7820 through the bus 7840 .

In some embodiments, the host may send a zero value indicator to the memory chip instead of a zero data value. This allows the memory chip (eg, a controller disposed on the memory chip) to store or refresh zero values in memory without having to receive zero data values. These updates may occur upon receipt of a zero-value indicator (eg, as part of a write command).

78B illustrates a memory chip 7810 that detects a 0 value stored in one or more specific addresses of a plurality of memory banks 7811A and 7811B at a memory bank level according to embodiments of the present disclosure. The memory chip 7810 may include a plurality of memory banks 7811A and 7811B and an IO bus 7812 . Although FIG. 78B shows two memory banks 7811A and 7811B implemented in the memory chip 7810, the memory chip 7810 may include any number of memory banks.

The IO bus 7812 may be configured to transmit and receive data to and from an external chip (eg, the host 7820 of FIG. 78A ) via the bus 7840B. Bus 7840B may function similarly to bus 7840A in FIG. 78A. IO bus 7812 may also transmit a zero-value indicator over zero-value indicator line 7830B, which may function similarly to zero-value indicator line 7830A in FIG. 78A. IO bus 7812 may also be configured to communicate with memory banks 7811A, 7811B via internal zero-value indicator line 7831 and bus 7841 . The IO bus 7812 may transfer the received data from an external chip to one of the memory banks 7811A, 7811B. For example, the IO bus 7812 may transmit data including a read command of data stored at a specific address of the memory bank 7811A through the bus 7841 . A multiplexer (mux) may be included between the IO bus 7812 and the memory banks 7811A, 7811B and may be connected by an internal zero value indicator line 7831 and a bus 7841A. The multiplexer may be configured to transfer received data from the IO bus 7812 to a particular memory bank, and may be further configured to transfer received data or a received zero value indicator from the particular memory bank to the IO bus 7812 .

In some cases, the host entity may be configured to receive only normal data transmissions, and may not be configured to interpret or respond to an initiated zero-value indicator. In this case, the disclosed embodiments (eg, controller/chip IO, etc.) may reproduce a zero value for the data line to the host IO instead of the zero-value indicator signal, thereby saving data transmission power internally of the chip. can

Memory banks 7811A and 7811B each include a control unit. The control unit may detect a value of 0 stored in the requested address of the memory bank. Upon detecting a stored zero value, the control unit may generate a zero value indicator and transmit it to the IO bus 7812 via an internal zero-value indicator line 7831, which may then be transmitted to an external zero-value indicator via an internal zero-value indicator line 7830B. can be transmitted to the chip.

79 illustrates a memory bank 7911 for detecting a zero value stored in one or more specific addresses of a plurality of memory mats at a memory mat level according to embodiments of the present disclosure. In some embodiments, memory bank 7911 can be organized into memory mats 7912A, 7912B, each memory mat can be individually controlled and accessed. Memory bank 7911 may include memory mat controllers 7913A, 7913B, which may include zero value detection logic 7914A, 7914B. Memory mat controllers 7913A and 7913B may enable reading and writing to locations on memory mats 7912A and 7912B, respectively. Memory bank 7911 may further include read disable elements, local sense amplifiers 7915A, 7915B, and/or global sense amplifiers 7916 .

Memory mats 7912A and 7912B may each include a plurality of memory cells. Each of the plurality of memory cells may store 1-bit binary information. For example, any and all memory cells may individually store a value of zero. If all memory cells of a particular memory mat store a value of 0, the value of 0 may be associated with the entire memory mat.

The memory mat controllers 7913A and 7913B may each be configured to access a dedicated memory mat, and may read data stored in the dedicated memory mat or write data to the dedicated mat.

In some embodiments, the zero value detection logic 7914A or 7914B may be implemented in the memory bank 7911 . One or more zero value detection logic 7914A, 7914B may be associated with a memory bank, a memory subbank, a memory mat, and a set of one or more memory cells. The zero value detection logic 7914A or 7914B may detect that a specific requested address (eg, the memory mat 7912A or 7912B) stores a zero value. Detection can be performed in several ways.

A first method involves utilizing a digital comparator to zero. The digital comparator may be configured to take two numbers as inputs in binary form and determine whether a first number (received data) is equal to a second number (0). If the digital comparator determines that the two numbers are equal, the zero-value detection logic may generate a zero-value indicator. The zero-value indicator can be a 1-bit signal and includes an amplifier (e.g., local sense amplifiers 7915A, 7915B), a transmitter, and a buffer that can transfer data bits to the next level (e.g., IO bus 7812 in Figure 78B). can be deactivated. The zero-value indicator may be further sent to the global sense amplifier 7916 via the zero-value indicator line 7931A or 7931B, but may bypass the global sense amplifier in some cases.

A second method of zero value detection may include utilizing an analog comparator. An analog comparator can function similarly to a digital comparator except that it utilizes the voltages of the two analog inputs for comparison. For example, all of the bits can be sensed, and the comparator can act as a logical OR function between the signals.

A third method of zero value detection may include utilizing signals transmitted from local sense amplifiers 7915A, 7915B into a global sense amplifier 7916, wherein the global sense amplifier 7916 selects a high (non-zero) among the inputs. ) and is configured to use this logic signal to control the next level of the amplifier. The local sense amplifiers 7915A, 7915B and the global sense amplifier 7916 sense low-power signals from a plurality of memory banks, and data stored in the plurality of memory banks is transmitted to at least one controller, such as a memory mat controller 7913A or 7913B. may include a plurality of transistors configured to amplify a small voltage swing to a high voltage level so as to be interpreted by For example, memory cells may be laid out in rows and columns on the memory bank 7911 . Each line may be attached to each memory cell in a row. The lines along the rows are called wordlines and are activated by selectively applying a voltage. The lines along the column are called bitlines, and two complementary bitlines can be attached to the sense amplifiers at the edges of the memory array. The number of sense amplifiers may correspond to the number of bitlines (columns) on the memory bank 7911 . To read a bit from a particular memory cell, the wordline along the row of cells is turned on, activating all memory cells in the row. The value (0 or 1) stored in each cell is then made available to the bitline associated with that particular cell. A sense amplifier at the end of the two complementary bitlines can amplify a small voltage to a normal logic level. The bits from the desired cell may then be latched from the cell's sense amplifier into a buffer and placed on an output bus.

A fourth method of zero-value detection may include utilizing an additional bit per each word stored in memory and stored at write time if the value is zero, and reading the data to know if the data is zero or not, this addition You can use bits. This method avoids writing all zeros to memory, which further saves energy.

As described above and described throughout this disclosure, some embodiments may include a memory unit (eg, memory unit 7800 ) that includes a plurality of processor subunits. These processor subunits may be spatially distributed on a single substrate (eg, a substrate of a memory chip such as the memory unit 7800 ). Also, each of the plurality of processor subunits may be dedicated to a corresponding one of the plurality of memory banks of the memory unit 7800 . In addition, such a memory bank dedicated to the processor sub-unit may also be spatially distributed on the substrate. In some embodiments, memory unit 7800 may be associated with a particular task (eg, one or more operations associated with executing a neural network, etc.), and each processor subunit of memory unit 7800 is responsible for performing a portion of such task. can be in charge For example, each processor subunit may be provided with instructions that may include data processing and memory operations, arithmetic and logical operations, and the like. In some cases, the zero value detection logic may be configured to provide a zero value indicator to one or more of the described processor subunits spatially distributed on the memory unit 7800 .

80 is a flowchart of an exemplary method 8000 of detecting a zero value stored at a specific address of a plurality of memory banks in accordance with embodiments of the present disclosure. Method 8000 may be performed by a memory chip (eg, memory chip 7810 in FIG. 78B ). Specifically, the controller of the memory unit (eg, the controller 7913A of FIG. 79 and the zero value detection logic unit (eg, the zero value detection logic unit 7914A)) may perform the method 8000 .

At step 8010, a read or write operation may be initiated in any suitable manner. In some cases, the controller may receive a request to read data stored at a specific address of a plurality of discrete memory banks (eg, the memory bank shown in FIG. 78 ). The controller may be configured to control at least one aspect of read/write operations for the plurality of discrete memory banks.

At 8020, one or more zero-value detection circuits may be used to detect the presence of a zero-value associated with a read or write command. For example, the 0 value detection logic unit (eg, the 0 value detection logic unit 7830 of FIG. 78 ) may detect a 0 value associated with a specific address associated with reading or writing.

In step 8030 , the controller may transmit a zero-value indicator to one or more circuits external to the memory unit in response to detecting a zero value by the zero-value detection logic in step 8020 . For example, the zero value detection logic may determine that the requested address stores a zero value and sends an indication that the value is zero outside the memory chip (or in the disclosed distributed processor including processor subunits distributed in an array of memory banks). In the case of a memory chip, etc., it may be transmitted to an entity (eg, one or more circuits) inside the memory chip. If a zero value is not detected as being associated with a read or write command, the controller may send a data value instead of a zero value indicator. In some embodiments, one or more circuitry through which a zero-value indicator is transmitted may be internal to the memory unit.

Although the disclosed embodiments have been described with respect to detecting a zero value, the same principles and methods may be applied to detecting other memory values (eg, 1, etc.). In some cases, in addition to the zero-value indicator, the detection logic may send one or more indicators of other values (eg, 1, etc.) associated with a read or write command, such indicators that any and all values corresponding to the value indicators are detected. may be transmitted in some cases. In some cases, values may be adjusted by the user (eg, updating one or more registers). Such updates can be particularly useful when characteristics about the data set may be known, and it is understood that certain values may be more present in the data than others (eg understood on the user's side). In this case, one, two, three, or more indicators may be associated with the largest number of data values associated with the data set.

Compensation for DRAM activation penalty

In certain types of memory (such as DRAM), memory cells may be placed in an array within a memory bank, and the values contained in the memory cells may be accessed and retrieved (read) one line at a time of the memory cells in the array at a time. have. This read process may include first opening (activating) lines (or columns) of memory cells to make available data values stored by the memory cells. Next, the memory cell values of the open line can be sensed simultaneously, and the column address is used to read the memory cell values to cycle an individual memory cell value or group of memory cell values (i.e., a word) and each memory cell value can be connected to an external data bus. This process takes time. In some cases, it takes 32 cycles of operation time to open a memory line for reading, and 32 cycles again to read data from an open line. If the next line to be read is opened only after the read operation of the currently open line is completed, a serious delay may occur. In this example, during the 32 cycles required to open the next line, no data is read, and as a result, a total of 64 cycles is required to read each line, rather than the 32 cycles it takes to repeat the line data. In the conventional memory system, the second line of the same bank cannot be opened while the first line is read or written. Thus, to reduce delay, the next line to open may be in a different bank or in a special bank for dual line access, as described in more detail below. The current line can all be sampled with the flip-flop or latch before opening the next line, and the next line can be opened while all processing is done on the flip-flop/latch. If the next expected line is in the same bank (and none of the above), the delay may be unavoidable and the system may need to wait. This mechanism corresponds to both standard memories and in particular memory processing devices.

The embodiments disclosed herein can reduce such a delay by predicting the next memory line to be opened before the read operation of the currently open memory line is completed. That is, if the next line to be opened can be predicted, the process of opening the next line may start before the read operation of the current line is completed. Depending on at what point in the process the next line prediction is made, the delay associated with the opening of the next line can be reduced from 32 cycles (the specific example described above) to less than 32 cycles. In one particular example, if the next line open is predicted 20 cycles earlier, the additional delay would be only 12 cycles. In another example, if the next line open is predicted 32 cycles earlier, there is no delay at all. As a result, instead of requiring a total of 64 cycles to open and read each row serially, by opening the next row while the current row is being read, the time to read each row can consequently be reduced.

The mechanism below requires that the current line and the predicted line be in the same bank, but if there is a bank that can support both activation and operation on the line at the same time, such a bank may be utilized.

A predictive address generator observes the rows accessed, identifies one or more patterns associated with the accesses (eg sequential line accesses, accesses to all even lines, accesses to all multiples of 3 lines, etc.), and based on the observed patterns, to include a pattern learning model that estimates the next row to be accessed. In another example, the predictive address generator may include a unit that applies a formula/algorithm to predict the next row to be accessed. In still other embodiments, the predictive address generator may generate a predictive next row (one associated with the predictive row) to be accessed based on input such as the current address/row being accessed, the previous 2, 3, 4, or more addresses/rows accessed, etc. It may include a learning neural network that outputs the above addresses). Using any and all of the above predictive address generators to predict the next memory line to be accessed can significantly reduce the latency associated with memory accesses. The predicted address/row generators described may be useful in any and all systems where access to memory to retrieve data is made. In some cases, the predictive address/row generators described and associated methods for predicting the next memory line access may be particularly suitable for systems running artificial intelligence models, where the AI models facilitate iterative memory access to predict the next row. Because it can be related to the pattern.

81A illustrates a system for activating a next row associated with the memory bank 8100 based on next row prediction according to embodiments of the present disclosure. The system 8100 may include a current and predictive address generator 8192 , a bank controller 8191 , and memory banks 8180A, 8180B. An address generator may be an entity that generates addresses for accesses in memory banks 8180A, 8180B, and may be based on any and all logic circuits, controllers, or microprocessors executing software programs. The bank controller 8191 may be configured to access a current row of the memory bank 8180A (eg, utilize a current row identifier generated by the address generator 8192). The bank controller 8191 may also be configured to activate a predicted next row to be accessed within the memory bank 8180B based on the predicted row identifier generated by the address generator 8192 . The example below illustrates two banks. In other examples, more banks may be used. In some embodiments, there may be a memory bank that allows accessing more than one row at a time (described below), so that the same process can be performed on a single bank. As described above, the activation of a row next to the predicted row to be accessed may start before completion of a read operation performed on the current row being accessed. Thus, in some cases, the address generator 8192 may predict the next row to be accessed and send the predicted next row identifiers (eg, one or more addresses) to the controller 8191 at any time before the access to the current row is complete. can send. Due to this timing, the bank controller can initiate the predicted activation of the next row at any time while the current row is accessed and before the current row is accessed. In some cases, the bank controller 8291 predicts the memory bank 8180 at the same time (or within a few clock cycles) of the time when the activation of the accessed current row is completed and/or the time when the read operation on the current row starts. Activation of the next row can be initiated.

In some embodiments, the operation on the current row associated with the current address may be a read or write operation. In some embodiments, the current row and the next row may be in the same memory bank. In some embodiments, the same memory bank may cause the next row to be accessed while the current row is being accessed. The current row and the next row may be in different memory banks. In some embodiments, the memory unit may include a processor configured to generate a current address and a predicted address. In some embodiments, the memory unit may include a distributed processor. A distributed processor may include a plurality of processor subunits of a processing array spatially distributed among a plurality of discrete memory banks of the memory array. In some embodiments, the predictive address may be generated by a series of flip-flops that samples the delayed generated address. The delay can be set via a multiplexer that selects between flip-flops storing the sampled address.

Here, if it is confirmed that the predicted next row is the actual next row for which the executing software requests access (eg, after completion of a read operation on the current row), the predicted next row may be the current row to be accessed. In the disclosed embodiments, since the process of activating the predicted next row can be initiated before the completion of the current row read operation, if it is verified that the predicted next row is the next row to be accessed, the next row to be accessed is already It can be fully or partially activated. This can significantly reduce the delay associated with line activation. Power reduction may occur when the next row is activated so that the activation ends before or at the end of the reading of the current row.

Current and predictive address generator 8192 identifies a row in memory bank 8180 to be accessed (eg, based on program execution) and predicts the next row to be accessed (eg, based on a pattern observed in row access, a predetermined pattern). (based on n+1, n+2), etc.) may include any and all logical elements, calculation units, memory units, algorithms, learning models, and the like. For example, in some embodiments, current and predictive address generator 8192 may include a counter 8192A, current address generator 8192B, and predictive address generator 8192C. The current address generator 8192B may be configured to generate the current address of the current row to be accessed in the memory bank 8180, for example, based on an output of the counter 8192A or based on a request from an operation unit. The address associated with the current row to be accessed may be provided to the bank controller 8191 . Predictive address generator 8192C may be configured to access a line based on an output of a counter 8192A, based on a predetermined access pattern (eg, with a counter 8192A), a learning neural network, or based on a pattern associated with an observed line access, etc. observing and based on the output of other types of pattern prediction algorithms that predict the next line to be accessed, determine the predicted address of the next row to be accessed within the memory bank 8180 . The address generator 8192 may provide the predicted next row address from the predicted address generator 8192C to the bank controller 8191 .

In some embodiments, current address generator 8192B and predictive address generator 8192C may be implemented internally or externally to system 8100 . An external host may also be implemented outside the system 8100 and further connected to the system 8100 . For example, current address generator 8192B may be software residing on an external host executing a program, and predictive address generator 8192C may be implemented internally or externally to system 8100 .

As described above, the predicted next row address may be determined using a learning neural network that predicts the next row to be accessed based on input(s) that may include one or more previously accessed row addresses. A learning neural network or other type of model may operate within the logic associated with predictive address generator 8192C. In some cases, the learning neural network or the like may be executed by one or more computations external to the predictive address generator 8192C but in communication with the predictive address generator 8192C.

In some embodiments, predictive address generator 8192C may comprise a duplicate or substantial clone of current address generator 8192B. Also, the current address generator 8192B and the predicted address generator 8192C operation timing may be fixed or adjusted with respect to each other. For example, in some cases, predictive address generator 8192C generates an address identifier associated with the next row to be accessed at a fixed time (eg, a fixed number of clock cycles) relative to when the current address generator 8192B is generating. It may be configured to output an address identifier associated with the predicted next row. In some cases, the predicted next row identifier is determined before or after activation of the current row to be accessed begins, before or after a read operation associated with the current row to be accessed begins, or before the read operation associated with the current row being accessed completes. It can be created at any time before. In some cases, the predicted next row identifier may be generated concurrently with the start of an activation of the current row to be accessed or the start of a read operation associated with the current row to be accessed.

In other cases, the time between generation of the predicted next row identifier and activation of the current row to be accessed or the initiation of a read operation associated with the current row may be adjustable. For example, in some cases, this time may be extended or shortened during operation of the memory unit 8100 based on values associated with one or more operating parameters. In some cases, a temperature (or other parameter value) associated with a memory unit or other component of a computing system may cause current address generator 8192B and predictive address generator 8192C to change relative timing of operation. In some embodiments, the prediction mechanism may be part of the logic.

Current and predicted address generator 8192 may generate a confidence level associated with determining a predicted next row to be accessed. This level of confidence (which may be determined by the predictive address generator 8192C as part of the prediction process) is determined during the read operation of the current row (i.e., before the current row read operation is complete and the identification of the next row to be accessed is verified). It can be used to determine whether or not to start activation of the next row predicted before). For example, in some cases, a confidence level associated with the predicted next row to be accessed may be compared to a threshold level. For example, if the confidence level is less than the threshold level, the memory unit 8100 may give up the predicted next row activation. On the other hand, if the confidence level exceeds the threshold level, the memory unit 8100 may initiate activation of the next predicted row in the memory bank 8180 .

The check of the confidence level against the threshold level of the predicted next row and the subsequent initiation or non-initiation of activation of the predicted next row may be in any and any suitable manner. For example, in some cases, if the confidence level associated with the predicted next row falls below a threshold level, the predicted address generator 8192C may abandon outputting the predicted next row result to a later logical element. In this case, alternatively, the current and predicted address generator 8192 may withhold the predicted next row identifier from the bank controller 8191, or the bank controller (or other logic) may read associated with the current row being read. It may be configured to utilize the confidence level in the predicted next row to determine whether to start the activation of the next predicted row before the operation is complete.

For example, in some cases, the confidence level may depend on whether one or more previous next row predictions have been found to be correct (eg, historical performance indicators). The confidence level may also be based on one or more characteristics of the input to the algorithm/model. For example, an input comprising an actual row access that follows a pattern may have a higher confidence level than an actual row access that follows less of the pattern. Also, in some cases, such as when randomness is detected for a stream of input containing recent row accesses, the confidence level generated may be low. Further, in the case where randomness is detected, the next row prediction process is batch aborted, the next row prediction is ignored by one or more components of the memory unit 8100 , or any other for abandoning activation of the predicted next row. An action may be taken.

In some cases, a feedback mechanism may be included with respect to the operation of memory 8100 . For example, periodically or even after each next row prediction, the accuracy of predicting the actual next row to be accessed by the predicted address generator 8192C may be determined. In some cases, if there is an error in the prediction of the next row to be accessed (or after a predetermined number of errors), the next row prediction operation of the prediction address generator 8192C may be stopped. In other cases, the prediction address generator 8192C may include a learning element such that one or more aspects of the prediction operation may be adjusted based on the received feedback regarding the accuracy of predicting the next row to approach. This capability may improve the operation of the predictive address generator 8192C to allow the predictive address generator 8192C to adapt to changing access patterns and the like.

In some embodiments, the timing of generation of the predicted next row and/or activation of the predicted next row may depend on the overall operation of the memory unit 8100 . For example, after turning on the power or after resetting the memory unit 8100, prediction of the next row to be accessed (or sending the predicted next row to the bank controller 8191) may be stopped (eg, a predetermined for a period of time or clock cycle, until a predetermined number of row accesses/reads are complete, until the predicted confidence level of the next row exceeds a predetermined threshold level, or any other suitable condition).

81B shows another configuration of the memory unit 8100 according to the disclosed embodiment. In system 8100B of FIG. 81B , cache 8193 may be associated with bank controller 8191 . Cache 8193 may be configured such that, for example, one or more rows of data are stored after being accessed and do not need to be activated again. Accordingly, the cache 8193 may allow the bank controller 8191 to access row data from the cache 8193 instead of accessing the memory bank 8180 . For example, cache 8193 may store the last X rows of data (or any other cache storage strategy), and bank controller 8191 may fill cache 8193 according to the predicted rows. Also, if the predicted row is already in cache 8193, the predicted row does not need to be reopened, and the bank controller (or cache controller implemented within cache 8193) can protect the predicted row from being swapped. can The cache 8193 may provide several advantages. First, the cache 8193 loads a row into the cache 8193 and the bank controller can access the cache 8193 to fetch row data, so there is no need for a special bank or more than one bank for next row prediction. Second, since the physical distance from the bank controller 8191 to the cache 8193 is shorter than the physical distance from the bank controller 8191 to the memory bank 8180, reading and writing from the cache 8193 can save energy. . Third, the cache 8193 is smaller and closer to the controller 8191 so the delay caused by the cache 8193 is lower than that of the memory bank 8180 . In some cases, for example, the identifier of the predicted next row generated by the predictive address generator may be stored in the cache 8193 as the predicted next row is activated in the memory bank 8180 by the bank controller 8191 . have. Based on program execution, etc., the current address generator 8192B may identify the actual next row in the memory bank 8180 to be accessed. The identifier associated with the actual next row to be accessed may be compared with the identifier of the predicted next row stored in the cache 8193 . When the actual next row to be accessed and the predicted next row to be accessed are the same, the bank controller 8191 may initiate a read operation on this row after activation of the next row to be accessed is completed (as a result of the next row prediction process) may be fully or partially activated). On the other hand, if the actual next row to be accessed (as determined by the current address generator 8192B) does not match the predicted next row identifier stored in the cache 8193, read for the fully or partially activated predicted next row. No action is initiated, and the system begins activating the actual next row to be accessed.

Dual Activation Bank

As discussed earlier, it is worth explaining the different mechanisms that allow building a bank with the ability to activate one row while other rows are being processed. Various embodiments may be provided for a bank that activates additional rows while other rows are being accessed. Although it is described that only two rows are activated in the embodiments, it will be natural that it can be applied to more rows. In the first proposed embodiment, the memory bank may be divided into memory subbanks, and the described embodiments are utilized for the activation of the next row predicted or required in another subbank while performing a read operation on a line in one subbank. can be For example, as shown in FIG. 81C , the memory bank 8180 may be configured to include multiple memory subbanks 8181 . The bank controller 8191 associated with the memory bank 8180 may include a plurality of sub-bank controllers in association with the corresponding sub-banks. The first subbank controller of the plurality of subbank controllers is configured to activate the first subbank of the plurality of subbanks while the second subbank controller of the plurality of subbank controllers can activate the next row of the second subbank of the plurality of subbanks. may be configured to enable access to data contained in the current row of the bank. Only one column decoder can be used when accessing only the words of one subbank at a time. Two banks can be tied to the same output bus and appear as a single bank. A single bank input can also be a single address and an additional row address to open a new row.

FIG. 81C illustrates the first and second subbank row controllers 8183A and 8183B for each memory subbank 8181 . As shown in FIG. 81C , the memory bank 8180 may include a plurality of subbanks 8181 . In addition, the bank controller 8191 may include a plurality of sub-bank controllers 8183A-B respectively associated with the corresponding sub-bank 8181 . The first sub-bank controller 8183A of the plurality of sub-bank controllers is configured to activate the first sub-bank controller 8183B of the sub-bank 8181 while the second sub-bank controller 8183B can activate the next row of the second portion of the sub-bank 8181 . It may be configured to enable access to data contained in the current row of the part.

Because activation of a row immediately adjacent to the row being accessed may distort the row accessed and/or cause errors in data read from the row being accessed, the disclosed embodiments ensure that the next row predicted to be active is the data accessed and may be configured to be spaced apart, for example, by at least two rows from the current row of the first subbank in which it is located. In some embodiments, the rows to be activated can be spaced apart by at least one mat so that activation can be performed on different mats. The second subbank controller may be configured to cause access to data included in the current row of the second subbank while the first subbank controller activates the next row of the first subbank. The next activated row of the first subbank may be spaced apart by at least two rows from the current row of the second subbank from which data is being accessed.

A predetermined distance between rows being read/accessed and rows being activated may be determined by hardware. For example, different portions of a memory bank may be coupled to different row decoders, and software may keep the data intact. The spacing between the current rows can be 2 or more (eg 3, 4, 5, or more). The distance may change over time, for example, based on an assessment of the distortion that has occurred to the stored data. Distortion can be assessed in a variety of ways. For example, it can be evaluated by calculating a signal-to-noise ratio, an error rate, an error code required to correct distortion, and the like. The two rows are far enough apart and can actually be active if two bank controllers are implemented on the same bank. The new architecture (implementing two controllers on the same bank) prevents the lines from being opened within the same mat.

81D illustrates an embodiment for next row prediction according to embodiments of the present disclosure. This embodiment may include an additional pipeline of flip-flops (address registers A to C). The delay needed to delay the entire execution to take advantage of activations and deferred addresses and predictions after the address generator can be the new address generated (beginning of the pipe, below address register C), and as the current address becomes the end of the pipeline , the pipeline can be implemented with any and all number of flip-flops (stages). In this embodiment, a duplicate address generator is not required. A selector (multiplexer in Fig. 81D) may be added to configure the delay while the address register provides the delay.

81E illustrates an embodiment of a memory bank according to embodiments of the present disclosure. The memory bank may be implemented such that activation of a new line does not damage the current line if the newly activated line is far enough away from the current line. 82E, the memory bank may include an additional memory mat (black) between every two lines of mats. Accordingly, the controller (eg, the row decoder) may activate lines spaced apart by the mat.

In some embodiments, the memory unit may be configured to receive a first address for processing and a second address for activation and access at a predetermined time.

81F illustrates another embodiment of a memory bank according to embodiments of the present disclosure. The memory bank may be implemented such that activation of a new line does not damage the current line if the newly activated line is far enough away from the current line. The embodiment shown in FIG. 81F may allow the row decoder to open line n and line n+1 by having all even lines implemented in the upper half of the memory bank and all odd lines being implemented in the lower half of the memory bank. With this implementation it may always be possible to access a continuous line that is far enough away.

A dual control memory bank in accordance with disclosed embodiments may enable access and activation of different portions of a single memory bank, even when the dual control memory bank is configured to output only one data unit at a time. For example, as described, dual control may enable a memory bank to access a first row while activating a second row (eg, a predicted next row or a predetermined next row to be accessed).

82 illustrates a dual control memory bank 8280 for reducing a memory row activation penalty (eg, delay) in accordance with embodiments of the present disclosure. The dual control memory bank 8280 includes a data input (DIN) 8290, a row address (ROW) 8291, a column address (COLUMN) 8292, a first command input (COMMAND_1) 8293, and a second command may include inputs including input (COMMAND_2) 8294 . The memory bank 8280 may include a data output (Dout) 8295 .

It is assumed that the address has a row address and a column address, and there are two row decoders. Other configurations of addresses are possible, the number of row decoders may be more than two, and there may be more column decoders than single column decoders.

A row address (ROW) 8291 may identify a row associated with an instruction, such as an active instruction. Since row activation entails reading or writing to the row, it may not be necessary to write while the row is open (after activation) or send the row address to read from.

The first command input (COMMAND_1) 8293 may be used to transmit a command (such as an active command) to a row accessed by the first row decoder. The second command input (COMMAND_2) 8294 may be used to send a command (such as an active command) to a row accessed by the second row decoder.

A data input (DIN) 8290 may be used to supply data when performing a write operation.

Since the entire row may not be read at once, single row segments may be read sequentially, and the column address (COLUMN) 8292 may indicate the segment to be read (ie, which column) of the row. For convenience of explanation, it is assumed that there are segments of 2Q, there are Q bits in the column input, and that Q is a positive integer greater than 1.

The dual control memory bank 8280 may operate in conjunction with or separately from the address prediction described above with reference to FIGS. 81A to 81B . Of course, in order to reduce the delay of the operation, the dual control memory bank may operate together with the address prediction according to the present disclosure.

83A, 83B, and 83C illustrate examples of accessing and activating a row of the memory bank 8180. In one example, as mentioned above, it is estimated that 32 cycles (segments) are required to read and activate one row. Also, to reduce the activation penalty (length expressed in delta), it may be advantageous to know in advance (at least before delta before the next row needs to be accessed) whether the next row should be opened. In some cases, the delta may be 4 cycles. Each memory bank shown in FIGS. 83A, 83B, and 83C may include two or more subbanks in which only one row may be open at a time in some embodiments. In some cases, even numbered rows may be associated with a first subbank and odd numbered rows may be associated with a second subbank. In this example, utilizing the disclosed predictive address embodiments may allow the initiation of activation of one row of a particular memory subbank before the end of a read operation on the row of another memory subbank is reached (prior to the delta period). Thereby, sequential memory accesses (eg reads of rows 1, 2, 3, 4, 5, 6, 7, 8 ... have to be performed, rows 1, 3, 5, etc. are associated with the first memory subbank and , rows 2, 4, 6, etc. are associated with a second memory subbank different therefrom) can be performed in a very efficient manner.

83A may be a diagram illustrating a state for accessing a memory row included in a different memory subbank. In the state shown in Fig. 83A:

a. Row A may be accessible by the first row decoder. The first segment (the leftmost shaded segment) can be accessed after the first row decoder activates row A.

b. Row B may be accessible by the second row decoder. In the state shown in Fig. 83A, row B is closed and has not yet been activated.

Prior to the state illustrated in FIG. 83A , the activation command and the address of row A may be transmitted to the first row decoder.

83B shows a state for accessing the row B after the row A is accessed. According to this example, row A may be accessible by the first row decoder. 83B, the first row decoder activated row A and accessed all segments except for the rightmost 4 segments (unshaded 4 segments). Since the delta (4 white segments in row A) is 4 cycles, the bank controller can force the second row decoder to activate row B before accessing the rightmost segments of row A. In some cases, row B may be activated in response to a predetermined access pattern (eg, sequential row access in which odd rows are assigned to a first subbank and even rows are assigned to a second subbank). In other cases, row B may be activated corresponding to any and all of the row prediction methods described above. When row B is accessed, the bank controller may cause the second row decoder to activate row B so that row B is already active (open) instead of waiting for row B to be activated to open row B when it is accessed.

The following operations may occur prior to the state shown in FIG. 83B .

a. Send the activation command and the address of row A to the first row decoder

b. read or write the first 28 segments of row A

c. After a read or write operation of the 28 segment of the row, an enable command for the address of row B is sent to the second row decoder

In some embodiments, even rows are located in half of one or more memory banks. In some embodiments, odd rows are located in half of one or more memory banks.

In some embodiments, a line of extra overlapping mats may be placed between each two mat lines to form a distance to allow activation. In some embodiments, lines adjacent to each other may not be active at the same time.

83C may be a diagram illustrating a state for accessing a row C (eg, the next odd-numbered row included in the first subbank) after accessing the row A. 83C , row B may be accessible by the second row decoder. As shown, the second row decoder activated row B and accessed all segments except for the rightmost 4 segments (the 4 unshaded segments). In this example, the delta is 4 cycles, so the bank controller can cause the first row decoder to activate row C before accessing the rightmost segments of row B. When row C is accessed, the bank controller may cause the first row decoder to activate row C, such that row C is already activated (opened) instead of waiting for row C to be activated to open row C. By operating in this manner, delays associated with memory read operations can be reduced or eliminated entirely.

Memory mat as register file

In computer architectures, processor registers constitute storage locations that are quickly accessible to the computer's processor (eg CPU). The register generally contains the memory unit closest to the processor core (L0). Registers can provide the fastest way to access certain types of data. A computer may include several types of registers, and each type of register may be distinguished according to the type of information it stores or the type of instruction operating on the information in a particular type of register. For example, a computer may have a data register that holds numeric information, operands, intermediate results, and settings, an address register that stores address information used by instructions to access main memory, and a general purpose register that stores both data and address information. It may include registers, status registers, and the like. A register file contains logical groups of registers for use by computer processing units.

In many cases, a computer's register file is located within a processing unit (eg, CPU) and implemented by logic transistors. However, in the disclosed embodiments, the arithmetic processing unit may not be in the existing CPU. Instead, these processing elements (eg, processor subunits) may be spatially distributed as a processing array within a memory chip (see above). Each processor subunit may be associated with one or more corresponding dedicated memory units (eg, memory banks). This architecture allows each processor subunit to be located in spatial proximity to one or more memory elements that store data to be operated by that particular processor subunit. As described in this disclosure, such an architecture can significantly increase the speed of certain memory intensive operations, such as by eliminating memory access bottlenecks experienced by typical CPU and external memory architectures.

However, the distributed processor memory chip architecture described herein may still take advantage of register files containing various types of registers that operate data from memory elements dedicated to the corresponding processor subunit. However, since a processor subunit may be distributed among the memory elements of a memory chip, one or more memory elements may be assigned to a corresponding processor subunit to serve as a register file or cache for the corresponding processor subunit rather than serving as main memory storage. It may be possible to add a (which can be an advantage from a particular process over the logic element of that particular manufacturing process).

Such an architecture can provide several advantages. For example, since a register file is part of a corresponding processor subunit, a processor subunit may be located spatially close to the associated register file. This arrangement can significantly increase operating efficiency. Conventional register files are implemented by logic transistors. For example, since each bit of a conventional register file consists of about 12 logic transistors, a register file of 16 bits consists of 192 logic transistors. These register files can require a large number of logic components to access the logic transistors and therefore can take up a lot of space. Compared to a register file implemented by a logic transistor, the register file of embodiments of the present disclosure requires significantly less space. This size reduction may be realized by implementing the register file of embodiments of the present disclosure that utilizes a memory mat comprising memory cells fabricated by a process optimized for fabrication of a memory structure rather than fabrication of a logical structure.

In some embodiments, a distributed processor memory chip may be provided. A distributed processor memory chip may include a substrate, a memory array disposed on the substrate and including a plurality of discrete memory banks, and a processing array disposed on the substrate and including a plurality of processor subunits. Each processor subunit may be associated with a corresponding dedicated discrete memory bank among the plurality of discrete memory banks. The distributed processor memory chip may also include a first plurality of buses and a second plurality of buses. Each bus of the first plurality of buses may couple one processor subunit of the plurality of processor subunits to a corresponding dedicated memory bank. Each bus of the second plurality of buses may couple one processor subunit of the plurality of processor subunits to another processor subunit of the plurality of processor subunits. In some cases, the second plurality of buses may couple one or more processor subunits of the plurality of processor subunits to two or more other processor subunits of the plurality of processor subunits. One or more processor subunits of the plurality of processor subunits may also include at least one memory mat disposed on the substrate. The at least one memory mat may be configured to serve as at least one register in a register file for one or more processor subunits of the plurality of processor subunits.

In some cases, a register file may be associated with one or more logical components to cause the memory mat to function as one or more registers of the register file. For example, such logic components may include switches, amplifiers, inverters, sense amplifiers, and the like. In an example where the register file is implemented by a DRAM mat, a logic component may be included to perform a refresh operation to prevent loss of stored data. Such logical components may include row and column multiplexers (“mux”). Also, the register file implemented by the DRAM mat may include a redundancy mechanism to counter the yield degradation.

84 shows a conventional computer architecture 8400 that includes a CPU 8402 and an external memory 8406 . During operation, values from memory 8406 may be loaded into registers associated with register file 8504 included in CPU 8402 .

85A illustrates an exemplary distributed processor memory chip 8500a in accordance with disclosed embodiments. Unlike the architecture of FIG. 84 , the distributed processor memory chip 8500a includes a memory element and a processor element disposed on the same substrate. That is, chip 8500a may include a processing array including a memory array and a plurality of processor subunits each associated with one or more dedicated memory banks included in the memory array. In the architecture of FIG. 85, the registers used by the processor subunits are provided by one or more memory mats disposed on the same substrate on which the memory array and processing array are formed.

As shown in FIG. 85A , a distributed processor memory chip 8500a may be formed by a plurality of processing groups 8510a , 8510b , 8510c disposed on a substrate 8502 . More specifically, the distributed processor memory chip 8500a may include a memory array 8520 and a processing array 8530 disposed on a substrate 8502 . The memory array 8520 may include a plurality of memory banks, such as memory banks 8520a, 8520b, and 8520c. The processing array 8530 may include a plurality of processor subunits, such as processor subunits 8530a, 8530b, and 8530c.

Additionally, each processing group 8510a, 8510b, 8510c may include a processor subunit and one or more corresponding memory banks dedicated to the processor subunit. 85A, each processor subunit 8530a, 8530b, 8530c may be associated with a corresponding dedicated memory bank 8520a, 8520b, 8532c. That is, processor subunit 8530a may be associated with memory bank 8520a, processor subunit 8530b may be associated with memory bank 8520b, and processor subunit 8530c may be associated with memory bank 8520c. can be related to

To enable each processor subunit to communicate with its corresponding dedicated memory bank(s), the distributed processor memory chip 8500a connects one of the processor subunits with its corresponding dedicated memory bank(s). 1 It may include a plurality of buses 8540a, 8540b, and 8540c. 85A, bus 8540a may couple processor subunit 8530a to memory bank 8520a, and bus 8540b may couple processor subunit 8530b to memory bank 8520b. bus 8540c may couple processor subunit 8530c to memory bank 8520c.

Also, to enable each processor subunit to communicate with other processor subunits, the distributed processor memory chip 8500a may include a second plurality of buses connecting one of the processor subunits with at least one other processor subunit. (8550a, 8550b) may be included. 85A, bus 8550a may couple processor subunit 8530a with processor subunit 8530b, and bus 8550b may connect processor subunit 8530b with processor subunit 8530c ) can be associated with

Each discrete memory bank 8520a, 8520b, 8520c may include a plurality of memory mats. 85A, memory bank 8520a may include memory mats 8522a, 8524a, 8526a, memory bank 8520b may include memory mats 8522b, 8524b, 8526b and , the memory bank 8520c may include memory mats 8522c, 8524c, and 8526c. As described above with reference to FIG. 10 , the memory mat may include a plurality of memory cells, and each cell may include a capacitor, a transistor, or other circuit for storing at least one bit of data. A conventional memory mat may include, for example, 512 bits by 512 bits, but embodiments disclosed herein are not limited thereto.

At least one of the processor subunits 8530a, 8530b, 8530c is at least one memory mat, such as a memory mat 8532a, 8532b, 8532c, configured to serve as a register file for the corresponding processor subunit 8530a, 8530b, 8530c may include That is, at least one memory mat 8532a, 8532b, 8532c provides at least one register of a register file used by one or more processor subunits 8530a, 8530b, 8530c. A register file may contain one or more registers. 85A, the memory mat 8532a in the processor subunit 8530a is the processor subunit 8530a (and/or any and all other processor subunits included in the distributed processor memory chip 8500a). may serve as a register file for (also referred to as 'register file 8532a') for Memory mat 8532c within processor subunit 8530c may serve as a register file for processor subunit 8530c.

At least one of the processor subunits 8530a, 8530b, 8530c may also include at least one logical component, such as a logical component 8534a, 8534b, 8534c. Each logical component 8534a, 8534b, or 8534c may be configured to enable a corresponding memory mat 8532a, 8532b, or 8532c to serve as a register file for the corresponding processor subunit 8530a, 8530b, or 8530c. can

In some embodiments, at least one memory mat may be disposed on a substrate, wherein the at least one memory mat is configured to provide at least one redundant register for one or more processor subunits of the plurality of processor subunits. It may contain one redundant memory bit. In some embodiments, the at least one processor subunit may include a mechanism to refresh the memory mat by suspending the current task and triggering a memory refresh operation at a specific time.

85B illustrates an exemplary distributed processor memory chip 8500b in accordance with disclosed embodiments. The memory chip 8500b shown in FIG. 85B is the memory chip 8500b shown in FIG. 85A except that the memory mats 8532a, 8532b, 8532c are not included in the corresponding processor subunits 8530a, 8530b, 8530c. 8500a) is practically the same. Instead, the memory mats 8532a , 8532b , 8532c of FIG. 85B are disposed outside but spatially adjacent to the corresponding processor subunits 8530a , 8530b , 8530c . In this way, the memory mats 8532a, 8532b, 8532c can still serve as register files for the corresponding processor subunits 8530a, 8530b, 8530c.

85C illustrates an apparatus 8500c according to a disclosed embodiment. The apparatus 8500c includes a substrate 8560 , a first memory bank 8570 , a second memory bank 8572 , and a processing unit 8580 . A first memory bank 8570 , a second memory bank 8572 , and a processing unit 8580 are disposed on a substrate 8560 . The processing unit 8580 includes a register file 8582 implemented by a processor 8584 and a memory mat. During operation of processing unit 8580 , processor 8584 may access register file 8582 to read or write data.

A distributed processor memory chip 8500a, 8500b or device 8500c may provide various functions based on the processor subunit's access to registers provided by the memory mat. For example, in some embodiments, the distributed processor memory chip 8500a or 8500b may include a processor subunit coupled to the memory to function as an accelerator to enable greater memory bandwidth utilization. In the embodiment shown in FIG. 85A , processor subunit 8530a may function as an accelerator (also referred to as 'accelerator 8530a '). Accelerator 8530a may utilize memory mat 8532a disposed within accelerator 8530a to provide one or more registers in a register file. Alternatively, in the embodiment shown in FIG. 85B , the accelerator 8530a may utilize the memory mat 8532a disposed outside the accelerator 8530a as a register file. In addition, accelerator 8530a may utilize any one of memory mats 8522b, 8524b, 8526b in memory bank 8520b or any one of memory mats 8522c, 8524c, 8526c in memory bank 8520c. More registers can be provided.

The disclosed embodiments may be particularly useful for certain types of image processing, neural networks, database analysis, compression and decompression, and the like. For example, in the embodiment of FIG. 85A or 85B, the memory mat may provide one or more registers in a register file for one or more processor subunits included on the same chip as the memory mat. One or more registers may be utilized for storage of frequently accessed data by the processor subunit(s). For example, during convolutional image processing, the convolution accelerator may repeatedly use the same coefficients multiple times for the entire image in memory. The proposed implementation for such a convolution accelerator puts all these coefficients in a 'neighbor' register file, ie in one or more registers contained within a memory mat dedicated to one or more processor subunits located on the same chip as the register file memory mat. may be storage. This architecture may place registers (and stored count values) close to the processor subunit that operates the count values. Since the register file implemented by the memory mat can function as an efficient cache that is spatially close, it is possible to significantly reduce loss in data transfer and delay in access.

In another example, the disclosed embodiments may include an accelerator capable of inputting words into registers provided by the memory mat. The accelerator can doubling the vector in a single cycle by treating the registers as a cyclic buffer. For example, in the device 8500c shown in FIG. 85C , the processor 8584 in the processing unit 8580 uses the register file 8582 implemented by the memory mat as a circular buffer to the data A1 , A2 , A3 ,　.　.　.　.) functions as an accelerator. The first memory bank 8570 stores data B1, B2, B3, 　. The second memory bank 8572 stores the multiplication result (C1, C2, C3, 　.　.　.　.). That is, Ci = Ai Х Bi. If there is no register file in the processing unit 8580, the processor 8584 can use the data (A1, A2, A3, 　. ,　.　.　.　) will require more memory bandwidth and more cycles to read, which can cause significant delay. On the other hand, in the present embodiment, data A1, A2, A3, 　. . . . . . . are stored in a register file 8582 formed in the processing unit 8580. Accordingly, the processor 8584 only needs to read data B1, B2, B3, 　. 　. 　. 　 from the external memory bank 8570. Accordingly, the memory bandwidth can be significantly reduced.

In memory processes, memory mats usually only allow one-way access (ie, single access). In one-way access, there is one port into memory. As a result, only one access operation, eg, read or write, from a specific address can be performed at a specific time. However, if the memory mat itself allows bidirectional access, bidirectional access can be a natural choice. In bidirectional access, two different addresses may be accessed at a specific time. A method of accessing the memory mat may be determined based on an area and a requirement. In some cases, the register file implemented by the memory mat may allow four-way access when connected to a processor that has to read two sources and has one destination register. In some cases, where the register file is implemented by the DRAM mat to store settings or cache data, the register file may only allow one-way access. A standard CPU may include a multi-way access mat, whereas a one-way access mat may be more desirable when DRAM is applied.

In cases where the controller or accelerator is designed in such a way that only a single access to the register is required, a register implemented as a memory mat can be used instead of a conventional register file. In single access, only one word can be accessed at a time. For example, a processing unit may access two words from two register files at a particular time. Each register file of the two register files can be implemented by a memory mat (eg DRAM mat) that only allows single access.

In most technologies, the memory mat IP, which is a closed block (IP) secured from the manufacturer, is equipped with wiring such as word lines and row lines for row and column access. However, the memory mat IP does not contain any logical components around it. Accordingly, the register file implemented by the memory mat disclosed in the present embodiments may include a logical component. The size of the memory mat may be selected based on the requested size of the register file.

Certain issues may arise when utilizing a memory mat to provide registers from a register file, and these issues may depend on the particular memory technology utilized to form that memory mat. For example, in memory production, not all manufactured memories will necessarily function properly after production. This is a known issue, especially when there is a high density of SRAM or DRAM on the chip. In memory technology, in response to this problem, one or more redundancy mechanisms may be utilized to keep the yield at an appropriate level. In the disclosed embodiment, since the number of memory instances (eg, memory banks) used to provide registers in the register file is quite small, the redundancy mechanism may not be as important as in typical memory applications. On the other hand, the same production issues that affect memory functionality can also affect whether a particular memory mat functions properly to provide one or more registers. As a result, overlapping elements may be included in the disclosed embodiments. For example, at least one redundant memory mat may be disposed on a substrate of a distributed processor memory chip. The at least one redundant memory mat may be configured to provide at least one redundant register for one or more processor subunits of the plurality of processor subunits. In another example, the mat may be larger than the required size (eg 620x620 rather than 512x512), and the redundancy mechanism may be built into a region of the memory mat outside the 512x512 region or equivalent.

Another issue could be timing related. In general, the loading timing of the word line and the bit line is determined by the size of the memory. Since the register file can be implemented by a single fairly small memory mat (e.g. 512 x 512 bits), it may take less time to load a word from the memory mat, and the timing may be sufficient to execute significantly faster than the logic. can

Refresh - Some types of memory, such as DRAM, require periodic refreshes. A refresh may be performed when the processor or accelerator is stopped. For a small memory mat, the time to refresh may be a small percentage of the time. Thus, even if the system is down for a short period of time, the gains from using the memory mat as a register may be worth the downtime given the overall performance. In one embodiment, the processing unit may include a counter that counts backwards from the number of corrections. When the counter reaches '0', the processing unit may abort the current work being performed by the processor (eg accelerator) and trigger a refresh operation in which the memory mat is refreshed line by line. When the refresh operation is finished, the processor can resume work, and the counter can be reset to count backwards from a predetermined number.

86 illustrates a flowchart 8600 of an exemplary method of executing at least one instruction in a distributed processor memory chip in accordance with disclosed embodiments. For example, at step 8602 , at least one data value may be retrieved from a memory array on a substrate of a distributed processor memory chip. In step 8604, the retrieved data value may be stored in a register provided by a memory mat of a memory array on a substrate of a distributed processor memory chip. At step 8606, a processor element, such as one or more of the distributed processor subunits on the distributed processor memory chip, may operate the stored data values from the memory mat registers.

Throughout this specification, since a register file may be a lowest level cache, it will be understood that all references to a register file apply equally to a cache.

processing bottleneck

Terms such as 'first', 'second', and 'third' are used only to indicate a difference between different terms. These terms do not indicate the order and/or timing and/or importance of the elements. For example, it may be possible that the second process precedes the first process, and so on.

The term 'coupled' may mean directly and/or indirectly connected.

'Memory/processing', 'memory and processing', and 'memory processing' are used in an interchangeable manner.

There may be multiple methods, computer readable media, memory/processing units, and/or systems that may be memory/processing units.

A memory/processing unit is a hardware unit with memory and processing capabilities.

The memory/processing unit may be a memory processing integrated circuit, may be included in a memory processing integrated circuit, or may include one or more memory processing integrated circuits.

The memory/processing unit may be a distributed processor as shown in PCT Patent Application Publication No. WO2019025892.

The memory/processing unit may include a distributed processor as shown in PCT Patent Application Publication No. WO2019025892.

The memory/processing unit may belong to a distributed processor as shown in PCT Patent Application Publication No. WO2019025892.

The memory/processing unit may be a memory chip as shown in PCT Patent Application Publication No. WO2019025892.

The memory/processing unit may include a memory chip as shown in PCT Patent Application Publication No. WO2019025892.

The memory/processing unit may belong to a memory chip as shown in PCT Patent Application Publication No. WO2019025892.

The memory/processing unit may be a distributed processor as shown in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may include a distributed processor as shown in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may belong to a distributed processor as shown in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may be a memory chip as shown in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may include a memory chip as shown in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may belong to a memory chip as shown in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may be an integrated circuit interconnected utilizing wafer to wafer bonds and multiple conductors.

Distributed Processors Memory chips, distributed memory processing integrated circuits, memory chips, and anything related to distributed processors may be implemented as a pair of integrated circuits interconnected utilizing wafer-to-wafer junctions and multiple conductors.

The memory/processing unit may be manufactured by a first manufacturing process that is more suitable for memory cells than logic cells. Accordingly, the first manufacturing process may be considered a memory flavored manufacturing process. A memory cell may include one or more transistors. A logic cell may include one or more transistors. A first manufacturing process may be applied to manufacturing the memory bank. A logic cell may include one or more transistors that together implement logic functions and may be used as the basic building blocks of blocks of larger logic circuits. A memory cell may include one or more transistors that together implement a memory function and may be used as the basic building block of a larger block of logic circuitry. Corresponding logic cells may implement the same logic function.

The memory/processing unit may be different from a processor, processing integrated circuit, and/or processing unit manufactured by a second manufacturing process that is more suitable for logic cells than memory cells. Accordingly, the second manufacturing process may be considered a logic flavored manufacturing process. The second manufacturing process may be used for manufacturing a CPU, GPU, or the like.

A memory/processing unit may be more suitable for performing less computationally intensive operations than a processor, processing integrated circuit, and/or processing unit.

For example, a memory cell manufactured by a first manufacturing process may exceed, or even significantly exceed (eg, 2, 3, 4, 5, 9) a critical dimension of a logic circuit manufactured by the first manufacturing process. , by a factor exceeding 7, 8, 9, 10, etc.).

The first manufacturing process may be an analog manufacturing process, a DRAM manufacturing process, or the like.

The size of a logic cell manufactured by the first manufacturing process may exceed the size of a corresponding logic cell manufactured by the second manufacturing process by at least two. A corresponding logic cell may have the same potential as a logic cell manufactured by the first manufacturing process.

The second manufacturing process may be a digital manufacturing process.

The second fabrication process may be any one of complementary metal-oxide-semiconductor (CMOS), bipolar, bipolar-CMOS (BiCOMS), double-diffused metal-oxide-semiconductor (DMOS), silicon on oxide fabrication process, and the like. can

A memory/processing unit may include multiple processor subunits.

The processor subunits of one or more memory/processing units may perform operations and/or distributed processing individually and/or cooperatively with each other. Distributed processing may be performed in a variety of ways, for example, in a flat manner or in a hierarchical manner.

The flat approach allows the processor subunits to perform the same operations (and output or not output the result of the processing in between).

A hierarchical approach may include execution of a sequence of processing operations in different layers. Here, the processing operation of a specific layer follows the processing operation of another layer. Processor subunits can be assigned (dynamic or static) to different layers and participate in hierarchical processing.

Distributed processing may also include other units, such as units that do not belong to and/or to the controller of a memory/processing unit.

The terms logic/logic and processor subunit may be used interchangeably.

Any and all processing/processing mentioned in this application may be executed in any and all manner (eg, distributed manner, non-distributed manner, etc.).

Hereinafter, various references and/or contents by reference are based on PCT Patent Application Publication No. WO2019025892 and PCT Patent Application No. PCT/IB2019/001005 (September 9, 2019). PCT Patent Application Publication No. WO2019025892 and/or PCT Patent Application No. PCT/IB2019/001005 provides examples of various methods, systems, processors, memory chips, etc., but are not limited thereto, and other methods, systems, processors, etc. may be provided have.

A processing system may be provided with one or more memory/processing units prior to the processor, each memory and processing unit (memory/processing unit) having a processing resource and a storage resource.

The processor may request or direct one or more memory/processing units to perform various processing tasks. Execution of various processing tasks offloads the processor, reduces latency, and in some cases reduces the total bandwidth of information between one or more memory/processing units and the processor.

The processor may provide different granularity of indications and/or requests. For example, a processor may send an indication directed at a particular processing resource or may send a higher layer indication directed at a memory/processing unit without specifying any processing resource.

A memory/processing unit may manage its processing and/or memory resources in any and all manner (eg, dynamic, static, distributed, centralized, offline, online, etc.). Management of resources may be autonomously executed under the control of the processor after setting by the processor or the like.

For example, a task may be divided into sub-tasks that may require execution or one or more instructions by one or more processing resources and/or memory resources of one or more memory/processing units. Each processing resource may be configured to execute (eg, independently or non-independently) at least one instruction. For example, an instruction of a sub-series is executed by a processing resource such as a processor subunit of PCT Patent Application Publication No. WO2019025892.

Allocation of at least memory resources may also be provided to an entity other than one or more memory/processing units, eg, a direct access memory (DMA) unit, which may be coupled to one or more memory/processing units.

Compilers may prepare configuration files for each type of operation executed by the memory/processing unit. The configuration file may contain memory allocations and processing resource allocations associated with job types. A configuration file may contain instructions that may be executed by different processing resources and/or may define memory allocations.

For example, a configuration file for a matrix multiplication operation (multiplying matrix A by matrix B, i.e. A*B = C) tells where to store the elements of matrix A, where to store the elements of matrix B, and where to store the elements of matrix C. It may indicate where to store the elements, where to store intermediate results generated during matrix multiplication, and may include instructions directed to processing resources for performing any and all mathematical operations related to matrix multiplication. The configuration file is an example of a data structure, and other data structures may be provided.

Matrix multiplication may be performed in any and all manner by one or more memory/processing units.

One or more memory/processing units may multiply matrix A by vector V. This can be done in any and every way. For example, maintaining a row or column of a matrix per processing resource (different rows of a column per different processing resource), cycling the result of a vector multiplication by a row or column of a matrix (between different processing resources), and a previous multiplication (second to the most recent iteration).

Assuming that matrix A is a 4x4 matrix, vector V is a 1x4 vector, and there are 4 processing resources, a first row of matrix A is stored in a first processor subunit, and a second row of matrix A is stored in a second processor subunit unit, wherein the third row of matrix A is stored in the third processor subunit, and the fourth row of matrix A is stored in the fourth processor subunit. The multiplication is initiated by sending the first through fourth elements of vector V to the first through fourth processing resources and multiplying the first through fourth elements of vector V by different vectors of A to provide a first intermediate result. Multiplication continues by cycling the first intermediate result, ie, each processing resource sends the first intermediate result computed by the first processing resource to a neighboring processing resource. Each processing resource multiplies the first intermediate result by a vector to provide a second multiplication result. This is repeated several times until the multiplication of matrix A by vector V is finished.

FIG. 90A is an example of a system 10900 including one or more memory/processing units (collectively designated 10910 ) and a processor 19020 . As shown above, the processor 10920 may send a request or indication (via link 10931 ) to one or more memory/processing units 10910 , which in turn may send such requests to one or more memory/processing units 10910 . and/or perform (or optionally perform) the instruction and send the result to the processor 10920 (via link 10932 ). Processor 10920 may further process the results to provide one or more outputs (via link 10933 ).

The one or more memory/processing units may include J memory resources 10912(1,1) - 10912(1,J), where J is a positive integer, and K processing resources 10911(1,1) - 10911(1,K)) (where K is a positive integer).

J may be the same as or different from K.

The processing resources 10911(1,1) - 10911(1,K) may be a processing group or processor subgroup, etc. as shown in PCT Patent Application Publication No. WO2019025892.

Memory resources 10912(1,1) - 10912(1,J) may be memory instances, memory mats, memory banks, etc. as shown in PCT Patent Application Publication No. WO2019025892.

There may be any and all connectivity and/or functional relationships between any and all resources (memory resources or processing resources) of one or more memory/processing units.

90B is an example of the memory/processing unit 10910(1).

In FIG. 90B , K processing resources 10911(1,1) - 10911(1,K)) (K being a positive integer) are connected in series with each other (see link 10915 ) to form a loop. Each processing resource is also coupled to a corresponding pair of dedicated memory resources. For example, processing resource 10911(1) is coupled to memory resources 10912(1) and 10912(2), and processing resource 10911(K) includes memory resources 10912(J-1) and 10912(J). )) is bound to The processing resources may be interconnected in any and all other ways. The number of memory resources allocated for each processing resource may not be two. Examples of connectivity between different resources are shown in PCT Patent Application Publication No. WO2019025892.

FIG. 90C is an example of a system 10901 including N (N is a positive integer) memory/processing units 10910(1) - 10910(N) and a processor 10920 . As shown above, processor 10920 may send a request or instruction (via link 10931(1) - 10931(N)) to memory/processing units 10920(1) - 10910(N) and , memory/processing units 10920(1) - 10910(N) then perform these requests and/or instructions and return the results to processor 10920 (via link 10932(1)-3232(N)) can send. Processor 10920 may further process the results to provide one or more outputs (via link 10933 ).

FIG. 90D is an example of a system 10902 including N (N is a positive integer) memory/processing units 10910(1) - 10910(N) and a processor 10920 . Figure 90D shows a preprocessor 10909 in front of memory/processing units 10910(1) - 10910(N). The preprocessor may perform various preprocessing operations such as frame extraction, header detection, and the like.

90E is an example of a system 10903 that includes one or more memory/processing units 10910 and a processor 10920 . FIG. 90E shows a preprocessor 10909 and a DMA controller 10908 in front of one or more memory/processing units 10910 .

90F illustrates a method 10800 for distributed processing of at least one information stream.

Method 10800 may begin with step 10810 in which one or more memory processing integrated circuits receive at least one information stream via a first communication channel, wherein each memory processing integrated circuit includes a controller, multiple processor subunits, and multiple memories. includes units.

After step 10810, steps 10820 and 10830 may be performed.

Step 10820 may include one or more memory processing integrated circuits buffering the information stream.

Step 10830 may include the one or more memory processing integrated circuits performing a first processing operation on the at least one information stream to provide a first processing result.

Step 10830 may include compression or decompression.

Accordingly, the total size of the information stream may exceed the total size of the first processing result. The total size of the information stream may reflect the amount of information received during a period of a particular period. The total size of the first processing result may reflect the amount of the first processing result output during any period of the same specific period.

Alternatively, the total size of the information stream (or any other information entity mentioned herein) may be less than the total size of the first processing result. In this case, compression is ensured.

After step 10830, step 10840 of transmitting the first processing result to one or more processing integrated circuits may be performed.

One or more memory processing integrated circuits may be fabricated by a memory additive manufacturing process.

One or more memory processing integrated circuits may be fabricated by a logic additive manufacturing process.

In a memory processing integrated circuit, each of the memory units may be coupled to a processor subunit.

After operation 10840 , operation 10850 in which one or more processing integrated circuits perform a second processing operation on the first processing result to provide a second processing result may be performed.

Step 10820 and/or step 10830 are directed by one or more processing integrated circuits, requested by one or more processing integrated circuits, executed by one or more processing integrated circuits after configuration of the one or more processing integrated circuits, or performed one or more processing It can be executed independently without the intervention of an integrated circuit.

The first processing operation may be less computationally intensive than the second processing operation.

Step 10830 and/or step 10850 includes (a) a mobile network processing operation, (b) another network-related processing operation (processing of a mobile network and another network), (c) a database processing operation, (d) a database analysis processing operation, ( e) an artificial intelligence processing operation, and any other processing operation.

Separate system memory/processing unit and distributed processing method

Disaggregated systems, distributed processing methods, processing/memory units, methods of operating discrete systems, methods of operating processing/memory units, and non-transitory computer-readable storage instructions for executing any of the above methods A medium may be provided. A separate system allocates different subsystems to perform different functions. For example, storage may be implemented primarily in one or more storage subsystems, while operations may be implemented primarily in one or more storage subsystems.

A separate system may be one separate server, one or more separate servers, one or more servers and another server.

A separate system may include one or more switching subsystems, one or more computing subsystems, one or more storage subsystems, and one or more processing/memory subsystems.

One or more processing/memory subsystems, one or more computing subsystems, and one or more storage subsystems may be coupled to each other via one or more switching subsystems.

One or more processing/memory subsystems may be included in one or more subsystems in separate systems.

87A shows various examples of a separate system.

There can be any and any number of any and any type of subsystem. A separate system may include one or more additional subsystems of a type not included in FIG. 87A, fewer types of subsystems, and the like.

The separate system 7101 includes two storage subsystems 7130 , a computing subsystem 7120 , a switching subsystem 7140 , and a processing/memory subsystem 7110 .

The separate system 7102 includes two storage subsystems 7130 , a computing subsystem 7120 , a switching subsystem 7140 , a processing/memory subsystem 7110 , and an accelerator subsystem 7150 .

The separate system 7103 includes a switching subsystem 7140 that includes two storage subsystems 7130 , a computing subsystem 7120 , and a processing/memory subsystem 7110 .

The separate system 7104 includes two storage subsystems 7130 , a computing subsystem 7120 , a switching subsystem 7140 including a processing/memory subsystem 7110 , and an accelerator subsystem 7150 . do

By including the processing/memory subsystem 7110 within the switching subsystem 7140 , it may be possible to reduce traffic within the separate systems 7101 , 7102 , reduce delay in switching, and the like.

Different subsystems of a separate system may communicate with each other utilizing various communication protocols. It has been found that utilizing Ethernet, or even utilizing RDMA over Ethernet communication protocols, can increase throughput and even reduce the complexity of various control and/or storage operations relating to the exchange of units of information between elements of separate systems. became

A separate system may perform distributed processing by having the processing/memory subsystem participate in the computation, in particular by executing memory intensive computations.

For example, assuming that N computing units must share information units among the N computing units (all-to-all sharing), then (a) the N information units are connected to one or more processing/memory subsystems. may be sent to one or more processing/memory units of may be transmitted in units. This would require about N transmission operations.

For example, FIG. 87B shows distributed processing of updating a model of a neural network (a model including weights assigned to nodes of the neural network).

Each computing unit of the N computing units PU( 1 ) through PU(N) 7120( 1 ) - 7120(N) is one of the discrete systems 7101 , 7102 , 7103 , 7104 . may belong to computing subsystem 7120 .

The N computing units compute N partial model updates 7121( 1 ) through 7121(N) (N different parts updated) and process/memory subsystem 7110 (via switching subsystem 7140 ). send to

The processing/memory subsystem 7110 computes the updated model 7122 and (via the switching subsystem 7140 ) N computing units PU( 1 ) through PU(N) 7120( 1 ) through 7120 ( N)).

87c, 87d, and 87e show examples of memory/processing units 7011, 7012, and 7013, respectively, and FIGS. 87f and 87g show memory/processing unit 9010 and Ethernet module and RDMA on Ethernet module An integrated circuit 7014, 7015 including one or more communication modules, such as 9022, is shown.

The memory/processing unit includes a controller 9020 , an internal bus 9021 , multiple pairs of logic 9030 , and a memory bank 9040 . The controller may be configured to operate as a communication module or may be coupled to the communication module.

The connectivity between the controller 9020 and multiple pairs of logic 9030 and memory bank 9040 may be implemented in other ways. The memory banks and logic may be arranged in different ways (not in pairs).

One or more memory/processing units 9010 of processing/memory subsystem 7110 may process model updates in parallel (using different logics and retrieving different parts of the model from different memory banks in parallel), and bulk The very high bandwidth of the memory resources of the memory bank and the connection between the memory bank and the logic allows these calculations to be performed in a very efficient manner.

The memory/processing units 7011 , 7012 , 7013 of FIGS. 87C-87E and the integrated circuits 7014 , 7015 of FIGS. 87C-87E include an Ethernet module ( FIGS. 87C-87G ) and an RDMA 7022 on the Ethernet module. ) (FIGS. 87e, 87g), including one or more communication modules.

Having such an RDMA and/or Ethernet module (either within the memory/processing unit or within the same integrated circuit as the memory/processing unit), communication between the different elements of a separate system is very fast, and in the case of RDMA, the It greatly simplifies communication between different elements.

Here, a memory/processing unit comprising RDMA and/or Ethernet modules may be advantageous in other circumstances, for example, even if the memory/processing unit is not included in a separate system.

Additionally, RDMA and/or Ethernet modules may be allocated per group of memory/processing units, eg for cost savings reasons.

The memory/processing units, groups of memory/processing units, and processing/memory subsystems may include other communication ports, such as PCIe communication ports.

Utilizing RDMA and/or Ethernet modules can be cost-effective as there is no need to connect memory/processing units with bridges that connect to network integrated circuits (NICs) that may have Ethernet ports.

Utilizing RDMA and/or Ethernet modules allows Ethernet (or RDMA over Ethernet) to be native within the memory/processing unit.

Here, Ethernet is only an example of a local area network (LAN) protocol, and PCIe is only an example of another communication protocol that can be used over a larger distance than Ethernet.

87H illustrates a method 7000 for distributed processing.

Method 7000 may include one or more processing iterations.

The processing iterations may be executed by one or more memory processing integrated circuits in separate systems.

The processing iterations may be performed by one or more processing integrated circuits in separate systems.

A processing iteration executed by one or more processing integrated circuits may be followed by a processing iteration executed by one or more processing integrated circuits.

The processing iteration executed by the one or more processing integrated circuits may be preceded by the processing iteration executed by the one or more processing integrated circuits.

Another processing iteration may be performed by other circuits in a separate system. For example, one or more preprocessing circuits may perform any and all types of preprocessing, including preparing information units for processing iterations by one or more memory processing integrated circuits, and the like.

Method 7000 may include step 7020 in which one or more memory processing integrated circuits in the separate system receive the unit of information.

Each memory processing integrated circuit may include a controller, multiple processor subunits, and multiple memory units.

One or more memory processing integrated circuits may be fabricated by a memory additive manufacturing process.

The information unit may carry part of the neural network model.

The information unit may carry a partial result of at least one database query.

The information unit may carry partial results of at least one aggregate database query.

Step 7020 may include receiving information units from one or more storage subsystems of the separate system.

Step 7020 may include receiving units of information from one or more computing subsystems of the separate system, wherein the one or more computing subsystems may include multiple processing integrated circuits fabricated by a logic additive manufacturing process.

In step 7030 after step 7020, the one or more memory processing integrated circuits may perform a processing operation on the information unit to provide a processing result.

The total size of the information unit may be greater than, equal to, or smaller than the total size of the processing result.

In step 7040 after step 7030, the one or more memory processing integrated circuits output processing results.

Step 7040 may include outputting the processing results to one or more computing subsystems of the separate system, wherein the one or more computing subsystems may include multiple processing integrated circuits manufactured by a logic additive manufacturing process.

Step 7040 may include outputting the processing result to one or more storage subsystems of the separate system.

The information units may be transmitted from different groups of processing units of the multiple processing integrated circuit and may be different parts of an intermediate result of a process executed in a distributed manner by the multiple processing integrated circuit. A group of processing units may include at least one processing integrated circuit.

Step 7030 may include processing the information units to provide a result of the overall process.

Step 7040 may include transmitting the result of the overall process to each of the multiple processing integrated circuits.

Different parts of the intermediate result may be different parts of the updated neural network model, where the result of the overall process is the updated neural network model.

Step 7040 may include transmitting the updated neural network model to each processing integrated circuit of the multiprocessing integrated circuit.

In step 7050 after step 7040, the multi-processing integrated circuit may perform other processing based at least in part on a result of processing with the multi-processing integrated circuit.

Step 7040 may include outputting a processing result by utilizing a switching subunit of the separated system.

Operation 7020 may include receiving an information unit that is a preprocessed information unit.

87I shows a method 7001 for distributed processing.

Method 7001 differs from method 7000 in that it includes a step 7010 in which the multiprocessing integrated circuit preprocesses information to provide preprocessed information units.

After step 7010, steps 7020, 7030, and 7040 may be performed.

Accelerate database analytics

A computer-readable medium is provided that stores apparatus, methods and instructions for performing at least filtering by a filtering unit belonging to the same integrated circuit as the memory unit. Here, the filter indicates which entries are relevant to a particular database query. An arbitrator or any other flow control manager may effectively reduce most traffic to and from the processor by sending relevant entries to the processor or not sending irrelevant entries to the processor.

For example, FIG. 91A shows an integrated circuit including a processor (CPU 9240) and a memory and filtering system (9220). The memory and filtering system 9220 may include a filtering unit 9224 coupled to one or more arbiters, such as an arbiter 9229 that transmits a memory unit entry 9222 and associated entries to the processor. Any and all arbitration processes may be applied. There may be any and all relationships between the number of entries, the number of filtering units, and the number of arbiters.

The arbiter may be replaced by any and all units capable of controlling the flow of information, such as a communication interface, a flow controller, and the like.

Filtering is based on one or more relevance/filtering criteria.

Relevance can be established per database query and can be expressed in any and all manner. For example, the memory unit may store an association flag 9224' indicating which entry is relevant. Also, there is a storage device 9210 that stores K database segments 9220(k), where k ranges from 1 to K. Here, the entire database is stored in a memory unit and may not be stored in a storage device (this solution is also referred to as a volatile memory storage database).

A memory unit entry may be too small to store the entire database, so it can receive one segment at a time.

The filtering unit may perform a filtering operation such as comparing the value of the field to a threshold value, comparing the value of the field to a predetermined value, determining whether the value of the field is within a predetermined range, and the like.

Thus, the filtering unit can perform known database filtering operations and can be a small and inexpensive circuit.

The output 9101 of the filtering operation (eg, the contents of an associated database entry) is sent to the CPU 9420 for processing.

The memory and filtering system 9220 may be replaced with a memory and processing system as shown in FIG. 91B .

The memory and processing system 9229 includes a processing unit 9225 coupled to a memory unit entry 9222 . The processing unit 9225 may perform a filtering operation and may at least partially participate in the execution of one or more additional operations on the associated record.

A processing unit may be a programmable unit tailored to perform a particular operation and/or configured to perform multiple operations. For example, the processing unit may be a pipelined processing unit, may include an ALU, may include multiple ALUs, and the like.

The processing unit 9225 may all perform one or more additional operations.

Alternatively, a portion of the one or more additional operations may be executed by the processing unit, and the processor (CPU 9240) may execute other portions of the one or more additional operations.

The output of the processing operation (eg, partial response 9102 or full response 9103 to database query) is sent to CPU 9420 .

Partial responses require additional processing.

92A illustrates a memory/processing system 9228 including a memory/processing unit 9227 configured to perform filtering and further processing.

Memory/processing system 9228 implements the processing unit and memory unit of FIG. 91 by memory/processing unit 9227 .

The role of the processor may include controlling the processing unit, executing at least a portion of one or more additional operations, and the like.

A combination of a memory entry and a processing unit may be implemented, at least in part, by one or more memory/processing units.

92B shows an example of a memory/processing unit 9010 .

Memory/processing unit 9010 includes a controller 9020 , an internal bus 9021 , and multiple pairs of logic 9030 and memory bank 9040 . The controller may be configured to operate as a communication module or may be coupled to the communication module.

The connectivity between the controller 9020 and multiple pairs of logic 9030 and memory bank 9040 may be implemented in other ways. The memory banks and logic may be arranged in different ways (not in pairs). Multiple memory banks may be coupled to and/or managed by a single logic.

Database query 9100 is received by the memory/processing system via interface 9211 . Interface 9211 may be a bus, port, input/output interface, or the like.

wherein the response to the database is provided by at least one (or a combination thereof) of one or more memory/processing systems, one or more memory and processing systems, one or more memory and filtering systems, and one or more processors located external to such systems, and the like. can be created

wherein the response to the database is generated by at least one (or a combination thereof) of one or more filtering units, one or more memory/processing units, one or more processing units, and one or more other processors (eg, one or more other CPUs), etc. can be

Any and all processes may include finding the relevant database entry and thereafter processing the relevant database entry. The processing may be executed by one or more processing entities.

A processing entity is a memory and processing unit of a processing system (eg, processing unit 9225 of memory and processing system 9229 ), a processor subunit (or logic) of a memory/processing unit (eg, FIGS. 91A , 91B , and It may be at least one of the CPU 9240 of FIG. 74 ).

The processing involved in generating a response to a database query may be generated by any one or combination of the following.

a. A processing unit 9225 of the memory and processing system 9229 .

b. Processing units 9225 in different memory and processing systems 9229 .

c. A processor subunit (or logic 9030 ) of one or more memory/processing units 9227 of memory/processing system 9228 .

d. Processor subunits (or logic 9030 ) of memory/processing units 9227 of different memory/processing systems 9228 .

e. Controllers of one or more memory/processing units 9227 of memory/processing system 9228 .

f. Controllers of one or more memory/processing units 9227 of different memory/processing systems 9228 .

Accordingly, the processing involved in responding to a database query may include (a) one or more controllers of one or more memory/processing units, (b) one or more processing units of memory/processing systems, (c) one or more memory/processing units, and (c) one or more memory/processing units. may be executed by any combination or sub-combination of one or more processor subunits, and (d) one or more other processors, etc.

Processing performed by more than one processing entity may be referred to as distributed processing.

Here, the filtering may be performed by a filtering entity in one or more filtering units and/or one or more processing units and/or one or more processing subunits. In this dimension, a processing unit and/or a processing subunit that performs a filtering operation may be referred to as a filtering unit.

The processing entity may be a filtering entity or may be different from a filtering unit.

A processing entity may perform processing operations on database entries deemed relevant by other filtering entities.

The processing entity may also perform filtering operations.

A response to a database query may utilize one or more filtering entities and one or more processing entities.

The one or more filtering entities and the one or more processing entities may belong to the same system (eg, memory/processing system 9228 , memory and processing system 9229 , memory and filtering system 9220 ) or may belong to different systems.

A memory/processing unit may include multiple processor subunits. Processor subunits may operate independently of each other, may partially cooperate with each other, may engage in distributed processing, and the like.

92C illustrates multiple memory and filtering system 9220 , another multiple processor (eg, CPU 9240 ), and storage device 9210 .

Multiple memory and filtering system 9220 may engage (concurrently or asynchronously) in the filtering of one or more database entries based on one or more filtering criteria in one or more database queries.

92D illustrates multiple memory and processing system 9229 , another multiple processor (eg, CPU 9240 ), and storage device 9210 .

Multiple memory and processing system 9229 may engage (concurrently or asynchronously) processing and at least in part filtering involved in responding to one or more database queries.

92E illustrates multiple memory/processing system 9228 , another multiple processor (eg, CPU 9240 ), and storage device 9210 .

Multiple memory/processing system 9228 may engage (simultaneously or asynchronously) processing and at least in part filtering involved in responding to one or more database queries.

92F illustrates a method 9300 for accelerating database analysis.

The method 9300 may begin with step 9310 where the memory processing integrated circuit receives a database query comprising at least one relevance criterion representing a database entry in the database that is relevant to the database query.

A database entry of a database that is related to a database query may be all, some, one, or none of the database entries of the database.

A memory processing integrated circuit may be a controller, multiple processor subunits, and multiple memory units.

After step 9310, step 9320 may be performed in which the memory processing integrated circuit determines a group of relevant database entries stored in the memory processing integrated circuit based on at least one relevance criterion.

After step 9320, step 9330 of transferring the group of relevant database entries to the one or more processing entities to continue processing without substantially transferring the extraneous data entries stored in the memory processing integrated circuit to the one or more processing entities can be performed.

The phrase 'substantially not sending' means either not sending at all (in response to a database query) or sending a negligible number of irrelevant entries. Insignificant can mean up to 1, 2, 3, 4, 5, 6, 7, 8, 9, 10%, or it can mean transmitting an amount that has no significant impact on bandwidth.

After step 9330, step 9340 may be performed to process the relevant database entries of the group to provide a response to the database query.

[002] 92G illustrates a database analysis acceleration method 9301 .

It is assumed that all of the processing necessary for filtering and responding to database queries is executed by the memory processing integrated circuit.

The method 9301 may begin with step 9310 where the memory processing integrated circuit receives a database query comprising at least one relevance criterion representing a database entry in the database that is relevant to the database query.

After step 9310, step 9320 may be performed in which the memory processing integrated circuit determines a group of relevant database entries stored in the memory processing integrated circuit based on at least one relevance criterion.

After step 9320, step 9331 is performed of transferring the relevant database entries of the group to the one or more processing entities for complete processing without substantially transferring the extraneous data entries stored in the memory processing integrated circuit to the one or more processing entities. can be

After step 9331 , step 9341 may be performed to completely process the relevant database entries of the group to provide a response to the database query.

After step 9341, step 9351 of outputting a response to the database query from the memory processing integrated circuit may be performed.

92H illustrates a method 9302 for accelerating database analysis.

It is assumed that only a portion of the processing necessary for filtering and responding to database queries is executed by the memory processing integrated circuit. The memory processing integrated circuit may output partial results to be processed by one or more other processing entities located external to the memory processing integrated circuit.

The method 9301 may begin with step 9310 where the memory processing integrated circuit receives a database query comprising at least one relevance criterion representing a database entry in the database that is relevant to the database query.

After step 9310, step 9320 may be performed in which the memory processing integrated circuit determines a group of relevant database entries stored in the memory processing integrated circuit based on at least one relevance criterion.

After step 9320, a step 9332 of transferring the relevant database entries of the group to the one or more processing entities for some processing is performed without substantially transferring the extraneous data entries stored in the memory processing integrated circuit to the one or more processing entities. can be

After step 9332, step 9342 may be performed to partially process the relevant database entries of the group to provide an intermediate response to the database query.

After step 9342, step 9352 of outputting an intermediate response to the database query from the memory processing integrated circuit may be performed.

After step 9352, step 9390 may be performed to continue processing the intermediate response to provide a response to the database query.

92I illustrates a method 9303 for accelerating database analysis.

It is assumed that the filtering is performed by the memory processing integrated circuit but the processing of the relevant database entries is not performed by the memory processing integrated circuit. The memory processing integrated circuit may output a group of related database entries to be fully processed by one or more other processing entities located external to the memory processing integrated circuit.

The method 9301 may begin with step 9310 where the memory processing integrated circuit receives a database query comprising at least one relevance criterion representing a database entry in the database that is relevant to the database query.

After step 9310, step 9320 may be performed in which the memory processing integrated circuit determines a group of relevant database entries stored in the memory processing integrated circuit based on at least one relevance criterion.

After step 9320, transferring the group of relevant database entries to one or more processing entities located external to the memory processing integrated circuit without substantially transferring the extraneous data entries stored in the memory processing integrated circuit to the one or more processing entities. 9333 may be performed.

After step 9333, step 9391 of completely processing the intermediate response to provide a response to the database query may be performed.

92J illustrates a method 9304 for accelerating database analysis.

Method 9303 may begin with step 9315, in which the integrated circuit receives a database query comprising at least one relevance criterion indicative of a database entry in the database relevant to the database query, wherein the integrated circuit includes a controller, a filtering unit, and multiple It includes a memory unit.

After step 9315, step 9325 in which the filtering unit determines a group of relevant database entries stored in the integrated circuit based on at least one relevance criterion may be performed.

After step 9325, transferring the relevant database entries of the group to one or more processing entities located external to the integrated circuit to continue processing without substantially transferring the extraneous data entries stored in the integrated circuit to the one or more processing entities. Step 9335 may be performed.

Step 9391 may be performed after step 9335.

92K illustrates a method 9305 for accelerating database analysis.

The method 9305 may begin with step 9314, in which the integrated circuit receives a database query comprising at least one relevance criterion representing a database entry in the database relevant to the database query, wherein the integrated circuit includes a controller, a processing unit, and multiple It includes a memory unit.

After step 9314, step 9324 may be performed in which the processing unit determines a group of relevant database entries stored in the integrated circuit based on at least one relevance criterion.

After step 9324, step 9334 may be performed in which the processing unit processes the relevant database entries of the group to provide processing results without the processing unit processing the irrelevant data entries stored in the integrated circuit.

After step 9334, step 9344 of outputting a processing result from the integrated circuit may be performed.

In any of methods 9300 , 9301 , 9302 , 9304 , 9305 , the memory processing integrated circuit generates an output. The output can be a group of related database entries, one or more intermediate results, or one or more (complete) results.

A retrieval of one or more relevant database entries and/or one or more results (full results or intermediate results) may be performed from a filtering entity and/or processing entity of the memory processing integrated circuit prior to output.

The retrieval may be controlled in one or more ways and may be controlled by one or more controllers and/or arbiters of the memory processing integrated circuit.

Output and/or retrieval may include controlling one or more parameters of the retrieval and/or output. The parameters may include search timing, search rate, search source, bandwidth, search order, output timing, output rate, output source, bandwidth, output order, type of search method, type of arbitration method, and the like.

Output and/or retrieval may perform a flow control process.

Outputs and/or retrievals (eg, application of flow control processes) may be output from one or more processing entities and correspond to indicators regarding completion of processing of a group's database entries. The indicator may indicate whether an intermediate result is ready to be retrieved from the processing entity.

The output may include an attempt to match the bandwidth used in the output process to a maximum allowed bandwidth, via a link coupling the memory processing integrated circuit to a requester unit. This link may be a link to the receiver of the output of the memory processing integrated circuit. The maximum allowed bandwidth may depend on the capabilities and/or availability of the link, the capabilities and/or availability of the recipient of the outputted content.

Outputting may include an attempt to output the outputted content in a best or suboptimal manner.

The output of the output content may include an attempt to keep fluctuations in the output traffic rate below a threshold value.

Any of methods 9300 , 9301 , 9302 , 9305 may include one or more processing entities generating a processing status indicator, which may indicate progress of further processing of the group's relevant database entry. .

The processing included in any and all methods described above may be performed by more than a single processing entity. In this case, since the processing is executed in a distributed manner, it can be regarded as distributed processing.

As indicated above, processing may be performed in a hierarchical manner or in a flat manner.

Any of methods 9300 - 9305 may be executed by multiple systems capable of responding to multiple database queries simultaneously or sequentially.

word embedding

As previously discussed, word embeddings are a generic name for a collection of language modeling and feature learning approaches in natural language processing (NLP) in which words or phrases in a vocabulary are mapped to vectors of elements. Conceptually, a mathematical mapping is involved from a space with many dimensions per word to a continuous vector space with many fewer dimensions.

Vectors can be processed mathematically. For example, vectors belonging to a matrix may be summed to provide a summation vector.

As another example, the covariance of a matrix (of a sentence) can be calculated. This may include multiplication of a matrix and its transposed matrix.

The memory/processing unit may store the vocabulary. In particular, portions of a vocabulary may be stored in multiple memory banks of a memory/processing unit.

Thus, the memory/processing unit can be accessed with access information (eg, a search key) that represents a word or phrase in a sentence. Accordingly, a vector representing a word or phrase of a sentence is retrieved from at least some of the memory banks of the memory/processing unit.

Different memory banks of the memory/processing unit can store different parts of the vocabulary and can be accessed in parallel (according to the index distribution of the sentence). Prediction can also reduce the penalty when more than a single line of a memory bank needs to be accessed sequentially.

Allocating words of the vocabulary between different memory banks of the memory/processing unit may be best or highly desirable in terms of improving the likelihood of parallel access to different memory banks of the memory/processing unit per sentence. These assignments may be learned per user, per general public, or per group of people.

In addition, the memory/processing unit may also be utilized to perform (by logic) at least a portion of the processing operations, thereby reducing the bandwidth required from the bus external to the memory/processing unit and performing multiple operations in an efficient manner (parallel). ) can be computed (utilizing multiple processors in a memory/processing unit in parallel).

A memory bank may be associated with logic.

At least some of the processing operations may be executed by one or more additional processors (eg, a vector processor including a vector adder, etc.).

A memory/processing unit may include one or more additional processors (logic pairs) that may be allocated as part or all of a memory bank.

Thus, a single additional processor may be allocated as all or part of a memory bank (logic pair). As another example, the additional processors may be arranged in a hierarchical manner such that some level of additional processors may process the output of lower level additional processors.

Here, the processing operation may be executed by the logic of the memory/processing unit without using an additional processor.

89A, 89B, 89C, 89D, 89E, 89F, and 89G illustrate examples of memory/processing units 9010, 9011, 9012, 9013, 9014, 9015, and 9019, respectively. Memory/processing unit 9010 includes a controller 9020 , an internal bus 9021 , and multiple pairs of logic 9030 and memory bank 9040 .

Here, logic 9030 and memory bank 9040 may be coupled to the controller and/or to each other in other ways. For example, it may be possible for multiple buses to be provided between the controller and the logic, for the logic to be placed in multiple layers, for a single logic to be shared by multiple memory banks (see the example of FIG. 89E ), and the like.

The page length of each memory bank within the memory/processing unit 9010 may be defined in any and all manner. For example, the length may be small enough, and the number of memory banks may be large enough to enable the output of a large number of vectors in parallel without wasting many bits on unrelated information.

Logic 9020 may include a full ALU, a partial ALU, a memory controller, a partial memory controller, and the like. A partial ALU (memory controller) unit is capable of executing only a fraction of the functions executable by a full ALU (memory controller). Any and all logic or subprocessors shown in this application may include full ALUs, partial ALUs, memory controllers, partial memory controllers, and the like.

The connectivity between the controller 9020 and multiple pairs of logic 9030 and memory bank 9040 may be implemented in other ways. The memory banks and logic may be arranged in different ways (eg, not in pairs).

There may be no additional vectors in the memory/processing unit 9010 , and processing of the vectors (from the memory bank) is performed by the logic 9030 .

89B shows an additional processor, such as vector processor 9050, coupled to internal bus 9021.

89C shows an additional processor, such as vector processor 9050, coupled to internal bus 9021. One or more additional processors execute (alone or in conjunction with logic) the processing operations.

89D shows host 9018 coupled to memory/processing unit 9010 via bus 9022 .

89D also shows vocabulary 9070 mapping word/phrase 9072 to vector 9073 . The memory/processing unit is accessed utilizing a search key 9071 representing each previously recognized word or phrase. The host 9018 may send the multiple search key 9071 representing the sentence to the memory/processing unit, which may output the result of the processing operation applied by the vector 9070 or the vector associated with the sentence. Words/phrases are not normally stored in memory/processing unit 9010 .

Memory controller functions for controlling the memory bank may be included in logic (completely or partially), included in controller 9020 (completely or partially), and/or one or more memory controllers (not shown) within memory/processing unit 9010 . city) can be included (completely or partially).

The memory/processing unit may be configured to maximize the throughput of vectors/results sent to the host 9018 or control the internal memory/processing unit traffic and/or control the memory/processing unit and the host computer (or external to the memory/processing unit). Any and all processes for controlling traffic between any other entity) may be applied.

Different logic 9030 may be coupled to the memory bank 9040 of the memory/processing unit and perform mathematical operations on the vectors (preferably in parallel) to produce the processed vectors. One logic 9030 may send the vector to another logic (see example line 38 in FIG. 89G ), and the other logic may apply mathematical operations to the received vector and the vector it computes itself. Logic can be arranged in hierarchies, and logic at a particular level can process vectors or intermediate results (creating applying mathematical operations) from previous levels of logic.

When the total size of the processed vector exceeds the total size of the result, a reduction in the output bandwidth (output bandwidth from the memory/processing unit) is ensured. For example, if K vectors are summed by the memory/processing unit to provide a single output vector, then a K:1 reduction in bandwidth is ensured.

The controller 9020 may be configured to open multiple memory banks in parallel by broadcasting the addresses of different vectors to be accessed.

The controller determines the order in which different vectors are retrieved from multiple memory banks (or any intermediate buffer or storage circuitry that stores different vectors; see buffer 9033 in Figure 89D) based at least in part on the order of words or phrases in the sentence. can be configured to control.

The controller 9020 may be configured to manage the retrieval of different vectors based on one or more parameters relating to an output external to the memory/processing unit 9010 of the vector. For example, the retrieval rate of different vectors from the memory bank may be set to be substantially equal to the allowable rate of outputting the different vectors from the memory/processing unit 9010 .

The controller may apply any traffic shaping process to output different vectors out of the memory/processing unit 9010 . For example, the controller 9020 may aim to output different vectors at a speed as close as possible to the maximum speed allowed by the host computer or the link coupling the memory/processing unit 9010 to the host computer. As another example, the controller may output different vectors while minimizing or at least substantially reducing traffic rate fluctuations over time.

The controller 9020 may belong to the same integrated circuit as the memory bank 9040 and the logic 9030, so that the retrieval states of different vectors (e.g., whether the vector is ready, whether the vector is ready but another vector from the same memory bank) Feedback on whether it is being retrieved or is about to be retrieved, etc.) can be readily received from different logic/memory banks. Feedback may be provided in any and all manner, eg, via dedicated control lines, shared control lines, etc., utilizing status bits, etc. (see status line 9039 in FIG. 89F ).

The controller 9020 can independently control the search and output of different vectors, thus reducing the intervention of the host computer. Alternatively, the host computer may not be aware of the controller's management capabilities and may continue to send detailed instructions. In this case, the memory/processing unit 9010 may ignore detailed instructions, hide management capabilities of the controller, and the like. The solution described above can be used based on a protocol manageable by the host computer.

It is very advantageous to perform processing operations in the memory/processing unit, even when such operations consume more power than processing operations within the host, even when these operations consume more power than transfer operations between the host and the memory/processing unit. It was found that there is (in terms of energy). For example, assuming that the vector is large enough, e.g. the energy consumption of transferring a data unit is 4pJ and the energy consumption of a processing operation (by the host) of the data unit is 0.1pJ, then the energy consumption of the data unit by the memory/processing unit is 0.1pJ. The processing was more effective when the energy consumption of processing of data units by the memory/processing unit was less than 5 pJ.

Each vector (a vector of matrices representing sentences) can be represented as a sequence of words (or other multi-bit segments). For convenience of description, it is assumed that a multi-bit segment is a word.

Further power reduction can be ensured if the vector contains zero value words. Instead of printing the entire zero-valued word, a zero-valued flag (sometimes passed to a dedicated control line) that is shorter than the word (eg 1 bit) may be output. Flags can be assigned different values (eg words with a value of 1).

88A shows a method 9400 for embedding, which may be a method for searching feature vector related information. The feature vector-related information may include a feature vector and/or a processing result of the feature vector.

The method 9400 may begin with step 9410 where the memory processing integrated circuit receives search information for a search of multiple requested feature vectors that may map to multiple sentence segments.

A memory processing unit may include a controller, multiple processor subunits, and multiple memory units. Each memory unit may be coupled to a processor subunit.

After operation 9410 , operation 9420 of retrieving multiple requested feature vectors from at least some of the multiple memory units may be performed.

The retrieval may include a request for a requested feature vector stored in the two or more memory units from the two or more memory units.

The request may be executed based on a known mapping between the sentence segment and the location of the feature vector mapped to the sentence segment.

The mapping may be uploaded during the boot process of the memory processing integrated circuit.

It may be advantageous to retrieve as many requested feature vectors as possible at once, but this depends on the number of memory units different from where the requested feature vectors are stored.

If more than one requested feature vector is stored in the same memory bank, a predictive search may be applied to reduce the penalty associated with retrieving information from the memory bank. Various methods for penalty reduction are shown in various parts of this application.

The search may include applying a predictive search of at least some requested feature vectors of a set of requested feature vectors stored in a single memory unit.

The requested feature vectors can be distributed among the memory units in an optimal way.

The requested feature vectors may be distributed among memory units based on expected search patterns.

Retrieval of multiple requested feature vectors may be performed according to a particular order, eg, according to the order of sentence segments of one or more sentences.

The retrieval of the multiple requested feature vectors may be performed, at least in part, out of order, wherein the retrieval may further include rearranging the order of the multiple requested feature vectors.

Retrieving the multiple requested feature vector may include buffering the multiple requested feature vector before the multiple requested feature vector is read by the controller.

Retrieving the multiple requested feature vectors may include generating a buffer status indicator that indicates when one or more buffers associated with the multiple memory units store the one or more requested feature vectors.

The method may include passing a buffer status indicator through a dedicated control line.

A dedicated control line may be allocated for each memory unit.

The buffer status indicator may be a status bit stored in one or more buffers.

The method may include passing the buffer status indicator over one or more shared control lines.

After step 9420 , step 9430 of processing the multi-requested feature vector to provide a processing result may be performed.

Additionally or alternatively, after step 9420, step 9440 of outputting from the memory processing integrated circuit an output that may include at least one of (a) the requested feature vector and (b) a result of processing the requested feature vector is performed can be At least one of (a) the requested feature vector and (b) the result of processing the requested feature vector is also referred to as feature vector related information.

When step 9430 is executed, step 9440 may include outputting (at least) a result of processing the requested feature vector.

If step 9430 is skipped, step 9440 may include outputting the requested feature vector and may not include outputting the processing result of the requested feature vector.

88B illustrates a method 9401 for embedding.

It is assumed that the output contains the requested feature vector but not the result of processing the requested feature vector.

The method 9401 may begin with step 9410 where the memory processing integrated circuit receives search information for a search of multiple requested feature vectors that may be mapped to multiple sentence segments.

After operation 9410 , operation 9420 of retrieving multiple requested feature vectors from at least some of the multiple memory units may be performed.

After step 9420 , step 9431 of outputting an output including the requested feature vector but not including a result of processing the requested feature vector from the memory processing integrated circuit may be performed.

88C illustrates a method 9402 for embedding.

It is assumed that the output contains the result of processing the requested feature vector.

The method 9402 may begin with step 9410 where the memory processing integrated circuit receives search information for a search of multiple requested feature vectors that may be mapped to multiple sentence segments.

After operation 9410 , operation 9420 of retrieving multiple requested feature vectors from at least some of the multiple memory units may be performed.

After step 9420, step 9430 of processing the multi-requested feature vector to provide a processing result may be performed.

After step 9430 , step 9442 of outputting an output that may include a result of processing the requested feature vector from the memory processing integrated circuit may be performed.

Outputting the output may include applying traffic shaping to the output.

Outputting the output may include, via a link coupling the memory processing integrated circuit to the requester unit, attempting to match the bandwidth used in the outputting process to a maximum allowed bandwidth.

Outputting the output may include attempting to keep fluctuations in the output traffic rate below a threshold.

Any of the steps of retrieving and outputting may be executed independently or in part independently by a controller and/or under the control of the host.

The host may send search commands of different granularity, ranging from the transmission of general search information independent of the location of the requested feature vector within multiple memory units to the transmission of detailed search information based on the location of the requested feature vector within multiple memory units. .

The host may control (or attempt to control) the timing of different seek operations within the memory processing integrated circuit, but may not care about such timing.

The controller may be controlled at various levels by the host, and may even override the host's detailed commands and at least independently control the retrieval and/or output.

Processing of the requested feature vector may be executed by at least one (or a combination thereof) of one or more memory/processing units and one or more processors located external to the one or more memory/processing units, and the like.

wherein the processing of the requested feature vector is performed by at least one (or a combination thereof) of one or more processor subunits, a controller, one or more vector processors, and one or more memory/processing units located external to the one or more memory/processing units. can be executed

The processing of the requested feature vector may be executed and generated by any one or a combination of the following.

a. Processor subunit (or logic 9030) of memory/processing unit

b. Processor subunit (or logic 9030) of multiple memory/processing units

c. Controller of memory/processing unit

d. Controller of multiple memory/processing units

e. One or more vector processors in the memory/processing unit

f. One or more vector processors in multiple memory/processing units

Accordingly, processing of the requested feature vector may result in (a) one or more controllers of one or more memory/processing units, (b) one or more processor subunits of one or more memory/processing units, (c) one or more memory/processing units, and (c) one or more memory/processing units. may be executed by any combination or sub-combination of one or more vector processors of

Processing performed by two or more processing entities may be referred to as distributed processing.

A memory/processing unit may include multiple processor subunits. The processor subunits can operate independently of each other, partially cooperate with each other, participate in distributed processing, and the like.

The processing may be executed in a flat manner in which all processor subunits perform the same operation (and may or may not output the result of the processing between operations).

The processing may be performed in a hierarchical fashion in which a sequence of processing operations at different levels is included. Here, a processing operation of a particular layer follows a processing operation of another level. Processor subunits can be assigned (dynamic or static) to different layers and participate in layer processing.

The processing of the requested feature vector may be executed by two or more processing entities (processor subunits, controllers, vector processors, other processors), and may be distributed processing in any and all manner (flat, hierarchical, or otherwise). For example, the processor subunits may output the processing result to the controller, and the controller may further process the result. One or more other processors located external to the one or more memory/processing units may further process the output of the memory processing integrated circuit.

Here, the search information may also include information for searching for a requested feature vector that is not mapped to a sentence segment. These feature vectors may map to one or more people, devices, or any other entity that may be associated with a sentence segment. For example, this may include the user of the device that detected the sentence segment, the device that detected the sentence segment, the user identified as the source of the sentence segment, the website accessed at the time the sentence was generated, the location where the sentence was captured, and the like.

Methods 9400 , 9401 , 9402 are applicable mutatis mutandis to requested search vectors and/or processing that are not mapped to sentence segments.

Examples of processing of feature vectors may include, but are not limited to, summing, weighted summing, averaging, subtracting, or application of any and any other mathematical function.

hybrid device

As both processor speed and memory size continue to increase, a significant limit to effective processing speed is the von Neumann bottleneck. The von Neumann bottleneck is caused by the throughput limitations of conventional computer architectures. In particular, the transfer of data from memory (ie, memory external to the logic die, such as external DRAM memory) to the processor is often a bottleneck compared to the actual computation performed by the processor. Accordingly, in memory-intensive processing, the number of clock cycles for reading and writing to the memory significantly increases. As clock cycles are spent reading and writing to memory and cannot be utilized to perform operations on data, the result is a loss in effective processing speed. Also, the computational bandwidth of a processor is generally greater than the bandwidth of the bus that the processor uses to access memory.

This bottleneck is particularly evident in memory-intensive processes such as neural networks and other machine learning algorithms, database construction, index searches, query operations, and other tasks that involve more read and write operations than data processing operations.

This disclosure describes solutions for alleviating or overcoming one or more of the problems listed above, among other problems in the prior art.

A hybrid device may be provided for memory intensive processing, wherein the hybrid device uses a base die, multiple processors, a first memory resource of at least one other die, and a second memory resource of at least one other die. may include

The base die and at least one other die are connected to each other by wafer on wafer bonding.

The multiple processors are configured to perform processing operations and retrieve retrieved information stored in the first memory resource.

The second memory resource is configured to transfer the additional information from the second memory resource to the first memory resource.

the overall bandwidth of the first path between the base die and the at least one other die is greater than the bandwidth of the second path between the at least one other die and the at least one other die, and the storage capacity of the first memory resource is greater than that of the second memory resource. It is several times smaller than the storage capacity.

The second memory resource is a high bandwidth memory (HBM) resource.

At least one other die is a stack of high-bandwidth memory (HBM) chips.

At least a portion of the second memory resource may belong to another die connected to the base die by a wafer-to-wafer junction and other connectivity.

At least a portion of the second memory resource may belong to another die connected to another die by a wafer-to-wafer junction and other connectivity.

The first memory resource and the second memory resource are cache memories of different levels.

The first memory resource is located between the base die and the second memory resource.

The first memory resource is located next to the second memory resource.

The other die is configured to perform additional processing, and the other die includes a plurality of processor subunits and a first memory resource.

Each processor subunit is coupled to a unique portion of a first memory resource allocated to the processor subunit.

The unique portion of the first memory resource is at least one memory bank.

A multiprocessor is a plurality of processor subunits included in a memory processing chip that also includes a first memory resource.

The base die includes multiple processors, the multiple processors being a plurality of processor subunits coupled via conductors formed in a wafer-to-wafer junction.

Each processor subunit is coupled to a unique portion of a first memory resource allocated to the processor subunit.

A hybrid integrated circuit capable of coupling at least a portion of a base die to a second memory resource contained in one or more other dies and coupled utilizing wafer-on-wafer (WOW) connectivity and other connectivity utilizing WOW connectivity. can An example of the second memory resource may be a high bandwidth memory (HBM) resource. In the various figures, a second memory resource is included in a stack of HBM memory units that may be coupled to a controller utilizing through silicon via (TSV) connectivity. The controller is included in the base die or coupled to at least a portion of the base die (eg, via micro bumps).

The base die may be a logic die but may also be a memory/processing unit.

WOW connectivity may be utilized to couple one or more portions of a base die to one or more portions of another die (WOW connected die), which may be a memory die or a memory/processing unit. WOW connectivity is a very high-throughput connectivity.

Stacks of high-bandwidth memory (HBM) chips can be coupled to a base die (either directly or via a WOW-connected die) and can provide high-throughput connectivity and highly expanded memory resources.

WOW connected dies may be coupled between a stack of HBM chips and a base die to form an HBM memory chip stack with TSV connectivity and WOW connected dies underneath.

A stack of HBM chips with TSV connectivity and with WOW connected dies underneath can provide a multi-layered memory layer that can be used as a lower level memory (eg level 3 cache) where the WOW connected dies can be accessed as a base die, where Fetch and/or pre-fetch operations from the higher level HBM memory stack fill the WOW connected die.

The HBM memory chip may be an HBM DRAM chip, although any and all other memory technologies may be used.

The combination of WOW connectivity and HBM chips can provide a multi-level memory architecture that can include multiple memory layers that can provide a balance between bandwidth and memory density.

The presented solution can serve as an additional and entirely new memory layer between the conventional DRAM memory/HBM and the internal cache of the logic die.

This could provide a new memory layer on DRAM that better manages memory reads in a faster way.

93A to 93I show hybrid integrated circuits 11011'-11019', respectively.

93A shows TSV connectivity and micro-bumps at the lowest level, comprising stacks of HDM DRAM memory chips 11032 connected to each other utilizing TSVs 11039 and coupled to a first memory controller 11031 on a base die. HBM DRAM stacking (collectively designated 11030) is shown.

93A also shows a second memory controller 11022 of a base die 11019 coupled to a DRAM wafer 11021 via one or more WOW interlayers 11023 having at least memory resources and coupled utilizing WOW technology. A wafer (collectively designated 11040) is shown. The one or more WOW interlayers may be composed of different materials but may have different pad connectivity and/or TSV connectivity.

Conductors 11022' pass through one or more WOW interlayers and electrically couple the DRAM die to the components of the base die.

The base die 11019 is coupled to an interposer 11018 , which in turn is coupled to the package substrate 11017 utilizing micro bumps. An array of micro bumps is provided on the lower surface of the package substrate.

Micro bumps can be replaced with other connectivity. The interposer 11018 and the package substrate 11017 may be replaced with other layers.

The first memory controller 11031 and/or the second memory controller 11032 are external (at least partially) to the outside of the base die 11019, such as in a DRAM wafer, between the DRAM wafer and the base die, and the stacking of HBM memory units; It may be located between the base dies, etc.

The first memory controller 11031 and the second memory controller 11032 may belong to the same controller or different controllers.

One or more of the HBM memory units may include logic as well as memory, and may be or include a memory/processing unit.

The first and second memory controllers may be coupled to each other by multiple buses 11016 to transfer information between the first memory resource and the first memory resource. 93A also shows a bus 11014 from the second memory controller to the components of the base die (eg, multiple processors). Also shown in FIG. 93A is a bus 11015 from the first memory controller to the components of the base die (eg, multiple processors as shown in FIG. 93C ).

93B shows a hybrid integrated circuit 11012 different from the hybrid integrated circuit 11011 of FIG. 93A by having a memory/processing unit 11021' instead of a DRAM die 11021.

93C shows an HBM memory chip stack (collectively designated 11040) with TSV connectivity and a WOW connected die underneath, including a DRAM die 11021 between a stack of HBM memory units and a base die 11018. A hybrid integrated circuit 11013 different from the hybrid integrated circuit 11011 of FIG. 93A is shown in a point.

DRAM die 11021 is coupled to first memory controller 11031 of base die 11019 utilizing WOW technology (see WOW intermediate layer 11023 ). One or more of the HBM memory dies 11032 may include both logic and memory and may be or include a memory/processing unit.

The bottommost DRAM die (shown as DRAM die 11021 in FIG. 93C ) may be an HBM memory die or different from the HBM memory die. As shown in the hybrid integrated circuit 11014 of FIG. 93D , the lowest DRAM die (DRAM die 11021) may be replaced with a memory/processing unit 11021'.

93e to 93g show hybrid integrated circuits 11015, 11016, and 11016', respectively, in which the base die 11019 has TSV connectivity and an HBM DRAM stack 11030 with micro-bumps at the lowest level and at least a memory resource. and multiple instances of wafer 11020 bonded utilizing WOW technology and/or multiple instances of HBM memory chip stack 11040 with TSV connectivity and WOW connected dies underneath.

93H illustrates a hybrid integrated circuit 11014' that differs from the hybrid integrated circuit 11014 of FIG. 93D by showing a memory unit 53, a level 2 cache memory (L2 cache 11052) and multiple processors 11051. it will be shown Multiple processors 11051 may be coupled to L2 cache 11052 and fed by coefficients and/or data stored in memory unit 11053 and L2 cache 11052 .

Any hybrid integrated circuit of the hybrid integrated circuit may be utilized for bandwidth intensive artificial intelligence (AI) processing.

The memory/processing unit 11021' of Figures 93D and 93H, when coupled to a memory controller with WOW technology, can perform AI calculations and retrieve data and counts from HBM DRAM stacks and/or WOW connected dies at very high rates. all can be received.

Any and all memory/processing units may include distributed memory arrays and processor arrays. Distributed memory arrays and processor arrays may include multiple memory banks and multiple processors. Multiple processors may form a processing array.

93c, 93d, and 93h, assuming that a hybrid integrated circuit 11013, 11014, or 11014' is required to perform general matrix-vector multiplications (GEMVs) that include a matrix-by-vector calculation , this type of computation is bandwidth intensive because it does not reuse the retrieved matrix data. Therefore, the entire matrix has to be searched and used only once.

GEMV is (i) multiplying a first matrix (A) by a first vector (V1) to provide a first intermediate vector, applying a first nonlinear operation (NLO1) to the first intermediate vector to provide a first intermediate result, , (ii) multiplying the second matrix (B) by the first intermediate result to give a second intermediate vector, applying a second nonlinear operation (NLO2) to the second intermediate vector to give a second intermediate result, etc. (N may be part of a sequence of mathematical operations including (until receiving an Nth intermediate result greater than 2).

Assuming each matrix is large (eg 1 gigabit), the computation requires 1 terabit of computational power and 1 terabit of bandwidth/throughput. Operations and computations can be executed in parallel.

Assume that the GEMV calculation represents N=4 and has the form: Result = NLO4(D*(NLO3(C*(NLO2(B*(NLO1(A*V1)))))))).

Also, assuming that DRAM die 11021 (or memory/processing unit 11021') does not have enough memory to store A, B, C, and D simultaneously, at least some of these matrices are the HDM DRAM die 11032 ) is stored in

It is assumed that the base die is a logic die that includes computation units such as processors, ALUs, and the like.

While the first die calculates A*V1 , the first memory controller 11031 retrieves the missing portions of another matrix from the one or more HBM DRAM die 11032 for the next calculation.

93h, (a) DRAM die 11021 has a bandwidth of 2 TB and a capacity of 512 Mb, (b) HBM DRAM die 11032 has a bandwidth of 0.2 TB and a capacity of 8 Gb, and (c) L2 Assume that the cache 11052 is SRAM with a bandwidth of 6 TB and a capacity of 10 Mb.

Multiplication of matrices results in data reuse, i.e. dividing a large matrix into segments (eg 5 Mb segments to fit into an L2 cache that can be used in a double buffer configuration) and dividing the first matrix segment into segments of the second matrix (another matrix segment). followed by a second matrix segment).

While multiplying the first matrix segment by the second matrix segment, another second matrix segment is fetched from the DRAM die 11021 (of the memory processing unit 11021 ′) into the L2 cache.

Assuming that the matrices are 1 Gb each, a DRAM die 11021 or memory/processing unit 11021' is supplied by matrix segments from the HBM DRAM die 11032 while fetch operations and calculations are being executed.

The DRAM die 11021 or memory/processing unit 11021 ′ sums the matrix segments, and the matrix segments are supplied to the base die 11019 through the WOW intermediate layer 11023 .

The memory/processing unit 11021 ′ performs the calculation instead of sending the calculated intermediate value to provide the result and transmits the result to reduce the amount of information transmitted to the base die 11019 through the WOW intermediate layer 11023 . can If multiple (Q) median values are processed to provide a result, the compression ratio may be Q:1.

93I shows an example of a memory processing unit 11019' implemented with WOW technology. A logic unit 9030 (which may be a processor subunit), a controller 9020 , and a bus 9021 are located in a first chip 11061 , and a memory bank 9040 assigned to a different logic unit is located in a second chip It is located in (11062). Here, the first chip and the second chip are connected to each other utilizing a conductor 11012 ′ passing through a WOW junction 11061 which may include one or more WOW interlayers.

93J is an example of a method 11100 for memory intensive processing. Memory intensive means that processing requires or is associated with high bandwidth memory consumption.

Method 11100 may begin with steps 11110 , 11120 , and 11130 .

Step 11110 includes multiple processors of the hybrid device comprising a base die, a first memory resource of at least one other die, and a second memory resource of at least one other die, performing a processing operation, wherein the base The die and at least one other die are connected to each other by wafer-on-wafer bonding.

Step 11120 includes the multi-processor retrieving the retrieved information stored in the first memory resource.

Step 11130 may include transmitting additional information from the second memory resource to the first memory resource, wherein the overall bandwidth of the first path between the base die and the at least one other die is between the at least one die and the at least one other die. greater than the overall bandwidth of the second path between different dies, and the storage capacity of the first memory resource is several times smaller than the storage capacity of the second memory resource.

Method 11100 may further include step 11140 in which another die including the plurality of processor subunits and the first memory resource performs additional processing.

Each processor subunit may be coupled to a unique portion of a first memory resource allocated to that processor subunit.

The unique portion of the first memory resource is at least one memory bank.

Step 11110, step 11120, step 11130, and step 11140 may be executed concurrently, in a partially overlapping manner, or the like.

The second memory resource may be a high bandwidth memory (HBM) resource or may be different from the HBM memory resource.

At least one other die is a stack of HBM memory chips.

communication chip

A database contains many entries containing several fields. Database processing may include one or more filtering parameters (eg, identifiers of one or more relevant fields or values of one or more relevant fields) and may also determine the type of operation to be executed, a variable or constant to be used when applying the operation, etc. It typically involves the execution of one or more queries that include one or more operational parameters. Data processing may include database analysis or other database processes.

For example, a database query may request that a statistical operation be performed (operation parameter) on all records in the database for which a particular field has a value within a predetermined range (a filtering parameter). As another example, a database query may request that records with certain fields less than a threshold (a filtering parameter) be deleted (a computational parameter).

Bulk databases are typically stored on storage devices. To respond to a query, the database is transferred to a memory unit, one database segment at a time.

Entries in the database segment are transferred from the memory unit to a processor that does not belong to the same integrated circuit as the memory unit. The entry is then processed by the processor.

For each database segment of the database stored in the memory unit, processing includes the following steps: (i) selecting a record of the database segment; (ii) transferring the write from the memory unit to the processor; (iii) the processor filtering the records to determine whether they are relevant records; and (iv) performing one or more additional operations (summing, application of any other mathematical and/or statistical operations) on the relevant records.

When all records are sent to the processor and the processor determines that the records are relevant, the filtering process ends.

If the relevant entries in the database are not stored in the processor, it is necessary to send these relevant records to the processor in order to continue processing after the filtering step (applying operations following processing).

When multiple processing operations follow a single filtering, the result of each operation may be sent to a memory unit, and then sent back to the processor.

This process is bandwidth and time consuming.

There is a growing need to provide an efficient way to perform database processing.

An apparatus may be provided that may include a database acceleration integrated circuit.

one or more database acceleration integrated circuit groups comprising one or more database acceleration integrated circuit groups that can be configured to exchange information and/or acceleration results (results of processing performed by the database acceleration integrated circuits) between the database acceleration integrated circuits of the one or more database acceleration integrated circuit groups A device may be provided.

The database acceleration integrated circuits of the database acceleration integrated circuit group may be connected to the same printed circuit board.

The database acceleration integrated circuits of the database acceleration integrated circuit group may belong to a modular unit of the computing system.

Different groups of database acceleration integrated circuits may be connected to different printed circuit boards.

Different groups of database acceleration integrated circuits may belong to different modular units of a computing system.

The apparatus may be configured to execute a distributed process by one or more groups of database acceleration integrated circuits.

The apparatus may be configured to use the at least one switch to exchange at least one of (a) information and (b) a database acceleration result between one or more groups of different groups of database acceleration integrated circuits.

The apparatus may be configured to execute a distributed process by a portion of a group of database acceleration integrated circuits in one or more groups.

The apparatus may be configured to perform a process of distributing the first data structure and the second data structure, wherein a total size of the first data structure and the second data structure is greater than a storage capacity of the multiple memory processing integrated circuit.

The apparatus may be configured to: (a) newly allocate different pairs of the first data structure part and the second data structure part to different database acceleration integrated circuits and (b) perform the distributed process by repeating the different pairs of processing multiple times. can

94A and 94B show examples of a storage system 11560 , a computer system 11510 , and one or more devices 11520 for database acceleration. The one or more devices 11520 for database acceleration monitor communication between the storage system 11560 and the computer system 11510 in various ways, such as by detecting or locating between the storage system 11560 and the computer system 11510 . can do.

The storage system 11560 may include many (eg, 20 or more, 50 or more, 100 or more, etc.) storage units (eg, disks) and may store, for example, 100 TB or more of information. The computer system 11510 may be a massive computer system and may include tens, hundreds, or thousands of processing units.

The computer system 11510 may include multiple compute nodes 11512 controlled by a manager 11511 .

The compute node may control one or more devices 11520 for database acceleration or may interact with one or more devices 11520 for database acceleration.

One or more devices 11520 for database acceleration may include one or more database acceleration integrated circuits (eg, database acceleration integrated circuits 11530 of FIGS. 94A and 94B ) and memory resources 11550 . A memory resource may belong to one or more chips dedicated to memory, but may also belong to a memory/processing unit.

94C and 94D illustrate examples of a computer system 11510 and one or more devices 11520 for database acceleration.

The one or more database acceleration integrated circuits of the one or more devices 11520 for database acceleration are located in a computer system (see FIG. 94c) or a management unit 11513 located within the one or more devices 11520 for database acceleration (see FIG. 94d). can be controlled by

94E shows an apparatus 11520 for database acceleration that includes a database acceleration integrated circuit 11530 and a multiple memory processing integrated circuit 11551 . Each memory processing integrated circuit may include a controller, multiple processor subunits, and multiple memory units.

The database acceleration integrated circuit 11530 includes a network communication interface 11531 , a first processing unit 11532 , a memory controller 11533 , a database acceleration unit 11535 , an interconnect 11536 , and a management unit 11513 . is shown as

Network The communication network 11531 may be configured to receive a large amount of information from a large number of storage units (eg, via a first port 11531( 1 ) of a network communication interface). Each storage unit can output information at rates of tens and hundreds of megabytes per second, and data transfer rates are expected to continue to increase in the future (eg, doubling every 2-3 years). The number of storage units may be 10, 50, 100, 200, or more. Vast amounts of information can be tens and hundreds of gigabytes per second or more, and can even range from terabytes to petabytes per second.

The first processing unit 11532 may be configured to provide first processing information by primary processing (pre-processing) a vast amount of information.

The memory controller 11533 may be configured to transmit the first processing information to the multi-memory processing integrated circuit via the massive throughput interface 11534 .

The multiple memory processing integrated circuit 11551 may be configured such that the multiple memory processing integrated circuit secondary processes (processes) at least a portion of the first processing information to provide the second processing information.

Memory controller 11533 may be configured to retrieve information retrieved from multiple memory processing integrated circuits. The retrieved information may include at least one of (a) at least a portion of the first processing information and (b) at least a portion of the second processing information.

The database acceleration unit 11535 may be configured to perform a data processing operation on the retrieved information to provide a database acceleration result.

The database acceleration integrated circuit may be configured to output the database acceleration results, for example, via the one or more second ports 11531( 2 ) of the network communication interface.

Fig. 94E also shows a management unit 11513 configured to manage at least one of retrieval of the retrieved information, primary processing (pre-process), secondary processing (process), and tertiary processing (database processing). The management unit 11513 may be located external to the database acceleration integrated circuit.

The management unit may be configured to perform the management according to the execution plan. The execution plan may be generated by the management unit or may be generated by an entity located external to the database acceleration integrated circuit. The execution plan includes at least one of (a) instructions to be executed by the various components of the database acceleration integrated circuit, (b) data and/or coefficients required for implementation of the execution plan, and (c) memory allocation of instructions and/or data. may include

The management unit allocates at least a portion of (a) a network communication network interface resource, (b) a decompression unit resource, (c) a memory controller resource, (d) a multiple memory processing integrated circuit resource, and (e) a data acceleration unit resource. and may be configured to perform management.

94E and 94G , the network communication network interface may include different types of network communication ports.

Different types of network communication ports include storage interface protocol ports (such as SATA ports, ATA ports, ISCSI ports, network file ports, Fiber Channel ports) and storage interface protocol ports over common networks (such as ATA over Ethernet, Ethernet over through Fiber Channel, NVME, Roce, etc.).

Different types of network communication ports may include storage interface protocol ports and PCIe ports.

94F shows a dotted line representing the flow of massive information, first processing information, searched information, and database acceleration results. 94F shows the database acceleration integrated circuit 11530 coupled to multiple memory resources 11550 . Multiple memory resources 11550 may not belong to a memory processing integrated circuit.

The apparatus 11520 for database acceleration may be configured such that the database acceleration integrated circuit 11530 executes multiple tasks simultaneously. That is, as the network communication interface 11531 can receive (simultaneously) multiple streams of information, the first processing unit 11532 can perform primary processing on multiple information units simultaneously, and the memory controller 11533 ) may transmit multiple primary processing information units to multiple memory processing integrated circuits 11551 at the same time, and database acceleration unit 11535 may process multiple retrieved information units simultaneously.

Apparatus for database acceleration 11520 may be configured to execute at least one of retrieval, primary processing, transmission, and tertiary processing based on an execution plan transmitted to the database acceleration integrated circuit by the compute node of the massive computer system. can

Apparatus for database acceleration 11520 may be configured to manage at least one of retrieval, primary processing, transmission, and tertiary processing in a manner that substantially optimizes utilization of the database acceleration integrated circuit. Optimization takes into account latency, throughput, and any other timing, storage, or processing and attempts to keep all components non-stop and bottleneck-free along the flow path.

The database acceleration integrated circuit may be configured to output a database acceleration result, eg, via the one or more second ports 11531(2) of the network communication interface.

Apparatus for database acceleration 11520 may be configured to substantially optimize bandwidth of traffic exchanged by a network communication network interface.

Apparatus for database acceleration 11520 may be configured to substantially avoid bottlenecking at least one of retrieval, primary processing, transmission, and tertiary processing in a manner that substantially optimizes utilization of the database acceleration integrated circuit. have.

The apparatus for database acceleration 11520 may be configured to allocate resources of the database acceleration integrated circuit according to a temporal I/O bandwidth.

94G illustrates an apparatus 11520 for database acceleration that includes a database acceleration integrated circuit 11530 and a multiple memory processing integrated circuit 11551 . 94G also shows various units coupled to database acceleration integrated circuit 11530 , such as remote RAM 11546 , Ethernet memory DIMMs 11547 , storage system 11560 , local storage unit 11561 , non-volatile memory (NVM). ) 11563 (the non-volatile memory may be an NVM express unit (NVME)) and the like are shown.

The database acceleration integrated circuit 11530 includes an Ethernet port 11531(4), an RDMA unit 11545, a serial scale-up port 11531(5), a SATA controller 11540, a PCIe port 11531(9), a first 1 processing unit 11532 , memory controller 11533 , database acceleration unit 11535 , interconnect 11536 , management unit 11513 , encryption engine 11537 for executing cryptographic operations, and level 2 SRAM (L2 SRAM) ) 11538 .

The database acceleration unit is shown comprising a DMA engine 11549 , a level 3 (L3) memory 11548 , and a database accelerator subunit 11547 . The database accelerator subunit 11547 may be a configurable unit.

Ethernet port 11531(4), RDMA unit 11545, serial scale-up port 11531(5), SATA controller 11540, PCIe port 11531(9) are each part of network communication interface 11531 can be considered as

Remote RAM 11546 , Ethernet memory DIMMs 11547 , and storage system 11560 are coupled to an Ethernet port 11531 ( 4 ), which in turn is coupled to an RDMA unit 11545 .

The local storage unit 11561 is coupled to the SATA controller 11540 .

PCIe port 11531 ( 9 ) is coupled to NVM 11563 . The PCIe port can also be utilized to exchange commands for management purposes, for example.

94H is an example of the database acceleration unit 11535 .

The database acceleration unit 11535 may be configured such that the database processing subunit 11573 concurrently performs database process instructions, wherein the database acceleration unit includes a group of database accelerator subunits sharing a shared memory unit 11575. can

Different combinations of database acceleration units 11535 may be dynamically linked together (via configurable links or interconnects 11576) to provide the execution pipeline required to execute database process operations that may include multiple instructions. can

Each database processor subunit may be configured to execute specific types of database process instructions (eg, filter, merge, accumulate, etc.).

94H also shows a separate database processing unit 11572 coupled to the cache 11571 . The database processing unit 11572 and the cache 11571 may be provided instead of the reconfigurable DB accelerator array 11574 or may be provided in addition to the reconfigurable DB accelerator array 11574 .

The apparatus facilitates scale-in and/or scale-out so that multiple database acceleration integrated circuits 11530 (and associated memory resources 11550 or multiple memory processing integrated circuits 11551) are , for example, may engage in distributed processing of database operations, allowing them to cooperate with each other.

94I illustrates a modular unit, such as blade 11580, including two database acceleration integrated circuits 1153 (and associated memory resources 11550). A blade may include one, two, or three or more memory processing integrated circuits 11551 and associated memory resources 11550 .

The blade may also include one or more non-volatile memory units, an Ethernet switch, a PCIe switch, and an Ethernet switch.

Multiple blades may communicate with each other utilizing any and all communication methods, communication protocols, and connectivity.

94I shows four database acceleration integrated circuits 11530 (and associated memory resources 11550) fully coupled to each other. Here, each database acceleration integrated circuit 11530 is connected to the remaining three database acceleration integrated circuits 11530 . Such connectivity may be ensured utilizing any and all communication protocols, such as, for example, RDMA protocol over the Internet.

Also shown in FIG. 94i is a database acceleration integrated circuit 11530 coupled to a unit 11531 comprising an associated memory resource 11550 and RAM and Ethernet ports.

94J, 94K, 94L, and 94M show four database acceleration integrated circuit groups 11580, each group comprising four database integrated circuits 11530 (fully coupled to each other) and associated memory resources. (11550) shows. The different groups are connected to each other via a switch 11590 .

The number of groups may be 2, 3, 4, or more. The number of database acceleration integrated circuits per group may be 2, 3, 4, or more. The number of groups may be equal to (or different from) the number of database acceleration integrated circuits per group.

94k shows two tables A and B that are too large (eg, 1 TB) to join together efficiently at the same time.

The tables are effectively broken down into patches, and the join operation is applied to a pair containing the patch in Table A and the patch in Table B.

A group of database-accelerated integrated circuits can process patches in a variety of ways.

For example, the apparatus may be configured to perform distributed processing by the following operations:

g. Allocating different first data structure portions (patches in Table A, eg first patch A0 to 16th patch A15) to one or more groups of different database acceleration integrated circuits.

h. (i) new assignment of different second data structure parts (patches in Table B, such as first patch B0 to 16th patch B15) to one or more groups of different database acceleration integrated circuits, and (ii) Perform multiple iterations of processing of the first data structure portion and the second data structure portion by the database acceleration integrated circuit.

The apparatus may be configured to execute the new assignment of the next iteration in a manner that at least partially overlaps in time with the processing of the current iteration.

The apparatus may be configured to effect the new allocation by exchanging the second data structure portion between the different database acceleration integrated circuits.

The exchange may be carried out in a manner that overlaps, at least in part, in time with the process.

The device exchanges a second data structure portion between a group of different database acceleration integrated circuits and, when there are no more second data structure portions to exchange, a new allocation by exchanging a second data structure portion between a different group of database acceleration integrated circuits. can be configured to run

In Figure 94k, some join operations of 4 cycles are shown. For example, referring to the database acceleration integrated circuit 11530 in the upper left group, 4 cycles are Join(A0, B0), Join(A0, B3), Join(A0, B2), and Join(A0, B1) includes the calculation of During these four cycles, A0 remains in the same database acceleration integrated circuit 11530 , while patches B0 , B1 , B2 , B3 in matrix B are interspersed between the components of the database acceleration integrated circuit 11530 in the same group. is rotated in

94L , the patches of the second matrix are rotated between different groups. That is, (a) patches B0, B1, B2 and B3 (the patches previously processed by the group on the top left) are sent from the group on the top left to the group on the bottom left, and (b) patches B4, B5, B6 and B7 (patches previously processed by the lower left group) are sent from the lower left group to the upper right group, (c) patches B8, B9, B10 and B11 (the patches previously processed in the upper right group) ) are sent from the upper right group to the lower right group, and (d) patches B12, B13, B14 and B15 (the patches previously processed in the lower right group) are sent from the lower right group to the upper left group. are sent

94n shows multiple blades 11580, SATA controller 11540, local storage unit 11561, NVME 11563, PCIe switch 11601, Ethernet memory DIMMs 11547, and Ethernet port 11531(4). An example of a system that includes

The blade 11580 may be coupled to each of a PCIe switch 11601 , an Ethernet port 11531 , and a SATA controller 11540 .

Two systems 11621 and 11622 are shown in Figure 94o.

The system 11621 may include one or more devices for database acceleration 11520 , a switching system 11611 , a storage system 11612 , and a compute system 11613 . The switching system 11611 provides connectivity between one or more devices 11520 , the storage system 11612 , and the compute system 11613 for database acceleration.

System 11622 may include a storage system and one or more devices 11615 for database acceleration, a switching system 11611 , and a compute system 11613 . The switching system 11611 provides connectivity between the compute system 11613 and one or more devices 11615 for storage system and database acceleration.

95A illustrates a method 11200 for database acceleration.

The method 11200 may begin with step 12100 in which a network communication network interface of the database acceleration integrated circuit retrieves a vast amount of information from a vast number of storage units.

Connecting to a large number of storage units (eg utilizing multiple different buses) allows the network communication network interface to receive large amounts of information, even when the throughput of a single storage unit is limited.

After step 11210, a large amount of information may be first processed to provide the primarily processed information. Primary processing may include buffering, extracting information from payloads, removing headers, decompressing, compressing, decrypting, filtering database queries, or performing any other processing operation. Primary processing may also be limited to buffering.

After step 11210, step 11220 may be performed in which the memory controller of the database acceleration integrated circuit transmits the primary processed information to the multi-memory processing integrated circuit through the massive throughput interface, wherein each memory processing integrated circuit includes a controller, multiple It may include a processor subunit, and multiple memory units. The memory processing integrated circuit may be a memory/processing unit or a distributed processor or memory chip as shown elsewhere in this application.

After operation 11220 , operation 11230 in which the multi-memory processing integrated circuit provides secondary processed information by secondary processing at least a portion of the primary processed information may be performed.

Step 11230 may include concurrent execution of multiple tasks by the database acceleration integrated circuit.

Step 11230 may include concurrent execution of a database processing instruction by the data processing subunit, wherein the database acceleration unit may include a group of database accelerator subunits sharing a shared memory unit.

After step 11230, step 11240 in which the memory controller of the database acceleration integrated circuit retrieves information retrieved from the multi-memory processing integrated circuit may be performed, wherein the retrieved information includes (a) at least a portion of the primary processed information and (b) ) may include at least one of at least a portion of the secondary processed information.

After step 11240 , step 11250 in which the database acceleration unit of the database acceleration integrated circuit provides a database acceleration result by performing a database processing operation on the retrieved information may be performed.

Step 11250 may include allocating resources of the database acceleration integrated circuit according to the time I/O bandwidth.

After step 11250, step 11260 of outputting a database acceleration result may be performed.

Step 11260 may include dynamically coupling database processing subunits to provide an execution pipeline necessary to execute database processing operations that may include multiple instructions.

Step 11260 may include outputting the database acceleration result to the local storage and retrieving the database acceleration result from the local storage.

Here, step 11210, step 11220, step 11230, step 11240, step 11250, step 11260 and any other steps of method 11100 may be executed in a pipelined manner. These steps may be performed concurrently or in an order different from that described above.

For example, step 11250 may be performed after step 1120 so that the primary processed information continues to be processed by the database acceleration unit.

For another example, the primary processed information may be sent to a multiple memory processing integrated circuit and then to a database acceleration unit (without being processed by the multiple memory processing integrated circuit).

As another example, the primary processed information and/or the secondary processed information may be output from the database acceleration integrated circuit without database processing in the data acceleration unit.

The method may include executing at least one of retrieval, primary processing, transmission, and tertiary processing based on an execution plan sent to the database acceleration integrated circuit by the compute node of the massive computing system.

The method may include managing at least one of retrieval, primary processing, transmission, and tertiary processing in a manner that substantially optimizes utilization of the database acceleration integrated circuit.

The method may include substantially optimizing a bandwidth of traffic exchanged by the network communication network interface.

The method may include substantially avoiding bottlenecking at least one of retrieval, primary processing, transmission, and tertiary processing in a manner that substantially optimizes utilization of the database acceleration integrated circuit.

Method 11200 may also include at least one of the following steps.

Step 11270 may include, by the management unit of the database acceleration integrated circuit, managing at least one of retrieval, primary processing, transmission, and tertiary processing.

The management may be executed according to the execution plan generated by the management unit of the database acceleration integrated circuit.

Management may be executed based on an execution plan not created in the database acceleration integrated circuit but received by the database acceleration integrated circuit.

The management comprises allocating at least a portion of (a) a network communication network interface resource, (b) a decompression unit resource, (c) a memory controller resource, (d) a multiple memory processing integrated circuit resource, and (e) a data acceleration unit resource. may include steps.

Operation 11271 may include, by the compute node of the massive computing system, controlling at least one of search, primary processing, transmission, and tertiary processing.

Step 11272 may include, by a management unit located external to the database acceleration integrated circuit, managing at least one of search, primary processing, transmission, and tertiary processing.

95B illustrates a method 11300 of operating a group of database acceleration integrated circuits.

The method 11300 may begin with step 11310 where the database acceleration integrated circuit performs a database acceleration operation. Step 11310 may include executing one or more steps of method 11200 .

Method 11300 may also include step 11320 exchanging at least one of (a) information and (b) database acceleration result between data acceleration integrated circuits of one or more groups of database acceleration integrated circuits.

The combination of steps 11310 and 11320 may result in execution of distributed processing by one or more groups of database acceleration integrated circuits.

The exchange may be performed using a network communication network interface of one or more groups of database acceleration integrated circuits.

The exchange can be carried out across multiple groups which can be connected to each other by a star-connection.

Step 11320 may include using the at least one switch to exchange at least one of (a) information and (b) database acceleration results between one or more groups of different groups of database acceleration integrated circuits.

Step 11310 may include step 11311 in which a portion of the database acceleration integrated circuit of the portion of one or more groups executes distributed processing.

Step 11311 may include performing a process of distributing the first data structure and the second data structure, wherein a total size of the first data structure and the second data structure is greater than a storage capacity of the multiple memory processing integrated circuit.

The step of performing distributed processing includes the steps of (a) newly allocating different pairs of the first data structure part and the second data structure part to different database acceleration integrated circuits, and (b) repeating the different pairs of processing several times. may include

Performance of distributed processing may include execution of a database join operation.

Step 11310 includes (a) allocating different first data structure parts to one or more groups of different database acceleration integrated circuits; performing step 11314 and step 11316 of the database acceleration integrated circuit processing the first data structure portions and the second data structure portions several times repeatedly.

Step 11314 may be executed in a manner that at least partially overlaps in time with the processing of the current iteration.

Step 11314 may include exchanging the second data structure portion between different database acceleration integrated circuits.

Step 11320 may be executed in a manner that at least partially overlaps in time with step 11310 .

step 11314 exchanging a second data structure portion between a group of different database acceleration integrated circuits and exchanging a second data structure portion between a different group of database acceleration integrated circuits when there are no more second data structure portions to exchange. may include

95C illustrates a method 11350 for database acceleration.

The method 11350 may begin with step 11352 in which a network communication network interface of the database acceleration integrated circuit retrieves a vast amount of information from a large number of storage units.

After operation 11352, operation 11354 of providing primary processed information by primary processing a large amount of information may be performed.

After operation 11352, operation 11354 in which the memory controller of the database acceleration integrated circuit transmits the primary-processed information to multiple memory resources through a massive throughput interface may be performed.

After step 11354, step 11356 of retrieving information retrieved from multiple memory resources may be executed.

After step 11356, step 11358 in which the database acceleration unit of the database acceleration integrated circuit performs a database processing operation on the retrieved information to provide a database acceleration result may be performed.

After step 11358, step 11359 of outputting a database acceleration result may be performed.

The method may also include a step 11355 of secondary processing the primary processed information to provide secondary processed information. The secondary processing may be executed by multiple processors located within one or more memory processing integrated circuits further comprising multiple memory resources. After step 11355, steps 11354 and 11356 are executed.

The total size of the secondary processed information may be smaller than the total size of the primary processed information.

The total size of the primary processed information may be smaller than the total size of the vast amount of information.

Primary processing may include filtering of database entries. Thus, by filtering out database entries that are not relevant to the query, bandwidth, storage resources, and processing resources can be saved before further processing and/or before storing the irrelevant database entries in multiple memory resources.

Secondary processing may include filtering of database entries. Filtering can be applied when the filtering condition is complex (eg contains multiple conditions) and multiple database entry fields must be received in case no filtering is performed. For example, (a) a person who is over a certain age and is like a banana, and (b) a person who is over a certain age and is like an apple may be the case.

database

The examples below relate to databases. The database may be a data center, may be part of a data center, or may not belong to a data center.

The database may be coupled to multiple users via one or more networks. The database may be a cloud database.

Multiple database accelerator boards may be provided that include a database including one or more management units and one or more memory/processing units.

96B shows a database 12020 comprising a management unit 12021 and multiple DB accelerator boards 12022 each including a communication/management processor (processor 12024 ) and multiple memory/processing units 12026 . .

The processor 12024 may support various communication protocols, such as protocols such as PCIe and ROCE.

The database instructions may be executed by the memory/processing unit 12026 , which processes the traffic between the different memory/processing units 12026 , between the different DB accelerator boards 12022 , and with the management unit 12021 . can be transmitted.

In particular, utilizing multiple memory/processing units 12026 with many internal memory banks can dramatically accelerate the execution of database commands and avoid communication bottlenecks.

96C shows a DB accelerator board 12022 including a processor 12024 and multiple memory/processing units 12026 . The processor 12024 includes multiple communication-only components such as a DDR controller 12033 for communicating with the memory/processing unit 12026 , an RDMA engine 12031 , a DB query database engine 12034 , and the like. A DDR controller is an example of a communication controller, and an RDMA engine is an example of an arbitrary communication engine.

A method of operating (or operating any portion of the system) of the system of FIGS. 96B , 96C , and 96D may be provided.

Here, the database acceleration integrated circuit 11530 may be associated with multiple memory resources not included in the multiple memory processing integrated circuit, or may not be associated with a processing unit. In this case, the processing is performed primarily and exclusively by the database acceleration integrated circuit.

94P illustrates a method 11700 for database acceleration.

Method 11700 may include step 11710 in which the network communication interface of the database acceleration integrated circuit retrieves information from the storage unit.

After step 11710, step 11720 of providing the primarily processed information by first processing the amount of information may be performed.

After operation 11720, operation 11730 in which the memory controller of the database acceleration integrated circuit transmits the primarily processed information to multiple memory resources through the throughput interface may be performed.

After step 11730, step 11740 of retrieving information from multiple memory resources may be performed.

After step 11740 , step 11750 in which the database acceleration unit of the database acceleration integrated circuit provides a database acceleration result by performing a database processing operation on the retrieved information may be performed.

After step 11750, step 11760 of outputting a database acceleration result may be performed.

Primary processing and/or secondary processing may include filtering of database entries to determine which database entries to continue processing.

Secondary processing includes filtering of database entries.

hybrid system

Memory/processing units are very effective when performing computations that can be memory intensive and/or when bottlenecks are related to retrieval operations. Processing intensive (and less memory intensive) processor units (eg GPUs, CPUs, etc.) can be more effective when the bottleneck is related to computational operations.

A hybrid system may include both one or more processor units and one or more memory/processing units, which may be fully or partially coupled to each other.

The memory/processing unit (MPU) may be manufactured by a first manufacturing process that is more suitable for memory cells than logic cells. For example, the critical dimension of a memory cell manufactured by the first manufacturing process may be less than or even significantly smaller than the critical dimension of a logic cell manufactured by the first manufacturing process (eg, exceeds 2, 3, 4, 5, as many arguments as 6, 7, 8, 9, 10, etc.). For example, the first manufacturing process may be an analog manufacturing process, a DRAM manufacturing process, or the like.

The processor may be manufactured by a second manufacturing process that is more suitable for the logic. For example, a critical dimension of a logic circuit manufactured by a second manufacturing process may be less than or even significantly smaller than a critical dimension of a logic circuit manufactured by a first manufacturing process. As another example, the critical dimension of a logic circuit manufactured by the second manufacturing process may be less than or even significantly smaller than the critical dimension of a memory cell manufactured by the first manufacturing process. For example, the second manufacturing process may be a digital manufacturing process, a CMOS manufacturing process, or the like.

Tasks can be assigned between different units in a static or dynamic manner, taking into account the advantages of each unit and the penalties associated with transferring data between the units.

For example, a memory-intensive process may be allocated to a memory/processing unit and a processing-intensive, non-memory intensive process may be allocated to a processing unit.

The processor may request or direct one or more memory/processing units to perform various processing tasks. Execution of various processing tasks may offload the process, reduce latency, in some cases reduce the overall bandwidth of information between one or more memory/processing units and the processor, and the like.

The processor may provide different granularity of indications and/or requests. For example, a processor may send an indication directed at a particular processing resource or a higher level indication directed at a memory/processing unit without designating any processing resource.

96D is an example of a hybrid system 12040 including one or more memory/processing units (MPUs) 12043 and a processor 12042 . As shown, the processor 12042 may send a request or instruction to one or more MPUs 12043 , which in turn perform (or optionally perform) the request and/or instruction and send the result to the processor. It can be sent to (12042).

Processor 12042 may further process the results to provide one or more outputs.

Each MPU includes memory resources, processing resources (eg, compact microcontroller 12044 ), and cache memory 12049 . A microcontroller may have limited computational power (eg it may primarily contain multiplication and accumulation units).

The microcontroller 12044 may apply the process for in-memory acceleration purposes, and may be a CPU or full DB processing engine or a subset thereof.

The MPU 12043 may include a microprocessor and a packet processing unit that may be coupled in a mesh, ring, or other topology for fast communication between banks.

There can be more than one DDR controller for fast communication between DIMMs.

The goal of the in-memory packet processor is to reduce BW, data movement, power consumption and improve performance. Utilizing them will dramatically improve performance/TCO over standard solutions.

Here, the management unit is optional.

Each MPU can perform artificial intelligence (AI) calculations and pass only the results to the processor, reducing the amount of traffic (especially when the MPU receives and stores neural network coefficients to be used for multiple computations) and a portion of the neural network is used It can act as an AI memory/processing unit as it does not need to receive coefficients from an external chip every time it processes new data.

The MPU may determine if the coefficient is zero and inform the processor that there is no need to perform multiplication involving the zero-valued coefficient.

Here, the primary processing and secondary processing may include filtering of database entries.

The MPU may be any and all memory processing units shown in this specification, PCT Application Publication No. WO2019025862 and PCT Patent Application No. PCT/IB2019/001005.

An AI computing system (and systems executable by the system) whose network interface card has AI processing capabilities and is configured to perform some AI processing tasks to reduce the amount of traffic to be sent over a network that combines multiple AI acceleration servers; can be provided.

For example, in some inference systems, the input is a network (eg multiple streams of IP cameras connected to an AI server). In this case, leveraging RDMA + AI on the processing and networking unit can reduce the load on the CPU and PCIe bus and provide processing on the processing and networking unit rather than by the GPU that is not included in the processing and networking unit. can

For example, instead of computing the initial result and sending it to the target AI acceleration server (the server that applies one or more AI processing operations), the processing and networking unit performs preprocessing that reduces the amount of values sent to the target AI acceleration server. can do. The target AI computing server is an AI computing server assigned to perform calculations on values provided by other AI acceleration servers. This reduces the bandwidth of traffic exchanged between AI acceleration servers and also reduces the load on the target AI acceleration server.

Target AI acceleration servers may be allocated in a dynamic or static manner, utilizing load balancing or other allocation algorithms. There may be more target AI acceleration servers than a single target AI acceleration server.

For example, if the target AI acceleration server adds multiple losses, the processing and networking unit may add the losses generated by the AI acceleration server and send the sum of the losses to the target AI acceleration server to reduce bandwidth. The same gain can be obtained in the case of performing a preprocessing operation such as a differential calculation, a synthesis, and the like.

97B shows a system 12060 including subsystems each including a switch 12061 for interconnecting an AI processing and networking unit 12063 with a server motherboard 12064 . The server motherboard has network capabilities and includes one or more AI processing and networking units 12063 with AI processing capabilities. AI processing and networking unit 12063 may include one or more NICs, and an ALU or other computational circuitry to perform preprocessing.

The AI processing and networking unit 12063 may be a chip or may include more than a single chip. It may be advantageous to have the AI processing and networking unit 12063 being a single chip.

AI processing and networking unit 12063 may include (only or primarily) processing resources. AI processing and networking unit 12063 may include in-memory computing circuitry, may not include in-memory computing circuitry, or may not include significant in-memory computing circuitry.

AI processing and networking unit 12063 may be an integrated circuit, may include one or more integrated circuits, may be part of an integrated circuit, or the like.

AI processing and networking unit 12063 passes traffic between the AI acceleration server including AI processing and networking unit 12063 and other AI acceleration servers (eg, utilizing DDR channels, network channels, and/or PCIe channels). ) can be done (see FIG. 97c). AI processing and networking unit 12063 may also be coupled to external memory, such as DDR memory. The processing and networking unit may include memory and/or may include a memory/processing unit.

97C , AI processing and networking unit 12063 is shown to include local DDR connections, DDR channels, AI accelerators, RAM memory, encryption/decryption engines, PCIe switches, PCIe interfaces, multi-core processing arrays, high-speed networking connections, and the like. is shown.

A method for operating (or operating any portion of) the system of any of the figures of FIGS. 97B and 97C may be provided.

Combinations of any and all steps of any and all methods mentioned in this application may be provided.

Any and all units, integrated circuits, memory resources, logic, processing subunits, controllers, and any combination of components mentioned herein may be provided.

Any reference to "including" and/or "comprising" may apply mutatis mutandis to "comprising" and "substantially consisting of".

The above description has been presented for purposes of illustration. This description is not exhaustive and is not limited to the precise form or embodiment disclosed. Modifications and adaptations will become apparent to those skilled in the art upon consideration of the present specification and practice of the disclosed embodiments. Also, although aspects of the disclosed embodiments have been described as being stored in memory, those skilled in the art will recognize that these aspects are, for example, hard disk or CD ROM, or other types of RAM or ROM, USB media, DVD, Blu-ray, UHD Blu-ray. , or other tangible computer readable media such as secondary storage such as optical drive media.

A computer program based on the disclosed description and method is of ordinary skill to those skilled in the art. Various programs or program modules may be created using techniques known to those skilled in the art or designed in conjunction with existing software. For example, a program section or program module can contain .Net Framework, .Net Compact Framework (and related languages such as Visual Basic, C, etc.), Java, C++, Objective-C, HTML, HTML/AJAX combination, XML, or Java applet. It can be designed with HTML containing (Java applet).

Moreover, although illustrated embodiments have been described herein, the scope of all embodiments is contemplated with equivalent elements, modifications, omissions, combinations (eg, combinations of aspects across various embodiments), applications, and/or variations. It is obvious to those skilled in the art. The limitations of the claims are to be interpreted broadly based on the language used in the claims and are not limited to the examples set forth herein or during the filing of the present invention. It is to be understood that the examples are not exclusive. Furthermore, the steps of the disclosed method may be modified in various ways, such as by rearranging the order of the steps and/or inserting or deleting steps. Accordingly, the specification and examples are to be considered for illustrative purposes only, and the true scope and spirit is defined by the following claims and their equivalents.

Claims (368)

Board;
a memory array disposed on the substrate and comprising a plurality of discrete memory banks;
a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks; and
An integrated circuit comprising a controller, comprising:
and the controller is configured to implement at least one security measure for the operation. According to claim 1,
and the controller is configured to take one or more solutions when the at least one security action is triggered. According to claim 1,
and the controller is configured to implement at least one security measure on the at least one memory location. 3. The method of claim 2,
The data includes weight data of a neural network model. According to claim 1,
and the controller is configured to implement at least one security measure comprising blocking access to one or more memory portions of the memory array that are not used for input data or output data operations. According to claim 1,
and the controller is configured to implement at least one security measure comprising blocking only a subset of the memory array. 7. The method of claim 6,
and said subset of said memory array is designated by a particular memory address. 7. The method of claim 6,
and said subset of said memory array is configurable. According to claim 1,
and the controller is configured to implement at least one security measure comprising controlling traffic to or from the integrated circuit. According to claim 1,
and the controller is configured to implement at least one security measure comprising uploading of mutable data, code, or fixed data. According to claim 1,
and the uploading of the mutable data, code, or fixed data occurs during a boot process. According to claim 1,
wherein the controller is configured to implement at least one security measure comprising uploading during the boot process a configuration file identifying a specific memory address for at least a portion of the memory array to be blocked upon completion of the boot process. Circuit. According to claim 1,
and the controller is further configured to request a password to unblock access to a memory portion of the memory array associated with one or more memory addresses. According to claim 1,
and the at least one security measure is triggered when an attempt to access at least one blocked memory address is detected. According to claim 1,
The controller calculates a checksum, a hash, a cyclic redundancy check (CRC), and a parity calculated for at least a portion of the memory array, the calculated checksum, the hash , CRC, or parity to a predetermined value. 16. The method of claim 15,
The controller is
and determine, as part of the at least one security measure, whether the calculated checksum, hash, CRC, or parity matches the predetermined value. According to claim 1,
and said at least one security measure comprises copying the program code in at least two different memory portions. 18. The method of claim 17,
and said at least one security measure comprises determining whether output results from execution of said program code of said at least two different memory portions differ from one another. 19. The method of claim 18,
and the output result comprises an intermediate output result or a final output result. 18. The method of claim 17,
and the at least two different memory portions are included within the integrated circuit. According to claim 1,
wherein said at least one security measure comprises determining whether an operating pattern differs from one or more predetermined operating patterns. 3. The method of claim 2,
and the one or more solutions include stopping execution of an operation. A method of protecting an integrated circuit from counterfeiting, comprising:
The method includes implementing, using a controller associated with the integrated circuit, at least one security measure for operation of the integrated circuit;
The integrated circuit comprises:
Board;
a memory array disposed on the substrate and comprising a plurality of discrete memory banks;
and a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks.
24. The method of claim 23,
and taking one or more remedies if the at least one security action is triggered.
Board;
a memory array disposed on the substrate and comprising a plurality of discrete memory banks;
a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks; and
An integrated circuit comprising a controller, comprising:
and the controller is configured to implement at least one security measure for operation of the integrated circuit, wherein the at least one security measure comprises copying program code in at least two different memory portions.
Board;
a memory array disposed on the substrate and comprising a plurality of discrete memory banks;
a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks; and
An integrated circuit comprising a controller configured to implement at least one security measure for operation of the integrated circuit.
27. The method of claim 26,
and the controller is further configured to take one or more solutions when the at least one security action is triggered.
Board;
a memory array disposed on the substrate and comprising a plurality of discrete memory banks;
a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with one or more of the plurality of discrete memory banks; and
a first communication port configured to establish a communication connection between the distributed processor memory chip and an external entity other than another distributed processor memory chip; and
and a second communication port configured to establish a communication connection between the distributed processor memory chip and the first additional distributed processor memory chip.
29. The method of claim 28,
and a third communication port configured to establish a communication connection between the distributed processor memory chip and a second additional distributed processing memory chip.
30. The method of claim 29,
and a controller configured to control communication through at least one of the first communication port, the second communication port, and the third communication port.
30. The method of claim 29,
and the first communication port, the second communication port, and the third communication port are each associated with a corresponding bus.
32. The method of claim 31,
and the corresponding bus is a bus common to each of the first communication port, the second communication port, and the third communication port.
32. The method of claim 31,
and the corresponding bus associated with each of the first communication port, the second communication port, and the third communication port are all coupled to the plurality of distributed memory banks.
32. The method of claim 31,
and at least one bus associated with the first communication port, the second communication port, and the third communication port is unidirectional.
32. The method of claim 31,
and at least one bus associated with the first communication port, the second communication port, and the third communication port is bidirectional.
31. The method of claim 30,
wherein the controller schedules a data transfer between the distributed processor memory chip and the first additional distributed processor memory chip, wherein the first additional distributed processor memory chip is received during a time period based on the data transfer and during which the data transfer is received. A distributed processor memory chip, configured to cause a processor subunit to execute program code associated therewith.
31. The method of claim 30,
the controller transmits a clock enable signal to the at least one processor subunit of the plurality of processor subunits of the distributed processor memory chip to enable the at least one processor subunit of the plurality of processor subunits A distributed processor memory chip configured to control one or more aspects of operation.
38. The method of claim 37,
The controller controls the clock enable signal sent to the at least one processor subunit of the plurality of processor subunits to time the one or more communication commands associated with the at least one processor subunit of the plurality of processor subunits. Distributed processor memory chip, characterized in that configured to control the.
31. The method of claim 30,
and the controller is configured to selectively initiate execution of program code by one or more processor subunits of the plurality of processor subunits of the distributed processor memory chip.
31. The method of claim 30,
wherein the controller is configured to utilize a clock enable signal to control the timing of data transmission from one or more processor subunits of the plurality of processor subunits to at least one of the second communication port and the third communication port. Distributed processor memory chip.
29. The method of claim 28,
and a communication rate associated with the first communication port is lower than a communication rate associated with the second communication port.
31. The method of claim 30,
The controller determines whether a first processor subunit of the plurality of processor subunits is ready to transmit data to a second processor subunit included in the first additional distributed processor memory chip, and the first processor subunit and use a clock enable signal to initiate transmission of data from the first processor subunit to the second processor subunit after determining that the data is ready to be transmitted to the second processor subunit. Distributed processor memory chip.
43. The method of claim 42,
The controller uses the clock enable signal to determine whether the second processor subunit is ready to receive the data, and after determining that the second processor subunit is ready to receive the data and initiate the transfer of the data from one processor subunit to the second processor subunit.
43. The method of claim 42,
the controller determines whether the second processor subunit is ready to receive the data, and determines whether the second processor subunit of the first additional distributed processor memory chip is ready to receive the data and wherein the distributed processor memory chip is further configured to buffer the data included in the transmission.
A method for transferring data between a first distributed processor memory chip and a second distributed processor memory chip, the method comprising:
whether a first processor subunit of a plurality of processor subunits disposed on the first distributed processor memory chip is ready to transmit data to a second processor subunit included in the second distributed processor memory chip; determining using a controller associated with at least one of a distributed processor memory chip and the second distributed processor memory chip; and
After determining that the first processor subunit is ready to transmit the data to the second processor subunit, the first processor subunit uses a clock enable signal controlled by the controller to transfer the data to the second processor subunit and initiating transmission of the data to a unit.
46. The method of claim 45,
determining, using the controller, whether the second processor subunit is ready to receive the data; and
initiating the transmission of the data from the first processor subunit to the second processor subunit by utilizing the clock enable signal after the second processor subunit determines that it is ready to receive the data; How to include more.
46. The method of claim 45,
determine whether the second processor subunit is ready to receive the data, and include in the transmission until after determining that the second processor subunit of the first additional distributed processor memory chip is ready to receive the data The method further comprising the step of buffering (buffering) the data.
Board;
a memory array disposed on the substrate and comprising a plurality of discrete memory banks;
a first communication port configured to establish a communication connection between the memory chip and an external entity other than the memory chip; and
and a second communication port configured to establish a communication connection between the memory chip and the first additional memory chip.
49. The method of claim 48,
and the first communication port is connected to at least one of a main bus inside the memory chip and at least one processor subunit included in the memory chip.
49. The method of claim 48,
and the second communication port is connected to at least one of a main bus inside the memory chip and at least one processor subunit included in the memory chip.
a memory array including a plurality of memory banks;
at least one controller configured to control at least one aspect of a read operation for the plurality of memory banks; and
at least one zero value detection logic configured to detect a multi-bit zero value associated with data stored at a particular address of the plurality of memory banks;
and the at least one controller is configured to send a zero-value indicator to the one or more circuits in response to detection of a zero value by the at least one zero-value detection logic unit.
52. The method of claim 51,
and the one or more circuits to which the zero-value indicator is sent are external to the memory unit.
52. The method of claim 51,
and the one or more circuits to which the zero-value indicator is sent are internal to the memory unit.
52. The method of claim 51,
and at least one read deactivation element configured to abort a read command associated with the particular address when the at least one zero value detection logic detects a zero value associated with the particular address.
52. The method of claim 51,
and the at least one controller is configured to transmit the zero-value indicator to the one or more circuits instead of transmitting the zero-valued data stored at the specific address.
52. The method of claim 51,
The size of the zero-value indicator is smaller than the size of the zero-value data.
52. The method of claim 51,
Energy expended by a first process comprising (a) detecting the zero-value, (b) generating the zero-value indicator, and (c) transmitting the zero-value indicator to the one or more circuits is the zero-valued data is less than the energy consumed by transmitting to the one or more circuits.
58. The memory unit of claim 57, wherein the energy consumed by the first process is less than half of the energy consumed by transferring the zero-valued data to the one or more circuits.
52. The method of claim 51,
at least one sense amplifier configured to exclude activation of at least one memory bank of the plurality of memory banks after detection of a zero value by the at least one zero value detection logic.
60. The method of claim 59,
wherein the at least one sense amplifier is configured to sense a low power signal from the plurality of memory banks and amplify a small voltage swing to a high voltage level such that data stored in the plurality of memory banks can be interpreted by the at least one controller. A memory unit comprising a plurality of transistors.
52. The method of claim 51,
Each memory bank of the plurality of memory banks is further organized into subbanks, wherein the at least one controller includes a subbank controller, and the at least one zero value detection logic includes zero value detection logic associated with the subbank. A memory unit, characterized in that.
62. The method of claim 61,
and at least one read disable element comprising a sense amplifier associated with each of said subbanks.
52. The method of claim 51,
a plurality of processor subunits spatially distributed within the memory unit, each processor subunit of the plurality of processor subunits being associated with at least one memory bank dedicated to the plurality of memory banks; wherein each processor subunit of the processor subunit is configured to access and operate data stored in a corresponding memory bank.
64. The method of claim 63,
and the one or more circuits include one or more of the processor subunits.
64. The method of claim 63,
and each processor subunit of the plurality of processor subunits is connected to two or more other processor subunits of the plurality of processor subunits by one or more buses.
52. The method of claim 51,
A memory unit further comprising a plurality of buses.

67. The method of claim 66,
and said plurality of buses are configured to transfer data between said plurality of memory banks.
68. The method of claim 67,
and at least one of the plurality of buses is configured to transmit the zero-value indicator to the one circuit.
receiving a read request of data stored in addresses of a plurality of memory banks from a circuit external to the memory unit;
activating a zero-value detection logic in response to the received request, so that the controller detects a zero value at the received address; and
and sending, by the controller, a zero-value indicator to the circuit in response to the zero-value detection by the zero-value detection logic unit.
70. The method of claim 69,
and if the zero detection logic detects a zero value associated with the requested address, configuring the read disable element to cause the controller to abort a read command associated with the requested address.
70. The method of claim 69,
and configuring a sense amplifier such that when the zero value detection logic detects a zero value, the controller excludes activation of at least one memory bank of the plurality of memory banks.
A non-transitory computer-readable medium storing a set of instructions executable by a controller of the memory unit to cause a memory unit to detect a value of zero at a particular address in a plurality of memory banks, the method comprising:
receiving a read request of data stored in addresses of a plurality of memory banks from a circuit external to the memory unit;
activating a zero-value detection logic in response to the received request, so that the controller detects a zero value at the received address; and
and in response to detection of the zero value by the zero value detection logic, the controller sending a zero value indicator to the circuitry.
73. The method of claim 72,
The method further comprises configuring a read disable element such that when the zero detection logic detects a zero value associated with the requested address, the controller aborts a read command associated with the requested address. A non-transitory computer-readable medium characterized in that
73. The method of claim 72,
The method further comprises configuring the sense amplifier such that when the zero value detection logic detects a zero value, the controller excludes activation of at least one memory bank of the plurality of memory banks. a non-transitory computer-readable medium.
a plurality of memory banks, at least one controller configured to control at least one aspect of a read operation for the plurality of memory banks, and at least one configured to detect a multi-bit zero value associated with data stored at a specific address of the plurality of memory banks a memory unit including one zero-value detection logic; and
a processing unit configured to send a read request to the memory unit to read data from the memory bank;
wherein the at least one controller and the at least one zero detection logic are configured to send a zero-value indicator to the one or more circuits in response to detection of a zero value by the at least one zero-value detection logic.
a memory array including a plurality of memory banks;
at least one controller configured to control at least one aspect of a read operation for the plurality of memory banks; and
at least one detection logic configured to detect a predetermined multi-bit value associated with data stored at a specific address of the plurality of memory banks;
and the at least one controller is configured to send a value indicator to one or more circuits in response to detection of the predetermined multi-bit value by the at least one detection logic unit.
77. The method of claim 76,
The predetermined multi-bit value is selectable by a user.
a memory array including a plurality of memory banks;
at least one controller configured to control at least one aspect of a write operation to the plurality of memory banks; and
at least one detection logic configured to detect a predetermined multi-bit value associated with data to be written to a particular address of the plurality of memory banks;
and the at least one controller is configured to send a value indicator to one or more circuits in response to detection of the predetermined multi-bit value by the at least one detection logic unit.
Board;
a memory array including a plurality of memory banks disposed on the substrate;
a plurality of processor subunits disposed on the substrate;
at least one controller configured to control at least one aspect of a read operation for the plurality of memory banks; and
at least one detection logic configured to detect a predetermined multi-bit value associated with data stored at a specific address of the plurality of memory banks;
and the at least one controller is configured to send a value indicator to one or more of the plurality of processor subunits in response to detection of the predetermined multi-bit value by the at least one detection logic. .
one or more memory banks;
bank controller; and
an address generator;
The address generator is:
providing to the bank controller a current address of a current row to be accessed within an associated memory bank of the one or more memory banks;
determine a predicted address of a next row to be accessed within the associated memory bank;
and provide the predicted address to the bank controller before the operation on the current row associated with the current address is complete.
81. The method of claim 80,
and the operation on the current row associated with the current address is a read operation or a write operation.
81. The method of claim 80,
and the current row and the next row are in the same memory bank.
83. The method of claim 82,
and the same memory bank allows the next row to be accessed while the current row is accessed.
81. The method of claim 80,
and the current row and the next row are in different memory banks.
81. The method of claim 80,
Further comprising a distributed processor,
wherein said distributed processor comprises a plurality of processor subunits of a processing array spatially distributed among a plurality of discrete memory banks of said memory array.
81. The method of claim 80,
and the bank controller is configured to access the current row and activate the next row prior to completion of the operation on the current row.
81. The method of claim 80,
The one or more memory banks each include at least a first sub-bank and a second sub-bank, and the bank controller associated with each of the one or more memory banks includes a first sub-bank controller associated with the first sub-bank and the second A memory unit comprising a second subbank controller associated with the subbank.
88. The method of claim 87,
the first subbank controller is configured to enable access to data included in a current row of the first subbank while the second subbank controller activates a next row in the second subbank unit.
89. The method of claim 88,
and the activated next row of a second subbank is spaced apart by at least two rows from the current row of the first subbank from which data is being accessed.
88. The method of claim 87,
the second subbank controller is configured to cause access to data contained in a current row of the second subbank while the first subbank controller activates a next row of the first subbank unit.
91. The method of claim 90,
and the activated next row of a first subbank is spaced apart by at least two rows from the current row of the second subbank from which data is being accessed.
81. The method of claim 80,
The predicted address is a memory unit, characterized in that determined using a learning neural network.
81. The method of claim 80,
The predicted address is determined based on the determined line access pattern.
81. The method of claim 80,
wherein the address generator comprises a first address generator configured to generate the current address and a second address generator configured to generate the predicted address.
95. The method of claim 94,
and the second address generator is configured to calculate the predicted address within a predetermined period of time after the first address generator generates the current address.
96. The method of claim 95,
and the predetermined time period is adjustable.
97. The method of claim 96,
and the predetermined period of time is adjusted based on a value of at least one operating parameter associated with the memory unit.
98. The method of claim 97,
and the at least one operating parameter comprises a temperature of the memory unit.
81. The method of claim 80,
wherein the address generator is further configured to generate a confidence level associated with the predicted address and cause the bank controller to relinquish access of the next row at the predicted address if the confidence level falls below a predetermined threshold. Characterized memory unit.
81. The method of claim 80,
and the predicted address is generated by a chain of flip flops that samples the delayed generated address.
101. The method of claim 100,
The delay is settable through a multiplexer that selects between flip-flops storing the sampled address.
81. The method of claim 80,
and the bank controller is configured to ignore the predicted address received from the address generator for a predetermined period after the reset of the memory unit.
81. The method of claim 80,
and the address generator is configured to abandon providing the predicted address to the bank controller after detecting a random pattern in row accesses associated with the associated memory bank.
one or more memory banks, wherein each memory bank of the one or more memory banks comprises:
multiple rows;
a first row controller configured to control a first subset of the plurality of rows;
a second row controller configured to control a second subset of the plurality of rows;
a single data input for receiving data to be stored in the plurality of rows; and
and a single data output for providing data retrieved from said plurality of rows.
105. The method of claim 104,
and the memory unit is configured to receive a first address for processing and a second address for activation and access at a predetermined time.
105. The method of claim 104,
and the first subset of the plurality of rows is comprised of even numbered rows.
107. The method of claim 106,
and the even numbered rows are arranged in half of the one or more memory banks.
107. The method of claim 106,
and odd numbered rows are arranged in half of the one or more memory banks.
105. The method of claim 104,
and the second subset of the plurality of rows consists of odd numbered rows.
105. The method of claim 104,
and the first subset of the plurality of rows is included in a first subbank of the memory bank adjacent a second subbank of the memory bank that includes the second subset of the plurality of rows.
105. The method of claim 104,
the first row controller is configured to cause access of data contained in a row in the first subset of the plurality of rows while the second row controller activates a row in the second subset of the plurality of rows A memory unit, characterized in that.
112. The method of claim 111,
and the activated row of the second subset of the plurality of rows is spaced apart by at least two rows from the row of the first subset of the plurality of rows from which data is being accessed.
105. The method of claim 104,
the second row controller is configured to cause access of data contained in a row in the second subset of the plurality of rows while the first row controller activates a row in the second subset of the plurality of rows A memory unit, characterized in that.
114. The method of claim 113,
and the activated row of the first subset of the plurality of rows is spaced apart by at least two rows from the row of the second subset of the plurality of rows from which data is being accessed.
105. The method of claim 104,
wherein each memory bank of said one or more memory banks includes a column input for receiving a column identifier indicating a portion of a row to be accessed.
105. The method of claim 104,
A memory unit, characterized in that one line of extra overlapping mats is disposed between each of the two mat lines to form a distance to allow activation.
105. The method of claim 104,
A memory unit characterized in that lines adjacent to each other are not activated at the same time.
Board;
a memory array disposed on the substrate and comprising a plurality of discrete memory banks;
a processing array disposed on the substrate and comprising a plurality of processor subunits each associated with a corresponding dedicated discrete memory bank of the plurality of discrete memory banks; and
at least one memory mat disposed on the substrate and configured to serve as at least one register in a register file for one or more processor subunits of the plurality of processor subunits.
119. The method of claim 118,
and the at least one memory mat is included in at least one processor subunit of the plurality of processor subunits of the processing array.
119. The method of claim 118,
wherein the register file is configured as a data register file.
119. The method of claim 118,
wherein the register file is configured as an address register file.
119. The method of claim 118,
wherein the at least one memory mat provides at least one register of a register file for one or more processor subunits of the plurality of processor subunits to store data to be accessed by the one or more processor subunits of the plurality of processor subunits; A memory chip comprising:
119. The method of claim 118,
the at least one memory mat is configured to provide at least one register of a register file for one or more processor subunits of the plurality of processor subunits, the at least one register of the register file comprising the plurality of processor subunits and store coefficients used by the plurality of processor subunits during execution of convolution accelerator operations by
119. The method of claim 118,
wherein the at least one memory mat is a DRAM memory mat.
119. The method of claim 118,
and the at least one memory mat is configured to communicate via a one-way access.
119. The method of claim 118,
and said at least one memory mat allows bidirectional access.
119. The method of claim 118,
at least one redundant memory mat disposed on the substrate, wherein the at least one redundant memory mat is configured to provide at least one redundant register for one or more processor subunits of the plurality of processor subunits memory chip with
119. The method of claim 118,
at least one redundant memory mat disposed on the substrate, wherein the at least one redundant memory mat is configured to provide at least one redundant register for one or more processor subunits of the plurality of processor subunits. A memory chip comprising redundant memory bits of
119. The method of claim 118,
a first plurality of buses each coupling one processor subunit of the plurality of processor subunits to a corresponding dedicated memory bank; and
and a second plurality of buses each connecting one processor subunit of the plurality of processor subunits to another processor subunit of the plurality of processor subunits.
119. The method of claim 118,
at least one processor subunit of the plurality of processor subunits comprises a counter configured to count backwards from a predetermined number, and when the counter reaches a value of zero, the at least one processor subunit of the plurality of processor subunits: A memory chip configured to suspend a current task and trigger a memory refresh operation.
119. The method of claim 118,
and at least one processor subunit of the plurality of processor subunits comprises a mechanism to refresh the memory mat by suspending a current task and triggering a memory refresh operation at a specific time.
119. The method of claim 118,
and the register file is configured to be used as a cache.
retrieving one or more data values from a memory array of a distributed processor memory chip;
storing the one or more data values in a register formed in a memory mat of the distributed processor memory chip; and
accessing the one or more data values stored in the register according to at least one instruction executed by a processor element;
wherein the memory array includes a plurality of discrete memory banks disposed on a substrate;
wherein the processor element is a processor subunit of a plurality of processor subunits included in a processing array disposed on the substrate, each processor subunit associated with a corresponding dedicated discrete memory bank of the plurality of discrete memory banks;
wherein the register is provided by a memory mat disposed on the substrate.
134. The method of claim 133,
wherein the processor element is configured to function as an accelerator;
accessing first data stored in the register:
accessing second data from the memory array; and
The method further comprising the step of performing an operation on the first data and the second data.
134. The method of claim 133,
The at least one memory mat comprises a plurality of word lines and bit lines,
and determining timing of loading the word line and the bit line according to the size of the memory mat.
134. The method of claim 133,
The method further comprising periodically refreshing the register.
134. The method of claim 133,
wherein the memory mat comprises a DRAM memory mat.
134. The method of claim 133,
wherein said memory mat is included in said plurality of discrete memory banks of said memory array.
Board;
a processing unit disposed on the substrate; and
a memory unit disposed on the substrate;
the memory unit is configured to store data to be accessed by the processing unit;
wherein the processing unit comprises a memory mat configured to serve as a cache for the processing unit.
A method for distributed processing of at least one information stream, the method comprising:
one or more memory processing integrated circuits receiving at least one information stream via a first communication channel, wherein each memory processing integrated circuit includes a controller, multiple processor subunits, and multiple memory units;
buffering, by the one or more memory processing integrated circuits, the at least one information stream;
performing, by the one or more memory processing integrated circuits, a primary processing operation on the at least one information stream to provide a primary processing result;
transmitting the primary processing result to a processing integrated circuit; and
performing, by the one or more memory processing integrated circuits, a secondary processing operation on the primary processing result to provide secondary processing results;
and a size of a logic cell of the one or more memory processing integrated circuits is less than a size of a logic cell of the processing integrated circuit.
140. The method of claim 140,
and each of said multiple memory units is coupled to at least one of said multiple processor subunits.
140. The method of claim 140,
The method according to claim 1, wherein a total size of information units of the at least one information stream received during a specific period of time is greater than a total size of primary processing results output during the specific period of time.
140. The method of claim 140,
and a total size of the at least one information stream is smaller than a total size of the primary processing result.
140. The method of claim 140,
The method of claim 1, wherein the memory additive manufacturing process is a DRAM manufacturing process.
140. The method of claim 140,
The processing integrated circuit is manufactured by a memory additive manufacturing process,
wherein the processing integrated circuit is fabricated by a logic additive manufacturing process.
140. The method of claim 140,
and a size of a logic cell of the one or more memory processing integrated circuits is at least twice a size of a corresponding logic cell of the processing integrated circuit.
140. The method of claim 140,
and a critical dimension of a logic cell of the one or more memory processing integrated circuits is at least twice a critical dimension of a corresponding logic cell of the processing integrated circuit.
140. The method of claim 140,
and a critical dimension of a memory cell of the one or more memory processing integrated circuits is at least twice a critical dimension of a corresponding logic cell of the processing integrated circuit.
140. The method of claim 140,
and requesting, by the processing integrated circuit, the one or more memory processing integrated circuits to perform the primary processing operation.
140. The method of claim 140,
and directing the processing integrated circuit to the one or more memory processing integrated circuits to perform the primary processing operation.
140. The method of claim 140,
and configuring the one or more memory processing integrated circuits to cause the processing integrated circuit to perform the primary processing operation.
140. The method of claim 140,
and the one or more memory processing integrated circuits executing the primary processing operation without intervention of the processing integrated circuit.
140. The method of claim 140,
wherein said primary processing operation is less memory intensive than said secondary processing operation.
140. The method of claim 140,
and a total throughput of the primary processing operation is greater than a total throughput of the secondary processing operation. 140. The method of claim 140,
wherein the at least one information stream comprises one or more preprocessed information streams. 156. The method of claim 155,
wherein the one or more preprocessed information streams are data extracted from network conveyed units. 140. The method of claim 140,
a portion of the primary processing operation is executed by the one processor subunit of the multiprocessor subunit, and another portion of the primary processing operation is executed by another processor subunit of the multiprocessor subunit. How to characterize. 140. The method of claim 140,
The method of claim 1, wherein the primary processing operation and the secondary processing operation include a mobile communication network processing operation. 140. The method of claim 140,
wherein the primary processing operation and the secondary processing operation comprise a database processing operation. 140. The method of claim 140,
wherein the primary processing operation and the secondary processing operation comprise a database analysis processing operation. 140. The method of claim 140,
The method of claim 1, wherein the primary processing operation and the secondary processing operation include an artificial intelligence processing operation. receiving, by one or more memory processing integrated circuits of a separate system comprising one or more computing subsystems separate from one or more storage subsystems, a unit of information, wherein each of the one or more memory processing integrated circuits comprises a controller, multiple a processor subunit, and multiple memory units, wherein the one or more computing subsystems include multiple processing integrated circuits, wherein a size of a logic cell of the one or more memory processing integrated circuits is a corresponding logic cell of the multiple processing integrated circuit. at least twice the size of ;
performing, by the one or more memory processing integrated circuits, a processing operation on the information unit to provide a processing result; and
outputting the processing result from the one or more memory processing integrated circuits.
163. The method of claim 162,
outputting the processing results to the one or more computing subsystems of the separate system.
163. The method of claim 162,
and receiving the unit of information to the one or more storage subsystems of the separate system.
163. The method of claim 162,
outputting the processing result to the one or more storage subsystems of the separate system.
163. The method of claim 162,
and receiving the unit of information to the one or more computing subsystems of the separate system.
166. The method of claim 166,
The information units transmitted from different groups of processing units of the multiple processing integrated circuits include different portions of intermediate results of a process executed by the multiple processing integrated circuits, and the group of processing units includes at least one processing integrated circuit. A method characterized in that
168. The method of claim 167,
The method further comprising the one or more memory processing integrated circuits outputting a result of the overall process.
169. The method of claim 168,
and transmitting the result of the overall process to each of the multiple processing integrated circuits. 169. The method of claim 168,
wherein the different parts of the intermediate result are different parts of an updated neural network model, and the result of the overall process is the updated neural network model.
169. The method of claim 168,
transmitting the updated neural network model to each of the multiprocessing integrated circuits.
163. The method of claim 162,
and outputting the processing result by utilizing a switching subunit of the separate system. 163. The method of claim 162,
wherein the one or more memory processing integrated circuits are included in a memory processing subunit of the separate system. 163. The method of claim 162,
wherein at least one of the one or more memory processing integrated circuits is included in one or more computing subsystems of the separate system.
163. The method of claim 162,
wherein at least one of the one or more memory processing integrated circuits is included in one or more memory subsystems of the separate system.
163. The method of claim 162,
wherein at least one of (a) said unit of information is received from at least one of said multiple processing integrated circuits and (b) said processing results are transmitted to one or more memory processing integrated circuits of said multiple processing integrated circuits. How to.
178. The method of claim 176,
and a critical dimension of a logic cell of the one or more memory processing integrated circuits is at least twice a critical dimension of a corresponding logic cell of the multiple processing integrated circuit.
178. The method of claim 176,
and a critical dimension of a memory cell of the one or more memory processing integrated circuits is at least twice a critical dimension of a corresponding logic cell of the multiple processing integrated circuit.
163. The method of claim 162,
The method of claim 1, wherein the information unit includes a preprocessed information unit.
180. The method of claim 179,
The method further comprising the step of the multiprocessing integrated circuit providing the preprocessed information unit.
163. The method of claim 162,
wherein the information unit carries parts of a model of a neural network.
163. The method of claim 162,
wherein the information unit carries a partial result of at least one database query.
163. The method of claim 162,
wherein the information unit carries partial results of at least one aggregate database query.
receiving, by a memory processing integrated circuit, the database query including at least one relevance criterion indicative of a database entry in a database that is relevant to the database query, wherein the memory processing integrated circuit comprises a controller, multiple processor subunits, and multiple memories includes units;
determining, by the memory processing integrated circuit, a group of relevant database entries stored in the memory processing integrated circuit based on the at least one relevance criterion; and
transferring the group of relevant database entries to the one or more processing entities to continue processing without substantially transferring the extraneous database entries stored in the memory processing integrated circuit to the one or more processing entities; and a database entry is different from the associated database entry.
185. The method of claim 184,
wherein the one or more processing entities are included in the multi-processor subunit of the memory processing integrated circuit.
185. The method of claim 185,
and the memory processing integrated circuit further processing the group of relevant database entries to complete the response to the database query.
187. The method of claim 186,
outputting the response to the database query from the memory processing integrated circuit.
187. The method of claim 187,
wherein said outputting comprises applying a flow control process.
190. The method of claim 188,
and applying the flow control process corresponds to an indicator output from the one or more processing entities regarding completion of processing of one or more database entries of the group.
185. The method of claim 185,
and the memory processing integrated circuit further processing the group of relevant database entries to provide an intermediate response to the database query.
190. The method of claim 190,
outputting the intermediate response to the database query from the memory processing integrated circuit.
192. The method of claim 191,
wherein said outputting comprises applying a flow control process.
193. The method of claim 192,
and applying the flow control process corresponds to an indicator output from the one or more processing entities regarding completion of partial processing of a database entry of the group.
185. The method of claim 185,
and generating, by the one or more processing entities, a processing status indicator indicating progress of further processing of the group's relevant database entries.
185. The method of claim 185,
further processing the group of relevant database entries utilizing the memory processing integrated circuit.
195. The method of claim 195,
The method of claim 1, wherein said processing is executed by said multi-processor subunit.
195. The method of claim 195,
The processing includes: one processing subunit of the multiple processing subunit calculating an intermediate result, sending the intermediate result to another processing subunit of the multiple processing subunit, and the other processing subunit adding A method comprising the step of performing a calculation.
195. The method of claim 195,
and the processing step is executed by the controller.
195. The method of claim 195,
and the processing step is executed by the multi-processor subunit and the controller.
185. The method of claim 184,
and the one or more processing entities are located external to the memory processing integrated circuit.
200. The method of claim 200,
and outputting the relevant database entry of the group from the memory processing integrated circuit.
201. The method of claim 201,
wherein said outputting comprises applying a flow control process.
202. The method of claim 202,
and applying the flow control process corresponds to an indicator output from the one or more processing entities and relating to the relevance of a database entry associated with the one or more processing entities.
185. The method of claim 184,
wherein the multiprocessor subunit comprises full arithmetic logic units.
185. The method of claim 184,
wherein the multiprocessor subunit comprises partial arithmetic logic units.
185. The method of claim 184,
wherein the multi-processor subunit comprises a memory controller.
185. The method of claim 184,
wherein the multiprocessor subunit comprises a partial memory controller.
185. The method of claim 184,
and outputting at least one of (i) a relevant database entry in the group, (ii) a response to the database query, and (iii) an intermediate response to the database query.
212. The method of claim 212,
wherein said outputting comprises applying traffic shaping.
212. The method of claim 212,
wherein the outputting includes attempting to match, via a link coupling the memory processing integrated circuit to a requester unit, the bandwidth used during the outputting to a maximum allowed bandwidth. Way.
212. The method of claim 212,
wherein outputting the output comprises maintaining a variation in output traffic rate below a threshold value.
185. The method of claim 184,
wherein the one or more processing entities include multiple processing entities, wherein at least one processing entity of the multiple processing entities belongs to the memory processing integrated circuit and at least another processing entity of the multiple processing entities is to the memory processing integrated circuit. A method characterized in that it does not belong.
185. The method of claim 184,
wherein the one or more processing entities belong to other memory processing integrated circuits.
receiving, by a multi-memory processing integrated circuit, the database query comprising at least one relevance criterion indicative of a database entry in a database that relates to the database query, wherein the multi-memory processing integrated circuit comprises: a controller, a multi-processor subunit; and multiple memory units;
determining, by each of the multiple memory processing integrated circuits, a group of relevant database entries stored in the memory processing integrated circuit based on the at least one relevance criterion; and
each of the multiple memory processing integrated circuits to continue processing without substantially transferring the unrelated database entries stored in the memory processing integrated circuit to one or more processing entities, the group of relevant database entries stored in the memory processing integrated circuit. to the one or more processing entities, wherein the unrelated database entry is different from the relevant database entry.
receiving, by an integrated circuit, the database query comprising at least one relevance criterion indicative of a database entry in a database that is related to the database query, wherein the integrated circuit includes a controller, a filtering unit, and multiple memory units;
determining, by the filtering unit, a group of relevant database entries stored in the integrated circuit based on the at least one relevance criterion; and
transferring the group's relevant database entries to the one or more processing entities located external to the integrated circuit to continue processing without substantially transferring the extraneous data entries stored in the integrated circuit to the one or more processing entities. A method for accelerating database analysis.
receiving, by an integrated circuit, the database query comprising at least one relevance criterion indicative of a database entry in a database that relates to the database query, wherein the integrated circuit includes a controller, a processing unit, and multiple memory units;
determining, by the processing unit, a group of relevant database entries stored in the integrated circuit based on the at least one relevance criterion;
processing, by the processing unit, the group of relevant database entries without the integrated circuit processing the extraneous data entries stored in the integrated circuit, and providing a processing result, wherein the extraneous database entries are different from the existing database entry; and
and outputting the processing result from an integrated circuit.
receiving, by a memory processing integrated circuit, retrieval information for retrieval of multiple requested feature vectors mapped to multiple sentence segments, wherein the memory processing integrated circuit comprises a controller, multiple processor subunits, and multiple memory units; each of the memory units coupled to the processor subunit;
retrieving the multiple requested feature vectors from at least some memory units of the multiple memory units, wherein the retrieving comprises simultaneously requesting the requested feature vectors stored in the two or more memory units from two or more memory units. included; and
outputting from the memory processing integrated circuit an output comprising at least one of (a) the requested feature vector and (b) a result of processing of the requested feature vector; .
218. The method of claim 217,
and the output includes the requested feature vector.
218. The method of claim 217,
and the output includes the result of the processing of the requested feature vector.
219. The method of claim 219,
and the processing is executed by the multi-processor subunit.
223. The method of claim 220,
wherein the processing comprises transmitting the requested feature vector from one processing subunit to another processing subunit.
223. The method of claim 220,
wherein said processing comprises calculating an intermediate result by one processing subunit, sending said intermediate result to another processing subunit, and calculating another intermediate result or processing result by said other processing subunit.
219. The method of claim 219,
and the processing is performed by the controller.
219. The method of claim 219,
and the processing is executed by the multi-processor subunit and the controller.
219. The method of claim 219,
wherein the processing is performed by a vector processor of the memory processing integrated circuit.
218. The method of claim 217,
and the controller is configured to simultaneously request the requested feature vector based on a known mapping between a sentence segment and the position of the feature vector mapped to the sentence segment.
12. The method of claim 11,
wherein the mapping is uploaded during a boot process of the memory processing integrated circuit.
218. The method of claim 217,
and the controller is configured to manage the retrieval of the multiple requested feature vectors.
218. The method of claim 217,
wherein the multiple sentence segments are in a specific order, and the output of the requested feature vector is performed according to the specific order.
229. The method of claim 229,
and the searching of the multiple requested feature vectors is performed according to the specific order.
229. The method of claim 229,
and wherein the searching of the multiple requested feature vectors is performed at least in part out of order, and wherein the searching further comprises rearranging the order of the multiple requested feature vectors.
218. The method of claim 217,
and the retrieval of the multiple requested feature vector comprises buffering the multiple requested feature vector before the multiple requested feature vector is read by the controller.
232. The method of claim 232,
and the retrieval of the multiple requested feature vectors comprises generating a buffer status indicator indicating when one or more buffers associated with the multiple memory units store one or more requested feature vectors.
234. The method of claim 233,
and passing the buffer status indicator over a dedicated control line.
234. The method of claim 234,
The method of claim 1, wherein one dedicated control line is allocated for each memory unit.
234. The method of claim 234,
wherein the buffer status indicator includes one or more status bits stored in one or more of the buffers.
234. The method of claim 234,
and passing the buffer status indicator over one or more shared control lines.
218. The method of claim 217,
wherein the search information is included in one or more search commands of a first resolution representing a specified number of bits.
238. The method of claim 238,
and managing the search via the controller at a higher resolution representing a smaller number of bits than the specified number of bits.
238. The method of claim 238,
and the controller is configured to manage the search on feature vector resolution.
238. The method of claim 238,
The method further comprising the step of the controller independently managing the search.
218. The method of claim 217,
wherein the multiprocessor subunit comprises full arithmetic logic units.
218. The method of claim 217,
wherein the multiprocessor subunit comprises partial arithmetic logic units.
218. The method of claim 217,
wherein the multi-processor subunit comprises a memory controller.
218. The method of claim 217,
wherein the multiprocessor subunit comprises a partial memory controller.
218. The method of claim 217,
wherein outputting the output comprises applying traffic shaping to the output.
217. The method of claim 217,
wherein outputting the output includes matching, via a link coupling the memory processing integrated circuit to a requester unit, the bandwidth used during the outputting to a maximum allowed bandwidth. Way.
218. The method of claim 217,
wherein outputting the output comprises maintaining a variation in output traffic rate below a threshold value.
218. The method of claim 217,
wherein said retrieving comprises applying a predictive search of at least some requested feature vectors of said requested feature vectors from a set of requested feature vectors stored in a single memory unit.
217. The method of claim 217,
and the requested feature vector is distributed among the memory units.
218. The method of claim 217,
The method of claim 1, wherein the requested feature vector is distributed among the memory units based on an expected search pattern.
performing processing operations by multiple processors included in a hybrid device comprising a base die, a first memory resource associated with at least one second die, and a second memory resource associated with at least one third die, wherein the base die and the at least one second die connected to each other by a wafer to wafer bond;
retrieving information stored in the first memory resource by utilizing the multiple processors; and
transmitting additional information from the second memory resource to the first memory resource, wherein an overall bandwidth of a first path between the base die and the at least one second die is between the at least one second die and the at least one greater than an overall bandwidth of a second path between the third dies of , and a storage capacity of the first memory resource is less than a storage capacity of the second memory resource. 252. The method of claim 252,
and the second memory resource comprises a high bandwidth memory (HBM) resource.
252. The method of claim 252,
and said at least one third die comprises a stack of high bandwidth memory (HBM) chips.
252. The method of claim 252,
and at least a portion of the second memory resource belongs to a third die of at least one third die coupled to the base die without wafer-to-wafer bonding.
252. The method of claim 252,
and at least a portion of the second memory resource belongs to a third die of at least one third die coupled to a second die of the at least one second die without wafer-to-wafer bonding.
252. The method of claim 252,
The method of claim 1, wherein the first memory resource and the second memory resource include different levels of cache memory.
252. The method of claim 252,
wherein the first memory resource is located between the base die and the second memory resource.
252. The method of claim 252,
wherein the first memory resource is not located above the second memory resource.
252. The method of claim 252,
The method further comprising: a second die of at least one second die comprising a plurality of processor subunits and the first memory resource performing additional processing.
260. The method of claim 260,
wherein at least one processor subunit is coupled to a dedicated portion of the first memory resource allocated to the processor subunit.
261. The method of claim 261,
and the dedicated portion of the first memory resource comprises at least one memory bank.
252. The method of claim 252,
wherein the multiple processors belong to a memory processing chip that also includes the first memory resource.
252. The method of claim 252,
wherein the base die includes the multiple processors, the multiple processors including a plurality of processor subunits coupled to the first memory resource via conductors formed of wafer-to-wafer junctions.
265. The method of claim 264,
wherein each processor subunit is coupled to a dedicated portion of the first memory resource allocated to the processor subunit.
base die;
multiple processors;
a first memory resource of at least one second die; and
a second memory resource of at least one third die;
the base die and the at least one second die are connected to each other by wafer-to-wafer bonding;
the multiple processors are configured to perform processing operations and retrieve information stored in the first memory resource;
the second memory resource is configured to transmit additional information from the second memory resource to the first memory resource;
an overall bandwidth of a first path between the base die and the at least one second die is greater than an overall bandwidth of a second path between the at least one second die and the at least one third die;
The hybrid device for memory-intensive processing, characterized in that the storage capacity of the first memory resource is smaller than the storage capacity of the second memory resource.
267. The method of claim 266,
and the second memory resource comprises a high bandwidth memory (HBM) resource.
267. The method of claim 266,
and the at least one third die is a stack of HBM memory chips.
267. The method of claim 266,
and at least a portion of the second memory resource belongs to a third die of the at least one third die coupled to the base die without wafer-to-wafer bonding.
267. The method of claim 266,
and at least a portion of the second memory resource belongs to a third die of the at least one third die coupled to a second die of the at least one second die without wafer-to-wafer bonding.
267. The method of claim 266,
The hybrid device, characterized in that the first memory resource and the second memory resource includes cache memories of different levels.
267. The method of claim 266,
The hybrid device of claim 1, wherein the first memory resource is located between the base die and the second memory resource.
267. The method of claim 266,
The first memory resource is a hybrid device, characterized in that located next to the second memory resource.
267. The method of claim 266,
and a second die of the at least one second die is configured to perform additional processing, the second die comprising a plurality of processor subunits and the first memory resource.
274. The method of claim 274,
wherein each processor subunit is coupled to a dedicated portion of the first memory resource allocated to the processor subunit.
275. The method of claim 275,
and the dedicated portion of the first memory resource comprises at least one memory bank.
267. The method of claim 266,
wherein the multiple processors include a plurality of processor subunits of a memory processing chip that also include the first memory resource.
267. The method of claim 266,
wherein the base die includes the multiple processors, the multiple processors including a plurality of processor subunits coupled to the first memory resource via conductors formed of wafer-to-wafer junctions.
287. The method of claim 278,
wherein each processor subunit is coupled to a dedicated portion of the first memory resource allocated to the processor subunit.
retrieving, by the network communication interface of the database acceleration integrated circuit, a quantity of information from the storage unit;
providing primary processed information by primary processing the amount of information;
utilizing a memory controller of the database acceleration integrated circuit and sending, via an interface, the primary processed information to multiple memory processing integrated circuits, each memory processing integrated circuit comprising: a controller, multiple processor subunits, and multiple memories includes units;
providing secondary processed information by secondary processing at least a portion of the primary processed information by utilizing the multi-memory processing integrated circuit;
retrieving, by the memory controller of the database acceleration integrated circuit, retrieved information from the multiple memory processing integrated circuit, wherein the retrieved information comprises (a) at least a portion of the primary processed information and (b) the secondary processing comprising at least one of at least a portion of the information;
providing a database acceleration result by performing a database processing operation on the retrieved information by utilizing a database acceleration unit of the database acceleration integrated circuit; and
and outputting the database acceleration result.
280. The method of claim 280,
and utilizing a management unit of the database acceleration integrated circuit to manage at least one of the retrieving, the primary processing, and the processing.
282. The method of claim 281,
and the managing is executed according to an execution plan generated by the management unit of the database acceleration integrated circuit.
282. The method of claim 281,
The method according to claim 1, wherein said managing is executed according to an execution plan received without being generated by said management unit of said database acceleration integrated circuit.
282. The method of claim 281,
The managing comprises at least one of (a) a network communication network interface resource, (b) a decompression unit resource, (c) a memory controller resource, (d) a multiple memory processing integrated circuit resource, and (e) a data acceleration unit resource. A method comprising the step of allocating
280. The method of claim 280,
wherein the network communication interface comprises at least two different types of network communication ports.
285. The method of claim 285,
wherein the at least two different types of network communication ports include a storage interface protocol port and a storage interface protocol port over a general network.
285. The method of claim 285,
wherein the at least two different types of network communication ports include a storage interface protocol port and a storage interface protocol port over Ethernet.
285. The method of claim 285,
wherein the at least two different types of network communication ports include a storage interface protocol port and a PCIe port.
280. The method of claim 280,
A method, comprising: a management unit comprising a compute node of a compute system and controlled by a manager of the compute system.
280. The method of claim 280,
The method further comprising the step of controlling, by a compute node of a compute system, at least one of the searching, the primary processing, the transmitting, and the tertiary processing.
280. The method of claim 280,
and executing multiple tasks concurrently utilizing the database acceleration integrated circuit.
280. The method of claim 280,
Using a management unit located outside the database acceleration integrated circuit, further comprising the step of managing at least one of the searching step, the primary processing step, the transmitting step, and the tertiary processing step Way.
280. The method of claim 280,
The method of claim 1, wherein the database acceleration integrated circuit belongs to a compute system.
280. The method of claim 280,
wherein said database acceleration integrated circuit does not belong to a compute system.
280. The method of claim 280,
At least one of the retrieving, the primary processing, the transmitting, and the tertiary processing based on an execution plan transmitted to the database acceleration integrated circuit by a compute node of a compute system. A method further comprising the step of executing.
280. The method of claim 280,
The performing the database processing operation includes the database processing subunit executing the database processing instruction at the same time, the database acceleration unit comprising a group of database accelerator subunits sharing a shared memory unit. How to.
297. The method of claim 296,
wherein each database subunit is configured to execute a specific type of database processing instruction.
297. The method of claim 297,
The method further comprising dynamically coupling database processing subunits to provide an execution pipeline necessary to execute a database processing operation comprising multiple instructions.
280. The method of claim 280,
wherein performing the database processing operation comprises allocating a resource of the database acceleration integrated circuit according to a time I/O bandwidth.
280. The method of claim 280,
The method further comprising outputting the database acceleration result to a local storage and retrieving the database acceleration result from the local storage.
280. The method of claim 280,
wherein the network communication interface comprises an RDMA unit.
280. The method of claim 280,
The method further comprising exchanging information between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits.
280. The method of claim 280,
The method further comprising exchanging database acceleration results between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits.
280. The method of claim 280,
The method further comprising exchanging at least one of (a) information and (b) database acceleration results between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits.
305. The method of claim 304,
A method, characterized in that a group of database acceleration integrated circuits are connected to a common printed circuit board.
305. The method of claim 304,
A method, characterized in that a group of database acceleration integrated circuits belong to a modular unit of a computing system.
305. The method of claim 304,
wherein different groups of database acceleration integrated circuits are connected to different printed circuit boards.
305. The method of claim 304,
A method, characterized in that different groups of database acceleration integrated circuits belong to a modular unit of a computing system.
305. The method of claim 304,
and the one or more groups of database acceleration integrated circuits performing distributed processing.
305. The method of claim 304,
and the exchanging step is performed utilizing a network communication interface of at least one group of said database acceleration integrated circuits.
305. The method of claim 304,
wherein the step of exchanging is carried out over multiple groups connected to each other by a star-connection.
305. The method of claim 304,
and using at least one switch to exchange at least one of (a) information and (b) database acceleration results between the one or more groups of different groups of database acceleration integrated circuits.
305. The method of claim 304,
and at least some of the one or more groups of database acceleration integrated circuits performing distributed processing.
305. The method of claim 304,
and performing distributed processing of the first data structure and the second data structure, wherein a total size of the first data structure and the second data structure is greater than a storage capacity of the multiple memory processing integrated circuit. How to.
314. The method of claim 314,
The step of performing the distributed processing includes (a) newly allocating different pairs of the first data structure part and the second data structure part to different database acceleration integrated circuits and (b) repeating the different pairs of processing several times. A method comprising a.
315. The method of claim 315,
The method of claim 1, wherein performing the distributed processing comprises a database join operation.
315. The method of claim 315,
The step of performing the distributed processing includes:
allocating different portions of the first data structure to the one or more groups of different database acceleration integrated circuits; and
new allocating different second data structure portions to the one or more groups of different database acceleration integrated circuits and processing the first data structure portions and the second data structure portions by the database acceleration integrated circuit multiple times A method comprising the steps of performing repeatedly.
317. The method of claim 317,
wherein said step of allocating anew of a next iteration is performed in a manner that at least partially overlaps in time with the processing of the current iteration.
317. The method of claim 317,
wherein said new allocating comprises exchanging portions of a second data structure between said different database acceleration integrated circuits.
319. The method of claim 319,
wherein said exchanging step is performed in a manner that overlaps at least partially in time with said processing step.
317. The method of claim 317,
The new allocating step may include exchanging a second data structure portion between a group of said different database acceleration integrated circuits and, upon completion of the exchanging step, exchanging a second data structure portion between a different group of database acceleration integrated circuits. A method comprising the step of:
280. The method of claim 280,
wherein the database acceleration integrated circuit is included in a blade comprising multiple database acceleration integrated circuits, one or more non-volatile memory units, an Ethernet switch, a PCIe switch and an Ethernet switch, and the multiple memory processing integrated circuits. .
database acceleration integrated circuits; and
each memory processing integrated circuit comprising a controller, multiple processor subunits, and multiple memory processing integrated circuits comprising multiple memory units;
the network communication interface of the database acceleration integrated circuit is configured to receive information from a storage unit;
the database acceleration integrated circuit is configured to primary process the amount of information to provide primary processed information;
the memory controller of the database acceleration integrated circuit is configured to transmit the primary processed information to the multi-memory processing integrated circuit through an interface;
wherein the multiple memory processing integrated circuit is configured to secondary process at least portions of the primary processed information to provide secondary processed information;
The memory controller of the database acceleration integrated circuit is configured to retrieve information from the multiple memory processing integrated circuit, wherein the retrieved information comprises (a) at least a portion of the primary processed information and (b) the secondary processed information. at least one of at least a portion of the information;
the database acceleration unit of the database acceleration integrated circuit is configured to perform a database process operation on the retrieved information to provide a database acceleration result;
and the database acceleration integrated circuit is configured to output the database acceleration result.
323. The method of claim 323,
an apparatus configured to utilize a management unit of the database acceleration integrated circuit to manage at least one of the retrieval, the primary processing, and the secondary processing of the retrieved information.
325. The method of claim 324,
and the management unit is configured to manage according to an execution plan generated by the management unit of the database acceleration integrated circuit.
325. The method of claim 324,
and the management unit is configured to manage according to an execution plan received without being generated by the management unit of the database acceleration integrated circuit.
325. The method of claim 324,
The management unit is configured to manage one or more of (a) a network communication network interface resource, (b) a decompression unit resource, (c) a memory controller resource, (d) a multiple memory processing integrated circuit resource, and (e) a data acceleration unit resource. A device configured to allocate and manage.
323. The method of claim 323,
wherein the network communication interface comprises different types of network communication ports.
339. The method of claim 328,
wherein the different types of network communication ports include a storage interface protocol port and a storage interface protocol port over a general network.
339. The method of claim 328,
wherein the different types of network communication ports include a storage interface protocol port and a storage interface protocol port over Ethernet.
339. The method of claim 328,
wherein the different types of network communication ports include a storage interface protocol port and a PCIe port.
323. The method of claim 323,
wherein the device is coupled to a management unit comprising a compute node of a compute system and controlled by a manager of the compute system.
323. The method of claim 323,
A device configured to be controlled by a compute node of a compute system.
323. The method of claim 323,
an apparatus configured to execute multiple tasks concurrently by the database acceleration integrated circuit.
323. The method of claim 323,
wherein the database acceleration integrated circuit belongs to a computing system. 323. The method of claim 323,
wherein the database acceleration integrated circuit does not belong to a computing system.
323. The method of claim 323,
an apparatus configured to execute at least one of retrieval, primary processing, forwarding, and tertiary processing based on an execution plan transmitted by a compute node of a computer system to the database acceleration integrated circuit.
323. The method of claim 323,
The database acceleration unit is configured to simultaneously perform database process instructions by the database processing subunit, and the database acceleration unit includes a group of database accelerator subunits sharing a shared memory unit.
339. The method of claim 338,
wherein each database processing subunit is configured to execute a specific type of database process instruction. 329. The method of claim 329,
wherein the apparatus is configured to dynamically couple database processing subunits to provide an execution pipeline necessary to execute a database process operation comprising multiple instructions. 323. The method of claim 323,
and the apparatus is configured to allocate resources of the database acceleration integrated circuit according to a time I/O bandwidth. 323. The method of claim 323,
wherein the device comprises local storage accessible by the database acceleration integrated circuit. 323. The method of claim 323,
wherein the network communication network interface comprises an RDMA unit.
323. The method of claim 323,
wherein the apparatus comprises one or more groups of database acceleration integrated circuits configured to exchange information between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits. 323. The method of claim 323,
wherein the apparatus comprises one or more groups of database acceleration integrated circuits configured to exchange acceleration results between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits. 323. The method of claim 323,
wherein the apparatus comprises one or more groups of database acceleration integrated circuits configured to execute at least one of (a) information and (b) a database acceleration result between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits. device characterized. 346. The method of claim 346,
A device, characterized in that a group of database acceleration integrated circuits are connected to the same printed circuit board. 346. The method of claim 346,
A device, characterized in that a group of database acceleration integrated circuits belong to a modular unit of a computing system. 346. The method of claim 346,
wherein different groups of database acceleration integrated circuits are connected to different printed circuit boards. 346. The method of claim 346,
A device according to claim 1, wherein different groups of database acceleration integrated circuits belong to a modular unit of a computing system. 346. The method of claim 346,
wherein said exchange is effected utilizing a network communication interface of said database acceleration integrated circuit of one or more groups. 346. The method of claim 346,
The device of claim 1, wherein the exchange is carried out over multiple groups connected to each other by a star-connection. 346. The method of claim 346,
and the apparatus is configured to use the at least one switch to exchange at least one of (a) information and (b) database acceleration results between different groups of database acceleration integrated circuits in the one or more groups. 346. The method of claim 346,
and the apparatus is configured to execute a distributed process by a portion of the database acceleration integrated circuits in a subset of the one or more groups. 346. The method of claim 346,
and perform distributed processing by utilizing the first data structure and the second data structure, wherein a total size of the first data structure and the second data structure is greater than a storage capacity of the multiple memory processing integrated circuit. device to do. 355. The method of claim 355,
The apparatus is configured to perform the distributed processing by (a) performing new assignment of different pairs of first data structure portions and second data structure portions to different database acceleration integrated circuits and (b) repeating the different pairs of processing multiple times; A device characterized in that it performs. 355. The method of claim 355,
wherein the distribution process comprises a database join operation. 355. The method of claim 355,
The device is:
allocating different portions of the first data structure to the one or more groups of different database acceleration integrated circuits; and
Newly allocating different second data structure portions to the one or more groups of different database acceleration integrated circuits and performing multiple iterations of processing the first data structure portions and the second data structure portions by the database acceleration integrated circuits to perform the distributed processing. 358. The method of claim 358,
and the apparatus is configured to execute the new assignment of the next iteration in a manner that at least partially overlaps in time with the processing of the current iteration. 358. The method of claim 358,
and the apparatus is configured to effect the new allocation by exchanging a second data structure portion between the different database acceleration integrated circuits. 360. The method of claim 360,
and said exchange is performed in a manner that at least partially overlaps in time with said processing by said database acceleration integrated circuit. 358. The method of claim 358,
The device is configured to exchange a second data structure portion between a group of said different database acceleration integrated circuits and by exchanging a second data structure portion between a different group of database acceleration integrated circuits when there are no more second data structure portions to exchange. A device configured to execute a new assignment. 323. The method of claim 323,
wherein the database acceleration integrated circuit is included in a blade comprising multiple database acceleration integrated circuits, one or more non-volatile memory units, an Ethernet switch, a PCIe switch and an Ethernet switch, and the multiple memory processing integrated circuits. . retrieving, by the network communication interface of the database acceleration integrated circuit, information from the storage unit;
providing primary processed information by primary processing the amount of information;
transmitting, by the memory controller of the database acceleration integrated circuit, the primary processed information to multiple memory resources through an interface;
retrieving information from the multiple memory resources;
performing, by the database acceleration unit of the database acceleration integrated circuit, a database processing operation on the retrieved information to provide a database acceleration result; and
and outputting the database acceleration result. 364. The method of claim 364,
secondary processing the primary processed information to provide secondary processed information, wherein the processing of the primary processed information is within one or more memory processing integrated circuits further comprising the multiple memory resources. A method, characterized in that it is executed by multiple processors located there. 364. The method of claim 364,
wherein said primary processing comprises filtering of database entries. 364. The method of claim 364,
wherein said secondary processing comprises filtering of database entries. 364. The method of claim 364,
wherein said primary processing and said secondary processing comprise filtering of database entries.

Images