[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN117194991A - High-dimensional data recommendation system and method based on GPU cluster - Google Patents

High-dimensional data recommendation system and method based on GPU cluster Download PDF

Info

Publication number
CN117194991A
CN117194991A CN202311452396.9A CN202311452396A CN117194991A CN 117194991 A CN117194991 A CN 117194991A CN 202311452396 A CN202311452396 A CN 202311452396A CN 117194991 A CN117194991 A CN 117194991A
Authority
CN
China
Prior art keywords
data
gpu
training
module
cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202311452396.9A
Other languages
Chinese (zh)
Other versions
CN117194991B (en
Inventor
王晓丹
王曦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan Bingji Technology Co ltd
Original Assignee
Sichuan Bingji Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan Bingji Technology Co ltd filed Critical Sichuan Bingji Technology Co ltd
Priority to CN202311452396.9A priority Critical patent/CN117194991B/en
Publication of CN117194991A publication Critical patent/CN117194991A/en
Application granted granted Critical
Publication of CN117194991B publication Critical patent/CN117194991B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the field of data processing, in particular to a high-dimensional data recommendation system and method based on a GPU cluster, which are characterized in that training data in a data training task are subjected to data splitting at a client, and a generated original data module sequence corresponding to the training data is sent to the GPU cluster; according to the original data module sequence of the corresponding training data, obtaining data training task characteristics, and generating a computing power container of the corresponding GPU cluster by the GPU cluster according to the data training task characteristics; distributing each data module in the original data module sequence in GPU units contained in a computing power container of the corresponding GPU cluster according to computing power occupation of the original data module sequence of the corresponding training data uploaded by the client; processing according to the data training task, generating data after data training, and generating recommended data according to the data after data training. By the technical scheme provided by the invention, the data processing efficiency can be improved.

Description

High-dimensional data recommendation system and method based on GPU cluster
Technical Field
The invention relates to the field of data processing, in particular to a high-dimensional data recommendation system and method based on a GPU cluster.
Background
With the rapid development of the internet, the demands of users for personalized recommendations are increasing. To meet this demand, high-dimensional data recommendation methods based on GPU clusters have been developed. However, in implementing this approach, many technical challenges need to be addressed. First, the processing of high-dimensional data is a difficult problem. Due to the extremely high data dimensions, conventional data processing methods often fail to process these data efficiently. Therefore, development of a new data processing method is required to accommodate the characteristics of high-dimensional data. Second, the use of GPU clusters also presents some technical challenges. Because of the special architecture of the GPU cluster, the data processing method needs to be optimized for the GPU cluster to ensure the high efficiency and accuracy of data processing. In addition, the choice of recommendation algorithm is also a critical issue. Different recommendation algorithms are suitable for different data types and user requirements. Therefore, the most suitable recommendation algorithm needs to be selected according to the actual situation.
Aiming at the problems, the data processing method based on the GPU cluster high-dimensional data recommendation needs to develop a complete technical scheme, which comprises links of data preprocessing, feature extraction, model training, model evaluation and the like. Meanwhile, optimization is required to be performed on the framework of the GPU cluster so as to improve the data processing efficiency and accuracy. In the data preprocessing stage, a proper data dimension reduction method is needed to convert high-dimension data into low-dimension data so as to reduce the complexity and the calculated amount of the data. At the same time, data cleaning and feature extraction are also required to obtain more accurate and useful data. In the model training stage, a proper recommendation algorithm needs to be selected, and optimization is performed according to the characteristics of the GPU cluster. In addition, model tuning and parameter adjustment are needed to improve the accuracy and generalization capability of the model. In the model evaluation stage, the effect of the recommended model needs to be evaluated by adopting proper evaluation indexes and methods. In summary, the data processing method based on GPU cluster high-dimensional data recommendation needs to solve many technical challenges, including links such as high-dimensional data processing, GPU cluster optimization, recommendation algorithm selection, and model evaluation.
Therefore, how to process data that needs to be recommended by high-dimensional data is a subject that needs to be studied by technicians in the current industry.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a high-dimensional data recommendation method based on a GPU cluster, which comprises the following steps:
step one, data splitting is carried out on training data in a data training task at a client to obtain an original data module sequence corresponding to the training data, and the client sends the generated original data module sequence corresponding to the training data to a GPU cluster;
step two, the GPU cluster obtains an original data module sequence corresponding to the training data, and obtains data training task characteristics according to the original data module sequence corresponding to the training data, and the GPU cluster generates an algorithm force container corresponding to the GPU cluster according to the data training task characteristics;
step three, according to the calculation power occupation of the original data module sequence of the corresponding training data uploaded by the client, if the calculation power occupation of the original data module sequence of the corresponding training data is within the capacity range of the calculation power container of the corresponding GPU cluster, distributing each data module in the original data module sequence in the GPU units contained in the calculation power container of the corresponding GPU cluster, and entering step four; if the computing power occupation of the original data module sequence corresponding to the training data is not in the capacity range of the computing power container corresponding to the GPU cluster, entering a step five;
step four, the GPU unit manager wakes up and sends the distributed data modules to the corresponding GPU units, the GPU units process the distributed data modules according to the data training tasks, after the data processing is completed, data after the data training is generated, the GPU units are released until all GPU units contained in the computing power container of the GPU cluster are completed, and step six is entered;
step five, the GPU unit manager calls GPU units corresponding to the calculation force difference values to the calculation force containers of the GPU clusters in the calculation force containers of the corresponding GPU clusters according to the calculation force difference values, generates corrected calculation force containers of the corresponding GPU clusters, distributes data training tasks to the calculation force containers of the corresponding GPU clusters, carries out data processing on an original data module sequence of the corresponding training data, generates data after the data training is completed, releases the GPU units, and enters step six;
step six, the GPU cluster uploads the original data module sequence corresponding to the training data to the data storage module, meanwhile, the GPU cluster generates basic data state information according to the information of the original data module sequence corresponding to the training data, the serial number is N, and recommended data are generated according to the data after the data training.
Further, the data splitting is performed on the training data in the data training task at the client to obtain an original data module sequence corresponding to the training data, which includes:
splitting training data in a data training task into a plurality of data modules at a client, numbering the data modules in sequence, generating a data module index table according to the numbers, packaging the data module index table and the first-ordered data module to generate a header file, and generating an original data module sequence of corresponding training data by the header file and the rest data modules.
Further, the GPU cluster obtains an original data module sequence corresponding to the training data, obtains data training task features according to the original data module sequence corresponding to the training data, and generates an computing power container corresponding to the GPU cluster according to the data training task features, including: the GPU cluster obtains the number of data modules according to the head files in the obtained original data module sequences corresponding to the training data, and obtains the number of required computing units according to the number of the data modules, wherein the number of the required computing units is the characteristic of the data training task;
obtaining the required number of GPU units according to the required number of the calculation units and the number of the unit calculation units of the GPU units, and generating a power calculation container corresponding to the GPU cluster by calling the required number of the GPU units in the GPU cluster according to the required number of the GPU units.
Further, the GPU cluster uploads the original data module sequence corresponding to the training data to the data storage module, and simultaneously generates basic data state information according to the information of the original data module sequence corresponding to the training data, the sequence number is N, and generates recommended data according to the data after the data training, including:
performing data operation on the original data module in the computing power container corresponding to the GPU cluster, performing state information modification on the basis of the basic data state information of the GPU cluster according to the content of the data operation, generating basic data state information with a sequence number added with one, and updating the basic data state information with the sequence number added with one into the data storage module to form a basic data state information sequence with the basic data state information; when the GPU unit accesses data, the data storage module firstly matches the original data according to the serial number of the basic data state information stored by the GPU cluster, then matches the corresponding basic data state information according to the number of the serial number plus one, and carries out corresponding data operation on the original data according to the content of the data operation in the corresponding basic data state information to generate recommended data.
The high-dimensional data recommendation system based on the GPU cluster is applied to the high-dimensional data recommendation method based on the GPU cluster, and comprises a GPU cluster module, a client, a data storage module, a communication module and a GPU management unit;
the client and the GPU cluster module are respectively in communication connection with the communication module; and the data storage module and the GPU management unit are respectively connected with the GPU cluster module.
The beneficial effects of the invention are as follows: according to the technical scheme provided by the invention, the data to be processed can be segmented and distributed to the specific GPU units, and the data processing efficiency is improved.
Drawings
FIG. 1 is a flow diagram of a high-dimensional data recommendation method based on GPU clusters;
fig. 2 is a schematic diagram of a high-dimensional data recommendation system based on GPU clusters.
Detailed Description
The technical solution of the present invention will be described in further detail with reference to the accompanying drawings, but the scope of the present invention is not limited to the following description.
For the purpose of making the technical solution and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the particular embodiments described herein are illustrative only and are not intended to limit the invention, i.e., the embodiments described are merely some, but not all, of the embodiments of the invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present invention. It is noted that relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The features and capabilities of the present invention are described in further detail below in connection with the examples.
As shown in fig. 1, the high-dimensional data recommendation method based on the GPU cluster includes the following steps:
step one, data splitting is carried out on training data in a data training task at a client to obtain an original data module sequence corresponding to the training data, and the client sends the generated original data module sequence corresponding to the training data to a GPU cluster;
step two, the GPU cluster obtains an original data module sequence corresponding to the training data, and obtains data training task characteristics according to the original data module sequence corresponding to the training data, and the GPU cluster generates an algorithm force container corresponding to the GPU cluster according to the data training task characteristics;
step three, according to the calculation power occupation of the original data module sequence of the corresponding training data uploaded by the client, if the calculation power occupation of the original data module sequence of the corresponding training data is within the capacity range of the calculation power container of the corresponding GPU cluster, distributing each data module in the original data module sequence in the GPU units contained in the calculation power container of the corresponding GPU cluster, and entering step four; if the computing power occupation of the original data module sequence corresponding to the training data is not in the capacity range of the computing power container corresponding to the GPU cluster, entering a step five;
step four, the GPU unit manager wakes up and sends the distributed data modules to the corresponding GPU units, the GPU units process the distributed data modules according to the data training tasks, after the data processing is completed, data after the data training is generated, the GPU units are released until all GPU units contained in the computing power container of the GPU cluster are completed, and step six is entered;
step five, the GPU unit manager calls GPU units corresponding to the calculation force difference values to the calculation force containers of the GPU clusters in the calculation force containers of the corresponding GPU clusters according to the calculation force difference values, generates corrected calculation force containers of the corresponding GPU clusters, distributes data training tasks to the calculation force containers of the corresponding GPU clusters, carries out data processing on an original data module sequence of the corresponding training data, generates data after the data training is completed, releases the GPU units, and enters step six;
step six, the GPU cluster uploads the original data module sequence corresponding to the training data to the data storage module, meanwhile, the GPU cluster generates basic data state information according to the information of the original data module sequence corresponding to the training data, the serial number is N, and recommended data are generated according to the data after the data training.
The data splitting is performed on training data in a data training task at the client to obtain an original data module sequence corresponding to the training data, and the data splitting comprises the following steps:
splitting training data in a data training task into a plurality of data modules at a client, numbering the data modules in sequence, generating a data module index table according to the numbers, packaging the data module index table and the first-ordered data module to generate a header file, and generating an original data module sequence of corresponding training data by the header file and the rest data modules.
The GPU cluster obtains an original data module sequence corresponding to training data, obtains data training task characteristics according to the original data module sequence corresponding to the training data, and generates an algorithm force container corresponding to the GPU cluster according to the data training task characteristics, and comprises the following steps: the GPU cluster obtains the number of data modules according to the head files in the obtained original data module sequences corresponding to the training data, and obtains the number of required computing units according to the number of the data modules, wherein the number of the required computing units is the characteristic of the data training task;
obtaining the required number of GPU units according to the required number of the calculation units and the number of the unit calculation units of the GPU units, and generating a power calculation container corresponding to the GPU cluster by calling the required number of the GPU units in the GPU cluster according to the required number of the GPU units.
The GPU cluster uploads an original data module sequence corresponding to training data to a data storage module, generates basic data state information according to information of the original data module sequence corresponding to the training data, has a sequence number of N, generates recommended data according to data after data training, and comprises the following steps:
performing data operation on the original data module in the computing power container corresponding to the GPU cluster, performing state information modification on the basis of the basic data state information of the GPU cluster according to the content of the data operation, generating basic data state information with a sequence number added with one, and updating the basic data state information with the sequence number added with one into the data storage module to form a basic data state information sequence with the basic data state information; when the GPU unit accesses data, the data storage module firstly matches the original data according to the serial number of the basic data state information stored by the GPU cluster, then matches the corresponding basic data state information according to the number of the serial number plus one, and carries out corresponding data operation on the original data according to the content of the data operation in the corresponding basic data state information to generate recommended data.
As shown in fig. 2, the GPU cluster-based high-dimensional data recommendation system applies the GPU cluster-based high-dimensional data recommendation method, and the GPU cluster-based high-dimensional data recommendation system comprises a GPU cluster module, a client, a data storage module, a communication module and a GPU management unit;
the client and the GPU cluster module are respectively in communication connection with the communication module; and the data storage module and the GPU management unit are respectively connected with the GPU cluster module.
The GPU cluster module is used for generating an algorithm force container corresponding to the GPU cluster according to the data training task characteristics;
the client splits the training data in the data training task to obtain an original data module sequence corresponding to the training data, and sends the generated original data module sequence corresponding to the training data to the GPU cluster.
The GPU management unit is used for distributing the data modules and dispatching the GPU units, and dispatching the GPU units to the generated computing power containers corresponding to the GPU clusters.
The data storage module is used for storing data generated after the computing power container corresponding to the GPU cluster performs data operation on the original data module.
The GPU cluster module further comprises a GPU cluster node and a calculation container generation module; the GPU cluster node comprises a plurality of GPU units, and the power calculation container generation module is used for generating power calculation containers corresponding to the GPU clusters according to the required power calculation.
The foregoing is merely a preferred embodiment of the invention, and it is to be understood that the invention is not limited to the form disclosed herein but is not to be construed as excluding other embodiments, but is capable of numerous other combinations, modifications and environments and is capable of modifications within the scope of the inventive concept, either as taught or as a matter of routine skill or knowledge in the relevant art. And that modifications and variations which do not depart from the spirit and scope of the invention are intended to be within the scope of the appended claims.

Claims (5)

1. The high-dimensional data recommendation method based on the GPU cluster is characterized by comprising the following steps of:
step one, data splitting is carried out on training data in a data training task at a client to obtain an original data module sequence corresponding to the training data, and the client sends the generated original data module sequence corresponding to the training data to a GPU cluster;
step two, the GPU cluster obtains an original data module sequence corresponding to the training data, and obtains data training task characteristics according to the original data module sequence corresponding to the training data, and the GPU cluster generates an algorithm force container corresponding to the GPU cluster according to the data training task characteristics;
step three, according to the calculation power occupation of the original data module sequence of the corresponding training data uploaded by the client, if the calculation power occupation of the original data module sequence of the corresponding training data is within the capacity range of the calculation power container of the corresponding GPU cluster, distributing each data module in the original data module sequence in the GPU units contained in the calculation power container of the corresponding GPU cluster, and entering step four; if the computing power occupation of the original data module sequence corresponding to the training data is not in the capacity range of the computing power container corresponding to the GPU cluster, entering a step five;
step four, the GPU unit manager wakes up and sends the distributed data modules to the corresponding GPU units, the GPU units process the distributed data modules according to the data training tasks, after the data processing is completed, data after the data training is generated, the GPU units are released until all GPU units contained in the computing power container of the GPU cluster are completed, and step six is entered;
step five, the GPU unit manager calls GPU units corresponding to the calculation force difference values to the calculation force containers of the GPU clusters in the calculation force containers of the corresponding GPU clusters according to the calculation force difference values, generates corrected calculation force containers of the corresponding GPU clusters, distributes data training tasks to the calculation force containers of the corresponding GPU clusters, carries out data processing on an original data module sequence of the corresponding training data, generates data after the data training is completed, releases the GPU units, and enters step six;
step six, the GPU cluster uploads the original data module sequence corresponding to the training data to the data storage module, meanwhile, the GPU cluster generates basic data state information according to the information of the original data module sequence corresponding to the training data, the serial number is N, and recommended data are generated according to the data after the data training.
2. The GPU cluster-based high-dimensional data recommendation method of claim 1, wherein the splitting of the training data in the data training task at the client to obtain the original data module sequence corresponding to the training data comprises:
splitting training data in a data training task into a plurality of data modules at a client, numbering the data modules in sequence, generating a data module index table according to the numbers, packaging the data module index table and the first-ordered data module to generate a header file, and generating an original data module sequence of corresponding training data by the header file and the rest data modules.
3. The high-dimensional data recommendation method based on the GPU cluster according to claim 2, wherein the GPU cluster obtains an original data module sequence corresponding to training data, obtains data training task features according to the original data module sequence corresponding to the training data, and generates an algorithm container corresponding to the GPU cluster according to the data training task features, and the method comprises the steps of: the GPU cluster obtains the number of data modules according to the head files in the obtained original data module sequences corresponding to the training data, and obtains the number of required computing units according to the number of the data modules, wherein the number of the required computing units is the characteristic of the data training task;
obtaining the required number of GPU units according to the required number of the calculation units and the number of the unit calculation units of the GPU units, and generating a power calculation container corresponding to the GPU cluster by calling the required number of the GPU units in the GPU cluster according to the required number of the GPU units.
4. The method for recommending high-dimensional data based on GPU clusters according to claim 1, wherein the GPU clusters upload the original data module sequence corresponding to the training data to the data storage module, and the GPU clusters generate the basic data status information according to the information of the original data module sequence corresponding to the training data, with the sequence number of N, and generate recommended data according to the data after the data training, comprising:
performing data operation on the original data module in the computing power container corresponding to the GPU cluster, performing state information modification on the basis of the basic data state information of the GPU cluster according to the content of the data operation, generating basic data state information with a sequence number added with one, and updating the basic data state information with the sequence number added with one into the data storage module to form a basic data state information sequence with the basic data state information; when the GPU unit accesses data, the data storage module firstly matches the original data according to the serial number of the basic data state information stored by the GPU cluster, then matches the corresponding basic data state information according to the number of the serial number plus one, and carries out corresponding data operation on the original data according to the content of the data operation in the corresponding basic data state information to generate recommended data.
5. The high-dimensional data recommendation system based on the GPU cluster is characterized by comprising a GPU cluster module, a client, a data storage module, a communication module and a GPU management unit, wherein the high-dimensional data recommendation method based on the GPU cluster is applied to any one of claims 1-4;
the client and the GPU cluster module are respectively in communication connection with the communication module; and the data storage module and the GPU management unit are respectively connected with the GPU cluster module.
CN202311452396.9A 2023-11-03 2023-11-03 High-dimensional data recommendation system and method based on GPU cluster Active CN117194991B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311452396.9A CN117194991B (en) 2023-11-03 2023-11-03 High-dimensional data recommendation system and method based on GPU cluster

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311452396.9A CN117194991B (en) 2023-11-03 2023-11-03 High-dimensional data recommendation system and method based on GPU cluster

Publications (2)

Publication Number Publication Date
CN117194991A true CN117194991A (en) 2023-12-08
CN117194991B CN117194991B (en) 2024-02-13

Family

ID=88994524

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311452396.9A Active CN117194991B (en) 2023-11-03 2023-11-03 High-dimensional data recommendation system and method based on GPU cluster

Country Status (1)

Country Link
CN (1) CN117194991B (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110209896A (en) * 2019-06-06 2019-09-06 江苏户传科技有限公司 A kind of product quality tracing system based on artificial intelligence big data platform
CN112241321A (en) * 2020-09-24 2021-01-19 北京影谱科技股份有限公司 Computing power scheduling method and device based on Kubernetes
CN112668650A (en) * 2020-12-30 2021-04-16 上海电气集团股份有限公司 Industrial data model generation method, system, device and medium
CN112988390A (en) * 2021-03-22 2021-06-18 上海超级计算中心 Calculation power resource allocation method and device
WO2022002068A1 (en) * 2020-06-29 2022-01-06 中兴通讯股份有限公司 Data processing method, system and device and storage medium
CN114145006A (en) * 2020-06-12 2022-03-04 华为技术有限公司 Method, device, storage medium and chip for scheduling artificial intelligence resources
CN114269445A (en) * 2019-08-26 2022-04-01 辉达公司 Content recommendation using one or more neural networks
US20220240408A1 (en) * 2021-01-22 2022-07-28 Nvidia Corporation Static data center power balancing and configuration
CN115543615A (en) * 2022-09-29 2022-12-30 上海商汤科技开发有限公司 Resource allocation method and device, electronic equipment and storage medium
US20230115163A1 (en) * 2021-12-31 2023-04-13 Beijing Baidu Netcom Science Technology Co., Ltd. Method for processing data, and electronic device, storage medium and program product
CN116069500A (en) * 2022-12-20 2023-05-05 中国电信股份有限公司 Model training task processing method and device, electronic equipment and readable medium
CN116450355A (en) * 2023-04-21 2023-07-18 重庆长安汽车股份有限公司 Multi-cluster model training method, device, equipment and medium
CN116954929A (en) * 2023-09-20 2023-10-27 四川并济科技有限公司 Dynamic GPU scheduling method and system for live migration

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110209896A (en) * 2019-06-06 2019-09-06 江苏户传科技有限公司 A kind of product quality tracing system based on artificial intelligence big data platform
CN114269445A (en) * 2019-08-26 2022-04-01 辉达公司 Content recommendation using one or more neural networks
CN114145006A (en) * 2020-06-12 2022-03-04 华为技术有限公司 Method, device, storage medium and chip for scheduling artificial intelligence resources
WO2022002068A1 (en) * 2020-06-29 2022-01-06 中兴通讯股份有限公司 Data processing method, system and device and storage medium
CN112241321A (en) * 2020-09-24 2021-01-19 北京影谱科技股份有限公司 Computing power scheduling method and device based on Kubernetes
CN112668650A (en) * 2020-12-30 2021-04-16 上海电气集团股份有限公司 Industrial data model generation method, system, device and medium
US20220240408A1 (en) * 2021-01-22 2022-07-28 Nvidia Corporation Static data center power balancing and configuration
CN112988390A (en) * 2021-03-22 2021-06-18 上海超级计算中心 Calculation power resource allocation method and device
US20230115163A1 (en) * 2021-12-31 2023-04-13 Beijing Baidu Netcom Science Technology Co., Ltd. Method for processing data, and electronic device, storage medium and program product
CN115543615A (en) * 2022-09-29 2022-12-30 上海商汤科技开发有限公司 Resource allocation method and device, electronic equipment and storage medium
CN116069500A (en) * 2022-12-20 2023-05-05 中国电信股份有限公司 Model training task processing method and device, electronic equipment and readable medium
CN116450355A (en) * 2023-04-21 2023-07-18 重庆长安汽车股份有限公司 Multi-cluster model training method, device, equipment and medium
CN116954929A (en) * 2023-09-20 2023-10-27 四川并济科技有限公司 Dynamic GPU scheduling method and system for live migration

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JAY H. PARK等: "HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism", 《2020 USENIX ANNUAL TECHNICAL CONFERENCE》, pages 307 - 321 *
沙章利: "基于AWS GPU集群的协同过滤算法的研究及应用", 《中国优秀硕士学位论文全文数据库信息科技辑》, no. 03, pages 138 - 7909 *

Also Published As

Publication number Publication date
CN117194991B (en) 2024-02-13

Similar Documents

Publication Publication Date Title
CN109739939A (en) The data fusion method and device of knowledge mapping
CN114186084B (en) Online multi-mode Hash retrieval method, system, storage medium and equipment
US10268749B1 (en) Clustering sparse high dimensional data using sketches
CN112396462A (en) Crowd circling method and device based on Clickhouse
CN113628043B (en) Complaint validity judging method, device, equipment and medium based on data classification
CN112214602A (en) Text classification method and device based on humor, electronic equipment and storage medium
CN117194991B (en) High-dimensional data recommendation system and method based on GPU cluster
CN108829846B (en) Service recommendation platform data clustering optimization system and method based on user characteristics
CN110795559A (en) Data processing method and device for customer service question answering
CN114119289A (en) Method and device for processing comprehensive energy monitoring data
CN107844536A (en) The methods, devices and systems of application program selection
CN113778681B (en) Data processing method and device based on cloud computing and storage medium
CN106599244B (en) General original log cleaning device and method
CN114298319B (en) Determination method and device for joint learning contribution value, electronic equipment and storage medium
CN111562990B (en) Lightweight serverless computing method based on message
CN113709314B (en) Intelligent seat outbound method and device, electronic equipment and computer storage medium
CN113076450B (en) Determination method and device for target recommendation list
CN112667398B (en) Resource scheduling method and device, electronic equipment and storage medium
CN112214683B (en) Mixed recommendation model processing method, system and medium based on heterogeneous information network
CN115048422A (en) Process recommendation method, device, equipment and storage medium
CN115237783A (en) Test data generation method and device
CN116107630B (en) Multi-platform adaptation method for big data operation and maintenance monitoring
JP7622182B1 (en) Information processing device, information processing method, and information processing program
CN118132686B (en) Service robot problem matching method and system for multi-service scene
CN111897910A (en) Information pushing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant