CN119272821A - Time sequence prediction model generation method, data processing method and device - Google Patents
Time sequence prediction model generation method, data processing method and device Download PDFInfo
- Publication number
- CN119272821A CN119272821A CN202411783260.0A CN202411783260A CN119272821A CN 119272821 A CN119272821 A CN 119272821A CN 202411783260 A CN202411783260 A CN 202411783260A CN 119272821 A CN119272821 A CN 119272821A
- Authority
- CN
- China
- Prior art keywords
- target
- information
- covariate
- time sequence
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 238000003672 processing method Methods 0.000 title abstract description 12
- 239000013598 vector Substances 0.000 claims abstract description 239
- 230000004927 fusion Effects 0.000 claims abstract description 77
- 238000012549 training Methods 0.000 claims abstract description 70
- 238000007499 fusion processing Methods 0.000 claims abstract description 52
- 238000012545 processing Methods 0.000 claims description 141
- 238000000605 extraction Methods 0.000 claims description 38
- 238000004458 analytical method Methods 0.000 claims description 25
- 238000004364 calculation method Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 7
- 238000010248 power generation Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 12
- 230000000875 corresponding effect Effects 0.000 description 86
- 238000003860 storage Methods 0.000 description 14
- 239000002910 solid waste Substances 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000005065 mining Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 229910052760 oxygen Inorganic materials 0.000 description 4
- 239000001301 oxygen Substances 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000005684 electric field Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 238000010998 test method Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000000546 chi-square test Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000010348 incorporation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000000528 statistical test Methods 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a method for generating a time sequence prediction model, a data processing method and a device. The method comprises the steps of obtaining historical target variable data and covariate data sets corresponding to preset prediction tasks, wherein the historical target variable data are historical prediction time sequence data, the covariate data sets are data sets related to prediction of the historical target variable data, determining target covariate information from the covariate data sets, wherein the target covariate information is information influencing the prediction of the historical target variable data, performing feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain fusion feature vectors, training a pre-training model according to the fusion feature vectors, and obtaining a target time sequence prediction model corresponding to the preset prediction tasks. The application solves the technical problems that the time sequence prediction model in the related technology is not combined with covariate information, and the time sequence data prediction accuracy is lower.
Description
Technical Field
The application relates to the field of artificial intelligence, in particular to a method for generating a time sequence prediction model, a data processing method and a device.
Background
With the rapid development of the deep learning technology, a time sequence prediction model is taken as an important branch of the deep learning technology, and has been widely applied in a plurality of fields such as industrial discrete manufacturing, retail sales volume prediction, power load prediction and the like. However, existing timing prediction models do not incorporate covariate information, resulting in lower accuracy of timing data predictions.
In view of the above problems, no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the application provides a generation method, a data processing method and a device of a time sequence prediction model, which at least solve the technical problem that the time sequence prediction model in the related technology is not combined with covariate information and has lower time sequence data prediction accuracy.
According to one aspect of the embodiment of the application, a generation method of a time sequence prediction model is provided, and the generation method comprises the steps of obtaining historical target variable data and covariate data sets corresponding to preset prediction tasks, wherein the preset prediction tasks at least comprise time sequence prediction tasks in the electric power field, the historical target variable data are historical prediction time sequence data, the covariate data sets are data sets related to prediction of the historical target variable data, determining target covariate information from the covariate data sets, wherein the target covariate information is information influencing the prediction of the historical target variable data, performing feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain fusion feature vectors, and training a pre-training model according to the fusion feature vectors to obtain a target time sequence prediction model corresponding to the preset prediction tasks.
Further, the mode of the target covariate information is at least one of text, numerical value and enumeration, the step of determining the target covariate information from the covariate data set comprises the steps of respectively calculating correlation coefficients between time sequence covariate data and historical target variable data in the covariate data set to obtain a calculation result if the mode of the target covariate information is numerical value, and the step of determining the target covariate information from the time sequence covariate data based on the calculation result.
Further, determining the target covariate information from the covariate data set includes performing bias analysis on timestamp information and event information in the covariate data set based on historical target variable data if the target covariate information is in a text and/or enumeration mode, obtaining an analysis result, and determining the target covariate information from the timestamp information and the event information based on the analysis result.
Further, feature embedding and fusion processing are carried out on the historical target variable data and the target covariate information to obtain a fusion feature vector, wherein the feature embedding processing is carried out on the historical target variable data and the target covariate information through a preset embedding module to obtain a feature vector corresponding to the historical target variable data and a feature vector corresponding to the target covariate information, and the fusion processing is carried out on the feature vector corresponding to the historical target variable data and the feature vector corresponding to the target covariate information to obtain the fusion feature vector.
Further, the preset embedding module comprises a first embedding module, a second embedding module, a third embedding module and a fourth embedding module, the types of information processed by the first embedding module, the second embedding module, the third embedding module and the fourth embedding module are different, if the mode of the target covariate information comprises texts, numerical values and enumeration, the characteristic embedding processing is respectively carried out on the historical target variable data and the target covariate information through the preset embedding module, and the characteristic vector corresponding to the historical target variable data and the characteristic vector corresponding to the target covariate information are obtained; the method comprises the steps of obtaining a first feature vector, obtaining time sequence covariate data with a mode of numerical value in target covariate information, obtaining a second feature vector by carrying out information extraction and embedding processing on the time stamp information with the mode of numerical value in the target covariate information through a second embedding module, obtaining a third feature vector by carrying out information extraction and embedding processing on event information with the mode of enumeration in the target covariate information through a third embedding module, obtaining a fourth feature vector, and obtaining a fifth feature vector by carrying out information extraction and embedding processing on event information with the mode of text in the target covariate information through a fourth embedding module.
Further, the fusion processing is carried out on the feature vector corresponding to the historical target variable data and the feature vector corresponding to the target covariate information, so that a fusion feature vector is obtained, wherein the fusion processing is carried out on the first feature vector, the second feature vector, the third feature vector, the fourth feature vector and the fifth feature vector, so that the fusion feature vector is obtained.
Further, training the pre-training model according to the fusion feature vector to obtain a target time sequence prediction model corresponding to the preset prediction task, wherein the step of inputting the fusion feature vector into the pre-training model and adjusting model parameters of the pre-training model according to a preset loss function to obtain the target time sequence prediction model.
According to another aspect of the embodiment of the application, a data processing method is provided, which comprises the steps of obtaining historical time sequence data corresponding to a preset prediction task and preset target covariate information, wherein the preset prediction task at least comprises a time sequence prediction task in the electric power field, the target covariate information is information influencing the prediction of the target prediction time sequence data, carrying out fusion processing on the historical time sequence data and the target covariate information through a target time sequence prediction model to obtain a target feature vector, wherein the target time sequence prediction model is obtained by adopting the method for generating the time sequence prediction model of any one of the above steps, and predicting the target prediction time sequence data corresponding to the preset prediction task according to the target feature vector.
According to another aspect of the embodiment of the application, a data processing method is provided, which comprises the steps of obtaining historical time sequence data corresponding to a preset prediction task uploaded by a client and preset target covariate information, wherein the preset prediction task at least comprises a time sequence prediction task in the electric power field, the target covariate information is information influencing the prediction of the target prediction time sequence data, carrying out fusion processing on the historical time sequence data and the target covariate information through a target time sequence prediction model in a cloud server to obtain a target feature vector, wherein the target time sequence prediction model is obtained by adopting the generation method of the time sequence prediction model of any item, obtaining the target prediction time sequence data corresponding to the preset prediction task according to the target feature vector prediction, and feeding the target prediction time sequence data back to the client.
According to another aspect of the embodiment of the application, a generating device of a time sequence prediction model is further provided, and the generating device comprises a first obtaining unit, a first determining unit and a second determining unit, wherein the first obtaining unit is used for obtaining historical target variable data and covariate data sets corresponding to a preset prediction task, the preset prediction task at least comprises a time sequence prediction task in the electric power field, the historical target variable data are historical prediction time sequence data, the covariate data sets are data related to prediction of the historical target variable data, the first determining unit is used for determining target covariate information from the covariate data sets, the target covariate information is information influencing the prediction of the historical target variable data, the first processing unit is used for performing feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain fusion feature vectors, and the second processing unit is used for training the pre-training model according to the fusion feature vectors to obtain the target time sequence prediction model corresponding to the preset prediction task.
Further, the first determining unit comprises a calculating subunit, a first determining subunit and a second determining subunit, wherein the calculating subunit is used for respectively calculating correlation coefficients between time sequence covariate data in the covariate data set and historical target variable data to obtain a calculation result if the mode of the target covariate information is a numerical value, and the first determining subunit is used for determining the target covariate information from the time sequence covariate data based on the calculation result.
Further, the first determining unit comprises an analyzing subunit, a second determining subunit and a third determining subunit, wherein the analyzing subunit is used for analyzing the deviation degree of the timestamp information and the event information in the covariate data set based on the historical target variable data to obtain an analysis result if the mode of the target covariate information is text and/or enumeration, and the second determining subunit is used for determining the target covariate information from the timestamp information and the event information based on the analysis result.
The first processing unit comprises a first processing subunit and a second processing subunit, wherein the first processing subunit is used for respectively carrying out feature embedding processing on the historical target variable data and the target covariate information through a preset embedding module to obtain feature vectors corresponding to the historical target variable data and feature vectors corresponding to the target covariate information, and the second processing subunit is used for carrying out fusion processing on the feature vectors corresponding to the historical target variable data and the feature vectors corresponding to the target covariate information to obtain fusion feature vectors.
The first processing subunit comprises a first processing module, a second processing module, a third processing module and a fourth processing module, wherein the first processing module is used for carrying out information extraction and embedding processing on historical target variable data through the first embedding module to obtain a first feature vector, the second processing module is used for carrying out information extraction and embedding processing on timestamp information with a numerical value in the target covariant information through the second embedding module to obtain a second feature vector, the third processing module is used for carrying out information extraction and embedding processing on time sequence covariant data with a numerical value in the target covariant information through the third embedding module to obtain a third feature vector, and carrying out information extraction and embedding processing on event information with a enumerated modal in the target covariant information through the third embedding module to obtain a fourth feature vector, and the fourth processing module is used for carrying out information extraction and embedding processing on event information with a text in the target covariant information through the fourth embedding module to obtain a fifth feature vector.
Further, the second processing subunit includes a fifth processing module, configured to perform fusion processing on the first feature vector, the second feature vector, the third feature vector, the fourth feature vector, and the fifth feature vector, to obtain a fused feature vector.
Further, the second processing unit comprises an adjusting subunit, which is used for inputting the fusion feature vector into the pre-training model, and adjusting model parameters of the pre-training model according to a preset loss function to obtain the target time sequence prediction model.
According to another aspect of the embodiment of the application, a data processing device is provided, which comprises a second acquisition unit, a third processing unit and a fourth processing unit, wherein the second acquisition unit is used for acquiring historical time sequence data corresponding to a preset prediction task and preset target covariate information, the preset prediction task at least comprises a time sequence prediction task in the electric power field, the target covariate information is information influencing the prediction of the target prediction time sequence data, the third processing unit is used for carrying out fusion processing on the historical time sequence data and the target covariate information through a target time sequence prediction model to obtain a target feature vector, the target time sequence prediction model is obtained by adopting the generation method of the time sequence prediction model of any item, and the fourth processing unit is used for predicting the target prediction time sequence data corresponding to the preset prediction task according to the target feature vector.
According to another aspect of the embodiment of the invention, the electronic device further provides an electronic device, which comprises a memory and a processor, wherein the memory stores an executable program, and the processor is used for running the program, and the method for generating the time sequence prediction model of any one of the above steps is executed when the program runs.
According to another aspect of the embodiment of the present invention, there is further provided a computer readable storage medium storing a program, where the program controls a device in which the storage medium is located to execute the method for generating the time sequence prediction model of any one of the above items when running.
According to another aspect of an embodiment of the present invention, there is also provided a computer program product comprising a computer program or instructions which, when executed by a processor, implement a method of generating a timing prediction model of any of the above.
In the embodiment of the application, a historical target variable data and a covariate data set corresponding to a preset prediction task are obtained, wherein the preset prediction task at least comprises a time sequence prediction task in the electric field, the historical target variable data is historical prediction time sequence data, the covariate data set is a set of data related to prediction of the historical target variable data, target covariate information is determined from the covariate data set, the target covariate information is information influencing the prediction of the historical target variable data, feature embedding and fusion processing are carried out on the historical target variable data and the target covariate information to obtain a fusion feature vector, a pre-training model is trained according to the fusion feature vector to obtain a target time sequence prediction model corresponding to the preset prediction task, the covariate information is effectively utilized, the model is trained through the fusion of multi-mode covariate information, and when the complex prediction task containing various types of covariate is processed, more accurate prediction can be better understood and made by using the covariate information, the accuracy and the robustness of the time sequence prediction model are improved, the accuracy and the efficiency of the time sequence prediction model are improved, the accuracy and the accuracy of the time sequence prediction model is improved, the accuracy of the time sequence prediction model is better supported, the relevant time sequence prediction data is achieved, and the accuracy of the time sequence prediction model is better than the relevant technology is achieved, and the accuracy of the prediction model is achieved, and the accuracy is achieved, and the problem is not achieved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
Fig. 1 is a schematic diagram of a computer terminal according to a first embodiment of the present application;
FIG. 2 is a flowchart of a method for generating a time series prediction model according to a first embodiment of the present application;
FIG. 3 is a schematic diagram of an alternative multimodal covariate fusion processing framework provided in accordance with an embodiment of the application;
FIG. 4 is a flow chart of a data processing method according to a second embodiment of the present application;
FIG. 5 is a flow chart of a data processing method according to a third embodiment of the present application;
Fig. 6 is a schematic diagram of a generating apparatus of a time-series prediction model according to a fourth embodiment of the present application;
FIG. 7 is a schematic diagram of a data processing apparatus according to a fifth embodiment of the present application;
Fig. 8 is a block diagram of an electronic device according to a sixth embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related region, and provide corresponding operation entries for the user to select authorization or rejection.
First, partial terms or terminology appearing in the course of describing embodiments of the application are applicable to the following explanation:
embedding embedding converts the high-dimensional sparse vectors into dense vectors, thereby facilitating downstream model processing, such as model processing after embedding for time series data, images, audio, etc.
Description information (prompt) prompt for large models to make the large model easier to understand.
Example 1
In accordance with an embodiment of the present application, there is also provided a method of generating a timing prediction model, it being noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
The method according to the first embodiment of the present application may be implemented in a mobile terminal, a computer terminal or a similar computing device. Fig. 1 shows a hardware block diagram of a computer terminal (or mobile device) for implementing a method of generating a timing prediction model. As shown in fig. 1, the computer terminal (or mobile device) 10 may include a processor set 102 (the processor set 102 may include, but is not limited to, a microprocessor (Microcontroller Unit, abbreviated MCU) or a programmable logic device (Field Programmable GATE ARRAY, abbreviated FPGA) or the like, and the processor set 102 may include a processor set, shown in fig. 1 as 102a,102b, 102 n), a memory 104 for storing data, and a transmission means 106 for communication functions. Among other things, a display, an input/output interface (I/O interface), a universal serial BUS (Universal Serial Bus, simply USB) port (which may be included as one of the ports of the BUS BUS), a network interface, a power supply, and/or a camera. It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 1 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
It should be noted that the one or more processors 102 and/or other data processing circuits described above may be referred to generally herein as "data processing circuits. The data processing circuit may be embodied in whole or in part in software, hardware, firmware, or any other combination. Furthermore, the data processing circuitry may be a single stand-alone processing module, or incorporated, in whole or in part, into any of the other elements in the computer terminal 10 (or mobile device).
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the method for generating a time sequence prediction model in the embodiment of the present application, and the processor 102 executes the software programs and modules stored in the memory 104, thereby executing various functional applications and data processing, that is, implementing the method for generating a time sequence prediction model described above. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 106 is arranged to receive or transmit data via a network. The specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, simply NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is configured to communicate with the internet wirelessly.
The display may be a touch screen type liquid crystal display that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).
In many fields such as industrial discrete manufacturing, retail sales prediction, and power load prediction, accurate prediction of the future state of a time series variable is of paramount importance. Traditional prediction methods, such as statistical analysis and classical machine learning algorithms, while capable of providing a degree of prediction capability based on historical data, often suffer from inherent limitations of models and inadequate understanding of complex timing patterns.
With the rapid development of deep learning technology, a deep learning model based on a self-attention mechanism can effectively capture long-term and short-term dependency relations in time sequence data due to strong sequence processing capacity and the self-attention mechanism, and quickly becomes a research hotspot in the field of time sequence prediction. However, despite its widespread use, this model has problems of high training costs and reliance on a large number of training samples, especially limited to a single domain of data training when training a randomly initialized model from scratch, resulting in insufficient generalization ability of the model in processing cross-domain time series data, and neglecting covariate information highly correlated with target variables, resulting in lower accuracy of time series data prediction.
In the above technical background, the present application provides a method for generating a time sequence prediction model as shown in fig. 2. Fig. 2 is a flowchart of a method for generating a time sequence prediction model according to an embodiment of the present application. The method comprises the following steps:
Step S201, obtaining historical target variable data and a covariate data set corresponding to a preset prediction task, where the preset prediction task at least includes a time sequence prediction task in the electric power field, the historical target variable data is historical prediction time sequence data, and the covariate data set is a set of data related to prediction of the historical target variable data.
Alternatively, the preset prediction task may be a time series prediction task in various fields of manufacturing, retail, electric power, traffic, and the like, such as solid waste furnace temperature prediction, electric power load prediction, and the like. For example, after a vertical scene (taking solid waste furnace temperature prediction as an example) is determined, historical target variable data and a covariate data set corresponding to a solid waste furnace temperature prediction task are obtained through a generation system of a time sequence prediction model, wherein the historical target variable data is the historical predicted furnace temperature, and the covariate data set comprises data related to the historical predicted furnace temperature such as oxygen concentration, ambient temperature, time and the like.
In step S202, target covariate information is determined from the covariate dataset, wherein the target covariate information is information that affects prediction of historical target variable data.
In order to enable the time sequence big model to have better fine tuning effect in the vertical field, multi-mode time sequence covariate mining is carried out, namely target covariate information is determined from a covariate data set, and the mode of the target covariate information is at least one of text type, numerical value type and enumeration type, wherein the target covariate information is information influencing the prediction of target time sequence variables (such as furnace temperature) and comprises the following types:
timing sequence covariates, namely timing sequence variables with the same frequency as the target timing sequence variables, such as oxygen concentration (numerical value) and weather (numerical value) in the prediction of the solid waste furnace temperature;
Time stamp information, namely time information (text) with the same frequency as the target time sequence variable or calendar information (numerical value) where the time stamp is positioned;
Event information, discrete variables, e.g., failure at a certain time (text), holiday information (text), holiday (enumeration), winter (enumeration).
And step S203, performing feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain a fusion feature vector.
In order to enable the time sequence large model to have a better fine tuning effect in the vertical field, multi-mode covariate fusion is carried out, namely characteristic embedding and fusion processing are carried out on historical target variable data and target covariate information, and fusion characteristic vectors are obtained. For example, the characteristic embedding processing is respectively performed on the historical prediction furnace temperature and the corresponding target covariate information (such as oxygen concentration and the like) through a preset embedding module, so that the characteristic vector corresponding to the historical prediction furnace temperature and the characteristic vector corresponding to the target covariate information are obtained, and the fusion processing is performed on the characteristic vector corresponding to the historical prediction furnace temperature and the characteristic vector corresponding to the target covariate information, so that the fusion characteristic vector is obtained.
And step S204, training the pre-training model according to the fusion feature vector to obtain a target time sequence prediction model corresponding to the preset prediction task.
Optionally, performing fine tuning training on the pre-training model according to the fusion feature vector, so as to obtain a target time sequence prediction model corresponding to the preset prediction task, for example, the target time sequence prediction model is a time sequence prediction model for predicting the solid waste furnace temperature.
Alternatively, the pre-training model may be a large language model or a time-series pre-training large model. For example, a deep learning model based on a self-attention mechanism is pre-trained in advance using time series data of a plurality of fields such as manufacturing, retail, electric power, traffic and the like, to obtain a time series pre-trained large model. The pre-training model can be determined by processing the training set with a large language model and a time sequence pre-training large model respectively, and selecting a model with smaller error between the model output result and the real label of the training set as the pre-training model.
In the scheme, the training is carried out on the pre-training model by fusing multi-mode covariate information, so that the model can better understand and utilize the covariate information to make more accurate predictions when processing complex prediction tasks containing various covariate, the accuracy and robustness of the time sequence prediction model are improved, the prediction precision and efficiency are improved, powerful support is provided for decisions in related fields, the purpose of realizing more accurate time sequence prediction is achieved, the technical effect of improving the prediction accuracy of the time sequence prediction model on time sequence data is achieved, and the technical problem that the time sequence prediction model in related technologies is not combined with covariate information and has lower time sequence data prediction accuracy is solved.
In order to mine proper covariate information, in the method for generating the time sequence prediction model provided by the embodiment of the application, the mode of the target covariate information is at least one of text, numerical value and enumeration, and the step of determining the target covariate information from the covariate data set comprises the steps of respectively calculating correlation coefficients between time sequence covariate data and historical target variable data in the covariate data set to obtain a calculation result if the mode of the target covariate information is numerical value, and determining the target covariate information from the time sequence covariate data based on the calculation result.
In order to mine proper covariate information, in the method for generating the time sequence prediction model provided by the embodiment of the application, determining target covariate information from the covariate data set comprises the steps of analyzing the deviation degree of timestamp information and event information in the covariate data set based on historical target variable data to obtain an analysis result if the mode of the target covariate information is text and/or enumeration, and determining the target covariate information from the timestamp information and the event information based on the analysis result.
Blind incorporation of potential covariates may instead degrade the predictive effect, so how to mine the appropriate covariate information for a particular vertical scenario would be critical. Alternatively, the mining of the target covariate information is divided into correlation mining and deviation mining, wherein the correlation mining is to judge the validity of the covariate according to the correlation coefficient of the covariate and the target variable, and is suitable for the information mining with the time sequence covariate as a numerical value, for example, in solid waste furnace temperature prediction, the correlation coefficients between all process variables (namely time sequence covariate data in the covariate data set) and the furnace temperature (namely historical target variable data) are respectively calculated to obtain a plurality of correlation coefficients (namely calculation results), then the target covariate information can be determined from the time sequence covariate data based on the calculation results, for example, the plurality of correlation coefficients are inverted index, and Top K covariates are selected as the target covariate information, wherein K is a positive integer. The correlation coefficient may be calculated by using a pearson correlation coefficient, a spearman level correlation coefficient, a kendel level correlation coefficient, or the like, which are widely used, and is not limited herein.
The deviation mining is to mine the value distribution difference of the target variable when the covariates are different in value, and is suitable for mining information with the mode of text and/or enumeration. Firstly, data preprocessing such as time stamp alignment and event information coding can be performed, the time stamp information of all sequence data is consistent through the time stamp alignment, the subsequent analysis is convenient, and the event information (such as holidays, abnormal alarms and the like) is converted into numerical value or category coding so that a model can be understood and processed. For the time stamp information, a time window may be defined based on a time scale of the predicted task, such as hours, days, weeks or months, and features may be extracted from the time stamp, such as a weekday or a weekend, summer or winter, etc., and then historical target variable data may be grouped according to the time features, and the groups may be compared to see if there is a significant difference in the distribution of the target variable at different feature values. For the event information, the event information can be classified into different categories, such as holidays, abnormal alarms and the like, and a front time window and a rear time window of the event are defined, such as a few hours or a few days before and after the occurrence of the event, and the target variable data before and after the event is used for comparison analysis to see whether a significant difference exists. The bias analysis may be implemented by a variety of statistical test methods, such as variance test, chi-square test, or non-parametric test. For example, a variance test method is used to compare whether there is a significant difference in the mean value of the target variable under different time characteristics or events, calculate the p-value (i.e., mean value), and if the p-value is less than a preset significance level (e.g., 0.05), consider the distribution of the target variable under this condition to have a significant difference, thereby identifying timestamp information and event information that statistically significantly affect the target variable. For example, in the power load prediction task, a variance test method is used to analyze the power load sequence in summer and the power load sequence in winter, and it is found that the power consumption in summer is significantly higher than the power consumption in winter, so whether summer or winter can be used as target covariate information.
Through correlation coefficient calculation and deviation analysis, proper covariate information can be selected, and redundant information is filtered.
In order to effectively combine covariate information, in the method for generating the time sequence prediction model provided by the embodiment of the application, feature embedding and fusion processing are carried out on historical target variable data and target covariate information to obtain fusion feature vectors, wherein the step of carrying out feature embedding processing on the historical target variable data and the target covariate information through a preset embedding module to obtain feature vectors corresponding to the historical target variable data and feature vectors corresponding to the target covariate information, and the step of carrying out fusion processing on the feature vectors corresponding to the historical target variable data and the feature vectors corresponding to the target covariate information to obtain fusion feature vectors.
After determining the target covariate information, it is also critical how to effectively combine the covariate information. Optionally, feature embedding processing is performed on the historical target variable data and the target covariate information through a preset embedding module, so that feature vectors corresponding to the historical target variable data and feature vectors corresponding to the target covariate information can be obtained, and then fusion processing is performed on the feature vectors corresponding to the historical target variable data and the feature vectors corresponding to the target covariate information, so that fusion feature vectors can be obtained. The preset embedding module at least comprises a position embedding module (positional embedding), a time embedding module (temporal embedding), a covariate embedding module (covariate embedding) and a description information embedding module (prompt embedding). positional embedding belongs to a foundation embedding, is used for processing target variables, and can be used in a pluggable manner with other embedding. temporal embedding is used for extracting information such as year, month, day, hour and the like of the time stamp to form a time sequence variable with the same length, then mapping the obtained time sequence variable to a vector with the same dimension as the time sequence embedding of the target variable by means of sine and cosine trigonometric functions or a learnable embedding or full-connection layer, and then performing point adding or point multiplying operation. covariate embedding is used for processing the time sequence covariates and the enumerated event information to obtain time sequence variables, and the time sequence covariates and the enumerated event information are connected with the target variables and then are sent into the model together for training reasoning. prompt embedding is used for processing event information of text type, such as holidays, abnormal alarms and the like, belonging to discontinuous events, and can perform dot addition or dot multiplication operation with a target variable through a prompt word and text embedding mode.
The fusion processing is carried out on the feature vector corresponding to the historical target variable data and the feature vector corresponding to the target covariate information, so that a fusion feature vector capable of comprehensively capturing various information dimensions in the time sequence prediction task is constructed, and richer and more comprehensive feature representation is provided for subsequent model training and prediction. The feature Fusion can adopt widely applied Early Fusion (Early Fusion), late Fusion (Late Fusion) or Deep Fusion (Deep Fusion) and other technologies, wherein the Early Fusion directly connects all feature vectors according to a certain sequence to form a long vector as a Fusion feature vector, the Late Fusion firstly processes each feature vector independently and then connects or averages the processed results to be used as the Fusion feature vector, and the Deep Fusion fuses the feature vectors through a neural network layer (such as a full connection layer or an attention mechanism layer) to allow a model to learn how to combine features of different information sources.
The historical target variable data and the target covariate information are subjected to feature embedding processing and fusion to obtain fusion feature vectors, so that effective combination of covariate information is realized, and a data basis is provided for model fine tuning training.
In order to effectively combine covariate information, in the method for generating the time sequence prediction model provided by the first embodiment of the application, a preset embedding module comprises a first embedding module, a second embedding module, a third embedding module and a fourth embedding module, the first embedding module, the second embedding module, the third embedding module and the fourth embedding module are used for processing information types different, if the mode of target covariate information comprises texts, numerical values and enumeration, characteristic embedding processing is respectively carried out on historical target variable data and target covariate information through the preset embedding module to obtain characteristic vectors corresponding to the historical target variable data and characteristic vectors corresponding to the target covariate information, information extraction and embedding processing are carried out on the historical target variable data through the first embedding module to obtain a first characteristic vector, information extraction and embedding processing are carried out on time stamp information with the mode of the numerical values in the target covariate information through the second embedding module to obtain a second characteristic vector, information extraction and embedding processing is carried out on the time sequence covariate data with the mode of the numerical values in the target covariate information through the third embedding module to obtain a third characteristic vector, characteristic vector is obtained through the characteristic vector extraction and the fourth embedding processing is carried out on the characteristic vector extraction information with the fourth embedding module in the target covariate information to obtain the characteristic vector.
In order to effectively combine covariate information, in the method for generating the time sequence prediction model provided by the embodiment of the application, fusion processing is carried out on the feature vector corresponding to the historical target variable data and the feature vector corresponding to the target covariate information to obtain a fusion feature vector, wherein the fusion processing is carried out on the first feature vector, the second feature vector, the third feature vector, the fourth feature vector and the fifth feature vector to obtain the fusion feature vector.
Optionally, the first embedding module is the aforementioned position embedding module, the second embedding module is the aforementioned time embedding module, the third embedding module is the aforementioned covariant embedding module, and the fourth embedding module is the aforementioned description information embedding module. For example, the method includes the steps of performing position information extraction and embedding processing on historical target variable data through positional embedding to obtain a feature vector (namely a first feature vector) corresponding to the historical target variable data, performing information extraction and embedding processing on timestamp information with a numerical value in the target covariate information through temporal embedding to obtain a second feature vector, performing information extraction and embedding processing on time sequence covariate data with a numerical value in the target covariate information through covariate embedding to obtain a third feature vector, performing information extraction and embedding processing on event information with a enumerated mode in the target covariate information through covariate embedding to obtain a fourth feature vector, and performing information extraction and embedding processing on event information with a text mode in the target covariate information through prompt embedding to obtain a fifth feature vector.
Optionally, the feature vector corresponding to the target covariate information includes a second feature vector, a third feature vector, a fourth feature vector, and a fifth feature vector. And carrying out fusion processing on the first feature vector, the second feature vector, the third feature vector, the fourth feature vector and the fifth feature vector to obtain a fusion feature vector. The fusion processing can adopt the operations of splicing fusion (splicing according to a specific sequence), element-level fusion (such as point adding or point multiplying), attention mechanism fusion (weighting fusion based on weights among different feature vectors), neural network fusion (fusion of feature vectors through designing a specific neural network structure such as a full-connection layer, a convolution layer or a cyclic neural network layer) and the like according to actual requirements, so as to form a comprehensive fusion feature vector.
The first feature vector, the second feature vector, the third feature vector, the fourth feature vector and the fifth feature vector are fused, so that effective combination of covariate information is realized, and a data base is provided for model fine tuning training.
In order to improve the prediction accuracy of the model, in the method for generating the time sequence prediction model provided by the embodiment of the application, training the pre-training model according to the fusion feature vector to obtain the target time sequence prediction model corresponding to the preset prediction task comprises the steps of inputting the fusion feature vector into the pre-training model, and adjusting model parameters of the pre-training model according to the preset loss function to obtain the target time sequence prediction model.
Optionally, after the fusion feature vector is obtained, the target time sequence prediction model can be obtained by inputting the fusion feature vector into the pre-training model and adjusting model parameters of the pre-training model according to a preset loss function. For example, a time sequence prediction model for predicting the solid waste furnace temperature can be obtained by inputting the fusion feature vector into a large language model or a time sequence pre-training large model for fine tuning training.
The training of the pre-training model is performed by fusing multi-mode covariate information, so that the model can better understand and utilize the covariate information to make more accurate predictions when processing complex prediction tasks containing various covariates, the accuracy and the robustness of the time sequence prediction model are improved, and the prediction precision and the prediction efficiency are improved.
In an alternative embodiment, the schematic diagram shown in fig. 3 may be used to implement a multimodal covariate fusion process. Fig. 3 is a schematic diagram of an alternative multimodal covariate fusion processing framework provided according to an embodiment of the application, and as shown in fig. 3, the information types of the embedding processing may be multimodal, such as time stamps, season and the like, holidays, events, weather and other text information. The embedding module is provided with a position embedding module, a time embedding module, a covariate embedding module and a descriptive information embedding module which are respectively used for processing different information types. The embedded layer is divided into a column dimension and a row dimension, wherein the column dimension refers to each column of the table data has respective codes, and the row dimension refers to the table data has unified codes. As shown in fig. 3, an example of an application scenario and covariate types in the scenario, which correspond to row and column dimensions, respectively, is given.
In the embodiment of the application, a historical target variable data and a covariate data set corresponding to a preset prediction task are obtained, wherein the preset prediction task at least comprises a time sequence prediction task in the electric field, the historical target variable data is historical prediction time sequence data, the covariate data set is a set of data related to prediction of the historical target variable data, target covariate information is determined from the covariate data set, the target covariate information is information influencing the prediction of the historical target variable data, feature embedding and fusion processing are carried out on the historical target variable data and the target covariate information to obtain a fusion feature vector, a pre-training model is trained according to the fusion feature vector to obtain a target time sequence prediction model corresponding to the preset prediction task, the covariate information is effectively utilized, the model is trained through the fusion of multi-mode covariate information, and when the complex prediction task containing various types of covariate is processed, more accurate prediction can be better understood and made by using the covariate information, the accuracy and the robustness of the time sequence prediction model are improved, the accuracy and the efficiency of the time sequence prediction model are improved, the accuracy and the accuracy of the time sequence prediction model is improved, the accuracy of the time sequence prediction model is better supported, the relevant time sequence prediction data is achieved, and the accuracy of the time sequence prediction model is better than the relevant technology is achieved, and the accuracy of the prediction model is achieved, and the accuracy is achieved, and the problem is not achieved.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method of the various embodiments of the present application.
Example 2
According to an embodiment of the present application, there is also provided a data processing method, as shown in fig. 4, including:
Step S401, acquiring historical time sequence data corresponding to a preset prediction task and preset target covariate information, wherein the preset prediction task at least comprises a time sequence prediction task in the electric power field, and the target covariate information is information influencing the prediction of the target prediction time sequence data;
Step S402, carrying out fusion processing on historical time sequence data and target covariate information through a target time sequence prediction model to obtain a target feature vector, wherein the target time sequence prediction model is obtained by adopting the generation method of any time sequence prediction model;
step S403, predicting according to the target feature vector to obtain target prediction time sequence data corresponding to the preset prediction task.
Alternatively, after the target timing prediction model is obtained, a corresponding data processing may be performed. For example, the preset prediction task is solid waste furnace temperature prediction, a time sequence prediction model for solid waste furnace temperature prediction is obtained by adopting the method for generating the time sequence prediction model of any one of the above steps, the solid waste furnace temperature (namely, historical time sequence data) in the past period of time and target covariate information (such as oxygen concentration, weather and the like) dug in the model training process are input into the time sequence prediction model, the model can be subjected to fusion processing to obtain a target feature vector, and then the solid waste furnace temperature (namely, target prediction time sequence data) in the future period of time is obtained according to the target feature vector prediction.
The training of the pre-training model is performed by fusing multi-mode covariate information, so that the model can better understand and utilize the covariate information to make more accurate predictions when processing complex prediction tasks containing various covariate, the accuracy and robustness of the time sequence prediction model are improved, the prediction precision and efficiency are improved, powerful support is provided for decisions in related fields, the aim of realizing more accurate time sequence prediction is achieved, the technical effect of improving the prediction accuracy of the time sequence prediction model on time sequence data is achieved, and the technical problem that the time sequence prediction model in related technologies does not combine covariate information and has lower time sequence data prediction accuracy is solved.
Example 3
According to an embodiment of the present application, there is also provided a data processing method, as shown in fig. 5, including:
step S501, historical time sequence data and predetermined target covariate information corresponding to a preset prediction task uploaded by a client are obtained, wherein the preset prediction task at least comprises a time sequence prediction task in the electric power field, and the target covariate information is information influencing the prediction of the target prediction time sequence data;
Step S502, fusion processing is carried out on historical time sequence data and target covariate information through a target time sequence prediction model in a cloud server to obtain a target feature vector, wherein the target time sequence prediction model is obtained by adopting the generation method of any time sequence prediction model;
in step S503, the target prediction timing data is fed back to the client.
In the cloud server, the specific method of data processing is the same as that in the second embodiment, and will not be described here again.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method of the various embodiments of the present application.
Example 4
According to an embodiment of the present application, there is also provided a generating apparatus of a time sequence prediction model for implementing the generating method of a time sequence prediction model, as shown in fig. 6, the apparatus includes a first obtaining unit 601, a first determining unit 602, a first processing unit 603, and a second processing unit 604.
A first obtaining unit 601, configured to obtain historical target variable data and a covariate data set corresponding to a preset prediction task, where the preset prediction task at least includes a time sequence prediction task in the electric power field, the historical target variable data is historical prediction time sequence data, and the covariate data set is a set of data related to prediction of the historical target variable data;
A first determining unit 602, configured to determine target covariate information from the covariate dataset, where the target covariate information is information that affects prediction of historical target variable data;
The first processing unit 603 is configured to perform feature embedding and fusion processing on the historical target variable data and the target covariate information, so as to obtain a fused feature vector;
The second processing unit 604 is configured to train the pre-training model according to the fused feature vector, so as to obtain a target time sequence prediction model corresponding to the preset prediction task.
In the generating device of the time sequence prediction model provided in the fourth embodiment of the present application, a first obtaining unit 601 obtains historical target variable data and a covariate data set corresponding to a preset prediction task, where the preset prediction task at least includes a time sequence prediction task in the electric power domain, the historical target variable data is historical prediction time sequence data, the covariate data set is a set of data related to prediction of the historical target variable data, a first determining unit 602 determines target covariate information from the covariate data set, where the target covariate information is information influencing prediction of the historical target variable data, a first processing unit 603 performs feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain a fusion feature vector, and a second processing unit 604 trains the pre-training model according to the fusion feature vector to obtain a target time sequence prediction model corresponding to the preset prediction task. In the scheme, the training is carried out on the pre-training model by fusing multi-mode covariate information, so that the model can better understand and utilize the covariate information to make more accurate predictions when processing complex prediction tasks containing various covariate, the accuracy and robustness of the time sequence prediction model are improved, the prediction precision and efficiency are improved, powerful support is provided for decisions in related fields, the purpose of realizing more accurate time sequence prediction is achieved, the technical effect of improving the prediction accuracy of the time sequence prediction model on time sequence data is achieved, and the technical problem that the time sequence prediction model in related technologies is not combined with covariate information and has lower time sequence data prediction accuracy is solved.
Optionally, in the generating device of the time sequence prediction model provided in the fourth embodiment of the present application, the first determining unit 602 includes a calculating subunit, configured to calculate correlation coefficients between time sequence covariate data in the covariate data set and historical target variable data respectively if the mode of the target covariate information is a numerical value, to obtain a calculation result, and the first determining subunit is configured to determine the target covariate information from the time sequence covariate data based on the calculation result.
Optionally, in the generating device of the time sequence prediction model provided in the fourth embodiment of the present application, the first determining unit 602 further includes an analysis subunit, configured to, if the modality of the target covariate information is text and/or enumeration, perform deviation analysis on the timestamp information and the event information in the covariate dataset based on the historical target variable data to obtain an analysis result, and the second determining subunit is configured to determine the target covariate information from the timestamp information and the event information based on the analysis result.
Optionally, in the generating device of the time sequence prediction model provided in the fourth embodiment of the present application, the first processing unit 603 includes a first processing subunit, configured to perform feature embedding processing on the historical target variable data and the target covariate information through a preset embedding module, to obtain a feature vector corresponding to the historical target variable data and a feature vector corresponding to the target covariate information, and a second processing subunit, configured to perform fusion processing on the feature vector corresponding to the historical target variable data and the feature vector corresponding to the target covariate information, to obtain a fused feature vector.
Optionally, in the generating device of the time sequence prediction model provided by the fourth embodiment of the application, the first processing subunit includes a first processing module, a second processing module, a third processing module and a fourth processing module, wherein the first processing module is used for carrying out information extraction and embedding processing on historical target variable data through the first embedding module to obtain a first feature vector, the second processing module is used for carrying out information extraction and embedding processing on timestamp information with a numerical mode in the target covariant information through the second embedding module to obtain a second feature vector, the third processing module is used for carrying out information extraction and embedding processing on time sequence covariant data with a numerical mode in the target covariant information through the third embedding module to obtain a third feature vector, and carrying out information extraction and embedding processing on event information with a enumerated mode in the target covariant information through the third embedding module to obtain a fourth feature vector, and the fourth processing module is used for carrying out information extraction and embedding processing on event information with a text mode in the target covariant information through the fourth embedding module to obtain a fifth feature vector.
Optionally, in the generating device of the time sequence prediction model provided in the fourth embodiment of the present application, the second processing subunit includes a fifth processing module, configured to perform fusion processing on the first feature vector, the second feature vector, the third feature vector, the fourth feature vector, and the fifth feature vector, to obtain a fused feature vector.
Optionally, in the generating device of the time sequence prediction model provided in the fourth embodiment of the present application, the second processing unit 604 includes an adjustment subunit, configured to input the fusion feature vector into the pre-training model, and adjust model parameters of the pre-training model according to a preset loss function, so as to obtain the target time sequence prediction model.
Here, the first obtaining unit 601, the first determining unit 602, the first processing unit 603, and the second processing unit 604 described above correspond to steps S201 to S204 in embodiment 1, and the above-described units are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the first embodiment described above. It should be noted that the above-described module may be operated as a part of the apparatus in the computer terminal 10 provided in the first embodiment.
It should be noted that, the preferred embodiment of the present application in the above examples is the same as the embodiment provided in example 1, the application scenario and the implementation process, but is not limited to the embodiment provided in example 1.
Example 5
There is also provided a data processing apparatus for implementing the above data processing method according to an embodiment of the present application, as shown in fig. 7, which includes a second acquisition unit 701, a third processing unit 702, and a fourth processing unit 703.
A second obtaining unit 701, configured to obtain historical time sequence data corresponding to a preset prediction task and predetermined target covariate information, where the preset prediction task at least includes a time sequence prediction task in the electric power field, and the target covariate information is information that affects prediction of the target prediction time sequence data;
A third processing unit 702, configured to perform fusion processing on the historical time sequence data and the target covariate information through a target time sequence prediction model, to obtain a target feature vector, where the target time sequence prediction model is obtained by adopting the method for generating the time sequence prediction model of any one of the above items;
the fourth processing unit 703 is configured to predict and obtain target prediction time sequence data corresponding to the preset prediction task according to the target feature vector.
In the data processing apparatus provided in the fifth embodiment of the present application, the second obtaining unit 701 obtains historical time sequence data corresponding to a preset prediction task and predetermined target covariate information, where the preset prediction task at least includes a time sequence prediction task in the power domain, the target covariate information is information that affects prediction of the target prediction time sequence data, the third processing unit 702 performs fusion processing on the historical time sequence data and the target covariate information through a target time sequence prediction model to obtain a target feature vector, where the target time sequence prediction model is obtained by adopting the method for generating the time sequence prediction model of any one of the above items, and the fourth processing unit 703 predicts according to the target feature vector to obtain target prediction time sequence data corresponding to the preset prediction task.
Here, the second acquiring unit 701, the third processing unit 702, and the fourth processing unit 703 correspond to steps S401 to S403 in embodiment 2, and the above-mentioned units are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to those disclosed in the second embodiment.
It should be noted that the preferred embodiment of the present application in the above example is the same as the embodiment provided in example 2, the application scenario and the implementation process, but is not limited to the embodiment provided in example 2.
Example 6
Embodiments of the present application may provide an electronic device, which may be any one of a group of electronic devices. Alternatively, in this embodiment, the electronic device may be replaced by a terminal device such as a mobile terminal.
Alternatively, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of the computer network.
In the embodiment, the electronic device can execute program codes of the method for generating the time sequence prediction model, wherein the program codes comprise the steps of obtaining historical target variable data and covariate data sets corresponding to preset prediction tasks, the preset prediction tasks at least comprise time sequence prediction tasks in the power field, the historical target variable data are historical prediction time sequence data, the covariate data sets are sets of data related to prediction of the historical target variable data, determining target covariate information from the covariate data sets, the target covariate information is information influencing the prediction of the historical target variable data, performing feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain fusion feature vectors, and training a pre-training model according to the fusion feature vectors to obtain the target time sequence prediction model corresponding to the preset prediction tasks.
The electronic equipment can execute program codes of the following steps in the generation method of the time sequence prediction model, wherein if the mode of the target covariate information is a numerical value, correlation coefficients between time sequence covariate data in the covariate data set and historical target variable data are calculated respectively to obtain a calculation result, and the target covariate information is determined from the time sequence covariate data based on the calculation result.
The electronic equipment can execute program codes of the following steps in the generation method of the time sequence prediction model, wherein if the mode of the target covariate information is text and/or enumeration, the timestamp information and the event information in the covariate data set are subjected to deviation analysis based on historical target variable data to obtain an analysis result, and the target covariate information is determined from the timestamp information and the event information based on the analysis result.
The electronic equipment can execute program codes of the following steps in the generation method of the time sequence prediction model, wherein the program codes respectively conduct feature embedding processing on the historical target variable data and the target covariate information through a preset embedding module to obtain feature vectors corresponding to the historical target variable data and feature vectors corresponding to the target covariate information, and conduct fusion processing on the feature vectors corresponding to the historical target variable data and the feature vectors corresponding to the target covariate information to obtain fusion feature vectors.
The electronic equipment can execute program codes of the following steps in the generation method of the time sequence prediction model, wherein the first characteristic vector is obtained by carrying out information extraction and embedding processing on historical target variable data through a first embedding module, the second characteristic vector is obtained by carrying out information extraction and embedding processing on timestamp information with a numerical mode in target covariant information through a second embedding module, the third characteristic vector is obtained by carrying out information extraction and embedding processing on the time sequence covariant data with the numerical mode in the target covariant information through a third embedding module, the fourth characteristic vector is obtained by carrying out information extraction and embedding processing on event information with the enumerated mode in the target covariant information through a third embedding module, and the fifth characteristic vector is obtained by carrying out information extraction and embedding processing on event information with the numerical mode in the target covariant information through a fourth embedding module.
The electronic device may further execute program codes for performing fusion processing on the first feature vector, the second feature vector, the third feature vector, the fourth feature vector, and the fifth feature vector to obtain a fused feature vector.
The electronic equipment can also execute the program codes of the following steps in the generation method of the time sequence prediction model, namely, the fusion feature vector is input into the pre-training model, and model parameters of the pre-training model are adjusted according to a preset loss function, so that the target time sequence prediction model is obtained.
Alternatively, fig. 8 is a block diagram of an electronic device according to an embodiment of the present application. As shown in fig. 8, the electronic device 80 may include one or more (only one is shown in fig. 8) processors 802, a memory 804. The electronic device 80 may further include a memory controller, through which the memory 804 is controlled and managed, and a peripheral interface through which the radio frequency module, the audio frequency module, the display screen, and the like are connected to the electronic device 80.
The memory may be used to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for generating a time sequence prediction model in the embodiments of the present application, and the processor executes the software programs and modules stored in the memory, thereby executing various functional applications and data processing, that is, implementing the method for generating a time sequence prediction model. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory located remotely from the processor, which may be connected to the terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The processor can call information and an application program stored in a memory through a transmission device to execute the following steps of obtaining historical target variable data and covariant data sets corresponding to preset prediction tasks, wherein the preset prediction tasks at least comprise time sequence prediction tasks in the electric power field, the historical target variable data are historical prediction time sequence data, the covariant data sets are sets of data related to prediction of the historical target variable data, determining target covariant information from the covariant data sets, wherein the target covariant information is information influencing the prediction of the historical target variable data, performing feature embedding and fusion processing on the historical target variable data and the target covariant information to obtain fusion feature vectors, and training a pre-training model according to the fusion feature vectors to obtain a target time sequence prediction model corresponding to the preset prediction tasks.
Optionally, the processor may further execute program code for calculating correlation coefficients between time sequence covariate data and historical target variable data in the covariate data set, respectively, if the modality of the target covariate information is a numerical value, to obtain a calculation result, and determining the target covariate information from the time sequence covariate data based on the calculation result.
Optionally, the processor may further execute program code for performing a deviation analysis on the timestamp information and the event information in the covariate dataset based on the historical target variable data if the target covariate information is in a text and/or enumeration mode, to obtain an analysis result, and determining the target covariate information from the timestamp information and the event information based on the analysis result.
Optionally, the processor may further execute program codes of performing feature embedding processing on the historical target variable data and the target covariate information through a preset embedding module to obtain feature vectors corresponding to the historical target variable data and feature vectors corresponding to the target covariate information, and performing fusion processing on the feature vectors corresponding to the historical target variable data and the feature vectors corresponding to the target covariate information to obtain fusion feature vectors.
Optionally, the processor may further perform information extraction and embedding processing on the historical target variable data by using a first embedding module to obtain a first feature vector, performing information extraction and embedding processing on timestamp information with a numerical value in the target covariate information by using a second embedding module to obtain a second feature vector, performing information extraction and embedding processing on the time sequence covariate data with a numerical value in the target covariate information by using a third embedding module to obtain a third feature vector, performing information extraction and embedding processing on event information with a enumerated mode in the target covariate information by using a third embedding module to obtain a fourth feature vector, and performing information extraction and embedding processing on event information with a text mode in the target covariate information by using a fourth embedding module to obtain a fifth feature vector.
Optionally, the processor may further execute program code to perform fusion processing on the first feature vector, the second feature vector, the third feature vector, the fourth feature vector, and the fifth feature vector to obtain a fused feature vector.
Optionally, the processor may further execute program code for inputting the fusion feature vector into a pre-training model, and adjusting model parameters of the pre-training model according to a preset loss function to obtain the target time sequence prediction model.
It will be appreciated by those skilled in the art that the structure shown in fig. 8 is only schematic, and the electronic device may also be a terminal device such as a smart phone (e.g. an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile internet device (Mobile INTERNET DEVICES, MID), a PAD, etc. Fig. 8 is not limited to the structure of the electronic device. For example, the electronic device 80 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 8, or have a different configuration than shown in FIG. 8.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program for instructing a terminal device related hardware, and the program may be stored in a computer readable storage medium, where the storage medium may include a flash disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, etc.
Example 7
Embodiments of the present application also provide a computer-readable storage medium. Alternatively, in this embodiment, the storage medium may be used to store the program code executed by the method for generating the time-series prediction model provided in the first embodiment.
Alternatively, in this embodiment, the storage medium may be located in any one of the electronic devices in the group of electronic devices in the computer network, or in any one of the mobile terminals in the group of mobile terminals.
Example 8
Embodiments of the present application also provide a computer program product. Alternatively, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the method for generating a time-series prediction model provided in the first embodiment.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and are merely a logical functional division, and there may be other manners of dividing the apparatus in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present application. The storage medium includes a U disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, etc. which can store the program code.
The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application, which are intended to be comprehended within the scope of the present application.
Claims (13)
1. A method of generating a timing prediction model, comprising:
Acquiring historical target variable data and covariate data sets corresponding to a preset prediction task, wherein the preset prediction task at least comprises a time sequence prediction task in the electric power field, the historical target variable data are historical prediction time sequence data, and the covariate data sets are data sets related to prediction of the historical target variable data;
determining target covariate information from the covariate dataset, wherein the target covariate information is information influencing the prediction of the historical target variable data;
performing feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain a fusion feature vector;
training the pre-training model according to the fusion feature vector to obtain a target time sequence prediction model corresponding to the preset prediction task.
2. The method of claim 1, wherein the modality of the target covariate information is at least one of text, numerical value, enumeration, determining target covariate information from the covariate dataset comprising:
If the mode of the target covariate information is the numerical value, calculating correlation coefficients between time sequence covariate data in the covariate data set and the historical target variable data respectively to obtain a calculation result;
and determining the target covariate information from the time sequence covariate data based on the calculation result.
3. The method of claim 2, wherein determining target covariate information from the covariate dataset comprises:
If the mode of the target covariate information is the text and/or the enumeration, performing deviation analysis on timestamp information and event information in the covariate dataset based on the historical target variable data to obtain an analysis result;
And determining the target covariate information from the timestamp information and the event information based on the analysis result.
4. The method of claim 1, wherein performing feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain a fused feature vector comprises:
Respectively carrying out feature embedding processing on the historical target variable data and the target covariate information through a preset embedding module to obtain feature vectors corresponding to the historical target variable data and feature vectors corresponding to the target covariate information;
And carrying out fusion processing on the feature vector corresponding to the historical target variable data and the feature vector corresponding to the target covariate information to obtain the fusion feature vector.
5. The method of claim 4, wherein the preset embedding module includes a first embedding module, a second embedding module, a third embedding module and a fourth embedding module, and the types of information processed by the first embedding module, the second embedding module, the third embedding module and the fourth embedding module are different, and if the mode of the target covariate information includes text, numerical value and enumeration, performing feature embedding processing on the historical target variable data and the target covariate information through the preset embedding module, respectively, where obtaining feature vectors corresponding to the historical target variable data and feature vectors corresponding to the target covariate information includes:
Information extraction and embedding processing are carried out on the historical target variable data through the first embedding module, so that a first feature vector is obtained;
Performing information extraction and embedding processing on the timestamp information with the numerical mode in the target covariate information through the second embedding module to obtain a second feature vector;
Performing information extraction and embedding processing on time sequence covariate data with a mode being a numerical value in the target covariate information through the third embedding module to obtain a third feature vector, and performing information extraction and embedding processing on event information with a mode being enumeration in the target covariate information through the third embedding module to obtain a fourth feature vector;
And carrying out information extraction and embedding processing on the event information with the text mode in the target covariate information through the fourth embedding module to obtain a fifth feature vector.
6. The method of claim 5, wherein performing fusion processing on the feature vector corresponding to the historical target variable data and the feature vector corresponding to the target covariate information to obtain the fused feature vector comprises:
And carrying out fusion processing on the first feature vector, the second feature vector, the third feature vector, the fourth feature vector and the fifth feature vector to obtain the fusion feature vector.
7. The method of claim 1, wherein training the pre-training model according to the fused feature vector to obtain the target time sequence prediction model corresponding to the preset prediction task comprises:
And inputting the fusion feature vector into the pre-training model, and adjusting model parameters of the pre-training model according to a preset loss function to obtain the target time sequence prediction model.
8. A method of data processing, comprising:
Acquiring historical time sequence data corresponding to a preset prediction task and preset target covariate information, wherein the preset prediction task at least comprises a time sequence prediction task in the electric power field, and the target covariate information is information influencing the prediction of the target prediction time sequence data;
performing fusion processing on the historical time sequence data and the target covariate information through a target time sequence prediction model to obtain a target feature vector, wherein the target time sequence prediction model is obtained by adopting the generation method of the time sequence prediction model according to any one of claims 1 to 7;
And predicting according to the target feature vector to obtain target prediction time sequence data corresponding to the preset prediction task.
9. A method of data processing, comprising:
Acquiring historical time sequence data corresponding to a preset prediction task uploaded by a client and preset target covariate information, wherein the preset prediction task at least comprises a time sequence prediction task in the electric power field, and the target covariate information is information influencing the prediction of the target prediction time sequence data;
Performing fusion processing on the historical time sequence data and the target covariate information through a target time sequence prediction model in a cloud server to obtain a target feature vector, wherein the target time sequence prediction model is obtained by adopting the generation method of the time sequence prediction model according to any one of claims 1 to 7;
and feeding the target prediction time sequence data back to the client.
10. A time series prediction model generation device, comprising:
The power generation system comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring historical target variable data and a covariate data set corresponding to a preset prediction task, the preset prediction task at least comprises a time sequence prediction task in the power field, the historical target variable data is historical prediction time sequence data, and the covariate data set is a set of data related to prediction of the historical target variable data;
a first determining unit configured to determine target covariate information from the covariate dataset, wherein the target covariate information is information that has an influence on prediction of the historical target variable data;
the first processing unit is used for carrying out feature embedding and fusion processing on the historical target variable data and the target covariate information to obtain a fusion feature vector;
And the second processing unit is used for training the pre-training model according to the fusion feature vector to obtain a target time sequence prediction model corresponding to the preset prediction task.
11. A data processing apparatus, comprising:
The second acquisition unit is used for acquiring historical time sequence data corresponding to a preset prediction task and preset target covariate information, wherein the preset prediction task at least comprises a time sequence prediction task in the electric power field, and the target covariate information is information influencing the prediction of the target prediction time sequence data;
A third processing unit, configured to perform fusion processing on the historical time sequence data and the target covariate information through a target time sequence prediction model, to obtain a target feature vector, where the target time sequence prediction model is obtained by adopting the method for generating the time sequence prediction model according to any one of claims 1 to 7;
and the fourth processing unit is used for predicting and obtaining target prediction time sequence data corresponding to the preset prediction task according to the target feature vector.
12. An electronic device, comprising:
a memory storing an executable program;
a processor for executing the program, wherein the program when executed performs the method of generating a time series prediction model according to any one of claims 1 to 7.
13. A computer program product comprising a computer program or instructions which, when executed by a processor, implement a method of generating a time series prediction model according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411783260.0A CN119272821A (en) | 2024-12-05 | 2024-12-05 | Time sequence prediction model generation method, data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411783260.0A CN119272821A (en) | 2024-12-05 | 2024-12-05 | Time sequence prediction model generation method, data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119272821A true CN119272821A (en) | 2025-01-07 |
Family
ID=94109788
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411783260.0A Pending CN119272821A (en) | 2024-12-05 | 2024-12-05 | Time sequence prediction model generation method, data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119272821A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115440032A (en) * | 2022-08-09 | 2022-12-06 | 浙江工业大学 | A long-term and short-term public traffic flow forecasting method |
| US11704576B1 (en) * | 2020-01-29 | 2023-07-18 | Arva Intelligence Corp. | Identifying ground types from interpolated covariates |
| CN117937425A (en) * | 2023-12-04 | 2024-04-26 | 西南交通大学 | A method and system for load forecasting of substation equipment based on segmented attention mechanism |
| CN118377979A (en) * | 2024-06-24 | 2024-07-23 | 杭州电子科技大学 | Interest point recommendation method and medium based on time sequence knowledge graph and electronic equipment |
| CN119005747A (en) * | 2024-08-05 | 2024-11-22 | 中国电信股份有限公司 | Carbon emission amount determination method, device, electronic equipment and nonvolatile storage medium |
-
2024
- 2024-12-05 CN CN202411783260.0A patent/CN119272821A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11704576B1 (en) * | 2020-01-29 | 2023-07-18 | Arva Intelligence Corp. | Identifying ground types from interpolated covariates |
| CN115440032A (en) * | 2022-08-09 | 2022-12-06 | 浙江工业大学 | A long-term and short-term public traffic flow forecasting method |
| CN117937425A (en) * | 2023-12-04 | 2024-04-26 | 西南交通大学 | A method and system for load forecasting of substation equipment based on segmented attention mechanism |
| CN118377979A (en) * | 2024-06-24 | 2024-07-23 | 杭州电子科技大学 | Interest point recommendation method and medium based on time sequence knowledge graph and electronic equipment |
| CN119005747A (en) * | 2024-08-05 | 2024-11-22 | 中国电信股份有限公司 | Carbon emission amount determination method, device, electronic equipment and nonvolatile storage medium |
Non-Patent Citations (4)
| Title |
|---|
| 中国计算机学会编: "CCF2020-2021中国计算机科学技术发展报告", 31 January 2022, 机械工业出版社, pages: 344 * |
| 姜明: "基于Web文本挖掘的在线评论情感分析研究", 30 April 2023, 东北财经大学出版社, pages: 216 - 217 * |
| 庞双子: "透过翻译的语言接触研究", 30 June 2022, 商务印书馆, pages: 155 - 156 * |
| 康志峰主编: "翻译研究与教学", 31 October 2022, 复旦大学出版社, pages: 103 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11244402B2 (en) | Prediction algorithm based attribute data processing | |
| CN111091196B (en) | Passenger flow data determination method and device, computer equipment and storage medium | |
| CN112256886B (en) | Probability calculation method and device in atlas, computer equipment and storage medium | |
| CN108133390A (en) | For predicting the method and apparatus of user behavior and computing device | |
| CN114971748B (en) | Prediction data generation method, model training method, computer device, and storage medium | |
| CN111178537B (en) | Feature extraction model training method and device | |
| CN113822460A (en) | Traffic flow prediction method and device, electronic equipment and storage medium | |
| CN111177568A (en) | Object pushing method based on multi-source data, electronic device and storage medium | |
| CN113761375A (en) | Message recommendation method, device, equipment and storage medium based on neural network | |
| CN112269875B (en) | Text classification method, device, electronic equipment and storage medium | |
| CN111652282B (en) | Big data-based user preference analysis method and device and electronic equipment | |
| CN109741108A (en) | Streaming application recommended method, device and electronic equipment based on context aware | |
| CN115757075A (en) | Task abnormity detection method and device, computer equipment and storage medium | |
| CN112651782B (en) | Behavior prediction method, device, equipment and medium based on dot product attention scaling | |
| CN107644042B (en) | Software program click rate pre-estimation sorting method and server | |
| CN119272821A (en) | Time sequence prediction model generation method, data processing method and device | |
| CN113139129B (en) | Virtual reading trajectory graph generation method and device, electronic equipment and storage medium | |
| CN117251781A (en) | Electric power auditing method and device based on data analysis and storage medium | |
| CN116861079A (en) | Ordering recommendation method based on long-term preference of user and related equipment thereof | |
| CN110874612A (en) | Time interval prediction method and device, computer equipment and storage medium | |
| CN114358186B (en) | A data processing method, device and computer readable storage medium | |
| CN116523622A (en) | Object risk prediction method and device, electronic equipment and storage medium | |
| CN115860835A (en) | Advertisement recommendation method, device and equipment based on artificial intelligence and storage medium | |
| CN114219663A (en) | Product recommendation method and device, computer equipment and storage medium | |
| CN116416018A (en) | Content output method, content output device, computer readable medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |