CN115168509A - Processing method and device of wind control data, storage medium and computer equipment - Google Patents
Processing method and device of wind control data, storage medium and computer equipment Download PDFInfo
- Publication number
- CN115168509A CN115168509A CN202210846082.6A CN202210846082A CN115168509A CN 115168509 A CN115168509 A CN 115168509A CN 202210846082 A CN202210846082 A CN 202210846082A CN 115168509 A CN115168509 A CN 115168509A
- Authority
- CN
- China
- Prior art keywords
- wind control
- data
- original
- processing
- wind
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Strategic Management (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Operations Research (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method and a device for processing wind control data, a storage medium and computer equipment, and relates to the technical field of Internet development. The main aim at solves the problem that the wind control data processing efficiency is low. The method mainly comprises the steps of obtaining a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin; performing pre-wind control processing on the original basic data based on the wind control task to generate original wind control data, and storing the original wind control data into a second data bin associated with the first data bin; and when a wind control processing request is received, acquiring the original wind control data from the second data bin, and processing the original wind control data based on the wind control indexes carried in the wind control processing request to obtain a wind control result. The method is mainly used for processing the wind control data.
Description
Technical Field
The invention relates to the technical field of internet development, in particular to a method and a device for processing wind control data, a storage medium and computer equipment.
Background
In order to ensure that different business applications can run safely and normally, wind control measures need to be taken to monitor the acquired big data in real time. For the service applied to the internet, the internet equipment can continuously generate data according to a set period or triggered by an external event, that is, each data is generated at a time point, and at this time, the data volume is only increased along with the increase of the internet-of-things equipment.
At present, the existing internet equipment generally directly stores the acquired wind control data in a distributed database, and when wind control judgment is performed, the wind control data is called from the distributed database according to wind control requirements for analysis and calculation. However, as the wind control demand increases, the processing pressure of system resources is increased by directly calling data, and a large number of wind control numbers reduce the speed of wind control processing, which cannot meet the requirement of accurate identification in a wind control scene.
Disclosure of Invention
In view of the above, the present invention provides a method and an apparatus for processing wind control data, a storage medium, and a computer device, and mainly aims to solve the problem of low processing efficiency of the existing wind control data.
According to an aspect of the present invention, there is provided a method for processing wind control data, including:
acquiring a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin;
performing pre-wind control processing on the original basic data based on the wind control task to generate original wind control data, and storing the original wind control data into a second data bin associated with the first data bin;
and when a wind control processing request is received, acquiring the original wind control data from the second data bin, and processing the original wind control data based on a wind control index carried in the wind control processing request to obtain a wind control result.
Further, the performing pre-wind control processing on the original basic data based on the wind control task, and generating original wind control data includes:
analyzing a wind control index in the wind control task, wherein the wind control index is used for representing an object which is expected to be subjected to operation processing according to data characteristics during wind control monitoring;
and determining a preposed wind control node matched with the wind control index, and carrying out preposed wind control processing on the original basic data according to the preposed wind control node to generate original wind control data.
Further, prior to storing the raw wind control data in a second data bin associated with the first data bin, the method further comprises:
determining at least one to-be-selected data bin to be associated, wherein the data storage format, the data storage amount and the data updating time interval of the to-be-selected data bin are matched with the wind control indexes;
randomly selecting a preset number of data bins to be selected, determining that a second data bin is associated with the first data bin, and storing the original wind control data into the second data bin; or the like, or, alternatively,
and sorting according to the data storage amount of the data bins to be selected, selecting the data bin to be selected with the largest data storage amount as a second data bin to be associated with the first data bin, and storing the original wind control data into the second data bin based on the original wind control data.
Further, before the obtaining of the wind control task expected to perform the wind control processing and the original base data with the wind control tag in the first data bin, the method further comprises:
when the original basic data are stored in the first data bin, determining at least one wind control task to be subjected to wind control processing, and determining a wind control label for the wind control task, wherein different wind control tasks are configured with matched wind control labels;
and marking the wind control label on the original basic data according to the wind control object in the wind control task to obtain the original basic data with the wind control label.
Further, the acquiring the wind control task expected to perform the wind control processing, and the raw base data with the wind control tag in the first data bin includes:
calling a wind control task triggered to be executed from a task manager according to a first preset time interval, and determining a wind control label based on a wind control object of the wind control task;
when the wind control label is inquired in the first data bin, calling original basic data marked with the wind control label from the first data bin;
and when the wind control label is not inquired in the first data bin, calling original basic data of an associated wind control label, of which the similarity with the wind control label meets a preset threshold value, from the first data bin.
Further, the obtaining the original wind control data from the second data bin, and processing the original wind control data based on the wind control indexes carried in the wind control processing request to obtain a wind control result includes:
analyzing a front wind control node matched with the wind control index, and determining a residual wind control node based on the front wind control node, wherein the front wind control node and the residual wind control node form a complete wind control process node;
and carrying out wind control processing on the original wind control data based on wind control processing strategies in the residual wind control nodes to generate a wind control result, wherein the wind control processing strategies comprise at least one of data operation, rule judgment and model prediction.
Further, the method further comprises:
after the original basic data in the first data bin are updated according to a second preset time interval, carrying out wind control label marking on the original basic data again, and obtaining the updated original wind control data again;
and updating the updated original wind control data to the associated second data bin, wherein the original wind control data in the second data bin is emptied according to the second preset time interval.
According to another aspect of the present invention, there is provided a wind-controlled data processing apparatus, comprising:
the acquisition module is used for acquiring a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin;
the first processing module is used for carrying out preposed wind control processing on the original basic data based on the wind control task to generate original wind control data and storing the original wind control data into a second data bin associated with the first data bin;
and the second processing module is used for acquiring the original wind control data from the second data bin when a wind control processing request is received, and processing the original wind control data based on the wind control indexes carried in the wind control processing request to obtain a wind control result.
Further, the first processing module comprises:
the first analysis unit is used for analyzing wind control indexes in the wind control task, and the wind control indexes are used for representing objects which are expected to be subjected to operation processing according to data characteristics during wind control monitoring;
and the first processing unit is used for determining a preposed wind control node matched with the wind control index, and performing preposed wind control processing on the original basic data according to the preposed wind control node to generate original wind control data.
Further, the apparatus further comprises:
the matching module is used for determining at least one to-be-selected data bin to be associated, and the data storage format, the data storage amount and the data updating time interval of the to-be-selected data bin are matched with the wind control indexes;
the first association module is used for randomly selecting a preset number of data bins to be selected, determining that a second data bin is associated with the first data bin, and storing the original wind control data into the second data bin; or the like, or, alternatively,
and the second association module is used for sequencing according to the data storage amount of the data bins to be selected, selecting the data bin to be selected with the largest data storage amount as a second data bin to be associated with the first data bin, and storing the original wind control data into the second data bin based on the second association module.
Further, the apparatus further comprises:
the determining module is used for determining at least one wind control task to be subjected to wind control processing when the original basic data is stored in the first data bin, and determining wind control labels for the wind control tasks, wherein different wind control tasks are configured with matched wind control labels;
and the marking module is used for marking the wind control label on the original basic data according to the wind control object in the wind control task to obtain the original basic data with the wind control label.
Further, the acquisition module comprises:
the system comprises a determining unit, a task manager and a control unit, wherein the determining unit is used for calling a wind control task triggered to be executed from the task manager according to a first preset time interval and determining a wind control label based on a wind control object of the wind control task;
the first calling unit is used for calling original basic data marked with the wind control label from the first data bin when the wind control label is inquired in the first data bin;
and the second calling unit is used for calling the original basic data of the associated wind control label, of which the similarity with the wind control label meets a preset threshold value, from the first data bin when the wind control label is not inquired in the first data bin.
Further, the second processing module comprises:
the second analysis unit is used for analyzing the preposed wind control nodes matched with the wind control indexes and determining the residual wind control nodes based on the preposed wind control nodes, wherein the preposed wind control nodes and the residual wind control nodes form a complete wind control process node;
and the second processing unit is used for carrying out wind control processing on the original wind control data based on wind control processing strategies in the residual wind control nodes to generate a wind control result, wherein the wind control processing strategies comprise at least one of data operation, rule judgment and model prediction.
Further, the apparatus further comprises:
the first updating module is used for re-marking the original basic data by the wind control label after the original basic data in the first data bin is updated according to a second preset time interval, and re-obtaining the updated original wind control data;
and the second updating module is used for updating the updated original wind control data to the associated second data bin, wherein the original wind control data in the second data bin is emptied according to the second preset time interval.
According to another aspect of the present invention, a storage medium is provided, where at least one executable instruction is stored, and the executable instruction causes a processor to perform an operation corresponding to the processing method of the wind control data.
According to still another aspect of the present invention, there is provided a computer apparatus including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the processing method of the wind control data.
By means of the technical scheme, the technical scheme provided by the embodiment of the invention at least has the following advantages:
the invention provides a method and a device for processing wind control data, a storage medium and computer equipment, wherein the wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin are obtained; performing pre-wind control processing on the original basic data based on the wind control task to generate original wind control data, and storing the original wind control data into a second data bin associated with the first data bin; when a wind control processing request is received, the original wind control data are obtained from the second data bin, the original wind control data are processed based on wind control indexes carried in the wind control processing request, a wind control result is obtained, and the wind control data are subjected to preposed wind control processing before the wind control processing request is received, so that the data processing difficulty and the data processing amount after the wind control processing request is received are greatly reduced, the data processing accuracy is guaranteed, the wind control data processing time is shortened, and the wind control data processing efficiency is effectively improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a flowchart illustrating a method for processing wind control data according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating another method for processing wind control data according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for processing wind control data according to another embodiment of the present invention;
fig. 4 is a block diagram illustrating a processing apparatus for processing wind control data according to an embodiment of the present invention;
fig. 5 shows a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The method comprises the steps that the acquired wind control data are usually and directly stored in a distributed database aiming at the existing Internet equipment, and when wind control judgment is carried out, the wind control data are called from the distributed database according to wind control requirements to be analyzed and calculated. However, as the wind control demand increases, the direct access to data increases the system resource processing pressure, and a large number of wind control data reduces the speed of wind control processing, which cannot meet the problem of accurate identification demand in a wind control scene. An embodiment of the present invention provides a method for processing wind control data, and as shown in fig. 1, the method includes:
101. the method comprises the steps of obtaining a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin.
In the embodiment of the invention, the wind control task can be a task of obtaining data for representing the risk degree or guiding the risk control based on the calculation and statistics of related data in any application scene. For example, user abnormal behavior control, bank transaction limit risk control, enterprise production and marketing risk control and the like. The task content and the task number of the wind control task which is expected to execute the wind control processing are determined based on a specific application scene, and can be customized according to specific requirements, and the embodiment of the invention is not specifically limited. The first data warehouse is used for storing all original basic data related to the wind control task, and may be a database, and may also be an information publishing component with a storage function, such as kafka, and the embodiment of the present invention is not particularly limited. The wind control label is a label which has a mapping relation with a wind control task expected to execute wind control processing. And the method is used for marking the corresponding relation between the current original basic data and which wind control tasks. The original basic data may be process data acquired from a website backend system or an enterprise operation system, for example, a user operation log, enterprise operation data, and the like, and the embodiment of the present invention is not particularly limited.
It should be noted that, original basic data are marked by using the wind control tag, and the original basic data with the marks are stored in the designated data bin, so that the data calling process is more convenient, the called data content is more accurate, and the efficiency and accuracy of data calling are improved.
102. And performing pre-wind control processing on the original basic data based on the wind control task to generate original wind control data, and storing the original wind control data into a second data bin associated with the first data bin.
In the embodiment of the invention, the original basic data is the original data acquired from the system, such as user browsing data, user operation logs and the like, and the data is more discrete and irregular and contains more repeated data and abnormal data. Therefore, the preliminary wind control processing is performed on the original basic data, and the method of the preliminary wind control processing may specifically be performing rule checking, deduplication processing, data cleaning, data aggregation processing, and the like on the original basic data, and the embodiment of the present invention is not particularly limited. After the primary basic data is subjected to the pre-wind control processing, the primary basic data subjected to the pre-wind control processing is stored in a second data bin for the convenience of calling and updating the data. The second data warehouse may be a storage system constructed based on a distributed system infrastructure, or may be a storage system constructed based on other system frameworks, which is not specifically limited in the embodiment of the present invention.
It should be noted that the second data bin is a data bin that is in an association relationship with the first data bin. For example, as data in a first data bin is updated, a second data bin may be updated synchronously with its stored content based on the association. By constructing the incidence relation between the first data bin and the second data bin, the synchronous updating of the data can be realized, and the validity and the accuracy of the wind control data are ensured. In addition, the original basic data are subjected to the preposed wind control processing, so that the interference data in the original basic data can be removed, the effectiveness of the data is improved, the data processing difficulty and the data processing amount of subsequent wind control processing nodes are reduced, and the processing efficiency of the whole wind control data is improved.
103. And when a wind control processing request is received, acquiring the original wind control data from the second data bin, and processing the original wind control data based on the wind control indexes carried in the wind control processing request to obtain a wind control result.
In the embodiment of the present invention, the wind control index is an object to be risk evaluated, and may include a preset condition for determining a risk state represented by the original wind control data, a preset threshold, a preset mapping relationship list, or the like. For example, in a user behavior abnormality control scenario, the wind control index is a mapping relation list of the number of times of user abnormal behaviors in the same IP address and the user behavior abnormality level. The specific content of the wind control index can be customized according to the actual application requirement, and the embodiment of the invention is not particularly limited. And processing the original wind control data based on the wind control indexes, namely extracting the risk state characteristics represented by the wind control data based on the wind control indexes to obtain a risk evaluation result. For example, in a bank loan business risk control scene, if the upper limit of loan in one week is 100 thousands for the wind control index, and the original wind control data is 180 thousands, the wind control result of the current loan amount is determined to be a high risk according to the wind control index.
It should be noted that, the wind control data is processed in advance before the wind control processing request is received. When a wind control processing request is received, the original wind control data only needs to be processed according to the carried wind control instruction, and a wind control result can be obtained. A large amount of data processing, complex data analysis and calculation are not needed, the response time of the wind control processing request is shortened, and when a large amount of wind control processing requests face, the accuracy and the speed of wind control result output can be guaranteed, so that the risk data processing efficiency is effectively improved.
For further explanation and limitation, in an embodiment of the present invention, as shown in fig. 2, the performing, in step 102, pre-wind control processing on the original basic data based on the wind control task, and generating original wind control data includes:
201. and analyzing the wind control indexes in the wind control task.
202. And determining a preposed wind control node matched with the wind control index, and carrying out preposed wind control processing on the original basic data according to the preposed wind control node to generate original wind control data.
In the embodiment of the invention, the wind control indexes obtained by analysis in the wind control task are used for representing objects which are expected to be operated and processed according to data characteristics in the wind control monitoring process. The analysis process of the wind control task can be matching based on preset wind control index keywords. For example, the similarity or the vector distance between the content of the wind control task and the preset wind control index keyword is calculated, and the embodiment of the present invention is not particularly limited. And dividing the processing process of the wind control data into a plurality of wind control nodes according to different data processing degrees. And all the wind control processing nodes which do not obtain the final wind control result can be used as preposed wind control nodes. After the wind control indexes are obtained through analysis according to the wind control tasks, the wind control indexes are matched with at least one preposed wind control node based on the wind control indexes, the preposed wind control node corresponding to the current wind control task is determined, and then the original basic data are processed according to a preposed wind control processing algorithm corresponding to the preposed wind control node. The pre-wind control processing algorithm can be customized according to a specific application scene, and embodiments of the present invention are not particularly limited.
It should be noted that, the processing process of the wind control data is divided into a plurality of wind control nodes according to different data processing degrees, each wind control node corresponds to different wind control processing strategies, so that different original wind control data are obtained, the diversity of the original wind control data can be increased, and different requirements on the wind control data are met.
In an embodiment of the present invention, for further explanation and definition, before storing the raw wind control data in a second data bin associated with the first data bin, the method further comprises:
at least one candidate data bin to be associated is determined.
Randomly selecting a preset number of data bins to be selected, determining that a second data bin is associated with the first data bin, and storing the original wind control data into the second data bin; or the like, or a combination thereof,
and sorting according to the data storage quantity of the data bins to be selected, selecting the data bin to be selected with the largest data storage quantity as a second data bin to be associated with the first data bin, and storing the original wind control data into the second data bin.
In the embodiment of the invention, because the original wind control data has diversity in the aspects of storage format, storage amount, data updating time interval and the like, matched data bins to be selected need to be determined from different types of data bins based on wind control indexes so as to realize classified storage according to the data characteristics of the original wind control data. After determining the candidate data bins, the number of the candidate data bins may be set in order to relieve the storage pressure when the data storage amount is large. The number of the data bins to be selected can be customized according to the actual application requirements, and the embodiment of the invention is not particularly limited. The method for determining the second data bin from the data bins to be selected may be a random selection method, or may be a method for ranking the storage capacity of the data bins to be selected, and the data bin to be selected with the largest data storage capacity is preferably selected.
It should be noted that the data storage format, the data storage amount, and the data update time interval of the data bin to be selected are matched with the wind control index. For example, the update cycle of the wind control indicator is one month, and a data bin with a data update time interval greater than or equal to one month needs to be selected as the candidate data bin. The original wind control data storage data bin is selected based on the wind control index, so that the matching degree of the data bin and the original wind control data can be improved, and the safety and the reliability of the original wind control data storage are guaranteed.
For further explanation and limitation, in an embodiment of the present invention, as shown in fig. 3, before the obtaining 101 of the wind-controlled task that is expected to perform the wind-controlled processing and the raw base data with the wind-controlled tag in the first data bin, the method further includes:
301. and when the original basic data is stored in the first data bin, determining at least one wind control task to be subjected to wind control processing, and determining a wind control label for the wind control task.
302. And marking the wind control label on the original basic data according to the wind control object in the wind control task to obtain the original basic data with the wind control label.
In the embodiment of the invention, after the original basic data is obtained, the type of the wind control processing which needs to be carried out on the original basic data is determined, and the mark is extracted and carried out. It is required to acquire a wind control task expected to be executed and set a wind control tag for the wind control task. And marking the original basic data matched with the wind control object in the corresponding wind control task by using the wind control label. And different wind control tasks are configured with matched wind control labels. For example, if the wind-controlled task is a loan amount wind-controlled task, then the wind-controlled tag may be the loan time, the borrower information, the loan amount, or a loan association. The wind control object is the content which needs to execute the wind control processing in the wind control task. For example, in the user abnormal behavior control scenario, the wind control objects are user operation time, user IP, user operation times, and the like. The wind control tag, the setting mode of the wind control object and the setting precision can be customized according to specific application requirements, and embodiments of the invention are not particularly limited.
For further explanation and limitation, step 103 of obtaining a wind control task that is expected to perform a wind control process, and the raw base data with a wind control tag in the first data bin includes:
calling a wind control task triggered to be executed from a task manager according to a first preset time interval, and determining a wind control label based on a wind control object of the wind control task;
when the wind control label is inquired in the first data bin, calling original basic data marked with the wind control label from the first data bin;
and when the wind control label is not inquired in the first data bin, original basic data of an associated wind control label, of which the similarity with the wind control label meets a preset threshold value, are called from the first data bin.
In the embodiment of the invention, the task manager stores at least one wind control task to be executed. Because the wind control evaluation is mostly based on data of a period of time for analysis processing, the wind control task to be executed is called according to a preset time interval. The preset time interval is determined based on a wind control scene, for example, the data output frequency of the abnormal operation behavior of the user is high, the data volume is large, and the preset time interval can be one hour or one natural day; the preset time interval of the data of the financial transaction amount category may be a natural month or a quarter, and the embodiment of the present invention is not limited in particular. And determining the wind control label based on the wind control object of the wind control task because the wind control label of the original basic data is marked based on the matching result of the wind control object. However, the original basic data has a condition that a plurality of wind control tasks are used in a cross mode, and the condition that the wind control tag in the first data bin is inquired and the wind control tag of the current wind control task is not matched exists, so that when the wind control tag is not inquired, the associated wind control tag which is strongly associated with the current wind control tag can be determined from the first data bin based on similarity calculation, and the original basic data corresponding to the associated wind control tag is used as a wind control processing object. The similarity calculation method may be a cosine similarity calculation method, and the embodiment of the present invention is not particularly limited.
In an embodiment of the present invention, for further explanation and limitation, the obtaining the original wind control data from the second data warehouse, and processing the original wind control data based on a wind control indicator carried in the wind control processing request to obtain a wind control result includes:
and analyzing the preposed wind control node matched with the wind control index, and determining the residual wind control node based on the preposed wind control node.
And carrying out wind control processing on the original wind control data based on the wind control processing strategies in the rest wind control nodes to generate a wind control result.
In the embodiment of the invention, the preposed wind control node and the rest wind control nodes form a complete wind control process node, namely, in the complete process node of the wind control processing, the completed wind control node is the preposed wind control node, and the unfinished wind control node is the rest wind control node. When a wind control processing request is received, in order to determine a wind control processing strategy, which flow node of the wind control flow nodes corresponds to the original wind control data needing wind control processing at present is determined based on the wind control index. And then processing the original wind control data according to the wind control processing strategies in the rest process nodes to obtain a final result of the wind control processing, namely a wind control result. The wind control processing strategy comprises at least one of data operation, rule judgment and model prediction. The data operation can be operation processing such as function operation, gaussian distribution operation, interpolation operation and the like on the original wind control data; the rule judgment can be to judge the original wind control data based on the corresponding relation between a preset judgment rule and a judgment result; the model prediction can be used for predicting the original wind control data based on a prediction model to obtain an evaluation result of the expected risk, and the embodiment of the invention does not specifically limit the content of the wind control processing strategy. The wind control result may include a risk level, a risk prediction level, and the like corresponding to the wind control object, and the embodiment of the present invention is specifically limited. The decoupling between each process node in the wind control process nodes can be realized by establishing the corresponding relation between the wind control processing strategy and the wind control process nodes, so that the flexibility of the wind control processing process is effectively improved.
In an embodiment of the present invention, for further explanation and limitation, the method further comprises:
and after the original basic data in the first data bin are updated according to a second preset time interval, carrying out wind control label marking on the original basic data again, and obtaining the updated original wind control data again.
Updating the updated original wind control data to the associated second data bin.
In the embodiment of the invention, in order to realize synchronous updating of the first data bin and the second data bin, after the original basic data is updated according to the second preset time interval, the original wind control data in the second data bin is updated according to the updating content of the original basic data. The implementation process of updating the original wind control data according to the updated original basic data is the same as the implementation process of performing pre-wind control processing according to the original basic data to obtain the original wind control data, and the embodiment of the invention is not described herein again. In order to delete invalid data in time, realize effective utilization of the storage space of the data warehouse and guarantee the accuracy of the original wind control data, the original wind control data in the second data warehouse is emptied according to a second preset time interval.
The invention provides a method for processing wind control data, which comprises the steps of obtaining a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin; performing pre-wind control processing on the original basic data based on the wind control task to generate original wind control data, and storing the original wind control data into a second data bin associated with the first data bin; when a wind control processing request is received, the original wind control data are obtained from the second data bin, the original wind control data are processed based on wind control indexes carried in the wind control processing request, a wind control result is obtained, and the wind control data are subjected to preposed wind control processing before the wind control processing request is received, so that the data processing difficulty and data processing amount after the wind control processing request is received are greatly reduced, the data processing accuracy is guaranteed, the wind control data processing time is shortened, and the wind control data processing efficiency is effectively improved.
Further, as an implementation of the method shown in fig. 1, an embodiment of the present invention provides a processing apparatus for wind control data, as shown in fig. 4, the apparatus includes:
an obtaining module 41, configured to obtain a wind control task that is expected to perform wind control processing, and original basic data with a wind control tag in a first data warehouse;
a first processing module 42, configured to perform pre-wind control processing on the original basic data based on the wind control task, generate original wind control data, and store the original wind control data in a second data bin associated with the first data bin;
and the second processing module 43 is configured to, when a wind control processing request is received, obtain the original wind control data from the second data warehouse, and process the original wind control data based on a wind control index carried in the wind control processing request to obtain a wind control result.
Further, the first processing module 42 includes:
the first analysis unit is used for analyzing the wind control indexes in the wind control task, and the wind control indexes are used for representing objects which are expected to be operated according to data characteristics during wind control monitoring;
and the first processing unit is used for determining a preposed wind control node matched with the wind control index, and carrying out preposed wind control processing on the original basic data according to the preposed wind control node to generate original wind control data.
Further, the apparatus further comprises:
the matching module is used for determining at least one to-be-selected data bin to be associated, and the data storage format, the data storage amount and the data updating time interval of the to-be-selected data bin are matched with the wind control indexes;
the first association module is used for randomly selecting a preset number of data bins to be selected, determining that a second data bin is associated with the first data bin, and storing the original wind control data into the second data bin; or the like, or, alternatively,
and the second association module is used for sorting according to the data storage quantity of the data bins to be selected, selecting the data bin to be selected with the largest data storage quantity as a second data bin to be associated with the first data bin, and storing the original wind control data into the second data bin based on the second association module.
Further, the apparatus further comprises:
the determining module is used for determining at least one wind control task to be subjected to wind control processing when the original basic data is stored in the first data bin, and determining wind control labels for the wind control tasks, wherein different wind control tasks are configured with matched wind control labels;
and the marking module is used for marking the wind control label on the original basic data according to the wind control object in the wind control task to obtain the original basic data with the wind control label.
Further, the obtaining module 41 includes:
the system comprises a determining unit, a task manager and a processing unit, wherein the determining unit is used for calling a wind control task triggered to be executed from the task manager according to a first preset time interval and determining a wind control label based on a wind control object of the wind control task;
the first calling unit is used for calling original basic data marked with the wind control label from the first data bin when the wind control label is inquired in the first data bin;
and the second calling unit is used for calling original basic data of the associated wind control label, of which the similarity with the wind control label accords with a preset threshold value, from the first data bin when the wind control label is not inquired in the first data bin.
Further, the second processing module 43 includes:
the second analysis unit is used for analyzing the preposed wind control nodes matched with the wind control indexes and determining the residual wind control nodes based on the preposed wind control nodes, wherein the preposed wind control nodes and the residual wind control nodes form a complete wind control process node;
and the second processing unit is used for carrying out wind control processing on the original wind control data based on wind control processing strategies in the residual wind control nodes to generate a wind control result, wherein the wind control processing strategies comprise at least one of data operation, rule judgment and model prediction.
Further, the apparatus further comprises:
the first updating module is used for carrying out wind control label marking on the original basic data again after the original basic data in the first data bin is updated according to a second preset time interval, and obtaining the updated original wind control data again;
and the second updating module is used for updating the updated original wind control data to the associated second data bin, wherein the original wind control data in the second data bin is emptied according to the second preset time interval.
The invention provides a processing device of wind control data, which is characterized in that a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin are obtained; performing pre-wind control processing on the original basic data based on the wind control task to generate original wind control data, and storing the original wind control data into a second data bin associated with the first data bin; when a wind control processing request is received, the original wind control data are obtained from the second data bin, the original wind control data are processed based on wind control indexes carried in the wind control processing request, a wind control result is obtained, and the wind control data are subjected to preposed wind control processing before the wind control processing request is received, so that the data processing difficulty and data processing amount after the wind control processing request is received are greatly reduced, the data processing accuracy is guaranteed, the wind control data processing time is shortened, and the wind control data processing efficiency is effectively improved.
According to an embodiment of the present invention, a storage medium is provided, where at least one executable instruction is stored, and the computer executable instruction may execute the processing method of the wind control data in any of the above method embodiments.
Fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the computer device.
As shown in fig. 5, the computer apparatus may include: a processor (processor) 502, a Communications Interface 504, a memory 506, and a communication bus 508.
Wherein: the processor 502, communication interface 504, and memory 506 communicate with one another via a communication bus 508.
A communication interface 504 for communicating with network elements of other devices, such as clients or other servers.
The processor 502 is configured to execute the program 510, and may specifically execute relevant steps in the above-described embodiment of the processing method of the wind control data.
In particular, program 510 may include program code that includes computer operating instructions.
The processor 502 may be a central processing unit CPU, or an Application Specific Integrated Circuit ASIC (Application Specific Integrated Circuit), or one or more Integrated circuits configured to implement an embodiment of the present invention. The computer device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 506 for storing a program 510. The memory 506 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 510 may specifically be used to cause the processor 502 to perform the following operations:
acquiring a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin;
performing pre-wind control processing on the original basic data based on the wind control task to generate original wind control data, and storing the original wind control data into a second data bin associated with the first data bin;
and when a wind control processing request is received, acquiring the original wind control data from the second data bin, and processing the original wind control data based on the wind control indexes carried in the wind control processing request to obtain a wind control result.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for processing wind control data is characterized by comprising the following steps:
acquiring a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin;
performing pre-wind control processing on the original basic data based on the wind control task to generate original wind control data, and storing the original wind control data into a second data bin associated with the first data bin;
and when a wind control processing request is received, acquiring the original wind control data from the second data bin, and processing the original wind control data based on a wind control index carried in the wind control processing request to obtain a wind control result.
2. The method of claim 1, wherein the pre-wind-control processing of the raw base data based on the wind control task, and generating raw wind control data comprises:
analyzing a wind control index in the wind control task, wherein the wind control index is used for representing an object expected to be subjected to operation processing according to data characteristics in wind control monitoring;
and determining a preposed wind control node matched with the wind control index, and carrying out preposed wind control processing on the original basic data according to the preposed wind control node to generate original wind control data.
3. The method of claim 2, wherein prior to storing the raw wind control data into a second data bin associated with the first data bin, the method further comprises:
determining at least one to-be-selected data bin to be associated, wherein the data storage format, the data storage amount and the data updating time interval of the to-be-selected data bin are matched with the wind control indexes;
randomly selecting a preset number of data bins to be selected, determining that a second data bin is associated with the first data bin, and storing the original wind control data into the second data bin; or the like, or, alternatively,
and sorting according to the data storage amount of the data bins to be selected, selecting the data bin to be selected with the largest data storage amount as a second data bin to be associated with the first data bin, and storing the original wind control data into the second data bin based on the original wind control data.
4. The method of claim 1, wherein prior to obtaining the wind-control task expected to perform the wind-control process and the raw base data with the wind-control tag in the first data bin, the method further comprises:
when the original basic data are stored in the first data bin, determining at least one wind control task to be subjected to wind control processing, and determining wind control labels for the wind control tasks, wherein different wind control tasks are configured with matched wind control labels;
and marking the wind control label on the original basic data according to the wind control object in the wind control task to obtain the original basic data with the wind control label.
5. The method of claim 4, wherein obtaining wind-control tasks expected to perform wind-control processing and raw base data with wind-control tags in a first data bin comprises:
calling a wind control task triggered to be executed from a task manager according to a first preset time interval, and determining a wind control label based on a wind control object of the wind control task;
when the wind control label is inquired in the first data bin, calling original basic data marked with the wind control label from the first data bin;
and when the wind control label is not inquired in the first data bin, calling original basic data of an associated wind control label, of which the similarity with the wind control label meets a preset threshold value, from the first data bin.
6. The method of claim 1, wherein the obtaining the original wind control data from the second data warehouse and processing the original wind control data based on a wind control indicator carried in the wind control processing request to obtain a wind control result comprises:
analyzing a front wind control node matched with the wind control index, and determining a residual wind control node based on the front wind control node, wherein the front wind control node and the residual wind control node form a complete wind control process node;
and carrying out wind control processing on the original wind control data based on wind control processing strategies in the residual wind control nodes to generate a wind control result, wherein the wind control processing strategies comprise at least one of data operation, rule judgment and model prediction.
7. The method of any one of claims 1-6, further comprising:
after the original basic data in the first data bin are updated according to a second preset time interval, carrying out wind control label marking on the original basic data again, and obtaining the updated original wind control data again;
and updating the updated original wind control data to the associated second data bin, wherein the original wind control data in the second data bin is emptied according to the second preset time interval.
8. A device for processing wind control data, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a wind control task expected to execute wind control processing and original basic data with a wind control label in a first data bin;
the first processing module is used for carrying out preposed wind control processing on the original basic data based on the wind control task to generate original wind control data and storing the original wind control data into a second data bin associated with the first data bin;
and the second processing module is used for acquiring the original wind control data from the second data bin when a wind control processing request is received, and processing the original wind control data based on a wind control index carried in the wind control processing request to obtain a wind control result.
9. A storage medium, wherein at least one executable instruction is stored, and the executable instruction causes a processor to execute an operation corresponding to the processing method of the wind control data according to any one of claims 1-7.
10. A computer device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the processing method of the wind control data according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210846082.6A CN115168509A (en) | 2022-07-19 | 2022-07-19 | Processing method and device of wind control data, storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210846082.6A CN115168509A (en) | 2022-07-19 | 2022-07-19 | Processing method and device of wind control data, storage medium and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115168509A true CN115168509A (en) | 2022-10-11 |
Family
ID=83495189
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210846082.6A Pending CN115168509A (en) | 2022-07-19 | 2022-07-19 | Processing method and device of wind control data, storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115168509A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116226213A (en) * | 2023-02-22 | 2023-06-06 | 广州集联信息技术有限公司 | Information recommendation system and method based on big data |
-
2022
- 2022-07-19 CN CN202210846082.6A patent/CN115168509A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116226213A (en) * | 2023-02-22 | 2023-06-06 | 广州集联信息技术有限公司 | Information recommendation system and method based on big data |
| CN116226213B (en) * | 2023-02-22 | 2023-11-10 | 广州集联信息技术有限公司 | Information recommendation system and method based on big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108647357B (en) | Data query method and device | |
| CN115544007A (en) | Label preprocessing method and device, computer equipment and storage medium | |
| CN109711849B (en) | Ether house address portrait generation method and device, electronic equipment and storage medium | |
| CN115168509A (en) | Processing method and device of wind control data, storage medium and computer equipment | |
| CN112836124A (en) | Image data acquisition method and device, electronic equipment and storage medium | |
| CN112613176A (en) | Slow SQL statement prediction method and system | |
| CN116303320A (en) | Real-time task management method, device, equipment and medium based on log file | |
| CN116225848A (en) | Log monitoring method, device, equipment and medium | |
| CN111400608B (en) | Data processing method and device, storage medium and electronic equipment | |
| CN114896955A (en) | Data report processing method and device, computer equipment and storage medium | |
| CN113779116A (en) | Object sorting method, related equipment and medium | |
| CN114238055A (en) | Task data processing method and device, electronic equipment and storage medium | |
| CN114579398A (en) | Log storage method, device, equipment and storage medium | |
| CN113535458A (en) | Abnormal false alarm processing method and device, storage medium and terminal | |
| CN112181825A (en) | Test case library construction method and device, electronic equipment and medium | |
| CN113052509A (en) | Model evaluation method, model evaluation apparatus, electronic device, and storage medium | |
| CN112527830A (en) | Data query method, device, equipment and storage medium | |
| CN112148491B (en) | Data processing method and device | |
| CN118333529B (en) | Inventory early warning method and device, electronic equipment and storage medium | |
| CN117972096B (en) | Method and system for processing interaction message of social platform | |
| CN117453493B (en) | GPU computing power cluster monitoring method and system for large-scale multi-data center | |
| CN111125685A (en) | Method and device for predicting network security situation | |
| CN115794609A (en) | Script sharing method and device, electronic equipment and storage medium | |
| CN115983222A (en) | EasyExcel-based file data reading method, device, equipment and medium | |
| CN116049508A (en) | Test element information generation method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |