CN116579842B - Credit data analysis method and system based on user behavior data - Google Patents

Abstract

The invention relates to the technical field of data processing, and discloses a credit data analysis method and system based on user behavior data, which are used for improving the accuracy rate of credit data analysis. Comprising the following steps: collecting a plurality of user behavior data and performing tag matching to determine tag data; integrating the data of the plurality of user behavior data and the tag data to obtain a user data set; performing data processing on the user data set to obtain a data set to be analyzed; performing first feature extraction processing on the data set to be analyzed through a filtering type feature extraction algorithm to obtain a first candidate feature set; performing second feature extraction processing on the first candidate feature set through a wrapped feature extraction algorithm to obtain a second candidate feature set; performing data drift detection and feature screening processing on the second candidate feature set to obtain a target feature set; and carrying out credit data analysis on the target feature set to obtain a credit data analysis result, and transmitting the credit data analysis result to a preset data processing terminal.

Description

Credit data analysis method and system based on user behavior data

Technical Field

The present invention relates to the field of data processing technology, and in particular to a credit data analysis method and system based on user behavior data.

Background Art

In recent years, the rapid development of Internet finance has made user credit data analysis increasingly important. It is difficult to effectively judge the risk of user default by evaluating the credit of users based on basic information alone. In addition, data with financial attributes that are strongly related to users is difficult to obtain and the cost of obtaining is very high, resulting in very limited effective data, which has caused great difficulties in building a high-accuracy credit data analysis system. And with the rapid development of Internet finance, the data dimension has shown explosive growth, resulting in high-dimensional and sparse data. In addition, in risk control modeling, structured data cleaning and processing are arduous, data transformation has matrix sparsity, resulting in excessive loss of information, and feature extraction is difficult. At the same time, higher-dimensional data has exceeded the data range that traditional scoring card models can handle.

However, machine learning models have obvious advantages in modeling data with the above characteristics. On the one hand, machine learning models can help filter out irrelevant and redundant feature data that affect the modeling effect. Feature selection can effectively reduce the dimension of data, reduce the computational complexity of the model, and improve the operation speed and accuracy of the model. On the other hand, machine learning models can also discover rules and patterns in high-dimensional sparse data and have strong generalization capabilities. Machine learning modeling can effectively improve the prediction and classification performance of the model while preventing the model from overfitting.

At the same time, the data drift problem has caused great difficulties in the actual production of machine learning models in recent years. Data drift refers to the gradual change of data distribution over time or space, and the data distribution that needs to be predicted or verified and the data used for training show obvious deviations, which will significantly reduce the prediction performance of the system model. Therefore, the accuracy of credit data analysis based on user behavior data is low.

Summary of the invention

In view of this, an embodiment of the present invention provides a credit data analysis method and system based on user behavior data, which solves the technical problem of low accuracy when performing credit data analysis based on user behavior data.

The present invention provides a credit data analysis method based on user behavior data, comprising: collecting multiple user behavior data, and performing label matching on the multiple user behavior data to determine the label data corresponding to each of the user behavior data; performing data integration on the multiple user behavior data and the label data corresponding to each of the user behavior data to obtain a user data set; performing data preprocessing on the user data set to obtain a data set to be analyzed; performing a first feature extraction process on the data set to be analyzed by a filtering feature extraction algorithm to obtain a first candidate feature set; performing a second feature extraction process on the first candidate feature set by a wrapping feature extraction algorithm to obtain a second candidate feature set; performing data drift detection and feature screening on the second candidate feature set to obtain a target feature set; performing credit data analysis on the target feature set by a preset target credit data analysis model to obtain a credit data analysis result, and transmitting the credit data analysis result to a preset data processing terminal.

In the present invention, the step of collecting multiple user behavior data, performing label matching on the multiple user behavior data, and determining the label data corresponding to each of the user behavior data includes: collecting multiple user behavior data, extracting time data on each of the user behavior data, and determining the time data corresponding to each of the user behavior data; based on the time data corresponding to each of the user behavior data, performing label matching on the multiple user behavior data, and determining the label data corresponding to each of the user behavior data.

In the present invention, the step of performing data preprocessing on the user data set to obtain the data set to be analyzed includes: performing outlier analysis on the user data set to determine the target outlier value, and performing missing value analysis on the user data set through the outlier value to determine the target missing value; based on the target missing value, performing data filling processing on the user data set to obtain the data set to be analyzed.

In the present invention, the step of performing a first feature extraction process on the data set to be analyzed by using a filtering feature extraction algorithm to obtain a first candidate feature set includes: removing redundant features from the data set to be analyzed by using the filtering feature extraction algorithm to obtain a feature set to be processed; performing feature correlation analysis on the feature set to be processed to obtain a feature correlation analysis result; and performing feature extraction on the feature set to be processed by using the feature correlation analysis result to obtain a first candidate feature set.

In the present invention, the step of performing second feature extraction processing on the first candidate feature set through a wraparound feature extraction algorithm to obtain a second candidate feature set includes: performing importance analysis on each first candidate feature in the first candidate feature set through a wraparound feature extraction algorithm to determine the importance data of each first candidate feature; performing second feature extraction processing on the first candidate feature set based on the importance data of each first candidate feature to obtain a second candidate feature set.

In the present invention, the step of performing data drift detection and feature screening processing on the second candidate feature set to obtain a target feature set includes: performing data drift detection on the second candidate feature set through a preset adversarial classifier to generate a data drift detection result; performing feature screening processing on the second candidate feature set through the data drift detection result to obtain a target feature set.

In the present invention, after the step of performing data drift detection and feature screening on the second candidate feature set to obtain the target feature set, and before the step of performing credit data analysis on the target feature set through a preset target credit data analysis model to obtain a credit data analysis result, and transmitting the credit data analysis result to a preset data processing terminal, it includes: performing initial hyperparameter analysis on the initial credit data analysis model to determine an initial hyperparameter combination; performing prior probability distribution analysis on the initial hyperparameter combination to determine prior probability distribution data; performing model training on the initial credit data analysis model through the second candidate feature set to generate a training set and a test set; performing posterior probability distribution analysis on the initial hyperparameter combination through the training set and the test set to determine posterior probability distribution data; performing iterative analysis on the initial hyperparameter combination based on the posterior probability distribution data to determine the optimal hyperparameter combination; and performing parameter configuration on the initial credit data analysis model based on the optimal hyperparameter combination to obtain the target credit data analysis model.

The present invention also provides a credit data analysis system based on user behavior data, comprising:

A data collection module is used to collect multiple user behavior data, and perform label matching on the multiple user behavior data to determine the label data corresponding to each user behavior data;

A data integration module, used to integrate the plurality of user behavior data and the label data corresponding to each of the user behavior data to obtain a user data set;

A data processing module, used for performing data preprocessing on the user data set to obtain a data set to be analyzed;

A first extraction module, configured to perform a first feature extraction process on the data set to be analyzed by using a filtering feature extraction algorithm to obtain a first candidate feature set;

A second extraction module, configured to perform a second feature extraction process on the first candidate feature set by using a wrapping feature extraction algorithm to obtain a second candidate feature set;

A feature screening module, used for performing data drift detection and feature screening processing on the second candidate feature set to obtain a target feature set;

The credit analysis module is used to perform credit data analysis on the target feature set through a preset target credit data analysis model to obtain a credit data analysis result, and transmit the credit data analysis result to a preset data processing terminal.

In the technical solution provided by the present invention, multiple user behavior data are collected and label matching is performed to determine the corresponding label data; multiple user behavior data and label data are integrated to obtain a user data set; the user data set is preprocessed to obtain a data set to be analyzed; a first feature extraction process is performed on the data set to be analyzed by a filtering feature extraction algorithm to obtain a first candidate feature set; a second feature extraction process is performed on the first candidate feature set by a wrapping feature extraction algorithm to obtain a second candidate feature set; data drift detection and feature screening are performed on the second candidate feature set to obtain a target feature set; credit data analysis is performed on the target feature set to obtain a credit data analysis result and the credit data analysis result is transmitted to a preset data processing terminal. In the embodiment of the present invention, more attention is paid to the impact of user behavior data on its credit status. Without the need to obtain high-cost and difficult-to-obtain financial attribute data that is strongly related to the user, a credit data analysis system based on user behavior data with a high accuracy is established. On the one hand, in the embodiment of the present invention, the categorical features are directly converted into numerical features, and there is no need to perform operations such as unique hot encoding on the categorical features to avoid increasing the data dimension, which is fast and efficient. On the other hand, the present invention reduces the impact of estimation bias by unbiased estimation of gradients, solves the problems of gradient bias and prediction offset, and thus effectively improves the generalization ability of the system model. Therefore, the present invention can predict the credit status of users at a faster training speed, and has more accurate prediction ability and better generalization performance, so as to further improve the accuracy of credit data analysis based on user behavior data.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a flow chart of a credit data analysis method based on user behavior data in an embodiment of the present invention.

FIG. 2 is a decile distribution diagram of redundant features in an embodiment of the present invention.

FIG. 3 is a decile distribution diagram of non-redundant features in an embodiment of the present invention.

FIG4 is a flow chart of performing a second feature extraction process on a first candidate feature set by using a wraparound feature extraction algorithm in an embodiment of the present invention.

FIG5 is a diagram showing the importance distribution of the remaining features in the filtering feature selection according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of a credit data analysis system based on user behavior data in an embodiment of the present invention.

Reference numerals:

3001, data acquisition module; 3002, data integration module; 3003, data processing module; 3004, first extraction module; 3005, second extraction module; 3006, feature screening module; 3007, credit analysis module; 3008, parameter analysis module; 3009, distribution analysis module; 3010, model training module; 3011, probability analysis module; 3012, iterative analysis module; 3013, parameter configuration module.

DETAILED DESCRIPTION

The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

In the description of the present invention, the terms "first", "second" and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

Please refer to FIG. 1 , which is a flow chart of a credit data analysis method based on user behavior data according to an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps:

S101, collecting multiple user behavior data, and performing label matching on the multiple user behavior data to determine label data corresponding to each user behavior data;

It should be noted that user behavior data includes information about apps used by users, information about devices used by users, and information about users’ recent location movements. Furthermore, multiple user behavior data are matched with labels to determine the label data corresponding to each user behavior data. It should be noted that in an embodiment of the present invention, user behavior data mainly includes a list of apps used by users during the collection period and the categories to which the corresponding apps belong. Apps are subsequently reclassified with reference to app classification information and related business cooperation. Before classification, because the actual collected app data has garbled characters and its authenticity needs to be verified, firstly, to address the garbled characters, app data with a character length greater than 20 is deleted, and then common Chinese characters are used to match the app names, and app data with a higher matching degree is retained. The garbled characters are processed through the above operations.

Table 1 - Specific meaning of each classification feature

Secondly, the present invention refers to the domestic professional mobile promotion data analysis platform Qimai Data to download real App data to verify the authenticity of the App data collected by the present invention. Finally, the usage frequency of all current users' single Apps is counted. Among the 319,071 Apps, almost 83% of the Apps are used only once, so the present invention retains the top 50,000 App data with higher usage frequency. The device information used by the user specifically includes the listing price, year, listing time limit this year, the latest active time of the device, the number of active days of the device during the data collection period, and other data. The user's location movement information during the collection period is roughly located as the province, district, and county to which the user belongs, and the longitude and latitude information of the user's location can be accurately obtained. According to the three parts of the behavior data collected, the user behavior data is processed based on a certain processing logic. Data collection and processing work consumes a lot of time and manpower, but the credibility, authenticity, validity, and versatility of the data have very significant advantages. A true and accurate data source is the key to modeling and the basis for the long-term applicability of the system model.

It should be noted that the behavioral data features in the data set to be analyzed include three parts: 1) Information on apps used by users, including four major categories: financial app usage preferences, other app usage preferences, financial tags, and other tags. Specifically, app usage preference features refer to the number of installations, new additions, and uninstalls of various apps on the user's device in the past 7 days, 15 days, 30 days, or 90 days, as well as the number of active days of various apps. Tag features refer to the number of times a certain type of feature is opened by a user; 2) Information on the device used by the user, including the listing price, year, the number of years since the device was listed, and the number of MAC addresses corresponding to the device in the past 30 days; 3) Information on the user's recent location movement, including the number of times the user has appeared in convenience stores recently. The user behavior data contains 94 features in six categories, as shown in Table 1, which details the specific meanings of some features in each category.

S102, integrating multiple user behavior data and label data corresponding to each user behavior data to obtain a user data set;

Specifically, multiple user behavior data and the label data corresponding to each user behavior data are merged to obtain a user data set, wherein the present invention also obtains the credit record data of the same batch of users, and defines the user performance period as three months. If the time data in the user's credit record data exceeds the preset first threshold during the performance period, it is defined as a positive sample. If the time data in the user's credit record data does not exceed the preset second threshold, it is defined as a negative sample. The modeling data only needs to obtain data that is clearly defined as positive samples and negative samples. The time range of this data set is from September 1, 2021 to December 31, 2021, including 142,793 data. The results of the data set integration and division are specifically shown in Table 2.

Table 2 - Dataset partitioning results

It should be noted that OOT in Table 2 is a cross-time validation set, and the cross-time validation set is the last sample of the modeling sample time slice.

S103, performing data preprocessing on the user data set to obtain a data set to be analyzed;

It should be noted that when preprocessing the user data set, the user behavior data and the label data of whether there is a breach of contract are first integrated to form the final data set, and the data set is divided at the same time. Then, outlier processing and missing value processing are performed, and the method of filling missing values is determined based on the model effect.

Specifically, when performing data preprocessing on a user data set, an outlier analysis is performed on the user data set to determine a target outlier, and a missing value analysis is performed on the user data set through the outlier to determine a target missing value, and based on the target missing value, data filling processing is performed on the user data set to obtain a data set to be analyzed.

It should be noted that since feature data with a high degree of missingness will affect the modeling effect, a direct deletion strategy is adopted for feature columns with a column feature missingness rate higher than 80%. Secondly, a box plot is used in combination with expert experience to detect and determine outliers. Data determined to be outliers are treated as missing values. Finally, a variety of methods such as fixed value filling, mean filling, previous data, and interpolation filling are used to fill missing values. The missing values of each column are filled based on the system model effect and the mean of the column is used to fill them.

S104, performing a first feature extraction process on the data set to be analyzed by using a filtering feature extraction algorithm to obtain a first candidate feature set;

S105, performing a second feature extraction process on the first candidate feature set by using a wraparound feature extraction algorithm to obtain a second candidate feature set;

In the embodiment of the present invention, the feature selection method adopts two methods: filtering and wrapping. Among them, when the filtering feature extraction algorithm is used for processing, it includes filtering out redundant features through three statistical methods: decile distribution, rank sum test, and standard score. Among them, in order to select features related to modeling, firstly, three statistical methods: decile distribution, rank sum test, and standard score are used to filter out redundant features; then, from the perspective of linearity and nonlinearity of features, the Pearson correlation system method and the maximum information coefficient method are used to update the feature set, and finally the first feature extraction processing of the data set to be analyzed is completed to obtain the first candidate feature set.

Furthermore, a second feature extraction process is performed on the first candidate feature set through a wraparound feature extraction algorithm to obtain a second candidate feature set, wherein when the server performs the second feature extraction process on the first candidate feature set through the wraparound feature extraction algorithm, a method of scoring feature importance of a classification tree model is combined to determine a better candidate feature set, and finally the better candidate feature set is used as the second candidate feature set.

S106, performing data drift detection and feature screening processing on the second candidate feature set to obtain a target feature set;

Specifically, in an embodiment of the present invention, HyperGBM is used to detect and process data drift of the user data. It should be noted that HyperGBM is a full-Pipeline automatic machine learning tool that can fully cover the entire process from data cleaning, preprocessing, feature processing and screening, model selection and hyperparameter optimization end-to-end, and perform feature screening processing at the same time to obtain a target feature set.

S107. Perform credit data analysis on the target feature set using a preset target credit data analysis model to obtain a credit data analysis result, and transmit the credit data analysis result to a preset data processing terminal.

By executing the above steps, multiple user behavior data are collected and label matching is performed to determine the corresponding label data; multiple user behavior data and label data are integrated to obtain a user data set; the user data set is preprocessed to obtain a data set to be analyzed; the first feature extraction process is performed on the data set to be analyzed by a filtering feature extraction algorithm to obtain a first candidate feature set; the second feature extraction process is performed on the first candidate feature set by a wrapping feature extraction algorithm to obtain a second candidate feature set; data drift detection and feature screening are performed on the second candidate feature set to obtain a target feature set; credit data analysis is performed on the target feature set to obtain a credit data analysis result and the credit data analysis result is transmitted to a preset data processing terminal.

In the embodiment of the present invention, more attention is paid to the impact of user behavior data on their credit status. Without the need to obtain high-cost and difficult-to-obtain data on financial attributes that are strongly related to users, a credit data analysis system based on user behavior data with high accuracy is established. On the one hand, in the embodiment of the present invention, categorical features are directly converted into numerical features, and there is no need to perform operations such as unique hot encoding on categorical features to avoid increasing data dimensions, which is fast and efficient. On the other hand, the present invention reduces the impact of estimation bias by unbiased estimation of the gradient compared to traditional gradient estimation methods, solves the problems of gradient bias and prediction offset, and thus effectively improves the generalization ability of the system model. Therefore, the present invention can predict the credit status of users at a faster training speed, and has more accurate prediction capabilities and better generalization performance, so as to further improve the accuracy of credit data analysis based on user behavior data.

In a specific embodiment, the process of executing step S101 may specifically include the following steps:

(1) Collect multiple user behavior data, extract time data for each user behavior data, and determine the time data corresponding to each user behavior data;

(2) Based on the time data corresponding to each user behavior data, label matching is performed on multiple user behavior data to determine the label data corresponding to each user behavior data.

It should be noted that user behavior data includes information about apps used by users, information about devices used by users, and information about recent location movements of users. Furthermore, label matching is performed on multiple user behavior data to determine the label data corresponding to each user behavior data. In an embodiment of the present invention, the user performance period is defined as three months. If the time data in the user's credit record data during the performance period exceeds a preset first threshold, it is defined as a positive sample. If the time data in the user's credit record data does not exceed a preset second threshold, it is defined as a negative sample.

In a specific embodiment, the process of executing step S103 may specifically include the following steps:

(1) Perform outlier analysis on the user data set to determine the target outlier, and perform missing value analysis on the user data set through the outlier to determine the target missing value;

(2) Based on the target missing values, the user data set is filled with data to obtain the data set to be analyzed.

In this step, it should be noted that the quality of the data can directly determine the prediction and generalization capabilities of the system, and data preprocessing is a prerequisite for ensuring data quality, so data preprocessing is crucial for modeling. The actual collected data will inevitably have a high degree of data missing due to the large data collection time span and complex collection method. The present invention believes that the feature data with extremely high missingness will affect the modeling effect, so the feature column with a column feature missingness higher than 80% adopts a direct deletion strategy. After the above operation, a total of 18 feature columns with extremely high missingness are deleted, and a data set containing 76 features is obtained. For the remaining data, because the data are all numerical types, a box plot is first drawn to detect outliers, and then the outliers are determined in combination with expert experience. Because there is no correlation between users and users, it is very likely that the number of users uninstalling and installing a certain type of App within a certain period of time is very large or very small, so the outliers detected by the box plot need to be determined according to expert experience. If the data is determined to be an outlier, it is treated as a missing value. The present invention adopts seven methods for filling missing values: fixed value filling, mean filling, median filling, mode filling, interpolation filling, previous data filling, and next data filling. According to the model effect, the filling effects of the four methods of fixed value filling, mean filling, median filling, and mode filling are significantly better than the three methods of interpolation filling, previous data filling, and next data filling by about 5% to 10%. This is because there is almost no correlation between the user data of the present invention, and it is not appropriate to use the previous user or next user data to complete the missing current user data. Among the four methods with better filling effects, the model effect of mean filling is about 1% to 2% better than the other three methods. Therefore, the missing value filling method of the present invention selects column mean filling.

Finally, the outlier analysis of the user data set is completed to determine the target outlier, and the missing value analysis of the user data set is performed through the outlier to determine the target missing value. Based on the target missing value, the user data set is filled with data to obtain the data set to be analyzed.

In a specific embodiment, the process of executing step S104 may specifically include the following steps:

(1) Redundant features are removed from the data set to be analyzed through a filtering feature extraction algorithm to obtain a feature set to be processed;

(2) Perform feature correlation analysis on the feature set to be processed to obtain feature correlation analysis results;

(3) Perform feature extraction on the feature set to be processed based on the feature correlation analysis results to obtain the first candidate feature set.

It should be noted that feature extraction is to select features that are helpful for modeling. The present invention uses filtering and wrapping methods to filter features that are not related to modeling. Among them, redundant features will increase the amount of calculation of the model, slow down the training speed, and even have the possibility of overfitting. Screening these features can reduce unnecessary resource consumption and improve the prediction performance of the system model. The goal of the three statistical feature selection methods of decile distribution, rank sum test and standard classification is to filter out redundant features. In addition, the degree of correlation between two variables can also be used as the basis for feature screening. The stronger the correlation between the variables, the greater the amount of information contained in the two variables, and then select one of the features with strong correlation. Therefore, the present invention uses the Pearson correlation coefficient method and the maximum information coefficient method to update the feature set from the perspective of linearity and nonlinearity of the features. As shown in Table 3, it is the strength relationship between the value of the correlation coefficient and the correlation between the features. In the present invention, the redundant features of the data set to be analyzed are eliminated by a filtering feature extraction algorithm to obtain a feature set to be processed, and the feature correlation analysis is performed on the feature set to be processed to obtain a feature correlation analysis result. The feature extraction of the feature set to be processed is performed by the feature correlation analysis result to obtain a first candidate feature set.

Table 3 - The relationship between the value of the correlation coefficient and the strength of the correlation between the features

Among them, the decile distribution is a feature selection method based on deciles, which can intuitively reflect the distinguishing effect of each feature on negative samples and positive samples. Figures 2 and 3 show the decile distribution of two features, redundant features and non-redundant features. In Figure 2, naw represents the number of express logistics or other express logistics APP installations in the past 7 days, and Deciles in Figure 2 represents the decile of redundant features; in Figure 3, ECA08 represents the number of users opening the e-commerce industry or e-commerce industry online behavior or vertical e-commerce-digital 3C APP in the past 7 days, and Deciles in Figure 3 represents the decile of non-redundant features. It can be observed from the figure that if the cumulative distribution of negative samples and positive samples of redundant features is the same, then the feature has no effect on distinguishing negative samples and positive samples, indicating that the feature is a redundant feature; on the contrary, if the cumulative distribution of negative samples and positive samples of non-redundant features is inconsistent, then the feature can distinguish negative samples and positive samples, indicating that the feature is not a redundant feature and the feature is retained. The rank sum test and the standard score are two statistical methods that belong to non-parametric statistical methods, and the decile distribution can intuitively reflect the distinguishing effect of each feature on negative samples and positive samples. Combining three statistical methods, 29 redundant features were eliminated and a candidate feature set consisting of 47 features was obtained. The Pearson correlation coefficient measures the linear correlation between different features by the covariance and standard deviation of two feature variables. The present invention defines the value range of the Pearson correlation coefficient as [0,1], and the relationship between the value of the correlation coefficient and the strength of the correlation between the features is shown in Table 3. By statistically analyzing extremely strongly correlated feature pairs, it is found that multiple extremely strongly correlated feature pairs can construct an extremely strongly correlated feature set, that is, any two features in this feature set satisfy the extremely strong correlation. Although the correlation coefficient of individual feature pairs in some feature sets is less than 0.8, it is higher than 0.7. Therefore, this special feature set is also considered by the present invention to satisfy the extremely strongly correlated feature set. Further, the correlation coefficient between each feature in the data to be analyzed is calculated, and the maximum information coefficient is measured by calculating the mutual information between the two variables and the joint probability MIC to measure the nonlinear correlation between different features. The value range of the MIC value is defined as [0,1]. The strength of the correlation between the value of the MIC and the feature is similar to the Pearson correlation coefficient method. The statistical results of extremely strong linear and nonlinear correlation feature pairs and feature sets in the Pearson correlation coefficient method and the maximum information coefficient method are shown in Table 4. Then, the decile distribution method is combined to finally determine the features selected from the extremely strong correlation feature pairs and feature sets. It should be noted that the feature selection results are bolded in Table 4. Finally, from the perspective of linearity and nonlinearity of the features, the Pearson correlation coefficient method and the maximum information coefficient method are used to eliminate 22 extremely correlated features, and the number of feature sets is updated to 25.

Table 4 - Statistics of extremely strong linear and nonlinear correlation feature pairs and feature sets

In a specific embodiment, as shown in FIG. 4 , the process of executing step S105 may specifically include the following steps:

S201, performing importance analysis on each first candidate feature in the first candidate feature set by using a wraparound feature extraction algorithm to determine importance data of each first candidate feature;

S202: Perform a second feature extraction process on the first candidate feature set based on the importance data of each first candidate feature to obtain a second candidate feature set.

Specifically, the importance of the features of the classification tree model can intuitively reflect the importance of each feature that affects the system decision by calculating the number of times the attributes are divided in all trees. As shown in Figure 5, it is an importance distribution diagram of the remaining features selected for the filtering feature, where Feature Importance in the figure represents the feature importance value of the classification tree model. According to the effect of the CatBoost model, the top 20 features with higher feature importance are finally selected as the better feature set. Finally, based on the importance data of each first candidate feature, the first candidate feature set is subjected to second feature extraction processing to obtain the second candidate feature set, as shown in Table 5, which is a description list of each feature in the second candidate feature set.

Table 5 - Description list of each feature in the second candidate feature set

It should be noted that the second candidate features and the categories to which the features belong and their specific meanings are shown in Table 5, and the feature order is described from high to low importance according to the classification tree model. First, it can be observed from Table 5 that the listing price of the device used by the user is relatively important in predicting whether the user has good credit.

According to the statistics, when the price of the device used is higher than RMB 2,500, about 73% of users have good credit, and when the price of the device used is lower than RMB 2,500, only 21% of users have good credit. At the same time, according to the statistics, when the number of times a user has appeared in a convenience store shopping place more than 10 times recently, about 67% of users have good credit, and when the number of times a user has appeared in a convenience store shopping place less than 10 times recently, about 26% of users have good credit. In summary, when evaluating whether a user has good credit, the price of the device used by the user or the number of times the user has appeared in a convenience store shopping place recently can be considered for comprehensive evaluation.

In a specific embodiment, the process of executing step S106 may specifically include the following steps:

(1) Performing data drift detection on the second candidate feature set through a preset adversarial classifier to generate a data drift detection result;

(2) Perform feature screening on the second candidate feature set based on the data drift detection results to obtain the target feature set.

In a specific embodiment, after executing step S106 and before executing step S107, the following steps are further included:

(1) Conduct initial hyperparameter analysis on the initial credit data analysis model and determine the initial hyperparameter combination;

(2) Perform a prior probability distribution analysis on the initial hyperparameter combination to determine the prior probability distribution data;

(3) Training the initial credit data analysis model using the second candidate feature set to generate a training set and a test set;

(4) Perform posterior probability distribution analysis on the initial hyperparameter combination through training sets and test sets to determine the posterior probability distribution data;

(5) Iteratively analyze the initial hyperparameter combination based on the posterior probability distribution data to determine the optimal hyperparameter combination;

(6) Based on the optimal hyperparameter combination, the initial credit data analysis model is configured with parameters to obtain the target credit data analysis model.

Specifically, in this step, the parameter adjustment method uses Bayesian optimization, which estimates the posterior distribution of the objective function based on the data using Bayesian theorem, and then selects the next sampled hyperparameter combination based on the distribution. Based on the full use of the information of the previous sampling point, it can better adjust the current parameters to quickly find the parameters that maximize the global objective function. Compared with grid search, Bayesian optimization has fewer iterations and runs faster. After a specific range of parameters is given, multiple parameters can be adjusted at once, so Bayesian optimization will not cause dimensionality explosion when there are too many parameters.

In an embodiment of the present invention, an initial hyperparameter analysis is performed on the initial credit data analysis model to determine the initial hyperparameter combination; a priori probability distribution analysis is performed on the initial hyperparameter combination to determine the priori probability distribution data; the initial credit data analysis model is trained through the second candidate feature set to generate a training set and a test set, and a posterior probability distribution analysis is performed on the initial hyperparameter combination through the training set and the test set to determine the posterior probability distribution data; the initial hyperparameter combination is iteratively analyzed based on the posterior probability distribution data to determine the optimal hyperparameter combination; the initial credit data analysis model is parameterized based on the optimal hyperparameter combination to obtain the target credit data analysis model. This model adjusts important parameters such as the learning rate, tree depth, sample sampling ratio, column sampling ratio, etc. The parameter setting range and the final adjustment results are shown in Table 6.

Table 6-Bayesian optimization parameter adjustment range and parameter adjustment results

Furthermore, as shown in Table 7, the initial effect of the system model training set KS is 0.1925, and the test set is 0.1523. After Bayesian optimization parameter adjustment, the system performance reaches a training set KS of 0.1728 and a test set KS of 0.1638. Parameter tuning effectively reduces the overfitting phenomenon of the system, and the test set effect is improved by about 7% compared with the initial system model KS.

Table 7-Comparison of credit data analysis model effects before and after parameter adjustment

It should be noted that the system evaluation index is KS (Kolmogorov-Smirnov), which is the maximum absolute value of the difference between the cumulative positive samples and the cumulative negative samples in each bin interval. In the risk control system, the size of the KS value represents the discrimination of the system. The larger the KS value, the stronger the risk ranking ability of the system. The system prediction ability refers to the prediction accuracy of the system. The better the prediction ability of the system, the stronger the system's discrimination ability; the generalization ability of the system refers to the system's prediction ability on a new data set with the same rules; the stability of the system refers to the fluctuation of the prediction results of the system under different random sampling results.

In an embodiment of the present invention, the prediction performance of five algorithms is also compared on a user behavior data set, as shown in Table 8, which is a performance comparison of different machine learning models.

First, it can be found that the performance of the integrated model is significantly better than that of a single system. This is because the integrated model aims to reduce the system prediction bias or variance, and combines several models according to a certain strategy to improve the system prediction performance. Secondly, the KS values of the CatBoost and LightGBM models using the boosting idea in the integrated model are significantly higher than those of the RandomForest using the bagging idea. This is because on the one hand, the GBDT algorithm can combine multiple base learners to effectively improve the generalization and robustness of the system model, and is more committed to improving the prediction accuracy of the model, while Random Forest only focuses on improving the generalization and robustness of the model. On the other hand, bagging builds different models based on the idea of parallelism, while boosting is based on the idea of serialization, with the goal of improving accuracy. The latter system fully considers the training results of the previous model. Finally, the CatBoost model shows obvious advantages over LightGBM in both training and testing stages, and has better generalization performance on OOT. This is because the CatBoost algorithm can quickly and efficiently process categorical features compared to LightGBM, and uses the Ordered boosting method to obtain an unbiased estimate of the gradient, solving the problems of gradient bias and prediction offset, thereby effectively improving the prediction performance and generalization ability of the system model. In terms of training time, Table 8 shows the average training time of the five models obtained through 50 trainings. First, it can be observed from Table 8 that the RandomForest model takes the longest to train, followed by CatBoost. Because the RandomForest tree considers all features at each split, it takes a long time to train. The advantage of CatBoost is that it can quickly process categorical features. If there are more categorical features in the data, the training time of CatBoost will be greatly reduced. Secondly, for the LightGBM and CatBoost models with better prediction performance, although the training time of the CatBoost model is about 10 times that of the LightGBM model, the prediction performance of the CatBoost model test set is 36.98% higher than that of LightGBM, and the OOT performance is also better than that of LightGBM. Finally, after discussion with experts, the training time of the CatBoost model of about 7s is acceptable in an actual production environment.

In terms of test time, the average test time of the five models obtained through 50 tests is shown in Table 8. From Table 8, it can be observed that the test time of the four models CatBoost, LightGBM, GaussianNaive Bayes and Gaussian Mixture Model is an order of magnitude better than that of Random Forest, among which the training time advantage of the CatBoost model is also obvious.

In summary, through the comparison of model effects of different algorithms, the present invention uses CatBoost to establish a credit data analysis system based on user behavior data, which not only shows excellent prediction ability, but also has good generalization ability and significant stability. As shown in Table 8, the KS of the test set reaches 0.1638, and the generalization ability is good. The stability of the system refers to the fluctuation of the prediction results of the system under different random sampling results. Because the present invention sets two parameters, the sampling ratio of samples and features, in the CatBoost model, the training objects of each iteration during the training process are different. The reason for this phenomenon is that different random seeds are set. Therefore, the stability of the system can be observed by observing the changes in the KS of the system under different random seeds.

Table 8 - Performance comparison of different machine learning models

The technical method of the present invention is introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

The embodiment of the present invention further provides a credit data analysis system based on user behavior data, as shown in FIG6 , the credit data analysis system based on user behavior data specifically includes:

The data collection module 3001 is used to collect multiple user behavior data, and perform label matching on the multiple user behavior data to determine the label data corresponding to each user behavior data;

A data integration module 3002 is used to integrate the plurality of user behavior data and the label data corresponding to each of the user behavior data to obtain a user data set;

The data processing module 3003 is used to perform data preprocessing on the user data set to obtain a data set to be analyzed;

A first extraction module 3004 is used to perform a first feature extraction process on the data set to be analyzed by using a filtering feature extraction algorithm to obtain a first candidate feature set;

A second extraction module 3005 is used to perform a second feature extraction process on the first candidate feature set by using a wrapping feature extraction algorithm to obtain a second candidate feature set;

A feature screening module 3006 is used to perform data drift detection and feature screening on the second candidate feature set to obtain a target feature set;

The credit analysis module 3007 is used to perform credit data analysis on the target feature set through a preset target credit data analysis model to obtain a credit data analysis result, and transmit the credit data analysis result to a preset data processing terminal.

Optionally, the data collection module 3001 is specifically used to: collect multiple user behavior data, and extract time data for each of the user behavior data to determine the time data corresponding to each of the user behavior data; based on the time data corresponding to each of the user behavior data, perform label matching on the multiple user behavior data to determine the label data corresponding to each of the user behavior data.

Optionally, the data processing module 3003 is specifically used to: perform outlier analysis on the user data set to determine target outliers, and perform missing value analysis on the user data set through the outliers to determine target missing values; based on the target missing values, perform data filling processing on the user data set to obtain the data set to be analyzed.

Optionally, the first extraction module 3004 is specifically used to: eliminate redundant features of the data set to be analyzed by using the filtering feature extraction algorithm to obtain a feature set to be processed; perform feature correlation analysis on the feature set to be processed to obtain a feature correlation analysis result; perform feature extraction on the feature set to be processed by using the feature correlation analysis result to obtain a first candidate feature set.

Optionally, the second extraction module 3005 is specifically used to: perform importance analysis on each first candidate feature in the first candidate feature set through a wraparound feature extraction algorithm to determine the importance data of each first candidate feature; perform second feature extraction processing on the first candidate feature set based on the importance data of each first candidate feature to obtain a second candidate feature set.

Optionally, the feature screening module 3006 is specifically used to: perform data drift detection on the second candidate feature set through a preset adversarial classifier to generate a data drift detection result; perform feature screening processing on the second candidate feature set through the data drift detection result to obtain a target feature set.

Optionally, the credit data analysis system based on user behavior data further includes:

The parameter analysis module 3008 is used to perform initial hyperparameter analysis on the initial credit data analysis model and determine the initial hyperparameter combination;

The distribution analysis module 3009 is used to perform a priori probability distribution analysis on the initial hyperparameter combination to determine a priori probability distribution data;

A model training module 3010, configured to perform model training on the initial credit data analysis model using the second candidate feature set to generate a training set and a test set;

The probability analysis module 3011 is used to perform a posterior probability distribution analysis on the initial hyperparameter combination through the training set and the test set to determine the posterior probability distribution data;

An iterative analysis module 3012 is used to iteratively analyze the initial hyperparameter combination based on the posterior probability distribution data to determine the optimal hyperparameter combination;

The parameter configuration module 3013 is used to configure the parameters of the initial credit data analysis model based on the optimal hyperparameter combination to obtain the target credit data analysis model.

Through the cooperation of the above modules, multiple user behavior data are collected and label matching is performed to determine the corresponding label data; multiple user behavior data and label data are integrated to obtain a user data set; the user data set is preprocessed to obtain a data set to be analyzed; the first feature extraction process is performed on the data set to be analyzed by a filtering feature extraction algorithm to obtain a first candidate feature set; the second feature extraction process is performed on the first candidate feature set by a wrapping feature extraction algorithm to obtain a second candidate feature set; the second candidate feature set is subjected to data drift detection and feature screening to obtain a target feature set; the target feature set is subjected to credit data analysis to obtain a credit data analysis result and transmit the credit data analysis result to a preset data processing terminal. In the embodiment of the present invention, more attention is paid to the impact of user behavior data on its credit status. Without the need to obtain high-cost and difficult-to-obtain financial attribute data that is strongly related to the user, a credit data analysis system based on user behavior data with a high accuracy is established. On the one hand, in the embodiment of the present invention, the categorical features are directly converted into numerical features, and there is no need to perform operations such as unique hot encoding on the categorical features to avoid increasing the data dimension, which is fast and efficient. On the other hand, the present invention reduces the impact of estimation bias by unbiased estimation of gradients, solves the problems of gradient bias and prediction offset, and thus effectively improves the generalization ability of the system model. Therefore, the present invention can predict the credit status of users at a faster training speed, and has more accurate prediction ability and better generalization performance, so as to further improve the accuracy of credit data analysis based on user behavior data.

The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the embodiments, a person skilled in the art should understand that the specific implementation modes of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims (4)

1. A credit data analysis method based on user behavior data, which is characterized by including: Collect multiple user behavior data, perform label matching on the multiple user behavior data, and determine the label data corresponding to each user behavior data; Perform data integration on the plurality of user behavior data and label data corresponding to each user behavior data to obtain a user data set; Perform data preprocessing on the user data set to obtain a data set to be analyzed; Perform a first feature extraction process on the data set to be analyzed through a filtering feature extraction algorithm to obtain a first candidate feature set, which specifically includes: removing redundant features from the data set to be analyzed through the filtering feature extraction algorithm. , obtain the feature set to be processed; perform feature correlation analysis on the feature set to be processed, and obtain the feature correlation analysis result; perform feature extraction on the feature set to be processed through the feature correlation analysis result, and obtain the first candidate feature set; Among them, when performing the first feature extraction process on the data set to be analyzed through the filtering feature extraction algorithm, it includes filtering out redundant features through the tenth distribution algorithm, the rank sum test algorithm and the standard dividing algorithm. Among them, the decile distribution algorithm, the rank sum test algorithm and the standard dividing algorithm are used to filter out redundant features. The bit distribution algorithm, rank sum test algorithm and standard classification algorithm screen out redundant features, and then use the Pearson correlation system method and the maximum information coefficient method to update the feature set to be processed to obtain the first candidate feature set; Perform a second feature extraction process on the first candidate feature set through a wrapped feature extraction algorithm to obtain a second candidate feature set, which specifically includes: performing a second feature extraction process on each first candidate feature set in the first candidate feature set through a wrapped feature extraction algorithm. Conducting importance analysis on the candidate features to determine the importance data of each first candidate feature; performing a second feature extraction process on the first candidate feature set based on the importance data of each first candidate feature to obtain the second candidate feature gather; Perform data drift detection and feature screening on the second candidate feature set to obtain a target feature set, which specifically includes: performing data drift detection on the second candidate feature set through a preset adversarial classifier to generate a data drift detection result ; Perform feature screening processing on the second candidate feature set through the data drift detection result to obtain a target feature set; Perform initial hyperparameter analysis on the initial credit data analysis model to determine an initial hyperparameter combination; perform a priori probability distribution analysis on the initial hyperparameter combination to determine a priori probability distribution data; use the second candidate feature set to determine the a priori probability distribution data. The initial credit data analysis model performs model training and generates a training set and a test set; performs posterior probability distribution analysis on the initial hyperparameter combination through the training set and the test set, and determines the posterior probability distribution data; based on the The posterior probability distribution data is used to iteratively analyze the initial hyperparameter combination to determine the optimal hyperparameter combination; parameter configuration is performed on the initial credit data analysis model based on the optimal hyperparameter combination to obtain the target credit data analysis Model; Perform credit data analysis on the target feature set through the target credit data analysis model to obtain credit data analysis results, and transmit the credit data analysis results to a preset data processing terminal. 2. The credit data analysis method based on user behavior data according to claim 1, characterized in that: collecting multiple user behavior data, performing tag matching on multiple user behavior data, and determining each of the user behavior data. The label data steps corresponding to user behavior data include: Collect multiple user behavior data, extract time data for each user behavior data, and determine the time data corresponding to each user behavior data; Based on the time data corresponding to each user behavior data, label matching is performed on multiple user behavior data to determine the label data corresponding to each user behavior data. 3. The credit data analysis method based on user behavior data according to claim 1, characterized in that the step of performing data preprocessing on the user data set to obtain the data set to be analyzed includes: Perform outlier analysis on the user data set to determine target outliers, and perform missing value analysis on the user data set through the outliers to determine target missing values; Based on the target missing values, data filling processing is performed on the user data set to obtain a data set to be analyzed. 4. A credit data analysis system based on user behavior data, used to perform the credit data analysis method based on user behavior data according to any one of claims 1 to 3, characterized in that it includes: A data collection module, used to collect multiple user behavior data, perform label matching on multiple user behavior data, and determine the label data corresponding to each user behavior data; A data integration module, configured to perform data integration on the plurality of user behavior data and label data corresponding to each of the user behavior data, to obtain a user data set; A data processing module, used to perform data preprocessing on the user data set to obtain a data set to be analyzed; The first extraction module is used to perform a first feature extraction process on the data set to be analyzed through a filtering feature extraction algorithm to obtain a first candidate feature set, specifically including: performing a first feature extraction process on the data set to be analyzed through the filtering feature extraction algorithm. The data set is subjected to redundant feature elimination to obtain a feature set to be processed; feature correlation analysis is performed on the feature set to be processed to obtain a feature correlation analysis result; and the feature set to be processed is processed based on the feature correlation analysis result. Feature extraction to obtain the first candidate feature set; Among them, when performing the first feature extraction process on the data set to be analyzed through the filtering feature extraction algorithm, it includes filtering out redundant features through the tenth distribution algorithm, the rank sum test algorithm and the standard dividing algorithm. Among them, the decile distribution algorithm, the rank sum test algorithm and the standard dividing algorithm are used to filter out redundant features. The bit distribution algorithm, rank sum test algorithm and standard classification algorithm screen out redundant features, and then use the Pearson correlation system method and the maximum information coefficient method to update the feature set to be processed to obtain the first candidate feature set; The second extraction module is used to perform a second feature extraction process on the first candidate feature set through a wrapped feature extraction algorithm to obtain a second candidate feature set, specifically including: performing a second feature extraction process on the first candidate feature set through a wrapped feature extraction algorithm. Conducting importance analysis on each first candidate feature in the feature set to determine the importance data of each first candidate feature; performing second feature extraction on the first candidate feature set based on the importance data of each first candidate feature Process to obtain the second candidate feature set; A feature screening module, used to perform data drift detection and feature screening processing on the second candidate feature set to obtain a target feature set, specifically including: performing data drift detection on the second candidate feature set through a preset adversarial classifier , generate a data drift detection result; perform feature screening processing on the second candidate feature set through the data drift detection result to obtain a target feature set; The credit analysis module is used to perform initial hyperparameter analysis on the initial credit data analysis model and determine the initial hyperparameter combination; perform a priori probability distribution analysis on the initial hyperparameter combination and determine the priori probability distribution data; through the second The candidate feature set conducts model training on the initial credit data analysis model, and generates a training set and a test set; performs posterior probability distribution analysis on the initial hyperparameter combination through the training set and the test set, and determines the posterior probability Distribution data; perform iterative analysis on the initial hyperparameter combination based on the posterior probability distribution data to determine the optimal hyperparameter combination; perform parameter configuration on the initial credit data analysis model based on the optimal hyperparameter combination to obtain The target credit data analysis model; Perform credit data analysis on the target feature set through the target credit data analysis model to obtain credit data analysis results, and transmit the credit data analysis results to a preset data processing terminal.