EA038056B1 - Computerized method for creating and managing scoring models - Google Patents

Abstract

The invention relates in general to the field of computing, and more particularly to methods for automatically creating credit scoring models and automatically implementing the same in the credit process. A computerized method for creating and managing scoring models involves obtaining data for a given period of time containing factors affecting a scoring model; then breaking down the data obtained into samples in order to create, validate and test a scoring model; then transforming factors by establishing a relationship between groups of values of a transformed factor and default rates; then excluding from the samples at least one transformed factor that correlates to at least one other factor; then generating a credit scoring model by training a binary multivariate logistic regression; and finally selecting an automatic cut-off zone for at least one scoring model for implementation thereof in the credit process. The technical result is an increase in the quality of the credit scoring models created.

Description

Technology area

This invention generally relates to the field of computing, and in particular to methods for the automatic development of credit scoring models and their automatic implementation in the credit process.

State of the art

Currently, financial institutions use standard statistical approaches to analyzing historical data to describe potential clients in terms of risk. This allows you to classify borrowers into good and bad and thus make the final decision on lending. Most credit institutions have established departments that develop credit scoring models based on their own statistics, taking into account the specifics of the client profile. However, these credit institutions often contact the credit bureaus, which is why the process of assessing the borrower's creditworthiness is greatly delayed and becomes inaccurate, since it depends on the algorithms used by the credit bureaus.

The essence of the invention

This invention is aimed at eliminating the disadvantages inherent in existing solutions known from the prior art.

The technical problem (or technical problem) in this invention is the implementation of the automatic development of credit scoring models with their subsequent implementation into the decision-making and monitoring system.

The technical result, which manifests itself in solving the above problem, is to improve the quality of the created credit scoring models. An additional technical result that manifests itself in solving a technical problem is an increase in the speed of developing credit scoring models. It also reduces the need for the amount of resources required to develop and maintain models, increase the speed and ease of implementation of models into the industrial circuit, as well as ensure monitoring of the operation of models and prompt response to changes.

The specified technical result is achieved due to the implementation of a method for the development and management of scoring models, in which data is obtained for a given period of time, containing factors affecting the scoring model;

after that, the obtained data are divided into samples for the development, validation and testing of the scoring model;

then, the transformation of factors is carried out by establishing relationships between the groups of values of the transformed factor and the levels of defaults;

then exclude from the samples at least one transformed factor that correlates with at least one other factor;

form a credit scoring model by training binary multiple logistic regression;

automatically select cut-off zones for at least one scoring model for its installation in the credit procedure.

In some embodiments, data is obtained for a predetermined period of time from a user's mobile communications device.

In some embodiments, the implementation of splitting the received data into samples obtains time-disjoint portions of the original population or random subsamples.

In some embodiments, the factors influencing the scoring model are annual income and / or outstanding debt and / or ownership of real estate and / or car ownership and / or seniority in last place and / or age.

In some embodiments, the factors influencing the scoring model are discrete or continuous.

In some embodiments, the implementation of the transformation of factors determines the degree of deviation of the default rate for the data group from the average default rate for the entire sample.

In some embodiments, the implementation of the transformation of factors by factors included in the excluded list, an algorithm for splitting the factor values with a new set of settings is started.

In some embodiments, when the transformed factors are excluded from the samples, a table is formed with the values of the pair correlation coefficients of the transformed factors.

In some embodiments, excluding transformed factors from the samples in the cycle selects the factor that has the largest number of factors correlated with it.

In some embodiments, when generating a credit scoring model, a logistic model is constructed using stepwise regression to select the resulting set of factors.

Brief Description of Drawings

Features and advantages of the present invention will become apparent from the detailed description of the invention which follows and the accompanying drawings.

FIG. 1 shows an example of an implementation of a method for developing and managing scoring models in the form of a block diagram.

FIG. 2 shows a high-level exemplary diagram of a method for developing and managing scoring models.

The main core is made up of two blocks - this is retraining and selection / adjustment of cut-off zones, and without adaptation of cut-off zones, it is impossible to organize automatic introduction of the model into the decision-making system. The results of these two blocks are integrated into a production environment (in this embodiment, SAS RTDM). In addition, each of these two blocks is subject to routine checks in the form of daily monitoring of the target indicator depending on the cut-off zones (approval level) and monthly model validation.

Detailed description of the invention

This invention can be implemented on a computer, in the form of an automated system (AS) or a computer-readable medium containing instructions for performing the above method.

The invention can be implemented as a distributed computer system.

In this solution, the system means a computer system, a computer (electronic computing machine), CNC (numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, well-defined sequence of computational operations (actions, instructions ).

A command processing device means an electronic unit or an integrated circuit (microprocessor) that executes machine instructions (programs).

A command processor reads and executes machine instructions (programs) from one or more storage devices. The role of data storage devices can be, but are not limited to, hard disks (HDD), flash memory, ROM (read only memory), solid state drives (SSD), optical drives.

A program is a sequence of instructions intended for execution by a computer control device or a command processing device.

Below will be described the terms and concepts necessary for the implementation of the invention.

Credit score is a method for modeling the credit risk of a borrower based on numerical statistical methods. The purpose of credit scoring is making decisions on granting loans to individuals or legal entities.

P-value is a value used when testing statistical hypotheses. In fact, this is the probability of error when the null hypothesis is rejected (errors of the first kind).

Representativeness is the correspondence of the characteristics of the sample to the characteristics of the population or the general population as a whole.

Representativeness determines how much it is possible to generalize the results of a study using a certain sample for the entire general population.

DR is the level of defaults. Calculated as the number of default cases in the group divided by the number of all cases in the group.

Bootstrap is a practical computer method for studying the distribution of statistics of probability distributions, based on the repeated generation of samples based on the available sample.

Default Probability - the probability of a default occurring on a transaction within one year from the date of the rating assignment / adjustment.

A sample is a set of transactions and their parameters that correspond to the given characteristics and represent a part of the analyzed general population.

Sample for training - a set of transactions and their parameters used to evaluate the model.

Sample for assessing stability - a set of transactions and their parameters used to assess the stability of the ranking ability of factors and their breakdowns.

Sample for testing - data for all available contracts for all available reporting dates. Defined in relation to the segment in which the model is developed.

General population - a collection of transaction-date pairs related to the selected segment.

Discrete factors are factors with a limited number of value options.

Continuous factors are factors with an unlimited number of possible variants of values.

Training sample - a set of deals and their parameters used to develop a model.

Factor transformation - replacing factor values with calculated values (rates, WOE) associated with an estimate of the probability of default related to the factor value.

Scoring score - the value of the indicator of the quality of transactions in terms of the probability of their default.

Test sample - a sample used to test the effectiveness of the resulting model (not involved in the development)

The transformation of factors is the same as the transformation of factors.

- 2 038056

PD is the value of the probability of default.

WOE (weight of evidence) is a value that characterizes the degree of deviation of the default rate for a group from the average default rate for the entire sample.

The computerized method for developing and managing scoring models, schematically shown in FIG. 1 includes the following steps.

Step 101: Get data for a given period of time containing factors that influence the scoring model.

User data may include the current state of accounts (including closed ones) - opening dates, current balances, term, currency, product type and name, number of renewals, current status, and so on, but not limited to.

Also, the obtained data may include monthly balances (at the end of each month) for each account for the last period of time (for example, for the last six months), all transactions for the same period with the amount, type and subtype, with the debit / credit attribute.

The above data, which is a sample, can be obtained from a user's mobile communication device, for example, such as a tablet, mobile phone, smartphone, or from an automated system of a financial and credit institution in which the data is stored.

Based on the received user data, the credit score is automatically determined, i.e. predicting non-repayment of the issued loan by the user. For this, a training sample is used: a set of objects (users), each of which is characterized by a set of characteristics (such as age, salary, type of loan, account balance, monthly balances, past defaults, etc.), as well as a target feature. The target attribute could be, for example, a loan delinquency. If this target feature is simply the fact that a loan is not repaid (takes the value 1 or 0, i.e. the financial institution knows about its customers, who repaid the loan and who did not), then this is a (binary) classification problem. If you know how long the client has delayed with the loan repayment, and you want to predict the same for new clients, then this will be a regression task.

For each group of accounts (deposits and other accounts), the following data or factors can be taken into account:

number of accounts;

number of accounts with the Active status;

number of accounts with the Closed status;

number of accounts with the Account arrested status;

Worst status for all client accounts;

the number of accounts in foreign currency;

number of accounts in precious metals;

minimum term on accounts;

average term on accounts;

maximum term on accounts;

minimum term for current accounts;

average term for active accounts;

maximum term for active accounts;

the weighted average of the current balance in rubles term of the agreement;

the total amount of current balances;

the maximum amount of the balance for all accounts;

the weighted average for the current balance share of foreign currency accounts;

the weighted average for the current balance of the share of accounts in precious metals;

time in days since the earliest account was opened.

A person skilled in the art will appreciate that the above dataset is exemplary and, in some embodiments, may differ from the above.

Next, at least one sample is formed to develop a scoring model. For this, the most relevant loans issued in one calendar year are used, which are in the portfolio for at least 12 months. Since scoring models are designed to predict the behavior of all borrowers, developing it solely on issued applications may lead to inaccurate results. In this case, the model will be trained on a biased sample, therefore, applications are analyzed for which the previous scoring model failed. In order to account for these rejections, a certain percentage of the worst claims for which the previous model was rejected is added to the sample for developing a scoring model. All such orders are considered default by default.

Step 102: split the obtained data into samples for training, validation and testing of the scoring model.

At this stage, the initial data set is divided into training, validation and test sets in a given ratio. In the future, the training sample is used at all stages of the process, the validation sample is used to select the most stable factors and the final quality check of the scoring model, and the test sample is used for comprehensive independent testing. Samples in non-3 038056 of which embodiments may be generated as sequential, time-disjoint portions of the original population, or as random subsamples.

Step 103: transformation of factors is carried out by establishing relationships between groups of values of the transformed factor and levels of defaults.

Factors used as inputs to scoring models and potentially related to a user's creditworthiness may include, but are not limited to, annual income, outstanding debt, ownership of real estate or a car, length of service in last place, age, etc. ...

Among the factors that describe the data of a loan application, most are usually discrete (education, gender, marital status, purpose of the loan, type of housing ownership, occupation, etc.). At the same time, if some factors lend themselves to some ordering (for example, education - we can assume that the higher the level, the greater the value of the variable), then for others there is no meaningful linear order (for example, marital status or the purpose of the loan). Consequently, such variables cannot be considered even approximately continuous, since their values are the numbers of answers to the corresponding questions, which can be arranged in an arbitrary order. If the used scoring model requires the use of continuous variables, then the discreteness of the variables can be bypassed by replacing them with a larger number of variables taking values from 0 to 1.

The transformation of each factor under consideration consists in replacing its values with a calculated value - WOE.

WOE - weight of evidence, characterizes the degree of deviation of the default level for a group of data from the average default level for the entire sample. Thus, each factor is replaced by its corresponding WOE factor as follows:

W OE factor _i = W OE; (f), where f is the factor under consideration, i is the number of the group of values of the factor f, WOEi (f) is the WOE value corresponding to the group of values i. In some embodiments, the implementation of the indicator WOE can be any value. Positive WOE values indicate that the segment under consideration has a lower default rate than the sample as a whole (the larger the WOE, the lower the default rate). A WOE value less than zero indicates that the segment in question has a higher default rate than the sample as a whole. The WOE values for group i can be determined as follows:

where NG (i) and NG are the number of non-default observations in group i and across the entire sample, respectively, NB (i) and NB are the number of default observations in group i and across the entire sample, respectively.

If NG (i) = 0 or NB (i) = 0, then the WOE value for the group is determined by the formula:

For continuous factors, the grouping is done so that observations with a comparable default rate (DR) fall within each range. As a result of the grouping process, the continuous factor is divided into several groups, for each of which it is possible to estimate the level of defaults on the basis of observations that fall within this range.

The grouping of variables with a discrete set of values is carried out in the same way as the grouping of continuous factors - based on the comparable default rate (DR). Each group can include one or more factor values. The default rate is calculated for all cases included in the group.

Using WOE factors has the following advantages.

Linearization of factors in accordance with the premises of logistic regression.

Automatic handling of missing values: they are either combined with the most similar group in terms of default, or act as a separate group. In the case when the missing value is not interpretable or is absent in the sample, then it belongs to the group with the worst risk level.

Automatic processing of anomalous values, since they are not able to negatively affect the model and their actual value is not used in the model. They will be included in the model as an element of one of the extreme groups, characterized by their WOE-value, based only on the ratio of default and non-default observations in the group.

- 4 038056

The ability to assess and control the logical direction of the relationship between factor values and the level of defaults (business logic), which helps to ensure that the final scores are meaningful (for example, older people tend to score more points than young people). Logical connections are confirmed by business experience, therefore, they allow you to get a more stable model.

Reduces the risk of overfitting. The model does not include every random change in the data, which would be the case for non-grouped attributes. Such a model is more flexible and able to withstand some changes in the population, which provides stability over a longer period of time.

The initial grouping of factor values can be done using one-way decision trees. This makes it possible to increase the discriminating ability of the obtained factors in comparison with manual groupings, since the obtained groups will be maximally homogeneous within and differ among themselves based on the statistical criterion used.

The discriminating power of a factor is understood as its ability to differentiate between default and non-default observations. The Gini index can be used to assess the discriminatory power of a variable.

Based on the practice available in the prior art, in terms of the interpretability of the factors used in scoring, it is necessary to pay attention not only to the ranking ability of the WoE-transformed factors, but also to their business logic. For this reason, at this stage, not only the automatic splitting of the factor values and the calculation of WoE for them occurs, but also the verification of the resulting splits into business logic. If the resulting partition does not pass this check, then the algorithm tries to obtain a new partition using alternative settings. The method for obtaining the final WoE factors includes the steps below.

First, partitions of factor values are started with the specified set of settings.

Then, the obtained groups are merged according to the proximity of the WoE values if the WoE distance between the groups does not exceed a predetermined threshold. For interval factors, the order of the groups ordered by factor values is also taken into account. The factors that have only one group left after the amalgamations are transferred to the list of excluded.

The next step is to merge small groups according to a predetermined threshold with the WoE closest group. For interval factors, the order of the groups ordered by factor values is also taken into account. The factors that have only one group left after the amalgamations are transferred to the list of excluded. After each merge, you need to return to the second point.

It is also important to merge the obtained groups according to the proximity of the WoE values if more groups are formed than the initially specified maximum number for this predictor. For interval factors, the order of the groups ordered by factor values is also taken into account. The factors that have only one group left after the amalgamations are transferred to the list of excluded. After each merge, you need to return to the second point.

In some embodiments, monotonicity, non-monotonicity conditions, and risk directions for interval variables are checked in accordance with the guide. Factors that do not meet the conditions from the reference book are transferred to the excluded list.

In some embodiments, the implementation checks the minimum number of groups allowed. If there are fewer groups available for a variable than the initially specified minimum allowable number, then it goes into the list of excluded.

In some embodiments, the implementation checks the conditions of the risk ratio in different groups for categorical and binary variables in accordance with the reference book (business logic check). Conditions are set using a special language that allows you to describe risk ratio patterns in groups of any complexity. Factors that do not meet the conditions from the reference book are transferred to the excluded list.

In some embodiments, the drop in the Gini coefficient is checked. If this coefficient for the predictor on the validation sample is less than the initially specified threshold value or falls in comparison with the Gini coefficient on the training sample by more than a specified number of percent, then such a factor goes into the list of excluded.

In some embodiments, the implementation checks the stability of the sequence of the groups, ordered by the WoE. There is a comparison of the training sample and 20 samples randomly selected from the combination of training and validation. Factors that are found to be unstable in the order of the groups, ordered by WoE, go to the excluded list.

For the factors included in the excluded list, it is necessary to run the algorithm for splitting the factor values with a new set of settings. If there are no available settings, or all of them have already been checked, then the formation of partitions is considered complete. The number of settings is determined by the capabilities of the used statistical package, for example, based on SAS Enterprise Miner. Thus, based on the results of applying the algorithm, WoE factors are formed. Initial factors that did not pass the test with any set of split settings are excluded from the process.

- 5 038056

Step 104: Eliminate from the samples at least one transformed factor that correlates with at least one other factor.

Pairwise correlation analysis is used to identify collinear relationships between variables. The presence of correlations between factors increases the standard deviations of the regression coefficients, which reduces their robustness and reliability in multivariate analysis. For correlation analysis, a correlation matrix is calculated - a table with the values of the pair correlation coefficients of the converted WOE factors. Analysis of this table makes it possible to determine the variables that have high linear relationships with other factors. The value from which the correlation coefficients are recognized as high is set in the reference book. The recommended value, starting from which the correlation coefficients are recognized as high, is in the range from 0.5 to 1 modulo. Of each pair of correlating factors, only one should be left based on either a higher individual predictive ability or a greater importance of the factor from the point of view of business logic. The system uses the following algorithm: the cycle selects the factor that has the largest number of factors correlated with it (the correlation value is higher than the selected threshold). If there are several of them, then the factor with the smallest value of the Gini index is selected from them. This factor is excluded from consideration. After that, the next factor is selected with the largest number of remaining factors correlated with it and the smallest value of the Gini index. Thus, at the exit from the cycle, there are factors without correlations above the selected threshold. This approach provides the largest number of uncorrelated factors in the final list of factors for modeling.

Step 105: build a credit scoring model by training binary multiple logistic regression as follows:

PD (Y l | X _lf X ₂ ..... X „) _{1 + exp (} _ _{(Po + pA + fcX2 +} ... _{+ PnW}

...

where Y is a dependent variable (a sign of default), Y = 1 is a default event, ХцХ ₂ ,, Xn is a set of independent explanatory WOE factors, β ₀ , β1, β ₂ , the probability of default.

en - coefficients of logistic regression, PD ...,

The values of the probability of default (PD - Probability of Default) are located in the interval [0, 1]. It shows the probability of default for each calculated rating. In some embodiments, the implementation of the values of the probability of default can range from 0 to 100 in percentage or numerical equivalent. Despite the absence of correlated pairs excluded in the previous step, multicollinearity can occur between the factors of the scoring model; therefore, at the stage of constructing a scoring model, it is necessary to check its absence. In addition, since the scoring model is developed on the basis of WoE factors, and the more WoE, the less the risk, it is necessary to check the correctness of the sign of the coefficient in the scoring model (all regression coefficients must be negative). In addition, it is required to ensure high stability of the model, therefore the significance of each of the factors included in it is checked using the statistical bootstrap procedure: each of the factors should be significant based on the Wald statistics in at least 85% of cases. The method of forming the final scoring model is as follows.

Based on all the factors that have reached this stage, a logistic model is built using stepwise regression to select the final set of factors.

For such factors, the Variance Inflation Factor (VIF) is calculated.To determine VIF, it is necessary to evaluate a linear regression model, where the factor under consideration will be the dependent variable, and the remaining factors included in the model will act as independent variables. The final VIF value for the factor can be found using the formula:

VIF = ------,

- R ² 'where R ² is the coefficient of determination of the model described above. A variable whose VIF value is greater than the specified value and the value of the Gini coefficient is minimum is excluded. The first and second steps are repeated until all factors included in the model have a VIF value below the specified value.

Then a check is carried out for the presence of factors with a positive sign of the regression coefficient. If they are found, the factor with the minimum value of the Gini coefficient is eliminated, after which it is necessary to return to the first step. If there are no such factors, then you should go to the next point.

Next, the training and validation samples are combined. Several dozen samples of the same size as the training one are randomly selected from their combination. On each of the obtained samples, the scoring model is trained with the current set of factors. If there are factors that are significant, according to Wald's statistics, in less than 85% of cases, then the one with the lowest Gini coefficient is excluded. After exclusion, you must return to the first step. If there are no such factors, then the scoring model is considered to be successfully built.

Thus, the algorithm makes it possible to automatically develop scoring models that meet all reasonable quality requirements. In addition, it ensures that each factor will correspond to the business logic described in a special reference book.

In some embodiments, automatic model validation is performed in accordance with any statistical model validation technique known in the art. At this stage, quantitative tests are calculated to assess the quality of the model. The validation process uses the test sample generated in step 102 and the population. If validation is passed, go to step 107, otherwise the system user is notified that the validation was not passed, as well as a detailed report on the identified deficiencies. By varying the algorithm settings, the user can adjust the modeling approaches and ensure the success of the next validation.

The choice of the optimal value of the cut-off threshold depends on the cost of committing an error of the first and second kind during classification. The model should more accurately classify bad borrowers, since in credit scoring, the price of a first-order error is higher. Decreasing the cutoff threshold in the model will increase the sensitivity, i.e. the model's ability to correctly identify those borrowers who will be in arrears. The optimal cutoff point can be taken as a balance point between sensitivity and specificity.

Step 106: automatically select cut zones for at least one scoring model to fit into the credit procedure.

Next, automatic selection of cut-off zones for scoring models by scoring points is carried out for their installation in the credit procedure. The algorithm for selecting clipping zones consists of two parts: external and internal. The outer part is responsible for iterative enumeration of cut-off levels, the inner part is responsible for calculating the expected approval level of a loan application corresponding to the current set of cut-off points. It is worth noting that not only the approval level can act as a criterion for the internal part of the algorithm, but any indicator of interest depending on the cut-off levels, for example, the level of risk or NPV of the portfolio. The algorithm works on a historical sample of data on loan applications. Due to the fact that the level of approval is characterized by seasonality within a week, in this invention we are talking about a target level of approval only within seven days, since otherwise, you will have to define it separately for each day of the week. Based on this, the number of days for which the history of applications is reviewed should be a multiple of seven. Suppose a combination of three models is used in the credit decision making process:

1) the quality of credit history or credit bureaus scoring (CRI-scoring);

2) personal data (application scoring);

3) tendency to fraud or FDC-scoring (Fraud Detection Card Scoring).

Suppose we have a combination of cutoff scores for the bid, FDC, and BCI scoring models. Let (t1, t2, t3) be the value of corrections for the cutoffs for the corresponding models, and (n1, n2, n3) - the number of successive repetitions of the adjustment for each of the corresponding models. Then the external algorithm for selecting cutoff points will be as follows. Sequentially, for each of the scoring models, it is necessary to carry out the following actions:

1) add the corresponding t correction from (t1, t2, t3) to the cutoff level for this model;

2) run the internals of the algorithm described below to calculate the expected level of approval;

3) if the deviation of the expected level of approval changed direction, then choose a combination of cut-off levels according to the models of application, FDC and CRI scoring, at which the deviation of the expected level of approval is the smallest (in fact, the choice is made from the last two tested combinations);

4) if the deviation of the expected level of approval from the target did not change direction and the first item was repeated less than n out of (n1, n2, n3) times, then go to the first item, i.e. to adjust the next scoring model.

In some embodiments, the above procedure is repeated until a target approval level is achieved or an upper / lower bound for scores is reached for each of the models.

As part of the internal part of the algorithm, the change in the approval level is assessed when the cut-off points change according to the running scoring models. As noted earlier, the effect of a change in cut-off zones can be assessed on various indicators, be it risk or profitability, but in any case, it is necessary to assess who will be approved within the new cut-off zones and who will be refused (or the likelihood of one of these events). In this regard, an algorithm for assessing changes in the level of approval will be considered.

As a rule, the decision-making system (DMS) in a financial and credit institution is a sequence of checks and application of rules and may include the following stages of the application process:

- 7 038056

1) failure according to minimum requirements, based on data from the Hunter system, stop list, etc .;

2) use of bid, BKI and FDC scoring;

3) application of models of trustworthiness;

4) underwriting;

5) refusals at subsequent stages.

For this reason, in order to assess the level of approval in case of redefining the actual scoring failures on applications, it is necessary to know the solution to them at each of the stages following the second stage (using the scoring). For any application approved for all working scoring models (application, BKI-, FDC-scoring, etc.), the necessary information is available about the process of its passage through the subsequent stages of the MSS. For applications for which at least one of the models was rejected, there is uncertainty regarding the subsequent stages, since such applications do not reach these stages. In order to eliminate this uncertainty, within the framework of the algorithm, refusals are simulated after the scoring stage for applications for which a refusal was previously received. The algorithm can be thought of as a sequence of the following actions.

1. For applications that have reached the scoring stage, a simulation of failures is performed for three types of models with new cutoff points. All applications for which an actual refusal was received at the scoring stage and approval for all models during the simulation are marked (they require a separate modeling of the probability of refusal at the subsequent stages of the PRM).

2. Modeling of the probability of failure is carried out at the stage of applying the reliability model. To build the model, applications are used that have successfully passed the scoring procedure before changing the cutoff points.

3. Modeling of the probability of failure at the underwriting stage is carried out. To do this, in addition, applications for which a refusal was received at the stage of applying the reliability models are excluded from the previous sample.

4. Modeling of the probability of failure at subsequent stages is carried out. In addition, applications for which a refusal was received at the underwriting stage are excluded.

5. The probability of refusal after passing the scoring procedure is calculated for marked applications that require separate modeling (clause 1).

In order to determine the probability of rejection for applications that were rejected at the scoring stage, the following formula is used:

Ppeject Pblag + (1 Pblag) * Punderr + (1 ^- Pblag) * (1 Punderr) * Pnext "where P _blag is the probability of rejection for the application according to the trustworthiness model; P _underr is the probability of rejection for the application at the underwriting stage; P _next is the probability of refusal for the application at subsequent stages.

To determine the likelihood of approval for an application, the probability of rejection is subtracted from one. After that, the approval level is calculated as the ratio of the number of approved applications (the sum of the approval probabilities) to the number of all applications. When this value is averaged over the portfolio under consideration, a level of approval is obtained for the selected cut-off zones.

If we choose the risk level as the target indicator, then the resulting value must be multiplied by the risk level obtained from the PD model. When this product is averaged, the level of risk in the portfolio issued for the selected cut-off zones is obtained.

After automatic selection of cut-off zones, these zones are optimized for different portfolio segments. The principle of operation of the optimization algorithm is based on an iterative optimization calculation of the optimal decision thresholds for individual customer segments in terms of the ratio Level of approval - level of risk. Below are the main prerequisites that are critical for the obtained results of the algorithm.

1. The level of risk is assessed as the average level of the probability of delinquency within each segment.

2. The forecast of the probability of delinquency is made on the basis of the latest available data, taking into account segmentation.

The main idea of the calculation algorithm is to iteratively shift the cut-off threshold for a particular client segment, which ultimately leads to an increase in the overall level of approval while maintaining the current level of risk.

At each iteration of the algorithm, the optimal client segment from the point of possible improvement of the AR / DR ratio is considered, within which the spin-spin operations take place in a given sequence with a predetermined step of 15 points (this step can be increased in accordance with the rules for forming a cycle, but no more than up to 60 points). Thus, an optimal neighborhood of the base cutoff threshold is sought, leading to an improvement in the overall AR / DR ratio.

Next, the scoring model (or models) and cut-off zones are introduced into the industrial environment.

As a result of automatic selection of cutoff levels, the target approval level may not be achieved with the required accuracy. As a result, it is necessary to adaptively adjust the scoring cutoffs to maximally approximate the target AR. To do this, 7 full days after the last change in the target approval level by 8 038056 or immediately after adjusting the cutoff points without changing the target approval level, an adaptive adjustment of the obtained cutoff points begins. It continues until the actual level of approval is within acceptable limits at least once. The correction is made according to the following scheme. Add the following value to all the clipping zones:

тт (20 ', Correction * ------: ---------- K), highAR - AR _t arget where Δ = AR - highAR, if the last AR out of the established boundaries occurred in a large direction;

—Min (20, Correction * --------------- t),

ARtarget ^ War where Δ = low _AR - AR, if lower. The size of the adjustment is set by experts in the reference book empirically.

Adjustments are run daily until the approval level returns within the acceptable range between highAR and lowAR values. In some embodiments, a monthly automatic validation of the scoring model is performed in accordance with a financial institution's methodology. If the model fails validation, it is sent for retraining.

In some embodiments, the level of approval is monitored on a daily basis. This methodology is suitable for monitoring not only the level of approval, but also other indicators, such as risk. The monitoring considers the moving average level of approval with a window of 7 days as a time series, the elements of which are modeled using independent normally distributed random variables. In order to maintain the level of approval at any target level, first of all, a criterion is needed with which it can be understood that a change has really occurred, since this indicator has natural fluctuations. Based on this, the CUSUM test can be used to identify deviations in the target approval level. To do this, it is determined that there is a moment of change in the level of approval (disagreement) - this is the moment when the distribution law in the stream of incoming data on the level of approval changes. In this invention, the change in the average value is considered. Let X _n , n> 1 be a sequence of observations that are modeled using independent normally distributed random variables, θ е [1, n] is an unknown moment in time at which the distribution of observations changes from f ₀ ~ N (μ ₀ , σ ² ) to f1 ~ N (μ1, σ ² ), and n is the current moment in time. Since the exact time of the disorder is unknown, the hypothesis H0 - there is no disorder on the segment [1, n], and H1 - the disorder occurred on the segment [1, n]. To distinguish between these two hypotheses, it is necessary to define a generalized likelihood ratio criterion:

V; Α ( ^χ ί) Г

T _n = max) log. ... ... > С _о , ^П Oe [l, n] Lj ^а fjxd ⁰ ι = θ where with the help of С ₀ the number of false alarms is controlled. This expression is known as the CUSUM test. The resulting test record will be computationally ineffective, but in the case of independent random variables, the statistics can be represented by a recurrent relation:

ΙΛ ^(x n)

T _n = _t ax [T _n _ _{1 +} 1od '/ ^, 0}

J0 \ ^x n)

Since we assume that f ₀ and f1 are normally distributed:

I'T FROM D1 + TO \ Ώ

T _n = max + x _n --- -, 0. ν \ ί /)

Let μ1 = μ ₀ ± δ, where δ is the permissible error, which is chosen depending on which deviation we consider acceptable. Then the expression for calculating CUSUM can be rewritten as:

Xn - Up to n-1 for downward deviations and for upward deviations.

The final solution is found from the condition max (T _n ⁺ , Tn ^- )> h = e (C0).

The described approach makes it possible to detect deviations of the approval level from the target level with minimal delay and a small number of false positives.

If the approval target changed less than 7 days ago, we cannot

- 9 038056

CUSUM test, as there is no 7-day moving average of approval observations not including the days before the adjustment. In addition, you need to insure yourself against incorrect operation of the CUSUM test. For this, an alternative, simpler test is used, based on setting the boundaries of the acceptable range for the observed indicator. Aspects of the present invention may also be implemented with a data processing device being a computer or system (or means such as a central / graphics processing unit or microprocessor) that reads and executes a program written to a memory device to perform the functions of the above-described embodiment (s) implementation, and the method shown in FIG. 1, the steps of which are performed by a computer or device by, for example, reading and executing a program recorded on a memory device in order to perform the functions of the above-described embodiment (s). For this purpose, the program is written to a computer, for example, via a network or from a recording medium of various types serving as a storage device (for example, a computer-readable medium). The data processing device may have additional features or functionality. For example, the data processing device may also include additional storage devices (removable and non-removable) such as, for example, magnetic disks, optical disks, or tape. Storage devices can include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules, or other data. Storage device, removable storage, and non-removable storage are examples of computer storage media. Computer storage media include, but are not limited to, random access memory (RAM), read only memory (ROM), electrically erasable programmable ROM (EEPROM), flash memory or other technology, CD ROM ( CD-ROM), versatile digital disks (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and to which the data processing device can access. The processing device may also include input device (s) such as a keyboard, mouse, stylus, voice input device, touch input device, and so on. Output device (s) such as display, speakers, printer, and the like may also be included in the system.

The data processing device contains communication connections that allow the device to communicate with other computing devices, such as over a network. Networks include local area networks and wide area networks along with other large scalable networks, including, but not limited to, corporate networks and extranets. A communication connection is an example of a communication medium. Typically, a communication medium can be implemented with machine-readable instructions, data structures, program modules, or other data in a modulated information signal such as a carrier wave or other transport mechanism, and includes any information delivery medium. The term “modulated information signal” means a signal that has one or more of its characteristics changed or set in such a way as to encode information in this signal. By way of example, but not limited to, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. The term computer-readable media, as used in this document, includes both storage media and communication media. The series of processes described in this document can be performed using hardware, software, or a combination of both. When processes are executed by software, a program in which a sequence of processes is recorded can be installed and executed in the memory of a computer embedded in specialized hardware, or a program can be installed and executed on a general-purpose computer that can execute various processes ...

For example, the program can be pre-recorded on a recording medium such as a hard disk or ROM (read only memory). Alternatively, the program can be temporarily or permanently stored (recorded) on a removable recording medium such as a floppy disk, CDROM (compact disk intended for playback only), MO (magneto-optical) disk, DVD (digital versatile disk), magnetic disk. or semiconductor memory. The removable recording medium can be distributed in the form of so-called retail software.

The program can be installed from the removable recording medium described above to a computer, or it can be cabled from a download site to a computer, or it can be transferred to a computer via network communication channels such as a LAN (Local Area Network) or the Internet.

The computer can receive the program transferred in this way and can install it on a recording medium such as a built-in hard disk. The processes described in this document can

- 10 038056 be executed sequentially in time, as described, or can be executed in parallel or separately, depending on the processing characteristics of the device executing the processes, or as required. The system described in this document is a logical collection of multiple devices and is not limited to the structure in which these devices are installed in a single housing.

Claims (10)

CLAIM 1. A computerized way to develop and manage scoring models, including the following steps: receive data for a given period of time, containing factors influencing the scoring model; split the obtained data into samples for the development, validation and testing of the scoring model; carry out the transformation of factors by establishing relationships between the groups of values of the transformed factor and the levels of defaults; exclude from the samples at least one transformed factor that correlates with at least one other factor; form a credit scoring model by training binary multiple logistic regression; automatically select cut-off zones for at least one scoring model for its installation in the credit procedure. 2. The method according to claim 1, characterized in that data is obtained for a predetermined period of time from a user's mobile communication device. 3. The method according to claim 1, characterized in that when performing the division of the obtained data into samples, parts of the original population that do not overlap in time or random subsamples are obtained. 4. The method according to claim 1, characterized in that the factors influencing the scoring model are annual income and / or the amount of outstanding debt and / or ownership of real estate and / or ownership of a car and / or length of service in the last place , and / or age. 5. The method according to claim 1, characterized in that the factors influencing the scoring model are discrete or continuous. 6. The method according to claim 1, characterized in that during the transformation of factors, the degree of deviation of the level of defaults for the data group from the average level of defaults for the entire sample is determined. 7. The method according to claim 1, characterized in that when transforming factors by factors included in the excluded list, an algorithm for splitting the factor values with a new set of settings is launched. 8. The method according to claim 1, characterized in that when the transformed factors are excluded from the samples, a table is formed with the values of the coefficients of pair correlations of the transformed factors. 9. The method according to claim 1, characterized in that when excluding transformed factors from the samples, the factor that has the largest number of factors correlated with it is selected in the cycle. 10. The method according to claim 1, characterized in that during the formation of the credit scoring model, a logistic model is built using stepwise regression to select the final set of factors.