JP2016122332A - Process evaluation device and process evaluation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a process evaluation device and a process evaluation program which can evaluate a process while considering a causal relation between variables and specify the place of a process showing abnormality from a large amount of data.SOLUTION: The process evaluation device includes: a reference causal model definition part for showing the flow of work tasks of business and development, and acquiring a reference causal model defined by a user who evaluates a process about the process characterized by variables that can quantitatively be measured; an evaluation causal model construction part for acquiring a plurality of variables of a process as evaluation data, using a causal model construction algorithm of a Bayesian network, and constructing an evaluation causal model; and a model score part for comparing a causal relation between variables in the reference causal mode with a causal relation between variables in the evaluation causal mode, and calculating scores corresponding to contradiction of the causal relation between the variables in the evaluation casual mode.SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to a process evaluation apparatus and a process evaluation program.

  In order to improve the efficiency and quality of business and development, it is necessary to optimize the process. In order to optimize, the current process is evaluated, an improvement point is identified, and an improvement plan is made.

  Traditionally, process evaluation methods include qualitative approaches that compare with best practices and quantitative approaches that capture variables that affect results through correlation.

  However, the correlation does not indicate a cause-and-effect relationship, and the evaluation of the process and the identification of the improvement part take time, and vary from person to person.

  Furthermore, when checking the work tasks for each process one by one, there is a problem that it takes time to make a decision, the result of improvement and the cause are intricately intertwined, and it is difficult to identify the cause.

JP 2006-127059 A

  A problem to be solved by the present invention is a process evaluation apparatus capable of evaluating a process while considering a causal relationship between variables, and capable of specifying a part of a process showing an abnormality from a large amount of data, and To provide a process evaluation program.

  The process evaluation apparatus according to the embodiment captures a reference causal model defined by a user who evaluates a process characterized by a variable that can be quantitatively measured, showing a flow of work tasks of work and development. In the reference causal model definition unit, the evaluation causal model construction unit that takes in a plurality of the variables of the process as evaluation data and constructs the evaluation causal model using the causal model construction algorithm of the Bayesian network, and the reference causal model A model scoring unit that compares a causal relationship between variables and a causal relationship between variables in the evaluation causal model, and obtains a score corresponding to a contradiction in the causal relationship between the variables in the evaluation causal model.

It is a block diagram which shows schematic structure of the process evaluation apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the relationship between a process and a variable. It is a figure which shows the example of the production | generation of the causal model by multivariable modeling. It is a figure explaining the function of the structure learning part of a Bayesian network. It is a figure explaining construction of an evaluation causal model by a causal model construction algorithm. It is a figure explaining the classification | category of the relationship between variables. It is a figure explaining the comparison of a reference causal model and an evaluation causal model. It is a figure explaining calculation of the correlation coefficient between two variables. It is a flowchart explaining the flow of the evaluation process of the evaluation causal model in a process evaluation apparatus. It is a figure which shows the standard causal model defined by the evaluator of the business process which is a user of a process evaluation apparatus in a standard causal model definition part. It is a figure which shows the evaluation causal model constructed | assembled in an evaluation causal model construction part. It is a figure explaining the kind of directed edge classified after the comparison in a model scoring part. It is a figure explaining the score calculation after weighting. It is a figure which shows the candidate of the improvement location ranked in the improvement location recommendation part.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same portions are denoted by the same reference numerals, and redundant description is omitted.

  First, main terms used in the present embodiment will be described.

  A “graph” refers to a set consisting of a set of vertices and edges connecting the vertices.

  “Causality” refers to the relationship between cause and effect.

  “Process” refers to a flow of work tasks (processes) for business and development.

  “Variable” refers to an index (quantitatively measurable) that characterizes a process.

  “Evaluation data” refers to a set of data composed of a plurality of variables.

  In the present embodiment, based on quantitative data, a reference causal model is compared with an evaluation causal model constructed using a structural learning part (causal model construction algorithm) of a Bayesian network, and the relationship between variables is compared. Evaluate the causal model by grasping the difference and quantitatively show the goodness of the process.

  FIG. 1 is a block diagram showing a schematic configuration of a process evaluation apparatus according to an embodiment of the present invention. This apparatus is realized using a general-purpose computer (for example, a personal computer (PC) or the like) and software operating on the computer. The computer includes an engineering workstation (EWS) suitable for CAD (Computer Aided Design) and CAE (Computer Aided Engineering). The present embodiment shows the flow of work tasks for business and development on such a computer, and relates to a process characterized by a variable that can be quantitatively measured, and a reference causal model defined by a user who evaluates the process. A function that captures multiple variables of the process as evaluation data and uses the Bayesian network causal model construction algorithm to construct an evaluation causal model, and the causal relationship between the variables in the reference causal model and the evaluation causal model A function for comparing causal relationships between variables and obtaining scores corresponding to contradictions in causal relationships between variables in the evaluation causal model can also be implemented as a program for realizing.

  As shown in FIG. 1, the process evaluation apparatus 1 according to the present embodiment mainly includes a reference causal model definition unit 11, an evaluation causal model construction unit 12, a model scoring unit 13, and an improvement point recommendation unit 14.

  The reference causal model definition unit 11 considers temporal order relations and relations between variables for the process variables to be evaluated, and generates a “reference causal model” (details will be described later) as a process evaluator ( User).

<Relationship between processes and variables>
The process shows the flow of work tasks and development work tasks (processes). In this embodiment, the process is grasped by “variables” as indicators (things that can be measured quantitatively) that characterize the process. To do.

  FIG. 2 is a diagram illustrating an example of the relationship between processes and variables. In the example shown in FIG. 2, for example, the process named “inspection item creation” is characterized by variables named “creation man-hour” and “number of items”. Further, the example shown in FIG. 2 indicates that “inspection item creation” of the process has a dependency relationship with processes named “inspection environment maintenance” and “test item review”.

  A set of actual data obtained by measuring a plurality of variables such as “manufacturing man-hours”, “number of items”, “number of indications” and the like is called “evaluation data”.

<Causal model>
Each variable (event) that constitutes the target variable (event) is represented by a node at the apex, and the dependency between nodes, that is, the cause-result relationship (causal relationship) between multiple variables is the cause variable. A “causal model” is represented by a network structure (graph) illustrated by “directed edges” having a direction from 1 to the resulting variable. When a node is considered to be the center, a node having a directed side toward the node is called a “parent”, and a node having a directed side extending from the node is a “child”.

  FIG. 3 is a diagram illustrating a generation example of a causal model by multivariable modeling. In the example shown in FIG. 3, the causal relationship between a plurality of variables named “number of requirements”, “number of specification defects”, “number of design document pages”, “number of steps”, and “number of test items” is the network structure. It is expressed by. For example, since the directed edge is drawn from the variable “requirement number” toward the variable “specification defect number” and the variable “design document page number”, the variable “requirement number” is the “cause”, and the variable It can be seen that the “specification defect count” and the variable “design document page count” are “results”.

<Standard causal model>
The “reference causal model” is created and defined by the user of the process evaluation apparatus (business process evaluator) according to the present embodiment in consideration of the temporal order relation and the relation between variables. The reference causal model is represented by a network structure as shown in FIG. 3, for example, and can be expressed by a list in which a set of parent nodes connected to each child node is arranged on the program.

  In the standard causal model, the directed edge is expressed by the temporal order relation of work and the relation between variables. Here, the relationship between the variables may be determined based on the experience of the user in charge of the business process. For example, in the relationship between the variable “number of requirements” and the variable “number of pages of design document”, when the variable “number of requirements” to be developed is determined, the design document to be created is determined accordingly, and the number of pages increases and decreases. The directed edge is drawn from the “number of records” to the variable “number of design document pages”.

  The evaluation causal model construction unit 12 takes in “evaluation data” of the process to be evaluated, and constructs an “evaluation causal model” (details will be described later) using the structure learning part (causal model construction algorithm) of the Bayesian network. .

<Evaluation causal model>
The evaluation causal model is a causal model mechanically constructed using the structure learning part (causal model construction algorithm) of the Bayesian network based on the actual data measured in the process evaluation, that is, the evaluation data.

<Bayesian network>
The Bayesian network is a probability model in which a qualitative dependency between a plurality of random variables (events) is represented by a graph structure, and a quantitative relationship between individual variables is represented by a conditional probability. Each variable (event) constituting the target phenomenon is represented by a node, and the dependency relationship between nodes, that is, the causal relationship is illustrated by a directed side having the direction of the variable resulting from the cause. The direction of the directed edge in the evaluation causal model is drawn by statistically examining the stochastic dependence between variables. The graph structure used in the Bayesian network is a directed graph without a cycle, and each node is a random variable in order to indicate whether a certain variable (event) has occurred.

  In general, in a Bayesian network, a “structural learning” part that constitutes a causal model construction algorithm for constructing a model by machine learning from real data (in this embodiment, evaluation data), and cause → result or result → probability of cause However, in this embodiment, only the “structure learning (causal model construction algorithm)” portion is used. FIG. 4 is a diagram for explaining the function of the structure learning portion of the Bayesian network. As shown in FIG. 4, when the structure learning part gives multivariate data, it constructs a causal model that best explains the data.

FIG. 5 is a diagram for explaining the construction of an evaluation causal model by a causal model construction algorithm. FIG. 5A shows an example of the evaluation data, and FIG. 5B shows the constructed evaluation causal model. As shown in FIG. 5A, a large amount of data is obtained from each sample for the variables “total number of steps”, “number of newly developed steps”, and “number of days delayed”. These evaluation data are given as input data to, for example, statistical software R.
For learning of the network structure in the statistical software R, a full search algorithm called Greedy Search Algorithm and Tree Selection Algorithm is implemented as a model search algorithm. When there are multiple models that explain a certain phenomenon, the criteria for selecting the optimal model from among them are the information criteria, AIC (Akaike information criterion), MDL (Minimum Description Length) and so on. AIC derives the information criterion from the approximate approach from the expected log likelihood, and makes the optimal model by minimizing the information amount between the estimated model distribution and the true distribution. MDL asymptotically selects a true model or a model that maximizes prediction accuracy.

  The Greedy Search Algorithm is a search algorithm that uses the approximate algorithm Greedy Algorithm. The “parent” combination for each node is determined by Greedy Search, and a directed graph is constructed.

  In Greedy Search, a graph is created by adding one parent node to each child node. Under the created graph, the conditional probability is calculated and the information criterion is evaluated. The conditional probability is represented by P (A | B) and represents the probability of event A under the condition that event B occurs. In the example shown in FIG. 5B, the conditional probability of “number of new development steps” is the largest, and there is no parent node. The conditional probability of “delay days” is P (delay days | new development step number), “delay days” is a child node, and “new development step number” is a parent node. Similarly, the conditional probability of “total number of steps” is P (total number of steps | new development step number), where “total number of steps” is a child node and “new development step number” is a parent node. In this way, when the evaluation becomes high, it is adopted as a parent node. Repeat this to construct a network in an exploratory manner.

  As described above, the graph structure configured by linking each node can be expressed by a list in which a set of parent nodes connected to each child node is arranged on the program.

  The model scoring unit 13 compares the reference causal model with the evaluation causal model, and when there is a difference in the directed sides indicating the relationship between the variables, the causal relationship between the variables in the evaluation causal model is inconsistent. The evaluation causal model is scored and evaluated.

<Classification of associations between variables>
In this embodiment, paying attention to the difference from the causal relationship in the reference causal model as a reference, the relationship between variables is (1) reverse, (2) disappearance, (3) occurrence (forward direction), (4) occurrence It is classified into four types (reverse direction). FIG. 6 is a diagram for explaining the classification of association between variables. FIG. 6A shows a reference causal model, and FIG. 6B shows four types of classification. The reference causal model shown in FIG. 6A shows that the variable B can result from the variable A. The variable C is independent of the variables A and B and is not causal.

(1) Inversion Unlike the standard causal model, the variable A is the result of the variable B, resulting in a contradiction in the causal relationship. For example, the relationship where the variable “number of steps” is “cause” and the variable “number of failures” occurs as “result” is the relationship where the variable “number of failures” is “cause” and the variable “number of steps” occurs as “result”. This is the case. When “reversal” occurs, it can be considered that the structural review has occurred due to the occurrence of a defect, and that the development process has been reversed.

(2) Disappearance Unlike the standard causal model, the variable B does not change corresponding to the variable A and the causal relationship is lost. For example, there is no relationship between the variable “test” and the variable “number of defects”. If “disappearance” occurs, it can be considered that the original test is not functioning, even if the test is performed hard, there is no effect in identifying the defect.

(3) Occurrence (forward direction)
Unlike the standard causal model, this is a case where a forward causal relationship occurs between irrelevant variables that are not causal but independent. When “occurrence” occurs, it can be understood that there is a skip in the process. Or it can be considered that the process is different from the ideal development flow.

(4) Occurrence (reverse direction)
Unlike the reference causal model, this is a case where a causal relationship in the opposite direction occurs between irrelevant variables that are not causal but independent. When “occurrence (reverse direction)” occurs, the same interpretation as when forward occurrence occurs holds true.

<Scoring of evaluation causal model>
The model scoring unit 13 compares the causal relationship between variables in the reference causal model and the causal relationship between variables in the evaluation causal model, and obtains a score corresponding to the contradiction in the causal relationship between variables in the evaluation causal model. In the present embodiment, the model scoring unit 13 compares the reference causal model with the evaluation causal model, counts the number of directed edges of inversion, disappearance, and occurrence (forward direction, reverse direction), and scores the evaluation causal model ( Score).

  FIG. 7 is a diagram illustrating a comparison between the reference causal model and the evaluation causal model. In the example shown in FIG. 7, “reverse” occurs between variable A and variable B and between variable C and variable E, “occurrence (forward direction)” occurs between variable C and variable D, and between variable F and variable C. “Generation (reverse direction)” occurs, and “disappearance” occurs between variable A and variable D. On the other hand, no contradiction occurs between the variables A and C, between the variables D and E, and between the variables F and E.

  Therefore, for example, if the ideal process is deducted as 100 points and 10 points / directed side for a combination in which the relationship between variables is changed, in the example shown in FIG. The evaluation of the process is scored with 60 points. In this way, it is possible to confirm to what extent the current process is compliant with the ideal process.

<Weighting>
For combinations in which the relationship between variables has changed, it is preferable that the user sets a weight according to the degree (degree) of contradiction in the causal relationship between variables. The weighting is preferably, for example, reverse: 10 points, occurrence (reverse direction): 7 points, disappearance: 5 points, occurrence (forward direction): 3 points.

  When weighting is performed, in the example shown in FIG. 7, there are two directed edges where reversal occurs, one directed edge where disappearance occurs, and one directed edge where occurrence (forward direction) occurs. Therefore, the deduction points are (10 × 2) + (5 × 1) + (3 × 1) = 25 points.

The improvement point recommendation unit 14 ranks candidates for improvement points in the process based on the relationship between the process and variables and the evaluation data. First, a correlation coefficient between two variables is calculated for each directed side. FIG. 8 is a diagram illustrating calculation of a correlation coefficient between two variables. In the example illustrated in FIG. 8, the correlation coefficient between two variables having a causal relationship with the variable “total number of inspections” as “cause” and the variable “total defect density” as “result” is calculated. When the variable “total number of inspections” is x and the variable “total defect density” is y, the correlation coefficient can be obtained by the following equation.

  According to the above equation, the correlation coefficient between the variable “total number of inspections” and the variable “total defect density” is 0.75767.

  It is preferable to rank the improvement location candidates after calculating the correlation coefficient in the following order. (1) Sorting in ascending order of weight among related types, (2) Sorting in descending order of correlation coefficient within the same related type, (3) Process and variable using relationship between process and variable Take the correspondence.

  As described above, it is possible to identify a process likely to be problematic from the focused variable by ranking the improvement points. In addition, it becomes possible to identify the cause of the causal contradiction based on past knowledge and formulate improvement measures.

<Process flow>
The flow of the evaluation process in the process evaluation apparatus 1 configured as described above will be described. FIG. 9 is a flowchart for explaining the flow of the evaluation process of the evaluation causal model in the process evaluation apparatus 1.

  First, the reference causal model defined by the user is input to the reference causal model definition unit 11 (step S91).

  Next, the evaluation data is taken into the evaluation causal model construction unit 12, and the evaluation causal model is constructed by the Bayesian network construction support software provided in the evaluation causal model construction unit 12 (step S92).

  Next, the model scoring unit 13 takes in the reference causal model defined by the reference causal model definition unit 11 and the evaluation causal model constructed by the evaluation causal model construction unit 12, compares them, and reverses, disappears, or occurs. Is counted (step S93).

  Subsequently, the model scoring unit 13 calculates the score by reflecting the weighting of reverse rotation / disappearance / occurrence (step S94).

  Next, the model scoring unit 13 displays the calculated score on a display unit prepared outside (step S95).

  Next, the improvement point recommendation unit 14 calculates a correlation coefficient between variables (step S96).

  Next, the improved part recommendation unit 14 ranks the improved part candidates, outputs them (step S97), and ends the evaluation causal model evaluation process.

[Example]
Next, examples of the present embodiment will be described.

  FIG. 10 is a diagram illustrating a reference causal model defined by a business process evaluator who is a user of the process evaluation apparatus 1 in the reference causal model definition unit 11. The evaluator defines a reference causal model for the variables of the process to be evaluated, taking into consideration the temporal order relationship and the relationship between the variables.

  FIG. 11 is a diagram showing an evaluation causal model constructed in the evaluation causal model construction unit 12. The evaluation causal model construction unit 12 constructs an evaluation causal model for the evaluation data of the process to be evaluated, using Bayesian network structure learning (causal model construction algorithm). In the causal model construction algorithm, a directed edge is drawn between related nodes in consideration of the stochastic dependency between variables.

  FIG. 12 is a diagram illustrating the types of directed sides classified after the comparison in the model scoring unit 13. In the example shown in FIG. 12, between the variable “number of single / join check” and the variable “total number of steps”, between the variable “total number of defects” and the variable “total number of steps”, the variable “total number of defects” and the variable “new development”. "Reverse" occurs between the variable "total defect density" and the variable "total number of inspections" between the "number of steps", between the variable "total number of steps" and the variable "total number of inspections", the variable "number of newly developed steps" and the variable “Disappearance” has occurred between “delayed days”, and “occurrence (reverse direction)” has occurred between the variable “total number of defects” and the variable “number of unit / join test”. On the other hand, between the variable “number of new development steps” and the variable “total number of steps”, between the variable “total defect density” and the variable “total defect density”, between the variable “total defect density” and the variable “total defect number”, and the variable “ There is no contradiction between the “total number of defects” and the variable “days late”, and between the variable “total number of defects” and the variable “days late”. Accordingly, there are four directional sides for reverse rotation, two directional sides for disappearance, and one directional side for occurrence (reverse direction).

  FIG. 13 is a diagram for explaining the score calculation after weighting. In the example shown in FIG. 13, weighting of reverse rotation: 10 points, disappearance: 5 points, occurrence (forward direction): 3 points, and occurrence (reverse direction): 7 points is performed. Assuming that the ideal process is 100 points, the score of this embodiment = 100−Σ (weight × number of directed sides) = 100−57 = 43 points. Although the goodness of the process can be quantitatively shown from the size of the numbers, it can be confirmed that the process of this example is at a low evaluation level.

  FIG. 14 is a diagram showing the improvement point candidates ranked in the improvement point recommendation unit 14. The improvement location recommendation unit 14 ranks the improvement location candidates. First, as shown in FIG. 12, a correlation coefficient between two variables is calculated for each directed edge where a contradiction in the causal relationship between variables occurs. The correlation coefficient between the two variables is based on the calculation formula described above. The ranking after calculating the correlation coefficient is as follows: (1) Sorting in ascending order of weight among related types, (2) Sorting in descending order of correlation coefficient within the same related type, (3) Process and Using the variable relationship, the process and the variable are matched.

  In the example shown in FIG. 14, a pair of variables in a reverse relationship, a pair of variables in an occurrence (reverse direction), and a pair of variables in an erasure relationship are roughly arranged in the order of weight. Among a pair of variables, a pair of variables having a large correlation coefficient is arranged at the top.

  According to the example shown in FIG. 14, it is possible to specify a process likely to have a problem such as “general inspection item creation” and “general inspection” from the focused variable. Also, the temporal order relationship between the variable “total defect density” and the variable “total number of inspections” is broken. From these facts, it is assumed that there is a problem with the quantity and quality of the process “Comprehensive Inspection”, so it is desirable to formulate an improvement measure regarding the method of creating the process “Comprehensive Inspection”. it can.

  As described above, according to the present embodiment, in a business process or development process in which a plurality of related variables exist, the process can be evaluated while considering the causal relationship between the variables. Furthermore, it is possible to identify the location of the process that shows an abnormality from a large amount of data, which can be used for selection of an improvement location.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 ... Process evaluation apparatus 11 ... Standard causal model definition part 12 ... Evaluation causal model construction part 13 ... Model scoring part 14 ... Improvement location recommendation part

Claims (12)

A reference causal model definition unit that captures a reference causal model defined by a user who evaluates the process regarding a process characterized by variables that can be quantitatively measured, showing the flow of work tasks of business and development,
An evaluation causal model construction unit that takes in a plurality of the variables of the process as evaluation data and constructs an evaluation causal model using a causal model construction algorithm of a Bayesian network;
A model scoring unit for comparing a causal relationship between variables in the reference causal model and a causal relationship between variables in the evaluation causal model, and obtaining a score corresponding to a contradiction in the causal relationship between the variables in the evaluation causal model;
A process evaluation apparatus provided.   The process evaluation apparatus according to claim 1, wherein the reference causal model is defined with respect to the variables of the process to be evaluated in consideration of temporal order relations and relationships between the variables.   The process evaluation apparatus according to claim 1, wherein the reference causal model is represented by a program list in which a set of parent nodes connected to each child node is arranged in a network structure.   4. The process evaluation according to claim 1, wherein the presence or absence of the contradiction of the causal relationship is determined by a state of a directed side having a direction from the causative variable to the resulting variable. apparatus.   The causal model construction algorithm of the Bayesian network implements a Greedy Search Algorithm in the model search algorithm, and selects an optimal model using an AIC (Akaike information criterion) as an information amount criterion. The process evaluation apparatus described in 1.   The model scoring unit is configured such that the causal relationship between the variables in the reference causal model is reversed, the causal relationship between the variables is reversed, the disappearance of the causal relationship between the variables is lost, and independent. 6. The generation of a forward direction in which a forward causal relationship has occurred between the variables of the above, and the occurrence of a reverse direction in which a reverse causal relationship has occurred between the independent unrelated variables. The process evaluation apparatus according to any one of the above.   The process according to claim 6, wherein the model scoring unit can set weights in the order of inversion of the classification, generation in the reverse direction, disappearance, and generation in the forward direction according to the degree of contradiction in the causal relationship between the variables. Evaluation device.   The process evaluation apparatus according to claim 7, wherein the model scoring unit counts the number of directed sides having contradictions in the causal relationship between the variables, and calculates the score of the evaluation causal model by reflecting the weighting. .   The process evaluation apparatus according to claim 8, further comprising: an improvement location recommendation unit that ranks candidates for improvement locations of the process for each directed side inconsistent in a causal relationship between the variables.   The process evaluation apparatus according to claim 9, wherein the ranking in the improvement location recommendation unit is sorted in descending order of weight according to the degree of contradiction in the causal relationship between the variables.   The improvement location recommendation unit calculates a correlation coefficient between the two variables for each inconsistency in the causal relationship between the variables, and sorts the correlation coefficients in descending order when the weights are the same. Item 13. The process evaluation apparatus according to Item 10. In process evaluation equipment that evaluates processes,
A function that shows the flow of work tasks for business and development, and for a process that is characterized by a variable that can be measured quantitatively, a function that captures a reference causal model defined by a user who evaluates the process,
A function of taking a plurality of the variables of the process as evaluation data and constructing an evaluation causal model using a causal model construction algorithm of a Bayesian network;
A function for comparing a causal relationship between variables in the reference causal model and a causal relationship between variables in the evaluation causal model, and obtaining a score corresponding to a contradiction in the causal relationship between the variables in the evaluation causal model;
Process evaluation program to realize.

Images