JP5957419B2 - QoE estimation apparatus, QoE estimation method and program - Google Patents

Description

  The present invention relates to a technique for evaluating and managing user experience quality (QoE: Quality of Experience) of an information communication service.

  Information communication services that use IP technology to communicate data, video, and sound using mobile networks and cloud contributors are provided. In order to provide a comfortable information communication service to users, it is important to appropriately monitor and manage the quality of experience (QoE) experienced by users. As one of the indexes for evaluating the quality of the information communication service, a response time (response time) from when the user makes a service start request until a response result is perceived is used.

  In ITU-T Recommendation G.1030 (Non-Patent Document 1), QoE (5 levels (5: very good, 4: good, 3: normal, 2) for response time of data application (specifically, Web browsing) The average rating method is recommended for: bad, 1: very bad). In this recommendation, the response time from the user request to the simplified workflow of “Search site display request”, “Search site display”, “Search start request” and “Search result display” in Web browsing The processing time until the server response (data) returns to the user terminal and the display time on the user terminal are classified into two. The response time of each process (the time from the “search site display request” until the server response returns to the user terminal, the time from when the server response returns to the display of the search site on the user terminal, the search start request to the server It is recommended that a method be used to estimate the QoE using the weighted response time as input, and the time until the response returns to the user terminal and the time from when the server response returns to the time when the search result is displayed. Has been.

  Further, as described in Non-Patent Document 2, it is reported that QoE varies depending on the type of service (application) such as Web access, telephone, e-mail, and file download even with the same response time.

ITU-T Recommendation G.1030 "Estimating end-to-end performance in IP networks for data applications" Nov. 2005 "A Study on Application Classification Considering Satisfaction with Waiting Time" IEICE Technical Report CQ2010-15 (2010-4)

  However, QoE for information communication services is considered to differ depending not only on the type of service (application) but also on various individual processes that occur within the same service. For example, a mail application (such as a mail application using web mail or cloud contributor) has different processes such as “launch application”, “login”, “mail reception”, “mail transmission”, “address search”, etc. Although it is executed, the user's tolerance and QoE for the response time of each process are considered to be different even if the response time is the same. For example, it is assumed that the user tends to allow even if it takes some time for “application activation”, but if it takes time for “address search”, the user tends not to allow this delay. In addition, among the response times of each individual process, the display time at the user terminal and the processing time at the server, which may vary depending on the type of application and the execution timing, often cannot be easily measured.

  In the prior art, one QoE can be estimated for the response time of the entire processing sequence of the information communication service only when the response time including the display time at the user terminal and the processing time at the server can be measured. It was not possible to estimate the QoE for the response times of different individual processes that make up the same service. In addition, QoE could not be estimated when response time including display time at the user terminal and processing time at the server could not be measured. In general, the packet transfer delay time can be easily measured by using a measuring device, while the response time of the individual processing is relatively difficult to acquire for the reasons described above. Moreover, since it is necessary to collect the QoE evaluation value with respect to the response time of each individual process from a user or an evaluator, it is assumed that acquisition is further difficult.

  In view of the above problems, an object of the present invention is to provide a technique for estimating QoE for each individual process in a series of processing steps of an information communication service from a packet transfer delay time.

  In order to solve the above-described problem, an aspect of the present invention provides a packet transfer delay time, a response time, and a QoE evaluation value measured or evaluated for each individual process constituting a processing sequence executed to provide an information communication service. A correlation between the packet transfer delay time and the QoE evaluation value for each individual process by statistically processing the acquired packet transfer delay time, response time and QoE evaluation value In response to acceptance of the QoE estimation request for the individual process, the measurement information acquisition unit acquires the measurement value of the packet transfer delay time of the individual process, and creates the estimation model And a QoE estimator that determines a QoE estimated value corresponding to the obtained measurement value of the packet transfer delay time based on the estimated model. That on QoE estimation apparatus.

  According to the present invention, it is possible to provide a technique for estimating QoE for each individual process in a series of processing steps of an information communication service from a packet transfer delay time.

FIG. 1 is a schematic diagram showing an information communication system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating exemplary measurement information output from the packet transfer delay measurement apparatus. FIG. 3 is a block diagram showing the configuration of the QoE estimation apparatus according to one embodiment of the present invention. FIG. 4 is a diagram illustrating an example data structure for storing measurement information for the mail service. FIG. 5 is a diagram showing another example data structure for storing measurement information for the mail service. FIG. 6 is a diagram illustrating an example of a response time average value estimation model. FIG. 7 is a diagram illustrating an example of a 95% value estimation model of response time. FIG. 8 is a diagram illustrating an example of a response time estimation model. FIG. 9 is a diagram showing the correlation between the response time and the QoE evaluation value. FIG. 10 is a diagram illustrating an example of the QoE estimation model. FIG. 11 is a diagram illustrating the degree of variation in the QoE evaluation value with respect to variation in response time. FIG. 12 is a diagram illustrating QoE estimation using an average response time estimation model. FIG. 13 is a diagram showing QoE estimation using a 95% value estimation model of response time. FIG. 14 is a flowchart showing QoE estimation processing according to an embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  An outline of an embodiment of the present invention to be described later is as follows. The QoE estimation device, for each individual process of a processing sequence executed for providing an information communication service to a user terminal, obtains a packet transfer delay time, a response time, and a QoE acquired in advance. By statistically processing the evaluation value, an estimation model indicating a correlation between the packet transfer delay time of each individual process and the QoE evaluation value is created in advance. Using the created estimation model, the QoE estimator can respond to subsequent QoE estimation requests and obtain packet response delay that is relatively easy to obtain without the need to obtain response times and QoE evaluation values that are difficult to obtain and measure. It is possible to obtain an estimated QoE value for the response time of each individual process from only the time.

  First, an information communication system according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a schematic diagram showing an information communication system according to an embodiment of the present invention.

  As shown in FIG. 1, the information communication system 10 includes a user terminal 50, a QoE estimation device 100, a response time measurement device 200, a packet transfer delay measurement device 250, an information communication service providing server 300, a network 350, and a network quality control device. 400 and an information communication service quality monitoring server 450.

  The user terminal 50 is, for example, a desktop PC (Personal Computer), a notebook PC, a tablet, a smartphone, and the like, and communicates with the information communication service providing server 300 and other network devices via the network 350 to perform various information communication with the user. Provide service. In the illustrated embodiment, only one user terminal 50 is shown, but there may typically be a plurality of user terminals 50.

  The user terminal 50 typically includes various hardware resources such as an auxiliary storage device, a memory device, a CPU (Central Processing Unit), a communication device, a display device, and an input device. The auxiliary storage device includes a hard disk, a flash memory, and the like, and stores programs and data for realizing various processes in the user terminal 50. These programs and data may be downloaded from various servers on the network 350, or may be provided from a storage medium connected to the interface of the user terminal 50, for example. The memory device is composed of a RAM (Random Access Memory) or the like, and reads and stores the program from the auxiliary storage device when an instruction to start the program is given. The CPU functions as a processor for processing information, and implements various functions in the user terminal 50 by executing programs stored in the memory device. The communication device includes various communication circuits for connecting to other devices such as the QoE estimation device 100 and the information communication service providing server 300. The display device includes a display or the like, and displays content received via the network 350, a GUI (Graphical User Interface) by a program, and the like. The input device typically includes operation buttons, a touch panel, a keyboard, a mouse, and the like, and is used by the user of the user terminal 50 to input various operation commands. Note that the user terminal 50 is not limited to the hardware configuration described above, and may have any other appropriate hardware configuration.

  The QoE estimation apparatus 100 estimates the user experience quality (QoE) with respect to the response time of the information communication service provided to the user terminal 50 by the information communication service providing server 300. Here, the response time of the information communication service is defined as the time from the user service start request to the information communication service until the response result is perceived by the user in the information communication system 10. Typically, the response time of the information communication service is the processing time of the information communication service providing server 300 for providing the requested information communication service, packet transfer between the user terminal 50 and the information communication service providing server 300. It consists of a delay time and a processing time related to display on the user terminal 50.

For example, when the mail service is an information communication service to be evaluated, the processing sequence executed to provide the mail service is “mailer activation”, “login”, “mail reception”,... , “Send mail” and “logout”. The QoE estimation device 100 measures the response time T _i for each individual processing i (i = 1, 2,..., N) from one or more of the response time measurement device 200, the packet transfer delay measurement device 250, and the user terminal 50. Value, measured value of packet transfer delay time, and QoE evaluation value dQoE _i in advance, and statistical processing of these measurement information makes it possible to correlate the packet transfer delay time and QoE evaluation value dQoE _i for each individual process. An estimation model indicating the above is created in advance. Thereafter, when receiving a QoE estimation request for individual processing, the QoE estimation apparatus 100 uses the created estimation model to obtain a QoE estimated value from a packet transfer delay time that is relatively easy to measure without acquiring a response time. Can be quickly derived.

  In the illustrated embodiment, the QoE estimation device 100 is shown as an independent device that is communicatively connected to the user terminal 50, and a CPU, an auxiliary storage device, a memory device, and a communication device for realizing a QoE estimation process described later. Consists of hardware resources such as However, the QoE estimation apparatus 100 of the present invention is not limited to this, and may be incorporated in the user terminal 50, for example. In this case, the QoE estimation apparatus 100 may be realized by a computer program executed on the user terminal 50, for example.

  The response time measuring apparatus 200 measures the response time of the information communication service provided to the user terminal 50 by the information communication service providing server 300. More specifically, the response time measuring apparatus 200 measures the response time for each process defined as the process time of each individual process in the process sequence executed to provide the information communication service to be evaluated, and Measure the overall response time of the entire sequence.

  For example, the response time measuring apparatus 200 measures the response time of each individual process in the process sequence executed to provide the information communication service according to the following procedure. That is, the response time measuring apparatus 200 receives the operation log of the user terminal 50 in order to measure (a) the response time for each individual process of the information communication service. Here, the operation log is, for example, a time when measurement is started by clicking a mouse in a browser log or the like, a time when a response from the information communication service distribution server 300 is displayed on the screen of the user terminal 50, or the like. Next, the response time measuring apparatus 200 calculates the response time by dividing each individual process using (b) the received operation log of the user terminal 50. Furthermore, the response time measuring device 200 stores the calculated response time information with the measurement start and measurement end time information added. In the illustrated embodiment, the response time measuring device 200 is shown as an independent device that is communicatively connected to the user terminal 50, but the present invention is not limited to this, and the response time measuring device 200 is For example, it may be incorporated in the user terminal 50.

  The packet transfer delay measuring device 250 measures the transfer times of various packets exchanged between the user terminal 50 and the network side. For example, the packet transfer delay measuring apparatus 250 measures the packet transfer time of the information communication service according to the following procedure. That is, the packet transfer delay measuring apparatus 250 receives the data stream of (a) information communication service, and specifies the measurement target stream (packet). The measurement target stream is identified by comparing the identification information (for example, IP address and port number) set in the packet transfer delay measuring apparatus 250 in advance with the header information of each packet of the received data stream. The (B) Next, the packet transfer delay measuring device 250 collects the packet transfer delay time from the header information of a plurality of packets included in the data stream measured for a certain time, and calculates the average value of the collected packet transfer delay time. For example, when TCP is used as a communication protocol, the packet transfer delay measuring device 250 measures a round trip transfer delay time (RTT) using a packet including data related to execution processing in an information communication service and an ACK packet. May be. (C) Next, for example, as shown in FIG. 2, the packet transfer delay measuring apparatus 250 includes time information regarding the start / end of the section measured to the average value of the calculated packet transfer delay time and the measurement target stream. Identification information (for example, IP address, port number, etc.) used for identification may be assigned and provided to the QoE estimation apparatus 100. This makes it possible to associate the response time for the packet with the packet transfer delay time.

  When the information communication service providing server 300 receives the service start request from the user terminal 50 via the network 350, the information communication service providing server 300 provides the information communication service related to the received service start request to the user terminal 50 via the network 350. The information communication service providing server 300 is, for example, a web server that provides a web service, a mail server that provides a mail service, or the like. When receiving the service start request, the information communication service providing server 300 executes various processes to provide the requested information communication service. When the processing is completed, the information communication service providing server 300 transmits information for providing the requested information communication service to the user terminal 50. When receiving the information, the user terminal 50 executes various processes such as a display process on the received information and displays a response result to the user. The display start time of the response result is the end time of the response time.

  The network 350 is a network operated by the Internet or an operator of the information communication system 10. When the user terminal 50 is wirelessly connected to the network 350, the network 350 communicates with the user terminal 50 via, for example, a base station (not shown) or an access point (not shown).

  The network quality control device 400 executes quality management such as quality monitoring and quality control of various services provided to the user at the user terminal 50. For example, the network quality control apparatus 400 controls the packet transfer delay time between the network side and the user terminal 50 according to the designated packet transfer delay time. The QoE estimation apparatus 100 preferably creates an estimation model indicating a correlation between a packet transfer delay time and a QoE evaluation value, which will be described later, based on data acquired under various network quality conditions. Therefore, the network quality control device 400 executes the processing sequence of the information communication service under various packet transfer delay time conditions, and each individual response time measurement device 200 executed under the various packet transfer delay times. Allows response time to be measured for processing. In addition, various QoE evaluation values can be acquired from the evaluator or the user with respect to different response times of the individual processes executed under such various conditions, and the network quality control apparatus 400 can be used for the QoE estimation apparatus 100. Makes it possible to create an estimation model based on various measurement information.

  The information communication service quality monitoring server 450 monitors the quality of the information communication service provided by the information communication service providing server 300. For example, the information communication service quality monitoring server 450 receives the QoE estimated value of each information communication service from the QoE estimation apparatus 100 and stores it as information indicating the quality of the information communication service.

  Next, the configuration of the QoE estimation apparatus according to an embodiment of the present invention will be described with reference to FIG. The QoE estimation apparatus 100 according to the present embodiment includes a packet transfer for each individual process constituting a process sequence executed to provide an information communication service from the response time measurement apparatus 200, the packet transfer delay measurement apparatus 250, and the user terminal 50. Estimated model that obtains the measured value of delay time, measured value of response time, and QoE evaluation value in advance, and shows the correlation between the packet transfer delay time of each individual processing and QoE evaluation value based on the acquired measurement information Is created in advance. Thereafter, the QoE estimation apparatus 100 determines the QoE estimated value from the packet transfer delay time acquired using the created estimation model without acquiring the response time of each individual process.

  FIG. 3 is a block diagram showing the configuration of the QoE estimation apparatus according to one embodiment of the present invention. As illustrated in FIG. 3, the QoE estimation apparatus 100 includes a measurement information acquisition unit 110, an estimation model creation unit 120, and a QoE estimation unit 130.

  The measurement information acquisition unit 110 acquires the packet transfer delay time, the response time, and the QoE evaluation value that are measured or evaluated for each individual process that constitutes the processing sequence executed to provide the information communication service. Specifically, the measurement information acquisition unit 110 receives the measurement value of the packet transfer delay time from the response time measurement device 200, the packet transfer delay measurement device 250, and the user device 50, and processes the information communication service at the same time as the measurement. The response time of each individual process of the sequence, the QoE evaluation value for the response time, and the time information of the time are acquired. Typically, the measurement information acquisition unit 110 has a plurality of response time measurement values for one packet transfer delay time measurement (for example, each network quality control condition set in the network quality control device 400). Get QoE evaluation value. This is because the response time is measured in a plurality of usage environments in consideration of differences in user terminals and the QoE evaluation value is evaluated by a plurality of evaluators. For example, the measurement information acquisition unit 110 manages the acquired measurement information in a data format as illustrated in FIGS. 4 and 5 and provides the measurement model generation unit 120 with the measurement information.

  The estimation model creation unit 120 statistically processes the packet transfer delay time, the response time, and the QoE evaluation value provided from the measurement information acquisition unit 110, so that the packet transfer delay time and the QoE evaluation value for each individual process are calculated. Create an estimation model that shows the correlation. In one embodiment, the estimated model creating unit 120 includes a response time estimated model creating unit 121 and a QoE estimated model creating unit 122.

  The response time estimation model creation unit 121 creates a response time estimation model indicating the correlation between the packet transfer delay time and the response time for each individual process by statistically processing the acquired packet transfer delay time and the response time. To do. In one embodiment, the response time estimation model creation unit 121 performs regression analysis on a plurality of pairs of measured values of the packet transfer delay time and response time of each individual process, and uses the regression equation derived from the regression analysis as a response. It may be determined as a time estimation model.

  Specifically, the response time estimation model creation unit 121 creates a response time estimation model that estimates the response time of each individual process of the information communication service processing sequence by inputting the packet transfer delay time according to the following procedure. First, the response time estimation model creation unit 121 acquires a packet transfer delay time measured for each individual process and a plurality of response times corresponding to the packet transfer delay time from the measurement information acquisition unit 110. This is because the packet transfer delay time is an average value of the packet transfer times measured a plurality of times with respect to the processing sequence of the information communication service, and thus a plurality of response times are acquired in association with a plurality of measurements. .

  Next, the response time estimation model creation unit 121 calculates an average value of a plurality of corresponding response times and a confidence width of 95% reliability (95% confidence width) for each packet transfer delay time, and calculates the calculated response. A value obtained by adding the calculated 95% confidence width to the average value of time is derived as a 95% value. Note that the present invention is not limited to the above-described 95% reliability range, and any other predetermined reliability range such as 99% reliability may be used.

  Next, the response time estimation model creation unit 121 derives and derives the regression equation of the average value and the regression equation of the 95% value from the average value and 95% value of the response time calculated for each individual process. The regression equation is an estimated model of the average value and 95% response time of each individual process. As a specific method for deriving the regression equation, any known method such as a least square method may be used.

Taking a mail service as a specific example, Fig. 6 shows an average value estimation model (regression equation) showing the relationship between the average value of response time and packet transfer delay time, and Fig. 7 shows the 95% value of response time and packet transfer delay. A 95% value estimation model (regression equation) showing the relationship with time is shown. In the illustrated embodiment, the average value estimation model g _i (x) and the 95% value estimation model h _i (x) are both expressed by an exponential function type of a × Exp (−x / b) + c. The Here, x is a packet transfer delay time, and a, b, and c are constant values estimated by regression analysis for each individual process (i = 1, 2,..., N). 5 and 6, activation (i = 1), login (i = 2), mail reception (i = 3), address search (i = 4), mail transmission (i = 5) and logout (i = 6). The average value estimation model g _i (x) and the 95% value estimation model h _i (x) of the present invention are not limited to the exponential function type described above, and represent the correlation between the response time and the packet transfer delay time. Any other functional type suitable for the above may be used. For example, a linear regression equation of ax + b may be used. In another embodiment, the regression analysis is performed by applying different function types, the correlation coefficients calculated by applying the derived regression equations are compared, and the function type having the highest correlation coefficient is adopted. Also good. In the above-described embodiment, the 95% value estimation model is used as h _i (x), but the present invention is not limited to this, and any other predetermined value such as a 99% value estimation model is used. A reliability estimation model may be used.

Next, the response time estimation model creation unit 121 identifies an individual process whose response time is likely to be affected other than the packet transfer delay time. Two estimation models, the average value estimation model g _i (x) and the 95% value estimation model h _i (x), for individual processing for which it is determined that the response time is likely to be affected by anything other than the packet transfer delay time Of these, the 95% value estimation model h _i (x) may be used as the response time estimation model. This makes it possible to reduce the possibility of undetected QoE reduction. Specifically, the response time estimation model creation unit 121 calculates an average value of 95% confidence width calculated for the packet transfer delay time data for each individual process, and the calculated average value is preset. A process larger than the threshold is specified. That is, an individual process whose regression prediction result is greater than the threshold is determined as an individual process that is highly likely to be affected by the response time other than the packet transfer delay time. In the specific example of FIG. 7, when the threshold is set to 1 second, the activation (i = 1) and logout (i = 6) exceed the threshold, and the response time may be affected other than the packet transfer delay time. It is determined as an individual process with high performance.

  In the above-described embodiment, the response time estimation model creation unit 121 performs the regression analysis on the measured values of the packet transfer delay time and the response time in order to determine the correlation between the packet transfer delay time and the response time. The determination of the correlation between the packet transfer delay time and the response time according to the present invention is not limited to this, and any other appropriate statistical process may be used.

The response time estimation model creation unit 121 uses the average value estimation model g _i (x) and the 95% value estimation model h _i (x) created in this way, and threshold determination using the average value of the 95% confidence range described above. The result (hereinafter, response time fluctuation determination information RJ _i ) is stored. For example, the response time estimation model creation unit 121 may be held in a data format as shown in FIG. That is, the response time fluctuation determination information (RJ _i ) may be set to “1” when the threshold is exceeded and “0” when the threshold is not exceeded.

  The QoE estimation model creation unit 122 creates a QoE estimation model indicating the correlation between the response time and the QoE evaluation value for each individual process by statistically processing the acquired response time and the QoE evaluation value. In one embodiment, the QoE estimation model creating unit 122 performs regression analysis on a plurality of pairs of measured response time values and QoE evaluation values of each individual process, and uses the regression equation derived from the regression analysis as a QoE estimation model. May be determined as

  Specifically, the QoE estimation model creation unit 122 receives the response time estimated using the response time estimation model created by the response time estimation model creation unit 121 according to the following procedure as an input, and performs QoE evaluation for each individual process. Create a QoE estimation model that estimates the value. First, the QoE estimation model creation unit 122 acquires a plurality of response times for the packet transfer delay time measured for each individual process and the QoE evaluation value dQoEi for the response time from the measurement information acquisition unit 110. Here, the QoE evaluation value is, for example, a five-level scale (5: very good, 4: good, 3: normal, 2: bad, 1: very bad), (5: delay is completely unknown, 4: delay I understand but I don't care, 3: I'm a little worried about the delay, 2: I'm worried about the delay, 1: I'm very worried about the delay).

Next, the QoE estimation model creation unit 122 calculates an average value of a plurality of corresponding response times for each packet transfer delay time, and calculates an average value of the QoE evaluation values dQoE _{i with} respect to the average value of the response times. . Next, the QoE estimation model creation unit 122 derives a regression equation from data of the calculated average value of response times and the average value of the QoE evaluation values dQoE _i . As a specific method for deriving the regression equation, any known method such as a least square method may be used. The regression equation derived in this way is determined as the QoE estimation model of the QoE evaluation value dQoE _i for each individual process. Taking a mail service as a specific example, FIG. 9 shows a dQoE _i estimation model (regression equation) showing the relationship between the average value of response times and the average value of dQoE _i . In the illustrated embodiment, the dQoE _i estimation model f _i (x) is represented by an exponential function type of a × Exp (−x / b) +1. Here, x is a response time, and a and b are constant values estimated by regression analysis for each individual process (i = 1, 2,..., N). The QoE estimation model creation unit 122 may hold the derived dQoE _i estimation model f _i (x) in a data format as shown in FIG.

Note that the dQoE _i estimation model f _i (x) of the present invention is not limited to the exponential function type described above, and any other functional type suitable for representing the correlation between the response time and the QoE evaluation value is used. May be. For example, a linear regression equation of ax + b may be used. In another embodiment, the regression analysis is performed by applying different function types, the correlation coefficients calculated by applying the derived regression equations are compared, and the function type having the highest correlation coefficient is adopted. Also good.

In response to acceptance of the QoE estimation request for the individual processing, the QoE estimation unit 130 causes the measurement information acquisition unit 110 to acquire a measurement value of the packet transfer delay time of the individual processing, and the estimation created by the estimation model creation unit 120 Based on the model, the estimated QoE value corresponding to the obtained measurement value of the packet transfer delay time is determined. Specifically, the QoE estimation unit 130 estimates dQoE _i for each individual process of the information communication service processing sequence according to the following procedure, and monitors the QoE with respect to the response time of the information communication service. That is, the QoE estimation unit 130 acquires the packet transfer delay time, the measurement target stream identification information, and the measurement time information from the measurement information acquisition unit 110. Next, the QoE estimation unit 130 acquires an estimation model corresponding to the individual processing of the measurement target from the estimation model creation unit 120 based on the identification information of the measurement target stream, and adds the average value of the packet transfer delay time to the acquired estimation model. To determine an estimate of dQoE _i .

As described above, when two estimation models of the average value estimation model g _i (x) and the 95% value estimation model h _i (x) are created in the estimation model creation unit 120, the QoE estimation unit 130 Of these two estimation models, which estimation model is used to determine the QoE estimation value is determined. In one embodiment, the QoE estimation unit 130 determines the average value estimation model g _i (x) and the 95% value estimation model h _i (x) according to the possibility that the response time is affected by other than the packet transfer delay time. May be used selectively. Specifically, the QoE estimation unit 130 refers to the data format variation determination information RJ _i indicating the response time estimation model as illustrated in FIG. 8, and performs individual processing that does not exceed the above-described variation determination threshold. That is, for individual processing with RJ _i = 0, the average value of the response time corresponding to the packet transfer delay time is identified using the average value estimation model g _i (x), and the dQoE _i estimation model f _i (x) Then, dQoE _i corresponding to the average value of the response time is derived as a QoE estimated value of the individual processing. On the other hand, for individual processing exceeding the threshold for fluctuation determination, that is, individual processing with RJ _i = 1, the average response time corresponding to the packet transfer delay time using the 95% value estimation model h _i (x) identifying the value to derive a DQoE _i corresponding to the average value of the response time as the QoE estimate of the individual processes in the derived DQoE _i estimation model f _i (x).

In one embodiment, the QoE estimation unit 130 calculates an average value estimation model g _i (x) and a 95% value estimation model h _i (x) according to the degree of the variation of the QoE evaluation value with respect to the variation of the response time. It may be used selectively. For example, when using the 95% value estimation model h _i (x), the QoE estimation unit 130 further has a response time at which dQoEi is the maximum value (dQoE _i = 5 in the specific example of FIG. 9), and dQoE _i is A response time that is a preset management target value (for example, dQoE _i = 3.5) is determined, and a difference between the two response times is calculated. Further, the QoE estimation unit 130 is preset with a value obtained by dividing a value (1.5 (= 5-3.5) in this example) obtained by subtracting the management target value from the maximum value of dQoE _i by the calculated response time difference. The 95% value estimation model h _i (x) is applied only when it is larger than the determination threshold (for example, 0.5) (95% value estimation model application determination QJ _i = 1), otherwise (95% value estimation model The average value estimation model g _i (x) may be applied to the application determination QJ _i = 0). That is, when RJ _i = 1 and QJ _i = 1, the QoE estimation unit 130 uses the 95% value estimation model h _i (x) and the dQoE _i estimation model f _i (x) (dQoE _i = f _{When i} (h _i (x)), RJ _i = 0 or QJ _i = 0, the average value estimation model g _i (x) and the dQoE _i estimation model f _i (x) are used (dQoE _i = f _i (g _i (x)) According to this embodiment, it is possible to apply an estimation model that estimates a large response time for individual processing in which dQoE _i fluctuates sensitively to response time fluctuations. Become.

In relation to the present embodiment, FIG. 11 shows the main processing for activation (i = 1) and logout (i = 6), which are individual processes in which RJ _i = 1 in the specific example of the mail service described above. An application example to which the embodiment is applied is shown. In the case of activation (n = 1), b1 / a1 is compared with a predetermined determination threshold, and in the case of logout (n = 6), b2 / a2 is compared with a predetermined determination threshold. When it is larger than the determination threshold, the 95% value estimation model application condition of the individual process i is QJ _i = 1, and when it is smaller, QJ _i = 0. In the example of FIG. 10, QJ _i = 1 in any individual processing, and the QoE estimation unit 130 applies the 95% value estimation model h _i (x). 12, the average value estimated relationship model g _i (x) a value obtained by estimating the DQoE _i using the DQoE _i was used to estimate model creation is shown in Figure 13, 95% value estimation models h _i ( relationship between the value obtained by estimating the DQoE _i and DQoE _i used for estimation model created using x) are shown. As described above, when dQoE _i is estimated using the 95% value estimation model h _i (x), a large difference occurs. For this reason, it is preferable to estimate dQoE _i using the 95% value estimation model h _i (x) only for individual processing with a large response time variation.

  Next, QoE estimation processing in the QoE estimation apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 14 is a flowchart showing QoE estimation processing according to an embodiment of the present invention. The QoE estimation process is a process for creating an estimation model indicating a correlation between the packet transfer delay time and the QoE estimation value from the previously acquired measurement value of the packet transfer delay time, the measurement value of the response time, and the QoE evaluation value (step S101 to S102) and processing (steps S103 to S104) for determining a QoE estimated value for the measured value of the packet transfer delay time based on the created estimation model (steps S103 to S104).

  As shown in FIG. 14, in step S101, the QoE estimation apparatus 100 measures or evaluates the packet transfer delay time and response time measured or evaluated for each individual process constituting the processing sequence executed to provide the information communication service. And obtain the QoE evaluation value. In one embodiment, the QoE estimating apparatus 100 acquires a measured value of packet transfer delay time, a measured value of response time, and a QoE evaluation value from the response time measuring apparatus 200, the packet transfer delay measuring apparatus 250, and the user terminal 50. In one embodiment, the network quality control device 400 executes the processing sequence of the information processing service for each packet transfer delay time a plurality of times, and the response time measurement device 200 executes each individual processing a plurality of times, A plurality of response times and QoE evaluation values are measured for each packet transfer delay time, and the plurality of response times and QoE evaluation values may be provided to the QoE estimation apparatus 100. In this case, the QoE estimation apparatus 100 may use the average values of the plurality of response times and the QoE evaluation values for subsequent processing.

  In step S102, the QoE estimation apparatus 100 performs statistical processing on the acquired packet transfer delay time, response time, and QoE evaluation value, thereby estimating the correlation between the packet transfer delay time and the QoE evaluation value for each individual process. Create a model. In one embodiment, the QoE estimation apparatus 100 includes a response time estimation model indicating a correlation between the packet transfer delay time and the response time, and a QoE estimation model indicating a correlation between the response time and the QoE evaluation value, respectively. An estimation model indicating the correlation between the packet transfer delay time and the QoE evaluation value may be created by deriving and combining the two derived estimation models. Regression analysis can be used as the statistical processing, and a regression equation derived by regression analysis may be used as each estimation model.

  In step S <b> 103, the QoE estimation apparatus 100 acquires a measured value of the packet transfer delay time of the individual process from the packet transfer delay measurement apparatus 250 in response to reception of the QoE estimation request for the individual process. The QoE estimation request may be received from, for example, an operator of the QoE estimation apparatus 100, the information communication service monitoring server 450, or the like.

  In step S104, the QoE estimation apparatus 100 determines a QoE estimated value corresponding to the acquired measurement value of the packet transfer delay time based on the created estimation model. The QoE estimation apparatus 100 returns the determined QoE estimation value to the request source of the QoE estimation request.

  The QoE estimation apparatus 100 may perform QoE monitoring based on such an estimation model in real time, that is, every time a measurement value of the packet transfer delay time is received from the packet transfer delay measurement apparatus 250. Alternatively, the QoE estimation apparatus 100 performs offline QoE monitoring based on such an estimation model, that is, temporarily stores a measurement value of the packet transfer delay time received from the packet transfer delay measurement apparatus 250, and holds the stored packet transfer. You may make it perform QoE monitoring at a certain timing about the measured value of delay time.

  Thus, by creating an estimation model indicating the correlation between the packet transfer delay time of each individual process and the QoE evaluation value from the packet transfer delay time, response time, and QoE evaluation value acquired in advance, the QoE estimation device 100 can obtain an estimated QoE value from the packet transfer delay time without requiring a response time and a QoE evaluation value that are relatively difficult to obtain and measure.

  As mentioned above, although the Example of this invention was explained in full detail, this invention is not limited to the specific embodiment mentioned above, In the range of the summary of this invention described in the claim, various deformation | transformation・ Change is possible.

DESCRIPTION OF SYMBOLS 10 Information communication system 50 User terminal 100 QoE estimation apparatus 110 Measurement information acquisition part 120 Estimation model creation part 121 Response time estimation model creation part 122 QoE estimation model creation part 130 QoE estimation part 200 Response time measurement apparatus 250 Packet transfer delay measurement apparatus 300 Information communication service providing server 350 Network 400 Network quality control device 450 Information communication service quality monitoring server

Claims (8)

A measurement information acquisition unit that acquires a packet transfer delay time, a response time, and a QoE evaluation value measured or evaluated for each individual process constituting a processing sequence executed to provide an information communication service;
An estimation model that creates an estimation model indicating a correlation between the packet transfer delay time and the QoE evaluation value for each individual process by statistically processing the acquired packet transfer delay time, response time, and QoE evaluation value The creation department;
In response to accepting a QoE estimation request for the individual process, the measurement information acquisition unit acquires a measurement value of the packet transfer delay time of the individual process, and based on the created estimation model, the acquired packet transfer delay A QoE estimator that determines a QoE estimate corresponding to the time measurement;
QoE estimation device having The estimated model creation unit
A response time estimation model that creates a response time estimation model indicating a correlation between the packet transfer delay time and the response time for each individual process by statistically processing the acquired packet transfer delay time and response time The creation department;
A QoE estimation model creation unit that creates a QoE estimation model indicating a correlation between the response time and the QoE evaluation value for each individual process by statistically processing the acquired response time and QoE evaluation value;
The QoE estimation apparatus according to claim 1, comprising:   The response time estimation model creation unit performs a regression analysis on a plurality of pairs of measured values of the packet transfer delay time and the response time of each individual process, and uses the regression equation derived from the regression analysis as the response time estimation The QoE estimation apparatus according to claim 2, which is determined as a model. The response time estimation model creation unit acquires a plurality of response time measurement values for the packet transfer delay time, and indicates a correlation between the average value of the acquired response time measurement values and the packet transfer delay time Deriving a first regression equation and a second regression equation indicating a correlation between a statistical value obtained by adding a confidence width of a predetermined reliability to the average value and the packet transfer delay time;
The said QoE estimation part selectively uses the said 1st regression equation and the said 2nd regression equation according to possibility that the said response time will receive influence other than the said packet transfer delay time. QoE estimation device.   5. The QoE according to claim 4, wherein the QoE estimation unit selectively uses the first regression equation and the second regression equation according to a degree of variation in the QoE evaluation value with respect to variation in the response time. Estimating device.   The QoE estimation model creation unit performs a regression analysis on a plurality of pairs of measured values of response times and QoE evaluation values of the individual processes, and determines a regression equation derived from the regression analysis as the QoE estimation model The QoE estimation apparatus according to any one of claims 2 to 5. A QoE estimation method executed by a QoE estimation device,
Obtaining a packet transfer delay time, a response time and a QoE evaluation value measured or evaluated for each individual process constituting a processing sequence executed to provide an information communication service;
Creating an estimation model indicating a correlation between the packet transfer delay time and the QoE evaluation value for each individual process by statistically processing the acquired packet transfer delay time, response time, and QoE evaluation value; ,
In response to accepting a QoE estimation request for the individual process, obtaining a measurement value of the packet transfer delay time of the individual process;
Determining a QoE estimate corresponding to the obtained measurement value of the packet transfer delay time based on the created estimation model;
QoE estimation method comprising: Obtaining a packet transfer delay time, a response time and a QoE evaluation value measured or evaluated for each individual process constituting a processing sequence executed to provide an information communication service;
Creating an estimation model indicating a correlation between the packet transfer delay time and the QoE evaluation value for each individual process by statistically processing the acquired packet transfer delay time, response time, and QoE evaluation value; ,
In response to accepting a QoE estimation request for the individual process, obtaining a measurement value of the packet transfer delay time of the individual process;
Determining a QoE estimate corresponding to the obtained measurement value of the packet transfer delay time based on the created estimation model;
A program that causes a processor to execute.

Images