CN105204981A - Application performance test method and device - Google Patents

Abstract

The invention discloses a method and device for testing application performance, wherein the method includes: obtaining the performance information of the mobile device during the operation of the mobile device under test and the collection time of the performance information; obtaining the performance information of the application under test during the operation of the mobile device User historical operation information and historical operation occurrence time; according to the mobile device performance information, the collection time, the user historical operation information and the historical operation occurrence time, it is determined that the application performance of the application under test changes with the collection time , the historical operation information of the user and the change of the historical operation time; output the change of the application performance of the tested application.

Description

Applied performance testing method and device

technical field

The invention relates to performance testing technology, in particular to an application performance testing method and a corresponding application performance testing device.

Background technique

Applications (also referred to as application programs) usually need to be tested for performance. For example, before an application developed for a mobile device such as a mobile phone or a tablet computer is officially put into use, the performance of the application needs to be tested.

Existing methods for testing application performance in mobile devices generally include: running the application under test in the mobile device, and obtaining performance information of the mobile device multiple times according to a predetermined time interval during the running of the application under test, and then, The performance of the application under test is obtained by analyzing all the performance information of the mobile device acquired within a period of time.

In the process of implementing the present invention, the inventors found that the types and quantities of applications have become increasingly numerous, and users' requirements for applications have also increased, which prompts applications to have better performance as much as possible to enhance their market competitiveness; in order to make The application has better performance, which requires a more refined performance test for the application to improve the application; however, the performance of the tested application obtained by using the existing application performance testing method is not very good Meet current application testing needs.

Contents of the invention

In view of the above problems, the present invention is proposed to provide an application performance testing method and device that overcome the above problems or at least partially solve the above problems.

According to one aspect of the present invention, a method for testing application performance is provided, including: obtaining the performance information of the mobile device during the running of the mobile device of the tested application and the collection time of the performance information; User historical operation information and historical operation occurrence time; according to the mobile device performance information, the collection time, the user historical operation information and the historical operation occurrence time, determine that the application performance of the application under test changes with the collection time , the historical operation information of the user and the change of the historical operation time; output the change of the application performance of the tested application.

According to another aspect of the present invention, an application performance testing device is provided, including: a performance monitoring module, adapted to obtain the performance information of the mobile device and the collection time of the performance information of the application under test during the operation of the mobile device; an operation monitoring module is adapted to acquire user history operation information and historical operation occurrence time of the application under test during the running process of the mobile device; the determining performance module is adapted to obtain the user's historical operation information and the The occurrence time of the historical operation determines the application performance of the application under test according to the collection time, the user historical operation information and the change of the historical operation occurrence time; the output performance module is adapted to output the application performance of the application under test Changes in application performance.

The application performance testing method and device of the present invention can know the impact of the running of the tested application on the performance of the mobile device by acquiring the mobile device performance information and the collection time of the tested application in the running process of the mobile device; During the process, the user's historical operation information and the time when the historical operation occurred can be used to know the impact of the user's historical operation on the performance of the mobile device through the association between the collection time and the historical operation time. In this way, the specific reasons for the performance fluctuation of the tested application and the Specific operations that cause abnormalities in the running of the application under test, etc.; thus, the present invention can better meet the current application testing requirements and is beneficial to the improvement of application performance.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings in the description are for the purpose of illustrating preferred embodiments only and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same parts. In the attached picture:

FIG. 1 shows a flowchart of an application performance testing method according to Embodiment 1 of the present invention;

FIG. 2 shows a line chart generated for the CPU usage percentage of a mobile device and historical user operation information according to Embodiment 1 of the present invention;

FIG. 3 shows a line chart generated for the occupied space of the mobile device memory and user history operation information according to Embodiment 1 of the present invention;

FIG. 4 shows a line graph generated for the battery power of the mobile device and user historical operation information according to Embodiment 1 of the present invention;

FIG. 5 shows a line graph generated for mobile device network traffic and user historical operation information according to Embodiment 1 of the present invention;

FIG. 6 shows a line graph generated for the CPU occupation percentage of the mobile device, the occupied space of the memory of the mobile device, the battery power of the mobile device, the network traffic of the mobile device, and the historical operation information of the user according to Embodiment 1 of the present invention;

FIG. 7 shows another line graph generated for the CPU occupation percentage of the mobile device, the occupied space of the memory of the mobile device, the battery power of the mobile device, the network traffic of the mobile device, and the historical operation information of the user according to Embodiment 1 of the present invention;

FIG. 8 shows a schematic diagram of an application performance testing device according to Embodiment 2 of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

Embodiment 1, application performance testing method. The method of this embodiment will be described in detail below with reference to FIG. 1 .

In FIG. 1 , S100 , acquire the performance information of the mobile device and the collection time of the performance information of the mobile device when the application under test is running on the mobile device.

Specifically, in this embodiment, the subject of acquiring the performance information of the mobile device and the collection time of the performance information of the mobile device should be the application performance testing device; the application performance testing device here can be a mobile device running the application under test, or it can be an independent Other devices based on the mobile device running the application under test, for example, the above other devices may be devices such as a server connected wirelessly or wired to the mobile device running the application under test.

In the case that the application performance test device is a mobile device running the application under test, the mobile device (such as the application performance test program installed in the mobile device, etc.) Collect mobile device performance information at regular intervals, and store the collected mobile device performance information and collection time locally in the mobile device performance log. After that, mobile devices (such as application performance test programs) can conveniently analyze application performance The performance information of the mobile device and the collection time of the performance information of the mobile device during the operation of the mobile device of the application under test are obtained accurately from the mobile device performance log stored locally.

In the case that the application performance testing device is other equipment connected to the mobile device running the application under test, such as a server, the mobile device (such as the application performance testing program installed in the mobile device, etc.) can be in the running process of the application under test, The performance information of the mobile device is regularly collected according to the preset time interval, and the collected mobile device performance information and collection time are stored locally in the mobile device performance log. After that, the mobile device (such as an application performance test program) can log the mobile device The performance log is uploaded to the server; for example, the mobile device (such as an application performance test program) actively uploads the locally stored mobile device performance log to the server after the application under test finishes running; another example, no matter whether the application under test is running or not After running, the mobile device uploads the locally stored mobile device performance log to the server after receiving the mobile device performance test information upload request transmitted from the server; The performance information of the mobile device and the collection time of the performance information of the mobile device during the operation of the mobile device by the tested application are obtained from the received mobile device performance log.

The above mobile device performance information usually includes multiple performance indicators of the mobile device, and the specific performance indicators included in the mobile device performance information can be determined according to the actual test requirements of the application; for example, the mobile device performance information can specifically include: mobile device CPU usage, Any one or any two or any three or all of mobile device memory usage, mobile device network traffic, and mobile device battery usage, etc.

The CPU usage of the above mobile device can be specifically: the CPU usage rate of the mobile device (that is, the percentage occupied by the mobile device) or the CPU idle rate of the mobile device (that is, the CPU idle percentage of the mobile device), and the CPU usage rate of the mobile device can be specifically defined as The CPU usage rate of the mobile device; the memory usage of the above mobile device can be specifically: the memory usage rate of the mobile device (that is, the percentage of memory occupied by the mobile device) or the idle rate of the memory of the mobile device (that is, the free percentage of the memory of the mobile device) or the memory usage of the mobile device The size of the space or the free space of the mobile device memory, and the mobile device memory occupancy rate can be specified as the mobile device memory occupancy rate of the application under test, and the occupied space of the mobile device memory can be specified as the mobile device memory occupancy space of the application under test size; the above-mentioned mobile device network traffic can be specifically the network traffic consumed by the application under test: and the mobile device network traffic can be any one or both of Wi-Fi data traffic and 2G/3G/4G data traffic; that is to say , the mobile device in this embodiment can distinguish the type of its current network traffic, so that the network traffic of the mobile device in the performance log of the mobile device can clearly indicate whether the network traffic currently consumed by the mobile device is Wi-Fi data traffic or 2G/ 3G/4G data traffic; the battery usage of the above-mentioned mobile device may specifically be the battery power of the mobile device (that is, the remaining power of the battery of the mobile device), and the battery power of the mobile device may be represented by the battery voltage.

The mobile device can use the performance monitoring module in the application performance test program installed inside it to collect the mobile device performance information of the tested application during the operation of the mobile device, and store the collected mobile device performance information and the corresponding collection time locally in the mobile device performance log. The performance monitoring module can collect the mobile device performance information of the tested application during the running of the mobile device by calling a corresponding API (Application Programming Interface, Application Programming Interface) in the mobile device operating system. The above application performance testing program can be used as a part of the function of an application installed in the mobile device, for example, the application performance testing program can be used as an optimization function of the optimization master installed in the mobile device.

A specific example of collecting the mobile device performance information of the application under test during the operation of the mobile device in this embodiment and locally storing the mobile device performance information and the collection time is as follows: after the application performance test program arrives at a preset time interval, Call SystemProperties.java in the operating system to obtain the CPU usage of the current mobile device, call ActivityManager.java in the operating system to obtain the occupied space of the current mobile device memory, and call ConnectivityManager.java in the operating system to obtain the current mobile device network Traffic, call BatteryManager.java in the operating system to obtain the battery power of the current mobile device. The above four calling operations can be performed at the same time under normal circumstances; when the application performance test program executes the above calling operations or receives the current mobile device returned by the operating system When the CPU usage of the device, the occupied space of the current mobile device memory, the current network traffic of the mobile device, and the current battery level of the mobile device are used, the current time of the mobile device is obtained (that is, the time stamp is given to the performance information of the mobile device), and then the application performance test is performed. The program stores the current mobile device CPU usage, current mobile device memory usage, current mobile device network traffic, current mobile device battery power, and the current time of the mobile device as a record in the mobile device performance log. In this embodiment, the API provided by the operating system of the mobile device can be used to obtain the current time of the mobile device, for example, the current time of the mobile device can be obtained by using System.currentTimeMillis() in the operating system.

The performance information log of the mobile device in this embodiment may be a file in txt format, or a file in other formats; and the performance information log of the mobile device may be stored in an SD card of the mobile device.

S110. Obtain historical operation information of the user during the running of the application under test on the mobile device and the time at which the historical operation occurred.

Specifically, in this embodiment, the subject that acquires user historical operation information and historical operation occurrence time should also be an application performance testing device; similarly, the application performance testing device in this embodiment can be a mobile device running the application under test, It may also be other devices independent of the mobile device running the application under test. For example, the above other devices may be devices such as a server connected wirelessly or wiredly to the mobile device running the application under test.

It should be noted that the subject that obtains the user's historical operation information and the time when the historical operation occurred is usually the same subject as the above-mentioned subject that obtains the mobile device performance information and the collection time of the mobile device performance information; in addition, although S100 and S110 are with The two steps of sequential numbering, however, there is no necessary execution sequence restriction between S100 and S110 in practical applications, these two steps should usually be executed at the same time, of course, can also be executed successively, but, within the entire time period of the test Preferably, part of the mobile device performance information and part of the user's historical operation information should be obtained at the same time.

In the case where the application performance test device is a mobile device running the application under test, the mobile device (such as the application performance test program installed in the mobile device, etc.) can monitor the user's operation in real time during the running of the application under test ( The user's operation here usually refers to the user's operation simulated by the test logic designed for application performance testing), and when it is detected that the user has performed the corresponding operation, the current time of the mobile device is obtained, and the monitored user The operation information corresponding to the executed operation and the above-mentioned current time are locally stored in the user history operation log as the user history operation information and the time when the history operation occurred; thus, when the mobile device needs to perform application performance analysis, it can conveniently store it from its locally stored The user history operation log obtains the user history operation information and the time when the application under test is running on the mobile device.

In the case that the application performance testing device is other equipment connected to the mobile device running the application under test, such as a server, the mobile device (such as the application performance testing program installed in the mobile device, etc.) can be in the running process of the application under test, Monitor the user's operation in real time (the user's operation here usually refers to the user's operation simulated by the test logic designed for application performance testing), and when it is detected that the user has performed the corresponding operation, obtain the current time of the mobile device ( That is, time stamp the user's historical operation information), and store the operation information corresponding to the operation performed by the user and the above-mentioned current time locally in the user's historical operation log as the user's historical operation information and the time when the historical operation occurred. After that, the mobile device can store the User history operation logs are uploaded to the server; for example, the mobile device actively uploads the locally stored user history operation logs to the server after the application under test finishes running; another example, the mobile device receives the mobile device performance test information transmitted by the server After uploading the request, upload the user history operation log stored locally to the server; in this way, when the server needs to perform application performance analysis, it can easily obtain the application under test from the received user history operation log during the running process of the mobile device. The user's historical operation information and the time when the historical operation occurred. Similarly, this embodiment can use the API provided by the operating system of the mobile device to obtain the current time of the mobile device, for example, use System.currentTimeMillis() in the operating system to obtain the current time of the mobile device.

The above-mentioned mobile device performance information log in S100 and user history operation log in S110 should usually be uploaded to the server at the same time. The log is uploaded to the server at the same time; for another example, after the mobile device receives the mobile device performance test information upload request transmitted from the server, it simultaneously uploads the locally stored mobile device performance information log and user history operation log to the server. In this embodiment, the above two logs can be uploaded to the server by setting the Upload() function in the application performance test program. In addition, the mobile device can clear records in the successfully uploaded mobile device performance information log and user history operation log.

The above-mentioned user history operation log usually contains user history operation information corresponding to various operations performed by the user (including the user's operation simulated by the test logic), and the various operations performed by the user may specifically include: click, slide, and input events etc. This embodiment does not limit the specific content of the operations performed by the user.

The mobile device can use the operation monitoring module in the application performance test program installed inside it to obtain the user's historical operation information and the operation time of the tested application during the operation of the mobile device, and the obtained user historical operation information and corresponding The operation occurrence time is stored locally in the user history operation log. The application performance testing program can usually obtain and identify UI (User Interface, user interface) controls in the application under test, and call corresponding UI controls in the application under test according to corresponding test logic, so as to test the application under test. When the application performance test program calls the corresponding UI control of the application under test, the operation monitoring module in the application performance test program can record the corresponding UI control information and the current call time as user historical operation information and operation occurrence time to the user historical operation in the log.

The application performance testing program in this embodiment may be an application program developed on the basis of Robotium. Robotium is a UI automation testing framework based on the Android platform. The API provided by Robotium (such as Solo.java) can be used in various ways to identify the UI controls of the application under test and operate the UI of the application under test, such as clicking operation, long press operation or slide operation, etc.

A specific example of the mobile device collecting the user operation information of the application under test during the running of the mobile device, and locally storing the user operation information and the time when the operation occurred is: after the application performance test program is running, it can query the operating system to know the All UI controls of the application (for example, the application performance test program can query all UI controls of the application under test by using the API provided by Robotium of the operating system), and then the application performance test program will call the corresponding test logic according to its preset test logic. UI controls, so that the application under test performs the corresponding operation for the corresponding UI control of the current call (such as the application performance test program can make the application under test perform the corresponding operation of the UI control by using the API provided by the Robotium of the operating system), thereby In this embodiment, the performance test is performed on the application under test by simulating the user's actual operation on the application under test; The monitoring module can record the operation information corresponding to the UI control, and at the same time obtain the current time of the mobile device (that is, add a corresponding time stamp for the UI control), and then, the operation monitoring module will use the operation information corresponding to the UI control and the current time as an item The records are stored in the user history operation log. The above-mentioned user history operation log can be a file in txt format, or a file in other formats.

S120. According to the above mobile device performance information, collection time, user historical operation information, and historical operation occurrence time, determine changes in the application performance of the tested application with collection time, user historical operation information, and historical operation occurrence time.

Specifically, the application performance of the application under test in this embodiment changes with the collection time, user historical operation information, and historical operation occurrence time can be reflected in the form of a line graph; of course, the change can also be reflected in other forms, Such as histograms, statistical tables, or text files.

Since the collection time and historical operation occurrence time in this embodiment can be fitted on the same time axis, the mobile device performance information and user historical operation information in this embodiment can be associated based on the time in the time axis of.

In this embodiment, the performance information of the mobile device and the collection time can be used to form a line graph, histogram, or statistical table, and then, the corresponding user historical operation information can be marked on the line graph, histogram, or statistical table by using the historical operation occurrence time. corresponding position in the table.

In this embodiment, when the obtained mobile device performance information contains multiple performance indicators, the change of each performance indicator with the collection time, user historical operation information, and historical operation occurrence time can be determined respectively; a specific example, In an actual application scenario where the obtained mobile device performance information includes mobile device CPU usage, mobile device memory usage, mobile device network traffic, and mobile device battery usage, this embodiment can read out from the mobile device performance information log The percentage of CPU consumed by all tested applications, the percentage of memory consumed by the tested application, the network traffic consumed by the tested application, the battery power of the mobile device, and the collection time corresponding to each information, and the read consumption of all the tested applications The percentage of CPU, the percentage of memory consumed by the application under test, the network traffic consumed by the application under test, the battery power of the mobile device, and the collection time corresponding to each information generate four polylines. These four polylines should usually be located in four different coordinate systems, of course It can also be located in the same coordinate system; no matter whether the four polylines are located in the same coordinate system or in four different coordinate systems, the abscissa of the coordinate system indicates the acquisition time, and the ordinate of the coordinate system indicates the performance index The specific quantitative value; then, read all the user's historical operation information and historical operation occurrence time from the user historical operation log, determine the position of the abscissa of each historical operation occurrence time in the coordinate system, and correspond to the historical operation occurrence time Mark the corresponding user historical operation information at the position of the broken line or the occurrence time of the historical operation at the position of the abscissa. This embodiment can mark based on the textual expression corresponding to the user historical operation information, or based on the identification corresponding to the user historical operation information characters, etc. to mark.

In this embodiment, an existing drawing plug-in can be used to draw the above-mentioned line graph, for example, the read data is provided to the drawing plug-in in a preset manner, so that the drawing plug-in can draw the above-mentioned line graph according to the data it receives.

S130. Output the change situation of the application performance of the application under test.

Specifically, the application performance testing device of this embodiment can display the application performance of the application under test with the collection time, user historical operation information, and historical operation time changes through its display screen (such as displaying the above-mentioned line graph), or through The printer outputs the change situation (such as printing a statistical table), of course, this embodiment can also output the change situation in other ways.

In this embodiment, in the actual application scenario where the display screen is used to output the change of the application performance of the application under test, various methods can be used to display the user's historical operation information marked in the coordinate system, such as the mouse pointing to the broken line of the application performance testing device When at a certain point in (or the touch position is at a certain point in the broken line), the textual expression or identifier corresponding to the user history operation information near this point is displayed explicitly, while the user history operation information at other positions The text expression or identifier corresponding to the information is in the hidden display state; for another example, when the mouse of the application performance testing device points to a certain point in the polyline (or the touch position is a certain point in the polyline), the mark is in the polyline The text expressions or identifiers corresponding to all the user history operation information are displayed explicitly.

In addition, when multiple polylines (such as four polylines) are located in the same coordinate system, the ordinates can respectively display the unit scales corresponding to the four polylines, such as using four ordinates to display the unit scales corresponding to the four polylines; Of course, this embodiment can also use other methods to display the unit scales corresponding to the four broken lines, for example, when the mouse of the application performance testing device points to a certain point in a certain broken line (or the touch position is a certain point in a certain broken line). point), the unit scale value of the ordinate of the coordinate system should display the unit scale value corresponding to the polyline; while the mouse of the application performance testing device does not point to any point in any polyline (or the touch position is not At any point in any polyline), the unit scale value of the ordinate of the coordinate system can be the unit scale value corresponding to a certain polyline set in advance.

A practical application of this embodiment will be described below with reference to the accompanying drawings 2-7. In this embodiment, the CPU usage percentage of the mobile device, the occupied space of the memory of the mobile device, the battery power of the mobile device, the network traffic of the mobile device, and the monitored historical user operation information collected during the running of the application under test are shown in Table 1 below. Show.

Table 1

For the above table 1, it needs to be specially explained that during the monitoring of user operations, a user operation may affect a continuous period of time. Therefore, the operation occurrence time corresponding to a user's historical operation information may correspond to multiple collection times, for example, The impact of the "input user name" operation monitored in the first second is a continuous process, so it can be considered that the performance information of multiple mobile devices collected before the next historical operation occurs is related to the "input user name" This user history operation information corresponds.

In this embodiment, when four coordinate systems are used to display the specific content in Table 1 above, the four generated line graphs are shown in FIGS. 2-5 . Specifically, this embodiment uses the collection time, historical operation occurrence time, and the line graph generated by the mobile device CPU occupation percentage and user historical operation information in Table 1 above, as shown in Figure 2, where the abscissa in Figure 2 represents time, The unit is seconds (s), and the ordinate represents the CPU occupation percentage, and the unit is a percentage value; this embodiment utilizes the collection time, the historical operation occurrence time, and the polyline generated by the occupied space of the mobile device memory and the user historical operation information in Table 1 above As shown in Fig. 3, the abscissa in Fig. 3 represents the time, the unit is second, and the ordinate represents the size of the memory footprint, the unit is MB (MByte, mega); The line graph generated by the battery power of the mobile device in Table 1 and the user's historical operation information is shown in Figure 4. The abscissa in Figure 4 represents time in seconds, and the ordinate represents battery voltage in Mv (millivolts); In this embodiment, the line graph generated by using the collection time, historical operation occurrence time, and the mobile device network traffic and user historical operation information in Table 1 above is shown in Figure 5. The abscissa in Figure 5 represents time in seconds, and the vertical Coordinates represent mobile device network traffic in KB.

In this embodiment, when a coordinate system is used to display the specific content in Table 1 above, the generated line chart is shown in Figure 6-7. Specifically, this embodiment uses the collection time, historical operation occurrence time, and the polyline generated by the CPU usage percentage of the mobile device, the occupied space of the mobile device memory, the battery power of the mobile device, the network traffic of the mobile device, and the historical user operation information in Table 1 above As shown in Figure 6 and Figure 7, the abscissa in Figure 6 and Figure 7 represents the time in seconds, and the ordinate can respectively represent the CPU usage percentage, memory footprint, battery voltage, and mobile device network traffic.

Embodiment 2, application performance testing device. The device of this embodiment will be described in detail below with reference to FIG. 8 .

The application performance testing device in Fig. 8 can be set in the mobile device running the application under test, also can be set in equipment such as a server connected wirelessly or wiredly with the mobile device running the application under test, or can be set partly in the mobile device running the application under test In the mobile device of the application under test, the other part is set in a device such as a server connected wirelessly or wired to the mobile device running the application under test.

The application performance testing device of this embodiment mainly includes: a performance monitoring module 200 , an operation monitoring module 210 , a performance determination module 220 and an output performance module 230 .

The performance monitoring module 200 may be completely set in the mobile device, may be completely set in the server, or may be partially set in the mobile device and partly set in the server. The performance monitoring module 200 is mainly adapted to acquire the performance information of the mobile device and the collection time of the performance information of the mobile device when the application under test is running on the mobile device.

Specifically, in the case that the performance monitoring module 200 is completely set on the mobile device running the application under test, the performance monitoring module 200 can regularly collect performance information of the mobile device according to a preset time interval during the running of the application under test, And the collected mobile device performance information and collection time are stored locally in the mobile device performance log. Afterwards, the performance determination module 220 can be conveniently obtained from the mobile device performance log stored locally on the mobile device when application performance analysis is required. The performance information of the mobile device and the collection time of the performance information of the mobile device during the operation of the mobile device by the application under test.

In the case where the performance monitoring module 200 is completely set on other devices connected to the mobile device running the application under test, such as a server, the mobile device (such as the application performance testing program installed in the mobile device, etc.) , regularly collect its performance information according to a preset time interval, and store the collected mobile device performance information and collection time locally in the mobile device performance log. After that, the mobile device (such as an application performance test program) can save the The mobile device performance log is uploaded to the server, so that the performance monitoring module 200 obtains the mobile device performance information and collection time from the information received by the server; if the performance monitoring module 200 sends a mobile device performance test information upload request to the mobile device through the server, the After the device receives the mobile device performance test information upload request transmitted from the server, it uploads its locally stored mobile device performance log to the server, so that the performance monitoring module 200 successfully obtains the mobile device performance information and collection time, and the performance monitoring module 200 can store the mobile device performance log received by the server locally in the server; in this way, when the performance determination module 220 in the server needs to perform application performance analysis, it can easily obtain the tested application from the locally stored mobile device performance log. The performance information of the mobile device and the collection time of the performance information of the mobile device during the operation of the mobile device.

In the case that the performance monitoring module 200 is partly set in the mobile device and partly set in other devices connected to the mobile device running the application under test, such as a server, the part of the performance monitoring module 200 set in the mobile device (hereinafter referred to as The first unit) regularly collects performance information of the application under test according to a preset time interval during the running process of the application under test, and locally stores the collected mobile device performance information and collection time in the mobile device performance log, and then, The first unit may upload the mobile device performance log to the server, so that the part of the performance monitoring module 200 set in the server (hereinafter referred to as the second unit) obtains the mobile device performance information and collection time from the information received by the server ; If the second unit sends a mobile device performance test information upload request to the mobile device through the server, after the mobile device receives the mobile device performance test information upload request transmitted by the server, the first unit will store the mobile device performance log locally stored in the mobile device Upload to the server, so that the second unit successfully obtains the mobile device performance information and collection time, and the second unit can store the mobile device performance log received by the server locally in the server; thus, the determining performance module 220 set in the server When application performance analysis is required, the performance information of the mobile device and the collection time of the performance information of the mobile device during the operation of the mobile device of the tested application can be easily obtained from the locally stored mobile device performance log.

The performance information of the mobile device in this embodiment usually includes multiple performance indicators of the mobile device, and the specific performance indicators included in the performance information of the mobile device can be determined according to the actual test requirements of the application; for example, the performance information of the mobile device can specifically include: the CPU of the mobile device Any one or any two or any three or all of usage, mobile device memory usage, mobile device network traffic, and mobile device battery usage.

The CPU usage of the above mobile device can be specifically: the CPU usage rate of the mobile device (that is, the percentage occupied by the mobile device) or the CPU idle rate of the mobile device (that is, the CPU idle percentage of the mobile device), and the CPU usage rate of the mobile device can be specifically defined as The CPU usage rate of the mobile device; the memory usage of the above mobile device can be specifically: the memory usage rate of the mobile device (that is, the percentage of memory occupied by the mobile device) or the idle rate of the memory of the mobile device (that is, the free percentage of the memory of the mobile device) or the memory usage of the mobile device The size of the space or the free space of the mobile device memory, and the mobile device memory occupancy rate can be specified as the mobile device memory occupancy rate of the application under test, and the occupied space of the mobile device memory can be specified as the mobile device memory occupancy space of the application under test size; the above-mentioned mobile device network traffic can be specifically the network traffic consumed by the application under test: and the mobile device network traffic can be any one or both of Wi-Fi data traffic and 2G/3G/4G data traffic; that is to say , the mobile device in this embodiment can distinguish the type of its current network traffic, so that the network traffic of the mobile device in the performance log of the mobile device can clearly indicate whether the network traffic currently consumed by the mobile device is Wi-Fi data traffic or 2G/ 3G/4G data traffic; the battery usage of the above-mentioned mobile device may specifically be the battery power of the mobile device (that is, the remaining power of the battery of the mobile device), and the battery power of the mobile device may be represented by the battery voltage.

The performance monitoring module 200 can collect the mobile device performance information of the tested application during the running of the mobile device by calling a corresponding API (Application Programming Interface, Application Programming Interface) in the operating system of the mobile device.

A specific example in which the performance monitoring module 200 of this embodiment collects the mobile device performance information of the application under test during the operation of the mobile device and locally stores the mobile device performance information and the collection time is: the performance monitoring module 200 After the interval arrives, call SystemProperties.java in the operating system to obtain the current mobile device CPU usage, call ActivityManager.java in the operating system to obtain the current mobile device memory usage, and call ConnectivityManager.java in the operating system to obtain the current mobile device usage. Device network traffic, call BatteryManager.java in the operating system to obtain the battery power of the current mobile device, the above four calling operations can be carried out at the same time under normal circumstances; when the performance monitoring module 200 executes the above calling operations or receives the return from the operating system When the current mobile device CPU usage rate, current mobile device memory usage rate, current mobile device network traffic, and current mobile device battery power, obtain the current time of the mobile device (that is, time stamp the performance information of the mobile device), and then, the performance monitoring module 200 stores the current CPU usage rate of the mobile device, the current memory usage rate of the mobile device, the network traffic of the current mobile device, the battery power of the current mobile device, and the current time of the mobile device as a record in the performance log of the mobile device. The performance monitoring module 200 of this embodiment can use the API provided by the operating system of the mobile device to obtain the current time of the mobile device, for example, use System.currentTimeMillis() in the operating system to obtain the current time of the mobile device.

The performance information log of the mobile device in this embodiment may be a file in txt format, or a file in other formats; and the performance information log of the mobile device may be stored in an SD card of the mobile device.

The operation monitoring module 210 may be completely set in the mobile device, may be completely set in the server, or may be partially set in the mobile device and partly set in the server. The operation monitoring module 210 is mainly adapted to obtain the user's historical operation information and the time when the tested application is running on the mobile device.

Specifically, in the case that the operation monitoring module 210 is completely set on the mobile device running the tested application, the operation monitoring module 210 can monitor the user's operation in real time during the running of the tested application (the user's operation here usually refers to The user's operation simulated by the test logic designed for application performance testing), and when it is detected that the user has performed the corresponding operation, the current time of the mobile device is obtained, and the operation information corresponding to the monitored user's operation is obtained And the above-mentioned current time is locally stored in the user historical operation log as user historical operation information and historical operation occurrence time; thus when the performance determination module 220 needs to perform application performance analysis, it can be conveniently obtained from the user historical operation log stored locally in the mobile device. Obtain the user's historical operation information and the time when the tested application is running on the mobile device.

In the case that the operation monitoring module 210 is completely set on other devices such as a server connected to the mobile device running the application under test, the operation monitoring module 210 can monitor the user's operation in real time during the running process of the application under test on the mobile device (here The user's operation usually refers to the user's operation simulated by the test logic designed for application performance testing), and when it is detected that the user has performed a corresponding operation, the operation monitoring module 210 obtains the current time of the mobile device (that is, gives the user Historical operation information is time stamped), and the operation information corresponding to the operation performed by the user and the above-mentioned current time are stored locally in the user historical operation log as user historical operation information and historical operation occurrence time. After that, the operation monitoring module 210 can log the user Upload the historical operation log to the server; for example, the operation monitoring module 210 actively uploads the locally stored user historical operation log to the server after the application under test runs; After the device performance test information upload request, the user history operation log stored locally on the mobile device is uploaded to the server; in this way, when the performance determination module 220 set in the server needs to perform application performance analysis, it can conveniently receive and locally store the log from the server. From the user history operation log of the tested application, the user history operation information and the time when the history operation occurred during the operation of the mobile device are obtained. Similarly, the operation monitoring module 210 in this embodiment can use the API provided by the operating system of the mobile device to obtain the current time of the mobile device, such as the operation monitoring module 210 using System.currentTimeMillis () in the operating system to obtain the current time of the mobile device the current time of .

In the case that the operation monitoring module 210 is partly set in the mobile device and partly set in other devices connected to the mobile device running the application under test, such as a server, the part of the operation monitoring module 210 set in the mobile device (hereinafter referred to as Unit 3) During the running of the application under test, monitor the user's operation in real time (the user's operation here usually refers to the user's operation simulated by the test logic designed for application performance testing), and monitor the user's execution When the corresponding operation is performed, the third unit acquires the current time of the mobile device (that is, time stamps the user's historical operation information), and uses the operation information corresponding to the operation performed by the user and the above-mentioned current time as the user's historical operation information and historical operation occurrence The time is locally stored in the user history operation log, and then the third unit can upload the user history operation log to the server, so that the part of the operation monitoring module 210 set in the server (hereinafter referred to as the fourth unit) receives from the server The user's historical operation information and historical operation occurrence time are obtained from the information of the user; for example, the fourth unit sends a mobile device performance test information upload request to the mobile device through the server, and the third unit receives the mobile device performance test information transmitted by the server on the mobile device After uploading the request, upload the user historical operation log locally stored on the mobile device to the server, so that the fourth unit can successfully obtain the user historical operation information and the time when the historical operation occurred, and the fourth unit can store the user historical operation log received by the server locally in the server; in this way, when the performance determination module 220 set in the server needs to perform application performance analysis, it can conveniently obtain the user history operation information of the application under test during the operation of the mobile device from the user history operation log stored locally in the server And the time when the historical operation occurred.

The mobile device performance information log and user history operation log in this embodiment should usually be uploaded to the server at the same time, for example, the performance monitoring module 200 and/or the operation monitoring module 210 actively upload the locally stored mobile device The performance information log and the user history operation log are uploaded to the server at the same time; for another example, after the performance monitoring module 200 and/or the operation monitoring module 210 receives the mobile device performance test information upload request transmitted from the server, the locally stored mobile Device performance information logs and user historical operation logs are uploaded to the server at the same time. The performance monitoring module 200 and/or the operation monitoring module 210 of this embodiment can upload the above two logs to the server by using the Upload() function. In addition, the performance monitoring module 200 and/or the operation monitoring module 210 may clear records in the successfully uploaded mobile device performance information log and user history operation log.

The above-mentioned user history operation log usually contains user history operation information corresponding to various operations performed by the user (including the user's operation simulated by the test logic), and the various operations performed by the user may specifically include: click, slide, and input events etc. This embodiment does not limit the specific content of the operations performed by the user.

The application performance testing program can usually obtain and identify the UI controls in the application under test, and call the corresponding UI controls in the application under test according to the corresponding test logic, so as to test the application under test. When the application performance test program invokes the corresponding UI control of the application under test, the operation monitoring module 210 can record the corresponding UI control information and the current calling time as user historical operation information and operation occurrence time in the user historical operation log.

The operation monitoring module 210 collects the user operation information of the application under test during the running of the mobile device, and stores the user operation information and the time of the operation locally. A specific example is: after the application performance test program runs, it can know All UI controls of the application under test (for example, the application performance test program can query all UI controls of the application under test by using the API provided by Robotium of the operating system), and then the application performance test program will call according to its preset test logic Corresponding UI controls, so that the application under test performs corresponding operations for the corresponding UI controls currently called (for example, the application performance test program can make the application under test perform the corresponding operations of the UI controls by using the API provided by Robotium of the operating system) , so as to test the performance of the application under test by imitating the actual operation of the user on the application under test; when the application performance test program calls the corresponding UI control and makes the application under test perform corresponding operations, the operation monitoring module 210 can record the UI The operation information corresponding to the control, and at the same time obtain the current time of the mobile device (that is, add a corresponding time stamp for the UI control), and then, the operation monitoring module 210 stores the operation information corresponding to the UI control and the current time as a record in the user history in the operation log. The above-mentioned user history operation log can be a file in txt format, or a file in other formats.

The determining performance module 220 may be completely set in the mobile device, and may also be completely set in the server. The determination performance module 220 is mainly adapted to determine the application performance of the application under test according to the performance information of the mobile device acquired by the performance monitoring module 200 and the operation monitoring module 210, collection time, user historical operation information and historical operation occurrence time. Operation information and changes in the time when historical operations occurred.

Specifically, the application performance of the application under test in this embodiment changes with the collection time, user historical operation information, and historical operation occurrence time can be reflected in the form of a line graph; of course, the change can also be reflected in other forms, Such as histograms, statistical tables, or text files.

The determining performance module 220 may first use the performance information of the mobile device and the collection time to form a line chart, a histogram, or a statistical table, etc., and then use the historical operation occurrence time to mark the corresponding user historical operation information on the line chart or histogram or corresponding position in the statistics table.

In the case that the obtained mobile device performance information contains multiple performance indicators, the determining performance module 220 can separately determine the change of each performance indicator with the collection time, user historical operation information and historical operation occurrence time; a specific example , in an actual application scenario where the obtained mobile device performance information includes mobile device CPU usage, mobile device memory usage, mobile device network traffic, and mobile device battery usage, the determining performance module 220 may read from the mobile device performance information log Take out the percentage of CPU consumed by all tested applications, the percentage of memory consumed by the tested application, the network traffic consumed by the tested application, the battery power of the mobile device, and the collection time corresponding to each information, and use all the read out measured The percentage of CPU consumed by the application, the percentage of memory consumed by the tested application, the network traffic consumed by the tested application, the battery power of the mobile device, and the collection time corresponding to each information generate four polylines. These four polylines should usually be located in four different coordinate systems , of course, can also be located in the same coordinate system; no matter whether the four polylines are located in the same coordinate system or in four different coordinate systems, the abscissa of the coordinate system indicates the acquisition time, and the ordinate of the coordinate system indicates The specific quantitative value of the performance index; then, determine the performance module 220 to read all user historical operation information and historical operation occurrence time from the user historical operation log, determine the position of the abscissa of each historical operation occurrence time in the coordinate system, and Mark the corresponding user historical operation information at the position of the broken line corresponding to the historical operation occurrence time or the abscissa position of the historical operation occurrence time, and the determining performance module 220 may mark based on the textual expression corresponding to the user historical operation information, or based on The identifier corresponding to the user's historical operation information is used for marking.

The determining performance module 220 can use the existing drawing plug-in to draw the above-mentioned line chart. For example, the determining performance module 220 provides the read data to the drawing plug-in in a preset manner, so that the drawing plug-in can draw a graph based on the data it receives. The line chart above.

The output performance module 230 can be completely set in the mobile device, and also can be completely set in the server. The output performance module 230 is mainly adapted to output the variation of the application performance of the tested application determined by the above-mentioned performance determination module 220 .

Specifically, the output performance module 230 can use the display screen of the application performance testing device to display the application performance of the application under test along with the collection time, user historical operation information and historical operation time changes (such as displaying the above-mentioned line graph), or through The printer outputs the change situation (such as printing a statistical table), of course, this embodiment can also output the change situation in other ways.

In the actual application scenario where the output performance module 230 uses the display screen to output changes in the application performance of the application under test, it can display the user's historical operation information marked in the coordinate system in various ways, such as pointing the mouse on the application performance testing device When a certain point in the broken line (or the touch position is a certain point in the broken line), the output performance module 230 makes the text expression or identifier corresponding to the user history operation information near the point displayed explicitly, so that Text expressions or identifiers corresponding to user historical operation information located in other locations are in a hidden display state; for another example, when the mouse of the application performance testing device points to a certain point in the polyline (or the touch position is a certain point in the polyline) , the output performance module 230 makes the text expressions or identifiers corresponding to all the user history operation information marked in the fold line be explicitly displayed.

In addition, when multiple polylines (such as four polylines) are located in the same coordinate system, the output performance module 230 can make the ordinates respectively display the unit scales corresponding to the four polylines. For example, the output performance module 230 uses four ordinates to display Display the unit scales corresponding to the four broken lines; of course, the output performance module 230 can also use other methods to display the unit scales corresponding to the four broken lines, for example, when the mouse of the application performance testing device points to a certain point in a certain broken line (or When the touch position is a certain point in a certain broken line), the output performance module 230 makes the unit scale value of the ordinate of the coordinate system should display the corresponding unit scale value of the broken line; When pointing to any point in any polyline (or the touch position is not at any point in any polyline), the output performance module 230 enables the unit scale value of the ordinate of the coordinate system to be a preset polyline The corresponding unit scale value.

A1, an application performance testing method, the method comprising:

Obtaining the performance information of the mobile device and the collection time of the performance information during the operation of the mobile device by the application under test;

Obtain the user's historical operation information and the time when the tested application is running on the mobile device;

According to the mobile device performance information, the collection time, the user historical operation information and the historical operation occurrence time, determine the application performance of the tested application with the collection time, the user historical operation information and the Changes in the time of historical operations;

Outputting changes in the application performance of the tested application.

A2. The application performance testing method according to A1, wherein the mobile device performance information includes: at least one of mobile device CPU usage, mobile device memory usage, mobile device network traffic, and mobile device battery usage.

A3. The application performance testing method according to A2, wherein the mobile device network traffic includes: at least one of Wi-Fi data traffic and 2G/3G/4G data traffic.

A4. The application performance testing method as described in A1 or A2 or A3, wherein said obtaining the performance information of the mobile device and the collection time of the performance information of the application under test during the operation of the mobile device includes:

receiving the mobile device performance log uploaded by the mobile device, and acquiring the mobile device performance information and the collection time from the mobile device performance log;

Wherein, the generation method of the mobile device performance log includes: the tested application is running on the mobile device, the mobile device periodically collects the mobile device performance information, and stores the collected mobile device performance information and collection time in the mobile device. in the device performance log.

A5. The application performance testing method as described in A1 or A2 or A3, wherein said obtaining user historical operation information and historical operation occurrence time of the application under test during the operation of the mobile device includes:

receiving the user historical operation log uploaded by the mobile device, and obtaining the user historical operation information and the time at which the historical operation occurred from the user historical operation log;

Wherein, the generation method of the user historical operation log includes: the mobile device monitors the user operation during the running process of the tested application, and uses the monitored user operation information and operation occurrence time information as the user historical operation information and historical operation occurrence information. Times are stored in the mobile device performance log.

A6. The application performance testing method as described in A1 or A2 or A3, wherein said outputting the variation of the application performance of the tested application includes:

Outputting changes in the application performance of the tested application in the form of a line graph;

Wherein, the abscissa of the line graph represents the collection time, the ordinate of the line graph represents the performance information of the mobile device, and the fold lines in the line graph are marked with corresponding User history operation information.

A7. The application performance testing method as described in A6, wherein, in the application scenario where the mobile device performance information includes: mobile device CPU usage, mobile device memory usage, mobile device network traffic, and mobile device battery usage, The line graph includes: a line graph of the CPU usage of the mobile device, a line graph of the memory usage of the mobile device, a line graph of the network traffic of the mobile device, and a line graph of the battery usage of the mobile device.

B8. An application performance testing device, the device comprising:

The performance monitoring module is adapted to obtain performance information of the mobile device and the collection time of the performance information during the operation of the mobile device of the application under test;

The operation monitoring module is suitable for obtaining the user's historical operation information and the historical operation time during the operation of the mobile device under test;

The determining performance module is adapted to determine the application performance of the tested application according to the performance information of the mobile device, the collection time, the user's historical operation information and the occurrence time of the historical operation. Historical operation information and changes in the time at which the historical operations occurred;

The output performance module is adapted to output the variation of the application performance of the tested application.

B9. The application performance testing device as described in B8, wherein the mobile device performance information includes: at least one of mobile device CPU usage, mobile device memory usage, mobile device network traffic, and mobile device battery usage.

B10. The application performance testing device as described in B9, wherein the mobile device network traffic includes: at least one of Wi-Fi data traffic and 2G/3G/4G data traffic.

B11. The application performance testing device as described in B8 or B9 or B10, wherein the performance monitoring module is specifically adapted to: receive a mobile device performance log uploaded by a mobile device, and obtain the mobile device performance log from the mobile device performance log. Equipment performance information and the collection time;

Wherein, the generation method of the mobile device performance log includes: the tested application is running on the mobile device, the mobile device periodically collects the mobile device performance information, and stores the collected mobile device performance information and collection time in the mobile device. in the device performance log.

B12. The application performance testing device as described in B8 or B9 or B10, wherein the operation monitoring module is specifically adapted to: receive the user history operation log uploaded by the mobile device, and obtain the user history operation log from the user history operation log Historical operation information and the time when the historical operation occurred;

Wherein, the generation method of the user historical operation log includes: the mobile device monitors the user operation during the running process of the tested application, and uses the monitored user operation information and operation occurrence time information as the user historical operation information and historical operation occurrence information. Times are stored in the mobile device performance log.

B13. The application performance testing device as described in B8, B9 or B10, wherein the output performance module is specifically adapted to: output the change of the application performance of the tested application in the form of a line graph;

Wherein, the abscissa of the line graph represents the collection time, the ordinate of the line graph represents the performance information of the mobile device, and the fold lines in the line graph are marked with corresponding User history operation information.

B14. The application performance testing device as described in B13, wherein, in the application scenario where the mobile device performance information includes: mobile device CPU usage, mobile device memory usage, mobile device network traffic, and mobile device battery usage, The line graph includes: a line graph of the CPU usage of the mobile device, a line graph of the memory usage of the mobile device, a line graph of the network traffic of the mobile device, and a line graph of the battery usage of the mobile device.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct such a system is apparent from the above detailed description. Furthermore, the present invention is not specific to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline this disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or its description. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method or method so disclosed may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. and form different embodiments. For example, in the following claims, any one of the claimed embodiments may be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in the application performance testing device according to the embodiment of the present invention. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, third, etc. does not indicate any order and these words can be interpreted as names.

Claims (10)

1. An application performance testing method, the method comprising: Obtaining the performance information of the mobile device and the collection time of the performance information during the operation of the mobile device by the application under test; Obtain the user's historical operation information and the time when the tested application is running on the mobile device; According to the mobile device performance information, the collection time, the user historical operation information and the historical operation occurrence time, determine the application performance of the tested application with the collection time, the user historical operation information and the Changes in the time of historical operations; Outputting changes in the application performance of the tested application. 2. The application performance testing method according to claim 1, wherein the mobile device performance information comprises: at least one of the mobile device CPU usage, the mobile device memory usage, the mobile device network traffic, and the mobile device battery usage . 3. The application performance testing method according to claim 2, wherein the mobile device network traffic comprises: at least one of Wi-Fi data traffic and 2G/3G/4G data traffic. 4. The application performance testing method according to claim 1, 2 or 3, wherein said obtaining the performance information of the mobile device and the collection time of the performance information of the application under test during the operation of the mobile device comprises: receiving the mobile device performance log uploaded by the mobile device, and acquiring the mobile device performance information and the collection time from the mobile device performance log; Wherein, the generation method of the mobile device performance log includes: the tested application is running on the mobile device, the mobile device periodically collects the mobile device performance information, and stores the collected mobile device performance information and collection time in the mobile device. in the device performance log. 5. The application performance testing method according to claim 1 , 2 or 3, wherein said obtaining user historical operation information and historical operation occurrence time of the application under test during the operation of the mobile device comprises: receiving the user historical operation log uploaded by the mobile device, and obtaining the user historical operation information and the time at which the historical operation occurred from the user historical operation log; Wherein, the generation method of the user historical operation log includes: the mobile device monitors the user operation during the running process of the tested application, and uses the monitored user operation information and operation occurrence time information as the user historical operation information and historical operation occurrence information. Times are stored in the mobile device performance log. 6. The application performance testing method according to claim 1, 2 or 3, wherein said outputting the variation of the application performance of the application under test comprises: Outputting changes in the application performance of the tested application in the form of a line graph; Wherein, the abscissa of the line graph represents the collection time, the ordinate of the line graph represents the performance information of the mobile device, and the fold lines in the line graph are marked with corresponding User history operation information. 7. The application performance testing method according to claim 6, wherein the mobile device performance information includes: application scenarios of mobile device CPU usage, mobile device memory usage, mobile device network traffic, and mobile device battery usage Wherein, the line graph includes: a line graph of the CPU usage of the mobile device, a line graph of the memory usage of the mobile device, a line graph of the network traffic of the mobile device, and a line graph of the battery usage of the mobile device. 8. An application performance testing device, the device comprising: The performance monitoring module is adapted to obtain performance information of the mobile device and the collection time of the performance information during the operation of the mobile device of the application under test; The operation monitoring module is suitable for obtaining the user's historical operation information and the historical operation time when the application under test is running on the mobile device; The determining performance module is adapted to determine the application performance of the tested application according to the performance information of the mobile device, the collection time, the user's historical operation information and the occurrence time of the historical operation. Historical operation information and changes in the time at which the historical operations occurred; The output performance module is adapted to output the variation of the application performance of the tested application. 9. The application performance testing device according to claim 8, wherein the mobile device performance information includes: at least one of the mobile device CPU usage, the mobile device memory usage, the mobile device network traffic, and the mobile device battery usage . 10. The application performance testing device according to claim 9, wherein the mobile device network traffic comprises: at least one of Wi-Fi data traffic and 2G/3G/4G data traffic.