CN113590449B - Python-based application performance data acquisition method - Google Patents
Python-based application performance data acquisition method Download PDFInfo
- Publication number
- CN113590449B CN113590449B CN202110915972.3A CN202110915972A CN113590449B CN 113590449 B CN113590449 B CN 113590449B CN 202110915972 A CN202110915972 A CN 202110915972A CN 113590449 B CN113590449 B CN 113590449B
- Authority
- CN
- China
- Prior art keywords
- python
- application
- probe
- performance data
- thread
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G06F11/3414—Workload generation, e.g. scripts, playback
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses an application performance data acquisition method based on Python.A Python probe appoints a start script when starting host application and automatically executes automatic hook operation of an import module in the start script, so that built-in service codes of the Python probe are injected into the host application, and some performance data are acquired in a point-burying mode; the method can quickly locate Python internal errors and request failures, stably acquire request records, database operations and Python calling relations, and greatly improve the robustness and stability of Python application; the risk problems can be avoided, and meanwhile the deployment efficiency of the Python applications can be greatly improved, so that the real-time monitoring on the health state of the application operation is completed on the premise of ensuring the stable operation of the applications.
Description
Technical Field
The invention belongs to the field of Application Performance Monitoring (APM), and particularly relates to a Python-based application performance data acquisition technology.
Background
In the prior art, a conventional probe collector needs a user to manually bury a point to collect application data, meanwhile, a service code needs to be modified to realize data collection, and the code also needs to be modified in different places according to different Python frames, so that a plurality of Python applications consume a lot of time when being used, and the use efficiency is greatly reduced.
Therefore, it is necessary to provide a Python-based application performance data acquisition technology, so that the robustness and stability of Python application are improved, and the use efficiency is improved.
Disclosure of Invention
The invention aims at the development requirements of the probe collector and is used for helping a user to quickly find problems, accurately position the problems and instantly solve the problems, and the probe automatically releases resources under the condition of insufficient server resources, thereby realizing the real-time monitoring of the health state of application operation on the premise of ensuring the stable operation of the application.
The technical scheme adopted by the invention for solving the technical problems comprises the following steps:
a Python-based application performance data acquisition method comprises the following steps:
s1, the host application starts the application through a script provided by the Python probe, and automatically executes the automatic hook operation of the import module in the starting script;
s2, the Python probe generates a hotid according to the configuration file content;
s3, after the hostid is successfully acquired, host registration operation is carried out;
s4, sending the Python probe heartbeat to the back end after successful registration, and judging the life state of the current Python probe by the back end;
s5, license application and occupation are carried out after the heartbeat is successfully sent;
and S6, after all the operations, the Python probe is buried in the host application for collecting performance data.
Preferably, after the step S1, that is, after the Python probe successfully starts the host application, 6 threads are started to perform probe logic operation, where the 6 threads specifically are:
the thread 1 is used for realizing the license check function and judging whether the current license is in an available state; the thread 2 is used for sending heartbeats per minute and reporting the current probe state; the thread 3 is used for synchronizing the configuration of the server; thread 4 is used for application registration and implements local caching according to request frequency; the thread 5 is used for assembling the data structure of the acquired performance data; thread 6 is used to send the assembled performance data to the backend service.
Preferably, the configuration of the thread 3 for the synchronization server specifically includes a fusing condition, a critical transaction, and a sampling rate.
Preferably, the host application is a Python application, and the step S2 further includes that after the Python application is started through a script provided by a Python probe, the Python probe automatically executes a hook (interception) operation when the Python application executes an import module, and the Python probe injects a built-in service code of the Python probe into the Python application through the operation of intercepting the Python application.
Preferably, the step S5 further includes the Python probe intercepting the Python application by burying the point.
Preferably, the operations of the Python application include database statement execution, request initiation, and method execution.
Preferably, the Python probe calculates the database statement execution time, the request response time, and the method execution time by using a Python code, and reports the performance data collected from the Python application to the back end of the Python probe.
Preferably, the Python application developer can see real-time Python application performance data on a platform page of the Python probe.
The invention has the technical effects that:
the invention relates to a Python-based application performance data acquisition method, wherein a Python probe is used for injecting a built-in service code of the Python probe into a host application by designating a start script when the host application is started and automatically executing an automatic hook operation of an import module in the start script, and acquiring some performance data in a point-burying mode, for example: the execution time of database statements, the request response time, the method execution time and the like, and the collected performance data are sent to the back end for data analysis, classification and display. After the Python probe is integrated into an application, only a script provided by the Python probe needs to be used for starting, and a service code does not need to be modified to perform point burying operation subsequently, so that internal errors and request failures of the Python can be quickly positioned, the acquisition request record, the database operation and the Python calling relation are stabilized, and the robustness and the stability of the Python application are greatly improved; the risk problems can be avoided, and meanwhile the deployment efficiency of the Python applications can be greatly improved, so that the real-time monitoring on the health state of the application operation is completed on the premise of ensuring the stable operation of the applications.
The invention will be further described with reference to the accompanying drawings.
Drawings
FIG. 1 is a flow chart of a Python-based application performance data acquisition method according to the present invention;
FIG. 2 is an application list diagram of a Python-based application performance data acquisition method of the present invention;
FIG. 3 is an application overview of a Python-based application performance data collection method of the present invention;
FIG. 4 is a request list diagram of a Python-based application performance data collection method of the present invention;
FIG. 5 is a detailed alarm event and alarm list for a Python-based application performance data collection method of the present invention;
FIG. 6 is a fusing threshold setting diagram of a Python-based application performance data acquisition method of the present invention;
FIG. 7 is a page diagram of an error overview of a technology company using a Python-based application performance data collection method of the present invention;
FIG. 8 is a page diagram of an error detail of a technology company using a Python-based application performance data collection method of the present invention;
fig. 9 is a page diagram of an error detail of a technology company using the Python-based application performance data collection method of the present invention.
Detailed Description
The embodiments of the present invention will be further described with reference to the accompanying drawings so that the present invention can be further understood by those skilled in the art without limiting the scope of the present invention.
With reference to fig. 1, fig. 1 is a flowchart of a Python-based application performance data acquisition method according to the present invention. In the embodiment, the host application starts the application through a script provided by a Python probe, and automatically executes automatic hook operation of an import module in the starting script; firstly, the Python probe generates a hotid according to the content of the configuration file, host registration operation is carried out after the hotid is successfully acquired, the Python probe heartbeat is sent to the back end after the successful registration, and the back end can judge the life state of the current Python probe. After the heartbeat is successfully sent, license application is carried out and occupation is carried out; after this operation the Python probe will have some buried points within the host application for collecting performance data.
The Python probe starts to start host application, 6 threads are started to perform probe logic operation after the Python probe is started successfully, and the thread 1 is used for realizing license check function and judging whether the current license is in an available state; the thread 2 is used for sending heartbeats per minute and reporting the current probe state; the thread 3 is used for synchronizing the configuration of the server, such as fusing condition, key transaction, sampling rate and the like; thread 4 is used to apply registration and implement local caching according to request frequency; the thread 5 is used for carrying out data structure assembly on the acquired performance data; the thread 6 is used for sending the assembled performance data to a back-end service;
the Python-based application performance data acquisition method in this embodiment includes the following processes: the Python application starts itself via a script provided by the Python probe (Python application); as the Python application is started through the script provided by the Python probe, the Python probe automatically executes hook (interception) operation when the Python application executes the import module, and the Python probe can inject the built-in service code of the Python probe into the Python application through the operation of intercepting the Python application; since the Python probe has been injected into the Python application, the Python probe can intercept some operations of the Python application in a point-burying manner, for example: database statement execution, request initiation, method execution, and the like; the Python probe can calculate the execution time of the database statement, the request response time, the method execution time and the like through the Python code because the Python probe intercepts the operation, and the Python probe reports the performance data collected from the Python application and sends the performance data to the rear end of the Python probe; since the Python probe is already at the back end of the Python probe sending the performance data collected from the Python application, the Python application developer can go to the platform page of the Python probe to see the real-time Python application performance data.
In this embodiment, the display of the platform page of the process and result of the Python-based application performance data acquisition method is combined with fig. 2 to 6, and fig. 2 is an application list diagram of the Python-based application performance data acquisition method according to the present invention; FIG. 3 is an application overview of a Python-based application performance data collection method of the present invention; FIG. 4 is a request list diagram of a Python-based application performance data collection method of the present invention; FIG. 5 is a detailed alarm event and alarm list for a Python-based application performance data collection method of the present invention; fig. 6 is a fusing threshold setting diagram of the Python-based application performance data acquisition method according to the present invention.
Products of a certain technology limited company have a huge user group, and tens of service lines such as a purchase-sale-storage system, an ERP system, a CRM system, a financial system, a budget system, a Saas system and the like and thousands of hosts are arranged in the products, when the system is abnormal or requests slowly, the problem is difficult to locate due to the fact that the end user of a certain region or a certain operator cannot visit a website and use except the abnormality of monitoring flow and access amount in the back-end monitoring of the traditional operation and maintenance, and even if the reduction of the user access amount is sensed from the flow abnormality, the problem is difficult to locate; and on the application performance monitoring platform, a unified view, fault management, root cause analysis, a strong report form, a friendly Web client side and multi-user role permission of IT resources can be provided, and the reason that the end user cannot access can be found and positioned very timely and accurately. Often, after seeing the fault from the platform or receiving the alarm information sent from the platform, users find the fault of the product; by using the acquisition technology of the invention, a customer can quickly find problems, accurately position the problems and instantly solve the problems on the premise of not influencing the normal and efficient operation of the application, and by combining with fig. 7-9, fig. 7 is the same, and fig. 7 is an error overview page diagram of a technology company using the Python-based application performance data acquisition method of the invention, and the errors are checked through the error overview page; FIG. 8 is a page diagram of an error detail of a technology company using a Python-based application performance data collection method of the present invention; the error details can be seen by clicking the error list, and the user can see the error information according to the error details of the user; FIG. 9 is a page diagram of an error detail of a technology company using a Python-based application performance data collection method of the present invention; the client may locate the problem by locating the stack details in an error or slow request.
The invention relates to a Python-based application performance data acquisition method, wherein a Python probe is used for injecting a built-in service code of the Python probe into a host application by designating a start script when the host application is started and automatically executing an automatic hook operation of an import module in the start script, and acquiring some performance data in a point-burying mode, for example: the execution time of database statements, the request response time, the method execution time and the like, and the collected performance data are sent to the back end for data analysis, classification and display. After the Python probe is integrated into an application, only a script provided by the Python probe needs to be used for starting, and a service code does not need to be modified to perform point burying operation subsequently, so that internal errors and request failures of the Python can be quickly positioned, the acquisition request record, the database operation and the Python calling relation are stabilized, and the robustness and the stability of the Python application are greatly improved; the risk problems can be avoided, and meanwhile the deployment efficiency of the Python applications can be greatly improved, so that the real-time monitoring on the health state of the application operation is completed on the premise of ensuring the stable operation of the applications.
The above examples are given for the purpose of illustrating the invention clearly and not for the purpose of limiting the same, and it will be apparent to those skilled in the art that, in light of the foregoing description, numerous modifications and variations can be made in the form and details of the embodiments of the invention described herein, and it is not intended to be exhaustive or to limit the invention to the precise forms disclosed.
Claims (7)
1. A Python-based application performance data acquisition method is characterized by comprising the following steps:
s1, the host application starts the application through a script provided by the Python probe, and automatically executes the automatic hook operation of the import module in the starting script;
s2, the Python probe generates a hotid according to the configuration file content;
s3, after the hostid is successfully acquired, host registration operation is carried out;
s4, sending the Python probe heartbeat to the back end after successful registration, and judging the life state of the current Python probe by the back end;
s5, license application and occupation are carried out after the heartbeat is successfully sent;
s6, after all the operations, the Python probe is used for collecting performance data by embedding points in the host application;
the step S2 further includes that after the Python application is started through a script provided by a Python probe, the Python probe automatically executes a hook interception operation when the Python application executes an import module, and the Python probe injects a built-in service code of the Python probe into the Python application through the operation of intercepting the Python application.
2. The Python-based application performance data collection method of claim 1, wherein after the step S1, that is, after the Python probe successfully starts the host application, 6 threads are started for performing probe logic operations, wherein the 6 threads are specifically:
the thread 1 is used for realizing the license check function and judging whether the current license is in an available state; the thread 2 is used for sending heartbeats per minute and reporting the current probe state; the thread 3 is used for synchronizing the configuration of the server; thread 4 is used for application registration and implements local caching according to request frequency; the thread 5 is used for assembling the data structure of the acquired performance data; thread 6 is used to send the assembled performance data to the backend service.
3. The Python-based application performance data collection method of claim 2, wherein the configuration of the thread 3 for the synchronization server specifically includes a fusing condition, a critical transaction, and a sampling rate.
4. The Python-based application performance data collection method according to claim 1, wherein the step S5 further comprises the Python probe intercepting Python application operation by means of a buried point.
5. The Python-based application performance data collection method of claim 4, wherein the Python application operations include database statement execution, request initiation, and method execution.
6. The Python-based application performance data collection method of claim 5, wherein the Python probe calculates database statement execution time, request response time and method execution time by Python code, and reports the collected performance data from Python application to the back end of Python probe.
7. The Python-based application performance data collection method of claim 6, wherein the Python application developer can see the Python application performance data in real time on the platform page of the Python probe.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110915972.3A CN113590449B (en) | 2021-08-11 | 2021-08-11 | Python-based application performance data acquisition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110915972.3A CN113590449B (en) | 2021-08-11 | 2021-08-11 | Python-based application performance data acquisition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113590449A CN113590449A (en) | 2021-11-02 |
| CN113590449B true CN113590449B (en) | 2022-03-04 |
Family
ID=78256958
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110915972.3A Active CN113590449B (en) | 2021-08-11 | 2021-08-11 | Python-based application performance data acquisition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113590449B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106301971A (en) * | 2016-11-17 | 2017-01-04 | 国家电网公司 | Electric power application performance monitoring system based on flow analysis |
| CN110650137A (en) * | 2019-09-23 | 2020-01-03 | 煤炭科学技术研究院有限公司 | Coal mine network abnormal behavior early warning method, system, equipment and readable storage medium |
| CN110704277A (en) * | 2019-09-27 | 2020-01-17 | 中电万维信息技术有限责任公司 | Method for monitoring application performance, related equipment and storage medium |
| CN112650527A (en) * | 2020-12-30 | 2021-04-13 | 中通天鸿（北京）通信科技股份有限公司 | Intelligent release system for automatically putting products on line and monitoring products |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6834276B1 (en) * | 1999-02-25 | 2004-12-21 | Integrated Data Control, Inc. | Database system and method for data acquisition and perusal |
| CN111031050B (en) * | 2019-12-16 | 2022-07-15 | 深圳市国电科技通信有限公司 | Monitoring method and device for electricity consumption information acquisition system |
-
2021
- 2021-08-11 CN CN202110915972.3A patent/CN113590449B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106301971A (en) * | 2016-11-17 | 2017-01-04 | 国家电网公司 | Electric power application performance monitoring system based on flow analysis |
| CN110650137A (en) * | 2019-09-23 | 2020-01-03 | 煤炭科学技术研究院有限公司 | Coal mine network abnormal behavior early warning method, system, equipment and readable storage medium |
| CN110704277A (en) * | 2019-09-27 | 2020-01-17 | 中电万维信息技术有限责任公司 | Method for monitoring application performance, related equipment and storage medium |
| CN112650527A (en) * | 2020-12-30 | 2021-04-13 | 中通天鸿（北京）通信科技股份有限公司 | Intelligent release system for automatically putting products on line and monitoring products |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113590449A (en) | 2021-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7640459B2 (en) | Performing computer application trace with other operations | |
| US6070190A (en) | Client-based application availability and response monitoring and reporting for distributed computing environments | |
| US6321263B1 (en) | Client-based application availability | |
| US7954011B2 (en) | Enabling tracing operations in clusters of servers | |
| Cohen et al. | Capturing, indexing, clustering, and retrieving system history | |
| US8028200B2 (en) | Tracing operations in multiple computer systems | |
| US20080098359A1 (en) | Manipulation of trace sessions based on address parameters | |
| US8060782B2 (en) | Root cause problem identification through event correlation | |
| US6175832B1 (en) | Method, system and program product for establishing a data reporting and display communication over a network | |
| US20100333072A1 (en) | Integrated performance and load testing tool for application servers | |
| US20100031252A1 (en) | Method And System For Monitoring The Performance Of An Application And At Least One Storage Device For Storing Code Which Performs The Method | |
| TW201411367A (en) | Method and system for monitoring execution of user request in distributed system | |
| CN111382023A (en) | Code fault positioning method, device, equipment and storage medium | |
| CN105119762A (en) | System and method of cloud platform for realizing transaction playback and transaction reworking based on logs | |
| CN105637488A (en) | Tracing source code for end user monitoring | |
| CN103248511B (en) | A kind of analysis methods, devices and systems of single-point service feature | |
| US11874728B2 (en) | Software application diagnostic aid | |
| US20060190488A1 (en) | System and method for determining information related to user interactions with an application | |
| CN110636116B (en) | Multidimensional data acquisition system and method | |
| CN112052135A (en) | Client program user operation record and exception reporting method based on C/S architecture | |
| CN113590449B (en) | Python-based application performance data acquisition method | |
| US7653742B1 (en) | Defining and detecting network application business activities | |
| CN101321319A (en) | Monitoring and recovery apparatus and method for invalid connection of business database | |
| CN114167181A (en) | Method and system for monitoring local and allopatric line fault tracing | |
| CN107515864B (en) | Method and equipment for monitoring workflow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |