CN114756435A - Log reading method, server, computer device and storage medium - Google Patents
Log reading method, server, computer device and storage medium Download PDFInfo
- Publication number
- CN114756435A CN114756435A CN202210514585.3A CN202210514585A CN114756435A CN 114756435 A CN114756435 A CN 114756435A CN 202210514585 A CN202210514585 A CN 202210514585A CN 114756435 A CN114756435 A CN 114756435A
- Authority
- CN
- China
- Prior art keywords
- log
- module
- log data
- application
- application service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/301—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is a virtual computing platform, e.g. logically partitioned systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the specification provides a log reading method, a server, a computer device and a storage medium. The method includes intercepting log data generated by the application service through a log interception module in the same virtual machine as the application service. Storing the log data to a specified file path pre-configured in the log intercepting module; therefore, at least part of log data in the designated file path is read based on the log acquisition module separated from the virtual machine, the log data are acquired in a reliable and non-intrusive mode, the influence on the progress of an application program is reduced, the integrity of the log data is ensured under the condition that the downstream storage is unreliable, and the accuracy of a monitoring result is improved.
Description
Technical Field
The embodiment of the specification relates to the technical field of computers, in particular to a log reading method, a server, computer equipment and a storage medium.
Background
The APM (Application Performance Management) is a solution for performing Performance Management and fault Management of an Application by monitoring the Application in real time. Specifically, the APM can monitor the application program, and when the application program has an abnormal problem, the application program can be analyzed to locate the abnormality.
However, in the conventional technology, the log analysis platform mainly obtains log data by the following two ways: one is that log data is sent to a log analysis platform through log agent by installing log acquisition tool log agent in a server running an application program; one is to send the log data directly to the log analysis platform within the application process.
Disclosure of Invention
In view of the above, embodiments of the present disclosure are directed to providing a log reading method, a server, a computer device, and a storage medium, which enable log data to be collected in a reliable and non-intrusive manner.
The embodiment of the present specification provides a server, including: the system comprises a virtual machine, an interface service module and a log acquisition module; a plurality of application services and a log interception module run in the virtual machine; the log intercepting module is used for intercepting log data generated by the application service and storing the log data to a specified file path pre-configured in the log intercepting module; the specified file path represents a memory address of the server; the log data stored in the specified file path corresponds to a path identifier; the interface service module is used for recording the path identifier and providing the path identifier for the log acquisition module; and the log acquisition module is used for reading the log data in the specified file path according to the path identifier.
An embodiment of the present specification provides a log reading method, including: intercepting log data generated by application service by adopting a log interception module; the log intercepting module and the application service are in the same virtual machine; storing the log data to a specified file path pre-configured in the log intercepting module; wherein the specified file path represents an address of a local memory; reading at least part of log data in the designated file path based on a log collection module separated from the virtual machine.
An embodiment of the present specification provides a log reading apparatus, including: the log data interception module is used for intercepting log data generated by the application service by adopting the log interception module; the log intercepting module and the application service are in the same virtual machine; the log data storage module is used for storing the log data to a specified file path pre-configured in the log interception module; wherein the specified file path represents an address of a local memory; and the log data reading module reads at least part of log data in the specified file path based on a log acquisition module separated from the virtual machine.
The present specification provides a computing device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method steps of the above embodiments when executing the computer program.
The present specification provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the method steps in the above embodiments.
The present specification embodiments provide a computer program product, which includes instructions that, when executed by a processor of a computer device, enable the computer device to perform the method steps in the above embodiments.
In the embodiment of the present specification, log data generated by an application service is intercepted by a log interception module in the same virtual machine as the application service. Storing the log data to a specified file path pre-configured in the log intercepting module; therefore, at least part of log data in the specified file path is read based on the log collection module separated from the virtual machine, so that the log data can be collected in a reliable and non-invasive mode, the influence on the progress of an application program is reduced, the integrity of the log data is ensured under the condition that the downstream storage is unreliable, and the accuracy of a monitoring result is improved.
Drawings
FIG. 1 is a diagram illustrating a log collection system in an example scenario, according to an embodiment;
FIG. 2 is a flowchart illustrating a log reading method according to an embodiment;
FIG. 3 is a flowchart illustrating a log reading method according to an embodiment;
FIG. 4 is a flowchart illustrating a log reading method according to an embodiment;
FIG. 5 is a block diagram illustrating a server according to an embodiment;
FIG. 6 is a diagram illustrating a daily log analysis configuration page, according to an embodiment;
FIG. 7 is a block diagram illustrating a log reading apparatus according to an embodiment;
fig. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art without any inventive work based on the embodiments in the present specification belong to the protection scope of the present specification.
In a specific scenario example, kubernets are installed, which provides a mechanism for application deployment, planning, updating, and maintenance. Kubernets provides a distributed architecture solution based on container technology. A Container (Linux Container) may be understood as a resource partition technology that divides resources (such as a processor, a memory, a disk, and the like) of a physical machine into a plurality of resource pools, so that an application may independently use a resource pool without being affected by other applications. The partitioned resource pool and the application using the portion of the resource pool can be referred to as a container. In kubenets, all containers operate in a Pod, and a Pod can carry one or more associated containers. Containers in the same Pod may be deployed on the same physical machine and can share resources. It is understood that a physical machine is a physical server, including hardware and systems, such as a Linux server.
In this scenario example, a Virtual machine jvm (java Virtual machine), an interface service module (apiserver), and a log collection module (logAgent) run on the Linux server. A plurality of application services, including, for example, Buy application services, may run in the virtual machine JVM. The Buy application service is bound to the runtime environment. For load balancing, two copies may be configured for the Buy application service, and then the Buy application service corresponds to two processes, and the two integration in kubenets may be referred to as two pod. The interface service module apiserver may record the names (pod names) of the two pods, which are respectively denoted as buy-1 and buy-2. The log collection module logAgent can communicate with an interface service module apiserver in Kubenetes. The log collection module logAgent can monitor the application installed with java-agent.
In this scenario example, the Buy application service is provided with additional information with a specified annnotion tag. And when detecting that the annnotion mark in the additional information of the Buy application service is an assigned mark, installing a log interception module java-agent for the Buy application service in a virtual machine JVM running the Buy application service. At this time, pod names (buy-1, buy-2) in the interface service module apiserver have corresponding mark information. The log interception module Java agent can be understood as a special Java program (Jar file) which is an interceptor running before the main method. The method can be started by attaching the java-agent parameters to the Buy application service, and realizes the functions of intercepting and modifying the byte codes during class loading.
In the scene example, the log framework is changed through the log interception module java-agent, and the log interception module java-agent is pre-configured with a specified file path for storing log data. The specified file path may be a local file system of the Linux server. For example, the file path is designated as/var/log/buy-1/stdout.log,/var/log/buy-2/stdout.log. It should be noted that, it is specified that buy-1 and buy-2 in the file path are pod names. And intercepting log data generated by the Buy application service by using a log interception module java-agent. And the log interception module java-agent stores the log data into a specified file path/var/log/buy-1/stdout.log,/var/log/buy-2/stdout.log.
In the example of the scenario, the log collection module logAgent communicates with the interface service module apiserver, and the interface service module apiserver records pod names (buy-1 and buy-2 with mark information). And the log acquisition module logAgent sends an application identifier query request to the interface service module apiserver. The application identifier query request carries tag information, and the interface service module apiserver obtains two application identifiers (pod names) based on the application identifier query request: buy-1 and buy-2. The interface service module apiserver returns application identifications buy-1 and buy-2 to the log collection module logAgent. Generating a specified file path/var/log/buy-1/stdout.log,/var/log/buy-2/stdout.log according to the application identifications buy-1 and buy-2, and reading log data from paths of/var/log/buy-1/stdout.log and/var/log/buy-2/stdout.log by a log collection module logAgent. And the log acquisition module logAgent sends the read Japanese data to a log analysis platform.
In the example of the scenario, a/var/log/{ $ PodName } }/stdout.log path may be agreed between the log collection module logAgent and the log interception module java-agent. { $ PodName } } is used to represent the application process identification (such as name). The log intercepting module java-agent pre-configures a specified file path for redirecting and storing log data according to the application process identifier, and the log collecting module logAgent reads data from the specified file path according to the application process identifier obtained by inquiry, so as to realize log collection.
Referring to fig. 1, an embodiment of the present disclosure provides a log collection system, and a log reading method provided by the present disclosure is applied to the log reading system. The log reading system may include a hardware environment formed by the log analysis platform 110 and the server 120. The log analysis platform 110 communicates with the server 120 over a network. The server 120 includes a virtual machine, an interface service module, and a log collection module. The system comprises a virtual machine, a log interception module and a server, wherein a plurality of application services and the log interception module run in the virtual machine; the log intercepting module is used for intercepting log data generated by the application service and storing the log data to a specified file path pre-configured in the log intercepting module; specifying a file path to represent a memory address of a server; the log data stored by the specified file path corresponds to a path identifier; the interface service module is used for recording the path identifier and providing the path identifier for the log acquisition module; and the log acquisition module is used for reading the log data in the specified file path according to the path identifier. The log collection module sends the read log data to the log analysis platform 110; the log analysis platform 110 receives the log data and monitors the application service according to the log data.
The log analysis platform 110 and the server 120 may be electronic devices with certain computing and processing capabilities. Which may have a network communication module, a processor, memory, etc. The server may also be a distributed server, and may be a system having a plurality of processors, memories, network communication modules, and the like that cooperate with one another. Alternatively, the server may be a server cluster formed by several servers. Or, with the development of scientific technology, the server can also be a new technical means capable of realizing the corresponding functions of the specification implementation mode. For example, it may be a new form of "server" implemented based on quantum computing.
Referring to fig. 2, an embodiment of the present disclosure provides a log reading method. The log reading method is applied to the server, and the log reading method can comprise the following steps.
Step S210: intercepting log data generated by application service by adopting a log interception module; and the log intercepting module and the application service are in the same virtual machine.
The log data may be obtained by writing information into a file in order to record a state and an operating process of the application service, and the recorded data is referred to as log data. The log intercepting module may be a process of intercepting an original write log of the application service based on a Hook mechanism (Hook) so as to make the server execute the write log preconfigured in the log intercepting module. An application service may be a running activity of a program with independent functionality with respect to a certain data set, such as a transaction process or an application process that may be understood to run on a server.
Specifically, in some cases, a plurality of application services are run in the server, and log data generated by each application service is stored in several or even more than ten storage paths. If the generated log data need to be sent to a log analysis platform, operation and maintenance personnel need to manually configure the storage paths, the operation and maintenance pressure is huge, and a large amount of manpower and time are needed. Therefore, in the present embodiment, a log interception module is provided in a virtual machine running an application service, and log data generated by the application service is intercepted in a non-intrusive manner. It should be noted that the log intercepting module may be understood as an extension program in the same virtual machine as the application service bubble.
Step S220: storing the log data to a specified file path pre-configured in a log interception module; wherein the specified file path represents an address of the local memory.
Specifically, in order to reduce operation and maintenance pressure, a specified file path is preconfigured in the log intercepting module, and when log data generated by the application service is intercepted by the log intercepting module, the intercepted log data can be stored in the preconfigured specified file path, so that the randomness and diversity of log data storage addresses are reduced, and the operation and maintenance pressure is reduced. Particularly, when the number of log files is large, frequent switching among random storage paths is not needed, the writing performance is improved, the resource contention is reduced, and the probability of log acquisition delay or log loss is reduced. Furthermore, when the downstream storage is unreliable, the local file system is used as a cache to store the log data, so that the integrity of the log data is ensured, and the monitoring accuracy is improved.
In this embodiment, a log frame (such as a logback) can be changed by the log interception module, and a code for setting a specified file path is added to the log interception module. The specified file path may be a fixed file path that has been previously present in the local storage, such as a certain storage path in C-disc. The specified file path may be a file path that needs to be generated using an identifier (e.g., $) in conjunction with application identification information (e.g., application process name), such as a buy process running on a server. Log interception module can output log data of application service under a/var/log/buy/stdout. As another example, a cart process is running on the server. Log interception module can output log data of application service under/var/log/cats/stdout.
Step S230: and reading at least part of log data in the designated file path based on a log collection module separated from the virtual machine.
The log collection module can be a process which is deployed on a server and has a log data transmission function, and is used for providing http service for the log analysis platform to pull logs. The log collection module is separated from the virtual machine, namely the log collection module and the application service are isolated from each other. The log collection module may be understood as a log collection probe, which faces the local file system and is used to read log data from the local file system. In some embodiments, the log collection module may employ a log collection tool loggent.
Specifically, in order to reduce operation and maintenance pressure, an agreement is established between the log intercepting module and the log collecting module, and the log collecting module can directly read log data from a specified file path based on the agreement without configuring a path for reading logs by the log collecting module. Therefore, based on the communication mechanism, the log collection module can acquire the specified file path which is pre-configured in the log interception module, and read at least part of log data stored in the specified file path.
According to the log reading method, log data generated by the application service is intercepted through the log intercepting module in the same virtual machine as the application service. Storing the log data to a specified file path pre-configured in a log interception module; therefore, at least part of log data in the designated file path is read based on the log acquisition module separated from the virtual machine, the log data are acquired in a reliable and non-invasive mode, the influence on the application program process is reduced, the integrity of the log data is ensured under the condition that the downstream storage is unreliable, and the accuracy of the monitoring result is improved.
In some embodiments, the log reading method may further include the steps of: and mounting a log intercepting module into a virtual machine running with application service.
In particular, the log interception module may be an executable file (such as a jar package). The method comprises the steps of obtaining a code storage path of a log interception module, mounting (attach) the log interception module to a virtual machine running with application service based on the code storage path, enhancing byte codes of enhanced classes when the virtual machine dynamically loads one class, intercepting log data generated by the application service, and storing the log data under a preset file path in the log interception module.
In the embodiment, the log intercepting module is mounted in the virtual machine running with the application service, so that the non-invasive collection of log data is realized in a byte code enhancement mode, the running of the application service is not influenced, and the service performance is improved.
In some embodiments, mounting the log interception module into a virtual machine running an application service may include: and mounting the log intercepting module into the virtual machine when detecting that the additional information of the application service is attached with a specified mark.
Specifically, the application service is provided with additional information, the additional information is monitored, and in the case that the additional information is detected to be accompanied by a specified mark, the log data of the application service needs to be intercepted. In order not to influence the operation of the application service, the log intercepting module is mounted into the virtual machine running the application service.
In the embodiment, whether the log intercepting module is installed for the application service is determined by detecting the accessory information, the code is not required to be rewritten by a user through byte code enhancement, the coding requirement on the user is reduced, and further, the influence on the memory of the transaction process can be reduced by mounting the log intercepting module into the virtual machine.
In some embodiments, the application service has an application identification. Storing the log data to a specified file path pre-configured in the log interception module may include: and storing the log data to a specified file path which is pre-configured based on the application identifier of the application service in the log interception module.
The application identifier of the application service may be an application name or an application ID. Specifically, if only the log interception module is used, log data needs to be directly sent to the log analysis platform in the application program process, and if the downstream storage is unreliable, the memory or the CPU of the application process is affected without discarding the log data, so that the application is inconsistent and cannot normally run. If the log data is lost, the log data obtained by the log analysis data is incomplete, and the loss of the log can cause inaccuracy of application monitoring. Therefore, in the embodiment, when the log interception module intercepts the log data generated by the application service, the log data is stored in the local file system, so that the negative effect when the downstream storage is unreliable is overcome. Further, in order to improve the performance of accessing log data, the specific file path pre-configured in the log interception module is determined based on the application identification of the application service. Illustratively, a buy process runs on the server. The specified file path may be a/var/log/buy/stdout. A cart process runs on the server. The specified file path may be a/var/log/cats/stdout.
In some embodiments, referring to fig. 3, reading at least a portion of log data within a specified file path based on a log collection module separate from the virtual machine may include the following steps.
Step S310: and sending an application identifier query request to the interface service component based on the log collection module, so that the interface service component returns the application identifier of the application service to the log collection module.
Step S320: and generating a specified file path according to the application identifier of the application service.
Step S330: and reading at least part of log data in the specified file path by adopting a log acquisition module.
Specifically, an interface service component is deployed in the server, and the interface service component is in communication with the log collection module. The interface service component can record the application identification of the application service. The application identification of the application service is based on the agreement between the log collection module and the log interception module. The log collection module may send an application identifier query request to the interface service component, and the interface service component receives the application identifier query request and queries to obtain an application identifier of the application service. And the interface service component returns the application identifier of the application service to the log acquisition module. The log collection module receives the application identifier of the application service, and determines a specified file path for storing the log data based on the application identifier of the application service, so that the log collection module reads at least part of the log data in the specified file path.
In the embodiment, the log acquisition module can quickly and accurately access the specified file path pre-configured based on the application identifier by querying the application identifier of the application service obtained by the interface service component, so that the performance of reading log data is improved, operation and maintenance personnel are not required to configure the reading path, and the operation and maintenance cost is reduced.
In some embodiments, the application service corresponds to a number of container groups; the application identification of the application service adopts the name identification of the container group.
In some cases, application services are bound by the runtime environment. With kubenets as an operating environment, two copies can be configured for application service, and then the application service corresponds to two application processes. The two application processes are referred to as two container groups (pod) in kubenets. In the kubenets environment, the interface service component apiserver records that the application service has a name identifier (pod name) of 2 container groups.
In this embodiment, the log collection module sends an application identifier query request to the interface service component, and the interface service component returns the name identifier of the container group to the log collection module. The log intercepting module identifies a pre-configured specified file path based on the name of the container group, and therefore the log collecting module determines the specified file path based on the application identification of the application service. The log collection module reads at least part of log data in the specified file path. Illustratively, the specified file path may be/var/log/{ $ podname } }/stdout. { $ podname } } represents the name identification of the specific container group. Such as two processes buy-1, buy-2 applying buy. "buy-1" and "buy-2" can be denoted as the name identifier (pod name) of the container group.
In some embodiments, the log reading method may include the steps of: and sending at least part of the read log data to a log analysis platform by using a log collection module. The log analysis platform is used for monitoring the application service according to at least part of log data.
Specifically, a log acquisition module is deployed in the server and used as a log acquisition probe which acquires log data in a specified file path. The log acquisition probe is a log data transmission channel and is used for transmitting the read log data to the log analysis platform. And the log analysis platform receives and analyzes the log data, and monitors the application service according to the analysis result.
Illustratively, an application service a is deployed in the server, and the application service a corresponds to a process XX and a process YY. The method comprises the step of having a log interception module in a virtual machine running with an application service A. The log interception module is pre-configured with a path/var/log/XX/stdout. And the log interception module intercepts log data generated by the process XX and stores the log data to a path/var/log/XX/stdout. And the log interception module intercepts log data generated by the process YY and stores the log data to a path/var/log/YY/stdout. The log collection module accesses/var/log/XX/stdout.log and/var/log/YY/stdout.log to read log data. And the log acquisition module sends the read log data to a log analysis platform. And the log analysis platform monitors the process XX and the process YY according to the log data.
In this embodiment, the log analysis platform may store corresponding log data through a log aggregation system (such as Loki). For example, the log aggregation system may employ a Loki log system. The Loki log system is particularly well suited to storing kubernets Pod logs. Metadata such as Pod tags may be automatically deleted and indexed. The Loki journal system consists of 3 parts: loki is the main server, responsible for storing log data and processing queries; the proxy is a specially-customized agent for Loki and is responsible for collecting log data and sending the log data to Loki; grafana is used for UI presentation. The user can view the log data stored in Loki through Grafana.
Referring to fig. 4, an embodiment of the present disclosure provides a log reading method. The log reading method may include the following steps.
Step S402: and mounting a log intercepting module into a virtual machine running with application service.
Specifically, when detecting that the additional information of the application service is accompanied by a specified mark, the log intercepting module is mounted into the virtual machine.
Step S404: intercepting log data generated by application service by adopting a log interception module; and the log intercepting module and the application service are in the same virtual machine.
Step S406: and storing the log data to a specified file path pre-configured in the log interception module.
Wherein the specified file path represents an address of a local memory; the application service has an application identification. Specifically, the log data is stored to a specified file path pre-configured based on the application identifier of the application service in the log interception module.
Step S408: and sending an application identifier query request to the interface service component based on the log collection module, so that the interface service component returns the application identifier of the application service to the log collection module.
Step S410: and generating a specified file path according to the application identifier of the application service.
Step S412: and reading at least part of log data in the specified file path by adopting a log acquisition module.
Step S414: sending at least part of the read log data to a log analysis platform by adopting a log acquisition module; the log analysis platform is used for monitoring the application service according to at least part of log data.
It should be understood that, although the steps in the above-described flowcharts are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the above-mentioned flowcharts may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or the stages is not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a part of the steps or the stages in other steps.
Referring to fig. 5, an embodiment of the present disclosure provides a server 500. The server 500 includes a virtual machine 510, an interface service module 520, and a log collection module 530.
Running in virtual machine 510 are a number of application services 512 and a log intercept module 514.
The interface service module 520 is configured to record the path identifier and provide the path identifier to the log collection module 530.
The log collection module 530 is configured to read log data in a path of the specified file according to the path identifier.
The log intercepting module 514 is configured to intercept log data generated by the application service, and store the log data to a specified file path pre-configured in the log intercepting module 514. The specified file path represents a memory address of the server. And the log data stored by the specified file path corresponds to a path identifier. The path identifier may be a file name of the log file, which is used to determine the specified file path. The path identification may be the path itself that specifies the path of the file. The path identification may also be an application name, which is used to determine the specified file path.
Specifically, the log interception module 514 is preconfigured with a specified file path. The interface service module 520 stores a path identifier corresponding to a specified file path. The interface service module 520 provides the path identification to the log collection module 530. The log collection module 530 accesses the specified file path according to the path identifier, and reads log data from the specified file path.
In this embodiment, the path identifier may be a path itself that specifies a file path. The appointed file path is appointed between the log intercepting module 514 and the log collecting module 530, the appointed file path is stored in the interface service module 520, the interface service module 520 and the log collecting module 530 can communicate, and the log collecting module 530 sends a request to the interface service module 520 to obtain the appointed file path. Therefore, the log collection module 530 accesses the specified file path and reads log data therein.
In this embodiment, the path identifier may be a file name of the log file. The log files generated by different application services are named by the respective corresponding file names, and the agreement is made between the log intercepting module 514 and the log collecting module 530 based on the naming. The interface service module 520 may store a file name of the log file, and the log collection module 530 sends a request to the interface service module 520 to obtain the file name of the log file. Therefore, the log collection module 530 determines a specified file path based on the file name of the log file, and accesses the specified file path to read log data therein.
In this embodiment, the path identifier may be an application name. Log files generated by different application services exist in a path directory corresponding to the application name. On this basis, agreement is made between the log intercept module 514 and the log collection module 530. The interface service module 520 may store an application name of the application service, and the log collection module 530 sends a request to the interface service module 520 to obtain the application name of the application service. Therefore, the log collection module 530 determines a specified file path based on the application name of the application service, and accesses the specified file path to read log data therein.
In the server, a virtual machine comprising an application service and a log interception module, an interface service module and a log acquisition module are arranged. The interface service module and the log collection module are communicated with each other, so that the log collection module can determine a specified file path for storing log data, which is pre-configured in the log interception module. The log data acquisition is realized in a non-intrusive mode, the memory of a transaction process is not occupied, the work of deploying the log acquisition module is simplified, the operation and maintenance cost is reduced, the integrity of the log data is ensured by storing the log data to a local file system, and the probability of losing the log data is reduced.
In some embodiments, the server further comprises an acquisition rule configuration module. And the acquisition rule configuration module is used for determining a pre-configured acquisition rule of the log data. And the log intercepting module is also used for intercepting log data meeting the pre-configured acquisition rule.
Specifically, the server comprises a collection rule configuration module, the collection rule configuration module can receive collection rule configuration operation sent by a user, and a pre-configured collection rule of log data is determined based on the collection rule configuration operation. The collection rule configuration module may send the preconfigured collection rule to the log interception module. And intercepting log data generated by the application service according to the pre-configured acquisition rule by a log interception module, namely intercepting the log data meeting the pre-configured acquisition rule.
For example, please refer to fig. 6. And the server accesses a log analysis configuration page of the log analysis platform, and displays the log analysis configuration page. And a log frame automatic acquisition switch control, a log level acquisition check control and a log length input box are arranged in the log analysis configuration page. The collection log level includes WARN, INFO, ERROR, DEBUG.
In some embodiments, the log collection module is further configured to send the read log data to a log analysis platform; the log analysis platform is used for monitoring application services according to log data.
Specifically, a log acquisition module is deployed in the server and used as a log acquisition probe, and the log acquisition probe acquires log data in a specified file path. The log acquisition probe is a log data transmission channel and is used for transmitting the read log data to the log analysis platform. And the log analysis platform receives and analyzes the log data, and monitors the application service according to the analysis result.
Illustratively, the log analysis platform may store corresponding log data via a log aggregation system (such as Loki). For example, the log aggregation system may employ a Loki log system. The Loki log system is particularly well suited to storing kubernets Pod logs. Metadata such as Pod tags may be automatically deleted and indexed. The Loki journal system consists of 3 parts: loki is the main server, responsible for storing log data and processing queries; the promtail is an agent specially customized for the Loki and is responsible for collecting log data and sending the log data to the Loki; grafana is used for UI presentation. The user can view the log data stored in Loki through Grafana.
Referring to fig. 7, an embodiment of the present disclosure provides a log reading apparatus. The log reading apparatus may include a log data intercepting module, a log data storing module, and a log data reading module.
The log data interception module is used for intercepting log data generated by the application service by adopting the log interception module; and the log interception module and the application service are in the same virtual machine.
The log data storage module is used for storing the log data to a specified file path pre-configured in the log interception module; wherein the specified file path represents an address of the local memory.
And the log data reading module is used for reading at least part of log data in the specified file path based on the log acquisition module separated from the virtual machine.
In some embodiments, the log reading apparatus may further include an interception module mounting module for mounting the log interception module into a virtual machine running the application service.
In some embodiments, the intercepting module is further configured to mount the log intercepting module into the virtual machine in the case that the additional information of the application service is detected to be accompanied by the specified mark.
In some embodiments, the application service has an application identification; and the log data storage module is also used for storing the log data to a specified file path which is pre-configured in the log interception module based on the application identifier of the application service.
In some embodiments, the log data reading module is further configured to send an application identifier query request to the interface service component based on the log collection module, so that the interface service component returns the application identifier of the application service to the log collection module; generating a specified file path according to the application identifier of the application service; and reading at least part of log data in the specified file path by adopting a log acquisition module.
In some embodiments, the application service corresponds to a number of container groups; the application identification of the application service is identified by the name of the container group.
In some embodiments, the log reading apparatus may further include a log data sending module, configured to send at least part of the read log data to the log analysis platform by using the log collection module; the log analysis platform is used for monitoring the application service according to at least part of log data.
For the specific limitation of the log reading device, reference may be made to the above limitation on the log reading method, which is not described herein again. The respective modules in the log reading apparatus can be wholly or partially implemented by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 8. The computer device comprises a processor, a memory, a communication interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a log reading method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key or a touch pad arranged on a shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing device to which the disclosed aspects apply, and that a computing device may in particular include more or less components than those shown, or combine certain components, or have a different arrangement of components.
In some embodiments, a computer device is provided, comprising a memory in which a computer program is stored and a processor which, when executing the computer program, carries out the method steps of the above embodiments.
In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the method steps in the above-described embodiments.
In some embodiments, there is also provided a computer program product comprising instructions which, when executed by a processor of a computing device, implement the method steps in the above embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The features of the above embodiments may be arbitrarily combined, and for the sake of brevity, all possible combinations of the features in the above embodiments are not described, but should be construed as being within the scope of the present specification as long as there is no contradiction between the combinations of the features.
The above description is only for the purpose of illustrating the preferred embodiments of the present disclosure and is not to be construed as limiting the present disclosure, and any modifications, equivalents and the like that are within the spirit and principle of the present disclosure are intended to be included within the scope of the present disclosure.
Claims (12)
1. A server, characterized in that the server comprises: the system comprises a virtual machine, an interface service module and a log acquisition module;
a plurality of application services and a log interception module run in the virtual machine; the log intercepting module is used for intercepting log data generated by the application service and storing the log data to a specified file path pre-configured in the log intercepting module; the specified file path represents a memory address of the server; the log data stored in the specified file path corresponds to a path identifier;
the interface service module is used for recording the path identifier and providing the path identifier for the log acquisition module;
and the log acquisition module is used for reading the log data in the specified file path according to the path identifier.
2. The server of claim 1, further comprising an acquisition rule configuration module;
the acquisition rule configuration module is used for determining a pre-configured acquisition rule of the log data;
the log intercepting module is further used for intercepting log data meeting the pre-configuration collecting rule.
3. The server according to claim 1, wherein the log collection module is further configured to send the read log data to a log analysis platform; the log analysis platform is used for monitoring the application service according to the log data.
4. A method for reading a log, the method comprising:
intercepting log data generated by application service by using a log interception module; the log intercepting module and the application service are in the same virtual machine;
storing the log data to a specified file path pre-configured in the log intercepting module; wherein the specified file path represents an address of a local memory;
reading at least part of log data in the designated file path based on a log collection module separated from the virtual machine.
5. The method of claim 4, further comprising:
and mounting the log intercepting module into a virtual machine running the application service.
6. The method of claim 5, wherein the mounting the log interception module into a virtual machine running the application service comprises:
and mounting the log intercepting module into the virtual machine under the condition that the additional information of the application service is detected to be attached with a specified mark.
7. The method according to any of claims 4 to 6, wherein the application service has an application identity; the storing the log data to a specified file path pre-configured in the log intercepting module includes:
storing the log data to a specified file path pre-configured based on the application identifier of the application service in the log interception module.
8. The method of claim 7, wherein reading at least a portion of log data within the specified file path based on a log collection module separate from the virtual machine comprises:
sending an application identifier query request to an interface service component based on the log acquisition module, so that the interface service component returns an application identifier of the application service to the log acquisition module;
generating the specified file path according to the application identifier of the application service;
and reading at least part of log data in the specified file path by adopting the log acquisition module.
9. The method of claim 7, wherein the application service corresponds to a plurality of container groups; and the application identifier of the application service adopts the name identifier of the container group.
10. The method of claim 4, wherein the method comprises:
sending the read at least part of log data to a log analysis platform by using the log acquisition module; wherein the log analysis platform is configured to monitor the application service according to the at least part of the log data.
11. A computer device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of any one of claims 4 to 10.
12. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of a computer device, enable the computer device to perform the method of any of claims 4 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210514585.3A CN114756435A (en) | 2022-05-12 | 2022-05-12 | Log reading method, server, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210514585.3A CN114756435A (en) | 2022-05-12 | 2022-05-12 | Log reading method, server, computer device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114756435A true CN114756435A (en) | 2022-07-15 |
Family
ID=82336015
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210514585.3A Pending CN114756435A (en) | 2022-05-12 | 2022-05-12 | Log reading method, server, computer device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114756435A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117707439A (en) * | 2023-08-22 | 2024-03-15 | 荣耀终端有限公司 | Log printing method and related device |
-
2022
- 2022-05-12 CN CN202210514585.3A patent/CN114756435A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117707439A (en) * | 2023-08-22 | 2024-03-15 | 荣耀终端有限公司 | Log printing method and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10884722B2 (en) | Cross-environment application of tracing information for improved code execution | |
| KR102541295B1 (en) | Operating system customization in an on-demand networked code execution system | |
| US10606565B2 (en) | Visual devops systems and methods | |
| US10459822B1 (en) | Iterative static analysis using stored partial results | |
| US11714675B2 (en) | Virtualization-based transaction handling in an on-demand network code execution system | |
| US10503623B2 (en) | Monitoring containerized applications | |
| US9229758B2 (en) | Passive monitoring of virtual systems using extensible indexing | |
| CN112364110A (en) | Metadata management method, device and equipment and computer storage medium | |
| CN110365724B (en) | Task processing method and device and electronic equipment | |
| US20130111018A1 (en) | Passive monitoring of virtual systems using agent-less, offline indexing | |
| CN114168179B (en) | Micro-service management method, micro-service management device, computer equipment and storage medium | |
| CN113342767A (en) | Log generation method, device, equipment and storage medium | |
| CN112866348A (en) | Database access method and device, computer equipment and storage medium | |
| CN112187509A (en) | Multi-architecture cloud platform execution log management method, system, terminal and storage medium | |
| CN114756435A (en) | Log reading method, server, computer device and storage medium | |
| US10073689B2 (en) | Managing application lifecycles within a federation of distributed software applications | |
| CN114691445A (en) | Cluster fault processing method and device, electronic equipment and readable storage medium | |
| US10853215B2 (en) | Intelligent configuration management of user devices | |
| CN111399999A (en) | Computer resource processing method and device, readable storage medium and computer equipment | |
| US11777810B2 (en) | Status sharing in a resilience framework | |
| CN114816668A (en) | Virtual machine kernel monitoring method, device, equipment and storage medium | |
| CN111488230A (en) | Method and device for modifying log output level, electronic equipment and storage medium | |
| CN113741912A (en) | Model management system, method, device and equipment | |
| US10203970B2 (en) | Dynamic configuration of native functions to intercept | |
| US20240338299A1 (en) | Detecting funtional anomalies associated with software services in a distributed computing environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |