CN112988528B - Log processing method, device and container group - Google Patents
Log processing method, device and container group Download PDFInfo
- Publication number
- CN112988528B CN112988528B CN201911281126.XA CN201911281126A CN112988528B CN 112988528 B CN112988528 B CN 112988528B CN 201911281126 A CN201911281126 A CN 201911281126A CN 112988528 B CN112988528 B CN 112988528B
- Authority
- CN
- China
- Prior art keywords
- log
- container
- configuration information
- processing
- acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000000605 extraction Methods 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000003362 replicative effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application discloses a log processing method, a log processing device and a container group, and relates to the field of cloud computing. One embodiment of the method comprises the following steps: firstly, acquiring configuration information corresponding to a container group in which a log acquisition container is positioned through the log acquisition container, wherein the configuration information is used for indicating the log acquisition container to carry out log processing on application containers in the container group, the log acquisition container is injected into the container group in advance, and a log storage catalog of the application containers in the container group is stored in the log acquisition container; then, according to the configuration information, the logs in the log storage catalog are processed through the log collection container. According to the method and the device, the log of the application container in the container group is processed through the log collection container, the log service does not need to be bound, the resource consumption for log processing is reduced, and the efficiency of log processing is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a log processing method, a log processing device and a container set.
Background
The log service is one-stop service for collecting, transmitting, storing and processing log data. Most software services need to discover, locate and track problems through log services, and log collection is the basis of log services. When monitoring alarm is needed according to the log information, index extraction aiming at the log information is indispensable.
At present, log collection of all containers in a node is performed by deploying log collection containers on the node. In the log acquisition process, dynamically discovering standard output and log files of all containers in the node through a service discovery module; and then, transmitting the acquired file to a storage device, and extracting indexes from the log file in the storage device through a log service.
Disclosure of Invention
The embodiment of the application provides a log processing method, a log processing device and a container group.
In a first aspect, an embodiment of the present application provides a log processing method, where the method includes: the method comprises the steps that configuration information corresponding to a container group in which a log collection container is located is collected through the log collection container, the configuration information is used for indicating the log collection container to conduct log processing on application containers in the container group, the log collection container is injected into the container group in advance, and a log storage catalog of the application containers in the container group is stored in the log collection container; and processing the logs in the log storage catalog through the log acquisition container according to the configuration information.
In some embodiments, the above method further comprises: and acquiring log attribute information representing the processing requirement of a user on the log through the log acquisition container, and generating indication configuration information according to the log attribute information.
In some embodiments, the configuration information is extracted configuration information used to characterize log index extraction.
In a second aspect, an embodiment of the present application provides a container set, where the container set includes an application container and a log collection container, where the log collection container stores a log storage directory of the application container in the container set; the log collection container is internally provided with a control module and a log processing module; the control module is used for collecting configuration information corresponding to the container group and controlling the log processing module to process the log in the log storage catalog according to the configuration information.
In some embodiments, the log collection container is further provided with a configuration collection module; the configuration acquisition module is used for acquiring log attribute information representing the processing requirement of a user on the log and generating configuration information indicating the log processing module to perform log processing according to the log attribute information.
In some embodiments, the configuration information is extracted configuration information used to characterize log index extraction.
In some embodiments, the control module is further configured to determine whether the configuration information has changed; in response to determining that the configuration information changes, controlling operation of the log processing module to process the log in the log storage directory information according to the change information of the configuration information.
In some embodiments, the log collection container is further provided with a self-monitoring module; the self-monitoring module is used for collecting the running state information and logs of each module in the log collecting container.
In a third aspect, an embodiment of the present application provides a log processing apparatus, where the apparatus includes: the information acquisition unit is configured to acquire configuration information corresponding to a container group in which the log acquisition container is located through the log acquisition container, wherein the configuration information is used for indicating the log acquisition container to carry out log processing on application containers in the container group, the log acquisition container is injected into the container group in advance, and a log storage catalog of the application containers in the container group is stored in the log acquisition container; and the log processing unit is configured to process the logs in the log storage catalog through the log collection container according to the configuration information.
In some embodiments, the apparatus further comprises: and the information generating unit is configured to acquire log attribute information representing the processing requirements of the user on the log through the log acquisition container and generate indication configuration information according to the log attribute information.
In some embodiments, the configuration information is extracted configuration information used to characterize log index extraction.
In a fourth aspect, embodiments of the present application provide a computer readable medium having a computer program stored thereon, wherein the program when executed by a processor implements a method as described in any of the implementations of the first aspect.
In a fifth aspect, embodiments of the present application provide an electronic device, including: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method as described in any of the implementations of the first aspect.
According to the log processing method, the log processing device and the container group, firstly, configuration information corresponding to the container group in which the log collection container is located is collected through the log collection container, the configuration information is used for indicating the log collection container to conduct log processing on the application container in the container group, the log collection container is injected into the container group in advance, and a log storage catalog of the application container in the container group is stored in the log collection container; then, according to the configuration information, the logs in the log storage catalog are processed through the log collection container. According to the method and the device, the log of the application container in the container group is processed through the log collection container, the log service does not need to be bound, the resource consumption for log processing is reduced, and the efficiency of log processing is improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings, in which:
FIG. 1 is an exemplary system architecture diagram in which an embodiment of the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a log processing method according to the present application;
fig. 3 is a schematic diagram of an application scenario of the log processing method according to the present embodiment;
FIG. 4 is a flow chart of yet another embodiment of a log processing method according to the present application;
FIG. 5 is a block diagram of one embodiment of a container group according to the present application;
FIG. 6 is a block diagram of one embodiment of a log processing device according to the present application;
FIG. 7 is a schematic diagram of a computer system suitable for use in implementing embodiments of the present application.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
FIG. 1 illustrates an exemplary architecture 100 in which the log processing methods and apparatus of the present application may be applied.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The terminal devices 101, 102, 103 may be hardware devices or software supporting network connections for data interaction and data processing. When the terminal device 101, 102, 103 is hardware, it may be various electronic devices supporting functions of information interaction, network connection, etc., including but not limited to smart phones, tablet computers, electronic book readers, laptop and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices. It may be implemented as a plurality of software or software modules, for example, for providing distributed services, or as a single software or software module. The present invention is not particularly limited herein.
The server 105 may be a server that provides various services, for example, a server that receives access requests of the terminal devices 101, 102, 103 to provide functions of image recognition, data processing, data matching, and the like. The server can store or process the received various data and feed back the processing result to the terminal equipment.
It should be noted that the log processing method provided by the embodiment of the present disclosure may be executed by the server 105. Accordingly, the log processing means may be provided in the server 105. An automated container manipulation platform, such as a Kubernetes platform, which is an open-source container orchestration engine, is installed in server 105, which can containerize applications in the server such that various applications support the orchestration scheduling of containers.
The server may be hardware or software. When the server is hardware, the server may be implemented as a distributed server cluster formed by a plurality of servers, or may be implemented as a single server. When the server is software, it may be implemented as a plurality of software or software modules, for example, for providing distributed services, or as a single software or software module. The present invention is not particularly limited herein.
It should be understood that the number of terminal devices and servers in fig. 1 is merely illustrative. There may be any number of terminal devices and servers, as desired for implementation.
With continued reference to FIG. 2, a flow 200 is shown according to one embodiment of the log processing method of the present application, including the steps of:
step 201, acquiring configuration information corresponding to a container group where a log acquisition container is located through the log acquisition container.
In this embodiment, an execution body (e.g., a server in fig. 1) installs an automated container manipulation platform that can containerize applications in the server so that various applications support orchestration scheduling of containers. For example, the automated container handling platform may be a Kubernetes platform.
In the Kubernetes platform, the programs running thereon are packaged into containers, each containing one or more applications, with application containerization allowing users to create a self-contained Linux execution environment. The programs in the container and all of their dependent settings may be packaged into a file and then shared across the network. Anyone can download the container and deploy the container on the owned infrastructure.
One container or a plurality of containers requiring cooperation may constitute a container group. The container group is the smallest basic unit for the Kubernetes platform to create or deploy an application. If a horizontal extension of an application instance is required, multiple container groups may be used to run instances separately, i.e., application extension is achieved by replicating the container groups.
The container group may be provided on a node on which the Kubernetes platform is installed. A node may be a physical server or virtual machine on which one or more container groups may be located. Multiple nodes with Kubernetes platform installed may form a cluster.
In the embodiment, the Kubernetes platform can automatically deploy and copy containers, can expand or shrink the container scale at any time, and can realize load balancing among containers.
In this embodiment, the log collection container is pre-injected into the container group, that is, the container group includes the application container and the log collection container. The log collection container stores a log storage catalog of an application container in a container group to which the log collection container belongs, and the log storage catalog of the application container stores a log file of the application container.
Log collection containers may be injected into the container group in a Sidecar mode. The Sidecar mode is a way to strip application functionality from the application itself as a separate process, which allows a user to add multiple functions to the application non-invasively, avoiding the addition of additional configuration code to the application to meet third party component requirements. Just as the sidecar is attached to the motorcycle, the log collection container is attached to the container group to extend the functional characteristics of the container group log processing, while the log collection container is loosely coupled to the application container.
In this embodiment, the configuration information is used to instruct the log collection container to log the application containers in the container group. For example, the configuration information may be log collection configuration information for instructing the log collection container to log the application containers in the container group. The log acquisition configuration information includes, but is not limited to, log keywords, log paths, log labels, output device information.
In some alternative implementations, the configuration information may also be log extraction configuration information used to characterize log index extraction. The log extraction configuration information includes, but is not limited to, log paths, log labels, index extraction rules, index names, index types. The index may be any index for representing the running condition of the application in the container, for example, may be a delay of a request, and the number of abnormal errors in the running process, and according to the extracted index, the running condition of each application may be further monitored and an abnormal alarm may be given.
For the extracted index, alarm monitoring can be performed through Prometheus. Prometheus is an open source monitoring alarm application developed by SoundCloud (a Germany website). The extracted index in this embodiment may be in a format supported by promethaus so that promethaus can directly use the extracted index.
The execution subject of this embodiment collects, through the log collection container, configuration information corresponding to a container group in which the log collection container is located.
Step 202, according to the configuration information, the logs in the log storage catalog are processed through the log collection container.
In this embodiment, the execution body may process the log in the log storage directory through the log collection container according to the configuration information after obtaining the configuration information.
When the configuration information is log collection configuration information for indicating the log collection container to collect the log of the application container in the container group, the execution body obtains a log file containing log keywords in the log collection configuration information under a log path in the log collection configuration information, and outputs the log file meeting the conditions to a designated output device for storage.
In some optional implementations, when the configuration information is log extraction configuration information for characterizing log index extraction, the execution body may obtain a log file conforming to an index extraction rule, an index name, or an index type under a log path in the log extraction configuration information. Furthermore, the running state of the corresponding application can be monitored through the statistical information of the log file after index extraction.
In this embodiment, the related operations in the steps 201 and 202 may be implemented by function modules, and the related operations are divided into a plurality of function modules, each function module completes a sub-function, and then the function modules are connected together to form a whole, so as to satisfy the required function of the whole log collection container. For example, a control module and a log processing module may be disposed in the log collection container, where the control module may collect configuration information corresponding to the container group, and control the log processing module to process the log in the log storage directory according to the configuration information.
Fig. 3 schematically shows an application scenario of the log processing method according to the present embodiment. The server cluster comprises a plurality of servers provided with an automatic container operation platform. The container group running in the server cluster comprises an application container and a log collection container. Each application in the server cluster runs in a corresponding container. The log collection containers in each container group collect configuration information corresponding to the container group in which the log collection container is located.
In fig. 3, a container group 302 running in a server 301 includes a log collection container 303, an application container 304, an application container 305, and an application container 306. The log collection container 306 obtains corresponding configuration information, where the configuration information is used to instruct the log collection container 303 to collect log files of the application container 304, the application container 305 and the application container 306, and the log collection container 303 performs log collection according to the stored log storage directory of the application container 304, the stored log storage directory of the application container 305 and the stored log storage directory of the application container 306, and outputs the collected log files to the designated device 307.
In the prior art, log collection of all containers in the node is carried out by deploying log collection containers on the node, when the number of container groups in the node increases, the collection pressure of the log collection containers is increased, and even the condition of missing collected logs can be caused. In the log acquisition process, dynamically discovering standard output and log files of all containers in the node through a service discovery module; and then, the acquired file is transmitted to a designated storage device, and index extraction is carried out on the log file in the storage device through log service, namely, in the prior art, subsequent processing of log acquisition is required to be carried out through binding of the log service.
In this embodiment, each container group includes an application container and a log collection container, and the log collection container processes the log of the application container in the container group, so that log service binding is not required, log processing is not required based on a storage device, resource consumption for log processing is reduced, and log processing efficiency is improved.
With continued reference to FIG. 4, there is shown a schematic flow 400 of another embodiment of a log processing method according to the present application, including the steps of:
in step 401, log attribute information characterizing the processing requirement of a user on a log is collected through a log collection container.
In this embodiment, the log attribute information is log attribute information representing a processing requirement of the user on the log, which is input by the user through the input device. For example, corresponding to configuration information is, including but not limited to: the method comprises the steps of a container to which a log belongs, a label to be added after the log is acquired, keywords for representing the content of the log and index information of the extracted log.
The execution subject can acquire log attribute information representing the processing requirements of a user on logs through the log acquisition container.
And step 402, generating configuration information according to the log attribute information.
In this embodiment, the execution body generates configuration information according to the log attribute information, so that the configuration information can be identified by the log collection container.
In some alternative implementations, the executing body mounts the generated configuration information to a designated configuration information storage directory so that the configuration information can be accurately obtained.
Step 403, collecting configuration information corresponding to the container group where the log collection container is located through the log collection container.
In this embodiment, step 403 is performed in a similar manner to step 201, and will not be described here again.
And step 404, processing the logs in the log storage catalog through the log collection container according to the configuration information.
In this embodiment, step 404 is performed in a similar manner to step 202, and will not be described here again.
As can be seen from fig. 4, compared with the embodiment corresponding to fig. 2, the flow 400 of the log processing method in this embodiment specifically illustrates that log attribute information that characterizes the processing requirement of the user on the log can be collected by the log collection container, and the indication configuration information is generated according to the log attribute information. Therefore, the execution main body acquires the log attribute information and generates the configuration information according to the log attribute information, so that the response time for the log processing requirement of the user is shortened, and the user experience is improved.
With continued reference to fig. 5, an exemplary architecture 500 of one embodiment of a container group according to the present application is shown. As shown in fig. 5, the container group 501 includes an application container 502 and a log collection container 503 in which a log storage directory of application containers in the container group is stored; a control module 5031 and a log processing module 5032 are arranged in the log acquisition container; the control module is used for collecting configuration information corresponding to the container group and controlling the log processing module to process the log in the log storage catalog according to the configuration information.
In some optional implementations, the log collection container is further provided with a configuration collection module; the configuration acquisition module is used for acquiring log attribute information representing the processing requirement of a user on the log and generating configuration information indicating the log processing module to perform log processing according to the log attribute information.
In some alternative implementations, the configuration information is extracted configuration information used to characterize log index extraction.
In some alternative implementations, the control module is further configured to determine whether the configuration information changes; in response to determining that the configuration information changes, controlling operation of the log processing module to process the log in the log storage directory information according to the change information of the configuration information. In particular, the configuration information change information entry may be change information for characterizing add configuration information, change information for characterizing delete configuration information, or change information for characterizing change configuration information.
In some alternative implementations, the log collection container is further provided with a self-monitoring module; the self-monitoring module is used for collecting the running state information and logs of each module in the log collecting container.
In this embodiment, the log collection container realizes the overall function by mutually cooperating with the plurality of functional modules, and when the configuration information changes, the configuration information can be updated only by restarting the log processing module in the log collection container, so that restarting of the application container in the whole node caused by restarting the log collection container due to the change of the configuration information is not required, and the invasiveness of the application container is reduced.
With continued reference to fig. 6, as an implementation of the method illustrated in the foregoing figures, the present disclosure provides an embodiment of a log processing apparatus, where the apparatus embodiment corresponds to the method embodiment illustrated in fig. 2, and the apparatus may be specifically applied to various electronic devices.
As shown in fig. 6, the log processing apparatus includes: an information acquisition unit 601 and a log processing unit 602.
The information collection unit 601 is configured to collect configuration information corresponding to a container group in which a log collection container is located through a log collection container, where the configuration information is used for indicating the log collection container to perform log processing on application containers in the container group, the log collection container is pre-injected into the container group, and a log storage catalog of the application containers in the container group is stored in the log collection container; the log processing unit 602 is configured to process the logs in the log storage directory through the log collection container according to the configuration information.
In some alternative implementations, the apparatus further includes: an information generating unit (not shown in the figure) configured to acquire log attribute information characterizing a processing requirement of a user for a log through the log acquisition container, and generate instruction configuration information according to the log attribute information.
In some alternative implementations, the configuration information is extracted configuration information used to characterize log index extraction.
Referring now to FIG. 7, there is illustrated a schematic diagram of a computer system 700 suitable for use in implementing the apparatus of embodiments of the present application (e.g., apparatus 101, 102, 103, 105 illustrated in FIG. 1). The apparatus shown in fig. 7 is merely an example, and should not be construed as limiting the functionality and scope of use of the embodiments herein.
As shown in fig. 7, the computer system 700 includes a processor (e.g., CPU, central processing unit) 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data required for the operation of the system 700 are also stored. The processor 701, the ROM702, and the RAM703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the method of the present application are performed when the computer program is executed by the processor 701.
It should be noted that the computer readable medium of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present application may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the client computer, partly on the client computer, as a stand-alone software package, partly on the client computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the client computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented by software, or may be implemented by hardware. The described units may also be provided in a processor, for example, described as: a processor includes an information acquisition unit and a log processing unit. The names of these units do not limit the unit itself in some cases, and for example, the information collecting unit may also be described as a unit that "collects configuration information corresponding to a container group in which the log collection container is located by the log collection container".
As another aspect, the present application also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the computer device to: the method comprises the steps that configuration information corresponding to a container group in which a log collection container is located is collected through the log collection container, the configuration information is used for indicating the log collection container to conduct log processing on application containers in the container group, the log collection container is injected into the container group in advance, and a log storage catalog of the application containers in the container group is stored in the log collection container; and processing the logs in the log storage catalog through the log acquisition container according to the configuration information.
The foregoing description is only of the preferred embodiments of the present application and is presented as a description of the principles of the technology being utilized. It will be appreciated by persons skilled in the art that the scope of the invention referred to in this application is not limited to the specific combinations of features described above, but it is intended to cover other embodiments in which any combination of features described above or equivalents thereof is possible without departing from the spirit of the invention. Such as the above-described features and technical features having similar functions (but not limited to) disclosed in the present application are replaced with each other.
Claims (10)
1. A log processing method, the method comprising:
collecting log attribute information representing the processing requirement of a user on a log through a log collecting container, and generating configuration information according to the log attribute information, wherein the log attribute information is the log attribute information representing the processing requirement of the user on the log, which is input by the user through an input device;
the method comprises the steps that configuration information corresponding to a container group where a log collection container is located is collected through the log collection container, the configuration information is used for indicating the log collection container to conduct log processing on application containers in the container group, the log collection container is injected into the container group in advance, and a log storage catalog of the application containers in the container group is stored in the log collection container;
and processing the logs in the log storage catalog through the log acquisition container according to the configuration information.
2. The log processing method according to claim 1, wherein the configuration information is extraction configuration information for characterizing log index extraction.
3. A container group comprising an application container and a log collection container, wherein the log collection container stores a log storage catalog of the application container in the container group;
the log collection container is internally provided with a control module, a log processing module and a configuration collection module;
the configuration acquisition module is used for acquiring log attribute information representing the processing requirement of a user on a log, and generating configuration information indicating the log processing module to perform log processing according to the log attribute information, wherein the log attribute information is the log attribute information representing the processing requirement of the user on the log, which is input by the user through an input device;
the control module is used for collecting configuration information corresponding to the container group and controlling the log processing module to process the log in the log storage catalog according to the configuration information.
4. A container group according to claim 3, wherein the configuration information is extraction configuration information for characterizing log index extraction.
5. A container group according to claim 3, wherein the control module is further adapted to determine whether the configuration information has changed; and in response to the determination that the configuration information changes, controlling the operation of the log processing module to process the log in the log storage directory information according to the change information of the configuration information.
6. A container set according to claim 3, wherein the log collection container is further provided with a self-monitoring module;
the self-monitoring module is used for collecting the running state information and logs of each module in the log collecting container.
7. A log processing apparatus, the apparatus comprising:
the information generation unit is configured to acquire log attribute information representing the processing requirement of a user on the log through the log acquisition container and generate configuration information according to the log attribute information, wherein the log attribute information is the log attribute information representing the processing requirement of the user on the log, which is input by the user through the input equipment;
the information acquisition unit is configured to acquire configuration information corresponding to a container group in which the log acquisition container is located through the log acquisition container, wherein the configuration information is used for indicating the log acquisition container to perform log processing on application containers in the container group, the log acquisition container is pre-injected into the container group, and a log storage catalog of the application containers in the container group is stored in the log acquisition container;
and the log processing unit is configured to process the logs in the log storage catalog through the log acquisition container according to the configuration information.
8. The log processing device according to claim 7, wherein the configuration information is extraction configuration information for characterizing log index extraction.
9. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1-2.
10. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911281126.XA CN112988528B (en) | 2019-12-13 | 2019-12-13 | Log processing method, device and container group |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911281126.XA CN112988528B (en) | 2019-12-13 | 2019-12-13 | Log processing method, device and container group |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112988528A CN112988528A (en) | 2021-06-18 |
| CN112988528B true CN112988528B (en) | 2024-04-05 |
Family
ID=76332565
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911281126.XA Active CN112988528B (en) | 2019-12-13 | 2019-12-13 | Log processing method, device and container group |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988528B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109660397A (en) * | 2018-12-21 | 2019-04-19 | 北京百度网讯科技有限公司 | For acquiring system, the method and apparatus of log |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7739230B2 (en) * | 2007-08-09 | 2010-06-15 | International Business Machines Corporation | Log location discovery and management |
-
2019
- 2019-12-13 CN CN201911281126.XA patent/CN112988528B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109660397A (en) * | 2018-12-21 | 2019-04-19 | 北京百度网讯科技有限公司 | For acquiring system, the method and apparatus of log |
Non-Patent Citations (2)
| Title |
|---|
| Design of Whole Chain Temperature Monitoring System for Raw Milk;Wenli Zhang;IEEE;全文 * |
| 基于Filebeat自动收集Kubernetes日志的分析系统;翟雅荣;于金刚;;计算机系统应用(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112988528A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10185626B2 (en) | Automatic application error detection and rollback | |
| US20110307523A1 (en) | Configuring cloud resources | |
| US20180285571A1 (en) | Automatic detection of an incomplete static analysis security assessment | |
| CN109359194B (en) | Method and apparatus for predicting information categories | |
| US20150195143A1 (en) | Discovering resources of a distributed computing environment | |
| CN113010405B (en) | Application program testing method and device | |
| US20160259795A1 (en) | Distributed version control of orchestration templates | |
| CN111563257A (en) | Data detection method and device, computer readable medium and terminal equipment | |
| CN107368407B (en) | Information processing method and device | |
| CN112084114B (en) | Method and apparatus for testing interfaces | |
| CN111444148B (en) | Data transmission method and device based on MapReduce | |
| CN112988528B (en) | Log processing method, device and container group | |
| CN110858240A (en) | Front-end module loading method and device | |
| CN110580216A (en) | application extraction and detection method and device | |
| US10796264B2 (en) | Risk assessment in online collaborative environments | |
| CN111241368B (en) | Data processing method, device, medium and equipment | |
| CN113569256A (en) | Vulnerability scanning method and device, vulnerability scanning system, electronic equipment and computer readable medium | |
| US20220122038A1 (en) | Process Version Control for Business Process Management | |
| CN109857838B (en) | Method and apparatus for generating information | |
| CN113407339A (en) | Resource request feedback method and device, readable storage medium and electronic equipment | |
| CN112230891A (en) | Interface document integration method and device, server and computer storage medium | |
| CN113312053A (en) | Data processing method and device | |
| CN112835609A (en) | Method and device for modifying dependent package download address | |
| CN112860538A (en) | Method and device for performing interface regression test based on online log | |
| CN116382597B (en) | Vehicle operation data storage method, apparatus and computer medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |