CN110019045A - Method and device is landed in log - Google Patents

Abstract

The present invention provides a log landing method and device. The method includes: receiving a log access request of a service, the log access request carrying a log category to be accessed and at least one target landing component; parsing the log access request , according to the log category, it is determined that the log of the business access belongs to the consumed log; according to the type of the target landing component, the configuration information of the target landing component is added to the configuration file of the log, and the log is written In the target landing component, log landing is completed. Based on the log landing method provided by the present invention, when the consumed logs are landed on a new landing component, the log landing can be realized only by adding the configuration information of the target landing component that needs to be landed in the configuration file of the log. It can quickly and effectively implement the log into the components for the business, thereby improving the efficiency of business access.

Description

Log landing method and device

technical field

The present invention relates to the field of computer technology, and in particular, to a log landing method and device.

Background technique

With the continuous development of computer technology, the amount of storage and calculation of various data is also increasing. Facing the increasingly huge amount of data, when the business needs data access, how to effectively and quickly write the required log data for each business is an urgent problem to be solved.

In the traditional technology, when a database cluster receives a data access request, it needs to land the data log in its respective target system. If different target systems have the same log requirements, the database cluster needs to implement the log for each target system. Relevant code, and will be developed repeatedly, and the logs will be repeatedly consumed. Using the above method will not only waste resources, but also reduce data access efficiency and slow down the running speed of the system.

SUMMARY OF THE INVENTION

The present invention provides a log landing method and device to overcome the above problems or at least partially solve the above problems.

According to an aspect of the present invention, a log landing method is provided, comprising:

Receive a log access request from the business, where the log access request carries the log category to be accessed and at least one target landing component;

Parse the log access request, and determine, according to the log category, that the service access log belongs to a consumed log;

The configuration information of the target landing component is added to the configuration file of the log according to the type of the target landing component, and the log is written into the target landing component to complete the log landing.

Optionally, the parsing the log access request, and determining that the service access log belongs to the consumed log according to the log category, including:

Parse the log access request, obtain the log category carried in the log access request, and determine whether the log category exists in the preset day category list;

If so, it is determined that the log of the service access belongs to the consumed log.

Optionally, after parsing the log access request, obtaining the log category, and judging whether the log category exists in the preset daily category list, the method further includes:

If not, it is determined that the log of the service access belongs to the unconsumed log;

Add the log category to the preset log category list, and consume logs from the specified cluster.

Optionally, adding the log category to the preset log category list and consuming logs from a specified cluster includes:

adding field information corresponding to the log category in the preset log category list;

Collect log information corresponding to the log category from the specified cluster.

Optionally, adding the configuration information of the target landing component to the configuration file of the log according to the type of the target landing component, writing the log into the target landing component, and completing the log landing, including: :

adding the field information of the target landing component to the landing component command specified in the configuration file of the log;

Determine the basic configuration information required by the target landing component to implement log landing according to the type of the target landing component;

The basic configuration information of the target landing component is added to the configuration file to provide an access interface of the target landing component, and the log is written into the target landing component to complete the log landing.

Optionally, after adding the configuration information of the target landing component in the configuration file according to the type of the target landing component, the method further includes:

Restart the application corresponding to the log category.

Optionally, the log access request also carries a landing path for writing the log into the target landing component;

The adding the configuration information of the target landing component in the configuration file of the log according to the type of the target landing component, writing the log into the target landing component, and after completing the log landing, further includes:

A message of successful log landing is returned to the service, and the service reads the log based on the landing path.

Optionally, the method is applied to a Kafka system.

According to another aspect of the present invention, there is also provided a log landing device, comprising:

a request receiving module, configured to receive a service log access request, where the log access request carries the log category to be accessed and at least one target landing component;

A parsing module, configured to parse the log access request, and determine according to the log category that the business access log belongs to a consumed log;

The landing module is configured to add configuration information of the target landing component in the configuration file of the log according to the type of the target landing component, write the log into the target landing component, and complete the log landing.

Optionally, the parsing module includes:

a judging unit, configured to parse the log access request, obtain the log category carried in the log access request, and determine whether the log category exists in the preset day category list;

The first determining unit is configured to determine that the log of the service access belongs to the consumed log when the log category exists in the preset day category list.

Optionally, the parsing module further includes:

The second determination unit is configured to be configured to, when the log category does not exist in the preset daily category list, determine that the log of the service access belongs to an unconsumed log; and add the log category to the preset log category list and consume logs from the specified cluster.

Optionally, the second determining unit is further configured to:

Field information corresponding to the log category is added to the preset log category list; log information corresponding to the log category is collected from a specified cluster.

Optionally, the landing module is further configured as:

adding the field information of the target landing component to the landing component command specified in the configuration file of the log;

Determine the basic configuration information required by the target landing component to implement log landing according to the type of the target landing component;

The basic configuration information of the target landing component is added to the configuration file to provide an access interface of the target landing component, and the log is written into the target landing component to complete the log landing.

Optionally, the above apparatus further includes: a restart module configured to restart the application corresponding to the log category.

Optionally, the log access request also carries a landing path for writing the log into the target landing component;

The device further includes: an information sending module configured to return a log landing success message to the service, and the service reads the log based on the landing path.

According to another aspect of the present invention, an electronic device is also provided, comprising:

processor; and

A memory arranged to store computer executable instructions which, when executed, cause the processor to perform a log landing method according to any of the above.

According to another aspect of the present invention, there is also provided a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, when the one or more programs are included as a plurality of application programs When the electronic device executes, the electronic device is caused to execute the log landing method according to any one of the above.

The present invention provides a more efficient log landing method and device. In the log landing method provided by the present invention, after receiving the log access request of the business, the log category and the log required to be accessed by the business are obtained by parsing the log access request. When it is determined that the log corresponding to the log category belongs to the consumed log, you can directly add the configuration information of the target landing component to the configuration file of the log to write the log into the target landing component to complete the log. landing. Based on the log landing method provided by the present invention, when landing the consumed log to a new landing component, the log landing can be realized by adding the configuration information of the target landing component that needs to be landing in the log configuration file, which can avoid the need for the same It can quickly and effectively implement the log into the components for the business, thereby improving the efficiency of business access.

The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.

The above and other objects, advantages and features of the present invention will be more apparent to those skilled in the art from the following detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered limiting of the invention. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

1 is a schematic flowchart of a log landing method according to an embodiment of the present invention;

2 is a schematic structural diagram of a Hamal unified landing frame according to an embodiment of the present invention;

3 is a schematic diagram of a data flow based on the Hamal unified landing framework according to an embodiment of the present invention;

4 is a schematic structural diagram of a log landing device according to an embodiment of the present invention;

5 is a schematic structural diagram of a log landing device according to a preferred embodiment of the present invention;

6 is a schematic block diagram of a computing device for executing the log landing method according to the present invention according to an embodiment of the present invention; and

7 is a schematic diagram of a storage unit for holding or carrying a program code for implementing the log landing method according to the present invention according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.

FIG. 1 is a schematic flowchart of a log landing method according to an embodiment of the present invention. As shown in FIG. 1 , the log landing method according to an embodiment of the present invention includes:

Step S102, receiving the log access request of the business, and the log access request carries the log category to be accessed and at least one target landing component;

Step S104, parse the above log access request, and determine according to the log category that the log of business access belongs to the consumed log;

Step S106, according to the type of the target landing component, add the configuration information of the target landing component to the configuration file of the log, write the log into the target landing component, and complete the log landing.

The embodiment of the present invention provides a more efficient log landing method. After receiving a log access request from a business, the log type required to be accessed by the business and the target landing component of the log are obtained by parsing the log access request. When the log corresponding to the category belongs to the consumed log, you can directly add the configuration information of the target landing component to the configuration file of the log to write the log into the target landing component to complete the log landing. Based on the log landing method provided by the embodiment of the present invention, when the consumed log is landed on a new landing component, it is only necessary to add the configuration information of the target landing component that needs to be landed in the log configuration file to realize the landing of the log, which can be avoided. The repeated consumption of the same log can quickly and effectively implement the log into the component for the business, thereby improving the efficiency of business access.

Consume logs, that is, pull data from the data storage system. The embodiments of the present invention are preferably applied to the Kafka system. Kafka is a distributed publish/subscribe based messaging system. In the embodiment of the present invention, Kafka is actually a bridge-like storage, the data in each database will enter Kafka, and Kafka only provides services for each business by reading these data. When each business needs to read data, it will pull data from Kafka. For example, when different businesses need to drop the same log into multiple landing components such as HDFS (Hadoop Distributed File System), Redis, HBase, etc., the same log may be consumed many times, resulting in a lot of repeated consumption and waste. resource.

In the log landing method provided by the embodiment of the present invention, when a request that a business needs to consume a certain log is received, the log required by the business will be dropped into a new landing component. At this time, you can first determine whether the log has been consumed, and if so, you can directly add the configuration information of the new landing component to the configuration file of the log. Taking a log as an example, the original business only needs HDFS. Now, if the business wants to have the same log in Redis, it only needs to add Redis-related configuration to the configuration file of the log.

Optionally, the above step S104 may further determine that the log belongs to the consumed log in the following manner: parsing the log access request, obtaining the log category carried in the log access request, and judging whether the log category exists in the preset day category list; , then it is determined that the logs accessed by the service belong to the consumed logs. If not, it is determined that the logs accessed by the business belong to unconsumed logs; the log category is added to the preset log category list, and logs are consumed from the specified cluster.

For the Kafka cluster, each message published to the Kafka cluster has a category, which is called topic, and each consumer must specify its topic. In this embodiment of the present invention, when a service data access request is received, the request may specify a topic consumed from Kafka. Topics or topic lists (category lists) can be pre-stored in the cluster. After receiving a request for a new service, it parses it, obtains the topic that needs to be consumed in the request, and determines whether the topic exists in the preset topic list. , if it exists, it can be directly determined that the topic has been consumed. If the topic is not in the preset topic list, add the topic to the preset topic list.

Preferably, adding the log category to the preset log category list and consuming logs from the specified cluster may include: adding field information corresponding to the log category in the preset log category list; collecting the log category corresponding to the log category from the specified cluster log information. For example, the topic field carried in the request can be added to the existing topic list. Then consume according to the protocol of parsing the topic, that is, realize the collection of logs. That is to say, the topic field in the topic list can be arbitrarily increased or decreased according to the needs of the business.

As mentioned in the above step S106, the configuration information of the target landing component can be added to the configuration file of the log according to the type of the target landing component, and the log is written into the target landing component to complete the log landing. Among them, adding the configuration information of the landing component to the configuration file of the log is actually the implementation of the interface. In this embodiment of the present invention, the access interface for the consumed logs may be a loadable interface. Since it will include multiple landing components such as Redis, HBase, HDFS, etc., when the business needs to land logs on one or more of these components, such as Redis, it only needs to implement a Redis interface to meet the business needs. In this embodiment, the relevant configuration information of Redis is added to the configuration file of the log, that is, the log implements the implementation of the interface available for Redis access, which is equivalent to the log required by the business will be written to Redis. , and then complete the landing of the log.

In practical applications, the configuration information of each landing component is different. For example, if the business needs to land logs to HDFS, the configuration information that needs to be added can include: the root directory of the landing, the compression method and the landing format. That is to say, each landing component will have its own configuration information, and then when the log landing needs to be performed, it can be achieved by providing the relevant information.

In addition, if the access request submitted by the business applies for the landing of logs to HDFS and Redis at the same time, the logs will be double-written to these two components, and when the logs are written to different landing components, you can use the landing components according to the landing components. The respective data storage formats convert the log format to meet the data format requirements of each landing component, and provide access services for businesses quickly and efficiently.

Optionally, after adding the configuration information of the landing component to the configuration file of the log, the application corresponding to the log category, that is, the Application, can be restarted. Application is equivalent to a business level. Each log actually belongs to a business. When a business needs to add landing components, it is only necessary to restart the Application corresponding to the log to make the new configuration information take effect.

Further, the log access request also carries the landing path for writing the log into the target landing component. After the configuration file of the log is modified, a message that the log landing is successful can be returned to the business, and the business can read the log based on the landing path. The message of successful log landing returned to the business may be an email sent to the business or other forms of messages, indicating that the specified log will be placed in the specified landing component according to the request of the business, which is not limited in the present invention.

As mentioned above, the log of the business request will be written to each landing component to realize the landing of the log. Preferably, each landing component in this embodiment can be implemented and managed by Flink. In addition to the above landing components, Flink can also implement a variety of source and sink components. The sink component is the target landing component mentioned above, such as HDFS, Redis, HBase, Hive, ELK, and so on.

Based on the log landing method provided by the above embodiments, the embodiments of the present invention can provide a landing architecture Hamal jointly implemented by Kafka and Flink, and based on the framework, data can be monitored and managed uniformly. As shown in FIG. 2 , the unified landing architecture Hamal based on Kafka and Flink provided by the embodiment of the present invention. The framework processes the data stream by Kafka, and saves the progress of consumer consumption through Kafka offset.

Offset is a record from which point to obtain log information. Although it is obtained from Kafka, it may not be obtained in real time. It is also possible to locate the data two days ago or a few days ago through offset. After the log two days ago or a few days ago, it will be landed on the sink component according to the needs of the business. When the log of the located time period has landed, the subsequent data can be landed in real time.

In the unified landing architecture provided by this embodiment, the offset mainly includes two threads: CheckPointer (checkpoint) and burrow (Lag Monitor).

CheckPointer function, once the user changes the data, the buffer becomes dirty. The user's commit does not mean that the changes have been written to the data file, that is, the WAL log (write-ahead log, write-ahead log) file, but the checkpoint process is required to complete this work. When a checkpoint operation occurs, all dirty pages are written to the table and index files; at this time a special checkpoint record is written to the WAL log file. When a crash occurs, the crash recovery program looks at the latest checkpoint record and decides where to start the REDO (redo log file) operation through a special checkpoint record in the WAL log file. Therefore, after a checkpoint is performed, we no longer need log segments prior to that point.

Burrow is an appropriate way to monitor the thorny problem of Kafka consumer health, providing users with a comprehensive overview of the consumer's state and providing a service for lazily verifying consumers without specifying thresholds. Burrow will monitor the deviation value of the data submitted by all consumers according to the topic partition, and calculate the status of the corresponding consumers according to the demand.

In the Hamal unified landing framework provided by the embodiment of the present invention, Kafka can be combined with the Hamal Framework (Hamal architecture) and Flink on Yarn Runtime (Flink on Yarn runtime), and based on multiple sink components implemented by Flink, the sink components may include : HDFS (Bucketing Sink), HBase (HBase Sink), Graylog (Graylog Sink), Redis, cassandra, etc. The implementation of the above components is like a plug-in. Once implemented, it can be used in any combination for the business. Unified development and management of each landing component is carried out. When a log access request from a business is received, the log type of the accessed log is carried in the request. And the landing component of the log, and then configure it. The data flow based on the above framework can be shown in Figure 3.

Kafka Consumer, a message consumer, is a client that reads messages from a Kafka broker. After the consumer pulls log data from the broker, it performs deserialization processing. The processed data is formatted according to the component type. Write each component to complete the log landing.

The above embodiment will be described in detail below through a preferred embodiment.

1. Receive a log access request from the business, and the request carries the landing of a specific log to hdfs, kafka and the corresponding landing path;

2. Modify the configuration file of the log, add hdfs and kafka fields to the landing component command of the log, and add the basic configuration information of hdfs and kafka;

3. Restart the application corresponding to the log; at the same time, the log can be written to hdfs and kafka according to the landing path carried in the log access request.

4. Feedback the successful landing information to the business, and the business can directly read the log according to the landing path in the log access request.

The process of the above preferred embodiment can be implemented by the following code:

//Specify which Kafka cluster to consume from

kafka.clusterId=shbt2

kafka.retries=5

kafka.auto.offset.reset=earliest

//The following are the topics that need to be consumed from kafka, which can be increased or decreased according to the business

hamal.source.kafka.topics=d.x.6.m,d.x.6,d.u.6.m,d.u.6,d.s.6.m,d.s.6,d.c.6.m,d.c.6

//Define the protocol of how to parse the above topic

hamal.source.kafka.message.deserializer.class=com.mvad.hamal.lib.source.io.RawlogDeserializer

//Whether to do some special processing in the process of data processing, such as filtering, sampling and other operations

hamal.processor.class=com.mvad.hamal.lib.processor.DSPProcessor

//Specify the two components that land on hdfs and kafka (kafka itself is also a component that can be landed)

hamal.sink.target=hdfs,kafka//You can also add a redis

//Configuration of kafka sink component

hamal.sink.kafka.clusterId=shbt2

hamal.sink.kafka.message.serializer.class=com.mvad.hamal.lib.source.io.D SPSampleKafkaMessageSerializer

//Configuration information of hdfs sink component

hamal.sink.hdfs.basedir=/mvad/rawlog/dsp

hamal.sink.hdfs.format=LzoThriftBlockWithMeta

hamal.sink.hdfs.bucketing.filename.prefix=dsp

hamal.sink.hdfs.bucketing.filename.suffix=.dsp.lzo

It can be seen from the above embodiment that if several components are configured in the configuration file of the log, the configuration information of each component needs to be correspondingly added, and the business can be flexibly configured by increasing or decreasing consumption topics or increasing or decreasing components.

Based on the same inventive concept, an embodiment of the present invention also provides a log landing device. As shown in FIG. 4 , a log landing device according to an embodiment of the present invention may include:

The request receiving module 410 is configured to receive the log access request of the business, and the log access request carries the log category to be accessed and at least one target landing component;

The parsing module 420 is configured to parse the log access request, and determine that the log accessed by the business belongs to the consumed log according to the log category;

The landing module 430 is configured to add the configuration information of the above-mentioned target landing component to the configuration file of the log according to the type of the target landing component, and write the log into the target landing component to complete the log landing.

In a preferred embodiment of the present invention, as shown in FIG. 5 , the parsing module 420 may include:

The judging unit 421 is configured to parse the log access request, obtain the log category carried in the log access request, and determine whether the log category exists in the preset daily category list;

The first determining unit 422 is configured to, when the log category exists in the preset daily category list, determine that the log accessed by the service belongs to the consumed log.

In a preferred embodiment of the present invention, as shown in FIG. 5 , the parsing module 420 may further include:

The second determining unit 423 is configured to be configured to, when the log category does not exist in the preset daily category list, determine that the log accessed by the business belongs to an unconsumed log; add the log category to the preset log category list, and select the log category from the list of preset log categories. Specifies the consumption log in the cluster.

In a preferred embodiment of the present invention, the second determining unit 423 may be further configured to: add field information corresponding to the log category in the preset log category list; collect log information corresponding to the log category from a specified cluster.

In a preferred embodiment of the present invention, the landing module 430 can also be configured as:

Add the field information of the target landing component to the landing component command specified in the log configuration file; determine the basic configuration information required by the target landing component to implement the log landing according to the type of the target landing component; add the target landing component to the configuration file. Basic configuration information to provide the access interface of the target landing component, write the log to the target landing component, and complete the log landing.

In a preferred embodiment of the present invention, as shown in FIG. 5 , the above-mentioned device may further include:

The restart module 440 is configured to restart the application corresponding to the log category.

In a preferred embodiment of the present invention, the log access request also carries a landing path for writing the log into the target landing component;

As shown in Figure 5, the above-mentioned device may also include:

The information sending module 450 is configured to return a message of successful log landing to the service, and the service reads the log based on the landing path.

Embodiments of the present invention provide a log landing method and device. After receiving a log access request from a business, the log type required by the business and the target landing component of the log are obtained by parsing the log access request. When the log corresponding to the category belongs to the consumed log, you can directly add the configuration information of the target landing component to the configuration file of the log to write the log into the target landing component to complete the log landing. Based on the log landing method provided by the embodiment of the present invention, when the consumed log is landed on a new landing component, it is only necessary to add the configuration information of the target landing component that needs to be landed in the log configuration file to realize the landing of the log, which can be avoided. The repeated consumption of the same log can quickly and effectively implement the log into the component for the business, thereby improving the efficiency of business access. In addition, each landing component in the embodiment of the present invention can be configured centrally. When a new business has different landing requests for the same log, it can efficiently and quickly provide access services for the business. After the configuration information is reached, the configuration will go online, which greatly improves the access efficiency.

Further, the consumption of one log in the embodiment of the present invention can be implemented in multiple systems, repeated consumption is avoided, resources are greatly saved, the degree of scalability and error tolerance of the system can be effectively improved, and high availability is also provided. High scalability.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or its description. This disclosure, however, should not be construed as reflecting an intention that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and further they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination, unless at least some of such features and/or procedures or elements are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include certain features, but not others, included in other embodiments, that combinations of features of different embodiments are intended to be within the scope of the invention within and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components of the log landing device according to the embodiment of the present invention. The present invention can also be implemented as apparatus or apparatus programs (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

An embodiment of the present invention also provides an electronic device, comprising: a processor; and a memory arranged to store computer-executable instructions, the executable instructions, when executed, cause the processor to execute the log landing according to any of the above method.

Embodiments of the present invention provide a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs, when executed by an electronic device including multiple application programs, cause the electronic device to execute The log landing method according to any one of the above.

For example, Figure 6 shows a computing device that can implement the log landing method. The computing device traditionally includes a computer program product or computer readable medium in the form of a processor 610 and a memory 620 . The memory 620 may be electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. The memory 620 has storage space 630 for storing program code 631 for performing any of the method steps in the above-described methods. For example, the storage space 630 for storing program codes may include respective program codes 631 for implementing various steps in the above methods, respectively. These program codes can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such a computer program product is typically a portable or fixed storage unit as shown in FIG. 7 . The storage unit may have storage segments, storage spaces, etc. arranged similarly to memory 620 in the computing device of FIG. 6 . The program code may, for example, be compressed in a suitable form. Typically, the storage unit includes computer readable code 631' for performing the method steps of the present invention, ie code readable by a processor such as 610, which when executed by a computing device, cause the computing device to execute the various steps in the method described above.

It should be noted that the above-described embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names.

By now, those skilled in the art will recognize that, although various exemplary embodiments of the present invention have been illustrated and described in detail herein, the present invention may still be implemented in accordance with the present disclosure without departing from the spirit and scope of the present invention. The content directly determines or derives many other variations or modifications consistent with the principles of the invention. Accordingly, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

According to one aspect of the present invention, A1. a log landing method is provided, comprising:

Receive a log access request from the business, where the log access request carries the log category to be accessed and at least one target landing component;

Parse the log access request, and determine, according to the log category, that the service access log belongs to a consumed log;

The configuration information of the target landing component is added to the configuration file of the log according to the type of the target landing component, and the log is written into the target landing component to complete the log landing.

A2. The method according to A1, wherein, by analyzing the log access request, determining that the service access log belongs to a consumed log according to the log category, including:

Parse the log access request, obtain the log category carried in the log access request, and determine whether the log category exists in the preset day category list;

If so, it is determined that the log of the service access belongs to the consumed log.

A3. The method according to A2, wherein after parsing the log access request, obtaining the log category, and judging whether the log category exists in the preset daily category list, the method further includes:

If not, it is determined that the log of the service access belongs to the unconsumed log;

Add the log category to the preset log category list, and consume logs from the specified cluster.

A4. The method according to A3, wherein the adding the log category to the preset log category list and consuming logs from a specified cluster includes:

adding field information corresponding to the log category in the preset log category list;

Collect log information corresponding to the log category from the specified cluster.

A5. The method according to A1, wherein the configuration information of the target landing component is added to the configuration file of the log according to the type of the target landing component, and the log is written to the target landing component , complete the log landing, including:

adding the field information of the target landing component to the landing component command specified in the configuration file of the log;

Determine the basic configuration information required by the target landing component to implement log landing according to the type of the target landing component;

The basic configuration information of the target landing component is added to the configuration file to provide an access interface of the target landing component, and the log is written into the target landing component to complete the log landing.

A6. The method according to A1, wherein after adding the configuration information of the target landing component in the configuration file according to the type of the target landing component, the method further includes:

Restart the application corresponding to the log category.

A7. The method according to any one of A1-A6, wherein the log access request also carries a landing path for writing the log into the target landing component;

The adding the configuration information of the target landing component in the configuration file of the log according to the type of the target landing component, writing the log into the target landing component, and after completing the log landing, further includes:

A message of successful log landing is returned to the service, and the service reads the log based on the landing path.

A8. The method according to any one of A1-A6, wherein the method is applied to a Kafka system.

According to another aspect of the present invention, B9. a log landing device is also provided, comprising:

a request receiving module, configured to receive a service log access request, where the log access request carries the log category to be accessed and at least one target landing component;

A parsing module, configured to parse the log access request, and determine according to the log category that the business access log belongs to a consumed log;

The landing module is configured to add configuration information of the target landing component in the configuration file of the log according to the type of the target landing component, write the log into the target landing component, and complete the log landing.

B10. The apparatus according to B9, wherein the parsing module comprises:

a judging unit, configured to parse the log access request, obtain the log category carried in the log access request, and determine whether the log category exists in the preset day category list;

The first determining unit is configured to determine that the log of the service access belongs to the consumed log when the log category exists in the preset day category list.

B11. The apparatus according to B10, wherein the parsing module further comprises:

The second determination unit is configured to be configured to, when the log category does not exist in the preset daily category list, determine that the log of the service access belongs to an unconsumed log; and add the log category to the preset log category list and consume logs from the specified cluster.

B12. The apparatus according to B11, wherein the second determining unit is further configured to:

Field information corresponding to the log category is added to the preset log category list; log information corresponding to the log category is collected from a specified cluster.

B13. The device according to B9, wherein the floor module is further configured to:

adding the field information of the target landing component to the landing component command specified in the configuration file of the log;

Determine the basic configuration information required by the target landing component to implement log landing according to the type of the target landing component;

The basic configuration information of the target landing component is added to the configuration file to provide an access interface of the target landing component, and the log is written into the target landing component to complete the log landing.

B14. The apparatus according to B9, further comprising: a restart module configured to restart the application corresponding to the log category.

B15. The device according to any one of B9-B14, wherein the log access request further carries a landing path for writing the log into the target landing component;

The device further includes: an information sending module configured to return a log landing success message to the service, and the service reads the log based on the landing path.

According to another aspect of the present invention, C16. an electronic device is also provided, comprising:

processor; and

A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the log landing method according to any of A1-A8.

According to another aspect of the present invention, D17. A computer-readable storage medium is also provided, wherein the computer-readable storage medium stores one or more programs, when the one or more programs are included with multiple applications When the electronic device of the program is executed, the electronic device is made to execute the log landing method according to any one of A1-A8.

Claims (10)

1. method is landed in a kind of log, comprising:

The log access of reception business is requested, the log category that accesses needed for carrying in the log access request and at least One target lands component；

The log access request is parsed, determines that the log of the business access belongs to according to the log category and has consumed day Will；

Matching for the target landing component is added in the configuration file of the log according to the type that the target lands component The log is written in the target landing component, completes log landing by confidence breath.

2. the method according to claim 1, wherein the parsing log access request is determined according to the log category The log of the business access, which belongs to, has consumed log, comprising:

The log access request is parsed, the log category carried in the log access request is obtained, judges the log category With the presence or absence of in default day list of categories；

If so, determining that the log of the business access belongs to has consumed log.

3. the log category is obtained according to the method described in claim 2, wherein, the parsing log access is requested, After judging that the log category whether there is in default day list of categories, further includes:

If not, it is determined that the log of the business access, which belongs to, does not consume log；

The log category is added in the default log category list, and consumes log from cluster-specific.

It is described the log category is added to the default log category to arrange 4. according to the method described in claim 3, wherein In table, and log is consumed from cluster-specific, comprising:

The corresponding field information of the log category is added in the default log category list；

Log information corresponding with the log category is collected from cluster-specific.

5. according to the method described in claim 1, wherein, the type according to target landing component is in the log The log is written in the target landing component configuration information that the target landing component is added in configuration file, complete It is landed at log, comprising:

The field information of the target landing component is added in the landing component order specified in the configuration file of the log；

Determine that target landing component realizes the required basic configuration letter of log landing according to the type that the target lands component Breath；

The basic configuration information of the target landing component is added, in the configuration file to provide the target landing component Access interface, the log is written in target landing component, log landing is completed.

6. according to the method described in claim 1, wherein, the type for landing component according to the target is literary in the configuration After the configuration information for adding the target landing component in part, further includes:

Restart application corresponding with the log category.

7. method according to claim 1-6, wherein also carry in the log access request by the day The landing path of the target landing component is written in will；

The type for landing component according to the target adds the target in the configuration file of the log and lands component Configuration information, the log is written in target landing component, after completing log landing, further includes:

Log is returned to the business and lands successful message, and the landing path is based on by the business and reads the log.

8. a kind of log apparatus for descending, comprising:

Request receiving module is configured to the log access request of reception business, carries required visit in the log access request The log category asked and at least one target land component；

Parsing module is configured to parse the log access request, the day of the business access is determined according to the log category Will, which belongs to, has consumed log；

Module is landed, the type for being configured to land component according to the target adds the mesh in the configuration file of the log Village component configuration information, the log is written in target landing component, log landing is completed.

9. a kind of electronic equipment, comprising:

Processor；And

It is arranged to the memory of storage computer executable instructions, the executable instruction makes the processor when executed It executes log described in any one of -7 according to claim 1 and lands method.

10. a kind of computer readable storage medium, wherein the computer-readable recording medium storage one or more program, One or more of programs are when the electronic equipment for being included multiple application programs executes, so that the electronic equipment executes root Method is landed according to log of any of claims 1-7.