CN115756754A - Containerized operation and maintenance method and system for stateful service - Google Patents
Containerized operation and maintenance method and system for stateful service Download PDFInfo
- Publication number
- CN115756754A CN115756754A CN202211490913.7A CN202211490913A CN115756754A CN 115756754 A CN115756754 A CN 115756754A CN 202211490913 A CN202211490913 A CN 202211490913A CN 115756754 A CN115756754 A CN 115756754A
- Authority
- CN
- China
- Prior art keywords
- service
- stateful
- configuration
- etcd
- initialization script
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Stored Programmes (AREA)
Abstract
The invention provides a containerization operation and maintenance method and system for stateful services, and relates to the technical field of containerization application. First, the data in the etcd is configured, and the dynamic configuration of the stateful service is written into the etcd. And then, constructing a service mirror image in a K8S format through dockerfile, reading a configuration item from the etcd storage system and adding the configuration item into the initialization script, so that the container under the management of the K8S platform can automatically generate corresponding configuration when being started each time. Thereafter, a service is started using the K8S platform to execute the initialization script, dynamically generating a configuration file. And finally, managing the running state of the service through the hypervisor based on the configuration file, and realizing the operation and maintenance management of the stateful service. The method and the device can efficiently integrate the stateful service and the K8S, realize the automatic process of creating, modifying and deleting the stateful service based on the Etcd, and improve the reliability and robustness of the middleware service.
Description
Technical Field
The invention relates to the technical field of containerization application, in particular to a containerization operation and maintenance method and system for stateful service.
Background
Containerization is a software development method that packages an application or service, its dependencies, and its configuration (abstracted into a deployment manifest file) together into a container image. Currently, kubernetes (K8S, an open source for managing containerization applications on multiple hosts in a cloud platform) has been the main direction of next generation operation and maintenance technology as the industry' S main container arrangement tool. Etcd (open source distributed key value storage) is often used to store and manage key information of a distributed system, and is also a tool used by K8S to manage configuration data, state data, and metadata. However, most of the existing deployed services based on the K8S are stateless, stateful services cannot be efficiently integrated with the K8S, the flows of automatic fault recovery, dynamic capacity expansion and the like of the services cannot be realized, and the service operation and maintenance cost is high.
Disclosure of Invention
The invention aims to provide a containerization operation and maintenance method and system for stateful service, which can efficiently integrate the stateful service and K8S, realize an automatic process of creating, modifying and deleting the stateful service based on Etcd, and simultaneously improve the reliability and robustness of middleware service.
The embodiment of the invention is realized by the following steps:
in a first aspect, an embodiment of the present application provides a containerization operation and maintenance method for stateful services, which includes:
writing the dynamic configuration of the stateful service into an etcd storage system, and storing the service according to a specified format;
constructing a service mirror image in a K8S format through dockerfile, specifying an initialization script, reading a configuration item from an etcd storage system and adding the configuration item into the initialization script;
starting a service by using a K8S platform to execute an initialization script and dynamically generating a configuration file;
and managing the running state of the service through the hypervisor based on the configuration file.
Based on the first aspect, in some embodiments of the present invention, the format defined above is: /{ service type }/{ service name }/{ service configuration item name }.
Based on the first aspect, in some embodiments of the present invention, the method further includes:
creating a new service through a K8S platform and creating a stateful object;
and starting a reuse instance function, configuring the resource quota of the service, and writing the name and the corresponding etcd address of the service into an environment variable of a stateful object.
Based on the first aspect, in some embodiments of the present invention, the method further includes: and mounting the data directory of the service on the directory of the host by configuring the affinity.
In a second aspect, an embodiment of the present application provides a containerization operation and maintenance system with stateful service, including:
the etcd configuration module is used for writing the dynamic configuration of the stateful service into the etcd storage system and storing the service according to a specified format;
the mirror image construction module is used for constructing a service mirror image in a K8S format through dockerfile, appointing an initialization script, reading a configuration item from the etcd storage system and adding the configuration item into the initialization script;
the script execution module is used for starting service by using the K8S platform to execute the initialization script and dynamically generating a configuration file;
and the hypervisor management module is used for managing the running state of the service through the hypervisor based on the configuration file.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory for storing one or more programs; a processor. The one or more programs, when executed by the processor, implement the method as described in any of the first aspects above.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method as described in any one of the above first aspects.
Compared with the prior art, the embodiment of the invention has at least the following advantages or beneficial effects:
the embodiment of the application provides a containerization operation and maintenance method and system for stateful services. And then, constructing a service mirror image in a K8S format through dockerfile, designating an initialization script, reading a configuration item from the etcd storage system and adding the configuration item into the initialization script, so that a container under the management of the K8S platform can automatically generate corresponding configuration when being started each time. And then, a new service is created through the operation of the K8S platform, a stateful object of the K8S is used, the container is ensured to be bound on a specified host machine through configuration affinity, a data directory of the service is mounted on the directory of the host machine, and the data is ensured not to be lost after the container is destroyed. And after the K8S platform starts the service, the container dynamically generates a configuration file according to the initialization script. And finally, based on the configuration file, managing the running state of the service in the container through the hypervisor, and realizing the operation and maintenance management of the stateful service. On the whole, by introducing the etcd storage and the super management, the service has the high availability characteristic of K8S, and the operation and maintenance requirements of the stateful service are met. After deployment, the effects of no manual intervention and self-healing of the service can be achieved, the operation and maintenance efficiency is improved, and the usability of the service is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
FIG. 1 is a block diagram illustrating the steps of a containerization operation and maintenance method for stateful services according to an embodiment of the present invention;
fig. 2 is a schematic flowchart illustrating an embodiment of a containerization operation and maintenance method for stateful services according to the present invention;
FIG. 3 is a block diagram of a containerization operation and maintenance system with stateful services according to the present invention;
fig. 4 is a block diagram of an electronic device according to an embodiment of the present invention.
An icon: 1. a memory; 2. a processor; 3. a communication interface; 11. an etcd configuration module; 12. a mirror image construction module; 13. a script execution module; 14. and (5) a super manager management module.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, as presented in the figures, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Examples
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the individual features of the embodiments can be combined with one another without conflict.
At present, most deployed services based on K8S are stateless, stateful services cannot be efficiently integrated with K8S, the automatic fault recovery, dynamic capacity expansion and other processes of the services cannot be realized, and the service operation and maintenance cost is high. In view of this, the embodiment of the present application provides a containerization operation and maintenance method for stateful services, which introduces etcd storage and hypervisor management, so that the services have the high availability of K8S, and meanwhile, the operation and maintenance requirements of the stateful services are met, and efficient integration of the stateful services and the K8S is realized.
Specifically, referring to fig. 1 and fig. 2, the method mainly includes the following steps:
step S1: the dynamic configuration of the stateful services is written into the etcd storage system and the services are stored in a prescribed format.
In the above steps, etcd is a distributed, consistent KV storage system for shared configuration and service discovery. The method adopts the Raft protocol to ensure the availability and consistency of the data of the distributed system, has the characteristics of simple interface and reliable data, and is suitable for storing key data in the distributed system. Such as for use in K8S (kubernets, an open source for managing containerized applications on multiple hosts in a cloud platform) to manage configuration data, state data, metadata, etc.
When in use, the data in the etcd needs to be configured in advance, the dynamic configuration of the stateful service is written into the etcd storage system, and the data in the etcd storage system is written into the storage system according to a specified format: /{ service type }/{ service name }/{ service configuration item name }, the service is stored. For example:
curl-X PUT
http://172.17.45.14:8030/v2/keys/zookeeper/zk-dubbo-welcome/max_client_cnxns-d value＝"102400"
the command indicates that the service for writing the configuration is zookeeper, the service name is zk-dubbo-welcor, the configuration item of the service is max _ client _ cnxns, and the configuration content is 102400. The corresponding format is etcd address/v 2/keys/service type/service name/configuration item, wherein the Value followed content is the Value of the configuration item.
Step S2: and constructing a service mirror image in a K8S format through dockerfile, designating an initialization script, reading a configuration item from the etcd storage system, and adding the configuration item into the initialization script.
In the above steps, when the service mirror image in the K8S format is manufactured, the mirror image may be constructed in the container by writing a dockerfile file. The dockerfile is a text file for constructing a docker image, and the text content includes a piece of instruction and description required for constructing the image. Through the docker build command, the required service image can be constructed according to the content of the Dockerfile. After the mirror image is constructed, the initialization script is designated as the script which is designated to be written, and then the part of the service configuration item which is read from the etcd storage system is added into the initialization script. So that each time the container is started, it can be automatically retrieved from the etcd storage system and the corresponding configuration generated.
And step S3: the configuration file is dynamically generated using the K8S platform boot service to execute the initialization script.
In the above steps, K8S is a tool for arranging containers, and in fact, is a tool for managing the full life cycle of the application. The method comprises the processes of application creation, application deployment, application service providing, capacity expansion and reduction application, application updating and the like, and can realize fault self-healing. For example, when one server fails, the K8S can automatically schedule the service on the server to another host for operation without human intervention.
Further, the step of using the K8S platform to start the service to execute the initialization script and dynamically generating the configuration file specifically includes:
first, a new service is created through the K8S platform, and a stateful object is created. And then, through configuration affinity, the container is ensured to be bound on a specified host, and the data directory of the service is mounted on the directory of the host, so that the data is ensured not to be lost after the container is destroyed. And meanwhile, starting a reuse instance function, configuring the resource quota of the service, and writing the name of the service and the corresponding etcd address into an environment variable of the stateful object. Therefore, identification is carried out through the environment variable, and each service is guaranteed not to acquire wrong configuration content.
The purpose of configuring the resource quota of the service is to limit the resource use condition of the thread, enable the system to run normally, and avoid that a certain thread in the K8S cluster is abnormal to occupy a large amount of system resources and affect other threads. The resource quota mainly comprises: computing Resource management (computer Resources), configuration Range management (Limit Range), and quota management (Resource requests). The computing resource management is mainly computing resources which are appointed to be used for containers in the threads, and the computing resources comprise a CPU (central processing unit), a memory and the like. The configuration range management mainly makes global unified limitation on the requests and the range configuration in the K8S cluster, which is equivalent to setting resource use limitation of threads within a certain range (a certain namespace) in batch. Quota management mainly provides an overall resource use limit for each namespace (namespace), and can limit the total number upper limit of a certain type of objects in the namespace and set the overall upper limit of computing resources which can be used by threads in the namespace, thereby improving the efficiency of resource allocation.
And step S4: and managing the running state of the service through the super based on the configuration file.
In the above steps, after the service is started, the container generates a corresponding configuration file according to the initialization script. And then, based on the configuration file, managing the running state of the service through the hypervisor, and realizing automatic management. The Supervisor is a set of general process management program developed by Python, can change a common command line process into a background daemon (background thread), monitors the process state, and can be automatically restarted when abnormal exit occurs. The method starts the managed processes as the subprocesses of the hypervisor in a fork/exec mode, so that the path of the executable file of the process to be managed is written in the configuration file of the hypervisor. And when the child process fails, the parent process can accurately acquire the information related to the failure of the child process and select whether to start and alarm.
In summary, in order to solve the problem that the existing K8S and stateful service cannot be integrated, the service has the high availability of the K8S and meets the operation and maintenance requirements of the stateful service by introducing the etcd storage and the hypervisor management. After deployment, the effects of no manual intervention and self-healing of the service can be achieved, the operation and maintenance efficiency is improved, and the usability of the service is ensured.
Based on the same inventive concept, the present invention further provides a containerization operation and maintenance system with stateful service, please refer to fig. 3, and fig. 3 is a block diagram of a containerization operation and maintenance system with stateful service according to an embodiment of the present application. The system comprises:
an etcd configuration module 11, configured to write dynamic configuration of stateful services into an etcd storage system, and store the services according to a specified format;
the mirror image construction module 12 is used for constructing a service mirror image in a K8S format through dockerfile, designating an initialization script, reading a configuration item from the etcd storage system and adding the configuration item into the initialization script;
the script execution module 13 is configured to use the K8S platform to start a service to execute the initialization script, and dynamically generate a configuration file;
and the hypervisor management module 14 is used for managing the running state of the service through the hypervisor based on the configuration file.
Referring to fig. 4, fig. 4 is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device comprises a memory 1, a processor 2 and a communication interface 3, wherein the memory 1, the processor 2 and the communication interface 3 are electrically connected with each other directly or indirectly to realize the transmission or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 1 may be used to store software programs and modules, such as program instructions/modules corresponding to a containerized operation and maintenance system with stateful services provided in the embodiments of the present application, and the processor 2 executes the software programs and modules stored in the memory 1, thereby executing various functional applications and data processing. The communication interface 3 may be used for communication of signaling or data with other node devices.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (7)
1. A containerization operation and maintenance method for stateful services is characterized by comprising the following steps:
writing the dynamic configuration of the stateful service into an etcd storage system, and storing the service according to a specified format;
constructing a service mirror image in a K8S format through dockerfile, designating an initialization script, reading a configuration item from an etcd storage system and adding the configuration item into the initialization script;
starting a service by using a K8S platform to execute an initialization script and dynamically generating a configuration file;
and managing the running state of the service through the super based on the configuration file.
2. The containerization operation and maintenance method of a stateful service of claim 1, wherein the specified format is: /{ service type }/{ service name }/{ service configuration item name }.
3. The containerization operation and maintenance method of a stateful service of claim 1, further comprising:
creating a new service through a K8S platform and creating a stateful object;
and starting a reuse instance function, configuring the resource quota of the service, and writing the name of the service and the corresponding etcd address into an environment variable of the stateful object.
4. The containerization operation and maintenance method of a stateful service of claim 1, further comprising: and mounting the data directory of the service on the directory of the host by configuring the affinity.
5. A containerized operation and maintenance system with stateful services, comprising:
the etcd configuration module is used for writing the dynamic configuration of the stateful service into the etcd storage system and storing the service according to a specified format;
the mirror image construction module is used for constructing a service mirror image in a K8S format through dockerfile, appointing an initialization script, reading a configuration item from the etcd storage system and adding the configuration item into the initialization script;
the script execution module is used for starting service by using the K8S platform to execute the initialization script and dynamically generating a configuration file;
and the hypervisor management module is used for managing the running state of the service through the hypervisor based on the configuration file.
6. An electronic device, comprising:
a memory for storing one or more programs;
a processor;
the one or more programs, when executed by the processor, implement the method of any of claims 1-4.
7. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211490913.7A CN115756754A (en) | 2022-11-25 | 2022-11-25 | Containerized operation and maintenance method and system for stateful service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211490913.7A CN115756754A (en) | 2022-11-25 | 2022-11-25 | Containerized operation and maintenance method and system for stateful service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115756754A true CN115756754A (en) | 2023-03-07 |
Family
ID=85338140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211490913.7A Pending CN115756754A (en) | 2022-11-25 | 2022-11-25 | Containerized operation and maintenance method and system for stateful service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115756754A (en) |
-
2022
- 2022-11-25 CN CN202211490913.7A patent/CN115756754A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8239505B2 (en) | Progressively implementing declarative models in distributed systems | |
| US10853160B2 (en) | Methods and systems to manage alerts in a distributed computing system | |
| CN107451147B (en) | Method and device for dynamically switching kafka clusters | |
| CN112287007B (en) | Industrial production data real-time processing method and system based on Flink SQL engine | |
| US9146756B2 (en) | Java management extensions (JMX) bean custom service extension plugin agent for an application server | |
| CN113204353B (en) | Big data platform assembly deployment method and device | |
| US9513934B2 (en) | Platform and software framework for data intensive applications in the cloud | |
| US10872007B2 (en) | Methods and systems to compound alerts in a distributed computing system | |
| US20230305876A1 (en) | Managing storage domains, service tiers, and failed servers | |
| CN110971700A (en) | Method and device for realizing distributed lock | |
| CN111858007A (en) | Task scheduling method and device based on message middleware | |
| CN115543548B (en) | Configuration method, device and equipment of container group and readable storage medium | |
| CN116339756A (en) | Application program mounting method, device, equipment and storage medium | |
| CN115543543A (en) | Application service processing method, device, equipment and medium | |
| US20230161643A1 (en) | Lifecycle management for workloads on heterogeneous infrastructure | |
| CN117608810B (en) | Service processing method, device, electronic equipment and storage medium | |
| CN117076096A (en) | Task flow execution method and device, computer readable medium and electronic equipment | |
| CN114443239A (en) | Method and device for filling container | |
| CN111399999A (en) | Computer resource processing method and device, readable storage medium and computer equipment | |
| CN115756754A (en) | Containerized operation and maintenance method and system for stateful service | |
| US11675678B1 (en) | Managing storage domains, service tiers, and failed service tiers | |
| US11663096B1 (en) | Managing storage domains, service tiers and failed storage domain | |
| CN116708583A (en) | Service execution method, storage medium, device and distributed system | |
| CN116010111A (en) | Cross-cluster resource scheduling method, system and terminal equipment | |
| CN110837405B (en) | Multi-instance graphic desktop operation and switching method based on container technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |