Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, are intended to fall within the scope of the present invention.
In the description of the present invention, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
Referring to fig. 1 to 9, a fast communication protocol includes: the data head comprises a packet head length, a protocol version number, a packet type, a flag bit, an application code length, an application code, a service name length, a service name, a method name length, a method name, a system tracking number length, a system tracking number, an application system Token length, an application system Token, a request serial number length, a request serial number, an encryption algorithm name length, an encryption algorithm name, a compression algorithm name length, a compression algorithm name, a file name length, a file name, a message coding length and a message coding; a data body comprising a data length, a field and a MAC check; and the file attachment comprises a file length, file content, total fragment number and fragment sequence number.
The fields include: a common field, wherein the common field consists of a field name and a field value, the field name comprises the length of the field name and the value of the field name, and the field value comprises the length of the field value and the value of the field value; the structure field consists of a flag bit, a structure name length, a structure name, a structure body length and a structure body, and the content of the structure body is the same as that of the data body; the cyclic field consists of a flag bit, a cyclic name length, a cyclic name, a cyclic number, an ith cyclic length and an ith cyclic body, and the content of the cyclic body is the same as that of the data body.
The field name length of the common field is 250 bytes at most; if the field name length is larger than 250 bytes, converting the field name length into a flag bit, wherein the flag bit uses 0xFE and 0xFF, the 0xFE corresponds to a structural field, and the 0xFF corresponds to a circulating field; when the field name length of the common field is less than or equal to 250 bytes, namely the field value length and the field value are included; when the field name length is greater than 250 bytes, a fragmentation process is required, where the length field flag is 0xFF and the fragment length is 250 bytes.
The packet types include: data transmission, file transmission, data and file transmission.
The communication protocol is mainly used for socket communication based on TCP. The main usage scenario is quick communication among a plurality of application systems in the same data center, and simultaneously supports communication among different data center application systems, but service registration information delay is needed to be paid attention, and the service registration information delay updating method can be used in an application scenario allowing service registration information delay updating.
The communication protocol comprises a data head, a data body and a file attachment.
Data head:
1) Header length, header length field is integer 2 bytes. The header length includes a protocol version number, a packet type, and a flag bit, and may include an application code length, an application code, a transaction code length, a transaction code, a service serial number length, a service serial number, a user identification length, a user identification, a signature length, and a signature. But the length does not include the header length field itself. The length of the field contained in the header is not more than 256.
2) Version number, version number field is byte type, length is 4bit, current value is 0x01.
3) The packet type, packet type field is byte type, length 4bit. Currently support two types, data transfer, file transfer, data and file transfer
0x01 |
Data transmission |
0x02 |
File attachment |
0x03 |
Data and file transfer |
4) The flag bit is byte type and has the length of 1 byte. The total 8 flag bits have the following specific meanings:
5) The application code length and the application code, the application code length field is integer and 1 byte, and the application code refers to the application system identification and character type.
6) The service name code length and the service name are the integer and 1 byte, and the service name refers to the name and character type of the application system service.
7) The method name length and the method name are integer and 1 byte, and the method name refers to the name and character type of calling the service method of the application system.
8) The length of the system tracking number and the length field of the system tracking number are integer and 1 byte, and the system tracking number indicates the uniqueness and character type of the request in the service sense.
9) The application system Token length and the application system Token, the application system Token length field is integer and 1 byte, and the application system Token is used for authenticating the application system.
10 Request serial number length and request serial number, the request serial number length field is integer and 1 byte, and the request serial number is the unique identification of each request. The encryption algorithm length and the encryption algorithm, the encryption algorithm length field is integer and 1 byte, and the encryption algorithm refers to a data encryption algorithm used for a data body or a file attachment, such as DSE, RSA and the like.
12 A compression algorithm length field is integer, 1 byte, and the compression algorithm refers to a data compression algorithm used for data volumes or file attachments, such as snpay, gzip, etc.
13A file name length and a file name, the file name length field being integer, 1 byte. The file name refers to the name of the transmission file.
14 Message coding length and message coding, wherein the message coding length field is integer and 1 byte.
Data body:
1) The data length, the data length field is integer, 4 bytes, the data length refers to the length of the data body.
2) The field type is divided into a common field, a result field and a circulation field.
(1) The common field is composed of a field name and a field value, wherein the field name comprises the length of the field name and the value of the field name, and the field value comprises the length of the field value and the value of the field value.
The field name length (byte type, 1 byte) of the normal field is 250 bytes at maximum, and more than 250 is illegal. If the field name length is greater than 250, the field name is converted into a flag bit, and 0xFE and 0xFF are used currently, wherein the 0xFE corresponds to a structural field, and the 0xFF corresponds to a circulating field.
When the field value length is less than or equal to 250, the field value length and the field value are included. When the field value length is greater than 250, a slicing process is required, wherein the length field flag is 0xFF, indicating that there is a subsequent slice, and the slice length is 250, and the subsequent slices are spliced into a completion value.
0xFF |
Value (Split) |
1Byte, a value of 0xFF indicates that there is a subsequent slice, otherwise, end |
Value (fragment i) |
(2) The structure field consists of a flag bit, a structure name length, a structure name, a structure body length and a structure body, and the content of the structure body is the same as that of the data body.
(3) The cycle field consists of a flag bit, a cycle name length, a cycle name, a cycle number, an ith cycle length and an ith cycle body, and the contents of the cycle body are the same as the data body.
3) MAC checking, byte type, length 4 bytes, and the algorithm is undetermined. When the value is 0, no check code is indicated.
File attachment:
when the packet type is 0x02, the message only contains the file attachment, and when the packet type is 0x03, the message comprises data and the file attachment. The file attachment format is as follows:
When the message comprises data and files, the data packet is sent first, and then the file packet is sent. The file length refers to the length of the content of the transmitted file, the total number of fragments is the total number of fragments of the file, the length is 1 byte, and the fragment sequence number refers to the sequence number of the current fragment, the byte type and the length is 1 byte.
Note that: 1) The current version does not support the breakpoint resume function.
2) In the file transmission process, the slicing processing is recommended, and the single transmission data is not more than 8k.
Referring to fig. 10, a method for controlling financial services based on a rapid communication protocol, to which the rapid communication protocol is applied, includes: a service registers its access address with a registration center at the time of starting and provides service information; the registry stores the service information so that other services can be queried and discovered; other services can query the registration center for the position information of the corresponding service when needing to interact with a specific service; the registry will return the address of the target service and other relevant information so that communication can be made directly between the two services.
Service registration and discovery involves a registry, service consumer, service provider; the service provider registers own address and service information to the registry when the service is started, and the service provider and the registry communicate through tcp socket; the service consumer inquires the address and other basic information of the service from the registry, and the service consumer and the registry are communicated through a tcp socket; the service consumer invokes the service interface of the service provider, and the service consumer and the service provider communicate through a tcp socket.
The service consumer invokes a service interface of a service provider, supporting a synchronous mode and an asynchronous mode; if the service interface is in the synchronous mode, the service provider needs to wait for the return of the calling result and then execute the subsequent operation; if the service interface is in asynchronous mode, subsequent operations may be performed upon receipt of a service provider's response.
Implementation of the support function
Service registration discovery
1) Introduction to service registration discovery
Service registration and discovery refers to how a service registers its location information and how other services discover and communicate with it in a distributed environment. Specifically, service registration and discovery refers to:
Service registration: the service registers its access address with the registry at start-up, as well as the service information provided. The registry stores this information so that other services can query and discover.
Service discovery: other services may query the registry for location information for a particular service when interaction with that service is desired. The registry will return the address of the target service and other relevant information so that the service can communicate directly.
2) Architecture and flow
Service registration discovery involves a registry, service consumer, service provider, and architecture diagram is shown in fig. 3:
The flow is as follows:
(1) The provider registers own address and service information with the registry when the service is started, and the provider and the registry communicate through tcp socket. The fields illustrate the following table:
The registration is based on the following messages sent by way of example:
{"appCode":"app1","operate":"register","serviceName":"com.example.service.UserService","address":"10.0.0.1:20880","methods":["methodA":{"args":[100,"test"],"return":"Sting","mode":"synchronize"}]"weight":"50"}
wherein the "operation" field refers to a specific operation type, and a "register" value is used at registration.
The message of the registry response is as follows:
{"result":"success","appCode":"app1","operate":"register","serviceName":"com.example.service.UserService","address":"10.0.0.1:20880"}
(2) The consumer inquires the address and other basic information of the service from the registry, and the consumer and the registry communicate through a tcp socket.
The conditional message sent by the consumer is as follows:
{“appCode”:“app1”“operate”:“query”,“serviceName”:“com.example.service.UserService”}
wherein the "method" field value "query" indicates the address and other basic information of the query-specific service.
The message of the registry response is as follows:
{“appCode”:“app1”,“result”:“success”,“operate”:”query”,
“serviceName”:“com.example.service.UserService”,
"address":["10.0.0.1:20880","10.0.0.2:20880"],"weights":[50,50],"methods":["methodA":{"args":[100,"test"],"return":"Sting","mode":"synchronize"}]}
when the instance information of the registry service changes, the consumer needs to be notified through a socket, and the notification content is as follows:
{"appCode":"app1","operate":"change","serviceName":"com.example.service.UserService","address":["10.0.0.1:20880:50","10.0.0.2:20880:50"],"weights":[50,50],"methods":["methodA":{"args":[100,"test"],"return":"Sting","mode":"synchronize"}]}
the response message of the consumer is as follows:
{"result":"success","appCode":"app1","operate":"change","serviceName":"com.example.service.UserService"]}
(3) The consumer service invokes the provider's service interface, which communicates with each other via tcp socket, the detailed description of the communication protocol being as described above. The consumer invokes the provider's service interface, supporting synchronous and asynchronous modes, and the service registration sets whether the mode of the method is asynchronous or synchronous. When a consumer calls a specific method, if the method is synchronous, the subsequent operation can be executed by waiting for the provider to return a call result, and if the method is asynchronous, the subsequent operation can be executed without waiting for the specific call result after receiving the response of the provider (receiving a request). The fields of the communication message are described as follows, the header fixed field is omitted here, and only the data body field is described:
Data message
The following describes the communication procedure using simplified data messages as examples:
1> asynchronous mode, consumer invokes provider message:
{“method”:“methodA”,“args”:[100,“test”],“address”:“10.0.0.1:20880”}
response by the service provider:
{“result”:“recieve sucess”,“method”:“methodA”,“args”:[100,“test”]}
Asynchronous notification of service provider:
{“result”:“handle sucess”,“method”:“methodA”,“param”:[100,“test”],“data”:“100”}
wherein the data field is the result data of the response. The interaction process is schematically shown in fig. 4.
2> Synchronization scheme
Consumer call provider message:
{“method”:“methodA”,“args”:[100,“test”]}
response by the service provider:
{“result”:“handle sucess”,“method”:“methodA”,“args”:[100,“test”],“data”:“100”}
An interaction process is schematically shown in fig. 5.
File message
Field name |
Meaning of |
Must be transported |
Description of the invention |
fileName |
File name |
Is that |
|
fileCode |
File encoding |
Is that |
|
fileLenth |
File length |
Is that |
|
content |
File content |
Is that |
|
shareCount |
Total number of fragments |
Is that |
Total number of file fragments |
shareIndex |
Fragment serial number |
Is that |
The number of fragments is the sequence number of fragments. |
The file message sent by the consumer is exemplified as follows:
message data header:
{“fileName”:“fname1”,“fileCode”:“fcode1”}
Data body:
{“fileLenth”:1000,“content”:da12vhju...,“shareCount”:0x05,“shareIndex”:0x01}
a schematic diagram of file fragment transmission is shown in fig. 6.
3) Registry data structure
The registry may use a ZooKeeper. The stored data structure in the ZooKeeper is as follows:
Root node: in ZooKeeper, a root node named/nRPC is created under the root node.
Application node: under/nRPC root nodes, each application system creates a node, the node name being the application code, for example: nRPC/app1
And the service interface nodes are used for each service interface under the application node, and the name of each service interface is the full-limit name of the service interface, such as/nRPC/app 1/com. Each service interface node will contain all provider nodes and consumer nodes for that service.
Provider node: for a provider node, a providers child node is created under the service interface node for storing all provider information that provides the service. The name of each provider node is a unique identifier, typically a combination of the provider's IP address and port number, e.g., 10.0.0.1:20880. The data content of each provider node typically includes information of the provider's address, weight, status, etc.
Consumer node: for a consumer node, a consumers child node is created under the service interface node for storing all consumer information to consume the service. Each consumer node is structured like a provider node, the node name is a combination of an IP address and a port, and the node data contains information of the address, status, etc. of the consumer.
The method comprises the following steps: for method nodes, a methods child node is created under the service interface node for storing all methods and their attributes. The node name is a combination of method name, parameter type and return value type, and the node data is a pattern value of the method, i.e., synchronous or asynchronous.
Dynamic node change: when a new service provider or consumer registers, a corresponding node is dynamically created and relevant information is written to the node. When a service provider or consumer goes offline, the corresponding node is dynamically deleted. Changes in service interface content, such as method increases and decreases, parameter changes, etc., also require registration information to be updated.
In general, using a ZooKeeper as a registry, corresponding application system nodes and service interface nodes are created under/nRPC root nodes, and provider and consumer nodes are created under each service interface node for storing information of service providers and consumers. The structure enables the registry to realize dynamic registration and discovery of the service and ensures the reliability and stability of service call.
4) Service center information synchronization
Registry service information synchronization is a critical process, and the registry is hierarchically designed to be internal to the application, at the application level, and at the enterprise level. The application-level registry ensures cross-system service invocation and cross-data center invocation, and the enterprise-level registry supports system service communication of different institutions.
Referring to fig. 7, the specific implementation of traffic routing is that first, resources (application services) with certain attribute characteristics are divided into a set, the set is labeled with a specific label, then traffic with the specific characteristic is routed to the application service set identifying the label, and other traffic is routed to other application services not identifying the label. The feature data may be an application system or a user, etc.
1) Communication security
The communication security mainly comprises security of a communication link and encryption of communication content, wherein the link security refers to link encryption based on security protocols such as SSL/TLS and the like, so that the communication content is ensured not to be stolen, tampered and the like; encryption and decryption of communication data content are summarized as follows: the sender inputs the plaintext, encrypts the plaintext, generates ciphertext, transmits the ciphertext, and the receiver decrypts the ciphertext to obtain the plaintext.
2) Authentication authorization
Authentication refers to validating the identity of a claimant. The common authentication method mainly comprises the following steps: identity card, user name and password, mobile phone short message, two-dimensional code scanning, gesture password, email, biological characteristics of user: fingerprint, iris, etc.
Authorization refers to obtaining the delegation rights of a user. In the field of information security, authorization refers to delegation of an executor by a resource owner, and endowing the executor with a resource operation authority within a specified range.
The application system can be authenticated by the application system code and the application system Token in the data message protocol, the user and the user Token field authenticate the user, and the legitimacy of the operation is checked according to the authority list of the application system and the user.
The flow limiting fusing is a flow control component facing the distributed service architecture, mainly uses flow as an access point, and helps developers to ensure the stability of micro services from multiple dimensions of flow limiting, flow shaping, fusing degradation, system load protection, hot spot protection and the like, so that avalanche effect on external services caused by various problems of software and hardware faults, flow surge, synchronous resource exhaustion, cache failure and the like is prevented.
One reference implementation for current limiting fusing is as follows:
The request is issued to the client in real time through the configuration center, the client intercepts the request, and whether the request is released or not is judged according to the treatment strategy. The client asynchronously collects flow monitoring data, sends the data to the current limiting center for aggregation calculation, and provides basis for operation and maintenance personnel to evaluate treatment effect and formulate treatment strategy. As shown in fig. 8.
And (3) strategy release: depending on the configuration center, the real-time issuing of the treatment strategy is realized.
Request interception: and intercepting the request by taking the flow as an access point, and performing request degradation processing which does not meet the policy.
And (3) flow monitoring: and collecting operation indexes related to the flow, performing real-time aggregation analysis and performing visual display.
Referring to fig. 9, the monitoring center is responsible for counting indexes such as call times, call time, call success rate and the like of the service, the calling party and the called Fang Xian collect data in the memory for a period of time (for example, one minute) and then send the data to the monitoring center server, and the monitoring center itself also needs to buffer the received monitoring data and then persistence in batches. And finally, displaying the monitoring statistical data in a report form.
The monitoring indexes such as calling times, calling time, calling success rate and the like can be subjected to statistical analysis according to multiple dimensions, including dimensions of an application system, a consumer, a provider and the like.
The above examples are only for illustrating the technical scheme of the present invention and are not limiting. It will be understood by those skilled in the art that any modifications and equivalents that do not depart from the spirit and scope of the invention are intended to be within the scope of the appended claims.