[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN106407011A - A routing table-based search system cluster service management method and system - Google Patents

A routing table-based search system cluster service management method and system Download PDF

Info

Publication number
CN106407011A
CN106407011A CN201610835665.3A CN201610835665A CN106407011A CN 106407011 A CN106407011 A CN 106407011A CN 201610835665 A CN201610835665 A CN 201610835665A CN 106407011 A CN106407011 A CN 106407011A
Authority
CN
China
Prior art keywords
search
cluster
client
service
request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610835665.3A
Other languages
Chinese (zh)
Other versions
CN106407011B (en
Inventor
姜平
何学勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Focus Technology Co Ltd
Original Assignee
Focus Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Focus Technology Co Ltd filed Critical Focus Technology Co Ltd
Priority to CN201610835665.3A priority Critical patent/CN106407011B/en
Publication of CN106407011A publication Critical patent/CN106407011A/en
Application granted granted Critical
Publication of CN106407011B publication Critical patent/CN106407011B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/202Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
    • G06F11/2023Failover techniques
    • G06F11/203Failover techniques using migration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/202Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
    • G06F11/2023Failover techniques
    • G06F11/2033Failover techniques switching over of hardware resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/951Indexing; Web crawling techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/54Organization of routing tables
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/74Address processing for routing
    • H04L45/745Address table lookup; Address filtering
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/2866Architectures; Arrangements
    • H04L67/30Profiles
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/80Database-specific techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5013Request control
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/508Monitor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

一种基于路由表的搜索系统集群服务管理的方法,通过对搜索服务器进行逻辑集群划分,将划分所得的集群信息存在路由表模块中,并构建多个对外提供集群服务的搜索应用请求转发器,这些请求转发器能统一从路由表信息中获取集群服务的分布,并面向搜索客户端提供服务结果应答和搜索请求转发;搜索客户端提出的搜索服务的索引数据被复制多份,分配在不同的搜索服务应用中;通过搜索路由表系统记录搜索服务在搜索服务应用的分配情况,同时记录所有搜索服务应用和由他们组合而成的搜索集群;搜索请求分发系统接收客户端发出的搜索服务请求,将搜索请求转发给相应的搜索服务应用,得到结果后回发给搜索客户端。

A routing table-based search system cluster service management method, by dividing the search server into logical clusters, storing the divided cluster information in the routing table module, and constructing multiple search application request transponders that provide cluster services externally, These request forwarders can uniformly obtain the distribution of cluster services from the routing table information, and provide service result responses and search request forwarding for search clients; the index data of search services proposed by search clients are copied in multiple copies and distributed in different In the search service application; record the distribution of search service in the search service application through the search routing table system, and record all search service applications and the search cluster formed by them at the same time; the search request distribution system receives the search service request sent by the client, The search request is forwarded to the corresponding search service application, and the result is sent back to the search client.

Description

一种基于路由表的搜索系统集群服务管理的方法及系统Method and system for cluster service management of search system based on routing table

技术领域technical field

本发明属于海量数据搜索技术领域,具体涉及一种针对多搜索集群对外提供搜索服务的管理方法及系统。The invention belongs to the technical field of massive data search, and in particular relates to a management method and system for providing search services for multi-search clusters.

背景技术Background technique

搜索系统从海量数据中检索相匹配的目标数据提供给用户浏览,是目前主流电子商务平台(B2B,B2C等)的主要应用场景。在数据日趋庞大,搜索匹配规则日趋复杂的今天,单靠一个搜索应用的集群对外服务,在追求高可用服务的今天,已经不能满足用户的需求。构建搜索系统集群服务,能够解决国内目前实际上网络分为电信,网通两个不同网络下的访问速度,尤其针对多数据中心(简称DC)部署的业务系统,搜索系统天然就面临一个跨DC部署应用的现实环境。The search system retrieves matching target data from massive data and provides it to users for browsing, which is the main application scenario of mainstream e-commerce platforms (B2B, B2C, etc.). Today, when the data is getting larger and the search matching rules are becoming more and more complex, relying on a single search application cluster for external services, in pursuit of high-availability services, can no longer meet the needs of users. Constructing a search system cluster service can solve the problem of access speed under two different networks, China Telecom and China Netcom, especially for business systems deployed in multiple data centers (referred to as DC). The search system naturally faces a cross-DC deployment The real-world environment of the application.

物理DC的区分如果还能够人工进行搜索系统集群服务管理的话,那么由于逻辑功能上的区分集群的需求,则不可能单单依靠根据网络硬件条件所划分的几个固定集群即可满足。例如搜索业务是随着用户需求可能随时需要调整搜索的匹配规则,包括不同的匹配规则可能需要在线上进行并行的用户实际应用试验,根据搜索结果和用户行为的反馈,再动态进行搜索规则和参数的调整。再例如搜索除了用户实际使用外,各大搜索引擎的爬虫也会自动进行爬虫抓取网站内容,而当这种抓取量对搜索服务器造成极大负荷的时候,我们想要人为的区分爬虫触发的搜索请求和真实用户访问的搜索请求,将两种请求进行不同的服务对待的时候。这都需要我们能够灵活的通过配置不同的搜索系统集群服务来满足这种管理要求。而本发明就是为了针对这种现实的技术应用场景,通过路由表技术的方式,将搜索集群对外服务的路由进行管理,达到集群服务调度的目的。目前一般的搜索系统,都是通过将搜索服务对外服务的地址,配置在固化的搜索客户端中,一般会配置多个对外服务地址,作为提高可用性的保证。但如果这些地址都不可用时,搜索客户端会直接报错。另外一般的搜索系统,在对外服务中是不允许同时存在两个同名的搜索服务,会出现同名错误。而本发明支持对外服务中的可以存在任意多个重名的搜索业务,且可以通过配置不同的访问策略,支持多个同名搜索业务的不同集群按照不同的业务逻辑对外提供搜索服务,并且本发明通过客户端缓存路由表信息的方式,即使客户端配置的服务地址全部出错,仍然能够与可用服务器建立连接,检索得到正常的搜索结果。If the distinction of physical DCs can also be managed manually for search system cluster services, it is impossible to rely solely on several fixed clusters divided according to network hardware conditions to meet the needs of distinguishing clusters in terms of logical functions. For example, in the search business, the matching rules of the search may need to be adjusted at any time according to the needs of users, including different matching rules that may require parallel user application tests online, and then dynamically perform search rules and parameters according to the feedback of search results and user behavior. adjustment. Another example is that in addition to the actual use of users, the crawlers of major search engines will also automatically crawl the content of the website. When the amount of crawling causes a huge load on the search server, we want to artificially distinguish crawler triggers. When the search requests of real users and the search requests accessed by real users are treated differently. This requires us to flexibly configure different search system cluster services to meet this management requirement. The present invention is aimed at such a realistic technical application scenario, and manages the routing of the external service of the search cluster through the routing table technology, so as to achieve the purpose of cluster service scheduling. At present, the general search system configures the external service address of the search service in the solidified search client, and generally configures multiple external service addresses as a guarantee for improving usability. But if none of these addresses are available, the search client will directly report an error. In addition, the general search system does not allow two search services with the same name to exist in the external service at the same time, and an error of the same name will occur. However, the present invention supports any number of search services with the same name in external services, and by configuring different access policies, different clusters that support multiple search services with the same name provide search services externally according to different business logics, and the present invention By caching the routing table information on the client side, even if all the service addresses configured on the client side are wrong, it can still establish a connection with an available server and retrieve normal search results.

发明内容Contents of the invention

为了解决搜索客户端配置的搜索地址出错和满足搜索需要同名搜索服务同时存在的问题,本发明目的在于,提供一种基于路由表的搜索系统集群服务管理的方法及系统,旨在提高搜索系统集群服务的可用性,即使客户端配置的搜索集群的服务节点出现故障,客户端仍然能够通过路由表找到其他可用的搜索集群服务节点;同时通过路由表存放不同集群的服务信息,使搜索系统可以根据不同访问策略调用若干集群下的同名搜索服务。同时本发明也能改良后应用与其他分布式应用系统的集群服务管理。In order to solve the problem of the wrong search address configured by the search client and the simultaneous existence of search services with the same name that meet the search requirements, the purpose of the present invention is to provide a method and system for managing search system cluster services based on routing tables, aiming at improving the search system cluster. Service availability, even if the service node of the search cluster configured by the client fails, the client can still find other available search cluster service nodes through the routing table; at the same time, the service information of different clusters is stored through the routing table, so that the search system can be based on different The access policy invokes search services with the same name under several clusters. At the same time, the invention can also improve cluster service management of the application and other distributed application systems.

本发明技术方案如下:一种基于路由表的搜索系统集群服务管理的方法,通过对搜索服务器进行逻辑集群划分,将划分所得的集群信息存在路由表模块中,并构建多个对外提供集群服务的搜索应用请求转发器,这些请求转发器能统一从路由表信息中获取集群服务的分布,并面向搜索客户端提供服务结果应答和搜索请求转发;搜索客户端client提出的搜索服务的索引数据被复制多份,分配在不同的搜索服务应用中;通过搜索路由表系统记录搜索服务在搜索服务应用的分配情况,同时记录所有搜索服务应用和由他们组合而成的搜索集群;搜索请求分发系统接收客户端发出的搜索服务请求,根据请求内容查询路由表系统router中关于本次搜索服务的路由信息,将搜索请求转发给相应的搜索服务应用replica-search server,得到结果后再把结果回发给搜索客户端,具体步骤包括:The technical solution of the present invention is as follows: a routing table-based search system cluster service management method, which divides the search server into logical clusters, stores the divided cluster information in the routing table module, and constructs multiple external cluster services Search application request forwarders, these request forwarders can uniformly obtain the distribution of cluster services from the routing table information, and provide service result responses and search request forwarding for search clients; the index data of search services proposed by the search client client is copied Multiple copies, allocated in different search service applications; record the distribution of search services in search service applications through the search routing table system, and record all search service applications and search clusters composed of them; the search request distribution system receives customers According to the search service request sent by the terminal, query the routing information about the search service in the routing table system router according to the request content, forward the search request to the corresponding search service application replica-search server, and send the result back to the search server after getting the result Client, the specific steps include:

步骤1:在搜索服务器上构建多个能备份搜索服务信息的搜索服务应用(replica-searh Server),并将实际的搜索服务(collection)分配在这些搜索服务器(replica-Searh Server)上;Step 1: Build multiple search service applications (replica-searh Server) that can back up search service information on the search server, and distribute the actual search service (collection) on these search servers (replica-Searh Server);

步骤2:若干个搜索服务应用(replica-Searh Server)集合,组成一个搜索系统集群(cluster);Step 2: Several search service applications (replica-Searh Server) are assembled to form a search system cluster (cluster);

步骤3:在路由表系统(router)中注册、记录及管理所有搜索系统集群内的搜索服务分布的信息;Step 3: Register, record and manage information about the distribution of search services in all search system clusters in the routing table system (router);

步骤4:配置与搜索客户端(client)指定连接的搜索请求分发系统,当客户端发出搜索请求时,能直接与搜索请求分发系统Dispatcher通信;当客户端与该指定连接的搜索请求分发系统成功建立通信后,客户端同时从路由表系统中自动获取其他可用的搜索请求分发系统的地址信息;Step 4: Configure the search request distribution system connected to the search client (client), when the client sends a search request, it can directly communicate with the search request distribution system Dispatcher; when the client successfully connects to the search request distribution system After the communication is established, the client automatically obtains the address information of other available search request distribution systems from the routing table system at the same time;

所述搜索服务信息会复制多个备份,并被分配到不同的搜索服务应用replica-search server中;所述搜索路由表系统router包含搜索服务应用replica-search server和其内搜索服务的属性信息及分布信息;所述搜索请求转发系统dispatcher接收客户端发出的搜索服务请求,并根据请求中的信息,通过根据在router查询所需搜索服务的地址信息,将搜索服务请求转发至相应的搜索服务集群及其内的搜索服务应用,获取到结果后经由搜索请求转发系统dispatcher返回给客户端;The search service information will be copied into multiple backups and distributed to different search service applications replica-search server; the search routing table system router includes the search service application replica-search server and the attribute information of the search service in it and Distributing information; the search request forwarding system dispatcher receives the search service request sent by the client, and according to the information in the request, forwards the search service request to the corresponding search service cluster by querying the address information of the required search service on the router And the search service application in it, after obtaining the result, return it to the client through the dispatcher of the search request forwarding system;

步骤5:搜索请求分发系统(dispatcher)收到来自搜索客户端Client的搜索服务请求后,检索路由表系统(router)内有关该搜索服务的搜索集群配置信息;Step 5: After the search request distribution system (dispatcher) receives the search service request from the search client client, it retrieves the search cluster configuration information about the search service in the routing table system (router);

所述搜索服务请求中包含cookie信息、指定的搜索集群、搜索服务名称,在没有启用搜索流量集群分发策略时,dispatcher将根据搜索服务请求中的信息决定搜索服务请求转发至哪一个搜索集群内;The search service request includes cookie information, a specified search cluster, and a search service name. When the search traffic cluster distribution strategy is not enabled, the dispatcher will determine which search cluster to forward the search service request to according to the information in the search service request;

步骤6:根据本次搜索请求中搜索服务在router内的集群配置信息,包括搜索服务对应的集群名称、搜索服务名称、是否启用搜索流量集群分发策略、集群内的搜索服务器名称及地址信息,进而确定dispatcher将client的搜索服务请求转发具体至哪一个集群和集群内的搜索服务应用replica-search server进行搜索服务;Step 6: According to the cluster configuration information of the search service in the router in this search request, including the cluster name corresponding to the search service, the search service name, whether to enable the search traffic cluster distribution policy, the name and address information of the search server in the cluster, and then Determine which cluster the dispatcher forwards the client's search service request to and the search service application within the cluster uses replica-search server for search service;

步骤7:搜索服务应用(replica-search server)根据收到的搜索请求,得到搜索服务结果,并将该结果返回给client端。Step 7: The search service application (replica-search server) obtains the search service result according to the received search request, and returns the result to the client.

步骤5搜索客户端client通过初始化配置可用的搜索请求分发dispatcher访问信息,初始化连接第一个可用的搜索请求分发系统dispatcher后,进入步骤5-1;Step 5: The client searches for the client by initializing and configuring the available search request distribution dispatcher access information, and after initializing and connecting to the first available search request distribution system dispatcher, enter step 5-1;

步骤5-1:搜索客户端client成功连接dispatcher后,获取与该dispatcher相连的Router内其他所有dispatcher的信息,并持久化存储在客户端本地,作为与dispatcher连接的备用信息;Step 5-1: After the client successfully connects to the dispatcher, obtain the information of all other dispatchers in the Router connected to the dispatcher, and store them locally in the client as backup information for connecting to the dispatcher;

步骤5-2:client发送搜索请求给已建立连接的dispatcher,搜索请求中包含用户的cookie信息、是否指定搜索集群cluster、所请求的搜索服务名称。搜索请求中的信息被作为dispatcher执行路由检索的参考信息;Step 5-2: The client sends a search request to the dispatcher that has established a connection. The search request includes the user's cookie information, whether to specify a search cluster cluster, and the name of the requested search service. The information in the search request is used as the reference information for the dispatcher to perform routing retrieval;

步骤5-3:dispatcher收到来自客户端client的请求信息,并根据请求信息检查router内是否有请求中的搜索服务的地址信息和已指定的搜索集群,如果上述两个信息都没有被找到,抛出异常,通知客户端client搜索失败;如果查询到相关信息,转步骤4;Step 5-3: The dispatcher receives the request information from the client, and checks whether there is the address information of the search service in the request and the specified search cluster in the router according to the request information. If neither of the above two information is found, An exception is thrown to notify the client that the search failed; if relevant information is found, go to step 4;

步骤5-4:搜索请求分发系统dispatcher在Router中已查询到客户端请求的搜索服务,但若客户端请求指定的集群信息为空或找不到对应的集群,且发现同名的搜索服务分布在其他多个集群中,搜索请求分发系统dispatcher则任意挑选一个集群作为目标集群,将搜索请求发送至该集群内包含该搜索服务的搜索服务应用replica-search server,搜索服务应用成功返回结果,由搜索请求分发系统传给客户端;Step 5-4: The search request distribution system dispatcher has found the search service requested by the client in the Router, but if the cluster information specified by the client request is empty or the corresponding cluster cannot be found, and the search service with the same name is found to be distributed in Among other multiple clusters, the dispatcher of the search request distribution system randomly selects a cluster as the target cluster, and sends the search request to the search service application replica-search server that contains the search service in the cluster. The search service application returns results successfully, and the search service The request distribution system is passed to the client;

步骤5-5:若在步骤5-4中,由搜索请求分发系统dispatcher发送至搜索服务应用replica-search server的搜索请求出错或者无响应时,dispatcher将继续在Router中查找下一个可用的搜索服务应用replica-search server;若继续出错,进而导致尝试次数达到设置的最大限制或者超过服务端超时时间设置,则搜索请求分发系统dispatcher直接抛出搜索异常信息给客户端client;Step 5-5: If in step 5-4, the search request sent by the search request distribution system dispatcher to the search service application replica-search server has an error or no response, the dispatcher will continue to search for the next available search service in the Router Apply replica-search server; if errors continue, resulting in the number of attempts reaching the set maximum limit or exceeding the server timeout setting, the search request distribution system dispatcher will directly throw search exception information to the client client;

步骤5-6:当客户端收到步骤5-5中抛出的异常时,客户端client自动在已持久化本地的搜索请求分发系统dispatcher信息列表中获取另一个可用dispatcher,并连接;重复步骤2-步骤6,直至dispatcher信息列表中记录的dispatcher都被尝试与客户端连接,若dispatcher还抛出搜索异常或者超过客户端超时时间设置,则客户端抛出异常,本次搜索请求失败;Step 5-6: When the client receives the exception thrown in step 5-5, the client client automatically obtains another available dispatcher from the dispatcher information list of the persistent local search request distribution system, and connects; repeat steps 2-Step 6, until the dispatcher recorded in the dispatcher information list is tried to connect with the client, if the dispatcher still throws a search exception or exceeds the client timeout setting, the client throws an exception, and this search request fails;

步骤5-7:若在步骤5-2中,客户端请求的搜索服务在路由表系统router中被标识为启用“搜索流量集群分配”策略,dispatcher根据搜索请求中的用户cookie信息,经计算后,与路由表系统router中已设置的各个集群的权重值比照,经比照一致的搜索集群将被选择为搜索请求的发送方,该搜索集群内包含该搜索服务的搜索服务应用replica-searchserver将作为接收本次搜索请求的目标搜索服务应用,搜索服务应用成功返回搜索结果,由搜索请求分发系统传给客户端;Step 5-7: If in step 5-2, the search service requested by the client is marked as enabling the "search traffic cluster distribution" policy in the routing table system router, the dispatcher calculates the search service based on the user cookie information in the search request , compared with the weight values of each cluster already set in the routing table system router, the search cluster that agrees with the comparison will be selected as the sender of the search request, and the search service application replica-searchserver that includes the search service in the search cluster will serve as The target search service application that receives this search request, the search service application successfully returns the search results, which are transmitted to the client by the search request distribution system;

所述步骤4中还包括客户端对dispatcher的缓存,具体步骤有:The step 4 also includes the cache of the dispatcher by the client, and the specific steps are as follows:

步骤4-1:客户端client首次成功连接指定的dispatcher后,将自动从路由表系统(Router)中远程获取当前所有可用的dispatcher列表信息,并通过文件存储Dispatcher信息实现在本地持久化缓存;Step 4-1: After the client successfully connects to the specified dispatcher for the first time, it will automatically remotely obtain all currently available dispatcher list information from the routing table system (Router), and store the Dispatcher information in a file to achieve local persistent caching;

步骤4-2:客户端client拥有Dispatcher列表信息后,当配置的Dispatcher地址不可用,客户端调用本地缓存的且已在router中注册的dispatcher列表,并自动从列表中依次选择其他dispatcher进行连接,直至连接成功;Step 4-2: After the client has the Dispatcher list information, when the configured Dispatcher address is unavailable, the client invokes the dispatcher list cached locally and registered in the router, and automatically selects other dispatchers from the list to connect. until the connection is successful;

步骤4-3:当本地缓存中所有记录的dispatcher都尝试连接后,若仍存在连接异常,则搜索客户端抛出异常,本次搜索请求失败;Step 4-3: After all dispatchers recorded in the local cache try to connect, if there is still a connection exception, the search client throws an exception, and the search request fails this time;

步骤6中启用所述的搜索流量集群分发策略,具体步骤包括:Enable the search traffic cluster distribution strategy described in step 6, and the specific steps include:

步骤6-1:在router存放的路由表信息中会通过一个字段来记录是否启用搜索集群流量分发;步骤6-2:当标示为启用搜索流量集群分发时,Dispatcher调用本次搜索业务请求,并按照用户特性进行流量切分;Step 6-1: In the routing table information stored in the router, a field will be used to record whether to enable search cluster traffic distribution; Step 6-2: When it is marked as enabling search traffic cluster distribution, Dispatcher calls this search service request, and Segment traffic according to user characteristics;

其中,搜索业务对应的多个集群的流量占比由人为设定,存储在路由表系统Router中,所有集群的流量占比总和为100%;Among them, the traffic proportions of multiple clusters corresponding to the search business are set manually and stored in the routing table system Router, and the sum of the traffic proportions of all clusters is 100%;

步骤6-3:执行搜索流量集群分发时,依据client端所发搜索请求中的用户特征信息,利用一致性哈希算法计算一个权重值;Step 6-3: When performing cluster distribution of search traffic, calculate a weight value using the consistent hash algorithm according to the user characteristic information in the search request sent by the client;

步骤6-4:根据步骤6-3得到的权重值,通过对照router中该搜索业务对应的所有集群的权重值,确认搜索请求转发的目的集群cluster;Step 6-4: According to the weight value obtained in step 6-3, by comparing the weight values of all the clusters corresponding to the search service in the router, confirm the destination cluster for forwarding the search request;

一种基于路由表的搜索系统集群服务管理的系统,其特征在于,系统包括搜索请求客户端client、搜索请求分发系统dispatcher、搜索路由表系统router、搜索服务应用replica-search server;所述四个部分之间都通过网络进行通信,各自有不同的服务端口,按照一定的规则匹配一起完成搜索服务;A routing table-based search system cluster service management system is characterized in that the system includes a search request client client, a search request distribution system dispatcher, a search routing table system router, and a search service application replica-search server; the four The parts communicate through the network, each has a different service port, and the search service is completed by matching according to certain rules;

搜索路由表系统router负责所有搜索服务应用replica-search server的登记注册及管理,并负责记录所有搜索服务应用replica-search server的属性信息及所有搜索服务在各搜索服务应用replica-search server内的分布情况;The search routing table system router is responsible for the registration and management of all search service application replica-search servers, and is responsible for recording the attribute information of all search service application replica-search servers and the distribution of all search services in each search service application replica-search server Condition;

搜索请求分发系统dispatcher在搜索客户端client和搜索服务应用replica-search server之间承担代理和转发角色,负责接收搜索请求客户端client的搜索请求,并根据搜索路由表系统router中的搜索服务路由信息、以及搜索服务和搜索集群的配置信息,对搜索请求进行不同策略的转发;The search request distribution system dispatcher assumes the role of proxy and forwarding between the search client client and the search service application replica-search server, is responsible for receiving the search request from the search request client client, and according to the search service routing information in the search routing table system router , and the configuration information of the search service and the search cluster, forwarding search requests with different strategies;

所有搜素服务应用划分为不同的搜索集群cluster,关于“每个搜索集群内的搜索服务应用的分布”、“每个搜索服务应用包含的搜索服务”等涉及到搜索集群cluster、搜索服务应用及搜索服务三者的对应关系,均被记录在搜索路由表系统router中;All search service applications are divided into different search cluster clusters. Regarding "distribution of search service applications in each search cluster" and "search services included in each search service application", it involves search cluster clusters, search service applications and The corresponding relationship among the search services is recorded in the search routing table system router;

所述搜索集群Cluster,是包含若干个replica-search server的集合;The search cluster Cluster is a set that includes several replica-search servers;

所述搜索服务,是指通过客户端向搜索服务器发出针对某个搜索业务索引的搜索请求,并得到搜索结果的服务模型。The search service refers to a service model in which a client sends a search request for a certain search service index to a search server and obtains search results.

所述搜索请求客户端client,是搜索服务调用请求的发起人,是搜索服务的直接使用人;客户端的初始dispatcher的连接地址由手工配置,当搜索客户端client成功与初始搜索请求分发系统dispatcher建立通讯关系,搜索客户端client将获取到其他有用的dispatcher地址信息;The search request client client is the initiator of the search service call request and the direct user of the search service; the connection address of the initial dispatcher of the client is manually configured, when the search client client successfully establishes with the initial search request distribution system dispatcher Communication relationship, search client client will get other useful dispatcher address information;

客户端从搜索路由表系统router内获取其他可用的搜索请求分发系统dispatcher地址信息,并不缓存关于搜索服务的详细完整的路由信息;客户端不需要知道服务端具体的搜索服务节点的分布情况,只需要知道当前所有可用的搜索服务转发代理dispatcher;The client obtains other available search request distribution system dispatcher address information from the search routing table system router, and does not cache detailed and complete routing information about the search service; the client does not need to know the specific distribution of search service nodes on the server. You only need to know all currently available search service forwarding agent dispatchers;

所述搜索请求分发系统dispatcher,是路由信息的实际执行者,能根据搜索路由表系统router内记录的集群分布信息及分布策略,执行搜索服务请求的转发和调度;The search request distribution system dispatcher is the actual executor of routing information, and can perform forwarding and scheduling of search service requests according to the cluster distribution information and distribution strategies recorded in the search routing table system router;

所述搜索服务应用replica-search server,是具备自我注册能力的搜索服务端,它包含搜索服务的索引数据,负责完成搜索服务的索引查询匹配;多个搜索服务应用构成搜索集群cluster,搜索服务应用replica-search server的集群通过自注册加入的方式成立,当第一个replica-search server启动时,它需要在router内配置地址,同时在router中增加一个初始配置信息,并在router配置信息中加入一个唯一标识id号,这个唯一标识id号会被当作一个新的搜索集群id号记录在搜索路由表系统router中;如果再有其他的搜索服务应用replica-search server想要加入已存在的搜索集群中,以现有的搜索集群id号配置在该搜索服务应用在搜索路由表系统router中;The search service application replica-search server is a search server end with self-registration capability, it contains index data of search service, and is responsible for completing index query matching of search service; multiple search service applications form a search cluster cluster, and search service application The cluster of replica-search server is established through self-registration and joining. When the first replica-search server starts, it needs to configure the address in the router. At the same time, add an initial configuration information to the router and add it to the router configuration information. A unique identification id number, this unique identification id number will be recorded as a new search cluster id number in the search routing table system router; if there are other search service applications replica-search server want to join the existing search In the cluster, configure the search service with the existing search cluster id number in the search routing table system router;

任意一个replica-search server只能从属于一个cluster,而任何一个cluster中,都不允许出现同名的replica-search server的集合collection。Any one replica-search server can only belong to one cluster, and any collection of replica-search servers with the same name is not allowed to appear in any cluster.

所述搜索客户端client:负责发送搜索请求,搜索请求中包含搜索客户端的用户特征信息,以及本次搜索服务对应的搜索服务信息(collection);The search client client: responsible for sending a search request, the search request includes user characteristic information of the search client, and search service information (collection) corresponding to this search service;

所述搜索请求分发系统dispatcher:即搜索路由信息的执行者,与搜索客户端直接通讯;同时负责从搜索路由表模块router中获取本次搜索业务在搜索服务应用内分布情况;The search request distribution system dispatcher: that is, the executor of the search routing information, directly communicates with the search client; at the same time, it is responsible for obtaining the distribution of the search service in the search service application from the search routing table module router;

所述搜索路由表系统router:用于管理所有处于启动状态的搜索服务应用,记录各个搜索服务应用的属性信息,以及每个搜索服务在搜索服务应用内的实际分布;同时用于将不同搜索服务应用集成为不同的集群,记录所有集群的属性信息及集群内的搜索服务应用分布情况;当有新的搜索服务应用启动时,需要及时在搜索搜索路由表系统router内注册和更新搜索业务分布信息;The search routing table system router: used to manage all search service applications in the startup state, record the attribute information of each search service application, and the actual distribution of each search service in the search service application; The application is integrated into different clusters, and the attribute information of all clusters and the distribution of search service applications in the cluster are recorded; when a new search service application is started, it is necessary to register and update the search service distribution information in the search routing table system router in a timely manner ;

所述搜索服务应用replica-search server:搜索服务应用存放实际的搜索服务的索引信息,若干个搜索服务应用可以组成一个搜索系统集群cluster,每一个搜索服务应用只能从属于一个集群cluster;The search service application replica-search server: the search service application stores the index information of the actual search service, and several search service applications can form a search system cluster cluster, and each search service application can only belong to one cluster cluster;

关于“每个搜索集群内的搜索服务应用的分布”、“每个搜索服务应用包含的搜索服务”等涉及到搜索集群cluster、搜索服务应用及搜索服务三者的对应关系,均被记录在搜索路由表系统router中;Regarding the "distribution of search service applications in each search cluster" and "search services included in each search service application", the correspondence between the search cluster cluster, search service applications, and search services is recorded in the search In the routing table system router;

所述搜索请求分发系统dispatcher并不具备搜索服务的能力,也不需要大量的计算和存储能力;本发明中的搜索请求分发系统dispatcher能支持高并发的大量的用户访问,能通过增加搜索服务应用replica-search server实现扩展整个搜索系统的服务能力时,而不是增加太多的dispatcher模块;The search request distribution system dispatcher does not have the ability to search services, and does not require a large amount of computing and storage capabilities; the search request distribution system dispatcher in the present invention can support a large number of high-concurrency user visits, and can increase the search service application When replica-search server realizes expanding the service capability of the entire search system, instead of adding too many dispatcher modules;

所述搜索服务应用集合而成的集群采用自注册方式组合,当第一个搜索服务应用启动时,在搜索路由表系统router配置地址信息,同时标识唯一的集群标识ID;当其他搜索服务应用欲加入已有的搜索集群中,以现有的集群标识ID作为该搜索服务应用在router中的地址信息。The cluster formed by the collection of search service applications is combined by self-registration. When the first search service application is started, the address information is configured in the search routing table system router, and the unique cluster ID is identified at the same time; when other search service applications want to Join an existing search cluster, and use the existing cluster ID as the address information of the search service application in the router.

本方明与现有技术相比,其有益效果:Compared with the prior art, this party has beneficial effects:

(1)本发明中的客户端除了配置的固定dispatcher地址外,还加入客户端持久化文件,可以自动定期更新服务端的所有的可用搜索服务请求转发应用(简称dispatcher)信息,这样大大的提高了搜索集群服务系统的可用性,一个或少数dispatcher的不可用,或是client初始化连接的dispatcher不可用,都不会导致client无法连接搜索服务。(1) In addition to the fixed dispatcher address of the configuration, the client in the present invention also adds the client persistent file, which can automatically and regularly update all available search service request forwarding application (dispatcher) information of the server, which greatly improves the The availability of the search cluster service system, the unavailability of one or a small number of dispatchers, or the unavailability of the dispatcher for the client's initial connection will not cause the client to fail to connect to the search service.

(2)本发明在client和dispatcher,searcher-server之间加入多级故障重试机制。若dispatcher无法服务,client会依次将所有缓存在client本地可用的多个dispatcher服务依次故障重试。如果searcher-server出现故障,dispatcher会按照router信息,将包含该collection的所有replica-searcher server依次故障重试。这种两级故障重试机制,确保了在发生服务器软硬件故障的条件下,只要有一丝可能,搜索服务就能正常对外使用,提高了用户的满意度和平台整体可靠观感。(2) The present invention adds a multi-level fault retry mechanism between the client, dispatcher, and searcher-server. If the dispatcher fails to serve, the client will sequentially fail and retry all the multiple dispatcher services cached locally and available on the client. If the searcher-server fails, the dispatcher will fail and retry all replica-searcher servers containing the collection in turn according to the router information. This two-level fault retry mechanism ensures that in the event of server software and hardware failures, as long as there is a slight possibility, the search service can be used normally, which improves user satisfaction and the overall reliability of the platform.

(3)本发明在router应用中的路由表信息内加入collection,replica集群的访问的策略控制,无需客户端进行任何修改,即可按照不同的collection,结合业务实际需求进行类似AB测试,及流量切分的搜索系统集群服务的管理。尤其对于搜索系统来说,不同搜索规则在线上并行发布使用,但对应的是同名的搜索collection的业务场景,是非常具有使用价值的;当发生服务器机房升级换代,搜索服务器的硬件维修替换时,能将线上的搜索流量切分到备用的搜索集群上,当原集群完成升级换代及硬件维护后,能立即平滑地完成两个集群的切换。(3) The present invention adds collection and replica cluster access policy control in the routing table information in the router application, without any modification on the client side, it can perform similar AB tests and flow according to different collections in combination with actual business needs Sharded search system cluster service management. Especially for the search system, different search rules are released and used in parallel online, but they correspond to the business scenario of the search collection with the same name, which is very useful; when the server room is upgraded and the hardware of the search server is repaired and replaced, It can split the online search traffic to the standby search cluster. When the original cluster is upgraded and hardware maintenance is completed, the switch between the two clusters can be smoothly completed immediately.

(4)本发明结合router的路由表,能将原来并不能同时对外一起提供服务的replica-search server组织起来,构建成一个虚拟的搜索集群,同时还能支持搜索系统服务管理的数据权限分权要求,实现在测试版上的搜索服务管理集群和正式应用的集群能进行权限隔离,大大方便了搜索平台的日常管理。(4) In combination with the routing table of the router, the present invention can organize the replica-search servers that cannot provide services together at the same time to form a virtual search cluster, and can also support the decentralization of data permissions for search system service management It is required that the search service management cluster on the beta version and the official application cluster can be separated in authority, which greatly facilitates the daily management of the search platform.

附图说明Description of drawings

图1为本发明实施例的系统架构图;Fig. 1 is a system architecture diagram of an embodiment of the present invention;

图2为本发明实施例的路由表系统结构图;Fig. 2 is a routing table system structural diagram of an embodiment of the present invention;

图3为本发明实施例的基于路由表的搜索系统请求集群服务方法的流程图;FIG. 3 is a flowchart of a method for requesting a cluster service by a routing table-based search system according to an embodiment of the present invention;

图4为本发明实施例的搜索流量集群分发方法的流程图。FIG. 4 is a flowchart of a search traffic cluster distribution method according to an embodiment of the present invention.

具体实施方式detailed description

为了使本发明的目的,技术方案和优点更加清楚,下面结合附图和具体实施例对本发明继续详细描述。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

如图1所示,为本发明实施例的系统结构。为了描述方便,我们假设整个搜索系统只包含了两个搜索索引,分别为Collection_1 107,Collection_2,这两个搜索索引即为搜索系统对外提供的搜索服务,这两个搜索所分别分布在三个Cluster 105中,每个Cluster中包含了2个Replica-Server 106。搜索Client端的Dispatcher_Route_List 102中存放了服务端3个Dispatcher 103的列表信息。每个Cluster中的的所有Replica-Server 106和包含其中的索引Collection 107的相关信息都会注册到Router 104中。Router 104模块是整个系统架构中的核心模块,它负责管理所有搜索集群的业务信息。Dispatcher 103则通过和Router 104之间的通讯来获取和感知搜索集群的业务状态和分布,以将来自Client的搜索请求转发到对应的Replica Server 106上。As shown in FIG. 1, it is the system structure of the embodiment of the present invention. For the convenience of description, we assume that the entire search system only includes two search indexes, namely Collection_1 107 and Collection_2. These two search indexes are the search services provided by the search system. These two search stations are respectively distributed in three Clusters. In 105, each Cluster contains 2 Replica-Server 106. Search Dispatcher_Route_List 102 on the client side for storing the list information of three Dispatchers 103 on the server side. Relevant information about all Replica-Servers 106 in each Cluster and the Index Collection 107 contained therein will be registered in Router 104 . The Router 104 module is the core module in the entire system architecture, and it is responsible for managing the business information of all search clusters. The Dispatcher 103 obtains and senses the service status and distribution of the search cluster through communication with the Router 104, so as to forward the search request from the Client to the corresponding Replica Server 106.

如图2所示,为本发明实施例的路由表结构图。路由表是存放在Router 104中的。它实际分为集群Key 201、2集群流量比202、搜索索引名称203、是否启用流量分配开关204、Replica-Server ID 205、Replica-Server的实际IP和端口206。As shown in FIG. 2 , it is a structural diagram of a routing table in an embodiment of the present invention. The routing table is stored in Router 104. It is actually divided into cluster key 201, 2 cluster traffic ratio 202, search index name 203, whether to enable traffic distribution switch 204, Replica-Server ID 205, the actual IP and port 206 of Replica-Server.

图2中举例了说明了路由表里的数据样式了,当然为了简便说明,数据样式做了简化。Cluster_A 207表示了存放在路由表中一条集群信息,路由表中的数据是由集群Key201和搜索索引名称203组合成为唯一主键,即相同的Cluster中不能有同名搜索索引。路由表中还包含了Replica—Server的具体信息209和210,209表示的是一个集群key和一个索引名称组成的主键可对应多个Replica-Server ID 205和Replica-Server的实际IP和端口206。Figure 2 exemplifies the data style in the routing table. Of course, for the sake of simplicity, the data style is simplified. Cluster_A 207 represents a piece of cluster information stored in the routing table. The data in the routing table is composed of the cluster Key 201 and the search index name 203 to form a unique primary key, that is, no search index with the same name can exist in the same Cluster. The routing table also includes specific information 209 and 210 of the Replica-Server, and 209 indicates that a primary key composed of a cluster key and an index name can correspond to multiple Replica-Server IDs 205 and the actual IP and port 206 of the Replica-Server.

图中208所表示的collection_2表示该业务对应的集群是具备了流量分配功能,分配的百分比为Cluster_B和Cluster_C各占50%。The collection_2 represented by 208 in the figure indicates that the cluster corresponding to the service has the traffic distribution function, and the distribution percentage is 50% for Cluster_B and 50% for Cluster_C.

如图3所示,为本发明实施例的基于路由表系统搜索请求集群服务方法的流程图,具体包括:As shown in Figure 3, it is a flowchart of a routing table system search request cluster service method based on an embodiment of the present invention, specifically including:

步骤301:Client发出搜索请求。Client端为整个搜索系统的服务客户端,负责向服务端发出搜索服务请求,接收搜索结果。Step 301: the client sends out a search request. The client is the service client of the entire search system, responsible for sending search service requests to the server and receiving search results.

步骤302:Client发出搜索请求时会首先判断是否已经配置了可连接的Dispatcher地址。Client不直接和搜索的实际服务器Replica-Search Server 106发生关联,所以必须在Client配置可以连接的Dispatcher地址Step 302: When the Client sends out a search request, it first judges whether a connectable Dispatcher address has been configured. Client is not directly associated with the actual search server Replica-Search Server 106, so the Dispatcher address that can be connected must be configured on Client

步骤303:如果没有配置Dispatcher地址,则Client直接抛出异常,表示无法进行搜索服务连接。搜索请求异常终止。Step 303: If the Dispatcher address is not configured, the Client directly throws an exception, indicating that the connection to the search service cannot be made. The search request terminated abnormally.

步骤304:如果已配置Dispatcher地址,则会与该地址进行连接,发出搜索服务请求,判断该Dispatcher是否能正常对外服务Step 304: If the Dispatcher address has been configured, it will connect to this address, send a search service request, and determine whether the Dispatcher can normally serve externally

步骤305:一旦成功建立连接后,假设该Client是第一次与Dispatcher进行通讯,则会将该Dispatcher所连接的Router上的所有Dispatcher信息全部获取,并传回客户端本地进行持久化,持久化以文件方式存储。Step 305: Once the connection is successfully established, assuming that the Client communicates with the Dispatcher for the first time, all the Dispatcher information on the Router to which the Dispatcher is connected will be obtained and sent back to the client for local persistence. Stored as a file.

步骤306:Dispatcher收到Client发过来的有效搜索查询请求后,会在Router上进行路由检索,获取所有相关该查询Collection名称的所有路由表信息Step 306: After the Dispatcher receives the valid search query request from the Client, it will search the route on the Router to obtain all the routing table information related to the query Collection name

步骤307:Dispatcher会解析Client发过来的搜索请求中是否明确指定了需要连接的Cluster信息。如果明确指明了目标Cluster信息,则在该Cluster的所有Replica-Server信息中选择一个可用的Replica-Server转发搜索请求。选择Cluster中的Replica-Server的方法有多种,有按照分配的负荷权重进行分配,也可以按照查询请求的条件进行Hash后,选择将特定Hash数值的请求转发给特定的Server。Step 307: The Dispatcher will analyze whether the search request sent by the Client clearly specifies the information of the Cluster to be connected. If the target Cluster information is clearly specified, an available Replica-Server is selected from all the Replica-Server information of the Cluster to forward the search request. There are many ways to select the Replica-Server in the Cluster, such as assigning according to the assigned load weight, or performing Hash according to the conditions of the query request, and choosing to forward the request of a specific Hash value to a specific Server.

步骤308:如果没有指定目标的Cluster信息,则会随机从当前可用的Cluster中选择一个(也可以采用的规则是选择最新创建的Cluster作为目标Cluster),然后在该Cluster中按照步骤307的算法选择一个Replica-Server进行搜索服务请求转发Step 308: If there is no specified target Cluster information, one will be randomly selected from the currently available Clusters (the rule that can also be adopted is to select the newly created Cluster as the target Cluster), and then select according to the algorithm of step 307 in the Cluster A Replica-Server forwards search service requests

步骤309:Replica-Server进行搜索实际查询服务执行,因为搜索应用是一个高CPU和高内存消耗的应用,且由于硬件资源的条件限制,搜索应用可能不一定能够得到需要的搜索结果,可能出现搜索实例超时或者内存溢出等异常情况Step 309: Replica-Server searches for the actual query service execution, because the search application is an application with high CPU and high memory consumption, and due to the limitation of hardware resources, the search application may not necessarily be able to obtain the required search results, and there may be a search Abnormal situations such as instance timeout or memory overflow

步骤310:如果出现搜索得到结果失败的情况,则表示当前所执行搜索服务请求的Replica-Server出现故障,需要更换Cluster中的其他可用Replica进行重试Step 310: If the result of the search fails, it means that the Replica-Server currently executing the search service request is faulty, and it is necessary to replace other available Replicas in the Cluster to try again

步骤311:如果所有当前Cluster下的Replica都尝试过出现无法服务的故障,则Dispatcher会将异常信息抛出给客户端Client,Client会放弃当前的Dispatcher,切换到本地所缓存的其他Dispatcher地址,进行重新搜索服务请求。重复步骤304-310Step 311: If all the Replicas under the current Cluster have tried to fail to service, the Dispatcher will throw an exception message to the client Client, and the Client will abandon the current Dispatcher, switch to other Dispatcher addresses cached locally, and perform Re-search the service request. Repeat steps 304-310

步骤312:如果replica执行成功,得到搜索请求的结果,则返回给client,表示一个完正的搜索请求服务流程完毕。Step 312: If the replica executes successfully and obtains the result of the search request, it returns to the client, indicating that a complete search request service process is completed.

如图4所示,为本发明实施例的搜索流量集群分发方法的流程图,具体包括As shown in FIG. 4 , it is a flowchart of a search traffic cluster distribution method according to an embodiment of the present invention, specifically including

步骤401:类似图3所描述的搜索请求服务流程的起点,Client发出搜索服务请求。Step 401: Similar to the starting point of the search request service process described in FIG. 3, the Client sends a search service request.

步骤402:类似图3部分的流程此处进行简化,不做具体描述。当Dispatcher收到Client发出的搜索请求后,会检索Router上的相关路由管理信息,当判断发现如果该次搜索请求需要进行流量集群分发,即将调用同一个搜索业务的请求按照用户特性进行流量切分,切分的相关百分比配置已经在Router的路由表信息中进行了约定。Step 402: The flow similar to the part in Fig. 3 is simplified here, and no specific description is given. When the Dispatcher receives the search request from the Client, it will retrieve the relevant routing management information on the Router. When it is judged that if the search request requires traffic cluster distribution, it will call the same search service request to perform traffic segmentation according to user characteristics. , the relevant percentage configuration of segmentation has been agreed in the routing table information of the Router.

步骤403:计算Client传入的搜索服务请求中包含的用户特征信息,一般来说,采用的用户端Cookie信息中包含的UUID组成的一个数值型字串。Step 403: Calculate the user characteristic information included in the search service request sent by the Client. Generally speaking, a numeric string composed of UUID included in the client Cookie information is used.

步骤404:针对用户的特征值然后进行取模计算,再结合一致性哈希的算法,进行不同的比例的权重计算,最后得出该次用户的搜索请求服务应该分配的目标对应Cluster结果。Step 404: Carry out modulo calculations for the user's feature values, and then combine the consistent hash algorithm to perform weight calculations of different proportions, and finally obtain the corresponding Cluster result of the target that the user's search request service should be allocated.

步骤405:在对应的Cluster包含的Replica中选择一个Replica作为本次搜索请求服务的执行方。选择的算法类似于图3中的步骤307Step 405: Select a Replica among the Replicas included in the corresponding Cluster as the executor of the search request service. The selected algorithm is similar to step 307 in Figure 3

步骤406:搜索服务端得到搜索结果,返回给Dispatcher,再传递给Client结果。如果出现搜索异常,则抛出异常给Client。此处重试的机制在图3的步骤中已经做过描述,此处不再赘述。Step 406: The search server obtains the search results, returns them to the Dispatcher, and then delivers the results to the Client. If a search exception occurs, an exception is thrown to the Client. The retry mechanism here has been described in the steps in FIG. 3 and will not be repeated here.

本发明还可有其他多种实施方式,在不违背离本发明精神及其实质的情况下,熟悉本领域的技术人员可根据本发明做出相应的改变和变形,这些改变和变形都应属于本发明所附的权利要求的保护范围。The present invention can also have other multiple implementation modes, without departing from the spirit and essence of the present invention, those skilled in the art can make corresponding changes and deformations according to the present invention, and these changes and deformations should belong to The scope of protection of the appended claims of the present invention.

Claims (9)

1.一种基于路由表的搜索系统集群服务管理的方法,其特征在于,通过对搜索服务器进行逻辑集群划分,将划分所得的集群信息存在路由表模块中,并构建多个对外提供集群服务的搜索应用请求转发器,这些请求转发器能统一从路由表信息中获取集群服务的分布,并面向搜索客户端提供服务结果应答和搜索请求转发;搜索客户端client提出的搜索服务的索引数据被复制多份,分配在不同的搜索服务应用中;通过搜索路由表系统记录搜索服务在搜索服务应用的分配情况,同时记录所有搜索服务应用和由他们组合而成的搜索集群;搜索请求分发系统接收客户端发出的搜索服务请求,根据请求内容查询路由表系统router中关于本次搜索服务的路由信息,将搜索请求转发给相应的搜索服务应用replica-searchserver,得到结果后再把结果回发给搜索客户端,具体步骤包括:1. A method for cluster service management of a search system based on a routing table, characterized in that, by dividing the search server into logical clusters, the cluster information obtained by the division is stored in the routing table module, and multiple external cluster services are constructed. Search application request forwarders, these request forwarders can uniformly obtain the distribution of cluster services from the routing table information, and provide service result responses and search request forwarding for search clients; the index data of search services proposed by the search client client is copied Multiple copies, allocated in different search service applications; record the distribution of search services in search service applications through the search routing table system, and record all search service applications and search clusters composed of them; the search request distribution system receives customers According to the search service request sent by the terminal, query the routing information about the search service in the routing table system router according to the request content, forward the search request to the corresponding search service application replica-searchserver, and send the result back to the search client after getting the result end, the specific steps include: 步骤1:在搜索服务器上构建多个能备份搜索服务信息的搜索服务应用(replica-searh Server),并将实际的搜索服务(collection)分配在这些搜索服务器(replica-Searh Server)上;Step 1: Build multiple search service applications (replica-searh Server) that can back up search service information on the search server, and distribute the actual search service (collection) on these search servers (replica-Searh Server); 步骤2:若干个搜索服务应用(replica-Searh Server)集合,组成一个搜索系统集群(cluster);Step 2: Several search service applications (replica-Searh Server) are assembled to form a search system cluster (cluster); 步骤3:在路由表系统(router)中注册、记录及管理所有搜索系统集群内的搜索服务分布的信息;Step 3: Register, record and manage information about the distribution of search services in all search system clusters in the routing table system (router); 步骤4:配置与搜索客户端(client)指定连接的搜索请求分发系统,当客户端发出搜索请求时,能直接与搜索请求分发系统Dispatcher通信;当客户端与该指定连接的搜索请求分发系统成功建立通信后,客户端同时从路由表系统中自动获取其他可用的搜索请求分发系统的地址信息;Step 4: Configure the search request distribution system connected to the search client (client), when the client sends a search request, it can directly communicate with the search request distribution system Dispatcher; when the client successfully connects to the search request distribution system After the communication is established, the client automatically obtains the address information of other available search request distribution systems from the routing table system at the same time; 所述搜索服务信息会复制多个备份,并被分配到不同的搜索服务应用replica-searchserver中;所述搜索路由表系统router包含搜索服务应用replica-search server和其内搜索服务的属性信息及分布信息;所述搜索请求转发系统dispatcher接收客户端发出的搜索服务请求,并根据请求中的信息,通过根据在router查询所需搜索服务的地址信息,将搜索服务请求转发至相应的搜索服务集群及其内的搜索服务应用,获取到结果后经由搜索请求转发系统dispatcher返回给客户端;The search service information will be copied into multiple backups and distributed to different search service applications replica-searchserver; the search routing table system router includes the search service application replica-search server and the attribute information and distribution of the search service within it information; the search request forwarding system dispatcher receives the search service request sent by the client, and according to the information in the request, forwards the search service request to the corresponding search service cluster and the address information of the required search service according to the router query The search service application in it returns the results to the client via the dispatcher of the search request forwarding system after obtaining the results; 步骤5:搜索请求分发系统(dispatcher)收到来自搜索客户端Client的搜索 服务请求后,检索路由表系统(router)内有关该搜索服务的搜索集群配置信息;Step 5: After the search request distribution system (dispatcher) receives the search service request from the search client Client, it retrieves the search cluster configuration information about the search service in the routing table system (router); 所述搜索服务请求中包含cookie信息、指定的搜索集群、搜索服务名称,在没有启用搜索流量集群分发策略时,dispatcher将根据搜索服务请求中的信息决定搜索服务请求转发至哪一个搜索集群内;The search service request includes cookie information, a specified search cluster, and a search service name. When the search traffic cluster distribution strategy is not enabled, the dispatcher will determine which search cluster to forward the search service request to according to the information in the search service request; 步骤6:根据本次搜索请求中搜索服务在router内的集群配置信息,包括搜索服务对应的集群名称、搜索服务名称、是否启用搜索流量集群分发策略、集群内的搜索服务器名称及地址信息,进而确定dispatcher将client的搜索服务请求转发具体至哪一个集群和集群内的搜索服务应用replica-search server进行搜索服务;Step 6: According to the cluster configuration information of the search service in the router in this search request, including the cluster name corresponding to the search service, the search service name, whether to enable the search traffic cluster distribution policy, the name and address information of the search server in the cluster, and then Determine which cluster the dispatcher forwards the client's search service request to and the search service application within the cluster uses replica-search server for search service; 步骤7:搜索服务应用(replica-search server)根据收到的搜索请求,得到搜索服务结果,并将该结果返回给client端。Step 7: The search service application (replica-search server) obtains the search service result according to the received search request, and returns the result to the client. 2.根据权利要求1所述的一种基于路由表的搜索系统集群服务管理的方法,其特征在于,所述步骤5中,搜索客户端client通过初始化配置可用的搜索请求分发dispatcher访问信息,初始化连接第一个可用的搜索请求分发系统dispatcher后,进入步骤5-1;2. the method for a kind of search system cluster service management based on routing table according to claim 1, it is characterized in that, in described step 5, search client client distributes dispatcher access information by initialization configuration available search request, initialization After connecting to the first available search request distribution system dispatcher, go to step 5-1; 步骤5-1:搜索客户端client成功连接dispatcher后,获取与该dispatcher相连的Router内其他所有dispatcher的信息,并持久化存储在客户端本地,作为与dispatcher连接的备用信息;Step 5-1: After the client successfully connects to the dispatcher, obtain the information of all other dispatchers in the Router connected to the dispatcher, and store them locally in the client as backup information for connecting to the dispatcher; 步骤5-2:client发送搜索请求给已建立连接的dispatcher,搜索请求中包含用户的cookie信息、是否指定搜索集群cluster、所请求的搜索服务名称。搜索请求中的信息被作为dispatcher执行路由检索的参考信息;Step 5-2: The client sends a search request to the dispatcher that has established a connection. The search request includes the user's cookie information, whether to specify a search cluster cluster, and the name of the requested search service. The information in the search request is used as the reference information for the dispatcher to perform routing retrieval; 步骤5-3:dispatcher收到来自客户端client的请求信息,并根据请求信息检查router内是否有请求中的搜索服务的地址信息和已指定的搜索集群,如果上述两个信息都没有被找到,抛出异常,通知客户端client搜索失败;如果查询到相关信息,转步骤4;Step 5-3: The dispatcher receives the request information from the client, and checks whether there is the address information of the search service in the request and the specified search cluster in the router according to the request information. If neither of the above two information is found, An exception is thrown to notify the client that the search failed; if relevant information is found, go to step 4; 步骤5-4:搜索请求分发系统dispatcher在Router中已查询到客户端请求的搜索服务,但若客户端请求指定的集群信息为空或找不到对应的集群,且发现同名的搜索服务分布在其他多个集群中,搜索请求分发系统dispatcher则任意挑选一个集群作为目标集群,将搜索请求发送至该集群内包含该搜索服务的搜索服务 应用replica-search server,搜索服务应用成功返回结果,由搜索请求分发系统传给客户端;Step 5-4: The search request distribution system dispatcher has found the search service requested by the client in the Router, but if the cluster information specified by the client request is empty or the corresponding cluster cannot be found, and the search service with the same name is found to be distributed in Among other multiple clusters, the dispatcher of the search request distribution system randomly selects a cluster as the target cluster, and sends the search request to the search service application replica-search server that contains the search service in the cluster. The search service application returns results successfully, and the search service The request distribution system is passed to the client; 步骤5-5:若在步骤5-4中,由搜索请求分发系统dispatcher发送至搜索服务应用replica-search server的搜索请求出错或者无响应时,dispatcher将继续在Router中查找下一个可用的搜索服务应用replica-search server;若继续出错,进而导致尝试次数达到设置的最大限制或者超过服务端超时时间设置,则搜索请求分发系统dispatcher直接抛出搜索异常信息给客户端client;Step 5-5: If in step 5-4, the search request sent by the search request distribution system dispatcher to the search service application replica-search server has an error or no response, the dispatcher will continue to search for the next available search service in the Router Apply replica-search server; if errors continue, resulting in the number of attempts reaching the set maximum limit or exceeding the server timeout setting, the search request distribution system dispatcher will directly throw search exception information to the client client; 步骤5-6:当客户端收到步骤5-5中抛出的异常时,客户端client自动在已持久化本地的搜索请求分发系统dispatcher信息列表中获取另一个可用dispatcher,并连接;重复步骤2-步骤6,直至dispatcher信息列表中记录的dispatcher都被尝试与客户端连接,若dispatcher还抛出搜索异常或者超过客户端超时时间设置,则客户端抛出异常,本次搜索请求失败;Step 5-6: When the client receives the exception thrown in step 5-5, the client client automatically obtains another available dispatcher from the dispatcher information list of the persistent local search request distribution system, and connects; repeat steps 2-Step 6, until the dispatcher recorded in the dispatcher information list is tried to connect with the client, if the dispatcher still throws a search exception or exceeds the client timeout setting, the client throws an exception, and this search request fails; 步骤5-7:若在步骤5-2中,客户端请求的搜索服务在路由表系统router中被标识为启用“搜索流量集群分配”策略,dispatcher根据搜索请求中的用户cookie信息,经计算后,与路由表系统router中已设置的各个集群的权重值比照,经比照一致的搜索集群将被选择为搜索请求的发送方,该搜索集群内包含该搜索服务的搜索服务应用replica-searchserver将作为接收本次搜索请求的目标搜索服务应用,搜索服务应用成功返回搜索结果,由搜索请求分发系统传给客户端。Step 5-7: If in step 5-2, the search service requested by the client is marked as enabling the "search traffic cluster distribution" policy in the routing table system router, the dispatcher calculates the search service based on the user cookie information in the search request , compared with the weight values of each cluster already set in the routing table system router, the search cluster that agrees with the comparison will be selected as the sender of the search request, and the search service application replica-searchserver that includes the search service in the search cluster will serve as The target search service application that receives this search request, the search service application successfully returns the search results, which are transmitted to the client by the search request distribution system. 3.根据权利要求1所述的一种基于路由表的搜索系统集群服务管理的方法,其特征在于,所述步骤4中还包括客户端对dispatcher的缓存,具体步骤有:3. the method for a kind of search system cluster service management based on routing table according to claim 1, it is characterized in that, also comprise the buffering of client to dispatcher in the described step 4, concrete steps have: 步骤4-1:客户端client首次成功连接指定的dispatcher后,将自动从路由表系统(Router)中远程获取当前所有可用的dispatcher列表信息,并通过文件存储Dispatcher信息实现在本地持久化缓存;Step 4-1: After the client successfully connects to the specified dispatcher for the first time, it will automatically remotely obtain all currently available dispatcher list information from the routing table system (Router), and store the Dispatcher information in a file to achieve local persistent caching; 步骤4-2:客户端client拥有Dispatcher列表信息后,当配置的Dispatcher地址不可用,客户端调用本地缓存的且已在router中注册的dispatcher列表,并自动从列表中依次选择其他dispatcher进行连接,直至连接成功;Step 4-2: After the client has the Dispatcher list information, when the configured Dispatcher address is unavailable, the client invokes the dispatcher list cached locally and registered in the router, and automatically selects other dispatchers from the list to connect. until the connection is successful; 步骤4-3:当本地缓存中所有记录的dispatcher都尝试连接后,若仍存在连接异常,则搜索客户端抛出异常,本次搜索请求失败。Step 4-3: After all dispatchers recorded in the local cache try to connect, if there is still a connection exception, the search client throws an exception, and the search request fails this time. 4.根据权利要求1所述的一种基于路由表的搜索系统集群服务管理的方法,其 特征在于,步骤6中启用所述的搜索流量集群分发策略,具体步骤包括:4. the method for a kind of search system cluster service management based on routing table according to claim 1, it is characterized in that, enable described search traffic cluster distribution strategy in step 6, concrete steps comprise: 步骤6-1:在router存放的路由表信息中会通过一个字段来记录是否启用搜索集群流量分发;步骤6-2:当标示为启用搜索流量集群分发时,Dispatcher调用本次搜索业务请求,并按照用户特性进行流量切分;Step 6-1: In the routing table information stored in the router, a field will be used to record whether to enable search cluster traffic distribution; Step 6-2: When it is marked as enabling search traffic cluster distribution, Dispatcher calls this search service request, and Segment traffic according to user characteristics; 其中,搜索业务对应的多个集群的流量占比由人为设定,存储在路由表系统Router中,所有集群的流量占比总和为100%;Among them, the traffic proportions of multiple clusters corresponding to the search business are set manually and stored in the routing table system Router, and the sum of the traffic proportions of all clusters is 100%; 步骤6-3:执行搜索流量集群分发时,依据client端所发搜索请求中的用户特征信息,利用一致性哈希算法计算一个权重值;Step 6-3: When performing cluster distribution of search traffic, calculate a weight value using the consistent hash algorithm according to the user characteristic information in the search request sent by the client; 步骤6-4:根据步骤6-3得到的权重值,通过对照router中该搜索业务对应的所有集群的权重值,确认搜索请求转发的目的集群cluster。Step 6-4: According to the weight value obtained in step 6-3, by comparing the weight values of all clusters corresponding to the search service in the router, confirm the destination cluster for forwarding the search request. 5.一种基于路由表的搜索系统集群服务管理的系统,其特征在于,系统包括搜索请求客户端client、搜索请求分发系统dispatcher、搜索路由表系统router、搜索服务应用replica-search server;所述四个部分之间都通过网络进行通信,各自有不同的服务端口,按照一定的规则匹配一起完成搜索服务;5. A system of routing table-based search system cluster service management, characterized in that the system includes a search request client client, a search request distribution system dispatcher, a search routing table system router, and a search service application replica-search server; The four parts communicate through the network, each has a different service port, and they match together to complete the search service according to certain rules; 搜索路由表系统router负责所有搜索服务应用replica-search server的登记注册及管理,并负责记录所有搜索服务应用replica-search server的属性信息及所有搜索服务在各搜索服务应用replica-search server内的分布情况;The search routing table system router is responsible for the registration and management of all search service application replica-search servers, and is responsible for recording the attribute information of all search service application replica-search servers and the distribution of all search services in each search service application replica-search server Condition; 搜索请求分发系统dispatcher在搜索客户端client和搜索服务应用replica-searchserver之间承担代理和转发角色,负责接收搜索请求客户端client的搜索请求,并根据搜索路由表系统router中的搜索服务路由信息、以及搜索服务和搜索集群的配置信息,对搜索请求进行不同策略的转发;The search request distribution system dispatcher assumes the role of proxy and forwarding between the search client client and the search service application replica-searchserver, and is responsible for receiving the search request from the search request client client, and according to the search service routing information in the search routing table system router, As well as the configuration information of the search service and the search cluster, different strategies are used to forward search requests; 所有搜索服务应用划分为不同的搜索集群cluster,关于“每个搜索集群内的搜索服务应用的分布”、“每个搜索服务应用包含的搜索服务”等涉及到搜索集群cluster、搜索服务应用及搜索服务三者的对应关系,均被记录在搜索路由表系统router中;All search service applications are divided into different search cluster clusters. Regarding "distribution of search service applications in each search cluster" and "search services included in each search service application", it involves search cluster clusters, search service applications, and search The corresponding relationship between the three services is recorded in the search routing table system router; 所述搜索请求客户端client,是搜索服务调用请求的发起人,是搜索服务的直接使用人;客户端的初始dispatcher的连接地址由手工配置,当搜索客户端client成功与初始搜索请求分发系统dispatcher建立通讯关系,搜索客户端client将获取到其他有用的dispatcher地址信息;The search request client client is the initiator of the search service call request and the direct user of the search service; the connection address of the initial dispatcher of the client is manually configured, when the search client client successfully establishes with the initial search request distribution system dispatcher Communication relationship, search client client will get other useful dispatcher address information; 客户端从搜索路由表系统router内获取其他可用的搜索请求分发系统dispatcher地址信息,并不缓存关于搜索服务的详细完整的路由信息;客户端不需要知道服务端具体的搜索服务节点的分布情况,只需要知道当前所有可用的搜索服务转发代理dispatcher;The client obtains other available search request distribution system dispatcher address information from the search routing table system router, and does not cache detailed and complete routing information about the search service; the client does not need to know the specific distribution of search service nodes on the server. You only need to know all currently available search service forwarding agent dispatchers; 所述搜索请求分发系统dispatcher,是路由信息的实际执行者,能根据搜索路由表系统router内记录的集群分布信息及分布策略,执行搜索服务请求的转发和调度;The search request distribution system dispatcher is the actual executor of routing information, and can perform forwarding and scheduling of search service requests according to the cluster distribution information and distribution strategies recorded in the search routing table system router; 所述搜索服务应用replica-search server,是具备自我注册能力的搜索服务端,它包含搜索服务的索引数据,负责完成搜索服务的索引查询匹配;多个搜索服务应用构成搜索集群cluster,搜索服务应用replica-search server的集群通过自注册加入的方式成立,当第一个replica-search server启动时,它需要在router内配置地址,同时在router中增加一个初始配置信息,并在router配置信息中加入一个唯一标识id号,这个唯一标识id号会被当作一个新的搜索集群id号记录在搜索路由表系统router中;如果再有其他的搜索服务应用replica-search server想要加入已存在的搜索集群中,以现有的搜索集群id号配置在该搜索服务应用在搜索路由表系统router中;The search service application replica-search server is a search server end with self-registration capability, it contains index data of search service, and is responsible for completing index query matching of search service; multiple search service applications form a search cluster cluster, and search service application The cluster of replica-search server is established through self-registration and joining. When the first replica-search server starts, it needs to configure the address in the router. At the same time, add an initial configuration information to the router and add it to the router configuration information. A unique identification id number, this unique identification id number will be recorded as a new search cluster id number in the search routing table system router; if there are other search service applications replica-search server want to join the existing search In the cluster, configure the search service with the existing search cluster id number in the search routing table system router; 任意一个replica-search server只能从属于一个cluster,而任何一个cluster中,都不允许出现同名的replica-search server的集合collection。Any one replica-search server can only belong to one cluster, and any collection of replica-search servers with the same name is not allowed to appear in any cluster. 6.根据权利要求5所述的一种基于路由表的搜索系统集群服务管理的系统,其特征在于,所述搜索客户端client:负责发送搜索请求,搜索请求中包含搜索客户端的用户特征信息,以及本次搜索服务对应的搜索服务信息(collection);6. The system for cluster service management of a search system based on a routing table according to claim 5, wherein the search client client is responsible for sending a search request, and the search request includes user characteristic information of the search client, And the search service information (collection) corresponding to this search service; 所述搜索请求分发系统dispatcher:即搜索路由信息的执行者,与搜索客户端直接通讯;同时负责从搜索路由表模块router中获取本次搜索业务在搜索服务应用内分布情况;The search request distribution system dispatcher: that is, the executor of the search routing information, directly communicates with the search client; at the same time, it is responsible for obtaining the distribution of the search service in the search service application from the search routing table module router; 所述搜索路由表系统router:用于管理所有处于启动状态的搜索服务应用,记录各个搜索服务应用的属性信息,以及每个搜索服务在搜索服务应用内的实际分布;同时用于将不同搜索服务应用集成为不同的集群,记录所有集群的属性信息及集群内的搜索服务应用分布情况;当有新的搜索服务应用启动时,需要及时在搜索搜索路由表系统router内注册和更新搜索业务分布信息;The search routing table system router: used to manage all search service applications in the startup state, record the attribute information of each search service application, and the actual distribution of each search service in the search service application; The application is integrated into different clusters, and the attribute information of all clusters and the distribution of search service applications in the cluster are recorded; when a new search service application is started, it is necessary to register and update the search service distribution information in the search routing table system router in a timely manner ; 所述搜索服务应用replica-search server:搜索服务应用存放实际的搜索服务的索引信息,若干个搜索服务应用可以组成一个搜索系统集群cluster,每一个搜索服务应用只能从属于一个集群cluster。The search service application replica-search server: the search service application stores the index information of the actual search service, and several search service applications can form a search system cluster cluster, and each search service application can only belong to one cluster cluster. 7.根据权利要求书5所述的一种基于路由表的搜索系统集群服务管理的系统,其特征在于,关于“每个搜索集群内的搜索服务应用的分布”、“每个搜索服务应用包含的搜索服务”等涉及到搜索集群cluster、搜索服务应用及搜索服务三者的对应关系,均被记录在搜索路由表系统router中。7. A routing table-based search system cluster service management system according to claim 5, characterized in that, regarding "the distribution of search service applications in each search cluster" and "each search service application contains The corresponding relationship among the search cluster, search service application and search service” is recorded in the search routing table system router. 8.根据权利要求书5所述的一种基于路由表的搜索系统集群服务管理的系统,其特征在于,所述搜索请求分发系统dispatcher并不具备搜索服务的能力,也不需要大量的计算和存储能力;本发明中的搜索请求分发系统dispatcher能支持高并发的大量的用户访问,能通过增加搜索服务应用replica-search server实现扩展整个搜索系统的服务能力时,而不是增加太多的dispatcher模块。8. A routing table-based search system cluster service management system according to claim 5, wherein the search request distribution system dispatcher does not have the ability to search for services, and does not require a large amount of calculation and Storage capacity; the search request distribution system dispatcher in the present invention can support a large number of high-concurrency user visits, and can expand the service capability of the entire search system by increasing the search service application replica-search server instead of adding too many dispatcher modules . 9.根据权利要求5所述的一种基于路由表的搜索系统集群服务管理的系统,其特征在于,所述搜索服务应用集合而成的集群采用自注册方式组合,当第一个搜索服务应用启动时,在搜索路由表系统router配置地址信息,同时标识唯一的集群标识ID;当其他搜索服务应用欲加入已有的搜索集群中,以现有的集群标识ID作为该搜索服务应用在router中的地址信息。9. A routing table-based search system cluster service management system according to claim 5, characterized in that, the cluster formed by the collection of search service applications is combined by self-registration, when the first search service application When starting, configure the address information in the search routing table system router, and identify a unique cluster ID at the same time; when other search service applications want to join the existing search cluster, use the existing cluster ID as the search service application in the router address information.
CN201610835665.3A 2016-09-20 2016-09-20 A kind of method and system of the search system cluster service management based on routing table Active CN106407011B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610835665.3A CN106407011B (en) 2016-09-20 2016-09-20 A kind of method and system of the search system cluster service management based on routing table

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610835665.3A CN106407011B (en) 2016-09-20 2016-09-20 A kind of method and system of the search system cluster service management based on routing table

Publications (2)

Publication Number Publication Date
CN106407011A true CN106407011A (en) 2017-02-15
CN106407011B CN106407011B (en) 2019-05-10

Family

ID=57997771

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610835665.3A Active CN106407011B (en) 2016-09-20 2016-09-20 A kind of method and system of the search system cluster service management based on routing table

Country Status (1)

Country Link
CN (1) CN106407011B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107770066A (en) * 2017-10-20 2018-03-06 成都精灵云科技有限公司 It is a kind of across main frame, travelling across VLAN, the Docker container methods of river diversion across cluster
CN109190004A (en) * 2018-08-30 2019-01-11 焦点科技股份有限公司 A method of search complexity is reduced to cope with excess load searching request based on specific policy
CN109302345A (en) * 2017-07-25 2019-02-01 迈普通信技术股份有限公司 FIB routing table issues time test system, method and apparatus
CN109474535A (en) * 2017-09-07 2019-03-15 北京京东尚科信息技术有限公司 The method, apparatus and system shunted for multi- search engine
CN109792406A (en) * 2018-07-27 2019-05-21 袁振南 Message delivery method, device and storage medium in server cluster
CN110618885A (en) * 2018-06-20 2019-12-27 北京金风慧能技术有限公司 System high availability method, device and system
CN111586079A (en) * 2019-02-15 2020-08-25 阿里巴巴集团控股有限公司 Communication method, system, control terminal, controlled terminal, device and storage medium
CN112653625A (en) * 2019-10-10 2021-04-13 北京京东尚科信息技术有限公司 Message routing method and device
CN113391759A (en) * 2020-03-13 2021-09-14 华为技术有限公司 Communication method and device
CN113495921A (en) * 2020-04-02 2021-10-12 北京京东振世信息技术有限公司 Routing method and device of database cluster
CN114070889A (en) * 2021-11-10 2022-02-18 北京百度网讯科技有限公司 Configuration method, traffic forwarding method, device, storage medium, and program product
CN115087968A (en) * 2020-03-24 2022-09-20 深圳市欢太科技有限公司 Data transmission method and device
CN117370497A (en) * 2023-10-18 2024-01-09 交通运输部规划研究院 Traffic planning statistics data linkage retrieval system and method based on multiple servers

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101685456A (en) * 2008-09-26 2010-03-31 华为技术有限公司 Search method, system and device
CN101930441A (en) * 2009-06-25 2010-12-29 李冰 Method for establishing Internet holographic search service
CN104754065A (en) * 2015-04-28 2015-07-01 湖南科技大学 Dynamic distribution Web resource management method and system based on content center network
CN105743980A (en) * 2016-02-03 2016-07-06 上海理工大学 Constructing method of self-organized cloud resource sharing distributed peer-to-peer network model

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101685456A (en) * 2008-09-26 2010-03-31 华为技术有限公司 Search method, system and device
CN101930441A (en) * 2009-06-25 2010-12-29 李冰 Method for establishing Internet holographic search service
CN104754065A (en) * 2015-04-28 2015-07-01 湖南科技大学 Dynamic distribution Web resource management method and system based on content center network
CN105743980A (en) * 2016-02-03 2016-07-06 上海理工大学 Constructing method of self-organized cloud resource sharing distributed peer-to-peer network model

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
余鑫: ""集群路由器关键技术研究"", 《中国博士学位论文全文数据库 信息科技辑》 *
刘昶言: ""云计算中混合云互通及其资源管理机制研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109302345B (en) * 2017-07-25 2022-01-25 迈普通信技术股份有限公司 System, method and device for testing issuing time of FIB routing table
CN109302345A (en) * 2017-07-25 2019-02-01 迈普通信技术股份有限公司 FIB routing table issues time test system, method and apparatus
CN109474535A (en) * 2017-09-07 2019-03-15 北京京东尚科信息技术有限公司 The method, apparatus and system shunted for multi- search engine
CN107770066B (en) * 2017-10-20 2020-06-02 成都精灵云科技有限公司 Cross-host, cross-VLAN and cross-cluster Docker container diversion method
CN107770066A (en) * 2017-10-20 2018-03-06 成都精灵云科技有限公司 It is a kind of across main frame, travelling across VLAN, the Docker container methods of river diversion across cluster
CN110618885A (en) * 2018-06-20 2019-12-27 北京金风慧能技术有限公司 System high availability method, device and system
CN110618885B (en) * 2018-06-20 2023-06-02 北京金风慧能技术有限公司 System high availability method, device and system
CN109792406A (en) * 2018-07-27 2019-05-21 袁振南 Message delivery method, device and storage medium in server cluster
CN109190004B (en) * 2018-08-30 2020-07-07 焦点科技股份有限公司 Method for reducing search complexity based on specific strategy
CN109190004A (en) * 2018-08-30 2019-01-11 焦点科技股份有限公司 A method of search complexity is reduced to cope with excess load searching request based on specific policy
CN111586079B (en) * 2019-02-15 2022-04-12 阿里巴巴集团控股有限公司 Communication method, system, control terminal, controlled terminal, device and storage medium
CN111586079A (en) * 2019-02-15 2020-08-25 阿里巴巴集团控股有限公司 Communication method, system, control terminal, controlled terminal, device and storage medium
CN112653625A (en) * 2019-10-10 2021-04-13 北京京东尚科信息技术有限公司 Message routing method and device
CN113391759A (en) * 2020-03-13 2021-09-14 华为技术有限公司 Communication method and device
CN113391759B (en) * 2020-03-13 2024-04-09 华为云计算技术有限公司 Communication method and equipment
CN115087968A (en) * 2020-03-24 2022-09-20 深圳市欢太科技有限公司 Data transmission method and device
CN113495921A (en) * 2020-04-02 2021-10-12 北京京东振世信息技术有限公司 Routing method and device of database cluster
CN113495921B (en) * 2020-04-02 2023-09-26 北京京东振世信息技术有限公司 Routing method and device for database cluster
CN114070889A (en) * 2021-11-10 2022-02-18 北京百度网讯科技有限公司 Configuration method, traffic forwarding method, device, storage medium, and program product
CN114070889B (en) * 2021-11-10 2023-11-14 北京百度网讯科技有限公司 Configuration method, traffic forwarding device, storage medium, and program product
CN117370497A (en) * 2023-10-18 2024-01-09 交通运输部规划研究院 Traffic planning statistics data linkage retrieval system and method based on multiple servers
CN117370497B (en) * 2023-10-18 2025-04-22 交通运输部规划研究院 Traffic planning statistics data linkage retrieval system and method based on multiple servers

Also Published As

Publication number Publication date
CN106407011B (en) 2019-05-10

Similar Documents

Publication Publication Date Title
CN106407011A (en) A routing table-based search system cluster service management method and system
CN112087312B (en) Method, device and equipment for providing edge service
JP3167893B2 (en) Method and apparatus for reducing network resource location traffic
JP7270755B2 (en) Metadata routing in distributed systems
US8195742B2 (en) Distributed client services based on execution of service attributes and data attributes by multiple nodes in resource groups
US10158579B2 (en) Resource silos at network-accessible services
US10922303B1 (en) Early detection of corrupt data partition exports
GB2632973A (en) Systems and methods for enabling a highly available managed failover service
CN100588172C (en) A system and method for realizing network reservation storage
US20120166394A1 (en) Distributed storage system and method for storing objects based on locations
CN111158949A (en) Disaster-tolerant architecture configuration method, switching method, device, device, and storage medium
US20240248810A1 (en) Systems and methods for cross-regional back up of distributed databases on a cloud service
US10862794B2 (en) Automated link aggregation group configuration system
US9760370B2 (en) Load balancing using predictable state partitioning
US9390156B2 (en) Distributed directory environment using clustered LDAP servers
CN113542373B (en) Route service discovery device and method for PAAS platform
CN113038394B (en) Session binding relationship processing method, apparatus, electronic device and readable medium
CN117131493A (en) Permission management system construction method, device, equipment and storage medium
WO2023100062A1 (en) Managing nodes of a dbms
US20210390082A1 (en) Distributed file system and method for accessing a file in such a system
JP4459999B2 (en) Non-stop service system using voting and information updating and providing method in the system
JP2003271440A (en) Contents delivery management system
US11757722B2 (en) Automatic switching fabric role determination system
JP6473425B2 (en) Node and data placement method
CN118921370A (en) Credit service processing system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant