CN112583864A

CN112583864A - Data migration method and device

Info

Publication number: CN112583864A
Application number: CN201910926762.7A
Authority: CN
Inventors: 廖锋; 余刚; 陈辉
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Group Guangdong Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Group Guangdong Co Ltd
Priority date: 2019-09-27
Filing date: 2019-09-27
Publication date: 2021-03-30
Anticipated expiration: 2039-09-27
Also published as: CN112583864B

Abstract

The embodiment of the invention provides a data migration method and a data migration device, wherein all routing data are stored and managed in a centralized manner by utilizing a pre-constructed routing service cluster, and in the data migration process, state information and node identification information in the routing data stored in the routing service cluster are updated in real time so as to quickly query routing data with high accuracy and high consistency from the routing service cluster; therefore, in the process of processing the routing service, the current state of the routing object can be inquired from the routing service cluster firstly, only the service of the routing object in the migration state can be influenced, but the routing service of the routing object in the normal state can not be influenced, so that the normal routing service can be provided for the user to the maximum extent, the user experience is improved, meanwhile, the data migration work can be automatically carried out in the process of processing the routing service, the data migration cost is reduced, and the data migration efficiency is improved.

Description

Data migration method and device

Technical Field

The present invention relates to the field of mobile communications technologies, and in particular, to a data migration method and apparatus.

Background

At present, with the rapid development of mobile communication technology, in a business support system (BOSS), a Business Data Service (BDS) is used as an important network element for storing user data, account data and the like, and a plurality of BDS nodes (i.e., business data service nodes) are deployed in different nodes based on a memory database, so as to provide high-speed data access service for systems such as pricing, accounting, information synchronous management and acquisition, and provide very high performance while ensuring data accuracy, so as to meet the real-time requirements of online charging and batch business. The number segment and the scattered number routing data of the user attributive BDS node are distributed in each module to be stored independently, the storage modes are various, and the BDS node comprises a memory bank, a physical bank, a cache and the like.

Currently, in the daily operation support activity of the BOSS, the performance of the existing BDS node is often insufficient to support the service requirement due to the increase of the user quantity and the improvement of the service complexity; therefore, the computing capacity of the BDS nodes is generally increased by means of newly-built BDS node capacity expansion, data equalization among existing BDS nodes, replacement of BDS nodes and the like, and data migration among BDS nodes is required in the process; the data migration process provided in the prior art mainly includes: operating the data migration process as an engineering project, establishing a cutting team, formulating a cutting scheme, and performing one-time large cutting if the related nodes are more than required; in addition, during the cutover period, the related system needs to interrupt service, namely, the BOSS application route is interrupted to access the user related data in the BDS node, so as to accurately perform a system operation of data export of the migrating BDS node (namely, a source data node), filtration of data to be migrated, data import of the migrating BDS node (namely, a target data node), data comparison, updating of routing data of each module, data cleaning of the migrating BDS node, service restart, test verification, business opening and the like; and after the cutting, service monitoring is required to be carried out for a period of time to check whether the processing is correct.

Therefore, the data migration method in the prior art has the problems of complex migration process, high migration cost and long consumed time, and in the data migration process, normal business service needs to be stopped, so that business response delay and complaint risk are increased, and user use experience is influenced.

Disclosure of Invention

The embodiment of the invention aims to provide a data migration method and a data migration device, and aims to solve the problems that the existing data migration method is complex in migration process, high in migration cost and long in consumed time, and in the data migration process, normal business service needs to be stopped, so that business response delay and complaint risk are increased.

In order to solve the above technical problem, the embodiment of the present invention is implemented as follows:

in a first aspect, an embodiment of the present invention provides a data migration method, including:

determining a data migration task to be processed;

updating state information in first routing data corresponding to the data migration task into a migration state, wherein the first routing data are stored in a pre-constructed routing service cluster, and routing data respectively corresponding to full numbers are stored in the routing service cluster;

migrating the user-related data aimed at by the data migration task from a source data node to a target data node;

and updating the state information in the first routing data to be in a normal state, and updating the node identification information in the first routing data to be the identification information of the target data node.

In a second aspect, an embodiment of the present invention provides a data migration apparatus, including:

the migration task determination module is used for determining a data migration task to be processed;

a first state updating module, configured to update state information in first routing data corresponding to the data migration task to a migration state, where the first routing data is stored in a pre-constructed routing service cluster, and routing data corresponding to full numbers are stored in the routing service cluster;

the data migration module is used for migrating the user-related data aimed at by the data migration task from a source data node to a target data node;

and the second state updating module is used for updating the state information in the first routing data to be in a normal state and updating the node identification information in the first routing data to be the identification information of the target data node.

In a third aspect, an embodiment of the present invention provides a computer device, including a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; the memory is used for storing a computer program; the processor is configured to execute the program stored in the memory, and implement the steps of the data migration method according to the first aspect.

In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the data migration method according to the first aspect.

According to the data migration method and device, all routing data are stored and managed in a centralized mode through the pre-established routing service cluster, and in the data migration process, state information and node identification information in the routing data stored in the routing service cluster are updated in real time, so that the routing data with high accuracy and high consistency can be inquired from the routing service cluster quickly; therefore, in the process of processing the routing service, the current state of the routing object can be inquired from the routing service cluster firstly, only the service of the routing object in the migration state can be influenced, but the routing service of the routing object in the normal state can not be influenced, so that the normal routing service can be provided for the user to the maximum extent, the user experience is improved, meanwhile, the data migration work can be automatically carried out in the process of processing the routing service, the data migration cost is reduced, and the data migration efficiency is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a first flowchart of a data migration method according to an embodiment of the present invention;

fig. 2 is a second flowchart of a data migration method according to an embodiment of the present invention;

fig. 3 is a third flowchart illustrating a data migration method according to an embodiment of the present invention;

fig. 4 is a schematic flowchart of a specific implementation of a data migration process according to an embodiment of the present invention;

fig. 5 is a schematic diagram illustrating a specific implementation principle of a data migration method according to an embodiment of the present invention;

fig. 6 is a schematic diagram illustrating an implementation principle of updating routing data in the data migration method according to the embodiment of the present invention;

FIG. 7 is a block diagram of a data migration apparatus according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

Detailed Description

In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The embodiment of the invention provides a data migration method and device, and the independent scheme is summarized, so that the embodiment of the invention realizes the purpose of ensuring that normal routing service is provided for users to the maximum extent, improves the user experience, simultaneously can ensure that the data migration work is automatically carried out in the routing service processing process, reduces the data migration cost and improves the data migration efficiency.

Fig. 1 is a first flowchart of a data migration method according to an embodiment of the present invention, where the method in fig. 1 can be executed by a business support system BOSS, as shown in fig. 1, the method at least includes the following steps:

s101, determining a data migration task to be processed, wherein the data migration task comprises: a migration object identifier, which may be a number identifier or a non-number identifier, such as a customer identifier, where the customer identifier corresponds to at least one number segment information;

specifically, the BDS node performance is increased to support the service requirement by increasing BDS nodes for capacity expansion or performing data equalization on the existing BDS nodes, and at this time, data migration is required, that is, data in one BDS node is migrated to another BDS node; the data migration method provided by the embodiment of the invention is suitable for the capacity expansion or data relocation scene in the BDS daily operation and maintenance, and specifically, the data migration method receives a data migration request input by a user in a foreground interface, generates a corresponding data migration task based on the data migration request, and adds the data migration task to a migration task queue, wherein a migration object targeted by the data migration task can be a single user number, namely, user-related data corresponding to the user number needs to be migrated, or a number segment, namely, user-related data corresponding to a plurality of user numbers contained in the number segment needs to be migrated.

S102, updating state information in first routing data corresponding to the data migration task into a migration state, wherein the first routing data are stored in a pre-constructed routing service cluster, and routing data respectively corresponding to full numbers are stored in the routing service cluster;

the routing service cluster is established in advance based on Redis, specifically, the number routing is used as a minimum unit, routing data is stored in a distributed mode in the routing service cluster, and the total number comprises: a single loose number and a plurality of loose numbers obtained by number section conversion; for example, a number in an interval of 13812340000-13812349999 is converted into a plurality of hash numbers, such as hash number 13812340001, each hash number corresponds to respective routing data, that is, a segment route is converted into a hash route, wherein the segment route refers to a route defining a segment number, and the hash route refers to a route defining a number, so that corresponding routing data can be located through a hash value of the number, and the routing data includes: status information and home node identification, etc.;

if the migration object targeted by the data migration task is at least one number segment, the first routing data includes: and the routing data of a plurality of hash numbers obtained by the number section conversion.

S103, migrating the user related data aimed at by the data migration task from a source data node to a target data node;

specifically, after the state information of the current number to be migrated is updated to the migration state, migration processing is performed on user-related data corresponding to the current number to be migrated, that is, the user-related data is migrated from one BDS node to another BDS node;

wherein the user-related data comprises: at least one item of three-user data, order relation data, balance data, free resource data, accumulated amount data and arrearage data; specifically, all entity records associated with the current number to be migrated are queried, and all the queried entity records are migrated to a target data node, wherein the number of base tables associated with one number in the BDS node can reach more than 50, and the BDS node stores user related data in a memory database form;

specifically, the user-related data in the memory database of the BDS node may be associated by a number, a client ID, an account ID, and a user ID, and a data tree is formed based on the identifier used for the query, and the types of the user-related data mainly include:

data of three households: associating based on the number, customer ID, account ID, user ID; subscription relationship data: associating based on the user ID; balance data: associating based on the account ID; free resource data: associating based on the user ID; cumulative amount data: associating based on the user ID; arrearage data: based on account ID association, associated subordinate node records can be inquired through some fields in the superior node records by combining the association relationship among data.

S104, updating the state information in the first routing data to a normal state, and updating the node identification information in the first routing data to the identification information of the target data node;

specifically, for a current number to be migrated, after data migration of user-related data of the number is completed, it is necessary to update a state information in routing data corresponding to the number from a migration state to a normal state, and update a home node identifier in the routing data corresponding to the number from an identifier of a source data node to an identifier of a destination data node;

specifically, in the process of data migration for a current migration number, if a routing service request is received, whether user-related data in a corresponding BDS node can be accessed can be known by querying state information in routing data corresponding to the routing service request in a routing service cluster; if the state information is in a transition state, the user related data in the corresponding BDS node is not allowed to be accessed until the state information is updated to be in a normal state; and if the state information is in a normal state, allowing access to the user related data in the corresponding BDS node.

In the embodiment provided by the invention, all routing data are centrally stored and managed by utilizing the pre-constructed routing service cluster, and in the data migration process, the state information and the node identification information in the routing data stored in the routing service cluster are updated in real time, so that the routing data with high accuracy and high consistency can be rapidly inquired from the routing service cluster; therefore, in the process of processing the routing service, the current state of the routing object can be inquired from the routing service cluster firstly, only the service of the routing object in the migration state can be influenced, but the routing service of the routing object in the normal state can not be influenced, so that the normal routing service can be provided for the user to the maximum extent, the user experience is improved, meanwhile, the data migration work can be automatically carried out in the process of processing the routing service, the data migration cost is reduced, and the data migration efficiency is improved.

Further, in the data migration process, the state information and the node identification information in the routing data stored in the routing service cluster are updated in real time, so that in the automatic data migration process between the nodes of the memory database, the routing service does not need to be stopped completely, and the routing service request can be responded without stopping the service based on the routing state information stored in the routing service cluster, and based on this, as shown in fig. 2, the migration method further includes:

s105, receiving a routing service request, wherein the routing service request carries a routing object identifier, and the routing object identifier includes: any one of number identification, customer identification, account identification and user identification;

the routing service types comprise: number routing and non-number routing, the non-number routing comprising: customer routing, account routing, user routing; for number routing service, the routing object identifier is a number identifier, and for non-number routing service, the routing object identifier is a customer identifier, an account identifier or a user identifier;

specifically, the basic model related in the telecommunication is a three-user model, the client ID is the unique identifier of a client table, and the information of a natural person is stored; the account ID is a unique identifier of an account table, basic information of a charge payment account corresponding to the current customer is stored, and a plurality of balance accounts are hung below the account ID; the user ID is the unique identification of the user table, i.e. one number corresponds to one user;

the number and the user ID are basically corresponding and belong to the minimum granularity, the difference between the number and the user ID is that the number can be repeatedly used, and the user ID is generally unique in the whole process; the account ID is of medium granularity; the client ID belongs to the maximum granularity; wherein, for an individual, a simple model with one-to-one correspondence of the three is adopted.

S106, in the routing service cluster, inquiring the state information in the second routing data corresponding to the routing object identifier;

considering that some routing services are processed by non-numbers, for example, for a printing bill service, a user-related data query is required according to an account ID; for non-number routing, pre-establishing corresponding relations between a client identifier, an account identifier and a user identifier and a main number respectively, searching for number routing corresponding to the non-number routing by using a two-stage routing model, and further determining state information and an attributive BDS node corresponding to the number routing, so that the non-number routing is converted into the number routing through the two-stage routing model;

in specific implementation, a primary routing table (i.e., primary and secondary routing mapping definition information) for non-number routing and a secondary routing table (i.e., primary routing key definition information) for number routing may be stored in the routing service cluster; specifically, the corresponding primary number is determined according to the account ID and based on the primary and secondary routing mapping definition information in the two-level routing model, and then the state information and the home node identifier are determined according to the primary number and based on the primary routing key value definition information in the two-level routing model.

In the process of processing the routing service, the BOSS application terminal inquires required routing data from the routing service cluster through a preset access interface, namely the routing service cluster provides high-performance routing access service through the preset access interface; in the data migration process, the BDS node updates the routing data in the routing service cluster through a preset updating interface; in addition, the routing service cluster also provides routing increment updating capability and maintenance capability.

S107, if the inquired state information is in a normal state, determining a home data node of the user related data corresponding to the route object identifier according to the node identifier information in the second route data;

specifically, if it is determined that the current state information of the route object to which the route service request is directed is a normal state, it indicates that the user-related data corresponding to the route object can be directly accessed currently, and the BDS node where the user-related data of the route object is located can be determined from the route service cluster;

s108, reading user related data corresponding to the routing object identification from the determined attribution data node;

the BDS node stores user related data by adopting a memory database, and is used for providing high-speed user data access to the outside, for example, for online charging service, a response is returned within 200 ms; specifically, after the user related data is obtained, based on the user related data, related services such as service pricing, rent calculation, recharging and payment are performed.

Further, if the routing object targeted by the routing service request is a current data migration object, at this time, the user-related data corresponding to the routing object cannot be directly accessed, and corresponding service processing needs to be performed in combination with the routing service type, based on which, the routing service request also carries the routing service type, where the routing service type includes: online charging service, tariff query service, etc.;

correspondingly, in the foregoing fig. 2, after querying, in the routing service cluster, the state information in the second routing data corresponding to the route object identifier at step S106, the method further includes:

s109, if the queried status information is a migration status, determining a service processing mode for the received routing service request according to the routing service type, where the service processing mode includes: any one of the business is released, failure prompt information is returned, and asynchronous processing flow is started; and the number of the first and second groups,

s110, based on the determined service processing mode, responding to the received routing service request.

For example, considering that the online charging service requires a response to be returned within 200ms, in order to ensure that the user service is continuously used, in a case that the type of the routing service is the online charging service, the service processing method is correspondingly to start an asynchronous processing flow, specifically, a successful response is directly returned to the client, and an offline charging method is adopted for processing, so that an emergency measure which does not affect service experience when a system fails can be realized.

In the embodiment provided by the invention, the automatic migration of data among the nodes of the memory database of continuous services is realized based on the two-stage routing service and the routing state information, namely under the condition of completely stopping the services, the automatic migration of the data among the nodes of the memory database is realized based on the two-stage routing service and the routing state information, in the process, only the service request related to the data being migrated is influenced, and the service request aiming at the routing object in a normal state can be normally responded; specifically, in the automatic data migration process, the processing mode of the front-end service is directed based on the state information of the current migration route, that is, when the data migration is achieved by updating the state of the route in real time, all services do not need to be stopped, only the service corresponding to the route key value currently being migrated is affected, and the service is immediately restored after the migration is completed, so that the services of all users are not stopped due to the data migration.

As shown in fig. 3, for the case of non-number routing, that is, the routing object identifier includes: the customer identifier, the account identifier, or the user identifier needs to convert a non-number route into a number route based on a two-level routing model, and then determines state information corresponding to the number route, based on which, the routing data of the full number is stored in a two-level routing model manner, where the two-level routing model includes: the primary and secondary routing mapping defines a sub-model and the primary routing key value defines a sub-model;

correspondingly, in the routing service cluster, the step S106 of querying the state information in the second routing data corresponding to the route object identifier specifically includes:

s1061, defining a sub-model based on primary and secondary route mapping, and determining a target number identifier corresponding to the route object identifier; defining a sub-model based on primary and secondary route mapping, and determining a corresponding primary key value type and a primary key value according to a secondary key value;

wherein, the primary and secondary route mapping definition submodel includes: the method comprises the steps of storing a corresponding relation between a primary key value and a secondary key value in a key value pair mode, wherein the primary key value is a number mark and the secondary key value is a non-number mark, or the primary key value is a non-number mark and the secondary key value is a number mark; specifically, the corresponding relation between the non-number identification and the number identification is determined according to the corresponding relation between the secondary key value and the primary key value;

in specific implementation, the primary and secondary routing map definition submodels may be represented in the form of:

s1062, defining a sub-model based on the main routing key value, and determining state information corresponding to the target number identifier; defining a sub-model based on a main routing key value, and determining corresponding state information according to a main key value type and the main key value;

wherein, the main routing key value definition sub-model includes: the corresponding relation between the primary key value stored in the form of key value pair and the route attribute information includes: at least one of status information, home data node, and data update time; specifically, for the case that the primary key value is a number identifier and the secondary key value is a non-number identifier, determining the route attribute information corresponding to the destination number identifier according to the corresponding relationship between the primary key value and the route attribute information;

in a specific implementation, the representation form of the primary routing key value definition sub-model may be:

correspondingly, in step S107, if the queried status information is in a normal status, determining, according to the node identifier information in the second routing data, the home data node of the user-related data corresponding to the route object identifier, specifically including:

s1071, defining a sub-model based on the main routing key value, and determining a target home node identifier corresponding to the target number identifier;

specifically, the main routing key value defines a sub-model, and when the value of the main key value is the target number identifier, the corresponding home node ID is the target home node identifier;

s1072, the target home node identification is determined as the home data node of the user related data corresponding to the routing object identification.

In the embodiment provided by the invention, the corresponding routing data can be inquired directly based on the main routing key value, or the corresponding main routing key value can be determined based on the auxiliary routing key value, and then the corresponding routing data can be inquired based on the main routing key value, and a matched routing data inquiry mode can be flexibly selected in combination with a specific service scene in the actual application process, wherein the corresponding routing data can be inquired directly based on the main routing key value aiming at the service request of the minimum granularity routing; for the service request of non-number routing, the corresponding relation between the customer identification and the main number identification, the corresponding relation between the account identification and the main number identification, or the corresponding relation between the customer identification and the main number identification can be known through the main-auxiliary routing mapping definition sub-model in the two-stage routing model, so as to realize the association between the non-number routing and the number routing (namely, the non-number routing is converted into the number routing); and then defining a sub-model by a main routing key value in the two-stage routing model, so that the routing attribute information corresponding to the main number identifier can be known, and further the BDS node identifier can be obtained.

The values of the relevant fields in the primary and secondary route mapping definition submodels and the primary route key value definition submodel are described as follows:

the values of the primary and secondary identifications can be: 1-primary key route, 2-secondary key route; the routing type is as follows: the type of the current route is represented, and the values thereof can be: 1-BDS routing, and subsequently, other routing types can be expanded according to actual conditions; (Primary) Key value type: the data classification representing the current primary key value or the secondary key value may be: 1-number, 2-customer ID, 3-account ID, 4-user ID;

state information: the current state of the number is represented, and the values thereof may be: 0-unassigned, 1-normal, 2-migratory; home node ID: representing the node identification of the attribution under the current routing type; update time represents: the last update time of current number routing is the number of seconds, a string of digits, since 1 month 1 of 1970.

In a specific embodiment, if the primary key value type is a number identifier and the secondary key value type is an account ID, the field values in the primary and secondary route mapping definition submodels are:

Key:21320087654321

Value:1,13812345678,1513728555

wherein, 2 in 213 represents primary and secondary identification (i.e. secondary route), "1" represents route type (i.e. BDS route), "3" represents key value type (i.e. account ID), and "20087654321" represents secondary key value (i.e. value of account ID); "13812345678" indicates the primary key value (i.e. the value of the number), and "1513728555" indicates the update time;

correspondingly, the field value in the main routing key value definition submodel is as follows:

Key:11113812345678

Value:1,2987,1513728000

wherein "1" in "111" represents a primary and secondary identifier (i.e. a primary route), "1" represents a route type (i.e. a BDS route), "1" represents a key value type (i.e. a number), and "13812345678" represents a primary key value (i.e. a value of the number); "2987" indicates status information, and "1513728000" indicates the update time.

Considering that one non-number identifier may correspond to a plurality of number identifiers, in order to ensure that a unique main route can be queried through a secondary route and that all user-related data corresponding to the non-number route can be read from a determined home node, based on this, a secondary key value type in a sub-model is defined as the non-number identifier for primary and secondary route mapping, and a primary key value type in the sub-model is defined as the number identifier for the primary route key value; correspondingly, before determining the data migration task to be processed in S101, the method further includes:

determining a primary number identifier corresponding to a non-number identifier assigned by a secondary key value in a primary and secondary route mapping definition sub-model;

step two, setting the value of the primary key value in the primary routing key value definition sub-model as a primary number identification; and the number of the first and second groups,

and step three, storing the user related data corresponding to the non-number identification in the attribution data node corresponding to the main number identification.

Specifically, if one client ID corresponds to multiple numbers, only the route of the primary number corresponding to the client ID may be configured, and it is necessary to ensure that all the user-related data of the client ID can be found in the BDS node where the primary number is located, so all the user-related data corresponding to the client ID need to be stored in the BDS node.

For the data migration process, if the migration object targeted by the data migration task includes: at least one number segment information; the number segment information corresponds to a plurality of numbers, the number is required to be firstly taken as a minimum migration unit to split the data migration task to obtain a plurality of data migration subtasks, and then data migration is performed on each data migration subtask in sequence, based on this, after determining the data migration task to be processed, the method further includes:

generating a data migration subtask corresponding to each user number in the at least one number segment information;

correspondingly, the step S102 of updating the state information in the first routing data corresponding to the data migration task to the migration state specifically includes:

sequentially selecting at least one data migration subtask from the plurality of data migration subtasks as a current processing object; if the source data nodes corresponding to the data migration subtasks are different, the data migration subtasks can be processed concurrently;

and updating the state information in the first routing data corresponding to the user number aimed at by the current processing object to be in a transition state.

Wherein, for a user with a single account, that is, a client corresponds to one account, but corresponds to multiple users, that is, corresponds to multiple numbers, at this time, it is necessary to migrate user-related data of the associated number to the target data node together, and based on this, after generating a data migration subtask corresponding to the user number for each user number in the at least one number segment information, the method further includes:

judging whether the data migration subtasks to be processed contain the data migration subtasks established aiming at the associated numbers;

if the judgment result is yes, continuing to execute the step of sequentially selecting at least one data migration subtask as the current processing object from the plurality of data migration subtasks;

and if the judgment result is negative, generating a data migration subtask corresponding to the associated number, and adding the data migration subtask to be processed.

In specific implementation, as shown in fig. 4, a specific implementation flow diagram of the data migration process is given, specifically:

s401, reading a data migration task to be processed currently;

s402, updating the read state of the data migration task into processing, and updating the starting time of the data migration task;

s403, sequentially selecting a current task detail to be processed from at least one task detail under the data migration task to be processed; wherein, one migration task may correspond to multiple task detail definitions, for example, migrating multiple number segments at a time;

s404, updating the state of the selected task detail into processing, and updating the starting time of the task detail;

s405, sequentially selecting at least one current subtask to be processed from a plurality of subtasks under the selected task detail; each task detail definition can be further refined into a plurality of subtasks, for example, a number segment can be split into a plurality of subtasks according to the number.

S406, updating the state of the selected subtask into processing, and updating the starting time of the subtask;

s407, updating the state information in the routing data corresponding to the selected subtask in the routing service cluster into a migration state;

s408, migrating the user related data aimed at by the currently selected subtask from the source data node to the target data node;

s409, updating the state information in the routing data corresponding to the selected subtask in the routing service cluster to be in a normal state, and updating the identification information in the routing data to be the target data node identification;

s410, updating the state of the currently selected subtask to be successfully processed, and updating the end time of the subtask;

s411, judging whether the currently selected subtask is the last subtask in the current processed task detail;

if not, returning to execute the step S405;

if yes, executing S412, updating the state of the currently processed task detail to processing completion, and updating the end time of the task detail; and the number of the first and second groups,

s413, judging whether the currently selected task detail is the last task detail under the currently processed data migration task;

if not, returning to execute the step S403;

if yes, executing S414, updating the read state of the data migration task to be processing completion, and updating the end time of the data migration task; and returning to execute the step S401.

In the embodiment of the invention, each migration processing starts with an unprocessed migration task, all unprocessed or in-process task details under the migration task are read, for each task detail, each primary key route is firstly divided into a plurality of subtasks, if the primary key route has a subtask in the current migration task, the primary key route is skipped, and then each unprocessed subtask is processed.

In one embodiment, as shown in fig. 5, the BOSS system includes: the system comprises a routing service cluster, an online migration processing end, a plurality of BOSS applications and a plurality of BDS nodes (data storage is carried out in the form of an in-memory database);

(A) the specific process for managing the migration task comprises the following steps:

A1. visual migration task definition, migration monitoring, migration task cancellation and the like are provided through a foreground interface, and more efficient automatic completion of data migration operation is achieved;

specifically, a certain data migration task can be started, suspended or cancelled through a foreground interface; the execution condition of the migration task can be monitored through a foreground interface; log management such as retrieval and export of the migration log can be performed through a foreground interface;

wherein, one migration task may correspond to multiple migration task detail definitions, for example, migrating multiple number segments at a time; each migration task detail definition can refine a plurality of migration subtasks, for example, one number segment can be split into a plurality of migration subtasks according to the number; in foreground interface operation or background operation, log information needs to be registered for each migration subtask;

in a specific implementation, the migration log may include the following fields: at least one of a migration log ID, an operation type, an operator, an operation association ID, operation start time, operation end time, migration success quantity and migration failure quantity;

correspondingly, the migration task may include the following fields: at least one of a migration task ID, a task current state, creation time, a creator, planned migration start time, actual migration start time, and actual migration completion time;

correspondingly, the fields contained in the migration task detail may be: at least one item of migration task detail ID, migration task ID, key value type, start key value, end key value, state information, migration start time and migration end time;

correspondingly, the migration subtask may include the following fields: at least one of a migration subtask ID, a migration task detail ID, a primary key value, state information, an associated upper level subtask ID, migration start time, migration end time and an end code;

specifically, the values of the key fields are as follows:

the operation type values may be: 1-newly building a task, 2-newly building a task detail, 3-suspending a task, 4-suspending a task detail, 5-suspending a subtask, 6-canceling a task, 7-canceling a task detail, 8-canceling a subtask and 9-automatically migrating a task;

the values of the state information may be: 1-waiting for treatment, 2-treatment, 3-treatment success and 4-treatment failure;

the value of the key value type may be: 1-number, 2-customer ID, 3-account ID, 4-user ID.

(B) The specific process for online data migration is as follows:

B1. and the online migration processing end reads the migration task regularly, if the migration task is not split, the migration task is split into a plurality of subtasks according to the number, the subtasks are concurrently processed according to the configuration, in order to ensure that the data is not messy, each migration node can be configured, only one subtask is in processing, and the plurality of subtasks of different migration nodes can be concurrently processed.

B2. And at the online migration processing end, when the subtask is processed, the subtask is locked, then the number route is read, the state information of the number route is updated to be 'in migration', if the route cannot be found, the subtask is skipped, and the subtask state is updated to be 'no user'.

B3. And the online migration processing end reads the user related data from the migration BDS node, if the associated user exists, the related data of the associated user needs to be migrated together, specifically, a new lower-level subtask needs to be judged and registered first, and if the associated user exists, the new lower-level subtask needs to be skipped.

B4. And the online migration processing end writes the read user related data into the migration BDS node, and meanwhile, the related data of the added related users also needs to be written.

B5. And after the data migration is successful, invalidating or deleting the corresponding user related data in the migrated BDS node.

B6. And after the data migration is successful, updating the state information of the corresponding routing data in the routing service cluster to be normal.

B7. And after the data migration is successful, updating the state of the subtask to be 'completed', and generating a corresponding migration log aiming at the subtask.

(C) The specific process for the routing service application is as follows:

and C1, the BOSS application accesses the routing service cluster to inquire routing data corresponding to the routing service request to be processed before receiving the routing service request and accessing the service or data of the BDS node.

C2, BOSS application, based on the inquired routing data, responding to the routing service request; and if the state is available, accessing the corresponding BDS node according to the BDS node number in the routing data.

C3. If the state is in transition or can not be inquired, returning failure or release (for online charging information) according to the current service scene; therefore, only the service of the current transferred number can be influenced, the services aiming at other numbers can be normally processed, and the service can be immediately recovered after the transfer is finished because the data granularity of each transfer is number level and the transfer time is short (second level).

Correspondingly, as shown in fig. 6, a schematic diagram of an implementation principle of updating routing data in the data migration method is given, taking a current migration number of 1xxxxxxxx 1 as an example, specifically:

A) before data migration: normal access BDS1 node

After inquiring the routing access service from the routing service cluster, each BOSS application learns that the state of the routing number is normal, and then normally accesses the affiliated BDS1 node.

B) During data migration: not having access to BDS1 node

Under the condition of not stopping service, setting the state of the number segment to be migrated as in migration, and after the BOSS application inquires the routing access service from the routing service cluster, knowing that the state of the routing number is in migration, performing service release or asynchronous processing, for example, when CBE is charged online, returning to play back the pass error code.

C) After data migration: normal access BDS2 node

After the data migration is completed, the route home node of the route number is updated to be the BDS2 node, and the state corresponding to the route number is normal, at this time, after each BOSS application inquires the route access service from the route service cluster, it learns that the state of the route number is normal, and then normal access is performed on the home BDS2 node.

Therefore, the number state mark and the centralized routing query ensure that only the service of the number in the migration is influenced in the data migration process, and other numbers can be normally processed.

The data migration method in the embodiment of the invention utilizes the pre-constructed routing service cluster to intensively store and manage all routing data, and updates the state information and the node identification information in the routing data stored in the routing service cluster in real time in the data migration process so as to quickly inquire the routing data with high accuracy and high consistency from the routing service cluster; therefore, in the process of processing the routing service, the current state of the routing object can be inquired from the routing service cluster firstly, only the service of the routing object in the migration state can be influenced, but the routing service of the routing object in the normal state can not be influenced, so that the normal routing service can be provided for the user to the maximum extent, the user experience is improved, meanwhile, the data migration work can be automatically carried out in the process of processing the routing service, the data migration cost is reduced, and the data migration efficiency is improved.

Corresponding to the data migration method provided in the foregoing embodiment, based on the same technical concept, an embodiment of the present invention further provides a data migration apparatus, and fig. 7 is a schematic diagram illustrating a module composition of the data migration apparatus provided in the embodiment of the present invention, where the data migration apparatus is configured to execute the data migration method described in fig. 1 to fig. 6, and as shown in fig. 7, the data migration apparatus includes:

a migration task determining module 701, configured to determine a data migration task to be processed;

a first state updating module 702, configured to update state information in first routing data corresponding to the data migration task to a migration state, where the first routing data is stored in a pre-constructed routing service cluster, and the routing service cluster stores routing data corresponding to all numbers respectively;

a data migration module 703, configured to migrate the user-related data targeted by the data migration task from a source data node to a target data node;

a second state updating module 704, configured to update the state information in the first routing data to a normal state, and update the node identification information in the first routing data to the identification information of the target data node.

In the embodiment of the invention, all routing data are centrally stored and managed by utilizing the pre-constructed routing service cluster, and in the data migration process, the state information and the node identification information in the routing data stored in the routing service cluster are updated in real time, so that the routing data with high accuracy and high consistency can be quickly inquired from the routing service cluster; therefore, in the process of processing the routing service, the current state of the routing object can be inquired from the routing service cluster firstly, only the service of the routing object in the migration state can be influenced, but the routing service of the routing object in the normal state can not be influenced, so that the normal routing service can be provided for the user to the maximum extent, the user experience is improved, meanwhile, the data migration work can be automatically carried out in the process of processing the routing service, the data migration cost is reduced, and the data migration efficiency is improved.

Optionally, the apparatus further comprises:

a routing request receiving module, configured to receive a routing service request, where the routing service request carries a routing object identifier, and the routing object identifier includes: any one of number identification, customer identification, account identification and user identification;

a state information query module, configured to query, in the routing service cluster, state information in the second routing data corresponding to the route object identifier;

and the first request response module is used for determining a home data node of the user related data corresponding to the routing object identifier according to the node identifier information in the second routing data if the state information is in a normal state, and reading the user related data from the home data node.

Optionally, the routing service request further carries a routing service type; the above-mentioned device still includes: a second request response module to:

if the state information is a migration state, determining a service processing mode aiming at the routing service request according to the routing service type, wherein the service processing mode comprises the following steps: any one of the business is released, failure prompt information is returned, and asynchronous processing flow is started;

and responding to the routing service request based on the service processing mode.

Optionally, the routing object identifier includes: customer identification, account identification, or user identification, wherein the routing data of the full number is stored in a two-level routing model, and the two-level routing model comprises: the primary and secondary routing mapping defines a sub-model and the primary routing key value defines a sub-model;

correspondingly, the state information query module is specifically configured to:

determining a target number identifier corresponding to the route object identifier based on the primary and secondary route mapping definition submodels;

determining state information corresponding to the target number identification based on the main routing key value definition sub-model;

correspondingly, the first request response module is specifically configured to:

determining a target home node identifier corresponding to the target number identifier based on the main routing key value definition sub-model;

and determining the target home node identification as a home data node of the user related data corresponding to the routing object identification.

Optionally, the primary and secondary routing map definition submodel includes: the method comprises the steps of storing a corresponding relation between a secondary key value and a primary key value in a key value pair mode, wherein the primary key value is a number mark and the secondary key value is a non-number mark, or the primary key value is a non-number mark and the secondary key value is a number mark;

the main routing key value definition sub-model comprises: the method comprises the following steps of storing a corresponding relation between a primary key value and route attribute information in a key value pair mode, wherein the route attribute information comprises: at least one of status information, home data node, and data update time.

Optionally, the migration object for which the data migration task is directed includes: at least one number segment information; the above-mentioned device still includes: a migration task splitting module to:

correspondingly, the first state updating module 702 is specifically configured to:

sequentially selecting at least one data migration subtask from the plurality of data migration subtasks as a current processing object;

Optionally, the secondary key value type in the primary and secondary route mapping definition submodels is a non-number identifier, and the primary key value type in the primary route key value definition submodel is a number identifier; the above-mentioned device still includes:

a primary number determining module, configured to determine a primary number identifier corresponding to a non-number identifier assigned by a secondary key value in the primary and secondary route mapping definition submodels;

a primary key value setting module, configured to set a value of a primary key value in the primary routing key value definition sub-model as the primary number identifier; and the number of the first and second groups,

and the user data storage module is used for storing the user related data corresponding to the non-number identification in the attribution data node corresponding to the main number identification.

The data migration device in the embodiment of the invention utilizes the pre-constructed routing service cluster to intensively store and manage all routing data, and updates the state information and the node identification information in the routing data stored in the routing service cluster in real time in the data migration process so as to quickly inquire the routing data with high accuracy and high consistency from the routing service cluster; therefore, in the process of processing the routing service, the current state of the routing object can be inquired from the routing service cluster firstly, only the service of the routing object in the migration state can be influenced, but the routing service of the routing object in the normal state can not be influenced, so that the normal routing service can be provided for the user to the maximum extent, the user experience is improved, meanwhile, the data migration work can be automatically carried out in the process of processing the routing service, the data migration cost is reduced, and the data migration efficiency is improved.

The data migration apparatus provided in the embodiment of the present invention can implement each process in the embodiment corresponding to the data migration method, and is not described here again to avoid repetition.

It should be noted that the data migration apparatus provided in the embodiment of the present invention and the data migration method provided in the embodiment of the present invention are based on the same inventive concept, and therefore specific implementation of the embodiment may refer to implementation of the data migration method, and repeated details are not described herein.

Corresponding to the data migration method provided in the foregoing embodiment, based on the same technical concept, an embodiment of the present invention further provides a computer device, where the computer device is configured to execute the data migration method, and fig. 8 is a schematic structural diagram of a computer device implementing various embodiments of the present invention, as shown in fig. 8. Computer devices may vary widely in configuration or performance and may include one or more processors 801 and memory 802, where one or more stored applications or data may be stored in memory 802. Wherein the memory 802 may be a transient storage or a persistent storage. The application program stored in memory 802 may include one or more modules (not shown), each of which may include a series of computer-executable instructions for a computing device. Still further, the processor 801 may be configured to communicate with the memory 802 to execute a series of computer-executable instructions in the memory 802 on a computer device. The computer apparatus may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input-output interfaces 805, one or more keyboards 806.

In this embodiment, the computer device includes a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; a memory for storing a computer program; a processor for executing the program stored in the memory, implementing the following method steps:

determining a data migration task to be processed;

The computer equipment in the embodiment of the invention utilizes the pre-constructed routing service cluster to intensively store and manage all routing data, and updates the state information and the node identification information in the routing data stored in the routing service cluster in real time in the data migration process so as to quickly inquire the routing data with high accuracy and high consistency from the routing service cluster; therefore, in the process of processing the routing service, the current state of the routing object can be inquired from the routing service cluster firstly, only the service of the routing object in the migration state can be influenced, but the routing service of the routing object in the normal state can not be influenced, so that the normal routing service can be provided for the user to the maximum extent, the user experience is improved, meanwhile, the data migration work can be automatically carried out in the process of processing the routing service, the data migration cost is reduced, and the data migration efficiency is improved.

The computer device provided by the embodiment of the present invention can implement each process in the embodiment corresponding to the data migration method, and is not described herein again to avoid repetition.

It should be noted that the computer device provided in the embodiment of the present invention and the data migration method provided in the embodiment of the present invention are based on the same inventive concept, and therefore specific implementation of the embodiment may refer to implementation of the data migration method, and repeated details are not described herein.

An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the following method steps are implemented:

determining a data migration task to be processed;

The computer-readable storage medium in the embodiment of the invention utilizes the pre-constructed routing service cluster to perform centralized storage and management on all routing data, and in the data migration process, the state information and the node identification information in the routing data stored in the routing service cluster are updated in real time so as to quickly query the routing data with high accuracy and high consistency from the routing service cluster; therefore, in the process of processing the routing service, the current state of the routing object can be inquired from the routing service cluster firstly, only the service of the routing object in the migration state can be influenced, but the routing service of the routing object in the normal state can not be influenced, so that the normal routing service can be provided for the user to the maximum extent, the user experience is improved, meanwhile, the data migration work can be automatically carried out in the process of processing the routing service, the data migration cost is reduced, and the data migration efficiency is improved.

The computer-readable storage medium provided in the embodiment of the present invention can implement each process in the embodiment corresponding to the data migration method, and is not described herein again to avoid repetition.

It should be noted that the computer-readable storage medium provided in the embodiment of the present invention and the data migration method provided in the embodiment of the present invention are based on the same inventive concept, and therefore specific implementation of the embodiment may refer to implementation of the foregoing data migration method, and repeated details are not described here.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method of data migration, comprising:

determining a data migration task to be processed;

2. The method of claim 1, further comprising:

receiving a routing service request, wherein the routing service request carries a routing object identifier, and the routing object identifier includes: any one of number identification, customer identification, account identification and user identification;

in the routing service cluster, inquiring the state information in the second routing data corresponding to the routing object identifier;

and if the state information is in a normal state, determining a home data node of the user related data corresponding to the routing object identifier according to the node identifier information in the second routing data, and reading the user related data from the home data node.

3. The method of claim 2, wherein the routing service request further carries a routing service type;

after querying the state information in the second routing data corresponding to the route object identifier in the routing service cluster, the method further includes:

4. The method of claim 2, wherein the routing object identification comprises: customer identification, account identification, or user identification, wherein the routing data of the full number is stored in a two-level routing model, and the two-level routing model comprises: the primary and secondary routing mapping defines a sub-model and the primary routing key value defines a sub-model;

in the routing service cluster, querying state information in the second routing data corresponding to the routing object identifier includes:

correspondingly, the determining the home data node of the user-related data corresponding to the route object identifier according to the node identifier information in the second route data includes:

5. The method of claim 4, wherein,

the primary and secondary route mapping definition submodel comprises: the method comprises the steps of storing a corresponding relation between a secondary key value and a primary key value in a key value pair mode, wherein the primary key value is a number mark and the secondary key value is a non-number mark, or the primary key value is a non-number mark and the secondary key value is a number mark;

6. The method of claim 1, wherein the migration object for which the data migration task is directed comprises: at least one number segment information;

after determining the data migration task to be processed, the method further comprises the following steps:

correspondingly, the updating the state information in the first routing data corresponding to the data migration task to the migration state includes:

7. The method according to claim 4, wherein the primary and secondary routing maps define the secondary key value type in the sub-model as a non-number identity, and the primary routing key value type in the sub-model as a number identity;

before determining the data migration task to be processed, the method further comprises the following steps:

determining a primary number identifier corresponding to a non-number identifier assigned by a secondary key value in the primary and secondary route mapping definition submodels;

setting the value of the primary key value in the primary routing key value definition sub-model as the primary number identification; and the number of the first and second groups,

and storing the user related data corresponding to the non-number identification in the attribution data node corresponding to the main number identification.

8. A data migration apparatus, comprising:

9. A computer device comprising a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; the memory is used for storing a computer program; the processor is used for executing the program stored on the memory and realizing the data migration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the data migration method according to any one of claims 1 to 7.