WO2017012475A1

WO2017012475A1 - Data acquisition method and device

Info

Publication number: WO2017012475A1
Application number: PCT/CN2016/089307
Authority: WO
Inventors: 王晨纯
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2015-07-17
Filing date: 2016-07-08
Publication date: 2017-01-26
Also published as: CN106357539B; CN106357539A

Abstract

Disclosed are a data acquisition method and device. The method comprises: a routing device receiving a data acquisition request back to the source by a content distribution network (CDN) cache server, and the routing device searching for data corresponding to the data acquisition request based on the data acquisition request; and the routing device generating a feedback message based on a search result, and sending the feedback message to the CDN cache server so that the CDN cache server acquires the data based on the feedback message. In this way, by means of a routing device, a domain name back to the source of a plurality of storage centres is realized, thereby saving resources and improving the user experience.

Description

Data acquisition method and device

The present application claims priority to Chinese Patent Application Serial No. No. No. No. No. No. No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No

Technical field

The embodiments of the present invention relate to the field of communications technologies, and in particular, to a data acquiring method and device.

Background technique

The traditional CDN domain name can only be returned to a storage center, and the content returned by the source station is cached on the CDN node and transparently transmitted to the client. This also means that the domain name of a CDN can only correspond to the file of a single storage center. Storage facility.

For users who want to have a CDN domain name supporting multiple storage center proxy services at the same time, there is currently no solution. Users can only migrate all file data to a centralized storage center, and then configure the CDN's return source address to that. Centralized storage center to go. In this way, a period of access interruption occurs in the middle, which results in the service being unavailable, and also limits the scalability of the storage center. As the traffic volume increases, the storage capacity of the storage center is bound to be insufficient.

It can be seen that in the prior art, a CDN domain name can only be returned to one storage center, which results in multiple domain names when data resources are distributed in multiple storage centers, and cannot be merged into one domain name to access, resulting in high user cost. , affecting the user's experience.

Summary of the invention

In the prior art, a domain name can only be returned to a storage center, which causes the cost of using multiple storage centers to be too high, and affects the user experience. The present application proposes a data acquisition method, including:

The routing device receives the data acquisition request from the source of the CDN cache server,

The routing device searches for data corresponding to the data acquisition request based on the data acquisition request;

The routing device generates a feedback message based on the search result;

The routing device sends the feedback message to the CDN cache server to cause the CDN cache server to obtain the data based on the feedback message.

Optionally, before the routing device receives the data acquisition request of the CDN cache server, the method further includes:

Modifying a return source address of the CDN cache server to an address of the routing device, so that when the CDN cache server returns to the source, the routing device receives a data acquisition request from the CDN cache server.

Optionally, the data acquisition request includes a URL of the data, where the data is stored in a storage center.

The routing device searches for data corresponding to the data acquisition request based on the data acquisition request, and specifically includes:

Determining, by the routing device, a URI in the URL based on a URL in the data acquisition request;

The routing device performs a data search based on the URI to determine a storage center where the data is located;

The routing device extracts a domain name of a storage center where the data is located.

Optionally, a data index table is pre-stored in the routing device, where the index table stores a correspondence between each storage center and data stored in each storage center.

The routing device performs a data search based on the URI to determine a storage center where the data is located, and specifically includes:

The routing device performs a lookup in the index table based on the URI to determine a storage center where the data is located.

Optionally, the routing device stores, in advance, a correspondence between a domain name of each storage center and the data acquisition request.

Determining, by the routing device, each storage center corresponding to the data acquisition request according to a correspondence relationship;

The routing device looks up the data in various storage centers based on the URI to determine a storage center in which the data resides.

Optionally, the routing device generates a feedback message based on the search result, which specifically includes:

The routing device generates a second URL based on the extracted domain name and the URI as a feedback message.

The application also proposes a data acquisition method, including:

The CDN cache server returns the received data acquisition request to the routing device;

The CDN cache server receives a feedback message that is sent by the routing device based on the data acquisition request, where the feedback message includes an address where the data corresponding to the data acquisition request is located;

The CDN cache server acquires data corresponding to the data acquisition request based on the feedback message.

Optionally, the feedback message includes a URL of the data, where the data is stored in a storage center;

The acquiring, by the CDN cache server, the data corresponding to the data acquisition request, based on the feedback message, specifically includes:

Determining, by the CDN cache server, a domain name of the storage center where the data is located and a URI of the data based on a URL in the feedback message;

The CDN cache server acquires the data in the determined storage center according to the URI.

Optionally, after the CDN cache server acquires the data corresponding to the data acquisition request based on the feedback message, the method further includes:

The CDN cache server caches the data and sends the data to a user terminal corresponding to the data acquisition request.

The application also proposes a routing device, including:

a receiving module, configured to receive a data acquisition request from a source of the CDN cache server,

a searching module, configured to search for data corresponding to the data acquisition request based on the data acquisition request;

a generating module, configured to generate a feedback message based on the search result;

And a sending module, configured to send the feedback message to the CDN cache server, so that the CDN cache server acquires the data based on the feedback message.

Optionally, it also includes:

And a modifying module, configured to modify a return source address of the CDN cache server to an address of the routing device, so that when the CDN cache server returns to the source, the routing device receives a data acquisition request of the CDN cache server back to the source.

The searching module is specifically configured to:

The searching module performs a data search based on the URI to determine a storage center where the data is located, and specifically includes:

A lookup is performed in the index table based on the URI to determine a storage center where the data is located.

Determining, according to the correspondence relationship, each storage center corresponding to the data acquisition request;

The data is looked up in various storage centers based on the URI to determine a storage center in which the data resides.

Optionally, the generating module is specifically configured to:

A second URL is generated as a feedback message based on the extracted domain name and the URI.

The application also proposes a CDN cache server, comprising:

a source module, configured to return the received data acquisition request to the routing device;

a receiving module, configured to receive a feedback message that is sent by the routing device based on the data acquisition request, where the feedback message includes an address where the data corresponding to the data acquisition request is located;

And an obtaining module, configured to acquire data corresponding to the data acquisition request based on the feedback message.

The obtaining module is specifically configured to:

Determining, according to a URL in the feedback message, a domain name of a storage center where the data is located and a URI of the data;

The data is obtained in the determined storage center according to the URI.

Optionally, the CDN cache server further includes:

And a sending module, configured to cache the data, and send the data to a user terminal corresponding to the data obtaining request.

The application also discloses a data acquisition system, including:

a routing device, configured to receive a data acquisition request from a source of the CDN cache server, search for data corresponding to the data acquisition request based on the data acquisition request, generate a feedback message based on the search result, and send the feedback message to the a CDN cache server, such that the CDN cache server acquires the data based on the feedback message;

a CDN cache server, configured to: return a received data acquisition request to the routing device; and receive, by the routing device, a feedback message that is based on the data acquisition request, where the feedback message includes data corresponding to the data acquisition request. And the address corresponding to the data acquisition request is obtained based on the feedback message.

Compared with the prior art, in the present application, the data acquisition request of the CDN cache server is received by the device, and the data corresponding to the data acquisition request is searched based on the data acquisition request; and the feedback message is generated based on the search result. A domain name can be returned to multiple storage centers, which reduces the cost of use and improves the user experience.

DRAWINGS

FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present application;

2 is a schematic flowchart of a data acquisition method according to an embodiment of the present application;

FIG. 3 is a schematic flowchart diagram of a data acquisition method according to an embodiment of the present application;

4 is a schematic structural diagram of a routing device according to an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of a routing device according to an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of a CDN cache server according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a data acquisition system according to an embodiment of the present application.

detailed description

For the defects of the background art, the embodiment of the present application discloses a data acquisition method, which is applied to a system as shown in FIG. 1 including multiple storage centers, in which the CDN cache server will data from the user terminal. The request source is sent back to the routing device, and after the routing device finds the data, the feedback message generated by the storage center based on the data is fed back to the CDN cache server, so that the CDN cache server caches the data and returns the data to the user terminal, thereby reducing the user and the user. The workload of the CDN cache server improves the user experience.

The method is shown in Figure 2 and includes the following steps:

Step 201: The routing device receives a data acquisition request from a source of the CDN cache server.

Before the routing device receives the data acquisition request from the CDN (Content Distribution Network) cache server, the method further includes:

The source address of the CDN cache server is modified to the address of the routing device, so that when the CDN cache server returns to the source, the routing device receives the data acquisition request from the source of the CDN cache server.

Specifically, when the terminal where the user is located sends a data acquisition request to the CDN cache server, if the data corresponding to the data acquisition request is not in the cache of the CDN cache server, the corresponding data is obtained by the other device based on the data acquisition request, by default. The CDN cache server has a preset source address, that is, a preset source address. When the CDN cache server receives the data acquisition request and does not cache the required data, it will obtain the source back to the source station based on the data. Obtain the data in the medium. To this end, the source address of the CDN cache server is modified in this application. Specifically, it may be modified by the routing device, or may be modified by any other device or manual setting, and modified to be the address of the routing device, so that when the source needs to be returned, the CDN cache server will data. The obtaining request is sent to the routing device, and then the routing device searches for the corresponding data based on the data obtaining request; for example, the original source address of the CDN cache server is address 1 (ie, the source station address), and the address of the routing device is address 2. Originally, when the source is needed, the CDN cache server returns the data acquisition request to address 1, and after changing the source address from address 1 to address 2, the CDN cache server will request the data acquisition when it needs to return the source. Returning the source to address 2, that is, sending a data acquisition request to the routing device.

Step 202: The routing device searches for data corresponding to the data acquisition request based on the data acquisition request.

The data acquisition request includes a URL (Uniform Resource Locator), and the data is stored in the storage center.

Therefore, the routing device searches for data corresponding to the data acquisition request based on the data acquisition request, and specifically includes:

The routing device determines a URI (Uniform Resource Identifier) in the URL based on the URL in the data acquisition request; the routing device performs data search based on the URI to determine a storage center where the data is located; and the storage device where the routing device extracts the data domain name.

That is, the data is searched by the URI, and the specific search has the following two methods:

In a first method, a data index table is pre-stored in the routing device, and a correspondence relationship between each storage center and data stored in each storage center is stored in the index table.

The routing device performs data search based on the URI to determine the storage center where the data is located, and specifically includes: the routing device searches the index table based on the URI, and determines the storage center where the data is located.

Specifically, in an embodiment of the present application, the routing device stores an index table in advance, and the index table may be as shown in Table 1.

Table 1

存储中心Storage center	存储中心中存储的数据Data stored in the storage center
存储中心1Storage center 1	数据A，数据B，数据CData A, Data B, Data C
存储中心2Storage center 2	数据F，数据GData F, data G
存储中心3Storage center 3	数据JData J
······	······
存储中心nStorage center n	数据KData K

The data acquisition request includes a URL, and the URL includes the domain name and the URI. Therefore, after receiving the data acquisition request, the routing device extracts the URI, and the URI can determine the data to be searched, for example, to check If the data G is found, it can be found in the storage center 2 through the index table, thereby searching for the data and recording the domain name of the storage center 2.

In the second manner, the mapping between the domain name of each storage center and the data acquisition request is pre-stored in the routing device;

The routing device performs data search based on the URI to determine the storage center where the data is located, and specifically includes: the routing device determines each storage center corresponding to the data acquisition request based on the correspondence relationship; the routing device searches for data in each storage center based on the URI to determine the data. Storage center.

Step 203: The routing device generates a feedback message based on the search result.

The routing device generates a feedback message based on the search result, which specifically includes:

The routing setting generates a second URI as a feedback message based on the extracted domain name (ie, the domain name of the storage center where the data is located) and the URI.

Specifically, the storage center includes a data center and a source station as an example. If the data is found in the data center, the domain name of the data center (for example, www.ABCD.com) and the URI (for example, 1.jpg) are used. Generating a second URL as a feedback message, for example, may be www.ABCD.com/1.jpg as a feedback message, and if it is determined that the data is stored in the source station, similarly, the domain name of the source station is extracted, for example, www. EFC.com, and URI, for example 1.jpg, generate a second URL as a feedback message, for example, www.EFC.com/1.jpg as a feedback message.

Step 204: The routing device sends a feedback message to the CDN cache server, so that the CDN cache service acquires the data based on the feedback message.

In step 203, a feedback message is generated, and then the routing device sends the generated feedback message to the CDN cache server, so that the CDN cache server receives the feedback message (either the second URL or the third URL). The corresponding data is found based on the feedback message (either in the data center or at the source station), the data is cached, and the data is presented to the terminal where the user is located.

The above is based on the routing device side. The second embodiment of the present application also provides a data acquisition method in the CDN cache server, as shown in FIG. 3, including:

Step 301: The CDN cache server returns the received data acquisition request to the routing device.

Step 302: The CDN cache server receives a feedback message that is sent by the routing device based on the data acquisition request, where the feedback message includes an address where the data corresponding to the data acquisition request is located;

Step 303: The CDN cache server acquires data corresponding to the data acquisition request based on the feedback message.

Specifically, the feedback message includes a URL of the data, and the data is stored in the storage center;

The CDN cache server obtains the data corresponding to the data acquisition request based on the feedback message, and specifically includes:

The CDN cache server determines the domain name of the storage center where the data is located and the URI of the data based on the URL in the feedback message;

The CDN cache server fetches data in the determined storage center based on the URI.

After the CDN cache server obtains the data corresponding to the data acquisition request based on the feedback message, the method further includes:

The CDN cache server caches the data and sends the data to the user terminal corresponding to the data acquisition request.

In order to further illustrate the application, the second embodiment of the present application further provides a data acquisition method in a specific scenario, including:

In this embodiment, the user has a lot of picture data, part of which is stored in the data center (data center domain name: user-bucket.oss.aliyuncs.com), and part of it is stored in the user-defined source station (source domain name is In :image-source.somesite.com), the user visits www.somesitecdn.com (the CDN domain name of the image). Since the user wants to migrate the image from the source station to the data center, the user needs to fill in his source domain name. Image-source.somesite.com, and register his current CDN domain name www.somesitecdn.com, and the routing device stores the mapping relationship between www.somesitecdn.com and image-source.somesite.com;

When the user visits www.somesitecdn.com, it passes through the CDN cache server. Since the source address of the CDN cache server is modified into a routing device in advance, the CDN cache server returns the access request to the routing device, whereby the routing device is based on The access request looks up the data to determine if the data is stored in the data center or the source station.

Specifically, for example, if a picture abc.jpg has been uploaded to the data center, then when the user visits www.somesitecdn.com/abc.jpg, the routing device finds the picture in the data center, then returns X-SECOND-PROXY- PASS: user-bucket.oss.aliyuncs.com/abc.jpg Header to the CDN cache server, CDN cache server then go to user-bucket.oss.aliyuncs.com/abc.jpg to capture the image, cache the image and return to user.

There is also a picture def.jpg, not in the data center, on the user's custom source station, then when the user visits www.sitesitecdn.com/def.jpg, the routing device finds that the picture is not in the data center, then returns X- SECOND-PROXY-PASS: image-source.somesite.com/def.jpg Header to the CDN cache server, CDN cache server to image-source.somesite.com/def.jpg to capture the image, cache the image and return to user.

In this way, regardless of the number of source stations, the CDN cache server performs multiple times of returning through the routing device, so that the user actually sees and accesses the same CDN domain name, and realizes that a CDN domain name can be returned to multiple source stations.

The third embodiment of the present application further discloses a routing device, as shown in FIG. 4, including:

The receiving module 401 is configured to receive a data acquisition request of a source back of the CDN cache server, where

The searching module 402 is configured to search for data corresponding to the data acquisition request based on the data acquisition request;

a generating module 403, configured to generate a feedback message based on the search result;

The sending module 404 is configured to send the feedback message to the CDN cache server, so that the CDN cache server obtains the data based on the feedback message.

As shown in FIG. 5, the routing device further includes:

The modifying module 505 is configured to modify the return source address of the CDN cache server from the source station address to the address of the routing device, so that when the CDN cache server returns the source, the routing device receives the data returned by the CDN cache server. Get the request.

Specifically, the data acquisition request includes a URL of the data, and the data is stored in a storage center.

The searching module 402 is specifically configured to:

Determining, by the routing device, a URI in the URL based on a URL in the data acquisition request; the routing device performs a data search based on the URI to determine a storage center where the data is located; and the routing device extracts the The domain name of the storage center where the data resides.

Specifically, the routing device is pre-stored with a data index table, where the index table stores a correspondence between each storage center and data stored in each storage center;

The searching module 402 performs a data search based on the URI to determine a storage center where the data is located, and specifically includes:

Specifically, the routing device stores in advance a correspondence between a domain name of each storage center and the data acquisition request;

The generating module 403 is specifically configured to:

A second URI is generated as a feedback message based on the extracted domain name and the URI.

The embodiment of the present application further discloses a CDN cache server, as shown in FIG. 6, comprising:

The source module 601 is configured to return the received data acquisition request to the routing device.

The receiving module 602 is configured to receive a feedback message that is sent by the routing device based on the data acquisition request, where the feedback message includes an address where the data corresponding to the data acquisition request is located;

The obtaining module 603 is configured to acquire data corresponding to the data acquisition request based on the feedback message.

The feedback message includes a URL of the data, and the data is stored in a storage center;

The obtaining module 603 is specifically configured to:

The data is obtained in the determined storage center according to the URI.

Specifically, the CDN cache server further includes:

The embodiment of the present application further discloses a data acquisition system, as described in FIG. 1 and FIG. 7, including:

Compared with the prior art, the data receiving request of the source of the CDN cache server is received by the routing device, and the data corresponding to the data obtaining request is searched based on the data obtaining request; and a feedback message is generated based on the search result, A domain name can be returned to multiple storage centers, which reduces the cost of use and improves the user experience.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by hardware, or by software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several The instructions are for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various implementation scenarios of the present application.

A person skilled in the art can understand that the drawings are only a schematic diagram of a preferred implementation scenario, and the modules or processes in the drawings are not necessarily required to implement the application.

A person skilled in the art may understand that the modules in the apparatus in the implementation scenario may be distributed in the apparatus for implementing the scenario according to the implementation scenario description, or may be correspondingly changed in one or more devices different from the implementation scenario. The modules of the above implementation scenarios may be combined into one module, or may be further split into multiple sub-modules.

The above serial numbers are only for the description, and do not represent the advantages and disadvantages of the implementation scenario.

The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any changes that can be made by those skilled in the art should fall within the protection scope of the present application.

Claims

A data acquisition method, comprising:

The routing device receives the data acquisition request from the source of the CDN cache server,

The routing device searches for data corresponding to the data acquisition request based on the data acquisition request;

The routing device generates a feedback message based on the search result;

The routing device sends the feedback message to the CDN cache server to cause the CDN cache server to obtain the data based on the feedback message.
The method of claim 1, wherein before the routing device receives the data acquisition request from the source of the CDN cache server, the method further includes:

Modifying a return source address of the CDN cache server to an address of the routing device, so that when the CDN cache server returns to the source, the routing device receives a data acquisition request from the CDN cache server.
The method according to claim 1, wherein the data acquisition request includes a URL of the data, and the data is stored in a storage center.

The routing device searches for data corresponding to the data acquisition request based on the data acquisition request, and specifically includes:

Determining, by the routing device, a URI in the URL based on a URL in the data acquisition request;

The routing device performs a data search based on the URI to determine a storage center where the data is located;

The routing device extracts a domain name of a storage center where the data is located.
The method of claim 3, wherein the routing device pre-stores a data index table, wherein the index table stores a correspondence between each storage center and data stored in each storage center; Performing a data search based on the URI to determine a storage center where the data is located, specifically including:

The routing device performs a lookup in the index table based on the URI to determine a storage center where the data is located; or

The routing device pre-stores the corresponding relationship between the domain name of each storage center and the data acquisition request; the routing device performs a data search based on the URI to determine a storage center where the data is located, and specifically includes:

Determining, by the routing device, each storage center corresponding to the data acquisition request according to a correspondence relationship;

The routing device looks up the data in various storage centers based on the URI to determine a storage center in which the data resides.
A data acquisition method, comprising:

The CDN cache server returns the received data acquisition request to the routing device;

The CDN cache server receives a feedback message that is sent by the routing device based on the data acquisition request, where the feedback message includes an address where the data corresponding to the data acquisition request is located;

The CDN cache server acquires data corresponding to the data acquisition request based on the feedback message.
The method according to claim 5, wherein the feedback message includes a URL of the data, and the data is stored in a storage center;

The acquiring, by the CDN cache server, the data corresponding to the data acquisition request, based on the feedback message, specifically includes:

Determining, by the CDN cache server, a domain name of the storage center where the data is located and a URI of the data based on a URL in the feedback message;

The CDN cache server acquires the data in the determined storage center according to the URI.
The method according to claim 5 or 6, wherein after the CDN cache server acquires the data corresponding to the data acquisition request based on the feedback message, the method further includes:

The CDN cache server caches the data and sends the data to a user terminal corresponding to the data acquisition request.
A routing device, comprising:

a receiving module, configured to receive a data acquisition request from a source of the CDN cache server,

a searching module, configured to search for data corresponding to the data acquisition request based on the data acquisition request;

a generating module, configured to generate a feedback message based on the search result;

And a sending module, configured to send the feedback message to the CDN cache server, so that the CDN cache server acquires the data based on the feedback message.
The routing device of claim 8 further comprising:

And a modifying module, configured to modify a return source address of the CDN cache server to an address of the routing device, so that when the CDN cache server returns to the source, the routing device receives a data acquisition request of the CDN cache server back to the source.
The routing device according to claim 8, wherein the data acquisition request includes a URL of the data, and the data is stored in a storage center.

The searching module is specifically configured to:

Determining, by the routing device, a URI in the URL based on a URL in the data acquisition request;

The routing device performs a data search based on the URI to determine a storage center where the data is located;

The routing device extracts a domain name of a storage center where the data is located.
A CDN cache server, comprising:

a source module, configured to return the received data acquisition request to the routing device;

a receiving module, configured to receive a feedback message that is sent by the routing device based on the data acquisition request, where the feedback message includes an address where the data corresponding to the data acquisition request is located;

And an obtaining module, configured to acquire data corresponding to the data acquisition request based on the feedback message.
The CDN cache server according to claim 11, wherein the feedback message includes a URL of the data, and the data is stored in a storage center;

The obtaining module is specifically configured to:

Determining, according to a URL in the feedback message, a domain name of a storage center where the data is located and a URI of the data;

The data is obtained in the determined storage center according to the URI.
The CDN cache server according to claim 11 or 12, further comprising:

And a sending module, configured to cache the data, and send the data to a user terminal corresponding to the data obtaining request.
A data acquisition system, comprising:

a routing device, configured to receive a data acquisition request from a source of the CDN cache server, search for data corresponding to the data acquisition request based on the data acquisition request, generate a feedback message based on the search result, and send the feedback message to the a CDN cache server, such that the CDN cache server acquires the data based on the feedback message;

a CDN cache server, configured to: return a received data acquisition request to the routing device; and receive, by the routing device, a feedback message that is based on the data acquisition request, where the feedback message includes data corresponding to the data acquisition request. And the address corresponding to the data acquisition request is obtained based on the feedback message.