CN111880840A - Data acquisition method and device, electronic equipment and storage medium - Google Patents
Data acquisition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111880840A CN111880840A CN202010722781.0A CN202010722781A CN111880840A CN 111880840 A CN111880840 A CN 111880840A CN 202010722781 A CN202010722781 A CN 202010722781A CN 111880840 A CN111880840 A CN 111880840A
- Authority
- CN
- China
- Prior art keywords
- data
- type
- acquired
- target
- supplementary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 230000000153 supplemental effect Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 abstract description 6
- 238000012423 maintenance Methods 0.000 abstract description 5
- 238000011161 development Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000009877 rendering Methods 0.000 description 3
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001364 causal effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/72—Code refactoring
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a data acquisition method, a data acquisition device, electronic equipment and a storage medium, and relates to the field of data processing. The method comprises the following steps: receiving a page acquisition request sent by a client; acquiring target data indicated by a page acquisition request; with the data type of the target data as a parent data type, searching a child data type of the target data in a pre-constructed parent-child relationship of the data type, and acquiring data corresponding to the child data type as supplementary data of the target data; and constructing a page by adopting the target data and the supplementary data, and sending the page to the client. Therefore, the whole process does not need to write codes of parent-child relations of the data types in a tiled mode and can be repeatedly used, and corresponding data acquisition codes do not need to be written for all the data types. Therefore, the problems of code redundancy and difficult maintenance caused by complex program code development and incapability of multiplexing codes among data types are solved.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a data acquisition method and apparatus, an electronic device, and a storage medium.
Background
The server acquires data, constructs the data and generates a final rendering result, wherein the format of the rendering result can be JSON (JavaScript Object Notation), XML format of extensible markup language (XML), and the like. The process of building data acquisitions can involve a complex series of engineering problems. Generally, the original data is stored in the form of indexes and entities, or a mixture of both.
In the prior art, when a user request is sent, a server side first obtains a type identifier set through an index, then obtains entity data according to the type identifier set, finally obtains a set of the entity data to construct data, and outputs a final rendering result according to a service requirement.
However, the inventor finds that in the actual data acquisition process, due to the association relationship among the data, some entity data may contain some additional entity data similar to foreign key association, the additional entity data also needs to be acquired, and a very large number of levels may be nested based on the association relationship. Therefore, developers need to know the association relationship among data, the data acquisition process is to read the data, analyze the read data, and if additional data is included, continue to acquire the additional data, and so on. When the incidence relation of the data is complex, the code development for acquiring the data is complex, and the codes among different data types cannot be multiplexed.
Disclosure of Invention
The present disclosure aims to provide a data acquisition method, an apparatus, an electronic device, and a storage medium, so as to solve the problems in the related art that when a parent-child relationship of data is complex, development of codes for acquiring data is complex, and codes between different data types cannot be reused.
In a first aspect, the present disclosure provides a data acquisition method, the method comprising:
receiving a page acquisition request sent by a client;
acquiring target data indicated by the page acquisition request;
with the data type of the target data as a parent data type, searching a child data type of the target data in a pre-constructed parent-child relationship of the data type, and acquiring data corresponding to the child data type as supplementary data of the target data;
and constructing a page by adopting the target data and the supplementary data, and sending the page to the client.
In an embodiment, the searching for the child data type of the target data in the pre-constructed parent-child relationship of the data type and obtaining the data corresponding to the child data type as the supplementary data of the target data includes:
for the target data, the following loop process is executed:
analyzing the subdata types contained in the data types of the target data in the parent-child relationship of the data types;
if the sub-data type contained in the data is analyzed, at least one piece of data which has an association relation with the target data is obtained from the data of the sub-data type and is used as the supplementary data;
comparing the supplementary data with the acquired data set, and finding out supplementary data which is not contained in the acquired data set as new target data; the acquired data set comprises data acquired before the current cycle is executed and required for constructing the page;
when the supplemental data is included in the acquired data set, the loop process is ended.
In one embodiment, the method further comprises:
adding each target data into the acquired data set before searching the subdata type of the target data in the pre-constructed parent-child relationship of the data type;
the comparing the supplemental data to the acquired data set comprises:
forming an intermediate data set by the supplementary data, and comparing the acquired data set with the intermediate data set to obtain a difference set of the acquired data set and the intermediate data set; the difference set is used for storing data which belongs to the intermediate data set but does not belong to the acquired data set;
when the supplementary data are all contained in the acquired data set, ending the loop process, including:
and if the difference set is empty, ending the circulation process.
In one embodiment, after the comparing the acquired data set and the intermediate data set to obtain a difference set between the acquired data set and the intermediate data set, the method further includes:
merging the intermediate data set into the acquired data set.
In one embodiment, prior to said merging said intermediate data set into said acquired data set, said method further comprises:
filtering the intersection of the intermediate data set and the acquired set from the intermediate data set.
In one embodiment, the parent-child relationship of the data type further includes a supplemental data acquisition condition; before searching the sub-data type of the target data in the pre-established parent-child relationship of the data type and acquiring the data corresponding to the sub-data type as the supplementary data of the target data, the method further comprises:
acquiring the acquisition condition of the supplementary data and the information to be matched with the acquisition condition; the acquisition condition includes at least one of the following conditions: the access right of the supplementary data is possessed, the data type of the target data meets the preset data type, and the target data is approved data;
and if the information to be matched meets the obtaining condition, searching the sub-data type of the target data in a pre-constructed parent-child relationship of the data type, and obtaining data corresponding to the sub-data type as supplementary data of the target data.
In a second aspect, the present disclosure provides a data acquisition apparatus, the apparatus comprising:
the receiving module is configured to execute the page obtaining request sent by the receiving client;
a target data acquisition module configured to execute acquisition of target data indicated by the page acquisition request;
the supplementary data acquisition module is configured to search a child data type of the target data in a pre-established parent-child relationship of the data type by taking the data type of the target data as a parent data type, and acquire data corresponding to the child data type as supplementary data of the target data;
and the page construction module is configured to perform page construction by adopting the target data and the supplementary data and send the page to the client.
In an embodiment, the supplementary data obtaining module is specifically configured to perform:
for the target data, the following loop process is executed:
analyzing the subdata types contained in the data types of the target data in the parent-child relationship of the data types;
if the sub-data type contained in the data is analyzed, at least one piece of data which has an association relation with the target data is obtained from the data of the sub-data type and is used as the supplementary data;
comparing the supplementary data with the acquired data set, and finding out supplementary data which is not contained in the acquired data set as new target data; the acquired data set comprises data acquired before the current cycle is executed and required for constructing the page;
when the supplemental data is included in the acquired data set, the loop process is ended.
In one embodiment, the apparatus further comprises:
an adding module configured to add each target data to the acquired data set before searching for a child data type of the target data in the pre-constructed parent-child relationship of the data type;
the supplemental data acquisition module, when performing the aligning the supplemental data with the acquired data set, is configured to:
forming an intermediate data set by the supplementary data, and comparing the acquired data set with the intermediate data set to obtain a difference set of the acquired data set and the intermediate data set; the difference set is used for storing data which belongs to the intermediate data set but does not belong to the acquired data set;
the supplementary data acquisition module, when executing the loop process when the supplementary data are all contained in the acquired data set, is configured to:
and if the difference set is empty, ending the circulation process.
In one embodiment, the apparatus further comprises:
a merging module configured to perform the comparing of the acquired data set and the intermediate data set, and merge the intermediate data set into the acquired data set after obtaining a difference set between the acquired data set and the intermediate data set.
In one embodiment, the apparatus further comprises:
a filtering module configured to filter an intersection of the intermediate data set and the acquired data set from the intermediate data set prior to performing the merging of the intermediate data set into the acquired data set.
In one embodiment, the parent-child relationship of the data type further includes a supplemental data acquisition condition; the device further comprises:
a condition acquisition module configured to execute an acquisition condition for acquiring the supplementary data and information to be matched that matches the acquisition condition; the acquisition condition includes at least one of the following conditions: the access right of the supplementary data is possessed, the data type of the target data meets the preset data type, and the target data is approved data;
and the determining module is configured to execute the steps of searching the sub-data type of the target data in a pre-established parent-child relationship of the data type and acquiring data corresponding to the sub-data type as supplementary data of the target data if the information to be matched is determined to meet the acquiring condition.
According to a third aspect of embodiments of the present disclosure, there is provided an electronic device comprising at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor; the instructions are executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.
According to a fourth aspect provided by an embodiment of the present disclosure, there is provided a computer storage medium storing a computer program for executing the method of the first aspect.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
the disclosure provides a data acquisition method, a data acquisition device, an electronic device and a storage medium. The method comprises the following steps: receiving a page acquisition request sent by a client; acquiring target data indicated by the page acquisition request; with the data type of the target data as a parent data type, searching a child data type of the target data in a pre-constructed parent-child relationship of the data type, and acquiring data corresponding to the child data type as supplementary data of the target data; and constructing a page by adopting the target data and the supplementary data, and sending the page to the client. Therefore, the codes for acquiring the parent-child relationship of the data types do not need to be written in a tiled mode in the whole process, the parent-child relationship of the data types can be reused, and corresponding data acquisition codes do not need to be written for each data type. Therefore, when the codes of the acquired data are developed, the parent-child relationship of the data types does not need to be considered, the parent-child relationship of the data types does not need to be repeatedly written in the program codes, and the problems of code redundancy and difficult maintenance caused by the fact that the program codes are complex to develop and the codes among different data types cannot be reused in the prior art are solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is one of a data acquisition tree diagram according to one embodiment of the present disclosure;
FIG. 2 is one of the flow diagrams for data acquisition according to one embodiment of the present disclosure;
FIG. 3 is one of the interface schematic diagrams according to one embodiment of the present disclosure;
FIG. 4 is a second schematic interface diagram according to an embodiment of the present disclosure;
FIG. 5 is a second tree diagram for data acquisition according to one embodiment of the present disclosure;
FIG. 6 is a second schematic flow chart of data acquisition according to one embodiment of the present disclosure;
FIG. 7 is a schematic diagram of an apparatus for data acquisition according to one embodiment of the present disclosure;
FIG. 8 is a schematic structural diagram of an electronic device according to one embodiment of the present disclosure.
Detailed Description
To further illustrate the technical solutions provided by the embodiments of the present disclosure, the following detailed description is made with reference to the accompanying drawings and the specific embodiments. Although the disclosed embodiments provide method steps as shown in the following embodiments or figures, more or fewer steps may be included in the method based on conventional or non-inventive efforts. In steps where no necessary causal relationship exists logically, the order of execution of the steps is not limited to that provided by the disclosed embodiments. The method can be executed in the order of the embodiments or the method shown in the drawings or in parallel in the actual process or the control device.
The term "plurality" in the embodiments of the present disclosure means two or more, and other terms are used similarly, it being understood that the preferred embodiments described herein are only for illustrating and explaining the present disclosure, and are not intended to limit the present disclosure, and that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
The inventor researches and discovers that some entity data can contain some extra entity data similar to foreign key association due to the existence of the association relationship among the data, the extra entity data also needs to be acquired, constructed and output, and the association relationship can be nested in a very large number of layers. The inventor finds that, when the data codes are obtained by writing in the related art, the data have no clear association relationship, what data needs to be obtained is completely analyzed by the data result obtained by the upper level. For example:
when video data in the video list is acquired, it is necessary to check whether the video data includes data identifiers of other data types, and if the video data includes data identifiers of other data types, for example, an author identifier, it is necessary to acquire author data corresponding to the author identifier. When the author data is acquired, whether data identification of other data types is contained in the author data or not needs to be analyzed. And if the data identifier contains the data identifier of other data types, continuing to acquire the data. And so on, the iteration of the layers is continued until the acquired data does not contain any unacquired data identification any more. Thus, in the prior art, the code that constructs the data acquisition is typically written in a tiled fashion. Resulting in complex code writing logic. And because program code for constructing data of different data types cannot be reused, the program code is written for each data type. Therefore, the program codes in the related art are not only redundant but also difficult to maintain.
Therefore, the data acquisition method is provided, by means of the incidence relation among the data, the data are abstractly described into the parent-child relation, and then the parent-child relation of different data types is expressed in the form of a directed graph constructed on the basis of the abstract description. When a page acquisition request of a client is received, searching data to be acquired in a pre-constructed parent-child relationship of a data type to construct a page. Therefore, the data needing to be acquired can be determined according to the pre-constructed parent-child relationship of the data types, the parent-child relationship among the data does not need to be covered in the program codes of the acquired data, so that the program codes of the data acquisition do not need to be written in a tiled mode, the parent-child relationship of the data types can be repeatedly used, the program codes of the data acquisition do not need to be written aiming at each data type, and the redundancy of the program codes is reduced. If the parent-child relationships between data have not changed, maintenance and updating of program code also does not have to care about the parent-child relationships between data.
The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings.
Firstly, constructing a data type parent-child relationship:
before data acquisition is carried out, a parent-child relation of a data type needs to be constructed, and can be represented through a directed graph. For example, as shown in FIG. 1, each vertex in FIG. 1 is a data type, and each edge in the directed graph, e.g., the A → B edge, represents that data type A is the data type B parent data type. By analogy, data type A is also the parent data type of data type C. Data type B is the parent data type of data type D. For example, in the multimedia network resource application scenario, if the data type a is a photo type, the data type B is a comment type, the data type C may be a photo author type, and the data type D may be a comment number.
It should be noted that the parent-child relationship of the data type in the directed graph may be constructed by writing program code. Or may be implemented by way of database tables. For example, when constructed in program code, each data type is a class, and parent-child relationships between classes are written in the program code. Finally, the parent-child relationship of each data type in the directed graph can be obtained through the program code of the directed graph.
When implemented as a database table, determining the parent-child relationship of a data type can be implemented by means of a lookup table. Tables 1 and 2 may be used as the association tables for the two data types. Table 1 has associated key values of the data in table 2, and the associated key values are used to point to one or more data types in table 2. For example, table 1 contains data type a, and table 2 contains data type B, data type C, and data type D. The data type B, the data type C, and the data type D are sub data types of the data type a.
After the construction of the directed graph is introduced, the maintenance of the parent-child relationships of the data types in the directed graph is explained as follows:
1) and when maintaining the parent-child relationship of the data types constructed by the program codes, if a new data type needs to be added, modifying the codes in the program codes and adding the corresponding data type. If the data type is not used any more, modifying the corresponding program code, and deleting the data type which is not used any more.
2) And when the parent-child relationship of the data types constructed by the database table is maintained, if a new data type is added, adding and modifying the corresponding data table. And if the data type is to be deleted, deleting the corresponding data type in the corresponding data table.
Secondly, acquiring data according to the parent-child relationship of the data types in the directed graph:
as shown in fig. 2, the present disclosure provides a data acquisition method, which may include the steps of:
step 201: receiving a page acquisition request sent by a client;
step 202: acquiring target data indicated by the page acquisition request;
step 203: with the data type of the target data as a parent data type, searching a child data type of the target data in a pre-constructed parent-child relationship of the data type, and acquiring data corresponding to the child data type as supplementary data of the target data;
step 204: and constructing a page by adopting the target data and the supplementary data, and sending the page to the client.
Thus, the present disclosure expresses parent-child relationships of different data types by constructing abstract descriptions of parent-child relationships between data into the form of directed graphs. When a page acquisition request of a client is received, searching data to be acquired in a pre-constructed parent-child relationship of a data type to construct a page. Therefore, the data needing to be acquired can be determined according to the pre-constructed parent-child relationship of the data types, the parent-child relationship among the data does not need to be covered in the program codes of the acquired data, so that the program codes of the data acquisition do not need to be written in a tiled mode, the parent-child relationship of the data types can be repeatedly used, the program codes of the data acquisition do not need to be written aiming at each data type, and the redundancy of the program codes is reduced. If the parent-child relationships between data have not changed, maintenance and updating of program code also does not have to care about the parent-child relationships between data.
In the above-mentioned parent-child relationship of the pre-constructed data type, searching for the sub-data type of the target data, and obtaining data corresponding to the sub-data type as supplementary data of the target data; in the disclosed embodiments, it may be implemented as:
for the target data, the following loop process is executed:
analyzing the subdata types contained in the data types of the target data in the parent-child relationship of the data types; if the sub-data type contained in the data is analyzed, at least one piece of data which has an association relation with the target data is obtained from the data of the sub-data type and is used as the supplementary data; comparing the supplementary data with the acquired data set, and finding out supplementary data which is not contained in the acquired data set as new target data; the acquired data set comprises data acquired before the current cycle is executed and required for constructing the page; when the supplemental data is included in the acquired data set, the loop process is ended.
For example, a discovery page of a video application is taken as an example to describe in detail, as shown in fig. 3, fig. 3 is a discovery page of a video application, a certain number of videos are displayed in the page for a user to select and browse, 4 videos are displayed in this embodiment of the application, and it should be noted that the number displayed in the page is not limited herein. After a user clicks a certain video, for example, clicks the video 1, the user enters a playing interface of the video 1, and as shown in fig. 4, information such as an author and comments of the video is displayed in the interface. It should be noted that, after any one of the videos in fig. 3 is clicked, the video author, comment, and other types of information are displayed. Taking the above 4 videos as an example, how to acquire data in the videos is described:
at this time, the target data are video 1, video 2, video 3, and video 4. And resolving the subdata types contained in the data types of the video 1, the video 2, the video 3 and the video 4 in the parent-child relationship of the data types. First, target data is added to an acquired data set, where the acquired data set includes: video 1, video 2, video 3, and video 4. Then, a loop process is performed on the target data, and the following description will be given by taking the video 2 as an example:
at this time, the data type of the video 2 is a video, and the sub-data type of the video is determined to be an author and a comment according to the parent-child relationship of the data type in fig. 5. The associated data corresponding to the video 2 is obtained from the author data set and the comment data set as the supplementary data, for example, the author of the video 2 is author 2, and the comment corresponding to the video 2 includes comment 1 and comment 2. The supplemental data includes author 2, comment 1, and comment 2.
In one embodiment, the comparing the supplemental data to the acquired data set may be implemented as: forming an intermediate data set by the supplementary data, and comparing the acquired data set with the intermediate data set to obtain a difference set of the acquired data set and the intermediate data set; the difference set is used for storing data which belongs to the intermediate data set but does not belong to the acquired data set; when the supplementary data are both included in the acquired data set, the loop process is ended, which may be implemented as: and if the difference set is empty, ending the circulation process.
Therefore, after the target data circulation process is finished, an intermediate data set is constructed by the supplementary data, and the acquired data set and the intermediate data set are compared to obtain a difference set. If the author of video 1 is author 1, the author of video 3 is author 3, and the author of video 4 is author 4. Video 1, video 3, and video 4 have no comments. The intermediate data set includes: author 1, author 2, author 3, author 4, comment 1, comment 2. The acquired data set at this time is: video 1, video 2, video 3, video 4. Then the difference set is determined as: author 1, author 2, author 3, author 4, comment 1, comment 2.
In one embodiment, the intermediate data set is merged into the acquired data set. The merged acquired data set comprises: author 1, author 2, author 3, author 4, comment 1, comment 2, video 1, video 2, video 3, video 4.
Then, the data in the determined difference set is searched and acquired continuously according to the parent-child relationship of the data types in fig. 5. For example, if none of author 1, author 2, author 3, and author 4 is interested in a user, there is no data to be acquired in the interested user type. For example, the author who has acquired comment 1 is author 2, and the author of comment 2 is also author 3. The intermediate data set at this time includes: author 2, author 3. And comparing the intermediate data set with the acquired data set. Since the merged acquired data set includes author 2 and author 3, the difference set is determined to be empty. The loop process is ended.
The above-mentioned whole data acquisition logic, the implementation process in the corresponding directed graph program code can be understood as follows: each data type is a class, and each class has member variables of the class. For example, if a video is a class, the member variables of the video have information such as author, comment, etc. And if the data of the video type is to be acquired, performing traversal iteration according to the parent-child relationship of the codes in the video type directed graph to obtain the data to be acquired.
Therefore, all data required in the page acquisition request can be acquired through a repeated iteration acquisition mode according to the parent-child relationship of the data type.
Prior to merging the intermediate data set into the acquired data set, in one embodiment, intersections of the intermediate data set and the acquired set are filtered out of the intermediate data set.
For example, the intermediate data set includes data a and data B. The acquired data set includes data B, data C, and data D. The data in the two aggregate intersections is data B. Then data B is filtered out of the intermediate data set.
The parent-child relationship of the data type also comprises a supplementary data acquisition condition; in one embodiment, acquiring conditions of the supplementary data and information to be matched with the acquiring conditions are acquired; the acquisition condition includes at least one of the following conditions: the access authority of the supplementary data is possessed, the data type of the target data meets the preset data type, and the target data is data which passes the auditing; and if the information to be matched meets the obtaining condition, searching the sub-data type of the target data in a pre-constructed parent-child relationship of the data type, and obtaining data corresponding to the sub-data type as supplementary data of the target data.
(1) When the acquisition condition is that the access right of the supplementary data is:
for example, the supplementary data is data requested by the visitor, and the acquisition condition is that the authentication request function is set. If the user a has set the verification request function, it indicates that the information to be matched is the set verification request function, and the information to be matched satisfies the obtaining condition, then the data of the access request can be obtained.
(2) The obtaining condition is that the data type of the target data meets a preset data type:
for example, the obtaining condition is that the video type is a music video type, and if the information to be matched is a music video, corresponding music data needs to be obtained from a music database.
(3) Acquiring the target data as the data which passes the examination:
for example, if the target data is an e-commerce video, and the acquired information to be matched is the audited data, it can be determined that the information to be matched meets the acquisition condition.
Therefore, the data can be more flexibly acquired by setting the acquisition condition, and unnecessary acquisition operation is avoided.
To further understand the technical solution provided by the present disclosure, the following detailed description with reference to fig. 6 may include the following steps:
step 601: receiving a page acquisition request sent by a client;
step 602: acquiring target data indicated by the page acquisition request;
step 603: adding each target data to the acquired data set;
step 604: analyzing a child data type contained in the data type of the target data in a parent-child relationship of the data type by taking the data type of the target data as a parent data type;
step 605: if the sub-data type contained in the data is analyzed, at least one piece of data which has an association relation with the target data is obtained from the data of the sub-data type and is used as the supplementary data;
step 606: forming an intermediate data set by the supplementary data, and comparing the acquired data set with the intermediate data set to obtain a difference set of the acquired data set and the intermediate data set; the difference set is used for storing data which belongs to the intermediate data set but does not belong to the acquired data set;
step 607: merging the intermediate data set into the acquired data set;
step 608: if the difference set is empty, ending the cycle process; otherwise, continuing to execute step 604 by using the data in the difference set as new target data;
step 609: filtering the intersection of the intermediate data set and the acquired set from the intermediate data set.
Based on the same inventive concept, the data acquisition method of the present disclosure as described above can also be implemented by a data acquisition apparatus. The effect of the device is similar to that of the method, and is not repeated herein.
Fig. 7 is a schematic structural diagram of a data acquisition device according to an embodiment of the present disclosure.
As shown in fig. 7, the data acquisition apparatus 700 of the present disclosure may include a receiving module 710, a target data acquisition module 720, a supplementary data acquisition module 730, and a page construction module 740.
A receiving module 710 configured to execute receiving a page obtaining request sent by a client;
a target data obtaining module 720, configured to perform obtaining the target data indicated by the page obtaining request;
a supplementary data obtaining module 730, configured to perform searching for a child data type of the target data in a pre-established parent-child relationship of the data type with the data type of the target data as a parent data type, and obtain data corresponding to the child data type as supplementary data of the target data;
and a page construction module 740 configured to perform page construction using the target data and the supplemental data, and send the page to the client.
In an embodiment, the supplementary data obtaining module 730 is specifically configured to perform:
for the target data, the following loop process is executed:
analyzing the subdata types contained in the data types of the target data in the parent-child relationship of the data types;
if the sub-data type contained in the data is analyzed, at least one piece of data which has an association relation with the target data is obtained from the data of the sub-data type and is used as the supplementary data;
comparing the supplementary data with the acquired data set, and finding out supplementary data which is not contained in the acquired data set as new target data; the acquired data set comprises data acquired before the current cycle is executed and required for constructing the page;
when the supplemental data is included in the acquired data set, the loop process is ended.
In one embodiment, the apparatus further comprises:
an adding module 750 configured to add each target data to the acquired data set before searching for a child data type of the target data in a parent-child relationship of the pre-constructed data type;
the supplemental data acquisition module 730, in performing the aligning the supplemental data with the acquired data set, is configured to:
forming an intermediate data set by the supplementary data, and comparing the acquired data set with the intermediate data set to obtain a difference set of the acquired data set and the intermediate data set; the difference set is used for storing data which belongs to the intermediate data set but does not belong to the acquired data set;
the supplementary data obtaining module 730, when executing the step of ending the loop process when the supplementary data are all contained in the obtained data set, is configured to:
and if the difference set is empty, ending the circulation process.
In one embodiment, the apparatus further comprises:
a merging module 760 configured to perform the comparing of the acquired data set and the intermediate data set, and merge the intermediate data set into the acquired data set after obtaining a difference set between the acquired data set and the intermediate data set.
In one embodiment, the apparatus further comprises:
a filtering module 770 configured to filter the intersection of the intermediate data set and the acquired data set from the intermediate data set prior to performing the merging of the intermediate data set into the acquired data set.
In one embodiment, the parent-child relationship of the data type further includes a supplemental data acquisition condition; the device further comprises:
a condition obtaining module 780 configured to execute obtaining conditions for obtaining the supplementary data and information to be matched that matches the obtaining conditions; the acquisition condition includes at least one of the following conditions: the access right of the supplementary data is possessed, the data type of the target data meets the preset data type, and the target data is approved data;
the determining module 790 is configured to perform, if it is determined that the information to be matched meets the obtaining condition, a step of searching a sub-data type of the target data in a pre-established parent-child relationship of the data type, and obtaining data corresponding to the sub-data type as supplementary data of the target data.
Having described a data method and apparatus of an exemplary embodiment of the present application, an electronic device according to another exemplary embodiment of the present application is next described.
As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method or program product. Accordingly, various aspects of the present application may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
In some possible implementations, an electronic device in accordance with the present application may include at least one processor, and at least one computer storage medium. The computer storage medium stores program code, which, when executed by a processor, causes the processor to perform the steps of the data acquisition method according to various exemplary embodiments of the present application described above in the present specification. For example, the processor may perform steps 201 and 204 as shown in FIG. 2.
An electronic device 800 according to this embodiment of the application is described below with reference to fig. 8. The electronic device 800 shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 8, the electronic device 800 is represented in the form of a general electronic device. The components of the electronic device 800 may include, but are not limited to: the at least one processor 801, the at least one computer storage medium 802, and the bus 803 that connects the various system components (including the computer storage medium 802 and the processor 801).
The computer storage media 802 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM)821 and/or cache storage media 822, and may further include read-only computer storage media (ROM) 823.
The computer storage media 802 may also include a program/utility 825 having a set (at least one) of program modules 824, such program modules 824 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 800 may also communicate with one or more external devices 804 (e.g., keyboard, pointing device, etc.), with one or more devices that enable a user to interact with the electronic device 800, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 800 to communicate with one or more other electronic devices. Such communication may be through input/output (I/O) interfaces 805. Also, the electronic device 800 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 806. As shown, the network adapter 806 communicates with other modules for the electronic device 800 over the bus 803. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
In some possible embodiments, various aspects of a data acquisition method provided by the present application may also be implemented in the form of a program product including program code for causing a computer device to perform the steps in a data acquisition method according to various exemplary embodiments of the present application described above in this specification when the program product is run on the computer device, for example, the computer device may perform steps 201 and 204 as shown in fig. 2.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a random access computer storage media (RAM), a read-only computer storage media (ROM), an erasable programmable read-only computer storage media (EPROM or flash memory), an optical fiber, a portable compact disc read-only computer storage media (CD-ROM), an optical computer storage media piece, a magnetic computer storage media piece, or any suitable combination of the foregoing.
The program product for data acquisition of the embodiments of the present application may employ a portable compact disc read-only computer storage medium (CD-ROM) and include program code, and may be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the consumer electronic device, partly on the consumer electronic device, as a stand-alone software package, partly on the consumer electronic device and partly on a remote electronic device, or entirely on the remote electronic device or server. In the case of remote electronic devices, the remote electronic devices may be connected to the consumer electronic device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external electronic device (e.g., through the internet using an internet service provider).
It should be noted that although several modules of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the modules described above may be embodied in one module according to embodiments of the application. Conversely, the features and functions of one module described above may be further divided into embodiments by a plurality of modules.
Further, while the operations of the methods of the present application are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk computer storage media, CD-ROMs, optical computer storage media, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable computer storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable computer storage medium produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A method for data acquisition, the method comprising:
receiving a page acquisition request sent by a client;
acquiring target data indicated by the page acquisition request;
with the data type of the target data as a parent data type, searching a child data type of the target data in a pre-constructed parent-child relationship of the data type, and acquiring data corresponding to the child data type as supplementary data of the target data;
and constructing a page by adopting the target data and the supplementary data, and sending the page to the client.
2. The method of claim 1, wherein the searching for the sub data type of the target data in the pre-established parent-child relationship of the data type and obtaining the data corresponding to the sub data type as the supplementary data of the target data comprises:
for the target data, the following loop process is executed:
analyzing the subdata types contained in the data types of the target data in the parent-child relationship of the data types;
if the sub-data type contained in the data is analyzed, at least one piece of data which has an association relation with the target data is obtained from the data of the sub-data type and is used as the supplementary data;
comparing the supplementary data with the acquired data set, and finding out supplementary data which is not contained in the acquired data set as new target data; the acquired data set comprises data acquired before the current cycle is executed and required for constructing the page;
when the supplemental data is included in the acquired data set, the loop process is ended.
3. The method of claim 2, further comprising:
adding each target data into the acquired data set before searching the subdata type of the target data in the pre-constructed parent-child relationship of the data type;
the comparing the supplemental data to the acquired data set comprises:
forming an intermediate data set by the supplementary data, and comparing the acquired data set with the intermediate data set to obtain a difference set of the acquired data set and the intermediate data set; the difference set is used for storing data which belongs to the intermediate data set but does not belong to the acquired data set;
when the supplementary data are all contained in the acquired data set, ending the loop process, including:
and if the difference set is empty, ending the circulation process.
4. The method of claim 3, wherein after comparing the acquired data set with the intermediate data set to obtain a difference between the acquired data set and the intermediate data set, further comprising:
merging the intermediate data set into the acquired data set.
5. The method of claim 4, wherein prior to said merging the intermediate data set into the acquired data set, the method further comprises:
filtering the intersection of the intermediate data set and the acquired set from the intermediate data set.
6. The method according to claim 2, wherein the parent-child relationship of the data type further includes a supplemental data acquisition condition; before searching the sub-data type of the target data in the pre-established parent-child relationship of the data type and acquiring the data corresponding to the sub-data type as the supplementary data of the target data, the method further comprises:
acquiring the acquisition condition of the supplementary data and the information to be matched with the acquisition condition; the acquisition condition includes at least one of the following conditions: the access right of the supplementary data is possessed, the data type of the target data meets the preset data type, and the target data is approved data;
and if the information to be matched meets the obtaining condition, searching the sub-data type of the target data in a pre-constructed parent-child relationship of the data type, and obtaining data corresponding to the sub-data type as supplementary data of the target data.
7. A data acquisition apparatus, characterized in that the apparatus comprises:
the receiving module is configured to execute the page obtaining request sent by the receiving client;
a target data acquisition module configured to execute acquisition of target data indicated by the page acquisition request;
the supplementary data acquisition module is configured to search a child data type of the target data in a pre-established parent-child relationship of the data type by taking the data type of the target data as a parent data type, and acquire data corresponding to the child data type as supplementary data of the target data;
and the page construction module is configured to perform page construction by adopting the target data and the supplementary data and send the page to the client.
8. The apparatus according to claim 7, wherein the supplementary data acquisition module is specifically configured to perform:
for the target data, the following loop process is executed:
analyzing the subdata types contained in the data types of the target data in the parent-child relationship of the data types;
if the sub-data type contained in the data is analyzed, at least one piece of data which has an association relation with the target data is obtained from the data of the sub-data type and is used as the supplementary data;
comparing the supplementary data with the acquired data set, and finding out supplementary data which is not contained in the acquired data set as new target data; the acquired data set comprises data acquired before the current cycle is executed and required for constructing the page;
when the supplemental data is included in the acquired data set, the loop process is ended.
9. An electronic device comprising at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor; the instructions are executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
10. A computer storage medium, characterized in that the computer storage medium stores a computer program for performing the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010722781.0A CN111880840A (en) | 2020-07-24 | 2020-07-24 | Data acquisition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010722781.0A CN111880840A (en) | 2020-07-24 | 2020-07-24 | Data acquisition method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111880840A true CN111880840A (en) | 2020-11-03 |
Family
ID=73201294
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010722781.0A Pending CN111880840A (en) | 2020-07-24 | 2020-07-24 | Data acquisition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111880840A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113901125A (en) * | 2021-09-17 | 2022-01-07 | 盐城金堤科技有限公司 | Data display method and device, storage medium and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021501A (en) * | 2016-05-20 | 2016-10-12 | 天脉聚源(北京)传媒科技有限公司 | Data storing method and device |
| CN107562467A (en) * | 2017-07-26 | 2018-01-09 | 阿里巴巴集团控股有限公司 | Page rendering method, apparatus and equipment |
| CN109522073A (en) * | 2018-11-06 | 2019-03-26 | 网宿科技股份有限公司 | Multi-layer page storage method, device and electronic equipment based on JSON |
| CN110263277A (en) * | 2019-06-18 | 2019-09-20 | 广州虎牙科技有限公司 | Display methods, update method, device, equipment and the storage medium of page data |
| CN110889067A (en) * | 2018-09-10 | 2020-03-17 | 广州虎牙信息科技有限公司 | Page rendering method, device and equipment and storage medium |
| CN110990746A (en) * | 2019-12-06 | 2020-04-10 | 北京同邦卓益科技有限公司 | Page loading method, device, system, storage medium and electronic equipment |
| CN111124379A (en) * | 2019-11-25 | 2020-05-08 | 贝壳技术有限公司 | Page generation method and device, electronic equipment and storage medium |
-
2020
- 2020-07-24 CN CN202010722781.0A patent/CN111880840A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021501A (en) * | 2016-05-20 | 2016-10-12 | 天脉聚源(北京)传媒科技有限公司 | Data storing method and device |
| CN107562467A (en) * | 2017-07-26 | 2018-01-09 | 阿里巴巴集团控股有限公司 | Page rendering method, apparatus and equipment |
| CN110889067A (en) * | 2018-09-10 | 2020-03-17 | 广州虎牙信息科技有限公司 | Page rendering method, device and equipment and storage medium |
| CN109522073A (en) * | 2018-11-06 | 2019-03-26 | 网宿科技股份有限公司 | Multi-layer page storage method, device and electronic equipment based on JSON |
| CN110263277A (en) * | 2019-06-18 | 2019-09-20 | 广州虎牙科技有限公司 | Display methods, update method, device, equipment and the storage medium of page data |
| CN111124379A (en) * | 2019-11-25 | 2020-05-08 | 贝壳技术有限公司 | Page generation method and device, electronic equipment and storage medium |
| CN110990746A (en) * | 2019-12-06 | 2020-04-10 | 北京同邦卓益科技有限公司 | Page loading method, device, system, storage medium and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 王开铸等: "《C语言数据结构程序设计》", vol. 1, 31 March 2003, 哈尔滨：哈尔滨工业大学出版社, pages: 306 - 310 * |
| 程光等: "《僵尸网络检测技术》", vol. 1, 南京：东南大学出版社, pages: 54 - 57 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113901125A (en) * | 2021-09-17 | 2022-01-07 | 盐城金堤科技有限公司 | Data display method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9898280B2 (en) | Automatic code review and code reviewer recommendation | |
| US11487721B2 (en) | Matching metastructure for data modeling | |
| EP3502896B1 (en) | Generation of an adapters configuration user interface using a data structure | |
| US9690822B2 (en) | System and method for metadata level validation of custom setup objects | |
| US9576036B2 (en) | Self-analyzing data processing job to determine data quality issues | |
| KR101755365B1 (en) | Managing record format information | |
| US8140573B2 (en) | Exporting and importing business objects based on metadata | |
| US10621211B2 (en) | Language tag management on international data storage | |
| EP3438813B1 (en) | Component management platform | |
| US11481200B1 (en) | Checking source code validity at time of code update | |
| US10083016B1 (en) | Procedurally specifying calculated database fields, and populating them | |
| US20130007709A1 (en) | Software configuration management | |
| JP5791149B2 (en) | Computer-implemented method, computer program, and data processing system for database query optimization | |
| US20200349129A1 (en) | Schema alignment and structural data mapping of database objects | |
| US10572371B2 (en) | Generating valid use cases from a parsed configuration file describing use case options | |
| US9442718B1 (en) | System for assisting in locating changes in source code version control system | |
| US8676627B2 (en) | Vertical process merging by reconstruction of equivalent models and hierarchical process merging | |
| EP4290393A1 (en) | Consolidation spaces providing access to multiple instances of application content | |
| CN115599386A (en) | Code generation method, device, equipment and storage medium | |
| US11422984B2 (en) | Clustering within database data models | |
| US20200097260A1 (en) | Software application developer tools platform | |
| CN111880840A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| US20230418803A1 (en) | Techniques for integrating data for multple instances of a data artifact | |
| US20150178075A1 (en) | Enhancing understandability of code using code clones | |
| CN116414855A (en) | Information processing method and device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201103 |
|
| RJ01 | Rejection of invention patent application after publication |