[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN116340364A - Data processing method, device, equipment and storage medium - Google Patents

Data processing method, device, equipment and storage medium Download PDF

Info

Publication number
CN116340364A
CN116340364A CN202310363821.0A CN202310363821A CN116340364A CN 116340364 A CN116340364 A CN 116340364A CN 202310363821 A CN202310363821 A CN 202310363821A CN 116340364 A CN116340364 A CN 116340364A
Authority
CN
China
Prior art keywords
field
fields
cache
target
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310363821.0A
Other languages
Chinese (zh)
Inventor
郑志杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Priority to CN202310363821.0A priority Critical patent/CN116340364A/en
Publication of CN116340364A publication Critical patent/CN116340364A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2379Updates performed during online database operations; commit processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/166Editing, e.g. inserting or deleting
    • G06F40/177Editing, e.g. inserting or deleting of tables; using ruled lines
    • G06F40/18Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application discloses a data processing method, a device, equipment and a storage medium, comprising the following steps: traversing the source data to obtain a plurality of fields of the source data and an initial field information set corresponding to each field. Based on the plurality of fields and the corresponding initial set of field information, a plurality of field objects and a plurality of target sets of field information are stored in a cache. For a target field of the plurality of fields, a set of item information associated with the target field in the source data is obtained. And finding a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache, so as to realize the storage of the source data in the cache. By the method for processing the table data, the source data can be stored in the cache, so that the data can be quickly read from the cache later, and the speed of creating or modifying the pivot table is improved.

Description

Data processing method, device, equipment and storage medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data processing method, device, equipment, and storage medium.
Background
To facilitate user analysis of the source data, a pivot table may be created from the source data. The pivot table can process and calculate the data of the data source, and a two-dimensional table is generated according to the configuration of a user, such as summation, counting and the like. The user can dynamically change the layout of the pivot table to analyze the data in different ways.
The user may update the data presented by the final two-dimensional table by changing the field configuration in the pivot row/column dimension. When the source data volume of the pivot table is large, when the pivot table is reconstructed each time, the process of traversing the source data to search the related data and performing operation is very time-consuming, and the construction speed of the pivot table is reduced.
Disclosure of Invention
In view of this, the present application provides a method, apparatus, device, and storage medium for processing data, so as to store source data in a cache, so that a pivot table can be created or modified according to data in the cache, and the speed of creating or modifying the pivot table is increased.
In order to achieve the above purpose, the technical scheme provided by the application is as follows:
in a first aspect of the present application, there is provided a method for processing data, the method comprising:
Traversing the source data to obtain one or more fields and one or more initial field information sets, wherein the initial field information sets are used for representing relevant information of the fields;
storing one or more field objects and one or more target field information sets in a cache based on the one or more fields and the one or more initial field information sets;
aiming at a target field in the one or more fields, acquiring an item information set corresponding to the target field in the source data;
and acquiring a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache.
In a second aspect of the present application, there is provided a data processing apparatus, the apparatus comprising:
the first acquisition unit is used for traversing the source data to acquire one or more fields and one or more initial field information sets, wherein the initial field information sets are used for representing relevant information of the fields;
a first storage unit configured to store one or more field objects and one or more target field information sets in a cache based on the one or more fields and the one or more initial field information sets;
A second obtaining unit, configured to obtain, for a target field in the one or more fields, a set of item information corresponding to the target field in the source data;
and the second storage unit is used for acquiring a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache.
In a third aspect of the present application, there is provided an electronic device, the device comprising: a processor and a memory;
the memory is used for storing instructions or computer programs;
the processor is configured to execute the instructions or the computer program in the memory, so that the electronic device performs the method described in the first aspect.
In a fourth aspect of the present application, there is provided a computer readable storage medium having instructions stored therein which, when run on a device, cause the device to perform the method of the first aspect described above.
In a fifth aspect of the present application, there is provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement the method of the first aspect.
From this, this application has following beneficial effect:
in the above implementation manner of the present application, in order to store source data in a cache and meet a data format of creating a pivot table, the source data needs to be traversed first, for example, may be table data, so as to obtain a plurality of fields in the source data and an initial field information set corresponding to each field, where the initial field information set is used to represent relevant information of the field. Such as field names, field identifications, data formats of the fields, etc. Based on the plurality of fields and the corresponding initial field information sets, a plurality of field objects and a plurality of target field information sets are stored in a cache, wherein the plurality of field objects are in one-to-one correspondence with the plurality of target field information sets. For a target field of the plurality of fields, a set of item information associated with the target field in the source data is obtained, e.g., cell data of a row/column in which the target field is located. And finding a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache, so as to realize the storage of the source data in the cache. By the method for processing the table data, the source data can be stored in the cache, so that the data can be quickly read from the cache later, and the speed of creating or modifying the pivot table is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application;
FIG. 2 is a flowchart of a method for creating a pivot table according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the following description will clearly and completely describe the technical solution in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
To facilitate user analysis of the source data, a pivot table may be created from the source data. The user may update the data presented by the final two-dimensional table by changing the field configuration in the pivot row/column dimension. When the source data volume of the pivot table is large, when the pivot table is reconstructed each time, the process of traversing the source data to search the related data and performing operation is very time-consuming, and the construction speed of the pivot table is reduced.
Based on this, the embodiment of the application provides a data processing method, so as to store source data in a cache, so that a pivot table can be created or modified according to data in the cache, and the speed of creating or modifying the pivot table is increased. In specific implementation, traversing the source data, and acquiring a plurality of fields in the source data and an initial field information set corresponding to each field, wherein the initial field information set is used for representing relevant information of the field. Such as field names, field identifications, data formats of the fields, etc. Based on the plurality of fields and the corresponding initial field information sets, a plurality of field objects and a plurality of target field information sets are stored in a cache, wherein the plurality of field objects are in one-to-one correspondence with the plurality of target field information sets. For a target field of the plurality of fields, a set of item information associated with the target field in the source data is obtained, e.g., cell data of a row/column in which the target field is located. And finding a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache, so as to realize the storage of the source data in the cache. By the data processing method, the source data can be stored in the cache, so that the data can be quickly read from the cache later, and the speed of creating or modifying the pivot table is improved.
In order to facilitate understanding of the technical solutions provided in the embodiments of the present application, the following description will be made with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a flowchart of a data processing method according to an embodiment of the present application.
The method may be performed by a processing device that may implement the processing of the table data. The processing device may be an electronic device or other device.
The method may comprise the steps of:
s101: traversing the source data to obtain one or more fields and one or more initial field information sets.
Wherein the one or more fields are in a one-to-one correspondence with one or more initial sets of field information, i.e., each field corresponds to an initial set of field information that may be used to represent information about the field. For example, the initial set of field information may include a field name, a field identification, a data format of the field, a data type, and the like.
Alternatively, the source data may be table data stored in the processing device, or table data acquired from other devices, or the like, and the top row or column of the table data may include fields of various types including, for example, a date of sale, a province of sale, a price of sale, and a sales amount. Based on this, one or more fields in the top row of source data may be obtained by traversing the source data acquisition fields in the following manner. That is, one or more fields in the top row are different types of fields. Alternatively, the first column of the source data is traversed to obtain one or more fields in the first column, i.e., the one or more fields in the first column are different types of fields. The initial field information set may directly obtain related information when traversing the fields in the source data, which is not described herein.
S102: one or more field objects and one or more target field information sets are stored in a cache based on the one or more fields and the one or more initial field information sets.
The field and the field object in the cache are in a corresponding relation, and the initial field information set and the target field information set are in a corresponding relation, so that the plurality of field objects and the plurality of target field information sets are in one-to-one correspondence, that is, each field object in the cache corresponds to one target field information set, and the target field information set can be used for representing relevant information of the field object. The field object stored in the cache may be the same content as the field, or may be a field converted according to actual requirements. For example, to reduce cache space, a field for a kanji name may be stored as a field object represented by a numeric identifier. In addition, the target field information set and the initial field information set may be the same content, or may be a field information set converted according to actual requirements.
In one possible implementation, the field object may be stored in the cache in such a way that when the initial set of field information includes a field name and a field identification of the field, a first set and a second set corresponding to the field may be created in the cache based on the field, the field name and the field identification. The first set and the second set may be sets with mapping relationships, and the sets may include multiple sets of key-values with mapping relationships, where each key corresponds to a value. For example, it may be a set of key-value pairs maps. The key of the first set may be set as a field name, the value of the first set is a field object corresponding to the field, the key of the second set is a field identifier, and the value of the second set is a field object corresponding to the field. Alternatively, when the initial field information set includes only the field name or the field identification of the field, a set including the field name or the field identification may be created according to the above steps.
The field objects are stored in a Map set mode, when the data is required to be searched from the cache in the subsequent construction of the data pivot table, the data pivot table can be searched according to the field names or the field identifications, the matched Map set is quickly obtained, and then the data pivot table is constructed based on the field objects in the Map set and the item information sets corresponding to the field objects. When searching is performed based on the data structure of the Map set, the searching dimension is 1 dimension, namely, the searching can be realized by determining the corresponding field name or field identification in the Map set, and when searching data from the source data, the searching dimension is n (the number of fields), namely, all field data in the data source need to be traversed one by one, so that the speed of constructing the data perspective table can be improved based on the method for caching the source data provided by the embodiment of the application.
In one possible implementation, an array may also be created in the cache, and a plurality of field objects corresponding to a plurality of fields of the source data may be added to the array. Based on the above, when all fields of the source data need to be acquired, the array is called in the cache, so that the speed of searching the data is improved.
S103: and aiming at a target field in the one or more fields, acquiring an item information set corresponding to the target field in the source data.
After each field in the source data is stored in the cache, the set of item information associated with the field may be retrieved so that each set of item information is also stored in the cache. The item information set may include information about the item, such as a name of the item, a type of the item, a number of lines in which the item is located, and the like.
The maximum information of the item, such as the maximum value, the minimum value, the maximum date, etc., can be increased according to the actual requirement, and the embodiment of the application does not limit the specific content of the item information set.
Alternatively, when the fields are stored in the same row, the item information set corresponding to each field may be related information of each cell data of the column where the field is located. For example, when the field is "sales province", the item information set corresponding to the field may be each province in the column where the field is located and related information of the stored province, for example, the item is Beijing, tianjin, shanghai, the type of the item is a character string, and the like. When the fields are stored in the same column, the item information set corresponding to each field may be related information of the cell data of the row where the field is located.
S104: and acquiring a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache.
Because each field in the source data is already stored in the cache, when the item information set is stored in the cache, the corresponding relationship between the field object and the item information set needs to be ensured. That is, for a target field of the multiple fields of the source data, after acquiring an item information set corresponding to the target field in the source data, acquiring a target field object corresponding to the target field in the cache, and when the item information set does not exist in the cache, storing the item information set under the target field object to realize the corresponding relationship between the target field object and the item information set.
Alternatively, when the item information set of the target field is stored, the target field may be one field or may be a plurality of fields. That is, for a set of item information of a field, the set of item information may be stored under a corresponding field object in the cache, or multiple sets of item information may be obtained by traversing multiple fields at a time, and for a corresponding multiple field object in the cache, the corresponding set of item information may be stored under each field object in turn.
Since the source data may include data in various formats, for example, the items in the item information set may be pictures, links, etc., when the item information set is stored, the item information set may be converted according to a data conversion rule to obtain the item information set satisfying the pivot table data format and stored in the cache. The data conversion rule may be set according to a requirement of a user for creating a pivot table, which is not limited in the embodiment of the present application. For example, the pictures may be converted to blank data, the links to plain text data for processing, and the drop down multiple options in the form to strings for processing.
Based on the method provided by the embodiment, the source data can be stored in the cache according to the specific data structure, and then the pivot table associated with the source data can be created according to the field object, the target field information set and the item information set stored in the cache, so that the speed of creating the pivot table is improved. The specific process of creating the pivot table is not a technical solution protected in the present application, and reference may be made to an existing solution for creating the pivot table, which is not described herein.
Based on this, the embodiment of the application provides a flowchart of a method for creating a pivot table, as shown in fig. 2.
The method may comprise the steps of:
s201: acquiring source data;
s202: storing the source data in a cache;
the step of storing the source data may refer to the above method embodiment, and is not described herein.
S203: creating a pivot table according to the data in the cache;
s204: when the source data is changed, updating the cache based on the changed data;
when the source data is changed, the corresponding data in the cache needs to be updated according to the changed data, and the corresponding data can be a field object, a target field information set, a term information set or the like.
S205: updating the pivot table based on the updated cache.
When the data in the cache is changed, the pivot table can be updated according to the updated cache.
Alternatively, the user may change the display data of the pivot table by changing the operation of the configuration field in the pivot table, for example, modifying a certain field in the row dimension or deleting a certain field in the column dimension, and the processing device may obtain, from the cached set, a target set matching the field name or the field identifier of the configuration field based on the field name or the field identifier of the configuration field. And updating the pivot table according to the field object in the target set and the item information set corresponding to the field object.
When the target set is a Map set, searching can be performed according to the field name or the field identification when searching is performed based on the data structure of the Map set, and the searching dimension is 1 dimension, namely searching can be realized by determining the corresponding field name or the field identification in the Map set, and the matched Map set can be rapidly obtained. Compared with the searching of the data from the source data, the searching dimension is n (the number of the fields), namely, all the field data in the data source need to be traversed one by one, so that the speed of constructing the data pivot table can be improved based on the caching mode of the source data provided by the embodiment of the application.
When the data size of the source data is large, the stored cache data occupies a large memory of the page, so that the rendering of the data in the page may be affected. Therefore, after deleting the pivot table, if the source data does not have other associated pivot tables, the cache can be deleted to release the page memory space, so that the speed and quality of page rendering are improved.
Based on the method embodiment, the embodiment of the application also provides a data processing device. Referring to fig. 3, fig. 3 is a schematic diagram of a data processing apparatus according to an embodiment of the present application.
The apparatus 300 includes:
a first obtaining unit 301, configured to traverse source data to obtain one or more fields and one or more initial field information sets, where the initial field information sets are used to represent relevant information of the fields;
a first storage unit 302, configured to store one or more field objects and one or more target field information sets in a cache based on the one or more fields and the one or more initial field information sets;
a second obtaining unit 303, configured to obtain, for a target field in the one or more fields, a set of item information corresponding to the target field in the source data;
And a second storage unit 304, configured to obtain a target field object corresponding to the target field in the cache, and store the item information set under the target field object when the item information set does not exist in the cache.
In one possible implementation manner, the one or more fields are in one-to-one correspondence with the one or more initial field information sets, for any one of the one or more fields and an initial field information set corresponding to the field, the initial field information set includes at least one of a field name and a field identifier of the field, and the first storage unit 302 is specifically configured to, when the initial field information set includes the field name and the field identifier of the field, create, in the cache, a first set and a second set corresponding to the field based on the field, the field name and the field identifier, a key of the first set is a field name of the field, a value of the first set is a field object corresponding to the field, a key of the second set is a field identifier of the field, and a value of the second set is a field object corresponding to the field.
In one possible implementation, the apparatus 300 further includes: a creation unit;
the creation unit is configured to create an array in the cache, and add the one or more field objects to the array.
In a possible implementation manner, the second storage unit 304 is specifically configured to convert the item information set based on a data conversion rule, where the data conversion rule is predetermined according to a data format of a pivot table, and store the converted item information set under the target field object.
In a possible implementation manner, the first obtaining unit 301 is specifically configured to traverse a first row of the source data to obtain one or more fields in the first row, where the one or more fields in the first row are fields of different types; or traversing the first column of the source data to obtain one or more fields in the first column, wherein the one or more fields in the first column are fields of different types.
In a possible implementation manner, the creating unit is further configured to create a pivot table associated with the source data based on the field object, the target field information set, and the item information set stored in the cache.
In one possible implementation, the apparatus 300 further includes: an updating unit;
the updating unit is used for responding to the operation of changing the configuration field in the pivot table by a user, and acquiring a target set matched with the field name or the field identifier of the configuration field from the cached set based on the field name or the field identifier of the configuration field; and updating the pivot table according to the field object in the target set and the item information set corresponding to the field object.
In a possible implementation manner, the updating unit is further configured to update the cache based on changed data when the source data is changed; and updating the pivot table based on the updated cache.
In one possible implementation, the apparatus 300 further includes: a deletion unit;
and the deleting unit is used for deleting the cache in response to the operation of deleting the pivot table by a user.
The beneficial effects of the data processing device provided in the embodiment of the present application can be seen in the above method embodiments, and are not described herein.
It should be noted that, for specific implementation of each unit in this embodiment, reference may be made to the related description in the above method embodiment. The division of the units in the embodiment of the application is schematic, which is merely a logic function division, and other division manners may be adopted in actual implementation. Each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. For example, in the above embodiment, the processing unit and the transmitting unit may be the same unit or may be different units. The integrated units may be implemented in hardware or in software functional units.
Referring to fig. 4, a schematic diagram of an electronic device 400 suitable for use in implementing embodiments of the present application is shown. The terminal device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 4 is only an example and should not be construed as limiting the functionality and scope of use of the embodiments herein.
As shown in fig. 4, the electronic device 400 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 401, which may perform various suitable actions and processes according to a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage means 408 into a Random Access Memory (RAM) 403. In the RAM403, various programs and data necessary for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM403 are connected to each other by a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
In general, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 407 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 408 including, for example, magnetic tape, hard disk, etc.; and a communication device 409. The communication means 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. While fig. 4 shows an electronic device 400 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via communications device 409, or from storage 408, or from ROM 402. The above-described functions defined in the methods of the embodiments of the present application are performed when the computer program is executed by the processing means 401.
The electronic device provided in the embodiment of the present application and the method provided in the foregoing embodiment belong to the same inventive concept, and technical details not described in detail in the present embodiment can be seen in the foregoing embodiment, and the present embodiment has the same beneficial effects as the foregoing embodiment.
The present embodiment provides a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the method provided by the above embodiment.
It should be noted that the computer readable medium described in the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal that propagates in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the method described above.
Computer program code for carrying out operations of the present application may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented by software, or may be implemented by hardware. Where the names of the units/modules do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
According to one or more embodiments of the present application, there is provided a data processing method, including:
traversing the source data to obtain one or more fields and one or more initial field information sets, wherein the initial field information sets are used for representing relevant information of the fields;
storing one or more field objects and one or more target field information sets in a cache based on the one or more fields and the one or more initial field information sets;
aiming at a target field in the one or more fields, acquiring an item information set corresponding to the target field in the source data;
and acquiring a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache.
According to one or more embodiments of the present application, the one or more fields are in one-to-one correspondence with the one or more initial field information sets, and for any one of the one or more fields and an initial field information set corresponding to the field, the initial field information set includes at least one of a field name and a field identification of the field, and storing one or more field objects in a cache based on the one or more fields and the one or more initial field information sets includes:
When the initial field information set comprises a field name and a field identification of the field, a first set and a second set corresponding to the field are created in the cache based on the field, the field name and the field identification, a keyword of the first set is the field name of the field, a value of the first set is a field object corresponding to the field, a keyword of the second set is the field identification of the field, and a value of the second set is a field object corresponding to the field.
According to one or more embodiments of the present application, the method further comprises:
an array is created in the cache and the one or more field objects are added to the array.
According to one or more embodiments of the present application, the storing the item information set under the target field object includes:
and converting the item information set based on a data conversion rule, wherein the data conversion rule is preset according to the data format of the pivot table, and storing the converted item information set under the target field object.
According to one or more embodiments of the present application, the traversing the source data to obtain one or more fields includes:
Traversing the head line of the source data to obtain one or more fields in the head line, wherein the one or more fields in the head line are fields of different types; or,
traversing the first column of the source data to obtain one or more fields in the first column, wherein the one or more fields in the first column are fields of different types.
According to one or more embodiments of the present application, the method further comprises:
a pivot table associated with the source data is created based on the field objects, the target field information set, and the item information set stored in the cache.
According to one or more embodiments of the present application, the method further comprises:
responding to the operation of changing the configuration field in the pivot table by a user, and acquiring a target set matched with the field name or the field identification of the configuration field from the cached set based on the field name or the field identification of the configuration field;
and updating the pivot table according to the field object in the target set and the item information set corresponding to the field object.
According to one or more embodiments of the present application, the method further comprises:
When the source data is changed, updating the cache based on the changed data;
and updating the pivot table based on the updated cache.
According to one or more embodiments of the present application, the method further comprises:
and deleting the cache in response to the operation of deleting the pivot table by the user.
According to one or more embodiments of the present application, there is provided a data processing apparatus, the apparatus comprising:
the first acquisition unit is used for traversing the source data to acquire one or more fields and one or more initial field information sets, wherein the initial field information sets are used for representing relevant information of the fields;
a first storage unit configured to store one or more field objects and one or more target field information sets in a cache based on the one or more fields and the one or more initial field information sets;
a second obtaining unit, configured to obtain, for a target field in the one or more fields, a set of item information corresponding to the target field in the source data;
and the second storage unit is used for acquiring a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache.
In one or more embodiments of the present application, the one or more fields are in one-to-one correspondence with the one or more initial field information sets, for any one of the one or more fields and an initial field information set corresponding to the field, the initial field information set includes at least one of a field name and a field identifier of the field, and the first storage unit is specifically configured to, when the initial field information set includes the field name and the field identifier of the field, create, in the cache, a first set and a second set corresponding to the field based on the field, the field name and the field identifier, a keyword of the first set is a field name of the field, a value of the first set is a field object corresponding to the field, a keyword of the second set is a field identifier of the field, and a value of the second set is a field object corresponding to the field.
In one or more embodiments of the present application, the apparatus further comprises: a creation unit;
the creation unit is configured to create an array in the cache, and add the one or more field objects to the array.
In one or more embodiments of the present application, the second storage unit is specifically configured to convert the item information set based on a data conversion rule, where the data conversion rule is predetermined according to a data format of a pivot table, and store the converted item information set under the target field object.
In one or more embodiments of the present application, the first obtaining unit is specifically configured to traverse a first row of the source data to obtain one or more fields in the first row, where the one or more fields in the first row are fields of different types; or traversing the first column of the source data to obtain one or more fields in the first column, wherein the one or more fields in the first column are fields of different types.
In one or more embodiments of the present application, the creating unit is further configured to create a pivot table associated with the source data based on the field object, the target field information set, and the item information set stored in the cache.
In one or more embodiments of the present application, the apparatus further comprises: an updating unit;
the updating unit is used for responding to the operation of changing the configuration field in the pivot table by a user, and acquiring a target set matched with the field name or the field identifier of the configuration field from the cached set based on the field name or the field identifier of the configuration field; and updating the pivot table according to the field object in the target set and the item information set corresponding to the field object.
In one or more embodiments of the present application, the updating unit is further configured to update the cache based on the changed data when the source data is changed; and updating the pivot table based on the updated cache.
In one or more embodiments of the present application, the apparatus further comprises: a deletion unit;
and the deleting unit is used for deleting the cache in response to the operation of deleting the pivot table by a user.
According to one or more embodiments of the present application, there is provided an electronic device, the device comprising: a processor and a memory;
the memory is used for storing instructions or computer programs;
the processor is configured to execute the instructions or the computer program in the memory, so that the electronic device executes the data processing method.
According to one or more embodiments of the present application, there is provided a computer-readable storage medium having instructions stored therein, which when executed on a device, cause the device to perform the method of processing data.
It should be noted that, in the present description, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different manner from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system or device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant points refer to the description of the method section.
It should be understood that in this application, "at least one" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
It is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (12)

1. A method of processing data, the method comprising:
traversing the source data to obtain one or more fields and one or more initial field information sets, wherein the initial field information sets are used for representing relevant information of the fields;
Storing one or more field objects and one or more target field information sets in a cache based on the one or more fields and the one or more initial field information sets;
aiming at a target field in the one or more fields, acquiring an item information set corresponding to the target field in the source data;
and acquiring a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache.
2. The method of claim 1, wherein the one or more fields are in one-to-one correspondence with the one or more initial field information sets, wherein for any one of the one or more fields and an initial field information set corresponding to the field, the initial field information set includes at least one of a field name and a field identification of the field, wherein storing one or more field objects in a cache based on the one or more fields and the one or more initial field information sets comprises:
when the initial field information set comprises a field name and a field identification of the field, a first set and a second set corresponding to the field are created in the cache based on the field, the field name and the field identification, a keyword of the first set is the field name of the field, a value of the first set is a field object corresponding to the field, a keyword of the second set is the field identification of the field, and a value of the second set is a field object corresponding to the field.
3. The method according to claim 2, wherein the method further comprises:
an array is created in the cache and the one or more field objects are added to the array.
4. The method of claim 1, wherein storing the set of item information under the target field object comprises:
and converting the item information set based on a data conversion rule, wherein the data conversion rule is preset according to the data format of the pivot table, and storing the converted item information set under the target field object.
5. The method of claim 1, wherein traversing the source data to obtain one or more fields comprises:
traversing the head line of the source data to obtain one or more fields in the head line, wherein the one or more fields in the head line are fields of different types; or,
traversing the first column of the source data to obtain one or more fields in the first column, wherein the one or more fields in the first column are fields of different types.
6. The method according to claim 2, wherein the method further comprises:
A pivot table associated with the source data is created based on the field objects, the target field information set, and the item information set stored in the cache.
7. The method of claim 6, wherein the method further comprises:
responding to the operation of changing the configuration field in the pivot table by a user, and acquiring a target set matched with the field name or the field identification of the configuration field from the cached set based on the field name or the field identification of the configuration field;
and updating the pivot table according to the field object in the target set and the item information set corresponding to the field object.
8. The method of claim 6, wherein the method further comprises:
when the source data is changed, updating the cache based on the changed data;
and updating the pivot table based on the updated cache.
9. The method of claim 6, wherein the method further comprises:
and deleting the cache in response to the operation of deleting the pivot table by the user.
10. A data processing apparatus, the apparatus comprising:
The first acquisition unit is used for traversing the source data to acquire one or more fields and one or more initial field information sets, wherein the initial field information sets are used for representing relevant information of the fields;
a first storage unit configured to store one or more field objects and one or more target field information sets in a cache based on the one or more fields and the one or more initial field information sets;
a second obtaining unit, configured to obtain, for a target field in the one or more fields, a set of item information corresponding to the target field in the source data;
and the second storage unit is used for acquiring a target field object corresponding to the target field in the cache, and storing the item information set under the target field object when the item information set does not exist in the cache.
11. An electronic device, the device comprising: a processor and a memory;
the memory is used for storing instructions or computer programs;
the processor for executing the instructions or computer program in the memory to cause the electronic device to perform the method of any of claims 1-9.
12. A computer readable storage medium having instructions stored therein which, when executed on a device, cause the device to perform the method of any of claims 1-9.
CN202310363821.0A 2023-04-06 2023-04-06 Data processing method, device, equipment and storage medium Pending CN116340364A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310363821.0A CN116340364A (en) 2023-04-06 2023-04-06 Data processing method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310363821.0A CN116340364A (en) 2023-04-06 2023-04-06 Data processing method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116340364A true CN116340364A (en) 2023-06-27

Family

ID=86884002

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310363821.0A Pending CN116340364A (en) 2023-04-06 2023-04-06 Data processing method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116340364A (en)

Similar Documents

Publication Publication Date Title
CN109656923B (en) Data processing method and device, electronic equipment and storage medium
CN115757400B (en) Data table processing method, device, electronic equipment and computer readable medium
CN111400625B (en) Page processing method and device, electronic equipment and computer readable storage medium
CN112906394A (en) Address recognition method, device, equipment and storage medium
CN111241137B (en) Data processing method, device, electronic equipment and storage medium
CN112307061A (en) Method and device for querying data
CN111262907B (en) Service instance access method and device and electronic equipment
CN111597439A (en) Information processing method and device and electronic equipment
CN111753238A (en) Data mapping method and device and electronic equipment
CN111782895B (en) Retrieval processing method and device, readable medium and electronic equipment
WO2022242441A1 (en) Spreadsheet importing method, apparatus, and device, and medium
CN113157695B (en) Data processing method and device, readable medium and electronic equipment
CN112100211B (en) Data storage method, apparatus, electronic device, and computer readable medium
CN111143355B (en) Data processing method and device
CN116340364A (en) Data processing method, device, equipment and storage medium
CN114040014B (en) Content pushing method, device, electronic equipment and computer readable storage medium
CN110780898A (en) Page data upgrading method and device and electronic equipment
CN113032808B (en) Data processing method and device, readable medium and electronic equipment
CN111240657A (en) Method and device for generating SDK and electronic equipment
CN111984645B (en) Data processing method, device, medium and electronic equipment
CN116483808B (en) Data migration method, device, electronic equipment and computer readable medium
CN109739652A (en) Real-time long connection method and device based on Go language
CN110619093B (en) Method, apparatus, electronic device, and computer-readable storage medium for determining an order of search items
CN111782831B (en) Media file display method and device, media and electronic equipment
CN114691613A (en) File management method and device, readable medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination