CN116383317A - Remote measurement data storage method and device, electronic equipment and storage medium - Google Patents
Remote measurement data storage method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116383317A CN116383317A CN202310366756.7A CN202310366756A CN116383317A CN 116383317 A CN116383317 A CN 116383317A CN 202310366756 A CN202310366756 A CN 202310366756A CN 116383317 A CN116383317 A CN 116383317A
- Authority
- CN
- China
- Prior art keywords
- telemetry
- source packet
- telemetry source
- type
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000013500 data storage Methods 0.000 title abstract description 23
- 238000005259 measurement Methods 0.000 title description 2
- 230000006870 function Effects 0.000 claims description 28
- 238000004458 analytical method Methods 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 7
- 238000012905 input function Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 3
- 230000006835 compression Effects 0.000 description 7
- 238000007906 compression Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000003780 insertion Methods 0.000 description 6
- 230000037431 insertion Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/1851—Systems using a satellite or space-based relay
- H04B7/18519—Operations control, administration or maintenance
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Astronomy & Astrophysics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Selective Calling Equipment (AREA)
Abstract
The application discloses a telemetry data storage method, a telemetry data storage device, electronic equipment and a telemetry data storage medium; the method comprises the following steps: if at least one telemetry source packet exists in the telemetry data stream, extracting one telemetry source packet from the telemetry data stream as a current telemetry source packet; classifying the current telemetry source packet to obtain the type of the current telemetry source packet; and according to the type of the current telemetry source packet, directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet. According to the embodiment of the application, the telemetry data can be quickly stored in the database, the storage space of the telemetry data can be effectively saved, and the storage efficiency of the telemetry data is greatly improved.
Description
Technical Field
The embodiment of the application relates to the technical field of aerospace, in particular to a telemetry data storage method, a telemetry data storage device, electronic equipment and a telemetry data storage medium.
Background
Telemetry data of satellites is of great importance for monitoring, analyzing the operating conditions of satellites and loads. With the construction of large-scale constellations, long-term telemetry data storage and querying of multiple satellites poses a serious challenge to commercial satellite companies.
The traditional satellite data processing flow is to analyze the received telemetry data and store the telemetry data in a corresponding database. This method is a normal and general method when storing telemetry data for a short period of time. However, as single satellite telemetry accuracy increases, parameters increase, constellation scale increases and run time increases, telemetry data in databases has a dramatically increasing trend. Compression and storage of telemetry data is a necessary solution.
For example, a telemetry source packet is 16 bytes, the data obtained after parsing is huge, if the data is compressed on the basis, the enumeration quantity of normal and abnormal compression can certainly be only according to the character string compression method, at least 7 bytes or 8 bytes, the actual source code represents only 1 bit, and the ratio of the original code stream cannot be reached in any way in the later compression.
Disclosure of Invention
The application provides a telemetry data storage method, a device, an electronic device and a storage medium, which can rapidly store telemetry data in a database, can effectively save the storage space of telemetry data and greatly improve the storage efficiency of telemetry data.
In a first aspect, an embodiment of the present application provides a method for storing telemetry data, the method including:
extracting a telemetry source packet from a telemetry data stream as a current telemetry source packet if at least one telemetry source packet is present in the telemetry data stream;
classifying the current telemetry source packet to obtain the type of the current telemetry source packet;
and directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet according to the type of the current telemetry source packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet.
In a second aspect, embodiments of the present application further provide a device for storing telemetry data, the device including: the device comprises an extraction module, a classification module and a storage module; wherein,,
the extraction module is used for extracting a telemetry source packet from the telemetry data stream as a current telemetry source packet if at least one telemetry source packet exists in the telemetry data stream;
the classification module is used for classifying the current telemetry source packet to obtain the type of the current telemetry source packet;
and the storage module is used for directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet according to the type of the current telemetry source packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet.
In a third aspect, an embodiment of the present application provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of storing telemetry data described in any of the embodiments of the present application.
In a fourth aspect, embodiments of the present application provide a storage medium having stored thereon a computer program which, when executed by a processor, implements a method of storing telemetry data as described in any of the embodiments of the present application.
The embodiment of the application provides a method, a device, electronic equipment and a storage medium for storing telemetry data, wherein if at least one telemetry source packet exists in a telemetry data stream, one telemetry source packet is extracted from the telemetry data stream and used as a current telemetry source packet; classifying the current telemetry source packet to obtain the type of the current telemetry source packet; and then according to the type of the current telemetry source packet, directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet. That is, in the technical solution of the present application, each telemetry source packet may be directly stored in a corresponding database according to a type, and since each telemetry source packet is not parsed during storage, the storage space may be reduced to the greatest extent. In the prior art, each telemetry source packet is firstly analyzed, then the analysis result is stored, and because the data obtained after the analysis is very huge, even if the data is compressed on the basis, the data occupies a large storage space. Therefore, compared with the prior art, the telemetry data storage method, device, electronic equipment and storage medium provided by the embodiment of the application can be used for rapidly storing telemetry data into a database, and can be used for effectively saving the storage space of the telemetry data, so that the storage efficiency of the telemetry data is greatly improved; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Drawings
FIG. 1 is a first flow chart of a method for storing telemetry data according to an embodiment of the present disclosure;
FIG. 2 is a second flow chart of a method for storing telemetry data according to an embodiment of the present application;
FIG. 3 is a third flow chart of a method for storing telemetry data according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a telemetry data storage device according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present application are shown in the drawings.
Example 1
Fig. 1 is a schematic flow chart of a first method for storing telemetry data according to an embodiment of the present application, where the method may be performed by a telemetry data storage device or an electronic device, and the device or the electronic device may be implemented in software and/or hardware, and the device or the electronic device may be integrated into any intelligent device with a network communication function. As shown in fig. 1, the method of storing telemetry data may include the steps of:
s101, if at least one telemetry source packet exists in the telemetry data stream, extracting one telemetry source packet from the telemetry data stream as a current telemetry source packet.
In this step, the electronic device may first receive a telemetry data stream issued by the satellite; each telemetry source packet is then extracted from the telemetry data stream according to a predetermined packet start position and packet end position. Specifically, assume that there are N telemetry source packets in the telemetry data stream; wherein N is a natural number greater than 1; the electronic device may extract telemetry source packets one by one in the telemetry data stream and then take the extracted telemetry source packet as the current telemetry source packet.
S102, classifying the current telemetry source packet to obtain the type of the current telemetry source packet.
In this step, the electronic device may classify the current telemetry source packet to obtain a type of the current telemetry source packet. A telemetry data stream contains telemetry data for a plurality of devices on a satellite, each device comprising its own set of telemetry data, typically distinguished by the class number of the telemetry source packet.
S103, according to the type of the current telemetry source packet, directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet.
In this step, the electronic device may directly store the current telemetry source packet into the database corresponding to the type of the current telemetry data packet according to the type of the current telemetry source packet, and repeatedly execute the above operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet. Specifically, the electronic device may first convert the format of the current telemetry source packet from a hexadecimal text format to a binary bit stream format through a predefined input function corresponding to the type of the current telemetry source packet; and then directly storing the current telemetry source packet in the binary bit stream format into a database corresponding to the type of the current telemetry source packet.
Current databases typically support custom data types, e.g., postgreSQL supports custom types defined by the C language. Insertion, replication and export of individual telemetry source packages is accomplished by defining a telemetry source package as a custom type and defining the necessary operating functions or operators thereon. The following definitions are generally required: 1) An input function responsible for converting hexadecimal text telemetry source packet data into an internal binary representation; 2) An output function responsible for converting the telemetry source code data of the internal binary representation to hexadecimal text; 3) Converting to JSON, which is responsible for converting telemetry source code data of the internal binary representation to JSON representation; 4) From JSON conversion, it is responsible for converting telemetry source code data of JSON representation to internal binary representation. The above functions 1) and 2) are inverse functions to each other, which are simply computer-represented transformations. And functions 3) and 4) are also inverse functions to each other, requiring parsing and encoding according to the definition of the telemetry source package.
In the method for storing telemetry data provided by the embodiment of the application, if at least one telemetry source packet exists in a telemetry data stream, extracting one telemetry source packet from the telemetry data stream as a current telemetry source packet; classifying the current telemetry source packet to obtain the type of the current telemetry source packet; and then according to the type of the current telemetry source packet, directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet. That is, in the technical solution of the present application, each telemetry source packet may be directly stored in a corresponding database according to a type, and since each telemetry source packet is not parsed during storage, the storage space may be reduced to the greatest extent. In the prior art, each telemetry source packet is firstly analyzed, then the analysis result is stored, and because the data obtained after the analysis is very huge, even if the data is compressed on the basis, the data occupies a large storage space. Therefore, compared with the prior art, the telemetry data storage method provided by the embodiment of the application can be used for rapidly storing telemetry data in a database, and can also be used for effectively saving the storage space of the telemetry data, so that the telemetry data storage efficiency is greatly improved; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example two
Fig. 2 is a second flow chart of a method for storing telemetry data according to an embodiment of the present application. Further optimization and expansion based on the above technical solution can be combined with the above various alternative embodiments.
As shown in fig. 2, the method of storing telemetry data may include the steps of:
s201, if at least one telemetry source packet exists in the telemetry data stream, extracting one telemetry source packet from the telemetry data stream as a current telemetry source packet.
S202, classifying the current telemetry source packet to obtain the type of the current telemetry source packet.
S203, according to the type of the current telemetry source packet, directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet.
S204, receiving a query request sent by target equipment; the query request at least comprises the type and the identification of the target telemetry source packet.
S205, searching the target telemetry source packet in a database corresponding to the type of the target telemetry source packet in response to the query request.
In this step, the electronic device may search for the target telemetry source package in a database corresponding to the type of the target telemetry source package in response to the query request. Specifically, the electronic device may first find the base data of the target telemetry source packet in the base column of the database corresponding to the type of the target telemetry source packet; if the basic data of the target telemetry packet is found in the basic column, the electronic equipment can continuously judge whether a derived column exists in the database corresponding to the type of the target telemetry source packet; if the derived column exists in the database corresponding to the type of the target telemetry source packet, the electronic equipment can search the derived data of the target telemetry source packet in the derived column; if the derived data of the target telemetry source packet is found in the derived column, the electronic device may use the base data and the derived data of the target telemetry source packet as a result of the search of the target telemetry source packet.
Optionally, if the derived column does not exist in the database corresponding to the type of the target telemetry source packet, or derived data of the target telemetry source packet is not found in the derived column, the electronic device may use the base data of the target telemetry source packet as a search result corresponding to the target telemetry source packet.
S206, if the target telemetry source packet is found in the database corresponding to the type of the target telemetry source packet, analyzing the target telemetry source packet through a predefined analysis function to obtain an analysis result corresponding to the target telemetry source packet.
In this step, if the target telemetry source packet is found in the database corresponding to the type of the target telemetry source packet, the electronic device may parse the target telemetry source packet through a predefined parsing function, so as to obtain a parsing result corresponding to the target telemetry source packet. Specifically, the electronic device may convert the format of the target telemetry source packet from a binary bit stream format to a JSON format through a predefined parsing function corresponding to the type of the target telemetry source packet; taking the current telemetry source packet in the JSON format as an analysis result corresponding to the target telemetry source packet; wherein the JSON format includes at least the following fields: code, name, type and description. For example, the part of the global navigation satellite system (Global Navigation Satellite System, abbreviated as GNSS) source package of a certain satellite is defined as shown in the following table 1:
TABLE 1
For example, a telemetry source packet is transmitted in hexadecimal notation as:
AA0409 … 234923E2608E6FFD29AFA37800AA0139 …; after parsing, JSON was obtained as:
the resulting data volume is large and there is a structural overhead for additional recording of the ID name, in addition to the decoding reasons.
If the hexadecimal representation of the telemetry source packet at the next time is received is:
AA0409 … 234923E2608E6FFD29AFA37700AB0138 …; after parsing, JSON was obtained as:
it can be seen that the difference in telemetry representation at the front and rear moments after parsing becomes larger, which is more obvious for equivalent engineering quantities. This is very disadvantageous for compression on a time scale. In general, the smaller the back-and-forth variation, the more advantageous it is for a time series to achieve a greater proportion of compression. In addition, the telemetry source packages are stored in the database according to the types, and the problem of how to store the telemetry source packages needs to be solved.
Current databases typically support custom data types, such as PostgreSQL supports custom types defined by the C language. Insertion, replication and export of individual telemetry source packages is accomplished by defining a telemetry source package as a custom type and defining the necessary operating functions or operators thereon. It is generally necessary to define: 1) An input function responsible for converting hexadecimal text telemetry source packet data into an internal binary representation. 2) An output function responsible for converting the telemetry source data of the internal binary representation to hexadecimal text. 3) To JSON, is responsible for converting the telemetry source data of the internal binary representation to JSON representation. 4) From JSON conversion, it is responsible for converting telemetry source code data of JSON representation to internal binary representation. The above functions 1) and 2) are inverse functions to each other, which are simply computer-represented transformations. And functions 3) and 4) are also inverse functions to each other, requiring parsing and encoding according to the definition of the telemetry source package.
It is not enough that a database row is stored only in the binary of the source packet, but also that important parameters are presented for quick querying, and in particular which important parameters are determined according to the subsequent usage scenario. Telemetry source packet collection time is generally an important parameter. Still other applications such as "active-standby state" may be frequently used, defining a specialized extraction resolution function. The above-mentioned input, output, conversion, extraction and analysis are generally defined using a programming language, such as C, golang. Compiled into a dynamic link library and loaded at database start-up. Then defining the functions of the database according to the derived functions of the dynamic link library, and defining the derived columns according to the use condition. The derived columns are in effect redundant fields of custom source packet types for speeding up the search or display. Thus, derived columns are defined only for telemetry fields that are frequently used.
For example, in Postgresql 14:
alter table tbl_xingwu_kuai
add c005 varchar generated always as xingwu_005(packsrc)STORED；
a derivative column c005 is created in the table tbl_xingwu_kuai representing the quick package of the star. The method is generated by a telemetry source packet field packsrc through a custom parsing function xingwu_005; wherein, the xingwu_005 function is a C language defined function, and is used for analyzing a program control enabling parameter from a telemetry source packet of the star rapid packet. The parameter input is a bit indication bit and the output is an "enable", "disable" flag. Thus, the type of derived column is defined as varchar.
S207, sending the analysis result corresponding to the target telemetry source packet to the target equipment.
In the method for storing telemetry data provided by the embodiment of the application, if at least one telemetry source packet exists in a telemetry data stream, extracting one telemetry source packet from the telemetry data stream as a current telemetry source packet; classifying the current telemetry source packet to obtain the type of the current telemetry source packet; and then according to the type of the current telemetry source packet, directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet. That is, in the technical solution of the present application, each telemetry source packet may be directly stored in a corresponding database according to a type, and since each telemetry source packet is not parsed during storage, the storage space may be reduced to the greatest extent. In the prior art, each telemetry source packet is firstly analyzed, then the analysis result is stored, and because the data obtained after the analysis is very huge, even if the data is compressed on the basis, the data occupies a large storage space. Therefore, compared with the prior art, the telemetry data storage method provided by the embodiment of the application can be used for rapidly storing telemetry data in a database, and can also be used for effectively saving the storage space of the telemetry data, so that the telemetry data storage efficiency is greatly improved; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example III
Fig. 3 is a third flow chart of a method for storing telemetry data according to an embodiment of the present application. Further optimization and expansion based on the above technical solution can be combined with the above various alternative embodiments.
As shown in fig. 3, the method of storing telemetry data may include the steps of:
s301, if at least one telemetry source packet exists in the telemetry data stream, extracting one telemetry source packet from the telemetry data stream as a current telemetry source packet.
S302, classifying the current telemetry source packet to obtain the type of the current telemetry source packet.
S303, checking the current telemetry source packet in at least one dimension to obtain a checking result of the current telemetry source packet in each dimension; wherein the at least one dimension includes, but is not limited to: integrity, legitimacy.
In this step, the electronic device may further perform verification on the current telemetry source packet in at least one dimension, to obtain a verification result of the current telemetry source packet in each dimension; wherein the at least one dimension includes, but is not limited to: integrity, legitimacy. The present application still requires parsing after a telemetry source packet is received, but the parsing herein is only for classifying the telemetry source packet and verifying the integrity and legitimacy of the telemetry source packet.
S304, if the verification results of the current telemetry source packet in all dimensions meet the requirements, converting the format of the current telemetry source packet from a hexadecimal text format to a binary bit stream format through an input function corresponding to the type of the predefined current telemetry source packet.
S305, according to the type of the current telemetry source packet, the current telemetry source packet in a binary bit stream format is directly stored in a database corresponding to the type of the current telemetry source packet, and the operation is repeatedly executed until each telemetry source packet in the telemetry data stream is stored in the database corresponding to the type of each telemetry source packet.
S306, extracting key information of each telemetry source packet from key fields of a database corresponding to the type of each telemetry source packet; wherein the key information includes, but is not limited to: acquisition time.
S307, updating the index in the database corresponding to the type of each telemetry source packet according to the key information of each telemetry source packet, and adding the updated index into the database corresponding to the type of each telemetry source packet.
The warehousing process of the telemetry source packet in the embodiment of the application comprises the following operation steps: 1) After the telemetry source packet is received, parsing is still required, although the parsing is only for purposes of classifying the telemetry source packet and verifying the integrity and legitimacy of the telemetry source packet. A telemetry data stream contains telemetry data of a plurality of devices on a satellite, each device forming its own set of telemetry data, typically distinguished by the class number of the telemetry source packet. One function of the parser is to extract individual telemetry source packages and identify the type of telemetry source package based on the class number. And then comparing to the known size, if necessary, calculating a checksum to determine whether the source packet is complete. Also, whether the source packet is valid is determined by judging the sequence number and the key field range, and only the complete and legal source packet can be processed continuously. 2) And calculating the acquisition time of the telemetry source packet so as to be used as the basis of telemetry inquiry. This step also extracts key fields if the underlying database does not use a derived column. The key fields here are fields that the user needs to extract as search, filtering, in order to generate the index in the database. 3) Insert row data of the database is prepared and inserted. Because the telemetry source packet is inserted into the database field entirely, the previously intercepted source packet data stream need only be converted to a hexadecimal string. Upon insertion into the database, an internally defined input function in the database will automatically restore the hexadecimal string to a binary representation. The whole hexadecimal character string inserting mode is far smaller than that in the traditional method, a plurality of fields after analysis are inserted, or the JSON character string is inserted, so that the data size to be transmitted is small. And thus more efficient. 4) If there are telemetry source packets, the source packets continue to be extracted. Otherwise, the insertion is ended. 5) After the insertion is completed, the database indexing command is run. This step may be eliminated if the database is automatically indexed during the insertion process. Since the batch insert operation is put in one transaction, the insert speed can be increased, and the database transaction is closed in this step.
At this time, a series of telemetry data are stored in the database, and only the direct query is needed when the data are queried. If a derived column is defined, it is only necessary to list the derived column in the select statement, and if not, the statement that extracts the field is called in the select statement. The derived columns can also be generated in advance and indexed, so that the speed of display and search can be improved.
By adopting the technical scheme provided by the application, the following advantages can be brought at least: 1) The method for directly storing the satellite telemetry data into the database is designed, and is more efficient than the traditional receiving, analyzing, compressing and storing modes. 2) The telemetry source packets of the same type are generally fixed in length and range from tens to hundreds of bytes, thereby being beneficial to the storage of databases. 3) The same type of telemetry source packets change less in time scale than after parsing, which is more intuitive in the original satellite-to-ground transmission data stream and is more conducive to further compression in time scale. 4) The original information can be reserved to the maximum extent by directly storing the original satellite-to-ground transmitted telemetry data stream, and the data information loss or distortion caused by the imperfection of analysis software in the traditional method is avoided. 5) When the database is queried, the original telemetry data flow is parsed, so that the characteristics of parallel processing of a modern database, a distributed database and the like can be fully utilized to accelerate calculation, and an additional design of own parallel calculation means is not needed.
In the method for storing telemetry data provided by the embodiment of the application, if at least one telemetry source packet exists in a telemetry data stream, extracting one telemetry source packet from the telemetry data stream as a current telemetry source packet; classifying the current telemetry source packet to obtain the type of the current telemetry source packet; and then according to the type of the current telemetry source packet, directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet. That is, in the technical solution of the present application, each telemetry source packet may be directly stored in a corresponding database according to a type, and since each telemetry source packet is not parsed during storage, the storage space may be reduced to the greatest extent. In the prior art, each telemetry source packet is firstly analyzed, then the analysis result is stored, and because the data obtained after the analysis is very huge, even if the data is compressed on the basis, the data occupies a large storage space. Therefore, compared with the prior art, the telemetry data storage method provided by the embodiment of the application can be used for rapidly storing telemetry data in a database, and can also be used for effectively saving the storage space of the telemetry data, so that the telemetry data storage efficiency is greatly improved; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example IV
Fig. 4 is a schematic structural diagram of a telemetry data storage device according to an embodiment of the present application. As shown in fig. 4, the telemetry data storage device includes: an extraction module 401, a classification module 402 and a storage module 403; wherein,,
the extracting module 401 is configured to extract a telemetry source packet from the telemetry data stream as a current telemetry source packet if at least one telemetry source packet exists in the telemetry data stream;
the classification module 402 is configured to classify the current telemetry source packet to obtain a type of the current telemetry source packet;
the storage module 403 is configured to directly store the current telemetry source packet into a database corresponding to the type of the current telemetry data packet according to the type of the current telemetry source packet, and repeatedly perform the above operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet.
The telemetering data storage device can execute the method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment may be found in the method for storing telemetry data provided in any embodiment of the present application.
Example five
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Fig. 5 illustrates a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the present application. The electronic device 12 shown in fig. 5 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 5, the electronic device 12 is in the form of a general purpose computing device. Components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, a bus 18 that connects the various system components, including the system memory 28 and the processing units 16.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from or write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, commonly referred to as a "hard disk drive"). Although not shown in fig. 5, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 18 through one or more data medium interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of the embodiments of the present application.
A program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods in the embodiments described herein.
The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and/or any devices (e.g., network card, modem, etc.) that enable the electronic device 12 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 22. Also, the electronic device 12 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 over the bus 18. It should be appreciated that although not shown in fig. 5, other hardware and/or software modules may be used in connection with electronic device 12, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processing unit 16 executes various functional applications and data processing, such as implementing the telemetry data storage method provided by embodiments of the present application, by running programs stored in the system memory 28.
Example six
Embodiments of the present application provide a computer storage medium.
Any combination of one or more computer readable media may be employed in the computer readable storage media of the embodiments herein. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present application may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
Note that the above is only a preferred embodiment of the present application and the technical principle applied. Those skilled in the art will appreciate that the present application is not limited to the particular embodiments described herein, but is capable of numerous obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the present application. Therefore, while the present application has been described in connection with the above embodiments, the present application is not limited to the above embodiments, but may include many other equivalent embodiments without departing from the spirit of the present application, the scope of which is defined by the scope of the appended claims.
Claims (10)
1. A method of storing telemetry data, the method comprising:
extracting a telemetry source packet from a telemetry data stream as a current telemetry source packet if at least one telemetry source packet is present in the telemetry data stream;
classifying the current telemetry source packet to obtain the type of the current telemetry source packet;
and directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet according to the type of the current telemetry source packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet.
2. The method according to claim 1, wherein the method further comprises:
receiving a query request sent by target equipment; wherein, the inquiry request at least comprises the type and the identification of the target telemetry source packet;
searching the target telemetry source packet in a database corresponding to the type of the target telemetry source packet in response to the query request;
if the target telemetry source package is found in the database corresponding to the type of the target telemetry source package, analyzing the target telemetry source package through a predefined analysis function to obtain an analysis result corresponding to the target telemetry source package;
and sending the analysis result corresponding to the target telemetry source packet to the target equipment.
3. The method of claim 1, wherein prior to directly storing the current telemetry source package in a database corresponding to a type of the current telemetry source package, the method further comprises:
checking the current telemetry source packet in at least one dimension to obtain a checking result of the current telemetry source packet in each dimension; wherein the at least one dimension includes, but is not limited to: integrity, legitimacy;
and if the verification results of the current telemetry source packet in all dimensions meet the requirements, executing the operation of directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry source packet.
4. The method of claim 1, wherein directly storing the current telemetry source package into a database corresponding to a type of the current telemetry source package comprises:
converting the format of the current telemetry source packet from a hexadecimal text format to a binary bit stream format through a predefined input function corresponding to the type of the current telemetry source packet;
and directly storing the current telemetry source packet in a binary bit stream format into a database corresponding to the type of the current telemetry source packet.
5. The method of claim 2, wherein parsing the target telemetry source package by a predefined parsing function to obtain a parsing result corresponding to the target telemetry source package, comprises:
converting the format of the target telemetry source packet from a binary bit stream format to a JSON format through a predefined analytic function corresponding to the type of the target telemetry source packet; taking a current telemetry source packet in a JSON format as an analysis result corresponding to the target telemetry source packet; wherein the JSON format includes at least the following fields: code, name, type and description.
6. The method of claim 2, wherein searching for the target telemetry source package in a database corresponding to a type of the target telemetry source package comprises:
searching basic data of the target telemetry source packet in a basic column of a database corresponding to the type of the target telemetry source packet;
if the basic data of the target telemetry packet is found in the basic column, judging whether a derived column exists in a database corresponding to the type of the target telemetry source packet;
if the database corresponding to the type of the target telemetry source packet is stored in the derivative column, searching the derivative data of the target telemetry source packet in the derivative column;
and if the derived data of the target telemetry source packet is found in the derived column, taking the basic data and the derived data of the target telemetry source packet as the finding result of the target telemetry source packet.
7. The method of claim 1, wherein after storing each telemetry source packet in the telemetry data stream in a database corresponding to a type of each telemetry source packet, the method further comprises:
extracting key information of each telemetry source packet from key fields of a database corresponding to the type of each telemetry source packet; wherein the key information includes, but is not limited to: collecting time;
and updating the index in the database corresponding to the type of each telemetry source packet according to the key information of each telemetry source packet, and adding the updated index into the database corresponding to the type of each telemetry source packet.
8. A device for storing telemetry data, the device comprising: the device comprises an extraction module, a classification module and a storage module; wherein,,
the extraction module is used for extracting a telemetry source packet from the telemetry data stream as a current telemetry source packet if at least one telemetry source packet exists in the telemetry data stream;
the classification module is used for classifying the current telemetry source packet to obtain the type of the current telemetry source packet;
and the storage module is used for directly storing the current telemetry source packet into a database corresponding to the type of the current telemetry data packet according to the type of the current telemetry source packet, and repeatedly executing the operation until each telemetry source packet in the telemetry data stream is stored into the database corresponding to the type of each telemetry source packet.
9. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of storing telemetry data of any of claims 1 to 7.
10. A storage medium having stored thereon a computer program which when executed by a processor implements a method of storing telemetry data according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310366756.7A CN116383317A (en) | 2023-04-07 | 2023-04-07 | Remote measurement data storage method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310366756.7A CN116383317A (en) | 2023-04-07 | 2023-04-07 | Remote measurement data storage method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116383317A true CN116383317A (en) | 2023-07-04 |
Family
ID=86972830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310366756.7A Pending CN116383317A (en) | 2023-04-07 | 2023-04-07 | Remote measurement data storage method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116383317A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070129009A1 (en) * | 2005-12-01 | 2007-06-07 | Electronics And Telecommunications Research Institute | Fault management system using satellite telemetering technology and method thereof |
| CN109857741A (en) * | 2019-01-31 | 2019-06-07 | 重庆零壹空间航天科技有限公司 | A kind of method of selecting and device of telemetry of launch vehicle data |
| CN114500614A (en) * | 2022-04-14 | 2022-05-13 | 北京航天驭星科技有限公司 | Satellite telemetry data processing method and device, electronic equipment and readable medium |
| CN115017150A (en) * | 2022-04-19 | 2022-09-06 | 北京电子工程总体研究所 | High-code-rate telemetering data processing method |
-
2023
- 2023-04-07 CN CN202310366756.7A patent/CN116383317A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070129009A1 (en) * | 2005-12-01 | 2007-06-07 | Electronics And Telecommunications Research Institute | Fault management system using satellite telemetering technology and method thereof |
| CN109857741A (en) * | 2019-01-31 | 2019-06-07 | 重庆零壹空间航天科技有限公司 | A kind of method of selecting and device of telemetry of launch vehicle data |
| CN114500614A (en) * | 2022-04-14 | 2022-05-13 | 北京航天驭星科技有限公司 | Satellite telemetry data processing method and device, electronic equipment and readable medium |
| CN115017150A (en) * | 2022-04-19 | 2022-09-06 | 北京电子工程总体研究所 | High-code-rate telemetering data processing method |
Non-Patent Citations (1)
| Title |
|---|
| 吴健鑫: "基于XML技术的卫星遥测数据处理系统设计与实现", 中国优秀硕士学位论文全文数据库（工程科技Ⅱ辑）, no. 2, pages 031 - 1671 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110347598B (en) | Test script generation method and device, server and storage medium | |
| CN111258966A (en) | Data deduplication method, device, equipment and storage medium | |
| CN109165209B (en) | Data verification method, device, equipment and medium for object types in database | |
| CN111309760A (en) | Data retrieval method, system, device and storage medium | |
| CN112231407B (en) | DDL synchronization method, device, equipment and medium of PostgreSQL database | |
| CN109947804B (en) | Data set query optimization method and device, server and storage medium | |
| CN112035443A (en) | Big data execution method, system, equipment and storage medium based on Linux platform | |
| CN111694866A (en) | Data searching and storing method, data searching system, data searching device, data searching equipment and data searching medium | |
| CN112039975A (en) | Method, device, equipment and storage medium for processing message field | |
| CN113962597A (en) | Data analysis method and device, electronic equipment and storage medium | |
| CN113760839A (en) | Log data compression processing method and device, electronic equipment and storage medium | |
| CN111240971B (en) | Method and device for generating wind control rule test case, server and storage medium | |
| CN108008947B (en) | Intelligent prompting method and device for programming statement, server and storage medium | |
| CN112162922A (en) | Method, device, server and storage medium for determining difference of new and old systems | |
| US8321429B2 (en) | Accelerating queries using secondary semantic column enumeration | |
| CN114385679A (en) | Meter structure inspection method, meter structure inspection device and electronic equipment | |
| CN111177089B (en) | Log processing code generation method and device, computer system and storage medium | |
| CN116405091B (en) | Method and device for analyzing telemetry data, electronic equipment and storage medium | |
| CN113760894A (en) | Data calling method and device, electronic equipment and storage medium | |
| CN116383317A (en) | Remote measurement data storage method and device, electronic equipment and storage medium | |
| CN111930708A (en) | Extension system and method of object tag based on Ceph object storage | |
| CN117271554A (en) | Distributed database view processing method, device, equipment and storage medium | |
| CN111352751A (en) | Data file generation method and device, computer equipment and storage medium | |
| WO2022236973A1 (en) | Data backfilling method and apparatus, electronic device, and storage medium | |
| CN116257545B (en) | Data query method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230704 |