CN109597865B - Massive geographic information data storage and retrieval method based on embedded platform - Google Patents
Massive geographic information data storage and retrieval method based on embedded platform Download PDFInfo
- Publication number
- CN109597865B CN109597865B CN201811392731.XA CN201811392731A CN109597865B CN 109597865 B CN109597865 B CN 109597865B CN 201811392731 A CN201811392731 A CN 201811392731A CN 109597865 B CN109597865 B CN 109597865B
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- geographic information
- information data
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention belongs to the technical field of data scheduling algorithms, and particularly relates to a storage and retrieval method of massive geographic information data based on an embedded platform; the method combines partitioned storage and classified storage aiming at geographic information data; the data is subjected to partition storage, indexes are established in a partition mode, the data are classified and stored into massive terrain data and raster data in different storage organization forms respectively, and the terrain data are stored in a differential integer mode; the raster data is stored by adopting a compression format of direct decompression; the retrieval mode of the massive geographic information data is that a plurality of partitioned files are organized into one file, an index is established for each file, and the index contains partitioned data information in the file; by the storage organization form and the retrieval mode, large-scale storage, reading and scheduling of massive geographic information data under an embedded platform are realized; and greatly improves the reading and dispatching speed of the geographic information data.
Description
Technical Field
The invention belongs to the technical field of data scheduling algorithms, and particularly relates to a massive geographic information data storage and retrieval method based on an embedded platform.
Background
With the development of geographic information technology and mapping technology, the data volume and quality of geographic information data are rapidly improved, the traditional geographic information data storage algorithm and scheduling algorithm use a typical file mode or a large-scale database to manage the geographic information data, which can cause large software scale, slow scheduling of the geographic information data and slow refreshing of display data, and can be only used at a large server end and a PC (personal computer).
Disclosure of Invention
The purpose of the invention is: the method for storing and retrieving the massive geographic information data based on the embedded platform is provided, and rapid data scheduling and display can be performed on the massive geographic information data in a large scale under the embedded platform.
In order to solve the technical problem, the technical scheme of the invention is as follows:
the storage and retrieval method of massive geographic information data based on an embedded platform combines partitioned storage and classified storage aiming at the geographic information data; the partitioned storage is used for carrying out partitioned area indexing on the data, and scheduling and displaying the specified geographic information data in a specified range; the classified storage is that different storage organization forms are respectively adopted for mass terrain data and raster data, and the terrain data adopts a differential integer storage mode; the raster data is stored in a compressed format with direct decompression.
The retrieval mode of the massive geographic information data in the storage and retrieval method is to organize a plurality of partition files into one file, establish an index for each file, and the index comprises partition data information in the file.
The specific scheduling mode of the partitioned storage is as follows: and under different display scales, different levels of geographic information data are called.
The storage mode of the integral difference is specifically as follows: and converting the difference value of the terrain data of the double-precision type into integer data for storage.
The compression format of the direct decompression is DDS format.
The index file comprises the following information: the file name of the index file, the longitude and latitude coordinates and the altitude of the four corner points of the tile.
The retrieval method specifically comprises the following steps: and obtaining the partition data content to be scheduled through the partition data information in the index file, and reading the geographic information data.
The invention has the beneficial effects that:
the storage and retrieval method of the massive geographic information data based on the embedded platform designs the partition display principle of the massive geographic information data and the massive geographic information retrieval mode based on file index, designs the storage organization form of massive terrain data and raster data, and realizes the storage, reading and scheduling of the massive geographic information data under the embedded platform in a large scale; and the reading and scheduling speed of the geographic information data is greatly improved, a key basic technology is provided for the research and development of geographic information software in the embedded field, and the application range of the geographic information software under an embedded platform is promoted.
Drawings
FIG. 1 a LOD hierarchy of geographic information data;
FIG. 2 is a geographic information data file naming diagram of the present invention.
Detailed Description
The invention is further illustrated with reference to the following figures and examples:
the invention discloses a storage and retrieval method of massive geographic information data based on an embedded platform, which comprises the following steps: the method comprises the following steps of a partitioning mode of massive geographic information data, a retrieval mode of the massive geographic information data, a storage organization mode of massive terrain data and a storage organization mode of massive grid data.
Partitioning mode of mass geographic information data: and establishing indexes by regions according to the geographic information data, and scheduling and displaying the specified geographic information data in a specified range. According to the actual display capability of the embedded platform, LOD (level of detail) grading processing is carried out on geographic information data in the global range. And under different display scales, different levels of geographic information data are called.
Retrieval mode of massive geographic information data: based on the consideration of improving the file reading speed, the data is reorganized and aggregated, and a plurality of partitioned files are organized into one file. In an embedded environment, scheduling of massive geographic information data needs to be achieved within a short time, and therefore an index file needs to be established before partition data is scheduled. And obtaining the partition data content to be scheduled through the partition data information in the index file, and then reading the geographic information data.
Storage organization form of massive terrain data: in an embedded environment, an excessive amount of terrain data may not be stored and severely affect the data update speed. Therefore, the integral processing of the difference value is adopted for the terrain data.
Storage organization form of mass raster data: the DDS data compression format can be directly transmitted to the video memory without CPU decompression, and the DDS data compression format can not be decompressed after the video memory, so that the use amount of the video memory is greatly reduced, the texture loading speed is improved, and great advantages are achieved. Therefore, in an embedded environment, a DDS compression format which can be directly decompressed by the GPU is adopted as a format of raster data, and the decompression efficiency is improved.
The specific description is as follows:
partitioning mode of mass geographic information data: the content of the massive geographic information data is various, and comprises contents such as terrain, grids, vectors and the like, and the amount of the geographic information data covering the global range is large and can reach TB magnitude. Under the magnitude, in the embedded environment, it is necessary to establish an index by regions for geographic information data, perform LOD (level of detail) classification processing on geographic information data in a global range, and display geographic data of different levels under different display scales according to the display capability of an actual display.
The earth is modeled according to a regular sphere model, and the geographic information data is organized according to longitude and latitude as coordinate axes. The geographic information data is divided into eight blocks in the global scope, as shown in fig. 1. Then, in each block, four blocks are equally divided, and the blocks are divided in sequence. Finally, an LOD data partitioning approach is formed with an initial level of 8 blocks, and then 4 blocks are split down each level.
Retrieval mode of massive geographic information data: the range of each partition data is defined by the geographic information data according to the partition principle, the number of each partition file of the geographic information data is very large under the condition of very large data information, and 44739240 partition data exist in the 11-level global range. In an embedded environment, an operating system has a limit on the total number of files, and the number of files is too large, which causes frequent file opening and closing operations during program operation, and affects the scheduling speed and the display speed of geographic information data. Based on the consideration of improving the file reading speed, the partitioned files are reorganized and aggregated, and a plurality of partitioned files are organized into one file. And recombining the data according to the idea of a quadtree.
Examples are: as shown in FIG. 2, 10920 partition data of level 4, partition data of X coordinate 6, partition data of Y coordinate 47, and partition data of quad-tree of 6 levels down are organized into one file named 4 \u6 \u47 \u6.
The retrieval mode of the massive geographic information data comprises the following steps: in an embedded environment, scheduling of massive geographic information data needs to be completed within a short time, so that index files need to be established when partition data is scheduled. And acquiring the information of the tiles from the index file, and reading the geographic information data. The index file mainly comprises longitude and latitude coordinates and altitude of four corner points of the tile, and the file name of the index file adopts an aggregate file name. When software is initialized, firstly reading index file information, constructing a data tree, and scheduling and reading tile data through the tile information of the data tree.
The storage organization form of the massive terrain data is as follows: in an embedded environment, because the storage space of a hard disk is limited, all terrain data of double-precision type cannot be stored, and because the updating condition is limited, the loading and updating speed of data can be seriously influenced by an overlarge amount of terrain data. The terrain data is thus subjected to difference processing and then processed as a double-byte integer.
Examples are as follows: if the height of a certain point is 2565.36 m, if the double-precision type storage is used, 8 bytes are needed, because the terrain data is stored in a partitioned mode, the difference value of (2565.36-2142.12) =423.24 m by taking 2142.12 m at the lower left corner of the block data as a reference, and then 423.24 × 100 multiples =42324 are adopted for the difference value storage and integral processing. The Short type of storage space is 0-65536 and can store this value, occupying 2 bytes.
The storage organization form of the massive grid data is as follows: in an embedded environment, the storage space is limited, and a compression algorithm is required to be adopted for raster data, so that the storage space is reduced. The traditional image compression algorithms include compression algorithms such as JPG and PNG, which meet the requirements on compression ratio, but have low decompression efficiency in an embedded environment and influence the data reading speed in the embedded environment. Due to the compression advantages of DDS. The DDS compression format is thus adopted as the format of the raster data.
Examples are: in the raster file of 4_6_47_6, file information is stored first, including data type, version information, quad-tree hierarchy, tile size, which refers to the pixel size of the raster, and 256 × 256 is used in an embedded environment, which can satisfy the balance of speed and effect. Then, four blocks are divided downwards from the tiles of 4-6-47, and the tiles are arranged in the order of upper left, upper right, lower left and lower right, and the tile data is stored in the order. In the index mode, the offset address and the length of the data content are stored firstly, and then the grid data are read.
Claims (2)
1. The storage and retrieval method of massive geographic information data based on the embedded platform is characterized in that: the storage and retrieval method combines partitioned storage and classified storage aiming at geographic information data; the partitioned storage is used for carrying out partitioned area indexing on the data, and scheduling and displaying the specified geographic information data in a specified range; the classified storage is that different storage organization forms are respectively adopted for mass terrain data and raster data, and the terrain data adopts a differential integer storage mode; the raster data is stored by adopting a compression format of direct decompression;
the retrieval mode of the massive geographic information data in the storage and retrieval method is that a plurality of partitioned files are organized into one file, an index is established for each file, and the index contains partitioned data information in the file;
the specific scheduling mode of the partitioned storage is as follows: under different display scales, geographic information data of different levels are called;
the storage mode of the integral difference is specifically as follows: performing difference on the terrain data of the double-precision type, converting the difference into integer data and storing the integer data;
the index file comprises the following information: indexing the file name of the file, and the longitude and latitude coordinates and the altitude of four corner points of the tile;
the compression format of the direct decompression is a DDS format.
2. The method for storing and retrieving massive geographic information data based on an embedded platform according to claim 1, wherein the method comprises the following steps: the retrieval method specifically comprises the following steps: and obtaining the partition data content to be scheduled through the partition data information in the index file, and reading the geographic information data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811392731.XA CN109597865B (en) | 2018-11-21 | 2018-11-21 | Massive geographic information data storage and retrieval method based on embedded platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811392731.XA CN109597865B (en) | 2018-11-21 | 2018-11-21 | Massive geographic information data storage and retrieval method based on embedded platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109597865A CN109597865A (en) | 2019-04-09 |
| CN109597865B true CN109597865B (en) | 2022-11-04 |
Family
ID=65960265
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811392731.XA Active CN109597865B (en) | 2018-11-21 | 2018-11-21 | Massive geographic information data storage and retrieval method based on embedded platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109597865B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262755A (en) * | 2019-06-20 | 2019-09-20 | 福建北峰通信科技股份有限公司 | A kind of file memory method of embedded system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20070040732A (en) * | 2005-10-12 | 2007-04-17 | 소니 가부시끼 가이샤 | Data management device and method for managing recording medium |
| CN103412962A (en) * | 2013-09-04 | 2013-11-27 | 国家测绘地理信息局卫星测绘应用中心 | Storage method and reading method for mass tile data |
| CN105718481A (en) * | 2014-12-05 | 2016-06-29 | 星际空间（天津）科技发展有限公司 | Massive topographic data organization and release method |
-
2018
- 2018-11-21 CN CN201811392731.XA patent/CN109597865B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20070040732A (en) * | 2005-10-12 | 2007-04-17 | 소니 가부시끼 가이샤 | Data management device and method for managing recording medium |
| CN103412962A (en) * | 2013-09-04 | 2013-11-27 | 国家测绘地理信息局卫星测绘应用中心 | Storage method and reading method for mass tile data |
| CN105718481A (en) * | 2014-12-05 | 2016-06-29 | 星际空间（天津）科技发展有限公司 | Massive topographic data organization and release method |
Non-Patent Citations (1)
| Title |
|---|
| 基于海量存储的三维地理信息系统应用与研究;宋娟等;《计算机技术与发展》;20171205(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109597865A (en) | 2019-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111291016B (en) | Hierarchical hybrid storage and indexing method for massive remote sensing image data | |
| US9877031B2 (en) | System and method for multi-resolution raster data processing | |
| CN107402958B (en) | Method for creating and accessing GKF three-dimensional spatial database | |
| CN101916299B (en) | Three-dimensional spatial data storage and management method based on file system | |
| CN110543716B (en) | Three-dimensional overhead cable hierarchical power grid optimization method, device and computer equipment | |
| CN104657436B (en) | Static tile pyramid parallel constructing method based on MapReduce | |
| Hwa et al. | Adaptive 4-8 texture hierarchies | |
| GB2507838A (en) | Compressing/decompressing vertex parameter data | |
| CN108804602A (en) | A kind of distributed spatial data storage computational methods based on SPARK | |
| Purss et al. | The OGC® Discrete Global Grid System core standard: A framework for rapid geospatial integration | |
| CN112380302B (en) | Thermodynamic diagram generation method and device based on track data, electronic equipment and storage medium | |
| US20230281182A1 (en) | R-tree index merging and updating method and apparatus based on hilbert curve, and medium | |
| US10013474B2 (en) | System and method for hierarchical synchronization of a dataset of image tiles | |
| CN115408382B (en) | Vector space information gridding storage management method for large-scale satellite data | |
| CN109597865B (en) | Massive geographic information data storage and retrieval method based on embedded platform | |
| CN112843700A (en) | Terrain image generation method and device, computer equipment and storage medium | |
| CN110765298B (en) | Tile coding method for decoupling geometric attributes of vector data | |
| CN117237503B (en) | Geographic element data accelerated rendering and device | |
| CN115471971B (en) | Basin simulation stage data processing method and device and computer readable storage medium | |
| US12061784B2 (en) | Generating aggregate data geospatial grid cells for encoding in vector tiles | |
| CN116185305A (en) | Service data storage method, device, computer equipment and storage medium | |
| CN115858659A (en) | Vector track display method and device, computer equipment and storage medium | |
| CN115658693A (en) | Efficient optimized storage method suitable for massive raster tile data | |
| CN118298114B (en) | Tree structure of point cloud double-layer spatial index and multi-resolution point cloud divide-and-conquer construction algorithm | |
| CN104809217B (en) | A kind of GIS raster datas cloud storage method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |