[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2024218875A1 - Encoding device, encoding method, and encoding program - Google Patents

Encoding device, encoding method, and encoding program Download PDF

Info

Publication number
WO2024218875A1
WO2024218875A1 PCT/JP2023/015510 JP2023015510W WO2024218875A1 WO 2024218875 A1 WO2024218875 A1 WO 2024218875A1 JP 2023015510 W JP2023015510 W JP 2023015510W WO 2024218875 A1 WO2024218875 A1 WO 2024218875A1
Authority
WO
WIPO (PCT)
Prior art keywords
removal
data
point
point cloud
encoding
Prior art date
Application number
PCT/JP2023/015510
Other languages
French (fr)
Japanese (ja)
Inventor
勇 五十嵐
隆一 谷田
小軍 ウ
真由子 渡邊
潤 島村
Original Assignee
日本電信電話株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電信電話株式会社 filed Critical 日本電信電話株式会社
Priority to PCT/JP2023/015510 priority Critical patent/WO2024218875A1/en
Publication of WO2024218875A1 publication Critical patent/WO2024218875A1/en

Links

Definitions

  • the disclosed technology relates to an encoding device, an encoding method, and an encoding program.
  • a point cloud is a collection of points that have three-dimensional coordinate values. Registration is a technique for aligning multiple point clouds measured at the same location at different times.
  • point cloud data related to point clouds will be encoded for transmission when performing registration. Furthermore, in order to increase the coverage rate in Japan of point cloud databases that store multiple point cloud data, it will be necessary to measure and accumulate huge amounts of point cloud data. Furthermore, it is expected that the transmission bandwidth will become tight as the number of devices that measure point clouds increases. Furthermore, it is expected that the reference point clouds that are used for comparison in alignment will be stored in an encoded state. It is also expected that it will become necessary to realize point cloud processing in combination with encoding.
  • Non-Patent Documents 1 and 2 disclose representative point cloud registration methods.
  • Non-Patent Document 3 discloses a method for removing points unsuitable for registration from a point cloud.
  • the measured point cloud may contain points that are unsuitable for registration.
  • a point cloud may contain points that are not due to valid measurements, and these points that are not due to valid measurements may have a negative effect on registration.
  • points that are due to valid measurements there are points that are suitable for registration and points that are not suitable for registration.
  • the point cloud registration methods described in Non-Patent Document 1 and Non-Patent Document 2 do not include a process for removing points that are unsuitable for registration.
  • Non-Patent Document 3 requires advanced processing such as processing based on the distance from surrounding points and statistical filtering.
  • the disclosed technology has been made in consideration of the above points, and aims to provide an encoding device, encoding method, and encoding program that can efficiently encode point cloud data by removing data of points that satisfy predetermined conditions.
  • a first aspect of the present disclosure is an encoding device that includes an input unit that inputs point cloud data including three-dimensional coordinate values and attribute values of each point that constitutes the point cloud, a removal unit that removes from the point cloud data removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among each point that constitutes the point cloud, an encoding unit that encodes post-removal data obtained by removing the removal point data from the point cloud data, and an output unit that outputs the encoded post-removal data.
  • a second aspect of the present disclosure is an encoding method, in which an input unit inputs point cloud data including three-dimensional coordinate values and attribute values of each point constituting the point cloud, a removal unit removes removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among each point constituting the point cloud from the point cloud data, an encoding unit encodes post-removal data obtained by removing the removal point data from the point cloud data, and an output unit outputs the encoded post-removal data.
  • the third aspect of the present disclosure is an encoding program that causes a computer to function as each part of the encoding device of the first aspect.
  • the disclosed technology makes it possible to efficiently encode point cloud data by removing data for points that satisfy predetermined conditions.
  • FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of an encoding device according to an embodiment.
  • FIG. 2 is a block diagram showing an example of a functional configuration of an encoding device according to an embodiment.
  • 1 is a flowchart showing an example of an encoding process in the encoding device according to the embodiment.
  • the encoding device 10 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, storage 14, and a communication I/F (Interface) 17. Each component is connected to each other so that they can communicate with each other via a bus 19.
  • CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • storage 14 storage 14
  • communication I/F Interface
  • the CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads the programs from the ROM 12 or storage 14, and executes the programs using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various arithmetic processing according to the programs stored in the ROM 12 or storage 14.
  • an encoding program is stored in the ROM 12 or storage 14.
  • the encoding program may be a single program, or may be a group of programs consisting of multiple programs or modules.
  • ROM 12 stores various programs and data.
  • RAM 13 temporarily stores programs or data as a working area.
  • Storage 14 is composed of a HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including the operating system and various data.
  • the communication I/F 17 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).
  • the encoding device 10 includes an input unit 101, a division unit 102, a calculation unit 103, a removal unit 104, an encoding unit 105, and an output unit 106.
  • the input unit 101 reads out the point cloud data stored in the ROM 12 or the storage 14. The input unit 101 then passes the point cloud data to the division unit 102.
  • the point cloud data is data that includes three-dimensional coordinate values and attribute values of each point that constitutes the point cloud. In this embodiment, brightness, color (e.g., RGB values), or normal vectors are applied as attribute values.
  • the division unit 102 divides the point cloud data passed from the input unit 101 into block data including three-dimensional coordinate values and attribute values of each point constituting a block divided from the point cloud.
  • the division unit 102 divides the point cloud data into an octree structure.
  • the division unit 102 may divide the point cloud data into a structure other than an octree structure.
  • the division unit 102 then passes the block data to the calculation unit 103 and the removal unit 104.
  • the calculation unit 103 calculates a representative value for each block data passed from the division unit 102.
  • the arithmetic mean, trimmed mean, median, or mode of the attribute values of each point constituting the block is applied as the representative value.
  • the calculation unit 103 then passes the representative value for each block data to the removal unit 104.
  • the removal unit 104 removes removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among the points that constitute the point cloud from the point cloud data.
  • points that constitute a block whose representative value satisfies a predetermined condition are used as the removal points. That is, in this embodiment, the removal unit 104 removes removal point data of removal points that are points that constitute a block whose representative value passed from the calculation unit 103 satisfies a predetermined condition from the block data passed from the division unit 102.
  • this is not limited to this example.
  • the removal unit 104 removes removal point data of removal points that satisfy the predetermined condition from the point cloud data passed from the input unit 101.
  • the removal unit 104 passes the post-removal data, in which the removal point data has been removed from the block data, to the encoding unit 105.
  • a condition related to an attribute value is applied as the predetermined condition.
  • a condition that the luminance is 0 may be applied as the predetermined condition.
  • a condition that the absolute value of the z component of the normal vector is greater than a predetermined value may be applied as the predetermined condition. This is because by leaving only blocks whose absolute value of the z component of the normal vector is equal to or less than a predetermined value, it is possible to leave only points that are measured on a surface close to vertical, that is, points that are relatively likely to be walls.
  • the predetermined condition is a condition that the representative value is equal to a predetermined reference value, or a condition that is determined based on the magnitude relationship between the representative value and a predetermined reference value.
  • the encoding unit 105 encodes the post-removal data passed from the removal unit 104.
  • the specific encoding method may be any method based on an octree structure, such as G-PCC (Geometry based Point Cloud Compression).
  • G-PCC Geometry based Point Cloud Compression
  • the output unit 106 outputs the coded post-removal data passed from the coding unit 105.
  • the output unit 106 outputs the coded post-removal data to a specified area of the storage 14.
  • the output unit 106 may output the coded post-removal data to a device configured separately from the coding device 10.
  • FIG. 3 is a flowchart showing the flow of the encoding process by the encoding device 10.
  • the encoding process is performed by the CPU 11 reading the encoding program from the ROM 12 or storage 14, expanding it into the RAM 13, and executing it.
  • step S101 the CPU 11 inputs point cloud data stored in the ROM 12 or storage 14 as the input unit 101.
  • step S102 the CPU 11, functioning as the division unit 102, divides the point cloud data into block data. Specifically, the CPU 11 divides the point cloud data into an octree structure.
  • step S103 the CPU 11, functioning as the calculation unit 103, calculates a representative value for each block data.
  • the CPU 11 calculates, for each block data, the arithmetic mean, trimmed mean, median, or mode of the attribute values of each point constituting the block.
  • step S104 the CPU 11, functioning as the removal unit 104, removes removal point data of removal points that constitute blocks whose representative values satisfy a predetermined condition from the block data.
  • step S105 the CPU 11, functioning as the encoding unit 105, encodes the post-removal data. Specifically, the CPU 11 encodes the post-removal data using a method based on an octree structure such as G-PCC.
  • step S106 the CPU 11 outputs the encoded post-removal data as the output unit 106, and ends this encoding process.
  • the encoding device 10 includes an input unit 101 that inputs point cloud data, a removal unit 104 that removes removal point data from the point cloud data, an encoding unit 105 that encodes the post-removal data, and an output unit 106 that outputs the encoded post-removal data. This makes it possible to encode point cloud data efficiently.
  • the encoding device 10 further includes a division unit 102 that divides the point cloud data into block data, and a removal unit 104 removes removal point data of removal points, which are points that constitute blocks whose representative values satisfy a predetermined condition, from the block data.
  • a division unit 102 that divides the point cloud data into block data
  • a removal unit 104 removes removal point data of removal points, which are points that constitute blocks whose representative values satisfy a predetermined condition, from the block data.
  • the CPU 11 of the encoding device 10 calculates the representative value for each block data as the calculation unit 103.
  • this is not limited to this example.
  • a device other than the encoding device 10 may calculate the representative value for each block data.
  • various processes that the CPU reads and executes software (programs) in the above embodiment may be executed by various processors other than the CPU.
  • processors in this case include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacture, and dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations designed specifically to execute specific processes.
  • the encoding process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA, etc.).
  • the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.
  • the encoding program is pre-stored (installed) in the ROM 12 or storage 14, but this is not limiting.
  • the program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory.
  • the program may also be downloaded from an external device via a network.
  • An encoding device comprising: Memory, at least one processor coupled to the memory; Including, The processor, Input point cloud data including three-dimensional coordinate values and attribute values of each point constituting the point cloud; removing, from the point cloud data, removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among the points constituting the point cloud; encoding post-removal data obtained by removing the removal point data from the point cloud data; outputting the encoded post-removal data;
  • the encoding device configured as follows.
  • a non-transitory storage medium storing a program executable by a computer to perform an encoding process,
  • the encoding process includes: Input point cloud data including three-dimensional coordinate values and attribute values of each point constituting the point cloud; removing, from the point cloud data, removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among the points constituting the point cloud; encoding post-removal data obtained by removing the removal point data from the point cloud data; outputting the encoded post-removal data;
  • Non-transitory storage media Non-transitory storage media.

Landscapes

  • Image Processing (AREA)

Abstract

An encoding device 10 comprises: an input unit 101 for inputting point cloud data including three-dimensional coordinate values and an attribute value of each point forming a point cloud; a removal unit 104 that removes, from the point cloud data, removal-point data including the three-dimensional coordinate values and the attribute value of a removal point that satisfies a predetermined condition from among the points forming the point cloud; an encoding unit 105 that encodes post-removal data obtained by removing the removal-point data from the point cloud data; and an output unit 106 that outputs the encoded post-removal data.

Description

符号化装置、符号化方法、及び符号化プログラムEncoding device, encoding method, and encoding program
 開示の技術は、符号化装置、符号化方法、及び符号化プログラムに関する。 The disclosed technology relates to an encoding device, an encoding method, and an encoding program.
 異なる時刻に同一の場所で計測された点群を接続する場合、又は点群を計測するのに利用した機器の位置を推定する場合、点群同士のレジストレーション(registration)が必要である。ここで、点群とは、三次元座標値を持つ点の集合である。また、レジストレーションとは、異なる時刻に同一の場所で計測された複数の点群を位置合わせする技術である。 When connecting point clouds measured at the same location at different times, or when estimating the position of the equipment used to measure the point clouds, registration between the point clouds is required. Here, a point cloud is a collection of points that have three-dimensional coordinate values. Registration is a technique for aligning multiple point clouds measured at the same location at different times.
 将来、レジストレーションをするに際して、点群に関する点群データを伝送等のために符号化する事が考えられる。また、複数の点群データが記憶された点群データベースの日本における網羅率を上げるには膨大な点群データの計測及び蓄積が必要である。また、点群を計測する機器が増えることにより、伝送用の帯域が逼迫することが考えられる。また、位置合わせの比較対象となる参照点群は符号化された状態で保存されると考えられる。また、符号化と組み合わせて点群処理を実現することが必要になると考えられる。 In the future, it is expected that point cloud data related to point clouds will be encoded for transmission when performing registration. Furthermore, in order to increase the coverage rate in Japan of point cloud databases that store multiple point cloud data, it will be necessary to measure and accumulate huge amounts of point cloud data. Furthermore, it is expected that the transmission bandwidth will become tight as the number of devices that measure point clouds increases. Furthermore, it is expected that the reference point clouds that are used for comparison in alignment will be stored in an encoded state. It is also expected that it will become necessary to realize point cloud processing in combination with encoding.
 例えば、非特許文献1及び非特許文献2には代表的な点群レジストレーション手法が開示されている。また、非特許文献3には点群からレジストレーションに不適な点を除去する方法が開示されている。 For example, Non-Patent Documents 1 and 2 disclose representative point cloud registration methods. Non-Patent Document 3 discloses a method for removing points unsuitable for registration from a point cloud.
 計測された点群の中にはレジストレーションに不適な点が含まれる場合がある。例えば、点群には有効な計測に因らない点が含まれることがあり、当該有効な計測に因らない点はレジストレーションに悪影響を及ぼしうる。また、有効な計測に因る点においても、レジストレーションに適した点とレジストレーションに適さない点とがある。しかし、非特許文献1及び非特許文献2に記載の点群レジストレーション手法にはレジストレーションに不適な点を除去する処理は含まれていない。 The measured point cloud may contain points that are unsuitable for registration. For example, a point cloud may contain points that are not due to valid measurements, and these points that are not due to valid measurements may have a negative effect on registration. Furthermore, even among points that are due to valid measurements, there are points that are suitable for registration and points that are not suitable for registration. However, the point cloud registration methods described in Non-Patent Document 1 and Non-Patent Document 2 do not include a process for removing points that are unsuitable for registration.
 また、1つ1つの点について除去するか否かを判定する場合、周囲の点の情報を用いて判定しなくてはならない場合がある。また、1つの点のみで除去するか否かを判定する場合でも、誤検出及び未検出等が発生する場合がある。例えば、非特許文献3に開示された技術では、周囲の点との距離に基づく処理、及び統計的フィルタリング等の高度な処理を要する。 In addition, when determining whether or not to remove each point, it may be necessary to use information about surrounding points. Even when determining whether or not to remove a point based on only one point, false positives and non-detections may occur. For example, the technology disclosed in Non-Patent Document 3 requires advanced processing such as processing based on the distance from surrounding points and statistical filtering.
 開示の技術は、上記の点に鑑みてなされたものであり、事前に予め定められた条件を満たす点のデータを除去することで、効率的に点群データを符号化できる符号化装置、符号化方法、及び符号化プログラムを提供することを目的とする。 The disclosed technology has been made in consideration of the above points, and aims to provide an encoding device, encoding method, and encoding program that can efficiently encode point cloud data by removing data of points that satisfy predetermined conditions.
 本開示の第1態様は、符号化装置であって、点群を構成する各点の3次元の座標値及び属性値を含む点群データを入力する入力部と、前記点群を構成する各点のうち予め定められた条件を満たす除去点の3次元の座標値及び属性値を含む除去点データを、前記点群データから除去する除去部と、前記点群データから前記除去点データを除去した除去後データを符号化する符号化部と、符号化した前記除去後データを出力する出力部と、を備える。 A first aspect of the present disclosure is an encoding device that includes an input unit that inputs point cloud data including three-dimensional coordinate values and attribute values of each point that constitutes the point cloud, a removal unit that removes from the point cloud data removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among each point that constitutes the point cloud, an encoding unit that encodes post-removal data obtained by removing the removal point data from the point cloud data, and an output unit that outputs the encoded post-removal data.
 本開示の第2態様は、符号化方法であって、入力部が、点群を構成する各点の3次元の座標値及び属性値を含む点群データを入力し、除去部が、前記点群を構成する各点のうち予め定められた条件を満たす除去点の3次元の座標値及び属性値を含む除去点データを、前記点群データから除去し、符号化部が、前記点群データから前記除去点データを除去した除去後データを符号化し、出力部が、符号化した前記除去後データを出力する。 A second aspect of the present disclosure is an encoding method, in which an input unit inputs point cloud data including three-dimensional coordinate values and attribute values of each point constituting the point cloud, a removal unit removes removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among each point constituting the point cloud from the point cloud data, an encoding unit encodes post-removal data obtained by removing the removal point data from the point cloud data, and an output unit outputs the encoded post-removal data.
 本開示の第3態様は、符号化プログラムであって、コンピュータを、上記第1態様の符号化装置の各部として機能させるためのプログラムである。 The third aspect of the present disclosure is an encoding program that causes a computer to function as each part of the encoding device of the first aspect.
 開示の技術によれば、事前に予め定められた条件を満たす点のデータを除去することで、効率的に点群データを符号化できる。 The disclosed technology makes it possible to efficiently encode point cloud data by removing data for points that satisfy predetermined conditions.
実施形態に係る符号化装置のハードウェア構成の一例を示す模式図である。FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of an encoding device according to an embodiment. 実施形態に係る符号化装置の機能構成の一例を示すブロック図である。FIG. 2 is a block diagram showing an example of a functional configuration of an encoding device according to an embodiment. 実施形態に係る符号化装置における符号化処理の一例を示すフローチャートである。1 is a flowchart showing an example of an encoding process in the encoding device according to the embodiment.
 以下、開示の技術の実施形態の一例を、図面を参照しつつ説明する。なお、各図面において同一又は等価な構成要素及び部分には同一の参照符号を付与している。また、図面の寸法比率は、説明の都合上誇張されており、実際の比率とは異なる場合がある。 Below, an example of an embodiment of the disclosed technology will be described with reference to the drawings. Note that the same reference symbols are used for identical or equivalent components and parts in each drawing. Also, the dimensional ratios in the drawings have been exaggerated for the convenience of explanation and may differ from the actual ratios.
 図1に示すように、符号化装置10は、CPU(Central Processing Unit)11、ROM(Read Only Memory)12、RAM(Random Access Memory)13、ストレージ14、及び通信I/F(Interface)17を有する。各構成は、バス19を介して相互に通信可能に接続されている。 As shown in FIG. 1, the encoding device 10 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, storage 14, and a communication I/F (Interface) 17. Each component is connected to each other so that they can communicate with each other via a bus 19.
 CPU11は、中央演算処理ユニットであり、各種プログラムを実行したり、各部を制御したりする。すなわち、CPU11は、ROM12又はストレージ14からプログラムを読み出し、RAM13を作業領域としてプログラムを実行する。CPU11は、ROM12又はストレージ14に記憶されているプログラムに従って、上記各構成の制御及び各種の演算処理を行う。本実施形態では、ROM12又はストレージ14には、符号化プログラムが格納されている。符号化プログラムは、1つのプログラムであってもよいし、複数のプログラム又はモジュールで構成されるプログラム群であってもよい。 The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads the programs from the ROM 12 or storage 14, and executes the programs using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various arithmetic processing according to the programs stored in the ROM 12 or storage 14. In this embodiment, an encoding program is stored in the ROM 12 or storage 14. The encoding program may be a single program, or may be a group of programs consisting of multiple programs or modules.
 ROM12は、各種プログラム及び各種データを格納する。RAM13は、作業領域として一時的にプログラム又はデータを記憶する。ストレージ14は、HDD(Hard Disk Drive)又はSSD(Solid State Drive)により構成され、オペレーティングシステムを含む各種プログラム、及び各種データを格納する。 ROM 12 stores various programs and data. RAM 13 temporarily stores programs or data as a working area. Storage 14 is composed of a HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including the operating system and various data.
 通信I/F17は、他の機器と通信するためのインタフェースであり、例えば、イーサネット(登録商標)、FDDI、Wi-Fi(登録商標)等の規格が用いられる。 The communication I/F 17 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).
 次に、符号化装置10の機能構成について説明する。図2に示すように、符号化装置10は、入力部101、分割部102、算出部103、除去部104、符号化部105及び出力部106を備えている。 Next, the functional configuration of the encoding device 10 will be described. As shown in FIG. 2, the encoding device 10 includes an input unit 101, a division unit 102, a calculation unit 103, a removal unit 104, an encoding unit 105, and an output unit 106.
 入力部101は、ROM12又はストレージ14に記憶された点群データを読み出す。そして、入力部101は、分割部102に対して点群データを受け渡す。点群データとは、点群を構成する各点の3次元の座標値及び属性値を含むデータである。なお、本実施形態では、属性値として輝度、色(例えば、RGB値)、又は法線ベクトルを適用している。 The input unit 101 reads out the point cloud data stored in the ROM 12 or the storage 14. The input unit 101 then passes the point cloud data to the division unit 102. The point cloud data is data that includes three-dimensional coordinate values and attribute values of each point that constitutes the point cloud. In this embodiment, brightness, color (e.g., RGB values), or normal vectors are applied as attribute values.
 分割部102は、入力部101から受け渡された点群データを、点群から分割したブロックを構成する各点の3次元の座標値及び属性値を含むブロックデータに分割する。本実施形態では、分割部102は、点群データを八分木構造に分割する。しかし、この例に限られない。分割部102は、点群データを八分木構造以外の構造に分割してもよい。そして、分割部102は、ブロックデータを算出部103及び除去部104に受け渡す。 The division unit 102 divides the point cloud data passed from the input unit 101 into block data including three-dimensional coordinate values and attribute values of each point constituting a block divided from the point cloud. In this embodiment, the division unit 102 divides the point cloud data into an octree structure. However, this is not limited to this example. The division unit 102 may divide the point cloud data into a structure other than an octree structure. The division unit 102 then passes the block data to the calculation unit 103 and the removal unit 104.
 算出部103は、分割部102から受け渡されたブロックデータ毎に代表値を算出する。本実施形態では、代表値として、ブロックを構成する各点の属性値の算術平均値、トリム平均値、中央値、又は最頻値を適用している。そして、算出部103は、除去部104に対して、ブロックデータ毎の代表値を受け渡す。 The calculation unit 103 calculates a representative value for each block data passed from the division unit 102. In this embodiment, the arithmetic mean, trimmed mean, median, or mode of the attribute values of each point constituting the block is applied as the representative value. The calculation unit 103 then passes the representative value for each block data to the removal unit 104.
 除去部104は、点群を構成する各点のうち予め定められた条件を満たす除去点の3次元の座標値及び属性値を含む除去点データを、点群データから除去する。本実施形態では、除去点として、代表値が予め定められた条件を満たすブロックを構成する点を適用している。すなわち、本実施形態では、除去部104は、算出部103から受け渡された代表値が予め定められた条件を満たすブロックを構成する点である除去点の除去点データを、分割部102から受け渡されたブロックデータから除去する。しかし、この例に限られない。例えば、分割部102が点群データをブロックデータに分割せず、算出部103がブロックデータ毎に代表値を算出しない場合、除去部104は予め定められた条件を満たす除去点の除去点データを、入力部101から受け渡された点群データから除去する。 The removal unit 104 removes removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among the points that constitute the point cloud from the point cloud data. In this embodiment, points that constitute a block whose representative value satisfies a predetermined condition are used as the removal points. That is, in this embodiment, the removal unit 104 removes removal point data of removal points that are points that constitute a block whose representative value passed from the calculation unit 103 satisfies a predetermined condition from the block data passed from the division unit 102. However, this is not limited to this example. For example, if the division unit 102 does not divide the point cloud data into block data and the calculation unit 103 does not calculate a representative value for each block data, the removal unit 104 removes removal point data of removal points that satisfy the predetermined condition from the point cloud data passed from the input unit 101.
 そして、除去部104は、除去点データをブロックデータから除去した除去後データを符号化部105に受け渡す。 Then, the removal unit 104 passes the post-removal data, in which the removal point data has been removed from the block data, to the encoding unit 105.
 なお、本実施形態では、予め定められた条件として、属性値に関する条件を適用している。例えば、属性値として輝度を適用している場合であって、点群を計測する計測装置が正常に計測できなかった点を輝度0として記録している場合は、予め定められた条件として輝度が0であるとの条件を適用してもよい。また、例えば、属性値として法線ベクトルを適用している場合、予め定められた条件として、法線ベクトルのz成分の絶対値が予め定められた値より大きいとの条件を適用してもよい。法線ベクトルのz成分の絶対値が予め定められた値以下のブロックのみを残すことで、鉛直に近い面で計測された点、すなわち壁の可能性が相対的に高い点のみを残すことができるからである。しかし、これらの例に限定されない。まとめると、予め定められた条件は、代表値と所定の基準値が等しい、又は代表値と所定の基準値との大小関係によって判定される条件である。 In this embodiment, a condition related to an attribute value is applied as the predetermined condition. For example, when luminance is applied as the attribute value, and a point that the measuring device that measures the point cloud could not measure normally is recorded as luminance 0, a condition that the luminance is 0 may be applied as the predetermined condition. Also, for example, when a normal vector is applied as the attribute value, a condition that the absolute value of the z component of the normal vector is greater than a predetermined value may be applied as the predetermined condition. This is because by leaving only blocks whose absolute value of the z component of the normal vector is equal to or less than a predetermined value, it is possible to leave only points that are measured on a surface close to vertical, that is, points that are relatively likely to be walls. However, the present invention is not limited to these examples. In summary, the predetermined condition is a condition that the representative value is equal to a predetermined reference value, or a condition that is determined based on the magnitude relationship between the representative value and a predetermined reference value.
 符号化部105は、除去部104から受け渡された除去後データを符号化する。具体的な符号化方法は、例えばG-PCC(Geometry based Point Cloud Compression)等の八分木構造に基づく方法であれば如何なる方法であってもよい。そして、符号化部105は、符号化した除去後データを出力部106に受け渡す。 The encoding unit 105 encodes the post-removal data passed from the removal unit 104. The specific encoding method may be any method based on an octree structure, such as G-PCC (Geometry based Point Cloud Compression). The encoding unit 105 then passes the encoded post-removal data to the output unit 106.
 出力部106は、符号化部105から受け渡された符号化した除去後データを出力する。本実施形態では、出力部106は、符号化した除去後データをストレージ14の所定の領域に出力する。しかし、この例に限られない。例えば、出力部106は、符号化装置10とは別体で構成された装置等に対して、符号化した除去後データを出力してもよい。 The output unit 106 outputs the coded post-removal data passed from the coding unit 105. In this embodiment, the output unit 106 outputs the coded post-removal data to a specified area of the storage 14. However, this is not a limitation. For example, the output unit 106 may output the coded post-removal data to a device configured separately from the coding device 10.
 次に、符号化装置10の作用について説明する。 Next, the operation of the encoding device 10 will be explained.
 図3は、符号化装置10による符号化処理の流れを示すフローチャートである。CPU11がROM12又はストレージ14から符号化プログラムを読み出して、RAM13に展開して実行することにより、符号化処理が行なわれる。 FIG. 3 is a flowchart showing the flow of the encoding process by the encoding device 10. The encoding process is performed by the CPU 11 reading the encoding program from the ROM 12 or storage 14, expanding it into the RAM 13, and executing it.
 ステップS101で、CPU11は、入力部101として、ROM12又はストレージ14に記憶された点群データを入力する。 In step S101, the CPU 11 inputs point cloud data stored in the ROM 12 or storage 14 as the input unit 101.
 ステップS102で、CPU11は、分割部102として、点群データをブロックデータに分割する。具体的に、CPU11は、点群データを八分木構造に分割する。 In step S102, the CPU 11, functioning as the division unit 102, divides the point cloud data into block data. Specifically, the CPU 11 divides the point cloud data into an octree structure.
 ステップS103で、CPU11は、算出部103として、ブロックデータ毎に代表値を算出する。本実施形態では、CPU11は、ブロックデータ毎に、ブロックを構成する各点の属性値の算術平均値、トリム平均値、中央値、又は最頻値を算出する。 In step S103, the CPU 11, functioning as the calculation unit 103, calculates a representative value for each block data. In this embodiment, the CPU 11 calculates, for each block data, the arithmetic mean, trimmed mean, median, or mode of the attribute values of each point constituting the block.
 ステップS104で、CPU11は、除去部104として、代表値が予め定められた条件を満たすブロックを構成する除去点の除去点データを、ブロックデータから除去する。 In step S104, the CPU 11, functioning as the removal unit 104, removes removal point data of removal points that constitute blocks whose representative values satisfy a predetermined condition from the block data.
 ステップS105で、CPU11は、符号化部105として、除去後データを符号化する。具体的に、CPU11は、除去後データをG-PCC等の八分木構造に基づく方法で符号化する。 In step S105, the CPU 11, functioning as the encoding unit 105, encodes the post-removal data. Specifically, the CPU 11 encodes the post-removal data using a method based on an octree structure such as G-PCC.
 ステップS106で、CPU11は、出力部106として、符号化した除去後データを出力し、本符号化処理を終了する。 In step S106, the CPU 11 outputs the encoded post-removal data as the output unit 106, and ends this encoding process.
 以上説明したように、本実施形態に係る符号化装置10は、点群データを入力する入力部101と、除去点データを点群データから除去する除去部104と、除去後データを符号化する符号化部105と、符号化した除去後データを出力する出力部106と、を備える。これにより、効率的に点群データを符号化できる。 As described above, the encoding device 10 according to this embodiment includes an input unit 101 that inputs point cloud data, a removal unit 104 that removes removal point data from the point cloud data, an encoding unit 105 that encodes the post-removal data, and an output unit 106 that outputs the encoded post-removal data. This makes it possible to encode point cloud data efficiently.
 また、本実施形態に係る符号化装置10は、点群データをブロックデータに分割する分割部102を更に備え、除去部104は、代表値が予め定められた条件を満たすブロックを構成する点である除去点の除去点データをブロックデータから除去する。これにより、固まって分布している点のみが除去対象になるため、有効な計測に因らない点と同様の特徴を持つ他の点を除去することを防ぐことができる。また、点単位または局所的な計測値のぶれを吸収又は平均化することができるため、過検出及び未検出等の発生を抑制することができる。 The encoding device 10 according to this embodiment further includes a division unit 102 that divides the point cloud data into block data, and a removal unit 104 removes removal point data of removal points, which are points that constitute blocks whose representative values satisfy a predetermined condition, from the block data. As a result, only points that are distributed in clusters are subject to removal, making it possible to prevent the removal of other points that have similar characteristics to points that are not due to valid measurements. Also, since it is possible to absorb or average out point-by-point or localized fluctuations in measurement values, it is possible to suppress the occurrence of overdetection and non-detection, etc.
<変形例>
 なお、本発明は、上述した実施形態に限定されるものではなく、この発明の要旨を逸脱しない範囲内で様々な変形や応用が可能である。
<Modification>
The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the spirit and scope of the present invention.
 例えば、上記実施形態では、符号化装置10が有するCPU11が、算出部103として、ブロックデータ毎に代表値を算出していた。しかし、この例に限られない。例えば、符号化装置10以外の装置がブロックデータ毎に代表値を算出してもよい。 For example, in the above embodiment, the CPU 11 of the encoding device 10 calculates the representative value for each block data as the calculation unit 103. However, this is not limited to this example. For example, a device other than the encoding device 10 may calculate the representative value for each block data.
 また、上記実施形態でCPUがソフトウェア(プログラム)を読み込んで実行した各種処理を、CPU以外の各種のプロセッサが実行してもよい。この場合のプロセッサとしては、FPGA(Field-Programmable Gate Array)等の製造後に回路構成を変更可能なPLD(Programmable Logic Device)、及びASIC(Application Specific Integrated Circuit)等の特定の処理を実行させるために専用に設計された回路構成を有するプロセッサである専用電気回路等が例示される。また、符号化処理を、これらの各種のプロセッサのうちの1つで実行してもよいし、同種又は異種の2つ以上のプロセッサの組み合わせ(例えば、複数のFPGA、及びCPUとFPGAとの組み合わせ等)で実行してもよい。また、これらの各種のプロセッサのハードウェア的な構造は、より具体的には、半導体素子等の回路素子を組み合わせた電気回路である。 Furthermore, various processes that the CPU reads and executes software (programs) in the above embodiment may be executed by various processors other than the CPU. Examples of processors in this case include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacture, and dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations designed specifically to execute specific processes. Furthermore, the encoding process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA, etc.). Moreover, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.
 また、上記実施形態では、符号化プログラムがROM12又はストレージ14に予め記憶(インストール)されている態様を説明したが、これに限定されない。プログラムは、CD-ROM(Compact Disk Read Only Memory)、DVD-ROM(Digital Versatile Disk Read Only Memory)、及びUSB(Universal Serial Bus)メモリ等の非一時的(non-transitory)記憶媒体に記憶された形態で提供されてもよい。また、プログラムは、ネットワークを介して外部装置からダウンロードされる形態としてもよい。 In the above embodiment, the encoding program is pre-stored (installed) in the ROM 12 or storage 14, but this is not limiting. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.
 以上の実施形態に関し、更に以下の付記を開示する。 The following notes are further provided with respect to the above embodiment.
 (付記項1)
 符号化装置であって、
 メモリと、
 前記メモリに接続された少なくとも1つのプロセッサと、
 を含み、
 前記プロセッサは、
 点群を構成する各点の3次元の座標値及び属性値を含む点群データを入力し、
 前記点群を構成する各点のうち予め定められた条件を満たす除去点の3次元の座標値及び属性値を含む除去点データを、前記点群データから除去し、
 前記点群データから前記除去点データを除去した除去後データを符号化し、
 符号化した前記除去後データを出力する、
 ように構成される符号化装置。
(Additional Note 1)
1. An encoding device, comprising:
Memory,
at least one processor coupled to the memory;
Including,
The processor,
Input point cloud data including three-dimensional coordinate values and attribute values of each point constituting the point cloud;
removing, from the point cloud data, removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among the points constituting the point cloud;
encoding post-removal data obtained by removing the removal point data from the point cloud data;
outputting the encoded post-removal data;
The encoding device configured as follows.
 (付記項2)
 符号化処理を実行するようにコンピュータによって実行可能なプログラムを記憶した非一時的記憶媒体であって、
 前記符号化処理は、
 点群を構成する各点の3次元の座標値及び属性値を含む点群データを入力し、
 前記点群を構成する各点のうち予め定められた条件を満たす除去点の3次元の座標値及び属性値を含む除去点データを、前記点群データから除去し、
 前記点群データから前記除去点データを除去した除去後データを符号化し、
 符号化した前記除去後データを出力する、
 非一時的記憶媒体。
(Additional Note 2)
A non-transitory storage medium storing a program executable by a computer to perform an encoding process,
The encoding process includes:
Input point cloud data including three-dimensional coordinate values and attribute values of each point constituting the point cloud;
removing, from the point cloud data, removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among the points constituting the point cloud;
encoding post-removal data obtained by removing the removal point data from the point cloud data;
outputting the encoded post-removal data;
Non-transitory storage media.
10 符号化装置
11 CPU
12 ROM
13 RAM
14 ストレージ
17 通信I/F
19 バス
101 入力部
102 分割部
103 算出部
104 除去部
105 符号化部
106 出力部
10 Encoding device 11 CPU
12 ROM
13 RAM
14 Storage 17 Communication I/F
19 Bus 101 Input section 102 Division section 103 Calculation section 104 Removal section 105 Encoding section 106 Output section

Claims (8)

  1.  点群を構成する各点の3次元の座標値及び属性値を含む点群データを入力する入力部と、
     前記点群を構成する各点のうち予め定められた条件を満たす除去点の3次元の座標値及び属性値を含む除去点データを、前記点群データから除去する除去部と、
     前記点群データから前記除去点データを除去した除去後データを符号化する符号化部と、
     符号化した前記除去後データを出力する出力部と、
     を備えた符号化装置。
    an input unit for inputting point cloud data including three-dimensional coordinate values and attribute values of each point constituting the point cloud;
    a removal unit that removes, from the point cloud data, removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among the points constituting the point cloud;
    an encoding unit that encodes post-removal data obtained by removing the removal point data from the point cloud data;
    an output unit that outputs the encoded post-removal data;
    An encoding device comprising:
  2.  前記点群データを、前記点群から分割したブロックを構成する各点の3次元の座標値及び属性値を含むブロックデータに分割する分割部を更に備え、
     前記除去部は、代表値が前記予め定められた条件を満たす前記ブロックを構成する点である前記除去点の前記除去点データを、前記ブロックデータから除去する、
     請求項1に記載の符号化装置。
    A dividing unit divides the point cloud data into block data including three-dimensional coordinate values and attribute values of each point constituting a block divided from the point cloud,
    the removal unit removes, from the block data, removal point data of the removal point, which is a point constituting the block and whose representative value satisfies the predetermined condition.
    The encoding device according to claim 1 .
  3.  前記分割部は、前記点群データを八分木構造に分割する、請求項2に記載の符号化装置。 The encoding device according to claim 2, wherein the division unit divides the point cloud data into an octree structure.
  4.  前記代表値は前記ブロックを構成する各点の前記属性値の算術平均値、トリム平均値、中央値、又は最頻値である、請求項2に記載の符号化装置。 The encoding device according to claim 2, wherein the representative value is an arithmetic mean value, trimmed mean value, median value, or mode value of the attribute values of each point constituting the block.
  5.  前記属性値は輝度、色、又は法線ベクトルである、請求項1に記載の符号化装置。 The encoding device according to claim 1, wherein the attribute value is a luminance, a color, or a normal vector.
  6.  前記予め定められた条件は、前記属性値に関する条件である、請求項1に記載の符号化装置。 The encoding device according to claim 1, wherein the predetermined condition is a condition related to the attribute value.
  7.  入力部が、点群を構成する各点の3次元の座標値及び属性値を含む点群データを入力し、
     除去部が、前記点群を構成する各点のうち予め定められた条件を満たす除去点の3次元の座標値及び属性値を含む除去点データを、前記点群データから除去し、
     符号化部が、前記点群データから前記除去点データを除去した除去後データを符号化し、
     出力部が、符号化した前記除去後データを出力する、
     符号化方法。
    The input unit inputs point cloud data including three-dimensional coordinate values and attribute values of each point constituting the point cloud;
    a removal unit removes removal point data including three-dimensional coordinate values and attribute values of removal points that satisfy a predetermined condition among the points constituting the point cloud from the point cloud data;
    an encoding unit encoding post-removal data obtained by removing the removal point data from the point cloud data;
    an output unit outputs the encoded post-removal data;
    Encoding method.
  8.  コンピュータを、請求項1~請求項6の何れか1項に記載の符号化装置の各部として機能させるための符号化プログラム。 An encoding program for causing a computer to function as each part of the encoding device described in any one of claims 1 to 6.
PCT/JP2023/015510 2023-04-18 2023-04-18 Encoding device, encoding method, and encoding program WO2024218875A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/015510 WO2024218875A1 (en) 2023-04-18 2023-04-18 Encoding device, encoding method, and encoding program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/015510 WO2024218875A1 (en) 2023-04-18 2023-04-18 Encoding device, encoding method, and encoding program

Publications (1)

Publication Number Publication Date
WO2024218875A1 true WO2024218875A1 (en) 2024-10-24

Family

ID=93152163

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/015510 WO2024218875A1 (en) 2023-04-18 2023-04-18 Encoding device, encoding method, and encoding program

Country Status (1)

Country Link
WO (1) WO2024218875A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200043199A1 (en) * 2018-07-13 2020-02-06 Electronics And Telecommunications Research Institute 3d point cloud data encoding/decoding method and apparatus
US20200302652A1 (en) * 2017-07-13 2020-09-24 Interdigital Vc Holdings, Inc. A method and apparatus for encoding/decoding a colored point cloud representing the geometry and colors of a 3d object
WO2020241723A1 (en) * 2019-05-28 2020-12-03 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
JP2022516851A (en) * 2019-03-22 2022-03-03 テンセント・アメリカ・エルエルシー Methods for interframe point cloud attribute coding and their device and computer programs
JP2022538857A (en) * 2019-07-01 2022-09-06 エルジー エレクトロニクス インコーポレイティド Point cloud data processing apparatus and method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200302652A1 (en) * 2017-07-13 2020-09-24 Interdigital Vc Holdings, Inc. A method and apparatus for encoding/decoding a colored point cloud representing the geometry and colors of a 3d object
US20200043199A1 (en) * 2018-07-13 2020-02-06 Electronics And Telecommunications Research Institute 3d point cloud data encoding/decoding method and apparatus
JP2022516851A (en) * 2019-03-22 2022-03-03 テンセント・アメリカ・エルエルシー Methods for interframe point cloud attribute coding and their device and computer programs
WO2020241723A1 (en) * 2019-05-28 2020-12-03 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
JP2022538857A (en) * 2019-07-01 2022-09-06 エルジー エレクトロニクス インコーポレイティド Point cloud data processing apparatus and method

Similar Documents

Publication Publication Date Title
JP6501240B2 (en) How to compress a point cloud
WO2020248187A1 (en) Point cloud coding method, point cloud decoding method and related device
KR102318023B1 (en) 3-Dimensional Model Generation Using Edges
JP5081276B2 (en) Pattern measuring device, pattern measuring method, and program
WO2021003726A1 (en) Data encoding method, data decoding method, devices and storage medium
JP2005037378A (en) Depth measurement method and depth measurement device
WO2021017471A1 (en) Point cloud filtering method based on image processing, apparatus, and storage medium
EP3692508B1 (en) Method and device for up-sampling a point cloud
US9632038B2 (en) Hybrid phase unwrapping systems and methods for patterned wafer measurement
WO2021062772A1 (en) Prediction method, encoder, decoder, and computer storage medium
KR100512760B1 (en) Method for generating 3d mesh from 3d points by using shrink-wrapping scheme of boundary cells
WO2024218875A1 (en) Encoding device, encoding method, and encoding program
KR101927861B1 (en) Method and apparatus for removing noise based on mathematical morphology from geometric data of 3d space
US9563969B2 (en) Method of generating implicit terrain data by performing lossless compression and lossy compression and electronic apparatus for performing the method
US20230237707A1 (en) Point cloud encoding and decoding method, encoder, decoder and codec system
CN109344750B (en) Complex structure three-dimensional object identification method based on structure descriptor
KR102151127B1 (en) Apparatus for generating 3-dimensional image and method for the same
CN115482268A (en) High-precision three-dimensional shape measurement method and system based on speckle matching network
CN112687010B (en) Digital metering method of end frame drilling jig
Qi et al. Robust slicing procedure based on surfel-grid
US20230237704A1 (en) Point cloud decoding and encoding method, and decoder, encoder and encoding and decoding system
KR102171203B1 (en) A method of matching a stereo image and an apparatus therefor
CN115049750B (en) Voxel map generation method, device, storage medium and equipment based on octree
CN117635875A (en) Three-dimensional reconstruction method, device and terminal
KR102060758B1 (en) Primitive fitting apparatus and method using point cloud