[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

JP5755935B2 - Component pitch measuring apparatus and component pitch measuring method - Google Patents

Component pitch measuring apparatus and component pitch measuring method Download PDF

Info

Publication number
JP5755935B2
JP5755935B2 JP2011107817A JP2011107817A JP5755935B2 JP 5755935 B2 JP5755935 B2 JP 5755935B2 JP 2011107817 A JP2011107817 A JP 2011107817A JP 2011107817 A JP2011107817 A JP 2011107817A JP 5755935 B2 JP5755935 B2 JP 5755935B2
Authority
JP
Japan
Prior art keywords
component
camera
pitch
component pitch
correlation value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2011107817A
Other languages
Japanese (ja)
Other versions
JP2012238777A (en
Inventor
弘健 江嵜
弘健 江嵜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fuji Corp
Original Assignee
Fuji Machine Manufacturing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fuji Machine Manufacturing Co Ltd filed Critical Fuji Machine Manufacturing Co Ltd
Priority to JP2011107817A priority Critical patent/JP5755935B2/en
Priority to CN201210146702.1A priority patent/CN102778206B/en
Publication of JP2012238777A publication Critical patent/JP2012238777A/en
Application granted granted Critical
Publication of JP5755935B2 publication Critical patent/JP5755935B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Length Measuring Devices By Optical Means (AREA)
  • Supply And Installment Of Electrical Components (AREA)

Description

本発明は、同一形状の部品が配列されたものにおける部品ピッチを計測する部品ピッチ計測装置及び部品ピッチ計測方法に関する発明である。   The present invention relates to a component pitch measuring apparatus and a component pitch measuring method for measuring a component pitch in a component in which components having the same shape are arranged.

例えば、部品実装機に部品を供給するフィーダとして、テープフィーダ、トレイフィーダ、バルクフィーダ等が知られている。ここで、トレイフィーダは、トレイ上に部品をマトリックス状(碁盤目状)に配列し、部品実装機の吸着ノズルでトレイ上の部品を吸着して回路基板に実装する。この場合、トレイ上に配列された部品を吸着ノズルで順番に吸着するためには、最初に吸着する部品の位置座標、X方向の部品ピッチ、Y方向の部品ピッチ、X方向の部品数、Y方向の部品数等のデータを含むトレイデータが必要となる。   For example, tape feeders, tray feeders, bulk feeders, and the like are known as feeders that supply components to a component mounting machine. Here, the tray feeder arranges the components on the tray in a matrix (a grid pattern), and sucks the components on the tray with the suction nozzle of the component mounting machine and mounts them on the circuit board. In this case, in order to sequentially pick up the components arranged on the tray with the suction nozzle, the position coordinates of the first picked-up component, the component pitch in the X direction, the component pitch in the Y direction, the number of components in the X direction, Y Tray data including data such as the number of parts in the direction is required.

従来、部品ピッチ等のデータは、部品供給元等から入手した設計データ等を参照して作業者が手作業で入力するようにしているが、トレイの寸法や部品ピッチ等は厳密に寸法管理されているわけではない。このため、作業者が入力した部品ピッチが実際の部品ピッチからずれている可能性があり、その結果、トレイ上の各列の部品の個数に応じて部品位置の誤差が累積的に増大して部品の吸着ミスが発生する可能性がある。   Conventionally, data such as component pitch is manually input by an operator with reference to design data obtained from a component supplier, etc., but the dimensions of the tray, component pitch, etc. are strictly controlled. I don't mean. For this reason, there is a possibility that the part pitch input by the operator is deviated from the actual part pitch. As a result, the error in the part position increases cumulatively according to the number of parts in each row on the tray. There is a possibility that component adsorption mistakes may occur.

この課題に対して、特許文献1(特開2001−345595号公報)のトレイデータ作成方法では、カメラをトレイ上の最初に吸着する部品(吸着開始点)の上方に移動させて、当該吸着開始点の部品を撮像して表示装置に表示し、その表示画面上で作業者が当該吸着開始点の部品の位置を手動で入力し、その後、カメラを当該部品の列の最後に吸着する部品(最終吸着点)の上方に移動させて、当該最終吸着点の部品を撮像して表示装置に表示し、その表示画面上で作業者が当該最終吸着点の部品の位置を手動で入力して、吸着開始点から最終吸着点までのカメラの移動量をその列の部品数で割り算することで部品ピッチを求めるようにしている。   In response to this problem, in the tray data creation method disclosed in Patent Document 1 (Japanese Patent Application Laid-Open No. 2001-345595), the camera is moved above the part (suction start point) to be first suctioned on the tray to start the suction. The component of the point is picked up and displayed on the display device, and the operator manually inputs the position of the component at the suction start point on the display screen, and then the component that sucks the camera at the end of the column of the component ( The final suction point) is moved above, the part at the final suction point is imaged and displayed on the display device, and the operator manually inputs the position of the part at the final suction point on the display screen, The part pitch is obtained by dividing the moving amount of the camera from the suction start point to the final suction point by the number of parts in the row.

また、特許文献2(特開2008−10594号公報)のトレイデータ補正方法では、吸着ノズルで吸着した部品の吸着位置ずれ量を画像認識し、この吸着位置ずれ量によって部品ピッチやトレイの傾きに関する補正値を求めて、予め設定されたトレイデータを補正するようにしている。   Further, in the tray data correction method disclosed in Patent Document 2 (Japanese Patent Laid-Open No. 2008-10594), the amount of suction position deviation of a component sucked by a suction nozzle is recognized as an image, and the component pitch and the inclination of the tray are related to the suction position deviation amount. A correction value is obtained, and preset tray data is corrected.

特開2001−345595号公報JP 2001-345595 A 特開2008−10594号公報JP 2008-10594 A

しかし、特許文献1のトレイデータ作成方法では、作業者が吸着開始点と最終吸着点を手動で入力したり、部品数を手動で入力しなければならないため、手間がかかるばかりか、作業者が勘違い等により間違った部品数等を入力して間違った部品ピッチを算出してしまう可能性もある。   However, in the tray data creation method of Patent Document 1, the operator must manually input the suction start point and the final suction point or manually input the number of parts. There is a possibility that the wrong part pitch may be calculated by inputting the wrong number of parts due to misunderstanding or the like.

また、特許文献2の技術は、予め作成した部品ピッチ等のトレイデータを吸着位置ずれ量に応じて補正する技術であるため、予め作業者が部品ピッチ等のトレイデータを作成しておく必要があり、手間がかかる。   Further, since the technique of Patent Document 2 is a technique for correcting tray data such as a component pitch created in advance according to the amount of suction position deviation, it is necessary for an operator to create tray data such as a component pitch in advance. Yes, it takes time.

そこで、本発明が解決しようとする課題は、部品ピッチの計測を自動化できる部品ピッチ計測装置及び部品ピッチ計測方法を提供することである。   Therefore, the problem to be solved by the present invention is to provide a component pitch measuring device and a component pitch measuring method capable of automating the measurement of the component pitch.

上記課題を解決するために、請求項1に係る発明は、同一形状の部品が配列されたものにおける部品ピッチを計測する部品ピッチ計測装置において、前記部品を撮像するカメラと、前記カメラを相対移動させる移動機構と、前記カメラを前記移動機構により前記部品の配列方向に相対移動させながら連続的に撮像し、それらの撮像画像の変動の周期性を評価してその変動の周期性と前記カメラの相対移動量との関係に基づいて部品ピッチを計測する部品ピッチ計測手段とを備え、前記部品ピッチ計測手段は、前記カメラの撮像毎にその撮像画像と前記部品の基準画像との相関値を演算し、その相関値のピーク点間の前記カメラの相対移動量に基づいて部品ピッチを計測することを特徴とするものである。ここで、「連続的に撮像」とは、計測対象となる部品ピッチと比べて十分に短い間隔で撮像を繰り返すことである。 In order to solve the above-described problem, the invention according to claim 1 is a component pitch measurement device that measures a component pitch in a component in which components having the same shape are arranged, and relatively moves the camera that images the component and the camera. A moving mechanism that moves the camera and continuously images the camera while relatively moving the camera in the arrangement direction of the parts, and evaluating the periodicity of fluctuations in the captured images and the periodicity of the fluctuations of the camera. A component pitch measuring unit that measures a component pitch based on the relationship with the relative movement amount, and the component pitch measuring unit calculates a correlation value between the captured image and the reference image of the component for each imaging of the camera. The component pitch is measured based on the relative movement amount of the camera between the peak points of the correlation value. Here, “continuous imaging” is to repeat imaging at intervals sufficiently shorter than the component pitch to be measured.

本発明のように、カメラを部品の配列方向に相対移動させながら連続的に撮像すると、カメラの視野内に1つの部品しか収まらない場合でも、配列された複数の部品の撮像画像を含む多くの撮像画像を取得することができる。これらの撮像画像は、カメラの相対移動に伴って周期的に変動し、先の部品を撮像してから次の部品を撮像するまでの撮像回数(カメラの相対移動量)を1周期として変動し、その1周期分のカメラの相対移動量が部品ピッチに相当する。この関係から、撮像画像の変動の周期性を評価すれば、その1周期分のカメラの相対移動量から部品ピッチを計測することができる。これにより、部品ピッチの計測を自動化することが可能となる。 As in the present invention, when images are continuously captured while relatively moving the camera in the arrangement direction of the components, even if only one component is contained in the field of view of the camera, many images including captured images of a plurality of arranged components are included. A captured image can be acquired. These captured images fluctuate periodically with the relative movement of the camera, and fluctuate with the number of times of imaging ( relative movement amount of the camera) from imaging the previous part to imaging the next part as one period. The relative movement amount of the camera for one cycle corresponds to the component pitch. From this relationship, if the periodicity of the fluctuation of the captured image is evaluated, the component pitch can be measured from the relative movement amount of the camera for one cycle. This makes it possible to automate the measurement of the component pitch.

本発明は、撮像画像の変動の周期性を評価する手法として相互相関法を用い、カメラの撮像毎にその撮像画像と部品の基準画像との相関値を演算し、その相関値の変動の1周期分である相関値のピーク点間のカメラの相対移動量に基づいて部品ピッチを計測するようにすれば良い。ここで、部品の基準画像は、部品ピッチを計測する部品と同じ部品をカメラの視野の中心(基準位置)に収めて撮像した画像である。相関値がピーク点となる撮像画像は、部品の基準画像と最も類似度が高い撮像画像であり、実際に部品をカメラの視野の中心に収めて撮像した画像であることを意味するため、相関値がピーク点となる撮像画像を撮像したカメラの位置から部品の位置が判明する。この関係から、相関値のピーク点間のカメラの相対移動量が部品ピッチに相当するため、相関値のピーク点間のカメラの相対移動量から部品ピッチを計測することができる。 The present invention is, cross-correlation method used as a method of evaluating the periodicity of the variation of the captured image, it calculates a correlation value between the reference image of the captured image and the part for each imaging camera, the fluctuation of the correlation value The component pitch may be measured based on the relative movement amount of the camera between the peak points of the correlation value corresponding to one period. Here, the reference image of the component is an image obtained by capturing the same component as the component for measuring the component pitch in the center (reference position) of the camera field of view. The captured image with the peak correlation value is the captured image with the highest degree of similarity to the reference image of the component, and means that the image is actually captured with the component placed in the center of the camera's field of view. The position of the component is determined from the position of the camera that captured the captured image having the peak value. From this relationship, since the relative movement amount of the camera between the peak points of the correlation value corresponds to the component pitch, the component pitch can be measured from the relative movement amount of the camera between the peak points of the correlation value.

このような相互相関法を用いれば、撮像画像内の部品とその背景部分との間の輝度の差が小さい等の理由により、部品認識精度が悪い撮像条件下であっても、相関値がピーク点となる撮像画像を撮像したカメラの位置から部品の位置が判明するため、相関値のピーク点間のカメラの相対移動量から部品ピッチを計測することができる Using such a cross-correlation method, the correlation value peaks even under imaging conditions where the component recognition accuracy is poor due to a small difference in brightness between the component in the captured image and its background. Since the position of the component is determined from the position of the camera that captured the captured image as a point, the component pitch can be measured from the relative movement amount of the camera between the peak points of the correlation value .

本発明に用いる部品の基準画像は、予め、部品ピッチを計測する部品と同じ部品をカメラの視野の中心に収めて撮像して得られた画像を基準画像として記憶手段に記憶しておくようにしても良い。この際、部品ピッチ計測時の撮像条件と同じ撮像条件(背景、照明条件、カメラ位置等)で基準画像を撮像することが好ましい。   The reference image of the component used in the present invention is stored in advance in the storage means as an image obtained by placing the same component as the component whose component pitch is measured in the center of the field of view of the camera. May be. At this time, it is preferable to capture the reference image under the same imaging conditions (background, illumination conditions, camera position, etc.) as the imaging conditions at the time of component pitch measurement.

或は、請求項のように、カメラを部品の配列方向に相対移動させながら連続的に撮像する際に、先頭の部品の撮像画像を基準画像として用いるようにしても良い。このようにすれば、部品ピッチを計測する際に、同じ撮像条件で撮像した先頭の部品の撮像画像を基準画像とすることができ、正確な基準画像を容易に取得することができる。この際、先頭の部品を精度良く画像認識できない場合は、作業者が先頭の部品の位置を指定すれば良い。先頭の部品の位置の指定は、例えば、作業者が表示装置に表示された撮像画像を見て先頭の部品の位置を手動で入力しても良いし、部品供給元等から入手した設計データ等を参照して先頭の部品の位置の座標を手動で入力しても良い。 Alternatively, as described in claim 2 , when continuously capturing images while moving the camera relatively in the component arrangement direction, the captured image of the leading component may be used as the reference image. In this way, when measuring the component pitch, the captured image of the first component imaged under the same imaging conditions can be used as the reference image, and an accurate reference image can be easily acquired. At this time, if the image of the leading part cannot be recognized with high accuracy, the operator may specify the position of the leading part. For example, the position of the head part may be specified by, for example, the operator manually entering the position of the head part by looking at the captured image displayed on the display device, or design data obtained from a part supplier, etc. The coordinates of the position of the leading part may be manually input with reference to FIG.

以上説明した請求項1,2に係る発明は、同一形状の部品が配列された種々の装置に適用して実施でき、例えば、トレイフィーダに適用しても良い。トレイフィーダにセットするトレイ上には、同一形状の部品がマトリックス状に配列されているため、請求項のように、カメラをX方向に相対移動させてX方向の部品ピッチを計測し、カメラをY方向に相対移動させてY方向の部品ピッチを計測するようにすれば良い。これにより、トレイ上のX方向とY方向の部品ピッチを自動的に計測することができる。 The inventions according to claims 1 and 2 described above can be applied to various devices in which parts having the same shape are arranged, and may be applied to, for example, a tray feeder. On the tray to be set in the tray feeder, since the same shaped parts are arranged in a matrix, as claimed in claim 3, the camera are relatively moved in the X direction to measure the X direction of the component pitch, camera May be relatively moved in the Y direction to measure the component pitch in the Y direction. Thereby, it is possible to automatically measure the component pitch in the X direction and the Y direction on the tray.

尚、請求項は、前記請求項1に記載の「部品ピッチ計測装置」の発明と実質的に同じ技術思想を「部品ピッチ計測方法」として記載したものである。 In addition, claim 4 describes the technical idea substantially the same as the invention of the “component pitch measuring device” described in claim 1 as a “component pitch measuring method”.

図1は本発明の一実施例における部品ピッチ計測装置の構成例を示すブロック図である。FIG. 1 is a block diagram showing a configuration example of a component pitch measuring apparatus according to an embodiment of the present invention. 図2はトレイ上の部品の配列を示す平面図である。FIG. 2 is a plan view showing the arrangement of components on the tray. 図3はカメラを部品の配列方向に移動させながら連続的に撮像して部品ピッチを計測する方法を説明する図(その1)である。FIG. 3 is a diagram (part 1) for explaining a method of measuring the component pitch by continuously capturing images while moving the camera in the component arrangement direction. 図4はカメラを部品の配列方向に移動させながら連続的に撮像して部品ピッチを計測する方法を説明する図(その2)である。FIG. 4 is a diagram (part 2) for explaining a method of measuring the component pitch by continuously capturing images while moving the camera in the component arrangement direction. 図5は部品ピッチ計測プログラムの処理の流れを示すフローチャートである。FIG. 5 is a flowchart showing the flow of processing of the component pitch measurement program.

以下、本発明を実施するための形態を部品実装機に適用して具体化した一実施例を説明する。
まず、図1に基づいて部品実装機のシステム構成を説明する。
部品実装機は、コンピュータにより構成された制御装置11と、キーボード、マウス等の入力装置12と、液晶ディスプレイ、CRT等の表示装置13と、部品実装機制御プログラムや後述する図5の部品ピッチ計測プログラム等を記憶する記憶装置14と、吸着ノズルを保持する装着ヘッドをX−Y−Z方向に移動させる装着ヘッド移動装置15(移動機構)と、部品を実装する回路基板を搬送する基板搬送装置16と、吸着ノズルに吸着した部品を下方から撮像するパーツカメラ17と、回路基板の基準位置マーク等を上方から撮像するマークカメラ18等を備え、マークカメラ18を装着ヘッドに取り付けることで、マークカメラ18を装着ヘッドと一体的にX−Y−Z方向に移動させる構成となっている。本実施例では、マークカメラ18を後述する部品ピッチ計測用のカメラとして使用する。
Hereinafter, an embodiment in which a mode for carrying out the present invention is applied to a component mounter will be described.
First, the system configuration of the component mounter will be described with reference to FIG.
The component mounter includes a control device 11 configured by a computer, an input device 12 such as a keyboard and a mouse, a display device 13 such as a liquid crystal display and a CRT, a component mounter control program, and a component pitch measurement shown in FIG. A storage device 14 that stores a program and the like, a mounting head moving device 15 (moving mechanism) that moves a mounting head that holds a suction nozzle in the XYZ directions, and a substrate transfer device that transfers a circuit board on which components are mounted 16, a parts camera 17 that picks up the part sucked by the suction nozzle from below, a mark camera 18 that picks up the reference position mark of the circuit board from above, and the like. The camera 18 is configured to move in the XYZ direction integrally with the mounting head. In the present embodiment, the mark camera 18 is used as a camera for component pitch measurement described later.

この部品実装機には、トレイフィーダ19、テープフィーダ、バルクフィーダ等のフィーダがセットされ、該フィーダから部品が供給される。図2に示すように、トレイフィーダ19にセットされるトレイ20上には、同一形状の部品21がマトリックス状に配列されている。この場合、トレイ20上に配列された部品21を吸着ノズルで順番に吸着するためには、最初に吸着する先頭の部品21の中心位置の座標(X0 ,Y0 )、X方向の部品ピッチ、Y方向の部品ピッチ、X方向の部品数、Y方向の部品数等のデータを含むトレイデータが必要となる。   In this component mounting machine, a feeder such as a tray feeder 19, a tape feeder, or a bulk feeder is set, and components are supplied from the feeder. As shown in FIG. 2, on the tray 20 set in the tray feeder 19, parts 21 having the same shape are arranged in a matrix. In this case, in order to suck the components 21 arranged on the tray 20 in order with the suction nozzle, the coordinates (X0, Y0) of the center position of the first component 21 to be sucked first, the component pitch in the X direction, Y Tray data including data such as the component pitch in the direction, the number of components in the X direction, and the number of components in the Y direction is required.

そこで、本実施例では、制御装置11によって後述する図5の部品ピッチ計測プログラムを実行することで、トレイ20上の部品21の配列をマークカメラ18で連続的に撮像して、それらの撮像画像を用いて、相互相関法によりX,Y方向の部品ピッチと部品数を自動的に計数する。以下、部品ピッチと部品数の計数方法を説明する。   Therefore, in the present embodiment, the control device 11 executes a component pitch measurement program shown in FIG. 5 to be described later, whereby the arrangement of the components 21 on the tray 20 is continuously captured by the mark camera 18 and the captured images thereof. Is used to automatically count the component pitch and the number of components in the X and Y directions by the cross-correlation method. Hereinafter, a method for counting the component pitch and the number of components will be described.

吸着ノズルで最初に吸着する先頭の部品21の中心位置の座標(X0 ,Y0 )に関しては、マークカメラ18で先頭の部品21を撮像した画像から該先頭の部品21の外形形状を所定以上の精度で認識可能である場合は、マークカメラ18で先頭の部品21を撮像した画像を処理して該先頭の部品21の外形形状を認識して該先頭の部品21の中心位置の座標(X0 ,Y0 )を算出すれば良い。   With respect to the coordinates (X0, Y0) of the center position of the leading part 21 that is first picked up by the picking nozzle, the outer shape of the leading part 21 is more than a predetermined accuracy from the image of the leading part 21 captured by the mark camera 18. If the image can be recognized by the mark camera 18, an image obtained by capturing the leading component 21 is processed to recognize the outer shape of the leading component 21, and the coordinates (X0, Y0) of the center position of the leading component 21 are recognized. ) May be calculated.

一方、マークカメラ18の撮像画像内の部品21とその背景部分との間の輝度の差が小さい等の理由により、先頭の部品21の外形形状を精度良く認識できない場合は、作業者が先頭の部品21の中心位置を指定すれば良い。先頭の部品21の中心位置の指定方法は、例えば、マークカメラ18で先頭の部品21を撮像してその画像を表示装置13に表示させ、作業者が表示装置13に表示された撮像画像を見て先頭の部品21の中心位置を手動で入力しても良いし、部品供給元等から入手した設計データ等を参照して先頭の部品21の中心位置の座標(X0 ,Y0 )を入力装置12で入力しても良い。   On the other hand, if the outer shape of the leading component 21 cannot be accurately recognized due to a small luminance difference between the component 21 in the captured image of the mark camera 18 and its background portion, the operator must The center position of the component 21 may be designated. The method of specifying the center position of the first component 21 is, for example, imaging the first component 21 with the mark camera 18 and displaying the image on the display device 13 so that the operator can view the captured image displayed on the display device 13. The center position of the first component 21 may be manually input, or the coordinates (X0, Y0) of the center position of the first component 21 are input by referring to design data obtained from a component supplier or the like. You may enter in.

その後、図3に示すように、マークカメラ18を先頭の部品21の中心位置の真上に移動させて先頭の部品21をマークカメラ18の視野の中心に収めて撮像し、その撮像画像を基準画像として記憶装置14に記憶した後、マークカメラ18を装着ヘッド移動装置15により部品21の配列方向(X方向又はY方向)に移動させながら連続的に撮像する。ここで、「連続的に撮像」とは、計測対象となる部品ピッチと比べて十分に短い間隔で撮像を繰り返すことである。   Thereafter, as shown in FIG. 3, the mark camera 18 is moved right above the center position of the leading part 21 to capture the leading part 21 in the center of the visual field of the mark camera 18, and the captured image is used as a reference. After being stored in the storage device 14 as an image, the mark camera 18 is continuously imaged while being moved in the arrangement direction (X direction or Y direction) of the components 21 by the mounting head moving device 15. Here, “continuous imaging” is to repeat imaging at intervals sufficiently shorter than the component pitch to be measured.

このように、マークカメラ18を部品21の配列方向に移動させながら連続的に撮像すると、マークカメラ18の視野内に1つの部品21しか収まらない場合でも、配列された複数の部品21の撮像画像を含む多くの撮像画像を取得することができる。これらの撮像画像は、マークカメラ18の移動に伴って周期的に変動し、先の部品21を撮像してから次の部品21を撮像するまでの撮像回数(マークカメラ18の移動量)を1周期として変動し、その1周期分のマークカメラ18の移動量が部品ピッチに相当する。この関係から、撮像画像の変動の周期性を評価すれば、その1周期分のマークカメラ18の移動量から部品ピッチを計測することができる。   As described above, when images are continuously captured while the mark camera 18 is moved in the arrangement direction of the components 21, even if only one component 21 can be accommodated in the field of view of the mark camera 18, captured images of the plurality of components 21 arranged. Many captured images including can be acquired. These captured images periodically change with the movement of the mark camera 18, and the number of times (the amount of movement of the mark camera 18) from when the previous component 21 is captured until the next component 21 is captured is set to 1. The period fluctuates, and the movement amount of the mark camera 18 for one period corresponds to the component pitch. From this relationship, if the periodicity of the fluctuation of the captured image is evaluated, the component pitch can be measured from the amount of movement of the mark camera 18 for one cycle.

この場合、撮像画像の変動の周期性を評価する手法は、例えば、周波数解析を用いても良いし、相互相関法を用いても良い。以下、相互相関法を用いて部品ピッチを計測する方法を説明する。   In this case, for example, frequency analysis may be used as a method for evaluating the periodicity of fluctuation of the captured image, or a cross-correlation method may be used. Hereinafter, a method for measuring the component pitch using the cross-correlation method will be described.

上述したように、マークカメラ18を部品21の配列方向に移動させながら連続的に撮像し、その撮像毎にその撮像画像と記憶装置14に記憶された基準画像(先頭の部品21の撮像画像)との相関値を算出して、その相関値のデータを記憶装置14に時系列的に記憶していく。この相関値は、撮像画像と基準画像との類似度が高いほど大きな値となる。   As described above, images are continuously captured while the mark camera 18 is moved in the arrangement direction of the components 21, and the captured image and the reference image (captured image of the leading component 21) stored in the storage device 14 for each imaging. And the correlation value data is stored in the storage device 14 in time series. The correlation value increases as the similarity between the captured image and the reference image increases.

そして、マークカメラ18の移動量がトレイ20の幅を越えた時点で、マークカメラ18の移動と撮像を終了して、記憶装置14に記憶されている相関値の時系列データを分析して相関値のピーク点をサーチし、その相関値の変動の1周期分である相関値のピーク点間のマークカメラ18の移動量(以下「カメラ移動量」という)を計算して、その計算値を部品ピッチとして記憶装置14に記憶する。   When the amount of movement of the mark camera 18 exceeds the width of the tray 20, the movement and imaging of the mark camera 18 are finished, and the time series data of the correlation values stored in the storage device 14 are analyzed and correlated. The peak point of the value is searched, the movement amount of the mark camera 18 between the peak points of the correlation value, which is one cycle of the fluctuation of the correlation value (hereinafter referred to as “camera movement amount”), is calculated, The component pitch is stored in the storage device 14.

ここで、相関値がピーク点となる撮像画像は、部品21の基準画像と最も類似度が高い撮像画像であり、実際に部品21をマークカメラ18の視野の中心に収めて撮像した画像であることを意味するため、相関値がピーク点となる撮像画像を撮像したマークカメラ18の位置から部品21の位置が判明する。この関係から、相関値のピーク点間のカメラ移動量が部品ピッチに相当するため、相関値のピーク点間のカメラ移動量から部品ピッチを計測することができる。   Here, the captured image whose correlation value is the peak is the captured image having the highest similarity to the reference image of the component 21, and is an image actually captured by capturing the component 21 in the center of the visual field of the mark camera 18. For this reason, the position of the component 21 is determined from the position of the mark camera 18 that has captured the captured image at which the correlation value is the peak point. From this relationship, since the camera movement amount between the peak points of the correlation value corresponds to the component pitch, the component pitch can be measured from the camera movement amount between the peak points of the correlation value.

この際、全てのピーク点間のカメラ移動量をそれぞれ計算して、各部品21毎に部品ピッチを記憶しても良いし、或は、全てのピーク点間のカメラ移動量の平均値を算出して、その平均値を共通の部品ピッチとして記憶しても良い。   At this time, the camera movement amount between all peak points may be calculated and the component pitch may be stored for each component 21 or the average value of the camera movement amounts between all peak points may be calculated. Then, the average value may be stored as a common component pitch.

また、各部品21の位置で相関値がピークとなるため、相関値のピーク点の数を計数して、そのピーク点の数を部品数として記憶装置14に記憶する。
以上説明した本実施例の部品ピッチの計測処理は、制御装置11によって図5の部品ピッチ計測プログラムに従って次のように実行される。
Further, since the correlation value has a peak at the position of each component 21, the number of peak points of the correlation value is counted, and the number of peak points is stored in the storage device 14 as the number of components.
The component pitch measurement process of the present embodiment described above is executed by the control device 11 as follows according to the component pitch measurement program of FIG.

図5の部品ピッチ計測プログラムは、トレイフィーダ19からトレイ20が所定の部品供給位置に引き出される毎に実行され、特許請求の範囲でいう部品ピッチ計測手段としての役割を果たす。本プログラムが起動されると、まず、ステップ101で、マークカメラ18を先頭の部品21の中心位置の真上に移動させて、次のステップ102で、マークカメラ18の視野の中心位置で先頭の部品21を撮像し、次のステップ103で、先頭の部品21の撮像画像を基準画像として記憶装置14に記憶する。   The component pitch measurement program of FIG. 5 is executed each time the tray 20 is pulled out from the tray feeder 19 to a predetermined component supply position, and serves as a component pitch measurement means in the claims. When this program is started, first, in step 101, the mark camera 18 is moved to a position just above the center position of the head component 21, and in the next step 102, the head of the mark camera 18 is displayed at the center position of the field of view. The component 21 is imaged, and in the next step 103, the captured image of the leading component 21 is stored in the storage device 14 as a reference image.

この後、ステップ104に進み、マークカメラ18を装着ヘッド移動装置15により部品21の配列方向(X方向又はY方向)に所定量移動させる。ここで、「所定量」は、連続的に撮像する間隔(カメラ移動量)を設定する値であり、計測対象となる部品ピッチと比べて十分に短い間隔に設定されている。この後、ステップ05に進み、マークカメラ18で撮像し、次のステップ106で、その撮像画像と記憶装置14に記憶された基準画像(先頭の部品21の撮像画像)との相関値を算出して、その相関値を記憶装置14に記憶する。   Thereafter, the process proceeds to step 104 where the mark camera 18 is moved by a predetermined amount in the arrangement direction (X direction or Y direction) of the components 21 by the mounting head moving device 15. Here, the “predetermined amount” is a value for setting a continuous image capturing interval (camera movement amount), and is set to a sufficiently short interval as compared with the component pitch to be measured. Thereafter, the process proceeds to step 05, where the image is captured by the mark camera 18, and in the next step 106, a correlation value between the captured image and the reference image (captured image of the first component 21) stored in the storage device 14 is calculated. Then, the correlation value is stored in the storage device 14.

この後、ステップ107に進み、先頭の部品21の位置からのカメラ移動量がトレイ20の幅を越えたか否かを判定し、カメラ移動量がトレイ20の幅を越えていないと判定されれば、上述したステップ104〜106の処理を繰り返す。これにより、マークカメラ18を部品21の配列方向に移動させながら連続的に撮像し、その撮像毎にその撮像画像と基準画像との相関値を算出して、その相関値のデータを記憶装置14に時系列的に記憶していく。   Thereafter, the process proceeds to step 107, where it is determined whether or not the camera movement amount from the position of the leading component 21 exceeds the width of the tray 20, and if it is determined that the camera movement amount does not exceed the width of the tray 20. , The above-described steps 104 to 106 are repeated. Thereby, the mark camera 18 is continuously imaged while moving in the arrangement direction of the components 21, the correlation value between the captured image and the reference image is calculated for each imaging, and the correlation value data is stored in the storage device 14. It memorizes in time series.

その後、マークカメラ18の移動量がトレイ20の幅を越えた時点で、ステップ107からステップ108に進み、記憶装置14に記憶されている相関値の時系列データを分析して相関値のピーク点をサーチする。この後、ステップ109に進み、相関値のピーク点間のカメラ移動量を計算して、その計算値を部品ピッチとして記憶装置14に記憶する。この際、全てのピーク点間のカメラ移動量をそれぞれ計算して、各部品21毎に部品ピッチを記憶しても良いし、或は、全てのピーク点間のカメラ移動量の平均値を算出して、その平均値を共通の部品ピッチとして記憶しても良い。   Thereafter, when the amount of movement of the mark camera 18 exceeds the width of the tray 20, the process proceeds from step 107 to step 108 where the time series data of the correlation values stored in the storage device 14 is analyzed and the peak value of the correlation value Search for. Thereafter, the process proceeds to step 109, the camera movement amount between the peak points of the correlation value is calculated, and the calculated value is stored in the storage device 14 as the component pitch. At this time, the camera movement amount between all peak points may be calculated and the component pitch may be stored for each component 21 or the average value of the camera movement amounts between all peak points may be calculated. Then, the average value may be stored as a common component pitch.

この後、ステップ110に進み、相関値のピーク点の数を計数して、そのピーク点の数を部品数として記憶装置14に記憶して、本プログラムを終了する。
本プログラムにより、マークカメラ18をX方向に移動させてX方向の部品ピッチを計測すると共にX方向の部品数を計数し、マークカメラ18をY方向に移動させてY方向の部品ピッチを計測すると共にY方向の部品数を計数する。
Thereafter, the process proceeds to step 110, the number of peak points of the correlation value is counted, the number of peak points is stored in the storage device 14 as the number of parts, and the program is terminated.
With this program, the mark camera 18 is moved in the X direction to measure the component pitch in the X direction, the number of components in the X direction is counted, and the mark camera 18 is moved in the Y direction to measure the component pitch in the Y direction. At the same time, the number of parts in the Y direction is counted.

以上説明した本実施例のように、相互相関法を用いて部品ピッチを計測すれば、撮像画像内の部品21とその背景部分との間の輝度の差が小さい等の理由により、部品認識精度が悪い撮像条件下であっても、相関値がピーク点となる撮像画像を撮像したマークカメラ18の位置から部品21の位置が判明するため、相関値のピーク点間のマークカメラ18の移動量から部品ピッチを精度良く計測することができる。   If the component pitch is measured using the cross-correlation method as in the present embodiment described above, the component recognition accuracy is due to a small difference in luminance between the component 21 in the captured image and its background portion. Even under poor imaging conditions, the position of the component 21 can be determined from the position of the mark camera 18 that captured the captured image with the correlation value being the peak point. Therefore, the amount of movement of the mark camera 18 between the peak points of the correlation value Therefore, the component pitch can be measured with high accuracy.

また、図4に示すように、照明変化等による部分的な輝度変化、部品21の回転ずれ、部品21上面のごみ・汚れ等があっても、その部品21の中心位置で相関値がピークとなるため、相関値のピーク点間のマークカメラ18の移動量から部品ピッチを精度良く計測することができ、照明変化等による部分的な輝度変化、回転ずれ、ごみ・汚れ等に対するロバスト性を向上することができる。   Further, as shown in FIG. 4, even if there is a partial luminance change due to a change in illumination, rotation deviation of the component 21, dust or dirt on the upper surface of the component 21, the correlation value has a peak at the center position of the component 21. Therefore, the component pitch can be accurately measured from the amount of movement of the mark camera 18 between the peak points of the correlation value, and the robustness against partial brightness change due to illumination change, rotation deviation, dust / dirt, etc. is improved. can do.

また、本実施例では、マークカメラ18を部品の配列方向に移動させながら連続的に撮像する際に、先頭の部品21の撮像画像を基準画像として用いるようにしたので、部品ピッチを計測する際に、同じ撮像条件で撮像した先頭の部品21の撮像画像を基準画像とすることができ、正確な基準画像を容易に取得することができる。   Further, in this embodiment, when continuously capturing images while moving the mark camera 18 in the component arrangement direction, the captured image of the leading component 21 is used as the reference image, so when measuring the component pitch. In addition, the captured image of the first component 21 imaged under the same imaging conditions can be used as a reference image, and an accurate reference image can be easily obtained.

但し、基準画像は、先頭の部品21の撮像画像に限定されず、例えば、予め、部品ピッチを計測する部品21と同じ部品をカメラの視野の中心(基準位置)に収めて撮像して得られた画像を基準画像として記憶装置14に記憶しておくようにしても良い。この際、部品ピッチ計測時の撮像条件と同じ撮像条件(背景、照明条件、カメラ位置等)で基準画像を撮像することが好ましい。   However, the reference image is not limited to the picked-up image of the leading component 21 and is obtained, for example, by previously capturing the same component as the component 21 for measuring the component pitch in the center (reference position) of the camera field of view. The stored image may be stored in the storage device 14 as a reference image. At this time, it is preferable to capture the reference image under the same imaging conditions (background, illumination conditions, camera position, etc.) as the imaging conditions at the time of component pitch measurement.

また、本実施例では、撮像画像の変動の周期性を評価する手法として、相互相関法を用いたが、本発明に関連する参考例として、周波数解析を用いて撮像画像の変動の周期性を評価する構成が考えられ、この場合でも、撮像画像の部品認識精度は要求されないため、部品認識精度が悪い撮像条件下や、照明変化等による部分的な輝度変化、回転ずれ、ごみ・汚れ等があっても、撮像画像の変動の1周期分のカメラ移動量から部品ピッチを計測することができる。
In this embodiment, the cross-correlation method is used as a method for evaluating the periodicity of the fluctuation of the captured image. However, as a reference example related to the present invention, the periodicity of the fluctuation of the captured image is determined using frequency analysis. configured to evaluate is considered, even in this case, since the component recognition accuracy of the captured image is not required, components and recognition accuracy is poor imaging conditions, partial brightness change due to illumination change or the like, the deviation rotary, dust, dirt, etc. is Even in such a case, the component pitch can be measured from the amount of camera movement for one cycle of fluctuation of the captured image.

また、本実施例では、トレイ20上にマトリックス状に配列された部品21のピッチを計測するようにしたが、例えば、テープフィーダの部品収納テープに収納された部品のピッチを計測したり、バルクフィーダ等の各種のフィーダから供給される部品のピッチを計測するようにしても良い。   In this embodiment, the pitch of the components 21 arranged in a matrix on the tray 20 is measured. For example, the pitch of the components stored in the component storage tape of the tape feeder is measured, or the bulk You may make it measure the pitch of the components supplied from various feeders, such as a feeder.

その他、本発明は、フィーダに限定されず、同一形状の部品が配列された装置における部品ピッチを計測するシステムに広く適用して実施できる等、要旨を逸脱しない範囲内で種々変更して実施できる。   In addition, the present invention is not limited to the feeder, and can be widely applied to a system for measuring a component pitch in an apparatus in which components having the same shape are arranged, and can be implemented with various modifications without departing from the gist. .

11…制御装置(部品ピッチ計測手段)、14…記憶装置、15…装着ヘッド移動装置(移動機構)、17…パーツカメラ、18…マークカメラ、19…トレイフィーダ、20…トレイ、21…部品   DESCRIPTION OF SYMBOLS 11 ... Control apparatus (part pitch measurement means), 14 ... Memory | storage device, 15 ... Mounting head moving apparatus (movement mechanism), 17 ... Parts camera, 18 ... Mark camera, 19 ... Tray feeder, 20 ... Tray, 21 ... Parts

Claims (4)

同一形状の部品が配列されたものにおける部品ピッチを計測する部品ピッチ計測装置において、
前記部品を撮像するカメラと、
前記カメラを相対移動させる移動機構と、
前記カメラを前記移動機構により前記部品の配列方向に相対移動させながら連続的に撮像し、それらの撮像画像の変動の周期性を評価してその変動の周期性と前記カメラの相対移動量との関係に基づいて部品ピッチを計測する部品ピッチ計測手段と
を備え
前記部品ピッチ計測手段は、前記カメラの撮像毎にその撮像画像と前記部品の基準画像との相関値を演算し、その相関値のピーク点間の前記カメラの相対移動量に基づいて部品ピッチを計測することを特徴とする部品ピッチ計測装置。
In a component pitch measuring device that measures a component pitch in an array of components having the same shape,
A camera for imaging the component;
A moving mechanism for relatively moving the camera;
The camera is continuously imaged while being relatively moved in the arrangement direction of the parts by the moving mechanism, and the periodicity of fluctuations of the captured images is evaluated, and the periodicity of the fluctuations and the relative movement amount of the camera are calculated . Component pitch measuring means for measuring the component pitch based on the relationship ,
The component pitch measurement means calculates a correlation value between the captured image and the reference image of the component for each imaging of the camera, and calculates a component pitch based on a relative movement amount of the camera between peak points of the correlation value. parts pitch measurement device, characterized in that the measuring.
前記部品ピッチ計測手段は、前記カメラを部品の配列方向に相対移動させながら連続的に撮像する際に、先頭の部品の撮像画像を前記基準画像として用いることを特徴とする請求項に記載の部品ピッチ計測装置。 2. The component pitch measurement unit according to claim 1 , wherein when capturing images continuously while relatively moving the camera in a component arrangement direction, the component pitch measurement unit uses a captured image of a leading component as the reference image. Component pitch measuring device. 前記部品は、トレイフィーダにセットされるトレイ上にマトリックス状に配列された部品であり、
前記部品ピッチ計測手段は、前記カメラをX方向に相対移動させてX方向の部品ピッチを計測し、前記カメラをY方向に相対移動させてY方向の部品ピッチを計測することを特徴とする請求項1又は2に記載の部品ピッチ計測装置。
The parts are parts arranged in a matrix on a tray set in a tray feeder,
The component pitch measurement means measures the component pitch in the X direction by relatively moving the camera in the X direction, and measures the component pitch in the Y direction by relatively moving the camera in the Y direction. Item 1. The component pitch measuring device according to Item 1 or 2 .
同一形状の部品が配列されたものにおける部品ピッチを計測する部品ピッチ計測方法において、
カメラを前記部品の配列方向に相対移動させながら連続的に撮像し、それらの撮像画像の変動の周期性を評価してその変動の周期性と前記カメラの相対移動量との関係に基づいて部品ピッチを計測する部品ピッチ計測方法であって、
前記カメラの撮像毎にその撮像画像と前記部品の基準画像との相関値を演算し、その相関値のピーク点間の前記カメラの相対移動量に基づいて部品ピッチを計測することを特徴とする部品ピッチ計測方法。
In the component pitch measurement method for measuring the component pitch in the arrangement of components of the same shape,
Images are taken continuously while relatively moving the camera in the arrangement direction of the parts, and the periodicity of fluctuations in the captured images is evaluated, and the parts are based on the relationship between the periodicity of the fluctuations and the relative movement amount of the camera. A component pitch measurement method for measuring pitch,
A correlation value between the captured image and the reference image of the component is calculated for each imaging of the camera, and a component pitch is measured based on a relative movement amount of the camera between peak points of the correlation value. Component pitch measurement method.
JP2011107817A 2011-05-13 2011-05-13 Component pitch measuring apparatus and component pitch measuring method Active JP5755935B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2011107817A JP5755935B2 (en) 2011-05-13 2011-05-13 Component pitch measuring apparatus and component pitch measuring method
CN201210146702.1A CN102778206B (en) 2011-05-13 2012-05-11 Part spacing measuring means and part spacing measuring method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2011107817A JP5755935B2 (en) 2011-05-13 2011-05-13 Component pitch measuring apparatus and component pitch measuring method

Publications (2)

Publication Number Publication Date
JP2012238777A JP2012238777A (en) 2012-12-06
JP5755935B2 true JP5755935B2 (en) 2015-07-29

Family

ID=47123206

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2011107817A Active JP5755935B2 (en) 2011-05-13 2011-05-13 Component pitch measuring apparatus and component pitch measuring method

Country Status (2)

Country Link
JP (1) JP5755935B2 (en)
CN (1) CN102778206B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106482653A (en) * 2015-08-28 2017-03-08 华硕电脑股份有限公司 Optical measurement device and gap method for measurement
JP6788606B2 (en) * 2015-11-17 2020-11-25 株式会社Fuji Implementation processing method
JP6647049B2 (en) * 2016-01-18 2020-02-14 ヤマハ発動機株式会社 Pitch measuring device, pitch measuring method and component mounting device
JP6882522B2 (en) * 2017-12-19 2021-06-02 株式会社Fuji Mounting device, detection device and detection method

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4596467A (en) * 1984-03-16 1986-06-24 Hughes Aircraft Company Dissimilar superimposed grating precision alignment and gap measurement systems
JP3274132B2 (en) * 1990-03-20 2002-04-15 株式会社日立製作所 How to create a reference pattern for visual inspection
US5371375A (en) * 1992-06-24 1994-12-06 Robotic Vision Systems, Inc. Method for obtaining three-dimensional data from multiple parts or devices in a multi-pocketed tray
WO1999025168A1 (en) * 1997-11-10 1999-05-20 Matsushita Electric Industrial Co., Ltd. Part mounting apparatus and part supply apparatus
US6072898A (en) * 1998-01-16 2000-06-06 Beaty; Elwin M. Method and apparatus for three dimensional inspection of electronic components
JP2000208990A (en) * 1999-01-19 2000-07-28 Matsushita Electric Ind Co Ltd Data preparation method and electronic parts mounting apparatus
JP2001345595A (en) * 2000-05-30 2001-12-14 Hitachi Kokusai Electric Inc Method for generating tray data of surface part mounter
JP2002324259A (en) * 2001-04-25 2002-11-08 Nippon Conlux Co Ltd Method and device for identifying coin
JP2006041260A (en) * 2004-07-28 2006-02-09 Juki Corp Method for correcting nozzle position of electronic part mounting device
JP4377782B2 (en) * 2004-09-14 2009-12-02 アイパルス株式会社 Mounting board inspection method and apparatus
JP4262232B2 (en) * 2005-10-17 2009-05-13 リンテック株式会社 measuring device
CN101424518B (en) * 2007-10-31 2011-06-15 比亚迪股份有限公司 Push button space detecting method and system
JP4834703B2 (en) * 2008-08-22 2011-12-14 ヤマハ発動機株式会社 Surface mount machine
JP5214478B2 (en) * 2009-01-22 2013-06-19 株式会社日立ハイテクインスツルメンツ Electronic component mounting method and electronic component mounting apparatus
CN101603818B (en) * 2009-07-14 2011-02-02 中国科学院安徽光学精密机械研究所 System and method for measuring width between perforated lines and width between perforated line and edge for tipping paper of cigarette

Also Published As

Publication number Publication date
JP2012238777A (en) 2012-12-06
CN102778206A (en) 2012-11-14
CN102778206B (en) 2017-05-17

Similar Documents

Publication Publication Date Title
JP6014315B2 (en) Measuring method of electronic component mounting device
JP5926881B2 (en) Image processing component data creation method and image processing component data creation device
US10806037B2 (en) Component mounting system
JP5510342B2 (en) Electronic component mounting method
CN109076728B (en) Substrate working machine
JP5755935B2 (en) Component pitch measuring apparatus and component pitch measuring method
JP6292825B2 (en) Substrate processing equipment
US20060016066A1 (en) Pick and place machine with improved inspection
JP2010147401A (en) Electronic component mounting apparatus and image distortion correcting method
WO2017081773A1 (en) Image processing device and image processing method for base plate
WO2018154691A1 (en) Substrate working device and image processing method
JP2011043998A (en) Method and apparatus for analyzing image
JP4892463B2 (en) Parts feed pitch setting method
JP4869214B2 (en) Component mounting time simulation method
JP6904978B2 (en) Parts mounting machine
JP6706625B2 (en) Board position search device and component mounter
JP7319264B2 (en) Control method, electronic component mounting device
JP2015225967A (en) Component mounting method and component mounting device
JP7253476B2 (en) Component recognition data creation method and component recognition data creation program
JP6418739B2 (en) Evaluation device, surface mounter, evaluation method
CN110463377B (en) Mounting apparatus and information processing method
JP7466746B2 (en) Countermeasure information presentation device and countermeasure information presentation method
JP7384916B2 (en) Information processing equipment and mounting equipment
JP4306353B2 (en) Pixel rate calculation method in image recognition apparatus
US11751371B2 (en) Component mounting device

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20140428

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20141113

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20141119

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20150114

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20150507

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20150528

R150 Certificate of patent or registration of utility model

Ref document number: 5755935

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

S533 Written request for registration of change of name

Free format text: JAPANESE INTERMEDIATE CODE: R313533

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250