JP2011183056A - Ultrasonic diagnostic apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain information on the position and attitude of an ultrasonic probe of high value in simpler constitution and without troublesome work. <P>SOLUTION: A head mount display (HMD) 11 includes a video camera unit 38 incorporating a CCD 39. The video camera unit 38 picks up an image (hand-side image) of the ultrasonic probe 10 and a part where the ultrasonic probe 10 is applied. A main control unit 50 on a processor unit 15 stores an ultrasonic image obtained by the ultrasonic probe 10 as related to the hand side image at almost similar imaging timing in a frame memory 56. The main control unit 50 reads a selected ultrasonic image with the hand side image from the frame memory 56 to be output to a memory device 18 in accordance with operation to a saving button 17b. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an ultrasonic diagnostic apparatus including a head mounted display.

  2. Description of the Related Art Conventionally, in the medical field, ultrasonic diagnosis in which an ultrasonic probe is applied to the body surface of a patient (subject) and the obtained ultrasonic image is observed on a monitor has been popular. In such an ultrasound diagnosis, the surgeon performs while looking at the ultrasound probe (hand) and the ultrasound image of the monitor alternately, which makes it difficult to move the line of sight. There was a problem of being neglected. In order to solve this problem, a transmissive head-mounted display (hereinafter abbreviated as HMD) is attached to an operator, an ultrasonic image is displayed on the screen of the HMD, and the hand and the ultrasonic image can be observed simultaneously. A possible ultrasonic diagnostic apparatus has been proposed (see Patent Document 1).

  On the other hand, where the ultrasound probe is placed on the body surface, in other words, for reference when interpreting the ultrasound image after inspection and to ensure the reproducibility of the procedure, that is, the position and posture of the ultrasound probe (body There is a need to memorize information such as the angle to the table. To meet this need, an ultrasonic diagnostic system that installs a video camera that reflects the procedure on the bedside, synthesizes the video from the video camera and an ultrasonic image, displays it on a monitor, and stores the synthesized image in a storage device. An apparatus has been proposed (see Patent Document 2).

  In Patent Document 2, two video cameras are prepared on the head side and the leg side of a patient, and when the hand becomes the blind spot of one video camera with the operator's arm or the like, the video camera is switched to the other video camera. Is disclosed. Furthermore, it is described that only one video camera may be used.

JP 2000-157542 A JP 2006-000400 A

  When there are two video cameras described in Patent Document 2, switching is troublesome. In the case of one, it is necessary to move the video camera when the hand becomes a blind spot, which is also troublesome. Considering use outside the hospital, such as home-visit medical care and business trips, video cameras are bulky and difficult to operate.

  Speaking from the main point of memorizing the position and orientation of the ultrasonic probe, it is only necessary to photograph at least a portion where the ultrasonic probe and the ultrasonic probe are assigned. Since the video is also filmed, the value of the information is low.

  In addition, since the composite image of the video camera image and the ultrasonic image is always stored, the capacity of the storage device is pressed.

  The present invention has been made in view of the above background, and an object of the present invention is to obtain information on the position and posture of a high-value ultrasonic probe without a troublesome operation with a simpler configuration.

  In order to achieve the above object, the ultrasonic diagnostic apparatus of the present invention transmits an ultrasonic wave to an observation site of a subject, receives a reflected wave from the observation site, and outputs a detection signal corresponding to the reflected wave. An ultrasonic probe with a built-in ultrasonic transducer, and a display unit that is attached to an operator and can observe the state of the subject and can display an ultrasonic image generated based on a detection signal A head-mounted display, which is equipped with an imaging means for capturing a hand image that reflects the position and orientation of the ultrasonic probe relative to the subject, and instructs to store ultrasonic images for a predetermined frame The operation input means, the storage means, and the ultrasound image and the hand image taken at substantially the same timing are associated in accordance with the operation input signal from the operation input means. Characterized in that it comprises a storage control means for storing in said memory means.

  Note that “substantially the same timing” refers to a case where the difference in imaging timing between the ultrasound image and the hand image stored in association is within 0.1 seconds, for example.

  It is preferable to include display control means for combining and displaying the ultrasonic image and the hand image on the display means. The display means may be a monitor provided in the ultrasonic diagnostic apparatus, a monitor different from that provided, or a display unit of the head mounted display.

  According to the present invention, an imaging unit for capturing a hand image that reflects the position and orientation of an ultrasound probe with respect to a subject is mounted on a head-mounted display, and an ultrasound image having substantially the same imaging timing according to an operation input and Since the hand images are stored in association with each other, it is possible to obtain information on the position and posture of the ultrasonic probe with high value without a troublesome work with a simpler configuration.

It is a figure which shows the outline of an ultrasonic diagnostic apparatus, and the mode of a technique. It is an external appearance perspective view of an ultrasonic probe. It is an external appearance perspective view of a head mounted display. It is a block diagram which shows the electric constitution of an ultrasonic diagnosing device. It is explanatory drawing which shows the example of a display of a monitor. It is a timing chart which shows the generation timing of an ultrasonographic image and a hand image. It is a flowchart which shows the procedure of an ultrasonic diagnosis.

  In FIG. 1, the ultrasonic diagnostic apparatus 2 includes an ultrasonic probe 10, a transmission type head mounted display (hereinafter abbreviated as HMD) 11, and an ultrasonic observation device 12. The ultrasound probe 10 is applied to the body surface such as the abdomen of the patient Pa held by the operator Op and rested on the bed 13. The HMD 11 is attached to the operator Op and can observe an ultrasonic image while securing a peripheral visual field. The ultrasonic observation device 12 includes a processor unit 15, a monitor 16, an operation unit 17, and a storage device 18 mounted on a portable cart 14 beside the bed 13. The storage device 18 incorporates a driver for a removable disk such as a hard disk drive or MO.

  In FIG. 2, the ultrasonic probe 10 includes a cable 25 connected to the processor unit 15, a gripping unit 26 gripped by the operator Op, and an ultrasonic transducer (hereinafter abbreviated as UT) array 27. Part 28. As is well known, the UT array 27 is a plurality of UTs arranged at a predetermined pitch, and a convex type is illustrated in the figure.

  In FIG. 3, the HMD 11 has a glasses-like form, and includes a lens part 35 that faces the left and right eyes of the operator Op, and a vine part 36 that is foldable on both the left and right sides of the lens part 35. The lens unit 35 is substantially transparent to ensure a peripheral visual field, and also serves as a screen (display unit) for displaying an ultrasonic image and observing the surgeon Op.

  At the joint between the left lens portion 35 and the vine portion 36, a video camera portion 38 with a photographing lens 37 exposed on the front surface is provided. The video camera unit 38 houses a CCD 39 for photographing the tip of the line of sight of the operator Op through the photographing lens 37.

  On the other hand, the right vine portion 36 is provided with a storage portion 41 from which a cable 40 connected to the processor portion 15 is connected. The housing 41 has a driver (not shown) for controlling the drive of the CCD 39, an analog signal processing circuit (AFE) 42, a digital signal processing circuit (DSP, see FIG. 4) 43, and an ultrasonic image on the lens unit 35. An electronic circuit such as a display control unit (not shown) for displaying is stored. Note that these electronic circuits may be incorporated in the processor unit 15 instead of the HMD 11.

  The AFE 42 amplifies the imaging signal output from the CCD 39, performs A / D conversion, and digitizes the imaging signal. The DSP 43 performs various signal processing such as color separation, color interpolation, gain correction, white balance adjustment, and gamma correction on the digital image pickup signal from the AFE 42 to generate an image (hereinafter referred to as a “hand image”). The hand image is generated, for example, at a frame rate of 30 frames / second (see FIG. 6).

  In FIG. 4, the main control unit 50 of the processor unit 15 controls the overall operation of the ultrasonic observation device 12. The main control unit 50 is connected to each unit via a data bus, an address bus, and a control line (not shown). The ROM 51 stores various programs (OS, application programs, etc.) and data (graphic data, etc.) for controlling the operation of the ultrasonic observer 12. The main control unit 50 reads necessary programs and data from the ROM 51, develops them in the RAM 52, which is a working memory, and sequentially processes the read programs.

  The main control unit 50 receives an operation input signal from the operation unit 17 and causes each unit to execute an operation corresponding to the operation input signal. The operation unit 17 is provided with a freeze button 17a and a save button 17b. The freeze button 17a is operated when switching the moving image display of the ultrasonic image to the still image display. The save button 17b is operated when an ultrasonic image and a hand image for a predetermined frame (for example, one to several frames) are stored in the storage device 18.

  The transmitter / receiver 53 includes a pulser and a receiver, and is provided for the number of UTs constituting the UT array 27. The transmission / reception unit 53 selects a pulsar to be driven from a plurality of pulsars, and sequentially switches them at a predetermined time interval. Specifically, for example, when the transmission / reception channel (the number of UTs) is 128 channels, the adjacent 48 channels out of 128 channels are set as one block, and the channels are driven with an arbitrary delay difference. Select and shift one to several channels to be driven for each transmission / reception of ultrasonic waves and reflected waves. The pulsar of the transmission / reception unit 53 transmits an excitation pulse for causing the UT to generate an ultrasonic wave based on the drive signal transmitted from the main control unit 50. By doing so, ultrasonic waves are scanned on the site to be observed of the patient Pa.

  The receiver of the transmission / reception unit 53 receives the reflected wave from the observation site of the patient Pa, amplifies the detection signal output from the UT, performs A / D conversion, and images the detection signal digitized by A / D conversion. The data is output to the processing unit 54. The image processing unit 54 performs orthogonal detection processing on the detection signal from the transmission / reception unit 53 to convert the detection signal into a complex baseband. Then, reception focus processing is performed by phasing addition on detection signals obtained by transmitting and receiving a plurality of times of ultrasonic waves and reflected waves, and an ultrasonic image (for example, a B-mode image) is generated. The ultrasonic image is generated at a frame rate of 10 frames / second, for example (see FIG. 6).

  As shown in FIG. 5, the display control unit 55 displays the ultrasound image PX and the hand image PY from the video camera unit 38 in parallel on the monitor 16 together with patient information and information on the examination site. Further, the display control unit 55 outputs the ultrasonic image PX to the HMD 11. The ultrasonic image PX from the display control unit 55 is displayed on the lens unit 35 of the HMD 11. Note that a composite image of the same ultrasonic image PX and hand image PY displayed on the monitor 16 may be output to the HMD 11. Alternatively, the composite image may be output to an external monitor different from the monitor 16.

  Since the lens part 35 of the HMD 11 is substantially transparent and functions as a screen for displaying an ultrasonic image, the line of sight of the operator Op is concentrated on the ultrasonic probe 10 during the procedure. Therefore, in the hand image PY, the point of the line of sight when the operator Op wears the HMD 11 and performs the procedure, that is, the portion where the ultrasonic probe 10 and the ultrasonic probe 10 are assigned is displayed.

  Returning to FIG. 4, a frame memory 56 is connected to the main controller 50. The frame memory 56 temporarily stores a plurality of frames, for example, 100 frames each of ultrasonic images and hand images. The frame memory 56 deletes the oldest of the ultrasonic images and hand images for a plurality of frames, and sequentially updates the latest ultrasonic images and hand images sequentially sent from the HMD 11 and the image processing unit 54. The frame memory 56 is a pair of an ultrasonic image and a hand image in which the imaging timing is substantially the same, that is, the difference between the reception start time of the reflected wave on the UT and the signal charge accumulation start time of the CCD 39 is within 0.1 seconds, for example. Are stored in association with each other. The difference between the reception start time of the reflected wave from the UT and the signal 39 charge accumulation start time of the CCD 39 is set to within 0.1 seconds. If there is a difference greater than 0.1 seconds, the difference between the ultrasonic image and the hand image This is because simultaneity cannot be maintained.

  When the freeze button 17a is operated, the update of the frame memory 56 is temporarily stopped. For this reason, the ultrasonic image and the hand image generated when the freeze button 17a is operated, and the ultrasonic image and the hand image for a plurality of previous frames (99 frames when storing 100 frames) are stored in the frame memory. Stocked at 56.

  The ultrasonic images and hand images for a plurality of frames stocked in the frame memory 56 by the operation of the freeze button 17a are displayed on the monitor 16 as candidates for selecting images for a predetermined frame to be stored in the storage device 18. . The displayed image can be sent and returned by operating the operation unit 17. The surgeon Op selects an appropriate image from the synthesized image of the ultrasonic image and the hand image displayed on the monitor 16, and operates the save button 17b. In response to the operation of the save button 17b, the main control unit 50 reads out an ultrasonic image for a predetermined frame selected by the operator Op and a hand image associated therewith from the frame memory 56, and stores them in the storage device 18. Output.

  Here, as shown in FIG. 6, the generation timing of the ultrasonic image and the hand image with substantially the same shooting timing is delayed by ΔT for the ultrasonic image than for the hand image. The generation timing of the ultrasound image and the hand image is the timing at which the ultrasound image and the hand image are output from the image processing unit 54 and the DSP 43, respectively. The reason why the generation timing is shifted by ΔT between the ultrasound image and the hand image having substantially the same imaging timing is that the time required for various processes in the transmission / reception unit 53 and the image processing unit 54 is longer than that of the AFE 42 and the DSP 43.

  For this reason, the generation timing of the ultrasonic image “1” and the local image “1” connected by the arrows is shifted by ΔT, but the imaging timing is substantially the same. The ultrasonic images “2”, “3”, “4”, “5”,... And the local images “4”, “7”, “10”, “13”,.

  The main control unit 50 takes into account a delay of ΔT of the generation timing of each image, and a pair of the hand image previously input from the HMD 11 and the ultrasonic image input from the image processing unit 54 after a delay of ΔT. That is, a pair of images having substantially the same shooting timing is stored in association with the frame memory 56. Similarly, the display control unit 55 displays the images with substantially the same shooting timing on the monitor 16 in parallel.

  Next, with reference to FIG. 7, a procedure for performing an ultrasonic diagnosis with the ultrasonic diagnostic apparatus 2 having the above-described configuration will be described. The operator Op operates the operation unit 17 to input information about the patient Pa, the date of the examination, and the like, and then inputs an operation input signal for instructing the start of the examination. Then, after mounting the HMD 11, the distal end portion 28 of the ultrasonic probe 10 is applied to the body surface of the patient Pa, and ultrasonic diagnosis is started.

  When receiving an operation input signal for starting inspection from the operation unit 17, the main control unit 50 drives and controls the transmission / reception unit 53 to cause the UT to transmit and receive ultrasonic waves and reflected waves. The detection signal output from the UT by this ultrasonic scanning is amplified and A / D converted by the transmission / reception unit 53, and then subjected to quadrature detection, reception focus processing, and the like by the image processing unit 54. Generated. The ultrasonic image generated by the image processing unit 54 is output to the main control unit 50 and the display control unit 55.

  Further, the HMD 11 is activated at the start of inspection, and the image light from the photographing lens 37 is picked up by the CCD 39. The imaging signal output from the CCD 39 is subjected to various processes by the AFE 42 and the DSP 43, thereby generating a hand image. The generated hand image is output to the main control unit 50 and the display control unit 55.

  The ultrasonic image and the hand image input to the main control unit 50 are temporarily stored in the frame memory 56 sequentially with a pair of substantially the same imaging timing associated therewith. Further, the display control unit 55 displays the ultrasonic image and the hand image in parallel on the monitor 16. Further, an ultrasonic image is sent from the display control unit 55 to the HMD 11 and displayed on the lens unit 35. The surgeon Op observes the state of the patient P through the lens unit 35 and observes an ultrasonic image displayed on the lens unit 35 (S10).

  The surgeon Op operates the freeze button 17a when a lesion candidate is displayed in the ultrasonic image (YES in S11). When the freeze button 17a is operated, the update of the frame memory 56 is temporarily stopped by the main control unit 50 (S12). Then, the display control unit 55 switches the display on the monitor 16 from a moving image to a still image obtained when the freeze button 17a is operated (S13). The surgeon Op operates the operation unit 17 to select an image to be stored in the storage device 18 while sending or returning a still image frame. Although illustration is omitted, when the freeze button 17a is operated again in the state of S13, the update pause of the frame memory 56 and the display of the candidate image are canceled, and the state returns to the state of S10.

  When an image to be stored in the storage device 18 is determined and the save button 17b is operated by the operator Op (YES in S14), the selected ultrasound image for a predetermined frame and substantially the same imaging timing associated therewith. Are output from the main control unit 50 to the storage device 18 and stored (S15). These series of processes are continued until an operation input signal instructing the end of the inspection is input from the operation unit 17 (YES in S16).

  As described above, since the video camera unit 38 incorporating the CCD 39 is mounted on the HMD 11, the ultrasonic probe 10 and the portion to which the ultrasonic probe 10 is applied, which is necessary for interpretation after inspection, etc. can be easily and reliably obtained. Can be taken. Since this place is a place where the surgeon Op gazes during the procedure, the hand is unlikely to become a blind spot as when a separate video camera is prepared. Therefore, the trouble of switching the video camera and moving the location of the video camera can be saved. Moreover, since the HMD 11 and the video camera unit 38 are integrated, it is easy to carry, and is suitable for use outside the hospital such as home-visit medical treatment or business trip examination.

  In accordance with the operation of the save button 17b, since the ultrasonic image and the hand image for a predetermined frame having substantially the same shooting timing are stored in the storage device 18, the storage device 18 is compared with the case where the moving image of the entire examination time is stored. There is no risk of squeezing the capacity. Each image stored in the storage device 18 can be used for reference at the time of interpretation after inspection and for ensuring reproducibility of the technique.

  An identification flag synchronized with the reception start time of the reflected wave of the UT and the charge accumulation start time of the CCD 39 is issued from the main control unit 50 to the image processing unit 54 and the DSP 43. You may associate both by adding to an image. Of course, when issuing the identification flag, the difference ΔT in the generation timing of each image is taken into consideration as in the above-described embodiment.

  In the above embodiment, the main control unit 50 stores the ultrasonic image and the hand image that are substantially the same in software shooting timing in association with each other. However, in addition to the main control unit 50, each image is stored in association with each other. Dedicated hardware may be provided.

  In the above embodiment, ultrasonic images and hand images for a plurality of frames are stored in the frame memory 56, and updating of the frame memory 56 is stopped when the freeze button 17a is operated, and each image for a plurality of frames is monitored. 16, the image to be displayed and stored is selected, but the present invention is not limited to this. The freeze button 17a and the frame memory 56 may be eliminated, or an ultrasonic image and a hand image for a predetermined frame may be directly stored in the storage device 18 by the operation of the save button 17b without going through the frame memory 56 by setting.

  In addition, although it has been described that a pair of an ultrasound image and a hand image corresponding to one frame having substantially the same shooting timing is stored, when the frame rate of the hand image is higher than that of the ultrasound image as in the above-described embodiment, Depending on the definition of “substantially the same timing”, there may be a plurality of hand images having substantially the same imaging timing with respect to one frame of ultrasonic image. For example, in FIG. 6, for the ultrasound image “3”, the hand image “7” and “5”, “6”, “8”, and “9” within 0.1 seconds before and after that are shown in the paragraph [0027]. ] Of “the shooting timing is substantially the same”. Therefore, all hand images that meet the definition, instead of one frame, may be stored in association with the ultrasound image.

  When selecting an image to be stored in the storage device 18, the candidate image may be displayed on the lens unit 35 of the HMD 11 instead of the monitor 16. In addition, the freeze button 17a, the save button 17b, and operation buttons for frame advancement and frame return of candidate images may be provided on the ultrasonic probe 10 instead of the operation unit 17. Selection of an image to be stored and operation of each button can be performed without moving the line of sight.

  Each member described in the above embodiment is an example, and can be changed in any way without departing from the gist of the present invention. For example, the ultrasonic probe may be a linear type instead of a convex type. Alternatively, the HMD is not in the form of glasses, but the lens part and the vine part are connected by a hinge that can move up and down, and the lens part can be moved between the position in front of the eye and the position in front of the forehead while the HMD is attached. It is good also as a structure. Further, the exchange of signals between the ultrasonic probe, the HMD and the processor may be made wireless.

2 Ultrasonic diagnostic equipment 10 Ultrasonic probe 11 Head mounted display (HMD)
12 Ultrasonic Observer 15 Processor Unit 16 Monitor 17 Operation Unit 17a Freeze Button 17b Save Button 18 Storage Device 27 Ultrasonic Transducer Array (UT Array)
38 Video camera section 39 CCD
50 Main control unit 55 Display control unit 56 Frame memory

Claims (2)

An ultrasonic probe including an ultrasonic transducer that transmits an ultrasonic wave to an observation site of a subject and receives a reflected wave from the observation site and outputs a detection signal corresponding to the reflected wave;
A head-mounted display attached to an operator and capable of observing the state of a subject and having a display unit capable of displaying an ultrasound image generated based on a detection signal, and the ultrasound probe for the subject A head-mounted display equipped with an imaging means for photographing a hand image reflecting the position and orientation of
Operation input means for instructing storage of ultrasonic images for a predetermined frame;
Storage means;
Ultrasound diagnosis, comprising: a storage control unit that associates an ultrasonic image and a hand image captured at substantially the same timing in accordance with an operation input signal from the operation input unit and stores the image in the storage unit apparatus.   The ultrasonic diagnostic apparatus according to claim 1, further comprising display control means for combining and displaying the ultrasonic image and the hand image on the display means.

Images