JP2017204162A - Display device and display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform processing according to a change of an indication body even when the indication body does not comprise a configuration for detecting the change of the indication body.SOLUTION: A projector 10 comprises: a detection section detecting a position of an indication body with respect to a display surface for displaying an image and contact of the indication body with the display surface; a drawing section generating a drawn image based on the position detected by the detection section; an imaging section outputting a picked-up image obtained by imaging a region including at least a part of the image displayed on the display surface; a processing section processing the drawn image according to a change of the indication body imaged in the picked-up image output from the imaging section when the detection section detects the contact of the indication body with the display surface; a superposition section generating an image obtained by superimposing the drawn image processed by the processing section on an external image supplied from an external device; and a display section displaying the image generated by the superposition section on the display surface.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display device and a display method.

  As an invention that draws a line according to the movement of the electronic pen and changes the display mode of the drawn line according to the writing pressure of the electronic pen, for example, there is an invention disclosed in Patent Document 1. In the present invention, the electronic pen detects the writing pressure with the strain sensor and outputs the detected writing pressure with the ultrasonic unit. The display control unit acquires the writing pressure output from the electronic pen, and displays a line having a thickness corresponding to the acquired writing pressure on the touch panel of the display device.

JP 2006-92410 A

  In the invention disclosed in Patent Document 1, the electronic pen includes components such as a strain sensor, an arithmetic circuit, and an ultrasonic unit, and an electronic circuit, and the configuration of the electronic pen is complicated.

  The present invention provides a technique that makes it possible to perform processing according to a change in the indicator even if the indicator does not have a configuration for detecting a change in the indicator.

The present invention provides a detection unit that detects a position of an indicator with respect to a display surface that displays an image and contact of the indicator with the display surface, and a drawing unit that generates a drawing image based on the position detected by the detection unit; An imaging unit that outputs a captured image obtained by imaging an area including at least a part of the image displayed on the display surface, and the detection unit detects contact of the indicator with the display surface. A processing unit that processes the drawing image in accordance with a change in the indicator reflected in the captured image output from the imaging unit, and a drawing image processed by the processing unit is supplied from an external device. A display device is provided that includes a superimposing unit that generates an image superimposed on the external image and a display unit that displays the image generated by the superimposing unit on the display surface.
According to the present invention, it is possible to perform processing according to a change in the indicator even if the indicator does not have a configuration for detecting a change in the indicator.

In this invention, the process which the said process part performs is good also as a structure which is a process which changes the aspect of the said drawing image.
According to this structure, the aspect of a drawing image can be changed according to the change of a pointer.

Moreover, in this invention, the said process part is good also as a structure which changes the aspect of the said drawing image according to the variation | change_quantity of the shape of the said indicator reflected in the said captured image output from the said imaging part.
According to this configuration, the change in the drawn image can be increased according to the change in the indicator.

Moreover, in this invention, the said process part is good also as a structure which changes the aspect of the said drawing image according to the variation | change_quantity of the inclination of the said indicator reflected in the said captured image output from the said imaging part.
According to this configuration, the change in the drawn image can be increased according to the change in the indicator.

Further, according to the present invention, a plurality of the imaging units may be provided, and the processing unit may be configured to process the drawn image according to a change in images output from the plurality of imaging units.
According to this configuration, it is possible to reliably detect a change in the indicator.

According to another aspect of the present invention, there is provided a detection step for detecting a position of an indicator on a display surface for displaying an image and a contact of the indicator with the display surface, and a drawing for generating a drawing image based on the position detected in the detection step. An image obtained by imaging an area including at least a part of the image displayed on the display surface when the detection step detects the step and the contact of the indicator with the display surface A processing step of processing the drawing image according to a change in the indicator reflected in an image; and a superimposing step of generating an image in which the drawing image processed in the processing step is superimposed on an external image supplied from an external device; And a display step of displaying the image generated in the superimposing step on the display surface.
According to the present invention, it is possible to perform processing according to a change in the indicator even if the indicator does not have a configuration for detecting a change in the indicator.

1 is a diagram showing an apparatus that constitutes a display system 1. FIG. The figure which showed the hardware constitutions of the projector 10 and the indicator 20. The functional block diagram of the control part 110 and the control part 210. FIG. The figure which showed an example of the time chart which detects a pointer. The figure which showed the state of the finger | toe 50 which touched the screen SC. The figure which showed an example of the image projected on the screen SC. The figure which showed the state of the finger | toe 50 which touched the screen SC. The figure which showed an example of the image projected on the screen SC. The figure which showed an example of the image projected on the screen SC.

[Embodiment]
(Configuration of the embodiment)
FIG. 1 is a diagram showing an apparatus constituting a display system 1 according to an embodiment of the present invention. The display system 1 includes a projector 10, an indicator 20, and a light emitting device 30 that project an image onto a screen SC serving as an image display surface.

  A projector 10 that is an example of a display device is connected to a PC (Personal Computer) 40 that is an example of an external device, and projects an image supplied from the PC 40 onto a screen SC. The projector 10 also has a drawing function for drawing an image at a position designated by the indicator 20 or the finger 50 and a PC operation function for using the indicator 20 or the finger 50 as a connected PC pointing device. .

  The projector 10 according to the present embodiment is installed obliquely above the screen SC and projects an image toward the screen SC. In the present embodiment, the projector 10 projects an image onto the screen SC, but may project an image onto a wall surface (display surface) instead of the screen SC. Further, in the present embodiment, the projector 10 is configured to be installed on the wall surface by a metal fitting, but may be installed on the ceiling.

  The pen-type indicator 20 functions as a pointing device when using the drawing function and the PC operation function described above, and is projected when the user operates the GUI (Graphical User Interface) of the PC that the projector 10 projects. This is used when the user draws on the image.

  The light emitting device 30 includes a light emitting unit that irradiates light (infrared light in the present embodiment) to a finger 50 (an example of an indicator) on the screen SC. The light emitting device 30 is installed above the upper end of the screen SC, and emits light by diffusing light downward in the range of the angle θ. The light emitted from the light emitting device 30 forms a light layer along the screen SC. In the present embodiment, the angle θ reaches approximately 180 degrees, and a light layer is formed on almost the entire screen SC. The surface of the screen SC and the light layer formed by the light emitting device 30 are preferably close to each other. The light layer has a thickness so that the finger 50 located away from the surface of the screen SC can also be irradiated. Moreover, you may irradiate the finger | toe 50 in the distant position by laminating | stacking a light emission part. The light emission from the light emitting device 30 is controlled by the projector 10.

  FIG. 2 is a diagram illustrating a hardware configuration of the projector 10 and the indicator 20. The indicator 20 includes a control unit 210, a communication unit 220, a light emitting unit 230, an operation unit 240, and a power source 250. The power source 250 is, for example, a dry battery or a secondary battery, and supplies power to the control unit 210, the communication unit 220, the light emitting unit 230, and the operation unit 240. The operation unit 240 includes a switch (not shown) that controls power supply from the power supply 250 to each unit. When the switch of the operation unit 240 is turned on, power is supplied from the power source 250 to each unit, and when the switch of the operation unit 240 is turned off, supply of power from the power source 250 to each unit is stopped. The light emitting unit 230 includes a light emitting diode that emits infrared light, and is provided at the tip of the indicator 20. Lighting and extinguishing of the light emitting unit 230 is controlled by the control unit 210. The light emitting unit 230 is a point light source, and light emitted from the light emitting unit 230 spreads on the spherical surface from the tip of the indicator 20. The communication unit 220 includes a light receiving element that receives infrared light. The communication unit 220 receives various signals transmitted by infrared light from the projector 10. The communication unit 220 converts various received signals into electric signals and supplies them to the control unit 210. The control unit 210 is connected to the light emitting unit 230 and the communication unit 220. The control unit 210 starts control of the light emitting unit 230 according to a signal supplied from the communication unit 220, and controls lighting and extinguishing of the light emitting diode of the light emitting unit 230.

  The projector 10 includes a control unit 110, a storage unit 120, an operation unit 130, and a projection unit 140. The projector 10 also includes a video processing unit 150, a video interface 160, a first imaging unit 171, a second imaging unit 172, and a communication unit 180. The control unit 110 is a microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). When the CPU executes a program stored in the ROM, in the projector 10, the control unit 110 controls each unit and projects an image on the screen SC, and a function that uses the finger 50 and the indicator 20 as a pointing device. A drawing function, a PC operation function, and the like are realized.

  The video interface 160 includes a plurality of connectors to which images are supplied, such as RCA, D-Sub, HDMI (registered trademark), USB (Universal Serial Bus), and the like. To supply. The video interface 160 may have a wireless communication interface such as a wireless LAN or Bluetooth (registered trademark), and may acquire an image by wireless communication.

  The storage unit 120 stores setting values related to the image quality of the projected image and information related to various functions. The operation unit 130 includes a plurality of buttons for operating the projector 10. The control unit 110 controls each unit in accordance with the operated button, thereby adjusting an image projected on the screen SC and setting various functions of the projector 10. The operation unit 130 also includes a light receiving unit (not shown) that receives an infrared light signal from a remote controller (not shown). The operation unit 130 converts a signal transmitted from the remote controller into an electric signal and supplies the electric signal to the control unit 110, and the control unit 110 controls each unit according to the supplied signal.

  The video processing unit 150 acquires an image supplied from the video interface 160. In addition, the video processing unit 150 acquires from the control unit 110 an on-screen image such as a GUI for operating the projector 10, a cursor indicating a position indicated by the indicator 20, and an image drawn by a drawing function. The video processing unit 150 includes a VRAM (Video RAM) 151 and has an area for developing an image supplied from the video interface 160 and an area for developing an on-screen image. Each image is developed in each area. To do. The video processing unit 150 has various image processing functions, performs image processing on the image developed in the VRAM 151, and adjusts the image quality of the projected image. In addition, when an on-screen image is supplied from the control unit 110, the video processing unit 150 supplies an image obtained by superimposing the on-screen image on the image (external image) supplied from the video interface 160 to the projection unit 140. . That is, the video processing unit 150 functions as a superimposing unit that superimposes an on-screen image on an external image supplied from an external device.

  The projection unit 140 that projects an image includes a light source 141, a light valve 142, a drive circuit 144, and a projection optical system 143. The projection unit 140 is an example of a display unit that displays an image. The light source 141 is a lamp that emits light, and the light emitted from the light source 141 is split into red, green, and blue light by a plurality of dichroic mirrors and mirrors (not shown), and each of the split red, green, and blue is split. Is guided to the light valve 142. The light source 141 may be a light emitting diode or a semiconductor laser device that emits laser light instead of a lamp.

  The drive circuit 144 acquires an image supplied from the video processing unit 150. The image supplied to the drive circuit 144 includes gradation data representing the gradation of the red component in the projected image, gradation data representing the gradation of the green component in the projected image, and the blue component in the projected image. It has gradation data representing gradation. The drive circuit 144 extracts gradation data of each color of red, green, and blue, and drives the light valve 142 based on the extracted gradation data of each color.

  The light valve 142 includes the above-described liquid crystal light valve on which red light is incident, the above-described liquid crystal light valve on which green light is incident, and the above-described liquid crystal light valve on which blue light is incident. The liquid crystal light valve is a transmissive liquid crystal panel and includes pixels arranged in a matrix with a plurality of rows and a plurality of columns. A liquid crystal light valve that receives red light is driven based on red gradation data, and a liquid crystal light valve that receives green light is driven based on green gradation data, and a liquid crystal light that receives blue light. The valve is driven based on the blue gradation data. In each liquid crystal light valve, each pixel is controlled by the drive circuit 144, and the transmittance of the pixel changes. By controlling the transmittance of the pixels, the light of each color transmitted through the liquid crystal light valve becomes an image corresponding to each gradation data. Red, green, and blue light images transmitted through the liquid crystal light valve are combined by a dichroic prism (not shown) and are incident on the projection optical system 143. The projection optical system 143 is an optical system that magnifies an incident image, and magnifies the incident image with a lens or a mirror and projects it onto the screen SC. When an image is projected on the screen SC, the image is displayed on the screen SC which is a display surface. Note that a reflective liquid crystal panel may be employed instead of the transmissive liquid crystal panel, or a digital mirror device or the like may be used.

  The first image capturing unit 171 and the second image capturing unit 172 generate an image in the captured range using a range including at least a part of the image projected on the screen SC by the projection unit 140, and output the generated image To do. The first imaging unit 171 is an imaging unit for specifying the positions of the indicator 20 and the finger 50. The first imaging unit 171 receives an infrared ray emitted from the light emitting unit 230, an infrared ray emitted from the light emitting device 30 and reflected by the finger 50 (such as a CMOS or CCD), and forms an image on the imaging element. An optical system for imaging, a diaphragm for limiting light incident on the image sensor, and the like are provided.

  The second imaging unit 172 is an imaging unit for detecting the deformation of the finger 50. The second imaging unit 172 includes an imaging device (such as CMOS or CCD) that receives visible light, an optical system that forms an image on the imaging device, and a diaphragm that restricts light incident on the imaging device. In the present embodiment, since the projector 10 is installed obliquely above the screen SC, the first imaging unit 171 and the second imaging unit 172 image a range including the screen SC from obliquely above.

  The communication unit 180 includes a light emitting diode that emits infrared light. The communication unit 180 controls the lighting and extinguishing of the light emitting diodes by the control unit 110, and transmits an infrared light signal for controlling the lighting and extinguishing of the light emitting unit 230. The communication unit 180 has a communication interface for communicating with a PC, and includes a communication interface such as a USB, a wired LAN, and a wireless LAN.

  FIG. 3 is a functional block diagram showing a configuration of functions realized by the control unit 110 executing a program and functions realized by the control unit 210. First, functions realized in the control unit 110 of the projector 10 will be described.

  The detection unit 113 shows the position of the light emitting unit 230 of the indicator 20 in the image projection area, the position of the finger 50 as an example of the indicator, and contact with the display surface of the indicator 20 or the finger 50, for example, as shown in FIG. Identifies periodically with a time chart. The period for specifying the position of the finger 50 and the position of the light emitting unit 230 has four phases from phase P11 to phase P14 as shown in FIG. When detecting the position of the finger 50, the position of the light emitting unit 230, and the contact of the finger 50 or the light emitting unit 230 with the projection surface, the phase P11 to the phase P14 are repeated. Phase P11 is a phase for synchronizing the timing at which the projector 10 performs imaging with the first imaging unit 171 with the timing at which the light emitting unit 230 of the indicator 20 emits light and the timing at which the light emitting device 30 emits infrared light. . In phase P11, the detection unit 113 controls the communication unit 180 so that an infrared light synchronization signal is output in a predetermined period te1.

  In the indicator 20, the control unit 210 emits light so that the light emitting unit 230 is turned on in a preset period te <b> 2 when a predetermined time has elapsed since the communication unit 220 received the synchronization signal and received the synchronization signal. The unit 230 is controlled. In the present embodiment, the light emitting unit 230 is controlled to light up from the start time of the phase P12, the phase P13, and the phase P14. In addition, the detection unit 113 controls the light emitting device 30 so that the light emitting device 30 emits infrared light in the period te2 from the start time of the phase P12 and the phase P14.

  In phase P12 to phase P14, the detection unit 113 controls the first imaging unit 171 to image a predetermined range including the screen SC at the set shutter speed. The exposure period in which exposure is performed by the electronic shutter function in the first imaging unit 171 starts from the start time of the phase P12 and the phase P14, and the time point when the exposure ends is determined by the set shutter speed. The image captured by the first imaging unit 171 during the exposure period of phase P12 to phase P14 is supplied to the detection unit 113.

  The detection unit 113 detects the position of the finger 50 and the light emitting unit 230 on the projected image and the contact of the light emitting unit 230 and the finger 50 with the projection surface using the image supplied from the first imaging unit 171. . Specifically, in the phase P12 and the phase P14, when the infrared light emitted from the light emitting device 30 is irradiated on the finger 50, the image obtained by the first imaging unit 171 is emitted from the light emitting device 30. The infrared light reflected by the finger 50 is reflected. Further, in phase P12 and phase P14, when the light emitting unit 230 is within the imaging range of the first imaging unit 171, the infrared light emitted by the light emitting unit 230 is also reflected in the image obtained by the first imaging unit 171. In the phase P13, since the light emitting device 30 does not emit light, the infrared light emitted from the light emitting unit 230 is reflected in the image obtained by the first imaging unit 171.

  In phase P12 to phase P14, the detection unit 113 specifies the position of the infrared light reflected in the image obtained by the first imaging unit 171 and the distance to the screen SC. The detection unit 113 identifies infrared light at a position close to the position of the infrared light whose position is identified in phase P13 among the infrared light whose positions are identified in phase P12 and phase P14, and The position is the position of the light emitting unit 230. The detection unit 113 sets the position of the finger 50 as the position of the infrared light far from the infrared light whose position is specified in the phase P13 among the infrared light whose positions are specified in the phases P12 and P14. Note that when the infrared light is not within the imaging range in the phase P13, the detection unit 113 sets the position specified in the phase P12 and the phase P14 as the position of the finger 50. These specified positions are used when various functions such as a drawing function and a PC operation function are executed.

  The drawing unit 112 generates an on-screen image of a drawing image to be projected according to the position detected by the detection unit 113. The processing unit 114 processes the drawn image generated by the drawing unit 112 in accordance with a change in the shape of the finger 50 shown in the image supplied from the second imaging unit 172. In the present embodiment, the process performed by the drawing unit 112 is a process of changing the line thickness (mode) of the drawn image.

  Next, functions realized in the control unit 210 of the indicator 20 will be described. The signal acquisition unit 211 acquires the synchronization signal received by the communication unit 220. The light emission control unit 212 acquires the synchronization signal from the signal acquisition unit 211, and when a predetermined time has elapsed since the acquisition of the synchronization signal, the light emission unit 230 is turned on in the period te2 in phase P12 to phase P14. 230 is controlled.

(Operation example of embodiment)
Next, an operation example of the present embodiment when the drawing function is on will be described. When an image is supplied from the PC 40, the video interface 160 acquires the image supplied from the PC 40 and supplies the acquired image to the video processing unit 150. When the image is supplied to the video processing unit 150, the supplied image is projected onto the screen SC.

  When drawing a projected image with the drawing function, the user brings the finger 50 close to a position on the screen SC where the drawing is performed. The second imaging unit 172 images the finger 50 approaching the screen SC and outputs an image in which the finger 50 is reflected. When the finger 50 contacts the screen SC, the infrared light emitted from the light emitting device 30 is reflected by the finger 50. The first imaging unit 171 images the infrared light emitted from the finger 50 and outputs an image in which the infrared light is reflected.

  The control unit 110 (detection unit 113) analyzes the image supplied from the first imaging unit 171 and identifies the position of the finger 50 based on the position of infrared light in the image. Further, the control unit 110 (processing unit 114) analyzes the image supplied from the second imaging unit 172 when infrared light is reflected in the image supplied from the first imaging unit 171, and the finger 50 moves the screen SC. The shape of the finger 50 when touching is obtained.

  FIG. 5A is a diagram illustrating an example of the finger 50 shown in the image (an example of the captured image) supplied by the second imaging unit 172, and FIG. 5B is a diagram when the screen SC is in contact with the screen SC. It is the figure which looked at the finger | toe 50 from the side of the screen SC. As shown in FIGS. 5A and 5B, when the finger 50 is in contact with the screen SC, the finger 50 maintains the shape before contacting the screen SC. The control unit 110 (processing unit 114) stores the shape of the finger 50 shown in FIG. 5A in the RAM.

  Next, the control unit 110 (drawing unit 112) generates an on-screen image according to the detected position of the indicator, and supplies the generated on-screen image to the video processing unit 150. When the on-screen image is supplied to the video processing unit 150, the on-screen image is superimposed on the image supplied from the PC 40 and projected. For example, when the user moves the finger 50 on the screen SC while maintaining the state of the finger 50 shown in FIG. 5, the thickness is a predetermined thickness as illustrated in FIG. An on-screen image of a line corresponding to 50 movement trajectories (hereinafter referred to as a drawing image G11 for convenience of explanation) is projected.

  Next, when the user increases the force with which the finger 50 is pressed against the screen SC, the finger 50 is deformed as shown in FIGS. 7A and 7B. FIG. 7A is a diagram illustrating an example of the finger 50 shown in the image supplied by the second imaging unit 172, and FIG. 7B is a diagram illustrating the finger 50 when touching the screen SC. It is the figure seen from the side.

  The control unit 110 (processing unit 114) analyzes the image supplied from the second imaging unit 172 and acquires the shape of the finger 50. Next, the control unit 110 compares the newly acquired shape of the finger 50 with the shape of the finger 50 stored in the RAM, and specifies the deformation amount of the finger 50. The control unit 110 determines the deformation amount of the finger 50, for example, based on the angle formed by the terminal and middle clauses of the finger 50 in the state shown in FIG. The difference between the angle formed by the last node and the middle node of the finger 50 in the state is defined as the deformation amount.

  Next, the control unit 110 (the processing unit 114) sets the thickness (aspect) of the line image to be drawn according to the movement of the finger 50 to a thickness (aspect) corresponding to the specified deformation amount, and detects the detected indicator. An on-screen image corresponding to the position is generated, and processing for supplying the generated on-screen image to the video processing unit 150 is performed. Here, in the present embodiment, the control unit 110 increases the thickness as the deformation amount specifying the thickness of the line to be drawn increases.

  When the on-screen image is supplied to the video processing unit 150, the on-screen image is superimposed on the image supplied from the PC 40 and projected. For example, when the user moves the finger 50 on the screen SC while maintaining the state of the finger 50 shown in FIG. 7 from the end of the drawn image G11, as illustrated in FIG. 8, it is thicker than the drawn image G11. A line on-screen image (hereinafter referred to as a drawn image G12 for convenience of description) is projected.

  When the thickness of the line to be drawn is reduced, the user weakens the force of pressing the finger 50 against the screen SC and sets the state of the finger 50 to the state shown in FIG. The controller 110 analyzes the image supplied from the second imaging unit 172 and acquires the shape of the finger 50. Next, the control unit 110 compares the newly acquired shape of the finger 50 with the shape of the finger 50 stored in the RAM, and specifies the deformation amount of the finger 50. The control unit 110 generates the on-screen image corresponding to the detected position of the indicator by setting the thickness of the line image to be drawn according to the movement of the finger 50 to the thickness corresponding to the specified deformation amount. The on-screen image is supplied to the video processing unit 150. Here, if the newly acquired shape of the finger 50 and the shape of the finger 50 stored in the RAM are the same, the control unit 110 determines the thickness of the line to be drawn when the finger 50 touches the screen SC. Return to the size.

  When the on-screen image is supplied to the video processing unit 150, the on-screen image is superimposed on the image supplied from the PC 40 and projected. For example, when the user moves the finger 50 on the screen SC while maintaining the state of the finger 50 shown in FIG. 5 from the end of the drawing image G12, as illustrated in FIG. 9, the same as the drawing image G11. An on-screen image having a thickness line (hereinafter referred to as a drawn image G13 for convenience of explanation) is projected.

  As described above, according to the present embodiment, it is possible to perform drawing according to a change in the state of the indicator without providing an electronic circuit or a component on the indicator.

[Modification]
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. For example, the present invention may be implemented by modifying the above-described embodiment as follows. In addition, you may implement each embodiment mentioned above and the following modifications, combining one or more suitably.

  In the embodiment described above, the thickness of the line to be drawn is changed according to the change in the angle between the last clause and the middle clause of the finger 50. However, the configuration for changing the thickness of the line to be drawn is described in the embodiment. It is not limited to the configuration of For example, the control unit 110 detects an angle formed by the finger 50 and the normal line of the screen SC from the image supplied from the second imaging unit 172, and changes the thickness of the line to be drawn according to the detected angle. It may be. This detected angle is also an example of the amount of change according to the present invention. In the case of this configuration, the control unit 110 may increase the thickness as the angle at which the thickness of the line to be drawn is detected increases. Further, in the case of detecting the angle formed by the finger 50 and the normal line of the screen SC, the difference between the angle when the finger 50 is in contact and the angle based on the thickness of the line to be drawn becomes large. It is good also as a structure thickened as it goes.

  In the above-described operation example, the thickness of the line to be drawn is changed according to the angle formed between the last clause and the middle clause when the last clause of the finger 50 is bent toward the back of the hand. May be changed according to the angle formed between the last clause and the middle clause when the last clause of the finger 50 is bent to the palm side.

  In the embodiment described above, the control unit 110 identifies the position and deformation amount of the finger 50 and draws a line according to the identified position and deformation amount, but is limited to the configuration of the embodiment. is not. For example, the control unit 110 analyzes the image supplied from the second imaging unit 172 and identifies the position and the deformation amount of the indicator that is an elastic body and reflects the infrared light in a pen shape. The line may be drawn according to the specified position and deformation amount. In the case of this configuration, the control unit 110 may increase the thickness of the line to be drawn according to the degree of bending of the indicator and the angle between the indicator and the normal line of the screen SC.

  In the present invention, the casing of the indicator 20 may be formed of an elastic member. In the case of this configuration, the control unit 110 analyzes the image supplied from the second imaging unit 172 and draws according to the degree of bending of the housing and the angle formed by the indicator 20 and the normal line of the screen SC. You may make it thicken the line to do.

In the embodiment and the modification described above, the control unit 110 performs a process of changing the thickness of the line to be drawn according to the deformation amount of the finger 50 and the angle formed by the finger 50 and the screen SC. The processing performed according to the amount of deformation of 50 and the angle formed by the finger 50 and the screen SC is not limited to the processing of changing the thickness of the line to be drawn. For example, the control unit 110 may change the density of the line to be drawn according to the deformation amount of the finger 50 and the angle formed by the finger 50 and the normal line of the screen SC. In the case of this configuration, the control unit 110 may be configured to increase the density of the color of the line to be drawn as the deformation amount of the finger 50 increases. Further, the control unit 110 may be configured to increase the color depth of the line to be drawn as the angle formed by the finger 50 and the normal line of the screen SC increases. The control unit 110 may change the color of the line to be drawn according to the deformation amount of the finger 50 and the angle formed by the finger 50 and the normal line of the screen SC.
For example, when the deformation amount of the finger 50 is less than a predetermined threshold value, the control unit 110 sets the color of the line to be drawn to black, and when the deformation amount of the finger 50 is equal to or greater than the predetermined threshold value. The line to be drawn may be red. In this configuration, the user may be able to set the line color when the deformation amount is less than the threshold and the line color when the deformation amount is greater than or equal to the threshold.

  In the present invention, the projector 10 may include a plurality of first imaging units 171 and specify the positions of the indicator 20 and the finger 50 using images obtained from the plurality of first imaging units 171. In addition, the projector 10 may include a plurality of second imaging units 172, and may specify a deformation amount of the indicator 20 or the finger 50 using images obtained from the plurality of second imaging units 172.

  In the embodiment described above, the projector 10 includes the second imaging unit 172, but is not limited to this configuration. For example, the second imaging unit 172 may be separated from the projector 10 and arranged on the side of the screen SC.

  In the embodiment described above, the control unit 110 changes the thickness of the line to be drawn according to the amount of deformation of the finger 50 from when the finger 50 contacts the screen SC. It is not limited. For example, when the change amount of the specified finger 50 is less than a predetermined threshold value, the control unit 110 does not change the thickness of the line, and the change amount of the finger 50 is equal to or greater than the predetermined threshold value. You may make it change the thickness of a line according to change amount.

  In the present invention, the control unit 110 may set the thickness of the line to be drawn to the first thickness when the amount of change of the finger 50 is within a predetermined first range. In this configuration, when the amount of change exceeds the first range, the thickness of the line to be drawn is made thicker than the first thickness, and the amount of change is a value below the first range. In such a case, the line to be drawn may be thinner than the first thickness.

In the present invention, the process of changing the thickness of the line to be drawn is executed according to the change amount of the finger 50, but the process executed according to the change amount of the finger 50 changes the thickness of the line to be drawn. It is not limited to processing. For example, when the PC operation function is turned on in the projector 10, the projection surface functions as a touch panel and the PC 40 can be operated with the finger 50.
When the PC operation function is on, the object is dragged when the finger 50 touches the position of the object, but the object selection range may be changed according to the amount of change of the finger 50. According to this configuration, it is possible to select a plurality of objects according to the amount of change of the finger 50 and move the plurality of objects.

DESCRIPTION OF SYMBOLS 1 ... Display system, 10 ... Projector, 20 ... Indicator, 30 ... Light-emitting device, 40 ... PC, 50 ... Finger, 110 ... Control part, 112 ... Drawing part, 113 ... Detection part, 114 ... Processing part, 120 ... Memory | storage , 130: operation unit, 140 ... projection unit, 150 ... video processing unit, 160 ... video interface, 171 ... first imaging unit, 172 ... second imaging unit, 180 ... communication unit, 210 ... control unit, 211 ... signal Acquisition unit, 212 ... light emission control unit, 220 ... communication unit, 230 ... light emission unit, 240 ... operation unit, 250 ... power source.

Claims (6)

A detection unit for detecting the position of the indicator with respect to the display surface displaying the image and the contact of the indicator with the display surface;
A drawing unit that generates a drawing image based on the position detected by the detection unit;
An imaging unit that outputs a captured image obtained by imaging an area including at least a part of the image displayed on the display surface;
A processing unit that processes the drawn image in accordance with a change in the indicator reflected in the captured image output from the imaging unit when the detection unit detects contact of the indicator with the display surface; ,
A superimposing unit that generates an image in which the rendered image processed by the processing unit is superimposed on an external image supplied from an external device;
A display unit configured to display an image generated by the superimposing unit on the display surface. The display device according to claim 1, wherein the process performed by the processing unit is a process of changing an aspect of the drawn image. The display device according to claim 2, wherein the processing unit changes a form of the drawn image in accordance with a change amount of a shape of the indicator reflected in the captured image output from the imaging unit. The display device according to claim 2, wherein the processing unit changes a form of the drawn image in accordance with a change amount of an inclination of the indicator reflected in the captured image output from the imaging unit. 5. The imaging unit according to claim 1, wherein the imaging unit includes a plurality of the imaging units, and the processing unit processes the drawn image according to a change in the captured image output from the plurality of imaging units. Display device. A detection step of detecting a position of the indicator with respect to a display surface for displaying an image and a contact of the indicator with the display surface;
A drawing step for generating a drawing image based on the position detected in the detection step;
When the contact of the indicator on the display surface is detected in the detection step, it appears in a captured image obtained by capturing an area including at least a part of the image displayed on the display surface. Processing steps for processing the drawn image in response to changes in the indicator;
A superimposing step for generating an image in which the rendered image processed in the processing step is superimposed on an external image supplied from an external device;
A display step of displaying the image generated in the superimposing step on the display surface.

Images