JP2011118330A - Image display device with imaging unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display device with an imaging unit, which can be manufactured at low cost and can converge light of sufficient light quantity to an imaging unit, wherein an image of a user opposed to an image display part can be obtained easily. <P>SOLUTION: An image display device with an imaging unit has: an image display unit 10 in which a plurality of pixels including light-emitting elements are arranged; a light transmissive sections 30 provided respectively to the plurality of pixels; an imaging unit 20 arranged at the back side of the image display unit 10; and a converging means 21 converging light passed through the plurality of light transmissive sections 30 to the imaging unit 20. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image display device with an imaging device, and more specifically to an image display device in which an imaging device is arranged on the back side of an image display unit.

  In recent years, as a flat panel display device (FP display device), an organic electroluminescence display device (hereinafter, sometimes simply referred to as “organic EL display device”) has been attracting attention. At present, a liquid crystal display (LCD) dominates as an FP display, but a member such as a backlight or a polarizing plate is required instead of a self-luminous device. Therefore, there are problems such as an increase in the thickness of the FP display device and insufficient luminance. On the other hand, the organic EL display device is a self-luminous device, and does not require a member such as a backlight in principle, and has many advantages over an LCD, such as being thin and having high luminance. In particular, an active matrix organic EL display device in which a switching element is arranged in each pixel can keep current consumption low by holding each pixel in a hold-on state, and it is relatively easy to enlarge the screen and increase the definition. Because it can be done, each company is developing and is expected to become the mainstream of next-generation FP display devices.

  In recent years, at least two sets of an image display device and an imaging device are used, and these are connected via a communication line or a network, so that a tele-telephone conference system (see FIG. Videophone devices) are beginning to spread. Then, by using a low-cost imaging device called a web camera, a personal computer, and software, a TV phone function can be easily realized. By the way, in this videophone system, the user's face on one side is photographed by the imaging device, the obtained image is displayed on the image display device on the other side, and the user's face on the other side is displayed on the imaging device. Then, the obtained image is displayed on the image display device on one side to have a conversation. On either side, the user talks while looking at the face of the other party displayed on his / her image display device, so the line of sight and face are directed to his / her image display device. In the conventional videophone system, since the imaging device is placed outside the display area of the image display device, the line of sight is not suitable for the imaging device. Therefore, the face of the user on the other side projected on the image display device does not look at him and feels a sense of incongruity. This uncomfortable feeling of not looking directly at the other party is one of the factors delaying the spread of TV phones.

  There have been many attempts to add functions other than image display to image display devices. For example, Japanese Patent Application Laid-Open No. 2005-176151 and Japanese Patent Application Laid-Open No. 2005-010407 disclose FP display devices with an imaging device.

JP 2005-176151 A JP2005-010407

  In the technique disclosed in Japanese Patent Application Laid-Open No. 2005-176151, a plurality of openings having minute lenses are provided between pixels constituting an image display unit of an image display device, and light passing through the plurality of openings is transmitted to a plurality of cameras. Take an image with. Here, the face of the user who looks at the image display device is imaged from a plurality of different angles, and the obtained plurality of images are processed to generate an image capturing the user from the front. However, in the technique disclosed in this patent publication, it is necessary to provide a microlens at the opening, and a high-precision microlens is required to accurately connect an image to the imaging device. Therefore, the manufacturing cost of the image display device is increased. In addition, the front face of the user is not imaged, but a front image is created from a plurality of images taken from different angles. The feeling of strangeness is great.

  In the technique disclosed in Japanese Patent Application Laid-Open No. 2005-010407, for example, as shown in FIGS. 15 and 16, an imaging apparatus based on light that has passed through one light transmission portion provided in a plurality of pixels. Therefore, it is difficult to collect a sufficient amount of light on the imaging device.

  Therefore, an object of the present invention is to manufacture at a low cost, to collect a sufficient amount of light on the imaging device, and to easily acquire an image of a user facing the image display unit. An object is to provide an image display device with an imaging device.

In order to achieve the above object, an image display device with an imaging device according to the first aspect of the present invention comprises:
An image display unit comprising a plurality of pixels including light emitting elements,
A light transmitting portion provided in each of the plurality of pixels;
An imaging device disposed on the back side of the image display unit, and
Condensing means for condensing the light that has passed through the plurality of light transmitting portions on the imaging device;
Have

An image display device with an imaging device according to the second aspect of the present invention for achieving the above-described object,
An image display unit comprising a plurality of pixels including light emitting elements,
A light transmission portion provided around at least one or more (preferably at least two) pixels;
An imaging device disposed on the back side of the image display unit, and
A condensing means for condensing the light that has passed through the light transmitting portion onto the imaging device;
Have The light transmitting portion may be provided around the entire pixel, or may be provided on a part of the periphery of the pixel (specifically, on two or more consecutive sides of the side corresponding to the boundary of the pixel). However, in the latter case, it is longer than 1/4 times the entire circumference of the pixel (in the case of two consecutive sides, it is more than 1/2 times the length of each side). It is preferable that a light transmission part is provided.

  In the image display device with an imaging device according to the first aspect or the second aspect of the present invention, the light that has passed through the light transmitting portions (the plurality of light transmitting portions) provided in each of the plurality of pixels is imaged. Light collected by the device or passed through a light transmission portion provided around at least one pixel is collected by the imaging device. Therefore, a high-precision microlens is not required to accurately connect an image to the imaging device, and the manufacturing cost of the image display device with the imaging device is not increased, and a sufficient amount of light is condensed on the imaging device. Can be made. And since the imaging device is arrange | positioned at the back side of an image display part, the user's face, eyes, operation | movement, etc. which are facing the display can be accurately imaged with an imaging device.

FIGS. 1A and 1B are schematic views of the image display device with an imaging device according to the first embodiment as viewed from the front and side, respectively, and FIG. 1C constitutes an image display unit. It is a figure which shows arrangement | positioning of a some pixel typically. FIG. 2 is a schematic partial cross-sectional view of the image display device with an imaging device of the embodiment. FIGS. 3A and 3B are conceptual diagrams of the image display device with an imaging device of the first embodiment and the image display device in which the imaging device is fixed to the outside of the image display unit. FIG. 4 is a schematic diagram for explaining a diffraction phenomenon caused by a slit. 5A and 5B respectively show an image obtained by photographing a transparent glass plate placed in front of the imaging device, and a certain shape, size, and distribution in front of the imaging device. It is the image obtained by arrange | positioning and image | photographing the transparent glass plate which provided the light transmissive part which has. FIG. 6 is a block diagram of an image display apparatus with an imaging apparatus according to the second embodiment. FIGS. 7A and 7B are diagrams schematically illustrating the shape of the light transmission portion in the second embodiment. FIG. 8 is a diagram schematically illustrating another shape of the light transmission portion in the second embodiment. FIG. 9 is a block diagram of an image display device with an imaging device according to the third embodiment. FIGS. 10A and 10B are diagrams schematically illustrating an arrangement of a plurality of pixels constituting an image display unit in the image display device with an imaging device according to the sixth embodiment. FIG. 11 is a diagram schematically illustrating an arrangement of a plurality of pixels constituting the image display unit in the image display device with an imaging device according to the sixth embodiment. FIGS. 12A and 12B are diagrams schematically illustrating an arrangement of a plurality of pixels constituting the image display unit in the image display device with an imaging device according to the seventh embodiment. FIGS. 13A and 13B are diagrams schematically illustrating the shape of the light transmission portion in the eighth embodiment. 14A and 14B are schematic diagrams of a notebook personal computer and a mobile phone, respectively.

Hereinafter, the present invention will be described based on examples with reference to the drawings. However, the present invention is not limited to the examples, and various numerical values and materials in the examples are examples. The description will be given in the following order.
1. 1. General description of an image display device with an imaging device according to the first and second aspects of the present invention. Example 1 (image display device with an imaging device according to the first aspect of the present invention)
3. Example 2 (modification of Example 1, first configuration of the present invention)
4). Example 3 (Modification of Example 2, Second Configuration of the Present Invention)
5). Example 4 (Modification of Example 1, Third Configuration of the Present Invention)
6). Example 5 (Modification of Example 1, Fourth Configuration of the Present Invention)
7). Example 6 (Modification of Example 1)
8). Example 7 (Modification of Example 1)
9. Example 8 (Image display device with imaging device according to second aspect of the present invention, etc.)

[Description of General Image Display Device with Imaging Device According to First and Second Aspects of the Present Invention]
The image display device with an imaging device according to the first aspect or the second aspect of the present invention (hereinafter, these may be collectively referred to simply as “image display device with an imaging device of the present invention”). In other words, when the light transmitting portion is very small, a diffraction phenomenon occurs in the light transmitting portion, resulting in blurring of the image formed on the imaging apparatus or lack of clarity. In such a case, it is preferable to further include a diffraction correction unit that corrects the diffraction that occurs in the light transmission unit with respect to the image information acquired through the imaging device. Such a configuration is referred to as a “first configuration of the present invention” for convenience.

In the first configuration of the present invention, a part or all of the light transmission part is periodically provided along the first direction and the second direction of the image display part,
When the length of the light transmission part along the first direction is L _tr-1 and the pitch of the pixels along the first direction is P _px-1 , the line aperture ratio L _{tr-1 in} the first direction / P _px-1 is
L _tr-1 / P _px-1 ≧ 0.5
Preferably, L _tr-1 / P _px-1 ≧ 0.8
Is preferably satisfied. Furthermore, when the length of the light transmission part along the second direction is L _tr-2 , and the pixel pitch along the second direction is P _px-2 , the line aperture ratio L in the second direction _tr-2 / P _px-2 is
L _tr-2 / P _px-2 ≧ 0.5
Preferably, L _tr-2 / P _px-2 ≧ 0.8
Is preferably satisfied. Note that the first direction and the second direction may be orthogonal to each other, or in some cases, may intersect at an angle other than 90 degrees. In the latter case, part or all of the light transmission part is periodically provided not only in the first direction and the second direction of the image display part but also in the third direction, the fourth direction,. In such a case, the length of the light transmission portion along at least two directions in each direction and the pitch of the pixels along these at least two directions are related to each other ( Specifically, it is preferable that 0.5 times or more is satisfied. There are no particular restrictions on the upper limit values of the line aperture ratios L _tr-1 / P _px-1 and L _tr-2 / P _px-2 as long as a light transmission part can be formed. Here, the length L _tr-1 of the light transmission part along the first direction is the length per one cycle of the line segment corresponding to the shape when the light transmission part is projected in the first direction. , And the pixel pitch P _px-1 along the first direction means the length per one period of the pixels along the first direction. Similarly, the length L _tr-2 of the light transmission part along the second direction is the length per cycle of the line segment corresponding to the shape when the light transmission part is projected in the second direction. The pixel pitch P _px-2 along the second direction means the length per one period of the pixels along the second direction.

  In the first configuration of the present invention, specifically, the inverse of MTF (Moduration Transfer Function) calculated based on the shape, size, and distribution of the light transmission part (and further based on the wavelength of external light in some cases). The conversion process is preferably performed on the image information. The shape, size, and distribution of the light transmission part may be stored in advance in the diffraction correction unit. The diffraction correction unit can be, for example, a circuit including a CPU having an input / output unit and a memory. In some cases, the diffraction correction unit is configured from a personal computer provided in an image display device with an imaging device. You can also. By considering the wavelength of external light, an optimal image can be obtained regardless of external light (external illumination environment).

  The image display device with an imaging device of the present invention including the above-described preferable configuration may further include a wavelength distribution measuring unit that measures the wavelength distribution of external light. Such a configuration is referred to as a “second configuration of the present invention” for convenience. By adopting such a configuration, it is possible to improve the accuracy of the image information acquired via the imaging device (for example, improve the accuracy of the color information), and to improve the accuracy of the MTF inverse conversion process. Can do. The wavelength distribution measuring means can be composed of a light receiving device such as a photosensor, for example. The control of the wavelength distribution measuring means may be performed by the image display device with an imaging device, or depending on the case, it may be performed by a personal computer provided in the image display device with the imaging device.

Furthermore, in the image display device with an imaging device of the present invention including the first configuration of the present invention, the second configuration of the present invention, and the preferred configuration described above,
Information sending means for sending image information acquired via the imaging device; and
Display means for displaying an image based on image information input from the outside on the image display unit;
Is further provided,
Send image information acquired via the imaging device to the outside by an information sending means,
An image based on image information input from the outside can be displayed on the image display unit by the display means. Such a configuration is referred to as a “third configuration of the present invention” for convenience. By connecting a plurality (two or more) of such image display devices with an imaging device via a communication line, a network, or the like, a so-called video conference call system (video phone device) can be constructed.

  Here, the information transmission means for transmitting the image information acquired via the imaging device and the display means for displaying an image based on the image information input from the outside on the image display unit are, for example, a television telephone conference system. A well-known means used in (Videophone device) may be used. It is preferable to arrange operation switches, buttons, a keyboard, and the like on the image display device with an imaging device (or image display unit). In some cases, the information sending means and the display means can be configured from a personal computer provided in the image display device with an imaging device. In this case, the personal computer may be connected to a communication line, a network, or the like.

Alternatively, in the image display device with an imaging device of the present invention including the first configuration of the present invention, the second configuration of the present invention, and the preferred configuration described above,
Storage means for storing image information acquired via the imaging device; and
Display means for displaying on the image display unit an image based on the image information acquired via the imaging device and the image information stored in the storage means;
Is further provided,
An image based on the image information acquired via the imaging device is displayed on the image display unit by the display unit, and an image based on the image information stored in the storage unit is displayed on the image display unit. Can do. Such a configuration is referred to as a “fourth configuration of the present invention” for convenience. Such an image display device with an imaging device functions as a so-called digital mirror, and compares, for example, a user's image captured in the past, a current user's image, and an image (the time difference when the image is captured). It is possible to compare two or more images and images.

  Here, the storage means for storing the image information acquired via the imaging device can be composed of, for example, a well-known non-volatile memory, a hard disk drive device, and a well-known image information processing circuit. The display means for displaying on the section can be constituted by, for example, a known image information display circuit. It is preferable to arrange operation switches, buttons, a keyboard, and the like on the image display device with an imaging device (or image display unit). In some cases, the storage means and the display means can be configured from a personal computer provided in the image display device with an imaging device.

Furthermore, in the image display device with an imaging device according to the first aspect of the present invention including the first configuration of the present invention, the second configuration of the present invention, and the preferred configuration described above,
[Case A] A mode in which the sizes of the plurality of light transmission portions are random (specifically, each of at least two light transmission portions adjacent to one light transmission portion has the size of this one light transmission portion. Different form)
It can be. Alternatively,
[Case B] Form in which the shapes of the plurality of light transmission portions are random (specifically, each of at least two light transmission portions adjacent to one light transmission portion is different from the shape of this one light transmission portion. Form)
It can be. Alternatively,
[Case C] Form in which the arrangement pitch of a plurality of light transmission parts is random (specifically, each of at least two light transmission parts adjacent to one light transmission part is arranged with this one light transmission part. Different form from the pitch)
It can be.

  Here, [Case A] may be adopted alone, [Case B] may be adopted alone, [Case C] may be adopted alone, or [Case A] [Case B] may be used in combination, [Case A] and [Case C] may be used in combination, or [Case B] and [Case C] may be used in combination. However, [Case A], [Case B], and [Case C] may be used in combination. The minimum value and the minimum shape of the light transmission part depend on the minimum processing dimension (for example, 0.5 μm) in the photolithography technique and the etching technique for providing the light transmission part.

  Although the sizes of the plurality of light transmission parts are random, specifically, at least two light transmission parts (for convenience, for convenience) adjacent to one light transmission part (referred to as “light transmission part A” for convenience). "Referred to as" light transmission part B, light transmission part C "), preferably two light transmission parts arranged in the horizontal direction, more preferably three light transmission parts adjacent to the light transmission part A (for convenience," Light transmission part B, light transmission part C, and light transmission part D ”, and more preferably four light transmission parts adjacent to the light transmission part A (for convenience,“ light transmission part B, light transmission part C, It is desirable to vary the sizes of the light transmission part D and the light transmission part E ”. That is, the sizes of the light transmission part A and the light transmission part B are made different, the sizes of the light transmission part A and the light transmission part C are made different, and the sizes of the light transmission part A and the light transmission part D are different. It is desirable that the light transmitting portion A and the light transmitting portion E are different in size. Similarly, the shapes of the plurality of light transmission parts are random. Specifically, at least two light transmission parts (light transmission part B, light transmission part) adjacent to one light transmission part (light transmission part A) are used. C), preferably two light transmission parts arranged in the horizontal direction, more preferably three light transmission parts adjacent to the light transmission part A (light transmission part B, light transmission part C, light transmission part D) More preferably, it is desirable to make the shapes of the four light transmission parts (light transmission part B, light transmission part C, light transmission part D, and light transmission part E) adjacent to the light transmission part A different. That is, the shapes of the light transmitting part A and the light transmitting part B are made different, the shapes of the light transmitting part A and the light transmitting part C are made different, and the shapes of the light transmitting part A and the light transmitting part D are made different. In addition, it is desirable that the shapes of the light transmitting portion A and the light transmitting portion E are different. Similarly, although the arrangement pitch of the plurality of light transmission parts is random, specifically, at least two light transmission parts (light transmission part B, light transmission adjacent to one light transmission part (light transmission part A)). Part C), preferably two light transmission parts arranged in the horizontal direction, more preferably three light transmission parts adjacent to the light transmission part A (light transmission part B, light transmission part C, light transmission part D) More preferably, it is desirable that the arrangement pitches of the four light transmission parts (light transmission part B, light transmission part C, light transmission part D, and light transmission part E) adjacent to the light transmission part A are made different. That is, the arrangement pitch of the light transmission part A and the light transmission part B is made different, the arrangement pitch of the light transmission part A and the light transmission part C is made different, and the arrangement pitch of the light transmission part A and the light transmission part D. Further, it is desirable that the light transmission portions A and the light transmission portions E have different arrangement pitches.

  Alternatively, in the image display device with an imaging device according to the first aspect of the present invention, including the first configuration of the present invention, the second configuration of the present invention, and the preferred configuration described above, the light transmission portion is a hollow structure. Or it is good also as a double cyclic structure (double hollow structure). Specifically, the light transmission part is composed of a light transmission part, or is composed of a first light transmission part and a second light transmission part, and the second light transmission part surrounds the first light transmission part. It can also be set as the form arrange | positioned. Optimizing the size, shape, arrangement state, positional relationship of the light transmission part, and the positional relationship between the first light transmission part and the second light transmission part of the light transmission part, the first light transmission part and the second light transmission part As a result, it is possible to reliably suppress the occurrence of the diffraction phenomenon. In addition, it may be combined with [Case A] described above, may be combined with [Case B], may be combined with [Case C], or may be combined with [Case A] and [Case B]. It may be combined with [Case A] and [Case C], may be combined with [Case B] and [Case C], and [Case A], [Case B] and [Case C] You may combine.

  In the image display device with an image pickup device of the present invention including the various preferable modes and configurations described above, the light emitting element is preferably a self-light emitting type light emitting element, and is formed of an organic electroluminescence element (organic EL element). More preferably. The liquid crystal display element constituting the liquid crystal display device controls the passage of light from the outside (light from the backlight or external light), and the pixel does not include a light emitting element. When the light emitting element is composed of an organic EL element, the organic layer (light emitting portion) constituting the organic EL element includes a light emitting layer made of an organic light emitting material. Layered structure of a layer and an electron transport layer, a layered structure of a light emitting layer serving as a hole transport layer and an electron transport layer, a hole injection layer, a hole transport layer, a light emitting layer, an electron transport layer, and an electron injection layer It can be composed of a laminated structure. When the electron transport layer, the light emitting layer, the hole transport layer, and the hole injection layer are “tandem units”, the organic layer is formed by stacking the first tandem unit, the connection layer, and the second tandem unit. It may also have a two-stage tandem structure, and may further have a three-stage or more tandem structure in which three or more tandem units are stacked. In these cases, the emission color is red, By changing green and blue for each tandem unit, an organic layer that emits white light as a whole can be obtained.

  By optimizing the thickness of the organic layer, for example, the light emitted from the light-emitting layer is resonated between the first electrode and the second electrode, and a part of this light is transmitted to the outside via the second electrode. It can also be set as the structure which radiate | emits.

Furthermore, in the image display device with an imaging device of the present invention including the various preferable modes and configurations described above, the image display unit includes:
(A) a first substrate;
(B) a drive circuit provided on the first substrate;
(C) an interlayer insulating layer covering the drive circuit;
(D) a light emitting section provided on the interlayer insulating layer;
(E) a protective layer provided on the light emitting part,
(F) a light shielding layer provided on the protective layer, and
(G) a second substrate covering the protective layer and the light shielding layer;
With
Each pixel includes the drive circuit and the light emitting unit,
The light shielding layer is provided with an opening,
The light transmitting part is configured by the opening, and the part of the protective layer and the part of the interlayer insulating layer located below the opening,
It can be set as the form by which the condensing means and the imaging device are arrange | positioned at the surface side of the 1st board | substrate which does not oppose the 2nd board | substrate.

Here, examples of the pixel arrangement include a stripe arrangement, a diagonal arrangement, a delta arrangement, and a rectangle arrangement. Further, as the first substrate or as the second substrate, a high strain point glass substrate, soda glass (Na ₂ O · CaO · SiO ₂ ) substrate, borosilicate glass (Na ₂ O · B ₂ O ₃ · SiO _2). ) Substrate, forsterite (2MgO · SiO ₂ ) substrate, lead glass (Na ₂ O · PbO · SiO ₂ ) substrate, various glass substrates with an insulating film formed on the surface, quartz substrate, insulating film formed on the surface Quartz substrate, silicon substrate with an insulating film formed on the surface, polymethyl methacrylate (polymethyl methacrylate, PMMA), polyvinyl alcohol (PVA), polyvinyl phenol (PVP), polyethersulfone (PES), polyimide, polycarbonate, polyethylene Organic polymer exemplified by terephthalate (PET) (flexible polymer composed of polymer material) Stick films or plastic sheets, the form of polymeric material such as a plastic substrate) may be mentioned. The drive circuit may be composed of, for example, one or a plurality of thin film transistors (TFTs). As a constituent material of the interlayer insulating _{layer, SiO 2, BPSG, PSG,} BSG, AsSG, PbSG, SiON, SOG ( spin on glass), low-melting glass, SiO ₂ based materials such glass paste; SiN-based materials; insulation, such as polyimide Resins can be used alone or in appropriate combination. When the pixel is composed of an organic EL element, the light emitting unit is as described above. As a material constituting the protective film, it is preferable to use a material that is transparent to the light emitted from the light emitting portion, is dense, and does not transmit moisture. Specifically, for example, amorphous silicon (α-Si), Amorphous silicon carbide (α-SiC), amorphous silicon nitride (α-Si _1-x N _x ), amorphous silicon oxide (α-Si _1-y O _y ), amorphous carbon (α-C), amorphous silicon oxide / silicon nitride (Α-SiON) and Al ₂ O ₃ can be mentioned. The light shielding film (black matrix) may be made of a known material. A color filter may be provided as necessary.

  In the image display device with an imaging device of the present invention including the various preferable configurations and forms described above (hereinafter, these may be collectively referred to as “the present invention”), the image display unit emits light. A plurality of pixel units including elements are arranged. Here, when the number of pixel units is represented by (M, N), VGA (640, 480), S-VGA (800, 600), XGA (1024) 768), APRC (1152,900), S-XGA (1280,1024), U-XGA (1600,1200), HD-TV (1920,1080), Q-XGA (2048,1536), ( (1920, 1035), (720, 480), (854, 480), (1280, 960), etc., but some of the image display resolutions can be exemplified, but the values are limited to these values. Not shall. In the image display unit that performs color display, one pixel unit includes, for example, three types of pixels: a red light emitting pixel that emits red light, a green light emitting pixel that emits green light, and a blue light emitting pixel that emits blue light. In addition to these three types of pixels, pixels that emit white light to improve brightness, pixels that emit complementary colors to expand the color reproduction range, and yellow to expand the color reproduction range Can be composed of four or more types of pixels, such as pixels that emit light, and pixels that emit yellow and cyan to expand the color reproduction range.

  In the image display device with an imaging device according to the first aspect of the present invention, the light transmission portion is provided in each of the plurality of pixels, but specifically, for example, although not limited thereto, It is desirable to be provided in three or more pixels, and the outer shape of the light transmission part is essentially arbitrary, and examples thereof include a rectangle such as a rectangle or a square. Further, in the image display device with an imaging device according to the second aspect of the present invention, the light transmission portion is provided around at least one or more pixels. However, it is desirable to be provided around three or more pixels, and the outer shape of the light transmission part is essentially arbitrary, and the “L” shape (the light transmission part is a pixel). (A form provided on two consecutive sides of the side corresponding to the boundary of the pixel), a “U” -shaped (light transmission part is provided on three consecutive sides of the side corresponding to the boundary of the pixel) ), “B” -shaped (form in which the light transmission part is provided on all sides corresponding to the boundary of the pixel), and a cross-shaped shape (the light transmission part corresponds to the boundary of the pixel) An example of a configuration provided on all sides and provided in common between adjacent pixels) It can be. Alternatively, it is preferable that the light transmission unit is provided in a pixel group including a projection image of a lens provided in the imaging device and is provided around the pixel group.

  In the present invention, the imaging device may be disposed on the back side of the image display unit, but is preferably disposed in the center of the image display unit. There may be one imaging device or a plurality of imaging devices. As the imaging device, for example, a well-known and commercially available solid-state imaging device including a CCD element and a CMOS sensor may be used. Note that a well-known and commercially available solid-state imaging device such as a video camera or a web camera can also be used. In these cases, the light collecting means and the imaging device are integrated.

  A well-known lens can be mentioned as a condensing means which condenses the light which passed the some light transmissive part to an imaging device. Specifically, the lens can be composed of either a biconvex lens, a plano-convex lens, or a meniscus convex lens, or can be composed of a reflecting mirror or a Fresnel lens, or a combination of these various convex lenses. In addition, a concave lens and these various convex lenses can be combined.

  In the present invention, it is preferable that a color filter is not disposed in the optical path of light that enters the image display unit, passes through the light transmission unit, exits from the image display unit, and enters the light collecting unit. It is preferable not to arrange an imaging system such as.

  The present invention can be used, for example, as an alternative to a monitor device that constitutes a personal computer, can be used as an alternative to a monitor device incorporated in a notebook personal computer, and can be used as a mobile phone or a PDA (portable information). Terminals, Personal Digital Assistant), monitor devices incorporated in game machines, and can be used as an alternative to conventional television receivers.

  Example 1 relates to an image display device with an imaging device according to the first aspect of the present invention. FIGS. 1A and 1B are conceptual views of the image display device with an imaging device according to the first embodiment viewed from the front and side, and the arrangement of a plurality of pixels constituting the image display unit is schematically illustrated in FIG. FIG. 2C is a schematic partial cross-sectional view of the image display device with an imaging device shown in FIG.

The image display device with an imaging device of Example 1 is
(A) An image display unit 10 in which a plurality of pixels 11 (11R, 11G, 11B) including light emitting elements are arranged,
(B) a light transmission unit 30 provided in each of the plurality of pixels 11;
(C) the imaging device 20 disposed on the back side of the image display unit 10, and
(D) Light condensing means 21 that condenses the light that has passed through the plurality of light transmitting portions 30 on the imaging device 20;
Have

  In Example 1 or Examples 2 to 8 to be described later, the light emitting element is a self-luminous light emitting element, specifically, an organic EL element, and the image display unit 10 is an XGA type organic color display. It consists of an EL display device. That is, when the number of pixel units is represented by (M, N), it is (1024, 768). One pixel unit is composed of three pixels: a red light emitting pixel 11R that emits red light, a green light emitting pixel 11G that emits green light, and a blue light emitting pixel 11B that emits blue light. 1 (C), FIG. 7 (A), (B), FIG. 8, FIG. 10 (A), (B), FIG. 11, FIG. 12 (A), (B), FIG. In (A) and (B), the outer edge of the pixel is indicated by a dotted line. The imaging device 20 is disposed on the back side of the image display unit 10, more specifically, on the center of the back side of the image display unit 10, and the number of the imaging device 20 provided is one. Here, the imaging device 20 and the light condensing means 21 are constituted by a well-known and commercially available video camera in which they are integrated, that is, provided with a CCD element. The image display device with an imaging device according to the first embodiment is used as an alternative to a monitor device that constitutes a personal computer. In addition, the image display device with an imaging device according to the first embodiment includes a personal computer.

  Although not limited, the light transmission unit 30 is provided in 6 × 3 = 18 pixels 11. One light transmitting portion 30 is provided for one pixel. The condensing unit 21 condenses the light that has passed through the light transmission unit 30 in the 6 × 3 = 18 pixels 11 on the imaging device 20. The shape of each light transmission part 30 is a rectangle.

  In Embodiment 1 or Embodiments 2 to 8 described later, the image display unit 10 is provided with a scanning signal supply IC that drives each scanning line and a video signal supply IC that supplies a video signal. A scanning line control circuit is connected to the scanning signal supply IC, and a signal line control circuit is connected to the video signal supply IC. A color filter is not arranged in the optical path of the light that enters the image display unit 10, passes through the light transmission unit 30, exits the image display unit 10, and enters the light collecting unit 21. There is no image system.

In Example 1 or Example 2 to Example 8 described later, specifically, the image display unit 10 includes:
(A) the first substrate 40,
(B) a drive circuit provided on the first substrate 40;
(C) an interlayer insulating layer 41 covering the drive circuit;
(D) a light emitting part (organic layer 63) provided on the interlayer insulating layer 41;
(E) a protective layer 64 provided on the light emitting portion (organic layer 63);
(F) a light shielding layer 65 provided on the protective layer 64, and
(G) a second substrate 67 covering the protective layer 64 and the light shielding layer 65;
It has.

  Each pixel 11 includes a drive circuit and a light emitting portion. The light shielding layer 65 is provided with an opening 65A. The opening 65A and the protective layer 64 positioned below the opening 65A are provided. The light transmitting portion 30 is configured by the portion, the second electrode 62 portion, the interlayer insulating layer 41 portion, and the like. The condensing means 21 and the imaging device 20 are arranged on the surface side of the first substrate 40 that does not face the second substrate 67.

More specifically, a drive circuit is provided on the first substrate 40 made of soda glass. The drive circuit is composed of a plurality of TFTs. The TFT includes a gate electrode 51 formed on the first substrate 40, a gate insulating film 52 formed on the first substrate 40 and the gate electrode 51, and a source provided on a semiconductor layer formed on the gate insulating film 52. The portion of the semiconductor layer located between the source / drain region 53 and the source / drain region 53 and above the gate electrode 51 is composed of a corresponding channel forming region 54. In the illustrated example, the TFT is a bottom gate type, but may be a top gate type. The gate electrode 51 of the TFT is connected to a scanning line (not shown). The interlayer insulating layer 41 (41A, 41B) covers the first substrate 40 and the drive circuit. The first electrode 61 constituting the organic EL element, SiO _X and SiN _Y, is provided on the interlayer insulating layer 41B made of polyimide resin or the like. The TFT and the first electrode 61 are electrically connected via a contact plug 42, a wiring 43, and a contact plug 44 provided in the interlayer insulating layer 41A. In the drawing, one TFT is shown for one organic EL element driving unit.

On the interlayer insulating layer 41, an insulating layer 45 having an opening 46 and having the first electrode 61 exposed at the bottom of the opening 46 is formed. The insulating layer 45 is excellent in flatness, and is made of an insulating material having a low water absorption rate, specifically, a polyimide resin in order to prevent the organic layer 63 from being deteriorated by moisture and maintain the light emission luminance. An organic layer 63 having a light emitting layer made of an organic light emitting material is formed over the portion of the insulating layer 45 surrounding the opening 46 from above the portion of the first electrode 61 exposed at the bottom of the opening 46. The organic layer 63 is composed of, for example, a stacked structure of a light-emitting layer that also serves as a hole transport layer and an electron transport layer, but is represented by one layer in the drawing. An insulating protective layer 64 made of amorphous silicon nitride (α-Si _1-x N _x ) is provided on the second electrode 62 based on the plasma CVD method for the purpose of preventing moisture from reaching the organic layer 63. It has been. A light shielding layer 65 made of black polyimide resin is formed on the protective layer 64, and a second substrate 67 made of soda glass is disposed on the protective layer 64 and the light shielding layer 65. The protective layer 64 and the light shielding layer 65 are bonded to the second substrate 67 by an adhesive layer 66 made of an acrylic adhesive. The first electrode 61 is used as an anode electrode, and the second electrode 62 is used as a cathode electrode. Specifically, the first electrode 61 is made of a light reflecting material made of aluminum (Al), silver (Ag), or an alloy thereof having a thickness of 0.2 μm to 0.5 μm, and the second electrode 62. Is made of a transparent conductive material such as ITO or IZO having a thickness of 0.1 μm, or a metal thin film (semi-transparent metal thin film) having a thickness of about 5 nm such as silver (Ag) or magnesium (Mg). . The second electrode 62 is not patterned and is formed in a single sheet. In some cases, an electron injection layer (not shown) made of LiF having a thickness of 0.3 nm may be formed between the organic layer 63 and the second electrode 62.

  In summary, the detailed configuration of the light-emitting elements of Example 1 or Examples 2 to 8 described later is as shown in Table 1 below.

[Table 1]
Second substrate 67: Soda glass adhesive layer 66: Acrylic adhesive light shielding layer 65: Black polyimide resin protective layer 64: SiN _x layer (thickness: 5 μm)
Second electrode (cathode electrode) 62: ITO layer (thickness: 0.1 μm) or semitransparent metal thin film electron injection layer: LiF layer (thickness: 0.3 nm)
Organic layer 63: As described above, first electrode (anode electrode) 61: Al—Nd layer (thickness: 0.2 μm)
Interlayer insulating layer 41: SiO ₂ layer TFT: constituting drive circuit First substrate 40: soda glass

  FIGS. 3A and 3B are conceptual diagrams of the image display device with an imaging device of the first embodiment and the image display device in which the imaging device is fixed to the outside of the image display unit. As shown in FIG. 3B, when the imaging device is fixed to the outside of the image display unit, the imaging device photographs the user of the image display device from an oblique direction, and the image is displayed on the image display unit. When the image is displayed, an image of the user taken from an oblique direction is displayed on the image display unit. Therefore, it is impossible to accurately display the user's face, and it is not possible to accurately determine where in the image display unit the user is gazing. Furthermore, when the user approaches the image display unit, there is a high possibility that the user will be outside the imaging range. On the other hand, in the image display device with an image pickup device of the first embodiment, as shown in FIG. 3A, the image pickup device is arranged at the center on the back side of the image display portion. The user of the image display apparatus can photograph the image pickup apparatus from the front, and when the image is displayed on the image display unit, the image of the user photographed from the front is displayed on the image display unit. . Therefore, the user's face can be accurately displayed, and it can be easily and accurately determined where in the image display unit the user is gazing. Further, even when the user approaches the image display unit, the user can be imaged.

  As described above, in the image display device with an imaging device according to the first embodiment, the light that has passed through the light transmission units 30 (the plurality of light transmission units 30) provided in each of the plurality of pixels 11 is input to the imaging device 20. Focused. Therefore, a high-precision microlens is not required to accurately tie an image to the imaging device 20, and the manufacturing cost of the image display device with the imaging device is not increased, and a sufficient amount of light is supplied to the imaging device 20. It can be condensed. In addition, since it is possible to specify the corresponding indication point of the image display unit from the orientation of the hand or pen, a so-called pointer function can be easily added to the image display device with an imaging device. Furthermore, in addition to the pointer function, the user's face, eyes, hand movements, peripheral brightness, etc. can also be known from the captured image, so various information can be obtained from the image display device with the image capturing device and used in various systems. Thus, the added value of the image display device with the imaging device can be increased.

  The second embodiment is a modification of the first embodiment and relates to the first configuration of the present invention. In the second embodiment, the captured image is processed to obtain a higher quality captured image.

  In general, when light passes through the minute light transmission part 30, a so-called diffraction phenomenon occurs in the light transmission part 30. FIG. 4 shows a schematic diagram for explaining the diffraction phenomenon caused by the slit. Here, the light transmission unit 30 acts as a slit, and due to the diffraction phenomenon, the same image as the image “C” appears as an image “A” and an image “B” at an equal pitch, resulting in blurring of the image. An image obtained by photographing with a transparent glass plate placed in front of the imaging device 20 is shown in FIG. 5A, and a light transmitting portion having a certain shape, size, and distribution in front of the imaging device 20 An image obtained by arranging and photographing a transparent glass plate provided with is shown in FIG. Blur is observed in the image shown in FIG. On the other hand, no blur is observed in the image shown in FIG. The intensity and distribution of the diffracted light depend on the shape, size, distribution, and wavelength of incident light (external light) of the light transmission unit 30. When the blur due to diffraction is small, it is not necessary to correct (compensate) the diffraction for the captured image. However, when a high-quality captured image is required, the influence of the diffracted light is corrected (compensated). )There is a need to.

  The image display device with an imaging device according to the second embodiment further includes a diffraction correction unit 100 that corrects (compensates) diffraction generated in the light transmission unit 30 with respect to image information acquired through the imaging device 20. .

The diffraction distribution P _diff can be calculated by Expression (1) when the pattern shape, size, distribution, and wavelength of incident light (external light) of the light transmitting unit 30 are determined. In the double integration, the integration is performed from −∞ to + ∞ with respect to x and y.

_{P diff (k x, k y} ) = ∬P at (x, y) · exp [-j (k x · x + k y · y)] dxdy
(1)
However,
k _x = (2π / λ) sin (θ _x )
k _y = (2π / λ) sin (θ _y )

Here, P _at (x, y) is a two-dimensional pattern on the xy plane of the light transmitting section 30, λ is the wavelength of incident light (external light), and θ _x and θ _y are the x direction and the y direction. Is the diffraction angle. In Example 2, the value of the wavelength λ of incident light (external light) was fixed at 525 nm in order to simplify the calculation.

Since Equation (1) is a two-dimensional Fourier transform of P _at (x, y), it can be calculated at high speed by using a fast Fourier transform (hereinafter abbreviated as “FFT”). Although P _diff (k _x , k _y ) includes phase information, actually, the imaging apparatus detects the diffracted light intensity H _diff (k _x , k _y ). The diffracted light intensity H _diff (k _x , k _y ) is equal to the square of the absolute value of P _diff (k _x , k _y ).

H _diff (k _x , k _y ) = | P _diff (k _x , k _y ) | ² (2)

  Here, assuming that the spatial resolution of the imaging device is modulated by diffracted light, an MTF (Moduration Transfer Function) is calculated from the following equation (3). Note that “FFT []” means executing a fast Fourier transform, and “IFFT []” means executing a fast inverse Fourier transform.

MTF (f _x , f _y ) = | FFT [H _diff (k _x , k _y )] | ² (3)

Here, f _x, f _y represents the x and y directions of spatial frequencies in each of the imaging elements constituting the imaging device. And between the image I _{cam (x, y)} passing through the light transmission part 30 on the imaging device 20 and the original image I _{ral (x, y)} when not passing through the light transmission part 30 The following relationship is established.

_{FFT [I cam (x, y} )] = FFT [I ral (x, y)] × MTF (f x, f y) (4)

That is, in the spatial frequency domain, the image I _{cam (x, y)} is a product of the original image I _{ral (x, y)} and MTF. Therefore, in order to obtain the original image I _{ral (x, y)} from the image I _{cam (x, y} ), a process based on the following equation (5) may be performed. In other words, the MTF inverse conversion process calculated based on the shape, size, distribution, and wavelength of the incident light (external light) of the light transmission unit 30 may be performed on the image information.

_{I ral (x, y) =} IFFT [FFT [I cam (x, y)] / MTF (f x, f y)] (5)

  Here, if the size, shape, and distribution of the light transmission part 30 are determined, the MTF may be scaled by the wavelength of the incident light (external light) obtained by Fourier transforming the two-dimensional pattern of the light transmission part 30. It can be easily obtained, and the original image can be easily restored from the relationship shown in Expression (5).

  FIG. 6 shows a block diagram of the image display device with an imaging device of the second embodiment. The image information acquired via the imaging device 20 is sent to the MTF inverse transform unit that constitutes the diffraction correction unit 100, and the MTF inverse transform unit outputs red (R), green (G), and blue (B) for each. Using the wavelength of external light (however, in the second embodiment, for simplicity of calculation, the above-described one kind of wavelength) and the MTF shape data of the light transmitting unit 30 obtained by the two-dimensional FFT are used. MFT inverse transform is performed to restore the original image, which is then sent to the control unit 12. The control unit 12 performs various detections such as detecting the user's line of sight and detecting the movement of the user's hand from the restored image, and reflects them on the image display unit. MTF shape data such as the size, shape, and distribution of the light transmission unit 30 is stored in the MFT shape storage unit that constitutes the diffraction correction unit 100. The control unit 12 is provided in an image display device with an imaging device (or an image display unit), or is configured by a personal computer provided in the image display device with an imaging device. The diffraction correction means 100 is also provided in an image display device with an imaging device (or an image display unit), or alternatively, a personal computer provided in the image display device with an imaging device.

  The image display in the image display unit 10 is performed under the control of the control unit 12. That is, display data and timing signals are sent from the control unit 12 to the display timing controller 13, and display data and horizontal timing signals are sent from the display timing controller 13 to a signal line control circuit (not shown). A vertical timing signal is sent to a scanning line control circuit (not shown). Then, the image display unit 10 displays an image based on a known method. On the other hand, an imaging timing signal, a shutter control signal, a gain control signal, and the like are sent from the control unit 12 to the imaging timing controller 14, and these signals are sent from the imaging timing controller 14 to the imaging device 20. Is controlled.

The shape of the light transmission part 30 is illustrated in FIGS. 7A and 7B and FIG. Part or all of the light transmission unit 30 is periodically provided along the first direction (horizontal direction) and the second direction (vertical direction) of the image display unit. Here, in the example shown in FIG. 7A, the light transmission part 30H extending along the first direction (horizontal direction) below the pixel is formed from three pixels (11R, 11G, and 11B). The light transmission part 30V provided across the configured one pixel unit and extending in the second direction (vertical direction) is provided in each of the pixels 11R, 11G, and 11B. It is provided between. In the example shown in FIG. 7B, the light transmission part 30H and the light transmission part 30 are connected. In the example shown in FIG. 8, the light transmission unit 30H is provided across one pixel unit composed of three pixels (11R, 11G, and 11B). Is different and consists of two parts. When the length of the light transmission part 30 along the first direction is L _tr−1 and the pitch of the pixels 11 along the first direction is P _px−1 , the line aperture ratio L _{tr in} the first direction. _-1 / P _px-1 is
L _tr-1 / P _px-1 ≧ 0.5
And when the length of the light transmitting portion 30 along the second direction is L _tr-2 and the pitch of the pixels 11 along the second direction is P _px-2 , the line in the second direction The aperture ratio L _tr-2 / P _px-2 is
L _tr-2 / P _px-2 ≧ 0.5
Satisfied. This can be explained from the definition of MTF.

The MTF is a diffraction distribution obtained from the two-dimensional pattern P _at (x, y) on the xy plane of the light transmitting section 30 as shown in the following expression (6) derived from the expressions (2) and (3). Is squared and the result is obtained as a result of further squaring.

MTF (f _x , f _y ) = | FFT [| P _diff (k _x , k _y ) | ² ] | ² (6)

From the so-called Wiener-Kinchin theorem, the Fourier transform of the autocorrelation function is equal to the power spectrum, so that MTF is equal to the square of the absolute value of the autocorrelation function of the diffraction distribution generated in the light transmission section 30. The condition that the autocorrelation function does not have a so-called uncorrelated point in the spatial frequency domain (that is, a point that becomes 0) is:
L _tr-1 / P _px-1 ≧ 0.5
L _tr-2 / P _px-2 ≧ 0.5
It is. When there is no point at which the MTF is 0, the equation (5) has no singular point, so that it is easy to reproduce the original image. Therefore, it is requested that the values of the line aperture ratio L _tr-1 / P _{px-1 in the first} direction and the line aperture ratio L _tr-2 / P _px-2 in the second direction are 0.5 or more. It is preferable to satisfy.

The third embodiment is a modification of the second embodiment and relates to the second configuration of the present invention. As described in the second embodiment, the expression (1) for _obtaining the diffraction distribution P _diff (k _x , k _y ) on the xy plane of the light transmitting unit 30 includes the wavelength λ of the incident light (external light). ing. Therefore, by measuring the wavelength distribution of external light, it is possible to adapt the MTF of each wavelength according to the external environment, and it is possible to correct and compensate for diffraction more accurately, and to obtain a higher quality captured image. Can be obtained.

  FIG. 9 shows a block diagram of an image display device with an imaging device according to the third embodiment. The third embodiment further includes wavelength distribution measuring means 110 that measures the wavelength distribution of external light. Specifically, the wavelength distribution measuring unit 110 includes a set of a photo sensor to which a red filter is attached, a photo sensor to which a green filter is attached, and a photo sensor to which a blue filter is attached. By providing such a wavelength distribution measuring means 110, the wavelength distribution of external light (light spectrum) can be obtained. And the wavelength distribution for each primary color (red, green, blue) of the captured image can be obtained by multiplying the obtained wavelength distribution of the external light and the spectral spectrum of the imaging device. Then, by weighting the image subjected to MTF inverse transform processing for each wavelength with the wavelength distribution of the captured image, correction and compensation for diffraction can be performed more accurately.

  In the third embodiment, by adopting such a configuration, it is possible not only to correct and compensate for diffraction more accurately, but also to improve the accuracy of image information acquired via the imaging device 20 ( For example, the accuracy of color information can be improved. When the blur due to diffraction is small, it is not necessary to correct (compensate) the diffraction for the captured image. However, even in such a case, the wavelength distribution for measuring the wavelength distribution of external light. By further including the measurement unit 110, it is possible to improve the accuracy of the image information acquired via the imaging device 20.

The fourth embodiment is also a modification of the first to third embodiments and relates to the third configuration of the present invention. In the image display device with an imaging device of Example 4,
Information sending means (not shown) for sending image information acquired via the imaging device 20, and
Display means (not shown) for displaying an image based on image information input from the outside on the image display unit 10;
Is further provided. Then, the image information acquired via the imaging device 20 is transmitted to the outside by the information transmitting unit, and an image based on the image information input from the outside is displayed on the image display unit 10 by the display unit.

  Here, the information transmission means for transmitting the image information acquired via the imaging device 20 and the display means for displaying an image based on the image information input from the outside on the image display unit 10 are a television telephone conference system. A well-known means used in (Videophone device) may be used. The information sending unit and the display unit are incorporated in the control unit 12, for example.

  By connecting a plurality of such image display devices with an image pickup device via a communication line, a network, or the like, a so-called video phone conference system (video phone device) can be constructed. And since the imaging device is arranged on the back side of the image display unit, the face of the user located in front of the image display unit can be imaged, and the face of the other user on the image display unit is displayed. Because they are facing each other, they do not give a sense of incongruity that the lines of sight of the conventional TV phone system do not match each other.

The fifth embodiment is a modification of the first to third embodiments and relates to the fourth configuration of the present invention. In the image display device with an imaging device of Example 5,
Storage means (not shown) for storing image information acquired via the imaging device 20, and
Display means (not shown) for displaying on the image display unit 10 an image based on the image information acquired through the imaging device 20 (including the acquired state) and the image information stored in the storage means;
Is further provided. Then, the display unit displays an image based on the image information acquired through the imaging device 20 (including the acquired state) on the image display unit 10 and also stores the image information stored in the storage unit. Is displayed on the image display unit 10. Note that the storage unit and the display unit are incorporated in the control unit 12, for example.

  Here, the storage means for storing the image information acquired via the imaging device 20 is, for example, a known nonvolatile memory or hard disk drive device provided in the imaging device 20, the image display unit 10, or the control unit 12, and The image pickup device 20, the image display unit 10, and the control unit 12 can be configured from a known image information processing circuit. The display means for displaying an image on the image display unit 10 can be configured by a known image information display circuit provided in the imaging device 20, the image display unit 10, and the control unit 12.

  Such an image display device with an imaging device functions as a so-called digital mirror. Then, for example, it is possible to compare the user's video and image captured in the past with the current user's video and image. That is, for example, the user's past video is called using an operation switch, button, keyboard, etc., the user's past video is displayed on the left side of the image display unit, and the user's current video is displayed on the right side. By displaying, the comparison result between the past and the current user can be displayed in separate windows in the image display unit 10. Thus, since the user can be compared with the past and the user himself / herself can recognize the difference, the imaging of the fifth embodiment can be used for so-called beauty applications such as makeup and facial expression confirmation. An image display device with a device can be used.

  The sixth embodiment is a modification of the first to fifth embodiments. As described above, when the light transmission part 30 is very small, a diffraction phenomenon occurs in the light transmission part 30, resulting in blurring of the image formed on the imaging device 20 or lack of clarity. Therefore, in Example 6, the sizes of the plurality of light transmission portions were random. Specifically, as schematically shown in FIG. 10A, each of at least two light transmission parts adjacent to one light transmission part has a configuration different from the size of this one light transmission part. . More specifically, 40 light transmission parts 30 are provided, and in all the light transmission parts, each of four light transmission parts adjacent to one certain light transmission part is the size of this certain light transmission part. And a different configuration. Here, in FIG. 10A, focusing on the light transmission portion 30A, two light transmission portions 30B and 30C arranged in the horizontal direction adjacent to the light transmission portion 30A and two arranged in the vertical direction. Each of the two light transmission portions 30D and 30E is different in size from the light transmission portion 30A. Further, when paying attention to the light transmission part 30B, each of the two light transmission parts 30A and 30f arranged in the horizontal direction adjacent to the light transmission part 30B and the two light transmission parts 30g and 30h arranged in the vertical direction, respectively. Is different from the light transmitting portion 30B in size. As a result, it is possible to avoid the occurrence of a diffraction phenomenon in the light transmission part 30. 10A, 10 </ b> B, and FIG. 11, for simplification of the drawings, in all the light transmitting portions 30, four light transmitting portions 30 adjacent to one certain light transmitting portion 30. Are not shown as being different from the size of the certain light transmitting portion 30.

  Alternatively, in Example 6, the shapes of the plurality of light transmission parts were random. Specifically, as schematically shown in FIG. 10B, each of at least two light transmission parts adjacent to one light transmission part has a configuration different from the shape of this one light transmission part. . More specifically, 40 light transmitting portions 30 are provided, and in all the light transmitting portions, each of the four light transmitting portions adjacent to one certain light transmitting portion has the shape of this certain light transmitting portion. And a different configuration. Here, in FIG. 10B, focusing on the light transmitting portion 30A, two light transmitting portions 30B and 30C arranged in the horizontal direction adjacent to the light transmitting portion 30A and two arranged in the vertical direction. Each of the two light transmission portions 30D and 30E is different in shape from the light transmission portion 30A. Further, when paying attention to the light transmission part 30B, each of the two light transmission parts 30A and 30f arranged in the horizontal direction adjacent to the light transmission part 30B and the two light transmission parts 30g and 30h arranged in the vertical direction, respectively. Is different in shape from the light transmitting portion 30B. Also by this, it was possible to avoid the occurrence of the diffraction phenomenon in the light transmission part 30.

  Alternatively, in Example 6, the arrangement pitch of the plurality of light transmission parts was random. Specifically, as schematically shown in FIG. 11, each of at least two light transmission parts adjacent to one light transmission part has a different arrangement pitch from the one light transmission part. More specifically, 40 light transmission parts 30 are provided, and in all the light transmission parts, each of the four light transmission parts adjacent to one certain light transmission part is connected to this certain light transmission part. The configuration is different from the arrangement pitch. Here, in FIG. 11, paying attention to the light transmission part 30A, the two light transmission parts 30B and 30C arranged in the horizontal direction adjacent to the light transmission part 30A and the two light transmission parts arranged in the vertical direction. Each of 30D and 30E is different in arrangement pitch from the light transmitting portion 30A. Also by this, it was possible to avoid the occurrence of the diffraction phenomenon in the light transmission part 30.

In addition, what is necessary is just to process the opening part 65A which comprises the light transmissive part 30 so that the structure and structure mentioned above may be obtained. Here, the minimum value and the minimum shape of the light transmission part 30 depend on the minimum processing dimension (for example, F: 0.5 μm) in the photolithography technique and the etching technique for providing the light transmission part 30. Therefore, the size of the light transmitting portion 30, to be defined by the aggregate of this unit the rectangular area F ² (or shapes derived on photolithography rectangular shape of the area F ²⁾ was defined as 1 unit, The shape of the light transmission part 30 is also defined by the aggregate of the units.

  In addition, the structure and structure of the light transmission part in Example 6 demonstrated above are applicable to Examples 1-5.

The seventh embodiment is also a modification of the first to fifth embodiments. As described above, when the light transmission part 30 is very small, a diffraction phenomenon occurs in the light transmission part 30, resulting in blurring of the image formed on the imaging device 20 or lack of clarity. Therefore, in Example 7, the light transmission portion has a hollow structure or a double annular structure (double hollow structure). Specifically, the arrangement of a plurality of pixels constituting the image display unit in the image display device with an imaging device according to the fifth embodiment is schematically shown in FIGS. 12A and 12B. The unit 30 includes a light transmission unit 30B (see FIG. 12A) or alternatively includes a first light transmission unit 30A and a second light transmission unit 30B so as to surround the first light transmission unit 30A. The second light transmission part 30B is arranged in the (see FIG. 12B). 12A and 12B, in order to clarify the light transmitting portion 30B, the first light transmitting portion 30A, and the second light transmitting portion 30B, the light transmitting portion 30B and the first light transmitting portion 30A. The second light transmission part 30B is hatched. The size, shape, and arrangement of the light transmission unit 30B, the first light transmission unit 30A, and the second light transmission unit 30B, the positional relationship of the light transmission unit 30B, and the first light transmission unit 30A and the second light transmission unit 30B. By optimizing the positional relationship, it was possible to reliably suppress the occurrence of the diffraction phenomenon. In addition, the structure and structure of the light transmission part in Example 7 can be applied to Examples 1 to 5. Also, from the viewpoint of avoiding having the point where the MTF described in the second embodiment becomes 0, it is more preferable that the opening has a hollow structure or a double ring structure. The reason is shown below. That is, the MTF corresponding to the outside of the rectangular opening shape in the hollowed structure or double ring _{structure, MTF out (f x, f} y), the MTF corresponding to the inside of the light shielding _{shape, MTF in (f x, f} y ) and when, MTF of the entire opening of the hollowed structure or a double ring structure (f _x, f _y) is
_{MTF (f x, f y)} = MTF out (f x, f y) + MTF in (f x, f y)
Since the is _{MTF out (f x, f y} ) a zero _{of, MTF in (f x, f} y) because it is possible to cancel by taking the sum of the. From the above, by making the shape of the opening portion a hollow shape, the occurrence of the diffraction phenomenon can be reduced, and of course, the reproduction of the image by correction and compensation for the diffraction phenomenon becomes easier. Furthermore, Example 7 and Example 6 may be combined.

Example 8 relates to an image display device with an imaging device according to the second aspect of the present invention. As schematically shown in FIG. 13A or 13B, the arrangement of the plurality of pixels 11 (11R, 11G, 11B) constituting the image display unit in the image display device with an image pickup apparatus of the eighth embodiment. The image display device with an imaging device of Example 8 is
An image display unit 10 comprising a plurality of pixels 11 including light emitting elements,
A light transmission portion 630 provided around at least one pixel 11;
An imaging device 20 disposed on the back side of the image display unit 10, and
Condensing means 21 for condensing the light that has passed through the light transmitting portion 630 on the imaging device 20;
Have

  In the example shown in FIG. 13A, the light transmitting portion 630 is provided around all of the pixels 11 and has a cross-like shape. In the example illustrated in FIG. 13B, the light transmission portion 630 is provided in a part of the periphery of the pixel 11 and has an “L” shape. In the image display device with an imaging device according to the eighth embodiment, the light transmission unit 630 is provided around at least one pixel 11, and specifically, around 6 × 3 = 18 pixels 11. Is provided.

  Except for the above points, the image display device with an imaging device can have the same configuration and structure as those of the image display devices with an imaging device according to the first to fifth embodiments, and thus detailed description thereof is omitted.

  As described above, in the image display device with an imaging device according to the eighth embodiment, the light that has passed through the light transmission unit 630 provided around at least one pixel 11 is collected on the imaging device 20. Therefore, a high-precision microlens is not required to accurately tie an image to the imaging device 20, and the manufacturing cost of the image display device with the imaging device is not increased, and a sufficient amount of light is supplied to the imaging device 20. It can be condensed. In addition, since it is possible to specify the corresponding indication point of the image display unit from the orientation of the hand or pen, a so-called pointer function can be easily added to the image display device with an imaging device. Furthermore, in addition to the pointer function, the user's face, eyes, hand movements, peripheral brightness, etc. can also be known from the captured image, so various information can be obtained from the image display device with the image capturing device and used in various systems. Thus, the added value of the image display device with the imaging device can be increased.

  As mentioned above, although this invention was demonstrated based on the preferable Example, this invention is not limited to these Examples. The configuration and structure of the image display device with an imaging device described in the embodiments are examples, and can be changed as appropriate. The image display device with an imaging device of the present invention can be used as an alternative to a monitor device incorporated in a notebook personal computer (see FIG. 14A), or can be used as a mobile phone (see FIG. 14B). ), A PDA, a monitor device incorporated in a game machine, or a conventional television receiver.

  In addition, if two imaging devices are arranged on the back side of the image display unit, it is possible not only to accurately grasp the face, eyes, movement, etc. of the user located in front of the image display unit. The distance from the image display unit to the user can be accurately measured based on the image information from each of the imaging devices. Alternatively, one pixel unit is added to the three types of pixels 11R, 11G, and 11B, a pixel that emits white light for improving luminance, a pixel that emits complementary colors to expand the color reproduction range, and a color reproduction range It is also possible to comprise four or more types of pixels, such as a pixel that emits yellow to enlarge the color and a pixel that emits yellow and cyan to enlarge the color reproduction range.

DESCRIPTION OF SYMBOLS 10 ... Image display part, 11, 11R, 11G, 11B ... Pixel, 12 ... Control part, 13 ... Display timing controller, 14 ... Imaging timing controller, 20 ... Imaging Device, 21 ... Condensing means, 30, 630 ... Light transmission part, 40 ... First substrate, 41 ... Interlayer insulating layer, 42, 44 ... Contact plug, 43 ... Wiring 45 ... insulating layer 46 ... opening 51 ... gate electrode 52 ... gate insulating film 53 ... source / drain region 54 ... channel forming region 61 ... First electrode, 62 ... second electrode, 63 ... organic layer, 64 ... protective layer, 65 ... light shielding layer, 65A ... opening, 66 ... adhesive layer, 67 ... Second substrate, 100 ... Diffraction correction means, 110 ... Wavelength distribution measurement means

Claims (14)

An image display unit comprising a plurality of pixels including light emitting elements,
A light transmitting portion provided in each of the plurality of pixels;
An imaging device disposed on the back side of the image display unit, and
Condensing means for condensing the light that has passed through the plurality of light transmitting portions on the imaging device;
An image display device with an imaging device. An image display unit comprising a plurality of pixels including light emitting elements,
A light transmission portion provided around at least one or more pixels;
An imaging device disposed on the back side of the image display unit, and
A condensing means for condensing the light that has passed through the light transmitting portion onto the imaging device;
An image display device with an imaging device.   The image display device with an imaging device according to claim 1, further comprising diffraction correction means for correcting diffraction generated in the light transmission portion with respect to the image information acquired through the imaging device. A part or all of the light transmission part is periodically provided along the first direction and the second direction of the image display part,
When the length of the light transmission part along the first direction is L _tr-1 and the pitch of the pixels along the first direction is P _px-1 ,
L _tr-1 / P _px-1 ≧ 0.5
The image display device with an imaging device according to claim 3 satisfying the above. When the length of the light transmission part along the second direction is L _tr-2 and the pixel pitch along the second direction is P _px-2 ,
L _tr-2 / P _px-2 ≧ 0.5
The image display device with an imaging device according to claim 4, wherein:   The image display device with an imaging device according to any one of claims 1 to 5, further comprising wavelength distribution measuring means for measuring a wavelength distribution of external light. Information sending means for sending image information acquired via the imaging device; and
Display means for displaying an image based on image information input from the outside on the image display unit;
Is further provided,
Send image information acquired via the imaging device to the outside by an information sending means,
The image display device with an imaging device according to claim 1, wherein an image based on image information input from the outside is displayed on the image display unit by a display unit. Storage means for storing image information acquired via the imaging device; and
Display means for displaying on the image display unit an image based on the image information acquired via the imaging device and the image information stored in the storage means;
Is further provided,
The display unit displays an image based on the image information acquired via the imaging device on the image display unit, and also displays an image based on the image information stored in the storage unit on the image display unit. The image display device with an imaging device according to claim 6.   2. The image display device with an imaging device according to claim 1, wherein each of at least two light transmissive portions adjacent to one light transmissive portion has a size different from that of the one light transmissive portion.   10. The image display device with an imaging device according to claim 1, wherein each of at least two light transmissive portions adjacent to one light transmissive portion has a shape different from that of the one light transmissive portion.   11. The imaging according to claim 1, wherein each of at least two light transmissive portions adjacent to one light transmissive portion has a different arrangement pitch from the one light transmissive portion. Image display device with device. The light transmission part is composed of a first light transmission part and a second light transmission part,
The image display device with an imaging device according to claim 1, wherein the second light transmission portion is disposed so as to surround the first light transmission portion.   The image display device with an imaging device according to any one of claims 1 to 12, wherein the light emitting element comprises an organic electroluminescence element. The image display section
(A) a first substrate;
(B) a drive circuit provided on the first substrate;
(C) an interlayer insulating layer covering the drive circuit;
(D) a light emitting section provided on the interlayer insulating layer;
(E) a protective layer provided on the light emitting part,
(F) a light shielding layer provided on the protective layer, and
(G) a second substrate covering the protective layer and the light shielding layer;
With
Each pixel includes the drive circuit and the light emitting unit,
The light shielding layer is provided with an opening,
The light transmitting part is configured by the opening, and the part of the protective layer and the part of the interlayer insulating layer located below the opening,
The image display device with an imaging device according to any one of claims 1 to 12, wherein a light condensing unit and an imaging device are arranged on a surface side of the first substrate that does not face the second substrate.