JP6906995B2 - Electronic devices, imaging devices, control methods, and programs - Google Patents

Description

The present invention relates to an electronic device, an image pickup device, and a control method, and a program, and more particularly to an electronic device, an image pickup device, a control method, and a program to which an image pickup device as a module can be attached and detached.

Electronic devices called smart devices that realize various desired functions by combining modules that are organized in functional units like blocks are known. Such a smart device is composed of a main body in which a plurality of slots are formed and a plurality of modules having different functions, and these various modules are attached to and detached from the slots of the main body in any combination. .. At this time, for example, if a module having a shooting function (imaging module) is installed in the slot of the main body, the shooting function can be used by the operation of the application program installed on the OS.

The imaging modules themselves that support such smart devices are also diverse. For example, the focal length of the optical lens and the size of the image sensor may be different as long as they satisfy certain standards such as means for attaching / detaching to / from the main body and means for communication. Further, the manufacturers that design these image pickup modules do not have to be limited to a specific company, and there may be a plurality of manufacturers such as a camera manufacturer and an electric appliance manufacturer. In addition, there are few restrictions on where and which component should be placed in the imaging module, and the degree of freedom in design is high. Therefore, the imaging module can be designed with an optimum layout that is convenient for each specification and manufacturer.

Further, as described above, since the combination of modules is relatively free, for example, a plurality of imaging modules can be mounted in different slots. In this case, by executing the plurality of imaging modules equipped with the application program for operating the compound eye camera function, the so-called image composition function and measurement function known as the compound eye camera function can be used.

Known image composition functions expected of a compound eye camera include a stereoscopic mode, a panoramic mode, a pan focus mode, a dynamic range expansion mode, a shallow focus mode, and a multi-zoom (high resolution) mode (for example, Patent Document 1). reference). These functions perform simultaneous shooting by matching or different shooting conditions in each imaging module, and synthesize one image from a plurality of obtained image data. In addition, a distance measuring technique can be mentioned as a measurement function using a compound eye camera (see, for example, Patent Document 2). This function captures a subject simultaneously with two left and right cameras and processes the obtained stereo image to three-dimensionally recognize the subject and measure the distance to the subject.

Since the image composition function and measurement function of such a compound eye camera are processed based on the parallax information of at least two imaging modules, it is important to obtain the distance between the optical axes of each imaging module, that is, the baseline length. .. Therefore, Patent Document 1 has a configuration in which the position and orientation of the imaging module are specified depending on which of the plurality of connecting portions provided on the main body side electrically detects the imaging module. At the same time, the baseline length can be obtained by determining from the detected signal which type of imaging module is connected among the two types of imaging modules.

On the other hand, in Patent Document 2, two cameras on the left and right are arranged in parallel at a predetermined interval, and if the baseline length shifts due to aging or the like in the process of use, a set of captured image data is stereo. It is controlled to be corrected by the image processing device. In this control, the images captured by the two cameras are geometrically transformed with conversion values set in advance according to the deviation between the images. At the same time, by performing linear interpolation on the converted image, it is possible to obtain an image equivalent to that in which the baseline length is correctly adjusted.

Japanese Patent No. 4448844 JP-A-3792832

By the way, in the configuration described in Patent Document 1, information such as the model of the imaging module and the position of the optical axis in the module must be registered in advance in the memory built in the main body side. However, in the above-mentioned smart device, the optimum position of the optical axis may be different in different types of imaging modules. Not only that, since the number of manufacturers is also unspecified, it is difficult to register all the information of the imaging module in the memory of the main body in advance. For example, if a user purchases another different type of imaging module for a smart device that he / she already owns, the information of the imaging module is newly added to the memory of the main unit by some means such as updating. The work of registering becomes necessary.

Further, the control described in Patent Document 2 is based on the assumption of a relatively small deviation caused by aging or the like. On the other hand, in the above-mentioned smart device, when the slot in which the image pickup module is mounted is changed, the position of the optical axis also moves accordingly, and the amount of change in the baseline length becomes relatively large. Therefore, it is not easy to detect a coincidence point from each image data and obtain correct parallax information. For example, if the amount of change in the baseline length exceeds the correctable range, the correct conversion process may not be performed and the reliability of the distance measurement result may be impaired. Even if the correctable range is expanded according to a relatively large amount of change, a large amount of memory capacity is required, and the calculation for detecting a coincidence point from two images becomes large and the calculation time is long. As a result, a time lag occurs between focusing and the start of exposure.

In view of these problems, the present invention presents an electronic device, an imaging device, and a control capable of always obtaining a correct baseline length even if a plurality of imaging modules are replaced with arbitrary slots at arbitrary timings. The purpose is to provide methods as well as programs.

The electronic device according to claim 1 of the present invention is an electronic device in which the first and second imaging modules are mounted in any of a plurality of mounting areas, respectively, and the first and second electronic devices mounted on the electronic device. and identification information for carrying out each imaging module, communicating with the main body of the electronic device has a memory for storing the coordinate information of the optical axis of each thereof imaging module, said first and second imaging module Acquisition means for acquiring coordinate information of the optical axes of the first and second imaging modules, respectively , a temperature sensor for measuring the temperature of the main body of the electronic device, and an imaging means for photographing in the compound eye mode. If, when a storage means for storing position information of the plurality of attachment regions, wherein the electronic device the first and second imaging module to the body of is mounted, and the electronic equipment measured by said temperature sensor It is characterized by including a calculation means for calculating the baseline length by acquiring the coordinate information of the optical axis and the position information of the attachment region when the change in the temperature of the main body of the main body exceeds the threshold value.

According to the present invention, even if a plurality of imaging modules are replaced with arbitrary slots at arbitrary timings, the correct baseline length can always be obtained.

FIG. 5 is an external view of a smart device as an electronic device according to the first embodiment. It is an external view of a module attached to the main body of a smart device. It is explanatory drawing which shows the method of attaching a module to the main body of a smart device. It is explanatory drawing which shows the coupling by the magnetic force of the EPM provided in the main body of a smart device, and the magnetic body provided in a module. It is a block diagram which shows the hardware composition of the main body of a plurality of modules and a smart device in which these are mounted. It is a flowchart which shows the procedure of the operation control processing of the smart device which a plurality of modules are attached, which is executed by the application program control module. It is a flowchart which shows the detailed procedure of the release process of step S1107 of FIG. It is a flowchart which shows the detailed procedure of the mounting process of step S1109 of FIG. It is a flowchart which shows the detailed procedure of the photographing application program execution process which is an example of the application program execution process of step S1111 of FIG. It is a flowchart which shows the procedure of the photographing execution processing of step S1405 of FIG. It is a flowchart which shows the procedure of the compound eye photography execution processing of step S1509 of FIG. It is a flowchart which shows the procedure of the distance measurement processing of step S1601 of FIG. It is explanatory drawing which showed the calculation method of the baseline length which concerns on Example 1 executed in step S1506 of FIG. It is explanatory drawing which shows the inclination of the optical axis with respect to the main body of the smart device of the image pickup module which concerns on Example 1. FIG. It is explanatory drawing which showed the calculation method of the baseline length which concerns on Example 2 executed in step S1506 of FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(Example 1)
FIG. 1 is an external view of a smart device 50 as an electronic device according to the first embodiment.

FIG. 1A is an external view of the main body of the smart device 50 as viewed from the front side and an external view as viewed from the back side.

As shown in FIG. 1A, a plurality of ribs 101a to 101c are formed on the front side of the main body of the smart device 50. Further, on the back side of the main body of the smart device 50, a plurality of ribs 101a, c to h are formed, and a spine 102 that divides the main body of the smart device 50 into left and right regions is formed. The ribs 101a to 102 and the spine 102 (guide portion) have both a guide function when mounting the module and a holding function for holding the module after mounting, and also have a function of increasing the rigidity of the main body of the smart device 50. is doing. Hereinafter, the ribs 101a to 101 and the spine 102 are collectively referred to as a frame structure. The front side and the back side of the main body of the smart device 50 are divided into mounting areas for a plurality of modules by ribs 101a to h and spines 102. Hereinafter, the mounting areas of these plurality of modules will be referred to as slots 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700, 1800, 1900.

Electropermanent magnets (EPM) 160 to 169, which control the electromagnetic attachment / detachment mechanism, are provided in the slots 1000 to 1900. The details of EPM 160 to 169 will be described later. In the vicinity of each EPM 160 to 169, there are main body side non-contact communication means (hereinafter referred to as main body side CMC) 140 to 149 for transmitting and receiving data between the main body of the smart device 50 and each module. It is equipped. That is, each of the slots 1000 to 1900 is provided with at least a pair of EPM 160 to 169 and CMC 140 to 149 on the main body side. As shown in FIG. 1A, a plurality of EPMs 160 to 169 and CMCs 140 to 149 on the main body side may be provided depending on the size of each slot 1000 to 1900.

On the front side of the main body of the smart device 50, EPM 160 and 163 are arranged near the end on the left side when facing the paper surface, and CMC 140 and 143 on the main body side are arranged on the right side of the EPM 160 and 163. EPM 161, 162a, 162b, 164a, 164b, 165a, 165b, 166 to 169 are arranged on the back side of the main body of the smart device 50 so as to be adjacent to the spine 102. EPM 165a, 165b, 167a, 167b, 168, 169 are provided in the area on the left side of the spine 102 when facing the paper surface, and CMC 145a, 145b, 147a, 147b, 148, 149 on the main body side are further arranged on the left side thereof. .. EPM 161, 162a, 162b, 164a, 164b, 166 are provided in the region on the right side of the spine 102, and CMC 141, 142a, 142b, 144a, 144b, 146 on the main body side are further arranged on the right side thereof.

FIG. 1B is an external view of the state in which the module is attached to the main body of the smart device 50 as viewed from the front side and an external view as viewed from the back side.

As shown in FIG. 1 (b), modules 150, 200, 300, 350, 400, 500, 600, 700, 800, 900 having various functions are provided on the front side and the back side of the main body of the smart device 50. It is attached. A module (hereinafter, referred to as a display operation module) 300 composed of an LCD panel 312 having a touch detection function on substantially the entire surface is mounted in a lower slot 1300 on the front side of the main body of the smart device 50. A power button 314a for switching the power ON / OFF of the smart device 50 is formed on the right side surface of the display operation module 300, and a volume control button 314b for adjusting the volume is also formed on the left side surface of the display operation module 300. Is formed. Further, the display operation module 300 is provided with a microphone 318 that detects the voice of the caller when the smart device 50 functions as a mobile wireless communication device. The microphone 318 also plays a role of collecting the sound of the moving image when the smart device 50 functions as a video camera. A speaker module 350 is attached to the upper slot 1000 on the front side of the main body of the smart device 50. The speaker module 350 is provided with a speaker unit 351 that outputs the received voice when the smart device 50 functions as a mobile wireless communication device. The speaker unit 351 also outputs music and operation sounds.

On the other hand, on the back side of the main body of the smart device 50, an image pickup module 500 having various shooting functions is placed in the upper slot 1500 on the left side of the spine 102, and an image pickup module 600 is placed in the upper slot 1600 on the right side of the spine 102. It is installed. When the slots in which the imaging modules 500 and 600 are mounted are substantially coplanar and the optical axes of the imaging modules 500 and 600 are substantially parallel when mounted in these slots, the compound eye mode described later can be selected by the user. Is set to. This makes it possible for the imaging modules 500 and 600 to frame the same subject in their respective shooting ranges, and to measure the subject distance using parallax, which will be described later, or to shoot at substantially the same time. Further, as described above, although the imaging modules 500 and 600 are standardized so that the means for attaching / detaching the smart device 50 to / from the main body and the communication means satisfy a certain standard, the components of each module are The arrangement is different.

A wireless LAN module 700 that wirelessly transmits and receives data to and from the outside is mounted in the slot 1700 formed below the upper slot 1500 on the left side of the spine 102. Further, a posture detection module 800 for detecting the posture of the smart device 50 is attached to the slot 1800 below the slot 1800. The attitude detection module 800 detects the attitude of the smart device 50 by using the angular velocity information acquired from the three-axis gyro sensor. A mobile communication module 900 having a single or a plurality of various long-distance communication functions such as TDMA, CDMA, and LTE is mounted in the lower slot 1900 on the left side of the spine 102. An application program control module 200 that controls the entire smart device 50 is mounted in the slot 1200 formed below the slot 1600 on the upper right side of the spine 102.

In order for the user to operate the function desired to be used in the smart device 50, it is necessary to install the application program dedicated to the predetermined module being installed in the application program control module 200. As a result, the desired function can be used in the smart device 50 via the application program control module 200. For example, when a call application dedicated to the mobile communication module 900 is installed, the mobile communication module 900 can be operated via the application program control module 200 to use the call function. In addition, an Internet connection application dedicated to the wireless LAN module 700 may be installed. In this case, the wireless LAN module 700 is operated via the application program control module 200 to enable the web browsing function via the Internet connection. Further, for example, when a photographing application dedicated to the imaging modules 500 and 600 is installed, the imaging modules 500 and 600 can be operated via the application program control module 200 to use the functions of the compound eye camera. Here, the function of the compound eye camera refers to an image composition function and a measurement function.

The image composition function expected of the compound eye camera includes the stereoscopic mode, panoramic mode, pan focus mode, dynamic range expansion mode, shallow focus mode, and multi-zoom (high resolution) mode described in Patent Document 1 described above. included. The dedicated photographing application can be arbitrarily set by each of the imaging modules 500 and 600. These compositing functions perform simultaneous shooting by matching or different shooting conditions of the imaging modules 500 and 600, and synthesize one image from the two obtained image data.

Further, in the above-mentioned photographing application, the distance-measuring technique described in the above-mentioned Patent Document 2 is applied as a measurement function. This function simultaneously captures a subject by the imaging modules 500 and 600 and processes the obtained stereo image to three-dimensionally recognize the subject and measure the distance to the subject. The specific operation flow for realizing the functions of such a compound eye camera will be described later with reference to FIGS. 11 and 12.

A power supply module 400 that supplies power to the smart device 50 is mounted in the slot 1400 below the slot 1200. Further, a recording module 150 for storing various data such as captured image data is attached to the lower slot 1100 on the right side of the spine 102.

FIG. 2 is an external view of modules 150, 200, 300, 350, 400, 500, 600, 700, 800, 900 attached to the main body of the smart device 50. FIG. 2A is an external view of the display operation module 300 and the speaker module 350 attached to the front side of the main body of the smart device 50 as viewed from the front side and an external view as viewed from the back side. FIG. 2B is an external view of each module attached to the back side of the main body of the smart device 50 as viewed from the front side and an external view as viewed from the back side. As described above, on the back side of the main body of the smart device 50, as shown in FIG. 1B, the image pickup module 500, the wireless LAN module 700, the attitude detection module 800, and the mobile communication module are on the left side of the spine 102. 900 is installed. Further, on the right side of the spine 102, an imaging module 600, an application program control module 200, a power supply module 400, and a recording module 150 are attached.

As shown in FIG. 2A, magnetic bodies 360, 356 are provided on the back surfaces of the display operation module 300 and the speaker module 350 at positions facing the EPM 163, 160 provided in the main body of the smart device 50. There is. As the material of the magnetic materials 360 and 356 used here, a soft magnetic material having a small coercive force and a large magnetic permeability is preferable, and in this embodiment, HIPERCOT M50, which is a soft magnetic alloy of iron, cobalt, and vanadium, is adopted. The same material is used for each of the magnetic materials described below.

Further, at a position facing the main body side CMC 143 and 140 provided on the main body of the smart device 50, a module side non-contact communication means (hereinafter, referred to as a module side CMC) 340 for transmitting and receiving data to and from the main body of the smart device 50. , 354 are provided. A pair of magnetic bodies 360 and 356 and module-side CMC 340 and 354 are provided adjacent to each other in the display operation module 300 and the speaker module 350, respectively.

On the other hand, as shown in FIG. 2B, magnetic bodies 560a and 560b are provided on the back surface of the image pickup module 500 at positions facing the EPM 165a and 165b provided in the main body of the smart device 50. Further, on the back surface of the image pickup module 600, a magnetic body 660a is provided at a position facing the EPM 166 provided in the main body of the smart device 50. A magnetic material 660b is also provided on the back surface of the image pickup module 600, but the main body of the smart device 50 does not have a magnetic material facing the magnetic material 660b. Therefore, the image pickup module 600 does not use the magnetic body 660b, but the magnetic body 660a is attached to the main body of the smart device 50 by being magnetically coupled to the EPM 166.

Similarly, on the back surface of the wireless LAN module 700, magnetic bodies 760a and 760b are provided at positions facing the EPM 167a and 167b provided in the main body of the smart device 50. Further, on the back surface of the posture detection module 800 and the mobile communication module 900, magnetic bodies 860 and 960 are provided at positions facing the EPM 168 and 169 provided in the main body of the smart device 50.

Further, on the back surface of the application program control module 200, magnetic bodies 260a and 260b are provided at positions facing the EPM 162a and 162b provided on the main body of the smart device 50. Further, on the back surface of the power supply module 400 and the recording module 150, magnetic materials 460a, 460b, 156a are provided at positions facing the EPMs, 164a, 164b, 161 provided in the main body of the smart device 50. .. A magnetic material 156b is also provided on the back surface of the recording module 150, but the main body of the smart device 50 does not have a magnetic material facing the magnetic material 156b. Therefore, the recording module 150 does not use the magnetic body 156b, but is mounted on the main body of the smart device 50 by magnetically binding the magnetic body 156a to the EPM 161.

Module side CMC540, 640, 740, 840, 940, 240a, 240b, 440a, 440b, 154 are provided at positions facing the main body side CMC 145b, 146, 147b, 148, 149, 142a, 142b, 144a, 144b, 141. .. Each module transmits / receives data to / from the main body of the smart device 50 by these module-side CMCs. The magnetic bodies 560b, 660a, 760b, 860, 960, 260a, 260b, 460a, 460b, 156a and the module side CMC 540, 640, 740, 840, 940, 240a, 240b, 440a, 440b, 154 are provided adjacent to each other. .. At least a pair is provided in the image pickup module 500, 600, the wireless LAN module 700, the posture detection module 800, the mobile communication module 900, and the recording module 150. Further, two pairs of the application program control module 200 and the power supply module 400 are provided.

FIG. 3 is an explanatory diagram showing a method of attaching modules 150, 200, 300, 350, 400, 500, 600, 700, 800, 900 to the main body of the smart device 50.

Here, FIG. 3A is an explanatory view showing a method of attaching the display operation module 300 and the speaker module 350 to the front side of the main body of the smart device 50. Further, FIG. 3B is an explanatory diagram showing a method of attaching the image pickup module 500, the wireless LAN module 700, the posture detection module 800, and the mobile communication module 900 to the back side of the main body of the smart device 50. Further, in FIG. 3B, the image pickup module 600, the application program control module 200, the power supply module 400, and the recording module 150 are also shown on the right side of the spine 102.

As shown in FIG. 3A, the display operation module 300 and the speaker module 350 are attached to the main body of the smart device 50 by sliding them from the side surface along the ribs 101a to 101c. At this time, the display operation module 300 and the speaker module 350 can be inserted from either the left side surface or the right side surface of the main body of the smart device 50.

As shown in FIG. 3B, the image pickup module 500, the wireless LAN module 700, the posture detection module 800, and the mobile communication module 900 are attached to the main body of the smart device 50 by sliding from the left side surface. By abutting each module against the spine 102 from the left side surface, the positions of the modules 500, 700, 800, 900 are determined with respect to the main body of the smart device 50. Further, the image pickup module 600, the application program control module 200, the power supply module 400, and the recording module 150 are attached to the main body of the smart device 50 by sliding from the right side surface. By abutting each module against the spine 102 from the right side, the positions of the modules 600, 200, 400, and 150 are determined with respect to the main body of the smart device 50.

In this embodiment, the slots provided on the back side of the main body of the smart device 50 as shown in FIG. 3B are roughly classified into three types according to the size. First, slots 1500, 1600, 1700, and 1100 are of the same type. For example, the imaging module 500 may be installed by selecting any of these four slots. In this embodiment, there are four slots into which the image pickup module 500 can be mounted, but the configuration is not limited to such a configuration as long as there are three or more slots. Further, the largest slots 1200 and 1400 are of the same type. For example, the application program control module 200 may be installed by selecting either of these two slots. Similarly, the smallest slots 1800 and 1900 are of the same type. For example, the posture detection module 800 may be installed by selecting either of these two slots.

FIG. 4 is a schematic diagram illustrating a magnetic force coupling between the EPM 165b provided in the main body of the smart device 50 and the magnetic body 560b provided in the imaging module 500.

Here, FIG. 4A is a partially enlarged view of a state in which the main body of the smart device 50 and the image pickup module 500 are not coupled by a magnetic force. Further, FIG. 4B is a partially enlarged view of a state in which the main body of the smart device 50 and the image pickup module 500 are coupled by a magnetic force. Although FIG. 4 shows a combination of the EPM 165b and the magnetic material 560b as an example, the combination of another EPM and the magnetic material is the same as that of FIG.

As shown in FIG. 4A, the EPM 165b has a structure in which both side surfaces of a permanent magnet 1651 having a fixed polarity and a permanent electromagnet 1652 are connected and held by magnetic bodies 1653a and 1653b. As the permanent magnet 1651 used here, for example, a neodymium magnet having a very high magnetic flux density is suitable. Further, the permanent electromagnet 1652 is composed of a reversible permanent magnet 1654 made of a hard magnetic material such as alnico, and a coil 1655 wound around the reversible permanent magnet 1654. When a current is passed through the coil 1655, the reversible permanent magnet 1654 is magnetized in one direction, and the magnetized state is maintained as it is even after the energization is completed. The energization time for the coil 1655 is about 1 to several seconds, which is a relatively short time. In this way, the permanent electromagnet 1652 becomes a permanent electromagnet having a variable polarity by changing the direction of the current flowing through the coil 1655 (polarity changing means).

When the coil 1655 is energized in the state shown in FIG. 4 (a), the reversible permanent magnet 1654 is magnetized, and the permanent electromagnet 1652 attracts the magnetic field lines of the permanent magnet 1651 whose polarity is fixed. To generate. As a result, the magnetic field lines of the permanent magnet 1652 and the magnetic field lines of the permanent magnet 1651 form a loop shape in which the magnetic field lines of the permanent magnet 1651 are closed to each other, and the magnetic force for attracting the magnetic body 560b of the imaging module 500 becomes very weak. Therefore, the image pickup module 500 is released from the EPM 165b without receiving an attractive force.

On the other hand, as shown in FIG. 4 (b), when the coil 1655 was energized in the direction opposite to that in FIG. 4 (a), the reversible permanent magnet 1654 was magnetized, and the polarity of the permanent electromagnet 1652 was fixed. A magnetic field line in a direction that repels the direction of the magnetic field line of the permanent magnet 1651 is generated. As a result, the magnetic field lines of the permanent magnet 1652 and the magnetic field lines of the permanent magnet 1651 are strengthened against each other, and the magnetic force that attracts the magnetic body 560b provided in the image pickup module 500 is greatly increased. Therefore, the image pickup module 500 receives the suction force from the EPM 165b and is fixed to the main body of the smart device 50. As described above, in this embodiment, by adopting EPM as the attachment / detachment means, both workability and reliability of attachment / detachment of each module are realized.

FIG. 5 is a block diagram showing a hardware configuration of a plurality of modules 200 to 600, 800 and a smart device 50 to which these modules are mounted.

Hereinafter, the configurations of the main body of the smart device 50, the application program control module 200, the display operation module 300, the power supply module 400, the image pickup modules 500 and 600, and the posture detection module 800 will be described with reference to FIG. There are a wide variety of modules that can be attached to the main body of the smart device 50, and the combinations shown in FIG. 5 are merely examples, and the present invention does not limit the combinations.

<Structure of the main body of the smart device 50>
The main body of the smart device 50 controls each module mounted on the main body of the smart device 50 under the integrated control by the application program control module 200. In the main body of the smart device 50, 110 is a system control circuit that controls the entire main body of the smart device 50. When executing various application programs in an environment in which a kernel or an OS is executed, the system control circuit 110 performs a cooperative operation in response to an instruction or request of the application control circuit 210 included in the application program control module 200. The system control circuit 110 can be operated in cooperation with the main body of the smart device 50 and each module, and various services and functions can be executed via the application control circuit 210.

The 112 is a memory that the system control circuit 110 directly accesses to read and write. Reference numeral 114 denotes a non-volatile memory that stores constants, variables, programs, position information of each slot, and the like for the operation of the system control circuit 110 and can be electrically erased and recorded. For example, a flash memory or the like is used. Here, the position information of each slot includes the position information of the slots 1100, 1200, 1400, 1500, 1600, 1700, 1800, and 1900 provided on the back side of the main body of the smart device 50. The position information of each slot specifies the coordinates of the abutting surfaces of the ribs 101a, c to h and the spine 102, which determine the position when the module is mounted in each slot. In this embodiment, each module is positioned by abutting the ribs 101a, c to h and the spine 102, but the present invention is not limited thereto. For example, the main body of the smart device 50 may be provided with a convex portion for positioning, and each module may be provided with a concave portion that fits with the convex portion. In this case, the position information of each slot includes the position information of the convex portion for positioning.

Reference numeral 116 denotes an identification information memory, which stores various identification information necessary for the main body of the smart device 50 to communicate with each module. Reference numeral 118 denotes a single or a plurality of temperature sensors for measuring the temperature of a predetermined portion of the main body of the smart device 50. Reference numeral 120 denotes a power supply control circuit that supplies a predetermined voltage and current required for each part of the main body of the smart device 50 via the system control circuit 110.

Reference numeral 122 denotes a power bus connected to the power control circuit 120 of the main body of the smart device 50 and the power terminals of the connectors 182 to 186, 188. The power supply terminals of the connectors 182 to 186, 188 are the power supply terminals of the power supply control circuits 220, 320, 410, 520, 620 of each module via the power supply terminals of the connectors 280, 380, 480, 580, 680, 880 of each module, respectively. , 820 is connected.

Reference numeral 130 denotes a switch interface circuit, which is connected to each module via main body side CMC 142a, 142b (not shown), 143, 144a, 144b (not shown), 145b, 146, 148 as shown in FIG. As a result, high-speed communication of data and messages is switched and relayed between each module and the main body of the smart device 50. The main body side CMC142a, 143, 144a, 145b, 146, 148 perform inductively coupled (inductively coupled) contact proximity communication. As a result, high-speed communication is performed with the module-side CMC240a, 340, 440a, 540, 640, 840, which are close to each other. The combination of the main body side CMC and the module side CMC close to the main body side CMC is appropriately changed according to the intention of the user, and the combination shown in FIG. 5 is only an example.

As shown in FIG. 5, the smart device 50 includes EPM 162a, 162b (not shown), 163, 164a, 164b (not shown), 165a, 165b (not shown), 166, 168. This attracts or does not adsorb the magnetic bodies 260a, 260b (not shown), 360, 460a, 460b (not shown), 560a, 560b (not shown), 660a, 860 of the module, respectively, by magnetic force control. As a result, each module is fixed (locked) or released (released) at the connection point between the frame structure of the main body of the smart device 50 and each module. The combination of each EPM on the main body side of the smart device 50 and the magnetic material on the module side connected to the EPM is appropriately changed according to the intention of the user, and the combination shown in FIG. 5 is only an example. do not have.

The connectors 182 to 186 and 188 are connected to the modules' connectors 280, 380, 480, 580, 680 and 880, respectively. As a result, the terminals related to the power supply (power bus, ground) can be mutually used between the main body of the smart device 50 and each module. Furthermore, each function such as the detection (Detect) signal terminal indicating the installation of the module, the activation (Wake) signal terminal indicating the wakeup of the module, and the RF signal terminal connecting the antenna wiring are also used mutually. It is possible. Here, the connectors 182 to 186, 188 in this embodiment are general small metal terminals formed on the ribs 101a to h of the main body of the smart device 50 and the side surface portions of the spine 102. It is not shown because it cannot be seen from the indicated position. The combination of the connectors 182 to 186, 188 and the connector on the module side connected to the connector 182 to 186, 188 is appropriately changed according to the intention of the user, and the combination shown in FIG. 5 is merely an example.

<Configuration of application program control module 200>
The application program control module 200 controls the entire system including the main body of the smart device 50 and each module mounted on the smart device 50 by the operation of the application control circuit 210. For example, the application control circuit 210 can control the LCD panel 312, which is a display unit, to display various information via the display operation control circuit 310 included in the display operation module 300. Further, the application control circuit 210 can acquire operation input information for the touch panel which is an operation input means and the operation (hereinafter referred to as “TP / button”) button 314 via the display operation control circuit 310 included in the display operation module 300. can. Then, it is possible to execute the processing by the kernel service, the OS service, and various application programs according to the operation input contents.

The 212 is a memory that the application control circuit 210 directly accesses to read and write. Reference numeral 214 denotes a non-volatile memory that stores constants, variables, programs, and the like for the operation of the application control circuit 210 and can be electrically erased and recorded. For example, a flash memory or the like is used. Reference numeral 216 is an identification information memory, which stores various identification information necessary for the application program control module 200 to communicate with the main body of the smart device 50 and each module. Reference numeral 220 denotes a power supply control circuit that supplies a predetermined voltage and current required for each part of the application program control module 200. Reference numeral 222 denotes a single or a plurality of temperature sensors for measuring the temperature at a predetermined position of the application program control module 200. Reference numeral 230 denotes an interface circuit, which relays high-speed communication of data and messages with the main body of the smart device 50 and each module via the module-side CMC240a.

Reference numeral 290 is a management table that stores information of a plurality of management files required for executing each dedicated application program. The information in the management file includes the types of modules that are indispensable when executing each dedicated application program, the combination of applicable modules that can make the best use of the desired functions, and the optimum slot for installing the applicable modules. The positional relationship is included. In addition, this embodiment includes the types of modules that are effective for adding functions, etc., although it is not essential for each dedicated application program, as the information of the management file, and enhances convenience by providing many choices to the user. ing. The information of such a management file is acquired by the application control circuit 210 from the management table 290. The present invention is not limited to this configuration, and the information of the management file may be stored in the memory 212 or the non-volatile memory 214. The management file information in this case is acquired by the application control circuit 210 from the memory 212 and the non-volatile memory 214.

<Configuration of display operation module 300>
The display operation module 300 displays various information and acquires operation inputs under the overall control of the application program control module 200 under the control of the main body of the smart device 50. In the display operation module 300, 310 is a display operation control circuit that controls the entire display operation module 300. As the display unit of the display operation module 300, a display device such as an LCD, an OLED, or an LED can be adopted, but in this embodiment, an LCD panel 312 is adopted. As the operation input means of the display operation module 300, the operation devices such as the touch panel (TP) and the operation buttons may be configured independently or integrally, but in this embodiment, they are configured independently. The TP / button 314 is adopted.

The LCD panel 312 displays various information to the user by the display operation control circuit 310 in response to the instruction of the application control circuit 210 of the application program control module 200. Further, the input operation such as the touch panel operation or the button operation by the user to the TP / button 314 and the audio signal detected by the microphone 318 are transmitted to the application control circuit 210 via the display operation control circuit 310.

Reference numeral 316 is an identification information memory, which stores various identification information necessary for the display operation module 300 to communicate with the main body of the smart device 50 and each module. Reference numeral 320 denotes a power supply control circuit that supplies a predetermined voltage and current required for each part of the display operation module 300. Reference numeral 322 denotes a single or a plurality of temperature sensors for measuring the temperature of a predetermined position of the display operation module 300. Reference numeral 330 denotes an interface circuit, which relays high-speed communication of data and messages with the main body of the smart device 50 and each module via the module-side CMC 340.

<Configuration of power supply module 400>
The power supply module 400 discharges and charges the battery 420 via the power supply bus 122 of the main body of the smart device 50 under the integrated control by the application program control module 200. In the power supply module 400, 410 is a battery control circuit that controls the entire power supply module 400 including discharge / charge control of the battery 420, and supplies a predetermined voltage / current required for each part of the power supply module 400. Reference numeral 416 denotes an identification information memory, which stores various identification information necessary for the power supply module 400 to communicate with the main body of the smart device 50 and each module.

The battery 420 corresponds to a Li-ion battery, a fuel cell, or the like. The battery 420 is discharged from the main body of the smart device 50 and each module by the battery control circuit 410 via the connector 480, and is charged from the main body of the smart device 50 and a charging module (not shown). Reference numeral 422 is a single or a plurality of temperature sensors for measuring the temperature at a predetermined position of the power supply module 400. Reference numeral 430 is an interface circuit, which relays high-speed communication of data and messages with the main body of the smart device 50 and each module via the module-side CMC440a.

<Configuration of imaging module 500>
The image pickup module 500 is an image pickup device as a module that is controlled by the main body of the smart device 50 and performs a desired image pickup process under the integrated control by the application program control module 200. In the image pickup module 500, 510 is a camera in which a plurality of optical lenses are arranged on the optical axis. Further, the camera 510 includes an aperture mechanism that adjusts the amount of light passing through, an AF mechanism that adjusts the focus by moving at least one optical lens in the optical axis direction, and a lens barrel that houses these components inside. It is composed of. Further, the camera 510 includes an image sensor that obtains image data by photoelectric conversion, an image processing circuit that processes the image data, and a drive control circuit that controls each mechanism.

The camera 510 automatically adjusts the aperture, shutter speed, and sensitivity of the image sensor to the optimum (AE), automatically adjusts the focus according to the subject distance (AF), and adjusts the color temperature to reproduce the appropriate color tone. Realizes control such as white balance (AWB). In addition, in this embodiment, camera shake is calculated from the angular velocity information acquired by the attitude detection module 800, and the camera shake correction (IS) is simply performed by following the exposure range cut out on the image sensor. Is possible. It should be noted that the present invention does not limit the general control method of such an image pickup apparatus, and since these are already known from the prior art documents and the like, detailed individual description thereof will be omitted.

The instruction to the camera 510 is given in response to an application program executed by the application control circuit 210 or an input to the TP / button 314 of the display operation module 300. The image data acquired by the camera 510 can be displayed on the LCD panel 312 by the application control circuit 210 controlling the main body of the smart device 50 and the display operation module 300.

Reference numeral 516 is an identification information memory, which stores various identification information necessary for the imaging module 500 to communicate with the main body of the smart device 50 and each module. Reference numeral 520 is a power supply control circuit that supplies a predetermined voltage and current required for each part of the image pickup module 500.

The 522 is a non-volatile memory that stores constants, variables, position information of components, optical axis error information, lens error information, and the like for the operation of the camera 510, and can be electrically erased and recorded. For example, a flash. Memory or the like is used. The position information of the component component referred to here includes the coordinate information of the optical axis seen from the outer shape of the image pickup module 500. As described above, the position of the image pickup module 500 is determined by abutting the outer shape of the image pickup module 500 against the main body of the smart device 50. In this embodiment, the imaging module 500 is positioned by abutting the ribs 101a to 101a of the main body of the smart device 50 and the spine 102, but the present invention is not limited thereto. For example, the main body of the smart device 50 may be provided with a convex portion for positioning, and the image pickup module 500 may be provided with a concave portion that fits with the convex portion. In this case, the position information of the component includes the coordinate information of the optical axis seen from the concave portion that fits with the convex portion.

The optical axis error information includes, for example, an error in the inclination of the optical axis of the camera 510, which occurs as a manufacturing error due to the accuracy of parts and assembly, which will be described in detail later. On the other hand, the error information of the lens includes, for example, an error of the focal length, an error of the F value, a distortion, and an angle error of the image sensor centered on the optical axis, which occur as a manufacturing error. By storing the information regarding such manufacturing errors in the non-volatile memory 522, it is possible to correct these errors in the processing of the application control circuit 210. As a result, the accuracy of the image composition function and the measurement function can be improved when the function of the compound eye camera described later is used.

Reference numeral 530 is an interface circuit, which relays high-speed communication of data and messages with the main body of the smart device 50 and each module via the module-side CMC 540.

<Configuration of imaging module 600>
The image pickup module 600 is an image pickup device as a module that is controlled by the main body of the smart device 50 and performs a desired image pickup process under the integrated control by the application program control module 200. In the image pickup module 600, 610 is a camera in which a plurality of optical lenses are arranged on the optical axis. Further, the camera 610 includes an aperture mechanism that adjusts the amount of light passing through, an AF mechanism that adjusts the focus by moving at least one optical lens in the optical axis direction, and a lens barrel that houses these components inside. It is composed of. Further, the camera 610 includes an image sensor that obtains image data by photoelectric conversion, an image processing circuit that processes the image data, and a drive control circuit that controls each mechanism. The components of such a camera 610 are the same as those of the above-mentioned camera 510. However, the arrangement and shape of the camera 610 in the image pickup module 600 are different from the arrangement and shape of the camera 510 in the image pickup module 500.

The camera 610 realizes the same control as the camera 510. Specifically, automatic exposure adjustment (AE) that optimally sets the aperture, shutter speed, and sensitivity of the image sensor, automatic focus adjustment (AF) according to the subject distance, and color temperature adjustment to reproduce the appropriate color tone. Realizes control such as automatic white balance (AWB). In addition, in this embodiment, camera shake is calculated from the angular velocity information acquired by the attitude detection module 800, and the camera shake correction (IS) is simply performed by following the exposure range cut out on the image sensor. Is possible. It should be noted that the present invention does not limit the general control method of such an image pickup apparatus, and since these are already known from the prior art documents and the like, detailed individual description thereof will be omitted.

The instruction to the camera 610 is given in response to an application program executed by the application control circuit 210 or an input to the TP / button 314 of the display operation module 300. The image data acquired by the camera 610 can be displayed on the LCD panel 312 by the application control circuit 210 controlling the main body of the smart device 50 and the display operation module 300.

Reference numeral 616 is an identification information memory, which stores various identification information necessary for the image pickup module 600 to communicate with the main body of the smart device 50 and each module. 620 is a power supply control circuit that supplies a predetermined voltage and current required for each part of the image pickup module 600.

The 622 is a non-volatile memory that stores constants, variables, position information of components, optical axis error information, lens error information, etc. for the operation of the camera 610, and can be electrically erased and recorded. For example, a flash. Memory or the like is used. The position information of the component component referred to here includes the coordinate information of the optical axis seen from the outer shape of the image pickup module 600. As described above, the position of the image pickup module 600 is determined by abutting the outer shape of the image pickup module 600 against the main body of the smart device 50. In this embodiment, the imaging module 600 is positioned by abutting the ribs 101a to 101a of the main body of the smart device 50 and the spine 102, but the present invention is not limited thereto. For example, the main body of the smart device 50 may be provided with a convex portion for positioning, and the image pickup module 600 may be provided with a concave portion that fits with the convex portion. In this case, the position information of the component includes the coordinate information of the optical axis seen from the concave portion that fits with the convex portion.

The optical axis error information includes, for example, an error in the inclination of the optical axis of the camera 610, which occurs as a manufacturing error due to the accuracy of parts and assembly, which will be described in detail later. On the other hand, the error information of the lens includes, for example, an error of the focal length, an error of the F value, a distortion, and an angle error of the image sensor centered on the optical axis, which occur as a manufacturing error. By storing the information regarding such manufacturing errors in the non-volatile memory 622, it is possible to correct these errors in the processing of the application control circuit 210. As a result, the accuracy of the image composition function and the measurement function can be improved when the function of the compound eye camera described later is used.

Reference numeral 630 is an interface circuit, which relays high-speed communication of data and messages with the main body of the smart device 50 and each module via the module-side CMC 640.

<Configuration of posture detection module 800>
The posture detection module 800 is controlled by the main body of the smart device 50 under the integrated control by the application program control module 200, and detects the posture of the smart device 50. In the attitude detection module 800, 810 is a gyro sensor that acquires angular velocity information from a three-axis gyro sensor. Reference numeral 816 is an identification information memory, which stores various identification information necessary for the posture detection module 800 to communicate with the main body of the smart device 50 and each module. Reference numeral 820 denotes a power supply control circuit that supplies a predetermined voltage and current required for each part of the attitude detection module 800. Reference numeral 822 is a single or a plurality of temperature sensors for measuring the temperature of a predetermined position of the posture detection module 800.

Reference numeral 830 is an interface circuit, which relays high-speed communication of data and messages with the main body of the smart device 50 and each module via the module-side CMC 840. The interface circuit 830 transmits the angular velocity information acquired by the gyro sensor 810 to the main body of the smart device 50. From there, the main body of the smart device 50 further transfers data to the application program control module 200 at high speed. In this way, the angular velocity information of the attitude detection module 800 is used for switching the display direction in the display operation module 300, for camera shake correction for the image pickup modules 500 and 600, and the like.

<Operation description of application program control module 200>
FIG. 6 is a flowchart showing a procedure of operation control processing of the smart device 50 equipped with a plurality of modules, which is executed by the application program control module 200.

The process of FIG. 6 starts when the application program control module 200, the main body of the smart device 50, and the display operation module 300 are in a low power consumption state, and there is a user operation on the power button 314a of the display operation module 300. do.

When there is such a user operation, the display operation control circuit 310 transmits a start (Wake) signal indicating wakeup to the application control circuit 210. Upon receiving the start (Wake) signal from the display operation control circuit 310, the application control circuit 210 executes the initial setting in step S1100. Further, in this embodiment, terminals for detection signals are provided in the connector portions of all the modules mounted on the main body of the smart device 50. As a result, for example, when a new module is installed in an empty slot, a detect signal is transmitted and the process proceeds to step S1100.

In step S1100, the application control circuit 210 resets and initializes predetermined flags, control variables, and the like, and initializes each part of the application program control module 200. Subsequently, the application control circuit 210 executes the software program read from the non-volatile memory 214 to sequentially start the kernel and the OS. After that, communication with the system control circuit 110 of the main body of the smart device 50 is initialized via the interface circuit 230, the module side CMC240a, the main body side CMC142a, and the switch interface circuit 130. By initializing the system control circuit 110, all the modules mounted on the main body of the smart device 50 are in an operable state. As a result, for example, in the display operation module 300, the display operation control circuit 310 causes the LCD panel 312, which is a display unit, to display a predetermined start-up screen. Then, the display operation module 300 reaches a state in which a user can give an input instruction to the TP / button 314 which is an operation input means.

When step S1100 is completed, the process proceeds to step S1101, and the application control circuit 210 determines whether or not the end message has been received in step S1101. This end message is transmitted to the application control circuit 210 in the display operation control circuit 310 in the following form. First, the end button is displayed on the LCD panel 312, which is a display unit, and the end button can be selected by the user with the TP / button 314. After that, when the end button is selected by the user, the display operation control circuit 310 transmits an end message to the application control circuit 210.

If it is determined that the end message has been received in step S1101, the process proceeds to step S1120. In step S1120, the application control circuit 210 performs termination processing. Specifically, the application control circuit 210 transmits a termination message to the system control circuit 110, and then saves flags, control variables, and the like to the non-volatile memory 214 as needed. At the same time, the OS and kernel are shifted to the end-of-operation state in which they operate with low power consumption. Then, the power supply to the application program control module 200, the main body of the smart device 50, and the display operation module 300 via the power supply control circuit 220 is changed to a low power consumption setting. Upon receiving the termination message, the system control circuit 110 performs a process of stopping all operations of the modules other than the application program control module 200, the main body of the smart device 50, and the display operation module 300.

After finishing the termination process of step S1120, the application control circuit 210 ends this process and reaches a so-called power-off state.

If the end message is not received in step S1101, the process proceeds to step S1102. In step S1102, the application control circuit 210 determines whether or not a sleep message for transitioning to the sleep state has been received from the display operation control circuit 310 of the display operation module 300. This sleep message is transmitted to the application control circuit 210 in the display operation control circuit 310 in the following form. First, the sleep button is displayed on the LCD panel 312, which is a display unit, and the sleep button can be selected by the user with the TP / button 314. After that, when the sleep button is selected by the user, the display operation control circuit 310 transmits a sleep message for transitioning to the sleep state to the application control circuit 210.

If it is determined in step S1102 that the sleep message for transitioning to the sleep state has been received, the process proceeds to step S1103. In step S1103, the application control circuit 210 performs sleep processing. Specifically, the application control circuit 210 transmits a sleep message to the system control circuit 110, and then saves flags, control variables, and the like to the non-volatile memory 214 as needed. At the same time, the OS and kernel are shifted to the sleep operation state in which they operate with low power consumption. Then, when the system control circuit 110 receives the sleep message, the system control circuit 110 performs a process of shifting the operation of all the modules of the smart device 50 to the sleep state, and then proceeds to step S1104.

When the initial settings are made in step S1100, the LCD panel 312 of the display operation module 300 displays the above-mentioned end button and sleep button, as well as the release button and application execution button described later. None of these buttons may be selected by the user until a predetermined time has elapsed after being displayed on the LCD panel 312. Further, when the start (Wake) signal transmitted from the display operation control circuit 310 is not received even after a lapse of a predetermined time after the user operates the power button 314a of the display operation module 300 at the start of this process. There is. In such a case, the process proceeds to step S1103 in the same manner as when the sleep message is received. Further, in step S1102, the application control circuit 210 integrates the time elapsed from the timing when the input instruction or the activation (Wake) signal by the process described later is last received. As a result of comparing this integrated time with a predetermined value, if the integrated time is longer, the sleep state is entered. The subsequent sleep processing is as described above.

In step S1104, the application control circuit 210 determines whether or not a start (Wake) signal transmitted from each module has been received via the connector 280. If the start (Wake) signal is not received in step S1104, the sleep operation state is continued until the start (Wake) signal is received. Here, the sleep operation state in this embodiment is different from the power-off state described above. For example, when the mobile communication module 900 receives a call signal conforming to the mobile communication standard, the application control circuit 210 immediately shifts the smart device 50 from the sleep operation state to the predetermined application execution state. Since the general control of such a mobile wireless communication system is already known, detailed description thereof will be omitted.

If the start (Wake) signal is received in step S1104, the process proceeds to step S1105. In step S1105, the application control circuit 210 returns flags, control variables, and the like from the non-volatile memory 214 as needed. At the same time, the return process is performed to shift the OS and kernel to the normal operating state in which the OS and the kernel operate at the normal power consumption, and change the power supply to all the modules of the smart device 50 via the power control circuit 220 to the normal power consumption setting. conduct. Further, in step S1105, the application control circuit 210 restores communication with the system control circuit 110 of the main body of the smart device 50. At this time, the system control circuit 110 performs a return process for all the modules other than the application control circuit 210, shifts the smart device 50 to the normal operating state, and returns to step S1101.

If the sleep message for transitioning to the sleep state is not received in step S1102, the process proceeds to step S1106. In step S1106, the application control circuit 210 determines whether or not a release message has been received from the display operation control circuit 310 of the display operation module 300. This release message is transmitted to the application control circuit 210 in the display operation control circuit 310 in the following form. First, the display operation control circuit 310 displays a release button on the LCD panel 312, which is a display unit, and makes the release button user-selectable by the TP / button 314. After that, when this release button is selected by the user and a user instruction as to which module is to be removed is input, the display operation control circuit 310 transmits a release message for shifting to the release state to the application control circuit 210.

If it is determined in step S1106 that the release message for shifting to the release state has been received, the process proceeds to step S1107. In step S1107, the application control circuit 210 executes a release process for normally terminating the function and releasing the EPM for the module intended to be removed by the user. Details of the release process will be described later with reference to FIG. 7. When step S1107 is completed, the process returns to step S1101.

If the release message for shifting to the release state is not received in step S1106, the process proceeds to step S1108. In step S1108, the application control circuit 210 determines whether or not a detect signal for each module has been received. Here, the detect signal is a signal for detecting that a module is newly attached to the main body of the smart device 50. Further, this detection signal is an electric signal transmitted from the newly mounted module to the application control circuit 210 via the system control circuit 110.

If the detect signal is received in step S1108, the process proceeds to step S1109. In step S1109, the application control circuit 210 executes a module setting process for fixing the corresponding module inserted in the main body of the smart device 50 and making it function appropriately. The details of the module setting process will be described later with reference to FIG. When step S1109 is completed, the process returns to step S1101.

If it is determined in step S1108 that the detect signal has not been received, the process proceeds to step S1110. In step S1110, the application control circuit 210 determines whether or not the application program-related message has been received from the display operation control circuit 310 of the display operation module 300. This application program-related message is transmitted to the application control circuit 210 in the display operation control circuit 310 in the following form. First, the display operation control circuit 310 displays an application execution button on the LCD panel 312, which is a display unit, and makes the application execution button user-selectable with the TP / button 314. After that, when the application execution button is selected by the user and the user inputs which application program to execute, the display operation control circuit 310 transmits an application program-related message to the application control circuit 210.

If it is determined that the application program-related message has been received in step S1110, the process proceeds to step S1111, and the application control circuit 210 executes the application program execution process in step S1111. The application program assumed in this embodiment includes various functions that can be realized by combining each module. For example, the combination of the mobile communication module 900 and the display operation module 300 enables a call function, and the combination of the wireless LAN module 700 and the display operation module 300 enables browsing of the web via an Internet connection. Further, for example, a general photographing function is realized only by the image pickup module 500, and by combining the image pickup module 600 with the image pickup module 600, an image composition function and a measurement function, which are functions of a compound eye camera, are realized. Details of the shooting application execution process, which is an example of such an application program, will be described later with reference to FIG. When step S1111 is completed, the process returns to step S1101.

If it is determined in step S1110 that the application program-related message has not been received, the process returns to step S1101.

FIG. 7 is a flowchart showing a detailed procedure of the release process of step S1107 of FIG.

In FIG. 7, first, in step S1201, the application control circuit 210 transmits a message instructing the system control circuit 110 to end the function of the module (hereinafter referred to as “release target module”) for which the removal instruction has been received by the user. Next, the process proceeds to step S1202, and the application control circuit 210 determines whether or not the information update message transmitted from the system control circuit 110 notifying that the module information of the release target module has been updated has been received.

If it is determined in step S1202 that the information update message has not been received, the application control circuit 210 performs a predetermined error processing in step S1203. After that, by controlling the EPM in step S1205, the locked state of the release target module is released and this process ends. In the error processing in step S1203, the error content may be displayed on the display operation module 300 or the like to notify the user.

If it is determined that the information update message has been received in step S1202, the process proceeds to step S1204. In step S1204, the application control circuit 210 updates the management information stored in the predetermined areas of the non-volatile memory 214 and the memory 212 managed by the OS and the kernel according to the content of the received information update message. The management information referred to here includes module management information, EPM control management information, and RF bus configuration management information. After that, by controlling the EPM in step S1215, the locked state of the release target module is released and this process ends.

FIG. 8 is a flowchart showing a detailed procedure of the mounting process of step S1109 of FIG.

In FIG. 8, first, in step S1301, the application control circuit 210 sets up a connection for message communication together with the system control circuit 110, and establishes a network link with the system control circuit 110. Next, the process proceeds to step S1302, and the application control circuit 210 acquires module information such as initialization from a module (hereinafter referred to as “mounting module”) mounted on the main body of the smart device 50 via the system control circuit 110. Further, the process proceeds to step S1303, and the application control circuit 210 verifies whether or not the module information acquired in step S1302 has no problem in the smart device 50. For example, is it possible to perform stable communication, is it possible to operate with the voltage of the power supply module 400 already installed, and if there is another standard set individually for the smart device 50, be satisfied with it. Whether or not it is verified.

If there is a problem in the result verified in step S1303, the application control circuit 210 finishes this process for the mounting module after performing a predetermined error process in step S1304. In the error processing, the error content or the like may be displayed on the display operation module 300 or the like to notify the user.

On the other hand, if there is no problem in the result of verification in step S1303, it is determined that the mounting module is normal, and the process proceeds to step S1305. In step S1305, the application control circuit 210 updates the management information stored in the predetermined areas of the non-volatile memory 214 and the memory 212 based on the module information of the mounting module to be initialized. The management information referred to here includes module management information, EPM control management information, and RF bus configuration management information.

Next, in step S1306, the application control circuit 210 transmits an EPM lock instruction message of the mounting module to be initialized to the system control circuit 110. As a result, the main body of the smart device 50 and the mounting module for initialization are fixed by the EPM and locked.

Subsequently, in step S1307, the application control circuit 210 transmits a communication start instruction message to the system control circuit 110, and notifies that message communication with the mounting module that has undergone a series of initialization processes has become possible. .. After that, in step S1308, the state change flag is switched to ON, and this process ends. The state change flag referred to here is a flag assigned to each module and held in the non-volatile memory 214, and is a reprocessing flag that can be switched between ON and OFF depending on the presence or absence of a state change of each module. The state change flag is turned ON when the state of each module mounted on the main body of the smart device 50 changes while the state change flag of each module is OFF. In addition to mounting each module, changes in the state include changes in the remaining battery level, failures, deterioration of performance, and the like. Further, when the temperature change measured by the temperature sensor 118 in the main body of the smart device 50 exceeds a certain threshold value, or when the humidity change measured by the humidity sensor (not shown in FIG. 5) exceeds a certain threshold value. Turns on the state change flags of all modules.

As a result of this processing, when the mounting module is normal (YES in step S1303) and the mounting module is locked (step S1306), the function of the mounting module becomes available in the smart device 50.

The mounting process of the present invention is not limited to the procedure shown in FIG. For example, in order to stabilize each communication, the mounting module may be locked by EPM before step S1301, and in that case, unlocking is performed after step S1304.

FIG. 9 is a flowchart showing an operation flow of the photographing application execution process, which is an example of the application program execution process of step S1111 of FIG.

In FIG. 9, when the shooting application is first started by the operation input of the display operation module 300 in step S1401, the application control circuit 210 acquires the information of the management file of the shooting application from the management table 290. This information includes the types of modules that are essential for running shooting applications, the combination of applicable modules that maximizes the shooting function, and the optimal positional relationship of each slot for mounting the relevant module. ..

Next, the process proceeds to step S1402, and the application control circuit 210 acquires module information from each module via the system control circuit 110. Then, in step S1403, based on the information in the management file of the imaging application, it is verified whether the necessary modules are installed and whether there is a problem in the combination thereof.

If there is a problem in the result verified in step S1403, the application control circuit 210 ends the main shooting application execution process after performing a predetermined error process in step S1404. In the error processing, the error content or the like may be displayed on the display operation module 300 or the like to notify the user. For example, if the imaging module is not installed in any of the slots at the time of step S1402 and the imaging application cannot be executed, the error content is notified in the error processing of step S1404, and the main shooting application execution process is terminated. In addition, even though the mounted image pickup module 500 has a camera shake correction (IS) function, camera shake may not be detected because, for example, the posture detection module 800 is not mounted. In this case, a part of the shooting function is restricted in the error processing in step S1404. In this case, it is not necessary to end the main shooting application execution process, and if there is a user operation input or the like in response to the notification of the error content, the shooting execution process in step S1405 may be performed depending on the situation.

If there is no problem in the result verified in step S1403, the process proceeds to step S1405 to perform the shooting execution process. After that, the application control circuit 210 stops the operation of each module required for the imaging application, and ends this process. The details of the shooting execution process will be described later with reference to FIG.

FIG. 10 is a flowchart showing a procedure of shooting execution processing executed in step S1405 of FIG.

In step S1501 of FIG. 10, the application control circuit 210 transmits a module start instruction message to the system control circuit 110. Upon receiving the message of the imaging module activation instruction via the system control circuit 110, the mounted imaging module (imaging modules 500 and 600 in this embodiment) performs a reset operation to complete the imaging preparation.

Next, in step S1502, the application control circuit 210 determines whether or not a plurality of imaging modules are mounted from the module information acquired in step S1402 of FIG. If there is only one imaging module, the process proceeds to step S1503. For example, when only the image pickup module 500 is attached, the application control circuit 210 uses the camera 510 to perform general shooting in the monocular mode, and proceeds to step S1510. The general shooting referred to here is the desired image data from the image sensor while controlling automatic exposure (AE), automatic focus adjustment (AF), automatic white balance (AWB), camera shake correction (IS), and the like. Is to get. It should be noted that the present invention does not limit the contents of such general photographing, and since these are already known from the prior art documents and the like, detailed description thereof will be omitted.

If it is determined in step S1502 that a plurality of imaging modules are mounted, the process proceeds to step S1504. In step S1504, the application control circuit 210 determines whether or not the state change flag is ON in order to detect whether or not the state of each mounted imaging module has changed. At this time, if all the state change flags are OFF, the process proceeds to step S1509. For example, when the image pickup modules 500 and 600 are mounted, the application control circuit 210 performs the compound eye shooting execution process via the cameras 510 and 610, and then ends this process. As described above, the functions of the compound eye camera include an image composition function and a measurement function. The compound eye photographing execution process will be described in detail later with reference to FIG.

As a result of the determination in step S1504, if the state change flag of any of the mounted imaging modules (for example, imaging modules 500 and 600) is ON, the process proceeds to step S1505.

In step S1505, the application control circuit 210 acquires the position information of each slot and the coordinate information of each optical axis. The position information of each slot is acquired from the memory 114 of the main body of the smart device 50. Similarly, the coordinate information of the optical axis in the camera 510 is acquired from the non-volatile memory 522 of the image pickup module 500, and the coordinate information of the optical axis in the camera 610 is acquired from the non-volatile memory 622 of the image pickup module 600.

Next, the process proceeds to step S1506, and the application control circuit 210 calculates the baseline length, which is the distance between the optical axis of the camera 510 and the optical axis of the camera 610. As mentioned above in the background art, it is important to obtain an accurate baseline length in order for at least two imaging modules to perform the functions of a compound eye camera. The baseline length referred to in the present invention is a variable used for the purpose of processing parallax information of at least two imaging modules, and indicates the distance between the two optical axes. It is described by expressions such as the amount of deviation and parallax. The details of the calculation method of the baseline length in this embodiment will be described later.

Subsequently, in step S1507, the application control circuit 210 updates the management information stored in the predetermined areas of the non-volatile memory 214 and the memory 212 together with the management information of the baseline length calculated in step S1506. The management information referred to here includes module management information, EPM control management information, and RF bus configuration management information, in addition to the baseline length management information newly obtained in step S1506.

After that, the process proceeds to step S1508, and the application control circuit 210 switches the state change flag to OFF. Here, the timing for turning on the state change flag is, for example, when a new module is mounted on the main body of the smart device 50 in step S1308 in the mounting process of FIG. On the other hand, the timing for turning off the state change flag is immediately after the management file is updated to the latest state, as in step S1508, for example. In this way, by switching the state change flag ON and OFF at an appropriate timing, the state change of the module is always managed in this embodiment.

Subsequently, the process proceeds to step S1509, and the application control circuit 210 performs a compound-eye imaging execution process for photographing in the compound-eye mode via, for example, the camera 510 and the camera 610.

Finally, in step S1510, the application control circuit 210 transmits a message for instructing the end of the imaging module to the system control circuit 110, and ends this process. When the system control circuit 110 receives the message of the image pickup module termination instruction, the system control circuit 110 changes the power supply to the image pickup modules 500 and 600 to the low power consumption setting via the power supply control circuit 220.

FIG. 11 is a flowchart showing the procedure of the compound eye photographing execution process in step S1509 of FIG.

First, in step S1601 of FIG. 11, the application control circuit 210 simultaneously controls the imaging modules 500 and 600 to perform distance measurement processing of the subject. The details of the distance measuring process will be described later with reference to FIG.

Next, the process proceeds to step S1602, and the focus adjustment based on the defocus amount obtained in step S1601 and the exposure adjustment for optimally setting the aperture, shutter speed, and sensitivity of the image sensor are executed. Then, the steps from the distance measurement process in step S1601 to the focus adjustment and exposure adjustment in step S1602 are repeated in the next step S1603 until there is a shooting instruction by the user. The shooting instruction by the user is given to the TP / button 314, and the display operation control circuit 310 transmits a shooting instruction message to the application control circuit 210 accordingly.

When the application control circuit 210 receives the shooting instruction message in step S1603, the process proceeds to step S1604. In step S1604, each of the imaging modules 500 and 600 performs an exposure process and outputs image data generated by photoelectric conversion of the imaging sensor, respectively. Each image data output in step S1604 is image-corrected in step S1605. At this time, for example, noise is removed or distortion is corrected for each image data. Further, by deforming or moving the image data, it is possible to correct the inclination of the image data and the unintended misalignment of the image data. Ideally, the optical axes of the imaging modules 500, 600 are preferably parallel. However, it is difficult to avoid manufacturing errors caused by the accuracy of parts and assembly. Therefore, in step S1605, the manufacturing error of the image pickup modules 500 and 600 is electronically corrected by the deformation and movement of the image data. This makes it possible to improve the accuracy of the image composition function and the measurement function, which will be described later.

Next, the process proceeds to step S1606, and the application control circuit 210 performs integrated processing of each image data corrected by step S1605. In the smart device 50 of the present invention, the viewpoints of the image pickup module 500 and the image pickup module 600 are different from each other, and each has a slight parallax. Therefore, if the pixels of each image data are combined with sub-pixel accuracy and integrated into one image data, image data having a higher resolution than the image data obtained in the monocular mode can be generated.

In general, when the depth of field of a camera made of an optical lens is shallow, high resolution can be obtained for a focused subject, but blurring occurs for a subject that differs in the depth direction with respect to the focal position. In order to solve this problem, the focal positions of the imaging modules 500 and 600 may be slightly different in step S1602, and the focal positions of the reference image data and the reference image data may be combined with sub-pixel accuracy in step S1606. good. As a result, the depth of field of the entire screen can be deepened to increase the resolution. In addition, in this compound eye shooting execution process, it is possible to perform image processing such as making the depth of field shallow to emphasize blurring or intentionally moving the focal position. Further, in step S1602, not only the focus adjustment but also the exposure adjustment can be controlled so that the exposure settings of the imaging modules 500 and 600 are slightly different from each other, so that the dynamic range of the entire screen can be expanded. .. The present invention does not limit the control method of such image integration processing, and since these are already known from the prior art documents and the like, detailed individual description will be omitted.

After generating the desired image data in step S1606, the process proceeds to step S1607, and the generated image data is stored in the recording module 150. When step S1607 is completed, the process proceeds to step S1608. In step S1608, if the user does not input an instruction to end shooting even after a certain period of time has elapsed after the process of step S1607, the process returns to step S1601 and the compound eye shooting execution process is repeated.

When a user's instruction to end shooting is input in step S1608, the display operation control circuit 310 transmits a shooting end instruction message to the application control circuit 210. Upon receiving this shooting end instruction message, the application control circuit 210 stops the operation of the imaging modules 500 and 600 and ends this process.

FIG. 12 is a flowchart showing the procedure of the distance measuring process in step S1601 of FIG.

First, in step S1701 of FIG. 12, the application control circuit 210 simultaneously controls the imaging modules 500 and 600 to perform the exposure process, and then in step S1702, the application control circuit 210 executes the correction process of the obtained image data. Since steps S1701 and S1702 have the same control contents as steps S1604 and S1605 described above in FIG. 11, detailed description thereof will be omitted.

Next, the process proceeds to step S1703, and the application control circuit 210 performs the block matching process.

The block matching referred to here is an algorithm for obtaining the corresponding positions of image data having two different viewpoints. To simplify the explanation, it is a method of searching for coordinates corresponding to a specific part in the same subject, for example, the apex of an edge from two image data. The reference image data is divided into small area areas, and a reference image is used for each area. It searches for the corresponding coordinates on the data. Specifically, in the block matching process, in order to evaluate the similarity between image data, an arbitrary region is cut out from the image data to be compared and an evaluation value is obtained. Examples of the evaluation value to be obtained include SAD (Sum of Absolute Difference), which is the sum of the brightness differences with respect to the cut-out region, and SSD (Sum of Spured Difference), which is the sum of squares of the brightness differences. Further, the normalized cross-correlation ZNCC (Zero-mean Normalized Cross-Correlation) or the like may be obtained as this evaluation value.

At this time, if the subject is unclear with no difference in brightness or the distance of the subject is too close to the length of the baseline length, the same specific location cannot be found on each image data. In such a case, the block matching process in step S1703 may fail. Therefore, when it is determined that the block matching process has failed (NO in step S1704), the application control circuit 210 proceeds to step S1705 and performs a predetermined error process. In the error processing, the error content may be displayed on the display operation module 300 or the like to notify the user.

On the other hand, if it is determined in step S1704 that the block matching process is successful, the process proceeds to step S1706, and the parallax amount on the image plane is calculated for the entire screen. The parallax amount in this embodiment is calculated over the entire screen based on the amount of deviation of the coordinates of the reference image data with respect to the reference image data associated by the block matching process. After that, the process proceeds to step S1707, and the parallax amount on the image plane calculated in step S1706 is converted into a distance.

The simplest method for converting the amount of parallax on the image plane into distance is the method using the calculation formula shown in Equation 1 below. Z is the distance to the subject, L is the baseline length between the imaging modules 500 and 600, f is the focal length of the cameras 510 and 610, and d is the parallax amount. In addition, in order to simplify the description here, the number of imaging modules is two, and the focal lengths of the respective cameras are the same, but the present invention is not limited thereto.
[Number 1]
Z = L × f ÷ d

In step S1707, the parallax amount on the image plane is converted into a distance over the entire screen to create a so-called distance map, and then in step S1708, the distance measuring range is specified so as to include a desired subject. The range-finding range is generally specified by the user's operation, such as displaying a movable range-finding frame on the LCD panel 312 as an index so that the subject to be photographed is included. In addition, the application control circuit 210 may determine the feature point of the subject, such as the face of a person, and automatically perform the determination.

When the distance measurement range is specified in step S1708, it becomes clear which subject is to be targeted, and which range of the distance map created in step S1707 is uniquely determined. Then, in step S1709, the difference between the target focal position on the distance map and the current focal position determined by the actual position of the optical lens is calculated as the defocus amount. The imaging modules 500 and 600 of this embodiment calculate the amount of movement of the optical lens based on this amount of defocus, and perform the focus adjustment described above in step S1602 of FIG. After that, the distance measurement process is completed.

This completes the description of the series of operation flows of the smart device 50. Information on the management table 290 provided in the application program control module 200 shown in FIG. 5 can be updated when the application program is updated by the wireless LAN module 700 or the like. Therefore, in the management table 290 in FIG. 5, it is possible to appropriately change or add the type of each module, the functions that can be realized, and other necessary information.

FIG. 13 is an explanatory diagram showing a method of calculating the baseline length executed in step S1506 of FIG.

In step S1506 of FIG. 10 described above, the application control circuit 210 acquires the position information of each slot from the memory 114 of the main body of the smart device 50. Here, the position information of each slot in this embodiment includes the position information of the slots 1500 and 1600, and specifies the positions of the abutting surfaces of the ribs 101a, c to h and the spine 102 that determine the position of the module. It is a thing. At the same time, in step S1506, the application control circuit 210 acquires the coordinate information of the optical axis in the camera 510 from the non-volatile memory 522 of the image pickup module 500. Further, the application control circuit 210 acquires the coordinate information of the optical axis in the camera 610 from the non-volatile memory 622 of the image pickup module 600. Here, the coordinate information of each optical axis in this embodiment specifies the coordinates of each optical axis as seen from the outer shape of each of the imaging modules 500 and 600.

Specifically, based on such position / coordinate information, the application control circuit 210 obtains the respective numerical values of X101 to X103 and Y101 and Y102 shown in FIG. X101 indicates the horizontal length from the abutting surface of the image pickup module 500 to the optical axis of the camera 510, and Y101 also indicates the vertical length. X102 indicates the horizontal length of the spine 102 in the main body of the smart device 50. X103 indicates the horizontal length from the abutting surface of the image pickup module 600 to the optical axis of the camera 610, and Y102 also indicates the vertical length.

L100 shown in FIG. 13 indicates the length of the center line connecting the optical axis of the camera 510 and the optical axis of the camera 610, and corresponds to the baseline length. The baseline length L100 is geometrically calculated from X101 to X103 and Y101 and Y102. Here, the baseline length L100 in this embodiment is obtained by the calculation formula shown in Equation 2 below.
[Number 2]
L100 = √ {(X101 + X102 + X103) ² + (Y101-Y102) ² }

As shown above, the calculation formula itself is relatively simple, and the memory capacity required for the memory 212 and the non-volatile memory 214 of the application program control module 200 is relatively small. Further, since the processing speed by the application control circuit 210 is very high, there is no risk of causing a time lag between focusing and the start of exposure.

FIG. 14 is an explanatory diagram showing an inclination of the optical axis with respect to the main body of the smart device 50 of the imaging module 500 according to this embodiment. FIG. 14A is a cross-sectional view of the imaging module 500 when it is divided in the horizontal direction in FIG. 13, and is a cross-sectional view of S1-S1 passing through the optical axis of the camera 510. On the other hand, FIG. 14B is a cross-sectional view of the image pickup module 500 when the image pickup module 500 is vertically divided in FIG. 13, and is a cross-sectional view of S2-S2 passing through the optical axis of the camera 510. The horizontal direction in FIG. 14 refers to the longitudinal direction of each of the ribs 101a to 101, and the vertical direction refers to the longitudinal direction of the spine 102 (not shown in this figure).

Θx shown in FIG. 14A schematically shows the horizontal tilt angle generated in the optical axis of the camera 510, and θy shown in FIG. 14B is the vertical generated in the optical axis of the camera 510. It schematically shows the tilt angle in the direction. As described above, in the main body of the smart device 50, it is preferable that at least the slots 1500 and 1600 are in the same plane and the optical axes of the cameras 510 and 610 are parallel. However, manufacturing errors caused by the accuracy of parts and assembly cannot be avoided, and although corrections such as lens position adjustment are attempted in the manufacturing process, minute horizontal tilt angles θx and vertical tilt angles θy remain, respectively. It ends up. Although FIG. 14 shows the inclination that occurs in the optical axis of the camera 510, such a manufacturing error also occurs in the camera 610.

Therefore, in this embodiment, the manufacturing error of the imaging modules 500 and 600 is measured in the manufacturing process. The tilts of the optical axes of the cameras 510 and 610 as the measurement results are stored in the non-volatile memory 522 of the image pickup module 500 and the non-volatile memory 622 of the image pickup module 600, respectively. By doing so, when the image data is corrected in step S1605 of FIG. 11 and step S1702 of FIG. 12, the application control circuit 210 is deformed or moved so as to approach the ideal system in consideration of each error. It becomes possible to do. In this way, in this embodiment, the accuracy of the image composition function and the measurement function can be improved when the function of the compound eye camera is used. Since a specific method for correcting an error from the inclination of the optical axis of the camera 510 and the inclination of the optical axis of the camera 610 is already known from Patent Document 2 and the like described above, detailed individual explanations are omitted. do.

(Example 2)
Up to this point, preferred embodiments of the present invention have been described according to Example 1 shown in FIGS. 1 to 14. From here on, the second embodiment in which only the combination of slots for mounting the modules is changed from the first embodiment will be described.

FIG. 15 is an external view of the smart device 50 as an electronic device according to the present embodiment, and is an explanatory view showing a method of calculating the baseline length executed in step S1506 of FIG.

In this embodiment, unlike the first embodiment, the image pickup module 500 is mounted in the slot 1600 and the image pickup module 600 is mounted in the slot 1100 on the back side of the main body of the smart device 50. Then, in the present embodiment, the recording module 150 is attached to the slot 1500 in which the imaging module 500 is mounted in the first embodiment. By doing so, the optical axis of the camera 510 and the optical axis of the camera 610 can be arranged more apart from each other, and the baseline length can be lengthened. Generally, when the baseline length is lengthened, it is easy to obtain a stereoscopic effect in the stereoscopic mode when shooting a distant subject, and it is possible to improve the distance measurement accuracy. Further, the positional relationship with the imaging modules 500 and 600 is also changed depending on the ease of holding the smart device 50 by the user.

In step S1506 of FIG. 10 described above, the application control circuit 210 acquires the position information of each slot from the memory 114 of the main body of the smart device 50. Here, the position information of each slot in the present embodiment includes the position information of the slots 1500, 1600 and the slot 1800, and specifies the positions of the abutting surfaces of the ribs 101a to 101a to h and the spine 102, which determine the position of the module. Is what you do. At the same time, in step S1506, the application control circuit 210 acquires the coordinate information of the optical axis in the camera 510 from the non-volatile memory 522 of the image pickup module 500. Further, the application control circuit 210 acquires the coordinate information of the optical axis in the camera 610 from the non-volatile memory 622 of the image pickup module 600. Here, the coordinate information of each optical axis in this embodiment specifies the coordinates of each optical axis as seen from the outer shape of each of the imaging modules 500 and 600.

Specifically, based on such position / coordinate information, the application control circuit 210 obtains the respective numerical values of X201 to X203 and Y201 and Y202 shown in FIG. X201 indicates the horizontal length from the abutting surface of the image pickup module 500 to the optical axis of the camera 510, and Y201 also indicates the vertical length. X202 indicates the horizontal lengths of the ribs 101f and 101h in the main body of the smart device 50. X203 indicates the horizontal length from the abutting surface of the image pickup module 600 to the optical axis of the camera 610, and Y202 also indicates the vertical length.

L200 shown in FIG. 15 indicates the length of the center line connecting the optical axis of the camera 510 and the optical axis of the camera 610, and corresponds to the baseline length. The baseline length L200 is geometrically calculated from X201 to X203 and Y201 and Y202. Here, the baseline length L200 in this embodiment is obtained by the calculation formula shown in Equation 3 below.
[Number 3]
L200 = √ {(X201 + X202 + X203) ² + (Y201-Y202) ² }

As shown above, the calculation formula itself is relatively simple, and the memory capacity required for the memory 212 and the non-volatile memory 214 of the application program control module 200 is relatively small. Further, since the processing speed by the application control circuit 210 is very high, there is no risk of causing a time lag between focusing and the start of exposure.

As described above, another preferred embodiment of the present invention has been described with reference to FIG. Since there are a wide variety of modules that can be mounted on the main body of the smart device 50, and the slots in which the modules are mounted can be freely selected by the user, the combination shown in FIG. 15 is merely an example, and the present invention presents the present invention. The combination is not limited.

The object of the present invention is also achieved by performing the following processing. That is, a program in which a storage medium in which a program code of software that realizes the functions of the above-described embodiment is recorded is supplied to a system or device, and a computer (or CPU, MPU, etc.) of the system or device is stored in the storage medium. This is the process of reading the code. In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the program code and the storage medium storing the program code constitute the present invention.

50 Smart device 101a to h Rib 102 Spine 118 Temperature sensor 1000 to 1900 Slot 200 Application program control module 210 Application control circuit 214 Non-volatile memory 500,600 Imaging module 510,610 Camera

Claims (13)

In an electronic device in which the first and second imaging modules are mounted in any of a plurality of mounting areas, respectively.
Each of the first and second imaging modules mounted on the electronic device stores a memory for storing identification information for communicating with the main body of the electronic device and coordinate information of an optical axis for each imaging module. I have
An acquisition means for acquiring coordinate information of the optical axis of the first and second imaging modules by communication with the first and second imaging modules, respectively.
A temperature sensor for measuring the temperature of the main body of the electronic device and
Shooting means for shooting in compound eye mode,
A storage means for storing the position information of the plurality of mounting areas, and
The optical axis when the first and second imaging modules are attached to the main body of the electronic device and when the temperature change of the main body of the electronic device measured by the temperature sensor exceeds a threshold value. An electronic device comprising a calculation means for calculating a baseline length by acquiring the coordinate information of the above and the position information of the mounting area. The compound eye mode is a mode in which images output from each of the first and second image pickup modules are combined, and the plurality of mounting areas are substantially coplanar, and the first and second image pickup modules are formed. The electronic device according to claim 1, wherein a user-selectable mode is set when the optical axes of the above are substantially parallel. The compound eye mode is a mode in which the subject distance is calculated based on the image data output from each of the first and second imaging modules. The electronic device according to claim 1, wherein a user-selectable mode is set when the optical axes of the second image pickup module are substantially parallel. The acquisition means further acquires error information of the respective optical axes from the first and second imaging modules, and further acquires the error information.
When there is a state change in either the first and second imaging modules and the main body of the electronic device, the coordinate information of the optical axis, the error information of the optical axis, and the position information of the mounting area are obtained. The electronic device according to any one of claims 1 to 3, further comprising acquiring and correcting image data output from the first and second imaging modules. Further, a detection means for detecting whether or not the first and second imaging modules are mounted in any of the plurality of mounting regions is further provided.
When it is detected by the detection means that each of the first and second imaging modules is mounted in any of the plurality of mounting areas, the imaging means performs imaging in the compound eye mode. The electronic device according to any one of claims 1 to 4. The electronic device according to any one of claims 1 to 5, wherein the first and second imaging modules have cameras having different arrangements and shapes. The electronic device according to any one of claims 1 to 6, wherein the plurality of mounting areas are at least three places, and the position information of each is different. A permanent electromagnet provided at a position corresponding to a magnetic body of the first and second imaging modules in each of the plurality of mounting regions, and a polarity changing means for individually changing the polarity of the permanent electromagnet are further provided. ,
The claim is characterized in that the polarity changing means releases and fixes the first and second imaging modules to any of the plurality of mounting areas by individually changing the polarity of the permanent electromagnet. Item 2. The electronic device according to any one of Items 1 to 7. A guide portion for dividing the plurality of mounting areas is further provided.
The guide unit is characterized in that it guides the first and second imaging modules when they are attached to any of the plurality of attachment areas, and holds the first and second imaging modules after the attachment. The electronic device according to any one of claims 1 to 8. The electronic device according to any one of claims 1 to 9, wherein when the baseline length is calculated, management information including the baseline length information is updated. An image pickup apparatus that functions as one of the first and second image pickup modules when mounted in any of the plurality of attachment areas according to any one of claims 1 to 10. ,
An imaging device having a memory for storing coordinate information of the optical axis viewed from each outer shape. In the control method of an electronic device in which the first and second imaging modules are mounted in any of a plurality of mounting areas, respectively.
The first and second imaging modules mounted on the electronic device each include a memory for storing identification information for communicating with the main body of the electronic device and coordinate information of the optical axis thereof.
An acquisition step of acquiring coordinate information of the optical axis of the first and second imaging modules by communication with the first and second imaging modules , respectively.
A measurement step for measuring the temperature of the main body of the electronic device, and
Shooting steps to shoot in compound eye mode,
A storage step for storing the position information of the plurality of mounting areas, and
When the first and second imaging modules are attached to the main body of the electronic device, and when the temperature change of the main body of the electronic device measured in the measurement step exceeds a threshold value, the optical axis A control method comprising a calculation step of acquiring the coordinate information of the above and the position information of the mounting area and calculating the baseline length. A program comprising executing the control method according to claim 12.

Images