KR102514702B1 - A mobile terminal for generating a photographed image and a method for generating a photographed image - Google Patents

Abstract

The present invention relates to a mobile terminal for generating a photographed image and a method for generating a photographed image, and more particularly, when a photographed image is generated in an electronic device such as a mobile terminal, synthesis of other previously photographed persons To create a photographic image that can provide the same effect as if users in different time and space were photographed together by including the object image to be overlapped with the rear surface of the real world object image of the photographing target to create a photographic image It relates to a mobile terminal and a method for generating a photographed image.

Description

A mobile terminal for generating a photographed image and a method for generating a photographed image}

The present invention relates to a mobile terminal for generating a photographed image and a method for generating a photographed image, and more particularly, when a photographed image is generated in an electronic device such as a mobile terminal, synthesis of other previously photographed persons To create a photographic image that can provide the same effect as if users in different time and space were photographed together by including the object image to be overlapped with the rear surface of the real world object image of the photographing target to create a photographic image It relates to a mobile terminal and a method for generating a photographed image.

Recent mobile terminals basically include a camera module, and digital cameras gradually include a communication module. Accordingly, a user posts a photographic image taken using a mobile terminal or a digital camera online in real time, such as a social network service, and shares it with other users.

A mobile terminal including such a camera module and a digital camera that can communicate each have various functions added to expand convenience, and one of them is a function of editing a captured image. However, the image editing function is only limited to application of filters to captured images, simple correction of images, and synthesis of predetermined icons. Accordingly, when taking a photo in a mobile terminal, a function of generating a single photo-capture image by including a photo-captured image already taken in the same mobile terminal or another mobile terminal in a newly captured real-world image is not provided.

United States Patent Publication No. US2015-0281591 (published on October 1, 2015) of IDEO LLC. discloses a video editing system capable of editing video data that is a continuous image. Prior Art 1 provides a video editing interface to a mobile terminal having a touch screen. The prior art 1 provides an editing interface capable of applying a filter or the like to an output image (video data) using an input method such as touch or drag & drop, so that the user can record the captured video. It provides effects that can easily edit data.

However, prior art 1 can perform editing, such as applying a filter function to image or video data stored in the mobile terminal itself, but has a problem in that it cannot perform synthesis of a plurality of images or video data that are each captured.

US Patent Publication No. US2007-0291338 (published on December 20, 2007) by Hewlett Packard relates to a photo editing menu system for a digital camera. It is characterized in that it can remotely transmit edited images with digital effects added without using a PC. In the prior art 2, not only photographing but also editing can be performed through a digital camera, which is a photographing device. Therefore, it is possible to solve the problem of transferring the captured photo file to a PC for editing. In addition, there is an effect of easily uploading a photograph taken and edited to a social network service through a function of remotely transmitting a file through a communication network.

Although there is an effect of editing a photographed image using an editing function included in a prior art 2-degree digital camera, editing that represents an effect photographed in the same place by synthesizing a plurality of different images photographed separately into one image. There is a problem that cannot be performed.

US Registered Patent Publication No. US 9,792,883 (registered on October 17, 2017) of Samsung Electronics discloses a technology in which a user not only simply takes a picture using a mobile terminal, but also synthesizes a user image captured at a predetermined location of a pre-stored background image. to provide. The prior art 3 provides an effect of utilizing the captured images in various ways by synthesizing the respective images to generate a single synthesized image and providing a synthesized image other than the captured image to the user.

However, prior art 3 only provides a compositing function for inserting a user image into a background image, which is each image taken by a mobile terminal, and users cannot apply a plurality of images each taken by users to taking pictures.

Even in the case of using the technical features presented in the prior art, in electronic devices such as mobile terminals and digital cameras, there is a method for selectively including people included in a plurality of captured images in real-time photographed images. can't provide

Patent Document 001 US Patent Publication US2015-0281591 (published on October 1, 2015) Patent Document 002 US Patent Publication No. US2007-0291338 (published on December 20, 2007) Patent Document 003 US Registered Patent Publication No. US9,792,883 (registered on October 17, 2017)

In order to solve the above problem, a mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention are a mobile terminal including a camera module or an electronic device such as a digital camera capable of communication. The purpose of generating a single photo-taking image is to share a photo taken with one or more electronic devices of another user, output the shared photo together with a real world image to a display, and capture the photo in real time.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention is a method for generating an object image for synthesis in which a background is excluded from a photograph taken by a user using a communicable electronic device such as a mobile terminal. Another object is to easily synthesize another user's real-time photograph and object image for synthesis as transmitted to the user's electronic device.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention are real-time photographs performed in a mobile terminal using an object image for synthesis created using a photograph taken in a mobile terminal or another mobile terminal. Another object is to generate a photographic image by further including another background image in a photograph to be synthesized while outputting the photographed image and generating the photographic image together.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention, when a plurality of users generate a photographed image using each mobile terminal, sequentially from any one terminal. A photographed image is created including an object image for synthesis to be transmitted. The photographed image is sequentially transmitted to the next mobile terminal so that the generated photographic image can be used as an object image for synthesis, and the received object image for synthesis is sequentially synthesized with the photograph taken by each mobile terminal to update the picture. do. Thereafter, another purpose is to enable the finally updated photo shoot image to have the same effect as a photo taken by each user at the same time and place.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention are composite outputs on a display later according to a user's pose and photographic composition included in an object image for synthesis, which is a photograph taken first. Another object is to provide consistent photo-taking images including object images for synthesis by inducing poses and photo compositions similar to those of object images for synthesis in photos taken including object images for synthesis.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention are provided in which a photograph taken and shared by a first user is used as an object image for compositing by other users so that each user can obtain a photographic image. Another purpose of allowing users who share each photographed image to check the final photographed image by allowing the final photographed image to be received in the mobile terminals of the first user and the users who created the photographed image after being created as a photographed image. there is

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention, when an object image for synthesis is received from a mobile terminal of a previous user, a real world image input from a camera module of the mobile terminal and a method for generating a photographed image Another object is to output the received object image for synthesis to a display so that the object image for synthesis can be used as a guide when generating a photographic image.

In order to solve the above problem, a method for generating a photographed image according to the present invention is a mobile terminal including a camera module, a display, a communication module, a processor, and a memory, wherein the processor performs the following steps to generate a photographed image. The generating method may include continuously outputting a real-world image including a real-world object image generated by continuously photographing a real-world scene by the camera module in real time on the display; outputting the object image for synthesis stored in the memory to the display; Arranging the object image for synthesis and the real-world object image on the display, when the object image for synthesis is selected to be overlapped with the rear surface of the real-world object image, the real-world object image continuously output in real time including overlapping and arranging the object image for synthesis on a rear surface; and generating a photographed image including the object image for synthesis and the real world object image disposed on the display.

In the method for generating a photographic image according to an embodiment of the present invention, after outputting the real-world image to the display, the real-world object image to be overlapped with the object image for synthesis is displayed in the real-world image. It is characterized in that it is carried out further including; the step of being selected.

In the method for generating a photographic image according to an embodiment of the present invention, the object image for synthesis is provided to be located on a first layer, and the real world object image is provided to be located on a second layer. and in the arranging, the object image for synthesis and the real world object image located on the first layer and the second layer are arranged on the display, wherein the object image for synthesis is located behind the real world object image. When a command to do so is received, the first layer is positioned behind the second layer so that the object image for synthesis is overlapped and disposed on the rear surface of the real world object image.

In the method for generating a photographic image according to an embodiment of the present invention, only the real-world object image is extracted from the real-world image, and the extracted real-world object image is placed in the second layer.

In the method for generating a photographed image according to an embodiment of the present invention, the real world object image and the transparent processed image are placed in the second layer. It is characterized by doing

In the method for generating a photographed image according to an embodiment of the present invention, a real world background image excluding the real world object image extracted from the real world image may be placed in a third layer.

In the method for generating a photographed image according to an embodiment of the present invention, the photographed image may be generated by further including the real world background image of the third layer.

In the method for generating a photographic image according to an embodiment of the present invention, in the generating of the photographic image, the memory includes one or more background images, and further includes a background image selected from among the background images. It is characterized in that the photographic image is generated.

In the method for generating a photographic image according to an embodiment of the present invention, the size or position of the object image for synthesis displayed on the display is controlled according to a control command input to the mobile terminal.

In the method for generating a photographic image according to an embodiment of the present invention, the object image for synthesis is generated in the mobile terminal or another mobile terminal and is provided through the communication module.

In the method for generating a photographic image according to an embodiment of the present invention, the object image for synthesis may include only an object image excluding a background image.

In the method for generating a photographic image according to an embodiment of the present invention, when the object image for synthesis is generated, the remaining images except for the object image for synthesis are created as background images.

In the method for generating a photographic image according to an embodiment of the present invention, the photographed image in which the object image for synthesis and the real world object image are arranged is used as an object image for synthesis, and another mobile terminal is used by using the communication module. Further comprising the step of transmitting, wherein the other mobile terminal takes a picture including the object image for synthesis and the real world object image of the other mobile terminal received using a processor performing the same steps as each of the above steps. It is characterized by generating an image.

A mobile terminal for generating a photographic image according to the present invention generates a photographic image using a camera module, a display, a communication module, a processor, a memory, and one or more programs stored in the memory and configured to be executed by the processor. The one or more programs are configured to continuously output a real-world image including a real-world object image generated by continuously photographing a real-world scene by the camera module in real-time to the display, and to store it in the memory for synthesis. An object image is output to the display, and the object image for synthesis and the real world image are disposed on the display, and when the object image for synthesis is selected to be overlapped with the rear surface of the real world object image, the object image for synthesis is continuously output in real time. A mobile device including instructions for causing the object image for synthesis to be overlapped and disposed on the rear surface of the real world object image, and for generating a photographic image including the object image for synthesis and the real world object image disposed on the display. characterizes the terminal.

In the mobile terminal for generating a photographic image according to an embodiment of the present invention, the one or more programs output the real world image to the display, and then overlap the object image for synthesis and arrange the real world image. It is characterized in that it includes a command for further performing selecting an object image from the real world image.

In a mobile terminal for generating a photographic image according to an embodiment of the present invention, the object image for synthesis is provided to be located on a first layer, and the real world object image is provided to be located on a second layer. , The arrangement is a command for arranging the object image for synthesis and the real-world object image located in the first layer and the second layer on the display, and the object image for synthesis is located on the back of the real-world object image. When received, the first layer is positioned behind the second layer so that the object image for synthesis is overlapped and disposed on the rear surface of the real world object image.

In the mobile terminal for generating a photographic image according to an embodiment of the present invention, only the real-world object image is extracted from the real-world image, and the extracted real-world object image is placed in the second layer.

In the mobile terminal for generating a photographed image according to an embodiment of the present invention, the real world image except for the real world object image is transparently processed, and the real world object image and the transparently processed image are stored in the second layer It is characterized in that it is located in.

In the mobile terminal for generating a photographic image according to an embodiment of the present invention, a real world background image excluding the real world object image extracted from the real world image is placed in a third layer.

In the mobile terminal for generating a photographed image according to an embodiment of the present invention, when generating the photographed image, the real world background image of the third layer is further included to generate the photographed image. to be

In the mobile terminal for generating a photographic image according to an embodiment of the present invention, when generating the photographic image, the memory includes one or more background images, and further includes a background image selected from among the background images. and generating the photographed image by doing so.

In the mobile terminal for generating a photographic image according to an embodiment of the present invention, the size or position of the object image for synthesis outputted to the display is controlled according to a control command input to the mobile terminal.

In the mobile terminal for generating a photographic image according to an embodiment of the present invention, the object image for synthesis is generated in the mobile terminal or another mobile terminal and is provided through the communication module.

In the mobile terminal for generating a photographed image according to an embodiment of the present invention, the object image for synthesis is characterized in that it includes only the object image excluding the background image.

In the mobile terminal for generating a photographed image according to an embodiment of the present invention, when the object image for synthesis is generated, the remaining images except for the object image for synthesis are created as background images.

In the mobile terminal for generating a photographed image according to an embodiment of the present invention, the communication module is used so that the photographed image in which the object image for synthesis and the real world object image are disposed can be used as an object image for synthesis. and transmits it to another mobile terminal, wherein the other mobile terminal receives the photographed image as an object image for synthesis using one or more programs including the same commands as the commands, and the object image for synthesis and the other mobile terminal It is characterized in that a photographed image including the real world object image of the terminal is created.

According to the configuration described above, the mobile terminal for generating a photographed image and the method for generating a photographed image according to the present invention include a photograph taken by a mobile terminal including a camera module or an electronic device such as a digital camera capable of communication. There is an effect of generating a single photo-taking image by sharing a photo with one or more electronic devices of another user, outputting the shared photo together with a real world image on a display, and capturing the photo in real time.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention is a method for generating an object image for synthesis in which a background is excluded from a photograph taken by a user using a communicable electronic device such as a mobile terminal. Another effect is that an object image for compositing can be easily synthesized with another user's real-time photograph as transmitted to the user's electronic device.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention are real-time photographs performed in a mobile terminal using an object image for synthesis created using a photograph taken in a mobile terminal or another mobile terminal. Other images that can change the background of the photograph taken in real time without additional editing by allowing the photographic image to be generated by further including other background images in the composited photo, while outputting to the shooting to be created as a photographic image together. It works.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention, when a plurality of users generate a photographed image using each mobile terminal, sequentially from any one terminal. A photographed image is created including an object image for synthesis to be transmitted. The photographed image is sequentially transmitted to the next mobile terminal so that the generated photographic image can be used as an object image for compositing, so that the received object image for compositing is sequentially synthesized with the photograph taken by each mobile terminal, and the photo is updated. do. After that, there is another effect that can create a final updated photo taken image as if each user took a photo at the same time and place.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention include a photographic image that is subsequently output to a display according to a user's pose and photographic composition included in an object image for synthesis of a photographed photograph. There is another effect capable of creating a photographic image that is consistently taken including the object image for synthesis by inducing a similar pose and photo composition to the object image for synthesis in a photo taken including the object image for synthesis.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention are provided in which a photograph taken and shared by a first user is used as an object image for compositing by other users so that each user can obtain a photographic image. After being created as a photographed image, the final photographed image is received in each mobile terminal of the first user and the users who created the photographed image, so that the object image for synthesis provided by the users who shared each photographed image is There are other effects that allow you to check the final photo shoot image used.

A mobile terminal for generating a photographed image and a method for generating a photographed image according to the present invention, when an object image for synthesis is received from a mobile terminal of a previous user, a real world image input from a camera module of the mobile terminal and a method for generating a photographed image By outputting the received object image for synthesis together on the display, the object image for synthesis is used as a shooting guide when creating a photo shoot image, and the expected composite form is checked in advance to obtain a better photo shoot image. There are other effects that

1 is a configuration diagram of a mobile terminal for generating a photographed image according to the present invention;
2 is a flowchart illustrating a method of generating a photographic image using a mobile terminal according to the present invention;
3 is a flowchart illustrating a method of generating a photographic image using a layer in a mobile terminal according to the present invention;
4 is an exemplary view showing an embodiment of outputting a real world image by photographing a real world scene using a mobile terminal according to the present invention;
5 is an exemplary view showing an embodiment of a real world image output to a mobile terminal according to the present invention;
6 is an exemplary view showing an embodiment of outputting a real world image and an object image for synthesis generated according to the present invention to a display;
7 is an exemplary view showing another embodiment of outputting a real world image and an object image for synthesis generated according to the present invention to a display;
8 is an exemplary view showing an embodiment of outputting a real-world object image and an object image for synthesis processed transparently to a display according to the present invention;
9 is an exemplary view showing another embodiment of outputting a real-world object image and an object image for synthesis processed transparently to a display according to the present invention;
10 is an exemplary view showing an embodiment for adjusting the position of a real-world object image using motion of a mobile terminal according to the present invention;
11 is an exemplary view showing overlapping of a real-world object image whose position is adjusted using a motion of a mobile terminal according to the present invention and an object image for synthesis;
12 is an exemplary view showing an embodiment in which a real-world object image and an object image for synthesis are overlapped and arranged according to the present invention;
13 is an exemplary view showing an embodiment for adjusting the position of a real-world object image using the movement of a real-world person photographed in a mobile terminal according to the present invention;
14 is an exemplary view showing overlapping of a real-world object image whose position is adjusted using a real-world person's motion according to the present invention and an object image for synthesis;
15 is an exemplary view showing an embodiment for adjusting the position of an object image for synthesis in a mobile terminal according to the present invention;
16 is an exemplary diagram illustrating overlapping of an object image for synthesis and a real-world object image whose location is adjusted in a mobile terminal according to the present invention.

The terms or words used in this specification and claims should not be construed as limited in their usual or dictionary meanings, and the inventors should use this invention based on the principle that terms and concepts can be defined in order to best explain their invention. It should be interpreted as meaning and concept consistent with the technical idea of the invention.

Therefore, the embodiments described in this specification and the configurations shown in the drawings correspond to a preferred embodiment of the present invention, and do not represent all the technical ideas of the present invention, so the corresponding configurations may replace them at the time of filing of the present invention. There may be many equivalents and variations.

Throughout the specification, the description that a certain part "includes" a certain component means that it may further include other components, not excluding other components unless otherwise specified. In addition, terms such as “..mother”, “..unit”, and “..system” described in the specification mean a unit that processes at least one function or operation, which is hardware or software, or a combination of hardware and software. It can be implemented as, and can be included in one device or each other device.

Hereinafter, a mobile terminal for photographing an update photograph and an update photographing method according to the present invention will be described with reference to drawings.

The present invention is a processor 103 in a mobile terminal 100 including a camera module 101, a display 102, a communication module 105, a microprocessor 103 (hereinafter referred to as 'processor') and a memory 104. performs the following steps to create a photographic image.

The processor 103 photographs a real world scene in the camera module 101 and outputs the real world image including the real world object image to the display 102; and displaying the object image for synthesis stored in the memory 104. ); and arranging the object image for synthesis and the real world image on the display 102, wherein, when the object image for synthesis is selected to be overlapped with the rear surface of the real world object image, including overlapping and arranging an object image for synthesis on the back of the real world object image; and generating a photographed image including an object image for synthesis and a real world object image arranged on the display 102 .

1 is a configuration diagram of a mobile terminal for generating a photographed image according to the present invention. As shown in FIG. 1, the present invention is stored in a camera module 101, a display 102, a communication module 105, a processor 103, a memory 104 and a memory 104, and the processor 103 One or more programs in the mobile terminal 100 including one or more programs configured to be executed by the user generate a photographed image using the following commands.

One or more programs capture a real world scene in the camera module 101, output a real world image including a real world object image to the display 102, and display an object image for synthesis stored in the memory 104 to the display 102. output, and the object image for synthesis and the real world image are placed on the display 102, but if the object image for synthesis is selected to be superimposed on the rear surface of the real world image, it is synthesized on the rear surface of the real world object image included in the real world image. A command for causing object images to be overlapped and for generating a photographic image including an object image for synthesis and a real-world object image arranged on the display 102 is included.

The mobile terminal 100 includes a camera module 101 capable of capturing a real world scene and generating a real world image including a real world object image, a display 102 capable of outputting the generated real world image, and a camera module 101. The processor 103 that drives and outputs the real world image to the display 102, the memory 104 that stores the received or generated object image for synthesis, and the object image for synthesis or the generated photographic image is sent to the outside. It includes a communication module 105 capable of receiving or transmitting from another mobile terminal (not shown).

One or more camera modules 101 are provided, and a real world scene input through the camera module 101 is generated as a real world image according to a command generated by the processor 103 . The real-world image includes one or more real-world object images, and the real-world object image is an image corresponding to a real-world object included in the real-world scene. At this time, the generated real world image may be stored in the memory 104 .

The camera module 101 continuously captures real-world scenes in real time to generate real-world images.

The camera module 101 may generate any one frame among continuously input real-world images as a photo-taking image by using a photo-taking command generated and transmitted by the processor 103 .

The camera module 101 may generate a plurality of consecutive frames among continuously input real-world images as a video capture image by using a video capture command generated and transmitted by the processor 103 .

The display 102 outputs images such as photos and videos according to a command transmitted from the processor 103 . The display 102 is composed of a touch screen (not shown) including a touch module (not shown), so that a user command input through the touch screen is converted into an electrical signal and transmitted to the processor 103 . The display 102 may be provided as a display other than a touch screen, and the mobile terminal 100 may further include an input module (not shown) such as a keypad and a pointing device.

The size of the display 102 is not limited, but is preferably made of a size that can be used in the mobile terminal 100 .

The processor 103 interprets an electrical signal of a user command input using the display 102 or the input module to generate a control command capable of controlling components included in the mobile terminal 100 . The processor 103 may drive the camera module 101 through one or more generated control commands. In addition, a photo capture image or a video capture image may be generated through another control command. The processor 103 generates various control commands capable of controlling the mobile terminal 100 .

The memory 104 consists of one or more and stores one or more programs driven by the processor 103 . One or more programs may drive the camera module 101, drive the communication module 105, extract a real-world object image from a real-world image, and generate a photographic image using the object image for synthesis and the real-world object image. there is. In addition, basic data such as an operating system for operating the mobile terminal 100, applications, and user information are stored.

The communication module 105 performs data communication with other external mobile terminals (not shown). The communication module 105 may receive an object image for synthesis created in another terminal or transmit a photographed image created in another terminal to another terminal. The communication module 105 may perform a phone call or the like in addition to data communication.

The mobile terminal 100 may be a cell phone, a smart phone, a smart pad, a laptop computer, a tablet PC, a digital camera, a portable multimedia player (PMP), etc. that are generally carried by individuals. The mobile terminal 100 includes a camera module, a display, a memory and a communication module.

2 is a flowchart illustrating a method of generating a photographed image using a mobile terminal according to the present invention. The processor 103 shown in FIG. 1 generates a photographic image by performing the following steps shown in FIG. 2 .

The processor 103 photographs a real-world scene in the camera module 101 and outputs the real-world image including the real-world object image to the display 102 (S110); and the object image for synthesis stored in the memory 104 Outputting to the display 102 (S120); and arranging the object image for synthesis and the real world object image on the display (S130), wherein the object image for synthesis is selected to be overlapped with the rear surface of the real world object image. In this case, the object image for synthesis is overlapped and disposed behind the real world object image; and generating a photographed image including an object image for synthesis and a real world object image disposed on the display 102 (S140).

In the present invention, "outputting a real-world image including a real-world object image to the display 102" may mean a step of outputting at least a real-world object image among real-world scenes.

In the present invention, "step of outputting an object image for synthesis to the display" means a step in which at least the object image for synthesis is output, and may include outputting additional other images or information including the object image for synthesis. can

Arranging the object image for synthesis and the real world object image on the display ( S130 ) may include selecting the object image for synthesis to be overlapped with the rear surface of the real world object image.

According to an embodiment, it may include checking whether an operation in which the object image for synthesis is overlapped with the rear surface of the real world object image is indicated. According to an embodiment, the method may include selecting a real-world object image overlapping a rear surface from real-world images.

3 is a flowchart illustrating a method of generating a photographic image using layers in a mobile terminal according to the present invention. In FIG. 3, a description is given of a method of providing each image output by the processor 103 to a plurality of layers and outputting the image to the display 102, which will be described with reference to FIG.

In step S110 of photographing a real world scene using the camera module 101 and outputting the real world image including the real world object image generated accordingly to the display 102 (S110), the real world object image output to the display 102 is provided to the second layer and output.

The second layer includes only information on real-world object images, includes information on all real-world images including real-world object images, or includes information about real-world object images and information about which the background excluding the real-world object images is transparent. can do.

A real-world scene is photographed using the camera module 101 and a real-world image is generated, which is described in FIG. 4 .

4 is an exemplary diagram illustrating an embodiment of outputting a real world image by capturing a real world scene using a mobile terminal according to the present invention. As shown in FIG. 4 , real-time continuous shooting of real-world scenes is performed through the camera module 101 of the mobile terminal 100 . The real world scene includes one or more objects such as a background 520 and a person 510 located in the real world. A real world scene photographed by the camera module 101 is continuously generated in real time as real world images, which are image data. Accordingly, the real world image 110 continuously output in real time from the display 102 includes a real world object image (object A) for the person 510 photographed from a camera module (not shown) and a real world background image for the background. can

When the position of the person 510 moves in the real world scene captured by the camera module 101, the real world object image of object A in the real world image is also displayed so that the position is moved corresponding to the moved position of the person 510 ( 102) in real time. In addition, the real world object image and real world background image of object A change according to the camera angle change through rotation or movement of the camera module 101, so that the real world object image and real world background image in the real world image 110 are changed. The location is moved and output.

An embodiment in which a real world scene is photographed using the camera module 101 and a real world image corresponding to the photographed real world scene is output to the display 102 will be described with reference to FIG. 5 .

5 is an exemplary diagram illustrating an embodiment of a real world image output to a mobile terminal according to the present invention. As shown in (a) of FIG. 5 , the real world image 110 that captures a real world scene and outputs it to the display 102 may include a real world object image (object A) and a real world background image.

The real world image of (a) may be configured by including both a real world object image and a real world background image in one image as shown in (c). Also, the real world image of (a) may be configured by combining a real world object image including the remaining transparent image as shown in (d) and a real world background image excluding the real world object image as shown in (e). Also, although not shown, a real world object image consisting of only object A excluding the other images may be combined with a real world background image excluding the real world object image as shown in (e).

As shown in (b) of FIG. 5 , the display 102 may output only the real world image 112 including the remaining images after the real world scene has been captured and the remaining transparent images have been processed as shown in (d). The output of real-world background images can be excluded.

Also, although not shown, a real world scene may be captured, only a real world object image consisting of object A may be output, and output of the other real world background images may be excluded.

In the case of the real world image 112 from which the output of the real world background image is excluded, a real world background image or another background image may be combined later.

As shown in (d), the real world image 112 including the remaining transparent images is transparent to the rest of the images except for the real world object image (object A) selected from the real world image 110 composed of one image as shown in (c). It can be created by doing

Alternatively, as in (c), it can be created by excluding or deleting the output of the remaining images except for the real world object image (object A) selected from the real world image 110 composed of one image.

In the present invention, the term "real world image" may be used to indicate an image that is consequently output to a display in a real world scene.

The step of selecting the real-world object image (object A) from the real-world image 110 may be performed before or after outputting the real-world image 110 to the display, in particular, after outputting the real-world image 110 . The selection of the real world object image (object A) is performed by the processor 103 recognizing the object A included in the real world image 110 and setting the recognized object A as the real world object image (object A).

Alternatively, selection of the real world object image (object A) recognizes the object A corresponding to the object selection signal in the real world image 110 output to the display 102 using the object selection signal input from the user, and the recognized object This is done by setting A as a real-world object image (object A). In this case, the object selection signal may be input by the user through an input module (not shown) provided in the mobile terminal 100 or a touch module included in the display 102 . Input of the object selection signal using the touch module is when the user touches the position of the object to be selected as an object in the real world image 110 output to the display 102, the information of the touched position is recognized as the object selection signal , the object at the touch position is set as the real world object image (object A). Alternatively, when the user touches and drags the outline of an object to be selected in the real world image 110 output on the display 102, the input information of the touch and drag is recognized as an object selection signal, , an object corresponding to touch and drag is set as a real-world object image (object A).

Even when an object is selected using an input module (not shown) provided in the mobile terminal 100, the object selected in the real world image 110 output to the display 102 according to the input information is displayed as a real world object image ( set to object A).

The display 102 outputs an image using one or more layers. The real world image 110 of (a) or the real world image 112 of (b) shown in FIG. 5 is provided to a second layer and output to the display 102 . The real world image 110 output to the display 102 is composed of one image including both the real world object image (object A) and the real world background image as shown in (a), or the real world object image of (d) and (e) ) of the real world background image may be combined, or two images of the real world object image in which only object A exists and the real world background image of (e) may be combined.

Also, as shown in (b), the real world image 112 is an image including a real world object image (object A) and an image in which transparency processing is performed on the remaining images as shown in (d). Alternatively, it is a real world image (not shown) including only object A.

Accordingly, the real world image 112 disposed on the second layer includes the object A, and may include other images or images to which transparency processing is performed on the remaining images, if necessary.

Meanwhile, when the real world image 110 in which the extracted real world object image and the real world background image are combined is output to the display 102, the real world object image is provided to the second layer and the real world background image is provided to the third layer. can

After step S110, in step S120 of outputting the object image for synthesis stored in the memory 104 to the display 102, the object image for synthesis output to the display 102 is provided to the first layer and output. In step S120, the object image for synthesis is output to the display 102 where the real world image is output, and the object image for synthesis output to the display 102 is similar to the embodiment shown in FIGS. 6 and 7 or FIGS. 8 and 9. It may be output in the form of other embodiments such as

6 is an exemplary diagram illustrating an embodiment of outputting a real world image and an object image for synthesis generated according to the present invention to a display. As shown in (a) of FIG. 6, on the display 102 provided in the mobile terminal 100, the real world image 110 generated by the camera module 101 capturing a real world scene is disposed on a second layer. output

As shown in (b), the object image for synthesis 210-1 (object B) is an image in which only the image of object B exists. The object image for synthesis 210-1 (object B) includes only image information about object B, and background information other than object B is not included. The object image for synthesis 210-1 (object B) is arranged on the first layer and output to the display 102.

When the real world image 110 including the real world object image (object A) of (a) and the object image for synthesis 210-1 (object B) of (b) are output together on the display 102, (c) As such, the real world image 110 provided to the second layer and the object image for synthesis (object B) provided to the first layer located in front of the second layer are included together and output to the display 102 .

7 is an exemplary diagram illustrating another embodiment of outputting a real world image and an object image for synthesis generated according to the present invention to a display. As shown in (a) of FIG. 7, on the display 102 provided in the mobile terminal 100, the real world image 110 generated by the camera module 101 capturing a real world scene is disposed on a second layer. output

As shown in (b), the image for synthesis 210-2 (object B) includes an object image for synthesis (object B) and an image obtained by performing a transparent process on the remaining images. The image for synthesis 210-2 (object B) also includes image information about object B and transparency processing information about the remaining transparent images. The composite image 210 - 2 (B) is arranged on the first layer and output to the display 102 .

According to one embodiment, the term "image for synthesis 210-2" used to indicate an image for synthesis including an object image for synthesis (object B) and the remaining transparent images includes the object image for synthesis. Therefore, it can be expressed as "object image for synthesis 210-2" or "object image for synthesis 210-2, object B" as needed.

When the real world image 110 including the real world object image (object A) of (a) and the image for synthesis 210-2 (object B) of (b) are output to the display 102 together, (c) and Likewise, the real world image 110 provided to the second layer and the object image for synthesis (object B) provided to the first layer located in front of the second layer are included together and output to the display 102 . At this time, since the transparently processed image of the rest of the images other than the object image for synthesis of the image for synthesis 210-2 (object B) is output transparently on the display 102, among the real world images 110 located in the second layer, The region where the object image (object B) is located is covered, and the real world object image (object A) except for the object image (object B) for synthesis and the real world background image are all normally output.

As shown in (c) of FIGS. 6 and 7, the real world image 110 continuously photographed and output in real time and the object image 210-1 or image 210-2 for synthesis are displayed on the display 102 , and the mobile terminal 100 displays the real world image 110 and the object image 210 for synthesis, which are automatically and continuously output to the display 102 in real time, when a command requesting generation of a photographic image is input. 1) Alternatively, the photographed images 130-1 and 130-2 are created by combining the images 210-2 for synthesis. The photographed images 130-1 and 130-2 to be combined are characterized in that they are either one frame or a plurality of frames, respectively, of the real world image 110 and the object image for synthesis (object B).

In the embodiments of FIGS. 6 and 7 , the object image 210-1 for synthesis or the image 210-2 for synthesis including the object image for synthesis is included in the real world image 110 shown in (a) of FIG. It was explained that it was output together.

In the present invention, the term "outputting the object image for synthesis to the display" includes outputting the image for synthesis to the display and consequently outputting the object image for synthesis.

Meanwhile, a method of generating the object image 210-1 for synthesis or the image 210-2 for synthesis may be performed in the following three forms. First, in order to generate the object image 210-1 for synthesis or the image 210-2 for synthesis, the mobile terminal 110 captures another real world scene to generate a real world image for synthesis, and the generated real world for synthesis. An object image for compositing can be extracted from an image. The extracted object image for synthesis (object B) may include only the object image for synthesis 210-1 excluding the other images except for the object image for synthesis (object B). Alternatively, an image for synthesis 210-2 including an object image for synthesis (object B) and an image in which transparency processing is performed on the remaining images may be generated.

Second, a real world image for synthesis capable of generating an object image for synthesis is captured by another mobile terminal (not shown), and the mobile terminal 100 receives and synthesizes the real world image for synthesis from another mobile terminal (not shown). It is possible to extract a dragon object image.

Third, a real world image for synthesis capable of generating an object image for synthesis is captured by another mobile terminal (not shown). Another mobile terminal (not shown) extracts an object image for synthesis from a real world image for synthesis. Another mobile terminal (not shown) may transmit the extracted object image for synthesis to the mobile terminal 100, and the mobile terminal 100 may receive and use the object image for synthesis.

8 is an exemplary diagram illustrating an embodiment of outputting a real world object image including a real world object image and a transparent background and an object image for synthesis to a display according to the present invention. As shown in (a) of FIG. 8, on the display 102 provided in the mobile terminal 100, the camera module 101 captures a real world scene and displays a real world object image and a transparent background among real world images. A real-world image 112 containing the data is output. In (a) of FIG. 8, the output of the real world image 112 is obtained by performing transparent processing on the remaining images except for the real world object image (object A) as shown in (d) of FIG. This is done by outputting only the generated real world image 112 to the display 102 .

Alternatively, in another embodiment, the output of the real world image (not shown) is displayed 102 by excluding the output of the other images except for the real world object image (object A) from the real world image that is captured and generated (102). ) by outputting

Accordingly, the real world image 112 including the object A shown in (a) of FIG. 8 and the remaining images subjected to the transparent processing or a real world image (not shown) including only the object A is disposed on the second layer and displayed. It is output to (102).

As shown in (b), the object image for synthesis 210-1 (B) is an image in which only the image of object B exists. The object image for synthesis 210-1 (B) includes only image information about object B, and background information other than object B is not included. The object image for synthesis 210-1 (object B) is arranged on the first layer and output to the display 102.

When the real world image 112 including only the real world object image (object A) of (a) and the object image for synthesis 210-1 (object B) of (b) are output together on the display 102, (c) As such, the real world image 110 provided to the second layer and the object image for synthesis (object B) provided to the first layer located in front of the second layer are included together and output to the display 102 .

9 is an exemplary view illustrating another embodiment of outputting a real world object image including a real world object image and a transparent background and an object image for synthesis to a display according to the present invention. As shown in (a) of FIG. 9 , on the display 102 provided in the mobile terminal 100, the camera module 101 captures a real-world scene and displays a real-world object image and a transparent background among real-world images. The real world image 112 including the real world image 112 is arranged on the second layer and output. In (a) of FIG. 9, the output of the real world image 112 is only the real world image 112 generated by performing the transparent process on the remaining images except for the object A as shown in (d) of FIG. This is done by outputting to the display 102.

Alternatively, the output of a real-world image (not shown) in another embodiment excludes the output of the remaining images except for the real-world object image (object A) from the real-world image generated by capturing, or deletes the remaining images to obtain only the real-world object image (object A). This is done by outputting to the display 102.

Accordingly, the real world image 112 consisting of the object A shown in (a) of FIG. 9 and the remaining images subjected to the transparent processing or a real world image (not shown) including only the object A is disposed on the second layer and displayed ( 102) is output.

As shown in (b), the image for synthesis 210-2 includes an object image for synthesis (object B) and an image obtained by performing a transparent process on the remaining images. The image for synthesis 210-2 (B) also includes image information on the object image (object B) for synthesis and transparency processing information on the remaining transparent images. The composite image 210 - 2 (B) is arranged on the first layer and output to the display 102 .

When the real world image 112 including the real world object image (object A) of (a) and the image for synthesis 210-2 (object B) of (b) are output to the display 102 together, (c) and Similarly, the real world image 110 provided to the second layer and the image for synthesis 210-2 (object B) provided to the first layer located in front of the second layer are included together and output to the display 102.

As shown in (c) of FIGS. 8 and 9 , the real world image 112 including object A continuously photographed and output in real time and the rest of the transparently processed image or only object A and the object image 210-1 for synthesis Alternatively, the image for synthesis 210-2 including the object image for synthesis (object B) and the rest of the transparent image is output to the display 102, and the mobile terminal 100 requests generation of a photographed image. is input, or the real world image 112 and the object image 210-1 for synthesis or the object image for synthesis (object B) and the remaining transparent images are automatically output to the display 102 continuously in real time. The photographic images 130-3 and 130-4 are created by combining the images for synthesis 210-2 including . The combined photographic images 130-1 and 130-2 are characterized in that they are either one frame or a plurality of frames of the real world image 110 and the object images 210-1 and 210-2 for synthesis.

In the embodiments of FIGS. 8 and 9 , it has been described that the object image 210-1 for synthesis or the image 210-2 for synthesis is output together with the real world image 112 shown in FIG. 5(b).

In this specification, the term “image for synthesis” or “image for synthesis 210-2” includes object images for synthesis, so “image for synthesis 210-2” or “image for synthesis 210-2, object B)”.

After the object image for synthesis is provided to the first layer in step S120 and output to the display 102, it is selected whether or not the output object image for synthesis is disposed behind the real world object image (S131).

In step S131, selection of the rear arrangement of the object image for synthesis is received from the user through an input module (not shown) of the mobile terminal 100, and the processor 103 converts the object image for synthesis into a real world object according to the input rear arrangement selection. Place it behind the image.

At this time, a process of selecting and extracting a real world object image serving as a reference for rear arrangement is performed from the real world image output to the display 102 before or after the rear arrangement is selected in step S131.

Referring to (a) of FIG. 5 , in order to extract a real-world object image (object A) from the real-world image 110, one or more objects present in the real-world image 110 must be recognized. Object A may exist in the real world image 110 , but when two or more objects exist, object A to be extracted from the real world image 110 must be selected. Alternatively, two or more objects may be selected.

An object selected from the real world image 110 may be automatically recognized and selected by the processor 103 or an object designated by the user may be selected.

A method of automatically selecting an object from the real world image 110 may use an object recognition method among image processing methods. Object recognition extracts feature points of each pixel in the real world image 110 . The contour of the object is determined using the set of extracted feature points. There is a method of recognizing and extracting a set of feature points corresponding to the contour of the determined object as an object.

In addition, when the object to be extracted is a person, there is a method of extracting the outline of the person through face recognition and recognizing and extracting the extracted image of the outline of the person as a person object.

Also, as a method of automatically recognizing an object in an image, any known image processing algorithm may be used.

A method of selecting an object by the user to extract the real world object image 111 from the real world image 110 is as follows. The user selects an object to be selected in the real world image 110 output to the display 102 of the mobile terminal 100 through an input such as a touch. The processor 103 analyzes the selected object and recognizes the outline of the object. Next, an object corresponding to the recognized hexagonal stone may be extracted from the real world image 110 .

At this time, when the user selects an object, the object at the touched position is selected by touching the position of the object output on the display 102 . Alternatively, the object may be selected by using continuous touch input in a drag and drop method of the outline of the object to be selected by the user.

The processor 103 may recognize an object using the selected object or its contour, and extract the recognized object from the real world image 110 .

In addition, a method other than a method for selecting a real world object image from the real world image 110 by the user may be used.

In addition, before or after the rear arrangement is selected in step S131, the size or position of the real world object image and the object image for synthesis output to the display 102 may be adjusted.

In addition, any one of a real world object image for which transparent processing has been performed on the background image shown in FIG. 8 or FIG. 9, a real world object image excluding the output of the background image, or a real world object image from which the background image has been deleted, and an object image for synthesis are included. If output, a background image to be included in the photographed image may be selected before or after the rear arrangement of step S131 is selected. The selected background image is arranged on the third layer and output to the display 102 . The selected background image output to the display 102 is included in the photographed image.

10 is an exemplary diagram illustrating an embodiment for adjusting the position of a real-world object image using the motion of a mobile terminal according to the present invention, and FIG. 11 is a real-world object whose position is adjusted using the motion of a mobile terminal according to the present invention. It is an exemplary diagram showing overlapping of an image and an object image for synthesis.

As shown in FIGS. 10 and 11 , when a real world object image (object A) is located in a specific position in the real world image 121 where the object image for synthesis (object B) is disposed, the user moves the mobile terminal 100 By rotating (changing the camera angle), the real-world object image (object A) can be placed at a specific location in the real-world image.

That is, as shown in (a) of FIG. 10 , a real world image 110 including an object image for synthesis (object B) and a real world object image (object A) is output to the display 102 of the mobile terminal 100 . When the real world object image (object A) in the real world image 110 is located at the center of the road, the user may move the angle of the camera module 101 provided in the mobile terminal 100 to the left. The real-world object image (object A) and the background image in the real-world image 110 output to the display 102 according to the angle moved to the left have the same effect as moved to the right, as shown in FIG. 10(b). indicate There is no movement of the object image (object B) for compositing. Accordingly, when the real-world object image (object A) is moved to the right and outputted, the same effect as when the object image for synthesis (object B) disposed in the real-world image 110 is separated from the object image (object B) is separated.

Meanwhile, the user may move the angle of the camera module 101 provided in the mobile terminal 100 to the right. The real-world object image (object A) and background image in the real-world image 110 output to the display 102 according to the angle moved to the right have the same effect as moved to the left, as shown in FIG. 10(c). indicate When the real-world object image (object A) is moved to the left and outputted, the same effect is obtained as if the distance to the object image for synthesis (object B) arranged in the real-world image 110 is adjacent to each other.

An embodiment in which a real world object image (object A) and an object image for synthesis (object B) are overlapped according to a change in the camera angle of the mobile terminal 100 is illustrated in FIG. 11 . When a command for overlapping a real-world object image (object A) output to the display 102 and an object image for synthesis (object B) is received, the processor 103 converts one object image to another object image. It can be placed in the front or in the back.

In step S131, when a rear arrangement selection is input to the mobile terminal 100 so that the object image for synthesis (object B) is superimposed on the rear surface of the real world object image (object A), the processor 103 ( As shown in a), the object image for synthesis (object B) provided in the first layer is arranged to overlap the rear surface of the real world object image (object A) provided in the second layer (S132).

Thereafter, in step S132, when the arrangement of the real world object image and the object image for synthesis is completed, and the adjustment of the size and position of the real world object image and the object image for synthesis is also completed, the mobile terminal 100 outputs to the display 102. A photographed image is created including an object image for synthesis and a real world object image (S140).

11(a) shows a real world object image (object A) included in the real world image 143 disposed on the second layer and an object image for synthesis (object B) disposed on the first layer overlapped with each other to create an object for synthesis. This is an embodiment in which an image (object B) is placed behind a real-world object image (object A).

An object image (object B) for synthesis is output to the display 102 of the mobile terminal 100 on which the real world image 143 is output. When the mobile terminal 100 rotates, the camera angle is changed so that the object image for synthesis (object B) and the real world object image (object A) may overlap each other. When the object image for synthesis (object B) is selected to be positioned behind the real-world object image (object A), the processor 103 converts the object image for synthesis (object B) provided to the first layer into a real-world object provided to the second layer. Arrange it so that it is located behind the image (object A).

Alternatively, first, the object image for synthesis (object B) provided in the first layer is selected so that the object image for synthesis (object B) is located on the back of the real-world object image (object A), and the real-world object image (object B) provided in the second layer is selected. Arrange it so that it is located on the back of A). Next, the camera angle may be changed so that the object image for synthesis (object B) and the real world object image (object A) overlap each other.

As shown in (c) of FIG. 8 or 9 , when a real world object image (object A) and an object image for synthesis (object B) are superimposed and output to the display 102, the processor 103 processes the image included in the real world image. A real world background image can be provided to the third layer and output. Alternatively, another background image stored in the memory 104 that does not include the real world background image may be provided to the third layer and output to the display 102 . Alternatively, the background image of the object image for synthesis may be provided to the third layer and output to the display 102 without including the real world background image.

The real world background image provided to the third layer, another background image stored in the memory 104, or a background image of an object image for synthesis is output to the display 102 and included in the generated photographic image.

In this case, the background image of the object image for synthesis is an image stored by extracting the object image for synthesis from the captured real world image and setting the remaining images as the background image to generate the object image for synthesis. Alternatively, the object image for synthesis may be generated through graphics or animation.

If, in step S131, if the rear arrangement selection is not input to the mobile terminal 100 so that the object image for synthesis (object B) is superimposed on the rear surface of the real world object image (object A), the processor 103 As shown in (b) of 11, a photographic image is created using the object image for synthesis (object B) provided to the second layer located in front of the real world object image (object A) provided to the second layer ( S140). Before generating the photographed image, the size and position of the real world object image and the object image for compositing may be adjusted.

11(b) shows that the real world object image (object A) included in the real world image 144 and the object image for synthesis (object B) overlap each other so that the object image for synthesis (object B) becomes a real world object image (object A). ) is an embodiment disposed in front of.

An object image (object B) for synthesis is output to the display 102 of the mobile terminal 100 on which the real world image 144 is output. When the mobile terminal 100 rotates, the camera angle is changed so that the object image for synthesis (object B) and the real world object image (object A) may overlap each other. When the object image for synthesis (object B) provided in the first layer is located in front of the real world object image (object A) provided in the second layer, the processor 103 may generate a photographic image in the arrangement state. there is.

If the real-world object image (object A) provided to the second layer is located behind the object image (object B) for synthesis, the arrangement of the object images provided to each layer is changed so that the object image (object B) for synthesis is created in the real world. It is rearranged so that it is located in front of the object image (object A).

The arrangement of each object image is changed according to a command input to the mobile terminal 100 by the user.

12 is an exemplary diagram illustrating an embodiment in which a real world object image and an object image for synthesis are overlapped and disposed according to the present invention. As shown in (a) of FIG. 12, when the object image 210 for synthesis provided in the first layer is superimposed on the back of the real world image 112 provided in the second layer, in (b) As shown, the object B of the object image for synthesis 210 may be positioned behind the object A of the real world image 112 . In this case, the photographed image created may provide the same effect as if the object B is located behind the object A and the object A is photographed together.

As shown in (c) of FIG. 12, when the object image 210 for synthesis provided in the first layer is superimposed on the front of the real world image 112 provided in the second layer, in (d) As shown, the object B of the object image for synthesis 210 may be positioned in front of the object A of the real world image 112 . In this case, the photographed image created may provide the same effect as if the object B is positioned in front of the object A and the object A is photographed together.

In the present invention, when combining a real-world object image including each object A and an object image for synthesis including object B into one photographed image so that the positions where each object is placed overlap, the real-world object image and The size and position of the object image for synthesis can be adjusted.

The real-world object image is generated through a real-world scene captured through the camera module 101 of the mobile terminal 100, and by enlarging or reducing the real-world image generated through control of the zoom function of the camera module 101. , the size of the real world object image can be controlled.

Position adjustment of the real-world object image is performed by changing the camera angle of the camera module 101 through a control device (not shown) that controls the movement of the user or the mobile terminal 100, and through this, the real-world object image included in the real-world scene When the position of the corresponding person is adjusted, the position of the real world object image within the real world image output to the display 102 may be adjusted.

The object image for synthesis is an image stored in the memory 104 output to the display 102. When the display 102 includes a touch module, the object image for synthesis is output by performing a touch or drag input to the display 102. The position of the object image can be changed.

When the touch module is included in the display 102, the enlargement and reduction of the object image for synthesis is generally performed according to the variation of the distance between two touched points while being used in the mobile terminal 100. For example, when the distance between two points input to the display 102 increases, the processor 103 enlarges the output object image for synthesis. If the distance between the two points becomes closer, the processor 103 reduces the output object image for synthesis.

Although it has been described that the position and size adjustment of the object image for synthesis is performed through the touch module provided in the display 102, the object image for synthesis can also be adjusted through a separate input module (not shown) provided in the mobile terminal 100. You can enter commands for positioning and resizing.

13 is an exemplary diagram illustrating an embodiment for adjusting the position of a real-world object image using the motion of a real-world person photographed in a mobile terminal according to the present invention. FIG. It is an exemplary diagram showing the overlapping of the position-adjusted real-world object image and the object image for synthesis.

As shown in FIGS. 13 and 14 , when the camera angle of the mobile terminal 100 is fixed in the real world image 110 in which the object image for synthesis (object B) is disposed, a person in the real world corresponding to object A ( 510) moves in the real world, the real world object image (object A) output from the display 102 may move to the periphery of the object image for synthesis (object B) output together on the display 102.

That is, in the first output real world image 110, the real world object image (object A) and the object image for synthesis (object B) are output as shown in (a). In this case, as shown in FIG. 4 , when the real world person 510 in the center of the real world surrounding background 520 moves to the left side of the person, as shown in FIG. Object A in the real world image 151 is moved to the right in the real world image and is output to be spaced apart from the object image for synthesis (object B) output on the left side of the display 102 .

Also, as in (c), when the real world person 510 corresponding to object A moves to the left in the real world, object A in the real world image 152 is output adjacent to the output object image (object B) for synthesis. do.

An embodiment in which a real world object image (object A) and an object image for synthesis (object B) are overlapped according to the movement of the real world person 510 located in the real world is shown in FIG. 14 . When the real world object image (object A) output to the display 102 and the object image for synthesis (object B) are overlapped, one object image may be positioned in front or behind the other object image. there is.

14(a) shows that the real world object image (object A) included in the real world image 153 and the object image for synthesis (object B) overlap each other so that the object image for synthesis (object B) is a real world object image (object A). ) It is an embodiment disposed on the rear side of.

A real world image 153 including a real world object image (object A) and an object image for synthesis (object B) are output to the display 102 of the mobile terminal 100 . The real world person 510 corresponding to the real world object image (object A) may be moved, and the object image for synthesis (object B) and the real world object image (object A) may be overlapped with each other. When the object image for synthesis (object B) is selected to be positioned behind the real-world object image (object A), the processor 103 converts the object image for synthesis (object B) provided to the first layer into a real-world object provided to the second layer. Arrange it so that it is located behind the image (object A).

Alternatively, first, the object image for synthesis (object B) provided in the first layer is selected so that the object image for synthesis (object B) is located on the back of the real-world object image (object A), and the real-world object image (object B) provided in the second layer is selected. Arrange it so that it is located on the back of A). Next, the real world person may move so that the object image for synthesis (object B) and the real world object image (object A) overlap each other.

As shown in (c) of FIG. 8 or 9 , when a real world object image (object A) and an object image for synthesis (object B) are superimposed and output to the display 102, the processor 103 processes the image included in the real world image. A real world background image can be provided to the third layer and output. Alternatively, another background image stored in the memory 104 that does not include the real world background image may be provided to the third layer and output to the display 102 . Alternatively, the background image for synthesis of the object image for synthesis may be provided to the third layer or the fourth layer and output to the display 102 without including the real world background image.

14(b) shows that the real world object image (object A) included in the real world image 154 and the object image for synthesis (object B) overlap each other so that the object image for synthesis (object B) is a real world object image (object A). ) is an embodiment disposed in front of.

A real world image 154 including a real world object image (object A) and an object image for synthesis (object B) are output to the display 102 of the mobile terminal 100 . In the real world scene, the real world person 510 may move, and the object image for compositing (object B) and the real world object image (object A) may overlap each other. When the object image for synthesis (object B) provided in the first layer is positioned in front of the real world object image (object A) provided in the second layer, a photographic image may be created in the arrangement state.

If the real-world object image (object A) provided to the second layer is located behind the object image (object B) for synthesis, the arrangement of the object images provided to each layer is changed so that the object image (object B) for synthesis is created in the real world. It is rearranged so that it is located in front of the object image (object A).

15 is an exemplary view showing an embodiment for adjusting the position of an object image for synthesis in a mobile terminal according to the present invention, and FIG. 16 is a graph showing the position-adjusted object image for synthesis and a real world object image in a mobile terminal according to the present invention. It is an example diagram showing overlapping.

As shown in FIGS. 15 and 16 , when the camera angle of the mobile terminal 100 is fixed in the real world image 110 in which the object image for synthesis (object B) is arranged, the object image for synthesis (object B) is arranged. ) can be moved to the periphery of the output real-world object image (object A). In the real world scene of FIG. 4 , when the real world person 510 located in the center of the real world surrounding background 520 is fixed, the real world image 110 is output as shown in FIG. 15 (a).

At this time, as shown in (b) of FIG. 15, the object image for synthesis (object B) is moved to the right side of the real world object image (object A) within the real world image 161, or as shown in (c), the real world image ( 162), the object image for synthesis (object B) may be moved to the left side of the real world object image (object A).

As described in FIG. 12, the movement of the object image for synthesis (object B) output to the display 102 is performed by performing a touch or drag input to the display 102 when the display 102 includes a touch module. The position of the output object image for synthesis can be changed. Alternatively, the location may be changed using an input module (not shown) provided in the mobile terminal 100 .

An embodiment in which a real-world object image (object A) and an object image for synthesis (object B) are overlapped according to the positional movement of the object image (object B) for synthesis is illustrated in FIG. 16 .

When the real world object image (object A) and the object image for synthesis (object B) are overlapped according to the positional movement of the object image for synthesis (object B) output on the display 102, one object image is converted to another object image for synthesis. It can be placed in front or behind the object image.

16(a) shows that the real world object image (object A) included in the real world image 163 and the object image for synthesis (object B) overlap each other so that the object image for synthesis (object B) is a real world object image (object A). ) It is an embodiment disposed on the rear side of.

A real world image 163 including a real world object image (object A) and an object image for synthesis (object B) are output on the display 102 of the mobile terminal 100, and the object image for synthesis (object B) is moved When the object image (object B) for compositing is selected to be overlapped with the real world object image (object A) and positioned behind the real world object image (object A) according to The object image (object B) is placed behind the real world object image (object A) provided in the second layer.

Alternatively, first, the object image for synthesis (object B) provided in the first layer is selected so that the object image for synthesis (object B) is located on the back of the real-world object image (object A), and the real-world object image (object B) provided in the second layer is selected. Arrange it so that it is located on the back of A). Next, the object image for synthesis (object B) may be moved so that it overlaps with the real world object image (object A).

As shown in (c) of FIG. 8 or 9 , when a real world object image (object A) and an object image for synthesis (object B) are superimposed and output to the display 102, the processor 103 processes the image included in the real world image. A real world background image can be provided to the third layer and output. Alternatively, another background image stored in the memory 104 may be provided to the third layer and output to the display 102 without including the real world background image. Alternatively, without including the real world background image, the background image for synthesis of the object image for synthesis is provided to the third layer or the fourth layer and output to the display 102 .

16(b) shows that the real world object image (object A) included in the real world image 164 and the object image for synthesis (object B) overlap each other so that the object image for synthesis (object B) becomes a real world object image (object A). ) is an embodiment disposed in front of.

A real world image 164 including a real world object image (object A) and an object image for synthesis (object B) are output to the display 102 of the mobile terminal 100 . The object image for synthesis (object B) may be moved and overlapped with the real world object image (object A). When the object image for synthesis (object B) provided in the first layer is positioned in front of the real world object image (object A) provided in the second layer, a photographic image may be created in the arrangement state.

If the real-world object image (object A) provided to the second layer is located behind the object image (object B) for synthesis, the arrangement of the object images provided to each layer is changed so that the object image (object B) for synthesis is created in the real world. It is rearranged so that it is located in front of the object image (object A).

As described above with reference to FIGS. 10 to 16 , using a real world image including a real world object image continuously output to the display 102 of the mobile terminal 100 in real time and an object image for synthesis output together with the real world image, You can create a photo shoot image. At this time, according to the change of the camera angle of the mobile terminal 100, the movement of the real-world person corresponding to the real-world object image, or the movement of the position of the object image for synthesis output together with the real-world image, real-world object images and composites at different locations are used. A photographed image may be generated using the object image.

15(c) is a result screen obtained by moving the object image for synthesis (object B) to the left side of the real world object image (object A) and additionally moving down the screen, and FIG. 16(a) shows the object image for synthesis The image (object B) is placed behind the real-world object image (object A), and it is the result of additional movement on the screen to highlight the perspective.

Alternatively, as shown in (a) and (b) of FIG. 11, when it is determined that the real world object image (object A) is disposed on either the front or rear side of the object image (object B) for synthesis, the real world person In the real world, it can be moved to highlight the three-dimensional effect or perspective in being located on the front or back of the object image (object B) for synthesis through movement.

Although not shown in each embodiment, when a real world object image and an object image for synthesis are overlapped and output, when one object image is placed on the front side and the other image is placed on the back side, according to each arrangement type Each object image may be enlarged or reduced so that a three-dimensional effect or perspective may be emphasized.

After step S140, the mobile terminal 100 of the present invention uses the real world object image (object A) and the object image for synthesis (object B) to generate a photographed image, which is an object for synthesis in another mobile terminal (not shown). The photographed image is transmitted to another mobile terminal so that it can be reused as an image.

Another mobile terminal receiving the photographed image may reuse the received photographed image as an object image for synthesis. Another mobile terminal may generate another photographic image by using a photographic image, which is an object image for synthesis, and a real world image generated by photographing a real world scene by a camera module of another mobile terminal.

If a photographed image transmitted from the mobile terminal 100 to another mobile terminal includes a background image, the other mobile terminal uses the photographed image excluding the background image as an object image for synthesis.

Also, when the mobile terminal 100 generates a photographic image by receiving an object image for synthesis from another mobile terminal, the mobile terminal 100 transmits the generated photographic image to the other mobile terminal that has provided the object image for synthesis. By transmitting, the user of another mobile terminal can check the generated photo-taking image.

In addition, when the mobile terminal 100 generates a final photographed image by using photographic images sequentially generated through a plurality of other mobile terminals as object images for compositing, the plurality of photographic images provided respectively The generated final photo-taking image is transmitted to each of the other mobile terminals so that the user of each other mobile terminal can check the final photo-taking image.

The description of the invention in this specification describes preferred embodiments, and the present invention is not limited to these embodiments. Those skilled in the art to which the present invention pertains can make various changes and modifications to the above embodiments without departing from the technical spirit of the present invention, and the technical spirit of the present invention is all of these various changes and modifications. include

100: mobile terminal 101: camera module
102: display 103: processor
104: memory 105: communication module
110, 112, 141 to 144, 151 to 154, 161 to 164: real world image
113 : real world background image
130-1, 130-2, 130-3, 130-4: photographing image
210-1: object image for synthesis 210-2: image for synthesis
510: real world character 520: real world background
A: Object AB: Object B

Claims (26)

A method of generating a photographic image by performing the following steps in a mobile terminal including a camera module, display, communication module, processor, and memory:
continuously outputting a real-world image including a real-world object image generated by continuously capturing a real-world scene by the camera module in real-time on the display;
outputting the object image for synthesis stored in the memory to the display;
Arranging the object image for synthesis and the real-world object image on the display, when the object image for synthesis is selected to be overlapped with the rear surface of the real-world object image, the real-world object image continuously output in real time including overlapping and arranging the object image for synthesis on a rear surface; and
generating a photographed image including the object image for synthesis and the real world object image arranged on the display;
Performing, providing the object image for synthesis to be located on a first layer, and providing the real-world object image to be located on a second layer,
In the placement step,
When a command is received to place the object image for synthesis and the real-world object image located on the first layer and the second layer on the display, and the object image for synthesis to be located behind the real-world object image, the A method of locating a first layer on a rear surface of the second layer so that the object image for synthesis is superimposed on the rear surface of the real world object image. The method of claim 1, after outputting the real world image to the display,
The method of claim 1 , further comprising: selecting the real world object image to be overlapped with the object image for synthesis from the real world image. delete According to claim 1,
and extracting only the real-world object image from the real-world image, and placing the extracted real-world object image in the second layer. According to claim 1,
The method of claim 1 , further comprising transparently processing images other than the real-world object image from the real-world image, and locating the real-world object image and the transparently-processed image in the second layer. According to claim 4,
Wherein the real-world background images other than the real-world object image extracted from the real-world image are placed in a third layer. 7. The method of claim 6, wherein in the generating of the photographic image,
The method of generating the photographed image by further including the real world background image of the third layer. The method of claim 1, wherein in the generating of the photographic image,
The method of claim 1 , wherein the memory includes one or more background images, and further includes a selected background image among the background images to generate the photographed image. According to claim 1,
and controlling the size or position of the object image for synthesis output to the display according to a control command input to the mobile terminal. According to claim 1,
The method of claim 1 , wherein the object image for synthesis is generated in the mobile terminal or another mobile terminal and is provided through the communication module. According to claim 10,
The method of claim 1 , wherein the object image for synthesis includes only an object image excluding a background image. According to claim 10,
When the object image for synthesis is generated, the remaining images except for the object image for synthesis are created as background images. According to claim 1,
Transmitting the photographic image in which the object image for synthesis and the real world object image are arranged as an object image for synthesis to another mobile terminal using the communication module,
Wherein the other mobile terminal generates a photographic image including the received object image for synthesis and a real world object image of the other mobile terminal using a processor performing the same steps as each of the above steps. . A mobile terminal for generating a photographic image using a camera module, a display, a communication module, a processor, a memory, and one or more programs stored in the memory and configured to be executed by the processor,
The one or more programs,
Real-world images including real-world object images generated by continuously photographing real-world scenes by the camera module in real-time are continuously output to the display in real-time;
outputting an object image for synthesis stored in the memory to the display;
When the object image for synthesis and the real world image are disposed on the display, and the object image for synthesis is selected to be overlapped with the rear surface of the real world object image, the rear surface of the real world object image continuously output in real time Object images for synthesis are overlapped and placed,
Including instructions for generating a photographic image including the object image for synthesis and the real world object image disposed on the display,
Providing the object image for synthesis to be located on a first layer and providing the real world object image to be located on a second layer;
The arrangement is
When a command is received to place the object image for synthesis and the real-world object image located on the first layer and the second layer on the display, and the object image for synthesis to be located behind the real-world object image, the A mobile terminal that places a first layer on a rear surface of the second layer so that the object image for synthesis is superimposed on the rear surface of the real world object image. 15. The method of claim 14, wherein the one or more programs,
and a command for further performing selecting, from the real world image, the real world object image to be overlapped and arranged with the object image for synthesis after outputting the real world image to the display. delete 15. The method of claim 14,
The mobile terminal, characterized in that extracting only the real-world object image from the real-world image and placing the extracted real-world object image in the second layer. 15. The method of claim 14,
The mobile terminal according to claim 1 , wherein the remaining images except for the real-world object image are transparently processed in the real-world image, and the real-world object image and the transparently processed image are positioned on the second layer. 18. The method of claim 17,
The mobile terminal of claim 1 , wherein a real world background image excluding the real world object image extracted from the real world image is placed in a third layer. The method of claim 19, when generating the photographic image,
The mobile terminal of claim 1 , further comprising the real world background image of the third layer to generate the photographed image. 15. The method of claim 14, when generating the photographic image,
The mobile terminal of claim 1 , wherein the memory includes one or more background images, and further includes a background image selected from among the background images to generate the photographed image. 15. The method of claim 14,
The mobile terminal characterized in that for controlling the size or position of the object image for synthesis output to the display according to the control command input to the mobile terminal. 15. The method of claim 14,
The mobile terminal according to claim 1 , wherein the object image for synthesis is generated in the mobile terminal or another mobile terminal and is provided through the communication module. 24. The method of claim 23,
The mobile terminal, characterized in that the object image for synthesis includes only the object image excluding the background image. 24. The method of claim 23,
The mobile terminal, characterized in that, when the object image for synthesis is generated, images other than the object image for synthesis are generated as background images. 15. The method of claim 14,
Transmitting the photographed image in which the object image for synthesis and the real world object image are arranged to another mobile terminal using the communication module so that it can be used as an object image for synthesis;
The other mobile terminal includes the object image for synthesis, in which the photographed image is received as an object image for synthesis using one or more programs including the same commands as the commands, and the real world object image of the other mobile terminal. A mobile terminal characterized in that for generating a photo-taking image to.

Images