KR101687616B1 - Method for providing video call service in network tv and the network tv - Google Patents

Abstract

A method of providing a video call service using a network TV according to an exemplary embodiment of the present invention includes receiving broadcast data through a broadcast network interface, controlling the received broadcast data to be output, Selecting a type and position of an additional image displayed in a first area in which the photographed user is displayed when the additional video service mode is being executed; Configuring a data set used in the video processing of the additional image, and transmitting the original video data photographed through the camera and the configured data set to a partner device in the video call mode .

Description

METHOD FOR PROVIDING VIDEO CALL SERVICE IN NETWORK TV AND THE NETWORK TV BACKGROUND OF THE INVENTION 1. Field of the Invention [

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a network television technology, and more particularly, to a method for providing a video call service using a network TV and a network TV.

Recently, various types of network TVs have been discussed. In an embodiment of the network TV, a function of receiving and processing a larger amount of information in real time via a wired / wireless Internet network is expected, unlike a conventional television receiving limited data only through a broadcasting network. More specific embodiments of the network TV may include Internet protocol television (IPTV), smart television, hybrid broadcast broadband television (HBBTV), digital television (DTV), and the like.

However, the specific functions of the network TV have not been defined yet, and a method for providing a more advanced video call system using network TV has not been developed yet.

For example, in the conventional TV, only the fixed service for transmitting the image captured by the camera is provided. Therefore, both the user who calls and the user who receives the telephone often have video data of stereotypes, which makes it difficult to distinguish them from mobile phone video calls.

In addition, according to the related art, there is a limitation that the data processing time of the video call is delayed due to the limited performance of the TV.

An embodiment of the present invention is to provide an additional video service by bypassing the concept of simply transferring an image photographed by a camera.

In addition, another embodiment of the present invention provides a differentiated service from a mobile phone video call by implementing various modes of a video call mode.

Further, another embodiment of the present invention is to define a solution for processing the above-mentioned additional video service at a higher speed.

A method of providing a video call service using a network TV according to an exemplary embodiment of the present invention includes receiving broadcast data through a broadcast network interface, controlling the received broadcast data to be output, Selecting a type and position of an additional image displayed in a first area in which the photographed user is displayed when the additional video service mode is being executed; Configuring a data set used in the video processing of the additional image, and transmitting the original video data photographed through the camera and the configured data set to a partner device in the video call mode .

The network TV providing video call service according to an embodiment of the present invention includes a call connection module for transmitting / receiving a video call connection signal, a broadcast network interface module for receiving broadcast data, And a controller for processing the video call connection signal. Further, in accordance with the control of the control unit, switching to the video communication mode is performed, and the original video data photographed by the camera and the first area where the original video data is displayed The identification information of the additional image being processed and the data set used in the video processing of the additional image, and to process the additional image based on the received data set.

According to an embodiment of the present invention, an additional video service is provided by bypassing the concept of simply transmitting an image captured by a camera.

In addition, according to another embodiment of the present invention, various modes of the video call mode are implemented to provide mobile phone video call and differentiated services.

According to another embodiment of the present invention, a solution for processing the additional video service described above at a higher speed is defined.

The effects of the more specific invention will be described later in detail in the following table.

1 is a diagram schematically illustrating an example of an overall broadcasting system including a network TV according to an embodiment of the present invention.
2 is a diagram showing an example of the network TV shown in FIG. 1 in more detail.
3 is a diagram illustrating an entire system including a network TV providing a video call service according to an embodiment of the present invention.
FIG. 4 is a diagram showing a first embodiment of the network TV shown in FIG.
5 is a view showing a second embodiment of the network TV shown in FIG.
FIG. 6 is a diagram illustrating a screen indicating an incoming call according to an exemplary embodiment of the present invention. Referring to FIG.
7 is a diagram illustrating a video call screen according to an embodiment of the present invention.
8 is a view illustrating a video call screen according to another embodiment of the present invention.
9 is a diagram illustrating a screen for indicating an additional video service mode according to an embodiment of the present invention.
10 is a diagram illustrating screen switching in the case where the additional video service mode corresponds to the first mode according to an embodiment of the present invention.
11 is a flowchart for explaining the process of FIG.
12 is a diagram illustrating screen switching in the case where the additional video service mode corresponds to the second mode according to an embodiment of the present invention.
13 is a flowchart for explaining the process of FIG.
FIG. 14 is a diagram illustrating screen switching when the additional video service mode corresponds to the third mode according to an embodiment of the present invention.
15 is a flowchart for explaining the process of FIG.
FIG. 16 is a diagram schematically illustrating a process in which a data set used in a video processing of an additional image is processed in a first device, according to an embodiment of the present invention.
FIG. 17 is a diagram schematically illustrating a process in which a data set used in a video processing of an additional image is processed in a second device, according to an embodiment of the present invention.
18 is a flowchart illustrating an entire process according to an embodiment of the present invention.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings. Furthermore, the suffix "module" and " part "for components used in the following description are given merely for ease of description, and the" module " It can be designed in hardware or software.

Meanwhile, the network TV described in this specification is an intelligent video display device that adds a computer support function to a broadcast reception function, for example, and is equipped with an Internet function while being faithful to a broadcast reception function, It can be equipped with a more convenient interface such as a space remote controller. Also, it can be connected to the Internet and a computer by the support of a wired or wireless Internet function, and can perform functions such as e-mail, web browsing, banking or game. A standardized general-purpose OS may be used for these various functions.

Therefore, the network TV described in the present invention can be freely added or deleted from various applications, for example, on a general-purpose OS kernel, so that various user-friendly functions can be performed. More specifically, the network TV may be, for example, an Internet TV, an HBBTV, a smart TV, a DTV, or the like, and may be applied to a smart phone in some cases.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

As used herein, terms used in the present invention are selected from general terms that are widely used in the present invention while taking into account the functions of the present invention, but these may vary depending on the intention or custom of a person skilled in the art or the emergence of new technologies. Also, in certain cases, there may be a term chosen arbitrarily by the applicant, in which case the meaning thereof will be described in the description of the corresponding invention. Therefore, it is intended that the terminology used herein should be interpreted based on the meaning of the term rather than on the name of the term, and on the entire contents of the specification.

1 is a diagram schematically illustrating an example of an overall broadcasting system including a network TV according to an embodiment of the present invention. Hereinafter, an overall broadcast system including a network TV according to an embodiment of the present invention will be schematically described with reference to FIG.

1, an overall broadcasting system including a network TV according to an embodiment of the present invention includes a content provider (CP) 10, a service provider (SP) 20, A network provider (NP) 30 and an HNED 40. The HNED 40 corresponds to, for example, a client 100 that is a network TV according to an embodiment of the present invention.

The content provider 10 produces and provides various contents. 1, a terrestrial broadcaster, a cable SO (System Operator), a MSO (Multiple System Operator), a satellite broadcaster, An Internet broadcaster, and the like may be exemplified. [0040] In addition, the content provider 10 may provide various applications in addition to the broadcast content.

The service provider 20 can provide the contents provided by the contents provider 10 in a service package. For example, the service provider 20 of FIG. 1 can package and provide the first terrestrial broadcast, second terrestrial broadcast, cable MSO, satellite broadcast, various Internet broadcasts, applications, etc. to the user.

Meanwhile, the service provider 20 can provide a service to the client 100 using a unicast or multicast method. The unicast method is a method of transmitting data 1: 1 between one sender and one receiver. For example, in a unicast method, when a receiver requests data from a server, the server can transmit data to the receiver upon request. The multicast method is a method of transmitting data to a plurality of receivers of a specific group. For example, the server can transmit data to a plurality of pre-registered receivers at once. An Internet Group Management Protocol (IGMP) protocol may be used for multicast registration.

The network provider 30 may provide a network for providing services to the client 100. The client 100 may be provided with a service by building a home network end user (HNED).

On the other hand, conditional access or content protection may be used as means for protecting transmitted contents. As an example of such a conditional access or content protection, a scheme such as a cable card (CableCARD) or a DCAS (Downloadable Conditional Access System) may be used, but the present invention is not limited thereto.

On the other hand, the client 100 can also provide the content through the network. In this case, conversely, conversely, conversely, the client 100 may be a content provider, and the content provider 10 may receive content from the client 100. [ In this case, there is an advantage that an interactive content service or a data service is possible.

2 is a diagram showing an example of the network TV shown in FIG. 1 in more detail. Hereinafter, an example of the network TV shown in FIG. 1 will be described in more detail with reference to FIG.

The network TV 200 according to an embodiment of the present invention includes a network interface 201, a TCP / IP manager 202 and a service delivery manager 203 A demultiplexer 205, a PSIP and / or SI decoder 204, an audio decoder 206, a video decoder 207, a display unit A / and OSD Module 208, a Service Control Manager 209, a Service Discovery Manager 210, a Metadata Manager 212, an SI & Metadata DB 211, A manager 214, and a service manager 213.

The network interface unit 201 receives packets received from the network and transmits packets to the network. That is, the network interface unit 201 receives services, contents, and the like from the service provider through the network.

The TCP / IP manager 202 is involved in delivering packets received from the network TV 200 and packets transmitted from the network TV 200, that is, from the source to the destination. Then, the TCP / IP manager 202 classifies the received packets so as to correspond to the appropriate protocols and classifies them into the service delivery manager 205, the service discovery manager 210, the service control manager 209, and the metadata manager 212 Lt; / RTI >

The service delivery manager 203 is responsible for controlling the received service data. For example, RTP / RTCP can be used to control real-time streaming data. When the real-time streaming data is transmitted using the RTP, the service delivery manager 203 parses the received data packet according to RTP and transmits it to the demultiplexer 205 or to the service manager 213 And stores it in the SI & Metadata DB 711. And feedbacks the network reception information to the server providing the service using the RTCP.

The demultiplexer 205 demultiplexes the received packets into audio, video, program specific information (PSI) data, and transmits them to the audio / video decoders 206 and 207 and the PSI & PSIP and / do.

The PSI & PSIP and / or SI Decoder 204 decodes service information such as PSI (Program Specific Information), for example. That is, the PSI & PSIP and / or SI decoder 204 receives and decodes the demultiplexed PSI section, the Program and Service Information Protocol (PSIP) section, or the SI (Service Information) section in the demultiplexer 205.

In addition, the PSI & PSIP and / or SI decoder 204 decodes the received sections to create a database of service information, and stores the database related to the service information in the SI & Metadata DB 211.

The audio / video decoder 206/207 decodes the video data and audio data received from the demultiplexer 205. [ The audio data decoded by the audio decoder 206 and the video data decoded by the video decoder 207 are provided to the user through the display unit 208.

The UI manager 214 and the service manager 213 manage the overall state of the network TV 200, provide a user interface, and manage other managers.

The UI manager 214 provides a GUI (Graphic User Interface) for the user using an OSD (On Screen Display) or the like, and receives a key input from the user to perform a receiver operation according to the input. For example, upon receipt of a key input from the user regarding channel selection, the key input signal is transmitted to the service manager 213.

The service manager 213 controls the manager associated with the service, such as the service delivery manager 203, the service discovery manager 210, the service control manager 209, and the metadata manager 212.

The service manager 213 also creates a channel map and selects a channel using the channel map according to the key input received from the user interface manager 214. The service manager 213 receives the service information of the channel from the PSI & PSIP and / or SI Decoder 204 and sets an audio / video PID (Packet Identifier) of the selected channel in the demultiplexer 205.

The service discovery manager 210 provides information necessary for selecting a service provider that provides the service. Upon receiving a signal regarding channel selection from the service manager 213, the service discovery manager 210 searches for the service using the information.

The service control manager 209 is responsible for selection and control of services. For example, when a user selects a service such as IGMP or RTSP when selecting a Live Broadcasting service such as an existing broadcasting method and selects a service such as VOD (Video On Demand), RTSP is used to select a service, control . The RTSP protocol may provide a trick mode for real-time streaming. Also, the service control manager 209 can initialize and manage a session through the IMC gateway using an IP Multimedia Subsystem (IMS) and a Session Initiation Protocol (SIP). The protocols are one embodiment, and other protocols may be used, depending on the implementation.

The metadata manager 212 manages the metadata associated with the service and stores the metadata in the SI & Metadata DB 211.

The SI & Metadata DB 211 selects the service information decoded by the PSI & PSIP and / or SI Decoder 204, the metadata managed by the metadata manager 212, and the service provider provided by the service discovery manager 210 Store the necessary information. In addition, the SI & Metadata DB 211 may store setup data and the like for the system.

The SI & Metadata DB 211 may be implemented using a nonvolatile RAM (NVRAM) or a flash memory.

Meanwhile, the IG 250 is a gateway that collects functions necessary for accessing an IMS-based IPTV service.

3 is a diagram illustrating an entire system including a network TV providing a video call service according to an embodiment of the present invention. Hereinafter, an overall system including a network TV providing a video call service according to an embodiment of the present invention will be described with reference to FIG.

3, the network TV 300 and 310 according to an embodiment of the present invention includes a camera 301. 311, databases 302 and 312, and the like. The cameras 301 and 311 may be used for photographing a user located in front of a TV screen and further include a microphone for video communication. Of course, a separate microphone may be used. Also, the databases 302 and 312 are all capable of storing data, for example, and are not particularly limited.

Further, the network TV 300 capable of video call service is connected to another network TV 310 through a network 340, and the network may be, for example, the Internet, a LAN, an intranet, a WAN, or the like. For example, instead of the other network TV 310, a mobile device 320, a PC 330, and the like are used.

FIG. 4 is a diagram showing a first embodiment of the network TV shown in FIG. Hereinafter, with reference to FIG. 4, the configuration modules of the network TVs 300 and 310 shown in FIG. 3 will be described in detail.

The first network TV 400 shown in FIG. 4 corresponds to a device executing an additional video service mode, and the second network TV 410 corresponds to a device for receiving and processing information for executing the additional video service mode .

The first network TV 400 includes a broadcast network interface module 401, a selection unit 402, a call connection module 403, a control unit 404, a photographing unit 405, a configuration unit 406, (407), and the like. The aforementioned modules can be designed by hardware or software, and may be designed by combining hardware and software as needed. In addition, the scope of rights of the present invention is not limited to this, and should be determined by the claims in principle.

The broadcast network interface module 401 receives the broadcast data, and the received broadcast data is output under the control of the controller 404.

Further, according to the operation of the call connection module 403, when the second network TV 410 receives a call or makes a call, the video call mode is executed. When the video call mode is executed, the photographing unit 405 photographs a user adjacent to the screen of the first network TV 400. The photographing unit may be a general camera, an RGB camera, an IR camera, or the like.

When the additional video service mode is being executed, the selecting unit 402 selects the type and position of the additional image displayed in the first area in which the photographed user is displayed.

The configuration unit 406 configures a data set used in the process of video-processing the additional image, and the transmission unit 407 transmits the original video data photographed through the photographing unit 405 and the configured data And transmits the set to the second network TV 410. As described above, the second network TV 410 corresponds to a partner device in the video call mode.

The additional image may be stored in the memory of the first network TV 400, and the memory corresponds to, for example, a database (DataBase).

According to an embodiment of the present invention, the data set is generated frame by frame, and the data set includes position information of the additional image, 3D conversion matrix information of the additional image, and scale information of the additional image do.

According to an embodiment of the present invention, the additional video service mode may include a first mode in which a specific image is simply added, a second mode in which a background is changed, or a third mode in which an image of a user is changed to an avatar image Corresponds to at least one or more.

If the additional video service mode corresponds to the first mode, the first network TV 400 overlaid the selected additional image with the original video data shot through the camera, So that data is displayed. The first mode will be described in more detail with reference to FIGS. 10 and 11. FIG.

When the additional video service mode corresponds to the second mode, the first network TV 400 separates and detects the user image and the background image from the original video data shot through the camera, The background image is replaced with the selected additional image, the user image of the original video data and the replaced additional image are synthesized, and the synthesized image is displayed. The above-described second mode will be described in more detail with reference to FIG. 12 and FIG.

If the additional video service mode corresponds to the third mode, the first network TV 400 detects a face image in the user image of the original video data shot through the camera, analyzes the detected face image , Reads the corresponding avatar image, and replaces the user image with the read avatar image. The above-mentioned third mode will be described in more detail in Fig. 14 and Fig.

The second network TV 410 includes a call connection module 411, a controller 412, a broadcast network interface module 413, and the like. The aforementioned modules can be designed by hardware or software, and may be designed by combining hardware and software as needed. In addition, the scope of rights of the present invention is not limited to this, and should be determined by the claims in principle.

The call connection module 411 transmits and receives a video call connection signal with the first network TV 400. The broadcast network interface module 413 receives broadcast data, and the controller 412 processes the received broadcast data and the video call connection signal.

Further, according to the control of the controller 412, the second network TV 410 is switched to the video call mode. In addition, the second network TV 410 receives original video data photographed by the camera from the partner device (for example, the first network TV 400) in the video communication mode and the original video data Identification information of the additional image displayed in the first area, and a data set used in the video processing of the additional image.

According to the control of the controller 412, the second network TV 410 processes the additional image based on the received data set.

According to another embodiment of the present invention, the second network TV 410 outputs the video-processed additional image and the original video data to the first area in accordance with the control of the controller 412 A display module, and a memory for storing an additional image mapped to the identification information of the additional image.

One embodiment of the present invention designed in this manner has the following effects. That is, the first network TV does not transform the image itself captured by the camera but only transmits information applied to the image synthesis. In addition, since the second network TV displays the corresponding image on the photographed original image using the received information, there is an advantage that a separate encoding / decoding process due to the added image is not necessary. Therefore, the present invention can be applied to a TV having poor performance, and an additional image processing speed can be improved.

5 is a view showing a second embodiment of the network TV shown in FIG. Hereinafter, with reference to FIG. 5, the configuration modules of the network TVs 300 and 310 shown in FIG. 3 will be described in detail.

The first network TV 500 shown in FIG. 5 corresponds to a device that executes the additional video service mode, and the second network TV 510 corresponds to a device that receives and processes information for executing the additional video service mode .

The first network TV 500 according to an exemplary embodiment of the present invention includes an image selection unit 501, a camera image processing unit 502, a sensation image synthesis unit 503, an information configuration unit 504, (505), and the like. The aforementioned modules can be designed by hardware or software, and may be designed by combining hardware and software as needed. In addition, the scope of rights of the present invention is not limited to this, and should be determined by the claims in principle.

For example, if the user of the first network TV 500 wishes to add another image to his or her image shot through the camera, the image selecting unit 501 accesses an image DB (not shown) Select additional images.

When the specific additional image is selected, the camera image processing unit 502 extracts only the user image from the video data received from the camera using a computer vision technique. To this end, object detection & object tracking techniques may be employed. Further, the camera image processing unit 502 separates the user image and the background image from the video data shot by the camera using background subtraction. In addition, the camera image processing unit 502 may be designed to recognize a facial expression among user images of the photographed video data. However, the background extracting method and the facial expression recognition method described above may employ a conventionally known technique.

The sensation image synthesis unit 503 uses, for example, an AR (Augmented Reality) technique. Accordingly, the real feeling image synthesis unit 503 synthesizes the original video data photographed by the camera with the specific additional image selected by the image selection unit, taking into consideration three-dimensional information such as the distance and direction from the camera to the user.

Further, the information constructing unit 504 constructs information for displaying the synthesized data in the counterpart device on a frame basis, and the information transmitting unit 505 transmits the configured information to the second network TV. The second network TV corresponds to a device of a video communication counterpart.

A specific method of configuring a data set for video processing of an additional image, which is one of the features of the present invention, will be described later in detail with reference to FIG.

The second network TV 510 according to an embodiment of the present invention includes an information receiving unit 511, an information extracting unit 512, an image output unit 513, and the like. The aforementioned modules can be designed by hardware or software, and may be designed by combining hardware and software as needed. In addition, the scope of rights of the present invention is not limited to this, and should be determined by the claims in principle.

The information receiving unit 511 receives the data set of the frame unit transmitted by the information transmitting unit 505 of the first network TV 500 and the information extracting unit 512 extracts Extract the dataset. Then, the video output unit 513 displays the video call screen added with the additional image using the extracted data set.

A specific method of receiving and processing a data set for video processing of an additional image, which is one of the features of the present invention, will be described later in detail with reference to FIG.

FIG. 6 is a diagram illustrating a screen indicating an incoming call according to an exemplary embodiment of the present invention. Referring to FIG. 7 is a diagram illustrating a video call screen according to an embodiment of the present invention. 8 is a view illustrating a video call screen according to another embodiment of the present invention. 9 is a diagram illustrating a screen for indicating an additional video service mode according to an embodiment of the present invention. Hereinafter, a general broadcast screen, a video call screen, and an additional video service mode will be sequentially described with reference to FIG. 6 to FIG.

As shown in FIG. 6, the network TV according to an exemplary embodiment of the present invention displays a general broadcasting screen 601, and when a call is received from a partner device, information 602 for displaying the information is displayed. However, although FIG. 6 exemplifies a situation in which a telephone call is received, the present invention may be applied to the reverse.

7, when the information 602 shown in FIG. 6 is accepted or confirmed, the general broadcasting screen 701, the user's own video data 702, and the other's video data 703, Is displayed. However, the video data is an image captured through a camera connected to a network TV or other device, for example.

8, the general broadcast screen is disappeared and the video data 801 of the user and the video data 802 of the other party are displayed as shown in FIG. 8, Is displayed.

Furthermore, as shown in FIG. 9, when the user starts a video call mode or when there is a user request, the video data 901 of the user and the video data 902 of the other party, Information 903 is displayed. However, the additional video service mode corresponds to a state in which, for example, video data shot by a camera during a video call can be edited in real time, and the scope of the present invention is not limited to the above- Should in principle be determined by the claims.

10 is a diagram illustrating screen switching in the case where the additional video service mode corresponds to the first mode according to an embodiment of the present invention. 11 is a flowchart for explaining the process of FIG. Hereinafter, a first mode for simply adding a specific image in the additional video service mode will be described with reference to FIGS. 10 and 11. FIG.

10, the video data 1000 for the caller in the video call mode and the video data 1010 for the other party are displayed at the beginning of the video call mode, and the additional video service mode Mode), the specific additional image 1021 is output in the form of overlaid on the video data 1020 for the caller. However, although the video data 1030 for the other party is maintained as it is in FIG. 10, the additional data may be added according to the operation of the partner network TV.

11, when a network TV according to an embodiment of the present invention enters an additional video service mode (first mode) in which a specific image is simply added, A specific additional image for editing the selected additional image is selected (S1101). The network TV overlays the selected additional image on the original video data shot through the camera (S1102), and controls the overlaid video data to be displayed (S1103).

Then, the network TV is designed to transmit the data set used in the overlay to the counterpart network TV (S1104). However, a more detailed description thereof will be described later with reference to FIG.

12 is a diagram illustrating screen switching in the case where the additional video service mode corresponds to the second mode according to an embodiment of the present invention. 13 is a flowchart for explaining the process of FIG. Hereinafter, a second mode for changing the background of the additional video service mode will be described with reference to FIGS. 12 and 13. FIG.

12, video data 1200 for a caller in a video call mode and video data 1210 for a counterpart are displayed in an initial mode of video communication mode, and an additional video service mode (second mode) The background image of the video data 1220 for the caller is output in the form of an alternative 1221. However, although the video data 1230 of the other party is maintained as it is in FIG. 12, the background image may be changed according to the operation of the partner network TV.

13, when the network TV according to the embodiment of the present invention enters the additional video service mode (the second mode) for changing the background, the network TV according to the embodiment of the present invention edits its own video data A specific background image is selected (S1301). The network TV separately detects the user image and the background image from the original video data shot through the camera (S1302), and replaces the separated background image with the selected background image (S1303).

Then, the network TV is designed to transmit the data set used for the replaced background image to the partner network TV (S1304). However, a more detailed description thereof will be described later with reference to FIG.

FIG. 14 is a diagram illustrating screen switching when the additional video service mode corresponds to the third mode according to an embodiment of the present invention. 15 is a flowchart for explaining the process of FIG. Hereinafter, a third mode for changing an image of a user to an avatar image in the additional video service mode will be described with reference to FIGS. 14 and 15. FIG.

14, the video data 1400 for the caller in the video call mode and the video data 1410 for the other party are displayed at the initial stage of the video communication mode, and an additional video service mode (Third mode), the image of some of the video data 1420 for the caller is output as the form 1421 in which the avatar image is replaced. However, although the video data 1430 for the other party is maintained as it is in FIG. 14, the background image may be changed according to the operation of the partner network TV.

15, when the network TV according to the embodiment of the present invention enters the additional video service mode (third mode) for changing the image of the user into the avatar image, (Step S1501). Further, the network TV analyzes the detected face image and reads the corresponding avatar image (S1502). For example, if the face is a smiley face, it is replaced with an avatar of a smiley face, and if it is an angry face, it is replaced with an avatar face. However, as another embodiment, the user may design the avatar image to be directly selected.

Then, the network TV replaces the user image with the read-out avatar image (S1503). Also, although not shown in FIG. 15, the data set used in the process of replacing with the avatar image is also transmitted to the partner device. However, a more detailed description thereof will be described later with reference to FIG.

FIG. 16 is a diagram schematically illustrating a process in which a data set used in a video processing of an additional image is processed in a first device, according to an embodiment of the present invention. Hereinafter, with reference to FIG. 16, a process of processing the data set used in the video processing of the additional image to the first device according to an embodiment of the present invention will be described. However, the first device means, for example, a network TV in a video call mode in which an additional image is to be added.

As shown in FIG. 16A, the first device has an image DB, and various types of image data are stored in the image DB. However, the first device may be connected to another external device or the like to update new image data. Image A shown in FIG. 16A is an example of an avatar image, image 2 is an example of a background image, and image k is an example of an additional image. Hereinafter, a case in which image k is selected as an example of the additional image will be described as an example, but a person skilled in the art can easily apply the design when another image is selected.

Further, as shown in Fig. 16B, the first device determines, at the request of the user, the position at which the selected specific additional image, image k, is to be displayed. Further, as shown in FIG. 16C, the first device uses the real image synthesis engine (AR engine) to select the additional image (image k) and the video data Vi input through the current camera, Dimensional image to generate a real image.

Further, as shown in (D) of FIG. 16, the first device configures the information used in the synthesis process into one table. The information means, for example, a data set used in the process of video-processing the additional image. 16 (D), the data set includes position information P (xi, yi) of the additional image, 3D conversion matrix information (Matrix Mi) of the additional image, And scale information Ci. The scale information of the additional image means, for example, the size of the converted additional image.

Thus, the first device transmits the aforementioned data set to the other party device of the video call without encoding the combined image of the additional image and the video data shot through the camera, so that even when the performance of the TV is relatively low, There is a unique effect of enabling video services.

FIG. 17 is a diagram schematically illustrating a process in which a data set used in a video processing of an additional image is processed in a second device, according to an embodiment of the present invention. Hereinafter, with reference to FIG. 17, a process of processing the data set used in the video processing of the additional image to the second device will be described as follows. The second device is, for example, a network TV in a video call mode for displaying an additional image added by a partner device.

First, when the second device receives the data set from the first device, the second device reads the image (image k) from its image database. Further, as shown in FIG. 17A, the second device converts the image using the Mi variable, adjusts the size using the Ci variable, and then converts the image into a specific position of the video frame Vi And outputs the converted additional image to P (xi, yi). In addition, since the above-described processes are applied to every frame of the image input to the user's camera, the synthesized image has an advantage of reflecting the movement of the user in real time.

18 is a flowchart illustrating an entire process according to an embodiment of the present invention. Hereinafter, the entire process according to an embodiment of the present invention will be described in more detail with reference to FIG. 18 is a view for explaining the embodiments of the present invention. The scope of the right should not be limited to the contents shown in FIG. 18, and the scope of rights of the present invention should be defined by the claims in principle.

The first network TV according to an embodiment of the present invention receives and outputs broadcast data (S1810). The second network TV according to an embodiment of the present invention also receives and outputs broadcast data (S1820), and transmits a video call request to the first network TV (S1821).

Both the first network TV and the second network TV execute the video communication mode (S1811, 1822), and the camera shooting is started (S1812, S1823). Then, the first network TV and the second network TV output general video call screens (S1813, S1824).

In response to a request from the user or the like, the first network TV executes the additional video service mode (S 1814). However, the step S1814 may be performed based on the screen shown in Fig.

The first network TV selects the type and location of the additional image to be added to the photographed video data (S1815). Further, the first network TV configures a data set used in the video processing of the additional image (S1816), and transmits the data set to the second network TV (S1817).

Meanwhile, the second network TV receives the data set from the first network TV (S1825), and video-processes the additional image using the received data set (S1826). Then, the second network TV synthesizes the video-processed additional image with the counterpart image data and outputs it (S1827).

Therefore, according to an embodiment of the present invention designed as described above, the image itself input by the camera is not distorted, and the transmitting side transmits the information applied to the image synthesis selected by the user. On the receiving side, So that it is not necessary to perform separate decoding or encoding due to the added image.

In addition, according to another embodiment of the present invention, in an Internet video call using a network TV, a user does not transmit his or her actual image as it is, adds a desired image, changes an actual background to another background, It is possible to quickly display the avatar image that has been set or automatically selected on the partner device.

In addition, according to another embodiment of the present invention, in synthesizing images selected by the user, three-dimensional information due to the distance and direction from the user and the camera can be utilized by introducing the real image synthesis technique. Therefore, there is an advantage that the added additional image can be transmitted more realistically to the other party.

In this specification, both the invention and the invention of the method are explained, and the description of the two inventions can be supplementarily applied as necessary.

The method inventions according to the present invention can all be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

300, 400: first network TV
310, 410: second network TV
320: Mobile device
330: PC

Claims (12)

A method for providing a video call service using a network TV,
Receiving broadcast data via a broadcast network interface;
Controlling the received broadcast data to be output;
Capturing a user through a camera when the video call mode is being executed;
Selecting the identification information of the additional image displayed in the first area in which the photographed user is displayed if the additional video service mode is being executed;
Configuring a data set used in the video processing of the additional image;
Transmitting the original video data photographed by the camera, the data set, and the identification information of the additional image to the partner device of the video call mode;
Extracting an additional image mapped to identification information of the received additional image from the additional image list already stored in the memory of the counterpart device; And
Video processing using the original video data and the extracted additional image based on the received data set,
Wherein the identification information of the additional image includes the type of the additional image,
Wherein the video processing comprises:
And detects the position information of the additional image included in the received data set, the 3D conversion matrix information of the additional image, and the scale information of the additional image, and applies the additional image to the original video data. To provide a video call service. The method according to claim 1,
The additional image may include:
And providing a video call service using a network TV stored in a database (DataBase) of the network TV. The method according to claim 1,
Wherein the additional video service mode comprises:
A method for providing a video call service using a network TV corresponding to at least one of a first mode for simply adding a specific image, a second mode for changing background, or a third mode for changing an image of a user to an avatar image . delete The method of claim 3,
If the additional video service mode corresponds to the first mode,
Overlaying the selected additional image on the original video data shot through the camera; And
Controlling the overlaid video data to be displayed
The method comprising the steps of: (a) The method of claim 3,
If the additional video service mode corresponds to the second mode,
Separating and detecting a user image and a background image from original video data shot through the camera;
Replacing the separated background image with the selected additional image;
Synthesizing a user image of the original video data and the substituted additional image; And
Controlling the synthesized image to be displayed
The method comprising the steps of: (a) The method of claim 3,
If the additional video service mode corresponds to the third mode,
Detecting a face image of a user image of original video data photographed through the camera;
Analyzing the detected face image and reading a corresponding avatar image; And
Replacing the user image with the read avatar image
The method comprising the steps of: (a) A network TV providing video call service,
A call connection module for transmitting and receiving a video call connection signal;
A broadcast network interface module for receiving broadcast data;
A memory for storing an additional image list mapped to the identification information of the additional image; And
A control unit for processing the received broadcast data and the video call connection signal,
/ RTI >
The switching to the video communication mode is executed according to the control of the control unit,
And a control unit for controlling the control unit so that original video data photographed by a camera, identification information of an additional image displayed in a first area where the original video data is displayed, Receiving a data set used in the processing,
Processing the additional image based on the received data set in accordance with the control of the control unit,
Wherein the data set is generated on a frame basis and includes position information of the additional image, 3D conversion matrix information of the additional image, and scale information of the additional image,
Wherein,
The additional image is read from the memory, the additional image is converted using the 3D conversion matrix information of the received additional image, the size is converted using the scale information of the additional image, Characterized in that the additional image is video processed using the additional image. 9. The method of claim 8,
A display module for outputting the video-processed additional image and the original video data in the first area in accordance with the control of the controller;
And a network TV. delete delete 9. The method of claim 8,
The network TV includes:
Smart TV, IPTV, HBBTV, and DTV.

Images