KR101476281B1 - System and method for handing over multimedia streams among heterogenous devices for n-screen service - Google Patents

Abstract

The present invention relates to a heterogeneous device handover system and method for a multimedia stream for an N-screen service, wherein a user of a multimedia content of an N-screen service not only moves indoor and outdoor including a house, an office or a vehicle, By using the N-Screen server equipped with N-Screen Server software program to divide the handover area into the kitchen and the room, it is possible to selectively input and output the multimedia stream to various N-screen capable heterogeneous devices, The present invention relates to a heterogeneous device handover system and method for a multimedia stream for an N-screen service that enables seamless use of a multimedia stream through a device of the device.

Description

[0001] SYSTEM AND METHOD FOR HANDLING OVER MULTIMEDIA STREAMS [0002] AMONG HETEROGENOUS DEVICES FOR N-SCREEN SERVICE [

The present invention relates to a heterogeneous device handover system and method for a multimedia stream for an N-screen service, and more particularly, N-screen service enables users to seamlessly use multimedia through a higher quality device by allowing users to input and output multimedia streams continuously with different types of N-screen capable smart devices using N- To a heterogeneous device handover system and method for a multimedia stream for screen services.

The N-screen service is a service that emerges in the changing communication environment through the spread of various smart devices including smart phones, smart TVs, PDAs, and tablet PCs. It can connect to broadcasting and communication networks and use network services Various heterogeneous multimedia terminal devices such as a smart phone, a mobile device, a computer (PC), a smart TV, and a multimedia device of a vehicle share a single screen, and even when a user is moving, A service that makes content available seamlessly. In order to provide such N-screen services, core technologies such as heterogeneous IP control technology, heterogeneous network handover technology, high-speed wireless transmission technology, and adaptive media support technology, which can provide seamless services even in the case of mobile terminals, do.

Among these core technologies, the handover technology refers to a technique in which a user of an N-screen service exchanges a communication channel, a line, a connection port, or a combination thereof while the N-screen service is in a state of being connected to a network, And the service state is maintained.

As described above, the heterogeneous network handover technology, which is a core technology of N-screen service, is also required. In IEEE (International Institute of Electrical and Electronics Engineers), the IEEE 802.21 WG (Working Group) standardization committee is provided. We are proposing a media independent handover technology standard that can be interworked and study it as an international standard. In addition, when receiving a content from a content providing server using a N-screen related technology and playing the content, if a terminal change service is requested, a patent to cause the content after playback stop is seamlessly and continuously provided to the changed terminal, And a content receiving terminal change service providing apparatus and method "(publication number: 10-2011-0026650).

However, these proposed standards and technologies do not provide a device separable handover technique according to the type of multimedia stream in the N-screen service of various multimedia contents. For audio video services such as video telephony, for example, voice may be used in a higher quality audio supporting multimedia device, and video may be used in a multimedia stream, which enables a user to selectively use in a higher quality video supporting multimedia device. Is not provided.

In addition, in the setting of the handover area of the N-screen service, a technique for allowing a user to freely set the handover area around a specific multimedia device in the handover area has not been specifically proposed yet.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to design and implement an N-screen server function with software, and to mount it on a multimedia terminal connectable to the Internet including a smart phone, a smart TV, The present invention provides a heterogeneous device handover system and method for a multimedia stream for an N-screen service that enables the terminals including a device to be used as an N-screen server.

 According to another aspect of the present invention, there is provided a method of recognizing a user handover time, the method comprising the steps of: receiving GPS position information, Wi-Fi specific service set identifier (SSID) information, Wi-Fi signal strength information, Wi- Screen service by allowing a handover of the N-screen service user to be performed more flexibly by grasping the handover point of the N-screen service user. The purpose.

In addition, the present invention provides a handover method for an N-screen service, which enables a user to freely set a handover area around a device capable of N-screen service available for each area in a plurality of handover areas, Over system and method therefor.

In order to maximize the quality of the N-screen service, the present invention also provides a heterogeneous device handover system and method for a multimedia stream for an N-screen service that supports separate handover of voice and video .

The present invention also relates to a method and apparatus for inter-disparate device-to-device communication between a heterogeneous device of a multimedia stream for N-screen service, which enables application of heterogeneous device handover technology to all multimedia contents including music, music video, Internet contents, CCTV image, video call, A handover system and a method thereof are provided.

A heterogeneous inter-device handover system of a multimedia stream for an N-screen service in accordance with an aspect of the present invention includes at least one multimedia device; At least one multimedia device; At least one N-screen server; At least one client, a multimedia server, or a combination thereof; And at least one handover server having a handover control function between a multimedia device including a mobile multimedia device, a client, a multimedia server, and an N-screen server. Also, the heterogeneous device handover system of the multimedia stream for the N-screen service may provide the N-screen service between the multimedia device, the N-screen support device, the client, or the multimedia server, Screen server device, a client, a multimedia server, or a combination thereof, by loading one of the at least one multimedia device with software that performs an N-screen server function and performing a function of an N-screen relay server Screen service, and when one of the at least one multimedia devices fails to establish a link session between the mobile communication network including 3G, 4G, Wibro, and LTE and the Wi-Fi network at the same time, at least one proxy server is added To (SSID) information, Wi-Fi signal strength information, Wi-Fi location-based service information, and a plurality of types of accesses Screen service user by detecting the handover time of the N-screen service user by using the information on the N-screen service in the plurality of handover areas, In order to maximize the quality of the N-screen service, it is necessary to support a separate handover of voice and video, and to provide a handover area of music, music video, Internet contents, CCTV video, video call , All multimedia content including video chat, Handover technique can be applied.

According to another aspect of the present invention, there is provided a method of inter-device handover of a multimedia stream for N-screen service, comprising: registering at least one multimedia device, client, or combination thereof in a handover server; Transmitting registration information of the multimedia device to the N-screen server; The handover server requests an INVITE to the multimedia device immediately according to the location information of the multimedia server, and requests an INVITE to the multimedia server and receives an INVITE from the client; Performing RTP streaming of the multimedia device between the handover server and the multimedia device, and performing client RTP streaming between the handover server and the client or between the handover server and the multimedia server. In addition, the method for inter-device handover of a multimedia stream for the N-screen service includes registering the location of the multimedia device in the multimedia device to the N-screen server, registering the location of the multimedia device in the N- ; Transmitting a UDP port from the N-screen server to the handover server and performing hole punching from the handover server to the N-screen server; Screen support device to open a video RTP session from the N-screen server to the N-screen support device, and the N-screen support device to create a video RTP session to the N-screen server, Screen support device, and performing audio and video RTP streaming between the N-screen server and the handover server, and performing an audio RTP session from the N-screen server to the multimedia device Requesting to open and creating an audio RTP session from the multimedia device to the N-screen server; And performing audio RTP streaming between the N-screen server and the multimedia device, requesting the N-screen server to destroy the RTP session of the multimedia device from the handover server, And the server is required to terminate the RTP session of the multimedia terminal.

DISCLOSURE OF THE INVENTION The present invention provides a heterogeneous device handover system and method for a multimedia stream for an N-screen service, which is applicable to various multimedia contents including music, movies, music videos, Internet contents, CCTV images, video calls, , The multimedia terminal including the mobile device having the N-screen server software installed on the internet can transmit the GPS position information, the SSID of the Wi-Fi system, the Wi-Fi signal strength information, the Bluetooth signal strength information, Screen service can be enjoyed at anytime and anywhere by recognizing the handover point and using the handover service between multimedia devices according to the situation according to the multimedia contents.

FIG. 1 is a block diagram illustrating an overall configuration of a system for explaining heterogeneous device handover of a multimedia stream for an N-screen service according to an exemplary embodiment of the present invention; FIG.
2 is a flowchart illustrating an N-screen server registration procedure of an N-screen support device according to an exemplary embodiment of the present invention;
3 is a flowchart illustrating an audio and video simultaneous handover procedure of multimedia contents according to an embodiment of the present invention.
4 is a flowchart illustrating an audio and video separation type handover procedure of multimedia contents according to an exemplary embodiment of the present invention.
FIG. 5 is a flowchart sequentially illustrating an audio and video separation type handover procedure of a multimedia server-based content according to an embodiment of the present invention; FIG.
FIG. 6 is a flowchart for sequentially illustrating an overall implementation scenario according to an embodiment of the present invention; FIG.
FIG. 7 is a flowchart sequentially illustrating flows between indoor handover regions of a music listening scenario according to an embodiment of the present invention; FIG.
8 is a conceptual diagram for performing handover between a mobile communication network and a Wi-Fi network according to an embodiment of the present invention.

Hereinafter, an embodiment of a heterogeneous device handover system and method of a multimedia stream for an N-screen service according to the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a configuration diagram illustrating an overall configuration of a system for explaining heterogeneous device handover of a multimedia stream for N-screen service according to an exemplary embodiment of the present invention. Referring to FIG.

1, a heterogeneous device handover system for a multimedia stream for an N-screen service according to the present invention includes a handover server 120, at least one mobile multimedia device (MMD) 120, Screen support device 150 to 154 and a client (client) (e.g., a mobile multimedia device such as a smartphone) 100, an N-screen server 110, 140, and at least one multimedia server (M / M (Multimedia Server) 130).

The handover server 120 is a server having a handover control function between the mobile multimedia device 100, the client 140 and the N-screen server 110, and the handover server registers the client and the multimedia device And transmits the registration information of the registered multimedia device to the N-screen server.

The N-screen server 110 is a device for interconnecting the mobile multimedia device 100, the multimedia server 130 and the client 140 and the N-screen support devices 150 to 154, And operates as an N-screen relay server.

The mobile multimedia device 100 refers to a device that is used by a user to determine a handover point by recognizing the location of a user, which is a portable multimedia playback device. For example, a smartphone and a tablet PC can be a mobile multimedia device 100. Such a mobile multimedia device 100 may be operated as an N-screen server by loading N-screen server software, in which case the N-screen server 110 becomes an N-screen relay server.

The N-screen support devices 150 to 154 are devices for outputting the multimedia contents received from the N-screen server 110, and are display devices and multimedia devices capable of connecting to the Internet and supporting the N-screen service.

The client 140 is a counterpart device that provides video telephony, video chat, or multimedia content. The multimedia server 130 may include a multimedia streaming service such as VOD (Video On Demand) or a server for supplying multimedia contents for the service It says. The audio device 153 or the display device 154 in the vehicle also becomes an N-screen support device in an automobile or the like as in the home handover area. At this time, the mobile multimedia device 100 operates as an N-screen relay server to perform handover between heterogeneous devices of a multimedia stream for an N-screen service.

Therefore, the connection relationship between the N-screen server and the mobile multimedia device is such that the mobile multimedia device outputs the content from the client or the multimedia server in the home H / O zone, When it is near, it can receive service of audio or video through high-quality or high-definition audio or smart TV with support of N-screen server. If there is only audio, high-quality audio can be output via audio, which is an N-screen support device, and the video can continue to be output through a mobile multimedia device (e.g., a smart phone). That is, an N-screen service that separates audio and video is possible. Similarly, when there is only an N-screen support device as a high-definition video device such as a smart TV, it is possible to output audio to a mobile multimedia device and output an image to a smart TV.

The above system configuration is merely an example of a schematic configuration of a heterogeneous device handover system of a multimedia stream for an N-screen service. Accordingly, it is possible that a normal technician can provide a handover and an N-screen service by using these models and concepts, so that the use of the modification having such a concept is naturally included in the technical scope of the present invention.

FIG. 2 is a flowchart illustrating an N-screen server registration procedure of an N-screen support device according to an embodiment of the present invention. As shown in FIG. 2, The heterogeneous device handover system requests the registration information specification of N-screen support devices 150 to 152 in the home when the N-screen server 110 starts the N-screen service (S201). At this time, the N-screen support devices 150 to 152 transmit the registration information of the device to the N-screen server 110 to complete the registration of the device (S202), and the handover of the N-screen support devices 150 to 152 Areas and information. This is because the N-screen server manages a detailed list of devices, such as the resolution of the N-screen support device, so that the content can be optimized for that device.

FIG. 3 is a flowchart illustrating an audio and video simultaneous handover procedure of multimedia contents according to an exemplary embodiment of the present invention. As shown in FIG. 3, multimedia contents such as music video and movie, in which audio and video are simultaneously streamed, When the N-screen service user uses the service in connection with the client 140 providing the content, the N-screen service user's mobile multimedia device 100 connecting with the client 140 first registers the registration information in the handover server 120 (S301). Here, the mobile multimedia device 100 generally refers to a mobile multimedia device such as a smart phone, and the mobile multimedia device requests registration immediately upon accessing a mobile communication network such as 3G or 4G and Wibro, WiMax, .

The handover server 120 then transmits the registration information of the mobile multimedia device 100 registered in the handover server 120 to the N-screen server 110 (S302). When the client 140 requests the handover server 120 to connect to the mobile multimedia device 100, the handover server 120 requests the corresponding mobile multimedia device 100 for INVITE (S303) . If a response to the INVITE is normally made, the RTP media stream starts to be transmitted through the handover server 120 and the multimedia stream between the mobile multimedia device 100 and the client 140 is transmitted (S304).

If the mobile multimedia device 100 detects movement of the handover area through the GPS module 100, the SSID of the Wi-Fi system, the Wi-Fi signal strength information, the Bluetooth signal strength information, and various approach detection sensor information during the multimedia streaming transmission (S313) The multimedia device 100 registers a set of information such as its local IP and the port number in use and location information with the N-screen server 110 and the N-screen server 110 transmits the requested information to the requested mobile multimedia device And requests the handover server 120 to update the information (S305). At this time, the N-screen server 110 generates a new UDP port to be used for handover by Wi-Fi detection and transmits the generated UDP port number to the handover server 120 (S306). The handover server 120 attempts hole punching using the received UDP port (S307).

For reference, UDP hole punching is one of the methods of NAT (Network Address Translation). Because of the network configuration of the private IP between Internet connection devices connected to Internet routers in the home, the public IP network is used when accessing the external network NAT traversal techniques are used to allow existing applications (VoIP, Internet game, Web Server, Telnet, FTP, etc.) to be used with a public IP network even in a NAT environment due to limited use of some applications.

The N-screen server 110 then requests a corresponding RTP session to the corresponding device detected on the handover among the N-screen support devices 150 to 152 registered in the N-screen server 110 (S308). The N-screen server 110 receives a new RTP session from the N-screen support devices 150 to 514 (S309).

At this time, the mobile multimedia device 100 simultaneously transmits the RTP session which is being used in the mobile communication network such as 3G, 4G and the like and the new RTP session which is hole-punched in the mobile multimedia device 100 and the N-screen support device 150 to 152 simultaneously So that the multimedia stream can be transmitted. After the mobile multimedia device 100 completes handover, message reception and handover device selection and confirmation, the N-screen server 110 relays the RTP media stream with the handover server 120 (S310 ). In this way, the handover RTP stream is transmitted to the heterogeneous device, which is one of the N-screen support devices, and the N-screen server requests the handover server to destroy the mobile multimedia device RTP session (S311). The N-screen server 110 also requests the mobile multimedia device 100 to destroy the mobile multimedia device RTP session (S312).

FIG. 4 is a flowchart illustrating an audio and video separation type handover procedure of multimedia contents according to an exemplary embodiment of the present invention. As shown in FIG. 4, in the execution of a multimedia service such as a video call, , And the video shows the flow of a separate handover service that makes it possible to use a service via an N-screen device such as a smart TV and a PC with a network camera.

A mobile multimedia device 100, such as a smart phone, which connects a video call with a client 140, which is the other party of the video call, and a video call with the client 140 in order to perform audio and video separation handover, Information in the handover server 120 (S401). Here, the smartphone, which is the mobile multimedia device 100, can directly request registration to the handover server 120 by accessing 3G, 4G, and other mobile communication networks. The handover server 120 then transmits the registration information of the mobile multimedia device 100 registered in the handover server 120 to the N-screen server 100 (S402). When the client 140 requests the handover server 120 to connect to the mobile multimedia device 100, the handover server 120 requests the corresponding mobile multimedia device 100 for INVITE (S403). If a response to the INVITE is normally made, the RTP media stream is transmitted from the handover server 120 to the mobile multimedia device 100 and from the handover server 120 to the client 140, A video call service between the smartphone 100 and the smartphone 100 is performed (S404). At this time, the N-screen server 110 updates the location and content information of the mobile multimedia device 100 from which content the service is being provided.

At this time, if the mobile multimedia device 100 recognizes and detects the handover time through the GPS position information, the SSID of the Wi-Fi system, the Wi-Fi signal strength information, the Bluetooth signal strength information, The multimedia device 100 registers a set of information such as its local IP and the port number in use with the N-screen server 110 and the N-screen server 110 transmits the requested mobile multimedia device information to the handover server 120 (S405). At this time, the N-screen server 110 generates a new UDP (User Datagram Protocol) port to be used for handover by Wi-Fi detection and transmits a UDP port number to be used to the handover server 120 (S406). The handover server 120 attempts hole punching using the received UDP port (S407). In accordance with UDP hole punching, which is a NAT passing solution, the N-screen server 110 requests a corresponding one of the N-screen support devices 150 to 154 registered therein to a new video RTP session (S408). The N-screen server 110 then receives a new video RTP session from the N-screen support devices 150 to 154 (S409). At this time, the handover server 120 can simultaneously transmit the multimedia stream to the RTP session in the network where the mobile multimedia device 100 is using the video call and the new RTP session of the hole-punched N-screen support device 150 to 154 (S409). When the mobile multimedia device 100 completes selection and confirmation of audio and video separation type handover, message reception and handover device notification that the optimal video device in the handover area is detected, the N-screen server 110 performs handover And transmits and receives video RTP streaming between the server 120 and the selected N-screen device for video transmission (S410). The N-screen server 110 then requests the mobile multimedia device 100 for a new audio-only RTP session (S411). The mobile multimedia device 100 transmits / receives a new audio RTP session to the handover server 120 (S412). The mobile multimedia device 100 and the handover server 120 transmit and receive audio RTP stream (S413). The N-screen server 110 requests the handover server 120 to terminate the video RTP session of the mobile multimedia device 100 and requests the mobile multimedia device 100 to terminate the video RTP session (S414).

As described above, the handover procedure through the separated audio and video streams implemented by the technique of the present invention in the N-screen service has been described.

5 is a flowchart for sequentially illustrating an audio and video separation type handover procedure of a multimedia server based content according to an exemplary embodiment of the present invention. For example, the VOD (Video on Demand) through the multimedia server 130 In the execution of the service, the audio represents the flow of the multimedia server 130 based separable handover service through the N-screen support devices 150 to 154, such as a smart phone or an audio device, and the video is a high-definition smart TV .

In more detail, the mobile multimedia device 100 requests registration of the registration information to the handover server 120 (S501). Here, when the mobile multimedia device 100 is a smart phone, the mobile multimedia device 100 can immediately request registration to the handover server 120 by connecting to the 3G or 4G mobile communication network. After completing authentication and registration, the handover server 120 transmits registration information of the mobile multimedia device 100 registered in the handover server to the N-screen server 110 (S502). Next, the mobile multimedia device 100 requests the handover server 120 to connect to the multimedia server 130, and the handover server 120 requests the multimedia server 130 for INVITE (S503). At this time, if a response to the INVITE is normally made, the RTP media stream transmission between the multimedia server 130 and the mobile multimedia device 100 is started through the handover server 120 (S504). If the mobile multimedia device 100 recognizes and detects the handover time through the GPS position information, the SSID of the Wi-Fi system, the Wi-Fi signal intensity information, the Bluetooth signal strength information, and various approach detection sensor information during the multimedia streaming (S515) The multimedia device 100 registers a set of information such as its local IP and the port number in use with the N-screen server 110 and the N-screen server 110 transmits the requested mobile multimedia device information to the handover server 120 in step S505. The N-screen server 110 generates a new UDP port to be used for handover by Wi-Fi detection and transmits a UDP port number to be used to the handover server 120 (S506). The handover server 120 attempts hole punching using the received UDP port (S507). The N-screen server 110 requests a new video RTP session to the corresponding N-screen support device among the N-screen support devices 150 to 154 registered in the N-screen server 110 (S508). The N-screen server 110 receives a new video RTP session from the N-screen support devices 150 to 154 (S509). At this time, the handover server 120 processes the multimedia multimedia device 100 so that the multimedia stream can be simultaneously transmitted through the RTP session in the multimedia streaming network and the new RTP session punched in the network. When the mobile multimedia device 100 completes selection and confirmation of audio and video separation type handover, message reception and handover device notification that the optimal video device in the handover area is detected, the N-screen server 110 transmits a handover Video RTP streaming with the selected video-only N-screen device 120 (S510). At this time, the N-screen server 110 requests the mobile multimedia device 100 for a new audio RTP session (S511), and the mobile multimedia device 100 sends a new video RTP session to the N-screen server 110 (S512 ). And transmits the RTP stream received from the handover server 120 to the N-screen server 110 to the mobile multimedia device 100 (S513). The N-screen server 110 requests the mobile multimedia device 100 and the handover server to terminate the RTP session of the mobile multimedia device RTP stream, respectively (S514).

FIG. 6 is a flowchart for sequentially explaining an overall implementation scenario according to an embodiment of the present invention.

As shown in FIG. 6, the user operates a multimedia content such as a CCTV video or a video phone using a smart phone (S601). At this time, when the user views the video on the smartphone and approaches the home or his / her own vehicle (S602), it is determined that the user is coming to the home or his / her own car by using the GPS position information of the mobile multimedia device 100, SSID, WiFi signal strength information, Bluetooth signal strength information, and various approach detection sensor information (S603). Screen information to the home N-screen server 110 or the N-screen server 110 of the vehicle (S604). In the N-screen service in the vehicle, the mobile multimedia device 100 can be operated as the N-screen server 110. The N-screen server 110 then operates the handover device at home. Or operates the multimedia device in the vehicle (S605). At this time, the smartphone simultaneously reproduces the multimedia stream simultaneously with the RTP session being used in the existing 3G or 4G mobile communication network and the new RTP session of the hole-punched smart TV (S606). At this time, the handover server 120 transmits information on the handover and the result of the TV setting to the user smart phone, and inquires whether to end the streaming of the smart phone or not to perform the handover (S607). For example, in CCTV content, "Do you want to end the video transmission of the current device? Yes / No "For video call content," Do you want to end the video call of the current device? YES ", " Yes / No or video only end " is received, and the user selects the video stopping " Yes "on the smartphone, the handover server 120 stops the video transmission to the smartphone. When selecting to send only the video, which is one of the options of the user, to the TV in the living room after the user enters the home, the smartphone micro-conversation is performed. The multimedia service is continuously used continuously through the server 110. In operation S608,

 FIG. 7 is a flowchart for sequentially illustrating the flow between indoor handover areas of a music listening scenario according to an embodiment of the present invention. As shown in FIG. 7, when a user uses an music streaming service, He / she is coming into the house listening to music (S701). For example, if a user enters an area for handover between heterogeneous devices, the user may be informed that he / she has entered the handover area on the screen of the smartphone. Yes / No "message window appears. The user presses the " Yes " button at the command to allow handover and enters the home (S702). If the user permits the handover as described above, the audio in the living room is turned on and the music that the user hears with the smartphone is reproduced in the living room audio (S703). If the user moves to the kitchen while listening to music while playing the audio in the living room, the user can enjoy the music while listening to the music (S708). If the handover area of the living room and the kitchen is set, , The SSID of the Wi-Fi system, the Wi-Fi signal strength information, the Bluetooth signal strength information, various approach sensor information, and recognizes that the handover area has entered the smartphone screen. "Do you want to play music in kitchen audio? Yes / No "message window appears. When the user selects a button of " Yes " in response to an instruction to permit handover, the handover device selection message is transmitted to the handover server 120 (S709) and music streaming is reproduced in the handover device in the kitchen. Or " No "button is pressed, the music streaming is reproduced directly from the device of the smart phone (S710).

If the user wishes to move the smartphone to the toilet while playing the audio in the living room (S711), if the handover area of the living room and the toilet is set, the mobile multimedia device 100 performs handover For example, if you recognize the area and recognize that you are in the handover area on the smartphone screen, for example, "Do you want to play music from the audio in the bathroom? Yes / No " message window is displayed (S712). However, if the N-screen support devices 150 to 154 are not installed in the toilet, the smartphone notifies that there is no suitable N-screen support device 150 to 154 in the handover area, and the music streaming service is reproduced in the smartphone (S713).

If the user wishes to move to the room while holding the smartphone while playing audio in the living room or kitchen (S704), if the handover area of the living room and the kitchen and the handover area of the room are set, the mobile multimedia device By recognizing the handover area and recognizing that you entered the handover area on the smartphone screen, you can select "Do you want to play music in your room, PC or TV? Yes / No " message window is displayed (S705). When the user presses the "Yes" button after selecting the appropriate device with the command to allow handover, the selected N-screen device in the room is turned on, the music heard on the smartphone is played on the PC, and the music playback on the smartphone is ended S706).

In the present invention, when a mobile multimedia device performs a handover between a client and a multimedia server, if the mobile multimedia device can not simultaneously open a multimedia session through a mobile communication network including 3G, 4G, Wibro, and LTE and a Wi-Fi network , And a proxy server is provided in the middle to support handover between the mobile communication network and the Wi-Fi network.

FIG. 8 is a conceptual diagram for performing a handover between a mobile communication network and a Wi-Fi network according to an embodiment of the present invention.

8 (a) shows a mobile multimedia device (e.g., smart phone) and a client or multimedia server having a mobile communication and a Wi-Fi network interface card (NIC) And in this case, it is conceptually impossible to simultaneously transmit and receive data with two links. In order to solve such a problem, a proxy server may be interposed between the mobile communication network and the Wi-Fi network as shown in FIG. 8 (b). That is, when the mobile multimedia device transmits and receives data through the a-link and then accesses the Wi-Fi area in the situation where the mobile multimedia device and the client or the multimedia server transmit and receive data through the a-link and the c- ), The proxy server keeps the original session for a period of time (for example, 2 minutes) while keeping the session over the a-link and the c-link. Next, the mobile multimedia device forms a b-link and requests handover to the session by the previous a-link to the proxy server. In this case, when the unique ID of the session is matched, the proxy server connects the a-link session to the b-link session and continuously performs handover between heterogeneous networks without changing the previous login information. It can be completed. Here, it is assumed that the c-link is always alive. The concept of this handover is that, in the present invention, even when the mobile multimedia device does not support handover, the handover is supported in the proxy server, Handover can be supported for the model.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. I will understand. Accordingly, the technical scope of the present invention should be defined by the following claims.

100: mobile multimedia device 110: N-screen server
120: Handover server 130: Multimedia server
140: Client
150 ~ 154: N-screen support device

Claims (15)

At least one multimedia device for processing multimedia content,
At least one N-screen support device which is an apparatus for outputting multimedia contents transmitted from an N-screen server,
At least one N-screen server for providing an N-screen service among at least one of the multimedia device, the N-screen support device, the client, the multimedia server,
At least one client that is a device that provides video telephony, video chat, or multimedia content,
A multimedia server for supplying multimedia contents, or a combination thereof; And
Having at least one handover control function between at least one of the multimedia device including a mobile multimedia device, at least one N-screen support device, the N-screen server, the client, the multimedia server, or a combination thereof And a handover server,
Registering at least one or more multimedia devices, clients, or a combination thereof, in a handover server,
Transmitting the registration information of the multimedia device to the N-screen server,
The handover server receives the INVITE from the client and requests the INVITE directly to the multimedia device according to the location information of the multimedia device, or receives the INVITE from the multimedia device according to the location information of the multimedia device, requests the INVITE to the multimedia server again,
Performing real-time transport protocol (RTP) streaming comprising at least one of video, audio, or a combination thereof between a multimedia device and a client or a multimedia server via a handover server, A heterogeneous handover system for multimedia streams for multimedia applications. delete The method according to claim 1,
The multimedia device, the N-screen support device, the client, the multimedia server, or a combination thereof may be implemented by one of the at least one multimedia device by installing software that performs an N-screen server function and performing a function of an N-screen relay server. Screen service between the different types of multimedia streams. The method of claim 3,
Wherein if at least one of the at least one multimedia device fails to simultaneously set up a link session between a mobile communication network including 3G, 4G, Wibro, LTE or a combination thereof and a Wi-Fi network, Handover between the mobile communication network and the Wi-Fi network. The method according to claim 1,
Screen service user's handover time is identified by using GPS position information, Wi-Fi specific service set identifier (SSID) information, Wi-Fi signal strength information, Wi-Fi location aware service information, So that the handover can be performed more flexibly. The method according to claim 1,
Screen service capable of freely configuring a handover area around a device capable of N-screen service existing in each of a plurality of areas in a plurality of handover areas, Delivery system. The method according to claim 1,
Characterized in that a separate handover of voice and video is supported in order to maximize the quality of the N-screen service. The method according to claim 1,
Screen service to multimedia content including multimedia, music, music video, Internet content, CCTV video, video call, video chat, or a combination thereof. Handover between heterogeneous devices. Registering at least one or more multimedia devices, clients or a combination thereof at a handover server;
Transmitting registration information of the multimedia device to the N-screen server;
The handover server receives the INVITE from the client and directly requests the multimedia device to perform the INVITE according to the location information of the multimedia device or receives the INVITE from the multimedia device according to the location information of the multimedia device and requests the INVITE to the multimedia server again; And
Performing real-time transport protocol (RTP) streaming including at least one of video, audio, or a combination thereof between a multimedia device and a client or a multimedia server through a handover server, A heterogeneous handover method of a multimedia stream for screen services. The method of claim 9,
If the multimedia device detects movement of the handover area through the information including the GPS position information, the SSID of the Wi-Fi system, the Wi-Fi signal strength information, the Bluetooth signal strength information, the plurality of proximity sensor information, or a combination thereof during the streaming,
The multimedia device registers information with its N-screen server, including its local IP, port number in use, location information, or a combination thereof, and the N-screen server requests the handover server to update the requested multimedia device information step; And
The N-screen server creates a new UDP port to be used for handover by Wi-Fi detection and transmits the generated UDP port number to the handover server. The handover server uses a UDP port to perform hole punching The method of claim 1, further comprising the steps of: attempting to perform a handover of a multimedia stream for an N-screen service. The method of claim 10,
The N-screen server requests an RTP session including at least one of new video, audio, or a combination thereof to the corresponding device detected on the handover among the registered N-screen support devices, Further comprising receiving a new RTP session including at least one of video, audio, or a combination thereof from the at least one RTP session. The method of claim 11,
Performing at least one of video, audio, or a combination thereof between the N-screen server and the N-screen support device, and performing at least one of video, audio, or a combination thereof between the N-screen server and the handover server And performing RTP streaming including the RTP stream including the RTP stream. The method of claim 12,
Audio, or a combination thereof at the same time in a multimedia device and an N-screen support device with an RTP session being used in a mobile communication network including a 3G, 4G, Wibro, LTE or a combination thereof and a new hole- The RTP stream including at least one of the RTP streams; And
When the multimedia device performs handover, message reception, and handover device selection and confirmation completion, the N-screen server relays the RTP stream including at least one of video, audio, or a combination thereof with the handover server, And transmitting RTP streaming comprising at least one of video, audio, or a combination thereof handed over to a heterogeneous device that is one of the screen support devices. ≪ RTI ID = 0.0 & Device handover method. The method of claim 12,
Screen server requests an RTP session that includes at least one of the new video, audio, or a combination thereof, and the multimedia device sends the N-screen server a new video, audio, or combination thereof Transmitting an RTP session to the multimedia device, the RTP stream including at least one of video, audio, or a combination thereof, received from the handover server to the N-screen server; A heterogeneous handover method of a multimedia stream for a service. The method according to claim 13 or 14,
Screen server to request the N-screen server to destroy an RTP session that includes at least one of video, audio, or a combination thereof of a multimedia device with a handover server, a multimedia device, or a combination thereof. A heterogeneous handover method of a multimedia stream for a service.

Images