KR20130137923A - Image display apparatus, mobile terminal and method for operating the same - Google Patents

Abstract

The present invention relates to an image display device, a mobile terminal, and a method for operating the same. According to an embodiment of the present invention, a method for operating an image display device comprises the steps of: entering into a wireless audio transmission mode; synchronizing with a mobile terminal by a first wireless communication mode; extracting audio data from multimedia data; and transmitting the extracted audio data to the mobile terminal by a second wireless communication mode different from the first wireless communication mode. Therefore, user convenience can be improved. [Reference numerals] (AA) Start;(BB) End;(S725) Enter into a wireless audio transmission mode;(S730) Sync with a mobile terminal by a first wireless communication mode;(S740) Extract an audio signal from multimedia data;(S750) Transmit the extracted audio signal to the mobile terminal by a second wireless communication mode different from the first wireless communication mode;(S760) Play multimedia data

Description

Image display apparatus, mobile terminal and method for operating the same

The present invention relates to an image display apparatus, a mobile terminal, and an operation method thereof, and more particularly, to an image display apparatus, a mobile terminal, and an operation method thereof, which can improve user convenience.

A video display device is a device having a function of displaying an image that a user can view. The user can view the broadcast through the video display device. A video display device displays a broadcast selected by a user among broadcast signals transmitted from a broadcast station on a display. Currently, broadcasting is changing from analog broadcasting to digital broadcasting around the world.

Digital broadcasting refers to broadcasting in which digital video and audio signals are transmitted. Digital broadcasting is more resistant to external noise than analog broadcasting, so it has less data loss, is advantageous for error correction, has a higher resolution, and provides a clearer picture. Also, unlike analog broadcasting, digital broadcasting is capable of bidirectional service.

An object of the present invention is to provide an image display apparatus, a mobile terminal, and an operation method thereof that can improve user convenience.

Another object of the present invention is to provide an image display apparatus, a mobile terminal, and a method of operating the same, which can easily transmit and reproduce audio data wirelessly between an image display apparatus and a mobile terminal.

In accordance with another aspect of the present invention, there is provided a method of operating an image display device, the method comprising: entering an audio wireless transmission mode, synchronizing with a mobile terminal by a first wireless communication method, and multimedia data; And extracting the audio data from the audio data and transmitting the extracted audio data to the mobile terminal by a second wireless communication scheme different from the first wireless communication scheme.

In addition, the operation method of the mobile terminal according to an embodiment of the present invention for achieving the above object, according to the audio wireless transmission mode, by the first wireless communication method, the step of synchronizing with the image display device, the first wireless A second wireless communication method different from the communication method includes receiving audio data from an image display device and reproducing the received audio data.

In addition, the video display device according to an embodiment of the present invention for achieving the above object, the display, a network interface unit for exchanging data with the mobile terminal, according to the audio wireless transmission mode, by the first wireless communication method, And a controller for synchronizing with the mobile terminal, extracting audio data from the multimedia data, and transmitting the extracted audio data to the mobile terminal by a second wireless communication method different from the first wireless communication method. do.

In addition, the mobile terminal according to an embodiment of the present invention for achieving the above object, according to the audio output module for outputting the audio signal to be reproduced, the wireless communication unit for exchanging data with the image display device, according to the audio wireless transmission mode, A control unit configured to control to synchronize with the image display apparatus by the first wireless communication scheme, and to control audio data received from the image display apparatus to be reproduced by the second wireless communication scheme different from the first wireless communication scheme. do.

According to an embodiment of the present invention, according to the audio wireless transmission mode, the video display device is synchronized with the mobile terminal by the first wireless communication method, and by the second wireless communication method different from the first wireless communication method. The extracted audio data is transmitted to the mobile terminal. By different wireless communication methods, audio wireless transmission can be efficiently performed by performing synchronization and transmission of audio data. Therefore, the user's ease of use is increased.

In particular, when using a transmission control protocol (TCP) based communication method as the first wireless communication method, reliability of data communication such as control data and feedback data can be ensured.

On the other hand, when using the real-time transmission protocol (RTP) and user datagram protocol (UDP) -based communication method as the second wireless communication method, low latency streaming of audio data and multi-user support are possible.

On the other hand, by executing the audio wireless transmission mode to the mobile terminal, only the user who wants to listen to the audio data can listen to the desired audio data, and the user who does not want to listen to the audio data can not listen to the audio data. Because of this, it is possible to provide a service according to the user's taste, after all, the user's ease of use can be increased.

1 is a view showing an image display system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a user listening to an audio signal using the image display system of FIG. 1.
3 is an internal block diagram of the image display device of FIG. 1.
4 is an example of an internal block diagram of the controller of FIG. 3.
5 is an internal block diagram of the wireless router of FIG. 1.
6 is an internal block diagram of the mobile terminal of FIG. 1.
7 is a flowchart illustrating a method of operating an image display apparatus according to an exemplary embodiment.
8 is a flowchart illustrating a method of operating a mobile terminal according to an embodiment of the present invention.
9 is a flowchart illustrating an operation of an image display system according to an exemplary embodiment.
10 to 14 are diagrams for describing various examples of an operating method of the image display apparatus of FIG. 7 or an operating method of the mobile terminal of FIG. 8.
15 is a flowchart illustrating a method of operating an image display apparatus according to another exemplary embodiment.
16 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention.
17 to 19 are views referred to for describing various examples of an operating method of the image display apparatus of FIG. 15 or an operating method of the mobile terminal of FIG. 16.
20 is a flowchart illustrating a method of operating an image display apparatus according to another exemplary embodiment.
21 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention.
22A to 23D are diagrams for describing various examples of an operating method of the image display apparatus of FIG. 20 or an operating method of the mobile terminal of FIG. 21.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The suffix "module" and " part "for components used in the following description are given merely for convenience of description, and do not give special significance or role in themselves. Accordingly, the terms "module" and "part" may be used interchangeably.

1 is a view showing an image display system according to an embodiment of the present invention.

Referring to FIG. 1, an image display system 10 according to an exemplary embodiment of the present invention may include an image display apparatus 100, a mobile terminal 600, and a network router 500.

The video display device 100 may receive content. For example, a broadcast signal may be received through an antenna, an external input signal may be received from an external device through an HDMI terminal, or a streaming signal may be received from a predetermined network router 500 through a network.

The video display apparatus 100 can process a received broadcast signal, an external input signal, or a streaming signal, display the video on the display 180, and output the audio.

According to an embodiment of the present invention, the video display device 100 transmits audio data wirelessly to an external device, for example, the mobile terminal 600, in the audio wireless transmission mode. To this end, the image display apparatus 100 may perform direct or indirect communication with the mobile terminal.

In FIG. 1, the image display apparatus 100 and the mobile terminal 600 exchange data through the network router 500.

The image display apparatus 100 may include a TV (TV), a monitor, a computer, a notebook computer, a tablet PC, and the like, which can be broadcastly displayed.

The network router 500 provides a network so that data can be exchanged between the video display device 100 and the mobile terminal 600.

For example, when the network router 500 is a wireless network router, each of the image display apparatus 100 and the mobile terminal using the same IP from the outside while using a plurality of virtual IPs internally. 600). And a wireless channel can be allocated corresponding to each virtual IP.

That is, the network router 500 allocates the first wireless channel to the video display device 100 in response to the first virtual IP, and corresponds to the second virtual IP in the first mobile terminal 600a. The second wireless channel may be allocated and the third mobile channel 600b may be allocated to the second mobile terminal 600b corresponding to the third virtual IP.

The network router 500 may control to exchange data between the video display device 100 and the mobile terminals 600a and 600b through each assigned channel.

Meanwhile, when an electronic device capable of wireless data communication is additionally added to the wireless network provided by the network router 500, the network router 500 may further virtualize the electronic device as described above. IP and wireless channel can be assigned.

The network router 500 may provide environment information of a corresponding wireless channel to the image display apparatus 100 and the mobile terminals 600a and 600b, respectively. For example, security information, channel name information, signal strength information, etc. of the corresponding wireless channel may be provided.

Meanwhile, the wireless network provided by the network router 500 may be a WiFi-based wireless network.

The mobile terminal 600 is a terminal capable of voice communication or data communication, and may be a mobile phone, a smart phone, a notebook computer, a tablet PC, or the like.

In relation to an embodiment of the present invention, the mobile terminal 600 may wirelessly receive audio data from an external device, for example, the image display apparatus 100, in an audio wireless reception mode. To this end, the mobile terminal 600 may perform direct or indirect communication with the image display apparatus 100.

According to an embodiment of the present invention, when the audio wireless transmission / reception mode is performed, first, synchronization is performed between the image display apparatus 100 and the mobile terminals 600a and 600b by a first wireless communication scheme. Perform. That is, control data is transmitted from the mobile terminals 600a and 600b to the image display apparatus 100 and feedback corresponding to the control data from the image display apparatus 100 to the mobile terminals 600a and 600b. Feedback data may be transmitted.

The control data includes at least one of start information or end information of an audio radio transmission mode, radio channel setting request information, radio channel change request information, allocated radio channel information, decodable information, radio channel environment information, or reproduction delay information. can do. The feedback data may include at least one of channel number information, encoding level information, encoding method information, reproduction time information, and current time information of the audio data to be transmitted.

On the other hand, in order to ensure the reliability of data communication, the first wireless communication method is a communication method based on bidirectional communication, and may be a transmission control protocol (TCP) based communication method.

For example, in the first mobile terminal 600a, when the audio wireless reception mode start signal is transmitted as the TCP packet data to the video display device 100 via the wireless router 500, the video display is performed. The apparatus 100 uses, as the TCP packet data, channel number information, encoding level information, encoding method information, etc., of the audio data to be transmitted among multimedia data reproduced on the image display apparatus, for the audio wireless transmission mode. ), It may be transmitted to the first mobile terminal 600a.

Next, when the synchronization is completed between the video display device 100 and the mobile terminals 600a and 600b, the video display device 100 transmits audio data to a second wireless communication method different from the first wireless communication method. By the wireless router 500, the mobile terminal 600a may be transmitted to the mobile terminals 600a and 600b.

On the other hand, for low latency streaming of audio data and multi-user support, the second wireless communication method is a one-way and multicast-based communication method, and a real-time transmission protocol (RTP) and user datagram protocol (UDP) based communication method. Can be.

For example, when synchronization is completed between the first and second mobile terminals 600a and 600b and the video display device 100, the video display device 100 may display a broadcast image that is being played on the video display device 100. The RTP / UDP packet including the audio signal may be transmitted to the first and second mobile terminals 600a and 600b via the wireless router 500.

Meanwhile, the transmitted audio signal may be adaptively changed according to whether it can be decoded in the mobile terminal, a wireless channel environment, a reproduction delay of the mobile terminal, and the like. For example, demultiplexed audio signals of stereo channels, demultiplexed audio signals of mono channels, decoded audio signals of stereo channels, decoded audio signals of mono channels, encoded audio signals of stereo channels, encoding of mono channels Various audio signals, such as a predetermined audio signal, may be transmitted from the image display apparatus 100 to the mobile terminals 600a and 600b.

Meanwhile, when audio data is transmitted, reproduction time information, current time, and the like of the audio data may be transmitted to the mobile terminals 600a and 600b. In this case, the reproduction time point information, the current time information, and the like, are transmitted to the mobile terminals 600a and 600b through the wireless router 500 in the form of an RTP / UDP packet together with the audio data. Alternatively, alternatively, the audio data may be transmitted to the mobile terminals 600a and 600b via the wireless router 500 as TCP packet data.

Meanwhile, unlike FIG. 1, wireless data communication is possible between the video display device 100 and each of the mobile terminals 600a and 600b without the network router 500. At this time, the WiFi Direct-based data communication may be performed between the devices.

FIG. 2 is a diagram illustrating a user listening to an audio signal using the image display system of FIG. 1.

Referring to the drawings, when entering the audio wireless transmission / reception mode under the video display system according to an embodiment of the present invention, the video data 50 among the multimedia data displayed on the video display device 100 continues to be played. The audio data of the multimedia data may be transmitted to the mobile terminal 600. The mobile terminal 600 reproduces the audio data received by the video display device 100.

Accordingly, the user 800 can easily listen to the audio data corresponding to the video data among the multimedia data displayed on the video display device 100 through the earphone 700 through the mobile terminal 600. do.

In particular, in the audio wireless transmission mode, the audio output unit 185 of the image display apparatus 100 may be a mute mode and may not output an audio signal. Accordingly, other users who do not want to listen to the surrounding audio data have an advantage of not having to listen to the audio signal when the multimedia data is reproduced in the image display apparatus 100.

Meanwhile, the drawing illustrates that an object 60 indicating a silent mode is displayed on the display 180 of the image display apparatus 100.

3 is an internal block diagram of the image display device of FIG. 1.

Referring to FIG. 3, the video display device 100 according to an embodiment of the present invention may include a broadcast receiver 105, an external device interface 130, a storage 140, a user input interface 150, It may include a sensor unit (not shown), the controller 170, the display 180, the audio output unit 185, and the remote control device 200.

The broadcast receiver 105 may include a tuner 110, a demodulator 120, and a network interface 135. Of course, the tuner unit 110 and the demodulation unit 120 may be provided, but the network interface unit 135 may not be included. Alternatively, the tuner unit 110 and the tuner unit 110 may be provided, And the demodulation unit 120 are not included.

The tuner unit 110 selects an RF broadcast signal corresponding to a channel selected by the user or all pre-stored channels among RF (Radio Frequency) broadcast signals received through the antenna. Also, the selected RF broadcast signal is converted into an intermediate frequency signal, a baseband image, or a voice signal.

For example, if the selected RF broadcast signal is a digital broadcast signal, it is converted into a digital IF signal (DIF). If the selected RF broadcast signal is an analog broadcast signal, it is converted into an analog baseband image or voice signal (CVBS / SIF). That is, the tuner unit 110 can process a digital broadcast signal or an analog broadcast signal. The analog baseband video or audio signal (CVBS / SIF) output from the tuner unit 110 can be directly input to the controller 170.

The tuner unit 110 may receive an RF broadcast signal of a single carrier according to an Advanced Television System Committee (ATSC) scheme or an RF broadcast signal of a plurality of carriers according to a DVB (Digital Video Broadcasting) scheme.

Meanwhile, the tuner unit 110 sequentially selects RF broadcast signals of all broadcast channels stored through a channel memory function among the RF broadcast signals received through the antenna in the present invention, and sequentially selects RF broadcast signals of the intermediate frequency signal, baseband image, . ≪ / RTI >

On the other hand, the tuner unit 110 may be provided with a plurality of tuners in order to receive broadcast signals of a plurality of channels. Alternatively, a single tuner that simultaneously receives broadcast signals of a plurality of channels is also possible.

The demodulator 120 receives the digital IF signal DIF converted by the tuner 110 and performs a demodulation operation.

The demodulation unit 120 may perform demodulation and channel decoding, and then output a stream signal TS. In this case, the stream signal may be a signal multiplexed with a video signal, an audio signal, or a data signal.

The stream signal output from the demodulation unit 120 may be input to the controller 170. The control unit 170 performs demultiplexing, video / audio signal processing, and the like, and then outputs an image to the display 180 and outputs audio to the audio output unit 185.

The external device interface unit 130 can transmit or receive data with the connected external device (not shown). To this end, the external device interface unit 130 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 130 can be connected to an external device such as a DVD (Digital Versatile Disk), a Blu ray, a game device, a camera, a camcorder, a computer , And may perform an input / output operation with an external device.

The A / V input / output unit can receive video and audio signals from an external device. Meanwhile, the wireless communication unit can perform short-range wireless communication with other electronic devices.

The network interface unit 135 provides an interface for connecting the video display device 100 to a wired / wireless network including the Internet network. To this end, the network interface unit 135 may include a wired communication unit (not shown) or a wireless communication unit (not shown).

For example, the network interface unit 135 may receive content or data from a server (not shown) provided by the Internet or a content provider or a network operator through a network.

Meanwhile, the network interface unit 135 may perform data communication with the network router 500 and exchange data within the network under the network router 500.

In accordance with an embodiment of the present invention, the network interface unit 135 may transmit control data in the form of TCP packets from the mobile terminal 600 to the network router 500 for synchronization with the mobile terminal 600 under an audio wireless transmission mode. Feedback data in the form of a TCP packet and corresponding thereto can be transmitted to the mobile terminal 600 via the network router 500.

In addition, the network interface unit 135 transmits audio data in the form of RTP / UDP packets corresponding to the multimedia data reproduced on the video display device to the mobile terminal 600 via the network router 500 after the synchronization is completed. Can transmit

The storage unit 140 may store a program for each signal processing and control in the control unit 170 or may store the processed video, audio, or data signals.

In addition, the storage unit 140 may perform a function for temporarily storing video, audio, or data signals input to the external device interface unit 130. [ In addition, the storage unit 140 may store information on a predetermined broadcast channel through a channel memory function such as a channel map.

Although the storage unit 140 of FIG. 3 is separately provided from the controller 170, the scope of the present invention is not limited thereto. The storage unit 140 may be included in the controller 170.

The user input interface unit 150 transmits a signal input by the user to the control unit 170 or a signal from the control unit 170 to the user.

(Not shown), such as a power key, a channel key, a volume key, and a set value, from the remote control apparatus 200, (Not shown) that senses a user's gesture to the control unit 170 or transmits a signal from the control unit 170 to the control unit 170 It is possible to transmit it to the sensor unit (not shown).

The control unit 170 demultiplexes the input stream or processes the demultiplexed signals through the tuner unit 110 or the demodulation unit 120 or the external device interface unit 130 so as to output the video or audio output Signals can be generated and output.

The video signal processed by the controller 170 may be input to the display 180 and displayed as an image corresponding to the video signal. Also, the image signal processed by the controller 170 may be input to the external output device through the external device interface unit 130.

The audio signal processed by the control unit 170 may be output to the audio output unit 185 as an audio signal. The audio signal processed by the controller 170 may be input to the external output device through the external device interface unit 130. [

Although not shown in FIG. 3, the controller 170 may include a demultiplexer, an image processor, and the like. This will be described later with reference to FIG. 4.

In addition, the control unit 170 can control the overall operation in the video display device 100. [ For example, the control unit 170 may control the tuner unit 110 to control the tuning of the RF broadcast corresponding to the channel selected by the user or the previously stored channel.

In addition, the controller 170 may control the image display apparatus 100 by a user command or an internal program input through the user input interface unit 150.

Meanwhile, the control unit 170 may control the display 180 to display an image. In this case, the image displayed on the display 180 may be a still image or a video, and may be a 2D image or a 3D image.

On the other hand, the control unit 170 can recognize the position of the user based on the image photographed from the photographing unit (not shown). For example, the distance (z-axis coordinate) between the user and the image display apparatus 100 can be grasped. In addition, the x-axis coordinate and the y-axis coordinate in the display 180 corresponding to the user position can be grasped.

Although not shown in the drawing, a channel browsing processing unit for generating a channel signal or a thumbnail image corresponding to an external input signal may be further provided. The channel browsing processing unit receives the stream signal TS output from the demodulation unit 120 or the stream signal output from the external device interface unit 130 and extracts an image from an input stream signal to generate a thumbnail image . The generated thumbnail image may be stream decoded together with the decoded image and input to the controller 170. The control unit 170 may display a thumbnail list having a plurality of thumbnail images on the display 180 using the input thumbnail image.

At this time, the thumbnail list may be displayed in a simple view mode displayed on a partial area in a state where a predetermined image is displayed on the display 180, or in a full viewing mode displayed in most areas of the display 180. The thumbnail images in the thumbnail list can be sequentially updated.

The display 180 converts a video signal, a data signal, an OSD signal, a control signal processed by the control unit 170, a video signal, a data signal, a control signal, and the like received from the external device interface unit 130, .

The display 180 may be a PDP, an LCD, an OLED, a flexible display, or the like, and may also be capable of a 3D display.

In order to view the three-dimensional image, the display 180 may be divided into an additional display method and a single display method.

The single display method can implement a 3D image only on the display 180 without a separate additional display, for example, glass, and examples thereof include a lenticular method, a parallax barrier, and the like Various methods can be applied.

On the other hand, the additional display method can implement a 3D image using an additional display as a viewing device (not shown) in addition to the display 180. For example, various methods such as a head mount display (HMD) type, Can be applied.

On the other hand, the spectacle type may be divided into a passive system such as a polarized glasses type and an active system such as a shutter glass type. In addition, the head mounted display can be divided into a passive type and an active type.

On the other hand, the viewing device (not shown) may be a 3D glass for stereoscopic viewing. The glass for 3D (not shown) may include a passive polarizing glass or an active shutter glass, and may be a concept including the head mount type described above.

The viewing apparatus (not shown) may be a glass capable of viewing different images for each user. For example, the first viewing device may be a glass for viewing only the first image, and the second viewing device may be a glass for viewing only the second image.

Meanwhile, the display 180 may be configured as a touch screen and used as an input device in addition to the output device.

The audio output unit 185 receives the signal processed by the control unit 170 and outputs it as a voice.

A photographing unit (not shown) photographs the user. The photographing unit (not shown) may be implemented by a single camera, but the present invention is not limited thereto, and may be implemented by a plurality of cameras. On the other hand, the photographing unit (not shown) may be embedded in the image display device 100 on the upper side of the display 180 or may be disposed separately. The image information photographed by the photographing unit (not shown) may be input to the control unit 170.

The control unit 170 can detect the gesture of the user based on each of the images photographed from the photographing unit (not shown) or the signals sensed from the sensor unit (not shown) or a combination thereof.

The remote control apparatus 200 transmits the user input to the user input interface unit 150. [ To this end, the remote control apparatus 200 can use Bluetooth, RF (radio frequency) communication, infrared (IR) communication, UWB (Ultra Wideband), ZigBee, or the like. Also, the remote control apparatus 200 can receive the video, audio, or data signal output from the user input interface unit 150 and display it or output it by the remote control apparatus 200.

Meanwhile, the video display device 100 may be a digital broadcast receiver capable of receiving a fixed or mobile digital broadcast.

Meanwhile, a block diagram of the image display apparatus 100 shown in FIG. 3 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the image display apparatus 100 actually implemented. That is, two or more constituent elements may be combined into one constituent element, or one constituent element may be constituted by two or more constituent elements, if necessary. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and apparatuses do not limit the scope of the present invention.

On the other hand, the image display apparatus 100 does not include the tuner 110 and the demodulator 120 shown in FIG. 3, unlike the network display unit 135 or the external device interface unit (not shown in FIG. 3). Through 135, broadcast content may be received and reproduced.

4 is an example of an internal block diagram of the controller of FIG. 3.

Referring to the drawings, the control unit 170 according to an embodiment of the present invention, the demultiplexer 310, the audio processor 315, the image processor 320, the processor 330, the OSD generator 340 May include a mixer 345. In addition, it may further include a data processor (not shown).

The demultiplexer 310 demultiplexes the input stream. For example, when an MPEG-2 TS is input, it can be demultiplexed into video, audio, and data signals, respectively. The stream signal input to the demultiplexer 310 may be a stream signal output from the tuner 110 or the demodulator 120 or the external device interface 130.

The audio processor 315 may perform voice processing of the demultiplexed voice signal. For this purpose, the audio processor 315 may include various decoders. Meanwhile, in order to efficiently transmit audio data to the mobile terminal 600, the audio processor 315 may further include an encoder (not shown).

In the audio wireless transmission mode of the image display apparatus 100 according to an exemplary embodiment of the present invention, audio data transmitted to the mobile terminal 600 is 1) demultiplexed by the demultiplexer 310 and the audio processor 315. 2) de-multiplexed by the demultiplexer 310, decoded by the audio processor 315, and then encoded audio data for transmission to the mobile terminal 600; The audio data may be demultiplexed by the demultiplexer 310 and decoded by the audio processor 315.

In the case of the audio data of 1), the audio data acquisition time to be transmitted to the mobile terminal 600 is short, but since it has not been decoded yet, if the corresponding mobile terminal 600 does not have a corresponding decoder, it is not decoded. There is a possibility.

2) In the case of audio data, since the decoded, that is, uncompressed audio data is encoded, the transmission efficiency in wireless transmission to the mobile terminal 600 may be improved, and in consideration of a decoder provided in the mobile terminal at the time of encoding, Although encoding may be performed, since the demultiplexing, decoding, and encoding should be performed in the image display apparatus 100, it may be difficult to synchronize reproduction between the image display apparatus and the mobile terminal.

3) In the case of audio data, since the demultiplexed audio data is decoded, uncompressed audio data may be received by the mobile terminal 600. Therefore, the mobile terminal 600 can immediately play back without a separate decoding step. However, during the wireless transmission between the image display device 100 and the mobile terminal 600, the amount of wireless data to be transmitted increases, and thus a wide frequency bandwidth is required on the wireless channel.

According to the embodiment of the present invention, audio data according to any one of 1) to 3) is adaptively transmitted according to the reproduction delay information of the wireless channel environment or the mobile terminal. This will be described later with reference to FIG. 20 or below.

Also, the audio processor 315 may process base, treble, volume control, and the like.

The image processing unit 320 may perform image processing of the demultiplexed image signal. For this, the image processing unit 320 may include a video decoder 325 and a scaler 335.

The video decoder 325 decodes the demultiplexed video signal and the scaler 335 performs scaling so that the resolution of the decoded video signal can be output from the display 180.

The processor 330 may control the overall operation in the image display apparatus 100 or in the control unit 170. [ For example, the processor 330 may control the tuner 110 to control the tuner 110 to select an RF broadcast corresponding to a channel selected by a user or a pre-stored channel.

In addition, the processor 330 may control the image display apparatus 100 by a user command or an internal program input through the user input interface unit 150. [

In addition, the processor 330 may perform data transfer control with the network interface unit 135 or the external device interface unit 130.

The processor 330 may control operations of the demultiplexing unit 310, the image processing unit 320, the OSD generating unit 340, and the like in the controller 170.

The OSD generator 340 generates an OSD signal according to a user input or itself. For example, based on a user input signal, a signal for displaying various information in a graphic or text form on the screen of the display 180 can be generated. The generated OSD signal may include various data such as a user interface screen of the video display device 100, various menu screens, a widget, and an icon. In addition, the generated OSD signal may include a 2D object or a 3D object.

The OSD generating unit 340 can generate a pointer that can be displayed on the display based on the pointing signal input from the remote control device 200. [ In particular, such a pointer may be generated by a pointing signal processor, and the OSD generator 340 may include a pointing signal processor (not shown). Of course, the pointing signal processor (not shown) may be provided separately without being provided in the OSD generator 340.

The mixer 345 may mix and output the OSD signal generated by the OSD generator 340 and the decoded image signal processed by the image processor 320.

The data processing unit (not shown) in the control unit 170 can perform data processing of the demultiplexed data signal. For example, if the demultiplexed data signal is a coded data signal, it can be decoded. The encoded data signal may be EPG (Electronic Program Guide) information including broadcast information such as a start time and an end time of a broadcast program broadcasted on each channel.

Meanwhile, the block diagram of the controller 170 shown in FIG. 4 is a block diagram for an embodiment of the present invention. Each component of the block diagram can be integrated, added, or omitted according to the specifications of the control unit 170 actually implemented.

5 is an internal block diagram of the wireless router of FIG. 1.

Meanwhile, referring to FIG. 5, the network router 500 may include a network interface unit 530, a storage unit 540, and a processor 520.

The network interface unit 530 may exchange data with the video display device 100 or the mobile terminal 600.

In detail, the network interface unit 530 receives the TCP packet based control data from the mobile terminal 600 in order to synchronize with the mobile terminal 600 in the audio wireless transmission mode, and displays the image display device ( 100, the TCP packet-based feedback data from the image display apparatus 100 may be received and transmitted to the mobile terminal 600.

Next, when the synchronization is completed, the network interface unit 530 receives the RTP / UDP packet-based audio data from the video display device 100 for wireless transmission of the audio data to the mobile terminal 600, and then performs the synchronization. May transmit to the mobile terminal 600.

The storage unit 540 may store an external IP address, an internally assigned virtual IP address, and a wireless channel name, frequency range, security information, and the like corresponding to each virtual IP for providing a wireless network. In addition, the device name of the video display device 100, the device name of the mobile terminal 600, and the like, which are network-connected, may be stored through the network router 500.

The processor 520 controls the overall operation of the network router 500.

As described above, data exchange between the image display apparatus 100 and the mobile terminal 600 may be controlled for the audio wireless transmission / reception mode, and when the electronic device is additionally added to the network, the corresponding electronic device In addition, a virtual IP and a wireless channel may be allocated. In addition, the image display apparatus 100 and the mobile terminal 600 may provide environment information of a corresponding wireless channel. For example, security information, channel name information, signal strength information, etc. of the corresponding wireless channel may be provided.

6 is an internal block diagram of the mobile terminal of FIG. 1.

6, the mobile terminal 600 includes a wireless communication unit 610, an A / V input unit 620, a user input unit 630, a sensing unit 640, an output unit 650, A memory 660, an interface unit 670, a control unit 680, and a power supply unit 690.

The wireless communication unit 610 according to an embodiment of the present invention transmits control data based on a TCP packet via the wireless router 500 to synchronize with the image display apparatus 100 in the audio wireless reception mode. And transmit the TCP packet-based feedback data from the image display apparatus 100 via the wireless router 500.

Meanwhile, when synchronization is completed, the wireless communication unit 610 may receive RTP / UDP packet based audio data from the video display device 100 via the wireless router 500 for the audio wireless reception mode. have.

The wireless communication unit 610 may include a broadcast receiving module 611, a mobile communication module 613, a wireless internet module 615, an NFC module 617, a GPS module 619, and the like.

The broadcast receiving module 611 may receive at least one of a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel. At this time, the broadcast channel may include a satellite channel, a terrestrial channel, and the like.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 611 may be stored in the memory 660.

The mobile communication module 613 transmits and receives a radio signal to at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to a voice call signal, a video call signal, or a text / multimedia message transmission / reception.

The wireless internet module 615 refers to a module for wireless internet access, and the wireless internet module 615 may be embedded or external to the mobile terminal 600. For example, the wireless internet module 615 may perform WiFi based wireless communication or WiFi Direct based wireless communication.

The NFC module 617 may perform near field communication. The NFC module 617 can receive data from the electronic device or transmit data to the electronic device when the electronic device is accessed within a predetermined distance, i.e., tagging, with the NFC tag or the NFC module.

In addition, Bluetooth, RFID (radio frequency identification), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

The global position system (GPS) module 619 may receive location information from a plurality of GPS satellites.

The A / V (Audio / Video) input unit 620 is for inputting an audio signal or a video signal, and may include a camera 621 and a microphone 623.

The user input unit 630 generates key input data that the user inputs to control the operation of the terminal. The user input unit 630 may include a key pad, a dome switch, and a touch pad (static / static). In particular, when the touch pad has a mutual layer structure with the display unit 651 described later, this may be referred to as a touch screen.

The sensing unit 640 senses the current state of the mobile terminal 600 such as the open / close state of the mobile terminal 600, the position of the mobile terminal 600, A sensing signal can be generated.

The sensing unit 640 may include a sensing sensor 641, a pressure sensor 643, a motion sensor 645, and the like. The motion sensor 645 can detect the movement or the position of the mobile terminal 600 using an acceleration sensor, a gyro sensor, a gravity sensor, or the like. In particular, the gyro sensor is a sensor for measuring the angular velocity, and it can sense the direction (angle) of rotation about the reference direction.

The output unit 650 may include a display unit 651, an audio output module 653, an alarm unit 655, and a haptic module 657.

The display unit 651 displays and outputs information processed by the mobile terminal 600.

Meanwhile, as described above, when the display unit 651 and the touch pad form a mutual layer structure and constitute a touch screen, the display unit 651 is an input device capable of inputting information by a user's touch in addition to the output device. May also be used.

The sound output module 653 outputs audio data received from the wireless communication unit 610 or stored in the memory 660. The sound output module 653 may include a speaker, a buzzer, and the like.

The alarm unit 655 outputs a signal for notifying occurrence of an event of the mobile terminal 600. For example, it is possible to output a signal in a vibration mode. .

The haptic module 657 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 657 is a vibration effect.

The memory 660 may store a program for processing and controlling the control unit 680, and may provide a function for temporarily storing input or output data (for example, a phone book, a message, a still image, a video, etc.). It can also be done.

The interface unit 670 serves as an interface with all external devices connected to the mobile terminal 600. The interface unit 670 may receive data from the external device or receive power and transmit the data to each component in the mobile terminal 600, and may transmit the data in the mobile terminal 600 to the external device. .

The controller 680 typically controls the operations of each unit to control the overall operation of the mobile terminal 600. For example, perform related controls and processing for voice calls, data communications, video calls, and the like. In addition, the controller 680 may include a multimedia playback module 681 for multimedia playback. The multimedia playback module 681 may be configured in hardware in the controller 680 or may be configured in software separately from the controller 680. On the other hand, the operation of the control unit 680 for the multimedia playback, etc. will be described later with reference to FIG.

The power supply unit 690 receives an external power source and an internal power source under the control of the controller 680 to supply power for operation of each component.

Meanwhile, the block diagram of the mobile terminal 600 shown in FIG. 6 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the mobile terminal 600 that is actually implemented. That is, two or more constituent elements may be combined into one constituent element, or one constituent element may be constituted by two or more constituent elements, if necessary. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and apparatuses do not limit the scope of the present invention.

7 is a flowchart illustrating a method of operating an image display apparatus according to an exemplary embodiment.

Referring to the drawing, first, when the image display apparatus 100 receives the audio wireless reception mode entry signal from the mobile terminal, the image display apparatus 100 enters the audio wireless transmission mode (S725). In operation S730, synchronization with the mobile terminal 600 is performed by the first wireless communication method.

The network interface unit 135 of the image display apparatus 100 may receive a TCP packet including an audio wireless transmission mode start signal through the wireless router 500. In an embodiment of the present invention, in order to secure the reliability of data communication, a transmission control protocol (TCP) based communication method may be adopted as the first wireless communication method.

The controller 170 of the image display apparatus 100 may control to enter the audio wireless transmission mode when receiving the audio wireless transmission mode start signal in the TCP packet. Then, it may be controlled to perform synchronization with the mobile terminal 600.

In detail, the control unit 170 of the image display apparatus 100 corresponds to the received audio wireless transmission mode start signal, and as feedback data, channel number information, encoding level information, encoding method information of audio data to be transmitted. Or the like in the form of TCP packets.

Here, the channel number information indicates information on whether a channel of audio data to be transmitted is a mono channel, a stereo channel, a 5.1 channel, or the like.

Meanwhile, the encoding level information may include 1) audio data to be transmitted is audio data demultiplexed by the demultiplexer 310 and not decoded by the audio processor 315, or 2) demultiplexed by the demultiplexer 310, Audio data decoded by the audio processor 315 and then decoded for transmission to the mobile terminal 600, or 3) demultiplexed by the demultiplexer 310, and decoded by the audio processor 315. Information may be indicated.

On the other hand, the encoding method information is effective when the encoding level information indicates the encoded audio data, and may indicate information about whether the encoding method is AC3 encoding or AAC encoding.

For example, when the image display apparatus 100 receives the audio wireless transmission mode start signal in a state in which a broadcast image of a predetermined channel is reproduced as multimedia data, the control unit of the image display apparatus 100 ( If the encoding method of the audio data to be transmitted is the AAC encoding method, the AAC may be controlled to transmit the AAC as the encoding method information, and when the number of channels is two, the stereo channel may be controlled to be transmitted as the channel number information.

Next, the network interface unit 135 of the image display apparatus 100 may receive a TCP packet including synchronization completion information indicating that synchronization with the mobile terminal 600 is completed. Accordingly, the synchronization is completed between the image display apparatus 100 and the mobile terminal 600. That is, under the audio wireless transmission mode, preparation for audio data transmission is completed.

Next, the image display apparatus 100 extracts an audio signal from the multimedia data (S740). The extracted audio signal is transmitted to the mobile terminal by the second communication method (S750).

After the synchronization is completed, the controller 170 of the video display device 100 extracts the audio signal to be transmitted. For example, when the multimedia data is a broadcast signal, the controller 170 of the video display device 100, in particular, the demultiplexer 310, may demultiplex the broadcast stream. Thus, it is divided into an audio signal, a video signal, and a data signal.

In the audio wireless transmission mode according to an embodiment of the present invention, audio data transmitted to the mobile terminal 600 is 1) demultiplexed by the demultiplexer 310, audio data not decoded by the audio processor 315 Or 2) demultiplexed by the demultiplexer 310, decoded by the audio processor 315, and then encoded audio data for transmission to the mobile terminal 600, or 3) by the demultiplexer 310. The audio data may be demultiplexed and decoded by the audio processor 315.

The controller 170 of the image display apparatus 100 selects audio data of any one of 1) to 3) and converts the selected audio data into an RTP / UDP packet in one-way multicasting. Can be controlled to be sent.

In an embodiment of the present invention, for low latency streaming of audio data and multi-user support, a real time transmission protocol (RTP) and a user datagram protocol (UDP) based communication scheme may be adopted as the second wireless communication scheme. .

Next, the video display device 100 reproduces the multimedia data (S760). In particular, the controller 170 of the image display apparatus 100 may decode the video signal among the demultiplexed audio signal, the video signal, and the data signal, and output the decoded video signal to be displayed on the display 180. Accordingly, the display 180 can display a broadcast image corresponding to the received broadcast signal. In this case, the audio output unit 185 of the image display apparatus 100 may not output the broadcast audio signal corresponding to the received broadcast signal. That is, the audio signal may not be output as a mute mode. Accordingly, other users who do not want to listen to the surrounding audio data have an advantage of not having to listen to the audio signal when the multimedia data is reproduced in the image display apparatus 100.

8 is a flowchart illustrating a method of operating a mobile terminal according to an embodiment of the present invention.

Referring to the drawings, first, the mobile terminal 600 enters the remote control mode for the video display device (S805). The remote control screen is displayed (S810). In the state where the remote control screen is displayed, it is determined whether there is an audio wireless reception mode start input (S815), and if so, enters the audio wireless reception mode (S816). The audio wireless reception screen is displayed (S818) and the first wireless communication method is used to synchronize with the image display device (S820).

When the remote control menu item or the remote control application item of the mobile terminal 600 is executed, the controller 680 of the mobile terminal 600 may control to enter the remote control mode for the image display apparatus 100. have. The controller 680 of the mobile terminal 600 may control the remote control screen 1310 to be displayed on the display unit 651 as shown in FIG. 13A.

The remote control screen 1310 of FIG. 13A illustrates an example of including a mute item, a channel control item, a volume control item, an audio wireless reception mode item (WiFi Sound) 1315, and the like. When the user selects the audio wireless reception mode item (WiFi Sound) 1315, the control unit 680 of the mobile terminal 600 determines that the audio wireless reception mode is selected, and determines the audio wireless reception mode. It is possible to control to display the audio wireless reception screen.

FIG. 13B illustrates an audio wireless reception screen, in which a WiFi sound screen 1330 indicating an audio wireless reception mode may be displayed on the display unit 651.

The WiFi Sound screen 1330 of FIG. 13B illustrates an object (shown as WiFi Sound) 1332 indicating an audio wireless reception mode, a channel adjustment item 1334, and a volume adjustment item 1336. .

Meanwhile, when the audio wireless reception mode item (WiFi Sound) 1315 in the remote control screen 1310 is selected, the controller 680 of the mobile terminal 600, as shown in FIG. 13 (b), displays the WiFi Sound screen ( In addition to controlling the display 1330, a TCP packet including the audio wireless reception mode start signal may be transmitted to the image display apparatus 100. That is, it may be controlled to perform synchronization with the image display apparatus 100.

Accordingly, the image display apparatus 100 enters the audio wireless transmission mode and synchronizes with the mobile terminal. That is, the video display device 100 may transmit, as feedback data, channel number information, encoding level information, encoding method information, etc. of audio data to be transmitted in the form of TCP packets. The controller 680 of the mobile terminal 600 may receive such channel number information, encoding level information, encoding scheme information, and the like through the wireless communication unit 610.

When the synchronization with the video display device 100 is completed, the controller 680 of the mobile terminal 600 may control to transmit a TCP packet including synchronization completion information indicating that the synchronization with the video display device 100 is completed. have.

Next, the mobile terminal 600 receives the audio signal from the video display device by the second wireless communication method (S850). After the synchronization is completed, the video display device 100 may transmit audio data in the form of an RTP / UDP packet, and the controller 680 of the mobile terminal 600 receives the audio data. In this case, the audio data may be 1) undecoded audio data, 2) encoded audio data, or 3) demultiplexed and decoded audio data.

Next, the mobile terminal 600 reproduces the received audio data (S865). In particular, the control unit 680 of the mobile terminal 600 reproduces the audio data in response to the audio data of any one of 1) to 3) described above, and outputs the reproduced audio data to the sound output module 653. Output through

Accordingly, the user of the mobile terminal 600 can listen to the audio signal output from the mobile terminal 600 while watching the image of the multimedia data reproduced by the image display apparatus 100.

On the other hand, unlike Figure 8, even if the remote control screen is not displayed on the mobile terminal 600, it is also possible to immediately enter the audio wireless reception mode. For example, in the mobile terminal 600, when a hot key indicating an audio wireless reception mode or an audio wireless reception mode app is executed, the mobile terminal 600 immediately enters the audio wireless reception mode. It is possible. By the hot key operation or the execution of the app, the WiFi Sound screen 1330, as shown in FIG. 13B, may be immediately displayed on the mobile terminal 600 without displaying the remote control screen.

9 is a flowchart illustrating an operation of an image display system according to an exemplary embodiment.

Referring to the drawings, an operation flowchart of the video display system of FIG. 9 corresponds to FIGS. 7 and 8.

Specifically, steps 905 to 918 (S905, S910, S915, S916, and S918) correspond to steps 805 to 818 (S805, S810, S815, S816, and S818 of FIG. 8), and 922. Steps 928 to S928 correspond to steps 922, S924, S926, and S928, step 830 of FIG. 8, and step 730 of FIG. 7.

In addition, operation 940 (S940) corresponds to operation 740 (S740) of FIG. 7, and operations 942 to 944 (S942 and S944) may include operations 750 (S750) and FIG. 8 of FIG. 7. It corresponds to step 850 (S850).

On the other hand, step 960 (S960) corresponds to step 760 (S760) of FIG. 7, and step 965 (S965) corresponds to step 865 (S865) of FIG.

FIG. 10 is a diagram referred to for describing TCP packet data exchange between an image display device and a mobile terminal.

The controller 680 of the mobile terminal 600 may control to transmit control data through the TCP transmitter 1125. The control data includes at least one of start information or end information of an audio radio reception mode, radio channel setting request information, radio channel change request information, allocated radio channel information, decodable information, radio channel environment information, or reproduction delay information. can do.

Among the control data, during the synchronization process, the start information, the radio channel setting request information, the allocated radio channel information, the decodable information, and the like of the audio wireless reception mode may be transmitted to the image display apparatus 100.

On the other hand, among the control data, during the audio data receiving process, the termination information of the audio wireless reception mode, the radio channel change request information, the radio channel environment information, or the reproduction delay information may be transmitted to the image display apparatus 100.

The controller 170 of the image display apparatus 100 may receive control data through the TCP receiver 1025. The controller 170 may control to transmit the feedback data through the TCP transmitter 1020. The feedback data may include at least one of channel number information, encoding level information, encoding method information, reproduction time information, and current time information of the audio data to be transmitted.

Among the control data, channel number information, encoding level information, encoding scheme information, and the like of audio data to be transmitted may be transmitted to the mobile terminal 600 during the synchronization process.

On the other hand, of the control data, during the audio data transmission process, the reproduction time information, etc. of the audio data to be transmitted may be transmitted to the mobile terminal 600.

Meanwhile, in FIG. 10, the synchronization unit 1010 of the image display apparatus 100 is provided separately from the control unit 170, and the current time information is separated from the feedback data through the TCP transmitter 1020. 600 to be transmitted. In addition, the synchronization unit 1110 of the mobile terminal 600 is provided separately from the control unit 680, and receives through the TCP receiver 1120, along with the feedback data.

Of course, alternatively, current time information may be included in the feedback data and transmitted.

Meanwhile, the TCP receiver 1025 and the TCP transmitter 1020 of the image display apparatus 100 may be provided in the network interface unit 135 of FIG. 3. In addition, the TCP receiver 1120 and the TCP transmitter 1125 of the mobile terminal 600 may be provided in the wireless communication unit 610 of FIG. 6.

FIG. 11 is a diagram referred to describe RTP / UDP packet data exchange between an image display device and a mobile terminal, and FIG. 12 is a diagram briefly illustrating a structure of RTP / UDP packet data.

The video display device 100 may receive multimedia data from the tuner unit 110, the external device interface unit 130, or the network interface unit 135.

In the case of stream data, the demultiplexer 310 demultiplexes the stream data, the decoder 1030 decodes the audio data, and the encoder 1040 encodes the decoded audio data. In this case, the demultiplexer 310, the decoder 1030, and the encoder 1040 may be provided in the controller 170.

As described above, the audio data that can be transmitted to the mobile terminal 600 is audio data that is demultiplexed by the demultiplexer 310 of ① and not decoded by the decoder 1030, or demultiplexer 310 of ②. The audio data may be demultiplexed at, decoded by the decoder 1030, and then encoded by the encoder 1040, or may be demultiplexed by the demultiplexer 310 of ③ and decoded by the decoder 1030. have.

As shown in FIG. 12, the RTP packet 1220 is added by the RTP transmitter 1045 of the image display apparatus 100, and the UDP packet 1230 is added by the UDP transmitter 1050. And as an IP packet 1240, it may be transmitted to the mobile terminal 600.

In response, the UDP receiver 1150 of the mobile terminal 600 separates the UDP packet 1230, and the RTP receiver 1145 separates the RTP packet 1220, thereby encoding the audio data 1210 in the encoder 1130. Or to an audio output module 653 (more precisely an audio reproduction buffer).

When the audio data transmitted from the video display device 100 is audio data of ① or ②, the audio data is encoded, and thus is decoded by the decoder 1130 and then provided to the sound output module 653.

As another example, when the audio data transmitted from the image display apparatus 100 is ③, since it is decoded, no separate decoding is necessary, and thus, the audio data may be provided to the sound output module 653.

Meanwhile, the RTP transmitter 1045 and the UDP transmitter 1050 of the image display apparatus 100 may be provided in the network interface unit 135 of FIG. 3. In addition, the UDP receiver 1150 and the RTP receiver 1145 of the mobile terminal 600 may be provided in the wireless communication unit 610 of FIG. 6.

On the other hand, FIG. 14 illustrates another example of a user interface in the mobile terminal for entering the audio wireless reception mode, similar to FIG. 13.

Referring to the drawings, unlike FIG. 13, when the 'TV Sharing' item 1305 is selected from the remote control screen 1310 of FIG. 14A, the TV Sharing screen is displayed as shown in FIG. 14B. In the drawing, the TV Sharing screen exemplarily includes remotely controllable items such as a WiFi Sound item 1322, a second TV item 1324, a Cloud TV item 1326, and the like.

When the WiFi Sound item 1322 is selected, the controller 680 of the mobile terminal 600 may control to display the WiFi Sound screen 1330. In addition, the controller 680 of the mobile terminal 600 may control to transmit the TCP packet including the audio wireless reception mode start signal (start) to the image display apparatus 100 separately from the WiFi Sound screen 1330. have. That is, it may be controlled to perform synchronization with the image display apparatus 100.

15 is a flowchart illustrating a method of operating an image display apparatus according to another exemplary embodiment.

Referring to the drawing, the controller 170 of the image display apparatus 100 receives an audio wireless reception mode entry signal from the mobile terminal 600 (S1510). As described in operation S710 of FIG. 7, the image display apparatus 100 may receive TCP packet data including an audio wireless transmission mode start signal. The TCP receiver 1025 may separate the TCP packet and transmit the audio wireless transmission mode start signal to the controller 170 of the image display apparatus 100.

Next, the controller 170 of the image display apparatus 100 transmits audio data related information to be transmitted to the mobile terminal 600 (S1520). For example, the TCP receiver 1020 may transmit a TCP packet including channel number information, encoding level information, encoding scheme information, and the like, of the audio data to be transmitted to the mobile terminal 600.

Next, the controller 170 of the image display apparatus 100 receives decodable information from the mobile terminal 600 (S1530). In operation S1540, audio data having an encoding level corresponding to the decodable information is transmitted.

The controller 680 of the mobile terminal 600 generates decodable information indicating whether or not audio data to be received in the mobile terminal 600 is decodable, using encoding method information among the received audio data related information. can do. The decodable information, in the form of a flag, may indicate decodable in the case of '1' and in the case of '0'.

The control unit 170 of the video display device 100 receives the decodable information through the TCP receiving unit 1020 and, when decodable, maintains the encoding scheme of the audio data to be transmitted, and stores the audio data in the mobile terminal. Control to transmit to 600.

When the decodable information indicates that the decodable information cannot be decoded, the controller 170 of the video display device 100 changes the encoding method of the audio data to be transmitted and controls the audio data to be transmitted to the mobile terminal 600.

For example, when the encoding method of audio data to be transmitted is an AAC encoding method, and the encoding method cannot be decoded by the mobile terminal 600, the video display device 100 may receive such non-decodeable information. Can be. Accordingly, the video display device 100 may change the audio encoding method to be transmitted to the AC3 encoding method and transmit the information to the mobile terminal 600 again. When the mobile terminal 600 can decode the AC3 encoding scheme, the mobile terminal 600 may transmit information indicating that decoding is possible to the video display device 100. Accordingly, the video display device 100 may control to transmit the AC3 encoding audio data to the mobile terminal 600.

The change of the encoding scheme may be performed by changing the source of audio data to be transmitted. That is, the encoding method can be changed by changing the audio data of (3) in FIG. 11 from the audio data of (1) in FIG.

For example, when the encoding method of audio data of ① is an AAC encoding method and the encoding method of the encoder 1040 is an AC3 encoding method, the audio data to be transmitted is selected from the audio data of ① in FIG. Audio data can be changed.

Next, the image display device reproduces the multimedia data (S1550). In particular, the controller 170 of the image display apparatus 100 may decode the video signal among the demultiplexed audio signal, the video signal, and the data signal, and output the decoded video signal to be displayed on the display 180. Accordingly, the display 180 can display a broadcast image corresponding to the received broadcast signal. In this case, the audio output unit 185 of the image display apparatus 100 may not output the broadcast audio signal corresponding to the received broadcast signal.

16 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention, and FIGS. 17 to 19 illustrate various examples of the method of operating the image display device of FIG. 15 or the method of operating the mobile terminal of FIG. It is a figure referred to for.

Referring to the drawing, the mobile terminal 600 transmits an audio wireless reception mode entry signal to the image display apparatus 100 in operation S1610. The audio data related information is received from the video display device 100 in operation S1620. The sixteenth operation S1610 corresponds to the seventeenth operation S1710 of FIG. 17, and the sixteenth operation S1620 corresponds to the operation 1720 of FIG. 17.

The TCP transmitter 1125 of the mobile terminal 600 may transmit the TCP packet data including the audio wireless reception mode start signal to the video display device 100. 19A illustrates that the audio wireless reception mode start signal SWi_s is transmitted from the mobile terminal 600 to the video display device 100.

The TCP receiving unit 1025 of the video display device 100 receives the TCP packet data including the audio wireless receiving mode start signal, separates the TCP packet, and reproduces the audio wireless receiving mode start signal (start). It may be delivered to the management unit 1070. The reproduction management unit 1070 of the video display device 100 determines the received audio wireless reception mode start signal start as the audio wireless transmission mode start signal start based on the video display device 100. .

The reproduction management unit 1070 of the image display apparatus 100 may control to transmit audio data related information to be transmitted to the mobile terminal 600. Accordingly, the TCP transmitter 1020 of the image display apparatus 100 may transmit a TCP packet including channel number information, encoding level information, encoding scheme information, and the like, of the audio data to be transmitted to the mobile terminal 600.

FIG. 19B illustrates that the audio data related information Inf_aud to be transmitted is transmitted from the video display device 100 to the mobile terminal 600.

The TCP receiving unit 1120 of the mobile terminal 600 receives the TCP packet data including the audio data related information, separates the TCP packet, and includes channel number information, encoding level information, encoding method information, etc. of the audio data to be transmitted. The data may be transmitted to the reproduction management unit 1170.

Next, the mobile terminal 600 transmits the decodable information to the image display apparatus 100 (S1630). In operation S1640, audio data corresponding to the decodable information is received from the image display apparatus 100. FIG. Then, the received audio data is reproduced (S1650). In operation 1630, operation S1630 corresponds to operation 1730 of FIG. 17, operation 1640, operation S1640 corresponds to operation 1740 of FIG. 17, operation 1650, and operation S1650. It corresponds to step 1750 of FIG. 17 (S1750).

The reproduction management unit 1170 of the mobile terminal 600 may determine whether audio data to be received by the decoder 1130 of the mobile terminal 600 can be decoded based on the received encoding scheme information. The reproduction management unit 1170 of the mobile terminal 600 may control to transmit the decodable information indicating whether or not it is decodable through the TCP transmission unit 1125.

19C illustrates that the decodable information Inf_able is transmitted from the mobile terminal 600 to the video display device 100.

The TCP receiver 1025 of the video display device 100 may receive the TCP packet data including the decodable information, separate the TCP packet, and transmit the decodable information to the reproduction management unit 1070.

The reproduction management unit 1070 of the image display apparatus 100 may control to transmit audio data corresponding to the received decodable information.

For example, in response to the received decodable information, the source of audio data to be transmitted may be changed. That is, the audio data of (3) in FIG. 11 can be changed from the audio data of (1) in FIG.

For example, assuming that the mobile terminal 600 can decode the AC3 coding scheme, the audio data coding scheme of ① is an AAC coding scheme, and the coding scheme of the encoder 1040 is an AC3 coding scheme. In this case, the reproduction management unit 1070 of the video display device 100 may control the conversion unit 1085 to change the audio data of (3) in FIG. 11 from the audio data of (1).

Accordingly, the RTP transmitter 1045 and the UDP transmitter 1050 of the video display device 100 may transmit the RTP packet and the data to which the UDP packet is added to the mobile terminal 600 to the audio data to be transmitted, respectively. .

On the other hand, the reproduction management unit 1070 of the video display device 100 controls the synchronization unit 1080 to transmit the reproduction time information and the current time information of the transmitted audio data to the mobile terminal 600. can do. Such reproduction time information and current time information are output from the synchronization unit 1080. Accordingly, the TCP transmitter 1020 of the video display device 100 may transmit a TCP packet including reproduction time point information, current time information, and the like to the mobile terminal 600.

FIG. 19D illustrates that audio data Audio_1, 2, or 3, playback time information Inf_pt, and the like are transmitted from the video display device 100 to the mobile terminal 600. FIG.

Meanwhile, the UDP receiver 1150 and the RTP receiver 1145 of the mobile terminal 600 may separate the UDP packet and the RTP packet, respectively, and transmit audio data to the buffer 1190.

The reproduction management unit 1170 of the mobile terminal 600 controls to reproduce the received audio data.

When the audio data transmitted from the video display device 100 is audio data of ① or ②, the audio data is encoded, and thus is decoded by the decoder 1130 and then provided to the sound output module 653.

As another example, when the audio data transmitted from the image display apparatus 100 is ③, since it is decoded, no separate decoding is necessary, and thus, the audio data may be provided to the sound output module 653.

Meanwhile, the demultiplexer 310, the decoder 1030, the encoder 1040, and the playback manager 1070 of FIG. 18 may be provided in the controller 170 of the image display apparatus 100. Among these, the reproduction management unit 1070 may correspond to the processor 330 of FIG. 4. In addition, the TCP transmitter 1020, the TCP receiver 1025, and the UDP transmitter 1050 of the RTP transmitter 1045 of FIG. 18 may be provided in the network interface unit 135 of FIG. 3. The buffer 1090 may correspond to the storage 140.

Meanwhile, the reproduction management unit 1170 of the mobile terminal 600 of FIG. 18 may be provided in the control unit 680 of the mobile terminal 600. Also, the TCP transmitter 1125, the TCP receiver 1120, and the UDP receiver 1150 of the RTP receiver 1145 of FIG. 18 may be provided in the wireless communication unit 610 of FIG. 6. The buffer 1190 may correspond to the memory 660.

In FIG. 18, the audio signal processed by the RTP receiver 1145 is output to the decoder 1130 or the sound output module 653 via the buffer 1130. However, the RTP receiver ( When the audio signal processed at 1145 is an encoded audio signal, the decoded audio signal may be stored in the buffer 1130 after passing through the decoder 1130. In other words, the position of the buffer 113 can be disposed between the sound output module 653 and the decoder 1130.

20 is a flowchart illustrating a method of operating an image display device according to another embodiment of the present invention. FIG. 21 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention. FIGS. 22A to 23D are views of FIG. 20. FIG. 21 is a view referred to for describing various examples of an operating method of an image display device or an operating method of a mobile terminal of FIG. 21.

Referring to the drawing, the video display device 100 transmits audio data to the mobile terminal 600 (S2005). Accordingly, the mobile terminal 600 receives the audio data (S2105).

The RTP transmitter 1045 and the UDP transmitter 1050 of the image display apparatus 100 may transmit an RTP / UDP packet including audio data to the mobile terminal 600. In response, the UDP receiver 1150 and the RTP receiver 1145 of the mobile terminal 600 may separate the UDP packet and the RTP packet, respectively, and transmit audio data to the buffer 1190.

Meanwhile, FIG. 22A illustrates that the first audio data Audio_a whose channel is a stereo channel and whose coding level is encoded audio data is transmitted from the video display device 100 to the mobile terminal 600. do.

On the other hand, the video display device 100 may transmit the reproduction time information and the current time information of the audio data to the mobile terminal 600 in addition to the transmitted audio data. Such reproduction time information and current time information may be output from the synchronization unit 1080. Accordingly, the TCP transmitter 1020 of the video display device 100 may transmit a TCP packet including reproduction time point information, current time information, and the like to the mobile terminal 600.

Meanwhile, the TCP receiver 1120 of the mobile terminal 600 receives the TCP packet data including the reproduction time point information, the current time information, and the like, separates the TCP packet, and reproduces the reproduction time point information and the current time information ( current time) and the like may be transmitted to the reproduction management unit 1170.

Next, the reproduction management unit 1170 of the mobile terminal 600 calculates the reproduction delay information (S2107). The reproduction management unit 1170 of the mobile terminal 600, based on the audio data received from the video display device, the reproduction time information, the current time information (current time), and the current time information from the synchronization unit 1180, The reproduction delay information of the audio data can be calculated.

Next, the video display device 100 reproduces the multimedia data (S2010). Correspondingly, the mobile terminal 600 reproduces the received audio data (S2110).

The reproduction management unit 1070 of the video display device 100 controls to reproduce the multimedia data. For example, the decoder 1030 may control to provide the decoded video data to the buffer 1090, and control the buffering of the buffer 1090 to control video playback timing of the multimedia data. The buffer 1090 outputs input video data to the display 180 according to the set buffering value. 22A illustrates that the buffering value of the buffer 1090 of the video display device 100 is 2 ms.

On the other hand, the reproduction management unit 1170 of the mobile terminal 600 controls to reproduce the audio data. In FIG. 22A, since the first audio data Audio_a is received, the reproduction management unit 1170 controls the decoder 1130 to provide encoded audio data temporarily stored in the buffer 1190 and to decode the decoder 1130. do. Then, the decoded audio data is provided to the sound output module 653 and controlled to be reproduced.

In operation 2107, the reproduction management unit 1170 of the mobile terminal 600 determines whether the calculated reproduction delay information is greater than or equal to a predetermined value (S2113). 100) and control to transmit (S2120). In response, the video display device 100 receives the reproduction delay information (S2020).

The calculated reproduction delay information may be transmitted to the video display device 100 through the TCP transmitter 1125. The TCP receiver 1025 of the video display device 100 may transmit the reproduction delay information to the reproduction management unit 1070.

Next, when receiving the reproduction delay information from the mobile terminal 600, the video display device 100 changes the number of channels of audio data to be transmitted (S2025). In operation S2030, the audio data having the changed number of channels is transmitted to the mobile terminal 600. Correspondingly, the mobile terminal 600 receives the changed audio data.

FIG. 22B illustrates the second audio data Audio_b in which the channel is a mono channel and the encoding method is encoded audio data when the video display device 100 receives the reproduction delay information Inf_d1 from the mobile terminal 600. ) Is transmitted from the image display apparatus 100 to the mobile terminal 600. In other words, audio data in which the number of channels is changed from two channels to one channel is transmitted.

The change in the number of channels may be performed by the converter 1085 of the image display apparatus 100.

The buffer 1190 of the mobile terminal 600 temporarily stores the changed audio data.

Next, the reproduction management unit 1170 of the mobile terminal 600 calculates the reproduction delay information on the audio data whose channel number is changed again (S2127). The reproduction management unit 1170 of the mobile terminal 600, based on the audio data received from the video display device, the reproduction time information, the current time information (current time), and the current time information from the synchronization unit 1180, The reproduction delay information of the audio data can be calculated.

The reproduction management unit 1170 of the mobile terminal 600 controls to reproduce the audio data having the changed number of channels. Since the number of channels has been changed from the stereo channel to the mono channel, the audio reproduction time can be partially reduced. Accordingly, the audio data can be reproduced in synchronization with the multimedia data reproduced by the video display device 100.

On the other hand, the reproduction management unit 1170 of the mobile terminal 600 determines whether the calculated reproduction delay information is greater than or equal to a predetermined value in operation S2127 (S2133), and if so, adds the reproduction delay information. In step S2135, the control unit 100 transmits the data to the video display device 100 (S2135). Correspondingly, the video display device 100 receives the reproduction delay information (S2035).

Next, when the image display apparatus 100 additionally receives the reproduction delay information, the operation 2040 or less may be performed as an example.

That is, the video display device 100 delays the playback time of the multimedia data (S2040). In addition, it is determined whether the reproduction delay information is additionally received (S2045), and when the reproduction delay information is further received, the audio data having a variable encoding level is transmitted to the mobile terminal 600 (S2050). The video display device 100 reproduces the multimedia data (S2070).

22C illustrates a case in which the video display device 100 delays the playback time in the video display device 100 when the video display device 100 further receives the reproduction delay information Inf_d2 from the mobile terminal 600.

That is, in FIG. 22C, the buffering value of the buffer 1090 of the image display apparatus 100 is changed from 2 ms to 10 ms. As the buffering value increases as described above, the output time of the video data output from the buffer 1090 to the display 180 is delayed, and thus the playback time of the multimedia data is delayed. The amount of increase in the buffering value may be determined in correspondence with the received reproduction delay information Inf_d2.

Since the reproduction time of the multimedia data is delayed, the second audio data Audio_b, in which the channel is a mono channel and the encoding method is encoded audio data, continues from the video display device 100 to the mobile terminal 600. Can be sent.

22D illustrates a third audio signal in which the channel is a mono channel and the encoding method is decoded audio data when the video display device 100 further receives the reproduction delay information Inf_d3 from the mobile terminal 600. The data Audio_c is transmitted from the video display device 100 to the mobile terminal 600. In other words, the encoding level is changed from the encoded audio data to the decoded audio data.

The change of the encoding level may be performed by the converter 1085 of the image display apparatus 100. That is, the converter 1085 outputs audio data of ① demultiplexed by the demultiplexer 310 or audio data of ② encoded by the encoder 1040 and then decoded by the decoder 1030. Data can be output.

The buffer 1190 of the mobile terminal 600 temporarily stores the received audio data, and the reproduction management unit 1170 of the mobile terminal 600 outputs the audio data directly without passing through a separate decoder 1130. To the module 653, so that the audio data is sound output.

Next, when reproduction delay information is additionally received in operation 2035 (S2035), as another example, the operation 2055 or less may be performed.

That is, the video display device 100 transmits audio data having a variable coding level. Correspondingly, the mobile terminal 600 receives audio data having a variable coding level (S2155). When the reproduction delay information is further received (S2060), the video display device 100 delays the reproduction time of the multimedia data (S2065). The English display device 100 reproduces the multimedia data (S2070). Correspondingly, the mobile terminal 600 receives audio data having a variable coding level (S2155), and reproduces the audio data (S2170).

Steps 2055 through 2065 correspond to steps 2040 through 2050 (S2040, S2045, and S2050), but after changing the number of audio channels, the image display apparatus changes the coding level. There is a difference in delaying the playback time of multimedia data.

Meanwhile, the above-described steps 2040 to 2050 (S2040, S2045, and S2050) may be performed in parallel with steps 2055 to 2065 (S2055, S2060, and S2065). That is, when steps 2040 to 2050 (S2040, S2045, S2050) are performed, steps 2055 to 2065 (S2055, S2060, S2065) may not be performed. Alternatively, when steps 2055 to 2065 are performed (S2055, S2060, S2065), steps 2040 to 2050 (S2040, S2045, S2050) may not be performed.

In FIG. 23A, similar to FIG. 22A, the first audio data Audio_1, in which the channel is a stereo channel and the encoding method is encoded audio data, is transmitted from the image display apparatus 100 to the mobile terminal 600. Illustrates that.

FIG. 23B shows the second audio data Audio_2 in which the channel is a mono channel and the encoding method is encoded audio data when the video display device 100 receives the reproduction delay information Inf_d1 from the mobile terminal 600. ) Is transmitted from the image display apparatus 100 to the mobile terminal 600. In other words, audio data in which the number of channels is changed from two channels to one channel is transmitted.

Next, FIG. 23C illustrates a third channel in which the video display device 100 further receives the reproduction delay information Inf_d2 from the mobile terminal 600, wherein the channel is a mono channel and the encoding method is decoded audio data. Audio data Audio_3 is transmitted from the video display device 100 to the mobile terminal 600. In other words, the encoding level is changed from the encoded audio data to the decoded audio data.

Next, FIG. 23D illustrates a case in which the video display device 100 delays the playback time in the video display device 100 when the video display device 100 further receives the reproduction delay information Inf_d3 from the mobile terminal 600.

That is, in FIG. 23D, the buffering value of the buffer 1090 of the image display apparatus 100 is changed from 2ms to 10ms. As the buffering value increases as described above, the output time of the video data output from the buffer 1090 to the display 180 is delayed, and thus the playback time of the multimedia data is delayed. The amount of increase in the buffering value may be determined in correspondence with the reproduction delay information Inf_d3 received.

10, data exchange between the image display apparatus 100 and the mobile terminal 600 is directly exchanged between the image display apparatus 100 and the mobile terminal 600, but is not limited thereto. . That is, as described above with reference to FIG. 1, data exchange between the image display apparatus 100 and the mobile terminal 600 may be performed through the network router 500.

The video display device, the mobile terminal, and an operation method thereof according to the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified in various ways. All or some of these may optionally be combined.

Meanwhile, the method of operating the video display device or the mobile terminal of the present invention may be embodied as a processor readable code on a processor-readable recording medium provided in the video display device or the mobile terminal. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

Claims (21)

Entering an audio wireless transmission mode;
Synchronizing with the mobile terminal by a first wireless communication scheme;
Extracting audio data from the multimedia data; And
And transmitting the extracted audio data to the mobile terminal according to a second wireless communication method different from the first wireless communication method. According to claim 1, And
Reproducing the multimedia data. The method of claim 1,
The synchronization step,
Receiving control data from the mobile terminal by the first wireless communication scheme; And
And transmitting feedback data corresponding to the control data to the mobile terminal by the first wireless communication method. The method of claim 3,
The control data,
At least one of start information or end information of the audio wireless transmission mode, wireless channel setting request information, wireless channel change request information, allocated wireless channel information, decodable information, wireless channel environment information, or reproduction delay information;
The feedback data is,
And at least one of channel number information, encoding level information, encoding method information, reproduction time information, and current time information of audio data to be transmitted. The method of claim 1,
Transmitting at least one of reproduction time information and current time information of the audio data to the mobile terminal by the first wireless communication method during the transmission of the audio data. How to operate. The method of claim 1,
The first wireless communication method is a communication method based on bidirectional communication, and the second communication method is an operation method of an image display apparatus, characterized in that the communication method based on one-way communication. The method of claim 1,
And the second wireless communication method is a multicast-based communication method. The method of claim 1,
The first wireless communication method is a transmission control protocol (TCP) based communication method, and the second communication method is a communication method based on a real time transmission protocol (RTP) and a user datagram protocol (UDP). Operation method of image display device. Synchronizing with the image display apparatus according to the first wireless communication method according to the audio wireless transmission mode;
Receiving audio data from the video display device by a second wireless communication method different from the first wireless communication method; And
Playing the received audio data. 10. The method of claim 9,
The synchronization step,
Transmitting control data to the video display device by the first wireless communication method; And
And receiving feedback data corresponding to the control data from the image display apparatus by the first wireless communication scheme. 10. The method of claim 9,
Receiving at least one of reproduction time point information and current time information of the audio data from the image display device, by the first wireless communication method, during the reception of the audio data. How to operate. 10. The method of claim 9,
Entering a remote control mode for the image display device;
Displaying a remote control screen according to the remote control mode;
The audio wireless transmission mode is performed based on selection of an audio wireless transmission mode item in the remote control screen. 10. The method of claim 9,
The first wireless communication method is a communication method based on bidirectional communication, and the second communication method is a method of operating a mobile terminal, characterized in that the communication method based on one-way communication. display;
A network interface unit for exchanging data with the mobile terminal; And
According to an audio wireless transmission mode, by a first wireless communication scheme, synchronizing with the mobile terminal, extracting audio data from multimedia data, and extracting by a second wireless communication scheme different from the first wireless communication scheme. And a controller which controls the transmitted audio data to be transmitted to the mobile terminal. 15. The method of claim 14,
The control unit,
And display the multimedia data. 15. The method of claim 14,
The network interface unit,
Receives control data from the mobile terminal by the first wireless communication method, and transmits feedback data corresponding to the control data to the mobile terminal by the first wireless communication method. Device. 15. The method of claim 14,
The network interface unit,
And at least one of reproduction time information and current time information of the audio data is transmitted to the mobile terminal by the first wireless communication method during the audio data transmission. An audio output module for outputting a reproduced audio signal;
A wireless communication unit for exchanging data with an image display device; And
Audio received from the video display device according to an audio wireless transmission mode by a first wireless communication method, and controlled to synchronize with the video display device, and by a second wireless communication method different from the first wireless communication method. And a control unit for controlling data to be reproduced. 19. The method of claim 18,
The wireless communication unit includes:
And transmitting control data to the video display device by the first wireless communication method, and receiving feedback data corresponding to the control data from the video display device by the first wireless communication method. Mobile terminal. 10. The method of claim 9,
The wireless communication unit includes:
And receiving at least one of reproduction time information and current time information of the audio data from the video display device during the audio data reception by the first wireless communication method. 10. The method of claim 9,
The control unit,
When entering the remote control mode for the image display apparatus, the remote control screen according to the remote control mode is controlled to be displayed, wherein the audio wireless transmission mode is selected in the audio wireless transmission mode item in the remote control screen. A mobile terminal characterized in that the control to be performed based on.

Images