CN108111898B

CN108111898B - Display method of graphical user interface of television picture screenshot and smart television

Info

Publication number: CN108111898B
Application number: CN201711381835.6A
Authority: CN
Inventors: 高斯太; 于芝涛; 周杉
Original assignee: Qingdao Hisense Media Network Technology Co Ltd
Current assignee: Qingdao Hisense Media Network Technology Co Ltd
Priority date: 2017-12-20
Filing date: 2017-12-20
Publication date: 2021-03-09
Anticipated expiration: 2037-12-20
Also published as: CN108111898A

Abstract

The application provides an intelligent television, and belongs to the technical field of televisions. In order to meet the requirements of a more intuitive user interface and a seamless user interaction function, an input screenshot operation instruction is responded, a plurality of option bar sets are displayed while the current playing content is displayed on the display screen, wherein the option bars are respectively used for displaying a screenshot picture thumbnail, identifying content-related recommended content based on the screenshot picture, and/or responding to a user control instruction input interface of screenshot-related operation.

Description

Display method of graphical user interface of television picture screenshot and smart television

Technical Field

The present application relates to a display receiving terminal, and in particular, to a smart television device and a method for displaying a graphical user interface of a television screen capture.

Background

Smart tvs generally focus on online interactive media, internet tv, and streaming media on demand, rather than traditional broadcast media. The system can provide richer contents and services for users, and television manufacturers are dedicated to developing various convenient functions which are convenient for users to use, so that the use experience of products is improved.

Therefore, it is very important to provide a simpler and more intuitive interface and a visual content link for the existing smart tv, and to seamlessly interface with the user habit so as to browse and/or execute various functions of the smart tv.

Disclosure of Invention

The application aims to provide the smart television to meet the requirements of a more visual user interface and a seamless user interaction function. This disclosure addresses these improved functional needs by way of various aspects, examples, and/or configurations thereof. Furthermore, while the present disclosure has been described in terms of exemplary embodiments, it should be appreciated that individual claims may be presented in terms of each aspect of the disclosure. The present disclosure may provide a number of advantages, depending upon the particular aspects, examples, and/or configurations.

First, the present application provides a smart tv, including: a display screen; a memory; and a processor in communication with the memory and the display screen, the processor configured to perform: responding to an input screenshot operation instruction, displaying a current playing content on the display screen, and simultaneously displaying a plurality of option bar sets, wherein the option bars are respectively used for displaying a screenshot picture thumbnail, identifying content-related recommended content based on the screenshot picture, and/or responding to a user control instruction input interface of screenshot correlation operation.

Optionally, the recommended content, including the keywords matched with the corresponding content identified by the image of the screenshot, and the brief description of the recommended content related to the keywords, and/or the source or type of the content.

Optionally, the option bar is associated with a silo corresponding service or application program according to the content of the option bar, and the content of the option bar is selected to activate the service or application program.

Optionally, the service or the application is any one of a web application, a video application, a television program application, and a shopping application.

Optionally, the method further includes any one of recommendation of web page text content, recommendation of video on demand, shopping web page link and recommendation of application program.

Second, the present application provides a method for displaying a graphical user interface of a television screen screenshot, the method comprising: responding to an input screenshot operation instruction, displaying a current playing content on the display screen, and simultaneously displaying a plurality of option bar sets, wherein the option bars are respectively used for displaying a screenshot picture thumbnail, identifying content-related recommended content based on the screenshot picture, and/or responding to a user control instruction input interface of screenshot correlation operation.

Third, the present application provides a smart tv, including: a display screen; a memory; and a processor in communication with the memory and the display screen, the processor configured to perform: responding to the input screenshot operation instruction, and displaying a graphical user interface on the display screen, wherein the screenshot operation instruction comprises the following steps: the system comprises a current playing content display area and a screenshot content display area, wherein the screenshot content display area comprises a screenshot display bar used for displaying a picture obtained by current screenshot, a screenshot sharing instruction bar and a two-dimensional code information display bar generated by screenshot.

Optionally, the screenshot sharing instruction bar can share the screenshot to the terminal connected with the smart television.

Optionally, the two-dimensional code information and the screenshot are uploaded to a third-party server synchronously.

Optionally, when it is determined that the terminal connected with the terminal is not connected with the smart television in response to the sharing screenshot input instruction, the two-dimensional code information of the screenshot picture is output, so that the terminal scans the two-dimensional code information and obtains the screenshot picture from the third-party service.

Optionally, when a sharing screenshot input instruction is responded and a terminal connected with the terminal is determined to be connected with the smart television, the screenshot is transmitted to the terminal through a transmission protocol between the terminal and the smart television.

Third, the present application provides a method for displaying a graphical user interface of a television screen shot, the method comprising: responding to the input screenshot operation instruction, and displaying a graphical user interface on the display screen, wherein the screenshot operation instruction comprises the following steps: the system comprises a current playing content display area and a screenshot content display area, wherein the screenshot content display area comprises a screenshot display field used for displaying a picture obtained by current screenshot and a plurality of option fields used for indicating a user and a screenshot-related control instruction input field.

Optionally, in response to a selection instruction for a screenshot picture, the display screen displays a plurality of pictures obtained by continuously screenshot; and responding to the selection of the picture, and storing the selected screenshot in a preset picture storage area.

Optionally, when a control instruction of sharing the screenshot is responded and it is judged that the electronic terminal connected with the network exists in the television equipment, the television equipment sends information capable of pushing the screenshot to the electronic terminal;

and when responding to the instruction that the electronic terminal can receive the screenshot, the television equipment sends the screenshot picture to the electronic terminal.

Optionally, when a control instruction of sharing the screenshot is responded and it is judged that the electronic terminal connected with the network does not exist, the television equipment displays the picture two-dimensional code information of the screenshot.

Optionally, after the electronic terminal can scan the two-dimensional code information, the screenshot picture can be obtained on a third-party service that the television device sends the screenshot picture.

Fourth, the present application provides a method for displaying a graphical user interface of a television screen screenshot, the method including, in response to an input screenshot operation instruction, displaying the graphical user interface on the display screen, including: the system comprises a current playing content display area and a screenshot content display area, wherein a plurality of option bars in the screenshot content display area are used for displaying pictures obtained by current screenshot and webpage type recommended texts and/or pictures related to image identification content of the screenshot.

Optionally, the content displayed in the option bar of the recommendation of the web page category includes an identifier of a search source, a brief description of the recommended text and/or picture, and/or a subject or keyword of a search result.

Optionally, in response to the selection of the option bar recommended by the user for one of the web page classes, one or more information windows are displayed in a floating manner above the screenshot content display area, and the information windows are used for displaying the detailed content of the selected option bar.

Optionally, in response to a user selection of one of the recommended options of the web page class, a plurality of information windows are displayed in a floating manner above the screenshot content display area, and the plurality of information windows are used for displaying a plurality of different pieces of web page information.

Optionally, in response to a user selection of one of the information windows, an information sub-window is displayed in a floating manner on the plurality of information windows, and the information sub-window is used for detailed contents of the selected information window.

Fourth, the present application provides a method for displaying a graphical user interface of a television screen screenshot, wherein, in response to an input screenshot operation instruction, displaying the graphical user interface on the display screen includes: the system comprises a current playing content display area and a screenshot content display area, wherein a plurality of option bars in the screenshot content display area are used for displaying a webpage recommendation content brief description related to the image identification content of the screenshot.

Optionally, the brief description of the recommended content of the web page class includes text and/or pictures.

Optionally, in response to the user selecting one of the option bars, at least one information window is displayed above the screenshot content display area in a floating mode, and the information window is used for detailed content of the selected option bar.

Optionally, the content displayed in the information window includes a website logo bar indicating a content source, a related keyword bar, a term detail bar, and/or an album bar recommended by related pictures.

Optionally, the entry detail column includes a keyword summary, basic information, and a text.

Fifth, the present application provides a method for displaying a graphical user interface of a television screen screenshot, wherein, in response to an input screenshot operation instruction, displaying the graphical user interface on the display screen includes: the system comprises a current playing content display area and a screenshot content display area, wherein at least one of a plurality of option bars in the screenshot content display area is used for displaying a program list of a television program channel identified from an image in screenshot.

Optionally, the content is displayed in a program menu option bar, including: the Chinese name or logo that identifies the television program channel, and the thumbnail of the program listings.

Optionally, in response to a selection of the program menu option bar by the user, at least one information window is displayed in a floating manner above the screenshot content display area, and the information window is used for displaying a detailed list of the program menu.

Optionally, at least one program in the detailed list of the program list prompts an icon capable of being reserved, and the program is reserved in response to the selection of the icon by the user.

Optionally, in response to the user selecting a program in the detailed list, activating the video-on-demand application, and updating and displaying the information window or displaying a sub-information window on the information window in an overlapping manner to present the current time or the related episode of the selected program available for selection by the video-on-demand application.

Optionally, in response to the user selecting the current time or the related episode, the currently played content display area is switched to the program playing of the selected video-on-demand.

Sixth, the present application provides a method for displaying a graphical user interface of a television screen screenshot, wherein, in response to an input screenshot operation instruction, displaying the graphical user interface on the display screen includes: the television program recommendation system comprises a current playing content display area and a screenshot content display area, wherein at least one of a plurality of option bars in the screenshot content display area is used for displaying video recommendations related to the television program identified by the image in the screenshot.

Optionally, the content is displayed in a video recommendation option bar, including: the name of the video, and the content profile picture of the video.

Optionally, in response to a user selection of the video recommendation tab, at least one information window is displayed for displaying a detailed description of the selected video.

Optionally, in response to the user selecting the video recommendation option bar, an on-demand video application program is activated, and the currently played content display area is switched to the corresponding video in the selected on-demand video.

Seventh, the present application provides a method for displaying a graphical user interface of a television screen screenshot, where, in response to an input screenshot operation instruction, the graphical user interface is displayed on the display screen, including: the system comprises a current playing content display area and a screenshot content display area, wherein at least one of a plurality of option bars in the screenshot content display area is used for displaying commodity shopping information recommendation related to an image identified product in a screenshot.

Optionally, in response to the user selecting the shopping information recommendation option bar, a shopping application program is activated, and at least one window is displayed to display the webpage content of the shopping application program.

Optionally, in response to the user selecting the shopping information recommendation option bar, at least one shopping information list window is displayed, and the window presents content including a related commodity profile sub option bar and a shopping application program activated sub option bar.

Optionally, in response to a user selection of the sub-option bar activated by the shopping application, activating the corresponding shopping application.

Optionally, after the corresponding shopping application is activated, the current content playing display area is switched to the page related to the commodity in the selected shopping application.

Optionally, in response to the user selecting the shopping related information recommendation option bar, displaying at least one shopping information list window, wherein the window presents content including a related commodity profile sub option bar, a shopping application program activated sub option bar and a two-dimension code information sub option bar.

Optionally, the two-dimensional code information represents a webpage of a commodity corresponding to the shopping application program.

Optionally, in response to the selection of the user on the sub-option bar of the two-dimensional code information, the two-dimensional code information is displayed in an enlarged manner so that other electronic terminals can scan the two-dimensional code information to obtain the webpage of the corresponding commodity in the shopping application program.

Optionally, in response to the selection of the user on the commodity profile sub-option bar, commodity details are displayed, wherein the commodity details comprise one or more of any commodity information combination of size, color and specification. The foregoing is a brief summary of the application to explain certain aspects of the application. This summary is not an extensive or exhaustive overview of the application and its various aspects, examples, and/or configurations. It is intended to neither identify key or critical elements of the application nor delineate the scope of the application but to present some concepts of the application in a simplified form as an introduction to the detailed description that follows. It should be understood that other aspects, examples, and/or configurations of the disclosure may utilize one or more features, alone or in combination, set forth above or described in detail below.

Drawings

FIG. 1A is a first view of an example of an environment or smart television;

FIG. 1B is a second view of an example of an environment or smart television;

fig. 2 is a first view of an example of a smart tv;

FIG. 3 is a block diagram of an example of smart television hardware;

FIG. 4 is a block diagram of an example of smart television software and/or firmware;

FIG. 5 is a second block diagram of an example of smart television software and/or firmware;

FIG. 6 is a third block diagram of an example of smart television software and/or firmware;

FIG. 7 is a block diagram of an example of a content data service;

FIG. 8 is a front view of an example smart television screen;

figure 9 is an illustrative pictorial example of a user interface for a content/silo selector; (ii) a

FIG. 10 illustrates an example GUI for selecting a trigger screenshot;

FIG. 11 illustrates another example GUI for selecting a trigger screenshot;

FIG. 12 illustrates another example GUI for selecting a trigger screenshot;

FIG. 13 illustrates an example GUI for selecting a select screenshot picture option bar;

FIG. 14 illustrates an example GUII selecting the "share screenshot" tab under the extranet;

FIG. 15A illustrates an example GUI for selecting the "Baidu encyclopedia" option bar;

FIG. 15B illustrates an example GUI for selecting a description column under the "Baidu encyclopedia" option column;

FIG. 15C illustrates an example GUI for selecting the album column under the "Baidu encyclopedia" option column;

FIG. 15D illustrates an example GUI for selecting a picture in the album column under the "Baidu encyclopedia" option column;

FIG. 16A illustrates an example GUI for selecting the "news" option bar;

FIG. 16B illustrates an example GUI for selecting the determine news event tab under the "news" tab;

FIG. 17A illustrates an example GUI for selecting the "program listings" option bar;

FIG. 17B illustrates an example GUI for a program reservation tab;

FIG. 17C illustrates an example GUI for selecting a program name option bar;

FIG. 17D is a flowchart illustrating an example GUI for selecting past programming;

FIG. 18 illustrates an example GUI for selecting a video on demand option bar;

FIG. 19A illustrates an example GUI for selecting a shopping option bar;

FIG. 19B illustrates an example GUI for selecting a system-owned shopping application tab;

FIG. 20 illustrates an example flow diagram for selecting a screenshot to get a screenshot and keyword GUI;

FIG. 21 illustrates an example GUI for a user to trigger a screenshot instruction;

FIG. 22 depicts a flowchart of a screenshot process, in accordance with the present application;

FIG. 23 depicts a flowchart of a process for obtaining a screenshot picture at a client in accordance with the present application;

24A-24B are flow diagrams illustrating entry into an encyclopedia viewing content according to some embodiments;

FIG. 25 depicts a flowchart of viewing a GUI under a news tab in accordance with the present application;

FIG. 26A depicts a flowchart of a program reservation process under a program listing, in accordance with the present application;

FIGS. 26B through 26C are flowcharts of a process for enabling selection of an updated end program for play under a program listing;

FIG. 27 is an exemplary process flow diagram for selecting video on demand under a video playback application;

28A-28B depict a flowchart for viewing a GUI under a shopping option bar in accordance with the present application;

FIG. 28C depicts a flowchart of a GUI for obtaining shopping information at a client in accordance with the present application;

FIG. 28D is an exemplary flow diagram of enabling entry into a system-owned shopping application.

In these figures, similar components and/or features may have the same reference label. Also, various components of the same type may be distinguished from other similar components by reference to a letter in the label. If only the first reference label is used in the specification, the description is applicable to any one of the similar components that are identical to the first reference label, regardless of whether the second reference label is identical.

Detailed Description

In the following description, numerous specific details are set forth to provide a more thorough explanation of embodiments of the present invention. It will be apparent, however, to one skilled in the art that the specific details may not be employed to practice embodiments of the present invention.

The term "live television" as used in this application refers to television production broadcast in real time or substantially synchronized with the time of occurrence of an event.

The term "video on demand" (VOD) as used in this application refers to systems and processes that allow a user to select and view/listen to video or audio content on demand. The VOD system may stream the content to view the real-time content or download it to a storage medium for later review.

The term "display screen" refers to a portion of one or more screens for displaying computer output content to a user. The display screen may be a single screen display screen or a multi-screen display screen (referred to as a composite display screen). A single physical screen may contain multiple display screens that are managed as separate logical display screens. Thus, different content may be displayed on separate display screens, albeit in a portion of the same physical screen.

The term "remote control" refers to a component of an electronic device (most commonly a television receiver, DVD player, and/or home theater system) that is generally capable of wirelessly controlling the device within a short line of sight. The remote control typically uses infrared and/or Radio Frequency (RF) signals, and may include WiFi, wireless USB, Bluetooth^TMConnections, motion sensor enabled functions, and/or voice control. Touch screen remote controls are hand-held remote controls, replacing most of the physical built-in hard keys in a typical remote control with a touch screen user interface.

The term "display image" used in the present application refers to image content formed on a display screen. A typical display image is television broadcast content. The display image may occupy all or a portion of the display screen.

The term "panel" as used in this application may refer to a user interface displayed on at least a portion of a display screen. The panel may be interactive (e.g., accept user input) or merely provide information (e.g., not accept user input). The panel may be translucent so that the panel may be obscured but not obscure the content on the display screen. The panels may vary according to user input from buttons or a remote control interface.

The term "silo" as used in this application may be a logical representation of an input, source, or application. The input may be an electronic device (e.g., DVD, video recorder, etc.) connected to the television through a port (e.g., HDMI, video/audio input port, etc.) or a network (e.g., local and wide area networks, etc.). Unlike a device, an input may be connected to one or more devices as an electrical or physical connection configuration. The source, and in particular the content source, may be a data service (e.g., media center, file system, etc.) that provides the content. The application may be a software service that provides a particular type of functionality (e.g., live television, video on demand, user applications, picture display, etc.). A silo, as a logical representation, may have other associated definitions or attributes, such as settings, functions, or other characteristics.

As used in this application, the terms "determine," "calculate," and "computer" and variations thereof are used interchangeably and include any type of methodology, procedure, mathematical operation or technique.

Hereinafter, when the present disclosure refers to "selecting," "selected," "to select," or "selecting" a user interface element in a GUI, these terms should be understood to include using a mouse or other input device, clicking or "hovering" over the user interface element, or using one or more fingers or styli to touch a screen, tap, or make a gestural action on the user interface element. The user interface elements may be virtual buttons, menu buttons, selectors, switches, sliders, erasers, knobs, thumbnails, links, icons, radio buttons, check boxes, and any other mechanism for receiving input from a user.

Smart Television (TV) environment:

reference is made to some embodiments of the smart tv 100 shown in fig. 1A and 1B. The smart tv 100 may be used for entertainment, business applications, social interactions, content creation and/or consumption, and/or further include one or more other devices for organizing and controlling communications with the smart tv 100. It can therefore be appreciated that smart tv can be used to enhance the user interaction experience, whether at home or at work.

In some instances, the smart tv 100 may be configured to receive and understand various user and/or device inputs. For example, a user may interact with the smart tv 100 through one or more physical or electronic controls, which may include a pressButtons, switches, touch screens/zones (e.g., capacitive touch screens, resistive touch screens, etc.), and/or other controls associated with the smart tv 100. In some cases, the smart tv 100 may include one or more interactive controls. Additionally or alternatively, the one or more controls may be associated with a remote control. The remote controller may communicate with the smart tv 100 through wired and/or wireless signals. It will thus be appreciated that the remote control may communicate via Radio Frequency (RF), Infrared (IR) and/or a specific wireless communication protocol (e.g., Bluetooth)^TMWi-Fi, etc.). In some cases, the physical or electronic controls described above may be configured (e.g., programmed) to suit the user's preferences.

Alternatively, a smart phone, tablet, computer, notebook, netbook, and other smart devices may be used to control the smart tv 100. For example, the smart tv 100 is controlled using an application running on the smart device. The application program may provide the user with various smart tv 100 controls in an intuitive User Interface (UI) on a screen associated with the smart tv 100 through configuration. The user's option input on the UI may be configured to control the smart device 100 via the application program using one or more communication functions associated with the smart device.

The smart television 100 may be configured to receive input through a variety of input devices including, but in no way limited to, video, audio, radio, light, tactile, and combinations thereof. Furthermore, these input devices may be configured to enable the smart tv 100 to see and recognize user gestures and react. For example, the user may speak with the smart tv 100 in a conversational manner. The smart television 100 receives and understands voice commands like intelligent personal assistants and voice-controlled navigation applications (such as Siri for apple, Skyvi for android, Robin, Iris, and others) that are smart devices.

In addition, the smart tv 100 may be configured as a communication device that may establish a network connection 104 and connect to a telephone network operated by a telephone company using a telephone line 120 in a number of different ways, including wired 108 or wireless 112, cellular network 116. These connections 104 enable the smart tv 100 to access one or more communication networks. A communication network encompasses any known communication medium or collection of communication media and may use any type of protocol to communicate information or signals between endpoints. The communication network may include wired and/or wireless communication technologies. The internet is an example of a communication network 132 that, together with many computers, computer networks, and other communication devices around the world, forms an Internet Protocol (IP) network, interconnected by many telephone systems and other means.

In some instances, the smart tv 100 may be equipped with a variety of communication tools. Various communication tools may allow the smart tv 100 to communicate over a Local Area Network (LAN)124, a Wireless Local Area Network (WLAN)128, and other networks 132. These networks may act as redundant connections to ensure network access. In other words, if one connection is interrupted, the smart tv 100 will re-establish and/or maintain the network connection 104 using another connection path. Moreover, the smart television 100 also uses these network connections 104 to send and receive information, as well as Electronic Program Guide (EPG)136 interactions, receive software updates 140, contact customer services 144 (e.g., to obtain help or services, etc.), and/or access remotely stored digital media library 148. In addition, these connections also allow the smart tv 100 to make phone calls, send and/or receive email messages, send and/or receive text messages (e.g., email and instant messages), surf the web using an internet search engine, blog through a blog service, and connect/interact with an online community maintained by a social media website and/or social networking service (e.g., Facebook, Twitter, LinkedIn, Pinterest, google plus, MySpace, etc.). When these network connections 104 are used in combination with other components of the smart tv 100 (described in more detail below), we can also hold video teleconferences, electronic conferences, and other types of communications on the smart tv 100. The smart tv 100 may capture and store images and sounds using a connected camera, microphone, and other sensors.

Additionally or alternatively, the smart tv 100 may create and save screenshots of media, images and data displayed on an associated screen of the smart tv 100.

As shown in fig. 1B, the smart tv 100 may interact with other electronic devices 168 via wired 108 and/or wireless 112 connections. As described herein, the components of the smart television 100 allow the device 100 to connect to devices 168, including but not limited to a DVD player 168a, a Blu-ray player 168b, a portable digital media device 168c, a smart phone 168d, a tablet device 168e, a personal computer 168f, an external cable box 168g, a keyboard 168h, a pointing device 168i, a printer 168j, a game controller and/or gamepad 168k, a satellite dish 168l, an external display device 168m, and other Universal Serial Bus (USB), Local Area Network (LAN), Bluetooth, a USB, a^TMA High Definition Multimedia Interface (HDMI) component device, and/or a wireless device. When connected to the external cable box 168g or the satellite dish 168l, the smart tv 100 may access more media content.

Furthermore, as described in detail below, the smart tv 100 may receive digital and/or analog signal broadcasts of a tv station. It may operate as one or more of cable television, internet protocol television, satellite television, web television, and/or smart television. The smart television 100 may also be configured to control and interact with other intelligent components, such as a security system 172, a door entry/controller 176, a remote video camera 180, a lighting system 184, a thermostat 188, a refrigerator 192, and other devices.

The smart television:

fig. 2 illustrates the components of the smart tv 100. As shown in fig. 2, the smart tv 100 may be supported by a movable base or support 204 that is connected to a frame 208. The frame 208 surrounds the edges of the display screen 212 without obscuring its front face. The display screen 212 may comprise a Liquid Crystal Display (LCD), plasma screen, Light Emitting Diode (LED) screen, or other type of screen.

The smart television 100 may include an integrated speaker 216 and at least one microphone 220. In some examples, a first region of the frame 208 includes a horizontal gesture capture region 224 and a second region includes a vertical gesture capture region 228. The gesture capture areas 224 and 228 contain areas that can receive input by recognizing user gestures, and in some examples, the user need not actually touch the surface of the screen 212 of the smart tv 100 at all. The gesture capture regions 224 and 228 do not contain pixels that may perform a display function or capability.

In some examples, one or more image capture devices 232 (e.g., cameras) are added to capture still and/or video images. The image capture device 232 may contain or be connected to other elements, such as a flash or other light source 236 and a ranging device 240 to assist in focusing of the image capture device. In addition, the smart tv 100 may also identify the respective users using the microphone 220, the gesture capture areas 224 and 228, the image capture device 232, and the ranging device 240. Additionally or alternatively, the smart tv 100 may learn and remember preferences of individual users. In some instances, learning and memory (e.g., recognizing and recalling stored information) may be associated with user recognition.

In some examples, an infrared transmitter and receiver 244 may be further configured to connect to the smart tv 100 via a remote control device (not shown) or other infrared device. Additionally or alternatively, the remote control device may transmit wireless signals by other means besides radio frequency, light and/or infrared.

In some examples, the audio jack 248 is hidden behind a foldable or removable panel. Audio jack 248 contains a pointed round cannula (TRS) connector, for example, to allow a user to use headphones, or other external audio device.

In some examples, the smart tv 100 also includes several buttons 252. For example, fig. 2 shows buttons 252 on the top of the smart tv 100, which may be located elsewhere. As shown, the smart tv 100 includes six buttons 252 (from a to f) that can be configured for a particular input. For example, the first button 252 may be configured as an on/off button for controlling the system power of the entire smart tv 100. The buttons 252 may be configured together or separately to control various aspects of the smart tv 100. Some non-limiting examples include, but are not limited to, overall system volume, brightness, image capture device, microphone, and video conferencing hold/end. Instead of separate buttons, two buttons may be combined into one wave button. Such a waving button may be useful in certain situations, such as controlling a function of volume or brightness.

In some instances, one or more buttons 252 may be used to support different user commands. For example, the duration of a normal press is typically less than 1 second, similar to a quick input. The duration of the medium press down is generally 1 second or more but not more than 12 seconds. The duration of the long press is typically 12 seconds or more. This function of the button is generally dependent on the application activated on the smart tv 100. For example, in a video conferencing application, a normal, medium, or long press may mean ending the video conference, increasing or decreasing the volume, increasing the input response speed, and switching the microphone on or off, depending on the particular button. Depending on the particular button, a normal, medium, or long press may also control image capture device 232 to increase or decrease zoom, take a picture, or record a video.

Hardware functions:

fig. 3 illustrates some components of a smart tv 100 according to an example of the present application. The smart tv 100 comprises a display screen 304.

One or more display controllers 316 may be used to control the operation of the display screen 304. The display controller 316 may control the operation of the display screen 304, including input and output (display) functions. The display controller 316 may also control the operation of the display screen 304 and interact with other inputs, such as infrared and/or radio input signals (e.g., door access/gate controllers, alarm system components, etc.). In accordance with other examples, the functionality of the display controller 316 may be incorporated into other components, such as the processor 364.

Processor 364 may include a general-purpose programmable processor or controller that executes application programming or instructions. In accordance with at least some examples, processor 364 includes multiple processor cores and/or executes multiple virtual processors. In accordance with other examples, processor 364 may comprise a plurality of physical processors. As a particular example, the processor 364 may comprise a specially configured Application Specific Integrated Circuit (ASIC) or other integrated circuit, a digital signal processor, a controller, a hardwired electronic or logic circuit, a programmable logic device or gate array, a special purpose computer, or the like. The processor 364 is generally configured to execute program code or instructions to perform various functions of the smart tv 100.

To support the connection function or capability, the smart tv 100 may include an encode/decode and/or compress/decompress module 366 to receive and manage digital tv information. The encode/decode compress/decompress module 366 may decompress and/or decode analog and/or digital information transmitted from the public television chain or in the private television network received via the antenna 324, the I/O module 348, the wireless connection module 328, and/or the other wireless communication module 332. Television information may be sent to the display screen 304 and/or an attached speaker that receives analog or digital received signals. Any encoding/decoding and compression/decompression may be performed based on a variety of formats (e.g., audio, video, and data). The encryption module 324 communicates with the encode/decode compression/decompression module 366 so that all data received or transmitted by a user or vendor is kept secret.

In some examples, the smart tv 100 includes an additional or other wireless communication module 332. For example, other wireless communication modules 332 may include Wi-Fi, Bluetooth^TMWiMax, infrared, or other wireless communication link. The wireless connection module 328 and the other wireless communication module 332 may each be interconnected with a common or dedicated antenna 324 and a common or dedicated I/O module 348.

In some examples, to support communication functions or capabilities, smart tv 100 may include wireless connection module 328. For example, wireless connection module 328 may include a GSM, CDMA, FDMA and/or analog cellular telephone transceiver capable of transmitting voice, multimedia and/or data over a cellular network.

An input/output module 348 and associated ports may be added to support communication with other communication devices, servers and/or peripherals, etc., over a wired network or link. Examples of the input/output module 348 include an Ethernet port, a Universal Serial Bus (USB) port, a Thunderbolt^TMOr a Light Peak interface, an Institute of Electrical and Electronics Engineers (IEEE)1394 port, or other interface.

An audio input/output interface/device 344 may be added to output analog audio to an interconnected speaker or other device, and to receive analog audio input from a connected microphone or other device. For example, the audio input/output interface/device 344 may include an associated amplifier and analog-to-digital converter. Alternatively or additionally, the smart tv 100 may include an integrated audio input/output device 356 and/or an audio jack to which an external speaker or microphone is connected. For example, adding an integrated speaker and integrated microphone provides support for near-end speech or speakerphone operation.

A port interface 352 may be added. The port interface 352 comprises a peripheral or general purpose port that provides support for the device 100 to connect to other devices or components (e.g., docking stations) that may or may not provide additional or different functionality to the device 100 after interconnection. In addition to supporting the exchange of communication signals between device 100 and other devices or components, docking port 136 and/or port interface 352 may provide power to device 100 or to output power from device 100. The docking port 352 also contains an intelligent component that includes a docking module that controls communication or other interaction between the smart television 100 and the connected devices or components. The docking module may interact with software applications to remotely control other devices or components (e.g., media centers, media players, and computer systems).

The smart tv 100 may also include a memory 308 for the processor 364 to execute application programming or instructions and for temporary or long-term storage of program instructions and/or data. For example, the memory 308 may include RAM, DRAM, SDRAM, or other solid state memory. In some examples, a data store 312 is added. Similar to the memory 308, the data storage 312 may include one or more solid-state memories. In some examples, data storage 312 may include a hard disk drive or other random access memory.

For example, hardware buttons 358 may be used for certain control operations. One or more image capture interfaces/devices 340 (e.g., cameras) may be added to capture still and/or video images. In some examples, the image capture interface/device 340 may include a scanner, code reader, or motion sensor. The image capture interface/device 340 may contain or be connected to other elements, such as a flash or other light source. The image capture interface/device 340 may interact with a user ID module 350 that helps identify the identity of the user of the smart tv 100.

The smart tv 100 may also include a Global Positioning System (GPS) receiver 336. According to some examples of the invention, the GPS receiver 336 may further include a GPS module to provide absolute positioning information to other components of the smart tv 100. It will therefore be appreciated that other satellite positioning system receivers may be used instead of or in addition to GPS.

The components of the smart television 100 may draw power through the main power source and/or the power control module 360. For example, the power control module 360 includes a battery, an ac-to-dc converter, power control logic, and/or ports for interconnecting the smart tv 100 to an external power source.

Firmware and software:

FIG. 4 shows an example of software system components and modules 400. Software system 400 may contain one or more layers including, but not limited to, an operating system kernel 404, one or more libraries 408, an application framework 412, and one or more applications 416. One or

more layers

404 and 416 may communicate with each other to perform the functions of the smart tv 100.

The Operating System (OS) kernel 404 contains the primary functions that allow software to interact with the hardware associated with the smart tv 100. The kernel 404 may comprise a series of software to manage computer hardware resources and provide services to other computer programs or software code. Operating system kernel 404 is a primary component of the operating system and acts as a man-in-the-middle between application programs and data processing done using hardware components. Portions of the operating system kernel 404 may contain one or more device drivers 420. The device driver 420 may be any code in an operating system to assist in operating or controlling devices or hardware connected to or associated with the smart television. The driver 420 may contain code to operate video, audio, and/or other multimedia components of the smart television 100. Examples of drivers include display screens, cameras, Flash, Binder (IPC), keyboards, WiFi, and audio drivers.

Library 408 may contain code or other components that are accessed and executed during operation of software system 400. Libraries 408 may include, but are not limited to, one or more operating system runtime libraries 424, a television system Hypertext Application Language (HAL) library 428, and/or a data services library 432. Operating system runtime library 424 may contain code required by operating system kernel 404 and other operating system functions performed during the operation of software system 400. The library may contain code that is initiated during the operation of software system 400.

The tv service hypertext application language 428 may contain code required by the tv service, for execution by the application framework 412 or the application 416. The tv service HAL library 428 is specific to the smart tv 100 operation controlling the different smart tv functions. Furthermore, the tv services HAL library 428 may also consist of instances of other types of application languages than hypertext application languages or different code types or code formats.

The data services library 432 may contain one or more components or code to execute components that implement data service functionality. Data service functions may be performed in the application framework 412 and/or the application layer 416. FIG. 6 shows examples of data service functions and component types that may be included.

The application framework 412 may contain a general abstraction for providing functionality that may be selected by one or more applications 416 to provide specific application functionality or software for those applications. Thus, the framework 412 can include one or more different services or other applications that can be accessed by the application 416 to provide general functionality on two or more applications. Such functionality includes, for example, management of one or more windows or panels, planes, activities, content, and resources. The application framework 412 may include, but is not limited to, one or more television services 434, a television services framework 440, television resources 444, and user interface components 448.

The television services framework 440 may provide additional abstractions for different television services. The television services framework 440 allows for conventional access and operation of services related to television functions. The television services 436 are general services provided in a television services framework 440, which television services framework 440 may be accessed through applications in the application layer 416. The television resources 444 provide code for accessing television resources including any type of stored content, video, audio, or other functionality provided by the smart television 100. Television resources 444, television services 436, and television services framework 440 serve to perform various television functions accompanying smart television 100.

The one or more user interface components 448 may provide general components for the display of the smart tv 100. The user interface component 448 can be accessed as a generic component through various applications provided by the application framework 412. The user interface component 448 may be accessed to provide services for panels and silos as described in figure 5.

The application layer 416 contains and executes applications associated with the smart tv 100. The application layer 416 may include, but is not limited to, one or more live television applications 452, video-on-demand applications 456, media center applications 460, application center applications 464, and user interface applications 468. The live television application 452 may provide live television through different signal sources. For example, the live television application 452 may provide television using input from cable television, radio broadcast, satellite service, or other types of live television services. The live television application 452 may then display a multimedia presentation or a video and audio presentation of the live television signal on the display screen of the smart television 100.

The video-on-demand application 456 may provide video from different storage sources. Unlike the live television application 452, the video on demand 456 provides a video display from some stored source. The video-on-demand source may be associated with a user or a smart tv or some other type of service. For example, video on demand 456 may be provided from an iTunes library stored in cloud technology, from local hard disk storage containing stored video programs, or some other source.

Media center application 460 may provide applications needed for various media presentations. For example, media center 460 may service the display of images or audio other than live television or video on demand but still accessible to the user. The media center 460 may obtain media displayed on the smart tv 100 by accessing different sources.

The application center 464 may provide, store, and use applications. The application may be a game, a productivity application or some other application commonly associated with computer systems or other devices but which may run in a smart tv. The application center 464 may obtain these applications from different sources, store them in local memory, and then execute them for the user on the smart tv 100.

The user interface application 468 may provide services for a particular user interface associated with the smart television 100. These user interfaces may include the silo and panels described in figure 5. An example of user interface software 500 is shown in FIG. 5. Here, the application framework 412 includes one or more code components that help control user interface events, while one or more applications in the application layer 416 affect the use of the user interface of the smart tv 100. Application framework 412 may include a silo switch controller 504 and/or input event transmitter 508. There may be more or fewer code components in the application framework 412 than shown in FIG. 5. Silo switch controller 504 contains code and languages that manage the switching between one or more silos. The silo can be a vertical user interface function on the intelligent television and contains information available for users. Switch controller 504 may manage the switching between the two silos upon the occurrence of an event at the user interface. The input event transmitter 508 may receive event information for the user interface from the operating system and then transmit to the input event transmitter 508. Such event information may include button selections on a remote control or television or other type of user interface input. The input event sender may then send these event information to the silo manager 532 or the panel manager 536 (depending on the event type). Silo switch controller 504 may interact with silo manager 532 to affect changes to the silo.

Application framework 416 may contain user interface application 468 and/or silo application 512. The application framework 416 may include more or fewer user interface applications than are necessary to control the smart tv 100 than shown in fig. 5. The user interface application may include a silo manager 532, a panel controller 536, and one or more panels 516-528. Silo manager 532 manages the display and/or functionality of the silo. Silo manager 532 may receive or transmit information from silo switch controller 504 or input event transmitter 508 to modify the displayed silo and/or to determine the type of input the silo receives.

The panel manager 536 may display panels in the user interface to manage switching between the panels or to affect user interface inputs received in the panels. Accordingly, the panel manager 536 may communicate with different user interface panels, such as the global panel 516, the volume panel 520, the settings panel 524, and/or the notification panel 528. The panel manager 536 may display these types of panels depending on the input from the input event transmitter 508. The global panel 516 may contain information related to the home screen or the user's highest level information. The volume panel 520 displays information related to audio volume controls or other volume settings. The information displayed by the settings panel 524 may relate to audio or video settings or other settable characteristics of the smart tv 100. The notification panel 528 may provide information related to user notifications. These notifications may be related to information such as video on demand displays, favorites, currently available programs, etc., or other information. The content of the notification is related to the media or some type of setting or operation or the smart tv 100. The panel manager 536 may communicate with the panel controller 552 of the silo application 512.

The panel controller 552 may control some of the several types described above. Thus, the panel controller 552 may communicate with the top panel application 540, the application panel 544, and/or the bottom panel 548. These several panels are different from each other when displayed in the user interface of the smart tv 100. Thus, the panel controls may set the panels 516-528 to a certain display orientation (determined by the top panel application 540, the application panel 544, or the bottom panel application 548) depending on the system configuration or the type of display currently in use.

FIG. 6 is an example of a data service 432 and data management operations. Data management 600 may include one or more code components associated with different types of data. For example, data service 432 may have several code components therein that may perform and correlate video on demand, electronic program guide, or media data. The data services 432 may have more or fewer component types than shown in fig. 6. Each of the different types of data may include the data model 604-612. These data models determine what information the data service stores and how it will store. Thus, the data model can manage any data, regardless of where they come from and how they will be received and managed in the smart tv system. Accordingly, the

data models

604, 608, and/or 612 may provide the ability to translate or influence the translation of data from one form to another form that is available to the smart tv 100.

Various data services (video on demand, electronic program guide, media) have a

data sub-service

620, 624 and/or 628 for communicating with one or more internal and/or external content providers 616.

Data subservices

620, 624, and 628 communicate with content provider 616 to obtain data, which is then stored in

databases

632, 636, and 640. The sub-services 620, 624, and 628 may communicate with the content provider, initiating or enabling one or more source plug-

ins

644, 648, and 652 to communicate with the content provider. The source plug-

ins

644, 648, and 652 differ for each content provider 616. Thus, if the data has multiple content sources, each data subservice 620, 624, and 628 may decide and then enable or launch a different source plug-in 644, 648, and/or 652. In addition, the content provider 616 may also provide information to the resource arbiter 656 and/or the thumbnail cache manager 660. The resource arbiter 656 may communicate with resources 664 external to the data services 432. Accordingly, the resource arbiter 656 may communicate with cloud storage, network storage, or other types of external storage in the resources 664. The information will then be provided to the data subservices 620, 624, 628 by the content provider module 616. Similarly, the thumbnail cache manager contains thumbnail information from one of the data subservices 620, 624, 628 and stores the information in the thumbnail database 666. Moreover, the thumbnail cache manager 660 may also extract or retrieve information from the thumbnail database 666 to provide to one of the data subservices 620, 624, 628.

Fig. 7 shows an exemplary content aggregation structure 1300. The structure may include a user interface and

content aggregation layer

1304 and 1308. User interface layer 1304 may include a television application 1312, a media player 1316, and applications 1320. The television application 1312 enables a viewer to view channels received via an appropriate transmission medium, such as cable, satellite, and/or the internet. The media player 1316 may view other types of media received over an appropriate transmission medium, such as the internet. The applications 1320 include other television-related (pre-installed) applications such as content viewing, content searching, device viewing and setup algorithms, and may also cooperate with the media player 1316 to provide information to the viewer.

The content source layer 1308 contains, as data services, a content source service 1328, a content aggregation service 1332, and a content presentation service 1336. The content source service 1328 manages content source investigators including local and/or network file systems, digital network device managers (which discover handheld or non-handheld devices (e.g., digital media servers, players, renderers, controllers, printers, uploaders, downloaders, network connection functions, and interoperability units) via known techniques such as multicast universal plug and play or UPnP discovery techniques, and retrieve, parse, and encrypt device descriptors for each device discovered, notify content source services for newly discovered devices, and provide information about previously discovered devices, such as indexes), internet protocol television or IPTV, digital television or DTV (including high definition and enhanced television), third party services (such as the services referenced above), and applications (such as android applications).

The content source investigator may track content sources, typically configured as binary. The content source service 1328 may launch a content source investigator and maintain an open and persistent communication channel. The communication includes a query or command and a response pair. The content aggregation service 1332 manages content metadata obtainers, such as video, audio, and/or image metadata obtainers. The content presentation service 1336 provides content indexing interfaces, such as an android application interface and a digital device interface.

The content source service 1328 may send communications 1344 to and receive from the content aggregation service 1332. The communication contains notifications regarding the latest and deleted digital devices and/or content and search queries and results. The content aggregation service 1332 can send communications 1348 to and receive from the content presentation service 1336, including device and/or content lookup notifications, advisories and notifications of content of interest, and search queries and results.

When a search is performed, particularly when the user is searching or browsing for content, content presentation service 1336 may receive a user request from user interface layer 1300, opening a socket and sending the request to content aggregation service 1332. The content aggregation service 1332 first returns results from the local database 1340. Local database 1340 contains indexes or data models and indexed metadata. The content source service 1328 further issues search and browse requests for all content source investigators and other data management systems. The results will be sent to the content aggregation service 1332, which updates the database 1340 to reflect the further search results and provides the original content aggregation database search results and the data update results reflecting the more content source service search results to the content presentation service 1336 through the previously opened socket. The content presentation service 1336 then provides the results to one or more components of the user interface layer 1300 for presentation to the viewer. When the search phase is over (e.g., the search phase is terminated by a user or user action), the user interface layer 1300 will disconnect the jack. As shown, media may be provided from the content aggregation service 1332 directly to the media player 1316 for display to the user.

As in fig. 8, video content (e.g., television programs, videos, televisions, etc.) is displayed on the front side of screen 212. The window 1100 obscures a portion of the screen 212 and the remainder of the displayed video content, and may also cause the portion of the screen 212 displaying the video content to move up or down and/or compress as the height of the window 1100 changes, and it may also be that the window 1100 is superimposed over the video content such that the change in height of the window 1100 does not affect the display position of the video content.

Window 1100 can include one or more items of information, such as: a front panel navigation bar associated with the currently displayed image and/or content, detailed information (e.g., title, date/time, audio/visual indicators, ratings and genres, etc.), a hotkey bar, an information entry bar associated with a browse request and/or a search request.

In some examples, window 1100 includes information regarding the appropriate information associated with the content (e.g., name, duration, and/or remaining content viewing time), setup information, television or system control information, application (active) icons (e.g., pre-installed and/or downloaded applications), application centers, media centers, web browsers, input sources.

Fig. 9 is an illustrative pictorial example of a user interface for a content/silo selector. The avatar 1400 includes a content source selector 1404. Content source selector 1404 includes icons for one or more silos 1408-1424.

Content source selector 1404 may include two or more icons 1408-1424 representing different silos. For example, icons 1408 through 1420 represent different content application silos. The different content application silos may comprise a live tv silo, represented by icon 1408. A live television silo is a logical representation of a broadcast television signal application that may provide television content to a user of the television 100. A Video On Demand (VOD) silo is represented by reference 1412. The VOD silo provides a path for access to video or other types of media that may be selected and provided to the user on demand. The media center silo is represented by icon 1416. The media center silo contains applications that provide images and/or movies developed or stored by users. The media center provides a way for the user to store his media using the smart tv 100. The application silo is represented by icon 1420. Application silos provide games and other user applications that can be accessed and used on television, and the like. The input source silo 1424 may be any type of device or other storage mechanism that is connected to the television 100 through an input port or other electrical connection, such as: HDMI, and other input interfaces, or input interface aggregation silos.

FIG. 10 illustrates an exemplary Graphical User Interface (GUI) when a user triggers a screenshot. When the video is being played on the display screen, after the screenshot command is triggered by clicking a remote controller, touching the screen or gestures, a graphical user interface is displayed on the display screen, including a current content playing display area 160 and a screenshot content display area 170. The screenshot content display area 170 is a set of a plurality of options, the current content playing display area 160 continues to play the current playing content, and an option in the screenshot content display area 170 displays a thumbnail of a screenshot and recommended content related to the content identified by an image of the screenshot, or a user control instruction input interface related to the screenshot function operation.

The screenshot content display area 170 is displayed in the screen edge area, such as: the right side or the lower side may be displayed in an overlapping manner on the video content, and the current content playing display area 160 may display only a portion of the video content that is not covered by the screenshot content display area 170, or the video content may be moved upward or leftward and compressed and displayed in the current content playing display area 160.

In some examples, the screenshot content display area 170 is displayed at the right edge of the display by default and may be selected by the user to move to the left, or to the lower side and change the layout direction. In some instances, the user may also place the system at any location on the selectable item in the system setting.

In some examples, the screenshot operation is performed under the condition of displaying any content, a picture which is completely consistent with the displayed content is obtained, or the screenshot operation instruction can be executed only when the picture or video is displayed on a screen, otherwise, the screenshot fails. In contrast, the former screenshot operation is a content screenshot operation performed after a content source is synthesized with a menu (OSD), and the latter is a content screenshot operation performed before a content source is synthesized with an OSD, which captures only an image of a current content source.

The screenshot content display area 170 includes a screenshot display field and an associated content display field, and/or a plurality of options fields associated with a control instruction input field.

FIG. 11 illustrates another exemplary GUI of a user triggered screenshot. In the illustrated embodiment, the GUI includes a current content play display area 160 and a screenshot content display area 170.

As shown in fig. 11, a screenshot display column 1710 is used for displaying a picture thumbnail obtained by successfully obtaining the current screenshot; and an associated control instruction input field including a "share screenshot" 1720 for displaying a picture indicating that the user shares the screenshot in the screenshot display field 1710 accordingly, and a prompt field of a "voice input search keyword" for indicating that the user inputs a related voice command keyword accordingly to perform a re-voice search.

A plurality of associated content display fields (1730-1790) are used to display the image recognition from the screenshot to determine the keywords, and a summary of the content related to the keywords, and/or the source or type of the content.

For example, when the current content playing display area 160 is playing drama "song", a screenshot instruction of the user is received, and the current playing drama scene is successfully screenshot, so that the screenshot display column 1710 captures the obtained thumbnail image of the picture.

And obtaining keywords related to the picture content of the screenshot through image recognition, such as: obtaining a key word 'Ju Juer' of a plot character; and the key word of Ju Er decorative actor Qiaoxin; obtaining a keyword 'oriental satellite' from the station caption of 'oriental satellite' identification; identifying the scenario of the scenario to obtain a keyword ' sister's deep sentiment '; and identifying the figure clothes of the screenshot to obtain a keyword 'pure color round collar gentlewoman woolen sweater' and the like. It should be noted that the image recognition process may be performed by local recognition, or may be performed by third party recognition to obtain the keyword.

According to the keywords, the associated content display bars (1730-.

Such as: the associated content display bar (1730) displays a keyword 'Jujuer', the content summary comprises '22 years old, five hundred powerful business trainees, good-family lambkin girls,' recommendation content from a Baidu webpage, and a 'Baidu' website icon representing the source of the recommendation content.

The associated content display bar 1740 displays the keyword "joxin," and a brief summary of the content includes that "while we are still young," joxin is not well distinguished by women second. ... "web page recommendations from online news," the website icon representing the source of the recommendation. Also as content bar 1750 ("Singlemicroblog" recommended content).

The associated content display box (1760) displays the keyword "eastern satellite television", and the brief description of the content includes the recommendation information of the program announcement list "15: 07 the year, say.

The associated content display column (1770) displays a keyword 'sister's love depth ', the brief description of the content comprises recommendation information of' sister 'love depth' drama pictures and a type 'video' typeface of the recommendation information.

An associated content display field (1780) is used to display the keyword "buy! "and content summary includes recommended photos like clothing, and the" Taobao "website icon represents the source of the recommended photos.

The associated content display field (1790) displays the keyword "plain round neck gentlewoman sweater", and the content summary includes recommended photos of similar clothes and price information, etc., and the "shopping" website icon represents the source of the recommended photos.

As shown in fig. 11, the options in the screenshot content display area 170 are associated with a corresponding service or application program of a silo according to the content, such as: the associated content display field and/or the associated control instruction input field correspond to associated related services or application programs. When a user selects one of the tabs, a corresponding service or application is activated in response to the user selection.

Such as: the "shared screenshot" 1720 is associated with a shared function application and the "speech input search keyword" is associated with a speech recognition application. A plurality of associated content display bars (1730-. An associated content display box (1760) is associated with the live television service program of the silo 'live television'. An associated content display bar (1770) is associated with the silo "video on demand application.

The screenshot content display 170 provides access without interfering with the current content playback display 160. That is, the user can still view the content of the current content playback display area 160 while the screenshot content display area 170 is displayed. For example, the screenshot content display area 170 may be semi-transparent to see through (i.e., still visible to the user) the content displayed in the current content playback display area 160. The screenshot content display area 170 may allow the user to quickly enter the core function of the smart tv 100, so that the user can always see the content activated by triggering the screenshot in the current content playing display area 160. Moreover, the screenshot content display area 170 may provide a consistent user experience with respect to all applications and contain important contextual content.

In terms of functionality and content sequencing, the screenshot content display area 170 includes a plurality of tabs including: webpage-like text content recommendations, such as: baidu encyclopedia 1730, news bagua 1740, microblog 1750; a video on demand referral 1770; shopping-type web pages or application program recommendations, including web-page-type panning shopping 1780 and built-in application-type shopping 1790; and screen two-dimensional code recognition 1798, and a bottom-most display of a new function preview 1799. The screenshot content display area 170 is represented by icons and text or shortcuts, respectively, for describing content searched from different media applications. The search content includes text including, but not limited to, the name (if a search for an item, showing its brand or similar brand), title, body, etc. resulting from the search.

In some examples, the user can set the text line number through system settings, such as: not more than 4 rows. The user may select a tab in the list to navigate to the corresponding content.

The user selected or pre-selected location is displayed in the screenshot content display area 170 in the focus box 150. For example, focus box 150 contains a box around the shared screenshot. Although blocks are used in the description, other methods or configurations may be used to select and/or identify icons. For example, in addition to boxes, the focus box 150 may also contain a background that adjusts 1720, etc. icons and text, thus making the colors, shades, or tints different. Alternatively or additionally, the focus frame 150 may contain icons and text that are enlarged or dilated.

In some instances, the content of multiple tabs may be displayed (e.g., scrolled or slid) in a moving manner based on user selection. Each option bar correspondingly displays a search or recommended content source, wherein the content source categories are as follows: the web page category, the video amount category, the television program category, the shopping category, the two-dimensional code category and the like.

The content in the option bar is presented as a circular list by scrolling or sliding, which is convenient for the user to find the interested content, such as: flip forward or backward, and when the user finishes viewing the last tab, either quiesce or automatically jump to the first tab.

For a GUI where the user triggers a screenshot, the search results, such as: web applications, video applications, television program applications, shopping applications, and the like. Specifically, one or a combination of several of them may be used. And applications of the above type may also be configured by the user, such as: i.e. the user may decide to display the type of application or to select a content source.

For example, the web page content source recommendation may be generated by a third-party search program, and the web page content recommendation related to the screenshot content is displayed by performing image recognition on the screenshot content, such as: baidu encyclopedia 1730, news bagua 1740, microblog 1750 and the like. Can be one or a plurality of combination modes of webpages such as Baidu encyclopedia 1730, news bagua 1740, microblog 1750 and the like.

And recommending the television program content, and displaying a program list 1760 of a television source under the station caption by a third-party program through identifying the image of the station caption in the video screenshot.

Video-like content recommendations, presented in the video recommendation tab 1770, may be displayed by a third-party program by image recognition of the screenshot content, with the video recommendation associated with the screenshot content. The video of the final search presentation may be a television show, movie, or web page short video related to the video content or the current screen display content.

And (4) recommending the shopping content, namely displaying online shopping information of the commodity related to the screenshot content by a third-party program through image recognition of the screenshot content. Online shopping information, such as: a panning application, a built-in shopping application (e.g., shopping for shares), or a tv shopping channel, etc. The method can be one or a plurality of combination modes of a Taobao application program, a built-in shopping application program (such as a shared shopping program) and a TV shopping channel.

The two-dimension code display is that a third-party program generates a corresponding two-dimension code according to screenshot content, so that a user can identify the two-dimension code through scanning, download a picture to another client, or perform other user operations and the like. And the two-dimension code information and the screenshot are synchronously uploaded to a third-party server.

Taking playing of "jubilation" of a tv show as an example, the following are: when video content of a plot 'Ju Er' is played, screenshot operation is triggered by clicking a remote controller or a touch screen or a gesture, and screenshot content is obtained and presented to a user GUI. The current content play display area 160 still plays the content of the video at the next moment, and the right screenshot content display area 170 appears with a plurality of tab lists.

In turn, the first tab displays a thumbnail 1710 of the screenshot picture.

The second tab displays the screenshot function tab 1720 "share screenshot".

The third option column is related content of a search option column 1730 of "Baidu encyclopedia" searched by a third-party program, the option column is divided into two columns, the left side of the first column displays "Jure Juer", the right side of the first column displays LOGO of a search source "Baidu encyclopedia", the second column displays search content (such as basic brief introduction, character and temperament and current life condition), the option column does not display all searched content, the text does not exceed four rows, and the rest part is hidden.

The fourth column is related content of the eight diagrams of news 1740 "internet news" searched by the third party program, and is divided into two columns, wherein the left side of the first column displays "johan xin", the right side of the first column displays LOGO of the search source "internet news", and the second column displays entries of the search content.

The fifth option column is a satellite television program option column 1760 related to the "eastern satellite television" content searched by the third-party program by identifying the station caption of the video playing source or the content display area, the option column is divided into two columns, the left side of the first column displays the "eastern satellite television", the right side displays the searched source "program list", and the second column displays the program list details including but not only the time list and the program title played at the determined time.

The sixth option bar is related to the video content option bar 1770 video related to the character which is still performed by the search obtained by the third party program through identifying the character in the video playing, the option bar is divided into two bars, the left side of the first bar displays another popular video 'sister' in which the character has performed, the right side displays a search source 'video', and the second bar displays a video picture.

The seventh option bar is a shopping option bar 1780 related contents of 'Taobao' which are obtained by a third party program through identifying clothing ornaments worn by a character in video playing and are the same as or similar to the clothing ornaments of the character, the option bar is divided into two bars, the left side of the first bar displays 'buying and buying', the right side of the first bar displays LOGO of a search source 'Taobao', and the second bar displays a commodity picture.

The eighth option bar is an option bar 1790 of the self-contained shopping application program of the system, related contents of the content option bar which is the same as or similar to the display of the figure and clothing ornaments under the shopping application program are searched, the option bar is divided into two bars, the left side of the first bar displays the names of commodities which are the same as or similar to the figure and clothing ornaments, the right side of the first bar displays the search source of 'shared shopping', the left side of the second bar displays a commodity cover map, and the right side of the second bar displays the names of the commodities and the prices of the commodities.

The ninth option bar is a screen two-dimensional code option bar 1798 provided by the third-party program, and converts the screen capture into a screen two-dimensional code, the option bar is divided into two bars, the left side of the first bar is identified by the screen two-dimensional code, the right side of the first bar displays the search source two-dimensional code, and the second bar displays the two-dimensional code capture.

The tenth tab displays a new function announcement tab 1799 "Coming song" indicating a new function announcement.

As previously described, the display of content in the options bar on the screenshot content display area 170 depends on the screenshot capture picture content. The plurality of tabs also provide customization information for the respective programs. The screenshot content display area 170 is assembled from application tabs. Such as: a function option bar and a content option bar. The function option bar corresponds to a bar, namely a function indication bar, such as: share screenshot 1720, and voice input search keyword. The content option bar comprises at least two bars, wherein the first bar is a navigation bar and displays an application icon and an application name, and the activated application can be selected by a user, such as: baidu encyclopedia, Internet news, New wave microblog and the like; the second column shows the content area, which may be updated, for the particular application search to obtain the relevant recommended content.

Further, in some examples, as shown in fig. 12, the displayed content in the tab may be a plain text, plain picture, or a co-occurrence type of tab.

Such as: in the plain text columns, each of the columns may be displayed in two columns, one above the other, and the name 1801 of the search source is displayed on the right side of the first column, for example: hundredth, online news, newwave microblog, treasure panning, etc.; the first column displays to the left the title, topic, or search keyword 1802 of the search result. The second column displays search text 1803. The body is typically in text form, with no more than 4 lines of the body, and the excerpt shown with an ellipsis at the end of the fourth line. The user can set the display line number, font, color and font size of the text in the system. The display mode is more suitable for hundredth encyclopedia or microblog.

The following steps are repeated: in the pure picture option bar, the option bar display is divided into two upper and lower bars, and the name 1801 of the search source is displayed on the right side of the first bar, such as: hundredth, online news, newwave microblog, treasure panning, etc. The first column displays on the left the title, topic, or keyword 1802 of the search result; the second column displays search pictures 1811, the number of which is no more than 3. The content displayed by the option bar can be of a type with both graphics and texts. In the option bar with coexisting text, the option bar display is divided into two columns, namely the name 1801 of the search source is displayed on the right side of the first column, such as: hundredth, news, newwave microblog, treasure panning, etc.; the first column displays the title, subject, or keyword 1802 of the search result to the left, and the second column displays the search picture along with the picture summary 1821. If the source of the entry is Taobao, the right side of the first column displays Taobao, the left side displays a title of Taobao, the left side of the second column displays a picture of a Taobao commodity, and the right side of the second column displays basic information (such as size, color, price and the like) of the commodity. In this embodiment, the option bar GUI may be at least one of plain text, plain picture, and coexisting text types.

In some instances, the GUI displays a function class option bar. In the GUI shown in fig. 12, the height of the function class card is half of that of the content class card, the function class card is a pure text by default, and the size of the function class card is consistent with that of the title of the content card (for example, the sharing screenshot 1831 and the new function preview icon Coming song 1832), and the function class card can also be freely designed (for example, voice input).

As previously described, the GUI may present display objects having a tab. The user selects the option bar to view various types of search contents, and when the option bar is selected through a remote controller, a gesture, or a touch screen, information according to the type of information selected in the current option bar is presented to the user. For example, when a menu bar is selected, which lists descriptions, summaries and related text or picture descriptors, the content description is displayed in the content display area, and the user can move the menu bar focus box to an icon or user interface pattern via a remote control, gesture or touch screen. While moving the focus box, the user may select the content corresponding to the tab again via a remote control, a gesture, or a touch screen, and the user may see more detailed content associated therewith, as shown in fig. 15A-15D.

As shown in fig. 13, when the user moves the focus frame to select the screenshot option 1710, the screenshot is displayed on the current content playing display area 160, and the 3-thumbnail of continuous screenshot is expanded and extends from right to left to the content display area for selection by the user, and the user may also pre-configure the number of continuously-captured images. On one hand, the user can reselect one of the screenshot pictures and select the screenshot picture to perform searching operation again, so that the user can accurately obtain the required picture when performing screenshot operation. On the other hand, the user may view, access, and/or push social content from their social network that is relevant to the selected screenshot.

When the user selects the "share screen shot" option 1720, the screen shot can be sent to other clients in two ways, namely, an "intranet" and an "extranet", for example: cell-phone, panel computer.

The "intranet" installs an application associated with the smart television 100 on another client by the user, such as: and the other client downloads and installs the maritime communication assistant APP, and the established same-network link is transmitted. At this time, the detail page of another client sharing the screenshot shows the user that if your mobile phone and the television are in the same network, the application program associated with the smart television can be directly started, for example: the "television micro assistant" takes the screenshot. And if the program of the mobile phone terminal runs in the background, the program receives a television push message and prompts the user. If the program is not in the background, the user starts the program, if the television is connected, the screenshot information is actively pulled, whether the information user is read or not is judged, and if the information user is not read, the user is prompted.

The external network uploads the picture to the server, and the user opens the webpage with the picture to obtain the picture. The mode of opening the webpage can be realized by scanning the formed two-dimensional code. FIG. 14 illustrates an exemplary GUI for selecting the "share screen shot" option bar under the extranet. When the user selects the "share screenshot" option, the "extranet" passes the picture to the server, and the GUI includes a current content play display area 160, a screenshot content display area 170, and a two-dimensional code window 202 that floats above the option. The two-dimensional code window displays the two-dimensional code of the webpage information with the screenshot, and the user obtains the screenshot in a mode of opening the webpage with the screenshot. If the user client is an Android system mobile phone, the scanning area can be directly aligned with the two-dimensional code of the display area to scan and download the picture; if the mobile phone is the iOS system mobile phone, the scanning area of the mobile phone is aligned with the display area of the television, and the user presses for a long time at the same time, so that the purpose of downloading pictures is achieved.

The window information 202 would require the user to connect any type of device to the television 100 or network associated with the television 100 in order for the user to retrieve media such as images and videos stored or created to the television 100. Once the device is scanned, the handset may be stored for content displayed through the media center silo.

FIG. 15A shows an example of a GUI in which the user selects the "Baidu encyclopedia" option bar 1730. In the illustrated embodiment, an encyclopedia information window 203 is displayed in the screenshot content display area 170 over the tab. The Baidu encyclopedia information window 203 is suspended above the screenshot content display area 170, for example: a Logo column 2031, an album column 2032, a related word column 2033, and an entry detail column 2034. The Logo column 2031 displays an encyclopo of department of hundred, and the album column 2032 displays a screen shot, a business card image, and an album. The related word column 2033 displays tag or semantic items, and the description column 2034 does not display all but replaces them with ellipses at the bottom of the screen. When the user selects the description bar 2034, a text introduction for the introduction to the screen shot and related information is displayed, as in fig. 15B, and the user can slide in to view the contents of interest to the user by clicking a remote controller, a gesture, or a touch screen.

Taking the display of the video playback in "song by song" as an example, the associated word bar 2033 is: jubilation, qieying, fan victory and beautiful, Andi, Xitong, Xiao, etc. The term detail column 2034 includes three sections, a business card summary, basic information, and a term body. The user obtains more comprehensive introduction information from a screenshot page to know about related resources.

Fig. 15C shows that when the user moves the focus frame to the album column 2032, the user selects the specific album column 2032, a new picture list window 204 is suspended above the hundred-degree encyclopedia information window 203 for showing the relevant picture list, and the focus frame can move up and down on the picture list window 204 to select a picture desired to be further viewed. Once the user has focused on any one of the pictures in the picture list window 204, another GUI is provided for interacting with the selected picture to the user, a picture window 205 floats above the picture list window 204, which displays a single picture with the view fully displayed therein as a full screen square, as described with reference to fig. 15D.

In some embodiments, in the case of encyclopedia incompetence, any one or more of the album column, keyword column, Logo column, and entry details column may not be included. If the album and the keyword are not displayed and only the entry details column is displayed, or if the album column and the entry details column are displayed without an associated word column.

Fig. 16A shows an example of a GUI when the user selects the news option bar 1740. In the example, a news event list window 301 is displayed in the GUI, the news event list window 301 being suspended above the screenshot content display area 170. And the news event list is a plurality of news event option columns, each news event option column describes a news event and comprises a date, a title (for example, the title is set not to exceed five lines, the part of the title exceeding the five lines is hidden, and the end of the fifth line is marked with "..") to indicate that the title is hidden), and a picture (for example, the picture is selected to be displayed or not displayed in the setting, and if the picture is displayed, at most three pictures can be displayed in the form of picture thumbnails).

Fig. 16B shows that when the user selects a select news event tab, the select activation determines a news event window 302, which is hovering above the news event list window 301. At this point, the news event depicted in the selected news event tab 3021 is fully expanded, including all of the news record's headlines, origination time, text, and pictures, while the unselected news event tab 3022 still displays only the headlines and a portion of the pictures. Not all displayed portions may be viewable through the scrolling list.

FIG. 17A shows an example of a GUI when the user selects the "program guide" tab 1760. In the example, a program listing window 401 is included in the GUI. The program listing window 401 is suspended above the screenshot content display area 170. When a television program channel is identified, a program list of the television program channel is displayed, such as: the next time slot or the name of the program being played, etc. Taking the oriental tv as an example, the program list window 401 displays a video program preview to be played by the oriental tv in the next time slot, including two columns, one column displaying the name 4011 of the program to be rebroadcast and the playing time point 4012 in the next stage, and one column displaying the icon 4013 of whether to make a program reservation. When the program reservation icon is a selection bar of a selected program to be watched, the user can be prompted to watch the selected program before the preset time. The program listings window 401 lists the program listings in a plurality of rows.

Fig. 17B shows an example of a GUI in which the user selects the program reservation 4013 option field. When the user selects the confirm program reservation icon 4013, the content display area shows a reservation reminder field 1601 indicating that the user is to be alerted before the program starts.

Fig. 17C shows an example of a GUI in which the user selects the program name 4013 option field. In an example, a third party video application is activated, the third party video application storing video content that has been played to completion in the past. A new program section or period list window 402 floats above the program listing window 401 for showing the number of sections or periods that the program has been played in the past, and the user can select to play the program. When a certain number of nodes and periods are selected, the current content play display area 160 displays a skip reminder column 1602 "skip after 5 seconds," activates another video application silo (e.g., video-on-demand application, or live-tv application) and plays the selected program through the video application, or the user may select "go back" to continue playing the program of the current silo, as described with reference to fig. 17D.

Through the selection of the user graphical interface, the smart television 100 can conveniently and quickly provide silo content for the user. Such as: live television signals, show menus to VOD or provide video last displayed in a VOD application, show one or more images or menus to select images within a media center, show menus or applications previously executed in an application silo, or show other types of silo content. Thus, a user may logically and contextually interact with the television 100 based on the silo and the content that the silo presents.

FIG. 18 shows an example of a GUI for selecting a video on demand option bar. In the example, the GUI includes a video options bar list window 501. The video tab list window 501 floats above the screenshot content display area 170, and the video tabs include a public rating, a movie poster, and a movie title. Under the option bar, a Video On Demand (VOD) silo is activated, and the video option bar list window 501 is displayed as a plurality of video option bars with high similarity to such videos or video contents with high interest degree of the user in currently provided media contents, such as: war movies, collusion and suspicion movies, dramas, comedy, etc. that the user likes.

In some examples, each row of the video taskbar list includes video information associated with the currently playing video wherein the video content most associated with the currently playing video is at the top of the video taskbar list. The video tab list window 501 is a scrolling video tab list (after the last related content is displayed, the user can jump to the first related content without having to go back to the first displayed content in turn) or a sliding video tab list.

FIG. 19A shows one example of a GUI for selecting a shopping option bar 1790. In the example, the GUI includes a shopping information list window 601. The shopping information list window 601 floats above the screenshot content display area 170. Shopping information list window 601 includes a plurality of item profile sub-options bar 6011, a "two-dimensional code scan" sub-option bar 6012, and a shopping application skip sub-option bar 6013. In some examples, when a user selects an item profile tab, the item tab is expanded and the shopping tab list window 601 displays item details (including size, color, specifications).

In some examples, when the user selects the "two-dimensional code scanning" option bar, the two-dimensional code is enlarged to the whole shopping option bar list window 601 and is square, and more commodity information is obtained from the mobile phone end through mobile phone scanning.

In some examples, when the user selects shopping application jump option bar 6013, the system shopping application (e.g., shared shopping) is in a pending state, jump option bar 1603, jump indication (e.g., jump after 5 seconds) and return indication are displayed at the bottom of current content playback display area 160. By selecting the skip indication, the system activates the shopping application and transitions to the TV shopping page to facilitate shopping on the TV, as described with reference to FIG. 19B.

As shown in fig. 20, the present embodiment provides an example of a method 2000 for implementing application display by screenshot. Although fig. 20 shows a general order of execution for method 2000, method 2000 may contain more or fewer steps, or the order of the steps may be arranged differently than method 2000 shown in fig. 20. The method 2000 is a set of computer-executable instructions executable by a computer system or processor, encoded or stored on a computer-readable medium, or embodied as circuitry in an Application Specific Integrated Circuit (ASIC) or Field Programmable Gate Array (FPGA) that can perform the method. In the following, we will explain the method 2000 in connection with the systems, components, modules, data structures, user interfaces, etc. described in connection with fig. 1-21.

In step 2002, the driver receives a screenshot signal from a remote controller, a gesture, or a touch screen. The screenshot signal will be transmitted to the input event sender. The input activity transmitter 508 determines whether the signal is from the player based on information from the drive 420.

In step 2003, if the player is in screenshot mode 1, the player performs screenshot operation, and if the player is not in screenshot mode 2.

And step 2005, judging whether the screenshot operation is successful, if so, executing step 2020, otherwise, executing step 2006.

In step 2006, "screen shot failed" is prompted. Then step 2007 is executed, in step 2007, whether the keyword is a third-party keyword is judged, if yes, a home page list is entered, but the screenshot related function is not available, and the screenshot event is ended 2008. If not, the page is also displayed at the current time, no screenshot and no sidebar are displayed to display any keyword, no function and content can be provided, and the screenshot event is ended 2009.

In step 2020, a determination is made as to whether there is a third party keyword. If not, step 2026 is performed, and the home page list is entered, and only the screenshot is obtained, and the logo may be identified, and only information related to the screenshot is provided. If yes, step 2022 is executed to optimize the keyword, and then step 2024 is executed to normally enter a home page list and display information of a side bar of a keyword option bar related to screenshot.

Figure 22 conceptually illustrates a process 2100 for some embodiments of screenshots. As shown, process 2100 first begins by receiving (at 2105) a command to upload an image to a photo sharing website to trigger a screenshot. The application may receive the command through a remote control button, a gesture, or a set of user interactions touching the screen, or through some variation.

The process then determines (at 2110) that multiple shots before and after the last command time point are taken in succession and displayed in thumbnail form on the shot content display area 170, and that not all of the shot pictures are displayed on the sidebar, but only one thumbnail is displayed. When the user selects the screenshot thumbnail (at 2115), one thumbnail on the sidebar expands to the current content play display area 160, exposing all screenshot pictures to the user to provide the selection. After the user selects the determined screenshot picture, the process determines that the picture (2120) is stored in a picture storage area (such as an album) of the smart tv 100, and creates a log.

Figure 23 conceptually illustrates a process 2200 of some embodiments for sharing a screenshot to another client. While fig. 14# refers to a website that shares screenshots, this can be a "photo sharing" website or a broader social media application (e.g., WeChat, QQ). As shown, process 2200 first begins with receiving (at 2105) an input to trigger the sharing of the screenshot, a command to upload the image to the photo sharing website. The application may receive the command through a remote control button, a gesture, or a set of user interactions touching the screen, or through some variation.

The process then determines (at 2210) whether another client is associated with the smart television, i.e., the client enables the smart television to exchange data directly with the client by installing an application associated with the smart television. And an application program for data exchange between a user and the client is specially installed in the intelligent television. The "intranet" is delivered through the internet link established by the user installing the application program associated with the smart television 100 at another client (for example, another client downloads and installs the hail letter assistant APP). At this time, the detail page of another client sharing the screenshot shows the user that if your mobile phone is in the same network as the television, the application program (such as "haixin television micro assistant") associated with the smart television can be directly started to obtain the screenshot.

When another client of the user is associated with the smart tv 100, the process determines (at 2220) whether the associated program is running in the background. The user may have opened the associated application, be it in the application interface or be running in the background, i.e. when the tv sends the screenshot message, the user may receive it in time. The process determines (at 2225) that the television prompts (at 2225) the user for a push message and sends the screenshot picture to the client prompting the user to receive it. At this time, the processing is finished, and the user receives the picture shared by the smart television at another client. Otherwise, when the associated application program is not started, the process 2222 requires the user to start the program and pull the screenshot information, and then the process is ended.

When another client of the user is not associated with the smart tv 100, the process 2215 outputs the two-dimensional code with the screenshot picture information in the screenshot content display area 170, and the another client scans the two-dimensional code on the smart tv 100 through a website sharing the screenshot or a broader social media application (e.g., WeChat, QQ), and then restores the two-dimensional code to the picture information, and ends the process.

Fig. 24A-24B are example process 2300A flow diagrams for selecting an encyclopedia. As shown, process 2300A begins by first receiving (at 2305) an input to trigger a share screen shot and a command for image recognition. The application may receive the command through a remote control button, a gesture, or a set of user interactions touching the screen, or through some variation.

The device searches on the third party application responsive to (at 2310) parsing of the screenshot content or the identified image, upon generating for display a search result in a GUI including a display object having a user selectable option bar to provide access to a different third party application. The tab generally describes the search content of the third party application. And the option bars displayed in the side bars form a list for the user to browse and view in a sliding or waterfall flow manner.

In some examples, the device detects (at 2315) that the user selects a web-like third party application (e.g., encyclopedia, fig. 15A), then a new window is provided in the sidebar of the GUI (at 2325) to provide access to the information window of the encyclopedia search. The information window 203 is a side information window that floats on the screenshot content display area 170 and corresponds to any one or more of an album column, a keyword column, and a term detail column. In some embodiments, in the case of encyclopedia incompetence, any one or more of the album column, keyword column, and entry details column may not be included. If the album and the keyword are not displayed and only the entry details column is displayed, or if the album column and the entry details column are displayed without an associated word column.

In the illustrated example, the user has selected the description bar 2034 focus box, resulting in the presentation of the GUI illustrated in FIG. 15B. In some examples, the location of album column 2302, keyword 2033 column, and Logo column 2301 do not change, while the contents of the itemization column will expand in their entirety, sliding up as selected by the user.

In the illustrated example, when the user selects the album bar 2032 focus box, the GUI is shown as in FIG. 15C. Process 2330 receives a user input selecting an album bar in an encyclopedia information window, and a picture list window 204 is provided in the sidebar of the GUI (at 2335) to provide access to the picture list for the encyclopedia search. The picture list window 204 is a side information window that floats on the screenshot content display area 170, includes thumbnails of at least three pictures and can be slid to view pictures not shown below (e.g., the GUI shown in fig. 15C).

In some instances, the user may simply view the picture thumbnails in the picture list window without further selection of pictures. In some instances, the user may exit from the picture list window 204 to the last option bar or exit the screenshot content display area 170 to browse the video content being played in the current content play display area 160 in full screen.

When the user is to further select a picture, the device detects (2345) a user command to select a picture option bar in the picture list window, and displays the selected picture window 205 in the GUI. A picture window 205 floats above the picture list window 204, and the picture window displays a single picture in which the picture has a full display view square, as described with reference to fig. 15D.

Fig. 25 is an exemplary process 3100 flow diagram for selecting a news tab. As shown, process 3100 begins by first receiving (at 3105) an input to trigger sharing of a screenshot and a command for image recognition. The application may receive the command through a remote control button, a gesture, or a set of user interactions touching the screen, or through some variation.

The device performs a search on the third party application in response (at 3110) to the parsing of the screenshot content or the identified image, and generates for display the search results to a GUI including a display object having a user selectable option bar to provide access to a different third party application. The tab generally describes the search content of the third party application. And the option bars displayed in the side bars form a list for the user to view in a sliding type or waterfall type or rolling type browsing mode.

In some examples, the device detects (at 3115) that the user selects a web-page-like third party application (e.g., internet news, fig. 16A), and provides a new window in the sidebar of the GUI (at 3125) to provide access to the news event list window. The news event list window 301 is a side information window that floats on the screenshot content display area 170 and corresponds to a plurality of news event options columns. A news event tab describes a news event and specifically includes a date, a title (no more than five rows are set, the part of the title beyond five rows is hidden, and the end of the fifth row is indicated with ".. -" to indicate that the title is hidden), a picture (a picture can be selected to be displayed or not in the setting, and if a picture is displayed, a maximum of three pictures can be displayed in the form of a picture thumbnail).

In some instances, the user may only view news headlines and/or profiles in the news event list window without further selection to view news. In some instances, the user may exit from the news event list window 301 to the previous option bar or exit the screenshot content display area 170 to browse the video content being played in the current content play display area 160 in full screen.

When the user wants to view the details of a news event, the apparatus detects (3135) a user's command to select a news event option column in the news event list window, and displays the selected news event window 302 in the GUI. In the illustrated example, the user selects the focus box corresponding to the first news event tab, resulting in the presentation of the GUI shown in FIG. 16B. A news event window 302 is activated and floats above the news event list window 301. The news event depicted in the selected news event tab 3021 is expanded in its entirety, including the title, origination time, text, and pictures of all the news recordings. While the unselected news event tab 3022 still displays only the title and a portion of the picture. Not all displayed portions may be viewable through the scrolling list.

Fig. 26A-26C are flow diagrams of an exemplary process 4100 for selecting a program listings tab. As shown in fig. 26A, a flowchart of a program reservation process under a program guide according to the present application is described. The process 4100 first begins with receiving (at 4105) a command to trigger input of a share screen shot and image recognition. The application may receive the command through a remote control button, a gesture, or a set of user interactions touching the screen, or through some variation.

The device searches on the third-party application in response to (at 4110) parsing the screenshot content or the identified image, and generates for display the search results to a GUI including a display object having a user-selectable option bar to provide access to a different third-party application. The tab generally describes the search content of the third party application. And the option bars displayed in the side bars form a list for the user to view in a sliding type or rolling type or waterfall type browsing mode.

In some examples, the device detects (at 4115) that the user selects a television program class third party application, and provides a new program listings window in the sidebar of the GUI (at 3125) to provide access to the program options bar. When the third-party application identifies that the source of the video played in the current content playing display area 160 is a satellite television video channel, the third-party application analyzes the station caption and displays the program list played under the satellite television corresponding to the image of the station caption, for example, as shown in fig. 16A. The program listings window 401 is a side information window that floats on the screenshot content display area 170 and corresponds to a plurality of program listings. Each program option bar includes a program name option bar and a reservation option bar.

The device receives (at 4130) a user request to reserve the program for the next time period, and displays a prompt for successful reservation in the content display area of the GUI (e.g., as shown in fig. 17B). In some instances, the user may choose to exit the program listings window 401 into the screenshot content display area 170, or exit the screenshot content display area 170, to facilitate full screen playback of the video content of the current content playback display area 160.

In some instances, when the user is to view the completed program content under the program name, the program name of the program option bar in the program list window 401 is selected and the apparatus receives (at 4230) the selected program name option bar. At this point, a third party video application is activated that stores video content that has been updated to completion in the past. As shown in the exemplary GUI of fig. 17C, a new program episode or number of episodes window 402 floats above the program listings window 401 to show the episodes or number of episodes the program has played in the past. The top two tabbed with markers preview the video content for the program.

Upon selection of a certain number of knots and times, the apparatus receives (at 4240) user input selecting a program set/period tab under the program name. The received user input may be a click on a remote control button, a gesture, or a touch screen. The current content play display area 160 of the main display panel (at 4245) displays a jump reminder column 1602 "jump 5 seconds later" to jump to the play of the program selected by the other silo (and third party video application) and display (at 4260) the video content of the selected past program in the GUI.

In some embodiments, the device detects (at 4250) that the user selects the "back" command in the jump prompt option bar, and the GUI exits (at 4255) the screenshot content display area 170 to continue playing the program of the current silo, as described with reference to figure 17D.

FIG. 27 is an exemplary process flow diagram for selecting video on demand under a video playback application; as shown, the process 5100 first begins with receiving (at 5105) a command to trigger input of a shared screenshot and image recognition. The application may receive the command through a remote control button, a gesture, or a set of user interactions touching the screen, or through some variation.

The device searches on the third party application responsive to (at 5110) parsing of the screenshot content or the identified image, upon generating for display a search result in a GUI including a display object having a user selectable option bar to provide access to a different third party application. The tab generally describes the search content of the third party application. And the option bars displayed in the side bars form a list for the user to browse and view in a sliding or waterfall flow manner.

In some examples, the device detects (at 5120) that the user selects a video-like third party application (an associated video application within the smart television, such as a gathering look; or jumps to a broader video playback application, such as arcade, PPTV, fig. 18), and provides a new window (such as a video menu bar list window, at 5125) in the sidebar of the GUI to provide access to the searched related video list window. The video tab list window 501 is a side information window that floats on the screenshot content display area 170 and includes a public rating, a movie poster, and a movie title. Under the option bar, the video-on-demand silo is activated, the video option bar list window 501 is displayed as a plurality of video option bars with high similarity to such videos, or the video option bar list window 501 is displayed as video content with high interest level of the user in the currently provided media content (such as war movies liked by the user, collusion movies, music dramas, comedy dramas, etc.). The video tab list window 501 is a scrolling video tab list (after the last related content is displayed, the user can jump to the first related content without having to go back to the first displayed content in turn) or a sliding video tab list.

In the illustrated example, when the device receives a request from a user to select a video menu bar focus box, the process may show a GUI similar to that of fig. 17D. Upon selection (at 5135) of a certain number of knots and number of periods, the device receives user input selecting the program set/period tab under the program name. The received user input may be a click on a remote control button, a gesture, or a touch screen. Then (at 5140) the current content play display area 160 of the main display panel displays a jump reminder bar 1602 "jump 5 seconds later" to jump to the play of the program selected by the other silo (and third party video application) and the video content of the selected past program is displayed in the GUI (at 5160).

In some embodiments, the device detects (at 5145) that the user selects the "back" command in the jump prompt option bar, and the GUI exits (at 5150) the screenshot content display area 170 to continue playing the program of the current silo. In some instances, the user may exit from the video menu bar list window 501 to the previous menu bar or exit the screenshot content display area 170 to browse the video content currently playing in the current content play display area 160 in full screen.

Figure 28A is a flowchart of an exemplary process 6100A for selecting a shopping-type third party application. As shown, process 5100 first begins with receiving (at 6105) an input to trigger sharing of a screenshot and a command for image recognition. The application may receive the command through a remote control button, a gesture, or a set of user interactions touching the screen, or through some variation.

The device searches on the third party application responsive to (at 6110) parsing of the screenshot content or the identified image, generating for display a search result in a GUI including a display object having a user selectable option bar to provide access to a different third party application. The tab generally describes the search content of the third party application. And the option bars displayed in the side bars form a list for the user to browse and view in a sliding or waterfall flow manner.

In some examples, as shown in fig. 28B, the device detects (at 6120) that the user selects a shopping-type third party application (associated shopping application within the smart tv, such as a share shopping, or jumps to a more socially popular shopping application, such as nao bao, kyoto, fig. 19A), and provides a new window (such as a shopping information list window, at 5125) in the sidebar of the GUI to provide access to the searched related items list window. The shopping information list window 601 is a side information window that floats on the screenshot content display area 170 and corresponds to an item profile option 6011 (the specific content display is determined by the specific collaboration), "two-dimensional code scan" option 6012, and a shopping application jump option 6013. Under this bar, the shopping application is activated and the shopping information list window 601 is either a scrolling video bar list (after the last related content is displayed, one can jump to the first related content without having to take the user back to the first displayed content in turn) or a sliding video bar list.

In some examples, when the user selects an item option, the device receives (at 6130) user input selecting an item option in the shopping information list window, the item option is expanded, and item details (including size, color, specifications) are specifically displayed (at 6135) on the shopping option list window 601 in the GUI.

FIG. 28C depicts a flowchart of a GUI for obtaining shopping information at a client in accordance with the present application. In some examples, the device detects (at 6230) that the user selects the two-dimensional code scanning option bar, displays a two-dimensional code window (at 6235) on the GUI, enlarges to the entire shopping option bar list window 601 and takes a square shape, and scans the two-dimensional code on the smart tv 100 through another client to obtain more commodity information from the other client.

FIG. 28D is an exemplary flow diagram of enabling entry into a system-owned shopping application. In some examples, the device receives (at 6330) a user input selecting shopping application buy-sharing in the shopping information list window, and when the jump option bar 6013, the system shopping application (e.g., buy-sharing) is in a pending state, the jump reminder option bar 1603 is displayed at the bottom of the current content playback display area 160 of the GUI (at 6335), including a jump indication (e.g., jump after 5 seconds) and a return indication. The system shopping application is activated and goes to the tv shopping page where the share and buy application content is displayed on the GUI to facilitate shopping on the tv, as described with reference to fig. 19B.

In some embodiments, the device detects (at 6345) that the user selects the "back" command in the jump prompt tab, and the GUI exits (at 6350) the screenshot content display area 170 to continue playing the program for the current silo. In some instances, the user may exit from the video menu bar list window 501 to the previous menu bar or exit the screenshot content display area 170 to browse the video content currently playing in the current content play display area 160 in full screen.

The exemplary systems and methods of the present application have been described in relation to an entertainment system. However, to avoid unnecessarily obscuring the present application, the foregoing description omits some known structures and devices. Such omissions are not to be construed as limiting the scope of the claims. Specific details are provided herein to facilitate an understanding of the present application. However, it should be understood that the present application may be embodied in many ways beyond the specific details detailed in the present application.

Moreover, while the exemplary aspects, examples, and/or configurations illustrated herein show various components of the system arranged together, some of the system components may be located remotely from a distributed network (e.g., a LAN and/or the Internet), or in a dedicated system. It should therefore be appreciated that the components of the system may be combined into one or more devices, such as a set-top box or television set, or other devices collocated on a particular node of a distributed network, such as an analog and/or digital telecommunications network, a packet-switched network, or a circuit-switched network. In light of the above description and for reasons of computational efficiency, the system components may be located anywhere within the distributed component network without affecting the operation of the system. For example, the various components may be located in a switch (e.g., a PBX and media server, a gateway), one or more communication devices, one or more user sites, or some combination thereof. Similarly, one or more functional portions of the system may be distributed between a telecommunications device and an associated computing device.

Further, it should be understood that the various connections between these elements may be wired or wireless connections, or any combination thereof, or any other known or later developed element capable of providing data and/or communicating data with the connected elements. These wired or wireless connections may also be secure connections capable of transmitting encrypted information. The transmission medium used for connection may be, for example, any suitable electronic signal carrier including coaxial cables, copper wire and fiber optics, and may be acoustic or light waves such as those generated during radio-wave and infra-red data transmissions.

Further, while some flow diagrams have been discussed and illustrated in a particular order of events, it should be understood that such order may be altered, augmented, and omitted without materially affecting the operation of the disclosed examples, configurations, and aspects.

A number of variations and modifications of the present application may be employed. It is possible to provide only some of the features of the present application without providing the remaining features.

In another example, the systems and methods of the present application, when implemented, may be used in conjunction with a special purpose computer, a programmed microprocessor or microcontroller and peripheral integrated circuit elements, an ASIC or other integrated circuit, a digital signal processor, a hardwired electronic or logic circuit (e.g., a discrete element circuit), a programmable logic device or gate array (e.g., PLD, PLA, FPGA, PAL), a special purpose computer, any comparable means, or the like. In general, any device or means capable of carrying out the methods described herein can be used to carry out the various aspects of the present disclosure. Exemplary hardware suitable for use with the disclosed examples, configurations, and aspects includes computers, handheld devices, telephones (e.g., cell phones, internet-enabled, digital, analog, hybrid, and others), and other hardware known in the art. Some such devices include a processor (e.g., one or more microprocessors), memory, non-volatile memory, input devices, output devices, and so forth. Moreover, the methods described herein may also be implemented using other software-implemented processes, including but not limited to distributed processing or component/object distributed processing, parallel processing, or virtual machine processing.

In another example, the disclosed methods may also be readily implemented in connection with software using object or object-oriented software development environments, as these environments may provide convenient source code that may be used on a variety of computer or workstation platforms. Alternatively, the disclosed system may be implemented partially or fully in hardware using standard logic circuits or VLSI devices. Whether software or hardware is used in implementing a system according to the present application depends on the speed and/or efficiency requirements of the system, the particular function and particular software or hardware system, or microprocessor or microcomputer system being used.

In another example, the disclosed methods may be implemented in part in software that may be stored on a storage medium, executed on a programmed general purpose computer, special purpose computer, microprocessor, equipped with a controller and memory. In these examples, the systems and methods herein may be implemented as programs embedded in a personal computer (e.g., an applet, or a CGI script), resources maintained on a server or computer workstation, routines embedded in a dedicated measurement system and system components, and so forth. Further, the system may also be implemented by physically integrating the system and/or method into a software and/or hardware system.

Although the present application describes components and functions implemented in certain aspects, examples, and/or configurations in accordance with particular standards and protocols, it is not intended that such aspects, examples, and/or configurations be limited by such standards and protocols. Other similar standards and protocols not mentioned in this application are also present and are considered to be included in this application. Moreover, the standards and protocols mentioned in this application and other similar standards and protocols not mentioned in this application will be periodically replaced by the fastest and more efficient equivalents which have essentially the same function. Such replacement standards and protocols having the same functions are considered equivalents to be included in this application.

The subject application includes the components, methods, processes, systems and/or apparatus described in detail herein in various aspects, examples, and/or configurations, including various aspects, examples, configuration examples, subcombinations, and/or subsets thereof. Those of skill in the art will understand how to make and use the disclosed aspects, examples, and/or configurations after understanding the present application. The subject application, in various aspects, examples, and/or configurations, includes providing devices and processes that improve performance, ease of implementation, reduce implementation costs, and/or the like, in the absence of items not depicted and/or described in this application or in various aspects, examples, and/or configurations, or in the absence of such items as may have been used in previous devices or processes.

The foregoing discussion is presented for purposes of illustration and description and is not intended to limit the application to the form or forms disclosed. In the foregoing detailed description, for example, various features of the application are grouped together in one or more aspects, examples, and/or configurations for the purpose of streamlining the disclosure. Various features of the various aspects, examples, and/or configurations of the application may be combined in other aspects, examples, and/or configurations than those described above. This method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in all features of a single foregoing disclosed aspect, example, and/or configuration. Thus, the following claims are hereby incorporated into this detailed description, with each claim standing on its own as a separate preferred example of the application.

Moreover, although the foregoing description has included description of one or more aspects, examples, and/or configurations and certain variations and modifications, other variations, combinations, and modifications are within the scope of the disclosure, as determined by the skilled artisan after understanding the disclosure. Applicants' intent is to obtain rights to include alternative aspects, examples, and/or configurations to the extent permitted, including alternate, interchangeable and/or equivalent structures, functions, ranges or steps to those included in the claims, whether or not such alternate, interchangeable and/or equivalent structures, functions, ranges or steps are disclosed in the application, and without intending to publicly dedicate any patentable subject matter.

Claims

1. A method for displaying a graphical user interface of a screenshot of a television picture,

when a video picture is displayed on a display screen, receiving an instruction for triggering screenshot;

responding to the screenshot operation instruction, and displaying a graphical user interface on the display screen, wherein the graphical user interface comprises a plurality of option bars, and at least one option bar is used for displaying a picture obtained by successfully capturing the screenshot; and the number of the first and second groups,

acquiring a keyword related to an identification object in a screenshot picture after the screenshot picture is subjected to image identification, displaying the keyword in at least one option bar in a graphical user interface, displaying a recommended text and/or the identification object picture related to the identification object of the screenshot picture in the at least one option bar, or displaying a user control instruction input interface related to the functional operation related to the identification object of the screenshot picture in the at least one option bar;

activating a service or application associated with content in a selected one of the plurality of tabs in response to a user input based on the user input selecting the content;

and responding to the selection of the user to display any option bar of the identification object picture or the key word or the recommended text, so as to update the detail content which is displayed in the window and is associated with the identification object picture or the key word or the recommended text in the selected option bar.

2. A method for displaying a graphical user interface of a screenshot of a television picture,

responding to the screenshot operation instruction, and displaying a graphical user interface on the display screen, wherein the graphical user interface comprises a plurality of option bars and at least one option bar is used for displaying a plurality of screenshot pictures obtained by screenshot success; and the number of the first and second groups,

acquiring a keyword related to an identification object in any screenshot picture obtained by image identification of the screenshot picture, displaying the keyword in at least one option bar in a graphical user interface, displaying a recommended text and/or the identification object picture related to the identification object of the screenshot picture in the at least one option bar, or displaying a user control instruction input interface related to the functional operation related to the identification object of the screenshot picture in the at least one option bar;

in response to selection of any one of the plurality of screenshot pictures, an object-related keyword is derived based on image recognition of the selected screenshot picture, and content related to an object of the selected screenshot picture is displayed based on search results by re-searching based on the keyword.

3. A method for displaying a graphical user interface of a television screen shot, comprising: the display method of any of claims 1-2, at least one of the plurality of tabs further comprising: the display content comprises identification of a search source, and/or brief description of the recommended text and/or the identification object picture, and/or subject or key words of the search result.

4. A method for displaying a graphical user interface of a screenshot of a television picture,

responding to the screenshot operation instruction, and displaying a graphical user interface on the display screen, wherein the graphical user interface comprises a plurality of option bars, and at least one option bar is used for displaying a plurality of screenshot pictures obtained by current continuous screenshot;

responding to selection of any one of the plurality of displayed screenshot pictures, acquiring a keyword related to an identification object in the screenshot picture obtained by image identification of the selected screenshot picture, displaying the keyword in at least one option bar in a graphical user interface, displaying a recommended text related to the identification object of the screenshot picture and/or the identification object picture in the at least one option bar, or displaying a user control instruction input interface related to the identification object of the screenshot picture in the at least one option bar;

and based on user input for selecting any content in a plurality of option bars, responding to the user input, and activating a service or an application program related to the content in the selected option bar, wherein the content in the plurality of option bars comprises a recommended text content brief description and/or a recognition object picture.

5. A method for displaying a graphical user interface of a screenshot of a television picture,

in response to selection of any one of the plurality of presented screenshot pictures, obtaining a keyword related to an identification object in the screenshot picture obtained by performing image identification on the selected screenshot picture, displaying the keyword in at least one option bar in a graphical user interface, displaying a recommended text and/or picture related to the identification object of the screenshot picture in the at least one option bar, or displaying a user control instruction input interface related to functional operation related to the identification object of the screenshot picture in the at least one option bar;

in response to the user selecting any one of the plurality of screenshot picture identification objects or keywords, updating and displaying one or more other in-option content associated with the identification objects or keywords for displaying the detailed content associated with the keywords in the selected option bar.

6. A method for displaying a graphical user interface of a television screen shot, comprising: the display method according to any one of claims 4 to 5, wherein the content displayed in the plurality of option bars comprises a website logo bar indicating a content source, a related keyword bar, a detailed vocabulary bar, and/or an album bar of related picture recommendations.

7. An intelligent television, comprising:

a display screen;

a memory;

and a processor in communication with the memory and the display screen, the processor performing: a method of displaying a graphical user interface comprising a screenshot of a television picture as claimed in any one of claims 1 to 6.