JP5461346B2 - Mobile terminal and remind method - Google Patents

Description

  The present invention relates to a mobile terminal and a remind method.

  With the advancement and popularization of mobile terminals, many mobile terminals can not only play moving pictures but also register various scenes of moving pictures as “favorites”. For example, if the user is interested in a beautiful scenery, actor, product, etc. that appeared in the scene, the user can register the scene as a “favorite”. A conventional technique for associating a specific scene with a “favorite” is disclosed in Patent Document 1.

  After viewing and saving a scene, the user remembers that he has registered the scene and gets useful information for the user, for example by looking at places where the view can be seen, actor profiles, new product details, etc. Can do.

JP 2009-182875 A

  However, as the number of scenes registered as “favorites” increases, it becomes difficult for the user to properly remember all of them. For example, assume that a user registers a scene with interest in a store that appears in a scene. Even if the user accidentally walks near the store at a later date, if the user cannot remember that he / she has registered, the user cannot notice the store. Furthermore, even if the user remembers that the scene has been registered, if the user does not know that the store is nearby, the user will pass through the neighborhood of the store. As a result, the user regrets, but there is a problem in that the registered information is not effectively used.

  An object of the present invention is to allow a user to remember that a scene displayed on a mobile terminal is stored as a “favorite” at an appropriate timing.

A mobile terminal according to an aspect of the present invention provides:
A mobile terminal capable of communicating through a network,
A display unit;
A storage unit for storing scene information including scene images and metadata displayed by the display unit;
Using the location information indicating the location of the mobile terminal, the peripheral information acquisition unit that acquires the peripheral information around the location from the information server via the network;
A priority determining unit that determines the priority of the scene indicated by the scene information by calculating the similarity between the word vector derived from the stored scene information and the word vector derived from the peripheral information; The display unit is a mobile terminal that displays an image of a scene having a relatively high priority.

  According to an aspect of the present invention, the user can be reminded at an appropriate timing that the scene displayed on the mobile terminal is stored as “favorite”.

The functional block diagram of the mobile terminal used in an Example. The flowchart which shows the operation example which registers a favorite scene. The figure which shows one scene of the moving image during reproduction | regeneration. The figure which shows a mode that the image and metadata are displayed. The figure which shows an example of a characteristic dictionary. The figure which shows an example of a word vector. The flowchart which shows the operation example which reminds the preserve | saved favorite scene. The figure explaining the calculation method of similarity. The figure which shows a mode that a favorite scene is reminded.

  Examples will be described from the following viewpoints.

1. 1. Mobile terminal 2. Registration of favorite scenes Remind your favorite scenes 4. Modified example

<1. Mobile terminal>
FIG. 1 shows a functional block diagram of a mobile terminal used in the embodiment. FIG. 1 shows functional elements particularly related to the present embodiment among various functions provided in a mobile terminal. The mobile terminal may be any user device that can communicate with various devices (particularly, a server that provides location information and peripheral information) via a network such as a public telephone network or the Internet. The mobile terminal is typically a mobile phone, but may be an information terminal, a smartphone, a personal digital assistant, a portable personal computer, or the like.

  The mobile terminal includes a video playback unit 22, a favorite scene storage unit 23, a scene information storage database 24, a scene information acquisition unit 25, a current position information acquisition unit 26, a peripheral information acquisition unit 27, a peripheral information storage database 28, a favorite scene priority A determination unit 29, a favorite scene recommendation reason generation unit 30, and a favorite scene rearrangement unit 31 are included.

  The moving image reproducing unit 22 can display still images and other information in addition to reproducing moving images. Therefore, although not shown, the moving image reproduction unit 22 includes at least a visual display unit and a loudspeaker unit (speaker) for reproducing auditory sounds.

  The favorite scene storage unit 23 stores a specific scene designated by the user as “favorite”. A specific procedure for registering a favorite scene will be described with reference to FIG. The scene is saved by saving the scene information in the scene information storage database 24. The scene information includes scene images and metadata. The image may be a moving image or a still image, but is typically a still image (part of a moving image). Metadata includes descriptive text about places, people, things, music, events, etc. that appear in the scene. The location is specified by the latitude and longitude information. Things are indicated by information about stores, products, vehicles, etc. appearing in the scene. Music is indicated by the author or name of the music or melody that appeared in the scene. The event is indicated by the place, date, etc., where the event took place. These specific examples of metadata are merely exemplary and may include any appropriate information known to those skilled in the art.

  The scene information storage database 24 stores the scene information acquired by the scene information acquisition unit 24.

  The scene information acquisition unit 25 acquires scene information from the scene stored in the favorite scene storage unit 23 and stores it in the scene information storage database 24. Further, the scene information acquisition unit 25 acquires scene information from the scene information storage unit database 24 in order to determine a priority described later.

  The current position information acquisition unit 26 acquires or calculates position information indicating the current position of the mobile terminal. The acquisition of the position information may be performed by any appropriate method known to those skilled in the art. For example, when the mobile terminal includes a GPS receiver (not shown), the position information of the mobile terminal may be calculated based on a signal received by the GPS receiver from a GPS satellite. Alternatively, the mobile terminal may calculate the position information by performing surveying using signals received from a plurality of fixed points (typically, base stations). In addition, the mobile terminal may acquire location information by using some information providing service. For example, the position information may be acquired from a position information providing server via a network. The position information is typically expressed by latitude and longitude, but may be expressed by other methods. For example, the position information may include coordinates in the height direction in addition to the coordinates in the horizontal plane.

  The peripheral information acquisition unit 27 uses the current position information of the mobile terminal to acquire peripheral information around the position. For example, the mobile terminal may acquire the peripheral information by accessing the peripheral information providing server via the network together with the current position information. The location information providing server and the peripheral information providing server may be separate network elements or the same network element. Peripheral information includes, for example, buildings, stores, facilities, companies, schools, parks, roads, rivers, ports, and other map information (including location information) that exist around the specified location, as well as their explanatory text. including. The mobile terminal provides not only the current position information but also some filtering information to the peripheral information providing server, and only the peripheral information that the user is particularly interested in is provided to the mobile terminal. You may do it. For example, information on Japanese restaurants that exist around the current location may be provided to the mobile terminal by filtering information that specifies only Japanese restaurants.

  The peripheral information storage database 28 stores the peripheral information acquired by the peripheral information acquisition unit 27.

  The favorite scene priority determination unit 29 determines the priority for each scene stored in the scene information storage unit database 24 by the method described later. In general, if a scene has a high priority, that scene is preferentially reminded to the current user. For example, the reminder is performed by displaying a thumbnail of an image of a scene together with some explanatory text (specifically, a reason for recommendation). Conversely, if the priority of a scene is low, there is little need to remind that scene to the current user.

  The favorite scene recommendation reason generation unit 30 generates information indicating a recommendation reason to be displayed together with an image when an image of the scene is presented to the user according to the priority determined by a method described later. As information indicating the reason for recommendation, information specified when the user registers a scene can be used. For example, it is assumed that the user is interested in a park that appears in a scene and registers the scene as a “favorite”. It is assumed that the user specifies that he / she is interested in a place at the time of registration (save) and saves it in the scene information storage unit database 24. For example, it is possible to specify that the user is interested in a place by pressing a “place” button among buttons indicating “place”, “product”, and “person” on the touch panel of the display unit. In this case, the favorite scene recommendation reason generation unit 30 displays that the user is interested in the location as the recommendation reason accompanying the scene image presented to the user. As an example, the word “location” is displayed next to the thumbnail of the image.

  The favorite scene rearrangement unit 31 determines the order in which the scenes are displayed according to the priority determined by the method described later.

<2. Register favorite scenes>
Hereinafter, an operation example of this embodiment will be described. The described operation is performed at the mobile terminal. The operation example will be described separately for an operation for registering a favorite scene and an operation for reminding a registered favorite scene.

  FIG. 2 shows an operation example for registering a favorite scene. In step 202, the user is playing a video on the mobile terminal. The embodiment is not limited to moving images, and still images may be displayed.

  In step 204, the user finds a favorite scene during moving image reproduction and saves the scene. FIG. 3 is a diagram showing one scene of a moving image. FIG. 4 shows various information (metadata) displayed together with the scene image shown in FIG. As shown in FIGS. 3 and 4, the scene is represented by an image 41 and metadata 42-47, and the user is not only the image 41 but also location information 42, person information 43, thing information 44, and other information 45. , Title 46 and URL 47 can be confirmed. The image 41 shows a moving image or a still image. The place information 42 or the information of the URL 47 is information included in the metadata of the scene. The place information 42 is information about places that appear in the scene. The person information 43 is information about a person appearing in the scene. The thing information 44 is information about things and events that appear in the scene. For example, information on the vehicle shown in the image is included in the thing information 44. The other information 45 is information about places other than those appearing in the scene, persons, and things, and includes, for example, information on the date and time when the moving image was broadcast on television, information about music that flowed in the scene, and the like. The title 46 indicates a title accompanying the scene. A URL (Universal Resource Locator) 47 indicates a site that distributes moving images. The example of FIG. 3 shows a state where the location information 42 is displayed on the right side together with the image 41.

  In principle, the metadata is prepared for each scene of the moving image, and as an example, the metadata is updated at a certain period when the moving image is reproduced (for example, every moving image segment for 5 minutes).

  When the user presses the save button 48 for a favorite scene, the scene is saved. From the viewpoint of reducing the time difference between when the favorite scene is displayed and when the save button 48 is pressed, the scene immediately before the user presses the save button 48 may be saved. Good.

  In step 206 of FIG. 2, scene information (scene images and metadata) is obtained from the stored scene. The acquired scene information is stored in the scene information storage database 24.

  In step 208, the user specifies a remind condition. As will be described later, the registered scene is later reminded to the user. Reminding is typically performed by displaying a thumbnail of an image of a saved scene and an accompanying description on the display unit. The accompanying description corresponds to the above “reason for recommendation”. For example, if the user is interested in a park that appears in a scene, when registering the scene, the “location” among the buttons indicating “location”, “product”, “person” on the touch panel of the display unit Specify that you are interested in a place by pressing the button. In this case, the recommendation reason is “place”. As an example, when reminding, the word “location” is displayed next to the thumbnail of the image.

  Note that step 208 is not an essential procedure in the present embodiment, and may be omitted.

  In step 210, position information is generated from the scene information of the registered scene, and the scene information is converted into a word vector. As described above, the scene information includes information such as location information 42, person information 43, and thing information 44. In this embodiment, location information 42 is included in any scene information. That is, in the metadata of the scene information, the location information is essential, but the person information 43, the thing information 44, and the other information 45 are optional. As an example, the position information is expressed by latitude and longitude. In the case of the example in FIG. 3, the latitude and longitude information of the park appearing in the scene is generated as the position information.

  The metadata of scene information is converted into a word vector by some information extraction algorithm. A word vector is also referred to as a feature vector. The information extraction algorithm is, for example, a morphological analysis algorithm. By performing morphological analysis on the metadata, the description of the metadata is classified for each part of speech. Part of speech generally includes general nouns, proper nouns, verbs, particles, and the like. As a result of the morphological analysis, general nouns and proper nouns are extracted from the scene metadata. By determining whether or not each extracted noun is associated with a predetermined word (feature element) in the feature dictionary, a word vector of the scene is generated.

  FIG. 5 shows an example of a feature dictionary. “Features” represents words that are elements or components of a word vector. “Matching target attribute” indicates which attribute of the metadata is to be checked when the metadata of the scene is associated with the feature element. Generally, matching target attributes include “Category” and “Description”. In the case of “category”, it is determined whether or not the metadata has a feature element based on whether or not words representing the category match instead of individual words in the explanatory text. In the case of “description”, it is determined whether each word in the description has a feature element. “Matching information” indicates individual words to be collated. One or more matching information exists for one feature element. The matching information is generally expressed by a keyword and a weight attached to the keyword. Since the number of characters in the description of the category is small, in the example shown in the figure, all weights are unified to 1 when comparing categories. On the other hand, in the case of comparison of explanatory texts, weights are set according to expressions of various words. As a result of the morphological analysis, for example, when the noun “park” is included in the description of the metadata, the word vector of the metadata of this scene has a characteristic element “park”.

  FIG. 6 shows an example of a word vector. One scene information is distinguished by one scene identifier (scene ID). In the illustrated example, word vectors for six scenes are shown. For each scene ID of S1-S6, it is shown whether or not a predetermined feature element is included. 1 indicates that the feature element is included. 0 indicates that the feature element is not included. For example, the word vector of the scene ID-S4 includes four characteristic elements “news”, “variety”, “drama”, and “park”.

  In step 210, position information and word vectors are derived from the stored scene metadata, and word vectors and position information are stored for each of the stored specific scenes.

<3. Remind your favorite scenes>
FIG. 7 shows an operation example for reminding a saved favorite scene. As described with reference to FIG. 2, the user stores several scenes as favorites. For each specific scene stored, word vectors and position information are stored.

  In step 702, the mobile terminal obtains peripheral information based on the current location information. The current location information may be calculated from information received by a GPS receiver mounted on a mobile terminal, may be obtained by accessing a location information providing server via a network, You may obtain | require by the method. The mobile terminal may acquire the peripheral information by accessing the peripheral information providing server via the network together with the current position information. The location information providing server and the peripheral information providing server may be separate network elements or the same network element. Peripheral information includes, for example, buildings, stores, facilities, companies, schools, parks, roads, rivers, ports, and other map information (including location information) that exist around the specified location, as well as their explanatory text. including. The mobile terminal provides not only the current position information but also some filtering information to the peripheral information providing server, and only the peripheral information that the user is particularly interested in is provided to the mobile terminal. You may do it.

  In step 704, the mobile terminal generates a word vector from the acquired peripheral information. The word vector generation method itself is the same as the method described in step 210 of FIG. First, explanatory text representing peripheral information is classified for each part of speech by morphological analysis, and general nouns and proper nouns are extracted. By determining whether or not these nouns correspond to predetermined words in the feature dictionary (FIG. 5), a word vector of peripheral information is generated.

  In step 706, the similarity between the word vector of the scene information and the word vector of the peripheral information is calculated. The similarity may be calculated by any appropriate method known to those skilled in the art. As an example, similarity between word vectors may be derived based on an angle formed by the vectors, as shown in the following equation.

In this case, sim (Xs, Xi) indicates the similarity between the word vector Xs of the scene information and the word vector Xs of the peripheral information. This similarity is also referred to as cosine similarity. Xs · Xi represents the inner product of the word vectors Xs and Xi. | Xs | indicates the size (square norm) of the word vector Xs. | Xi | indicates the size (square norm) of the word vector Xi.

FIG. 8 shows how the similarity is calculated. For simplification, it is assumed that the word vector of the scene IDS1-S6 is expressed by the four components in the broken line frame portion on the left side of FIG. Further, it is assumed that the word vector Xi of the peripheral information is (sports, news, variety, music) = (0.4, 0.7, 0.2, 0.7). In this case, the similarity sim (Xs1, Xi) between the scene ID-S1 and the word vector of the peripheral information is (0.11) / √2√ (0.4 ² +0.7 ² +0.2 ² +0.7 ² ) ≒ 0.72. Similarly, similarities can be calculated for other scenes, and the following calculation results are obtained.

Similarity sim (Xs1, Xi) = 0.72 between the scene ID-S1 and the word vector of the peripheral information
Similarity sim (Xs2, Xi) = 0.85 between scene ID-S2 and word vector of peripheral information
Similarity sim (Xs3, Xi) = 0.91 between the scene ID-S3 and the word vector of the peripheral information
Similarity sim (Xs4, Xi) = 0.59 between the scene ID-S4 and the word vector of the peripheral information
Similarity sim (Xs5, Xi) = 0.72 between the scene ID-S5 and the word vector of the peripheral information
Similarity sim (Xs6, Xi) = 0.72 between the scene ID-S6 and the word vector of the peripheral information.

  In this way, in step 706 of FIG. 7, the similarity between each word vector Xs of the saved scene and the word vector Xi of the peripheral information is calculated.

  In step 708 of FIG. 7, the mobile terminal determines the priority of the stored scene based on the calculated similarity. The priority may be determined based only on the similarity, or may be determined in consideration of a criterion different from the similarity, as will be described in the modification. For convenience of explanation, it is assumed that the priority is determined from the similarity. In other words, if the word vectors of the scene information and the peripheral information are similar, the priority of the scene is high, and if the word vectors are not similar, the priority of the scene is low.

  In step 710, the mobile terminal reminds the user that the scene has been registered by displaying the stored scene according to the priority. Reminding is performed in order from the scene with the highest priority.

  FIG. 9 shows how a favorite scene is reminded. For example, it is assumed that the user is interested in an XXX park that appears in a scene of a moving image and registers the scene as a scene ID-Sx. Suppose the user is in the vicinity of the XXX park for another errand later. In this case, the similarity between the word vector of the scene ID-Sx and the word vector of the peripheral information at the current position of the user is high. The mobile terminal displays a thumbnail of the image in order to display that a scene with a high similarity is registered. As a result, the user can remember that he / she is interested in the location of the scene, and by clicking on the image, the details of the location are presented to the user together with the map information.

  It should be noted that scenes that are reminded to the user are limited to those whose priority or similarity is a certain value or more. This is because even if the user is reminded in a situation that is not closely related to the registered scene, the significance of reminding is weak.

  Furthermore, it is possible for the user to specify conditions for reminding. For example, when a scene is registered, the user registers any one of “place”, “person”, and “thing” along with the scene. It is assumed that the user has set a mobile terminal that desires a reminder about a place and does not strongly desire a reminder about others. In this case, the mobile terminal extracts only the saved scenes for the “location” from the saved scenes, determines each priority level, and performs reminding. Thereby, it becomes possible to narrow down to what the user desires and to perform a reminder.

<4. Modification>
As described above, the priority of a scene may be determined by taking into account not only the similarity between word vectors but also other information. For example, the distance between the location indicated in the scene information and the current position may be taken into account. This is because the closer the distance is, the more suitable the situation should be for reminding the scene. For example, a first value (d1 = 1, 2,..., 5) that expresses the similarity between the word vectors of the scene information and the peripheral information in five levels, and a value that expresses the proximity of the distance in five levels ( The priority may be determined by a total point (D) obtained by adding d2 = 1, 2,..., 5). If they are most similar, d1 = 5, and if they are closest, d2 = 5 and 2 ≦ D ≦ 10.

D = d1 + d2
Alternatively, the first value (d1) and the second value (d2) may be weighted and added.

D = αd1 + βd2
Here, α and β are predetermined constants. In determining the priority, not only the distance is taken into consideration in addition to the similarity, but also other information (for example, time) may be further taken into consideration.

  Although the present invention has been described with reference to particular embodiments, they are merely exemplary and those skilled in the art will appreciate various variations, modifications, alternatives, substitutions, and the like. Although specific numerical examples have been described in order to facilitate understanding of the invention, these numerical values are merely examples and any appropriate values may be used unless otherwise specified. Although specific mathematical formulas have been described to facilitate understanding of the invention, these mathematical formulas are merely examples, unless otherwise specified, and any appropriate mathematical formula may be used. The classification of the examples or items is not essential to the present invention, and the items described in two or more items may be used in combination as necessary, and the items described in one item may be combined with other items. It may be applied (as long as it is not inconsistent) to the matters described. For convenience of explanation, an apparatus according to an embodiment of the present invention has been described using a functional block diagram, but such an apparatus may be realized by hardware, software, or a combination thereof. Software can be stored on random access memory (RAM), flash memory, read only memory (ROM), EPROM, EEPROM, registers, hard disk (HDD), removable disk, CD-ROM, database, server or any other suitable storage medium May be. The present invention is not limited to the above embodiments, and various modifications, modifications, alternatives, substitutions, and the like are included in the present invention without departing from the spirit of the present invention.

22 Movie playback section
23 Favorite scene storage
24 Scene information storage database
25 Scene information acquisition unit
26 Current location information acquisition unit
27 Peripheral information acquisition department
28 Peripheral information storage database
29 Favorite Scene Priority Determination Unit
30 Favorite Scene Recommendation Reason Generator
31 Favorite Scene Sorting Section
41 images
42 Location information
43 Personal information
44 Things Information
45 Other information
46 Title
47 URL

Claims (12)

A mobile terminal capable of communicating through a network,
A display unit;
A storage unit for storing scene information including scene images and metadata displayed by the display unit;
Using the location information indicating the location of the mobile terminal, the peripheral information acquisition unit that acquires the peripheral information around the location from the information server via the network;
A priority determining unit that determines the priority of the scene indicated by the scene information by calculating the similarity between the word vector derived from the stored scene information and the word vector derived from the peripheral information; And the display unit displays an image of a scene having a relatively high priority.   The mobile terminal according to claim 1, wherein the metadata includes a description of a place, person, thing, music, or event that appears in a scene.   The mobile terminal according to claim 1 or 2, wherein the word vector derived from scene information is derived by determining whether or not a predetermined word is included in metadata of scene information.   The movement according to any one of claims 1 to 3, wherein the word vector derived from the peripheral information is derived by determining whether or not a predetermined word is included in the peripheral information. Terminal.   The priority determination unit further calculates a distance between the position of the mobile terminal and the position indicated by the scene information, and determines the priority based on the similarity and the distance. The mobile terminal according to any one of claims.   The mobile terminal according to claim 1, wherein the similarity is derived from an inner product between word vectors and a size of each word vector. A reminding method in a mobile terminal capable of communicating through a network,
Storing scene information including images and metadata of the scene displayed by the display unit of the mobile terminal;
Using the location information indicating the location of the mobile terminal, obtaining peripheral information around the location from the information server via the network,
Determining the priority of the scene indicated by the scene information by calculating the similarity between the word vector derived from the stored scene information and the word vector derived from the peripheral information, The reminding method in a mobile terminal, wherein the display unit displays an image of a scene having a relatively high priority.   The reminding method according to claim 7, wherein the metadata includes a description of a place, person, thing, music, or event appearing in a scene.   The reminding method according to claim 7 or 8, wherein the word vector derived from scene information is derived by determining whether or not a predetermined word is included in metadata of scene information.   The reminder according to any one of claims 7 to 9, wherein the word vector derived from the surrounding information is derived by determining whether or not a predetermined word is included in the surrounding information. Method.   11. When determining the priority, the distance between the position of the mobile terminal and the position indicated by the scene information is further calculated, and the priority is determined based on the similarity and the distance. The remind method of any one of these.   The reminding method according to any one of claims 7 to 11, wherein the similarity is derived from an inner product of word vectors and a size of each word vector.