JP2022051367A - Information processing system and information processing method - Google Patents

Abstract

To efficiently learn a model that generates vectors by using data possessed by a plurality of terminal devices.SOLUTION: An information processing system includes: a server that provides a common model that generates a vector indicating a user's context from data acquired by a terminal device; and a plurality of terminal devices that each generate a local model obtained by updating parameters of the common model received from the server by using data possessed by each terminal device, and transmit update information which is information about the local model to the server. The server uses the update information received from the plurality of terminal devices to update the parameters of the common model, and provides an updated common model to the plurality of terminal devices.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing system and an information processing method.

A technique for generating information indicating the context of a user or the like is disclosed. For example, as an example of the context, a technique for automatically generating information representing a user's situation is provided by a machine learning model (also simply referred to as a “model”) (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2019-22009

However, it cannot be said that the above-mentioned conventional technique can efficiently learn a model that generates information indicating the context. For example, in the above-mentioned prior art, the case of targeting a plurality of users is not considered. Therefore, in the case of learning the model using the data possessed by the plurality of terminal devices used by each of the plurality of users, it is not always possible to learn the model efficiently.

The present application has been made in view of the above, and an object thereof is to efficiently learn a model for generating a vector by using data possessed by a plurality of terminal devices.

The information processing system according to the present application is a server that provides a common model that generates a vector indicating a user's context from data acquired by a terminal device, and parameters of the common model received from the server using the data possessed by each. The server has a plurality of terminal devices that generate an updated local model and transmit update information, which is information about the local model, to the server, and the server receives the update information from the plurality of terminal devices. The information is used to update the parameters of the common model, and the updated common model is provided to the plurality of terminal devices.

According to one aspect of the embodiment, the model for generating the vector can be efficiently learned by using the data possessed by the plurality of terminal devices.

FIG. 1 is an explanatory diagram showing an outline of information processing according to an embodiment. FIG. 2 is a diagram showing a configuration example of an information processing system according to an embodiment. FIG. 3 is a diagram showing a configuration example of the terminal device according to the embodiment. FIG. 4 is a diagram showing an example of a model information storage unit. FIG. 5 is a diagram showing a configuration example of the server device according to the embodiment. FIG. 6 is a diagram showing an example of a model information storage unit. FIG. 7 is a diagram showing an example of a community information storage unit. FIG. 8 is a diagram showing an example of a model. FIG. 9 is a diagram showing an example of a case where a part of the data is missing. FIG. 10 is a diagram showing an example of a community having a plurality of layers. FIG. 11 is a flowchart showing a processing procedure according to the embodiment. FIG. 12 is a diagram showing an example of a hardware configuration.

Hereinafter, an embodiment for implementing the information processing system and the information processing method according to the present application (hereinafter, referred to as “embodiment”) will be described in detail with reference to the drawings. The information processing system and the information processing method according to the present application are not limited by this embodiment. Further, in the following embodiments, the same parts are designated by the same reference numerals, and duplicate description is omitted.

[1. Information processing overview]
First, with reference to FIG. 1, an outline of information processing performed by the information processing system 1 according to the embodiment will be described. FIG. 1 is an explanatory diagram showing an outline of information processing according to an embodiment. FIG. 1 shows an example of learning a model by a server device 100 and a plurality of terminal devices 10. In the following, the model provided by the server device 100 to the terminal device 10 may be described as a common model or a global model, and the model in which each terminal device 10 updates the parameters of the global model may be described as a local model or a local model.

The model generated by the information processing system 1 may have any configuration as long as the learning process described later can be applied. That is, any network configuration such as SVM (Support Vector Machine) or DNN (Deep Neural Network) can be adopted as the network configuration of the model. For example, the network configuration of the global model may be CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long short-term memory), or the like. Further, the network configuration of the global model may be a configuration in which a plurality of networks are combined, such as a combination of CNN and RNN. The RNN and LSTM may be a neural network based on an attention mechanism. A specific example of the network configuration will be described later.

In FIG. 1, in order to simplify the explanation, a model M1 is trained in which the user's position information is input as data indicating the user's position and an embedding vector (embedded vector) indicating the user's context is output (generated). The case of doing is shown. In the following, the embedding vector may be simply referred to as a vector. That is, the model M1 converts (vectorizes) the user's position information into a vector indicating the user's context. The details of the context will be described later, but the context means the situation of the subject to which the vector is generated. For example, in the case of the model M1, it means the situation of the user who uses the terminal device 10.

In the following, the user identified by the user ID "U1" may be referred to as "user U1". As described above, in the following, when "user U * (* is an arbitrary numerical value)" is described, it means that the user is a user identified by the user ID "U *". For example, when "user U2" is described, the user is a user identified by the user ID "U2". Further, in the example shown in FIG. 1, the terminal device 10 will be described as the terminal devices 10-1 to 10-3 according to the user who uses the terminal device 10. For example, the terminal device 10-1 is a terminal device 10 used by a user (user U1) identified by the user ID "U1". Further, for example, the terminal device 10-2 is a terminal device 10 used by being identified by the user ID "U2" (user U2). Further, in the following, when the terminal devices 10-1 to 10-3 will be described without particular distinction, they will be referred to as the terminal device 10. Further, when the users such as users U1 to U3 are described without particular distinction, they may be described as user U.

First, the server device 100 provides the terminal device 10-1 used by the user U1 with the model M1 which is a global model (step S11-1). Further, the server device 100 provides the model M1 to the terminal device 10-2 used by the user U2 (step S11-2). Further, the server device 100 provides the model M1 to the terminal device 10-3 used by the user U3 (step S11-3). The order of steps S11-1 to S11-3 may be any order, and step S11-3 may be performed before S11-1. Hereinafter, when steps S11-1 to S11-3 are described without distinction, they are collectively referred to as step S11. Although three users U1 to U3 are illustrated in FIG. 1, the server device 100 is not limited to the users U1 to U3, and a large number of users such as users U4 and U5 (for example, 1 million users and 1000 users). The model M1 may be provided to the terminal device 10 of (10,000 users, etc.). For example, the model M1 may be provided to the terminal devices 10 of all users to which the model M1 is applied.

In step S11, when the server device 100 does not provide the terminal device 10 with the model M1 which is a global model, the network configuration of the model M1 and model parameters such as weights and biases of the model M1 (also simply referred to as "parameters"). ) To the terminal device 10. Further, in step S11, when the model M1 has already been provided to the terminal device 10, the server device 100 transmits parameters such as weights and biases of the model M1 to the terminal device 10. As a result, the information processing system 1 can suppress an increase in the amount of data transmitted by the server device 100 to the terminal device 10 and suppress an increase in the amount of communication.

The terminal device 10-1 that has received the model M1 from the server device 100 updates the parameters of the model M1 and generates the model M1a, which is a local model in which the parameters of the model M1 are updated (step S12-1). The terminal device 10-2 that has received the model M1 from the server device 100 updates the parameters of the model M1 and generates the model M1b, which is a local model in which the parameters of the model M1 are updated (step S12-2). The terminal device 10-3 receiving the model M1 from the server device 100 updates the parameters of the model M1, and the terminal device 10-3 generates the model M1c which is a local model in which the parameters of the model M1 are updated (step). S12-3). The order of steps S12-1 to S12-3 may be any order, and step S12-3 may be performed before S12-1. Hereinafter, when steps S12-1 to S12-3 are described without distinction, they are collectively referred to as step S12.

Here, the models M1a, M1b, and M1c have the same network configuration as the model M1, but only the parameters such as weights are updated. As described above, in step S12, the terminal device 10 generates a local model by updating the parameters of the model M1 which is a global model by learning processing using the data (position information) possessed by the terminal device 10. For example, the terminal device 10 acquires position information using positioning technology such as GNSS (Global Navigation Satellite System) or indoor positioning technology, and stores the acquired position information in a storage unit 40 or the like. Possess.

The terminal device 10 performs a learning process using the acquired position information and generates a local model. For example, the terminal device 10-1 updates the parameters of the model M1 and generates the model M1a by using the vector generated based on the position information of the user U1 and the context of the user U1. As a result, the terminal device 10-1 can generate the model M1a updated to the parameters reflecting the information of the user U1. By the same processing, the terminal devices 10-2 and 10-3 can generate the models M1b and M1c updated to the parameters reflecting the information of the users U2 and U3.

Then, the terminal device 10-1 transmits information indicating the difference between the generated model M1a and the model M1 to the server device 100 (step S13-1). The terminal device 10-2 transmits information indicating the difference between the generated model M1b and the model M1 to the server device 100 (step S13-2). The terminal device 10-3 transmits information indicating the difference between the generated model M1c and the model M1 to the server device 100 (step S13-3). The order of steps S13-1 to S13-3 may be any order, and step S13-3 may be performed before S13-1. Hereinafter, when steps S13-1 to S13-3 are described without distinction, they are collectively referred to as step S13.

As described above, in step S13, each terminal device 10 transmits only the information of the difference between the local model and the global model updated by itself to the server device 100. Each terminal device 10 transmits the parameter updated by itself to the server device 100 as a difference. As a result, the information processing system 1 does not need to transmit the user data held by the terminal device 10 to the server device 100, so that the privacy and personal information of the user can be protected. Further, the information processing system 1 can suppress an increase in the amount of data transmitted by the terminal device 10 to the server device 100, and can suppress an increase in the amount of communication. Note that each terminal device 10 may transmit the difference between the parameters of the local model updated by itself and the parameters of the global model to the server device 100.

Then, the server device 100 updates the model M1 which is a global model by using the difference information received from each terminal device 10 (step S14). The server device 100 updates the parameters of the model M1 which is a global model based on the parameters received from each terminal device 10. As a result, the server device 100 can update the global model by reflecting the learning result in each terminal device 10. In the following, for the sake of explanation, the model M1 after the parameter update may be described as "model M1'" with a dash. Since the process of updating the global model using the difference information is the same as the process of updating the global model in federated learning, detailed description thereof will be omitted.

Then, the server device 100 provides the terminal device 10-1 with a model M1'which is an updated global model (step S15-1). Further, the server device 100 provides the terminal device 10-2 with a model M1'which is an updated global model (step S15-2). Further, the server device 100 provides the terminal device 10-3 with a model M1'which is an updated global model (step S15-3). The order of steps S15-1 to S15-3 may be any order, and steps S15-3 may be performed before S15-1. Hereinafter, when steps S15-1 to S15-3 are described without distinction, they are collectively referred to as step S15.

After that, the information processing system 1 can update the model M1 by repeating steps S12 to S15. As a result, the information processing system 1 can efficiently learn the model for generating the vector by using the data possessed by the plurality of terminal devices. again,

[1-1. When using the model (inference)]
In the above-mentioned processing, the time of learning the model has been described, but the time of using the model (at the time of inference) will be described. In FIG. 1, after the learning process, the terminal device 10 performs inference processing using the received model M1'(updated model M1). For example, at the time of inference, the terminal device 10 inputs the data (position information) possessed by the terminal device 10 to the local model, and transmits the vector output by the local model to the server device 100. Then, the server device 100 analyzes the vector received from the terminal device 10 to determine the context of the user who uses the terminal device 10.

For example, the server device 100 arranges the vector in a specific space. By arranging the vector in a specific space, the server device 100 can clearly grasp the similarity and change of the user's own behavior and / or the similarity and difference of the behavior between the user and another user. ..

First, the advantage of converting position information into a vector will be described. Conventionally, the terminal device 10 has transmitted the position information to the server device 100, but at this time, the terminal device 10 rounds the position information in order to consider the privacy of the user and reduce the amount of data communication. (Rounding process) was performed, and the position information was rounded and converted into coarse position information before transmission. For example, if the position information is the latitude / longitude information of GPS (Global Positioning System) coordinates, the values below the second decimal place at the decimal angle of latitude and longitude are rounded up or down (rounding is also possible). It was converted to the position information of the value.

In this case, when the server device 100 tries to estimate the user's behavior based on the location information, the rounded location information is a rough value, so that a wide area unit such as an area or a large-scale facility can be specified. However, there were cases where it was not possible to identify narrow spaces (and / or differences between indoors and outdoors) such as adjacent small buildings and rooms. Therefore, although it is possible to estimate the rough behavior of the user, it may not be possible to estimate the detailed behavior of the user. For example, if a small building A and a building B (for example, a convenience store and a fast food store) are adjacent to each other within the range indicated by the user's rounded location information, the user can use the building A and the building B. Sometimes it was not possible to identify from the location information where you were, or whether you were inside or outside the building (or site). What the server device 100 (server side) really wants to know is not the user's location information itself, but the user's context indicated by the location information.

This time, when the terminal device 10 converts (vectorizes) its own position information into a vector, it converts the raw data of the position information before the rounding process (rounding process) into a vector. Therefore, the terminal device 10 can generate a vector after identifying a narrow place (and / or a difference between indoors and outdoors) such as an adjacent small building or room. Therefore, the server device 100 can know the detailed context of the user based on the vector. That is, the server device 100 can acquire specific and detailed information regarding the user's behavior from the terminal device 10 rather than receiving the conventional rounded position information. Further, since it is the server device 100 (server side) that defines the vector value of the vector and the content that it means, even if a third party sees only the vector value of the vector, what the vector value means. It's safe because I don't understand if it's done.

[1-2. context〕
As described above, the context means the situation of the subject for which the vector is generated, and in the case of the model M1, it means the situation of the user who uses the terminal device 10. When the vector is generated for the user, the context (also referred to as "user context") is the content of the content provided to the user, the content of the content that the user has reacted to, the attributes of the user, the current position of the user, the current time, and so on. It may be information indicating various user situations such as the physical environment in which the user is placed, the social environment in which the user is placed, the motor state of the user, and the estimated emotions and psychological states of the user.

Further, the user context may be periodic information indicating the periodic behavior of the user. The user's periodic actions include actions that are routinely performed for each day of the week (routine), etc., actions that go to a specific place such as a workplace or a favorite store, and specific actions that have become habitual. Point to. For example, the context may be periodic information indicating the average user behavior for each day of the week or time of day on a weekly basis.

Further, the context means the situation of a group in which a plurality of users are elements (hereinafter, also referred to as a community) or a community in which a plurality of communities are elements.

When the target of vector generation is a community, the context (also referred to as "community context") may be statistical information of a group of users belonging to the community. The community context includes the content of the content provided to the user group belonging to the community, the content content that the user group belonging to the community reacts to, the attributes of the user group belonging to the community, the current position and current time of the user group belonging to the community, and the user group belonging to the community. The situation of the user group belonging to various communities such as the physical environment in which it is placed, the social environment in which the user group belonging to the community is placed, the motor state of the user group belonging to the community, and the emotional and psychological states of the estimated user group belonging to the community. It may be the information (statistical information) to show. Further, the community context may be periodic information (statistical information) that statistically indicates the periodic behavior of the user group to which the community belongs.

It should be noted that the above example of the context is merely an example, and any information may be used as long as it indicates the situation of the subject to which the vector is generated.

[1-3. Other model examples]
In FIG. 1, a model M1 having position information as an input has been described as an example, but if the model outputs the context of the subject for which the vector is to be generated, any data can be adopted as the input. This point will be described below.

For example, the information processing system 1 may generate a model (hereinafter, also referred to as “sequence model”) that outputs a vector indicating a user's context by inputting sequence data in which position information is arranged in time series. In this case, the information processing system 1 may generate a sequence model having a network configuration such as RNN or LSTM. Even if the data is the same in the sequence model, the output changes in the order in which it is input. For example, if you are moving in the order of "Position 1"-> "Position 2"-> "Position 3", "Attendance" (outward route) is output, and "Position 3"-> "Position 2"-> "Position 1". If you are moving in, output "I came home" (return trip). By using such a sequence model, the information processing system 1 can generate a vector based on the characteristics of the transition of position information.

In this way, when a vector is generated from the sequence data using the sequence model, it is possible to generate a vector indicating whether or not the user has performed a predetermined action from the context of the user indicated by the sequence data. For example, in the information processing system 1, when the sequence data of the location information indicates that the user has moved from the area where the user is at home to the area where the user is at work, the element corresponding to the action of "going to work" is a predetermined value. You can generate a sequence model that outputs a vector that takes.

When the above-mentioned sequence model is used, the server device 100 uses the vector value of the vector based on the position information to allow the user to "be at home", "go to work", "go shopping", or "move by train". To determine the user context such as. At this time, the server device 100 may determine the action for each time zone based on the values indicated by the plurality of elements of the multidimensional vector. For example, the server device 100 "goes to work in the morning" when the vector based on the location information includes a value indicating "going out in the morning" and a value indicating "going to work". May be determined.

Here, the value of each element of the vector corresponds to the user's behavior. For example, in the case of a three-dimensional vector, the value of the first element indicates "whether or not you went to work", the value of the second element indicates "whether or not you ate out", and the value of the third element indicates "on the train". It may be specified to indicate "whether or not". In this case, the vector may be a One-hot vector. A one-hot vector is a vector such as (1,0,0) in which one component is 1 and the remaining components are all 0. However, in reality, the vector is not limited to the One-hot vector.

Further, the value of each element of the vector may correspond to a plurality of actions of the user. For example, it is stipulated that the value of the first element indicates "whether or not he went to work and ate out", and the value of the second element indicates "whether or not he did not go to work and ate out". You may.

The values of a plurality of elements contained in a certain range of the vector may correspond to the user's behavior. For example, if the value up to the third element (upper three digits) is "100", it may indicate "going to work", and if it is "101", it may indicate "going to work by train".

Further, the value of each element of the vector may be an evaluation value. For example, if the value up to the third element (upper three digits) is "000", it may be specified to indicate "not moved", and if it is "100", it may be specified to indicate "moved 100 kilometers". Further, the value of any element of the vector may be a value indicating the moving direction. For example, if the values from the 4th element to the 7th element are "0000", it means that the moving direction is "north", if it is "0001", it means that the moving direction is "north-northeast", and if it is "0010". "0011" indicates that the movement direction is "northeast", "0011" indicates that the movement direction is "east-northeast", "0100" indicates that the movement direction is "east", and "1000" indicates that the movement direction is "east". May indicate "south", and "1100" may indicate that the direction of movement is "west". Further, the value of each element of the vector may be a score of a healthy life.

It is assumed that any setting can be applied to the value of each element of the vector at the time of learning the model. For example, it is assumed that an arbitrary setting can be appropriately made depending on the circumstances such as what kind of behavior information the server side wants to acquire from the user side and use it for the service provided to the user.

Further, the terminal device 10 may generate correct answer data corresponding to the data input to the model. For example, the terminal device 10 displays a screen for designating its own context to the user, generates a vector corresponding to the designated context based on the context designated by the user (also referred to as "designated context"), and performs correct answer data. May be used as. For example, the terminal device 10 may display a screen for designating a context such as "home", "working", "shopping", "moving by train", and accept the designation of the designated context from the user.

In this case, the terminal device 10 compares the list with the designated context specified by the user by using the list showing the correspondence between the context and the vector, and is associated with the context that matches (or is similar to) the designated context. The vector may be used as the correct answer data. The terminal device 10 may generate correct answer data by using a conversion model that converts information indicating a user's designated context into a vector indicating the designated context. Then, the terminal device 10 learns the local model using the generated correct answer data and the corresponding data (input data). The above is an example, and the terminal device 10 may acquire the correct answer data corresponding to the data by using any information as long as the correct answer data corresponding to the data can be acquired.

[2. Information processing system configuration example]
Next, the configuration of the information processing system 1 including the server device 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing a configuration example of the information processing system 1 according to the embodiment. As shown in FIG. 2, the information processing system 1 according to the embodiment includes a plurality of terminal devices 10 and a server device 100. These various devices are connected so as to be communicable by wire or wirelessly via the network N. The network N is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network) such as the Internet.

Further, the number of each device included in the information processing system 1 shown in FIG. 2 is not limited to that shown in the figure. For example, in FIG. 2, for simplification of the illustration, only three terminal devices 10-1, 10-2, and 10-3 are shown, but these are merely examples and are not limited to 4 It may be more than one.

The terminal device 10 is an information processing device used by a user (user). For example, the terminal device 10 is a smart device such as a smartphone or tablet terminal, a feature phone, a PC (Personal Computer), a PDA (Personal Digital Assistant), a car navigation system, a wearable device such as a smart watch or a head-mounted display (Wearable Device). , Smart glasses, etc. For example, the terminal device 10 has a screen such as a liquid crystal display having a touch panel function, and receives various operations for display data such as contents such as tap operation, slide operation, scroll operation, etc. from a user with a finger, a stylus, or the like. The operation performed on the area of the screen on which the content is displayed may be an operation on the content. Further, the terminal device 10 may be not only a smart device but also an information processing device such as a desktop PC (Personal Computer) or a notebook PC.

The terminal device 10 includes a wireless communication network such as LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation: 5th generation mobile communication system), Bluetooth (registered trademark), and wireless LAN (Local Area Network). It is possible to connect to the network N via short-range wireless communication such as, and communicate with the server device 100.

The server device 100 is, for example, a PC, a server device, a mainframe, a workstation, or the like. The server device 100 may be realized by cloud computing.

[3. Configuration example of terminal device]
Next, the configuration of the terminal device 10 will be described with reference to FIG. FIG. 3 is a diagram showing a configuration example of the terminal device 10. As shown in FIG. 3, the terminal device 10 includes a communication unit 11, a display unit 12, an input unit 13, a positioning unit 14, a sensor unit 20, a control unit 30 (controller), and a storage unit 40. Be prepared.

(Communication unit 11)
The communication unit 11 is connected to the network N (see FIG. 2) by wire or wirelessly, and transmits / receives information to / from the server device 100 via the network N. For example, the communication unit 11 is realized by a NIC (Network Interface Card), an antenna, or the like.

(Display unit 12)
The display unit 12 is a display device that displays various information such as position information. For example, the display unit 12 is a liquid crystal display (LCD) or an organic EL display (Organic Electro-Luminescent Display). Further, the display unit 12 is a touch panel type display, but the display unit 12 is not limited thereto.

(Input unit 13)
The input unit 13 is an input device that receives various operations from the user U. The input unit 13 has, for example, a button for inputting characters, numbers, and the like. When the display unit 12 is a touch panel type display, a part of the display unit 12 functions as an input unit 13. The input unit 13 may be a microphone or the like that accepts voice input from the user U. The microphone may be wireless.

(Positioning unit 14)
The positioning unit 14 receives a signal (radio wave) transmitted from a GPS satellite, and acquires position information (for example, latitude and longitude) indicating the current position of the terminal device 10 which is its own device based on the received signal. do. That is, the positioning unit 14 positions the terminal device 10. GPS is only an example of GNSS (Global Navigation Satellite System).

In addition, the positioning unit 14 can position the position by various methods other than GPS. For example, the positioning unit 14 may position the position by using various communication functions of the terminal device 10 as an auxiliary positioning means for position correction or the like as described below.

(Wi-Fi positioning)
For example, the positioning unit 14 positions the position of the terminal device 10 by using the Wi-Fi (registered trademark) communication function of the terminal device 10 or the communication network provided by each communication company. Specifically, the positioning unit 14 determines the position of the terminal device 10 by performing Wi-Fi communication or the like and measuring the distance to a nearby base station or access point.

(Beacon positioning)
Further, the positioning unit 14 may position the position by using the Bluetooth (registered trademark) function of the terminal device 10. For example, the positioning unit 14 positions the terminal device 10 by connecting to a beacon transmitter connected by the Bluetooth (registered trademark) function.

(Geomagnetic positioning)
Further, the positioning unit 14 positions the position of the terminal device 10 based on the geomagnetic pattern of the structure measured in advance and the geomagnetic sensor included in the terminal device 10.

(RFID positioning)
Further, for example, when the terminal device 10 has an RFID (Radio Frequency Identification) tag function equivalent to a contactless IC card used in a station ticket gate, a store, or the like, or has a function of reading an RFID tag. In this case, the used position is recorded together with the information that the payment or the like has been made by the terminal device 10. The positioning unit 14 may position the position of the terminal device 10 by acquiring such information. Further, the position may be determined by an optical sensor included in the terminal device 10, an infrared sensor, or the like.

The positioning unit 14 may position the position of the terminal device 10 by using one or a combination of the above-mentioned positioning means, if necessary.

(Sensor unit 20)
The sensor unit 20 includes various sensors mounted on or connected to the terminal device 10. The connection may be a wired connection or a wireless connection. For example, the sensor unit 20 may be a detection device other than the terminal device 10, such as a wearable device. In the example shown in FIG. 3, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, a pressure sensor 23, a temperature sensor 24, a sound sensor 25, an optical sensor 26, a magnetic sensor 27, and an image sensor ( It is equipped with a camera) 28.

It should be noted that the above-mentioned sensors 21 to 28 are merely examples and are not limited thereto. That is, the sensor unit 20 may be configured to include a part of the sensors 21 to 28, or may include other sensors such as a humidity sensor in addition to or in place of the sensors 21 to 28. ..

The acceleration sensor 21 is, for example, a 3-axis acceleration sensor, and detects the physical movement of the terminal device 10 such as the moving direction, the speed, and the acceleration of the terminal device 10. The gyro sensor 22 detects the physical movement of the terminal device 10 such as the inclination in the triaxial direction based on the angular velocity of the terminal device 10 and the like. The barometric pressure sensor 23 detects, for example, the barometric pressure around the terminal device 10.

Since the terminal device 10 includes the above-mentioned acceleration sensor 21, gyro sensor 22, pressure sensor 23, etc., technologies such as pedestrian autonomous navigation (PDR) using each of these sensors 21 to 23, etc. It becomes possible to determine the position of the terminal device 10 by using. This makes it possible to acquire indoor position information, which is difficult to acquire with a positioning system such as GPS.

For example, a pedometer using an acceleration sensor 21 can calculate the number of steps, the walking speed, and the walking distance. Further, the gyro sensor 22 can be used to know the traveling direction of the user U, the direction of the line of sight, and the inclination of the body. Further, from the atmospheric pressure detected by the atmospheric pressure sensor 23, it is possible to know the altitude at which the terminal device 10 of the user U exists and the number of floors of the floor.

The air temperature sensor 24 detects, for example, the air temperature around the terminal device 10. The sound sensor 25 detects, for example, the sound around the terminal device 10. The optical sensor 26 detects the illuminance around the terminal device 10. The magnetic sensor 27 detects, for example, the geomagnetism around the terminal device 10. The image sensor 28 captures an image of the surroundings of the terminal device 10.

The barometric pressure sensor 23, the temperature sensor 24, the sound sensor 25, the optical sensor 26, and the image sensor 28, respectively, detect the barometric pressure, the temperature, the sound, and the illuminance, and capture an image of the surroundings, so that the terminal device 10 It is possible to detect the surrounding environment and situation of. Further, it is possible to improve the accuracy of the position information of the terminal device 10 from the surrounding environment and the situation of the terminal device 10.

(Control unit 30)
The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM, an input / output port, and various circuits. Further, the control unit 30 may be configured by hardware such as an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 includes a reception unit 31, an acquisition unit 32, a processing unit 33, a learning unit 34, an estimation unit 35, and a transmission unit 36.

(Receiver 31)
The receiving unit 31 receives various information provided from the server device 100 via the communication unit 11. For example, the receiving unit 31 receives the global model provided by the server device 100 via the communication unit 11. The receiving unit 31 stores (stores) the received global model in the model information storage unit 41.

(Acquisition unit 32)
The acquisition unit 32 acquires the position information of the terminal device 10 at a constant / periodic / predetermined timing. For example, the acquisition unit 32 acquires the position information of the terminal device 10 positioned by the positioning unit 14. Further, the acquisition unit 32 estimates and acquires the position information of the terminal device 10 based on various information detected by the sensors 21 to 28.

Further, the acquisition unit 32 stores (saves) the acquired position information in the storage unit 40. For example, the acquisition unit 32 stores the acquired position information as a position history (for example, location history, action history, etc.).

(Processing unit 33)
The processing unit 33 controls the entire terminal device 10 including the display unit 12 and the like. For example, the processing unit 33 can output various information transmitted by the transmitting unit 36 and various information received by the receiving unit 31 from the server device 100 to the display unit 12 for display. The processing unit 33 generates correct answer data corresponding to the data.

The processing unit 33 generates input data to be input to the model. When the data such as the position information acquired by the acquisition unit 32 is used as the input data of the model as it is, the processing unit 33 uses the data acquired by the acquisition unit 32 as the input data without performing the process of generating the input data.

When the processing unit 33 learns a model that can handle missing data, the processing unit 33 uses specific data (for example, n / a (value indicating not applicable)) as the value of the missing portion of the data (also simply referred to as data). Etc.) to generate input data that complements the missing part. When learning a model that can handle missing data, the processing unit 33 may generate input data using the estimated data estimated by the estimation unit 35.

When the processing unit 33 has acquired the position information every minute, the processing unit 33 arranges the position information every minute in chronological order and divides the sequence data into 30 minutes (for example, 8:01 to 8:30). Generate as input data. For example, the processing unit 33 generates sequence data in which position information for each minute in 30 minutes is arranged in chronological order as input data. For example, the processing unit 33 generates sequence data in which the position information for the first minute in 30 minutes and the difference for each minute with respect to the position information are arranged in chronological order as input data. Note that the sequence data is only an example, and may be, for example, input data in which raw data of position information is input as it is in chronological order.

Further, the processing unit 33 may generate a vector by using the local model learned by the learning unit 34. The processing unit 33 may generate a vector by using a local model at the time of inference.

When there is a defect in the data at the time of inference, the processing unit 33 uses specific data as the data of the missing portion, inputs the input data complementing the missing portion to the local model, and generates a vector. You may. Further, if there is missing data at the time of inference, the processing unit 33 may input the input data using the estimated data estimated by the estimation unit 35 into the local model and generate a vector.

(Learning Department 34)
The learning unit 34 generates a local model. The learning unit 34 uses the possessed data to generate a local model in which the parameters of the global model received from the server device 100 are updated. The learning unit 34 generates a local model using the estimated estimation data.

The learning unit 34 generates a local model using data including sensor information detected by the sensor. The learning unit 34 generates a local model using data including position information.

For example, the learning unit 34 performs learning processing using a pair of the data acquired by the acquisition unit 32 and the vector which is the correct answer data corresponding to the data, and generates a local model in which the parameters of the global model are updated. The vector, which is the correct answer data, is provided by any means. For example, the correct answer data may be a vector generated based on the information indicating the context specified by the user of the terminal device 10.

In the learning process in which the learning unit 34 updates the parameters of the model, an appropriate method is adopted according to the network configuration of the model. For example, when the network configuration of the model is DNN, the learning unit 34 updates the parameters of the model by the method of deep learning. The learning unit 34 updates the parameters of the model by a learning process performed by using a pair (combination) of the data input to the model and the correct answer data indicating the output when the data is input, that is, the supervised learning method. do. The above is an example, and the learning unit 34 may update the parameters of the model by any learning process as long as the parameters of the model can be updated.

When generating a model that allows missing data, the learning unit 34 learns a model that can handle the missing data. For example, the learning unit 34 learns a model that can deal with the missing data by using specific data as the data of the missing portion of the data and using the data that complements the missing portion.

The learning unit 34 may learn a model that inputs input data including estimated data that estimates missing data. The learning unit 34 learns a local model that inputs input data including the estimated data estimated by the estimation unit 35.

(Estimation unit 35)
The estimation unit 35 estimates the missing data. The estimation unit 35 generates estimated data (estimated value) that estimates (value) the data of the missing portion. The estimation unit 35 may estimate the data of the defective portion by using an estimation model for estimating the data of the defective portion.

For example, when the estimation unit 35 has a defect in a part of the sensor information, the estimation unit 35 estimates the data corresponding to the defect. The estimation unit 35 estimates the data corresponding to the missing sensor based on the past information of the sensor whose data is missing. When the data indicating the position is lost, the estimation unit 35 estimates the data indicating the missing position based on the past position information detected by the position detecting sensor (positioning unit 14 or the like) and the information of the acceleration sensor. do. For example, when the data indicating the position of the user at the date and time X is missing, the estimation unit 35 sets the position information of the user at the date and time Y before the date and time X detected by the sensor that detects the position, and the date and time Y to the date and time X. Based on the information of the acceleration sensor 21 and the gyro sensor 22, the data indicating the position of the user at the missing date and time X is estimated.

For example, the estimation unit 35 estimates the data corresponding to the missing sensor based on the information of the sensor other than the sensor for which the data is missing. For example, when the terminal device 10 does not have the temperature sensor 24 and the data indicating the temperature is missing, the estimation unit 35 estimates the data indicating the temperature based on the information of the pressure sensor 23 and the optical sensor 26. For example, the estimation unit 35 estimates data indicating temperature by using an estimation model that inputs a combination of a barometric pressure value and an illuminance value and outputs a temperature value corresponding to the combination. The estimation unit 35 estimates data indicating the temperature by using an estimation model trained using a pair of a combination of atmospheric pressure and illuminance and a temperature corresponding to the combination as training data. In this case, the estimation unit 35 estimates that the value output by the estimation model by inputting the atmospheric pressure value and the illuminance value at the estimation target date and time into the estimation model is the temperature at the estimation target date and time.

The above is an example, and the estimation unit 35 may perform estimation by any method as long as the missing data can be estimated. The estimation unit 35 may estimate the data of the defective portion to be specific data. For example, the estimation unit 35 may estimate and interpolate the data of the missing portion by using a list showing the correspondence between the type of data and the data to be complemented when the data of the type is missing. For example, the estimation unit 35 uses a list in which the values of the plurality of levels used as the values of the missing sensor are associated with the values of the other sensors, and the data of the missing portion is obtained from the values of the other sensors. It may be estimated and interpolated.

(Transmitting unit 36)
The transmission unit 36 transmits information to the server device 100 via the communication unit 11. The transmission unit 36 transmits update information, which is information about the local model, to the server device 100. The transmission unit 36 transmits the difference between the global model and the local model to the server device 100 as update information.

Further, the transmission unit 36 may transmit various information input by the user U using the input unit 13, various information processed by the processing unit 33, and the like to the server device 100. Further, the transmission unit 36 transmits the vector generated by the processing unit 33 to the server device 100.

(Memory unit 40)
The storage unit 40 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an optical disk. To. Various programs, various data, and the like are stored in the storage unit 40. As shown in FIG. 3, the storage unit 40 has a model information storage unit 41.

(Model information storage unit 41)
The model information storage unit 41 according to the embodiment stores information about the model. For example, the model information storage unit 41 stores a common model (global model) received from the server device 100 and a local model (local model) in which the parameters of the global model are updated by the own device. FIG. 4 is a diagram showing an example of a model information storage unit according to an embodiment. The model information storage unit 41 shown in FIG. 4 includes items such as "model ID", "use", "type", and "model data".

The "model ID" indicates identification information for identifying the model. For example, the model identified by the model ID "M1" corresponds to the model M1 shown in the example of FIG. "Use" indicates the use of the corresponding model. "Type" indicates the type of the model. For example, "type" indicates whether the model is a global model or a local model. Further, the "model data" indicates the data of the corresponding model associated with the model ID. In FIG. 4 and the like, an example in which conceptual information such as "MDT1" is stored in "model data" is shown, but in reality, the model is configured such as model configuration (network configuration) information and parameter information. Contains various information. For example, "model data" includes information including nodes at each layer of the network, functions adopted by each node, connection relationships of the nodes, and connection coefficients set for connections between the nodes.

In FIG. 4, the model identified by the model ID “M1” (model M1) shows that its use is “vector transformation”. The model M1 indicates that the type is "global" and is a global model. Further, it is shown that the model data of the model M1 is the model data MDT1.

Further, the model (model M1a) identified by the model ID "M1a" indicates that the use is "vector conversion". Further, the model M1a indicates that the type is "local" and is a local model. Further, it is shown that the model data of the model M1a is the model data MDT1a.

The model information storage unit 41 is not limited to the above, and may store various model information depending on the purpose.

Further, the storage unit 40 is positioned by, for example, various information input by the user U using the input unit 13, various information detected by the sensors 21 to 28 mounted on or connected to the terminal device 10, and the positioning unit 14. The position information and the like of the terminal device 10 are stored. Further, various information processed by the processing unit 33 and various information received by the receiving unit 31 are stored.

[4. Server device configuration example]
Next, the configuration of the server device 100 according to the embodiment will be described with reference to FIG. FIG. 5 is a diagram showing a configuration example of the server device 100 according to the embodiment. As shown in FIG. 5, the server device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. Further, the communication unit 110 is connected to the network N (see FIG. 2) by wire or wirelessly.

(Memory unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. As shown in FIG. 5, the storage unit 120 has a model information storage unit 121 and a community information storage unit 122.

(Model information storage unit 121)
The model information storage unit 121 according to the embodiment stores information about the model. For example, the model information storage unit 121 stores a common model (global model) learned for users and communities. FIG. 6 is a diagram showing an example of a model information storage unit according to an embodiment. The model information storage unit 121 shown in FIG. 6 includes items such as "model ID", "use", "type", "target", and "model data".

The "model ID" indicates identification information for identifying the model. "Use" indicates the use of the corresponding model. "Type" indicates the type of the model. "Object" indicates an object for which the model is used. Further, the "model data" indicates the data of the corresponding model associated with the model ID.

In FIG. 6, the model (model M1) identified by the model ID “M1” indicates that its use is “vector transformation”. The model M1 indicates that the type is "global" and is a global model. It is shown that the target of the model M1 is a model for users such as users U1, U2, and U3. Further, it is shown that the model data of the model M1 is the model data MDT1.

Further, the model (model M11) identified by the model ID "M11" indicates that the use is "vector conversion". The model M11 indicates that the type is "global" and is a global model. It is shown that the target of the model M11 is a model for a group (user group) such as community GP1, GP2, GP3 and the like. Further, it is shown that the model data of the model M11 is the model data MDT11.

The model information storage unit 121 is not limited to the above, and may store various model information depending on the purpose.

(Community Information Memory Unit 122)
The community information storage unit 122 according to the embodiment stores various information about the community (group). For example, the community information storage unit 122 stores the hierarchy of the community and the components of the community. FIG. 7 is a diagram showing an example of a community information storage unit according to an embodiment. The community information storage unit 122 shown in FIG. 7 includes items such as "community ID", "hierarchy", and "element".

The "community ID" indicates identification information for identifying a community. "Hierarchy" indicates the hierarchy of the community. For example, in the "hierarchy", "*" of "H * (* is an arbitrary value)" becomes larger as the hierarchy becomes higher, such as "H1" and "H2", and the hierarchy H2 is a higher hierarchy of the hierarchy H1. .. Further, the "element" indicates an element belonging to each community, that is, a component of the community.

In FIG. 7, the community (community GP1) identified by the community ID “GP1” is the hierarchy H1, and its constituent elements are the users U1, U2, U3, and the like. The user is a base element constituting the community, and may be managed as a hierarchy H0.

Further, the community identified by the community ID "GP11" (community GP11) is the hierarchy H2, and the constituent elements thereof are the community GP1, GP2, GP3 and the like. That is, it is shown that the community GP11 of the hierarchy H2 has the communities GP1, GP2, GP3 and the like of the hierarchy H1 as elements.

The community information storage unit 122 is not limited to the above, and may store various information depending on the purpose.

(Control unit 130)
Returning to FIG. 5, the explanation will be continued. The control unit 130 is a controller, and for example, the server device 100 is provided with a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. It is realized by executing various programs (corresponding to an example of an information processing program) stored in the internal storage device using a storage area such as a RAM as a work area. In the example shown in FIG. 5, the control unit 130 has a providing unit 131, an acquisition unit 132, a learning unit 133, and a determination unit 134.

(Providing section 131)
The providing unit 131 transmits the global model to the terminal device 10 via the communication unit 110. The providing unit 131 provides a global model that generates a vector indicating the user's context from the data acquired by the terminal device 10. The providing unit 131 provides the updated global model to a plurality of terminal devices 10.

(Acquisition unit 132)
The acquisition unit 132 receives information for updating the global model from the terminal device 10 via the communication unit 110. The acquisition unit 132 acquires update information, which is information about the local model, from each of the plurality of terminal devices 10. The acquisition unit 132 acquires the difference between the global model and the local model from each of the plurality of terminal devices 10.

The acquisition unit 132 receives the vector generated from the data acquired by the terminal device 10 from the terminal device 10. The acquisition unit 132 acquires various types of information from the storage unit 120.

(Learning Department 133)
The learning unit 133 generates a global model. The learning unit 133 learns a vector conversion model that outputs a vector by inputting data as a global model. The learning unit 133 may generate a global model in which each parameter is set to a predetermined initial value. The learning unit 133 may generate a global model using the data acquired in the past. For example, the learning unit 133 may learn the global model by using the pair of the data acquired in the past and the vector which is the correct answer data corresponding to the data.

The learning unit 133 updates the parameters of the global model by using the update information received from the plurality of terminal devices 10. The learning unit 133 updates the parameters of the global model by using the difference information received from the plurality of terminal devices 10.

As described above, any network configuration can be adopted as the network configuration of the global model generated by the learning unit 133. For example, the network configuration of the global model may be CNN, RNN, LSTM, or the like. Further, the network configuration of the global model may be a configuration in which a plurality of networks are combined, such as a combination of CNN and RNN. The RNN and LSTM may be a neural network based on an attention mechanism. Further, the learning unit 133 may use a similar natural language processing model.

(Judgment unit 134)
The determination unit 134 analyzes the vector acquired by the acquisition unit 132 and determines the context of the user who uses the terminal device 10. For example, the determination unit 134 arranges the vector in a specific space and determines the similarity or change in the behavior of the user himself / herself and / or the similarity or difference in the behavior between the user U and another user.

[5. When there is data loss]
Here, the case where there is a data loss will be described with reference to FIGS. 8 and 9. FIG. 8 is a diagram showing an example of a model. FIG. 9 is a diagram showing an example of a case where a part of the data is missing.

First, the whole model will be described with reference to FIG. For example, in FIG. 8, the sensor a may be a sensor (positioning unit 14 or the like) for detecting a position, the sensor b may be a sound sensor 25, and the sensor c may be an image sensor 28.

The model MX shown in FIG. 8 has an encoder EN1 that converts the sensor information SD1 of the sensor a into the intermediate representation IR1, an encoder EN2 that converts the sensor information SD2 of the sensor b into the intermediate representation IR2, and an intermediate representation IR3 of the sensor information SD3 of the sensor c. Includes an encoder EN3 that converts to, and a network NN1 that outputs the vector VD1 with the intermediate representations IR1, IR2, IR3 as inputs. The network NN1 is an integrator having a configuration such as an RNN.

Encoders EN1 to EN3 are blocks that process the raw data of each sensor into an intermediate expression. Further, the network NN1 is a block that aggregates intermediate expressions to generate a vector indicating a context. That is, the model MX is a model in which the sensor information SD1 to SD3 detected by the three sensors a, b, and c are used as input data, and the vector VD1 indicating the context corresponding to the input data is output. In the model MX, each block of the encoders EN1 to EN3 and the network NN1 may be individually learned, and the trained encoders acquired from the external device are used for the encoders EN1 to EN3, and only the network NN1 has information. It may be the target of the learning process of the processing system 1. For example, the information processing system 1 may learn only the network NN1.

When learning using a plurality of terminal devices 10, when a certain terminal device 10 does not have at least one of the sensor a, the sensor b, and the sensor c, or at least one of the sensor a, the sensor b, and the sensor c. If one cannot detect it, the model MX may not be properly learned.

Therefore, the information processing system 1 adjusts the model MX so that learning can be performed appropriately even if the input data is missing. For example, as shown in FIG. 9, the information processing system 1 accepts specific data (for example, n / a (value indicating not applicable) or the like) indicating that the sensor information could not be acquired by the network NN1 as an input. enable. FIG. 9 shows a case where the sensor information SD3 of the sensor c could not be acquired. In this case, the information processing system 1 inputs a value indicating not applicable to the network NN1 as the intermediate representation IR31 of the sensor c, and the model MX sets the model MX. Output the vector VD2.

In this way, the information processing system 1 learns the model MX so that the vector can be generated even when the sensor information cannot be acquired. As a result, in the information processing system 1, when the plurality of terminal devices 10 include the terminal device 10 which does not have some of the required sensors, or when some of the sensor information cannot be acquired. However, it is possible to generate a model that can be used for inference. The information processing system 1 may perform estimation processing on the missing data to supplement the data.

As described above, the information processing system 1 makes it possible to input specific data indicating n / a (not available) into the model when some of the sensor information is missing. As a result, the information processing system 1 makes full use of the sensor information if it is available to generate a vector, and makes it possible to generate a vector even if the sensor information is missing. The information processing system 1 reflects the difference between the parameters of the global model and the parameters of the local model in the form of adding to the global model regardless of whether there is a defect or not.

Reasons for the lack of information on a specific sensor during learning include, for example, limitation of the function of the terminal device 10 itself, privacy preference of the user, and the like. For this reason, it is very likely that the same restrictions apply not only when learning but also when using the model (inference).

However, the information processing system 1 targets various terminal devices 10 such as a terminal device 10 in which all types of sensor information are available, for example, a terminal device 10 in which an input of a specific sensor such as an environmental sound or a gyro is missing. , The difference of the local model can be reflected in the global model. Then, the information processing system 1 can perform processing on the terminal device 10 having a data defect at the time of inference. As described above, the information processing system 1 can appropriately process the terminal device 10 whose sensor information cannot be acquired and whose data is missing, both at the time of learning and at the time of inference.

[6. About the community model]
The model generated by the information processing system 1 is not limited to the above. For example, the information processing system 1 may generate a model whose application range is each community model. This point will be described with reference to FIG.

FIG. 10 is a diagram showing an example of a community having a plurality of layers. The example of FIG. 10 shows an example of a model generated for each community when the communities are layered.

The community GP1 described as community # 1 in FIG. 10 is a community having a plurality of users such as users U1, U2, and U3 as components. That is, the community GP1 is a community of the hierarchy H1 (first hierarchy community) in which a plurality of users (hierarchy H0) are included as components. For example, the community GP1 corresponds to an area in which users U1 to U3 and the like are located.

The community GP2 described as community # 2 in FIG. 10 is a community having a plurality of users such as users U11, U12, and U13 as components. That is, the community GP2 is a community of the hierarchy H1 (first hierarchy community) in which a plurality of users (hierarchy H0) are included as components.

The community GP11 described as the community # 11 in FIG. 10 is a community having a plurality of communities such as the communities GP1 and GP2 as components. That is, the community GP 11 is a community of the hierarchy H2 (second hierarchy community) in which a plurality of first hierarchy communities (layer H1) are included as components. For example, the community GP11 corresponds to an area including the area of the community GP1 and the area of the community GP2.

The community GP21 described as the community # 21 in FIG. 10 is a community having a plurality of communities such as the community GP11 and the GP12 as components. That is, the community GP21 is a community of the hierarchy H3 (third hierarchy community) in which a plurality of second hierarchy communities (layer H2) are included as components. In this way, each community is hierarchically organized based on the user.

FIG. 10 shows that the model M1, which is the global model shown in FIG. 1, is a first-layer model applied to the community GP1 whose components are the users U1, U2, U3, and the like. That is, the model M1 indicates that the users U1, U2, U3, etc. belonging to the community GP1 are models for which the vector is generated. The model M1 is learned as described with reference to FIG. 1 and is used for inference targeting users U1, U2, U3, etc. belonging to the community GP1.

It is shown that the model M2, which is a global model, is a first-layer model applied to the community GP2 whose components are the users U11, U12, U13, and the like. That is, the model M2 indicates that the users U11, U12, U13, etc. belonging to the community GP2 are models for which the vector is generated. The model M2 is learned in the same manner as the model M1 and is used for inference targeting the users U11, U12, U13, etc. belonging to the community GP2.

As described above, in FIG. 10, the server device 100 provides a different global model as the first layer model for each of the plurality of first groups (first layer communities) including the terminal device 10. For example, the server device 100 updates the parameters of each of the plurality of first-tier models using the difference information received from the terminal device 10 of the first-tier community. The server device 100 updates the parameters of the model M1 by using the difference information received from the terminal device 10 used by the users U1, U2, U3, and the like. Further, the server device 100 updates the parameters of the model M2 by using the difference information received from the terminal device 10 used by the users U11, U12, U13 and the like. Since the learning of the local model is the same as that in FIG. 1, detailed description thereof will be omitted.

FIG. 10 shows that the model M11, which is a global model, is a second-tier model applied to the community GP11 whose components are the community GP1, GP2, and the like. Further, it is shown that the model M21, which is a global model, is a third-layer model applied to the community GP21 whose components are the community GP11, GP12, and the like. Hereinafter, the model of the second layer or higher will be described by taking the second layer model as an example.

In the second layer model of the second layer community, the vector generation target may be a user belonging to the community under the community. In this case, the second layer model takes the data acquired by the terminal device 10 used by the user as an input, and outputs a vector indicating the context of the user.

For example, the model M11 is a model in which the users U1, U2, U3 belonging to the community GP1 under the community GP11, and the users U11, U12, U13 and the like belonging to the community GP2 are the target of vector generation. When the model M1 and the model M2 have been learned, the information processing system 1 may integrate the model M1 and the model M2 to generate the model M11. In this case, the server device 100 may generate the model M11 by integrating (for example, averaging) the parameters of the model M1 and the parameters of the model M2. That is, the server device 100 updates the parameters of M11, which is the second layer model of the upper layer of the first layer model, by using a plurality of first layer models such as model M1 and model M2.

When the model M1 and the model M2 have not been learned, the model M11 is learned by performing the same processing as in FIG. 1 for the users U1, U2, U3, U11, U12, U13, and the like. The model M11 is used for inference for users U1, U2, U3 belonging to the community GP1 under the community GP11, and users U11, U12, U13 and the like belonging to the community GP2. Further, the models of the third and higher layers are learned in the same manner as in the case of the second layer model. All tier models may have the same network configuration. Further, in the model of all layers, the input may be the user's data and the output may be a vector indicating the user's context.

In this way, the information processing system 1 can learn an applicable range, that is, a community model in which the scale of the community is different, and can provide an appropriate service by using the community model according to the scale in which the inference is performed. In the example of FIG. 10, the case where the user and the community are configured so as not to be included in the community of each layer is shown as an example, but the user or the community is included in the community of each layer in duplicate. You may. For example, the user U1 may be included in the community GP6 in addition to the community GP1.

Further, in the second layer model of the second layer community, the vector generation target may be the community under the community, that is, the first layer community. In this case, the second layer model takes the data of the first layer community as input and outputs a vector indicating the context of the first layer community. The data of the first-tier community may be any information as long as it can generate a vector indicating the community context of the first-tier community, and is statistical information about a group of users belonging to the first-tier community. It may be information about the range occupied by the first-tier community. The data of the first layer community used as the input of the second layer model may be generated by using the vector output by the model of the first layer community (first layer model). Further, the models of the third and higher layers are learned in the same manner as in the case of the second layer model.

The server device 100 may generate correct answer data to be used at the time of learning by an appropriate method, or may acquire it from an external device. The server device 100 may acquire information indicating the community context from an external device. The server device 100 compares the list with the acquired community context (acquisition context) using a list showing the association between the community context and the vector, and associates the list with the community context that matches (or is similar to) the acquisition context. The obtained vector may be used as the correct answer data. In addition, the community is not limited to the community related to the area, and any structure can be adopted. For example, the community may be a community related to user attributes such as gender and age. In this case, for example, users of the same gender may belong to the same community, or users of similar age may belong to the same community.

[7. Processing procedure]
Next, the processing procedure by the information processing system 1 according to the embodiment will be described with reference to FIG. FIG. 11 is a flowchart showing a processing procedure according to the embodiment.

As shown in FIG. 11, the information processing system 1 provides a common model in which the server device 100 generates a vector indicating the user's context from the data acquired by the terminal device 10 (step S101).

Then, the information processing system 1 generates a local model in which the parameters of the common model received from the server are updated by using the data possessed by each of the plurality of terminal devices 10 (step S102).

Then, in the information processing system 1, a plurality of terminal devices 10 transmit update information, which is information about the local model, to the server device 100 (step S103).

Then, in the information processing system 1, the server device 100 updates the parameters of the common model using the update information received from the plurality of terminal devices 10, and provides the updated common model to the plurality of terminal devices 10. Step S104).

[8. Other examples]
The terminal device 10 and the server device 100 described above may be implemented in various different forms other than the above-described embodiment. Therefore, a modified example of the embodiment will be described below.

In the above embodiment, the server device 100 uses the vector value of the vector based on the position information, such as "going to the mountain", "going to the sea", "going to a rural area", "going to a foreign country", etc. You may judge the behavior of. Further, the user may determine an action such as "going on a trip" or "on a business trip". In addition, the user may determine an action such as "day trip" or "stayed".

Further, in the above embodiment, when the terminal device 10 does not want to notify the server device 100 (server side) of detailed information regarding the user's behavior, when accumulating the position information or prior to embedding, the position information is stored. Fraction processing (rounding processing) may be performed to round the position information and convert it into position information with coarse accuracy. For example, when the position information is the latitude / longitude information of the GPS coordinates, the terminal device 10 rounds up or rounds down the value after the decimal point (minutes or less in the degree, minute, and second system) at the decimal angle of the latitude and longitude (even rounding off). Yes) and convert to rough value position information. Thereby, it is possible to limit the content of the information transmitted from the terminal device 10 to the server device 100 (server side) regarding the information regarding the user's behavior. In addition, the amount of position information as learning data for machine learning can be reduced, and the processing load can be reduced.

Further, in the above embodiment, the value of each element of the vector is expressed by "0" and "1" for convenience, but in reality, a numerical value of "2" or more, a numerical value of a plurality of digits, or a numerical value of a plurality of digits is used. It may be a numerical value having an integer part and a decimal part. That is, it may be a real number. As a result, a plurality of information and detailed information can be expressed in each element of the multidimensional vector.

Further, in the above embodiment, the terminal device 10 stores the acquired position information as a position history (for example, location history, action history, etc.), but when the storage of the position history is restricted or prohibited. , May be accumulated in a vectorized state. For example, the terminal device 10 sets the location information as a vector value indicating "at home", "at work", "at the store", "outdoor", etc., based on the location indicated by the individual location information. It may be converted to and accumulated. Then, when the same vector values are continuous along the time series, they may be collectively made into a vector.

[9. effect〕
As described above, the information processing system 1 according to the present application is a server (server in the embodiment) that provides a common model (global model in the embodiment) that generates a vector indicating a user's context from the data acquired by the terminal device 10. Using the device 100) and the data possessed by each, a local model (local model in the embodiment) in which the parameters of the common model received from the server are updated is generated, and the update information which is the information about the local model is transmitted to the server. It has a plurality of terminal devices 10 and the like. The server updates the parameters of the common model using the update information received from the plurality of terminal devices 10, and provides the updated common model to the plurality of terminal devices 10.

As a result, the information processing system 1 can efficiently learn the model for generating the vector by using the data possessed by the plurality of terminal devices 10.

Further, in the information processing system 1, the plurality of terminal devices 10 transmit the difference between the common model and the local model to the server as update information. In this way, the information processing system 1 reduces the amount of communication by transmitting only the difference between the plurality of terminal devices 10 to the server device 100, and transmits the data possessed by the plurality of terminal devices 10 to the outside. Can be suppressed.

Further, in the information processing system 1, when the possessed data has a defect, the plurality of terminal devices 10 perform predetermined processing on the missing data and generate a local model. As described above, in the information processing system 1, even when there is a defect in the data, since the plurality of terminal devices 10 can learn the local model, the model can be learned efficiently.

Further, in the information processing system 1, the plurality of terminal devices 10 estimate the missing data and generate a local model using the estimated estimated data. As described above, in the information processing system 1, even if there is a defect in the data, the plurality of terminal devices 10 can estimate the missing data and learn the local model, so that the model can be efficiently learned. ..

Further, in the information processing system 1, the plurality of terminal devices 10 generate a local model using data including sensor information detected by a sensor (positioning unit 14 or sensor unit 20 in the embodiment). As described above, in the information processing system 1, since the plurality of terminal devices 10 can learn the local model using the sensor information, it is possible to efficiently learn the model that outputs the vector indicating the context based on the sensor information. ..

Further, in the information processing system 1, the plurality of terminal devices 10 generate a local model using data including position information. As described above, in the information processing system 1, since the plurality of terminal devices 10 can learn the local model using the position information, the model that outputs the vector indicating the context based on the position information can be efficiently learned. ..

Further, in the information processing system 1, the common model inputs data and outputs a vector. In this way, the information processing system 1 can efficiently learn a model that outputs a vector by inputting data.

Further, in the information processing system 1, the server provides a common model different for each of a plurality of first groups (communities in the embodiment) including the terminal device 10 as a first-layer common model, and the servers of the plurality of first groups. The parameters of each of the plurality of first-tier common models are updated using the update information received from the terminal device 10, and the plurality of updated first-tier common models are used to support the plurality of first-tier common models. Update the parameters of the second layer common model, which is the upper layer model. In this way, the information processing system 1 can efficiently learn a plurality of models corresponding to a plurality of layers.

Further, in the information processing system 1, the server updates the parameters of the plurality of second layer common models corresponding to each of the plurality of second groups including the first group, and is common to the plurality of second layers after the update. Using the model, the parameters of the third layer common model, which is the upper layer model of the plurality of second layer common models, are updated. In this way, the information processing system 1 can efficiently learn a plurality of models corresponding to a plurality of layers.

[10. Hardware configuration]
Further, the terminal device 10 and the server device 100 according to the above-described embodiment are realized by, for example, a computer 1000 having a configuration as shown in FIG. Hereinafter, the server device 100 will be described as an example. FIG. 12 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to the output device 1010 and the input device 1020, and the arithmetic unit 1030, the primary storage device 1040, the secondary storage device 1050, the output I / F (Interface) 1060, the input I / F 1070, and the network I / F 1080 are bused. It has a form connected by 1090.

The arithmetic unit 1030 operates based on a program stored in the primary storage device 1040 or the secondary storage device 1050, a program read from the input device 1020, or the like, and executes various processes. The arithmetic unit 1030 is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used by the arithmetic unit 1030 for various calculations. Further, the secondary storage device 1050 is a storage device in which data used by the calculation device 1030 for various calculations and various databases are registered, and is a ROM (Read Only Memory), an HDD (Hard Disk Drive), and an SSD (Solid). State Drive), flash memory, etc. The secondary storage device 1050 may be an internal storage or an external storage. Further, the secondary storage device 1050 may be a removable storage medium such as a USB memory or an SD (Secure Digital) memory card. Further, the secondary storage device 1050 may be a cloud storage (online storage), NAS (Network Attached Storage), a file server, or the like.

The output I / F 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various information such as a display, a projector, and a printer. For example, USB (Universal Serial Bus) or DVI. (Digital Visual Interface), HDMI (Registered Trademark) (High Definition Multimedia Interface), etc. Further, the input I / F 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, a keypad, a button, a scanner, and the like, and is realized by, for example, USB.

Further, the output I / F 1060 and the input I / F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

Further, the output device 1010 and the input device 1020 may be integrated like a touch panel. In this case, the output I / F 1060 and the input I / F 1070 may also be integrated as input / output I / F.

The input device 1020 is, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like.

The network I / F 1080 receives data from another device via the network N and sends it to the arithmetic unit 1030, and also transmits the data generated by the arithmetic unit 1030 to the other device via the network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I / F 1060 and the input I / F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

For example, when the computer 1000 functions as the server device 100, the arithmetic unit 1030 of the computer 1000 realizes the function of the control unit 130 by executing the program loaded on the primary storage device 1040. Further, the arithmetic unit 1030 of the computer 1000 may load a program acquired from another device via the network I / F 1080 onto the primary storage device 1040 and execute the loaded program. Further, the arithmetic unit 1030 of the computer 1000 may cooperate with other devices via the network I / F 1080 to call the functions and data of the program from other programs of the other devices and use them.

[11. others〕
Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Further, the above-mentioned components include those that can be easily assumed by those skilled in the art, those that are substantially the same, that is, those in a so-called equal range. Furthermore, the components described above can be combined as appropriate. Further, various omissions, replacements or changes of the components can be made without departing from the gist of the above-described embodiment.

Further, among the processes described in the above-described embodiment, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or part of it can be done automatically by a known method. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown in the figure.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

For example, the above-mentioned server device 100 may be realized by a plurality of server computers, or may be realized by calling an external platform or the like by API (Application Programming Interface), network computing, or the like depending on the function. It can be changed flexibly.

Further, the above-described embodiments and modifications can be appropriately combined as long as the processing contents do not contradict each other.

Further, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

1 Information processing system 10 Terminal equipment 14 Positioning unit 30 Control unit 31 Reception unit 32 Acquisition unit 33 Processing unit 34 Learning unit 35 Estimating unit 36 Transmission unit 40 Storage unit 41 Model information storage unit 100 Server device 110 Communication unit 120 Storage unit 121 Model Information storage unit 122 Community information storage unit 130 Control unit 131 Providing unit 132 Acquisition unit 133 Learning unit 134 Judgment unit

Claims (10)

A server that provides a common model that generates a vector indicating the user's context from the data acquired by the terminal device, and
Using the data possessed by each, a plurality of terminal devices that generate a local model in which the parameters of the common model received from the server are updated and transmit update information that is information about the local model to the server, and a plurality of terminal devices.
Have,
The server
An information processing system characterized in that parameters of the common model are updated using the update information received from the plurality of terminal devices, and the updated common model is provided to the plurality of terminal devices. The plurality of terminal devices are
The information processing system according to claim 1, wherein the difference between the common model and the local model is transmitted to the server as the update information. The plurality of terminal devices are
The information processing system according to claim 1 or 2, wherein when the retained data has a defect, the missing data is subjected to predetermined processing to generate the local model. The plurality of terminal devices are
The information processing system according to claim 3, wherein the missing data is estimated, and the estimated local model is generated using the estimated estimated data. The common model is
The information processing system according to any one of claims 1 to 4, wherein the local model is generated using the data including the sensor information detected by the sensor. The plurality of terminal devices are
The information processing system according to any one of claims 1 to 5, wherein the local model is generated using the data including position information. The common model is
Output the vector with the data as input.
The information processing system according to any one of claims 1 to 6, wherein the information processing system is characterized by the above. The server
The common model, which is different for each of the plurality of first groups including the terminal device, is provided as the first layer common model, and the update information received from the plurality of first group terminal devices is used to provide the plurality of firsts. Each parameter of the one-layer common model is updated, and the updated parameters of the second-layer common model, which is a higher-level model of the plurality of first-layer common models, are used by using the updated first-layer common model. The information processing system according to any one of claims 1 to 7, wherein the information processing system is updated. The server
The parameters of the plurality of second-tier common models, each corresponding to each of the plurality of second groups including the first group, are updated, and the plurality of second-tier common models after the update are used. The information processing system according to claim 8, wherein the parameters of the third layer common model, which is a model of the upper layer of the layer common model, are updated. The server provides a common model that generates a vector indicating the user's context from the data acquired by the terminal device.
Using the data possessed by each of the plurality of terminal devices, a local model in which the parameters of the common model received from the server are updated is generated, and the update information which is the information about the local model is transmitted to the server.
Information characterized in that the server updates the parameters of the common model using the update information received from the plurality of terminal devices, and provides the updated common model to the plurality of terminal devices. Processing method.

Images