JP7550335B1 - system - Google Patents

Abstract

[Problem] The present invention aims to provide a means for verifying family and acquaintances, and a means for linking with the police for numbers with a history of misuse, in a system in which a generative AI answers calls from unregistered or unnotified phone numbers and documents and sends the message after the call ends.Solution: The system includes a response means in which a generative AI makes the initial response when a call is made from an unregistered or unnotified phone number, a sending means in which the message is documented in chat GPT after the call ends and sent via a messenger app or email, a confirmation means for verifying the call using a special question or password if the caller claims to be a family member or acquaintance, a connection means for the delivery company to connect to the person using a password for the delivery company, and a linking means for checking the phone number during the call and linking with the police if the number has a history of misuse. [Selected Figure] Figure 1

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description of the chatbot character and an associated instruction sentence, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

JP 2022-180282 A

The objective of this invention is to provide a means of identifying family and acquaintances and a means of coordinating with the police for numbers with a history of misuse in a system in which a generative AI answers calls from unregistered or withheld phone numbers and documents and transmits the message after the call ends.

The system includes a response means whereby a generative AI responds first when a call is received from an unregistered or withheld phone number, a sending means whereby the purpose of the call is documented in chat GPT after the call is ended and sent via a messenger app or email, a verification means whereby if the caller claims to be a family member or acquaintance, they are verified using special questions or passwords, a connection means whereby the delivery company connects to the person using a company password, and a linking means whereby the phone number is checked during the call and contacted by the police if the number has a history of misuse.

1 is a conceptual diagram showing an example of a configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. FIG. 11 is a conceptual diagram showing an example of a configuration of a data processing system according to a second embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 13 is a conceptual diagram showing an example of a configuration of a data processing system according to a third embodiment. FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped.

Below, an example of an embodiment of a system related to the technology disclosed herein is described with reference to the attached drawings.

First, let us explain the terminology used in the following explanation.

In the following embodiments, the signed processor (hereinafter simply referred to as the "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. The processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

In the following embodiments, a signed random access memory (RAM) is a memory in which information is temporarily stored and is used as a working memory by the processor.

In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) with a code is an interface including a communication processor and an antenna, etc. The communication I/F controls communication between multiple computers. Examples of communication standards applied to the communication I/F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same concept as "A and/or B" is also applied when three or more things are expressed by connecting them with "and/or."

[First embodiment]
FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with an indicator (e.g., a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form of expression that the user can perceive (e.g., voice and/or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. Data generation model 58 and emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the reception output process is performed by the processor 46. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a ChatGPT server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance). Next, the identification process by the identification processing unit 290 of the data processing device 12 will be described.

(Example 1)
The embodiment of the present invention is a system that includes a response means in which a generative AI responds to an incoming call from an unregistered or withheld phone number, and a sending means that documents the matter in chat GPT after the call ends and sends it via a messenger app or email. In addition, the system includes a confirmation means that uses a special question or password to confirm if the caller claims to be a family member or acquaintance, a connection means that connects the delivery company to the person using a password for the delivery company, and a linking means that checks the phone number during the call and links to the police if the number has a history of abuse.
The response means, the transmission means, the confirmation means, the connection means, and the linking means are realized, for example, by the specific processing unit 290 of the data processing device 12. In addition, some or all of these means may be realized, for example, by the control unit 46A of the smart device 14. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The response means includes a speech understanding function that utilizes voice recognition technology, and responds to calls from unregistered or withheld numbers in natural language generated by a pre-trained AI. This AI has learned a large amount of call data and has the ability to generate responses that can respond to various scenarios. When a call is received, the AI analyzes the content of the call in real time and selects an appropriate response to proceed with the conversation. The response content is customized to the nature of the call, so questions from sales calls are answered politely and suspicious calls are answered carefully. In addition, the response means can also use speech synthesis technology and speech content understanding technology to respond to calls from unregistered or withheld phone numbers, and a pre-trained AI can respond appropriately. This AI can learn multiple response scenarios, understand the intention of the call, and provide a natural dialogue according to the situation. The AI instantly selects a response based on the content of the call and responds customized to the purpose of the call, so inquiries from sales calls can be handled appropriately and suspicious calls can be answered carefully.
The transmission means includes voice recognition technology that converts the contents of the call into text and natural language generation technology that generates a summary of the call based on that text. When the call ends, the AI converts the contents of the call into text data and summarizes and documents the contents. The generated document is sent to the user via a communication means such as a messenger app or email. This process allows the user to check the details of the call at a later date and ensures that important information is not overlooked. The transmission means can also convert the contents of the completed call into text using voice recognition technology, then summarize it using natural language processing technology, and send the generated document to the user via a messenger app or email. The user can check the details of the call at any time through the sent document, and important information is recorded and will not be overlooked.
The verification means has the function of verifying the identity of the caller by using specific questions or passwords when the caller claims to be a family member or acquaintance. In this method, the AI uses questions and passwords set by the user, and verifies the identity of the caller by providing the correct response. If identity verification is successful, the AI continues the call and transfers the call to the user if necessary. This process prevents unauthorized access and fraud while ensuring smooth communication if the caller is the real person. The verification means can also have the AI present questions and passwords set by the user to callers claiming to be family members or acquaintances, and continue the call or transfer the call to the user only if an accurate answer is given. This function prevents unauthorized access and fraud, while enabling smooth communication if the caller is confirmed to be the real person.
The connection means has a function that allows a person in a specific occupation, such as a delivery worker, to talk to the person by using a secret code for the occupation. With this method, AI receives the secret code for the occupation, verifies that it is the correct code, and then connects the call to the person. The secret code for the occupation is important for ensuring the security of the call, and if an incorrect code is entered, the call will not be connected. The connection means can also have a function where AI receives a secret code specific to the occupation, such as a delivery worker, verifies its validity, and then connects the call to the user. The security of the call is ensured by this secret code, and if the incorrect code is entered, the call will be disconnected.
The linking means has a function of checking the phone number in real time during a call, and automatically linking to the police or related organizations if the number has a history of misuse. This means refers to a list of suspicious numbers stored in a database, and checks whether the incoming number is included in the list. If it is determined that the call is from a suspicious number, the system immediately starts a reporting protocol to the police, and takes appropriate action depending on the situation. The linking means can also monitor the number being called in real time, and automatically link to related organizations such as the police if the number has a history of misuse in the past. When a suspicious number is detected, a reporting protocol to the police is activated, and a prompt response is taken.
These measures are implemented by combining multiple AI technologies and collaboration functions to enhance user security and convenience. The data processing device and the control unit of the smart device cooperate to provide a platform for flexibly realizing these measures. Each measure is designed not only to function independently but also to work together to provide more advanced services. Through these measures, users can automate responses to calls from unregistered or unnotified phone numbers and protect themselves from potential risks that occur in daily life. In addition, these measures are built around AI technology to improve user security while simultaneously streamlining daily communication. As an alternative to sensor-free data collection, a process in which users manually enter information and the system uses that information is possible. Through these measures, users can automate responses to unregistered or unnotified calls and strengthen protection from unforeseen events.
Further functions can be added to the system of this embodiment to enhance its convenience. For example, the response means can be added with a function to analyze the user's emotions during a conversation, and the AI can infer the emotions from the tone of the other party's voice and manner of speaking and respond accordingly. This allows for a more careful and empathetic response when the other party is feeling angry or anxious, improving the quality of the call.
The sending method will add a feature that automatically integrates documented calls into users' calendars and reminders, helping them remember appointments and tasks generated during calls and improving productivity.
The verification method has a function to verify the identity of the user using biometric information. For example, biometric information obtained from a smartwatch or fitness tracker can be used to verify that the person on the other end of the line is actually a family member or acquaintance. This reduces the hassle for users while achieving a higher level of security.
The connection method will add a function that allows the delivery company to scan a QR code (registered trademark) or NFC tag to authenticate the user and connect the call directly to the user, eliminating the need for a password and making it possible to communicate with the delivery company more quickly and safely.
The collaboration will add a function to detect suspicious keywords from voice data during phone calls and automatically analyze the content. If a call is deemed suspicious based on the keywords, the system will first warn the user before reporting it to the police, helping them make more accurate decisions.
With these additional functions, the system of this embodiment not only automatically answers calls, but also more efficiently supports security and schedule management in daily life. In addition, by linking various AI technologies with databases, it provides greater security and convenience to the user's life. Furthermore, by integrating these technologies with the user's smart devices and IoT devices in the home, it becomes possible to realize a more personalized experience.
The system of this embodiment can be further expanded, and in order to increase convenience and security, the response means can be added with a function to analyze the caller's voiceprint and compare it with the voices of registered family members and acquaintances. This makes it possible to more accurately determine whether the caller is the real person and to continue the call safely without using specific questions or passwords.
The transmission means can incorporate a function that uses voice recognition and natural language processing to translate the contents of the call into multiple languages selected by the user, facilitating communication between users who speak different languages, facilitating use in international business situations and within multilingual households.
The verification method can include a feature that allows users to verify their identity by making specific gestures or movements on the camera, making it possible to easily and quickly verify the identity of a user during a call, further enhancing security.
Regarding the means of connection, AI will be able to add a function to verify the identity of the caller by comparing their location information with delivery schedule information, making it possible to confirm that the delivery company is calling from a location where the goods are actually being delivered, thereby guaranteeing the authenticity of the call.
The collaboration method will add a function that automatically notifies not only the police but also the user's designated emergency contact when a suspicious call is detected, enabling a rapid response in the event of an emergency.
These additional functions allow for flexible response to user needs while maintaining the basic structure of the existing system. In addition, by linking with smart devices and IoT devices in the home, the system provides a seamless experience in the environment in which the user uses the system on a daily basis. With these additional functions, the system of this embodiment goes beyond automatic answering of calls, improving the security and convenience of users in all aspects of their daily lives.
(Example 2)
The embodiment of the present invention is a system that has a sending means that can send the matter to family members when documenting the matter in chat GPT after the call ends and sending it by messenger app or email. Specifically, the matter can be sent to family members by selecting the document created after the call ends and specifying "family members" as the destination.
The transmission means is realized, for example, by the specific processing unit 290 of the data processing device 12. The transmission means may also be realized, for example, by the control unit 46A of the smart device 14. The correspondence between each means and the device or control unit is not limited to the above example, and various modifications are possible.
The means for documenting the contents after the end of the call includes a conversation content analysis means using a voice recognition technology. This analysis means converts the voice data of the call into text and extracts the main points of the call. In addition, the conversation content analysis means using the voice recognition technology has a function of converting the voice data generated after the end of the call into text and extracting the main points of the call from the text data. The voice recognition technology utilizes a model based on deep learning and is trained to support various languages and accents. In addition, in order to support various languages and accents, this analysis means utilizes a deep learning model and proceeds with learning based on a large amount of voice data. The conversation content analysis means performs morphological analysis and syntactic analysis to identify important information exchanged in the conversation and content requiring action. In particular, a keyword extraction function is used to identify words and phrases frequently used in the conversation, and the main points of the call are documented based on these. In addition, in order to clarify the main points of the conversation, a morphological analysis function and a syntactic analysis function are combined to identify important information exchanged during the conversation and content requiring action. The keyword extraction function identifies words and phrases that appear frequently in the conversation, and documents based on this information.
The means for formatting the documented content into a message includes a text editor function, which organizes the parsed text and arranges the document structure. The message formatting means provides an interface that allows a user to easily check the content and add edits or additional information as necessary. The means for formatting the documented content into a message also includes a text editor function, which organizes the parsed text and arranges the document layout. The means for formatting the message provides an interface that allows a user to check the content of the message and add edits or additional information.
The sending means sends a message to a specific family member using a destination designation function. This destination designation function works in conjunction with the user's contact list to automatically send a message based on the contact information of the selected family member. The sending means also has a function of confirming message transmission and managing a transmission history, so that the user can track the status of the sent message and resend it as necessary. The message is sent in conjunction with a messenger app or an email app. The linking function with the messenger app or the email app performs authentication using the user's account information and transmits the message safely. The sending means also encrypts the message as a security measure and performs an authentication process at the time of transmission to ensure privacy protection. The sending means also transmits the message in an appropriate format depending on the message transmission method selected by the user. For example, the message is sent as an instant message in the messenger app, and as an email in the email app. The message sending means has a destination designation function for sending a message to a family member selected by the user, and works in conjunction with the user's contact list to automatically send a message based on the contact information of the family member. The sending means also includes a function for checking the sending status of a message and managing the sending history, so that the user can track the status of the sent message and resend it if necessary. The message sending process has a function for linking with a messenger application or an email application, authenticating the user based on the account information, and sending the message securely. The sending means protects privacy by encrypting the contents of the message and implementing security measures such as authentication at the time of sending. In addition, the message is sent in an appropriate format according to the sending method selected by the user, such as an instant message in the messenger application or an email in the email application.
An example of data collection that does not include sensors is when a user manually takes notes on a phone call. In this case, the user records the main points of the call after the call ends, and sends the text data to the family as a message. The call contents manually recorded by the user are organized using a text editor function and sent to the family as a formatted message by the sending means. This manual recording is suitable for situations where automatic documentation using voice recognition technology cannot be applied, or when the user wants to convey specific information in his or her own words. Another possible method of data collection that does not include sensors is a scenario in which a user manually takes notes on a phone call after the call ends, and sends the information to the family as a message. This manual recording is organized using a text editor function and sent as a formatted message. This is used in situations where voice recognition technology cannot be applied, or when the user wants to convey specific information in his or her own words.
In an embodiment of the present invention, functions beyond documenting the contents of a call can be considered. For example, a function can be added that automatically registers appointments and events mentioned during a call in a calendar in cooperation with a schedule management system based on the text generated by speech recognition. This can save the user the trouble of manually managing the schedule after the call. In addition, it can suggest adding events to a calendar shared among family members, making it easier for all family members to share the schedule.
It is also possible to automatically generate task lists based on documented text data and post them to a shared platform that can be accessed by all family members, where each family member can update the progress of tasks and check off completed tasks, creating an environment for the whole family to share information and cooperate.
In addition, the company will add a feature that performs sentiment analysis on written messages and reflects the emotional nuances expressed during a call in the text, helping message recipients to more accurately understand the caller's intent. For example, if emotions such as joy or anxiety are expressed during a call, those emotions will be expressed in the text with specific emojis and formats, enhancing the richness of communication.
Another possible function would be to use voice recognition and analysis to automatically generate FAQs and a list of frequently asked questions from the content of calls, building a knowledge base that family members can refer to when they have similar inquiries. This knowledge base would be shared among family members and updated every time a new call occurs, improving the efficiency of communication between family members.
In addition, the text generated after the call ends can be sent not only via messenger apps or email, but also played back in audio format, making it easier for visually impaired family members or children with difficulty reading and writing to receive the information.
Finally, to protect the privacy of family members, the documented content after the call is completed can be automatically anonymized or masked to identify sensitive information in the document, providing an environment in which information can be shared with peace of mind. This allows for a balance between respecting each individual's privacy and sharing only the information that is necessary.
The embodiment of the present invention relates to automatic documentation and transmission of call contents, and new functions can be added to the document. For example, the document can be analyzed, and action items can be automatically generated after the call ends, and a reminder can be set on the user's smart device as a task that needs to be addressed. The reminder can be prioritized based on deadlines and importance mentioned in the call, helping the user to take action without forgetting.
Additionally, to enhance communication among family members, it would be beneficial to add the ability to automatically attach relevant images, videos, and links based on specific words or phrases in a written message, allowing for visual information to be shared in addition to text-based messages, making communication richer.
In addition, the system can enhance security by automatically detecting and blurring certain personal or confidential information when documenting call transcripts, in accordance with privacy protection guidelines and natural language processing techniques.
When converting call contents into text, it is also effective to incorporate a translation function into multiple languages to meet the diverse needs of users. In a multicultural environment where family members speak different languages, it is possible to automatically translate the call contents and send messages in a language that each member can easily understand.
To further enhance convenience, a schedule function will be added that allows users to customize the timing of message sending. Users can set messages to be sent at a specific date and time, rather than just immediately, allowing them to optimize the timing at which their family members receive information.
Finally, quick action buttons for replying or confirming messages have been provided so that recipients can easily take action on received messages, enabling quick feedback and efficient communication, making information sharing between family members even smoother.
(Example 3)
The embodiment of the present invention is a system that checks phone numbers during a call, and has a means for linking with the police when linking with the police if the number has a history of misuse. Specifically, the system is equipped with a function that checks the incoming number against a database during a call, and automatically notifies the police if it detects that the number has a history of misuse.
The linking means is realized, for example, by the specific processing unit 290 of the data processing device 12. The linking means may also be realized, for example, by the control unit 46A of the smart device 14. The correspondence between each means and the device or control unit is not limited to the above example, and various modifications are possible.
The extraction means is provided for extracting caller information contained in the signal of each call sent through the communication network, and can accurately obtain the caller's telephone number from the call data using digital signal processing technology. The extraction means also has a signal analysis function for analyzing Caller ID information and identifying the telephone number. The system can also accurately identify the called number by utilizing the signal extraction function for extracting the caller's telephone number from the call signal and an analysis algorithm specialized in decoding Caller ID information.
The matching means has a function of matching the extracted telephone number with a database that collects telephone numbers suspected of fraudulent use and organizes them by category, and the matching process can be performed in real time through a database management system, and rapid matching can be performed while the call is in progress using high-speed search algorithms and indexing technology.The matching means also has a high-performance database search function and indexing function for rapid search and matching by referring to a database that lists numbers with a history of misuse.
The linking means has a function of automatically reporting to the police when a number with a history of abuse is detected, includes an interface function with the reporting system, generates report information in a data format based on the protocol with the police reception system when reporting, and transmits it to the police reception system using a secure communication channel. The contents of the report information include information such as the detected abused phone number, the date and time of the call, the duration of the call, and the telecommunications carrier used by the caller, and uses encryption technology and authentication systems to protect personal information and ensure the accuracy of the report. In addition, the linking means is compatible with multiple communication protocols and reporting systems, has an adapter function for smooth data exchange between the systems, and has a function of adjusting the format of the report information to meet the requirements of the police reception system in the reporting process, and selecting an appropriate reporting protocol to make the report. When the reporting process is activated, the system transmits report information to the police reception system and obtains a report reception confirmation. This confirmation is used as information for the system to record that the report was made correctly and store it as a report history. The report history is used for future analysis and improvement, contributing to the efficiency and accuracy of the report process. This linking function uses a report data generation function to create a report format containing the necessary information, and transmits the information to the police reception system via a secure report transmission function. Data encryption and system authentication functions are implemented to ensure the security and reliability of reports. In addition, the system's linking means has an adapter function that is compatible with multiple report protocols and enables data exchange with different report systems, and this adapter function automatically adjusts to a format suitable for the protocol requirements at the time of reporting through a report protocol selection function, and transmits report information and confirms receipt. The report history recording function records the success of reports, which is used to analyze and improve system performance.
As an example of data collection without sensors, a function for users to report suspicious calls on an application or web interface is provided. The user enters their experience of the call and any suspicious points they noticed during the call into a form, and the information is registered in a database. The data collected through this manual report can be added to a list of phone numbers that are automatically matched in the database, contributing to improved accuracy in detecting numbers with a history of abuse. As an example of data collection without sensors, an input function is provided for users to report suspicious calls. Through an interactive reporting form, users register the contents of suspicious calls in a database, and the information collected is used to detect numbers with a history of abuse. This manual reporting system contributes to the expansion of the number matching database by providing additional data based on the user's experience and intuition.
The system can be equipped with a function to strengthen the database update mechanism. For example, newly confirmed abused numbers will be automatically added to the database even after they are reported. In addition, when a suspicious call is reported, the credit information of the number will be checked against other databases and combined with information based on user reports to more accurately identify abuse history. To ensure the integrity of the database, a regular cleaning process will be carried out to eliminate incorrect and outdated information. In addition, to strengthen cooperation with the reporting system, a function will be introduced to directly link with the crime database provided by the police, obtain crime information in real time during the matching process, and improve the accuracy of the matching results.
To improve the immediacy of reports, the matching process begins the moment the call is initiated, and if a number with a history of abuse is detected, the caller will be given the option to sound an alarm or automatically hang up. In addition, when the call is hung up, a recording of the call can be saved on the caller's behalf to assist police investigations. When police intervene, a detailed report including the caller's location and call history is automatically generated, helping to speed up criminal investigations.
The user interface will be updated to include a feature that allows users to check the progress of the reporting process in real time to increase transparency in the reporting system. Users will be able to check the results of their reports and feedback from the police, which will increase trust in the system. The system will also provide statistical data and trend analysis on abused numbers, providing information to users to be cautious about calls.
Furthermore, machine learning technology will be introduced to identify numbers with a history of abuse, and new numbers likely to be abused will be predicted by analyzing various indicators such as call patterns and call frequency. This will enable proactive updates to the database, strengthening measures to prevent unknown criminal activity. A function will also be added that allows users to provide direct feedback on the effectiveness of the reporting system, which will help improve the system. Feedback will be provided anonymously, protecting user privacy while collecting valuable information that will contribute to improving the system.
To make the phone number check during a call more effective, AI will learn the patterns of various fraudulent scams that users may face and implement a function to flag in real time when certain words or phrases are detected during a call. This will allow malicious behavior to be inferred and detected from the content of the call, rather than simply matching the phone number with abuse history in the database. In addition, the reporting process will be simplified by providing shortcuts and buttons on the smartphone interface that allow users to easily report calls that they suspect are fraudulent. This will allow the database to be updated more quickly and improve protection for other users.
To strengthen cooperation with the police, the reporting system will be integrated with location tracking capabilities to help police respond quickly based on reported information, helping them track and capture criminals. Artificial intelligence will also be built into the reporting system to analyze crime patterns from report data and provide police with information to plan preventive vigilance activities. Using such predictive analytics will help prevent future crimes.
To facilitate data sharing with the police, the system will receive real-time information on fraud cases and other crimes known to the police and update the database. This will ensure that the reporting system operates based on the latest crime information and strengthen measures to protect users. In addition, the system will be equipped with a block list function, allowing users to register suspicious numbers and block calls from them, allowing users to directly control the risks themselves.
Educational programs will include online courses and workshops to provide users with information to recognize and prevent fraud methods. This will provide users with the knowledge to protect themselves and raise security awareness in society as a whole. Examples of fraud cases that were prevented by using the reporting system will also be shared to help users understand the effectiveness of the system.
Finally, through a system update, if a call is deemed likely to be fraudulent, a warning message will be automatically sent to the user to warn them against fraud. This will allow users to immediately recognize the possibility of fraud and take appropriate action.

The process flow for each example is explained below.

(Example 1)
Step 1: When a call comes in from an unregistered or withheld phone number, the generative AI answers.
Step 2: After the call, document the matters discussed in Chat GPT.
Step 3: Send a written message via messenger app or email. If the caller claims to be a family member or acquaintance, verify the call using special questions or a secret code. The delivery company will connect you using a secret code. Check the phone number during the call, and if the number has a history of abuse, contact the police.
(Example 2)
Step 1: After the call, document the matters you wish to discuss in Chat GPT.
Step 2: Select the documented matter and designate it as being sent to family members.
Step 3: Send your message to your family via messenger app or email.
(Example 3)
Step 1: During a call, the incoming number is checked against a database to detect if it is a number with a history of abuse.
Step 2: If the number has a history of abuse, the call will automatically be reported to the police.
Step 3: After the call is over, the matter is documented in the chat GPT. The documented matter is used for communication with the police.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

(Example 1)
The embodiment of the present invention is a system that includes a response means in which a generative AI responds when a call is received from an unregistered or unnotified phone number, and a transmission means in which the matter is documented in chat GPT after the call is ended and sent via a messenger app or email. Furthermore, as an emotion recognition means that combines an emotion engine that recognizes the user's emotions, the tone of the user's voice and choice of words are analyzed during the call to estimate the emotion. Based on the estimated emotion, the generative AI's response and the expression of the documented matter are adjusted. For example, if the user shows an anxious emotion, a calmer expression is used to provide a sense of security.
The response means and the transmission means are realized, for example, by the specific processing unit 290 of the data processing device 12. The emotion recognition means detects the tone of the user's voice using, for example, the microphone 38B of the smart device 14, analyzes it using the control unit 46A, and estimates the emotion using the specific processing unit 290 of the data processing device 12. In addition, a part or all of the response means, the transmission means, and the emotion recognition means may be realized, for example, by the control unit 46A of the smart device 14. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The response means includes a dialogue generation means that combines natural language processing and speech synthesis. The dialogue generation means automatically responds to unregistered or unnamed incoming calls, enabling a dialogue with the user. When an unregistered or unnamed incoming call is detected, the generative AI responds using a pre-trained conversation model. This conversation model is trained based on a variety of dialogue data so that it can handle various scenarios, and provides appropriate answers and guidance to the callee's questions and requests. In addition, the response means has an AI dialogue generation function, and can automatically provide an appropriate response to an incoming call from an unregistered or unnamed phone number. This function combines a dialogue management function for understanding the callee's purpose and request at the early stage of the call and providing a corresponding response, and a voice generation function for converting the generated response into a natural voice. The dialogue management function can analyze the callee's intention based on the detection of specific keywords and phrases and generate an appropriate response. The voice generation function can convert a text-based response into voice in real time, providing the callee with a natural conversation experience.
The transmission means includes a chatbot and a documentation means using natural language understanding technology. After the call is ended, the contents of the call are accurately documented using an advanced natural language understanding model such as chat GPT. The documentation means has the ability to extract and summarize the main points of the call, and can generate a document that conveys the purpose concisely and clearly. The generated document is sent to the user via a messenger app or email. This process includes a link to the user's email address or messenger account, and the document is automatically sent in the appropriate format. The transmission means can also convert the contents of the call into text and provide it in an accessible form to the user. After the call is ended, the call content documentation function is activated and can extract and summarize the main points of the conversation. This summarized text is sent to the user's designated email address or messenger app through an automatic delivery function. This automatic delivery function includes an email delivery function and an app link function to properly format the document and ensure that it is delivered to the designated destination.
The emotion recognition means includes a voiceprint analysis means for performing voice analysis, and a language analysis means for estimating emotions from the choice of words. The voiceprint analysis means can detect characteristics of the user's voice, such as tone, pitch, and speed, using the microphone of the smart device, and can estimate the user's emotional state from these voice characteristics. The language analysis means can analyze the content of the user's speech and extract emotional context from the words and phrases used. These analysis results are used by the generative AI when making a response or adjusting the expression of documented matters. For example, if the user shows anxiety, the expression of the response or document is adjusted to be more gentle and reassuring. In addition, the emotion recognition means has a function of estimating emotions from the characteristics of the user's voice and language use during a call. The voiceprint analysis function can extract characteristics such as the pitch, strength, and speed of the voice from the voice data collected by the microphone, and can estimate emotions by analyzing these characteristics. The language emotion analysis function can process the language data during a call, analyze the emotional meaning of the words and phrases used, and grasp the user's emotional state. These analytics can be used to adjust the tone of the AI's responses and the wording of documented calls to provide an appropriate emotional response to the user.
Data collection methods that do not involve sensors include text data entered by users themselves and feedback on system usage. These are provided to the system through the user input acceptance function and feedback collection function, and are used as a valuable source of information for service improvement.
These means are intended to respond quickly and effectively to user requests and improve the quality of communication. In addition, the implementation of each means is flexibly performed by the data processing device and the control unit of the smart device, and can be modified in various ways to improve the efficiency and usability of the system.
The system of the present invention can be equipped with an additional function of a summary prediction means for predicting the caller's intention before answering an unregistered or unnamed call. This allows the response means to generate a more accurate dialogue, realizing a meaningful exchange for the user. In addition, when the generative AI responds, it can have a language selection function based on the caller's country or region, enabling automatic response in multiple languages.
As for the delivery method, in addition to documenting the contents of the call, it can also highlight important keywords and phrases to help users quickly understand the document, and automatically generate action items based on the documented content and add them to the user's task list.
The emotion recognition means can detect changes in the user's emotions during a call in real time and dynamically adjust the tone and tempo of the conversation of the response means. Protocols can also be built in that, when a particular emotion is detected, prompt the user to contact an expert who can provide special support or advice according to the emotion.
The response tool can be enhanced with a personalization feature that analyzes the caller's past call history and related data to provide a more personalized response, making it possible to provide more relevant information to the user and increase the usefulness of the response.
Regarding sending methods, a function can be added that suggests follow-up actions based on the documented content of the call. For example, tasks and events included in the call content can be automatically registered in a calendar app to help users manage their time.
Furthermore, the emotion recognition means may be capable of detecting the user's stress level or tension and providing appropriate advice on how to reduce stress or links to relaxation content, thereby supporting the user's mental health and promoting overall well-being.
The system of this embodiment is considered to have a number of additional functions for further improving its functionality. For example, a voiceprint recognition means can be added to identify a caller by analyzing the caller's voiceprint and comparing it with previous call data for an unregistered or unnamed call. This allows the response means to respond more appropriately based on information if the caller has previously interacted with the system. The voiceprint recognition means also functions as a security measure, providing a reliable call experience for the user.
Regarding the means of transmission, when documenting the contents of the call, the function of structuring the contents of the call and classifying the text according to the importance of the information will be considered. This will allow users to grasp important information more quickly when reading the document. In addition, the documented content can be customized according to the user's preferences and past behavior patterns to provide a more personal experience.
The emotion recognition means detects the user's stress level during a call, and if it is estimated that the user is under high stress, it provides guidance on relaxation methods and psychological support related to the content of the call, thereby providing a service that allows users to relax and reduce stress through the call.
Additionally, the response tool will automatically suggest follow-up actions to users based on the content of the call, for example providing links to additional information about products or services suggested during the call or suggesting next steps of action to help users make decisions.
To improve response methods, we are considering a function that automatically changes the response style according to the caller's intentions. For example, if the caller indicates an emergency situation, we will adjust the AI to provide quick and accurate instructions. This will create a system that can immediately respond to the caller's needs.
These additional functions are intended to improve the user experience and enable accurate understanding of call content and rapid response. Each function is expected to maximize the capabilities of data processing devices and smart device control units, further enhancing the usability of the system.
(Example 2)
The embodiment of the present invention is a system that includes an adjustment means for documenting the matter in chat GPT after the call ends, and adjusting the expression of the document using an emotion engine that recognizes the user's emotions when sending it by messenger app or email. Specifically, the documented matter is input to the emotion engine, and the user's emotions are analyzed. Based on the analysis results, the expression of the document is appropriately adjusted. For example, if the user is feeling happy or excited, the user's emotions are shared by using brighter and more lively expressions.
The adjustment means is realized, for example, by the specific processing unit 290 of the data processing device 12. In addition, a part or all of the adjustment means may be realized, for example, by the control unit 46A of the smart device 14. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The conversation content extraction means converts the contents of the call into text data by making full use of voice recognition technology. After receiving the voice signal, the conversation content extraction means uses noise reduction means to remove background noise and improve the accuracy of the voice-to-text conversion. Furthermore, the conversation content extraction means uses cutting-edge voice recognition technology to accurately convert the contents of the call into text data, and in the process, the noise reduction means removes background noise and improves the conversion accuracy.
The text processing means corrects syntactic errors in the converted text data and performs grammar checking via the grammar checking means to maintain linguistic fluency. The text processing means also performs adjustments to ensure that the grammar checking means of the converted text data ensures accuracy of grammar and maintains the natural flow of the document.
The emotion recognition means includes an emotion extraction means based on text mining and emotion analysis techniques, which estimates the user's emotion from the wording and context of the generated text data. The emotion extraction means also identifies words, phrases, and grammatical patterns that express various emotions and classifies them into emotion categories such as positive, negative, and neutral. The emotion emphasis means also adjusts the tone and phrasing of the text according to the extracted emotions, making modifications to better convey the user's emotional state. The emotion extraction means also reads the user's emotion from the language patterns expressed in the document and adjusts the tone of the document based on the emotion.
The adjustment means changes the expression of the text based on the emotion data analyzed by the emotion recognition means. When a positive emotion such as joy or excitement is detected, the expression enhancement means replaces the vocabulary used with brighter and more lively words to give the message a favorable impression. When a negative emotion such as sadness or anger is detected, the expression mitigation means softens the tone of the message and selects phrases that show empathy and understanding. When the user's emotion is positive, the expression enhancement means energizes the document, and when the user's emotion shows negative emotion, the expression mitigation means uses a more gentle expression.
The message sending means includes a function for linking with a messenger application or an email client and sends the adjusted text in an appropriate format. Furthermore, the transmission protocol selection means selects an optimal transmission protocol according to the platform and settings of the recipient and ensures delivery of the message. Furthermore, the user interface presentation means provides a preview screen for a final review of the document before sending, allowing the user to make final corrections as necessary. Furthermore, the message sending means sends the adjusted text in a format suitable for the messenger application or the email client, and the preview screen provides an interface for the user to make a final check of the document.
The above processes are realized by a specific processing unit built into the control unit of the smart device or the data processing device, depending on the device and settings used by the user. Moreover, these means are designed as modularized components, and can be easily replaced or expanded as system components. Furthermore, the correspondence between each means is set flexibly, and various changes can be made to accommodate system upgrades and customization. Moreover, this series of processes is modularized so that it can flexibly respond to devices and environments, and is designed to facilitate system upgrades and customization.
This embodiment can be further expanded to add a function for deeper understanding of the user's emotions and improving the quality of communication. For example, by incorporating a facial expression recognition function during video chat into the emotion engine, emotions can be analyzed from visual information as well, resulting in more accurate emotion judgment. To improve the accuracy of emotion recognition, the tone and pitch of the user's voice can also be analyzed and reflected in the text. Furthermore, by analyzing the user's past communication history and response patterns, the system can learn the individual's emotional expression style and adjust the text accordingly in a more personalized manner.
The text processing means has a function to suggest phrases inspired by literary works and poetry to enhance the diversity and creativity of expressions, allowing users to express their emotions more richly. It can also select appropriate expressions according to the environment and situation in which the communication takes place, taking into account the social context and cultural background.
The messaging tool will add a function to predict the emotional impact of the text sent on the recipient, allowing users to communicate more responsibly. In addition, AI can predict the recipient's reaction and use that information to suggest the next communication strategy the user should take.
Overall, these features will help users to reflect emotions accurately and sensitively in their communications, contributing to building deeper human relationships. These advanced methods can be effectively used by individuals as well as corporate customer support and CRM systems to deepen customer relationships. Furthermore, these systems are expected to be applied in fields where human emotions play an important role, such as user education and counseling.
The system leverages an emotion engine to provide document expression adjustments based on the user's emotions. To extend this functionality, sensors are integrated to capture the user's biometric information, allowing for a more accurate reading of emotions from physiological responses such as heart rate and skin conductivity. Data from the sensors is analyzed in real time, allowing the tone of the document to be adjusted on the fly.
In addition, it is equipped with a personalized learning function that continuously analyzes the user's daily communication to understand the user's unique expression style and preferences, allowing the system to suggest more natural and personalized documents that reflect the user's personality.
It will also enhance multilingual support, allowing it to capture emotional nuances in different languages, enhancing international communication and understanding among multilingual users.
To protect user privacy, the company will also add features to anonymize and encrypt emotion data and strengthen security, allowing users to use the system with peace of mind.
Aiming for applications in the fields of education and counseling, we will develop a function that uses the results of emotion recognition to support communication skills training. The training program will include practicing emotional expression and learning appropriate communication techniques.
Finally, to improve the usability of the system, we make the user interface rich and intuitive, supporting a variety of gestures and voice commands, and allowing users to easily intervene in the document adjustment process, allowing users to fine-tune the expression at their own will and achieve more personal communication.
(Example 3)
The embodiment of the present invention is a system that includes an emotion engine that recognizes the user's emotions during a call and an adjustment means that adjusts the response and dialogue content to match the user's emotions. Specifically, the user's tone of voice and choice of words are input to the emotion engine during a call to estimate the user's emotions. The generative AI's responses and dialogue content are adjusted based on the estimated emotions. For example, if the user shows sad emotions, words of empathy are used to provide encouragement and support.
The adjustment means detects the tone of the user's voice using the microphone 38B of the smart device 14, analyzes the tone using the control unit 46A, and estimates the emotion using the specific processing unit 290 of the data processing device 12. In addition, a part or all of the adjustment means may be realized by, for example, the specific processing unit 290 of the data processing device 12. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The voice data collection means includes a microphone with high sensitivity and noise canceling function. The microphone is equipped with a digital signal processing algorithm to remove ambient noise and clearly record the user's voice. The voice data collection means also employs a highly sensitive microphone that captures subtle acoustic characteristics from the user's speech and combines this with digital signal processing technology to effectively remove ambient noise. The microphone accurately captures the characteristics of the user's voice and provides basic data for detecting changes in emotions.
The voice feature extraction means extracts features that may reflect human emotions from the voice signal. This extraction means includes a spectrogram analysis function and a pitch tracking function for performing acoustic feature analysis, and an emotion identification function using an acoustic model. The voice feature extraction means also uses acoustic analysis tools such as a spectrum analysis function and a pitch analysis function to extract features that suggest emotions from the voice signal, and supplies these data to the emotion estimation model.
The emotion analysis means includes an emotion estimation model based on machine learning, which estimates the user's emotional state using the features extracted by the voice feature extraction means as input. The emotion estimation model is composed of classifiers such as neural networks, support vector machines, and decision trees trained based on training data, and classifies the user's emotions into categories such as positive, negative, and neutral. The emotion estimation model improves its accuracy through continuous learning, and evolves to be able to respond to subtle changes in the user's perception of emotions. In addition, the emotion analysis means has an emotion estimation model that utilizes machine learning technology, and analyzes the user's emotions based on the extracted voice features. This model combines various machine learning algorithms to identify emotion categories from the user's utterance, and estimates the user's emotional state based on this. The estimated emotional state is important information for adjusting the system's response to the user's utterance and behavior.
The dialogue adjustment means includes a response generation function using a generative AI model, which dynamically adjusts the content of the dialogue according to the emotion estimated by the emotion analysis means. The response generation function utilizes natural language generation technology to generate a response sentence using words and tones appropriate for the user's emotion. For example, if the user is expressing sad emotion, a response including empathetic expressions and comforting words is generated. This response generation function has a context-aware processing function for taking into account the context of the conversation and providing content that matches the user's emotion and the purpose of the communication. In addition, the generated response is constructed to be natural to the user and to meet the emotional needs. The response generation function is realized by a machine learning model trained on a large-scale conversation dataset and can adapt to the user's language and speaking style. In addition, the dialogue adjustment means includes a response generation function using a generative AI model, which generates dialogue content appropriately corresponding to the estimated emotion. This function selects words and a dialogue tone that correspond to the user's emotion and provides a response that matches the user's current emotion and the context of the conversation. The response generation function is designed to perform context-aware processing and provide the information and support required by the user in an appropriate form. This feature is capable of generating responses that adapt to a user's language and emotions based on models trained from large amounts of conversational data.
Non-sensor-based data collection methods include text information entered directly by users into the system and metadata obtained from call records. These are used as supplementary information when analyzing user behavioral patterns and preferences, and contribute to improving the accuracy of the emotion engine and optimizing response generation functions. User-entered text information is also used as training data for emotion estimation models as part of the emotion analysis method.
The system has the ability to analyze the user's emotions in real time during a call and automatically adjust the dialogue content, so it can integrate facial expression recognition technology in addition to voice data to capture even more subtle changes in emotions. It uses a webcam or a camera on a smart device to analyze the user's facial expressions and improve the accuracy of emotion analysis. This added facial expression recognition function will more accurately recognize the user's emotions and make it possible to adjust the dialogue according to even more subtle changes in emotions.
It is also conceivable to incorporate wearable devices to capture the user's physiological signals. Measuring physiological responses such as heart rate and electrodermal activity can provide deeper insight into emotions that cannot be captured by tone of voice or facial expressions alone. Integrating this data can further improve the accuracy of emotion analysis and generate more appropriate dialogue responses.
It would also be effective to add customization features to the dialogue adjustment method that take into account the user's cultural background and personal values. A user profile can be constructed and the tone and content of the dialogue can be further personalized based on that information. This makes it possible to provide detailed support tailored to each individual user.
Furthermore, in order to promote the evolution of the emotion estimation model, we will build a mechanism to collect emotion data through crowdsourcing and incorporate feedback from a variety of users to continuously update the model, resulting in a flexible system that can handle a variety of emotional expressions and languages.
Applications in the fields of education and mental health care can also be considered. For example, in the field of education, it could be used to present educational materials adapted to students' emotions or in counseling sessions. In mental health care, it could recognize the user's emotions and provide dialogue support to reduce stress and anxiety. This would enable a more effective approach to the problems the user is facing.
Regarding privacy protection of the system, we will strengthen security measures to safely store users' emotional data and minimize the risk of information leakage by using appropriate access control and encryption technology. This will allow users to use the system with peace of mind.
Ultimately, it is hoped that the personalized interaction experience provided by this system will be developed into services that improve the quality of users' lives.
The embodiment of the present invention is applicable not only to telephone calls but also to video conferencing and online education platforms. For example, a lecturer can grasp the emotions of students in real time and adjust the progress of the curriculum according to the emotions, thereby providing a more effective learning experience. Even in video conferencing, dialogue management that reflects the emotions of participants is performed, promoting a productive and positive meeting environment.
The system could also be enhanced to monitor health conditions based on emotional responses: for example, if a user's tone of voice or manner of speaking indicates negative emotions over a period of time, a mental health professional could be notified to intervene if necessary.
Furthermore, to further improve the emotion engine, a function will be incorporated that will analyze the emotional patterns of the user's daily life and provide advice on long-term emotion management and stress reduction based on that information. By analyzing the user's daily rhythm and activity patterns and predicting emotional ups and downs, the system will suggest content for relaxation and motivation at appropriate times.
This system is also expected to be applied in the field of customer support. For example, if call center operators could grasp the customer's emotions in real time and detect negative emotions such as dissatisfaction or anger, they could take immediate action to address the issue, contributing to improving customer satisfaction.
In the fields of games and entertainment, dynamic changes in content based on the user's emotions are expected to have the effect of increasing immersion and enjoyment. In-game characters will react to the player's emotions, changing the story development and dialogue, creating a more personalized experience.
In addition, we will integrate voice assistants and virtual reality (VR) to realize more natural dialogue based on the user's emotions. Voice assistants will understand the user's emotions and provide information and services tailored to individual needs. In the VR environment, the scenario and environment will change according to the user's emotions, providing an experience that matches the emotions in real time.
This system also has the potential to revolutionize the design of user interfaces (UI) and user experiences (UX). Using emotion recognition technology, it can provide UI and UX optimized for the user's emotions, increasing user satisfaction. For example, websites and applications can grasp the user's emotions in real time and adjust the way content is presented and the form of interaction.
Ultimately, it is hoped that the emotion regulation function provided by this system will become widely used throughout society as a tool to improve the quality of human relationships and increase the effectiveness of communication.

The process flow for each example is explained below.

(Example 1)
Step 1: When a call comes in from an unregistered or withheld phone number, the generative AI answers.
Step 2: After the call, document the matters discussed in Chat GPT.
Step 3: Input the documented requirements into the emotion engine to analyze the user's emotions.
Step 4: Based on the analysis results, the representation of the document is adjusted.
Step 5: Send the adjusted document via messenger app or email.
(Example 2)
Step 1: After the call, document the matters you wish to discuss in Chat GPT.
Step 2: Input the documented requirements into the emotion engine to analyze the user's emotions.
Step 3: Adjust the representation of the document based on the analysis results.
Step 4: Select the adjusted document and specify the recipient (e.g., family).
Step 5: Send the document to the specified recipient via messenger app or email.
(Example 3)
Step 1: During a call, the user's tone of voice, choice of words, etc. are input into the emotion engine to analyze the user's emotions.
Step 2: Based on the analysis results, adjust the generative AI's responses and dialogue content.
Step 3: After the call, document the matter in the chat GPT. The documented matter is used to input the emotion engine.

The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input for the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization. The identification processing unit 290 performs the above-mentioned identification processing while using the data generation model 58.

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart device 14.

[Second embodiment]

Figure 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical able-bodied person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is performed in a secure state.

Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. Data generation model 58 and emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a ChatGPT server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance). Next, the identification process by the identification processing unit 290 of the data processing device 12 will be described.

(Example 1)
The embodiment of the present invention is a system that includes a response means in which a generative AI responds to an incoming call from an unregistered or withheld phone number, and a sending means that documents the matter in chat GPT after the call ends and sends it via a messenger app or email. In addition, the system includes a confirmation means that uses a special question or password to confirm if the caller claims to be a family member or acquaintance, a connection means that connects the delivery company to the person using a password for the delivery company, and a linking means that checks the phone number during the call and links to the police if the number has a history of abuse.
The response means is realized by the specific processing unit 290 of the data processing device 12, and responds to an incoming call from an unregistered or unnotified phone number using a generative AI. The transmission means is realized by the specific processing unit 290 of the data processing device 12 as a function of documenting the matter in chat GPT after the call is ended and sending it by a messenger app or email. The confirmation means can be realized by the control unit 46A of the smart glasses 214 as a function of confirming family members and acquaintances using dedicated questions or passwords. The connection means can be realized by the control unit 46A of the smart glasses 214 as a function of a delivery company connecting to the person using a password for the delivery company. The linking means is realized by the specific processing unit 290 of the data processing device 12 as a function of checking the phone number during a call and linking to the police if the number has a history of misuse. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The response means includes a speech understanding function that utilizes voice recognition technology, and responds to calls from unregistered or withheld numbers in natural language generated by a pre-trained AI. This AI has learned a large amount of call data and has the ability to generate responses that can respond to various scenarios. When a call is received, the AI analyzes the content of the call in real time and selects an appropriate response to proceed with the conversation. The response content is customized to the nature of the call, so questions from sales calls are answered politely and suspicious calls are answered carefully. In addition, the response means can also use speech synthesis technology and speech content understanding technology to respond to calls from unregistered or withheld phone numbers, and a pre-trained AI can respond appropriately. This AI can learn multiple response scenarios, understand the intention of the call, and provide a natural dialogue according to the situation. The AI instantly selects a response based on the content of the call and responds customized to the purpose of the call, so inquiries from sales calls can be handled appropriately and suspicious calls can be answered carefully.
The transmission means includes voice recognition technology that converts the contents of the call into text and natural language generation technology that generates a summary of the call based on that text. When the call ends, the AI converts the contents of the call into text data and summarizes and documents the contents. The generated document is sent to the user via a communication means such as a messenger app or email. This process allows the user to check the details of the call at a later date and ensures that important information is not overlooked. The transmission means can also convert the contents of the completed call into text using voice recognition technology, then summarize it using natural language processing technology, and send the generated document to the user via a messenger app or email. The user can check the details of the call at any time through the sent document, and important information is recorded and will not be overlooked.
The verification means has the function of verifying the identity of the caller by using specific questions or passwords when the caller claims to be a family member or acquaintance. In this method, the AI uses questions and passwords set by the user, and verifies the identity of the caller by providing the correct response. If identity verification is successful, the AI continues the call and transfers the call to the user if necessary. This process prevents unauthorized access and fraud while ensuring smooth communication if the caller is the real person. The verification means can also have the AI present questions and passwords set by the user to callers claiming to be family members or acquaintances, and continue the call or transfer the call to the user only if an accurate answer is given. This function prevents unauthorized access and fraud, while enabling smooth communication if the caller is confirmed to be the real person.
The connection means has a function that allows a person in a specific occupation, such as a delivery worker, to talk to the person by using a secret code for the occupation. With this method, AI receives the secret code for the occupation, verifies that it is the correct code, and then connects the call to the person. The secret code for the occupation is important for ensuring the security of the call, and if an incorrect code is entered, the call will not be connected. The connection means can also have a function where AI receives a secret code specific to the occupation, such as a delivery worker, verifies its validity, and then connects the call to the user. The security of the call is ensured by this secret code, and if the incorrect code is entered, the call will be disconnected.
The linking means has a function of checking the phone number in real time during a call, and automatically linking to the police or related organizations if the number has a history of misuse. This means refers to a list of suspicious numbers stored in a database, and checks whether the incoming number is included in the list. If it is determined that the call is from a suspicious number, the system immediately starts a reporting protocol to the police, and takes appropriate action depending on the situation. The linking means can also monitor the number being called in real time, and automatically link to related organizations such as the police if the number has a history of misuse in the past. When a suspicious number is detected, a reporting protocol to the police is activated, and a prompt response is taken.
These measures are implemented by combining multiple AI technologies and collaboration functions to enhance user security and convenience. The data processing device and the control unit of the smart device cooperate to provide a platform for flexibly realizing these measures. Each measure is designed not only to function independently but also to work together to provide more advanced services. Through these measures, users can automate responses to calls from unregistered or unnotified phone numbers and protect themselves from potential risks that occur in daily life. In addition, these measures are built around AI technology to improve user security while simultaneously streamlining daily communication. As an alternative to sensor-free data collection, a process in which users manually enter information and the system uses that information is possible. Through these measures, users can automate responses to unregistered or unnotified calls and strengthen protection from unforeseen events.
Further functions can be added to the system of this embodiment to enhance its convenience. For example, the response means can be added with a function to analyze the user's emotions during a conversation, and the AI can infer the emotions from the tone of the other party's voice and manner of speaking and respond accordingly. This allows for a more careful and empathetic response when the other party is feeling angry or anxious, improving the quality of the call.
The sending method will add a feature that automatically integrates documented calls into users' calendars and reminders, helping them remember appointments and tasks generated during calls and improving productivity.
The verification method has a function to verify the identity of the user using biometric information. For example, biometric information obtained from a smartwatch or fitness tracker can be used to verify that the person on the other end of the line is actually a family member or acquaintance. This reduces the hassle for users while achieving a higher level of security.
The connection method will add a function that allows the delivery company to scan a QR code (registered trademark) or NFC tag to authenticate the user and connect the call directly to the user, eliminating the need for a password and making it possible to communicate with the delivery company more quickly and safely.
The collaboration will add a function to detect suspicious keywords from voice data during phone calls and automatically analyze the content. If a call is deemed suspicious based on the keywords, the system will first warn the user before reporting it to the police, helping them make more accurate decisions.
With these additional functions, the system of this embodiment not only automatically answers calls, but also more efficiently supports security and schedule management in daily life. In addition, by linking various AI technologies with databases, it provides greater security and convenience to the user's life. Furthermore, by integrating these technologies with the user's smart devices and IoT devices in the home, it becomes possible to realize a more personalized experience.
The system of this embodiment can be further expanded, and in order to increase convenience and security, the response means can be added with a function to analyze the caller's voiceprint and compare it with the voices of registered family members and acquaintances. This makes it possible to more accurately determine whether the caller is the real person and to continue the call safely without using specific questions or passwords.
The transmission means can incorporate a function that uses voice recognition and natural language processing to translate the contents of the call into multiple languages selected by the user, facilitating communication between users who speak different languages, facilitating use in international business situations and within multilingual households.
The verification method can include a feature that allows users to verify their identity by making specific gestures or movements on the camera, making it possible to easily and quickly verify the identity of a user during a call, further enhancing security.
Regarding the means of connection, AI will be able to add a function to verify the identity of the caller by comparing their location information with delivery schedule information, making it possible to confirm that the delivery company is calling from a location where the goods are actually being delivered, thereby guaranteeing the authenticity of the call.
The collaboration method will add a function that automatically notifies not only the police but also the user's designated emergency contact when a suspicious call is detected, enabling a rapid response in the event of an emergency.
These additional functions allow for flexible response to user needs while maintaining the basic structure of the existing system. In addition, by linking with smart devices and IoT devices in the home, the system provides a seamless experience in the environment in which the user uses the system on a daily basis. With these additional functions, the system of this embodiment goes beyond automatic call response and improves the security and convenience of the user in all aspects of daily life.
(Example 2)
The embodiment of the present invention is a system that has a sending means that can send the matter to family members when documenting the matter in chat GPT after the call ends and sending it by messenger app or email. Specifically, the matter can be sent to family members by selecting the document created after the call ends and specifying "family members" as the destination.
The sending means is realized by the specific processing unit 290 of the data processing device 12 as a function of selecting the document generated after the call is ended and sending the message to the family by specifying the family as the destination. This sending means can also be realized by the control unit 46A of the smart glasses 214. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The means for documenting the contents after the end of the call includes a conversation content analysis means using a voice recognition technology. This analysis means converts the voice data of the call into text and extracts the main points of the call. In addition, the conversation content analysis means using the voice recognition technology has a function of converting the voice data generated after the end of the call into text and extracting the main points of the call from the text data. The voice recognition technology utilizes a model based on deep learning and is trained to support various languages and accents. In addition, in order to support various languages and accents, this analysis means utilizes a deep learning model and proceeds with learning based on a large amount of voice data. The conversation content analysis means performs morphological analysis and syntactic analysis to identify important information exchanged in the conversation and content requiring action. In particular, a keyword extraction function is used to identify words and phrases frequently used in the conversation, and the main points of the call are documented based on these. In addition, in order to clarify the main points of the conversation, a morphological analysis function and a syntactic analysis function are combined to identify important information exchanged during the conversation and content requiring action. The keyword extraction function identifies words and phrases that appear frequently in the conversation, and documents based on this information.
The means for formatting the documented content into a message includes a text editor function, which organizes the parsed text and arranges the document structure. The message formatting means provides an interface that allows a user to easily check the content and add edits or additional information as necessary. The means for formatting the documented content into a message also includes a text editor function, which organizes the parsed text and arranges the document layout. The means for formatting the message provides an interface that allows a user to check the content of the message and add edits or additional information.
The sending means sends a message to a specific family member using a destination designation function. This destination designation function works in conjunction with the user's contact list to automatically send a message based on the contact information of the selected family member. The sending means also has a function of confirming message transmission and managing a transmission history, so that the user can track the status of the sent message and resend it as necessary. The message is sent in conjunction with a messenger app or an email app. The linking function with the messenger app or the email app performs authentication using the user's account information and transmits the message safely. The sending means also encrypts the message as a security measure and performs an authentication process at the time of transmission to ensure privacy protection. The sending means also transmits the message in an appropriate format depending on the message transmission method selected by the user. For example, the message is sent as an instant message in the messenger app, and as an email in the email app. The message sending means has a destination designation function for sending a message to a family member selected by the user, and works in conjunction with the user's contact list to automatically send a message based on the contact information of the family member. The sending means also includes a function for checking the sending status of a message and managing the sending history, so that the user can track the status of the sent message and resend it if necessary. The message sending process has a function for linking with a messenger application or an email application, authenticating the user based on the account information, and sending the message securely. The sending means protects privacy by encrypting the contents of the message and implementing security measures such as authentication at the time of sending. In addition, the message is sent in an appropriate format according to the sending method selected by the user, such as an instant message in the messenger application or an email in the email application.
An example of data collection that does not include sensors is when a user manually takes notes on a phone call. In this case, the user records the main points of the call after the call ends, and sends the text data to the family as a message. The call contents manually recorded by the user are organized using a text editor function and sent to the family as a formatted message by the sending means. This manual recording is suitable for situations where automatic documentation using voice recognition technology cannot be applied, or when the user wants to convey specific information in his or her own words. Another possible method of data collection that does not include sensors is a scenario in which a user manually takes notes on a phone call after the call ends, and sends the information to the family as a message. This manual recording is organized using a text editor function and sent as a formatted message. This is used in situations where voice recognition technology cannot be applied, or when the user wants to convey specific information in his or her own words.
In an embodiment of the present invention, functions beyond documenting the contents of a call can be considered. For example, a function can be added that automatically registers appointments and events mentioned during a call in a calendar in cooperation with a schedule management system based on the text generated by speech recognition. This can save the user the trouble of manually managing the schedule after the call. In addition, it can suggest adding events to a calendar shared among family members, making it easier for all family members to share the schedule.
It is also possible to automatically generate task lists based on documented text data and post them to a shared platform that can be accessed by all family members, where each family member can update the progress of tasks and check off completed tasks, creating an environment for the whole family to share information and cooperate.
In addition, the company will add a feature that performs sentiment analysis on written messages and reflects the emotional nuances expressed during a call in the text, helping message recipients to more accurately understand the caller's intent. For example, if emotions such as joy or anxiety are expressed during a call, those emotions will be expressed in the text with specific emojis and formats, enhancing the richness of communication.
Another possible function would be to use voice recognition and analysis to automatically generate FAQs and a list of frequently asked questions from the content of calls, building a knowledge base that family members can refer to when they have similar inquiries. This knowledge base would be shared among family members and updated every time a new call occurs, improving the efficiency of communication between family members.
In addition, the text generated after the call ends can be sent not only via messenger apps or email, but also played back in audio format, making it easier for visually impaired family members or children with difficulty reading and writing to receive the information.
Finally, to protect the privacy of family members, the documented content after the call is completed can be automatically anonymized or masked to identify sensitive information in the document, providing an environment in which information can be shared with peace of mind. This allows for a balance between respecting each individual's privacy and sharing only the information that is necessary.
The embodiment of the present invention relates to automatic documentation and transmission of call contents, and new functions can be added to the document. For example, the document can be analyzed, and action items can be automatically generated after the call ends, and a reminder can be set on the user's smart device as a task that needs to be addressed. The reminder can be prioritized based on deadlines and importance mentioned in the call, helping the user to take action without forgetting.
Additionally, to enhance communication among family members, it would be beneficial to add the ability to automatically attach relevant images, videos, and links based on specific words or phrases in a written message, allowing for visual information to be shared in addition to text-based messages, making communication richer.
In addition, the system can enhance security by automatically detecting and blurring certain personal or confidential information when documenting call transcripts, in accordance with privacy protection guidelines and natural language processing techniques.
When converting call contents into text, it is also effective to incorporate a translation function into multiple languages to meet the diverse needs of users. In a multicultural environment where family members speak different languages, it is possible to automatically translate the call contents and send messages in a language that each member can easily understand.
To further enhance convenience, a schedule function will be added that allows users to customize the timing of message sending. Users can set messages to be sent at a specific date and time, rather than just immediately, allowing them to optimize the timing at which their family members receive information.
Finally, quick action buttons for replying or confirming messages have been provided so that message recipients can easily take action on received messages, enabling quick feedback and efficient communication, making information sharing between family members even smoother.
(Example 3)
The embodiment of the present invention is a system that checks phone numbers during a call, and has a means for linking with the police when linking with the police if the number has a history of misuse. Specifically, the system is equipped with a function that checks the incoming number against a database during a call, and automatically notifies the police if it detects that the number has a history of misuse.
The police cooperation means is realized by the specific processing unit 290 of the data processing device 12 as a function of checking the incoming number against a database during a call and automatically reporting to the police if it is detected that the number has a history of misuse. This cooperation means can also be realized by the control unit 46A of the smart glasses 214. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The extraction means is provided for extracting caller information contained in the signal of each call sent through the communication network, and can accurately obtain the caller's telephone number from the call data using digital signal processing technology. The extraction means also has a signal analysis function for analyzing Caller ID information and identifying the telephone number. The system can also accurately identify the called number by utilizing the signal extraction function for extracting the caller's telephone number from the call signal and an analysis algorithm specialized in decoding Caller ID information.
The matching means has a function of matching the extracted telephone number with a database that collects telephone numbers suspected of fraudulent use and organizes them by category, and the matching process can be performed in real time through a database management system, and rapid matching can be performed while the call is in progress using high-speed search algorithms and indexing technology.The matching means also has a high-performance database search function and indexing function for rapid search and matching by referring to a database that lists numbers with a history of misuse.
The linking means has a function of automatically reporting to the police when a number with a history of abuse is detected, includes an interface function with the reporting system, generates report information in a data format based on the protocol with the police reception system when reporting, and transmits it to the police reception system using a secure communication channel. The contents of the report information include information such as the detected abused phone number, the date and time of the call, the duration of the call, and the telecommunications carrier used by the caller, and uses encryption technology and authentication systems to protect personal information and ensure the accuracy of the report. In addition, the linking means is compatible with multiple communication protocols and reporting systems, has an adapter function for smooth data exchange between the systems, and has a function of adjusting the format of the report information to meet the requirements of the police reception system in the reporting process, and selecting an appropriate reporting protocol to make the report. When the reporting process is activated, the system transmits report information to the police reception system and obtains a report reception confirmation. This confirmation is used as information for the system to record that the report was made correctly and store it as a report history. The report history is used for future analysis and improvement, contributing to the efficiency and accuracy of the report process. This linking function uses a report data generation function to create a report format containing the necessary information, and transmits the information to the police reception system via a secure report transmission function. Data encryption and system authentication functions are implemented to ensure the security and reliability of reports. In addition, the system's linking means has an adapter function that is compatible with multiple report protocols and enables data exchange with different report systems, and this adapter function automatically adjusts to a format suitable for the protocol requirements at the time of reporting using a report protocol selection function, and transmits report information and confirms receipt. The report history recording function records the success of reports, which is used to analyze and improve system performance.
As an example of data collection without sensors, a function for users to report suspicious calls on an application or web interface is provided. The user enters their experience of the call and any suspicious points they noticed during the call into a form, and the information is registered in a database. The data collected through this manual report can be added to a list of phone numbers that are automatically matched in the database, contributing to improved accuracy in detecting numbers with a history of abuse. As an example of data collection without sensors, an input function is provided for users to report suspicious calls. Through an interactive reporting form, users register the contents of suspicious calls in a database, and the information collected is used to detect numbers with a history of abuse. This manual reporting system contributes to the expansion of the number matching database by providing additional data based on the user's experience and intuition.
The system can be equipped with a function to strengthen the database update mechanism. For example, newly confirmed abused numbers will be automatically added to the database even after they are reported. In addition, when a suspicious call is reported, the credit information of the number will be checked against other databases and combined with information based on user reports to more accurately identify abuse history. To ensure the integrity of the database, a regular cleaning process will be carried out to eliminate incorrect and outdated information. In addition, to strengthen cooperation with the reporting system, a function will be introduced to directly link with the crime database provided by the police, obtain crime information in real time during the matching process, and improve the accuracy of the matching results.
To improve the immediacy of reports, the matching process begins the moment the call is initiated, and if a number with a history of abuse is detected, the caller will be given the option to sound an alarm or automatically hang up. In addition, when the call is hung up, a recording of the call can be saved on the caller's behalf to assist police investigations. When police intervene, a detailed report including the caller's location and call history is automatically generated, helping to speed up criminal investigations.
The user interface will be updated to include a feature that allows users to check the progress of the reporting process in real time to increase transparency in the reporting system. Users will be able to check the results of their reports and feedback from the police, which will increase trust in the system. The system will also provide statistical data and trend analysis on abused numbers, providing information to users to be cautious about calls.
Furthermore, machine learning technology will be introduced to identify numbers with a history of abuse, and new numbers likely to be abused will be predicted by analyzing various indicators such as call patterns and call frequency. This will enable proactive updates to the database, strengthening measures to prevent unknown criminal activity. A function will also be added that allows users to provide direct feedback on the effectiveness of the reporting system, which will help improve the system. Feedback will be provided anonymously, protecting user privacy while collecting valuable information that will contribute to improving the system.
To make the phone number check during a call more effective, AI will learn the patterns of various fraudulent scams that users may face and implement a function to flag in real time when certain words or phrases are detected during a call. This will allow malicious behavior to be inferred and detected from the content of the call, rather than simply matching the phone number with abuse history in the database. In addition, the reporting process will be simplified by providing shortcuts and buttons on the smartphone interface that allow users to easily report calls that they suspect are fraudulent. This will allow the database to be updated more quickly and improve protection for other users.
To strengthen cooperation with the police, the reporting system will be integrated with location tracking capabilities to help police respond quickly based on reported information, helping them track and capture criminals. Artificial intelligence will also be built into the reporting system to analyze crime patterns from report data and provide police with information to plan preventive vigilance activities. Using such predictive analytics will help prevent future crimes.
To facilitate data sharing with the police, the system will receive real-time information on fraud cases and other crimes known to the police and update the database. This will ensure that the reporting system operates based on the latest crime information and strengthen measures to protect users. In addition, the system will be equipped with a block list function, allowing users to register suspicious numbers and block calls from them. This will allow users to directly control the risks themselves.
Educational programs will include online courses and workshops to provide users with information to recognize and prevent fraud methods. This will provide users with the knowledge to protect themselves and raise security awareness in society as a whole. Examples of fraud cases that were prevented by using the reporting system will also be shared to help users understand the effectiveness of the system.
Finally, through a system update, if a call is deemed likely to be fraudulent, a warning message will be automatically sent to the user to warn them against fraud. This will allow users to immediately recognize the possibility of fraud and take appropriate action.

The process flow for each example is explained below.

(Example 1)
Step 1: When a call comes in from an unregistered or withheld phone number, the generative AI answers.
Step 2: After the call, document the matters discussed in Chat GPT.
Step 3: Send a written message via messenger app or email. If the caller claims to be a family member or acquaintance, verify the call using special questions or a secret code. The delivery company will connect you using a secret code. Check the phone number during the call, and if the number has a history of abuse, contact the police.
(Example 2)
Step 1: After the call, document the matters you wish to discuss in Chat GPT.
Step 2: Select the documented matter and designate it as being sent to family members.
Step 3: Send your message to your family via messenger app or email.
(Example 3)
Step 1: During a call, the incoming number is checked against a database to detect if it is a number with a history of abuse.
Step 2: If the number has a history of abuse, the call will automatically be reported to the police.
Step 3: After the call is over, the matter is documented in the chat GPT. The documented matter is used for communication with the police.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

(Example 1)
The embodiment of the present invention is a system that includes a response means in which a generative AI responds when a call is received from an unregistered or unnotified phone number, and a transmission means in which the matter is documented in chat GPT after the call is ended and sent via a messenger app or email. Furthermore, as an emotion recognition means that combines an emotion engine that recognizes the user's emotions, the tone of the user's voice and choice of words are analyzed during the call to estimate the emotion. Based on the estimated emotion, the generative AI's response and the expression of the documented matter are adjusted. For example, if the user shows an anxious emotion, a calmer expression is used to provide a sense of security.
The response means is realized by the specific processing unit 290 of the data processing device 12, and the generative AI responds to an incoming call from an unregistered or unnotified phone number. The transmission means is realized by the specific processing unit 290 of the data processing device 12 as a function of documenting the matter in chat GPT after the call is ended and sending it by messenger app or email. The emotion recognition means is realized by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214 as a function of analyzing the tone of the user's voice and choice of words during the call and estimating emotions. The function of adjusting the response of the generative AI and the expression of the documented matter based on the estimated emotion is also realized by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The response means includes a dialogue generation means that combines natural language processing and speech synthesis. The dialogue generation means automatically responds to unregistered or unnamed incoming calls, enabling a dialogue with the user. When an unregistered or unnamed incoming call is detected, the generative AI responds using a pre-trained conversation model. This conversation model is trained based on a variety of dialogue data so that it can handle various scenarios, and provides appropriate answers and guidance to the callee's questions and requests. In addition, the response means has an AI dialogue generation function, and can automatically provide an appropriate response to an incoming call from an unregistered or unnamed phone number. This function combines a dialogue management function for understanding the callee's purpose and request at the early stage of the call and providing a corresponding response, and a voice generation function for converting the generated response into a natural voice. The dialogue management function can analyze the callee's intention based on the detection of specific keywords and phrases and generate an appropriate response. The voice generation function can convert a text-based response into voice in real time, providing the callee with a natural conversation experience.
The transmission means includes a chatbot and a documentation means using natural language understanding technology. After the call is ended, the contents of the call are accurately documented using an advanced natural language understanding model such as chat GPT. The documentation means has the ability to extract and summarize the main points of the call, and can generate a document that conveys the purpose concisely and clearly. The generated document is sent to the user via a messenger app or email. This process includes a link to the user's email address or messenger account, and the document is automatically sent in the appropriate format. The transmission means can also convert the contents of the call into text and provide it in an accessible form to the user. After the call is ended, the call content documentation function is activated and can extract and summarize the main points of the conversation. This summarized text is sent to the user's designated email address or messenger app through an automatic delivery function. This automatic delivery function includes an email delivery function and an app link function to properly format the document and ensure that it is delivered to the designated destination.
The emotion recognition means includes a voiceprint analysis means for performing voice analysis, and a language analysis means for estimating emotions from the choice of words. The voiceprint analysis means can detect characteristics of the user's voice, such as tone, pitch, and speed, using the microphone of the smart device, and can estimate the user's emotional state from these voice characteristics. The language analysis means can analyze the content of the user's speech and extract emotional context from the words and phrases used. These analysis results are used by the generative AI when making a response or adjusting the expression of documented matters. For example, if the user shows anxiety, the expression of the response or document is adjusted to be more gentle and reassuring. In addition, the emotion recognition means has a function of estimating emotions from the characteristics of the user's voice and language use during a call. The voiceprint analysis function can extract characteristics such as the pitch, strength, and speed of the voice from the voice data collected by the microphone, and can estimate emotions by analyzing these characteristics. The language emotion analysis function can process the language data during a call, analyze the emotional meaning of the words and phrases used, and grasp the user's emotional state. These analytics can be used to adjust the tone of the AI's responses and the wording of documented calls to provide an appropriate emotional response to the user.
Data collection methods that do not involve sensors include text data entered by users themselves and feedback on system usage. These are provided to the system through the user input acceptance function and feedback collection function, and are used as a valuable source of information for service improvement.
These means are intended to respond quickly and effectively to user requests and improve the quality of communication. In addition, the implementation of each means is flexibly performed by the data processing device or the control unit of the smart device, and can be modified in various ways to improve the efficiency and usability of the system.
The system of the present invention can be equipped with an additional function of a summary prediction means for predicting the caller's intention before answering an unregistered or unnamed call. This allows the response means to generate a more accurate dialogue, realizing a meaningful exchange for the user. In addition, when the generative AI responds, it can have a language selection function based on the caller's country or region, enabling automatic response in multiple languages.
As for the delivery method, in addition to documenting the contents of the call, it can also highlight important keywords and phrases to help users quickly understand the document, and automatically generate action items based on the documented content and add them to the user's task list.
The emotion recognition means can detect changes in the user's emotions during a call in real time and dynamically adjust the tone and tempo of the conversation of the response means. Protocols can also be built in that, when a particular emotion is detected, prompt the user to contact an expert who can provide special support or advice according to the emotion.
The response tool can be enhanced with a personalization feature that analyzes the caller's past call history and related data to provide a more personalized response, making it possible to provide more relevant information to the user and increase the usefulness of the response.
Regarding sending methods, a function can be added that suggests follow-up actions based on the documented content of the call. For example, tasks and events included in the call content can be automatically registered in a calendar app to help users manage their time.
Furthermore, the emotion recognition means may be capable of detecting the user's stress level or tension and providing appropriate advice on how to reduce stress or links to relaxation content, thereby supporting the user's mental health and promoting overall well-being.
The system of this embodiment is considered to have a number of additional functions for further improving its functionality. For example, a voiceprint recognition means can be added to identify a caller by analyzing the caller's voiceprint and comparing it with previous call data for an unregistered or unnamed call. This allows the response means to respond more appropriately based on information if the caller has previously interacted with the system. The voiceprint recognition means also functions as a security measure, providing a reliable call experience for the user.
Regarding the means of transmission, when documenting the contents of the call, the function of structuring the contents of the call and classifying the text according to the importance of the information will be considered. This will allow users to grasp important information more quickly when reading the document. In addition, the documented content can be customized according to the user's preferences and past behavior patterns to provide a more personal experience.
The emotion recognition means detects the user's stress level during a call, and if it is estimated that the user is under high stress, it provides guidance on relaxation methods and psychological support related to the content of the call, thereby providing a service that allows users to relax and reduce stress through the call.
Additionally, the response tool will automatically suggest follow-up actions to users based on the content of the call, for example providing links to additional information about products or services suggested during the call or suggesting next steps of action to help users make decisions.
To improve response methods, we are considering a function that automatically changes the response style according to the caller's intentions. For example, if the caller indicates an emergency situation, we will adjust the AI to provide quick and accurate instructions. This will create a system that can immediately respond to the caller's needs.
These additional functions are intended to improve the user experience and enable accurate understanding of call content and rapid response. Each function is expected to maximize the capabilities of data processing devices and smart device control units, further enhancing the usability of the system.
(Example 2)
The embodiment of the present invention is a system that includes an adjustment means for documenting the matter in chat GPT after the call ends, and adjusting the expression of the document using an emotion engine that recognizes the user's emotions when sending it by messenger app or email. Specifically, the documented matter is input to the emotion engine, and the user's emotions are analyzed. Based on the analysis results, the expression of the document is appropriately adjusted. For example, if the user is feeling happy or excited, the user's emotions are shared by using brighter and more lively expressions.
The adjustment means is realized by the specific processing unit 290 of the data processing device 12 as a function of documenting the matter in the chat GPT after the call is ended and adjusting the expression of the document using an emotion engine that recognizes the user's emotions when sending it by a messenger app or email. The function of inputting the documented matter to the emotion engine and analyzing the user's emotions is also realized by the specific processing unit 290 of the data processing device 12. The function of appropriately adjusting the expression of the document based on the analysis result is realized by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214. The correspondence relationship between each means and the device or control unit is not limited to the above-mentioned example, and various changes are possible.
The conversation content extraction means converts the contents of the call into text data by making full use of voice recognition technology. After receiving the voice signal, the conversation content extraction means uses noise reduction means to remove background noise and improve the accuracy of the voice-to-text conversion. Furthermore, the conversation content extraction means uses cutting-edge voice recognition technology to accurately convert the contents of the call into text data, and in the process, the noise reduction means removes background noise and improves the conversion accuracy.
The text processing means corrects syntactic errors in the converted text data and performs grammar checking via the grammar checking means to maintain linguistic fluency. The text processing means also performs adjustments to ensure that the grammar checking means of the converted text data ensures accuracy of grammar and maintains the natural flow of the document.
The emotion recognition means includes an emotion extraction means based on text mining and emotion analysis techniques, which estimates the user's emotion from the wording and context of the generated text data. The emotion extraction means also identifies words, phrases, and grammatical patterns that express various emotions and classifies them into emotion categories such as positive, negative, and neutral. The emotion emphasis means also adjusts the tone and phrasing of the text according to the extracted emotions, making modifications to better convey the user's emotional state. The emotion extraction means also reads the user's emotion from the language patterns expressed in the document and adjusts the tone of the document based on the emotion.
The adjustment means changes the expression of the text based on the emotion data analyzed by the emotion recognition means. When a positive emotion such as joy or excitement is detected, the expression enhancement means replaces the vocabulary used with brighter and more lively words to give the message a favorable impression. When a negative emotion such as sadness or anger is detected, the expression mitigation means softens the tone of the message and selects phrases that show empathy and understanding. When the user's emotion is positive, the expression enhancement means energizes the document, and when the user's emotion shows negative emotion, the expression mitigation means uses a more gentle expression.
The message sending means includes a function of linking with a messenger application or an email client and sends the adjusted text in an appropriate format. Furthermore, the transmission protocol selection means selects an optimal transmission protocol according to the platform and settings of the recipient and ensures delivery of the message. Furthermore, the user interface presentation means provides a preview screen for a final review of the document before sending, allowing the user to make final corrections as necessary. Furthermore, the message sending means sends the adjusted text in a format suitable for the messenger application or the email client, and the preview screen provides an interface for the user to make a final check of the document.
The above processes are realized by a specific processing unit built into the control unit of the smart device or the data processing device, depending on the device and settings used by the user. Moreover, these means are designed as modularized components, and can be easily replaced or expanded as system components. Furthermore, the correspondence between each means is set flexibly, and various changes can be made to accommodate system upgrades and customization. Moreover, this series of processes is modularized so that it can flexibly respond to devices and environments, and is designed to facilitate system upgrades and customization.
This embodiment can be further expanded to add a function for deeper understanding of the user's emotions and improving the quality of communication. For example, by incorporating a facial expression recognition function during video chat into the emotion engine, emotions can be analyzed from visual information as well, resulting in more accurate emotion judgment. To improve the accuracy of emotion recognition, the tone and pitch of the user's voice can also be analyzed and reflected in the text. Furthermore, by analyzing the user's past communication history and response patterns, the system can learn the individual's emotional expression style and adjust the text accordingly in a more personalized manner.
The text processing means has a function to suggest phrases inspired by literary works and poetry to enhance the diversity and creativity of expressions, allowing users to express their emotions more richly. It can also select appropriate expressions according to the environment and situation in which the communication takes place, taking into account the social context and cultural background.
The messaging tool will add a function to predict the emotional impact of the text sent on the recipient, allowing users to communicate more responsibly. In addition, AI can predict the recipient's reaction and use that information to suggest the next communication strategy the user should take.
Overall, these features will help users to reflect emotions accurately and sensitively in their communications, contributing to building deeper human relationships. These advanced methods can be effectively used by individuals as well as corporate customer support and CRM systems to deepen customer relationships. Furthermore, these systems are expected to be applied in fields where human emotions play an important role, such as user education and counseling.
The system leverages an emotion engine to provide document expression adjustments based on the user's emotions. To extend this functionality, sensors are integrated to capture the user's biometric information, allowing for a more accurate reading of emotions from physiological responses such as heart rate and skin conductivity. Data from the sensors is analyzed in real time, allowing the tone of the document to be adjusted on the fly.
In addition, it is equipped with a personalized learning function that continuously analyzes the user's daily communication to understand the user's unique expression style and preferences, allowing the system to suggest more natural and personalized documents that reflect the user's personality.
It will also enhance multilingual support, allowing it to capture emotional nuances in different languages, enhancing international communication and understanding among multilingual users.
To protect user privacy, the company will also add features to anonymize and encrypt emotion data and strengthen security, allowing users to use the system with peace of mind.
Aiming for applications in the fields of education and counseling, we will develop a function that uses the results of emotion recognition to support communication skills training. The training program will include practicing emotional expression and learning appropriate communication techniques.
Finally, to improve the usability of the system, we make the user interface rich and intuitive, supporting a variety of gestures and voice commands, and allowing users to easily intervene in the document adjustment process, allowing users to fine-tune the expression at their own will and achieve more personal communication.
(Example 3)
The embodiment of the present invention is a system that includes an emotion engine that recognizes the user's emotions during a call and an adjustment means that adjusts the response and dialogue content to match the user's emotions. Specifically, the user's tone of voice and choice of words are input to the emotion engine during a call to estimate the user's emotions. The generative AI's responses and dialogue content are adjusted based on the estimated emotions. For example, if the user shows sad emotions, words of empathy are used to provide encouragement and support.
The adjustment means is realized by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214 as a function of adjusting the response and the content of the dialogue to match the user's emotions using an emotion engine that recognizes the user's emotions during a call. A function of inputting the user's tone of voice and choice of words into the emotion engine during a call and estimating the user's emotions is also realized by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214. A function of adjusting the response of the generative AI and the content of the dialogue based on the estimated emotions is realized by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The voice data collection means includes a microphone with high sensitivity and noise canceling function. The microphone is equipped with a digital signal processing algorithm to remove ambient noise and clearly record the user's voice. The voice data collection means also employs a highly sensitive microphone that captures subtle acoustic characteristics from the user's speech and combines this with digital signal processing technology to effectively remove ambient noise. The microphone accurately captures the characteristics of the user's voice and provides basic data for detecting changes in emotions.
The voice feature extraction means extracts features that may reflect human emotions from the voice signal. This extraction means includes a spectrogram analysis function and a pitch tracking function for performing acoustic feature analysis, and an emotion identification function using an acoustic model. The voice feature extraction means also uses acoustic analysis tools such as a spectrum analysis function and a pitch analysis function to extract features that suggest emotions from the voice signal, and supplies these data to the emotion estimation model.
The emotion analysis means includes an emotion estimation model based on machine learning, which estimates the user's emotional state using the features extracted by the voice feature extraction means as input. The emotion estimation model is composed of classifiers such as neural networks, support vector machines, and decision trees trained based on training data, and classifies the user's emotions into categories such as positive, negative, and neutral. The emotion estimation model improves its accuracy through continuous learning, and evolves to be able to respond to subtle changes in the user's perception of emotions. In addition, the emotion analysis means has an emotion estimation model that utilizes machine learning technology, and analyzes the user's emotions based on the extracted voice features. This model combines various machine learning algorithms to identify emotion categories from the user's utterance, and estimates the user's emotional state based on this. The estimated emotional state is important information for adjusting the system's response to the user's utterance and behavior.
The dialogue adjustment means includes a response generation function using a generative AI model, which dynamically adjusts the content of the dialogue according to the emotion estimated by the emotion analysis means. The response generation function utilizes natural language generation technology to generate a response sentence using words and tones appropriate for the user's emotion. For example, if the user is expressing sad emotion, a response including empathetic expressions and comforting words is generated. This response generation function has a context-aware processing function for taking into account the context of the conversation and providing content that matches the user's emotion and the purpose of the communication. In addition, the generated response is constructed to be natural to the user and to meet the emotional needs. The response generation function is realized by a machine learning model trained on a large-scale conversation dataset and can adapt to the user's language and speaking style. In addition, the dialogue adjustment means includes a response generation function using a generative AI model, which generates dialogue content appropriately corresponding to the estimated emotion. This function selects words and a dialogue tone that correspond to the user's emotion and provides a response that matches the user's current emotion and the context of the conversation. The response generation function is designed to perform context-aware processing and provide the information and support required by the user in an appropriate form. This feature is capable of generating responses that adapt to a user's language and emotions based on models trained from large amounts of conversational data.
Non-sensor-based data collection methods include text information entered directly by users into the system and metadata obtained from call records. These are used as supplementary information when analyzing user behavioral patterns and preferences, and contribute to improving the accuracy of the emotion engine and optimizing response generation functions. User-entered text information is also used as training data for emotion estimation models as part of the emotion analysis method.
The system has the ability to analyze the user's emotions in real time during a call and automatically adjust the dialogue content, so it can integrate facial expression recognition technology in addition to voice data to capture even more subtle changes in emotions. It uses a webcam or a camera on a smart device to analyze the user's facial expressions and improve the accuracy of emotion analysis. This added facial expression recognition function will more accurately recognize the user's emotions and make it possible to adjust the dialogue according to even more subtle changes in emotions.
It is also conceivable to incorporate wearable devices to capture the user's physiological signals. Measuring physiological responses such as heart rate and electrodermal activity can provide deeper insight into emotions that cannot be captured by tone of voice or facial expressions alone. Integrating this data can further improve the accuracy of emotion analysis and generate more appropriate dialogue responses.
It would also be effective to add customization features to the dialogue adjustment method that take into account the user's cultural background and personal values. A user profile can be constructed and the tone and content of the dialogue can be further personalized based on that information. This makes it possible to provide detailed support tailored to each individual user.
Furthermore, in order to promote the evolution of the emotion estimation model, we will build a mechanism to collect emotion data through crowdsourcing and incorporate feedback from a variety of users to continuously update the model, resulting in a flexible system that can handle a variety of emotional expressions and languages.
Applications in the fields of education and mental health care can also be considered. For example, in the field of education, it could be used to present educational materials adapted to students' emotions or in counseling sessions. In mental health care, it could recognize the user's emotions and provide dialogue support to reduce stress and anxiety. This would enable a more effective approach to the problems the user is facing.
Regarding privacy protection for the system, we will strengthen security measures to safely store users' emotional data and minimize the risk of information leakage by using appropriate access control and encryption technology. This will allow users to use the system with peace of mind.
Ultimately, it is hoped that the personalized interaction experience provided by this system will be developed into services that improve the quality of users' lives.
The embodiment of the present invention is applicable not only to telephone calls but also to video conferencing and online education platforms. For example, a lecturer can grasp the emotions of students in real time and adjust the progress of the curriculum according to the emotions, thereby providing a more effective learning experience. Even in video conferencing, dialogue management that reflects the emotions of participants is performed, promoting a productive and positive meeting environment.
The system could also be enhanced to monitor health conditions based on emotional responses: for example, if a user's tone of voice or manner of speaking indicates negative emotions over a period of time, a mental health professional could be notified to intervene if necessary.
Furthermore, to further improve the emotion engine, a function will be incorporated that will analyze the emotional patterns of the user's daily life and provide advice on long-term emotion management and stress reduction based on that information. By analyzing the user's daily rhythm and activity patterns and predicting emotional ups and downs, the system will suggest content for relaxation and motivation at appropriate times.
This system is also expected to be applied in the field of customer support. For example, if call center operators could grasp the customer's emotions in real time and detect negative emotions such as dissatisfaction or anger, they could take immediate action to address the issue, contributing to improving customer satisfaction.
In the fields of games and entertainment, it is expected that dynamically changing content according to the user's emotions will have the effect of increasing immersion and enjoyment. In-game characters will react to the player's emotions, changing the story development and dialogue content, creating a more personalized experience.
In addition, we will integrate voice assistants and virtual reality (VR) to realize more natural dialogue based on the user's emotions. Voice assistants will understand the user's emotions and provide information and services tailored to individual needs. In the VR environment, the scenario and environment will change according to the user's emotions, providing an experience that matches the emotions in real time.
This system also has the potential to revolutionize the design of user interfaces (UI) and user experiences (UX). Using emotion recognition technology, it can provide UI and UX optimized for the user's emotions, increasing user satisfaction. For example, websites and applications can grasp the user's emotions in real time and adjust the way content is presented and the form of interaction.
Ultimately, it is hoped that the emotion regulation function provided by this system will become widely used throughout society as a tool to improve the quality of human relationships and increase the effectiveness of communication.

The process flow for each example is explained below.

(Example 1)
Step 1: When a call comes in from an unregistered or withheld phone number, the generative AI answers.
Step 2: After the call, document the matters discussed in Chat GPT.
Step 3: Input the documented requirements into the emotion engine to analyze the user's emotions.
Step 4: Based on the analysis results, the representation of the document is adjusted.
Step 5: Send the adjusted document via messenger app or email.
(Example 2)
Step 1: After the call, document the matters you wish to discuss in Chat GPT.
Step 2: Input the documented requirements into the emotion engine to analyze the user's emotions.
Step 3: Adjust the representation of the document based on the analysis results.
Step 4: Select the adjusted document and specify the recipient (e.g., family).
Step 5: Send the document to the specified recipient via messenger app or email.
(Example 3)
Step 1: During a call, the user's tone of voice, choice of words, etc. are input into the emotion engine to analyze the user's emotions.
Step 2: Based on the analysis results, adjust the generative AI's responses and dialogue content.
Step 3: After the call, document the matter in the chat GPT. The documented matter is used to input the emotion engine.

The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input for the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization. The identification processing unit 290 performs the above-mentioned identification processing while using the data generation model 58.

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart glasses 214.

[Third embodiment]

Figure 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical able-bodied person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is performed in a secure state.

Figure 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Figure 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. Data generation model 58 and emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a ChatGPT server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance). Next, the identification process by the identification processing unit 290 of the data processing device 12 will be described.

(Example 1)
The embodiment of the present invention is a system that includes a response means in which a generative AI responds to an incoming call from an unregistered or withheld phone number, and a sending means that documents the matter in chat GPT after the call ends and sends it via a messenger app or email. In addition, the system includes a confirmation means that uses a special question or password to confirm if the caller claims to be a family member or acquaintance, a connection means that connects the delivery company to the person using a password for the delivery company, and a linking means that checks the phone number during the call and links to the police if the number has a history of abuse.
The response means is realized by the specific processing unit 290 of the data processing device 12, and responds using a generative AI when a call is received from an unregistered or unnotified phone number. The transmission means is realized by the specific processing unit 290 of the data processing device 12 as a means for documenting the matter in chat GPT after the call is ended and sending it by a messenger app or email. The confirmation means may be realized by the control unit 46A of the headset type terminal 314 as a means for confirming family members or acquaintances using dedicated questions or passwords. The connection means may be realized by the control unit 46A of the headset type terminal 314 as a means for a delivery company to connect to the person using a password for the delivery company. The linking means is realized by the specific processing unit 290 of the data processing device 12 as a means for checking the phone number during a call and linking to the police if the number has a history of misuse. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The response means includes a speech understanding function that utilizes voice recognition technology, and responds to calls from unregistered or withheld numbers in natural language generated by a pre-trained AI. This AI has learned a large amount of call data and has the ability to generate responses that can respond to various scenarios. When a call is received, the AI analyzes the content of the call in real time and selects an appropriate response to proceed with the conversation. The response content is customized to the nature of the call, so questions from sales calls are answered politely and suspicious calls are answered carefully. In addition, the response means can also use speech synthesis technology and speech content understanding technology to respond to calls from unregistered or withheld phone numbers, and a pre-trained AI can respond appropriately. This AI can learn multiple response scenarios, understand the intention of the call, and provide a natural dialogue according to the situation. The AI instantly selects a response based on the content of the call and responds customized to the purpose of the call, so inquiries from sales calls can be handled appropriately and suspicious calls can be answered carefully.
The transmission means includes voice recognition technology that converts the contents of the call into text and natural language generation technology that generates a summary of the call based on that text. When the call ends, the AI converts the contents of the call into text data and summarizes and documents the contents. The generated document is sent to the user via a communication means such as a messenger app or email. This process allows the user to check the details of the call at a later date and ensures that important information is not overlooked. The transmission means can also convert the contents of the completed call into text using voice recognition technology, then summarize it using natural language processing technology, and send the generated document to the user via a messenger app or email. The user can check the details of the call at any time through the sent document, and important information is recorded and will not be overlooked.
The verification means has the function of verifying the identity of the caller by using specific questions or passwords when the caller claims to be a family member or acquaintance. In this method, the AI uses questions and passwords set by the user, and verifies the identity of the caller by providing the correct response. If identity verification is successful, the AI continues the call and transfers the call to the user if necessary. This process prevents unauthorized access and fraud while ensuring smooth communication if the caller is the real person. The verification means can also have the AI present questions and passwords set by the user to callers claiming to be family members or acquaintances, and continue the call or transfer the call to the user only if an accurate answer is given. This function prevents unauthorized access and fraud, while enabling smooth communication if the caller is confirmed to be the real person.
The connection means has a function that allows a person in a specific occupation, such as a delivery worker, to talk to the person by using a secret code for the occupation. With this method, AI receives the secret code for the occupation, verifies that it is the correct code, and then connects the call to the person. The secret code for the occupation is important for ensuring the security of the call, and if an incorrect code is entered, the call will not be connected. The connection means can also have a function where AI receives a secret code specific to the occupation, such as a delivery worker, verifies its validity, and then connects the call to the user. The security of the call is ensured by this secret code, and if the incorrect code is entered, the call will be disconnected.
The linking means has a function of checking the phone number in real time during a call, and automatically linking to the police or related organizations if the number has a history of misuse. This means refers to a list of suspicious numbers stored in a database and checks whether the incoming number is included in the list. If it is determined that the call is from a suspicious number, the system immediately starts a reporting protocol to the police and takes appropriate action depending on the situation. The linking means can also monitor the number being called in real time, and automatically link to related organizations such as the police if the number has a history of misuse in the past. When a suspicious number is detected, a reporting protocol to the police is activated and a prompt response is taken.
These measures are implemented by combining multiple AI technologies and collaboration functions to enhance user security and convenience. The data processing device and the control unit of the smart device cooperate to provide a platform for flexibly realizing these measures. Each measure is designed not only to function independently but also to work together to provide more advanced services. Through these measures, users can automate responses to calls from unregistered or unnotified phone numbers and protect themselves from potential risks that occur in daily life. In addition, these measures are built around AI technology to improve user security while simultaneously streamlining daily communication. As an alternative to sensor-free data collection, a process in which users manually enter information and the system uses that information is possible. Through these measures, users can automate responses to unregistered or unnotified calls and strengthen protection from unforeseen events.
Further functions can be added to the system of this embodiment to enhance its convenience. For example, the response means can be added with a function to analyze the user's emotions during a conversation, and the AI can infer the emotions from the tone of the other party's voice and manner of speaking and respond accordingly. This allows for a more careful and empathetic response when the other party is feeling angry or anxious, improving the quality of the call.
The sending method will add a feature that automatically integrates documented calls into users' calendars and reminders, helping them remember appointments and tasks generated during calls and improving productivity.
The verification method has a function to verify the identity of the user using biometric information. For example, biometric information obtained from a smartwatch or fitness tracker can be used to verify that the person on the other end of the line is actually a family member or acquaintance. This reduces the hassle for users while achieving a higher level of security.
The connection method will add a function that allows the delivery company to scan a QR code (registered trademark) or NFC tag to authenticate the user and connect the call directly to the user, eliminating the need for a password and making it possible to communicate with the delivery company more quickly and safely.
The collaboration will add a function to detect suspicious keywords from voice data during phone calls and automatically analyze the content. If a call is deemed suspicious based on the keywords, the system will first warn the user before reporting it to the police, helping them make more accurate decisions.
With these additional functions, the system of this embodiment not only automatically answers calls, but also more efficiently supports security and schedule management in daily life. In addition, by linking various AI technologies with databases, it provides greater security and convenience to the user's life. Furthermore, by integrating these technologies with the user's smart devices and IoT devices in the home, it becomes possible to realize a more personalized experience.
The system of this embodiment can be further expanded, and in order to increase convenience and security, the response means can be added with a function to analyze the caller's voiceprint and compare it with the voices of registered family members and acquaintances. This makes it possible to more accurately determine whether the caller is the real person and to continue the call safely without using specific questions or passwords.
The transmission means can incorporate a function that uses voice recognition and natural language processing to translate the contents of the call into multiple languages selected by the user, facilitating communication between users who speak different languages, facilitating use in international business situations and within multilingual households.
The verification method can include a feature that allows users to verify their identity by making specific gestures or movements on the camera, making it possible to easily and quickly verify the identity of a user during a call, further enhancing security.
Regarding the means of connection, AI will be able to add a function to verify the identity of the caller by comparing their location information with delivery schedule information, making it possible to confirm that the delivery company is calling from a location where the goods are actually being delivered, thereby guaranteeing the authenticity of the call.
The collaboration method will add a function that automatically notifies not only the police but also the user's designated emergency contact when a suspicious call is detected, enabling a rapid response in the event of an emergency.
These additional functions allow for flexible response to user needs while maintaining the basic structure of the existing system. In addition, by linking with smart devices and IoT devices in the home, the system provides a seamless experience in the environment in which the user uses the system on a daily basis. With these additional functions, the system of this embodiment goes beyond automatic answering of calls, improving the security and convenience of users in all aspects of their daily lives.
(Example 2)
The embodiment of the present invention is a system that has a sending means that can send the matter to family members when documenting the matter in chat GPT after the call ends and sending it by messenger app or email. Specifically, the matter can be sent to family members by selecting the document created after the call ends and specifying "family members" as the destination.
The sending means is realized by the specific processing unit 290 of the data processing device 12 as a means for documenting the matter in chat GPT after the call ends and sending it to the family by messenger app or e-mail. The sending means may also be realized by the control unit 46A of the headset type terminal 314. The correspondence between each means and the device or control unit is not limited to the above-mentioned example, and various changes are possible.
The means for documenting the contents after the end of the call includes a conversation content analysis means using a voice recognition technology. This analysis means converts the voice data of the call into text and extracts the main points of the call. In addition, the conversation content analysis means using the voice recognition technology has a function of converting the voice data generated after the call is ended into text and extracting the main points of the call from the text data. The voice recognition technology utilizes a model based on deep learning and is trained to support various languages and accents. In addition, in order to support various languages and accents, this analysis means utilizes a deep learning model and proceeds with learning based on a large amount of voice data. The conversation content analysis means performs morphological analysis and syntactic analysis to identify important information exchanged in the conversation and content requiring action. In particular, a keyword extraction function is used to identify words and phrases frequently used in the conversation, and the main points of the call are documented based on these. In addition, in order to clarify the main points of the conversation, a morphological analysis function and a syntactic analysis function are combined to identify important information exchanged during the conversation and content requiring action. The keyword extraction function identifies words and phrases that appear frequently in the conversation, and documents based on this information.
The means for formatting the documented content into a message includes a text editor function, which organizes the parsed text and arranges the document structure. The message formatting means provides an interface that allows a user to easily check the content and add edits or additional information as necessary. The means for formatting the documented content into a message also includes a text editor function, which organizes the parsed text and arranges the document layout. The means for formatting the message provides an interface that allows a user to check the content of the message and add edits or additional information.
The sending means sends a message to a specific family member using a destination designation function. This destination designation function works in conjunction with the user's contact list to automatically send a message based on the contact information of the selected family member. The sending means also has a function of confirming message transmission and managing a transmission history, so that the user can track the status of the sent message and resend it as necessary. The message is sent in conjunction with a messenger app or an email app. The linking function with the messenger app or the email app performs authentication using the user's account information and transmits the message safely. The sending means also encrypts the message as a security measure and performs an authentication process at the time of transmission to ensure privacy protection. The sending means also transmits the message in an appropriate format depending on the message transmission method selected by the user. For example, the message is sent as an instant message in the messenger app, and as an email in the email app. The message sending means has a destination designation function for sending a message to a family member selected by the user, and works in conjunction with the user's contact list to automatically send a message based on the contact information of the family member. The sending means also includes a function for checking the sending status of a message and managing the sending history, so that the user can track the status of the sent message and resend it if necessary. The message sending process has a function for linking with a messenger application or an email application, authenticating the user based on the account information, and sending the message securely. The sending means protects privacy by encrypting the contents of the message and implementing security measures such as authentication at the time of sending. In addition, the message is sent in an appropriate format according to the sending method selected by the user, such as an instant message in the messenger application or an email in the email application.
An example of data collection that does not include sensors is when a user manually takes notes on a phone call. In this case, the user records the main points of the call after the call ends, and sends the text data to the family as a message. The call contents manually recorded by the user are organized using a text editor function and sent to the family as a formatted message by the sending means. This manual recording is suitable for situations where automatic documentation using voice recognition technology cannot be applied, or when the user wants to convey specific information in his or her own words. Another possible method of data collection that does not include sensors is a scenario in which a user manually takes notes on a phone call after the call ends, and sends the information to the family as a message. This manual recording is organized using a text editor function and sent as a formatted message. This is used in situations where voice recognition technology cannot be applied, or when the user wants to convey specific information in his or her own words.
In an embodiment of the present invention, functions beyond documenting the contents of a call can be considered. For example, a function can be added that automatically registers appointments and events mentioned during a call in a calendar in cooperation with a schedule management system based on the text generated by speech recognition. This can save the user the trouble of manually managing the schedule after the call. In addition, it can suggest adding events to a calendar shared among family members, making it easier for all family members to share the schedule.
It is also possible to automatically generate task lists based on documented text data and post them to a shared platform that can be accessed by all family members, where each family member can update the progress of tasks and check off completed tasks, creating an environment for the whole family to share information and cooperate.
In addition, the company will add a feature that performs sentiment analysis on written messages and reflects the emotional nuances expressed during a call in the text, helping message recipients to more accurately understand the caller's intent. For example, if emotions such as joy or anxiety are expressed during a call, those emotions will be expressed in the text with specific emojis and formats, enhancing the richness of communication.
Another possible function would be to use voice recognition and analysis to automatically generate FAQs and a list of frequently asked questions from the content of calls, building a knowledge base that family members can refer to when they have similar inquiries. This knowledge base would be shared among family members and updated every time a new call occurs, improving the efficiency of communication between family members.
In addition, the text generated after the call ends can be sent not only via messenger apps or email, but also played back in audio format, making it easier for visually impaired family members or children with difficulty reading and writing to receive the information.
Finally, to protect the privacy of family members, the documented content after the call is completed can be automatically anonymized or masked to identify sensitive information in the document, providing an environment in which information can be shared with peace of mind. This allows for a balance between respecting each individual's privacy and sharing only the information that is necessary.
The embodiment of the present invention relates to automatic documentation and transmission of call contents, and new functions can be added to the document. For example, the document can be analyzed, and action items can be automatically generated after the call ends, and a reminder can be set on the user's smart device as a task that needs to be addressed. The reminder can be prioritized based on deadlines and importance mentioned in the call, helping the user to take action without forgetting.
Additionally, to enhance communication among family members, it would be beneficial to add the ability to automatically attach relevant images, videos, and links based on specific words or phrases in a written message, allowing for visual information to be shared in addition to text-based messages, making communication richer.
In addition, the system can enhance security by automatically detecting and blurring certain personal or confidential information when documenting call transcripts, in accordance with privacy protection guidelines and natural language processing techniques.
When converting call contents into text, it is also effective to incorporate a translation function into multiple languages to meet the diverse needs of users. In a multicultural environment where family members speak different languages, it is possible to automatically translate the call contents and send messages in a language that each member can easily understand.
To further enhance convenience, a schedule function will be added that allows users to customize the timing of message sending. Users can set messages to be sent at a specific date and time, rather than just immediately, allowing them to optimize the timing at which their family members receive information.
Finally, quick action buttons for replying or confirming messages have been provided so that recipients can easily take action on received messages, enabling quick feedback and efficient communication, making information sharing among family members even smoother.
(Example 3)
The embodiment of the present invention is a system that checks phone numbers during a call, and has a means for linking with the police when linking with the police if the number has a history of misuse. Specifically, the system is equipped with a function that checks the incoming number against a database during a call, and automatically notifies the police if it detects that the number has a history of misuse.
The linking means is realized by the specific processing unit 290 of the data processing device 12 as a means for checking a telephone number during a call and linking with the police if the number has a history of misuse. The linking means may also be realized by the control unit 46A of the headset type terminal 314. The correspondence between each means and the device or control unit is not limited to the above-mentioned example, and various modifications are possible.
The extraction means is provided for extracting caller information contained in the signal of each call sent through the communication network, and can accurately obtain the caller's telephone number from the call data using digital signal processing technology. The extraction means also has a signal analysis function for analyzing Caller ID information and identifying the telephone number. The system can also accurately identify the called number by utilizing the signal extraction function for extracting the caller's telephone number from the call signal and an analysis algorithm specialized in decoding Caller ID information.
The matching means has a function of matching the extracted telephone number with a database that collects telephone numbers suspected of fraudulent use and organizes them by category, and the matching process can be performed in real time through a database management system, and rapid matching can be performed while the call is in progress using high-speed search algorithms and indexing technology.The matching means also has a high-performance database search function and indexing function for rapid search and matching by referring to a database that lists numbers with a history of misuse.
The linking means has a function of automatically reporting to the police when a number with a history of abuse is detected, includes an interface function with the reporting system, generates report information in a data format based on the protocol with the police reception system when reporting, and transmits it to the police reception system using a secure communication channel. The contents of the report information include information such as the detected abused phone number, the date and time of the call, the duration of the call, and the telecommunications carrier used by the caller, and uses encryption technology and authentication systems to protect personal information and ensure the accuracy of the report. In addition, the linking means is compatible with multiple communication protocols and reporting systems, has an adapter function for smooth data exchange between the systems, and has a function of adjusting the format of the report information to meet the requirements of the police reception system in the reporting process, and selecting an appropriate reporting protocol to make the report. When the reporting process is activated, the system transmits report information to the police reception system and obtains a report reception confirmation. This confirmation is used as information for the system to record that the report was made correctly and store it as a report history. The report history is used for future analysis and improvement, contributing to the efficiency and accuracy of the report process. This linking function uses a report data generation function to create a report format containing the necessary information, and transmits the information to the police reception system via a secure report transmission function. Data encryption and system authentication functions are implemented to ensure the security and reliability of reports. In addition, the system's linking means has an adapter function that is compatible with multiple report protocols and enables data exchange with different report systems, and this adapter function automatically adjusts to a format suitable for the protocol requirements at the time of reporting using a report protocol selection function, and transmits report information and confirms receipt. The report history recording function records the success of reports, which is used to analyze and improve system performance.
As an example of data collection without sensors, a function for users to report suspicious calls on an application or web interface is provided. The user enters their experience of the call and any suspicious points they noticed during the call into a form, and the information is registered in a database. The data collected through this manual report can be added to a list of phone numbers that are automatically matched in the database, contributing to improved accuracy in detecting numbers with a history of abuse. As an example of data collection without sensors, an input function is provided for users to report suspicious calls. Through an interactive reporting form, users register the contents of suspicious calls in a database, and the information collected is used to detect numbers with a history of abuse. This manual reporting system contributes to the expansion of the number matching database by providing additional data based on the user's experience and intuition.
The system can be equipped with a function to strengthen the database update mechanism. For example, newly confirmed abused numbers will be automatically added to the database even after they are reported. In addition, when a suspicious call is reported, the credit information of the number will be checked against other databases and combined with information based on user reports to more accurately identify abuse history. To ensure the integrity of the database, a regular cleaning process will be carried out to eliminate incorrect and outdated information. In addition, to strengthen cooperation with the reporting system, a function will be introduced to directly link with the crime database provided by the police, obtain crime information in real time during the matching process, and improve the accuracy of the matching results.
To improve the immediacy of reports, the matching process begins the moment the call is initiated, and if a number with a history of abuse is detected, the caller will be given the option to sound an alarm or automatically hang up. In addition, when the call is hung up, a recording of the call can be saved on the caller's behalf to assist police investigations. When police intervene, a detailed report including the caller's location and call history is automatically generated, helping to speed up criminal investigations.
The user interface will be updated to include a feature that allows users to check the progress of the reporting process in real time to increase transparency in the reporting system. Users will be able to check the results of their reports and feedback from the police, which will increase trust in the system. The system will also provide statistical data and trend analysis on abused numbers, providing information to users to be cautious about calls.
Furthermore, machine learning technology will be introduced to identify numbers with a history of abuse, and new numbers likely to be abused will be predicted by analyzing various indicators such as call patterns and call frequency. This will enable proactive updates to the database, strengthening measures to prevent unknown criminal activity. A function will also be added that allows users to provide direct feedback on the effectiveness of the reporting system, which will help improve the system. Feedback will be provided anonymously, protecting user privacy while collecting valuable information that will contribute to improving the system.
To make the phone number check during a call more effective, AI will learn the patterns of various fraudulent scams that users may face and implement a function to flag in real time when certain words or phrases are detected during a call. This will allow malicious behavior to be inferred and detected from the content of the call, rather than simply matching the phone number with abuse history in the database. In addition, the reporting process will be simplified by providing shortcuts and buttons on the smartphone interface that allow users to easily report calls that they suspect are fraudulent. This will allow the database to be updated more quickly and improve protection for other users.
To strengthen cooperation with the police, the reporting system will be integrated with location tracking capabilities to help police respond quickly based on reported information, helping them track and capture criminals. Artificial intelligence will also be built into the reporting system to analyze crime patterns from report data and provide police with information to plan preventive vigilance activities. Using such predictive analytics will help prevent future crimes.
To facilitate data sharing with the police, the system will receive real-time information on fraud cases and other crimes known to the police and update the database. This will ensure that the reporting system operates based on the latest crime information and strengthen measures to protect users. In addition, the system will be equipped with a block list function, allowing users to register suspicious numbers and block calls from them. This will allow users to directly control the risks themselves.
Educational programs will include online courses and workshops to provide users with information to recognize and prevent fraud methods. This will provide users with the knowledge to protect themselves and raise security awareness in society as a whole. Examples of fraud cases that were prevented by using the reporting system will also be shared to help users understand the effectiveness of the system.
Finally, through a system update, if a call is deemed likely to be fraudulent, a warning message will be automatically sent to users to warn them of fraud. This will allow users to immediately recognize the possibility of a fraud and take appropriate action.

The process flow for each example is explained below.

(Example 1)
Step 1: When a call comes in from an unregistered or withheld phone number, the generative AI answers.
Step 2: After the call, document the matters discussed in Chat GPT.
Step 3: Send a written message via messenger app or email. If the caller claims to be a family member or acquaintance, verify the call using special questions or a secret code. The delivery company will connect you using a secret code. Check the phone number during the call, and if the number has a history of abuse, contact the police.
(Example 2)
Step 1: After the call, document the matters you wish to discuss in Chat GPT.
Step 2: Select the documented matter and designate it as being sent to family members.
Step 3: Send your message to your family via messenger app or email.
(Example 3)
Step 1: During a call, the incoming number is checked against a database to detect if it is a number with a history of abuse.
Step 2: If the number has a history of abuse, the call will automatically be reported to the police.
Step 3: After the call is over, the matter is documented in the chat GPT. The documented matter is used for communication with the police.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

(Example 1)
The embodiment of the present invention is a system that includes a response means in which a generative AI responds when a call is received from an unregistered or unnotified phone number, and a transmission means in which the matter is documented in chat GPT after the call is ended and sent via a messenger app or email. Furthermore, as an emotion recognition means that combines an emotion engine that recognizes the user's emotions, the tone of the user's voice and choice of words are analyzed during the call to estimate the emotion. Based on the estimated emotion, the generative AI's response and the expression of the documented matter are adjusted. For example, if the user shows an anxious emotion, a calmer expression is used to provide a sense of security.
The response means is realized by the specific processing unit 290 of the data processing device 12, and provides a function of the generative AI responding to an incoming call from an unregistered or unnotified phone number. The transmission means is realized by the specific processing unit 290 of the data processing device 12 as a function of documenting the matter in chat GPT after the call is ended and sending it by a messenger app or email. The emotion recognition means is realized by the control unit 46A of the headset type terminal 314 as a function of analyzing the tone of the user's voice and choice of words during the call and estimating emotions, but may also be realized by the specific processing unit 290 of the data processing device 12. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The response means includes a dialogue generation means that combines natural language processing and speech synthesis. The dialogue generation means automatically responds to unregistered or unnamed incoming calls, enabling a dialogue with the user. When an unregistered or unnamed incoming call is detected, the generative AI responds using a pre-trained conversation model. This conversation model is trained based on a variety of dialogue data so that it can handle various scenarios, and provides appropriate answers and guidance to the callee's questions and requests. In addition, the response means has an AI dialogue generation function, and can automatically provide an appropriate response to an incoming call from an unregistered or unnamed phone number. This function combines a dialogue management function for understanding the callee's purpose and request at the early stage of the call and providing a corresponding response, and a voice generation function for converting the generated response into a natural voice. The dialogue management function can analyze the callee's intention based on the detection of specific keywords and phrases and generate an appropriate response. The voice generation function can convert a text-based response into voice in real time, providing the callee with a natural conversation experience.
The transmission means includes a chatbot and a documentation means using natural language understanding technology. After the call is ended, the contents of the call are accurately documented using an advanced natural language understanding model such as chat GPT. The documentation means has the ability to extract and summarize the main points of the call, and can generate a document that conveys the purpose concisely and clearly. The generated document is sent to the user via a messenger app or email. This process includes a link to the user's email address or messenger account, and the document is automatically sent in the appropriate format. The transmission means can also convert the contents of the call into text and provide it in an accessible form to the user. After the call is ended, the call content documentation function is activated and can extract and summarize the main points of the conversation. This summarized text is sent to the user's designated email address or messenger app through an automatic delivery function. This automatic delivery function includes an email delivery function and an app link function to properly format the document and ensure that it is delivered to the designated destination.
The emotion recognition means includes a voiceprint analysis means for performing voice analysis, and a language analysis means for estimating emotions from the choice of words. The voiceprint analysis means can detect characteristics of the user's voice, such as tone, pitch, and speed, using the microphone of the smart device, and can estimate the user's emotional state from these voice characteristics. The language analysis means can analyze the content of the user's speech and extract emotional context from the words and phrases used. These analysis results are used by the generative AI when making a response or adjusting the expression of documented matters. For example, if the user shows anxiety, the expression of the response or document is adjusted to be more gentle and reassuring. In addition, the emotion recognition means has a function of estimating emotions from the characteristics of the user's voice and language use during a call. The voiceprint analysis function can extract features such as the pitch, strength, and speed of the voice from the voice data collected by the microphone, and can estimate emotions by analyzing these characteristics. The language emotion analysis function can process the language data during a call, analyze the emotional meaning of the words and phrases used, and grasp the user's emotional state. These analytics can be used to adjust the tone of the AI's responses and the wording of documented calls to provide an appropriate emotional response to the user.
Data collection methods that do not involve sensors include text data entered by users themselves and feedback on system usage. These are provided to the system through the user input reception function and feedback collection function, and are used as a valuable source of information for service improvement.
These means are intended to respond quickly and effectively to user requests and improve the quality of communication. In addition, the implementation of each means is flexibly performed by the data processing device or the control unit of the smart device, and can be modified in various ways to improve the efficiency and usability of the system.
As an additional function, the system of the present invention can be equipped with an outline prediction means for predicting the caller's intention before answering an unregistered or unnamed call. This allows the response means to generate a more accurate dialogue, realizing a meaningful exchange for the user. In addition, when the generative AI responds, it can have a language selection function based on the caller's country or region, enabling automatic response in multiple languages.
As for the delivery method, in addition to documenting the contents of the call, it can also highlight important keywords and phrases to help users quickly understand the document, and automatically generate action items based on the documented content and add them to the user's task list.
The emotion recognition means can detect changes in the user's emotions during a call in real time and dynamically adjust the tone and tempo of the conversation of the response means. Protocols can also be built in that, when a particular emotion is detected, prompt the user to contact an expert who can provide special support or advice according to the emotion.
The response tool can be enhanced with a personalization feature that analyzes the caller's past call history and related data to provide a more personalized response, making it possible to provide more relevant information to the user and increase the usefulness of the response.
Regarding sending methods, a function can be added that suggests follow-up actions based on the documented content of the call. For example, tasks and events included in the call content can be automatically registered in a calendar app to help users manage their time.
Furthermore, the emotion recognition means may have the ability to detect the user's stress level or tension and provide appropriate advice on how to reduce stress or links to relaxation content, thereby supporting the user's mental health and promoting overall well-being.
The system of this embodiment is considered to have a number of additional functions for further improving its functionality. For example, a voiceprint recognition means can be added to identify a caller by analyzing the caller's voiceprint and comparing it with previous call data for an unregistered or unnamed call. This allows the response means to respond more appropriately based on information if the caller has previously interacted with the system. The voiceprint recognition means also functions as a security measure, providing a reliable call experience for the user.
Regarding the means of transmission, when documenting the contents of the call, the function of structuring the contents of the call and classifying the text according to the importance of the information will be considered. This will allow users to grasp important information more quickly when reading the document. In addition, the documented content can be customized according to the user's preferences and past behavior patterns to provide a more personal experience.
The emotion recognition means detects the user's stress level during a call, and if it is estimated that the user is under high stress, it provides guidance on relaxation methods and psychological support related to the content of the call, thereby providing a service that allows users to relax and reduce stress through the call.
Additionally, the response tool will automatically suggest follow-up actions to users based on the content of the call, for example providing links to additional information about products or services suggested during the call or suggesting next steps of action to help users make decisions.
To improve response methods, we are considering a function that automatically changes the response style according to the caller's intentions. For example, if the caller indicates an emergency situation, we will adjust the AI to provide quick and accurate instructions. This will create a system that can immediately respond to the caller's needs.
These additional functions are intended to improve the user experience and enable accurate understanding of call content and rapid response. Each function is expected to maximize the capabilities of data processing devices and smart device control units, further enhancing the usability of the system.
(Example 2)
The embodiment of the present invention is a system that includes an adjustment means for documenting the matter in chat GPT after the call ends, and adjusting the expression of the document using an emotion engine that recognizes the user's emotions when sending it by messenger app or email. Specifically, the documented matter is input to the emotion engine, and the user's emotions are analyzed. Based on the analysis results, the expression of the document is appropriately adjusted. For example, if the user is feeling happy or excited, the user's emotions are shared by using brighter and more lively expressions.
The adjustment means is realized by the specific processing unit 290 of the data processing device 12 as a function of recognizing the user's emotions and adjusting the expression of the document when documenting the matter in chat GPT after the call ends and sending it by messenger app or email. This adjustment means provides a function of appropriately adjusting the expression of the document based on the analysis result by the emotion engine. The adjustment means may also be realized by the control unit 46A of the headset type terminal 314. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The conversation content extraction means converts the contents of the call into text data by making full use of voice recognition technology. After receiving the voice signal, the conversation content extraction means uses noise reduction means to remove background noise and improve the accuracy of the voice-to-text conversion. Furthermore, the conversation content extraction means uses cutting-edge voice recognition technology to accurately convert the contents of the call into text data, and in the process, the noise reduction means removes background noise and improves the conversion accuracy.
The text processing means corrects syntactic errors in the converted text data and performs grammar checking via the grammar checking means to maintain linguistic fluency. The text processing means also performs adjustments to ensure that the grammar checking means of the converted text data ensures accuracy of grammar and maintains the natural flow of the document.
The emotion recognition means includes an emotion extraction means based on text mining and emotion analysis techniques, which estimates the user's emotion from the wording and context of the generated text data. The emotion extraction means also identifies words, phrases, and grammatical patterns that express various emotions and classifies them into emotion categories such as positive, negative, and neutral. The emotion emphasis means also adjusts the tone and phrasing of the text according to the extracted emotions, making modifications to better convey the user's emotional state. The emotion extraction means also reads the user's emotion from the language patterns expressed in the document and adjusts the tone of the document based on the emotion.
The adjustment means changes the expression of the text based on the emotion data analyzed by the emotion recognition means. When a positive emotion such as joy or excitement is detected, the expression enhancement means replaces the vocabulary used with brighter and more lively words to give the message a favorable impression. When a negative emotion such as sadness or anger is detected, the expression mitigation means softens the tone of the message and selects phrases that show empathy and understanding. When the user's emotion is positive, the expression enhancement means energizes the document, and when the user's emotion shows negative emotion, the expression mitigation means uses a more gentle expression.
The message sending means includes a function for linking with a messenger application or an email client and sends the adjusted text in an appropriate format. Furthermore, the transmission protocol selection means selects an optimal transmission protocol according to the platform and settings of the recipient and ensures delivery of the message. Furthermore, the user interface presentation means provides a preview screen for a final review of the document before sending, allowing the user to make final corrections as necessary. Furthermore, the message sending means sends the adjusted text in a format suitable for the messenger application or the email client, and the preview screen provides an interface for the user to make a final check of the document.
The above processes are realized by a specific processing unit built into the control unit of the smart device or the data processing device, depending on the device and settings used by the user. Moreover, these means are designed as modularized components, and can be easily replaced or expanded as system components. Furthermore, the correspondence between each means is set flexibly, and various changes can be made to accommodate system upgrades and customization. Moreover, this series of processes is modularized so that it can flexibly respond to devices and environments, and is designed to facilitate system upgrades and customization.
This embodiment can be further expanded to add a function for deeper understanding of the user's emotions and improving the quality of communication. For example, by incorporating a facial expression recognition function during video chat into the emotion engine, emotions can be analyzed from visual information as well, resulting in more accurate emotion judgment. To improve the accuracy of emotion recognition, the tone and pitch of the user's voice can also be analyzed and reflected in the text. Furthermore, by analyzing the user's past communication history and response patterns, the system can learn the individual's emotional expression style and adjust the text accordingly in a more personalized manner.
The text processing means has a function to suggest phrases inspired by literary works and poetry to enhance the diversity and creativity of expressions, allowing users to express their emotions more richly. It can also select appropriate expressions according to the environment and situation in which the communication takes place, taking into account the social context and cultural background.
The messaging tool will add a function to predict the emotional impact of the text sent on the recipient, allowing users to communicate more responsibly. In addition, AI can predict the recipient's reaction and use that information to suggest the next communication strategy the user should take.
Overall, these features will help users to reflect emotions accurately and sensitively in their communications, contributing to building deeper human relationships. These advanced methods can be effectively used by individuals as well as corporate customer support and CRM systems to deepen customer relationships. Furthermore, these systems are expected to be applied in fields where human emotions play an important role, such as user education and counseling.
The system leverages an emotion engine to provide document expression adjustments based on the user's emotions. To extend this functionality, sensors are integrated to capture the user's biometric information, allowing for a more accurate reading of emotions from physiological responses such as heart rate and skin conductivity. Data from the sensors is analyzed in real time, allowing the tone of the document to be adjusted on the fly.
In addition, it is equipped with a personalized learning function that continuously analyzes the user's daily communication to understand the user's unique expression style and preferences, allowing the system to suggest more natural and personalized documents that reflect the user's personality.
It will also enhance multilingual support, allowing it to capture emotional nuances in different languages, enhancing international communication and understanding among multilingual users.
To protect user privacy, the company will also add features to anonymize and encrypt emotion data and strengthen security, allowing users to use the system with peace of mind.
Aiming for applications in the fields of education and counseling, we will develop a function that uses the results of emotion recognition to support communication skills training. The training program will include practicing emotional expression and learning appropriate communication techniques.
Finally, to improve the usability of the system, we make the user interface rich and intuitive, supporting a variety of gestures and voice commands, and allowing users to easily intervene in the document adjustment process, allowing users to fine-tune the expression at their own will and achieve more personal communication.
(Example 3)
The embodiment of the present invention is a system that includes an emotion engine that recognizes the user's emotions during a call and an adjustment means that adjusts the response and dialogue content to match the user's emotions. Specifically, the user's tone of voice and choice of words are input to the emotion engine during a call to estimate the user's emotions. The generative AI's responses and dialogue content are adjusted based on the estimated emotions. For example, if the user shows sad emotions, words of empathy are used to provide encouragement and support.
The adjustment means is realized by the control unit 46A of the headset terminal 314 as a function of recognizing the user's emotions during a call and adjusting the response and the content of the dialogue to match the user's emotions. This adjustment means provides a function of inputting the user's tone of voice and choice of words into the emotion engine during a call, estimating the user's emotions, and adjusting the generative AI's response and the content of the dialogue. The adjustment means may also be realized by the specific processing unit 290 of the data processing device 12. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The voice data collection means includes a microphone with high sensitivity and noise canceling function. The microphone is equipped with a digital signal processing algorithm to remove ambient noise and clearly record the user's voice. The voice data collection means also employs a highly sensitive microphone that captures subtle acoustic characteristics from the user's speech and combines this with digital signal processing technology to effectively remove ambient noise. The microphone accurately captures the characteristics of the user's voice and provides basic data for detecting changes in emotions.
The voice feature extraction means extracts features that may reflect human emotions from the voice signal. This extraction means includes a spectrogram analysis function and a pitch tracking function for performing acoustic feature analysis, and an emotion identification function using an acoustic model. The voice feature extraction means also uses acoustic analysis tools such as a spectrum analysis function and a pitch analysis function to extract features that suggest emotions from the voice signal, and supplies these data to the emotion estimation model.
The emotion analysis means includes an emotion estimation model based on machine learning, which estimates the user's emotional state using the features extracted by the voice feature extraction means as input. The emotion estimation model is composed of classifiers such as neural networks, support vector machines, and decision trees trained based on training data, and classifies the user's emotions into categories such as positive, negative, and neutral. The emotion estimation model improves its accuracy through continuous learning, and evolves to be able to respond to subtle changes in the user's perception of emotions. In addition, the emotion analysis means has an emotion estimation model that utilizes machine learning technology, and analyzes the user's emotions based on the extracted voice features. This model combines various machine learning algorithms to identify emotion categories from the user's utterance, and estimates the user's emotional state based on this. The estimated emotional state is important information for adjusting the system's response to the user's utterance and behavior.
The dialogue adjustment means includes a response generation function using a generative AI model, which dynamically adjusts the content of the dialogue according to the emotion estimated by the emotion analysis means. The response generation function utilizes natural language generation technology to generate a response sentence using words and tones appropriate for the user's emotion. For example, if the user is expressing sad emotion, a response including empathetic expressions and comforting words is generated. This response generation function has a context-aware processing function for taking into account the context of the conversation and providing content that matches the user's emotion and the purpose of the communication. In addition, the generated response is constructed to be natural to the user and to meet the emotional needs. The response generation function is realized by a machine learning model trained on a large-scale conversation dataset and can adapt to the user's language and speaking style. In addition, the dialogue adjustment means includes a response generation function using a generative AI model, which generates dialogue content appropriately corresponding to the estimated emotion. This function selects words and a dialogue tone that correspond to the user's emotion and provides a response that matches the user's current emotion and the context of the conversation. The response generation function is designed to perform context-aware processing and provide the information and support required by the user in an appropriate form. This feature is capable of generating responses that adapt to a user's language and emotions based on models trained from large amounts of conversational data.
Non-sensor-based data collection methods include text information entered directly by users into the system and metadata obtained from call records. These are used as supplementary information when analyzing user behavioral patterns and preferences, and contribute to improving the accuracy of the emotion engine and optimizing response generation functions. User-entered text information is also used as training data for emotion estimation models as part of the emotion analysis method.
The system has the ability to analyze the user's emotions in real time during a call and automatically adjust the dialogue content, so it can integrate facial expression recognition technology in addition to voice data to capture even subtler changes in emotions. It uses a webcam or a camera on a smart device to analyze the user's facial expressions and improve the accuracy of emotion analysis. This added facial expression recognition function will more accurately recognize the user's emotions and make it possible to adjust the dialogue according to even subtler emotional changes.
It is also conceivable to incorporate wearable devices to capture the user's physiological signals. Measuring physiological responses such as heart rate and electrodermal activity can provide deeper insight into emotions that cannot be captured by tone of voice or facial expressions alone. Integrating this data can further improve the accuracy of emotion analysis and generate more appropriate dialogue responses.
It would also be effective to add customization features to the dialogue adjustment method that take into account the user's cultural background and personal values. A user profile can be constructed and the tone and content of the dialogue can be further personalized based on that information. This makes it possible to provide detailed support tailored to each individual user.
Furthermore, in order to promote the evolution of the emotion estimation model, we will build a mechanism to collect emotion data through crowdsourcing and incorporate feedback from a variety of users to continuously update the model, resulting in a flexible system that can handle a variety of emotional expressions and languages.
Applications in the fields of education and mental health care can also be considered. For example, in the field of education, it could be used to present educational materials adapted to students' emotions or in counseling sessions. In mental health care, it could recognize the user's emotions and provide dialogue support to reduce stress and anxiety. This would enable a more effective approach to the problems the user is facing.
Regarding privacy protection for the system, we will strengthen security measures to safely store users' emotional data and minimize the risk of information leakage by using appropriate access control and encryption technology. This will allow users to use the system with peace of mind.
Ultimately, it is hoped that the personalized interaction experience provided by this system will be developed into services that improve the quality of users' lives.
The embodiment of the present invention is applicable not only to telephone calls but also to video conferencing and online education platforms. For example, a lecturer can grasp the emotions of students in real time and adjust the progress of the curriculum according to the emotions, thereby providing a more effective learning experience. Even in video conferencing, dialogue management that reflects the emotions of participants is performed, promoting a productive and positive meeting environment.
The system could also be enhanced to monitor health conditions based on emotional responses: for example, if a user's tone of voice or manner of speaking indicates negative emotions over a period of time, a mental health professional could be notified to intervene if necessary.
Furthermore, to further improve the emotion engine, a function will be incorporated that will analyze the emotional patterns of the user's daily life and provide advice on long-term emotion management and stress reduction based on that information. By analyzing the user's daily rhythm and activity patterns and predicting emotional ups and downs, the system will suggest content for relaxation and motivation at appropriate times.
This system is also expected to be applied in the field of customer support. For example, if call center operators could grasp the customer's emotions in real time and detect negative emotions such as dissatisfaction or anger, they could take immediate action to address the issue, contributing to improving customer satisfaction.
In the fields of games and entertainment, dynamic changes in content based on the user's emotions are expected to have the effect of increasing immersion and enjoyment. In-game characters will react to the player's emotions, changing the story development and dialogue, creating a more personalized experience.
In addition, we will integrate voice assistants and virtual reality (VR) to realize more natural dialogue based on the user's emotions. Voice assistants will understand the user's emotions and provide information and services tailored to individual needs. In the VR environment, the scenario and environment will change according to the user's emotions, providing an experience that matches the emotions in real time.
This system also has the potential to revolutionize the design of user interfaces (UI) and user experiences (UX). Using emotion recognition technology, it can provide UI and UX optimized for the user's emotions, increasing user satisfaction. For example, websites and applications can grasp the user's emotions in real time and adjust the way content is presented and the form of interaction.
Ultimately, it is hoped that the emotion regulation function provided by this system will become widely used throughout society as a tool to improve the quality of human relationships and increase the effectiveness of communication.

The process flow for each example is explained below.

(Example 1)
Step 1: When a call comes in from an unregistered or withheld phone number, the generative AI answers.
Step 2: After the call, document the matters you wish to discuss in Chat GPT.
Step 3: The documented requirements are input into the emotion engine to analyze the user's emotions.
Step 4: Based on the analysis results, the representation of the document is adjusted.
Step 5: Send the adjusted document via messenger app or email.
(Example 2)
Step 1: After the call, document the matters you wish to discuss in Chat GPT.
Step 2: Input the documented requirements into the emotion engine to analyze the user's emotions.
Step 3: Adjust the representation of the document based on the analysis results.
Step 4: Select the adjusted document and specify the recipient (e.g., family).
Step 5: Send the document to the specified recipient via messenger app or email.
(Example 3)
Step 1: During a call, the user's tone of voice, choice of words, etc. are input into the emotion engine to analyze the user's emotions.
Step 2: Based on the analysis results, adjust the generative AI's responses and dialogue content.
Step 3: After the call, document the matter in the chat GPT. The documented matter is used to input the emotion engine.

The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input for the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization. The identification processing unit 290 performs the above-mentioned identification processing while using the data generation model 58.

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.
[Fourth embodiment]

Figure 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical able-bodied person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is performed in a secure state.

The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and legs. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. Data generation model 58 and emotion identification model 59 are used by the identification processing unit 290.

In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a ChatGPT server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance). Next, the identification process by the identification processing unit 290 of the data processing device 12 will be described.

(Example 1)
The embodiment of the present invention is a system that includes a response means in which a generative AI responds when a call is received from an unregistered or withheld phone number, and a sending means that documents the matter in chat GPT after the call ends and sends it via a messenger app or email. In addition, the system includes a confirmation means that uses a special question or password to confirm if the caller claims to be a family member or acquaintance, a connection means that connects the delivery company to the person using a password for the delivery company, and a linking means that checks the phone number during the call and links to the police if the number has a history of abuse.
The response means is realized by the specific processing unit 290 of the data processing device 12, and the generative AI responds to an incoming call from an unregistered or unnotified phone number. The transmission means is realized by the specific processing unit 290 of the data processing device 12 as a means for documenting the matter in chat GPT after the call is ended and sending it by messenger app or email. The confirmation means is realized by, for example, the control unit 46A of the robot 414, and confirms family and acquaintances using dedicated questions and passwords. The connection means is realized by the control unit 46A of the robot 414 as a means for a delivery company to connect to the person using a password for the delivery company. The linking means is realized by the specific processing unit 290 of the data processing device 12 as a means for checking the phone number and linking to the police if the number has a history of misuse. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The response means includes a speech understanding function that utilizes voice recognition technology, and responds to calls from unregistered or withheld numbers in natural language generated by a pre-trained AI. This AI has learned a large amount of call data and has the ability to generate responses that can respond to various scenarios. When a call is received, the AI analyzes the content of the call in real time and selects an appropriate response to proceed with the conversation. The response content is customized to the nature of the call, so questions from sales calls are answered politely and suspicious calls are answered carefully. In addition, the response means can also use speech synthesis technology and speech content understanding technology to respond to calls from unregistered or withheld phone numbers, and a pre-trained AI can respond appropriately. This AI can learn multiple response scenarios, understand the intention of the call, and provide a natural dialogue according to the situation. The AI instantly selects a response based on the content of the call and responds customized to the purpose of the call, so inquiries from sales calls can be handled appropriately and suspicious calls can be answered carefully.
The transmission means includes voice recognition technology that converts the contents of the call into text and natural language generation technology that generates a summary of the call based on that text. When the call ends, the AI converts the contents of the call into text data and summarizes and documents the contents. The generated document is sent to the user via a communication means such as a messenger app or email. This process allows the user to check the details of the call at a later date and ensures that important information is not overlooked. The transmission means can also convert the contents of the completed call into text using voice recognition technology, then summarize it using natural language processing technology, and send the generated document to the user via a messenger app or email. The user can check the details of the call at any time through the sent document, and important information is recorded and not overlooked.
The verification means has the function of verifying the identity of the caller by using specific questions or passwords when the caller claims to be a family member or acquaintance. In this method, the AI uses questions and passwords set by the user, and verifies the identity of the caller by providing the correct response. If identity verification is successful, the AI continues the call and transfers the call to the user if necessary. This process prevents unauthorized access and fraud while ensuring smooth communication if the caller is the real person. The verification means can also have the AI present questions and passwords set by the user to callers claiming to be family members or acquaintances, and continue the call or transfer the call to the user only if an accurate answer is given. This function prevents unauthorized access and fraud, while enabling smooth communication if the caller is confirmed to be the real person.
The connection means has a function that allows a person in a specific occupation, such as a delivery worker, to talk to the person by using a secret code for the occupation. With this method, AI receives the secret code for the occupation, verifies that it is the correct code, and then connects the call to the person. The secret code for the occupation is important for ensuring the security of the call, and if an incorrect code is entered, the call will not be connected. The connection means can also have a function where AI receives a secret code specific to the occupation, such as a delivery worker, verifies its validity, and then connects the call to the user. The security of the call is ensured by this secret code, and if the incorrect code is entered, the call will be disconnected.
The linking means has a function of checking the phone number in real time during a call, and automatically linking to the police or related organizations if the number has a history of misuse. This means refers to a list of suspicious numbers stored in a database and checks whether the incoming number is included in the list. If it is determined that the call is from a suspicious number, the system immediately starts a reporting protocol to the police and takes appropriate action depending on the situation. The linking means can also monitor the number being called in real time, and automatically link to related organizations such as the police if the number has a history of misuse in the past. When a suspicious number is detected, a reporting protocol to the police is activated and a prompt response is taken.
These measures are implemented by combining multiple AI technologies and collaboration functions to enhance user security and convenience. The data processing device and the control unit of the smart device cooperate to provide a platform for flexibly realizing these measures. Each measure is designed not only to function independently but also to work together to provide more advanced services. Through these measures, users can automate responses to calls from unregistered or unnotified phone numbers and protect themselves from potential risks that occur in daily life. In addition, these measures are built around AI technology to improve user security while simultaneously streamlining daily communication. As an alternative to sensor-free data collection, a process in which users manually enter information and the system uses that information is possible. Through these measures, users can automate responses to unregistered or unnotified calls and strengthen protection from unforeseen events.
Further functions can be added to the system of this embodiment to enhance its convenience. For example, the response means can be added with a function to analyze the user's emotions during a conversation, and the AI can infer the emotions from the tone of the other party's voice and manner of speaking and respond accordingly. This allows for a more careful and empathetic response when the other party is feeling angry or anxious, improving the quality of the call.
The sending method will add a feature that automatically integrates documented calls into users' calendars and reminders, helping them remember appointments and tasks generated during calls and improving productivity.
The verification method has a function to verify the identity of the user using biometric information. For example, biometric information obtained from a smartwatch or fitness tracker can be used to verify that the person on the other end of the line is actually a family member or acquaintance. This reduces the hassle for users while achieving a higher level of security.
The connection method will add a function that allows the delivery company to scan a QR code (registered trademark) or NFC tag to authenticate the user and connect the call directly to the user, eliminating the need for a password and making it possible to communicate with the delivery company more quickly and safely.
The collaboration will add a function to detect suspicious keywords from voice data during phone calls and automatically analyze the content. If a call is deemed suspicious based on the keywords, the system will first warn the user before reporting it to the police, helping them make more accurate decisions.
With these additional functions, the system of this embodiment not only automatically answers calls, but also more efficiently supports security and schedule management in daily life. In addition, by linking various AI technologies with databases, it provides greater security and convenience to the user's life. Furthermore, by integrating these technologies with the user's smart devices and IoT devices in the home, it becomes possible to realize a more personalized experience.
The system of this embodiment can be further expanded, and in order to increase convenience and security, the response means can be added with a function to analyze the caller's voiceprint and compare it with the voices of registered family members and acquaintances. This makes it possible to more accurately determine whether the caller is the real person and to continue the call safely without using specific questions or passwords.
The transmission means can incorporate a function that uses voice recognition and natural language processing to translate the contents of the call into multiple languages selected by the user, facilitating communication between users who speak different languages, facilitating use in international business situations and within multilingual households.
The verification method can include a feature that allows users to verify their identity by making specific gestures or movements on the camera, making it possible to easily and quickly verify the identity of a user during a call, further enhancing security.
Regarding the means of connection, AI will be able to add a function to verify the identity of the caller by comparing their location information with delivery schedule information, making it possible to confirm that the delivery company is calling from a location where the goods are actually being delivered, thereby guaranteeing the authenticity of the call.
The collaboration method will add a function that automatically notifies not only the police but also the user's designated emergency contact when a suspicious call is detected, enabling a rapid response in the event of an emergency.
These additional functions allow for flexible response to user needs while maintaining the basic structure of the existing system. In addition, by linking with smart devices and IoT devices in the home, the system provides a seamless experience in the environment in which the user uses the system on a daily basis. With these additional functions, the system of this embodiment goes beyond automatic answering of calls, improving the security and convenience of the user in all aspects of daily life.
(Example 2)
The embodiment of the present invention is a system that has a sending means that can send the matter to family members when documenting the matter in chat GPT after the call ends and sending it by messenger app or email. Specifically, the matter can be sent to family members by selecting the document created after the call ends and specifying "family members" as the destination.
The sending means is realized by the specific processing unit 290 of the data processing device 12 as a means for documenting the matter in chat GPT after the call ends and sending it to the family by messenger app or e-mail. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The means for documenting the contents after the end of the call includes a conversation content analysis means using a voice recognition technology. This analysis means converts the voice data of the call into text and extracts the main points of the call. In addition, the conversation content analysis means using the voice recognition technology has a function of converting the voice data generated after the end of the call into text and extracting the main points of the call from the text data. The voice recognition technology utilizes a model based on deep learning and is trained to support various languages and accents. In addition, in order to support various languages and accents, this analysis means utilizes a deep learning model and proceeds with learning based on a large amount of voice data. The conversation content analysis means performs morphological analysis and syntactic analysis to identify important information exchanged in the conversation and content requiring action. In particular, a keyword extraction function is used to identify words and phrases frequently used in the conversation, and the main points of the call are documented based on these. In addition, in order to clarify the main points of the conversation, a morphological analysis function and a syntactic analysis function are combined to identify important information exchanged during the conversation and content requiring action. The keyword extraction function identifies words and phrases that appear frequently in the conversation, and documents based on this information.
The means for formatting the documented content into a message includes a text editor function, which organizes the parsed text and arranges the document structure. The message formatting means provides an interface that allows a user to easily check the content and add edits or additional information as necessary. The means for formatting the documented content into a message also includes a text editor function, which organizes the parsed text and arranges the document layout. The means for formatting the message provides an interface that allows a user to check the content of the message and add edits or additional information.
The sending means sends a message to a specific family member using a destination designation function. This destination designation function works in conjunction with the user's contact list to automatically send a message based on the contact information of the selected family member. The sending means also has a function of confirming message transmission and managing a transmission history, so that the user can track the status of the sent message and resend it as necessary. The message is sent in conjunction with a messenger app or an email app. The linking function with the messenger app or the email app performs authentication using the user's account information and transmits the message safely. The sending means also encrypts the message as a security measure and performs an authentication process at the time of transmission to ensure privacy protection. The sending means also transmits the message in an appropriate format depending on the message transmission method selected by the user. For example, the message is sent as an instant message in the messenger app, and as an email in the email app. The message sending means has a destination designation function for sending a message to a family member selected by the user, and works in conjunction with the user's contact list to automatically send a message based on the contact information of the family member. The sending means also includes a function for checking the sending status of a message and managing the sending history, so that the user can track the status of the sent message and resend it if necessary. The message sending process has a function for linking with a messenger application or an email application, authenticating the user based on the account information, and sending the message securely. The sending means protects privacy by encrypting the contents of the message and implementing security measures such as authentication at the time of sending. In addition, the message is sent in an appropriate format according to the sending method selected by the user, such as an instant message in the messenger application or an email in the email application.
An example of data collection that does not include sensors is when a user manually takes notes on a phone call. In this case, the user records the main points of the call after the call ends, and sends the text data to the family as a message. The call contents manually recorded by the user are organized using a text editor function and sent to the family as a formatted message by the sending means. This manual recording is suitable for situations where automatic documentation using voice recognition technology cannot be applied, or when the user wants to convey specific information in his or her own words. Another possible method of data collection that does not include sensors is a scenario in which a user manually takes notes on a phone call after the call ends, and sends the information to the family as a message. This manual recording is organized using a text editor function and sent as a formatted message. This is used in situations where voice recognition technology cannot be applied, or when the user wants to convey specific information in his or her own words.
In an embodiment of the present invention, functions beyond documenting the contents of a call can be considered. For example, a function can be added that automatically registers appointments and events mentioned during a call in a calendar in cooperation with a schedule management system based on the text generated by speech recognition. This can save the user the trouble of manually managing the schedule after the call. In addition, it can suggest adding events to a calendar shared among family members, making it easier for all family members to share the schedule.
It is also possible to automatically generate task lists based on documented text data and post them to a shared platform that can be accessed by all family members, where each family member can update the progress of tasks and check off completed tasks, creating an environment for the whole family to share information and cooperate.
In addition, the company will add a feature that performs sentiment analysis on written messages and reflects the emotional nuances expressed during a call in the text, helping message recipients to more accurately understand the caller's intent. For example, if emotions such as joy or anxiety are expressed during a call, those emotions will be expressed in the text with specific emojis and formats, enhancing the richness of communication.
Another possible function would be to use voice recognition and analysis to automatically generate FAQs and a list of frequently asked questions from the content of calls, building a knowledge base that family members can refer to when they have similar inquiries. This knowledge base would be shared among family members and updated every time a new call occurs, improving the efficiency of communication between family members.
In addition, the text generated after the call ends can be sent not only via messenger apps or email, but also played back in audio format, making it easier for visually impaired family members or children with difficulty reading and writing to receive the information.
Finally, to protect the privacy of family members, the documented content after the call is completed can be automatically anonymized or masked to identify sensitive information in the document, providing an environment in which information can be shared with peace of mind. This allows for a balance between respecting each individual's privacy and sharing only the information that is necessary.
The embodiment of the present invention relates to automatic documentation and transmission of call contents, and new functions can be added to the document. For example, the document can be analyzed, and action items can be automatically generated after the call ends, and a reminder can be set on the user's smart device as a task that needs to be addressed. The reminder can be prioritized based on deadlines and importance mentioned in the call, helping the user to take action without forgetting.
Additionally, to enhance communication among family members, it would be beneficial to add the ability to automatically attach relevant images, videos, and links based on specific words or phrases in a written message, allowing for visual information to be shared in addition to text-based messages, making communication richer.
In addition, the system can enhance security by automatically detecting and blurring certain personal or confidential information when documenting call transcripts, in accordance with privacy protection guidelines and natural language processing techniques.
When converting call contents into text, it is also effective to incorporate a translation function into multiple languages to meet the diverse needs of users. In a multicultural environment where family members speak different languages, it is possible to automatically translate the call contents and send messages in a language that each member can easily understand.
To further enhance convenience, a schedule function will be added that allows users to customize the timing of message sending. Users can set messages to be sent at a specific date and time, rather than just immediately, allowing them to optimize the timing at which their family members receive information.
Finally, quick action buttons for replying or confirming messages have been provided so that recipients can easily take action on received messages, enabling quick feedback and efficient communication, making information sharing between family members even smoother.
(Example 3)
The embodiment of the present invention is a system that checks phone numbers during a call, and has a means for linking with the police when linking with the police if the number has a history of misuse. Specifically, the system is equipped with a function that checks the incoming number against a database during a call, and automatically notifies the police if it detects that the number has a history of misuse.
The means for coordinating with the police is realized by the specific processing unit 290 of the data processing device 12 as a function for checking the incoming number against a database during a call and automatically reporting to the police if a number with a history of misuse is detected. The correspondence between each means and the device or control unit is not limited to the above example, and various modifications are possible.
The extraction means is provided for extracting caller information contained in the signal of each call sent through the communication network, and can accurately obtain the caller's telephone number from the call data using digital signal processing technology. The extraction means also has a signal analysis function for analyzing Caller ID information and identifying the telephone number. The system can also accurately identify the called number by utilizing the signal extraction function for extracting the caller's telephone number from the call signal and an analysis algorithm specialized in decoding Caller ID information.
The matching means has a function of matching the extracted telephone number with a database that collects telephone numbers suspected of fraudulent use and organizes them by category, and the matching process can be performed in real time through a database management system, and rapid matching can be performed while the call is in progress using high-speed search algorithms and indexing technology.The matching means also has a high-performance database search function and indexing function for rapid search and matching by referring to a database that lists numbers with a history of misuse.
The linking means has a function of automatically reporting to the police when a number with a history of abuse is detected, includes an interface function with the reporting system, generates report information in a data format based on the protocol with the police reception system when reporting, and transmits it to the police reception system using a secure communication channel. The contents of the report information include information such as the detected abused phone number, the date and time of the call, the duration of the call, and the telecommunications carrier used by the caller, and uses encryption technology and authentication systems to protect personal information and ensure the accuracy of the report. In addition, the linking means is compatible with multiple communication protocols and reporting systems, has an adapter function for smooth data exchange between the systems, and has a function of adjusting the format of the report information to meet the requirements of the police reception system in the reporting process, and selecting an appropriate reporting protocol to make the report. When the reporting process is activated, the system transmits report information to the police reception system and obtains a report reception confirmation. This confirmation is used as information for the system to record that the report was made correctly and store it as a report history. The report history is used for future analysis and improvement, contributing to the efficiency and accuracy of the report process. This linking function uses a report data generation function to create a report format containing the necessary information, and transmits the information to the police reception system via a secure report transmission function. Data encryption and system authentication functions are implemented to ensure the security and reliability of reports. In addition, the system's linking means has an adapter function that is compatible with multiple report protocols and enables data exchange with different report systems, and this adapter function automatically adjusts to a format suitable for the protocol requirements at the time of reporting using a report protocol selection function, and transmits report information and confirms receipt. The report history recording function records the success of reports, which is used to analyze and improve system performance.
As an example of data collection without sensors, a function for users to report suspicious calls on an application or web interface is provided. The user enters their experience of the call and any suspicious points they noticed during the call into a form, and the information is registered in a database. The data collected through this manual report can be added to a list of phone numbers that are automatically matched in the database, contributing to improved accuracy in detecting numbers with a history of abuse. As an example of data collection without sensors, an input function is provided for users to report suspicious calls. Through an interactive reporting form, users register the contents of suspicious calls in a database, and the information collected is used to detect numbers with a history of abuse. This manual reporting system contributes to the expansion of the number matching database by providing additional data based on the user's experience and intuition.
The system can be equipped with a function to strengthen the database update mechanism. For example, newly confirmed abused numbers will be automatically added to the database even after they are reported. In addition, when a suspicious call is reported, the credit information of the number will be checked against other databases and combined with information based on user reports to more accurately identify abuse history. To ensure the integrity of the database, a regular cleaning process will be carried out to eliminate incorrect and outdated information. In addition, to strengthen cooperation with the reporting system, a function will be introduced to directly link with the crime database provided by the police, obtain crime information in real time during the matching process, and improve the accuracy of the matching results.
To improve the immediacy of reports, the matching process begins the moment the call is initiated, and if a number with a history of abuse is detected, the caller will be given the option to sound an alarm or automatically hang up. In addition, when the call is hung up, a recording of the call can be saved on the caller's behalf to assist police investigations. When police intervene, a detailed report including the caller's location and call history is automatically generated, helping to speed up criminal investigations.
The user interface will be updated to include a feature that allows users to check the progress of the reporting process in real time to increase transparency in the reporting system. Users will be able to check the results of their reports and feedback from the police, which will increase trust in the system. The system will also provide statistical data and trend analysis on abused numbers, providing information to users to be cautious about calls.
Furthermore, machine learning technology will be introduced to identify numbers with a history of abuse, and new numbers likely to be abused will be predicted by analyzing various indicators such as call patterns and call frequency. This will enable proactive updates to the database, strengthening measures to prevent unknown criminal activity. A function will also be added that allows users to provide direct feedback on the effectiveness of the reporting system, which will help improve the system. Feedback will be provided anonymously, protecting user privacy while collecting valuable information that will contribute to improving the system.
To make the phone number check during a call more effective, AI will learn the patterns of various fraudulent scams that users may face and implement a function to flag in real time when certain words or phrases are detected during a call. This will allow malicious behavior to be inferred and detected from the content of the call, rather than simply matching the phone number with abuse history in the database. In addition, the reporting process will be simplified by providing shortcuts and buttons on the smartphone interface that allow users to easily report calls that they suspect are fraudulent. This will allow the database to be updated more quickly and improve protection for other users.
To strengthen cooperation with the police, the reporting system will be integrated with location tracking capabilities to help police respond quickly based on reported information, helping them track and capture criminals. Artificial intelligence will also be built into the reporting system to analyze crime patterns from report data and provide police with information to plan preventive vigilance activities. Using such predictive analytics will help prevent future crimes.
To facilitate data sharing with the police, the system will receive real-time information on fraud cases and other crimes known to the police and update the database. This will ensure that the reporting system operates based on the latest crime information and strengthen measures to protect users. In addition, the system will be equipped with a block list function, allowing users to register suspicious numbers and block calls from them. This will allow users to directly control the risks themselves.
Educational programs will include online courses and workshops to provide users with information to recognize and prevent fraud methods. This will provide users with the knowledge to protect themselves and raise security awareness in society as a whole. Examples of fraud cases that were prevented by using the reporting system will also be shared to help users understand the effectiveness of the system.
Finally, through a system update, if a call is deemed likely to be fraudulent, a warning message will be automatically sent to the user to warn them against fraud. This will allow users to immediately recognize the possibility of fraud and take appropriate action.

The process flow for each example is explained below.

(Example 1)
Step 1: When a call comes in from an unregistered or withheld phone number, the generative AI answers.
Step 2: After the call, document the matters discussed in Chat GPT.
Step 3: Send a written message via messenger app or email. If the caller claims to be a family member or acquaintance, verify the call using special questions or a secret code. The delivery company will connect you using a secret code. Check the phone number during the call, and if the number has a history of abuse, contact the police.
(Example 2)
Step 1: After the call, document the matters you wish to discuss in Chat GPT.
Step 2: Select the documented matter and designate it as being sent to family members.
Step 3: Send your message to your family via messenger app or email.
(Example 3)
Step 1: During a call, the incoming number is checked against a database to detect if it is a number with a history of abuse.
Step 2: If the number has a history of abuse, the call will automatically be reported to the police.
Step 3: After the call is over, the matter is documented in the chat GPT. The documented matter is used for communication with the police.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

(Example 1)
The embodiment of the present invention is a system that includes a response means in which a generative AI responds when a call is received from an unregistered or unnotified phone number, and a transmission means in which the matter is documented in chat GPT after the call is ended and sent via a messenger app or email. Furthermore, as a combination means for combining an emotion engine that recognizes the user's emotions, the tone of the user's voice and choice of words are analyzed during the call to estimate the user's emotions. Based on the estimated emotions, the generative AI's response and the expression of the documented matter are adjusted. For example, if the user shows an anxious emotion, a calmer expression is used to provide a sense of security.
The response means is realized by the specific processing unit 290 of the data processing device 12, and the generative AI responds to an incoming call from an unregistered or unnotified phone number. The transmission means is realized by the specific processing unit 290 of the data processing device 12 as a means for documenting the matter in chat GPT after the call is ended and sending it by a messenger app or email. The emotion recognition means is realized by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414 as a means for analyzing the tone of the user's voice and choice of words during the call and estimating the emotion. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The response means includes a dialogue generation means that combines natural language processing and speech synthesis. The dialogue generation means automatically responds to an unregistered or unnotified incoming call, enabling a dialogue with the user. When an unregistered or unnotified incoming call is detected, the generative AI responds using a pre-trained conversation model. This conversation model is trained based on various dialogue data so that it can handle various scenarios, and provides appropriate answers and guidance to the callee's questions and requests. In addition, the response means has an AI dialogue generation function, and can automatically provide an appropriate response to an incoming call from an unregistered or unnotified phone number. This function combines a dialogue management function for understanding the callee's purpose and request at the early stage of the call and providing a corresponding response, and a voice generation function for also converting the generated response into a natural voice. The dialogue management function can analyze the callee's intention based on the detection of specific keywords and phrases and generate an appropriate response. The voice generation function can convert a text-based response into voice in real time, providing the callee with a natural conversation experience.
The transmission means includes a chatbot and a documentation means using natural language understanding technology. After the call is ended, the contents of the call are accurately documented using an advanced natural language understanding model such as chat GPT. The documentation means has the ability to extract and summarize the main points of the call, and can generate a document that conveys the purpose concisely and clearly. The generated document is sent to the user via a messenger app or email. This process includes a link to the user's email address or messenger account, and the document is automatically sent in the appropriate format. The transmission means can also convert the contents of the call into text and provide it in an accessible form to the user. After the call is ended, the call content documentation function is activated and can extract and summarize the main points of the conversation. This summarized text is sent to the user's designated email address or messenger app through an automatic delivery function. This automatic delivery function includes an email delivery function and an app link function to properly format the document and ensure that it is delivered to the designated destination.
The emotion recognition means includes a voiceprint analysis means for performing voice analysis, and a language analysis means for estimating emotions from the choice of words. The voiceprint analysis means can detect characteristics of the user's voice, such as tone, pitch, and speed, using the microphone of the smart device, and can estimate the user's emotional state from these voice characteristics. The language analysis means can analyze the content of the user's speech and extract emotional context from the words and phrases used. These analysis results are used by the generative AI when making a response or adjusting the expression of documented matters. For example, if the user shows anxiety, the expression of the response or document is adjusted to be more gentle and reassuring. In addition, the emotion recognition means has a function of estimating emotions from the characteristics of the user's voice and language use during a call. The voiceprint analysis function can extract features such as the pitch, strength, and speed of the voice from the voice data collected by the microphone, and can estimate emotions by analyzing these characteristics. The language emotion analysis function can process the language data during a call, analyze the emotional meaning of the words and phrases used, and grasp the user's emotional state. These analytics can be used to adjust the tone of the AI's responses and the wording of documented calls to provide an appropriate emotional response to the user.
Data collection methods that do not involve sensors include text data entered by users themselves and feedback on system usage. These are provided to the system through the user input reception function and feedback collection function, and are used as a valuable source of information for service improvement.
These means are intended to respond quickly and effectively to user requests and improve the quality of communication. In addition, the implementation of each means is flexibly performed by the data processing device or the control unit of the smart device, and can be modified in various ways to improve the efficiency and usability of the system.
The system of the present invention can be equipped with an additional function of a summary prediction means for predicting the caller's intention before answering an unregistered or unnamed call. This allows the response means to generate a more accurate dialogue, realizing a meaningful exchange for the user. In addition, when the generative AI responds, it can have a language selection function based on the caller's country or region, enabling automatic response in multiple languages.
As for the delivery method, in addition to documenting the contents of the call, it can also highlight important keywords and phrases to help users quickly understand the document, and automatically generate action items based on the documented content and add them to the user's task list.
The emotion recognition means can detect changes in the user's emotions during a call in real time and dynamically adjust the tone and tempo of the conversation of the response means. Protocols can also be built in that, when a particular emotion is detected, prompt the user to contact an expert who can provide special support or advice according to the emotion.
The response tool can be enhanced with a personalization feature that analyzes the caller's past call history and related data to provide a more personalized response, making it possible to provide more relevant information to the user and increase the usefulness of the response.
Regarding sending methods, a function can be added that suggests follow-up actions based on the documented content of the call. For example, tasks and events included in the call content can be automatically registered in a calendar app to help users manage their time.
Furthermore, the emotion recognition means may be capable of detecting the user's stress level or tension and providing appropriate advice on how to reduce stress or links to relaxation content, thereby supporting the user's mental health and promoting overall well-being.
The system of this embodiment is considered to have a number of additional functions for further improving its functionality. For example, a voiceprint recognition means can be added to identify a caller by analyzing the caller's voiceprint and comparing it with previous call data for an unregistered or unnamed call. This allows the response means to respond more appropriately based on information if the caller has previously interacted with the system. The voiceprint recognition means also functions as a security measure, providing a reliable call experience for the user.
Regarding the means of transmission, when documenting the contents of the call, the function of structuring the contents of the call and classifying the text according to the importance of the information will be considered. This will allow users to grasp important information more quickly when reading the document. In addition, the documented content can be customized according to the user's preferences and past behavior patterns to provide a more personal experience.
The emotion recognition means detects the user's stress level during a call, and if it is estimated that the user is under high stress, it provides guidance on relaxation methods and psychological support related to the content of the call, thereby providing a service that allows users to relax and reduce stress through the call.
Additionally, the response tool will automatically suggest follow-up actions to users based on the content of the call, for example providing links to additional information about products or services suggested during the call or suggesting next steps of action to help users make decisions.
To improve response methods, we are considering a function that automatically changes the response style according to the caller's intentions. For example, if the caller indicates an emergency situation, we will adjust the AI to provide quick and accurate instructions. This will create a system that can immediately respond to the caller's needs.
These additional functions are intended to improve the user experience and enable accurate understanding of call content and rapid response. Each function is expected to maximize the capabilities of data processing devices and smart device control units, further enhancing the usability of the system.
(Example 2)
The embodiment of the present invention is a system that includes an adjustment means for documenting the matter in chat GPT after the call ends, and adjusting the expression of the document using an emotion engine that recognizes the user's emotions when sending it by messenger app or email. Specifically, the documented matter is input to the emotion engine, and the user's emotions are analyzed. Based on the analysis results, the expression of the document is appropriately adjusted. For example, if the user is feeling happy or excited, the user's emotions are shared by using brighter and more lively expressions.
The adjustment means is realized by the specific processing unit 290 of the data processing device 12 as a means for documenting the matter in chat GPT after the call ends and adjusting the expression of the document using an emotion engine that recognizes the user's emotion when sending it by messenger app or email. The process of inputting the documented matter to the emotion engine and analyzing the user's emotion is performed according to the specific processing program 56 executed by the specific processing unit 290. The correspondence between each means and the device or control unit is not limited to the above example, and various changes are possible.
The conversation content extraction means converts the contents of the call into text data by making full use of voice recognition technology. After receiving the voice signal, the conversation content extraction means uses noise reduction means to remove background noise and improve the accuracy of the voice-to-text conversion. Furthermore, the conversation content extraction means uses cutting-edge voice recognition technology to accurately convert the contents of the call into text data, and in the process, the noise reduction means removes background noise and improves the conversion accuracy.
The text processing means corrects syntactic errors in the converted text data and performs grammar checking via the grammar checking means to maintain linguistic fluency. The text processing means also performs adjustments to ensure that the grammar checking means of the converted text data ensures accuracy of grammar and maintains the natural flow of the document.
The emotion recognition means includes an emotion extraction means based on text mining and emotion analysis techniques, which estimates the user's emotion from the wording and context of the generated text data. The emotion extraction means also identifies words, phrases, and grammatical patterns that express various emotions and classifies them into emotion categories such as positive, negative, and neutral. The emotion emphasis means also adjusts the tone and phrasing of the text according to the extracted emotions, making modifications to better convey the user's emotional state. The emotion extraction means also reads the user's emotion from the language patterns expressed in the document and adjusts the tone of the document based on that.
The adjustment means changes the expression of the text based on the emotion data analyzed by the emotion recognition means. When a positive emotion such as joy or excitement is detected, the expression enhancement means replaces the vocabulary used with brighter and more lively words to give the message a favorable impression. When a negative emotion such as sadness or anger is detected, the expression mitigation means softens the tone of the message and selects phrases that show empathy and understanding. When the user's emotion is positive, the expression enhancement means energizes the document, and when the emotion indicates a negative emotion, the expression mitigation means uses a more gentle expression.
The message sending means includes a function for linking with a messenger application or an email client and sends the adjusted text in an appropriate format. Furthermore, the transmission protocol selection means selects an optimal transmission protocol according to the platform and settings of the recipient and ensures delivery of the message. Furthermore, the user interface presentation means provides a preview screen for a final review of the document before sending, allowing the user to make final corrections as necessary. Furthermore, the message sending means sends the adjusted text in a format suitable for the messenger application or the email client, and the preview screen provides an interface for the user to make a final check of the document.
The above processes are realized by a specific processing unit built into the control unit of the smart device or the data processing device, depending on the device and settings used by the user. Moreover, these means are designed as modularized components, and can be easily replaced or expanded as system components. Furthermore, the correspondence between each means is set flexibly, and various changes can be made to accommodate system upgrades and customization. Moreover, this series of processes is modularized so that it can flexibly respond to devices and environments, and is designed to facilitate system upgrades and customization.
This embodiment can be further expanded to add a function for deeper understanding of the user's emotions and improving the quality of communication. For example, by incorporating a facial expression recognition function during video chat into the emotion engine, emotions can be analyzed from visual information as well, resulting in more accurate emotion judgment. To improve the accuracy of emotion recognition, the tone and pitch of the user's voice can also be analyzed and reflected in the text. Furthermore, by analyzing the user's past communication history and response patterns, the system can learn the individual's emotional expression style and adjust the text accordingly in a more personalized manner.
The text processing means has a function to suggest phrases inspired by literary works and poetry to enhance the diversity and creativity of expressions, allowing users to express their emotions more richly. It can also select appropriate expressions according to the environment and situation in which the communication takes place, taking into account the social context and cultural background.
The messaging tool will add a function to predict the emotional impact of the text sent on the recipient, allowing users to communicate more responsibly. In addition, AI can predict the recipient's reaction and use that information to suggest the next communication strategy the user should take.
Overall, these features will help users reflect their emotions more accurately and sensitively in their communications, contributing to building deeper human relationships. These advanced methods can be effectively used by individuals as well as corporate customer support and CRM systems to deepen customer relationships. Furthermore, these systems are expected to be applied in areas where human emotions play an important role, such as user education and counseling.
The system leverages an emotion engine to provide document expression adjustments based on the user's emotions. To extend this functionality, sensors that capture the user's biometric information can be integrated to more accurately read emotions from physiological responses such as heart rate and skin conductivity. Data from the sensors is analyzed in real time, allowing the tone of the document to be adjusted instantly.
In addition, it is equipped with a personalized learning function that continuously analyzes the user's daily communication to understand the user's unique expression style and preferences, allowing the system to suggest more natural and personalized documents that reflect the user's personality.
It will also enhance multilingual support, allowing it to capture emotional nuances in different languages, enhancing international communication and understanding among multilingual users.
To protect user privacy, the company will also add features to anonymize and encrypt emotion data and strengthen security, allowing users to use the system with peace of mind.
Aiming for applications in the fields of education and counseling, we will develop a function that uses the results of emotion recognition to support communication skills training. The training program will include practicing emotional expression and learning appropriate communication techniques.
Finally, to improve the usability of the system, we make the user interface rich and intuitive, supporting a variety of gestures and voice commands, and allowing users to easily intervene in the document adjustment process, allowing users to fine-tune the expression at their own will and achieve more personal communication.
(Example 3)
The embodiment of the present invention is a system that uses an emotion engine that recognizes the user's emotions during a call and includes an adjustment means for adjusting the response and dialogue content to match the user's emotions. Specifically, the user's tone of voice and choice of words are input to the emotion engine during a call to estimate the user's emotions. The response and dialogue content of the generative AI are adjusted based on the estimated emotions. For example, if the user shows sad emotions, words of empathy are used to provide encouragement and support.
The adjustment means is realized by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414 as a means for adjusting the response or content of the dialogue to match the user's emotions using an emotion engine that recognizes the user's emotions during a call. The process of inputting the user's tone of voice and choice of words during a call to the emotion engine and estimating the user's emotions is performed according to the specific processing program 56 executed by the specific processing unit 290. The correspondence between each means and the device or control unit is not limited to the above example, and various modifications are possible.
The voice data collection means includes a microphone with high sensitivity and noise canceling function. The microphone is equipped with a digital signal processing algorithm to remove ambient noise and clearly record the user's voice. The voice data collection means also employs a highly sensitive microphone that captures subtle acoustic characteristics from the user's speech and combines this with digital signal processing technology to effectively remove ambient noise. The microphone accurately captures the characteristics of the user's voice and provides basic data for detecting changes in emotions.
The voice feature extraction means extracts features that may reflect human emotions from the voice signal. This extraction means includes a spectrogram analysis function and a pitch tracking function for performing acoustic feature analysis, and an emotion identification function using an acoustic model. The voice feature extraction means also uses acoustic analysis tools such as a spectrum analysis function and a pitch analysis function to extract features that suggest emotions from the voice signal, and supplies these data to the emotion estimation model.
The emotion analysis means includes an emotion estimation model based on machine learning, which estimates the user's emotional state using the features extracted by the voice feature extraction means as input. The emotion estimation model is composed of classifiers such as neural networks, support vector machines, and decision trees trained based on training data, and classifies the user's emotions into categories such as positive, negative, and neutral. The emotion estimation model improves its accuracy through continuous learning, and evolves to be able to respond to subtle changes in the user's perception of emotions. In addition, the emotion analysis means has an emotion estimation model that utilizes machine learning technology, and analyzes the user's emotions based on the extracted voice features. This model combines various machine learning algorithms to identify emotion categories from the user's utterance, and estimates the user's emotional state based on this. The estimated emotional state is important information for adjusting the system's response to the user's utterance and behavior.
The dialogue adjustment means includes a response generation function using a generative AI model, which dynamically adjusts the content of the dialogue according to the emotion estimated by the emotion analysis means. The response generation function utilizes natural language generation technology to generate a response sentence using words and tones appropriate for the user's emotion. For example, if the user is expressing sad emotion, a response including empathetic expressions and comforting words is generated. This response generation function has a context-aware processing function for taking into account the context of the conversation and providing content that matches the user's emotion and the purpose of the communication. In addition, the generated response is constructed to be natural to the user and to meet the emotional needs. The response generation function is realized by a machine learning model trained on a large-scale conversation dataset and can adapt to the user's language and speaking style. In addition, the dialogue adjustment means includes a response generation function using a generative AI model, which generates dialogue content appropriately corresponding to the estimated emotion. This function selects words and a dialogue tone that correspond to the user's emotion and provides a response that matches the user's current emotion and the context of the conversation. The response generation function is designed to perform context-aware processing and provide the information and support required by the user in an appropriate form. This feature is capable of generating responses that adapt to a user's language and emotions based on models trained from large amounts of conversational data.
Non-sensor-based data collection methods include text information entered directly by users into the system and metadata obtained from call records. These are used as supplementary information when analyzing user behavioral patterns and preferences, and contribute to improving the accuracy of the emotion engine and optimizing response generation functions. User-entered text information is also used as training data for emotion estimation models as part of the emotion analysis method.
The system has the ability to analyze the user's emotions in real time during a call and automatically adjust the dialogue content, so it can integrate facial expression recognition technology in addition to voice data to capture even subtler changes in emotions. It uses a webcam or a camera on a smart device to analyze the user's facial expressions and improve the accuracy of emotion analysis. This added facial expression recognition function will more accurately recognize the user's emotions and make it possible to adjust the dialogue according to even subtler emotional changes.
It is also possible to incorporate wearable devices to capture the user's physiological signals. Measuring physiological responses such as heart rate and electrodermal activity can provide insight into deeper emotions that cannot be captured by tone of voice or facial expressions alone. Integrating this data can further improve the accuracy of emotion analysis and generate more appropriate dialogue responses.
It would also be effective to add customization features to the dialogue adjustment method that take into account the user's cultural background and personal values. A user profile can be constructed, and the tone and content of the dialogue can be further personalized based on that information. This makes it possible to provide detailed support tailored to each individual user.
Furthermore, in order to promote the evolution of the emotion estimation model, we will build a mechanism to collect emotion data through crowdsourcing and incorporate feedback from a variety of users to continuously update the model, resulting in a flexible system that can handle a variety of emotional expressions and languages.
Applications in the fields of education and mental health care can also be considered. For example, in the field of education, it could be used to present educational materials adapted to students' emotions or in counseling sessions. In mental health care, it could recognize the user's emotions and provide dialogue support to reduce stress and anxiety. This would enable a more effective approach to the problems the user is facing.
Regarding privacy protection for the system, we will strengthen security measures to safely store users' emotional data and minimize the risk of information leakage by using appropriate access control and encryption technology. This will allow users to use the system with peace of mind.
Ultimately, it is hoped that the personalized interaction experience provided by this system will be developed into services that improve the quality of users' lives.
The embodiment of the present invention is applicable not only to telephone calls but also to video conferencing and online education platforms. For example, a lecturer can grasp the emotions of students in real time and adjust the progress of the curriculum according to the emotions, providing a more effective learning experience. In video conferencing, dialogue management reflecting the emotions of participants is also performed, promoting a productive and positive meeting environment.
The system could also be enhanced to monitor health conditions based on emotional responses: for example, if a user's tone of voice or manner of speaking indicates negative emotions over a period of time, a mental health professional could be notified to intervene if necessary.
Furthermore, to further improve the emotion engine, a function will be incorporated that will analyze the emotional patterns of the user's daily life and provide advice on long-term emotion management and stress reduction based on that information. By analyzing the user's daily rhythm and activity patterns and predicting emotional ups and downs, the system will suggest content for relaxation and motivation at appropriate times.
This system is also expected to be applied in the field of customer support. For example, if call center operators could grasp the customer's emotions in real time and detect negative emotions such as dissatisfaction or anger, they could take immediate action to address the issue, contributing to improving customer satisfaction.
In the fields of games and entertainment, dynamic changes to content based on the user's emotions are expected to have the effect of increasing immersion and enjoyment. In-game characters will react to the player's emotions, changing the story development and dialogue, creating a more personalized experience.
Furthermore, we will integrate voice assistants and virtual reality (VR) to realize more natural dialogue based on the user's emotions. Voice assistants will understand the user's emotions and provide information and services tailored to individual needs. In the VR environment, the scenario and environment will change according to the user's emotions, providing an experience that matches the emotions in real time.
This system also has the potential to revolutionize the design of user interfaces (UI) and user experiences (UX). Using emotion recognition technology, it can provide UI and UX optimized for the user's emotions, increasing user satisfaction. For example, websites and applications can grasp the user's emotions in real time and adjust the way content is presented and the form of interaction.
Ultimately, it is hoped that the emotion regulation function provided by this system will become widely used throughout society as a tool to improve the quality of human relationships and increase the effectiveness of communication.

The process flow for each example is explained below.

(Example 1)
Step 1: When a call comes in from an unregistered or withheld phone number, the generative AI answers.
Step 2: After the call, document the matters you wish to discuss in Chat GPT.
Step 3: Input the documented requirements into the emotion engine to analyze the user's emotions.
Step 4: Based on the analysis results, the representation of the document is adjusted.
Step 5: Send the adjusted document via messenger app or email.
(Example 2)
Step 1: After the call, document the matters you wish to discuss in Chat GPT.
Step 2: Input the documented requirements into the emotion engine to analyze the user's emotions.
Step 3: Adjust the representation of the document based on the analysis results.
Step 4: Select the adjusted document and specify the recipient (e.g., family).
Step 5: Send the document to the specified recipient via messenger app or email.
(Example 3)
Step 1: During a call, the user's tone of voice, choice of words, etc. are input into the emotion engine to analyze the user's emotions.
Step 2: Based on the analysis results, adjust the generative AI's responses and dialogue content.
Step 3: After the call, document the matter in the chat GPT. The documented matter is used to input the emotion engine.

The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating the user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization. The identification processing unit 290 performs the above-mentioned identification processing while using the data generation model 58.

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the robot 414.

The emotion identification model 59, which serves as an emotion engine, may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. Emotions that represent states and actions arising from a state of mind are arranged on the outer sides of the concentric circles. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

These emotions are distributed in the three o'clock direction of emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a sense of calm.

The inside of emotion map 400 represents what is going on inside the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (expressed in behavior) the emotions become.

Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and remaining battery power, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map is lined with emotions that belong to an area called "reaction" where sensation is dominant. The right half of the emotion map is lined with emotions that belong to an area called "situation" where situation recognition is dominant.

The emotion map defines two emotions that encourage learning. The first is the negative emotion around the middle of "repentance" or "reflection" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input to a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple learning data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, "peace of mind," "calm," and "reassuring," have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

In the above embodiment, an example has been described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

The specific processing program 56 may also be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed in the computer 22 in response to a request from the data processing device 12.

It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The various processors listed below can be used as hardware resources for executing specific processes. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource for executing specific processes by executing software, i.e., a program. Examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically to execute specific processes. All of these processors have built-in or connected memory, and all of these processors execute specific processes by using the memory.

The hardware resource that executes the specific process may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The hardware resource that executes the specific process may also be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured by combining one or more CPUs with software, and this processor functions as a hardware resource that executes a specific process. Secondly, there is a configuration in which a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute a specific process, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the various processors mentioned above as hardware resources.

More specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processing described above is merely an example. It goes without saying that unnecessary steps can be deleted, new steps can be added, and the processing order can be changed without departing from the spirit of the invention.

The above description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. Also, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards described in this specification are incorporated by reference into this specification to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

The following is further disclosed regarding the above embodiment.

(Appendix 1)
When a call is received from an unregistered or withheld phone number, the generative AI responds first (by converting the generative AI's response into voice and responding), and after the call ends, the purpose of the call is documented in Chat GPT and sent via a messenger app or email.
If the person claims to be a family member or acquaintance, you can ask them special questions or use a password to verify their identity.
The delivery company will provide a way to contact the recipient using a secret code provided by the company.
The system checks phone numbers during calls and includes a means to contact the police if the number has a history of misuse.
(Appendix 2)
A system as described in Appendix 1, characterized in that it has a means for documenting the matter in chat GPT after the call ends and sending it to family members via a messenger app or email.
(Appendix 3)
A system as described in Appendix 1, characterized in that the system checks a telephone number during a call and, if the number has a history of misuse, contacts the police, by including a means for contacting the police.

(Appendix 1)
A system comprising a means for a generative AI to respond to an incoming call from an unregistered or withheld phone number, and a means for documenting the matter in chat GPT after the call ends and sending it via a messenger app or email, and further comprising a means for combining with an emotion engine that recognizes the user's emotions.
(Appendix 2)
The system according to claim 1, further comprising a means for documenting the contents of the call in chat GPT after the call ends, and adjusting the expression of the document using an emotion engine that recognizes the user's emotions when sending the document via a messenger app or email.
(Appendix 3)
2. The system according to claim 1, further comprising: means for adjusting responses and dialogue content to match the user's emotions using an emotion engine that recognizes the user's emotions during a call.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (6)

1. A system comprising:
A response means for the generative AI to respond when a call is received from an unregistered caller or a call is received from a withheld caller;
A method of transmitting the contents of a call using voice recognition technology to convert the contents of the call into text, summarizing and documenting the converted text using generative AI , and sending it via a messenger app or email.
When a call is received, the AI receives the secret code used by the delivery company and confirms that it is the correct code. If so, it will connect to the user .
and a linking means for, when a signal transmitted through a communication network includes caller information during a call, checking the telephone number indicated by the caller information , referring to a database that lists numbers with a history of misuse and that is included in the system, and notifying the police if the incoming number is a number with a history of misuse ;
The response means is a system that estimates the emotions of the caller from the tone of voice and manner of speaking, and responds according to the estimation result . 2. The system according to claim 1, wherein the sending means sends the summary to a family member when sending the summary via a messenger app or by email. 2. The system according to claim 1, wherein the linking means notifies an emergency contact designated by the user when reporting to the police. 2. The system according to claim 1, wherein the connection means performs authentication by having a delivery company scan a QR code or an NFC tag, and connects the call directly to the user . A system according to any one of claims 1 to 4, characterized in that the sending means has a function of highlighting important keywords and phrases in the document in addition to documenting the contents of the call. A system according to any one of claims 1 to 4,
an emotion recognition means for estimating an emotion of the other party based on the converted text;
and an adjustment means for adjusting the documentation of the call content based on the emotion of the call partner estimated by the emotion recognition means.

Images