KR20230051850A - System for providing account book service using Text Mining - Google Patents

Abstract

The present invention relates to a household account book service providing system using text mining that automatically organizes a household account book using a receipt image. A user terminal that performs a reading or editing function for the household account book file created through; and providing the smart household account book application to the user terminal, binarizing a receipt image provided from the user terminal for each pixel, extracting a text area having a pixel value equal to or greater than a preset threshold, and applying text mining technology to the extracted text area. The receipt text information is converted into receipt text information, and the receipt text information is organized into household account book format information including purchase date, purchase method, purchase product list, receipt issuer, consumption amount information, and point accumulation information to create a household account book file, and the user terminal It may be a system including a household account book management server provided to .

Description

System for providing account book service using text mining {System for providing account book service using Text Mining}

The present invention relates to a household account book service providing system using text mining that automatically organizes a household account book using receipt images.

The information described in this section merely provides background information on an embodiment of the present invention and does not constitute prior art.

The method of writing consumption habits using a handwritten household account book is a great help in the process of reflecting on consumption habits and correcting consumption patterns as the user writes out his or her own consumption details. It has the advantage of being able to change the . However, the handwritten household account book has disadvantages in that it is cumbersome and time-consuming because it is necessary to check and prepare various receipts one by one and to calculate the details of consumption by the user himself.

In order to overcome the disadvantages of these handwritten household account books, a lot of household account book documents made with simple household account book programs or Excel files have been used. The contents were recorded and understood.

The household account book application extracts the consumption history included in the SMS text message provided to the smartphone when making a payment through a credit card company or bank company, and automatically registers it in the household account book application. However, in the case of consumption activities performed by cash payment rather than card payment, consumption-related information cannot be registered in the household account book application, so the user must separately execute the corresponding application on the smartphone to register the consumption information.

In addition, the conventional household account book application provides a function of extracting strings such as purchase date, product item, amount, and total amount through text recognition of a receipt using a camera of a smartphone and recognizing them as data values that can be processed.

In the case of a household account book application with a receipt recognition function, in the case of a receipt issued by a mart, detailed expenditure categories are very diverse, so it is not easy to classify and infer the expenditure categories. It is inconvenient to have to input them one by one.

In a mart, not only foodstuffs but also various consumer goods, household appliances, sports equipment, etc. are very diverse, so it can be seen that it is impossible to accurately classify and fill out detailed expenditure categories. In particular, a large amount of money is consumed in a mart, and it is difficult to accurately determine how much food, consumer goods, and home appliances are consumed among the amounts. In this way, there is a demand for means capable of creating a household account book by accurately classifying detailed expenditure categories or product categories through consumption details of the mart.

In order to solve the above problems, the present invention provides a household account book service providing system using text mining, which automatically organizes a household account book using a receipt image and accumulates points to facilitate receipt management according to an embodiment of the present invention. It aims to provide

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the household account book service providing system using text mining according to an embodiment of the present invention executes a smart household account book application for household account book arrangement, takes a picture of a receipt, and provides a receipt image, A user terminal that performs a reading or editing function for the household account book file created through the smart household account book application; and providing the smart household account book application to the user terminal, binarizing a receipt image provided from the user terminal for each pixel, extracting a text area having a pixel value equal to or greater than a preset threshold value, and extracting a text area based on natural language processing in the extracted text area. Text mining technology is applied to convert receipt text information, and a household account book file is created by arranging the receipt text information into household account book format information including purchase date, purchase method, purchase product list, receipt issuer, consumption amount information, and point accumulation information and a household account book management server providing the information to the user terminal.

According to one aspect of the present invention, the household account book management server binarizes the receipt image pixel by pixel to extract a text area having a pixel value equal to or greater than a preset threshold value, and applies natural language processing-based text mining technology to the text area. a text mining module that recognizes text by using; a text formatting module that designates a text format to represent the texts extracted from the text mining module as strings and provides them as receipt text information; a text parsing module extracting a purchase product list to be filled in each item of a household account book form layer from the receipt text information provided by the text formatting module; a household account book form mapping module for mapping purchased product list information to the household account book form layer; and a file generation module for generating a household account book file including purchased product list information, income details information, and user data mapped to the household account book form layer in the household account book form mapping module.

According to one aspect of the present invention, the text mining module classifies recognized text into predefined detailed expenditure categories using a pre-learned artificial intelligence model.

According to one aspect of the present invention, the user terminal performs a function of registering an electronic receipt through a smart household account book application and a function of registering a payment-related text message transmitted from a card company server or a bank server, thereby generating payment-related text messages and electronic receipts. It is to be arranged on the household account book file.

According to one aspect of the present invention, the household account book management server, when registering a receipt image through the smart household account book application, provides a preset accumulation point, provides a current status of accumulation points according to an inquiry or payment request of the user terminal, and has already Points are awarded according to the set payment method.

A method for providing a household account book service using text mining performed by a household account book management server providing a household account book service using text mining according to an embodiment of the present invention includes: a) providing a smart household account book application for organizing household account books to a user terminal; registering a receipt image through the smart household account book application; b) The receipt image is binarized for each pixel to extract a text area having a pixel value equal to or greater than a preset threshold, and converted into receipt text information by applying text mining technology based on natural language processing to the extracted text area, and expressed as a string converting receipt text information by performing a text formatting and text parsing process; and c) generating a household account book file by arranging the receipt text information into household account book format information including date of purchase, means of purchase, list of purchased products, receipt issuer, consumption amount information, and point accumulation information, and providing the information to the user terminal. is to do

According to one aspect of the present invention, step c) classifies the recognized text in the receipt text information into predefined detailed expenditure categories using a pre-learned artificial intelligence model.

According to the problem solving means of the present invention described above, the present invention can easily manage receipts by accumulating points while automatically organizing a household account book using images of physical receipts and electronic receipts, and text messages and receipts related to payment and deposit and withdrawal can be managed together, and various contents about expenditure or income can be provided to the user by analyzing the household account book file.

1 is a diagram illustrating the configuration of a household account book service providing system using text mining according to an embodiment of the present invention.
2 is a diagram showing the configuration of a household account book management server according to an embodiment of the present invention.
3 is a flowchart illustrating a method of providing a household account book service using text mining according to an embodiment of the present invention.
4 is an exemplary view illustrating a receipt image and a household account book file according to an embodiment of the present invention.
5 is an exemplary view illustrating processing details of a receipt image according to an embodiment of the present invention.
6 is an exemplary view illustrating an analysis result of a household account book file according to an embodiment of the present invention.
7 is a flowchart illustrating a text formatting process according to an embodiment of the present invention.
8 is a flowchart illustrating a text parsing process according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that it may further include other components, not excluding other components, unless otherwise stated, and one or more other characteristics. However, it should be understood that it does not preclude the possibility of existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

In this specification, a 'terminal' may be a wireless communication device with guaranteed portability and mobility, and may be, for example, any type of handheld-based wireless communication device such as a smart phone, a tablet PC, or a laptop computer. Also, the 'terminal' may be a wired communication device such as a PC capable of accessing other terminals or servers through a network. In addition, a network refers to a connection structure capable of exchanging information between nodes such as terminals and servers, such as a local area network (LAN), a wide area network (WAN), and the Internet (WWW : World Wide Web), wired and wireless data communications network, telephone network, and wired and wireless television communications network.

Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasonic communication, visible light communication (VLC: Visible Light Communication), LiFi, and the like, but are not limited thereto.

The following examples are detailed descriptions for better understanding of the present invention, and do not limit the scope of the present invention. Therefore, inventions of the same scope that perform the same functions as the present invention will also fall within the scope of the present invention.

In addition, each configuration, process, process or method included in each embodiment of the present invention may be shared within a range that does not contradict each other technically.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating the configuration of a household account book service providing system using text mining according to an embodiment of the present invention.

Referring to FIG. 1 , a household account book service providing system using text mining includes at least one user terminal 100 and a household account book management server 200, but is not limited thereto.

The user terminal 100 executes the smart household account book application for organizing the household account book, photographs a real receipt or electronic receipt to provide a receipt image, and performs a function of viewing or editing the household account book file created through the smart household account book application. The user terminal 100 may perform a function of registering an electronic receipt and a function of registering a text message related to payment transmitted from a card company server or a bank server through a smart household account book application. It can be arranged on top.

The household account book management server 200 provides a smart household account book application to the user terminal 100, converts a receipt image provided from the user terminal 100 into receipt text information using text mining based on natural language processing, and converts the receipt text information into A household account book file is generated by arranging the household account book form information including date of purchase, means of purchase, list of purchased products, receipt issuer, consumption amount information, and point accumulation information, and then provided to the user terminal 100 .

The household account book management server 200 may be a computer body for a server in a general sense, and may be implemented in various types of devices capable of performing a server role. Specifically, the household account book management server 200 may be implemented in a computing device including a communication module (not shown), a memory (not shown), a processor (not shown), and a database (not shown). It may be implemented in a PDA, tablet PC, PC, notebook PC, and other user terminal devices.

2 is a diagram showing the configuration of a household account book management server according to an embodiment of the present invention.

Referring to FIG. 2, the household account book management server 200 includes a text mining module 210, a text formatting module 220, a text parsing module 230, a household account book form mapping module 240, a file creation module 250, and a database. (260), but is not limited thereto.

The text mining module 210 extracts a text area from the receipt image and extracts text from the text area. At this time, the text mining module 210 may use an artificial intelligence-based image analysis API provided by Google, that is, the Google Vision AI API. The Google Vision AI API can use OCR to recognize text in images in over 50 languages and various file formats, and uses a pre-trained AI model to convert the recognized text into detailed spending categories associated with predefined household accounts. can be quickly classified as On the other hand, the text mining module 210 applies the binarization technology to the receipt image even without using the OCR processing technology, extracts the text area by judging the pixels above the threshold value as text based on the pixel value, and then performs text mining on the text area By applying the technology, we use natural language processing technology based on linguistics, statistics, machine learning, etc. to standardize semi-structured/unstructured text data, and to discover meaningful information from the technology for extracting features and extracted features. do.

The text formatting module 220 designates a text format to represent the texts extracted by the text mining module 210 as strings and provides them as receipt text information.

Receipts are printed out when paying in cash or by card for orders at POS, kiosk, etc. , 4 items of amount (amount, total, amount, etc.) are exposed. These receipts may have different formatting methods such as order and text for expressing the four items depending on the payment settlement service provider.

Accordingly, the text formatting module 220 utilizes artificial intelligence to process various receipt formats. That is, the text formatting module 220 learns an artificial intelligence-based receipt recognition model, and inputs the receipt text information processed in the text mining module 210 to the learned receipt recognition model, which receipt is the original receipt of the receipt text information. A predicted value (receipt format value) is output to see if the format is being used.

In order to learn the receipt recognition model, the text formatting module 220 collects a training dataset of output receipt image-label pairs according to a format possessed by each payment provider, and converts the collected training dataset according to the receipt format. It is used to apply Image Classification technology. That is, the text formatting module 220 performs preprocessing on the receipt image of the training dataset, and the receipt recognition model is learned using the training dataset. At this time, the receipt recognition model can use CNN (Convolution Neural Network), which is a representative deep learning model for image processing, and CNN extracts spatial features of images through convolution, ReLU, and pooling processes, and uses them to fully connect layers. classified through

At this time, the text formatting module 220 can maintain high performance even without long-time learning by using AlexNet, VggNet, Googlenet, ResNet, etc. in addition to CNN.

The text parsing module 230 extracts a purchased product list used in a household account book template from receipt text information. At this time, the text parsing module 230 can easily extract the purchase product list according to the four items (menu name, price, quantity, amount) exposed on the receipt based on the receipt format predicted through the receipt recognition model.

The household account book format mapping module 240 maps purchased product list information to a household account book form layer designated by a user or a household account book form layer set as a default value, and the file generation module 250 maps the purchased product list information mapped to the household account book form layer as well as import Generates and stores a household account book file including details and other user data.

When a receipt issuer or a product purchaser issues an electronic receipt to the user terminal 100 after purchasing a product, the user terminal 100 registers the electronic receipt through a smart household account book application to include the expenditure details in the electronic receipt in the household account book file and text messages (payment text messages, deposit/withdrawal approval messages, etc.) transmitted from the card company server or bank server can be automatically registered so that the details of expenditure and income are included in the household account book file.

In addition, the database 260 stores receipt images, receipt text information, purchase product list information, electronic receipts, text messages, and the like, and stores various data accumulated while performing a method of providing a household account book service using text mining.

Meanwhile, the household account book management server 200 interworks with a smart household account book application to provide preset accumulation points when registering a receipt image. At this time, the household account book management server 200 allows certain points to be accumulated according to the number of registered receipt images or the total amount.

In addition, the household account book management server 200 provides the current status of accumulated points according to the inquiry request of the user terminal 100, and pays the accumulated points to a pre-registered user payment account or purchases products by accessing an affiliate site according to the payment request. make it possible The household account book management server 200 may accumulate points for each sales outlet and each mart in the receipt through prior partnership with sales outlets such as marts and food companies, map information providing sites, etc.

In addition, the household account book management server 200 maps the map information of the receipt issuing place (purchasing place) on a map information providing site, etc., stores the phone number, purchase name, etc. together, and registers it in My Page as favorite store information. .

The above-described modules are only one embodiment for explaining the present invention, and may be implemented in various modifications without being limited thereto. In addition, the above-described modules are stored in a memory as a computer-readable recording medium that can be controlled by the processor of the household account book management server 200 . In addition, at least some of the above-described modules may be implemented as software, firmware, hardware, or a combination of at least two of them, and may include a module, program, routine, instruction set, or process for performing one or more functions.

3 is a flowchart illustrating a method of providing a household account book service using text mining according to an embodiment of the present invention, FIG. 4 is an exemplary view illustrating a receipt image and a household account book file according to an embodiment of the present invention, and FIG. 5 is an exemplary diagram illustrating processing details of a receipt image according to an embodiment of the present invention, and FIG. 6 is an exemplary diagram illustrating an analysis result of a household account book file according to an embodiment of the present invention.

3 to 6, in the method of providing a household account book service using text mining, when a receipt is captured using a user's camera and a receipt image is stored in a state in which a smart household account book application is executed in a user terminal, the smart household account book application generates a receipt The image is automatically registered, and the text recognition process of the receipt image is performed (S10).

At this time, the smart household account book application interworks with the household account book management server 200 to perform natural language processing-based text mining text formatting, text parsing, household account book format mapping, and file creation. That is, the smart household account book application processes the receipt image through text mining based on natural language processing, performs text formatting to express the receipt image as a string (S20, S30), performs text parsing on the text formatted receipt text information, and then parses The resulting text information is classified into a purchase product list (S40, S50).

The smart household account book application creates a household account book file by projecting the purchased product list onto the household account book form layer (S60). At this time, the household account book file includes the date of purchase, means of purchase (card, cash, etc.), list of purchased products, receipt issuer, consumption amount information, and point accumulation information. In addition, the text mining module 210 may classify the recognized text into detailed expenditure categories such as transportation/vehicle, eating out, housing/communication, and expenses for congratulations and condolences, etc., defined in advance by using the pre-learned artificial intelligence model.

Accordingly, in the household account book file, text information (movable dumplings, private taxis, etc.) corresponding to products may be automatically classified and organized into detailed expenditure categories (transportation/vehicle>public transportation expenses, eating out>meals, etc.).

For example, as shown in FIG. 4, when a smart household account book application registers a receipt image, the receipt issuer ('Garden Dumpling'), purchase date ('2019--8-09 00:00:00'), and total amount ('15,000 won'), premium information is extracted.

As shown in FIG. 5, the smart household account book application stores spending time information, registration time information, and processing details together with receipt images in the database 260, and when a search request is made from the user terminal 100, the receipt image is stored in the database ( 260) and display it.

Meanwhile, as shown in FIG. 6 , the household account book management server 200 analyzes the expenditure details and income details by day, month, and year, and then converts the analysis result into a graph or an Excel graph for the expenditure ratio and expenditure amount for each category. It can be provided in the form of a report, etc.

7 is a flowchart illustrating a text formatting process according to an embodiment of the present invention.

Referring to FIG. 7 , in the text formatting process, the text mining engine can extract coordinate values of where each text is located on a receipt image through the text mining module 210, and blank information (DetectedBreak value) for spaces and newlines. can be extracted (S31).

For example, the text mining module 210 has a structure of fullTextAnnotations>pages>blocks>paragraphs, fullTextAnnotations contains information about all sections recognized in a receipt image, Within the entire partition information, partitioning is pages, partitioning in pages is blocks, and partitioning in blocks is paragraphs, words, and symbols. At this time, the boundingBox in Blocks, Paragraphs, words, and symbols contains the coordinate values of the section, and the text in symbols, the lowest unit of the section, contains the recognized text. Accordingly, the text formatting module 220 extracts the text and rearranges it according to the coordinate values of the bounding box.

The text formatting module 220 calculates all coordinate value differences between lines based on the coordinate information, and sets a value that minimizes the coordinate value difference as a new line reference value (S32).

The text formatting module 220 stores individual texts in the form of a text list based on the coordinate information and space information and classifies them based on the X and Y axes of the coordinate information (S33).

The text formatting module 220 applies a newline reference value to the classified text list, rearranges it into a string array, and stores the receipt text information (S34). At this time, the text formatting module 220 reorganizes the text list into a string array in such a way that the first text in the text list and the next text are sequentially concatenated. If the difference is greater than the newline standard value, it is stored at the next character array index after performing a newline.

That is, in the state in which the text formatting module 220 declares a variable called the empty string "text", based on the coordinate values (x, y) (coordinate information) of the section provided as a result of the processing of the text mining module 210 From the lowest unit word (on the X axis) of the division corresponding to the second quadrant on the XY coordinate plane, Word > Symbol > text is continuously attached to the text variable of 1.

At this time, if the symbol's Property > DetectedBreak > Type is checked and the type other than "EOL_SURE_SPACE" is found, Symbol.text + " " is attached to the text of 1. If there is "EOL_SURE_SPACE" in Property > DetectedBreak > Type in my last Symbol element of Word, it is assumed that this next word is a new word, and the coordinates of the concatenated text of 1 and the entire concatenated text are stored separately in an arbitrary array. Save and initialize the text variable. Then, the difference between the y value of the current word and the y value of the next word to be newline (first difference value) is calculated.

After repeating this process, sort the array storing the stored text and coordinate values in ascending order based on the y value and the x value, access the stored random array from the beginning, and convert the text of this element to another string To store in a variable, if the difference between the y value of the current element and the y value of the next element is greater than the first difference value, which is the difference between the y values stored at 4, the newly stored string variable is stored in another string array, and then this string variable is stored. Initialize and repeat this process.

The text formatting module 220 arranges strings corresponding to preset arrangement conditions in the receipt text information to facilitate text parsing (S35). That is, the text formatting module 220 removes special characters unnecessary for parsing Hangul, alphabets, numbers, and symbol characters (% & * + -, etc.), removes product order numbers in front of product names, and The number 1 should be recognized, but the part recognized as a special character vertical line (|) is supplemented with 1, two-space spaces are changed to one-space spaces, and text with only special characters or spaces is removed.

8 is a flowchart illustrating a text parsing process according to an embodiment of the present invention.

As shown in FIG. 8 , the text parsing module 230 performs regular expression text preprocessing on receipt text information. That is, the date of purchase, phone number, and payment card information are preprocessed with a date regular expression (eg, 2019-09-01, etc.), a phone number regular expression, and a card information regular expression (S41).

The text parsing module 230 sets the starting point of the receipt text information and performs string array index check (S42). In general, in the case of a string containing 'order number', 'product name', 'unit price', and 'product code', which are the texts preceding the product list, the index following the string corresponds to the product list. Texts including 'order number', 'product name', 'unit price', and 'product code' are collected and set as a starting group, and when a text in the starting group is found, the corresponding position can be set as a starting point.

The text parsing module 230 classifies character strings including product name, product code, quantity, unit price, discount, coupon, and discount to check the product list text (S43). Each character string (row) is classified into 'row with product name only', 'product name + unit price row', 'product code only row', 'product code + unit price row', and 'discount row'.

That is, a row containing text in the form of 'discount', 'surcharge', 'coupon', or '-number' in the string is classified as a 'discount row'. After creating a new string array by dividing the rows that are not discount rows based on blanks, if the length of the newly created string array is 4 and all data are numbers, classify them as 'product code + unit price row' , If the last (nth) text of the string, the n-1st text, and the n-2nd text are numbers, it is classified as 'product name + unit price row'. In addition, if the length of the string array is 4 or more, it is classified as 'product name row'.

On the other hand, when the length of the newly created string array is 3, if the text is all numbers, it is classified as a 'unit price row', and if some characters are included in the text, it is classified as a 'product name row'. If, as a result of examining the data of the string array based on blanks among the newly created string arrays, unclassified strings are classified as 'rows with only product names'.

On the other hand, if the length of the character array is 2, the text parsing module 230 classifies it as "product name + unit price row" if the first text is not a number and the second text is a number, and if both the first text and the second text are numbers, the first text is a number. If the text is a product code, it is classified as "product code + unit price row", and if the first text is not a product code, it is classified as "unit price row". On the other hand, if both the first and second texts are not numbers, it is classified as "product name row".

When the length of the character array is 1, the text parsing module 230 classifies the corresponding text into "product code row" if the text is a product code, "unit price row" if it is a number, and "product name row" if it is not a number.

The text parsing module 230 combines rows classified as 'product name only row', 'product name + unit price row', 'product code only row', 'product code + unit price row', and 'discount row' to purchase products compose a list

The text parsing module 230 stops inspecting the product list text when text corresponding to the end point exists, and provides a purchased product list combining each of the classified rows (S44 and S45). At this time, frequently used texts at the bottom of the receipt, such as 'taxable goods', 'taxable amount', 'total amount', 'order total', etc. are collected and set as an end group, and when the text in the end group is found, the corresponding position set as the end point.

Meanwhile, each of the steps of FIGS. 3, 7, and 8 may be divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted if necessary, and the order of steps may be changed.

The embodiments of the present invention described above may be implemented in the form of a recording medium including instructions executable by a computer, such as program modules executed by a computer. Such recording media includes computer readable media, which can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Computer readable media also includes computer storage media, both volatile and nonvolatile, implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. , including both removable and non-removable media.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

100: user terminal
200: household account book management server
210: text mining module
220: text formatting module
230: text parsing module
240: household account book form mapping module
250: file creation module
260: database

Claims (7)

In the household account book service providing system using text mining,
A user terminal that executes a smart household account book application for organizing household account book, takes a receipt and provides a receipt image, and performs a function of viewing or editing a household account book file created through the smart household account book application; and
The smart household account book application is provided to the user terminal, and a receipt image provided from the user terminal is binarized for each pixel to extract a text area having a pixel value equal to or greater than a preset threshold, and text mining technology is applied to the extracted text area and converts the receipt text information into receipt text information, and organizes the receipt text information into household account book format information including purchase date, purchase method, purchased product list, receipt issuer, consumption amount information, and point accumulation information to generate a household account book file and display the information on the user terminal. A household account book service providing system using text mining, comprising a household account book management server that provides. According to claim 1,
The household account book management server,
a text mining module that binarizes the receipt image pixel by pixel, extracts a text area having a pixel value equal to or greater than a predetermined threshold, and recognizes text by applying a natural language processing-based text mining technology to the text area;
a text formatting module that designates a text format to represent the texts extracted from the text mining module as strings and provides them as receipt text information;
a text parsing module extracting a purchase product list to be filled in each item of a household account book form layer from the receipt text information provided by the text formatting module;
a household account book form mapping module for mapping purchased product list information to the household account book form layer; and
and a file generation module for generating a household account book file including purchased product list information, income detail information, and user data mapped to the household account book form layer in the household account book form mapping module. A household account book service providing system using text mining. According to claim 2,
The text formatting module learns an artificial intelligence-based receipt recognition model for classifying receipt formats, and inputs the receipt text information processed by the text mining module to the learned receipt recognition model to correspond to the original receipt of the receipt text information A household account book service providing system using text mining, which outputs a predicted value of a receipt format that According to claim 3,
The text formatting module collects a learning dataset of receipt image-label pairs for each receipt format in order to classify receipt formats having different expression methods including exposure items including product name, unit price, quantity, and amount, and the collected learning data. A system for providing a household account book service using text mining, wherein the receipt recognition model is learned after performing preprocessing on a dataset. According to claim 2,
The text mining module,
A household account book service providing system using text mining, which classifies recognized text into predefined detailed expenditure categories using a pre-learned artificial intelligence model. According to claim 1,
The user terminal performs a registration function of electronic receipts and a registration function of text messages related to payment transmitted from a card company server or a bank server through a smart household account book application, so that payment related text messages and electronic receipts are organized on the household account book file A household account book service providing system using text mining. According to claim 1,
The household account book management server,
When registering a receipt image through the smart household account book application, a preset accumulation point is provided, the current status of the accumulation point is provided according to the inquiry or payment request of the user terminal, and the accumulation point is paid according to a pre-set payment method. A household account book service provision system using mining.

Images