JP7563118B2 - Information processing device, program, and image processing system - Google Patents

Description

The present invention relates to an information processing device, a program, and an image processing system.

It is possible for a scanner or other device to read confidential information such as personal information and store the image data. However, since users need to use the image data for work or other purposes, situations may arise where confidential information is leaked.

Therefore, there has been known a technique for detecting and concealing confidential information from image data generated by reading a document on which confidential information is written using a scanner or the like (see, for example, Patent Document 1). Patent Document 1 discloses a technique for masking certain personal information rather than hiding all of the personal information.

However, the conventional technology has the problem that it cannot restore the concealed secret information. In addition, because the concealed secret information cannot be restored, the conventional technology requires the user to re-scan the original document when the concealed secret information is needed. In addition, the conventional technology uses a method of concealment in which the information is painted over with black, which mars the aesthetic appearance and consumes a large amount of color material.

In view of the above problems, the present invention aims to provide an information processing device that can restore concealed secret information.

In view of the above problems, the present invention is an information processing device that conceals secret information contained in image data, comprising: a secret information detection unit that detects the secret information from image data containing one or more pieces of secret information; a digital watermark generation unit that converts the secret information and attributes of the secret information detected by the secret information detection unit into a digital watermark and forms the digital watermark on the image data; and a restoration unit that restores the secret information from the digital watermark, and is characterized in that the information processing device has an output unit that receives a purpose for disclosing the secret information, and outputs the secret information restored by the restoration unit from the digital watermark if a user identified by login has a security level corresponding to the purpose of the disclosure .

It is possible to provide an information processing device that can restore concealed secret information.

11A and 11B are diagrams illustrating a method for masking secret information in a comparative technique. 1A to 1C are diagrams for explaining an outline of a masking method according to an embodiment of the present invention. FIG. 1 is a diagram showing an overview of an example of an image processing device. FIG. 1 is a diagram illustrating a hardware configuration of an example of an image processing device. FIG. 2 is an example of a functional block diagram for explaining functions of the image processing apparatus by dividing them into blocks. 4A and 4B are diagrams illustrating an example of an original image and management information. 1 is a diagram illustrating an example of a digital watermark formed in a character string. FIG. 11 is a flowchart illustrating an example of a procedure for an image processing device to generate a mask image. 13 is a flowchart illustrating an example of a process in which a staff member selects and outputs personal information. FIG. 13 is a diagram showing an example of a mask attribute list screen. FIG. 13 is a diagram showing an example of a restored image displayed on an operation panel. 11 is a flowchart illustrating an example of a process in which the image processing device displays confidential information according to the security level of an employee and the purpose of using the personal information. FIG. 13 is a diagram showing an example of a disclosure purpose setting screen. FIG. 1 illustrates an example of the configuration of an image processing system. 1A and 1B are diagrams illustrating an example of an original image of a health insurance card and management information generated from the health insurance card. 13A and 13B are diagrams illustrating an example of a masked image and a restored image in the case of a health insurance card.

Below, an image processing device and an information management method performed by the image processing device will be described as an example of a form for implementing the present invention.

<How information on comparative technologies is managed>
Below, we will explain the operations of a bank as an example of an industry that handles confidential information. Banks may store copies of driver's licenses and other documents to verify identity. However, it is rare for a bank to need all the information on a driver's license for its operations, and banks have been managing excessive personal information. On the other hand, if a bank only manages some of the personal information, it will have to request the customer to present their driver's license again when it needs information that it does not manage.

In order to address these problems and make it easier to understand the information management method of this embodiment, we will first explain a comparative technology that can be compared with the technology of this embodiment.

Figure 1 is a diagram explaining a method for masking confidential information in a comparison technique. Figure 1(a) shows an original before masking, which is a driver's license in this case. The image of the original before masking is called an original image 301. A lot of confidential information is written on the driver's license, such as the name, date of birth, and address.

When an image processing device optically reads (images) a driver's license, the image processing device detects confidential information from the image data generated by the read operation. The image processing device converts the confidential information into character codes using, for example, an OCR (Optical Character Reader) and searches for characters that can identify the type of document, such as a driver's license. If the image processing device determines that the document is a driver's license, it searches for confidential information in a predefined format and masks each item on the license where confidential information exists.

Figure 1 (b) shows masked image data 1001. Confidential information such as the date of birth has been masked. In the comparative technology, the image data items are filled in with a solid color, so the user cannot display the confidential information in the image data once it has been masked. For example, the image processing device cannot display the date of birth on a driver's license. If the date of birth is required, the bank employee must borrow the original (driver's license) from the customer and scan it again. In addition, the masking method involves filling in the image with black, which mars the aesthetic appearance and consumes a large amount of color material.

<Information Management Method of the Present Embodiment>
2A and 2B are diagrams for explaining an outline of the masking method of this embodiment. Fig. 2A shows a driver's license (original image 301) before masking, similar to Fig. 1A.

When the image processing device optically reads (takes an image of) the driver's license, all confidential information is recognized from the image data generated by reading. The image processing device replaces each piece of confidential information with a character string. In FIG. 2, the character string is "personal information." The character string may be other letters or numbers. The character string may also be alphabets or symbols. The character string does not have to be a meaningful word.

The image data in which the confidential information has been replaced is called a mask image 302. Figure 2(b) shows the mask image 302. The image processing device stores the original image data in Figure 2(a) and the mask image 302 in Figure 2(b) in association with bank customers.

Microdots are formed in the character string in the mask image 302. The technology of expressing information using microdots is called digital watermarking. An example of information contained in the digital watermark in this embodiment is shown in FIG. 2C. Secret information and information for managing the secret information are included in the digital watermark. As an example, the following management information is converted into the digital watermark. Please refer to Table 1 for details of the management information.
"Mask number, mask attribute, mask content, position information, size information"
Next, a method in which the image processing apparatus restores the management information from the character string will be described. In this embodiment, the mask image 302 contains the secret information of the original image 301, so that the image processing apparatus can restore the secret information from the mask image 302. For example, when a customer makes a loan contract at a bank counter, there is no need to present a driver's license again.

There are two ways to restore the data: A. Any bank employee can restore the data, and B. An employee can restore only the confidential information that corresponds to the employee's security level.

For example, in method A, the image processing device detects the mask attribute (attribute of confidential information) selected by the employee from the digital watermark and outputs the mask content (confidential information).

In method B, the confidential information disclosed by the image processing device is controlled according to the compatibility between the security level of the staff member and the security level of the purpose requiring the confidential information. In this way, once the purpose and staff member are determined, the confidential information to be disclosed by the image processing device is specified. The image processing device outputs confidential information that is not to be disclosed as a string of characters, and if the confidential information may be disclosed, the image processing device restores it and outputs the masked content.

Figure 2(d) shows a restored image 303 in which some of the secret information has been restored. In Figure 2(d), the secret information has been restored except for personal information 2, 4, 5, 6, and 7.

As described above, the image processing device can conceal secret information by replacing it with a digital watermark. During restoration, the image processing device can analyze the digital watermark and extract the secret information. The image processing device can output the secret information any number of times, but because it has been replaced with a digital watermark, the leakage and spread of personal information can be suppressed. In addition, digital watermarks have the advantage that they are less unnatural to look at and require less color material.

<Terminology>
Confidential information is information that is important to an individual or organization and is not publicly available. In this embodiment, personal information is an example of confidential information. Not only information that is generally kept secret, such as a name or address, but also information that is predefined, such as a specific word, is confidential information.

A digital watermark is information embedded into an image using information hiding (data hiding) technology. In this embodiment, an example is described in which an image processing device embeds information using minute dots, but the method of generating a digital watermark is not limited to this. It is sufficient if the information cannot be deciphered by a human, but can be restored by the image processing device using a specified algorithm.

When the secret information is made invisible, it may be that the secret information is deleted or that the secret information is overwritten with other information. It is sufficient that the information cannot be read by a human without data processing.

Concealment means to keep something secret or not to reveal it. Concealment also includes making something obscure or making the contents unreadable. The image processing device may conceal confidential information by overwriting it with other information, or by deleting the confidential information. In this embodiment, this is described using the term mask.

<Configuration example>
3 is a diagram showing an overview of the image processing device 100 according to this embodiment. The image processing device 100 has an automatic document feeder 110, an operation panel 940, a scanner 130, a printer 140, and a paper feed bank 160. The automatic document feeder 110 guides a set document from a transport path to a contact glass, and causes the scanner 130 to read the document.

The operation panel 940 displays a screen that prompts the operator to input instructions to the image processing device 100, and accepts input from the operator. The scanner 130 outputs image data obtained by optically reading an image formed on a document. The printer 140 forms an image on a sheet-like material using ink or toner and outputs the image. The paper supply bank 160 stores the sheet-like material output by the image processing device 100.

In addition, the image processing device 100 may have a finisher. The finisher performs binding processes such as stapling and sorting on the sheet-like material on which an image has been formed. The image processing device 100 may also be connected to a personal computer 200 via a network. The image processing device 100 prints image data received from the personal computer 200 on the sheet-like material. The image processing device 100 may also be connected to a facsimile telephone line PN via a PBX 300.

As shown in FIG. 4, the image processing device 100 has the functions of an information processing device such as a computer.

<Hardware configuration of image processing device>
4 is a diagram showing the hardware configuration of the image processing device 100. As shown in FIG. 4, the image processing device 100 includes a controller 910, a short-range communication circuit 920, an engine control unit 930, an operation panel 940, and a network I/F 950.

Of these, the controller 910 has a CPU 901, which is the main part of the computer, a system memory (MEM-P) 902, a north bridge (NB) 903, a south bridge (SB) 904, an ASIC (Application Specific Integrated Circuit) 906, a local memory (MEM-C) 907, which is a storage unit, a HDD controller 908, and a HD 909, which is also a storage unit, and is configured such that the NB 903 and the ASIC 906 are connected by an AGP (Accelerated Graphics Port) bus 921.

Of these, the CPU 901 is a control unit that performs overall control of the image processing device 100. The NB 903 is a bridge that connects the CPU 901 with the MEM-P 902, the SB 904, and the AGP bus 921, and includes a memory controller that controls reading and writing to the MEM-P 902, a PCI (Peripheral Component Interconnect) master, and an AGP target.

The MEM-P 902 is composed of a ROM 902a, which is a memory for storing programs and data that realize the various functions of the controller 910, and a RAM 902b, which is used for expanding the programs and data, and as a drawing memory during memory printing. The programs stored in the RAM 902b may be provided by recording them in an installable or executable format on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

SB904 is a bridge for connecting NB903 with PCI devices and peripheral devices. ASIC906 is an IC (Integrated Circuit) for image processing applications that has hardware elements for image processing, and acts as a bridge connecting AGP bus921, PCI bus922, HDD controller908, and MEM-C907. This ASIC906 is composed of a PCI target and AGP master, an arbiter (ARB) that is the core of ASIC906, a memory controller that controls MEM-C907, multiple DMACs (Direct Memory Access Controllers) that rotate image data using hardware logic, and a PCI unit that transfers data between scanner unit931 and printer unit932 via PCI bus922. In addition, ASIC 906 may be connected to a USB (Universal Serial Bus) interface or an IEEE 1394 (Institute of Electrical and Electronics Engineers 1394) interface.

MEM-C907 is a local memory used as an image buffer for copying and a code buffer. HD909 is a storage for storing image data, font data used during printing, and forms. HD909 controls the reading and writing of data from and to HD909 under the control of CPU901. AGP bus921 is a bus interface for a graphics accelerator card proposed to speed up graphic processing. AGP bus921 can speed up the graphics accelerator card by directly accessing MEM-P902 with high throughput.

The short-range communication circuit 920 also includes a short-range communication circuit antenna 920a. The short-range communication circuit 920 is a communication circuit such as NFC or Bluetooth (registered trademark).

The engine control unit 930 is further composed of a scanner unit 931 and a printer unit 932. The operation panel 940 displays current settings and a selection screen. The operation panel 940 includes a panel display unit 940a such as a touch panel that receives input from the operator, and hard keys 940b including a numeric keypad that receives settings for image formation conditions such as density settings and a start key that receives a copy start command. The controller 910 controls the entire image processing device 100, and controls, for example, drawing, communication, and input from the operation panel 940. The scanner unit 931 or the printer unit 932 includes an image processing unit such as error diffusion and gamma conversion.

The image processing device 100 can sequentially switch between the document box function, copy function, printer function, and facsimile function using the application switching key on the operation panel 940. When the document box function is selected, the image processing device 100 goes into document box mode. When the copy function is selected, the image processing device 100 goes into copy mode. When the printer function is selected, the image processing device 100 goes into printer mode. When the facsimile mode is selected, the image processing device 100 goes into facsimile mode.

The network I/F 950 is an interface for data communication using the network N3. The short-range communication circuit 920 and the network I/F 950 are electrically connected to the ASIC 906 via the PCI bus 922.

<Functions of the image processing device>
5 is an example of a functional block diagram for explaining functions of the image processing device 100 by dividing them into blocks. The image processing device 100 has a reading unit 21, a display control unit 22, an operation receiving unit 23, an authentication unit 24, an output unit 25, an image processing control unit 26, a secret information detection unit 27, a digital watermark generation unit 28, a restoration unit 29, an OCR unit 30, and a face recognition unit 31. These functions of the image processing device 100 are functions or means realized by any of the components shown in FIG. 4 operating in response to an instruction from the CPU 901 in accordance with a program loaded from the HD 909 or the ROM 902a to the RAM 902b.

The reading unit 21 controls the scanner unit 931 to capture an image of the original document. The reading unit 21 generates image data of the original document image 301 by appropriately correcting gamma correction, density, contrast, etc. Note that the image processing device 100 may obtain image data from an external source or read image data from a storage medium. In this case, the reading unit 21 is not necessary, and the image processing device 100 may be an information processing device.

The display control unit 22 displays, on the panel display unit 940a, an operation screen for the user to input operations related to copying, scanning, facsimile, etc., and a screen showing a job in progress. In this embodiment, a mask image 302, a restored image 303, etc. may be displayed.

The operation reception unit 23 receives user operations on the panel display unit 940a or the hard keys 940b. The display control unit 22 transitions the operation screen in response to the user operations.

The authentication unit 24 authenticates the user. Authentication refers to judging whether the user is a legitimate authorized person. In this embodiment, authentication refers to whether the staff member has the authority to use the image processing device 100. If authentication is successful, the user logs into the image processing device 100. The staff member's authority (security level) is identified by the login. Login refers to the act of authentication to access system resources using pre-registered account information when using various services on a computer or the Internet. Account information includes a user ID and password, an IC card number, biometric authentication information, etc.

Note that authentication may be performed by an authentication server on the network, rather than by the image processing device 100.

The output unit 25 controls the printer unit 932 to form an image on a sheet-like material and discharge the sheet (printing, etc.). The output unit 25 also outputs the composite confidential information, the mask image 302, the restored image 303, etc. on the panel display unit 940a via the display control unit 22.

The image processing control unit 26 calls the confidential information detection unit 27, the digital watermark generation unit 28, the restoration unit 29, the OCR unit 30, and the face recognition unit 31 to control the generation of the digital watermark, the generation of the mask image 302, and the restoration of the digital watermark.

The OCR unit 30 performs character recognition on the document image 301 read by the reading unit 21, and converts the confidential information in the image into a string of characters including one or more characters. Each character is represented by a character code. Characters include numbers, letters, and symbols.

The face recognition unit 31 recognizes human faces when a document contains a photograph of a face. Machine learning using deep learning is known as a form of face recognition. The face recognition unit 31 learns the weights between nodes using the backpropagation method, using as training data image data that contains a face, image data that does not contain a face, and a teacher signal indicating the presence or absence of a face for each image data. The face recognition unit 31 can detect areas that contain a face from the image data. The face recognition unit 31 may also determine the presence or absence of a face after cutting out the area of the photograph in advance.

The confidential information detection unit 27 judges whether the character string generated by the OCR unit 30 is confidential information or not. For example, mask attributes such as "name", "date of birth", and "address" are stored in advance as a dictionary. The confidential information detection unit 27 judges whether the character string recognized by OCR has a mask attribute or not depending on whether it is registered in this dictionary. The confidential information detection unit 27 identifies the position of the circumscribing rectangle of the confidential information that is located at a fixed position for the attribute (mask attribute) of the confidential information (character string). The fixed position is the closest to the mask attribute, for example, the right side, left side, bottom side, top side, etc.

Since the relative positions of the mask attribute and the confidential information vary depending on the type of manuscript, it is preferable that the positions of the mask attribute and the confidential information be prepared in advance in a table or the like in association with the type of manuscript. In this case, the confidential information detection unit 27 determines the type of manuscript based on a character string indicating the type of manuscript, which is included in the character string generated by the OCR unit 30. For example, if the character string generated by the OCR unit 30 includes the character string "license", the confidential information detection unit 27 determines that the manuscript is a driver's license. If the character string generated by the OCR unit 30 includes the character string "health insurance", the confidential information detection unit 27 determines that the manuscript is a health insurance card. Once the type of manuscript is known, the position of the confidential information in the image data can be determined from the table, and the mask attribute of this confidential information can also be determined.

The digital watermark generating unit 28 replaces the confidential information with a predetermined character string, and further forms a digital watermark by forming minute dots in this character string. The predetermined character string may be, for example, "personal information n", where n may be an integer starting from 1. The information formed by the digital watermark includes a mask number, mask attributes, mask contents, position information, and size information. Note that the digital watermark generating unit 28 may form only a digital watermark in the mask image 302 without a character string. However, the presence of a character string makes it easier to conceal the existence of the digital watermark.

The restoration unit 29 analyzes the digital watermark formed in the character string and restores the secret information. The restoration unit 29 deletes the character string in which the digital watermark was formed ("Personal information n"), and places the restored secret information in the mask image 302.

The image processing device 100 also has a storage unit 39 realized by the HD 909 or ROM 902a shown in FIG. 4. The storage unit 39 has a management information storage unit 41, a staff information storage unit 42, an image data storage unit 43, and a target information storage unit 44 (see Tables 1 to 4).

表１は、管理情報記憶部４１が記憶する管理情報の一例を示す。管理情報は、画像処理装置１００がマスク画像３０２を生成する場合に一時的に作成され、マスク画像３０２の作成後は削除される。これにより、漏洩リスクを低減できる。

Table 1 shows an example of the management information stored in the management information storage unit 41. The management information is temporarily created when the image processing device 100 generates the mask image 302, and is deleted after the mask image 302 is created. This can reduce the risk of leakage.

The management information is a list of confidential information obtained by the OCR unit 30 from a document such as a driver's license. Table 1 is an example of management information when a driver's license is read. The management information includes a mask number, mask attributes, mask contents, position information, and size information.

Mask number: The mask number is identification information for the secret information that is assigned so as not to overlap within one document image 301. The mask numbers are assigned, for example, from left to right and from top to bottom in the order in which the secret information appears.
Mask attribute: The mask attribute is the name of the confidential information item. The confidential information detection unit 27 searches the character string obtained by OCR in a dictionary to detect the mask attribute. The dictionary lists character strings contained in a manuscript containing confidential information, such as name, address, birth, validity, and delivery.
- Mask content: The mask content is confidential information. The confidential information detection unit 27 identifies the position of the confidential information based on the relative position to the mask attribute or the type of document. The mask content other than the face photo is a character string. The face photo is image data such as JPEG. Of the management information in Table 1, the confidential information detection unit 27 does not need to delete only the face photo even after creating the mask image 302. This is because a digital watermark cannot store the capacity of a face photo.
Position information: The position information is, for example, the X and Y coordinates of the upper left vertex of a circumscribing rectangle of the secret information.
Size information: The size information is the width and height of the circumscribing rectangle of the secret information.

表２は、職員情報記憶部４２が記憶する職員情報の一例を示す。職員情報は、職員に関する情報である。職員情報は、職員ＩＤ、職員名、パスワード、及び、セキュリティレベルを有している。
・職員ＩＤ…職員ＩＤは職員の識別情報である。本実施形態では、メールアドレスが職員情報であるが、一意であれば任意の文字列等でよい。
・職員名…職員名は職員の氏名等である。
・パスワード…職員のみが知る秘密の文字列である。
・セキュリティレベル…セキュリティレベルは職員がどの秘匿情報にアクセスできるかの権限を示す。「職員のセキュリティレベル≧（後述する）目的のセキュリティレベル」を満たす秘匿情報であれば、職員がアクセスできる。セキュリティレベルは、例えば、１～５の数字で表される。数字が大きいほど秘匿性が高い。

Table 2 shows an example of staff information stored in the staff information storage unit 42. The staff information is information about staff members, and includes a staff ID, a staff name, a password, and a security level.
Staff ID: The staff ID is identification information for a staff member. In this embodiment, the staff member information is an email address, but any unique character string may be used.
・Staff name: Staff name is the name of the staff member.
・Password: A secret string of characters known only to employees.
・Security level: The security level indicates the authority of which confidential information an employee can access. Employees can access confidential information if it satisfies "employee security level ≥ desired security level (described later)". Security levels are expressed, for example, by numbers from 1 to 5. The higher the number, the higher the confidentiality.

表３は、画像データ記憶部４３が記憶する画像データ情報の一例を示す。画像データ情報は、原稿画像３０１やマスク画像３０２を有している。画像データ情報は、顧客ＩＤ、顧客氏名、原稿画像３０１、及び、マスク画像３０２を有している。
・顧客ＩＤ…顧客ＩＤは、銀行の顧客の識別情報である。
・顧客氏名…顧客氏名は顧客の氏名等である。
・原稿画像３０１…原稿画像３０１は該顧客の原稿画像３０１である。原稿画像３０１は秘匿性が高いので、マスク画像３０２とは別の記憶手段（ネットワークにつながっていないとよい。）に暗号化の上、保存される。
・マスク画像３０２…マスク画像３０２は該顧客のマスク画像３０２である。

Table 3 shows an example of image data information stored in the image data storage unit 43. The image data information includes an original image 301 and a mask image 302. The image data information includes a customer ID, a customer name, the original image 301, and the mask image 302.
Customer ID: The customer ID is identification information for a bank customer.
· Customer name: The customer name is the name of the customer, etc.
Original image 301: The original image 301 is an original image of the customer. Since the original image 301 requires high confidentiality, it is encrypted and stored in a storage means (preferably not connected to a network) separate from the mask image 302.
Mask image 302...The mask image 302 is a mask image 302 of the customer.

表４（ａ）は、目的情報記憶部４４が記憶する目的別セキュリティ情報の一例を示す。目的別セキュリティ情報は、目的とセキュリティレベルの項目を有している。
・目的…目的は、例えば銀行の業務において、職員がどのような目的でマスク画像３０２を使用するかを示す。
・セキュリティレベル…対応する目的が必要とする職員のセキュリティレベルを示す。

Table 4(a) shows an example of purpose-specific security information stored in the purpose information storage unit 44. The purpose-specific security information has items of purpose and security level.
Purpose: The purpose indicates for what purpose an employee uses the mask image 302 in bank operations, for example.
-Security level: Indicates the level of security required for personnel to fulfill the corresponding objective.

Table 4(b) shows an example of the disclosable information stored in the objective information storage unit 44. The disclosable information is associated with personal information that may be disclosed depending on the consistency between the security level of the employee and the objective security level. The disclosable information has a relationship between the security level and the items of personal information to be disclosed.
- Security level relationship: The security level relationship indicates the relative level of security between the staff member and the objective.
Personal information to be disclosed: Personal information to be disclosed is confidential information to be disclosed by the image processing device 100 in the case of a corresponding magnitude relationship. Although the personal information to be disclosed in Table 4(b) is represented by personal information n in the case of a driver's license, the personal information to be disclosed may be registered with a mask attribute.

<An example of a manuscript>
6A shows an example of a document read by the image processing device 100. This document is a driver's license. The confidential information on the driver's license includes 1. name, 2. date of birth, 3. address, 4. date of issue, 5. expiration date, 6. license conditions, 7. whether the driver is a good driver, 8. license number, 9. photograph, 10. license type, 11. prefecture of the public safety commission to which the driver belongs, 12. previous year of issue, 13. previous month of issue, 14. previous date of issue, etc. These numbers 1 to 14 correspond to the mask numbers in Table 1.

The secret information detection unit 27 identifies the mask attribute based on the character string generated by the OCR unit 30. Alternatively, the secret information detection unit 27 determines that it is a driver's license based on the character string generated by the OCR unit 30. The secret information detection unit 27 identifies the secret information based on the format of the license, and detects the secret information of the license (mask content in Table 1) that corresponds to each mask attribute. The secret information detection unit 27 obtains the circumscribing rectangle of the secret information, or the position and size of the secret information determined by the format. As a result, one row of data in Table 1 is obtained for each piece of secret information.

Next, the digital watermark generation unit 28 replaces the secret information with "personal information 1" to "personal information 14". Figure 6 (b) is an example of a mask image 302 in which the digital watermark generation unit 28 has replaced the secret information with "personal information 1" to "personal information 14". The secret information of mask numbers 1 to 14 has been replaced with "personal information 1" to "personal information 14". The digital watermark generation unit 28 then forms a digital watermark for personal information 1 that includes management information for mask number 1 in Table 1 (one row's worth of management information). The digital watermark generation unit 28 also forms a digital watermark for personal information 2 to 14 that includes management information for each row of mask numbers 2 to 14 in Table 1.

The digital watermark generation unit 28 does not have to convert all the information in one row of Table 1 into a digital watermark. In this case, however, the digital watermark generation unit 28 must convert at least the mask number into a digital watermark, and the management information of Table 1 must be stored in the image processing device 100 or in storage on the network.

For the face photo of mask number 9, the image processing device 100 stores a file of the face photo. The digital watermark generation unit 28 includes the URL and file path of the face photo file in the digital watermark of mask number 9. Alternatively, the image processing device 100 does not need to store a file of the face photo. In this case, the restoration unit 29 crops the face photo from the original image 301, so the digital watermark of the face photo includes the position and size.

<An example of a digital watermark>
Fig. 7 is a diagram for explaining a digital watermark formed in a character string. Fig. 7 shows a method for generating a digital watermark in which information is embedded by adding minute dots to the outline of a character or by removing the dots. In Fig. 7, two minute dots 401 and 402 are added to the character "D." This is an example of a method for generating a digital watermark, which will be briefly explained.

(i) The electronic watermark generating unit 28 detects edges in an area where, for example, the character string "D" is formed (in the example of a driver's license, the area where the electronic watermark generating unit 28 has drawn "personal information n" in the working memory).

(ii) The outer edge of the letter "D" is detected by the edges. The digital watermark generation unit 28 adds or removes tiny dots on the inside and outside of the outer edge of the "D" in order from a starting point determined by an algorithm. For example, adding a tiny dot corresponds to a "1" and removing a tiny dot corresponds to a "0". Thus, digital signals of 1 and 0 are formed in the "D". The digital watermark generation unit 28 encodes each number and letter with a bit string of a fixed length and forms it on the outer edge of the "D".

(iii) During restoration, the restoration unit 29 detects the outer edge of the letter "D" by edge detection. The restoration unit 29 detects the addition or loss of minute dots from the inside and outside of the outer edge of the "D" in order from a starting point determined by an algorithm. Therefore, the restoration unit 29 can restore a digital signal of 1s and 0s. The restoration unit 29 converts each bit string of a fixed length into numbers or letters.

<Mask image generation flow>
Fig. 8 is an example of a flow chart showing the procedure for image processing device 100 to generate mask image 302. Fig. 8 shows the process when a bank employee operates image processing device 100. The bank employee inputs information about the customer for which mask image 302 is to be created (customer name and customer ID) into image processing device 100.

First, the staff member selects the app "Personal Information Protection Scan" on the operation panel (S1). The operation reception unit 23 receives the selection, and the app "Personal Information Protection Scan" is launched.

The employee places the document (e.g. a driver's license) on the contact glass and presses a start button or the like. This causes the reading unit 21 to scan the document and generate a document image 301 (S2).

The OCR unit 30 performs character recognition (OCR) on the original image 301, and the confidential information detection unit 27 identifies the mask attribute and the area (position and size) of the confidential information. In addition, the face recognition unit 31 performs face recognition on the original image 301 and identifies the area of the face photo (S3).

The secret information detection unit 27 acquires mask attributes and secret information from the identified area (S4). The secret information detection unit 27 also assigns mask numbers in a fixed order, such as from left to right or from top to bottom.

The digital watermark generation unit 28 creates a character string of "personal information n" (n is the mask number) for each mask number. The digital watermark generation unit 28 then draws the character string of "personal information n" in the working memory, and forms confidential information, etc. in this character string as a digital watermark (S5). The digital watermark may include "mask number, mask attribute, mask content, position information, and size information."

The digital watermark generating unit 28 deletes the secret information from the area where the acquired secret information existed (or may overwrite the secret information with "personal information n"), and replaces each area of secret information with a character string (in the form of an image) of "personal information n" in which a digital watermark has been formed (S6). This process is called "masking". The image data in which the secret information has been replaced with "personal information n" is called a mask image 302.

The image processing control unit 26 associates the customer ID, customer name, original image 301, and mask image 302 and stores them in the image data storage unit 43 (S7).

As a result, the image processing device 100 can read a document containing multiple pieces of confidential information, identify the confidential information, and generate and store a mask image 302 in which the contents of the confidential information are embedded.

<Confidential information output flow>
Next, a flow of a user outputting confidential information will be described. Below, A. A method in which any bank employee can restore the confidential information, and B. A method in which an employee can restore only confidential information according to the employee's security level will be described.

<<A. How any bank employee can restore it>>
FIG. 9 is an example of a flow chart illustrating a process in which a staff member selects and outputs personal information.

The staff member inputs the customer ID or customer name of the customer whose personal information he/she wishes to use into the image processing device 100 (S11). The staff member may log into the image processing device 100, but method A does not take into account the security level of the staff member.

The staff member also selects the personal information to be output from the list of mask attributes (S12). The operation reception unit 23 receives the customer's customer ID or customer name, and the selection of mask attributes. Figure 10 shows an example of the mask attribute list screen. The list of mask attributes may be a dictionary used to detect mask attributes.

The image processing control unit 26 may display a mask image 302 that is stored in association with the customer name or customer ID. The staff member can select personal information 1 to 14 (mask attribute) that he or she wishes to display from the mask image 302. The operation reception unit 23 accepts the selection of personal information 1 to 14.

The image processing control unit 26 retrieves the mask image 302 of the corresponding customer, which is stored in association with the customer ID, from the image data storage unit 43 (S13).

The restoration unit 29 restores the digital watermark from the acquired mask image 302 (S14). The personal information to be restored may be that selected by the employee, so the restoration unit 29 restores the digital watermark of the mask image 302 in a fixed order until it finds the mask attribute selected by the employee. Finally, the restoration unit 29 restores only the personal information with the mask attribute selected by the employee, and discards any other personal information even if it is restored.

The output unit 25 displays only the personal information with the mask attribute selected by the staff member on the operation panel via the display control unit 22. Alternatively, the restoration unit 29 replaces "personal information n" in the mask image 302 with the restored personal information based on the position and size of the personal information included in the digital watermark. The replacement can be performed by the restoration unit 29 deleting "personal information n" including the digital watermark from the image data, and forming the personal information in the image data based on the position information. As a result, the restoration unit 29 generates a restored image 303, and the output unit 25 displays the restored image 303 on the operation panel (S15). The output unit 25 may print the restored image 303.

In this way, the image processing device 100 can restore and display only the personal information selected by the user.

Figure 10 is an example of a mask attribute list screen. The mask attribute list screen has a message saying "Please select attributes to display," an OK button, and a list of mask attributes. The staff member selects one or more mask attributes and presses the OK button.

<<Example of restored image display>>
Fig. 11 shows an example of a restored image 303 displayed on the operation panel. In Fig. 11, among the 14 pieces of personal information, personal information 1, 3, and 9 have been replaced with confidential information. That is, since the user selected the name, address, and face photo (personal information 1, 3, and 9) on the operation panel, the confidential information corresponding to personal information 1, 3, and 9 is displayed.

<<B. A method for allowing employees to restore only confidential information that corresponds to their security level>>
FIG. 12 is an example of a flow chart illustrating a process in which the image processing device 100 displays confidential information according to the security level of the staff member and the purpose of using the personal information.

The staff member logs in by inputting the staff ID and password into the image processing device 100 (S21). The operation reception unit 23 receives the input, and the authentication unit 24 determines whether the authentication was successful or not. Here, it is assumed that the authentication was successful. The user ID and security level are identified by the login.

The staff member inputs the customer ID or customer name of the customer whose personal information he/she wishes to use into the image processing device 100 (S22).

The employee selects the purpose for which the personal information is to be used on the operation panel (S23). The operation reception unit 23 accepts the selection. An example of the disclosed purpose setting screen is shown in FIG. 13. The purposes are insurance contracts, loan contracts, and opening fixed-term savings accounts, etc., as shown in Table 4(a). The restoration unit 29 obtains the security level associated with the selected purpose.

The image processing control unit 26 retrieves the mask image 302 of the corresponding customer, which is stored in association with the customer ID, from the image data storage unit 43 (S24).

The restoration unit 29 compares the security level of the staff member with the security level associated with the purpose. As a result of the comparison, the restoration unit 29 identifies a mask attribute from the disclosable information in Table 4 (b) that corresponds to "staff security level ≧ purpose security level" or "staff security level < purpose security level" (S25).

The restoration unit 29 restores the digital watermark from the acquired mask image 302 (S26). Since the personal information to be restored may be one that is consistent with the security level, the restoration unit 29 restores the digital watermark of the mask image 302 in a fixed order until a mask attribute that is consistent with the security level is found. Finally, the restoration unit 29 restores only the personal information with a mask attribute that is consistent with the security level, and discards any personal information that is not consistent with the security level even if it is restored.

The output unit 25 displays, via the display control unit 22, on the operation panel only the masked personal information that may be disclosed when the security level held by the staff member is equal to or higher than the security level corresponding to the purpose of disclosure. When the security level held by the staff member is lower than the security level corresponding to the purpose of disclosure, the output unit 25 outputs a predetermined portion of the personal information.

The output unit 25 may replace "personal information n" in the mask image 302 with the restored personal information based on the position and size of the personal information included in the digital watermark. As a result, the restoration unit 29 generates a restored image 303, and the output unit 25 displays the restored image 303 on the operation panel (S27). The output unit 25 may print the restored image 303.

As a result, the image processing device 100 can output personal information from among the multiple pieces of personal information contained in the mask image 302, according to the security level of the employee and the security level of the purpose for which the personal information will be used.

Figure 13 is an example of a disclosure purpose setting screen. The disclosure purpose setting screen has a message saying "Please select the purpose of disclosure" and one or more purposes. The employee selects the purpose of disclosure.

<Other configuration examples>
In the above embodiment, the image processing device 100 generates the mask image 302 and restores the secret information by itself. However, the image processing device 100 and a server can perform similar processes.

Figure 14 shows an example of the configuration of an image processing system. The image processing system has a server 210 and an image processing device 100 that can communicate via a network. In this configuration, the image processing device 100 executes a web application provided by the server 210, and a staff member operates the web application. The image processing device 100 transmits an original image 301 to the server 210 in response to the staff member's operation. As the server 210 is an information processing device, it has the functions of Figure 5 and can generate a mask image 302.

During restoration, the server 210 restores the personal information selected by the employee through the web app or the personal information according to the security level, generates a restored image 303, and transmits it to the image processing device 100.

With this configuration, the processing can be consolidated in the server 210, so the image processing device 100 does not need to have the functions shown in FIG. 5 individually. This helps prevent increases in costs.

The image processing device 100 is not limited to a device such as an MFP, but may be any device having an imaging means and a communication means. For example, information processing devices such as smartphones, tablet terminals, PCs (Personal Computers), game consoles, and PDFs (Personal Digital Assistants) have cameras. The information processing device can generate a mask image 302 by transmitting an original image 301 captured by the camera to the server 210. During restoration, the server 210 restores personal information selected by the user by operating a web app or personal information corresponding to a security level from the mask image 302 to generate a restored image 303 and transmits it to the image processing device 100.

<Examples of documents other than driver's licenses>
In this embodiment, a driver's license has been mainly used as an example. However, the document on which confidential information is recorded is not limited to a driver's license, and this embodiment can be applied to any document on which confidential information is recorded.

Using Figures 15 and 16, we will explain the case where a health insurance card is used as an example of a document containing confidential information in the image processing device 100 of this embodiment. Figure 15 shows an example of a document image 301 of a health insurance card and management information generated from the health insurance card. Figure 16 shows an example of a mask image 302 and a restored image.

As shown in FIG. 15(a), the health insurance card contains personal information such as the type of insured person, date of issue, symbol, number, name, date of birth, date of qualification, gender, business location, business name, insurer location, insurer number, insurer name, insurer seal, and issue serial number.

Also, as shown in FIG. 15(b), the image processing device 100 can read the health insurance card and perform character recognition to recognize all personal information, and detect and list the attributes, content, position, and size of the personal information.

As shown in FIG. 16(a), the image processing device 100 converts each detected piece of personal information into a digital watermark and forms the digital watermark in a predetermined character string. The image processing device 100 deletes the personal information from the original image 301 and forms a character string with a digital watermark. In this way, the image processing device 100 can generate a mask image 302 of the health insurance card.

The mask attributes to be disclosed are regulated according to the compatibility between the purpose of the certificate and the security level of the employee. As a result, when an employee uses mask image 302 in a certificate or the like, once the purpose and employee are confirmed, the mask attributes to be disclosed by image processing device 100 can be specified. Image processing device 100 leaves undisclosed mask attributes as "personal information n" and displays the mask contents of mask attributes that may be disclosed by restoring the digital watermark.

By doing this, as shown in FIG. 16(b), the image processing device 100 can display only the personal information required for the certificate. This makes it possible to prevent the leakage and spread of personal information.

In FIG. 16(b), the image processing device 100 has restored the name of personal information 5, the date of birth of personal information 6, the name of the business establishment of personal information 10, the name of the insurer of personal information 13, and the insurer seal of personal information 14 to the restored image 303.

As a result, similar to a driver's license, the image processing device 100 can display or print only the necessary personal information out of multiple pieces of personal information, and by not restoring other personal information, it is possible to prevent the leakage of personal information.

<Major Effects>
As described above, the image processing device 100 of this embodiment replaces the secret information with a digital watermark. During restoration, the image processing device 100 can analyze the digital watermark and extract the secret information. The image processing device 100 can output the secret information any number of times, but since the secret information has been replaced with a digital watermark, the leakage and spread of personal information can be suppressed. In addition, a digital watermark has the advantage that it is less unnatural to look at and requires less color material.

<Other application examples>
The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

For example, this embodiment can be applied not only to banking operations, but also to operations that require identity verification, such as securities companies, insurance companies, real estate procedures, local governments, or mobile phone contracts.

In addition, a digital watermark containing secret information may be formed outside the area of the corresponding secret information. The digital watermark only needs to be present on the sheet-like member.

For example, the configuration example in FIG. 5 and the like is divided according to the main functions in order to make it easier to understand the processing by the image processing device 100. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the image processing device 100 can also be divided into more processing units depending on the processing content. It can also be divided so that one processing unit includes more processes.

Each function of the embodiments described above can be realized by one or more processing circuits. In this specification, the term "processing circuit" includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), and conventional circuit modules designed to execute each function described above.

100 Image processing device 210 Server 301 Original image 302 Mask image 303 Reconstructed image

Patent No. 4335930

Claims (9)

An information processing device that conceals confidential information included in image data,
a secret information detection unit that detects one or more pieces of secret information from image data including the one or more pieces of secret information;
a digital watermark generating unit that converts the secret information and an attribute of the secret information detected by the secret information detecting unit into a digital watermark and forms the digital watermark in the image data;
a restoration unit that restores the secret information from the digital watermark,
Accept the purpose of disclosing the confidential information,
An information processing device characterized in that it has an output unit that outputs the secret information restored from the electronic watermark by the restoration unit when a user identified by login has a security level corresponding to the purpose of the disclosure . The information processing device according to claim 1, characterized in that the digital watermark generation unit forms the digital watermark in an area of the image data where the secret information exists. the secret information detection unit detects position information of the secret information,
3. The information processing apparatus according to claim 2, wherein the digital watermark generating section forms a digital watermark including the secret information and position information of the secret information in an area of the image data where the secret information exists. the restoration unit restores the secret information and position information of the secret information from the digital watermark;
The restoration unit removes the digital watermark from the image data,
4. The information processing apparatus according to claim 3, wherein the digital watermark generating section forms the secret information in the image data based on position information of the secret information. The digital watermark generating unit forms the digital watermark in a predetermined character string,
5. The information processing apparatus according to claim 1, wherein the electronic watermark formed in the character string is formed in an area of the image data where the secret information exists. The digital watermark generating unit further forms a digital watermark including an attribute of the secret information in the image data,
displaying a list of attributes of the confidential information, and accepting a selection of the attributes to be disclosed from the list of attributes of the confidential information;
the output unit outputs only the secret information having the same attribute as the received attribute from among the attributes of the secret information restored from the digital watermark by the restoration unit.
5. The information processing apparatus according to claim 4. If the security level of the user identified by the login is equal to or higher than the security level corresponding to the purpose of the disclosure,
the output unit outputs the secret information that may be disclosed when a security level held by the user is equal to or higher than a security level corresponding to a purpose of the disclosure;
If the user has a security level that is less than the security level required for the purpose of the disclosure,
The information processing apparatus according to claim 1 , wherein the output unit outputs a predetermined part of the secret information. An information processing device for concealing confidential information contained in image data,
a secret information detection unit that detects one or more pieces of secret information from image data including the one or more pieces of secret information;
a digital watermark generating unit that converts the secret information and an attribute of the secret information detected by the secret information detecting unit into a digital watermark and forms the digital watermark in the image data;
a restoration unit that restores the secret information from the digital watermark;
Accept the purpose of disclosing the confidential information,
A program for causing the restoration unit to function as an output unit that outputs the secret information restored from the electronic watermark when a user identified by login has a security level corresponding to the purpose of the disclosure . An image processing system for concealing confidential information contained in image data, comprising:
A reading unit that reads a document including confidential information and generates image data;
a secret information detection unit that detects one or more pieces of secret information from image data including the one or more pieces of secret information;
a digital watermark generating unit that converts the secret information and an attribute of the secret information detected by the secret information detecting unit into a digital watermark and forms the digital watermark in the image data;
a restoration unit that restores the secret information from the digital watermark ,
Accept the purpose of disclosing the confidential information,
An image processing system characterized in that it has an output unit that outputs the secret information restored from the electronic watermark by the restoration unit when a user identified by login has a security level corresponding to the purpose of the disclosure .

Images