FR2960999A1 - Word constructionless communication method for dumb and/or deaf, involves displaying detail or photo or pictogram on screen and/or vocalizing message using synthesis or recording voice - Google Patents

Abstract

The method involves selecting a relative photo or pictogram appropriate by action on corresponding key, and storing the photo or pictogram in a selection memory. An appropriate tool is selected if the action is carried out for the selected photo. An additional mode or detail is selected if the action is not carried out for the selected photo, and the additional mode or detail is stored in another selection memory. The detail or photo or pictogram is displayed on a screen and/or a message is vocalized using synthesis or recording voice. An independent claim is also included for a device for word constructionless communication in a portable cellular phone to a dumb and/or deaf.

Description

PHOTOGRAPHIC OR PICTOGRAPHIC COMMUNICATION METHOD FOR MUTE MUTE MUTE MUTE AND ITS IMPLEMENTATION DEVICE

The present invention relates to a communication method allowing in particular nonverbal communication equivalent to a sentence construction and its preferentially portable implementation device. It applies more particularly, but not exclusively, to people who have suffered an aphasia stroke and who have lost the use of the word and who express themselves only non-verbally. Other persons are concerned, provisionally or permanently and totally or partially dumb and / or dumb using or not using sign language or any person preferring not to show an accent, a stuttering, a difficulty to articulate . The invention can also be applied to people who no longer have vocal cords or language following an operation for cancer, children with autism and people. Finally, paralyzed and tracheotomized persons may use a device with a guiding device.

In general, during their daily activities and movements, dumb and / or dumb people need to get in touch with other people.

To communicate, they most often use an erasable slate 30 or a sign. This solution is not rewarding, not fast and can be cumbersome or impossible in case of paralysis.

There are also devices with physical keys for composing syllables and other more sophisticated devices using a display and a data entry mode but they generally require the use of the letters of a keyboard to form a sentence.

Some devices use pictograms and link a message to a drawing. These devices require long handling, detrimental to the conversation.

The object of the invention is to overcome these drawbacks, in particular: by reducing the time required to formulate a message; a photo or pictogram can save the design of a sentence but a personal photo is more effective than a long speech; - lowering the level of design or development of the message to allow those who can not master the complexity of language or writing to avoid the concept in favor of the need directly shown what is to communicate; this mode does not require mastering the alphabet. - by allowing to view the detail of the photo or pictogram shown on the finger or by a geometric figure moving on the screen which also allows to refine the object of the communication; - allowing the use of a personal photo database or a pictograph database of a third party depending on the circumstances; - by allowing to magnify by means of a zoom a detail;

In addition, the system is physically and visually adjustable for greater comfort for people with a vision disorder associated or with a motor disability or tremor that hinders the accuracy of the gesture. The size of the screen or the direct or indirect control of the movement can be adapted according to the motor handicap. For example, the users are elderly persons who have had a degenerative-type cerebral lesion or a cerebral accident with aphasia, persons who no longer have vocal cords or tongue after an operation of a cancer of the throat. or larynx, deaf or dumb deaf people using or not sign language, children with autism, intellectually limited persons, stutterers or voiceless persons.

In order to adapt to the age and the declining or ascending level of speech learning, the device comprises selection means 10 and a variable level of complexity for the reproduction of the information attached to the pictures in the vocal reproduction or the speech. use of third-party means for selection and eventual movement to detail. The most direct means being finger pointing, any other indirect means is compatible with the device including mouse, joystick, contactor, up to 15 scroll defined as a systematic and timed scanning for example vertical or horizontal of the entire 'screen.

The device according to the invention has many advantages in particular: it reduces the time required to formulate a message; a photo or pictogram can save the design of a sentence but a personal photo is more effective than a long speech; it is universal and crosses at the same time the barrier of the language and of a required intellectual level. It lowers the level of conception or elaboration of the message to enable those who can not master the complexity of language or writing to show what is to be communicated; by making it possible to visualize the detail of the photograph or pictogram shown on the finger or by a geometrical figure moving on the screen which also makes it possible to refine the object of the communication; 30 by allowing the use of a database of personal photos or a third party depending on the circumstances; - by allowing to magnify by means of a zoom a detail; it has dimensions that vary according to the material used and may have sufficiently small dimensions to carry it on one's hand very easily, for example in the hand, stored in a shirt pocket or worn on the belt; - it makes it possible to prepare communications in advance in a unique or repetitive manner; 5 - it can be presented otherwise than in a sound manner, especially with a person not knowing the difficulties of speech of the user; - it can be used by several in the same place or in dialogue by internet by showing the image and possibly a detail of it. 10 - it allows a facilitated conversation with a hearing person or not.

The invention proposes a communication method for dumb and / or deaf-mutes allowing to choose within any classification characterizing families of photos or pictograms in general themes. In a second step, the method makes it possible to show the photo or the pictogram in full screen or to show a finger or a scanning system a talking detail. The user can choose a form of magnifying glass without any vocalization to identify a detail that speaks alone. By way of example, the user can show the wheel of a wheelchair to signify that it is deflated or punctured, show a closet with clothes to indicate what he wants to wear or a shelf or a drawer. personal furniture to signify where to pick up an object, to show a self-portrait of face or back or profile to designate a pain or a need for massage.

The method is characterized in that each step of selecting and storing is followed by a display step and in that it comprises a possible preliminary step of delimiting with a material such as a stylus stereotyped shapes or specific to delimit a detail and possibly include a voice message,

The method is characterized in that a bank of photos or pictograms can be used or personal photos can be taken by means of a digital camera, allowing a classification of the general themes characterizing families of photos including personal or pictograms corresponding to everyday situations, objects and persons of the entourage, to actions, to categories of needs, to persons or objects, to categories of situations such as shops, categories Parks.

The method is characterized in that nonverbal communication by the pointing device is preferentially used rather than voice synthesis which is not triggered automatically. .

The method is characterized in that the possible means (2) pointing or scanning and / or selection include pointing objects such as a stylus but also peripheral type to the computer type joystick or mouse or of virtual type with directional arrows operated on the screen or finally by a systematic scanning system of the entire screen at a chosen rate.

The method is characterized in that a magnification function is operable by operating with a material such as a stylus logos to zoom in on the photo to go to the detail or to exclude objects from the photo or pictogram. ;

The method is characterized in that in the case of an impossibility of using a material such as a stylus, a mode of automatic scanning for example horizontal or vertical of geometrical figures makes it possible to move on the screen until detail to refine the object of communication by stopping scanning by a motion switch;

The method is characterized in that the method also makes it possible, at the time of programming, to define with a material such as a stylet the outlines of an element of the photograph by means of a conventional or single geometric figure of closed polygonal shape. by a specific multipoint mode of delimitation and to be able to associate possibly a vocalized or juxtaposed message next to the identified detail such as a comic comment bubble;

The method is characterized in that the information corresponds to sounds recorded for example musical or emitted by a person by a microphone on the computer or mobile equipment.

The finger can be used as a stylus on a touch screen whether it is of pocket size or of a usual size, this method being implemented by a device comprising: input and / or selection means comprising keys, pointing or scanning means and / or selection comprising keys that can be physical or virtual, and possibly a stylus; A processing module, a management module of the database, a display module comprising at least one screen; - A memory module, - a storage module of at least one database, 20 The method is characterized in that it comprises: o the photo mode or full-screen pictogram, o the mode pointing a detail of the photo or pictogram, o a mode of vocalization of the photo and / or the detail of the photo 25 or pictogram, o a silent full screen mode displaying the text or the detail relating to the photo or pictogram, o a mode that encircles a detail by means of a geometric shape, o a mode that allows a magnification function of the detail

said method comprising the steps of: a) displaying general themes corresponding to the categories of photos or pictograms; 2960999 -7- b) an action on a key corresponding to a general theme; c) a scrolling of photos or pictograms relating to the general theme thus selected; d) a selection of the appropriate relative photo by action on the corresponding key; e) storage in memory of the selection; f) a possible action to zoom or a mode of displacement in the case where a detail must be shown; g) a possible selection of the mode or additional detail; h) storage in memory of the selection; i) a repetition of steps f) to h) until there is no further validation; j) a display of the sentence or message thus constructed on the screen and / or a vocalization of said message by means of a synthetic voice.

Of course, each step of selection and storage may be followed by a display step.

The method is characterized in that the keys are virtual.

The method is characterized in that it further comprises a speech synthesis module or recording, control means of said speech synthesis and means for listening to the sentence or recording. such as headphones, headphones or speaker.

The method is characterized in that it comprises wired or wireless data transmission means to a second display module with second input and / or selection means for sending the photo or detail.

The method is characterized in that it is shaped to be used with a mobile phone or integrated in mobile phones called "smartphone". The system also makes it possible, at the time of programming, to define the contours of an element of the photograph or to assign a classic geometric figure triangle, square, round, or a single or unclassified figure allowing the entourage of the object by segments forming a figure 5 closed by a mode of creating lines between the points automatically created is a geometric figure of polygonal type. This possibility allows you to choose a detail. In a second programming time, a message can be associated with the detail, this being for example vocalized by synthetic voice or by a recording of the entourage. Subsequently, the user shows a finger or moves the cursor from one point to another of the screen to vocalize the message corresponding to the detail. For the user, the photo is then presented with details surrounded by circular or triangular or square classical geometric figures or by a custom-made contour providing a multi-label photo appearance.

This designation may be silent and is considered non-verbal communication. The designation may also be accompanied by a voice message. In the latter case, the system makes it possible, at the time of programming, to define the contours of an element of the photograph or to assign a classical geometrical figure to a detail and to assign a message struck on the keyboard of the photograph. computer for this message to be read by synthetic voice or by human voice recording. The message displayed in letters can be juxtaposed or a bubble info appears next to the detail at the touch. The text of the highlighted detail can be vocalized or shown in local display or in full screen to ensure discretion. In a variant of the device, the detail can be zoomed magnetically for the visually impaired but also in general to compensate for the relative narrowness of the screen to allow a reading for several interlocutors at a time or to be able to go even more directly or precisely to the communication.

The speech synthesis may be triggered automatically each selection or by using a specific key after verifying the validity of the message.

The device may include visual means for tracking the speech utterance and the progression of the sentence so that the disabled person, especially the deaf person, can know when the sentence is finished.

The device can be fully controlled by the 4 arrow keys of the numeric keypad or by a form of joystick or trackball or any other scrolling system that allows a command to close the eyelid or by a contactor to the skin.

The device also makes it possible to paste on the same photo elements that have a relationship with a general theme of the photo, always in the sense of a nonverbal communication or conversely with a comment. Embodiments of the invention will be described hereinafter by way of non-limiting examples, with reference to the accompanying drawings, in which: FIG. 1 represents a diagram of an architecture of a device according to the invention;

Figure 2 shows a flowchart of a method according to the invention;

FIG. 3 illustrates the method of the invention.

FIG. 1 illustrates a device according to the invention comprising the following elements: input and / or selection means 1, pointing means 2, a processing unit 3, a database management module 3, a display module 4 comprising at least one screen; at least one RAM memory and a ROM 5 containing executable programs corresponding to the various operating modes of the keyboard, to a database management system, voice synthesis, at least one database, a battery 7, - optionally listening means 8 such as a speaker.

The method is implemented as follows (FIG. 2): a) a display of general themes corresponding to the categories of photos; (Block 9), b) an action on a key corresponding to a general theme (Block 10), 5 2960999 -11- c) a scrolling of photos or pictograms relating to the general theme thus selected (Block Il), d) a selection the appropriate relative pictogram or pictogram by pressing the corresponding key (Block 12), 5 e) a storage in memory of the selection (Block 13), f) if there is additional information (Block 14), scrolling (Block 15), otherwise the method directly performs step k), g) a possible action to zoom or a mode of movement in case a detail is to be shown for additional information relating to the selected information ; h) a selection of the silent or vocalized mode of the additional detail (Block 16), i) a storage in memory of the selection (Block 17), j) if there is additional information (Block 18), a repetition of the 15 steps f) (Block 15) to h) (Block 17) otherwise, k) a possible display of the sentence thus constructed (Block 19); I) an eventual vocalization of the sentence (Block 20).

FIG. 3 represents for the purposes of the figure a pictogram 20 representing the preferentially personal photograph of a man 1. The picture is represented in full screen 2 of a display module. At this point the picture can only be shown and / or a voice tag. If the user wishes to make a detail speak like thumb 4 to signify that he is in pain for example, he can grasp the magnifying glass 3 to surround a circle with the thumb 4. This way of using the process is silent . In another mode, the photo can be labeled by geometric figures chosen from the vertical choice, the triangle 5 being for example chosen to delimit the foot in order to be content to show it and / or to associate a vocalization of the word foot . The device further comprises keys 7 for controlling a speech synthesis, a play key, a pause key and a stop key. In this case, the pressure on the key such as in choice 7 of the vocalization of the word foot can be completed by the display of the word foot in the information bubble 8. Finally, a simple magnification by The effect of the + 9 key also makes it possible to better define the detail or to exclude elements of the photo or pictogram. This display may be the only way to communicate without vocalisation but in writing mode. This is of great interest for the rehabilitation of patients who have lost the word, especially patients suffering from aphasia.

Of course, the invention is not limited to the examples described above.

By way of non-limiting example, the dimensions of the apparatus may be for example 10 centimeters in width to fit in a pocket. For persons suffering from paralysis and deficiency of the voice, the devices according to the invention may comprise larger screens, having for example a width of at least 30 centimeters, in order to allow these persons with disabilities to use rods. or tutors (or equivalent means) to enable them to press virtual keys having larger dimensions.

Claims (15)

CLAIMS1) Communication method without construction of phrases for dumb and / or deaf-mute, including non-verbal or vocalized communication by showing all or part of a photo or a pictogram, this method being implemented by a device comprising: input and / or selection means (1) comprising keys, - pointing means (2) - a processing module (3), - a database management module (3), - a module display device (4) comprising at least one screen - a memory module (5), - a storage module of at least one database (6), characterized in that it comprises the following steps: a ) a display of general themes corresponding to the categories of photos or pictograms; (9), b) an action on a key corresponding to a general theme (10), c) a scrolling of photos or pictograms relating to the general theme 20 thus selected (11), d) a selection of the appropriate relative photo or pictogram by pressing the corresponding key (12), e) a storage in memory of the selection (13), f) if there is a possible action to be performed for the selected photograph 25 (14), a choice of the tool adapted (15), otherwise the method goes directly to step j), g) a possible selection of the mode or additional detail (16), h) storage in memory of the selection (17), i) a repetition of the steps f) to h) until there is no further validation; j) a display of the detail or the pictogram or pictogram thus constructed on the screen and / or a vocalization of said message by means of a synthetic voice or a recording. 2960999 - 14 - 2) Method according to one of claims 1, characterized in that each step of selection and storage is followed by a display step and in that it comprises a possible preliminary step of delimiting by a material such as a stylus of stereotyped or specific forms for delimiting a detail and possibly associating a vocalized message with it, 3) Method according to one of claims 1 to 2, characterized in that a bank of photos or pictograms can be used or personal photos can be taken by means of a digital camera, allowing a classification of themes generals characterizing the families of photos including personal or pictograms corresponding to current situations, objects and people of the entourage, to actions, to categories of needs, to persons or objects, to categories of situations such as only stores, categories of places. 4) Method according to one of claims 1 to 3, characterized in that the nonverbal communication through the pointing device is preferably used rather than the speech synthesis that is not triggered automatically. 5) Method according to claim 4, characterized in that the possible means (2) for pointing or scanning and / or selection comprises pointing objects such as a stylus but also peripheral type to the computer type joystick or mouse or virtual type with directional arrows operated on the screen or finally by a systematic scanning system of the entire screen at a chosen pace. 30 6) Method according to claim 5, characterized in that a magnification function is used by operating a material such as a stylus logos to zoom 2960999 -15- on the photo to go to the detail or to exclude objects in the photo or pictogram; 7) Method according to claim 5, characterized in that in the case of an impossibility of using a material such as a stylus, a mode of automatic scanning for example horizontal or vertical geometric figures can move on the screen to detail to refine the object of communication by stopping scanning by a motion switch; 10 8) A method according to claim 5, characterized in that the method also allows the time of programming to define by a material such as a stylus the outlines of an element of the photo by means of a conventional geometric figure or unique 15 of polygonal shape closed by a specific mode multipoint delimitation and be able to possibly include a voice message or juxtaposed next to the identified detail such as a comic comment bubble; 9) A method according to claim 1 and 2, characterized in that the information corresponds to sounds recorded for example musical or emitted by a person by a microphone on the computer or mobile equipment. 10) Device for implementing the method of claim 1, characterized in that it comprises: - input and / or selection means (1) comprising keys, pointing means (2) a module of processing (3), - a database management module (3), - a display module (4) comprising at least one screen; - a memory module (5), - a storage module of at least one database (6), 2960999 -16- 11) Device according to claim 10, characterized in that it comprises: o the photo mode or pictogram full screen, o the mode pointing a detail of the photo or pictogram, 5 o a voice mode of the photo and / or detail of the picture or pictogram, o a silent full screen mode displaying the text or detail relating to the picture or pictogram, o a mode which circles a detail by means of a geometric shape, o a mode which allows a function retail magnification 12) Device according to one of the preceding claims, characterized in that the keys are virtual. 13) Device according to one of claims 10 to 12, characterized in that it further comprises a voice synthesis module or recording, control means of said voice synthesis and listening means of the sentence or registration. 14) Device according to one of claims 10 to 13, characterized in that it comprises wired or wireless data transmission means to a second display module with second input and / or selection means for sending the picture or the detail. 15) Device according to one of claims 10 to 14, characterized in that it is shaped to be used with a mobile phone or integrated in mobile phones called "smartphone". 15 20 25 30