WO2000079504A2

WO2000079504A2 - Portable sensing devices

Info

Publication number: WO2000079504A2
Application number: PCT/GB2000/002407
Authority: WO
Inventors: Alexander Woodhatch; Christos Papavassiliou; Marina Coriolano-Lykourezos; Yang Li; Qiang Li; Zhihong Yu
Original assignee: Alexander Woodhatch; Christos Papavassiliou; Coriolano Lykourezos Marina; Yang Li; Qiang Li; Zhihong Yu
Priority date: 1999-06-21
Filing date: 2000-06-21
Publication date: 2000-12-28
Also published as: WO2000079504A3; AU5549700A; GB9914465D0; EP1171859A2

Abstract

The portable sensing device comprises a sensing pen with a symbology reading head (1) that reads substantially invisible symbologies that correspond to units of text and pictures printed on the same media sheet or displayed on a screen. It has integrated function buttons (2, 3, 4, 5) that allow a user to determine the nature of the records that the computer software will retrieve from the database of languages or from the text specific database.

Description

PORTABLE SENSING DEVICES:

This invention relates to a light or magnetic sensing pen reader that reads substantially invisible bar codes or matrix or other symbologies with integrated function buttons and radio frequency attachment that is used to teach languages and other subjects.

Learning a foreign language is presently achieved through the use of books, tapes, CDs, teachers, TV and radio programmes, optical code / barcode language learning methods, and exposure to environments where the language to be learnt is spoken. Each of these methods is functional but also has practical disadvantages.

There are disadvantages inherent in each of the commonly used methods. When using the CD ROM alone, constant reading from computer screens is very uncomfortable, students have to sit in a particular location, and there are only limited contexts and vocabulary with which to interact. When using a text book alone, it only offers written guidance, it forces students to flick through pages to find grammar and vocabulary pages, and there are functional, but not always interesting, contexts from which to learn. Tapes require distracting fast-forward and rewind operations that are time consuming and there are generally functional, but not always interesting, contexts. Written and oral lessons from a teacher are another common means for learning languages. However, teachers have to meet the general needs of all students in the class, and they cannot meet all individual needs of their students. Teachers cannot know exactly all that which a student needs to focus on.

OS Patent No. 5,480,306 issued on Jan.2, 1996 to Chih-Yuan Liu, entitled 'Language Learning Apparatus and Method Utilizing Optical Code as Input Medium' discloses the use of optical code / bar code as the input medium for accessing pre-generated human voice sound data from memory locations that correspond to the text or visual media that a user interacts with when learning a foreign language. The main disadvantage of this invention is in the visibility of the optical code. If a page that a student is to learn from is covered with bar codes and optical codes, it is distracting and visually unappealing. Moreover, the code takes up space on the page that could be better used for educational purposes by adding extra text or pictures or blank space. Moreover, this invention only discloses sound data for educational feedback, limiting the value of the invention because it does not include text feedback or text in the memoiy locations.

In accordance with the present invention there is provided a portable sensing device comprising a sensor head capable of reading a first data set substantially invisible to a user, selecting means for enabling a user to select a desired function and means for generating a second data set in dependence on the first data set read by the sensor head and on the function selected by the user.

Preferred embodiments of the invention will now be described with reference to the accompanying drawings which illustrate schematically a portable sensing device in the form of a sensing pen in accordance with a preferred embodiment of the present invention. Hardware:

• an ultra-violet sensitive ink, or infra red sensitive ink, or magnetic ink, or screen bar code [matrix code] 100 symbology reading device [in a pen shape],

integrated into the stem of the pen shaped device: function-giving buttons bio-metric sensors

105 screen on-off switch microphone detachable cable link to a computer or other suitable device detachable radio frequency/infra red device to transmit the scanned symbology

110 radio frequency/infra red receivers attachable to a computer, the portable sensing- device perr and other suitable devices to receive the transmitted signals from the scanner ultra-violet sensitive ink, or infra red sensitive ink, or human-eye-visible ink, or screen bar code^* [matrix code] symbology reading internal electronics battery holding area

115 wrist-strap type battery holder and external power source head phones with integrated bio-metric sensors socket for insertion of head phone/microphone lead connector external power source power-intake socket loudspeakers

120

Software:

• software drivers to make the interaction between these hardware components possible 125 • a database with all parts of most spoken human languages cross-referenced and annotated

• an interface that allows a user to interact with the database records

• annotation workbenches for sound, text and music

• Internet based language-teaching software

• Network based:

130 • language teaching software

• interface software

• Bio-metric data analysis and response software

• Other software needed to enhance to functionality of the system

135 Texts:

• Texts with substantially invisible bar code or matrix or other symbologies printed in inks sensitive to> ultraviolet, infra-red. magnetic readers. Alternatively, the texts can be displayed on a screen.

140

The invention extends to the configuration of these elements, an example of which can be seen in the enclosed drawings, together with software, speakers / head phones, and texts to immediately gain access to any information about suitably prepared texts a learner can study in order to learn their own or a foreign language, or music, or other academic discipline.

145

150

155

160

165

170

175 Specific embodiments of the invention will now be described by way of example with reference to the accompanying drawings in which: -

180 Figure 1 shows in perspective the portable sensing device with: the symbology reading head connected to the internal electromcs for reading an ultra-violet sensitive ink, or infra red sensitive ink, or magnetic ink, or screen bar code [matrix code] symbology 1 function buttons that toggle for: grammar 2 185 pronunciation 3 translation 4 unit selection of text or picture (every word, phrase, sentence, paragraph, passage or picture) 5 a screen 6 a detachable radio frequency or infra-red transceiving attachment 7 190 the location of the battery hold 9 a location for a bio-metric sensor 10 an on-off switch 11 ear phone socket 12 a location for loud speakers 13 195 external power source power-intake socket 14 a location for a microphone 15

Figure 2 shows the portable sensing device with:

200 the symbology reading head connected to the internal electronics for reading an ultra-violet sensitive ink, or infra red sensitive ink. or magnetic ink, or screen-based bar code [matrix codel symbology 1 function buttons that toggle for: grammar 2 pronunciation 3 205 translation 4 unit selection of text or picture (every word, phrase, sentence, paragraph, passage or picture) 5 a screen 6 a detachable radio frequency or infra-red transceiving attachment 7 the location of the battery hold 9 210 a location for a bio-metric sensor 10 an on-off switch 11 ear phone socket 12 a location for loud speakers 13 external power source power-intake socket 1 215 a location for a microphone 15 Figure 3 shows the portable sensing device with:

220 the symbology reading head connected to the internal electronics for reading an ultra-violet sensitive ink, or infrared sensitive ink, or magnetic ink, or screen-based bar code [matrix code] symbology 1 function buttons that toggle for: grammar 2 pronunciation 3 225 translation 4 unit selection of text or picture (every word, phrase, sentence, paragraph, passage or picture) 5 a screen 6 a detachable cable attachment 8 the location of the battery hold 9 230 a location for a bio-metric sensor 10 an on-off switch 11 ear phone socket 12 a location for loud speakers 13 external power source power-intake socket 14 235 a location for a microphone 15

An alternative version of the portable sensing device is a reading pen as substantially described above bert without function buttons. Instead, a language learner uses a separate sheet of symbologies that offer a menu of options to the user. These symbologies would be sent along with scanned symbologies corresponding to the units of te^t and pictures in the same way as the currently selected set of function buttons options are. 240

Technical features of the portable sensing device:

The portable sensing device is comprised of the components as shown in the drawings, which show certain configurations of these components that fulfill the requirements needed to make the portable sensing device 245 function.

As the drawings show the portable sensing device is based around a symbology reading portable sensing device. The reading head and internal electronics are those used to read substantially invisible bar codes, matrix codes and other symbologies with either infra-red, ultra violet light, or magnetic ink.

250

These internal electronics are joined within the outer casing of the portable sensing device by the internal battery hold 9, and software drivers (not shown). If an internal battery hold 9 is not desirable for any reason, a wrist-strap type battery holder can be attached onto the user's wrist or other body part and hold batteries that are connected to the Light Pen's external power source power-intake socket 14.

255

Function buttons (2, 3, 4, and 5) are integrated into the outer casing. These can be toggled on and off (2, 3, and 4) or through the options for word, phrase, sentence, paragraph and passage (5). They interact with the internal electronics via software drivers that cause the internal electronics to transmit data specific to the current function button selections. In the portable sensing device example shown above, there are 25 different options made up of these

260 five options - grammar, pronunciation, translation, spoken grammar, and spoken translation - for each of these units of text - word, phrase, sentence, paragraph and passage. Other examples of functions are picture selection, translation into a particular language, grammar comparison, pronunciation analysis, communication with teacher, communication with classmates, setting time and date, language selection, select button tones, contact teacher, schedule a lesson, request feedback at later point in time, voice note recording, voice note playback, power level

265 monitoring, power charge requirement, volume playback, email reception, stop playback, start playback, scrolling images on the screen, adjusting the controls on using the electrode array sensors / radar sensing vocal fold movement detectors, analysis displays of pronunciation exercises, accessing log of previous swipes, brightness of screen, contrast of screen, screen backlight on and off, and any number of interrogations of the software via oral questioning.

270

This data is scanned with the symbology reading head 1, transmitted along with a scanned symbology so that the software on the computer that receives the transmitted scanned data produces the required records.

The current active function button selections are also shown on the screen 6 so that the portable sensing device user 275 can see instantly the options that are currently active. Therefore the software driver that causes the internal electronics to transmit additional data from the active function button selections also sends data corresponding to the current active function button selections to the screen for display.

An alternative means to select a function is to speak into the portable sensing device's microphone 15 where the 280 user's voice data will be interpreted by software to activate a function.

The transmission of the scanned data is achieved either with a detachable cable attachment 8 that links the portable sensing device to a computer, or via a detachable radio frequency or infra-red transmitting device 7 that transmits the data to a receiving device directly attached to a computer, or to a receiving device that forms part of a netwprk of 285 receivers located remotely from a server that serves the data requests of many portable sensing device users. The transmission means 7 will also offer access to the Internet and teaching and other communication facilities v a, for example, the Internet or a corporate / educational system's intranet.

There is also provided an "on-off switch" 11 that allows a user to turn the portable sensing device on and off. This 290 switch is connected to the battery hold and to the external power source power-intake socket 14.

Biometric sensors 10 that measure a user's pulse rate and other detectable physiological and biological conditions can also be integrated into the portable sensing device pen. Through the use of software drivers, a user's physiological metrics can be transmitted to the computer through the cable or radio frequency or infrared 295 attachment. The readings can be analyzed by user-calibrated software, and responded to by the software in order to maximize the learning efficacy of the recurring interaction between the user and the system made possible by the present invention. An example of this mechanism would be when a user's pulse rate is too last relative to his or her normal pulse rate indicating agitation or impatience, soothing music or comments can be generated by the software to bring about a more relaxed state in the user so as to enable him or her to learn with greater concentration. 300 Sensors attached to the headphones that measure biometric data from a user's head can complement the biometric sensors 10 in the portable sensing device itself.

A microphone (not shown) can be integrated into the stem of the portable sensing device so that a user can interact with the software drivers, databases and interfaces with his or her voice. An example of this is when a student 305 wishes to test his or her pronunciation accuracy they can speak directly into the microphone and receive processed feedback from the software. Another example is when a user wants to access other menu options offered by the interface software he or she can access the option with their voice.

Besides the user's voice-data and bio-metric data, the portable sensing device transmits via infra-red, radio J10 frequency or cable attachments the scanned symbologies together with the notification of the currently active function button selections to a computer which is running the portable sensing device system interface and database software. The interface software will receive this data, determine which records the user is seeking, search the database for these records, retrieve them, and display them on screen or send them for playing on external or integrated loud speakers 13 or on head-phones via cable or radio frequency transmission. 315

The interface software will also provide various menu options from the screen to allow a user to further investigate and learn language components.

As an alternative, for users that are remote from a screen and rely on sound for answers from the software after 320 interacting with printed material or sending voice-records via the portable sensing device, complete menu options that are found on the screen menus provided by the interface software can also be accessed by interacting with printed symbologies labeled to correspond to the respective screen menu items. These symbologies can be printed in an easy to reach location in the material the portable sensing device user is interacting with, e.g., inside the front and/or back covers, or on book marks that may be laminated. 325

Use of the portable sensing device:

The language learner may use the portable sensing device in the following way. The user:

330 1. holds the portable sensing device as they might hold a pen;

2. views a suitably prepared text on paper or on screen;

3. reads through the text until he or she wants to know something about the language in the text;

4. selects an option set from the available menu items by using the function buttons or oral instruction of the software through the microphone;

335 5. checks the option set by looking at the integrated screen 6;

6. scans the symbology relating to the text that is to be inquired about;

7. hears through headphones/external or integrated speakers 13, and / or reads from a screen, the retrieved records that were requested

8. optionally interacts with software on screen for further study of the language

340 9. optionally uses the in-built microphone or the microphone in the head set to interact with the software Advantages of the portable sensing device:

The portable sensing device's advantages over existing language learning methods are essentially that there is less 345 time staring at screens when learning, there is instant access to any information the user requires, there is little or no dictionary work required, grammar is easily referenced, and the system reduces resistance to study as a user knows in advance of studying that their mental energy will be expended on learning interesting subject matter rather than laborious referencing work. Thus, it allows a user to tackle any text with confidence and pleasure, and also allows user-specific training. 350

The present invention does not need to interact with optical code / barcodes as visible code is distracting and aesthetically less than optimal.

It affects the way a teacher teaches. The teacher's role becomes one of motivator and live practise partner/leader in 355 class room. The system saves a teacher repeating answers and allows a teacher to focus on more specialised points and give more examples where necessary. With faster progress, homework is made easier to do and yields greater results. The system equates to having your own personal tutor with you when reading whom you can ask any question and get an immediate reply. Teachers are saved from having to recall each individual pupil's specific needs as reports can be generated for teachers analysing areas on which to concentrate efforts based on a student's 360 symbology snipes.

A student may know generally what they need to study but they may not know what exactly they need to study, what out of the subject matter they should have learnt that they have actually forgotten. Likewise, a teacher may know generally what a particular student or a class need to know or work on but may not know on a day by day

365 basis exactly what their students have learnt, and what exactly they need to work on in class. Also, with the advent of distance learning through the Internet, an on-line teacher may not know the student they are teaching and it may take valuable online time and effort trying to establish what the most pertinent topics and problem areas that need to be covered during the lesson are. Moreover, where the service offered does not include regular interaction with a teacher, then this method of analysis will offer precise written feedback to students that could be, for example,

370 emailed to them.

The present invention therefore offers the following functionality. The software that the apparatus interacts with generates records of the user's swipes with associated function selection, pronunciation practise analysis, and other interaction made possible by the means described. Answers to test questions are recorded and logged. These are then

375 analysed in various ways, including by reference to the study course they are working on, their own voice and speech patterns and accents, and their previous knowledge, initial competence and proficiency in the subject they are studying, and what other educational materials they are working with. From this analysis, it is possible to generate tests and exercises that focus on an individual's specific needs, reports for teachers in classrooms on what the priority areas for their class are, and numerous other educationally beneficial reports and services.

Claims

CLAIMS:

1. A portable sensing device comprising a sensor head capable of reading a first data set which is substantially 385 invisible to the user, selecting means for enabling a said user to select a desired function and means for generating a second data set in dependence on the first data set read by the sensor head and on the function selected by the user.

2. A sensing device as claimed in claim I, arranged to read the first data set in the form of a magnetically 390 encoded pattern.

3. A sensing device as claimed in claim 1, arranged to read the first data set in the form of a pattern which affects incident electromagnetic radiation.

395 4. A sensing device as claimed in claim 3, further comprising a source of said electromagnetic radiation.

5. A sensing device as claimed in claim 4, wherein said electromagnetic radiation is in the ultraviolet range.

6. A sensing device as claimed in claim 5, wherein the selecting means comprises plurality of user-operable 400 switches.

7. A sensing device as claimed in claim 6, wherein the plurality of user-operable switches are arranged to be operated by the user's fingers.

405 8. A sensing device as claimed in claim 1, wherein the selecting means comprises means for reading a representation of the desired function in accordance with the selective positioning of the sensor head by the user in relation to a plurality of such representations.

9. A sensing device as claimed in claim 8, wherein the means for reading the representation comprises the 410 sensor head.

10. A sensing device as claimed in claim 1, wherein the selecting means comprises means for receiving an audio input from the user.

415 11. A sensing device as claimed in claim 10, wherein the means for receiving an audio input comprises a microphone.

12. A sensing device as claimed in any preceding claim, wherein the sensor head is arranged to read visible data printed on a page or displayed on a screen. 420

13. A sensing device as claimed in any preceding claim, wherein the sensor head is arranged to read invisible data printed on a page or displayed invisibly on a screen.

14. A sensing device as claimed in claim 13, wherein the sensor head is arranged to read invisible data in the 425 form of infrared radiation reflected from the page surface.

15. A sensing device as claimed in claim 13 or claim 14, wherein the sensor head is arranged to read invisible data in the form of ultraviolet radiation reflected from the page surface.

430 16. A sensing device as claimed in claim 13 , wherein the sensor head is arranged to read visible data in the form of infrared radiation reflected from the page surface.

17. A sensing device as claimed in any preceding claim, wherein the sensor head is arranged to read data in the form of magnetic ink on a page surface.

435

18. A sensing device as claimed in any preceding claim, wherein the sensor head is arranged to read data in the form of a barcode.

19. A sensing device as claimed in any preceding claim, wherein the sensor head is arranged to read data in the 440 form of a matrix code.

20. A sensing device as claimed in any preceding claim, substantially in the shape of a pen.

21. A sensing device as claimed in any preceding claim, further comprising means for indicating to the user 445 which function has been selected.

22. A sensing device as claimed in claim 21, wherein the indicating means comprises a visual display.

23. A sensing device as claimed in claim 21 or 22, wherein the indicating means comprises an audio 450 transducer.

24. A sensing device as claimed in any one of the claims 21 to 23 , wherein the indicating means is arranged to provide an audio output for connection to an audio transducer.

455 25. A sensing device as claimed in any preceding claim, further comprising means for transmitting the second data set to a remote location.

26. A sensing device as claimed in claim 25, further comprising means for receiving a third data set from the remote location. 460

27. A sensing device as claimed in claim 26, wherein the means for receiving a third data set is arranged to receive a said third data set generated in response to the transmission of the second data set.

465

28. A sensing device as claimed in any preceding claim, further comprising one or more bio-sensors for measuring physiological characteristics of the user.

29. A sensing device as claimed in claim 28, when dependent on claim 27, wherein the transmitting means is 470 arranged for transmitting additionally data derived from one or more bio-sensors.

30. A computer-based system comprising a memory having data stored therein and a sensing device as claimed in any preceding claim, further comprising means for selectively accessing the stored data in accordance with the second data set.

475

31. A system as claimed in claim 30, wherein the data are stored within the memory in predefined regions such that each region may be addressed by a corresponding function selected by a user.

32. A system as claimed in claim 30 or claim 31 , further comprising output means for outputting the selected 480 data stored in the memory.

33. A system as claimed in claim 32, wherein the output mean is arranged to output the selected data in the form of an audible signal.

485 34. A system as claimed in claim 33, wherein the stored data represent elements of speech and the output means is arranged to output the selected data in the form of audible speech.

35. A system as claimed in claim 33, wherein the stored data represent elements of speech and the output means is arranged to output the selected data in the form of music.

490

36. A system as claimed in any one of claims 30 to claim 34, wherein the output means is arranged to output the selected data in the form of a visual display.

37. A system as claimed in claim 32, wherein the stored data represent words and output means is arranged to 495 output the selected data in the form of legible words.

38. A system as claimed in claim 32, wherein the stored data represent words and output means is arranged to output the selected data in the form of legible music notation.

500 39. A system as claimed in claim 33 or claim 34, wherein the stored data represent respectively elements of speech or words in a plurality of languages and the output means is arranged to output data from the memory in a selected language in accordance with the function selected by the user. 505

40. A system as claimed in any one of claims 30 to 39, when dependent on claim 29, further comprising means for providing an audible output when a physiological characteristic measured by the one or more biosensors meets a predetermined criterion.

510 41. A system as claimed in claim 40, wherein the portable sensing device comprises the means for providing an audible output.

42. A system as claimed in claim 40 or claim 41, wherein the audible output comprises music.

515 43. A system as claimed in claim 1, that interacts with software that stores distinct relationships between plurality components and elements of plurality human languages.

44. A system as claimed in claim 43, where the software stores plurality annotations relating to the subject material the user interacts with.

520

45. A system as claimed in claim 30, wherein the data are stored within the memory in predefined regions but where the contents of the memoiy are reconfigured when copied out of these memory locations to generate the required response for the user.

525

530

535

540