WO2008092183A1 - Organisational structure and data handling system for cochlear implant recipients - Google Patents
Organisational structure and data handling system for cochlear implant recipients Download PDFInfo
- Publication number
- WO2008092183A1 WO2008092183A1 PCT/AU2007/000110 AU2007000110W WO2008092183A1 WO 2008092183 A1 WO2008092183 A1 WO 2008092183A1 AU 2007000110 W AU2007000110 W AU 2007000110W WO 2008092183 A1 WO2008092183 A1 WO 2008092183A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- audio signal
- digitized
- source audio
- cochlear implant
- recipient
- Prior art date
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N1/00—Electrotherapy; Circuits therefor
- A61N1/18—Applying electric currents by contact electrodes
- A61N1/32—Applying electric currents by contact electrodes alternating or intermittent currents
- A61N1/36—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation
- A61N1/36036—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation of the outer, middle or inner ear
- A61N1/36038—Cochlear stimulation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
- H04R25/505—Customised settings for obtaining desired overall acoustical characteristics using digital signal processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
- H04R25/554—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired using a wireless connection, e.g. between microphone and amplifier or using Tcoils
Definitions
- This invention relates to a system and apparatus for processing and distributing signals of a form suitable for cochlear implant devices.
- Implanted hearing prostheses that stimulate the inner ear can be used to assist people with total or partial hearing loss.
- such devices employ electronic analogue and digital techniques to process, and transform received sound or signals representing sound, into one or more discrete channels of mechanical or electrical stimulus information. The resulting stimulus is then conveyed to neural receptors within the recipient's cochlea associated with the sense of hearing.
- persons (hearing prosthesis recipients) with severe to profound hearing loss are able to perceive a potentially beneficial approximation of sound.
- a signal processing system for supplying signals of a form suitable for at least one cochlear implant system including an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal includes: a digital signal processor receiving said digitized source audio signal, the digital signal processor adapted to modify the digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the digitized source audio signal including at least spectral and temporal content suitable for use with the predetermined implanted cochlear device.
- the signal processing system further includes; a digitized audio signal distribution processor operatively connected to the digital signal processor for making said output form of the source audio signal, available to at least one cochlear implant system.
- the digital signal processor is configured to provide an output form of the source audio signal using a mapping for a predetermined implanted cochlear device and further wherein the cochlear implant system and includes a body worn or over the ear digital signal processing device adapted to receive said output form of the source audio signal and transform the output form to a signal suitable for the predetermined implanted cochlear device.
- a method for supplying signals of a form suitable for at least one cochlear implant system including an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal, the method steps included: digital signal processing the digitized source audio signal to modify the digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the digitized source audio signal including at least spectral and temporal content suitable for use with the predetermined implanted cochlear device.
- the method for supplying signals according to claim B further includes the step of operatively connecting said digital signal processor to a digitized audio signal distribution processor for making said a predetermined output form of the source audio signal available to a predetermined cochlear implant device.
- the method for supplying signals includes the step of configuring the digital signal processor to provide an output form of the source audio signal using a mapping for a predetermined implanted cochlear device wherein the predetermined cochlear implant system includes a body worn or over the ear digital signal processing device adapted to receive said output form of the source audio signal and transform the output form to a signal suitable for the predetermined implanted cochlear device.
- the method includes the step of storing and selectively outputting one or more output forms of respective source audio signals in a memory device in the body worn or over the ear signal processing device.
- the method for supplying signals further includes the step of applying at least one digital filter in the digital signal processor to the digitized source audio signal. Further to the previous method step for supplying signals wherein the digital filter modifies the frequency content of the source audio signal during the process of modifying the digitized audio signal according to the predetermined setting.
- the method for supplying signals wherein the source audio signal includes cochlear implant specific auditory habilitation audio signals includes cochlear implant specific auditory habilitation audio signals.
- a modified audio signal distribution system for signals suitable for at least one cochlear implant system includes an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal, the system including: a first networked computer having multiple stored digitized source audio signals accessible from the network; a second networked computer having access to the first networked computer and adapted to process one or more of the multiple stored digitized source audio signals to include at least spectral and temporal content suitable for a predetermined cochlear implant system including a predetermined implanted cochlear device; and a third networked computer having access to at least the second networked computer and associated with a predetermined cochlear implant system, for receiving the one or more multiple stored digitized source audio signals including at least spectral and temporal content suitable for the predetermined cochlear implant system.
- the modified audio signal distribution system further includes: a fourth networked computer adapted to authorise a financial transaction between at least the user of the second networked computer and one or more other parties.
- the modified audio signal distribution system further includes: a fifth networked computer associated with a predetermined cochlear implant system for receiving and storing the one or more multiple stored digitized source audio signals including at least spectral and temporal content suitable for the predetermined cochlear implant system before providing the one or more multiple stored digitized source audio signals to the predetermined cochlear implant system.
- a body worn or over the ear digital signal processing device for use with an implanted cochlear device for receiving an output form of a digitized source audio signal generated by a digital signal processor that modifies a digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the source audio signal including at least spectral and temporal content suitable for use with an implanted cochlear device
- said digital signal processor is physically remote the user of a cochlear implant system
- the body worn or over the ear digital signal processing device includes: a receiver adapted to receive the output form of the digitized source audio signal, and a digital signal processor to transform the output form to a signal suitable for a predetermined cochlear implant device implanted in the user.
- the body worn or over the ear digital signal processing device a further includes: a digital store for storing at least one received output form of a digitized source audio signal and for making a stored received output form available to the digital signal processor.
- the body worn or over the ear digital signal processing device includes: a radio signal generator and modulator for forming a radio signal modulated according to a protocol suitable for communicating the output form of a digitized source audio signal to the predetermined implanted cochlear device, and a radio signal transmitter for transmitting the modulated radio signal to the predetermined implanted cochlear device.
- the body worn or over the ear digital signal processing device has a device address addressable by a networked computer.
- the address is an address that conforms to Internet Protocol addressable by a networked computer.
- Fig. IA depicts 4 various levels of processing that can be applied to an external aural source and an illustrative result for various hearing recipients;
- Fig. IB depicts frequency verses amplitude graphs of the aural source and the manner in which the various frequencies are handled by one or other processes intended to provide input to a cochlear device
- Fig. 1C depicts a flow diagram of steps for supplying and processing an audio source obtained on-line in a manner suitable for a cochlear implant recipient
- Fig. 2 depicts a block diagram of a system configuration for supplying and processing an audio source obtained on-line in such a way that it is suitable for a cochlear implant recipient;
- Fig. 3 depicts a block diagram of an alternative arrangement of the system depicted in Fig. 2 wherein the recipient uses a body worn device to deliver the processed audio to their cochlear implant;
- Fig. 4 depicts a block diagram of a computer device for processing an audio source obtained on-line in such a way that it is suitable for a cochlear implant recipient
- Fig.5 depicts a block diagram of an alternative arrangement of the system depicted in Fig.4 wherein the recipient uses a body worn device to deliver the processed audio to their cochlear implant;
- Fig. 6 depicts a block diagram of a computer device for processing an audio source on a storage medium in such a way that it is suitable for a cochlear implant recipient;
- Fig. 7 depicts a block diagram of a prior art cochlear implant system
- Fig.8 depicts a block diagram of system element for supplying and processing an audio source obtained on-line in such a way that it is suitable for a cochlear implant recipient.
- the following description provides illustrative details of an organisational structure, system, method and various apparatus for supplying and processing an audio source in such a way (using predetermined settings) that the audio source is suitable for a predetermined cochlear implant recipient.
- Fig IA there is shown a series of different scenarios which will be discussed to illustrate the environment of the disclosure provided by this specification and how it may be beneficial to cochlea implant recipients.
- the uppermost illustration depicts the situation where a recipient of an aural stimulation (an external aural condition) who is unaffected by any significant aural impediment such as cochlea damage or disease, thus the recipient is capable of receiving and appreciating the full or the majority of the rich spectral content of the aural content that is part of the aural stimulation that may for example contain a combination of speech (singing) and musical content.
- an aural stimulation an external aural condition
- any significant aural impediment such as cochlea damage or disease
- the illustration immediately below the uppermost illustration depicts the situation where the recipient of an aural stimulation (an external aural condition) who is affected by a disability such as cochlea damage or disease, and thus the recipient is incapable of properly processing the aural content and is unlikely to be pleased with the aural experience (although it is accepted that hearing something instead of nothing may be considered an improvement).
- aural stimulation an external aural condition
- a disability such as cochlea damage or disease
- the illustration second below the uppermost illustration depicts the situation where the recipient of an aural stimulation (an external aural condition) is affected by a disability such as cochlea damage or disease, but is aurally compensated by the use of prosthesis (such as for example a cochlea implant), in which case the recipient is then capable of processing a modified version of the aural stimulation and much better off than the recipient depicted in the illustration immediately below the uppermost illustration.
- aural stimulation an external aural condition
- prosthesis such as for example a cochlea implant
- the illustration third below the upper most illustration depicts the situation where the recipient of an aural stimulation (an external aural condition) is affected by a disability such as cochlea damage or disease, but is aurally compensated by the use of a conditioning and modifying agent that knows the particular characteristics of the recipient's prosthesis (such as for example a cochlea implant), in this case the recipient is capable of processing a modified version of the aural stimulation and very much better off than the recipient depicted in the illustration immediately below the uppermost illustration.
- a conditioning and modifying agent that knows the particular characteristics of the recipient's prosthesis (such as for example a cochlea implant), in this case the recipient is capable of processing a modified version of the aural stimulation and very much better off than the recipient depicted in the illustration immediately below the uppermost illustration.
- the illustration fourth below the uppermost illustration depicts the situation where the recipient of an aural stimulation (an external condition) is affected by a disability such as cochlea damage or disease, but is aurally compensated by the use of a conditioning and modifying agent that knows the particular characteristics of the recipient's prosthesis (such as for example a cochlea implant that is reporting the characteristics independently of the recipient and the conditioned and modifying agent provides the relevant signal direct to the recipient), in this case the recipient is capable of processing a modified version of the aural stimulation and very much better off than the recipient depicted in the illustration immediately below the uppermost illustration.
- a conditioning and modifying agent that knows the particular characteristics of the recipient's prosthesis (such as for example a cochlea implant that is reporting the characteristics independently of the recipient and the conditioned and modifying agent provides the relevant signal direct to the recipient), in this case the recipient is capable of processing a modified version of the aural stimulation and very much better off than the recipient depicted in the illustration immediately below the uppermost illustration.
- the specification describes the various organisational structures, systems, methods and apparatus for supplying and processing an audio source in such a way that the audio source is suitable for a predetermined cochlear implant recipient.
- the structure and method augments and/ or replaces the limited processing capability of an implanted and/ or body- worn component of a hearing impaired recipient's hearing prosthesis, with one or more data processing resources, preferably located remote of the recipient.
- the source audio may include, but is not limited to: diagnostic, therapeutic, habilitation, educational and entertainment material.
- the enhanced data processing capabilities can be further enhanced if implemented through the conveyance and exchange of one or more parameters describing the recipient, their prosthetic hearing attributes, requirements and preferences, as well as the near real time capability and availability of computing resources and processing required of the data or the ability for dedicated computing resources and processing to perform tasks upon demand and within a short time deliver a desired product to a requesting recipient that is suitable for that recipient and that recipient alone.
- a system for supplying and processing an audio source in such a way that it is suitable for a cochlear implant recipient will be described that includes computer networks, most notably the world wide computer network referred to as the Internet.
- Computer networks can be used to connect cochlear implant recipients or their carers to computer servers that can supply a multitude of digital audio source material from multiple sources and then process chosen digital audio source material for a particular cochlear implant recipient.
- the transaction can be automated and the financial aspect of the transaction can be handled using known Internet based services.
- the processed audio data can be listened to at the time or can be stored for later use.
- the processing required to customize the source audio can be shared or distributed amongst various computer equipment with in the network. It is anticipated that the bulk of processing will be done by computers associated with the supplier of the service as the processing power and efficiencies will be more readily achievable when done by a bulk supplier and processor. However, as computing power increases it is conceivable that some if not all the necessary processing can be done on a computer associated directly with the cochlear implant recipient. Alternatively, the supplier of such a service can do partial processing and the remainder, being a large or small proportion, is done by another computer associated with the cochlear implant recipient or even by a device worn by the cochlear implant recipient.
- An example arrangement of a body worn computer device includes a wireless linked, portable electronic media player containing adapted hardware and software that uploads to other computers, one or more of a recipient's ID, customisation parameters, prosthesis type and capability, password, banking details and financial merit transfer authorisation from one or a number of various parts of the recipient's prosthesis to a recipient's wireless enabled personal computer (PC) where it is then conveyed via any convenient means, including the Internet, to a computing resource associated with a third party or cochlear company sponsored website server able to receive and decrypt this data to allow the recipient to browse and select for purchase, audio material that is then processed and customised for the particular recipient by taking account of the recipient's preferences and cochlear implant characteristics.
- PC personal computer
- Processed data which may be partially processed for a number of reasons, in that it is more expeditious to communicate a reduced data set or that different degrees of further processing is needed by a recipient, is then conveyed via a communications medium such as for example, the Internet back to the recipient, for example in the case of the Internet the recipient's personal computer or digital data device such as a data capable wireless personal assistant or mobile cellular telephone that has a wired or wireless connection to the cochlear media player of the recipient, where further processing and refinement may be undertaken before this customised audio material is conveyed to their personal cochlear implant device where it is converted into stimulus current data, for inductive transmission to the recipient's implanted cochlea stimulator electrodes.
- a communications medium such as for example, the Internet back to the recipient, for example in the case of the Internet the recipient's personal computer or digital data device such as a data capable wireless personal assistant or mobile cellular telephone that has a wired or wireless connection to the cochlear media player of the recipient, where further processing and refine
- an implant recipient is provided access to a website allowing them to browse select and purchase pre-recorded or live broadcast for example audio material, which prior to being downloaded, via the Internet, to their special personal media player or sound processor, is intensively processed and customised for delivery to the recipient's implanted cochlea stimulator electrodes.
- Graph 300 in figure IB discloses a selective representation of speech components in which the vertical axis 301 (which is the same for all four graphs depicted in figure IB) refers to relative amplitude and the horizontal axis 302 (which is the same for all four graphs depicted in figure IB) refers to frequency.
- the vocal sounds 303 and 304 are generated, as are their harmonics 305.
- the spectral content of chaotic musical sounds such as cymbals and brush drums, include many frequencies and their harmonics 401 that are close in frequency and often higher in amplitude than those associated with the vocal sounds 303 and 304 and the important harmonics of the voice of the accompanying vocalist as depicted in graph 300 of figure IB.
- graph 500 of Fig IB there is disclosed a corresponding spectral range allotted to each one of 22 tonotopically-arranged electrodes of a cochlear device. Note that speech sound harmonics important for vocal comprehension are missing from the part of the spectrum indicated at reference numeral 504. Hence, graph 500 shows the limited spectral delivery of 5 channels of neural stimulation that would be created by a cochlear implant system in response to the combined speech and cymbal sounds disclosed in the graphs 300 and 400 immediately above.
- the spectral content of chaotic musical sounds such as cymbals and brush drums such as those depicted in Figure 400, include many emissions (401) that are close in frequency and often higher in amplitude than those associated with the vocal sounds of an accompanying singer.
- Figure 500 plots the amplitude of the stimulating current that would be applied to each one of the 22 spatially dispersed neural stimulating electrodes of a typical cochlear implant, not incorporating the principles described in this specification, in response to a combination of vocal and cymbal sounds as depicted in figures 300 and 400.
- the horizontal axis 502 of figure 500 represents the range of sound frequencies that are assigned by the prosthesis to stimulate each spatially disposed electrode located within the user's cochlea.
- the vertical axis 501 of figure 500 represents the relative amplitude of the stimulating current that is subsequently delivered to each of these electrodes.
- the vocal harmonic emissions vital for speech recognition as depicted by 305 in figure 300 is clearly absent in neural stimulation delivered by a cochlea implant without this invention. Instead of delivering the electrical stimulation necessary to evoke a perception of sound in the vocal harmonic frequency range depicted by 504 in figure 500, stimulation representing the more dominant cymbal sounds shown by 505 are delivered. Speech perception by a user is difficult if not impossible in circumstances where a small limited number of electrodes are available to simultaneously convey complex sound information.
- Data processing software using Fast Fourier Transform algorithms or similar methods are used within the software incorporated into any one or a number of devices (104, 107, 110, 115 and 119) as shown in figure 1C.
- Detailed statistical analysis of the high resolution spectral information extracted from incoming sound data is undertaken so that only those spectral peaks 305 that are an exact integer multiple of any lower frequency peak within the fundamental frequency range of normal human speech (303 and 304), are conveyed as stimulation to the user. In this way, the randomly dispersed, higher amplitude spectral components of complex sound unrelated to speech are ignored.
- Graph 600 in figure IB illustrates the electrical stimulation to an implanted cochlea simulation device and even the associated external processing and power delivery device that would result from the use of the suggested processing when supplied the sounds represented by graphs 300 and 400 in figure IB.
- the all-important harmonics of speech are represented by electrical stimulation in the cochlea as shown by the stimulation currents shown as 601 in graph 600 of figure IB.
- the voice component of the combined voice and musical performance may not be not as important to the cochlea user as the musical component, so the suggested processing can be arranged to provide a result that best suits the representation of the musical content and excludes the voice content.
- Implementation of the system, and devices can then be made to suit the individual user rather than being (so called "factory set") to suit the majority of cochlea implant users.
- a personal desktop, portable or handheld computer incorporating commercially available, speech to text conversion software converts sound data containing the lyrics of a musically simple song into text.
- the principals employed by the spelling and grammar-checking utilities incorporated in common word processing applications are used to automatically correct interpretation errors in a manner that is relevant to the context of the incoming text.
- the resulting text is then processed by commercially available, text to speech software.
- the resulting reconstructed speech signal can be adjusted in tonal quality and pitch to suit a predetermined user, before being conveyed to the hearing-impaired predetermined prosthesis user with none of the confusing background music that accompanied the original speech.
- the intense processing and customisation referred to above takes account of recipient preferences, perceptive attributes and stimulation and processing strategy MAP settings, to refine the audio content for the recipient.
- An implantable hearing prosthesis and a supporting environment whereby electronic data is used to convey one or more recipient-attributes to one or a plurality of data processing resources external to their implant and body worn sound or speech processor, where such attributes might be applied to process sound data in a manner that benefits a recipient's listening experience.
- These recipient-attributes can include, but is not limited to, the following: attributes that uniquely identify the recipient;
- attributes that can identify and or describe the type, model, revision, repair, modification, upgrade and replacement or renewal status, technical specification and history of certain parts of their cochlear implant system and associated accessories, for example, the implanted stimulator and electrode system, the external processor, data transmitting coil, associated cable system, power system and battery or batteries;
- electrical characteristics of the neural stimulus which may include, but is not limited to; the charge, current, voltage, pulse timing and morphology that is necessary to be applied to each electrode in order to evoke a perception of sound that is; the lowest perceivable by the recipient, the highest that is comfortable for the recipient;
- attributes which describe the sound data processing strategies, like CIS, ACE, etc. as preferred by the recipient and or their healthcare professional. A recipient may persist in using one strategy, while their health care professional advises another for longer-term benefits;
- characteristics of the recipient's evoked sound perception and comfort as it may relate to the apparent pitch and loudness when two or more electrodes are stimulated in the various combinations that are available, sequentially and simultaneously;
- characteristics that describe a recipient's ability to distinguish the pitch or frequency of one or more sounds in silence and in a combination with a wide variety of conflicting background and foreground sounds characteristics that describe a recipient's dynamic, static and temporal preferences for the manner in which sounds that change in amplitude are translated into stimulation.
- Some recipients for example are more sensitive to stimulation after periods of sleep with their prosthesis inactive. Such recipients may therefore prefer a slow onset and amplitude limiting for loud sudden sounds during early morning that they may not wish for at other times;
- resources may include various features of their hearing prosthesis, its accessories and supporting environment, the recipient's carer, guardian, clinic, audiologist, health care professional as well as remotely accessible information in the form of audio and audio plus visual material for the delivery of therapy, education and entertainment.
- the data processing resources referred to may include, but are not limited to wire or wireless linked resources with data processing and storage capabilities that are less constrained in size, weight and available power, than those incorporated in the implanted and body worn parts of the recipient's cochlear implant devices.
- Such resources may include, but are not limited to; a handheld media player, mobile cell phone or PDA with accessible computing resources, a nearby personal computer, or a remotely located computing resource accessible via the Internet.
- FIG. 1C graphically shows the flow, processing and exchange of sound, user and prosthesis data associated with one embodiment of the invention.
- Incoming aural sounds 101 of a musical performance are captured, edited and duplicated for commercial distribution and sale 102 in the form of digital sound data files via the internet or via analogue or digital storage media such as compact disks 103.
- This server 104 automatically arranges the purchase and conveyance of the data representing the user's choice of audio material 103 to the user via one or more wired or wireless linked data processing and conveyance resources (107 and 110).
- These (107 and 110) resources could include, web enabled PCs, PDAs and media players. Data transactions related to the purchase of goods and transfer of financial merit have been omitted from diagram 100 in order to ensure clarity for the reader although these mechanisms are known to those skilled in the relevant art.
- Data related to a users hearing needs and sound media requests as represented by the stripped arrows (120,118,112, 109, 106) as well the user selected sound data 103 and derivatives as represented by arrows (103, 105, 108, 111, 117 and 121) includes additional data to control the processing and conveyance of sound data and the manner in which these tasks are automatically assigned to devices (104, 107, 110, 115 and 119) for the benefit of the prosthesis user and aural data recipient.
- Tables 1 and 2 describe an example of the content and format of data packets used by one embodiment of the invention.
- a gradual lightening in the solid colour shading applied to the data flow arrows (103, 105, 108, 111, 117 and 121) is used here to symbolise the progressive refinement or customisation of the sound data during its path to the user. Whilst this shading may appear to imply a sharing of processing tasks, in reality sound data processing tasks are automatically assigned and modified in response to the constantly changing status of all the devices involved in the data path.
- Fig. 1C also depicts a flow diagram of steps for supplying and processing an original audio source in a manner suitable for a cochlear implant recipient.
- An original sound source or audio plus associated video is recorded and prepared for distribution in a multitude of forms, including but not limited to, pre-recorded music in formats playable on a variety of devices, such as for example, Compact Disc players, digital audio players having decoders of MP3, WAV, etc. formats of audio.
- the distribution of audio particularly music is distributed in many ways and increasingly being made available for downloading in digital format, for example only, the music download service provided by Apple Computer known as iTunes supplies over the Internet a multitude of music audio having a proprietary music format playable only on a proprietary hardware device known as an iPod, both iTunes and iPod being trade marks of Apple Computer Inc.
- iTunes supplies over the Internet a multitude of music audio having a proprietary music format playable only on a proprietary hardware device known as an iPod, both iTunes and iPod being trade marks of Apple Computer Inc.
- This by way of example is one form of a media player suited to the needs of non-hearing impaired individuals.
- Audio sources may include radio station broadcasts and more likely "podcasts" of off-air recordings, digital audio sources tailored for replay on an iPod, audio of live and pre-recorded television programs, book readings that are increasingly being made available to the sight impaired, and other source material too numerous to list.
- the system proposed differs in that the original audio is selectively and uniquely converted into a form that will enhance the listening experience of a cochlear implant recipient.
- the recipient can select the source of audio source from the vast array of available sources from one or more multimedia market places 104 that are available on-line via the Internet.
- the processing of the source audio can take place as described previously in a service provider's computer server, the recipient's own computer or as is more preferable partially in the service provider's computer and partially in the computer or computers associated with the cochlear implant recipient and more preferably predominantly in the service provider's computer server.
- Table 1 illustrates a data structure used in one example of the invention.
- a user's authorization for transfer of purchase requests personal information like credit card details plus authorisation to transfer financial merit.
- Interactive or other information such as media product advertisements to be conveyed to the user.
- Primary Content Descriptor Format, compression status and processing requirements.
- the cochlear implant recipient or their carer with the assistance of the various computer devices most likely their personal computer provides cochlear implant recipient details, purchase and transaction authorising details to the service provider's computer server and those same devices associated with the cochlear implant recipient receives one or more audio sources that are processed and that need to then be stored and provided to the cochlear implant system.
- cochlear implant recipient details, purchase and transaction authorising details to the service provider's computer server and those same devices associated with the cochlear implant recipient receives one or more audio sources that are processed and that need to then be stored and provided to the cochlear implant system.
- a wireless link is made between the personal computer and/ or other device that receives the processed audio source to the cochlear implant recipient's body worn or over the ear device.
- These devices perform the further processing as required and transform the data into a modulated radio frequency transmission designed to transmit the data to an embedded radio frequency receiver that along with any yet further decoding and processing is used to stimulate the cochlea of the cochlear implant recipient.
- the embedded portion of the cochlear implant system is known technology and specific details are not required however for completeness an extremely brief description is provided.
- the radio frequency signal is demodulated and at the same time the radio frequency signal is also converted to direct current to power the embedded device.
- the demodulated signal is also demultiplexed to create several discrete channels of electrical stimulation parameters.
- the electrodes that were carefully implanted in the cochlea are stimulated with one or more combinations of signals to provide a sense of hearing to the cochlear implant recipient.
- a hearing prosthesis recipient 210 operates a special electronic media storage and player device 212, referred to herein as a "media player".
- the media player 212 is configured to convey electronic data 214 to and from the recipient's cochlear implant system which in this embodiment is wireless enabled, using a wireless card 16 or such like to convey electronic data 214' to and from the media player as well as the computer device such as a personal computer 218, which may in particular be a notebook PC, wherein the computer device has the ability to communicate with other computers, and in a preferred arrangement having access to the world wide network of computers commonly referred to as the Internet 220.
- the recipient operates the PC 218 to access a dedicated Internet website 222 which is run by a supplier of a service to process a chosen digital audio source.
- the recipient browses the supplier's site 222 and chooses to view a list of audio material that is available for sale.
- the audio material may in one example be made available for distribution via the Internet by the provider of the Internet web site 222 that works in conjunction with a supplier of such material that typically also provides a site 226 containing the selection of audio material.
- the recipient or their carer can be provided a wide array of items to select from which can include by way of example only a music track, a speech only track, an audio version of a music video or any audio only version of a video product or a complete version of a video product.
- the family of the recipient may want to watch and listen to that particular feature video and the service provider can supply a version of the audio processed specifically for the listening of a cochlear implant recipient and arrange in the delivered product to have the two versions of audio play in synchronism with each other so that the whole family can enjoy the video together.
- the recipient or their carer selects a preferred item and payment can be made from an account that provides a credit facility or from a pre-paid account. If credit is being used then the user is required to authorize an appropriate credit card payment using Internet based transaction tools and services well know to those skilled in the art and which are not detailed herein.
- This type of transaction may or may not involve the recipient's own nominated financial resource 224 that will in most cases be used to authenticate the transaction that has been initiated by the recipient.
- the computer hosting the dedicated website 222 conveys a JAVA script applet to the recipient's computer device such as a notebook personal computer 218, which then executes that applet. In most case this occurs automatically and since the source of the applet is can be made to be a trusted source of such downloaded executable software this will likely happen without specific intervention or instruction from the personal computer user. However, appropriate instructions for downloading JAVA tools are provided if these are not available on the recipient's computer device.
- the applet can be configured to conduct a search for all the hardware interfaced or connected in some way to that computer device 218 and in particular for the presence of the dedicated media player 212 and confirmation that it is in communications wirelessly or otherwise with the recipient's prosthesis.
- the media player 212 interrogates the recipient's cochlear implant or parts there of, in such a manner as to allow the cochlear implant to convey one or more of the following information that is available from with in the part or parts being used by it's recipient, such as the recipient's ID, the serial number of one or more of the parts of the cochlear implant, their type and other characteristics pertaining to the recipient.
- This data is then uploaded via the Internet to the computer hosting the dedicated website 222.
- the encryption can be invoked and equally many ways in which the veracity of the data can be ensured.
- the host computer 222 processes and customises the purchased audio material for the recipient with the use of the uploaded and decrypted recipient and prosthesis data.
- the purchased audio material may have been sourced from the data stores of the host computer or obtained from other sources, including a third party music vendor 226, or other third party source of audio such as a radio station, broadcasting network, and Internet site that may be providing visual content to the recipient using their personal computer and other such sources.
- cookies data files
- the use of cookies, or like techniques allows the service provider behind the web site to provide users a more useful interaction with that site and the services it provides. It allows for the automation of a number of potentially repetitious steps involved in an interaction with a web site and provides marketing information on the market as a whole, in most cases in the later case, without identifying individual users.
- a local or remote computer system to the host computer is configured to process and download data representing sound using one or a plurality of specific attributes that pertain to an implantable hearing prosthesis, their guardian, carer or health care professional.
- Various data including one or more of, the processed data, a description of the data content, the degree to which the data has been processed and the ID of the recipient's prosthesis, is downloaded to the recipient's notebook PC.
- the processing and customisation of the audio signal can be performed by a number of processing resources and that may depend on their availability, their capacity, and the particular type and quantity of processing that needs to be done. There is therefore additional data included in the data being exchanged between processors to allow process sharing and allocation and in particular how much needs to be done and conversely how much has been done to the audio data at each step of the process from source to recipient.
- the previously downloaded JAVA applet reads the data downloaded from the company sponsored web site to check its content integrity, and need for further data processing and customisation.
- the downloaded audio content is then processed by the recipient's notebook PC 218 if required, then conveyed to the recipient's media player 212 where it is stored in fixed or removable digital data memory.
- removable memory provides for allowing the recipient to have available a large variety of audio at the ready should they wish to listen to it.
- the cochlear implant recipient no different to a person that has a media player such as an iPod (TM of Apple Computers Inc.) multimedia player that can hold thousands of songs or a multimedia player (often referred to as an MP3 player) that has the ability to receive, store and play a variety of music and video files that include a removable memory for swapping over genre of audio or video content when the user wants to, but in this case the selected audio tracks are tailored to the cochlear implant users' own particular needs.
- a media player such as an iPod (TM of Apple Computers Inc.) multimedia player that can hold thousands of songs or a multimedia player (often referred to as an MP3 player) that has the ability to receive, store and play a variety of music and video files that include a removable memory for swapping over genre of audio or video content when the user wants to, but in this case the selected audio tracks are tailored to the cochlear implant users' own particular needs.
- An indication as to the successful completion of the download is conveyed from the media player to the recipient by way of an indication on the device and/ or to the recipient's cochlear implant and/ or the recipient's PC.
- the media player 212' may be operated independently from the Internet and recipient's PC much like a stored audio player, the examples given earlier including an iPod or MP3 player.
- this media player 212' can now allow a recipient to search, select and play the one or more downloaded audio materials contained within the media player or a removable digital storage memory device at their discretion.
- a removable digital memory can be used to store for extended period of time the downloaded audio.
- DRM Digital Rights Management
- this procedure can be performed by a carer or guardian using instructions in the recipient's product literature and or that provided by a dedicated website.
- the system and arrangement of the cochlear implant depicted in Fig. 3 differs from that depicted in Fig. 2 in that the cochlear implant recipient uses a body worn speech processor which can also perform the tasks of a media player as described above.
- the processing allocated to this type of processor is of two clear types, that associated with the media player functionality and that associated with the implant processes. Clearly, there are tasks that the one or more processors within such a device can be done simultaneously and as such the body worn device is be designed to perform them accordingly.
- the recipient's speech processor is adapted to initiate the process of receiving digital audio signals representing an Internet news broadcast 330 sourced from a third party vendor supplier, that has been customized for the recipient by a proprietary web hosted computer 322.
- the recipient's PC then downloads this data to the recipient's body worn media player and speech processor 328 via the Internet and/ or an associated web wireless hotspot service 320 and then the wireless link, provided in this example by a Bluetooth 332 data transceiver device fitted to both the PC 318 and the body worn device 328.
- the speech processor in the body worn device translates received sound data into neural stimulation data that is then delivered in a timely manner to the recipient's implanted neural stimulator in the manner briefly described previously.
- the recipient may signal to the body worn device that wireless access is available by pressing a button on their speech processor.
- This causes the electronic controller incorporated within the body worn device, which is part of their cochlear implant, to energise an inbuilt wireless link transceiver that may for example be in the form of a WAN service, which in turn determines whether an appropriate signal is present and whether the recipients body worn device has Internet access.
- the recipient Prior to this action, the recipient has accessed a dedicated website to establish an account and authorize online payment for one or more Internet audio programs such as for example, CNN or BBC World News.
- the URL or IP address of this subscriber accessed dedicated website is conveyed via a wireless data link to the recipient's cochlear implant, where is it stored in a volatile data memory along with downloaded software that enables the cochlear implant to send and receive data via the Internet, for example including but not limited to, using TCPIP and FTP protocols.
- the recipient By operating a control button on their body worn speech processor 328, the recipient causes their cochlear implant to recall the previously stored URL or IP address and subsequently gain access to this dedicated website.
- Other data including one or more of, but not limited to; the recipient's ID, their prosthesis ID and associated parameters, are conveyed to the computer host 322.
- the computing resource at the host location accesses the recipient's choice of audio material, which in this instance happens to be BBC World News as might be supplied in a streaming format from www.bbc.co.uk Data representing the sound of the chosen news service is returned to the computer hosting the dedicated website.
- the said digital audio data is then processed in a manner that customizes it for the benefit of the recipient's listening experience, taking account of the recipient's cochlear implant, its various operating parameters and recipient preferences and settings.
- such processing may be used to remove all musical content as well as to increase the pitch frequency of a newsreader's voice in order to aid a recipient's understanding.
- This processed data is then conveyed to the recipient's body worn speech processor via the Internet and the Internet wireless hotspot service.
- the recipient's prosthesis On receiving this data, the recipient's prosthesis authenticates the data and determines readiness and suitability for delivery of the data as neural stimulation to the recipient via their implant. If no further processing or customisation of the data is required, the data is then conveyed in a timely manner to the recipient's implanted neural stimulator.
- about 30 seconds of sound data storage is provided temporarily within the cochlear implant, in order to avoid interruptions due to temporary breaks in the data transmission path between vendor and recipient. Timing information included with the sound data is used to ensure the data is presented to the recipient at the appropriate rate.
- Fig.4 depicts a block diagram of a computer device for processing, an audio source obtained on-line or from some other source, in such a way that it is suitable for a cochlear implant recipient.
- the personal computer 400 PC or PDA or other suitable computer device
- the audio source may have been obtained from a service available on the Internet 402 or the source may be a pre-recorded audio track available from a CD or other media purchased from a store, or from a multitude of other sources.
- the processing of the audio source into a form suitable for its use by a cochlear implant device is very specialised and very processor intensive.
- the computer device 400 needs to be capable of executing the program 404 that performs that function.
- the program can be downloaded to the computer or it can be provided on digital memory media, which is shown by way of example as a CD in Fig. 4.
- the source dependent may be many ways of processing the source dependent not only the type of source it is to begin with, but more particularly dependent on the cochlear implant device characteristics and the recipient's preferences.
- the recipient specific parameters such as ID, stimulation MAP data and user preferences may have influence on the type and extent of processing performed.
- the processed audio is provided to a media player that may further process the audio or may merely store the processed or even partially processed audio for playback at the time or at a later time.
- the media player communicates to the over the ear sound processor 408 which provides a modulated radio frequency signal so as to transmit 410 the cochlea stimulus signals to the implanted portion of the cochlear implant device.
- Fig. 5 depicts a block diagram of an alternative arrangement of the system depicted in Fig. 4 wherein the recipient uses a body worn device to deliver the processed audio to their cochlear implant so that the stimulus data rate can be higher as can be the radio frequency radiation strength, thereby improving stimulus effectiveness and quality. To facilitate these improvements different protocols to control the data flow rate and quality may be used.
- Fig. 6 depicts a block diagram of a computer device for processing an audio source on a storage medium in such a way that it is suitable for a cochlear implant recipient.
- communication is provided directly from the computer device (which as stated previously could be a personal computer or a PDA or a mobile phone or other computationally suitable computer device) to a wireless enabled over the ear sound processor device.
- Communications is preferably achieved using a wireless connection, which for example could be based on IEEE 802.11 standards (WiFi) or Bluetooth or other low power wireless communications technology, otherwise a wired connection although more physically restrictive is possible.
- WiFi IEEE 802.11 standards
- Bluetooth Bluetooth
- Fig. 7 depicts a block diagram of a prior art cochlear implant system 700, mainly to illustrate that the input to such a device receives air compression and rarefaction sound waves that are transformed into electrical signals by one or more microphones 702 located on the exterior of an over the ear or body worn audio processor.
- the user may also input analogue electrical audio signals directly to the audio front-end 704.
- the output from the audio front-end is provided to a signal analysis module 706 that may perform one or more signal transformations including Fast Fourier Transforms and feature extraction algorithms.
- the specific audio frequencies are mapped 708 to specific channels that are destined to stimulate one more cochlea located electrodes in one or more combinations at one or more stimulation rates at one or more amplitudes.
- the previously described processing is performed external to the implant recipient and then transferred to the implanted portion 710 using modulated radio frequency transmission generated by the external portion of the cochlear device.
- the radio frequency transmission not only provides the modulated codes containing stimulation information but the radio frequency power is captured and reused to power the electrode 712 stimulus as a current 714 of required duration and strength.
- Fig. 8 depicts a block diagram of system elements for supplying and processing an audio source obtained on-line in such a way that it is suitable for multiple cochlear implant recipients with the majority of processing done external of the cochlear implant device. Not only is the processing done externally of a particular cochlear implant recipient but external of a multitude of cochlear implant recipients 1 and 2 in this embodiment.
- An audio source 802 and 804 as previously described is provided to the processor server located at a host server, which may be for example, owned and operated by a cochlear device manufacturer, where further processing 806 and 808 is performed in accordance with the parameters and preferences of one or more cochlear implant recipients.
- the processing associated with the channel to electrode mapping in the illustrated embodiment more readily performed by the remote server while also in the illustrated embodiment amplitude mapping is performed in the user cochlear implant device.
- Fig. 8 illustrates two users 1 and 2 each having a user sound processor 808- and 808-2 respectively in the body worn or over the ear external portions of the cochlear implant device.
- the implanted portions 810-1 and 810-2 respectively of the cochlear implant device include as previously noted electrodes 812-1 and 812-2 respectively and current sources 814-1 and 814-2 respectively.
- the hearing prosthesis is configured with the ability to convey and receive electronic data via the Internet and or other intervening, wired and wireless pathways.
- This conveyance includes the exchange of IP or URL data that allows the cochlear implant to locate and communicate with one or a number of specific Internet web sites.
- the cochlear implant is provided with the ability to store and retrieve data that may include, but is not limited to, a national and or international telephone number, the Internet IP address or unique resource locator of one or a plurality of Internet websites.
- the hearing prosthesis is provided with the ability to convey recipient asserted data or commands from the recipient to a remote device.
- this could extend to the provision of a pressure sensitive actuator incorporated into an cochlear implant device whereby the recipient by pressing this control initiates a wireless message from the device so as to signal authorisation for the remote purchase of goods from an Internet site, so configured as to accommodate such purchasing.
- the cochlear implant manufacturer or other parties such that volume and or tonal qualities of accompanying soundtrack music and environmental sounds are customised, reduced of removed completely. Whether this is achieved through processing of the sound data, or in obtaining and remixing the original elements of the sound track, or the reconstitution of speech from subtitle text included with the material the product is being modified in a way that enhances the enjoyment of a cochlear implant recipient.
- This type of processing may include the provision of only music that will enhance the recipient's watching pleasure since the speech aspect of the recorded media is displayed in the form of sub-titles.
- the hearing prosthesis is configured to capture information including a recipient's preferences and/ or selection and or listening habits and history of audio material conveyed to them via a cochlear implant.
- a recipients selection preferences, purchase and listening history of audio material downloaded via the Internet could be stored within a part of their prosthesis system for conveyance to another party such as a music program vendor as required.
- An implantable hearing prosthesis or any part there of, including it's support environment, able to encrypt, decrypt, store, retain, retrieve, compress, decompress, convey and validate electronic data in a manner that may be used to increase the privacy, security and integrity of any type of data or information pertaining to a recipient, their prosthesis or any part there of, including that -which may control the electrical currents that may be applied to the recipient, their healthcare clinic, their bank or credit card provider, their password or information conveying their authorisation to transfer money or financial merit for the purpose of obtaining goods and services.
- a hearing prosthesis or part there of configured to receive and or convey neural stimulation data requiring little or no extra data processing before the data can be translated into one or more electrical neural stimulation currents for delivery to a recipient so as to evoke an approximation of sound perception.
- An Internet website configured to allow an implantable cochlear implant recipient to browse, select, purchase, process and download data representing audio, or audio plus visual information, whereby such processing takes account of uploaded and or locally stored recipient attributes, to benefit the recipient's listening experience.
- This said audio or audio plus visual data may include, but is not limited to; diagnostic, therapeutic, habilitation, educational and entertainment material
- Processing data related to a recipient and prosthesis A local or remote computer system that is configured to process and download data representing sound using one or a plurality of specific attributes that pertain to an cochlear implant, their guardian, carer or health care professional.
- a cochlear implant system data storage partition of storing a description, summary or extract of the material selected by a recipient can be retrieved from the data store for presentation to someone other than the recipient. This feature is intended allow a recipient's friend, guardian, carer or health care professional to share, understand or appreciate the nature of material selected by the recipient.
- An cochlear implant system or component parts there of configured with an area of protected, volatile and or non volatile data and or program memory which is locally accessible to the cochlear implant system, and remotely accessible via a network, such as, but not limited to the Internet.
- This protected area or memory may be accessed by a remote source to store, retrieve and or execute software and or data so as to benefit a recipient in a safe manner.
- An Internet website or other remotely hosted computing resource with a capacity to store electronic data that is configured in a manner that retains data from a recipients cochlear implant system, such that the recipient and or their cochlear implant system may receive this memory capacity for any purpose relating to the operation of the said cochlear implant system and or data that is conveyed by the cochlear implant system.
- data could include the online Internet browsing history of the recipient; such that they could press a back button several times in order to access previously Internet accessed audio content.
- this same site or the recipient's cochlear implant system can store Internet like bookmarks as is performed by most Internet browsing programs.
- Cochlear implant system with "Cookie” storage & handling can be configured to capture, store, retrieve and convey what is generally referred to as Internet cookies. Such data or pieces of information conveyed for storage by the prosthesis or part there of, from any source, allow data such as that from Internet web sites and or accessories used in association with the prosthesis, to be exchanged.
- a cookie can be stored in a cochlear implant system by an Internet web site while it is being accessed by the recipient for audio content, might be used to identify the recipient and their purchase history when next they next visit that particular web site.
- a recipient's listening history of a particular type of audio content or that supplied by a particular vendor could be captured and stored by a cochlear implant system, then conveyed back to the vendor when next that cochlear implant system has Internet access.
- the last used volume settings of a body-worn element of a cochlear implant system could for example be stored within a recipient's implant so that when they switch to their ear worn or outdoor use (i.e. spill proof/ water proof) speech processor, the stored settings are automatically transferred to the newly selected processor.
- Sound Data "watermark” label and tracking system The storage and or conveyance of sound information or data in any form, which is labelled or accompanied by information or data that allows this sound information or data to be identified in some manner.
- sound or audio material could be; conveyed as sound or electronic data, processed, modified, stored or retrieved, in any manner while still retaining this identifying information.
- a recipient plays an item of audio material through the speakers of their CD player, low level, high frequency acoustic tones, with an accurately defined frequency, included at the beginning and or end the material could be received by the microphone of the cochlear implant system, then decoded and stored by the cochlear implant system as an identifying number.
- Any sound data processing applied to refine this data for the benefit of a recipient could be configured so as to retain this identifying data so that it may be reconstituted after such said sound processing.
- the material is identified by the cochlear implant system, which in turn can be made to keep track of the date and time this audio material was listened to.
- This data pertaining to the recipient's listening habits and choice of material could then be conveyed by any means including the Internet, to the vendor of the audio material or to the recipient's healthcare professional, on request or as required.
- Cochlear implant system with universal or configurable interface A cochlear implant system or any part or parts there of, can be configured in a manner that allows it to access any other computing resource via a wired and or wireless link such as, but not limited to the RS232, USB, USB2, Firewire, I 2 C, IRda, Bluetooth, acoustically coupled DTMF tones.
- a wired and or wireless link such as, but not limited to the RS232, USB, USB2, Firewire, I 2 C, IRda, Bluetooth, acoustically coupled DTMF tones.
- a cochlear implant system can be configured to convey via any wired or wireless network, including the Internet, information describing any or all of the following attributes; its total and or remaining data and or program and or application storage capacity as pertaining to any temporary, permanent and or removable, volatile and or non volatile electronic data storage memory. Prosthesis with battery life meter
- the cochlear implant system can be configured to convey via any wired or wireless network, including the Internet, information describing any or all of the following attributes; its total and or remaining battery life and or energy delivery capacity.
- the present invention may be implemented using hardware, software or a combination thereof and may be implemented in one or more computer systems or other processing systems.
- the invention is directed toward one or more computer systems capable of carrying out the functionality described herein.
- Various software embodiments are described in terms of the example computer systems. After reading this description, it will become apparent to a person skilled in the relevant art(s) how to implement the invention using other computer systems and/ or computer architectures.
- computer program medium and “computer useable medium” are used to generally refer to media such as removable storage drive, a hard disc installed in a removable hard drive bay, and storage media such as compact discs and digital versatile discs.
- computer program products are a means for providing executable software and data to the computer systems described.
- the software may be stored in a computer program product and loaded into the computer system using removable storage drive, hard disc and storage media.
- the software may be communicated from a remote source and stored in the computer system.
- the computer software contains instructions to be performed by the processor/ s in the computer system.
- the invention is implemented in hardware using, for example, hardware components such as application specific integrated circuits (ASIC's). Implementation of the hardware state machine so as to perform the functions described herein will be apparent to persons skilled in the relevant art(s).
Landscapes
- Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Prostheses (AREA)
Abstract
A signal processing system is described for supplying signals of a form suitable for at least one cochlear implant system including an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal including a digital signal processor receiving said digitized source audio signal, the digital signal processor adapted to modify the digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the digitized source audio signal including at least spectral and temporal content suitable for use with the predetermined implanted cochlear device.
Description
ORGANISATIONAL STRUCTURE AND DATA HANDLING SYSTEM FOR COCHLEAR IMPLANT RECIPIENTS
This invention relates to a system and apparatus for processing and distributing signals of a form suitable for cochlear implant devices.
Background
Sound vibrations that impinge on the outer and middle ear of a person with normal hearing, are conveyed to a tiny shell-like organ called the cochlea where they are spectrally dispersed across a vast number of neural sensors that line the narrow pathways therein. Profound deafness often arises when outer, middle or inner ear vibration pathways are severed or when the neural sensory mechanism of the cochlear is damaged or diseased.
Implanted hearing prostheses that stimulate the inner ear can be used to assist people with total or partial hearing loss. In general, such devices employ electronic analogue and digital techniques to process, and transform received sound or signals representing sound, into one or more discrete channels of mechanical or electrical stimulus information. The resulting stimulus is then conveyed to neural receptors within the recipient's cochlea associated with the sense of hearing. In this way, persons (hearing prosthesis recipients) with severe to profound hearing loss are able to perceive a potentially beneficial approximation of sound.
The fidelity of sound perceived by the recipient of an implanted hearing prosthesis and their ability to distinguish the pitch of different musical notes is however, severely limited by the relatively small number of discrete electrodes, which can be fitted within the narrow the confines of their cochlea.
The broad dispersal of applied electrical stimulation further conspires to degrade the recipient's ability to discriminate sounds that are close in frequency. As a consequence, most recipients have difficulty making any sense of all but the simplest forms of music and environmental sounds. When added to speech, music and environmental sound can significantly confound a recipient's speech perception
This inability renders most sounds, including recorded material, almost incomprehensible to the recipient and thereby limits their ability to learn and adapt to their newly acquired prosthetic hearing. This is especially relevant when the recipient is an infant or person with little or no prior experience of sound and its relationship to everyday events.
Currently the process of learning to use an implanted hearing prosthesis requires extensive and long-term habilitation provided by health care professionals.
While complex speech signal processing is currently provided to benefit vocal communication, the size, weight and battery power of body worn prostheses severely limits the computing power available to process such sounds as orchestral music or complex environmental sounds in real time for a recipient.
There is a need amongst cochlear implant recipients to hear the latest and topical audio that is taken for granted by those that do not have such implants and the system and methods of the invention provide a service and product that at least ameliorates the limitations described above or provides an alternative to present conditions that cater to the individual and special needs of cochlear implant recipients.
Brief Description of the Invention
In a broad aspect of the invention a signal processing system for supplying signals of a form suitable for at least one cochlear implant system including an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal includes: a digital signal processor receiving said digitized source audio signal, the digital signal processor adapted to modify the digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the digitized source audio signal including at least spectral and temporal content suitable for use with the predetermined implanted cochlear device.
Preferably the signal processing system further includes; a digitized audio signal distribution processor operatively connected to the digital signal processor for making said output form of the source audio signal, available to at least one cochlear implant system.
Yet further in the signal processing system the digital signal processor is configured to provide an output form of the source audio signal using a mapping for a predetermined implanted cochlear device and further wherein the cochlear implant system and includes a body worn or over the ear digital signal processing device adapted to receive said output form of the source audio signal and transform the output form to a signal suitable for the predetermined implanted cochlear device.
In another broad aspect of the invention a method for supplying signals of a form suitable for at least one cochlear implant system including an implanted cochlear device wherein a digitized source audio signal is a digitized stored
audio signal or a digitized real time audio signal, the method steps included: digital signal processing the digitized source audio signal to modify the digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the digitized source audio signal including at least spectral and temporal content suitable for use with the predetermined implanted cochlear device.
Preferably, the method for supplying signals according to claim B further includes the step of operatively connecting said digital signal processor to a digitized audio signal distribution processor for making said a predetermined output form of the source audio signal available to a predetermined cochlear implant device.
Yet further preferably, the method for supplying signals includes the step of configuring the digital signal processor to provide an output form of the source audio signal using a mapping for a predetermined implanted cochlear device wherein the predetermined cochlear implant system includes a body worn or over the ear digital signal processing device adapted to receive said output form of the source audio signal and transform the output form to a signal suitable for the predetermined implanted cochlear device.
Further to the previous method step for supplying signals the method includes the step of storing and selectively outputting one or more output forms of respective source audio signals in a memory device in the body worn or over the ear signal processing device.
Yet further the method for supplying signals further includes the step of applying at least one digital filter in the digital signal processor to the digitized source audio signal.
Further to the previous method step for supplying signals wherein the digital filter modifies the frequency content of the source audio signal during the process of modifying the digitized audio signal according to the predetermined setting.
Yet further the method for supplying signals wherein the source audio signal includes cochlear implant specific auditory habilitation audio signals.
In yet another broad aspect of the invention a modified audio signal distribution system for signals suitable for at least one cochlear implant system includes an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal, the system including: a first networked computer having multiple stored digitized source audio signals accessible from the network; a second networked computer having access to the first networked computer and adapted to process one or more of the multiple stored digitized source audio signals to include at least spectral and temporal content suitable for a predetermined cochlear implant system including a predetermined implanted cochlear device; and a third networked computer having access to at least the second networked computer and associated with a predetermined cochlear implant system, for receiving the one or more multiple stored digitized source audio signals including at least spectral and temporal content suitable for the predetermined cochlear implant system.
Preferably the modified audio signal distribution system further includes: a fourth networked computer adapted to authorise a financial transaction between at least the user of the second networked computer and one or more other parties.
Yet further preferably the modified audio signal distribution system further includes: a fifth networked computer associated with a predetermined cochlear implant system for receiving and storing the one or more multiple stored digitized source audio signals including at least spectral and temporal content suitable for the predetermined cochlear implant system before providing the one or more multiple stored digitized source audio signals to the predetermined cochlear implant system.
In a yet further aspect of the invention a body worn or over the ear digital signal processing device for use with an implanted cochlear device for receiving an output form of a digitized source audio signal generated by a digital signal processor that modifies a digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the source audio signal including at least spectral and temporal content suitable for use with an implanted cochlear device wherein said digital signal processor is physically remote the user of a cochlear implant system the body worn or over the ear digital signal processing device includes: a receiver adapted to receive the output form of the digitized source audio signal, and a digital signal processor to transform the output form to a signal suitable for a predetermined cochlear implant device implanted in the user.
Preferably the body worn or over the ear digital signal processing device a further includes: a digital store for storing at least one received output form of a digitized source audio signal and for making a stored received output form available to the digital signal processor.
Yet further preferably the body worn or over the ear digital signal processing device includes: a radio signal generator and modulator for forming a radio signal modulated according to a protocol suitable for communicating the output form of a digitized source audio signal to the predetermined implanted cochlear device, and a radio signal transmitter for transmitting the modulated radio signal to the predetermined implanted cochlear device.
Further preferably the body worn or over the ear digital signal processing device has a device address addressable by a networked computer.
Preferably the address is an address that conforms to Internet Protocol addressable by a networked computer.
Throughout the specification unless the context applies otherwise, the words "comprise" and "include" and variations such as "comprising" and
"including" will be understood to imply the inclusion of the stated integer or group of integers but not the exclusion of any other integer or group of integers.
A detailed description of one or more preferred embodiments of the invention is provided below along with accompanying figures that illustrate by way of example the principles of the invention. While the invention is described in connection with such embodiments, it should be understood that the invention is not limited to any embodiment. On the contrary, the scope of the invention is limited only by the appended claims and the invention encompasses numerous alternatives, modifications and equivalents.
For the purpose of example, numerous specific details are set forth in the following description in order to provide a thorough understanding of the
present invention. The present invention may be practiced according to the claims without some or all of these specific details.
For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the present invention is not unnecessarily obscured.
BRIEF DESCRIPTION OF THE FIGURES
Fig. IA depicts 4 various levels of processing that can be applied to an external aural source and an illustrative result for various hearing recipients;
Fig. IB depicts frequency verses amplitude graphs of the aural source and the manner in which the various frequencies are handled by one or other processes intended to provide input to a cochlear device
Fig. 1C depicts a flow diagram of steps for supplying and processing an audio source obtained on-line in a manner suitable for a cochlear implant recipient;
Fig. 2 depicts a block diagram of a system configuration for supplying and processing an audio source obtained on-line in such a way that it is suitable for a cochlear implant recipient;
Fig. 3 depicts a block diagram of an alternative arrangement of the system depicted in Fig. 2 wherein the recipient uses a body worn device to deliver the processed audio to their cochlear implant;
Fig. 4 depicts a block diagram of a computer device for processing an audio source obtained on-line in such a way that it is suitable for a cochlear implant recipient;
Fig.5 depicts a block diagram of an alternative arrangement of the system depicted in Fig.4 wherein the recipient uses a body worn device to deliver the processed audio to their cochlear implant;
Fig. 6 depicts a block diagram of a computer device for processing an audio source on a storage medium in such a way that it is suitable for a cochlear implant recipient;
Fig. 7 depicts a block diagram of a prior art cochlear implant system; and
Fig.8 depicts a block diagram of system element for supplying and processing an audio source obtained on-line in such a way that it is suitable for a cochlear implant recipient.
DETAILED DESCRIPTION
The following description provides illustrative details of an organisational structure, system, method and various apparatus for supplying and processing an audio source in such a way (using predetermined settings) that the audio source is suitable for a predetermined cochlear implant recipient.
Referring to Fig IA, there is shown a series of different scenarios which will be discussed to illustrate the environment of the disclosure provided by this specification and how it may be beneficial to cochlea implant recipients.
The uppermost illustration depicts the situation where a recipient of an aural stimulation (an external aural condition) who is unaffected by any significant aural impediment such as cochlea damage or disease, thus the recipient is capable of receiving and appreciating the full or the majority of the rich
spectral content of the aural content that is part of the aural stimulation that may for example contain a combination of speech (singing) and musical content.
The illustration immediately below the uppermost illustration depicts the situation where the recipient of an aural stimulation (an external aural condition) who is affected by a disability such as cochlea damage or disease, and thus the recipient is incapable of properly processing the aural content and is unlikely to be pleased with the aural experience (although it is accepted that hearing something instead of nothing may be considered an improvement).
The illustration second below the uppermost illustration depicts the situation where the recipient of an aural stimulation (an external aural condition) is affected by a disability such as cochlea damage or disease, but is aurally compensated by the use of prosthesis (such as for example a cochlea implant), in which case the recipient is then capable of processing a modified version of the aural stimulation and much better off than the recipient depicted in the illustration immediately below the uppermost illustration.
The illustration third below the upper most illustration depicts the situation where the recipient of an aural stimulation (an external aural condition) is affected by a disability such as cochlea damage or disease, but is aurally compensated by the use of a conditioning and modifying agent that knows the particular characteristics of the recipient's prosthesis (such as for example a cochlea implant), in this case the recipient is capable of processing a modified version of the aural stimulation and very much better off than the recipient depicted in the illustration immediately below the uppermost illustration.
The illustration fourth below the uppermost illustration (the bottommost illustration) depicts the situation where the recipient of an aural stimulation (an external condition) is affected by a disability such as cochlea damage or disease, but is aurally compensated by the use of a conditioning and modifying agent that knows the particular characteristics of the recipient's prosthesis (such as for example a cochlea implant that is reporting the characteristics independently of the recipient and the conditioned and modifying agent provides the relevant signal direct to the recipient), in this case the recipient is capable of processing a modified version of the aural stimulation and very much better off than the recipient depicted in the illustration immediately below the uppermost illustration.
The specification describes the various organisational structures, systems, methods and apparatus for supplying and processing an audio source in such a way that the audio source is suitable for a predetermined cochlear implant recipient.
The structure and method augments and/ or replaces the limited processing capability of an implanted and/ or body- worn component of a hearing impaired recipient's hearing prosthesis, with one or more data processing resources, preferably located remote of the recipient.
The processing and then conveyance of recipient specific data, together with data describing the sound data processing history and the degree to which further processing is necessary, and additionally the way in which this data is passed and shared between data processing resources, form elements of this invention.
The source audio may include, but is not limited to: diagnostic, therapeutic, habilitation, educational and entertainment material.
The enhanced data processing capabilities can be further enhanced if implemented through the conveyance and exchange of one or more parameters describing the recipient, their prosthetic hearing attributes, requirements and preferences, as well as the near real time capability and availability of computing resources and processing required of the data or the ability for dedicated computing resources and processing to perform tasks upon demand and within a short time deliver a desired product to a requesting recipient that is suitable for that recipient and that recipient alone.
A system for supplying and processing an audio source in such a way that it is suitable for a cochlear implant recipient will be described that includes computer networks, most notably the world wide computer network referred to as the Internet. Computer networks can be used to connect cochlear implant recipients or their carers to computer servers that can supply a multitude of digital audio source material from multiple sources and then process chosen digital audio source material for a particular cochlear implant recipient. The transaction can be automated and the financial aspect of the transaction can be handled using known Internet based services.
The processed audio data can be listened to at the time or can be stored for later use.
Depending on the computing equipment available, the processing required to customize the source audio can be shared or distributed amongst various computer equipment with in the network. It is anticipated that the bulk of processing will be done by computers associated with the supplier of the
service as the processing power and efficiencies will be more readily achievable when done by a bulk supplier and processor. However, as computing power increases it is conceivable that some if not all the necessary processing can be done on a computer associated directly with the cochlear implant recipient. Alternatively, the supplier of such a service can do partial processing and the remainder, being a large or small proportion, is done by another computer associated with the cochlear implant recipient or even by a device worn by the cochlear implant recipient.
An example arrangement of a body worn computer device includes a wireless linked, portable electronic media player containing adapted hardware and software that uploads to other computers, one or more of a recipient's ID, customisation parameters, prosthesis type and capability, password, banking details and financial merit transfer authorisation from one or a number of various parts of the recipient's prosthesis to a recipient's wireless enabled personal computer (PC) where it is then conveyed via any convenient means, including the Internet, to a computing resource associated with a third party or cochlear company sponsored website server able to receive and decrypt this data to allow the recipient to browse and select for purchase, audio material that is then processed and customised for the particular recipient by taking account of the recipient's preferences and cochlear implant characteristics.
Processed data which may be partially processed for a number of reasons, in that it is more expeditious to communicate a reduced data set or that different degrees of further processing is needed by a recipient, is then conveyed via a communications medium such as for example, the Internet back to the recipient, for example in the case of the Internet the recipient's personal computer or digital data device such as a data capable wireless personal
assistant or mobile cellular telephone that has a wired or wireless connection to the cochlear media player of the recipient, where further processing and refinement may be undertaken before this customised audio material is conveyed to their personal cochlear implant device where it is converted into stimulus current data, for inductive transmission to the recipient's implanted cochlea stimulator electrodes.
In another example, an implant recipient is provided access to a website allowing them to browse select and purchase pre-recorded or live broadcast for example audio material, which prior to being downloaded, via the Internet, to their special personal media player or sound processor, is intensively processed and customised for delivery to the recipient's implanted cochlea stimulator electrodes.
A non-limiting example of how the spectral content can be modified to improve the listening experience for the recipient is provided below.
Graph 300 in figure IB discloses a selective representation of speech components in which the vertical axis 301 (which is the same for all four graphs depicted in figure IB) refers to relative amplitude and the horizontal axis 302 (which is the same for all four graphs depicted in figure IB) refers to frequency. The vocal sounds 303 and 304 are generated, as are their harmonics 305.
Referring to graph 400 of figure IB, the spectral content of chaotic musical sounds such as cymbals and brush drums, include many frequencies and their harmonics 401 that are close in frequency and often higher in amplitude than those associated with the vocal sounds 303 and 304 and the important
harmonics of the voice of the accompanying vocalist as depicted in graph 300 of figure IB.
As a result of the higher amplitude of the chaotically distributed harmonics of the music, these sounds are conveyed to an aurally compromised hearing prosthesis user instead of the vocal harmonics that are vital for vocal recognition.
This is illustrated in graph 500 of Fig IB there is disclosed a corresponding spectral range allotted to each one of 22 tonotopically-arranged electrodes of a cochlear device. Note that speech sound harmonics important for vocal comprehension are missing from the part of the spectrum indicated at reference numeral 504. Hence, graph 500 shows the limited spectral delivery of 5 channels of neural stimulation that would be created by a cochlear implant system in response to the combined speech and cymbal sounds disclosed in the graphs 300 and 400 immediately above.
The following example describes how in one embodiment of the invention sound data is processed or customised so as to benefit the user of a cochlear implant hearing prosthesis.
The spectral content of chaotic musical sounds such as cymbals and brush drums such as those depicted in Figure 400, include many emissions (401) that are close in frequency and often higher in amplitude than those associated with the vocal sounds of an accompanying singer.
Figure 500 plots the amplitude of the stimulating current that would be applied to each one of the 22 spatially dispersed neural stimulating electrodes of a typical cochlear implant, not incorporating the principles described in this
specification, in response to a combination of vocal and cymbal sounds as depicted in figures 300 and 400. The horizontal axis 502 of figure 500 represents the range of sound frequencies that are assigned by the prosthesis to stimulate each spatially disposed electrode located within the user's cochlea. The vertical axis 501 of figure 500 represents the relative amplitude of the stimulating current that is subsequently delivered to each of these electrodes.
The vocal harmonic emissions vital for speech recognition as depicted by 305 in figure 300 is clearly absent in neural stimulation delivered by a cochlea implant without this invention. Instead of delivering the electrical stimulation necessary to evoke a perception of sound in the vocal harmonic frequency range depicted by 504 in figure 500, stimulation representing the more dominant cymbal sounds shown by 505 are delivered. Speech perception by a user is difficult if not impossible in circumstances where a small limited number of electrodes are available to simultaneously convey complex sound information.
Complex and energy demanding sound data processing, as enabled by the system and devices described in this specification, can be shared amongst available resources and used to distinguish the important harmonic components of speech or other sounds of interest to the user from those that are less important to the user.
Data processing software using Fast Fourier Transform algorithms or similar methods are used within the software incorporated into any one or a number of devices (104, 107, 110, 115 and 119) as shown in figure 1C. Detailed statistical analysis of the high resolution spectral information extracted from incoming sound data is undertaken so that only those spectral peaks 305 that
are an exact integer multiple of any lower frequency peak within the fundamental frequency range of normal human speech (303 and 304), are conveyed as stimulation to the user. In this way, the randomly dispersed, higher amplitude spectral components of complex sound unrelated to speech are ignored.
Graph 600 in figure IB illustrates the electrical stimulation to an implanted cochlea simulation device and even the associated external processing and power delivery device that would result from the use of the suggested processing when supplied the sounds represented by graphs 300 and 400 in figure IB. In this case the all-important harmonics of speech are represented by electrical stimulation in the cochlea as shown by the stimulation currents shown as 601 in graph 600 of figure IB.
The computing power and electrical energy necessary to undertake such data processing in real time is currently well beyond the capability of existing cochlea implant hearing prostheses. The description herein suggests such data processing be undertaken by conveying details of users hearing needs in a manner that allows the said processing to be assigned to and shared between other, less limited computing resources than those contained within small sized, battery powered hearing prostheses currently in use.
In a contrary example the voice component of the combined voice and musical performance may not be not as important to the cochlea user as the musical component, so the suggested processing can be arranged to provide a result that best suits the representation of the musical content and excludes the voice content.
Implementation of the system, and devices can then be made to suit the individual user rather than being (so called "factory set") to suit the majority of cochlea implant users.
A second more simply explained example of the type of complex sound data processing that might be undertaken to benefit a user is described as follows:
A personal desktop, portable or handheld computer incorporating commercially available, speech to text conversion software, converts sound data containing the lyrics of a musically simple song into text. The principals employed by the spelling and grammar-checking utilities incorporated in common word processing applications are used to automatically correct interpretation errors in a manner that is relevant to the context of the incoming text.
The resulting text is then processed by commercially available, text to speech software. The resulting reconstructed speech signal can be adjusted in tonal quality and pitch to suit a predetermined user, before being conveyed to the hearing-impaired predetermined prosthesis user with none of the confusing background music that accompanied the original speech.
In addition to, and/ or instead of, either partly or fully, the processing normally undertaken by a recipient's body worn sound or speech processor, the intense processing and customisation referred to above, takes account of recipient preferences, perceptive attributes and stimulation and processing strategy MAP settings, to refine the audio content for the recipient.
This can include:
(i) the removal of complex sounds from particular musical instrument in an orchestral recording. Spectrally diverse, atonal cymbal crashes for example, may confuse or overwhelm a spectrally limited listener. Removing such sounds may therefore add to the recipient's listening pleasure.
(ii) the extraction of lyrics from a song, and to then present these as spoken dialog or as a solo voice with or without the tonal qualities of the music. Some infants or children, unable to make any sense the words sung to a popular nursery rhyme like "Twinkle, twinkle little star", might well benefit if these words were translated to mono-tonal dialogue.
(iii) emphasising important dialogue accompanying a film or audiovisual event, so as to minimise confusion from accompanying music and environmental sounds.
Examples of recipient attributes
An implantable hearing prosthesis and a supporting environment, whereby electronic data is used to convey one or more recipient-attributes to one or a plurality of data processing resources external to their implant and body worn sound or speech processor, where such attributes might be applied to process sound data in a manner that benefits a recipient's listening experience.
These recipient-attributes can include, but is not limited to, the following:
attributes that uniquely identify the recipient;
(ii) attributes that can identify and or describe the type, model, revision, repair, modification, upgrade and replacement or renewal status, technical specification and history of certain parts of their cochlear implant system and associated accessories, for example, the implanted stimulator and electrode system, the external processor, data transmitting coil, associated cable system, power system and battery or batteries;
(iii) attributes describing a recipient's history of implantation and prosthesis usage, their personal control settings and preferences for its use;
(iv) attributes describing the batteries and energy sources used by the prosthesis including their type, age, status, use history, internal impedance, open circuit and under load terminal voltage, state of charge and remaining energy and estimated available usage time;
(v) the electrical impedance characteristics of each stimulating electrode and current return path electrode, especially in the manner in which this limits the maximum rate and amplitude of the stimulating currents and charge that may be delivered to the recipient;
(vi) electrical characteristics of the neural stimulus, which may include, but is not limited to; the charge, current, voltage, pulse timing and morphology that is necessary to be applied to each electrode in order to evoke a perception of sound that is; the lowest perceivable by the recipient, the highest that is comfortable for the recipient;
(vii) attributes, which describe the sound data processing strategies, like CIS, ACE, etc. as preferred by the recipient and or their healthcare professional. A recipient may persist in using one strategy, while their health care professional advises another for longer-term benefits;
(viii) attributes, which describe the recipient's spectral response emphasis and de-emphasis tone control setting preferences;
(iv) characteristics of the recipient's evoked sound perception and comfort as it may relate to the apparent pitch and loudness when two or more electrodes are stimulated in the various combinations that are available, sequentially and simultaneously;
(v) characteristics that describe a recipient's ability to understand speech in silence and in a combination with a wide variety of conflicting background and foreground sounds;
(vi) characteristics that describe a recipient's ability to interpret the meaning of various environmental sounds in silence and in a combination with a wide variety of conflicting background and foreground sounds;
(vii) characteristics that describe a recipient's ability to distinguish the pitch or frequency of one or more sounds in silence and in a combination with a wide variety of conflicting background and foreground sounds;
(viii) characteristics that describe a recipient's dynamic, static and temporal preferences for the manner in which sounds that change in amplitude are translated into stimulation. Some recipients, for example are more sensitive to stimulation after periods of sleep with their prosthesis inactive. Such recipients may therefore prefer a slow onset and amplitude limiting for loud sudden sounds during early morning that they may not wish for at other times;
(ix) characteristics that describe the recipient's ability to enjoy and or benefit from live and recorded audio material such as dialogue and various types of music and environmental sounds. A recipient with little ability to make sense of a bass baritone singer for example may prefer listening to a soprano instead; and
(x) characteristics that describe the recipient's determination, resourcefulness and ability to learn, access and make use of resources that benefit their hearing ability. Such resources may include various features of their hearing prosthesis, its accessories and supporting environment, the recipient's carer, guardian, clinic, audiologist, health care professional as well as remotely accessible information in the form of audio and audio plus visual material for the delivery of therapy, education and entertainment.
The data processing resources referred to, may include, but are not limited to wire or wireless linked resources with data processing and storage capabilities that are less constrained in size, weight and available power, than those incorporated in the implanted and body worn parts of the recipient's cochlear implant devices. Such resources may include, but are not limited to; a handheld media player, mobile cell phone or PDA with accessible
computing resources, a nearby personal computer, or a remotely located computing resource accessible via the Internet.
Figure 1C graphically shows the flow, processing and exchange of sound, user and prosthesis data associated with one embodiment of the invention.
Incoming aural sounds 101 of a musical performance are captured, edited and duplicated for commercial distribution and sale 102 in the form of digital sound data files via the internet or via analogue or digital storage media such as compact disks 103.
User identification, hearing prosthesis attributes, purchase instructions and money transfer authorisation data originating from the user 122 and their prosthesis (115 and 119) as shown by the stripped arrows (120, 118, 112, 109, and 106), is conveyed via several wireless and or cable linked devices (110 and 107) to an internet web based server 104.
This server 104 automatically arranges the purchase and conveyance of the data representing the user's choice of audio material 103 to the user via one or more wired or wireless linked data processing and conveyance resources (107 and 110). These (107 and 110) resources could include, web enabled PCs, PDAs and media players. Data transactions related to the purchase of goods and transfer of financial merit have been omitted from diagram 100 in order to ensure clarity for the reader although these mechanisms are known to those skilled in the relevant art.
Data related to a users hearing needs and sound media requests as represented by the stripped arrows (120,118,112, 109, 106) as well the user selected sound data 103 and derivatives as represented by arrows (103, 105,
108, 111, 117 and 121) includes additional data to control the processing and conveyance of sound data and the manner in which these tasks are automatically assigned to devices (104, 107, 110, 115 and 119) for the benefit of the prosthesis user and aural data recipient. Tables 1 and 2 describe an example of the content and format of data packets used by one embodiment of the invention.
A gradual lightening in the solid colour shading applied to the data flow arrows (103, 105, 108, 111, 117 and 121) is used here to symbolise the progressive refinement or customisation of the sound data during its path to the user. Whilst this shading may appear to imply a sharing of processing tasks, in reality sound data processing tasks are automatically assigned and modified in response to the constantly changing status of all the devices involved in the data path.
The way in which the arrangement of this embodiment responds to changes in the computing power, communications bandwidth and available battery power for devices (104, 107,110, 115 and 119) depends largely on choices made by a competent designer and the manner in which it is configured or programmed by the commands 114 entered by a health care professional 113 that are conveyed to the prosthesis by the programming and diagnostic system 115.
Fig. 1C also depicts a flow diagram of steps for supplying and processing an original audio source in a manner suitable for a cochlear implant recipient. An original sound source or audio plus associated video is recorded and prepared for distribution in a multitude of forms, including but not limited to, pre-recorded music in formats playable on a variety of devices, such as for example, Compact Disc players, digital audio players having decoders of
MP3, WAV, etc. formats of audio. The distribution of audio particularly music is distributed in many ways and increasingly being made available for downloading in digital format, for example only, the music download service provided by Apple Computer known as iTunes supplies over the Internet a multitude of music audio having a proprietary music format playable only on a proprietary hardware device known as an iPod, both iTunes and iPod being trade marks of Apple Computer Inc. This by way of example is one form of a media player suited to the needs of non-hearing impaired individuals.
Other audio sources may include radio station broadcasts and more likely "podcasts" of off-air recordings, digital audio sources tailored for replay on an iPod, audio of live and pre-recorded television programs, book readings that are increasingly being made available to the sight impaired, and other source material too numerous to list.
The system proposed differs in that the original audio is selectively and uniquely converted into a form that will enhance the listening experience of a cochlear implant recipient. Thus the recipient can select the source of audio source from the vast array of available sources from one or more multimedia market places 104 that are available on-line via the Internet.
The processing of the source audio, which is typically already in digital form, can take place as described previously in a service provider's computer server, the recipient's own computer or as is more preferable partially in the service provider's computer and partially in the computer or computers associated with the cochlear implant recipient and more preferably predominantly in the service provider's computer server.
Table 1 illustrates a data structure used in one example of the invention.
File Header Information:
Data file or data packet name, label, format, type, file size, creation and modification dates, checksum & error correction information.
Data Transmission & Processing Status:
The identity, network address or URL of the originating source, plus that of interim and final destination devices. Encryption and related security information.
Requested data transfer & processing priority, rate & mode.
Current status and capability of data processing resources existing in the path used to convey this data packet or file
User Needs and Preferences:
Characteristics pertaining to the user's hearing needs and preferences, their prosthesis and it's settings, and the abilities of their primary care giver or guardian.
Authorizations :
A user's authorization for transfer of purchase requests, personal information like credit card details plus authorisation to transfer financial merit.
User or Prosthesis Initiated Requests:
The identity of sound or other media data or information requested by the user.
End of file Indicator:
End of file character or data packet indicator
Table 1 - Example content of the data packets conveyed from a hearing prosthesis incorporating this invention:
File Header Information:
Data file or data packet name, label, format, type, file size, creation and modification dates, checksum & error correction information.
Data Transmission & Processing Status:
The identity, network address or URL of the originating source, plus that of interim and final destination devices. Encryption and related security information.
Requested data transfer, processing priority, rate & mode.
User Needs and Preferences:
Characteristics pertaining to the user's hearing needs and preferences, their prosthesis and it's settings, and the abilities of their primary care giver or guardian.
Authorizations :
Financial or merit transaction approval status feedback.
Media use, transmission and coping authorizations & related expiry dates.
Auxiliary or Optional Content:
Interactive or other information such as media product advertisements to be conveyed to the user.
Primary Content Descriptor:
Format, compression status and processing requirements.
Media or Primary Data Content:
Data representing requested sound or media content.
End of file Indicator:
End of file character or data packet indicator.
Table 2 -Example content of the data packets conveyed to a hearing prosthesis incorporating this invention:
The cochlear implant recipient or their carer with the assistance of the various computer devices most likely their personal computer provides cochlear implant recipient details, purchase and transaction authorising details to the service provider's computer server and those same devices associated with the cochlear implant recipient receives one or more audio sources that are processed and that need to then be stored and provided to the cochlear implant system. There are a number of different distribution arrangements as will be described in greater detail herein.
In one such arrangement a wireless link is made between the personal computer and/ or other device that receives the processed audio source to the cochlear implant recipient's body worn or over the ear device. These devices perform the further processing as required and transform the data into a modulated radio frequency transmission designed to transmit the data to an embedded radio frequency receiver that along with any yet further decoding
and processing is used to stimulate the cochlea of the cochlear implant recipient.
The embedded portion of the cochlear implant system is known technology and specific details are not required however for completeness an extremely brief description is provided. The radio frequency signal is demodulated and at the same time the radio frequency signal is also converted to direct current to power the embedded device. The demodulated signal is also demultiplexed to create several discrete channels of electrical stimulation parameters. Finally, the electrodes that were carefully implanted in the cochlea are stimulated with one or more combinations of signals to provide a sense of hearing to the cochlear implant recipient.
Media Player Referring to Fig. 2, a hearing prosthesis recipient 210 operates a special electronic media storage and player device 212, referred to herein as a "media player". The media player 212 is configured to convey electronic data 214 to and from the recipient's cochlear implant system which in this embodiment is wireless enabled, using a wireless card 16 or such like to convey electronic data 214' to and from the media player as well as the computer device such as a personal computer 218, which may in particular be a notebook PC, wherein the computer device has the ability to communicate with other computers, and in a preferred arrangement having access to the world wide network of computers commonly referred to as the Internet 220.
The recipient operates the PC 218 to access a dedicated Internet website 222 which is run by a supplier of a service to process a chosen digital audio source. The recipient then browses the supplier's site 222 and chooses to view a list of audio material that is available for sale. The audio material may in one
example be made available for distribution via the Internet by the provider of the Internet web site 222 that works in conjunction with a supplier of such material that typically also provides a site 226 containing the selection of audio material.
The recipient or their carer can be provided a wide array of items to select from which can include by way of example only a music track, a speech only track, an audio version of a music video or any audio only version of a video product or a complete version of a video product. In the later case the family of the recipient may want to watch and listen to that particular feature video and the service provider can supply a version of the audio processed specifically for the listening of a cochlear implant recipient and arrange in the delivered product to have the two versions of audio play in synchronism with each other so that the whole family can enjoy the video together.
The recipient or their carer selects a preferred item and payment can be made from an account that provides a credit facility or from a pre-paid account. If credit is being used then the user is required to authorize an appropriate credit card payment using Internet based transaction tools and services well know to those skilled in the art and which are not detailed herein. This type of transaction may or may not involve the recipient's own nominated financial resource 224 that will in most cases be used to authenticate the transaction that has been initiated by the recipient.
In one example of a system for facilitating the transaction type described above, the computer hosting the dedicated website 222 conveys a JAVA script applet to the recipient's computer device such as a notebook personal computer 218, which then executes that applet. In most case this occurs automatically and since the source of the applet is can be made to be a trusted
source of such downloaded executable software this will likely happen without specific intervention or instruction from the personal computer user. However, appropriate instructions for downloading JAVA tools are provided if these are not available on the recipient's computer device.
The applet can be configured to conduct a search for all the hardware interfaced or connected in some way to that computer device 218 and in particular for the presence of the dedicated media player 212 and confirmation that it is in communications wirelessly or otherwise with the recipient's prosthesis.
Once the presence of the media player is confirmed in one of many possible ways known to those skilled in the art, the media player 212 interrogates the recipient's cochlear implant or parts there of, in such a manner as to allow the cochlear implant to convey one or more of the following information that is available from with in the part or parts being used by it's recipient, such as the recipient's ID, the serial number of one or more of the parts of the cochlear implant, their type and other characteristics pertaining to the recipient.
This data, part or all of which may be encrypted by the prosthesis and or media player or the PC or the applet executing on the PC 218, is then uploaded via the Internet to the computer hosting the dedicated website 222. There are many ways in which the encryption can be invoked and equally many ways in which the veracity of the data can be ensured.
Once the host computer 222 has authenticated the recipient's purchase details, the host computer processes and customises the purchased audio material for the recipient with the use of the uploaded and decrypted recipient and prosthesis data. The purchased audio material may have been sourced from
the data stores of the host computer or obtained from other sources, including a third party music vendor 226, or other third party source of audio such as a radio station, broadcasting network, and Internet site that may be providing visual content to the recipient using their personal computer and other such sources.
In one example of a way in which to track use patterns it is typical for web sites to store data files (often referred to as cookies) on computers within the network that read data from the web site. The use of cookies, or like techniques, allows the service provider behind the web site to provide users a more useful interaction with that site and the services it provides. It allows for the automation of a number of potentially repetitious steps involved in an interaction with a web site and provides marketing information on the market as a whole, in most cases in the later case, without identifying individual users.
Other data pertaining to this recipient may have been supplied to the host computer previously and that data can also be applied during processing of a recipient's request for items from the host computer 222.
A local or remote computer system to the host computer is configured to process and download data representing sound using one or a plurality of specific attributes that pertain to an implantable hearing prosthesis, their guardian, carer or health care professional.
Various data including one or more of, the processed data, a description of the data content, the degree to which the data has been processed and the ID of the recipient's prosthesis, is downloaded to the recipient's notebook PC. The processing and customisation of the audio signal can be performed by a
number of processing resources and that may depend on their availability, their capacity, and the particular type and quantity of processing that needs to be done. There is therefore additional data included in the data being exchanged between processors to allow process sharing and allocation and in particular how much needs to be done and conversely how much has been done to the audio data at each step of the process from source to recipient.
The previously downloaded JAVA applet, reads the data downloaded from the company sponsored web site to check its content integrity, and need for further data processing and customisation.
The downloaded audio content is then processed by the recipient's notebook PC 218 if required, then conveyed to the recipient's media player 212 where it is stored in fixed or removable digital data memory. The use of removable memory provides for allowing the recipient to have available a large variety of audio at the ready should they wish to listen to it. This makes the cochlear implant recipient no different to a person that has a media player such as an iPod (™ of Apple Computers Inc.) multimedia player that can hold thousands of songs or a multimedia player (often referred to as an MP3 player) that has the ability to receive, store and play a variety of music and video files that include a removable memory for swapping over genre of audio or video content when the user wants to, but in this case the selected audio tracks are tailored to the cochlear implant users' own particular needs.
An indication as to the successful completion of the download is conveyed from the media player to the recipient by way of an indication on the device and/ or to the recipient's cochlear implant and/ or the recipient's PC.
On completion of a successful download, the media player 212' may be operated independently from the Internet and recipient's PC much like a stored audio player, the examples given earlier including an iPod or MP3 player.
Operating in a stand-alone mode, this media player 212' can now allow a recipient to search, select and play the one or more downloaded audio materials contained within the media player or a removable digital storage memory device at their discretion. A removable digital memory can be used to store for extended period of time the downloaded audio.
It is also possible to incorporate Digital Rights Management (DRM) devices in each relevant device such as the PC 218 the media player 212/212' and if necessary the implanted prosthesis. However, this is not anticipated since the form of the audio being processed will likely be far different to its original form and most unsuitable for listeners not using an implanted prosthesis.
However, in any event other types of rights management implementations can be used so that prior to playing the processed audio material, software embedded within the media player checks the number of times the material has been played or is allowed to be played, and or whether the play expiry date of the material has elapsed. In which case, replay of the audio material is prevented, and an indication as to the reason for not being played is conveyed to the recipient as a signal, tone or message via their media prosthesis, media player and or notebook PC.
In cases where the recipient is a person unable to accomplish such purchasing and downloading, this procedure can be performed by a carer or guardian
using instructions in the recipient's product literature and or that provided by a dedicated website.
Speech Processor initiated Web News Reception
The system and arrangement of the cochlear implant depicted in Fig. 3 differs from that depicted in Fig. 2 in that the cochlear implant recipient uses a body worn speech processor which can also perform the tasks of a media player as described above. The processing allocated to this type of processor is of two clear types, that associated with the media player functionality and that associated with the implant processes. Clearly, there are tasks that the one or more processors within such a device can be done simultaneously and as such the body worn device is be designed to perform them accordingly.
In the example depicted in Fig. 3, the recipient's speech processor is adapted to initiate the process of receiving digital audio signals representing an Internet news broadcast 330 sourced from a third party vendor supplier, that has been customized for the recipient by a proprietary web hosted computer 322. The recipient's PC then downloads this data to the recipient's body worn media player and speech processor 328 via the Internet and/ or an associated web wireless hotspot service 320 and then the wireless link, provided in this example by a Bluetooth 332 data transceiver device fitted to both the PC 318 and the body worn device 328. The speech processor in the body worn device translates received sound data into neural stimulation data that is then delivered in a timely manner to the recipient's implanted neural stimulator in the manner briefly described previously.
Referring further to Fig. 3, upon the recipient entering an area that is signposted as an Internet wireless access "hotspot", the recipient may signal
to the body worn device that wireless access is available by pressing a button on their speech processor. This causes the electronic controller incorporated within the body worn device, which is part of their cochlear implant, to energise an inbuilt wireless link transceiver that may for example be in the form of a WAN service, which in turn determines whether an appropriate signal is present and whether the recipients body worn device has Internet access.
Prior to this action, the recipient has accessed a dedicated website to establish an account and authorize online payment for one or more Internet audio programs such as for example, CNN or BBC World News.
At the same time, the URL or IP address of this subscriber accessed dedicated website is conveyed via a wireless data link to the recipient's cochlear implant, where is it stored in a volatile data memory along with downloaded software that enables the cochlear implant to send and receive data via the Internet, for example including but not limited to, using TCPIP and FTP protocols.
By operating a control button on their body worn speech processor 328, the recipient causes their cochlear implant to recall the previously stored URL or IP address and subsequently gain access to this dedicated website.
Other data, including one or more of, but not limited to; the recipient's ID, their prosthesis ID and associated parameters, are conveyed to the computer host 322. The computing resource at the host location in turn accesses the recipient's choice of audio material, which in this instance happens to be BBC World News as might be supplied in a streaming format from www.bbc.co.uk
Data representing the sound of the chosen news service is returned to the computer hosting the dedicated website. The said digital audio data is then processed in a manner that customizes it for the benefit of the recipient's listening experience, taking account of the recipient's cochlear implant, its various operating parameters and recipient preferences and settings. As an example, such processing may be used to remove all musical content as well as to increase the pitch frequency of a newsreader's voice in order to aid a recipient's understanding.
This processed data, together with other data describing it's content, format, rate and or time of delivery, as well as the recipient's ID and that of their prosthesis, is then conveyed to the recipient's body worn speech processor via the Internet and the Internet wireless hotspot service.
On receiving this data, the recipient's prosthesis authenticates the data and determines readiness and suitability for delivery of the data as neural stimulation to the recipient via their implant. If no further processing or customisation of the data is required, the data is then conveyed in a timely manner to the recipient's implanted neural stimulator. By way of example only, about 30 seconds of sound data storage is provided temporarily within the cochlear implant, in order to avoid interruptions due to temporary breaks in the data transmission path between vendor and recipient. Timing information included with the sound data is used to ensure the data is presented to the recipient at the appropriate rate.
If further processing is indicated, then hardware and software incorporated within the recipient's speech processor performs this as required before the data is conveyed to the implanted part of the cochlear implant.
Fig.4 depicts a block diagram of a computer device for processing, an audio source obtained on-line or from some other source, in such a way that it is suitable for a cochlear implant recipient. In this embodiment the personal computer 400 (PC or PDA or other suitable computer device) has previously obtained a source of audio material and it is has been stored in the memory (fixed or removable) associated with the computer. The audio source may have been obtained from a service available on the Internet 402 or the source may be a pre-recorded audio track available from a CD or other media purchased from a store, or from a multitude of other sources.
The processing of the audio source into a form suitable for its use by a cochlear implant device is very specialised and very processor intensive. The computer device 400 needs to be capable of executing the program 404 that performs that function. The program can be downloaded to the computer or it can be provided on digital memory media, which is shown by way of example as a CD in Fig. 4.
There may be many ways of processing the source dependent not only the type of source it is to begin with, but more particularly dependent on the cochlear implant device characteristics and the recipient's preferences. The recipient specific parameters such as ID, stimulation MAP data and user preferences may have influence on the type and extent of processing performed.
In the embodiment depicted in Fig. 4 the processed audio is provided to a media player that may further process the audio or may merely store the processed or even partially processed audio for playback at the time or at a later time. In Fig. 4 the media player communicates to the over the ear sound processor 408 which provides a modulated radio frequency signal so as to
transmit 410 the cochlea stimulus signals to the implanted portion of the cochlear implant device.
Fig. 5 depicts a block diagram of an alternative arrangement of the system depicted in Fig. 4 wherein the recipient uses a body worn device to deliver the processed audio to their cochlear implant so that the stimulus data rate can be higher as can be the radio frequency radiation strength, thereby improving stimulus effectiveness and quality. To facilitate these improvements different protocols to control the data flow rate and quality may be used.
Fig. 6 depicts a block diagram of a computer device for processing an audio source on a storage medium in such a way that it is suitable for a cochlear implant recipient. In the embodiment depicted communication is provided directly from the computer device (which as stated previously could be a personal computer or a PDA or a mobile phone or other computationally suitable computer device) to a wireless enabled over the ear sound processor device. Communications is preferably achieved using a wireless connection, which for example could be based on IEEE 802.11 standards (WiFi) or Bluetooth or other low power wireless communications technology, otherwise a wired connection although more physically restrictive is possible.
Fig. 7 depicts a block diagram of a prior art cochlear implant system 700, mainly to illustrate that the input to such a device receives air compression and rarefaction sound waves that are transformed into electrical signals by one or more microphones 702 located on the exterior of an over the ear or body worn audio processor. The user may also input analogue electrical audio signals directly to the audio front-end 704. The output from the audio front-end is provided to a signal analysis module 706 that may perform one or more signal transformations including Fast Fourier Transforms and feature
extraction algorithms. The specific audio frequencies are mapped 708 to specific channels that are destined to stimulate one more cochlea located electrodes in one or more combinations at one or more stimulation rates at one or more amplitudes. The previously described processing is performed external to the implant recipient and then transferred to the implanted portion 710 using modulated radio frequency transmission generated by the external portion of the cochlear device. The radio frequency transmission not only provides the modulated codes containing stimulation information but the radio frequency power is captured and reused to power the electrode 712 stimulus as a current 714 of required duration and strength.
Fig. 8 depicts a block diagram of system elements for supplying and processing an audio source obtained on-line in such a way that it is suitable for multiple cochlear implant recipients with the majority of processing done external of the cochlear implant device. Not only is the processing done externally of a particular cochlear implant recipient but external of a multitude of cochlear implant recipients 1 and 2 in this embodiment.
An audio source 802 and 804 as previously described is provided to the processor server located at a host server, which may be for example, owned and operated by a cochlear device manufacturer, where further processing 806 and 808 is performed in accordance with the parameters and preferences of one or more cochlear implant recipients. The processing associated with the channel to electrode mapping in the illustrated embodiment more readily performed by the remote server while also in the illustrated embodiment amplitude mapping is performed in the user cochlear implant device.
Fig. 8 illustrates two users 1 and 2 each having a user sound processor 808- and 808-2 respectively in the body worn or over the ear external portions of
the cochlear implant device. The implanted portions 810-1 and 810-2 respectively of the cochlear implant device include as previously noted electrodes 812-1 and 812-2 respectively and current sources 814-1 and 814-2 respectively.
VARIOUS ENHANCMENTS Internet Enabled Prosthesis with stored URLs
In this enhancement, the hearing prosthesis is configured with the ability to convey and receive electronic data via the Internet and or other intervening, wired and wireless pathways. This conveyance includes the exchange of IP or URL data that allows the cochlear implant to locate and communicate with one or a number of specific Internet web sites.
Contact Number Storage In this enhancement, the cochlear implant is provided with the ability to store and retrieve data that may include, but is not limited to, a national and or international telephone number, the Internet IP address or unique resource locator of one or a plurality of Internet websites.
Prosthesis authorised financial transactions
In this enhancement, the hearing prosthesis is provided with the ability to convey recipient asserted data or commands from the recipient to a remote device. As an example this could extend to the provision of a pressure sensitive actuator incorporated into an cochlear implant device whereby the recipient by pressing this control initiates a wireless message from the device so as to signal authorisation for the remote purchase of goods from an Internet site, so configured as to accommodate such purchasing.
Proprietary Soundtrack Customisation
In this enhancement, the there is provided a process where recorded media containing movie sounds tracks are processed by the cochlear implant manufacturer, or other parties such that volume and or tonal qualities of accompanying soundtrack music and environmental sounds are customised, reduced of removed completely. Whether this is achieved through processing of the sound data, or in obtaining and remixing the original elements of the sound track, or the reconstitution of speech from subtitle text included with the material the product is being modified in a way that enhances the enjoyment of a cochlear implant recipient. This type of processing may include the provision of only music that will enhance the recipient's watching pleasure since the speech aspect of the recorded media is displayed in the form of sub-titles.
Prosthesis capture and storage of recipient actions In this enhancement, the hearing prosthesis is configured to capture information including a recipient's preferences and/ or selection and or listening habits and history of audio material conveyed to them via a cochlear implant. In this way, a recipients selection preferences, purchase and listening history of audio material downloaded via the Internet, could be stored within a part of their prosthesis system for conveyance to another party such as a music program vendor as required.
Data Security and Integrity
An implantable hearing prosthesis, or any part there of, including it's support environment, able to encrypt, decrypt, store, retain, retrieve, compress, decompress, convey and validate electronic data in a manner that may be used to increase the privacy, security and integrity of any type of data or information pertaining to a recipient, their prosthesis or any part there of, including that -which may control the electrical currents that may be applied
to the recipient, their healthcare clinic, their bank or credit card provider, their password or information conveying their authorisation to transfer money or financial merit for the purpose of obtaining goods and services.
Conveyance and Handling of Stimulus Data.
A hearing prosthesis or part there of, configured to receive and or convey neural stimulation data requiring little or no extra data processing before the data can be translated into one or more electrical neural stimulation currents for delivery to a recipient so as to evoke an approximation of sound perception.
Distributed sound data processing and customisation
An Internet website configured to allow an implantable cochlear implant recipient to browse, select, purchase, process and download data representing audio, or audio plus visual information, whereby such processing takes account of uploaded and or locally stored recipient attributes, to benefit the recipient's listening experience. This said audio or audio plus visual data may include, but is not limited to; diagnostic, therapeutic, habilitation, educational and entertainment material
Processing data related to a recipient and prosthesis A local or remote computer system that is configured to process and download data representing sound using one or a plurality of specific attributes that pertain to an cochlear implant, their guardian, carer or health care professional.
Data that includes title and summary information
A cochlear implant system data storage partition of storing a description, summary or extract of the material selected by a recipient can be retrieved
from the data store for presentation to someone other than the recipient. This feature is intended allow a recipient's friend, guardian, carer or health care professional to share, understand or appreciate the nature of material selected by the recipient.
Network Accessible Memory
An cochlear implant system or component parts there of, configured with an area of protected, volatile and or non volatile data and or program memory which is locally accessible to the cochlear implant system, and remotely accessible via a network, such as, but not limited to the Internet. This protected area or memory may be accessed by a remote source to store, retrieve and or execute software and or data so as to benefit a recipient in a safe manner.
Dedicated website storage of recipient & prosthesis data
An Internet website or other remotely hosted computing resource with a capacity to store electronic data that is configured in a manner that retains data from a recipients cochlear implant system, such that the recipient and or their cochlear implant system may receive this memory capacity for any purpose relating to the operation of the said cochlear implant system and or data that is conveyed by the cochlear implant system. Such data could include the online Internet browsing history of the recipient; such that they could press a back button several times in order to access previously Internet accessed audio content. Likewise this same site or the recipient's cochlear implant system can store Internet like bookmarks as is performed by most Internet browsing programs.
Cochlear implant system with "Cookie" storage & handling
The cochlear implant system can be configured to capture, store, retrieve and convey what is generally referred to as Internet cookies. Such data or pieces of information conveyed for storage by the prosthesis or part there of, from any source, allow data such as that from Internet web sites and or accessories used in association with the prosthesis, to be exchanged.
A cookie can be stored in a cochlear implant system by an Internet web site while it is being accessed by the recipient for audio content, might be used to identify the recipient and their purchase history when next they next visit that particular web site. Likewise a recipient's listening history of a particular type of audio content or that supplied by a particular vendor could be captured and stored by a cochlear implant system, then conveyed back to the vendor when next that cochlear implant system has Internet access.
The last used volume settings of a body-worn element of a cochlear implant system, could for example be stored within a recipient's implant so that when they switch to their ear worn or outdoor use (i.e. spill proof/ water proof) speech processor, the stored settings are automatically transferred to the newly selected processor.
Sound Data "watermark" label and tracking system The storage and or conveyance of sound information or data in any form, which is labelled or accompanied by information or data that allows this sound information or data to be identified in some manner. In this way sound or audio material could be; conveyed as sound or electronic data, processed, modified, stored or retrieved, in any manner while still retaining this identifying information.
When for example, a recipient plays an item of audio material through the speakers of their CD player, low level, high frequency acoustic tones, with an accurately defined frequency, included at the beginning and or end the material could be received by the microphone of the cochlear implant system, then decoded and stored by the cochlear implant system as an identifying number. Any sound data processing applied to refine this data for the benefit of a recipient could be configured so as to retain this identifying data so that it may be reconstituted after such said sound processing. When this audio is presented to the recipient in any form, the material is identified by the cochlear implant system, which in turn can be made to keep track of the date and time this audio material was listened to. This data pertaining to the recipient's listening habits and choice of material could then be conveyed by any means including the Internet, to the vendor of the audio material or to the recipient's healthcare professional, on request or as required.
Cochlear implant system with universal or configurable interface A cochlear implant system or any part or parts there of, can be configured in a manner that allows it to access any other computing resource via a wired and or wireless link such as, but not limited to the RS232, USB, USB2, Firewire, I2C, IRda, Bluetooth, acoustically coupled DTMF tones.
Prosthesis with memory capacity meter
A cochlear implant system can be configured to convey via any wired or wireless network, including the Internet, information describing any or all of the following attributes; its total and or remaining data and or program and or application storage capacity as pertaining to any temporary, permanent and or removable, volatile and or non volatile electronic data storage memory.
Prosthesis with battery life meter
The cochlear implant system can be configured to convey via any wired or wireless network, including the Internet, information describing any or all of the following attributes; its total and or remaining battery life and or energy delivery capacity.
The present invention may be implemented using hardware, software or a combination thereof and may be implemented in one or more computer systems or other processing systems. In the embodiments described herein, the invention is directed toward one or more computer systems capable of carrying out the functionality described herein. Various software embodiments are described in terms of the example computer systems. After reading this description, it will become apparent to a person skilled in the relevant art(s) how to implement the invention using other computer systems and/ or computer architectures.
In this specification, the terms "computer program medium" and "computer useable medium" are used to generally refer to media such as removable storage drive, a hard disc installed in a removable hard drive bay, and storage media such as compact discs and digital versatile discs. These computer program products are a means for providing executable software and data to the computer systems described.
In an embodiment where the invention is implemented using software, the software may be stored in a computer program product and loaded into the computer system using removable storage drive, hard disc and storage media. The software may be communicated from a remote source and stored in the computer system. The computer software contains instructions to be performed by the processor/ s in the computer system.
In alternative embodiments, the invention is implemented in hardware using, for example, hardware components such as application specific integrated circuits (ASIC's). Implementation of the hardware state machine so as to perform the functions described herein will be apparent to persons skilled in the relevant art(s).
The reference to any prior art in this specification is not, and should not be taken as, an acknowledgment or any form of suggestion that such prior art forms part of the common general knowledge.
It will be appreciated by those skilled in the art that the invention is not restricted in its use to the particular application described. Neither is the present invention restricted in its preferred embodiment with regard to the particular elements and/ or features described or depicted herein. It will be appreciated that various modifications can be made without departing from the principles of the invention. Therefore, the invention should be understood to include all such modifications within its scope.
Claims
1. A signal processing system for supplying signals of a form suitable for at least one cochlear implant system including an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal including: a digital signal processor receiving said digitized source audio signal, the digital signal processor adapted to modify the digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the digitized source audio signal including at least spectral and temporal content suitable for use with the predetermined implanted cochlear device.
2. A signal processing system according to claim 1 further including; a digitized audio signal distribution processor operatively connected to said digital signal processor for making said output form of the source audio signal, available to at least one cochlear implant system.
3. A signal processing system according to claim 1 wherein the digital signal processor is configured to provide an output form of the source audio signal using a mapping for a predetermined implanted cochlear device and further wherein the cochlear implant system includes: a body worn or over the ear digital signal processing device adapted to receive said output form of the source audio signal and transform the output form to a signal suitable for the predetermined implanted cochlear device.
4. A signal processing system according to claims 3 wherein the body worn or over the ear signal processing device further comprises: a memory device for storing and selectively outputting one or more output forms of respective source audio signals.
5. A signal processing system according to claim 2 wherein said digitized audio signal distribution processor is a networked computer capable of communicating a stored version of said output form or a real time version of said output form of the source audio signal to another networked computer including a body worn or over the ear signal processing device.
6. A signal processing system according to claim 1 wherein the digital signal processor includes at least one digital filter.
7. A signal processing system according to claim 5 wherein said digital filter modifies the frequency content of the source audio signal during the process of modifying the digitized audio signal according to the predetermined setting.
8. A signal processing system according to claim 1 wherein the source audio signal includes cochlear implant specific auditory rehabilitation audio signals.
9. A method for supplying signals of a form suitable for at least one cochlear implant system including an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal the method steps including: digital signal processing the digitized source audio signal to modify the digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the digitized source audio signal including at least spectral and temporal content suitable for use with the predetermined implanted cochlear device.
10. A method for supplying signals according to claim 9 further includes the step; operatively connecting said digital signal processor to a digitized audio signal distribution processor for making said a predetermined output form of the source audio signal available to a predetermined cochlear implant device.
11. A method for supplying signals according to claim 9 further includes the step; configuring the digital signal processor to provide an output form of the source audio signal using a mapping for a predetermined implanted cochlear device wherein the predetermined cochlear implant system includes a body worn or over the ear digital signal processing device adapted to receive said output form of the source audio signal and transform the output form to a signal suitable for the predetermined implanted cochlear device.
12. A method for supplying signals according to claim 11 further includes the step; storing and selectively outputting one or more output forms of respective source audio signals in a memory device in the body worn or over the ear signal processing device.
13. A method for supplying signals according to claim 10 further includes the step; communicating a stored version of said output form or a real time version of said output form of the source audio signal from said digitized audio signal distribution processor to a networked computer including a body worn or over the ear signal processing device.
14. A method for supplying signals according to claim 9 further includes the step; applying at least one digital filter in the digital signal processor to the digitized source audio signal.
15. A method for supplying signals according to claim 14 wherein the digital filter modifies the frequency content of the source audio signal during the process of modifying the digitized audio signal according to the predetermined setting.
16. A method for supplying signals according to claim 9 wherein the source audio signal includes cochlear implant specific auditory habilitation audio signals.
17. A modified audio signal distribution system for signals suitable for at least one cochlear implant system including an implanted cochlear device wherein a digitized source audio signal is a digitized stored audio signal or a digitized real time audio signal, the system including: a first networked computer having multiple stored digitized source audio signals accessible from the network; a second networked computer having access to the first networked computer and adapted to process one or more of the multiple stored digitized source audio signals to include at least spectral and temporal content suitable for a predetermined cochlear implant system including a predetermined implanted cochlear device; and a third networked computer having access to at least the second networked computer and associated with a predetermined cochlear implant system, for receiving the one or more multiple stored digitized source audio signals including at least spectral and temporal content suitable for the predetermined cochlear implant system.
18. A modified audio signal distribution system according to claim 17 further including: a fourth networked computer adapted to authorise a financial transaction between at least the user of the second networked computer and one or more other parties.
19. A modified audio signal distribution system according to claim 17 further including: a fifth networked computer associated with a predetermined cochlear implant system for receiving and storing the one or more multiple stored digitized source audio signals including at least spectral and temporal content suitable for the predetermined cochlear implant system before providing the one or more multiple stored digitized source audio signals to the predetermined cochlear implant system.
20. A body worn or over the ear digital signal processing device for use with an implanted cochlear device for receiving an output form of a digitized source audio signal generated by a digital signal processor that modifies a digitized source audio signal according to a predetermined setting that converts the digitized source audio signal to an output form of the source audio signal including at least spectral and temporal content suitable for use with an implanted cochlear device wherein said digital signal processor is physically remote the user of a cochlear implant system the body worn or over the ear digital signal processing device including: a receiver adapted to receive the output form of the digitized source audio signal, a digital signal processor to transform the output form to a signal suitable for a predetermined cochlear implant device implanted in the user.
21. A body worn or over the ear digital signal processing device according to claim 20 further including: a digital store for storing at least one received output form of a digitized source audio signal and for making a stored received output form available to the digital signal processor.
22. A body worn or over the ear digital signal processing device according to claim 20 further including: a radio signal generator and modulator for forming a radio signal modulated according to a protocol suitable for communicating the output form of a digitized source audio signal to the predetermined implanted cochlear device, a radio signal transmitter for transmitting the modulated radio signal to the predetermined implanted cochlear device.
23. A body worn or over the ear digital signal processing device according to claim 20 having a device address addressable by a networked computer.
24. A body worn or over the ear digital signal processing device according to claim 23 wherein the address is an address that conforms to Internet Protocol addressable by a networked computer.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/AU2007/000110 WO2008092183A1 (en) | 2007-02-02 | 2007-02-02 | Organisational structure and data handling system for cochlear implant recipients |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/AU2007/000110 WO2008092183A1 (en) | 2007-02-02 | 2007-02-02 | Organisational structure and data handling system for cochlear implant recipients |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2008092183A1 true WO2008092183A1 (en) | 2008-08-07 |
Family
ID=39673551
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/AU2007/000110 WO2008092183A1 (en) | 2007-02-02 | 2007-02-02 | Organisational structure and data handling system for cochlear implant recipients |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2008092183A1 (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010068997A1 (en) * | 2008-12-19 | 2010-06-24 | Cochlear Limited | Music pre-processing for hearing prostheses |
WO2014108201A1 (en) | 2013-01-11 | 2014-07-17 | Advanced Bionics Ag | Method and system for neural hearing stimulation |
US9473852B2 (en) | 2013-07-12 | 2016-10-18 | Cochlear Limited | Pre-processing of a channelized music signal |
US10124168B2 (en) | 2014-07-07 | 2018-11-13 | Advanced Bionics Ag | System for combined neural and acoustic hearing stimulation |
US10376698B2 (en) | 2014-08-14 | 2019-08-13 | Advanced Bionics Ag | Systems and methods for gradually adjusting a control parameter associated with a cochlear implant system |
CN110755742A (en) * | 2019-10-18 | 2020-02-07 | 上海力声特医学科技有限公司 | Cochlear implant system and method for operating the same |
CN112151056A (en) * | 2020-09-27 | 2020-12-29 | 浙江诺尔康神经电子科技股份有限公司 | Intelligent cochlear sound processing system and method with customization |
US10904680B2 (en) | 2017-02-23 | 2021-01-26 | Advanced Bionics Ag | Battery-based systems and methods for managing sound processor programming for a cochlear implant system |
US20210264937A1 (en) * | 2018-07-25 | 2021-08-26 | Cochlear Limited | Habilitation and/or rehabilitation methods and systems |
WO2022117892A1 (en) * | 2020-12-04 | 2022-06-09 | Axiles Bionics Bv | Data transfer method for a prosthesis or orthosis and system thereof |
US11491331B2 (en) | 2007-05-31 | 2022-11-08 | Cochlear Limited | Acoustic output device with antenna |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5479522A (en) * | 1993-09-17 | 1995-12-26 | Audiologic, Inc. | Binaural hearing aid |
US20020068986A1 (en) * | 1999-12-01 | 2002-06-06 | Ali Mouline | Adaptation of audio data files based on personal hearing profiles |
US20030097037A1 (en) * | 2001-11-21 | 2003-05-22 | Miller Douglas Alan | Method and apparatus for audio input to implantable hearing aids |
US20040073275A1 (en) * | 2002-10-11 | 2004-04-15 | Maltan Albert A. | Cochlear implant sound processor with permanently integrated replenishable power source |
US20050031145A1 (en) * | 2001-10-03 | 2005-02-10 | Maltan Albert A. | Novel hearing aid design |
US7024000B1 (en) * | 2000-06-07 | 2006-04-04 | Agere Systems Inc. | Adjustment of a hearing aid using a phone |
US20060227987A1 (en) * | 2005-04-08 | 2006-10-12 | Phonak Ag | Hearing device with anti-theft protection |
-
2007
- 2007-02-02 WO PCT/AU2007/000110 patent/WO2008092183A1/en active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5479522A (en) * | 1993-09-17 | 1995-12-26 | Audiologic, Inc. | Binaural hearing aid |
US20020068986A1 (en) * | 1999-12-01 | 2002-06-06 | Ali Mouline | Adaptation of audio data files based on personal hearing profiles |
US7024000B1 (en) * | 2000-06-07 | 2006-04-04 | Agere Systems Inc. | Adjustment of a hearing aid using a phone |
US20050031145A1 (en) * | 2001-10-03 | 2005-02-10 | Maltan Albert A. | Novel hearing aid design |
US20030097037A1 (en) * | 2001-11-21 | 2003-05-22 | Miller Douglas Alan | Method and apparatus for audio input to implantable hearing aids |
US20040073275A1 (en) * | 2002-10-11 | 2004-04-15 | Maltan Albert A. | Cochlear implant sound processor with permanently integrated replenishable power source |
US20060227987A1 (en) * | 2005-04-08 | 2006-10-12 | Phonak Ag | Hearing device with anti-theft protection |
Cited By (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11491331B2 (en) | 2007-05-31 | 2022-11-08 | Cochlear Limited | Acoustic output device with antenna |
US12011593B2 (en) | 2007-05-31 | 2024-06-18 | Cochlear Limited | Acoustic output device with antenna |
US11819690B2 (en) | 2007-05-31 | 2023-11-21 | Cochlear Limited | Acoustic output device with antenna |
US9042583B2 (en) | 2008-12-19 | 2015-05-26 | Cochlear Limited | Music pre-processing for hearing prostheses |
WO2010068997A1 (en) * | 2008-12-19 | 2010-06-24 | Cochlear Limited | Music pre-processing for hearing prostheses |
WO2014108201A1 (en) | 2013-01-11 | 2014-07-17 | Advanced Bionics Ag | Method and system for neural hearing stimulation |
US9473852B2 (en) | 2013-07-12 | 2016-10-18 | Cochlear Limited | Pre-processing of a channelized music signal |
US9848266B2 (en) | 2013-07-12 | 2017-12-19 | Cochlear Limited | Pre-processing of a channelized music signal |
US10124168B2 (en) | 2014-07-07 | 2018-11-13 | Advanced Bionics Ag | System for combined neural and acoustic hearing stimulation |
US11642535B2 (en) | 2014-08-14 | 2023-05-09 | Advanced Bionics Ag | Systems and methods for gradually adjusting a control parameter associated with a cochlear implant system |
US11154720B2 (en) | 2014-08-14 | 2021-10-26 | Advanced Bionics Ag | Systems and methods for gradually adjusting a control parameter associated with a cochlear implant system |
US10376698B2 (en) | 2014-08-14 | 2019-08-13 | Advanced Bionics Ag | Systems and methods for gradually adjusting a control parameter associated with a cochlear implant system |
US10904680B2 (en) | 2017-02-23 | 2021-01-26 | Advanced Bionics Ag | Battery-based systems and methods for managing sound processor programming for a cochlear implant system |
US11889269B2 (en) | 2017-02-23 | 2024-01-30 | Advanced Bionics Ag | Systems and methods for remote loading of a sound processing program onto a sound processor included within a cochlear implant system |
US20210264937A1 (en) * | 2018-07-25 | 2021-08-26 | Cochlear Limited | Habilitation and/or rehabilitation methods and systems |
US12009008B2 (en) * | 2018-07-25 | 2024-06-11 | Cochlear Limited | Habilitation and/or rehabilitation methods and systems |
CN110755742A (en) * | 2019-10-18 | 2020-02-07 | 上海力声特医学科技有限公司 | Cochlear implant system and method for operating the same |
CN112151056A (en) * | 2020-09-27 | 2020-12-29 | 浙江诺尔康神经电子科技股份有限公司 | Intelligent cochlear sound processing system and method with customization |
CN112151056B (en) * | 2020-09-27 | 2023-08-04 | 浙江诺尔康神经电子科技股份有限公司 | Intelligent cochlea sound processing system and method with customization function |
WO2022117892A1 (en) * | 2020-12-04 | 2022-06-09 | Axiles Bionics Bv | Data transfer method for a prosthesis or orthosis and system thereof |
NL2027046B1 (en) * | 2020-12-04 | 2022-07-06 | Axiles Bionics Bv | Data transfer method for a prosthesis or orthosis and system thereof |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2008092183A1 (en) | Organisational structure and data handling system for cochlear implant recipients | |
US11848022B2 (en) | Personal audio assistant device and method | |
EP1216599B1 (en) | Producing and storing hearing profiles and customized audio data | |
US7181297B1 (en) | System and method for delivering customized audio data | |
CN101237814B (en) | Digital playback device, method and apparatus for spectrally modifying a digital audio signal | |
EP2109934B2 (en) | Personalized sound system hearing profile selection | |
CN105409243B (en) | The pretreatment of channelizing music signal | |
US20100208631A1 (en) | Inaudible methods, apparatus and systems for jointly transmitting and processing, analog-digital information | |
TW201216255A (en) | Method and system for self-managed sound enhancement | |
WO2007019446A2 (en) | Secure telerehabilitation system and method | |
US20080107294A1 (en) | Programmable Hearing Health Aid Within A Headphone Apparatus, Method Of Use, And System For Programming Same | |
US20180082672A1 (en) | Information processing apparatus and information processing method thereof | |
WO2012114169A1 (en) | Method and system for audio dubbing distribution | |
Bosman et al. | Evaluation of an abutment‐level superpower sound processor for bone‐anchored hearing | |
WO2008092182A1 (en) | Organisational structure and data handling system for cochlear implant recipients | |
CN105284095A (en) | A media player for receiving media content from a remote server | |
CN106998390A (en) | A kind of alarm clock setting method and device | |
Thibodeau | Five important questions about FM systems and Cochlear implants | |
KR20040048551A (en) | Method of service for education and therapeutic sound contents via the Internet using Study Pillow combined with the Function of MP3 Player plus Bone Conduction Listening Head Mat | |
KR20100137974A (en) | System for providing externalized sound and method thereof |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 07701442 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 07701442 Country of ref document: EP Kind code of ref document: A1 |