IMPROVED LISTENING ENHANCEMENT SYSTEM AND METHOD
All new material has been underlined and material that is to be deleted, pending the approval of the Examiner as to reliance on CIP status, is bracketed and in italics
The present invention covers the separation and or blending of predominantly voice[ band ] bandwidth audio from[ background ] remaining audio in an audio program in such a way that the end user can adjust the volume of the mostly voice bandwidth audio and the[ background ] remaining audio separately to his hearing needs or taste The present invention includes a variety of means for separating the majority of voice band from the background audio such as analog or digital bandpass filtering, [ or ]adaptive speech filtering using various digital signal processing techniques or obtaining a previously recorded, mostly voice, signal The applications for this invention include all audio playback or received modalities including but not limited to, video recording, television and radio broadcasting, as well as, tape[ and ]. DVD. CD music recording and specific play back means for adjusting or blending the separate volume levels by the end user While the present invention is intended for the hearing impaired as an improvement over closed caption broadcasting, non hearing impaired listeners can also benefit from this invention [ since most individuals have some minor form of hearing deficiencies] by obtaining a voice to remaining audio mixture that satisfies that specific user
FIELD OF INVENTION
The present invention pertains to the application of mostly separate audio signal controls to adjust the background and voice band components in a[ program] signal to optimize the mixture of these two signals to the listeners personal taste This invention can be utilized with CDs, television, radio, tape recorded audio programming or any kind of audio playback or received modalities but not limited to hearing aids, in which the listener can personally adjust or blend background and voice band signals to optimize the combination of both for his particular hearing needs
BACKGROUND OF THE INVENTION
As one ages and progresses through life, over time due to many factors such as age, genetics, disease, environmental effects and others, ones hearing becomes compromised Most of the time the deterioration is specific to certain frequency ranges Traditionally one compensates for this loss by increasing the volume of the audio But this simply increases the volume[ of the sound frequencies which can be heard while confusing or negating the frequencies that are deficient. Therefore the net effect is a marginal improvement at best in one understanding and appreciation of the audio] of all audible frequencies in the total signal The resulting increase in total signal volume will provide no improvement in speech intelligibility as long as the ratio of voice to background noise remains the same
Dolby discussed a method of creating a "center channel" for dialog in cinema sound in U S Patent No 4,024,344 This technique involved correlating the left and right stereophonic channels and adjusting the gain on either the combined and/or the separate left and right channel depending on the degree of correlation between the left and right channel The assumption being that the strong correlation between the left and right channel would indicate the presence of dialog The center channel, which is[ band passed fιlter]the filtered summation of the left and right channels, would be amplified or attenuated depending on the degree of correlation between the left and right channels The problem with this approach is that the individual listener cannot adjust the degree to which the center channel is amplified, attenuated or blended to optimize his hearing abilities
The separation of voice from background audio in television signals is discussed in Shiraki, U S Patent No 5, 197,100 The technique employed by Shiraki involves the use of band pass filtering in combination with[ summering ] summing and subtracting circuits to form a " voice channel" that would be differentiated from the rest of the audio programming The problem with the approach of Shiraki is that the individual listener cannot adjust the degree to which the center channel is amplified, attenuated or blended to optimize his hearing abilities
O 99/08380
This invention contemplates the use of "pure voice" as recorded in a studio and its mixing with whatever background is desired, e g, an actual or simulated crowd noise at a football game while the announcer, in a soundproof booth, records the play-by-play of the game The separate tracks are separately provided and mixed by the end user, not by any audio engineer, so as to better provide quality and intelligible listening specifically attuned to the end user's particular aural characteristics Up to the development of the instant invention, consumers have had no input regarding their preferential mix of vocals to remaining audio, whether it be background noise or music Only the audio engineer has had any input by utilizing his or her own preference and bias in mixing the audio inputs for broadcast or recording In general, audio programming for television ( entertainment, educational, informative, etc ) movies and music is created in a manner that is suitable for obtaining the mostly "pure voice" or preferred signal referred to in this disclosure In the studio music recording industry, e g , each component of the overall sound is usually recorded separately Vocals, guitar, bass and drums are all recorded onto separate tracks, independent of each other Such an arrangement permits the audio engineer to mix all components of the sound into a two track stereo recording with complete control over each instrument and voice Each of the other entertainment mediums such as TV and movie production often employ a similar mixing of pre-recorded sound components resulting in a stereo signal with a voice-to-recording audio mix that is ultimately determined by a single person, the audio engineer It should be noted that this approach can be used to mix a pure tone or musical instrument recording with background orchestration to provide a blend suitable to the end user's tastes and ear characteristics For purposes of this disclosure mostly pure voice is recognized as "preferred signal"
SUMMARY OF INVENTION
There are[ several ] many interfering audio signals that make understanding voice program audio difficult for individuals with hearing deficiencies In fact, any remaining audio signal component occurring simultaneously with voiced audio can affect speech intelligibility [ One interfering signal is the] Some examples occurring in general audio background include[s] music, traffic noise, wind, running water, etc A significant portion of these interfering sounds may not reside in the voice frequency band ( roughly 200 to 6000Hz ) and if [filtered] removed from the audio signal by filtering would result in an audio signal which is predominantly voice It is acknowledged, however, that certain background sounds that fall within the voice frequency band will remain along with the voice components If a listener is provided a means of adjusting the volume of the frequency limited audio, relative to the unfiltered audio program, an optimal combination of mostly voice and background is achieved by the end user in a ratio of the unfiltered audio to the frequency limited audio specifically customized for the end user
This invention provides a means for the end listener to adjust and blend the voice band audio and the background audio independently, to compensate for his hearing loss or obtain a more tailored audio program, specific to his or her needs both objects being paramount in this invention The end listener, by way of example only, in this case is the person [watching] listening to his television, video tapeA DVD [ or ] CD program, [or listening to his] music in his home or car , or any other audio programming which contains both a preferred audio signal (such as voice ) and other audio programming
The voice band signal and the full audio signal is adjusted, by way of example only and not limiting the scope of this invention, by sending the two signals to separate variable gain amplifiers that the listener adjusts a feature such as a button, [ with a dial or Jknob, dial or other equivalent adjustment on the audio receiver, tape player , VCR, DVD or CD player, or any other such audio playback or receiving modality such as, but not limited to, hearing aids or a headset which can be controlled by remote control such as by infra-red or by radio signals
OBJECTS OF THE INVENTION
Accordingly, it is an object of this invention to provide a listener of audio of whatever variety the ability to adjust the mostly voice band audio relative to the [background ] remaining audio for his particular hearing needs
It is another object of this invention to provide a system and method for splitting the audio signal of any reproducing or broadcast medium into a total audio signal and a voice frequency band limited signal to allow the listener to adjust one signal relative to the other to satisfy his own particular hearing needs
Yet another object of this invention is to provide a novel circuit means for splitting an audio signal into two components, one containing mostly voice band and the other [background] remaining audio, which can be background noise, and providing an adjustment capability whereby the listener can adjust the total signal, later summed, to his own needs including those incident to hearing loss in certain frequency ranges
Still another object of this invention is to provide a system and method for the separation and blending of voice band audio from background or remaining audio in an audio program in such a way that the end listener can adjust the volume of the mostly voice band audio and the background separately to satisfy his hearing needs or taste
It yet another object of this invention to provide a system for separating the voice band audio from the background audio in an audio program of any nature using a variety of means such as analog or digital bandpass filtering or adaptive speech filtering using digital signal processing techniques
A further object of this invention is to provide mostly pure voice to an end user for his or her blending with other audio, whether the other audio be background, music or another blend of audios
An added object of this invention is to provide a system and method by which an end user can blend mostly pure voice with other audio to "fine tune" the mix relationship to the particular characteristics of his or her own ear
Still a further object of this invention is to provide a method of obtaining mostly pure voice from standard audio formatting techniques prior to it being mixed with other audio by the audio engineer
These and other objects of this invention when reference is had to the accompanying description and drawings, in which
Figure 1 is a schematic of the system using a band pass filter to create a primarily voice band signal that is adjusted together with an unfiltered audio background signal, and
Figure 2 is a schematic of the system of this invention utilizing an adaptive speech filter, and
Figure 3 is a schematic of the system of this invention using an analog band pass filtering with Butterworth filters, and
Figure 4 is a schematic of the system utilizing a digital components including an anti-aliasing filter, converters, low pass filters, attenuators and processing compressors, and
Figure 5 is a schematic of the system showing the use of a band stop filter to create a background audio signal
DETAILED DESCRIPTION OF THE INVENTION
Figure 1 illustrates the use of band pass filtering to create a primarily voice band signal that is adjusted, together with the unfiltered audio signal to provide the listener with the ability to adjust the amplitude of the voice bandwidth limited signal with the full spectrum audio The incoming audio signal 1 is split [with part of the signal going to a] such that the same signal goes to a voice band pass filter 2 and [part going directly] to a variable gain amplifier 3 The band pass filtering is most easily accomplished using multistage analog filters built from discrete components ( such as resistors, capacitors and op- amps ) Such a filter design is commonplace and readily available to those skilled in the art Alternatively, the band pass filtering can be done using digital signal processing [77z/.y requires converting the analog audio signal to a digital signal and using an FFT algorithm on the incoming audio signal and subsequently performing a convolution with a band pass filter function ] Digital filtering can be performed in either the time domain or frequency domain In the time domain the sampled input can be continuously convolved with the coefficient of an FIR filter or filtered directly with an IIR filter Frequency domain filtering is more complicated and requires methods to avoid circular convolutions Again, filtering techniques with digital signal processors ( DSPs) are well known and available to those skilled in the art After the signal is filtered it is fed into a variable gain amplifier 4 enabling the user to adjust the amplitude of this signal relative to the unfiltered audio signal which is also adjustable The signals from the variable gain amplifiers are then sent to a summing amplifier 5 to produce the[ final ] user adjusted audio output
Fig 2 illustrates a technique for creating the voice signal using an adaptive speech filter Several adaptive speech filters exist in the open literature and have met with varying success in reducing and/or eliminating background noise from the speech portion of an audio signal The incoming audio signal 6 is again split [ A portion of the ] The signal is sent to a band pass filter 7 to remove the information below and above the frequency band information The filtered signal is then sent to an adaptive filter 8 to remove the stationary audio from the voice audio which is in general nonstationary audio
The output from the adaptive speech filter is then sent to a variable gain amplifier 9 for adjustment of the amplitude by the listener The unfiltered [portion ] of the signal is sent through a delay 10 so that the signals are in phase when recombined The output from the delay is then sent to a variable gain amplifier 1 1 to adjust its volume relative to voice band signal Both signals are summed with a summing amplifier 12 to produce the final audio output There are many so-called "speech extraction" algorithms currently on the market and still in development stages This invention does not seek to create such an algorithm Rather, the intent of this invention is to employ existing speech extraction algorithms for the purpose of creating a signal that is used to allow the end user of such an algorithm to adjust the volume level of its output with respect to its input ( the total audio signal ) or any remaining audio signal
Figure 3 illustrates an example of an analog approach to band pass filtering using Butterworth filters [ Butterworth filters are often used as the best compromise between flatness of response in the pass band and minimal phase delay ] Butterworth filters provide a design solution to filtering whose magnitude response is maximally flat in both the passband and stopband In figure 3 an audio signal 13 [ is split, part of the signal ] goes through a five pole high pass Butterworth filter 14 with a corner frequency determined by specific values of the resistors and capacitors in the circuit For a frequency cutoff of approximately 200 Hz, the component values are as follows
This high pass filter produces the lower corner frequency of the band pass filter, rapidly attenuating any frequencies below 200 Hz The signal output from the high pass filter is then sent to a low pass filter 15 This filter is also a five pole Butterworth filter For a cut off frequency of approximately 3000 Hz, the following component values are chosen
C3 00012μF
This low pass filter forms the upper [frequency] corner frequency of the band pass filter, in this case 3000 Hz This example is similar to telephony applications where the full range of voice frequencies are limited to just the range required intelligibility The selection of this range should not in anyway limit the scope of the instant invention Each [file] five pole filter produces approximately 30 dB of attenuation per octave, which should provide sufficient isolation of the desired voice band audio signal
The filter design discussed above is provided for example only, and in no way limits the bandpass filter approach discussed herein, to a ten pole Butterworth design Other filter designs and orders can be easily implemented for providing a user adjusted, mostly voice signal, without loss of generality
The output of the low pass filter is then sent to a variable gain amplifier 16, which is used to adjust the volume of the voice band limited audio signal[,] [ a] A separate variable gain amplifier 17 is used to adjust the volume on the unfiltered audio signal The two signals are then added together at a summing junction of another variable gain amplifier 18 which controls the volume of the total combined signalf.?] The variable gain amplifier can be[ simple ] implemented using potentiometers with isolation amplifiers or digitally controlled variable gain amplifiers It is understood that many methods of analog filtering are available in the literature and these are all equivalents and the instant example should not be considered the only method for providing such filtering nor is it intended to limit the scope of the present invention
Figure 4 illustrates a digital approach to the current invention Initially the audio signal 20 is sent to an anti-aliasing filter 21 This filter is designed in a manner similar to the low pass stage of the band pass filter from figure 3 The filtered signal is then digitized with [ Sigma Delta ] an A/D converter 22 that converts the time varying voltage into a string of digital pulses whose time varying frequency is representative of the time varying audio signal This signal is sent to digital low pass filter 23 [which converts the] from which a digital [stream into a] 16 bit signal [that can be] is_processed by the DSP 24 The DSP groups the pulses through six digital band pass filters 25, 26, 27, 28, 29, 30 with center frequencies as shown The output from each of these filters is weighted by attenuators 31, 32, 33, 34, 35, 36 that the user can control The three lower frequency weighted signals are added together to create a low pass channel and the three higher weighted channels are summed to produce a high pass channel Each of these channels are processed respectively in a high frequency 37 and a low frequency 38 signal compressor The high frequency compressor provides adaptive gain to provide listening comfort for soft and loud high frequency sounds without disrupting high frequency amplitude contrasts in speech and sudden loud sounds The low frequency compressor provides syllabic compression to provide some differentiation between speech and background sound to control perception of loudness The low [frequencies ] frequency and high frequency signals are then sent to another DSP 39 to be summed and oversampled to be converted back into an analog signal using [ a Sigma Delta ] a digital to analog[ filter 41 ] converter 40 [ to produce a final audio signal 42 This signal is then sent to a speaker or converter 40 ] The output of the converter is then sent to an [ antialiasing output ] low pass smoothing filter 41 to produce a final audio signal 42 This signal is then sent to a speaker or may require additional analog filtering to remove any remaining noise The above description and Figure 4, present a means for subdividing and compressing the voice band limited signal used in the end user adjustment process Although presented in a digital format, each filter and compressor stage can also be realized using analog electronics It is understood that many methods of digital filtering are available in the literature and the example used should not be construed as to be limiting of the scope of the instant invention
Figure 5 illustrates the use of a band stop filter to create a background audio signal, where the voice band frequencies are attenuated or removed from the signal, and a voice band signal that is formed by using a band pass filter as before The incoming audio signal 43, is split with part of the signal going to a band stop filter 44 The output from
the band stop filter is sent to a variable gain amplifier 45 The other part of the split signal is sent to a band pass filter 46 to form the voice band signal The output from the band pass filter is then sent to a variable gain amplifier 47 The listener can adjust the volume of the background signal with a knob that controls the gain of the variable gain amplifier The corner frequencies for the band pass and band stop filters are the same or very similar so that the frequencies that are attenuated by the band stop filter are the same frequencies that are parsed by the band pass filter The signals from the variable gain amplifiers are then sent to a summing amplifier 48 to produce the final audio output 49 The summing amplifier may be adjusted by the listener and be a variable gain amplifier The band pass and band stop filtering is most easily accomplished using multi-stage analog filters built from discrete components ( resistors, capacitors, and op-amps ) Such filter design is well established and readily available to those skilled in the art Alternatively, the band pass filtering can be done using digital signal processing This would require converting the analog audio signal to a digital signal and then performing a convolution with a band pass filter function The filtered signal would then be converted back into an analog signal Again, filtering techniques with digital signal processors are well known and available to those skilled in the art
In the case where mostly pure voice is used, it is possible to obtain the pure voice signal without its being incorporated into an audio program The pure voice signal can be obtained separately from the remaining audio ( voice and background ) signal These two signals are the two signals which are derived or extracted by the methods of bandpass filtering and speech extraction as previously described However, where pure voice is used the two signals are independent of one another and the speech intelligibility of the mostly pure voice depends only on extraneous noise that is present in the vocal track For musical recordings this is equivalent to microphone hiss or electronic noise because most recording studios are sound proofed However, for live sports broadcasts, the announcers microphone will also measure the crowd noise as well as his own voice, and thus speech intelligibility of the "pure voice" track will begin slightly degraded
The method of simultaneous delivery of the mostly voice signal and the remaining audio signal is different for different mediums and is not the specific focus of this invention The instant invention seeks to claim that keeping the voice or vocal
components of any programming separate from the remaining audio programming ( background or non-voiced components ) when the recording process initiates, can drastically improve the ability of the user adjustment hardware to improve speech intelligibility This is primarily because the mostly voice signal that is being adjusted already has excellent intelligibility as a result of the initial recording Once the two signals, mostly voice and remaining audio, have been delivered to a playback device operated by an end user, each signal is then sent through a variable gain amplifier and summed as described earlier for the other methods of obtaining mostly voice
While the instant invention does not have as a specific object to claim new methods of delivering another audio track to the end user, it does contemplate and claim that providing an audio track with mostly pure voice only to an end user for mixing by him or her with remaining audio is unique The ability of the end user to mix audios requires that two signals be delivered to the end user simultaneously This delivery mechanism will be different for different mediums and may include airwaves, such as the additional bandwidth now existing in total aural carrier for television programming Additional tracks on CDs or tapes can be used DVDs now come equipped with multiple tracks for storing alternative information but have no means of allowing end user adjustment of separate signals to his or her hearing preferences
While many changes and modifications can be made to the invention within the scope of the appended claims, such changes and modifications are within the scope of the claims and covered thereby