US20010016813A1

US20010016813A1 - Distributed recogniton system having multiple prompt-specific and response-specific speech recognizers

Info

Publication number: US20010016813A1
Application number: US09/221,582
Authority: US
Inventors: Deborah W. Brown; Randy G. Goldberg; Richard R. Rosinski; William R. Wetzel
Original assignee: AT&T Corp
Current assignee: Nuance Communications Inc; AT&T Properties LLC
Priority date: 1998-12-29
Filing date: 1998-12-29
Publication date: 2001-08-23
Anticipated expiration: 2018-12-29
Also published as: US6377922B2

Abstract

A speech recognition system recognizes spoken utterances received as a speech signal from a user. A prompt for requesting a spoken utterance from the user is assigned a response identifier which indicates at least one of a plurality of speech recognizers to best recognize a particular type of spoken utterance. The system includes a processor for receiving the speech signal from the user in response to the prompt. The processor also directs the speech signal to the at least one speech recognizer indicated by the response identifier. The speech recognizer generates a plurality of spoken utterance choices from the speech signal and a probability associated with each of the plurality of choices. At least one of the spoken utterance choices is selected based on the associated probabilities.

Description

FIELD OF THE INVENTION

The present invention is directed to a speech recognition system. More particularly, the present invention is directed to a speech recognition system that uses multiple speech recognizers to increase its accuracy.

BACKGROUND OF THE INVENTION

Speech recognition systems are increasingly being used to translate human spoken words or utterances into their written equivalent and meaning. Speech recognition systems can avoid the need for spoken utterances to be manually entered into a computer, or to be recognized by a human. Therefore, speech recognition systems are desirable for many businesses because these systems can minimize the number of human operators needed to handle calls from customers.

One drawback to speech recognition systems however, is that they can provide inaccurate results. An exact correspondence between the spoken utterance and an output recognized by a speech recognizer is difficult to attain due to, for example, the deterioration of speech signals that routinely occurs over conventional telephone lines and algorithmic limitations. Such deterioration present in the speech signals may cause a speech recognizer to produce a recognized output that does not correspond to the spoken utterance. Because of limitations introduced into the speech signal by the telephone lines, the speech recognizer may confuse similar sounding letters and numbers. Thus, a speech recognizer may confuse the letter “B” with the number “3” or the letter “C”. For example, given that a user utters the numbers “123” into a telephone, the speech recognizer may produce “12B” as the output.

Additionally, various speech recognizers have their own strengths and weaknesses with respect to accurately identifying spoken utterances. For example, one speech recognizer may perform better at recognizing a sequence of alpha-numeric characters while other speech recognizers perform better at recognizing proper nouns such as for examples, names of places, people and things. Also, some speech recognizers can execute certain tasks faster or require less processing time than other speech recognizers.

If such speech recognition systems are utilized, it is important that the speaker communicate accurate information to the system with maximum machine assistance and minimum user intervention. For example, it is desirable that the user be prompted as few times as possible to repeat questionable information or to supply additional information for the speech recognition system to reach the correct result.

Based on the foregoing there is a need for a speech recognition system that has an increased recognition accuracy without the necessity of relying on human operator intervention or requiring additional input from the user.

SUMMARY OF THE INVENTION

One embodiment of the present invention is a speech recognition system for recognizing spoken utterances received as a speech signal from a user. A prompt for requesting a spoken utterance from the user is assigned a response identifier which indicates at least one of a plurality of speech recognizers to best recognize a particular type of spoken utterance. The system includes a processor for receiving the speech signal from the user in response to the prompt. The processor also directs the speech signal to the at least one speech recognizer indicated by the response identifier. The speech recognizer generates a plurality of spoken utterance choices from the speech signal and a probability associated with each of the plurality of choices. At least one of the spoken utterance choices is selected based on the associated probabilities.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a speech recognition system in accordance with one embodiment of the present invention. [0008]
FIG. 2 is a flowchart illustrating the steps performed by a speech recognition system in accordance with one embodiment of the present invention. [0009]

DETAILED DESCRIPTION

One embodiment of the present invention is a speech recognition system that uses multiple speech recognizers to enhance its recognition accuracy. FIG. 1 is a block diagram illustrating a [0010] speech recognition system 100 in accordance with one embodiment of the present invention. Speech recognition system 100 includes an input/output (I/O) interface 101. I/O interface 101 interfaces system 100 to a user. I/O interface 101 can be a remote interface, such as a telephone line connection as shown in FIG. 1. I/O interface can also be a local interface, for example, a microphone and a speaker.
In the embodiment shown in FIG. 1, I/[0011] O interface 101 is coupled through a network 200 to a telephone 300. Telephone 300 enables a user of speech recognition system 100 to access the system by participating in a telephone call between telephone 300 and system 100. The user can transmit a speech signal to system 100 through telephone 300 as well as receive signals from system 100. Network 200 can be any network that enables the user at telephone 300 to dial a telephone number associated with system 100. For example, network 200 can be the Public Switched Telephone Network (“PSTN”), a local area network, the Internet, or an intranet.
[0012] Speech recognition system 100 also includes an analog-to-digital (“A/D”) converter 103 coupled to I/O interface 101 and a processor 110. A/D converter 103 converts analog speech signals from spoken utterances received from I/O interface 101 into digital signals. Alternatively, a digital signal may be sent through a digital network. An example is where A/D converter 103 is located locally. These digital signals are then received by processor 110. Processor 110 includes a memory 111 and a central processing unit (CPU) 11 that executes a supervisory operating system to coordinate the interactions between some of the different system elements. CPU 11 also executes application software in response to information supplied by the user.
Memory [0013] 111 may be a combination of random access memory (RAM), read only memory (ROM) and/or erasable programmable read only memory (EPROM) components. One or more system components may directly access memory 111, without the use of processor 110. In addition, one or more of the system components may incorporate its own local memory.
[0014] Speech recognition system 100 further includes a recognizer switch 104, a data storage device 108 and speech recognizers A, B and C, labeled 105, 106 and 107, respectively. Each speech recognizer includes speech processing software and hardware that receives a speech signal generated from a human utterance and generates one or more choice of words that represent the utterance. For each choice of words, a probability that the choice of words is the correct choice may also be generated by the speech recognizers. As illustrated, three speech recognizers are shown, but many such recognizers can be included in system 100. Database 108 stores prerecorded prompts used to communicate with the user by soliciting responses from the user. For example, database 108 may store prompts which request a user's account number or requests a method of payment for a particular transaction.
In one embodiment of the present invention, the speech recognizers utilize a Hidden Markov Model to generate a list of “N-best” choices for the solicited response. An example of this method of speech recognition is disclosed in U.S. Pat. No. 5,241,619, herein incorporated by reference. In another embodiment, each speech recognizer executes post-processing routines to generate multiple choices and associated probabilities for each choice. Further, other known methods can be used by the speech recognizers to generate multiple choices and probabilities assigned to these choices. [0015]
Speech recognizers [0016] 105-107 preferably have different capabilities or effectiveness in handling specific recognition situations. That is, the recognizers may provide varying degrees of reliability under different circumstances. For example, one speech recognizer may provide the most reliable recognition of numbers or digits, another may provide high reliability for recognition of letters of the alphabet, still another may provide high reliability in a specific limited vocabulary. These capabilities may be determined by testing each of the recognizers before implementation into recognition system 100. This testing process may include suppling each speech recognizer with an identical spoken utterance. A plurality of these identical spoken utterances may be supplied to each speech recognizer. The plurality of identical spoken utterances includes may types of spoken utterances such as digits, letters of the alphabet, limited vocabulary, etc. Each recognizer then processes the spoken utterances and returns an output signal. The output signals of the speech recognizers are compared with correct signals which represents the spoken utterances to determine which speech recognizer or speech recognizers correctly recognized the spoken utterances. After employing multiple types of spoken utterances such as digits, letters of the alphabet, limited vocabulary, etc., the recognizer that provides the highest reliability for recognizing a particular type of spoken utterance is assigned to recognize that particular type of spoken utterance. Additionally, there are other factors taken into consideration when assigning a speech recognizers to recognize a particular type of spoken utterance. These factors may include the cost associated with a speech recognizer recognizing a particular type of spoken utterance and the speed at which the speech recognize can recognize the particular type of spoken utterance. According to an embodiment of the present invention, recognizer A may be assigned to recognize digits, recognizer B may be assigned to recognize letters of the alphabet and recognizer C may be assigned to recognize limited vocabulary.
In continuing with the description of an embodiment of the present invention, processor [0017] 110 is programmed to select from a plurality of prompts stored in database 108. A prompt can be in the form of recorded or computer generated voice or in the form of a textual prompt if the customer has a local display that is connected to speech recognition system 100. The selected prompt is presented to the user to obtain a spoken utterance. Associated with each prompt is a response identifier which indicates the speech recognizer to be used for recognizing the spoken utterance. If a spoken utterance is supplied by the user, processor 110 along with switch 104 directs the response to the speech recognizer indicated by the response identifier. According to the present invention, a response identifier can indicate more than one speech recognizer if it is anticipated that the spoken utterance will contain multiple types of input utterances.
Each spoken utterance from the user initiates the next prompt in a flexible schedule of prompts to retrieve user information. For the purpose of this application the flexible schedule of prompts for [0018] recognition system 100 is implemented in an airline reservation and information system in a manner now described.
A user dialing a predetermined number associated with the airline reservation and information system is connected to [0019] system 100 via network 200. Processor 110 instructs the user with a stored prompt from database 108 requesting the user to speak his account number. For example, in the airline reservation embodiment, the prompt could be “What is your account number?” The account number can consist of all numbers or a combination of alpha-numeric characters. Associated with the stored prompt is the response identifier. The response identifier assigns at least one speech recognizer to a stored prompt in anticipation of the spoken utterance. Therefore, if system 100 anticipates receiving an alpha-numeric character as the spoken utterance for the user's account number, the response parameter will assign speech recognizer A to recognize digits and speech recognizer B to recognize letters of the alphabet.
Once the user supplies [0020] recognition system 100 with a spoken utterance in response to the request for his account number, a speech signal is generated in response to the user's utterance. Processor 110 processes the speech signal and forwards the user's response to the speech recognizer associated with the response identifier. The assigned speech recognizer decodes the received speech signal. In one embodiment, the assigned speech recognizer utilizes a Hidden Markov Model to generate one word choice or to generate a list of “N-best” choices. A probability for each word choice that that word choice is the correct word choice can also be generated. In another embodiment, the assigned speech recognizer generates one word choice, and then optionally executes post-processing routines to generate multiple choices and associated probabilities for each choice. Further, other known methods can be used by the assigned speech recognizer to generate one word choice or multiple word choices from the received speech signal.
In the case where multiple speech recognizers have been assigned, each speech recognizer provides a word choice and an assigned probability. These probabilities can be compared to determine which is higher, thus indicating the recognizer that best recognized the user's utterance. In another embodiment, an output signal from each recognizer is supplied to a comparator. The comparator compares the output signals to determine if a match between the output signals occurs. If a match occurs, the matched output signal is used to generate a word choice. Alternatively, if a match does not occur, each output signal is used to generate a different word choice with an assigned probability. [0021]
After the word choices and the probabilities have been assigned, one of the word choices is then selected using known methods of selecting the word choice with the highest probability. Once a word choice is selected, known methods can be used to confirm that the selected word choice is the correct word choice. For example, using another stored prompt from [0022] database 108, system 100 can ask the user “Is this your correct account number? Say Yes or No.” The response from the user will confirm if the selected word choice was correct.
Alternatively, a predetermined probability threshold could be used to filter out word choices falling below a predetermined probability value. In this case, each word choice having an assigned probability below the predetermined probability value would be discarded and only word choices above the predetermined probability threshold would be presented to the user for verification. [0023]

In response to a verified spoken utterance,

speech recognition system

100 could further request, “Which airport do you wish to depart from?” Associated with this prompt is the response identifier assigning speech recognize C which most accurately recognizes words in a limited vocabulary. For the purpose to the airline reservation and information system example of the present invention, a list of all the names of the airports would be stored by speech recognizer C. Table 1 below is an example of stored prompts and associated response identifier according to the present invention.

TABLE 1


Stored Prompts	Response Identifier

What is your account number?	Recognizer A & Recognizer B
Which airport do you wish to depart from?	Recognizer C
What is your telephone number?	Recognizer A
Is this information correct? Say Yes or	Recognizer C
No.
Please spell your last name.	Recognizer B

FIG. 2 is a flowchart illustrating some of the steps performed by one embodiment of [0025] speech recognition system 100 when a user dials the telephone number associated with speech recognition system 100 from telephone 300. The call is connected to network 200 or to other call processing hardware in the manner previously described.
At [0026] step 300, processor 110 selects a prompt stored in database 108 to present to the user. Associated with the stored prompt is a response identifier. The response identifier assigns at least one speech recognizer which performs best at recognizing a particular type of human utterance. In step 300, the prompt is a request for the user's account number.
At [0027] step 301, processor 110 receives a speech signal generated by the user's utterance in response to the request for the user's account number. For example, the user's account number “CBA123 ” will be spoken by the user if this is the account number assigned to the user.
At [0028] step 302, speech recognizer A and speech recognizer B receive the speech signal representing “CBA123” from processor 100. At step 303, each speech recognizer generates a plurality of choices of possible user account numbers based on the received speech signal. These choices are generated using the speech recognition hardware and software previously discussed. Associated with each choice is a probability as to whether that choice is the correct account number. Table 2 below is an example of some of the choices and associated probabilities that might be generated in response to the user's utterance of “CBA123”. The list of choices in Table 2 can include the choices for both recognizer A and recognizer B or a separate list of choices and assigned probabilities can be created for each recognizer.

TABLE 2

Account Number Probabilities

CBA123 .01

ZBA123 .003

BBA023 .006

GCK123 .005
At [0029] step 304, the user account number with the highest probability is presented to the user (e.g., “CBA123” in the example of Table 2). In addition, the user may be asked whether the presented user account number is the correct account number. For example, a prompt with a response identifier assigning Recognizer C, can request the user to verify that “CBA123” is the correct account number by asking the user, “Is your account number CBA123? Say Yes or NO.”
At [0030] step 305, based on the response from the user at step 304, processor 110 determines whether the presented user account number is the correct account number. If it is, then speech recognition system 100 has successfully recognized the correct user account number.
However, if it is not the correct account number, at [0031] step 306, the account number with the next highest probability (i.e., “ZBA123” in the example of Table 2) maybe presented to the user. Steps 305 and 306 are repeated until the correct account number is successfully recognized.
As disclosed, the present invention utilizes multiple speech recognizers to increase the accuracy of spoken utterances from a user. A prompt, used to solicit a response from a user in the form of a spoken utterance, is assigned to a speech recognizer designed to best recognize a particular type of spoken utterance. Thus, this allows speech recognition to proceed more quickly and accurately and with less disruption to the user. Several embodiments of the present invention are specifically illustrated and/or described herein. However, it will be appreciated that modifications and variations of the present invention are covered by the above teachings and within the purview of the appended claims without departing from the spirit and intended scope of the invention. [0032]

Claims

What is claimed is:

1. A method for recognizing a spoken utterance of a user comprising the steps of:

assigning at least one of a plurality of speech recognizers to a prompt for requesting a spoken utterance;

presenting a user with said assigned prompt;

receiving said spoken utterance from said user in response to said prompt and generating a speech signal from said received spoken utterance; and

directing said speech signal to the at least one assigned speech recognizer;

2. The method of

claim 1

, further comprising the steps of:

generating a plurality of spoken utterance choices from the speech signal and a probability associated with each of said plurality of choices; and

selecting the spoken utterance based on the associated probabilities.

3. The method of

claim 2

, wherein the step of selecting the spoken utterance comprises the steps of:

presenting a highest probability spoken utterance choice to the user; and

determining whether the presented highest spoken utterance choice is correct.

4. The method of

claim 3

, wherein the step of selecting the spoken utterance comprises the steps of:

presenting a next highest probability spoken utterance choice to the user;

determining whether the presented next highest spoken utterance choice is correct; and

repeating the steps of presenting a next highest probability and determining whether the presented next highest probability is correct if it is determined at the step of determining whether the presented next highest probability is correct, that the next highest spoken utterance choice is not correct.

5. The method of

claim 2

, wherein the step of generating a plurality of spoken utterance choices comprises using a hidden Markov Model to generate N-best choices.

6. The method of

claim 2

, wherein the step of generating a plurality of spoken utterance choices comprises the steps of:

recognizing a first spoken utterance; and

post-processing the first spoken utterance to generate the plurality of spoken utterance choices;

7. The method of

claim 1

, wherein the step of assigning comprises assigning at least one speech recognizer that provides a highest reliability for recognizing a particular type of spoken utterance.

8. The method of

claim 7

, wherein said particular type of spoken utterance includes numbers.

9. The method of

claim 7

, wherein said particular type of spoken utterance includes letters of the alphabet.

10. The method of

claim 7

, wherein said particular type of spoken utterance includes a limited vocabulary.

11. A speech recognition system for recognizing a spoken utterance comprising:

a processor for presenting a user with a prompt; and

a plurality of speech recognizers coupled to said processor, wherein each speech recognizer generates a plurality of spoken utterance choices from a speech signal and a probability for each spoken utterance choice;

wherein said prompt is assigned at least one speech recognizer that provides a highest reliability for recognizing a particular type of spoken utterance.

12. The speech recognition system of

claim 11

, further comprising a database coupled to said processor, said database having stored therein a plurality of prompts assigned to at least one speech recognizer that provides a highest reliability for recognizing a particular type of spoken utterance.

13. The speech recognition system of

claim 11

, further comprising an input/output interface to receive a speech signal from a user.

14. The speech recognition system of

claim 13

, wherein said input/output interface is coupled through a network and a telephone.

15. The speech recognition system of

claim 13

, wherein said input/output interface comprises a microphone.

16. The speech recognition system of

claim 14

, wherein said network is the Internet.

17. The speech recognition system of

claim 14

, wherein said network is a public switched telephone network.

18. The speech recognition system of

claim 14

, wherein each speech recognizer generates a plurality of spoken utterance choices using a hidden Markov Model to generate N-best choices.

19. The speech recognition system of

claim 11

, wherein each prompt is assigned at least one speech recognizer that provides a highest reliability for recognizing a particular type of spoken utterance.

20. The speech recognition system of

claim 19

, wherein said particular type of spoken utterance includes numbers.

21. The speech recognition system of

claim 19

22. The speech recognition system of

claim 19

23. A method for recognizing a plurality of utterances spoken by a user, the method comprising the steps of:

generating a first prompt requesting a first utterance;

receiving said first utterance;

generating a first speech signal based on said received first utterance;

directing said first speech signal to a first speech recognizer assigned to recognize speech received in response to said first prompt;

generating a second prompt requesting a second utterance;

receiving said second utterance;

generating a second speech signal based on said received second utterance; and

directing said second speech signal to a second speech recognizer assigned to recognize speech received in response to said second prompt.