RU2195708C2 - Inscription-bearing audio/video presentation structure, method for ordered linking of oral utterances on audio/video presentation structure, and device for linear and interactive presentation - Google Patents

Abstract

FIELD: educational technique and presentation of inscriptions such as subtitles on image. SUBSTANCE: method includes generation of plurality of audio/video presentation structures each incorporating plurality of descriptive segments with a number of human or other utterances in each segment; superposition of recorded words on such segments which constitute inscriptions corresponding to these utterances, one each time, to tie each utterance and each recorded word together; provision for at least one segment in mentioned presentation structures for each word in group of written words, and broadcasting or any other introduction of such presentation structures on territory, degree and time of such presentations being sufficient to familiarize given group of inhabitants with these written words in relevant language. EFFECT: enhanced self-descriptiveness of presented inscriptions on image. 17 cl, 34 dwg

Description

BACKGROUND OF THE INVENTION
Previous sound performances included the arrangement of subtitles (US Pat. No. 3,199,115 and US Pat. No. 5,097,349) or legends (signatures) of the type of ovals enclosed in words (US Pat. No. 1,240,774 and US Pat. .

 Although the subtitles in the prototype from time to time coincided with the pronunciation of one word in another language, this happens by chance, infrequently and without a controlled pattern to perform a specific connection of consecutive sounds with a sequence of recordings. In addition, the location for subtitle words was excluded from the illustrated action.

 Through well-known coding cards, each of which displays a word, they tried to teach reading through repeated coercive and unnatural exercises. Although some effect is ultimately achieved, the use of such cards requires longer periods of study and the personal presence of a competent mentor, whether it be a mother or a school teacher. In addition, such cards do not provide the bond strength that the present invention provides by providing links to a narrative audio-visual medium that is attracted to the student outside the subject of his training.

 US patent 5241671 discloses the presence on the computer screen of the text of an article with some underlined, and some not underlined words. When the user selects a word from the text, its definition appears in a window on the screen and the word is pronounced soundfully. A display of the speech sound illustration may also occur on the screen.

 Subtitled presentations on demand provide platforms where groupings of words are displayed. On-demand subtitle systems display groups of words along the bottom of the screen or at other locations remote from speakers or actors. On-demand subtitle words appear next to, below or above the visible image frame with a different background, and this background is usually white. To provide assistance to the deaf, on separate adjoining sites, a display of symbols of the sign language with sound representations is also shown. These display techniques on sites bounded by rectangular frames can be intrusive to viewers.

 Methodical audio-visual representations are transmitted on television, which include instructors facing the television camera and pronouncing the words with the corresponding written words in front of the speaker at the time of their utterance. Audience-listeners get tired of such methodological forms and, in particular, younger audience-listeners lose interest in the subject represented in this way.

Summary of the invention
In short, the present invention comprises a sound representation and a method for creating it, which uses the natural arrangement of commonly visible actions with their usual and common series of frames or image fragments, presented along with speech and other oral expressions, and these representations also have a series of recordings on them, related or corresponding to a series of pronunciations in the sequence in which their viewer-listener hears. We call this the “aesthetic” (well located) captioning. The spoken word and the written word in this case correspond to each other if they represent the same word. A spoken word in one language with a written word having the same meaning in another language are related words in this context.

 In accordance with some embodiments of the present invention, a distinctive feature is that each inscription appears side by side, on or in conjunction with the speaker’s head, so that the viewer-listener can simultaneously observe the written word, the spoken word and the accompanying facial expressions, lips and head movements, and so that due to the proximity to the mouth and the location on the same line with it, it seems that the word appears from the mouth. In accordance with other embodiments of the invention, each inscription appears near, on, or in conjunction with the hand or hands of a person using the sign language. In accordance with yet another embodiment of the invention, a Braille recording is “displayed” on a separate device along with the spoken words of the speaker.

 The present invention is used with non-methodical audio-visual representations, usually created for entertainment, informational or other purposes, and these audio-visual representations have no educational purposes. It can be used with such materials either as an element of a new product, or as a modification of previously created sound representations. In addition, the present invention can be used for newly created materials having the purpose of literacy, and which are intended for the application of the present invention; such newly created literacy materials embodying the present invention will provide the opportunity to obtain less boring and less frightening learners than genuine audio learning materials.

 A characteristic feature is that the sound presentation according to the invention can be presented linearly or in combination by programming and using a computer platform covering several media to create a presentation that can be interactively driven by the viewer / listener in order to provide additional instructions.

 Another characteristic feature of this method is that it is useful in a public event in which a sufficient number of representations is created, the placement of words on audio-visual representations, as disclosed here, using basic words in the language and repeating or otherwise representing such words for the population in order to educate part of the population to learn such words when reading.

Brief Description of the Drawings
Figure 1 is a prototype.

 FIG. 2 represents a series of vertical projections of the speaker with written words appearing in the same plane at the speaker's mouth.

 Figure 3 represents a series of vertical projections of the speaker with all the written words appearing in the same plane at the speaker's mouth.

 FIG. 4 is a flowchart showing steps and items of equipment intended for use in the present invention.

 FIG. 5 is yet another block diagram illustrating an interaction mechanism in accordance with the present invention, including a simultaneous audio expression / recording.

 6 is a flowchart illustrating the following steps and equipment elements for using the present invention.

 7 is a flowchart illustrating a method expanding the audio portion of a sound presentation to facilitate coordination of sound and recording.

 FIG. 8 is a front elevational view of a speaker with a word near his mouth.

 FIG. 9 is a partial schematic top view of FIG. 8 with dialog planes shown.

 FIG. 10 is a perspective view of a screen of a television receiver with speakers in various positions.

 11 is another perspective view of another speaker.

 FIG. 12a-12b are flowcharts of a method for performing an aesthetic subtitle introduction according to the present invention.

 FIG. 13a-13c are flowcharts of other systems and method of performing aesthetic captioning in accordance with the present invention.

 FIG. 14 is a flowchart of other systems and method of performing aesthetic captioning in accordance with the present invention.

 FIG. 15a-15c are representations of a waveform extension corresponding to one aspect of the present invention.

 FIG. 16 is a flowchart of other systems and method of performing aesthetic captioning in accordance with the present invention.

 FIG. 17 is a flowchart of the system and method of FIG. 16 illustrating further details regarding a computer workstation.

 FIG. 18 is a block diagram showing further details regarding the computer workstation shown in FIG.

 FIG. 19a-19d are representations of applying aesthetic captioning.

 FIG. 20a-20c represent a quadrant arrangement achieved by the aesthetic introduction of subtitles in accordance with the present invention.

 FIG. 21 is a block diagram illustrating an intuitive application of aesthetic captioning in accordance with the present invention.

 FIG. 22 is a schematic illustration of a multi-media platform according to the present invention.

 FIG. 23 is a block diagram of an interactive feature of the present invention.

 FIG. 24 is a block diagram of the interactive word pronunciation of FIG. 23.

 FIG. 25 is a schematic representation of a blocking zone according to the present invention.

 FIG. 26 is a schematic representation of one embodiment of the present invention using a sign language.

Description of preferred embodiments of the invention
Figure 1 shows the screen 1 of the prototype with the usual sound image 2 (shaded area), which has a known area bounded by a rectangular frame 3 inside the image 2, having the words "in the house"; a known area in the image bounded by a rectangular frame 4 for sign language, and a lower elongated portion 5 in the image for verbal accompaniment, with the words "at eleven." Section 5 contains words that move in the direction of arrow A. Sounds that include a conversation associated with the image 2 mostly come from the source 6 of the sound source.

 In accordance with the present invention, words or other symbols are usually placed on a portion of an image of a sound presentation within a portion 6 of a sound source; however, words can also be superimposed on this portion of image 2, where the attention of the listener is directed due to his or her interest in sound representation, for example, where there is an action, regardless of whether the position of such an action coincides with the sound source or not.

The present invention, in one preferred embodiment, places words in a coordinate system of speakers in a sound representation (i.e., in planes not parallel to the plane of the screen). Since the viewer's coordinate system is the plane of the screen, words moving from such a plane into the actor’s world are more easily and expressively visible and appear to the viewer as three-dimensional objects. Consider FIG. 2 and 3, on which the speaker S of the sound presentation has a head H and a mouth M, from which the written word “look” (“look”) appears in the plane P ₁ when pronouncing such a word. The plane P _{1 is} approximately perpendicular to the line passing through the ears of the speaker (not shown). Each word preferably appears within a short period of time in which the word is spoken or spoken; however, the word may additionally appear immediately before or immediately after pronouncing it, provided that its appearance does not mix with words and sounds pronounced before or after. The criterion is that when each word is pronounced, the corresponding written word or record is associated with it for the viewer-listener. The present invention provides representations of an expressive sequence of spoken words (or other expressions) together with a coordinated sequence of recorded words, numbers or other inscriptions, for communicating such a spoken word or expression and inscription, one at a time, from the point of view of the viewer-listener.

 The purpose of coordinating the presentation of a variety of written words or inscriptions, one at a time, with the corresponding spoken words, is to provide the viewer-listener with the opportunity to combine such sounds and types in a natural arrangement in order to remember that the sound and the view are to be combined so that further presentation either a sound or a view aroused the feeling of another in the viewer-listener. Although this goal is intended to teach literacy, learn a foreign language and education, another advantage of the invention is to increase clarity of understanding in the sense that the viewer-listener can accept and understand the word verbally or visually or by both pathogens, depending on his or her ability , attentiveness, or position relative to an apparatus displaying a sound representation. The second advantage is the translation of foreign sounds, with increased understanding, provided by the location of the recorded translation near or near the mouth; and the third advantage is to achieve the simultaneous presentation of a bilingual inscription by presenting two entries, one in the speaker’s language and the other in the other language, and both appear simultaneously with the pronunciation. When the utterances can be transmitted sequentially so quickly that visual matching with the inscriptions is not practical, this portion of the audio / visual medium exposed to this effect can be expanded in a discrete manner with respect to sound, and expanded visually using either digital or analog means to enable understood communication.

 You can display at the same time two-grams, three-grams or quadragrams (a sequence of two, three or even four words), where the task is to understand the sound accompaniment of a sound representation by a deaf or not speaking a language, and the presentation of one word is too fast; in both cases, the task is for the captions on the screen to be offered in a language that the viewer already understands. In this case, the number of words should be the smallest number of words that will still allow us to adequately understand the phrase in question in the time window. This approach is the substitution of on-demand subtitles or subtitles of foreign films, where the goal is limited to understanding the narrative or entertainment program, as opposed to pronunciation connections.

 According to another embodiment of the present invention, one or more words are arranged in accordance with the hand or hands of a person speaking the sign language, so that there is a correspondence between such words and one element of the symbol language. Thus, the viewer-listener is provided with the opportunity to associate in a natural arrangement such words for the purpose of remembering that the words are associated with this element of the symbol language.

 In accordance with yet another embodiment of the invention, words can be arranged in a sound presentation so that they are only visible to a viewer using a special reading device. This is similar to the three-dimensional representations that are visible only when the viewer wears a special type of glasses. In fact, the preferred method of this embodiment is to use a special type of glasses.

 The words in the present invention are displayed as an integer part and superimposed on a pictorial presentation scene. Visual scenes include elements such as figures of people, furniture, sky, background cityscape, and the like. Words can be superimposed on one or more figurative elements and as a result of this prevent viewing of part of the figurative element or prevent partial viewing of a portion of the figurative element where the recorded word is transparent or translucent, or the word is composed of wire-frame type letters.

 Since presenting more than one written word to the viewer-listener at a time when the words are pronounced makes it difficult, if not impossible, to correctly associate the correct sound with the corresponding written word, it is important that each sound or its corresponding recorded word appears in this way so that the listener-viewer easily linked the two elements. In order to avoid distraction and confusion, each spoken word should be accompanied by its only written connection with the possible exception of an additional pictogram of such a word, language representation of the signs of such a word, or a foreign translation of such a word. Such a written word or words may be displayed before, during or after pronouncing the word, provided that such a display does not occur during the pronunciation of the previous word or connecting word.

 More than one word or symbol may appear during a pronunciation, provided that each word or symbol must be associated with the pronunciation. For example, if the word "thank-you" is spoken (thanks), then the word "thank-you" and the word "merei" may appear at the same time.

 During a sound presentation, there are periods of pronunciation time in which words are pronounced, and periods of non-pronunciation between them. In the “one word at a time” method of the present invention, a written word appears only during a period including 1) a period of non-pronunciation following the pronunciation of the previous word, 2) a pronunciation of the word and 3) a period of non-pronunciation of the word following the pronunciation of the word until pronouncing the next word.

 By presenting alphabetically based words in the pictorial arrangement of such words, the illiterate learner must remember the logograms using that part of the brain that records entire visual images, much like film perceives light to create a photograph.

 The invented segmented representation of the alphabetical word with the simultaneous accompaniment of either a pronunciation or a pictographic link, or both at the same time, creates a restored link in the student’s memory between the written word (which is perceived as a logogram, although it’s written normally) and simultaneously represented by a link ( links). After several repetitions, subsequent representations of the alphabet-based word (logogram) are resurrected in the memory of the student of the link (s), that is, the spoken word. This, of course, determines the action of reading, the training of which is the purpose of the present invention.

 The same process of pairing spoken and written words is used in the opposite way also for teaching a student who knows a given language the ability to speak it. In this case, the reference is the written word or logogram, and the purpose of the study is the spoken word.

 The key to the intensity of learning, especially the child, is that the associations presented in the environment, that is, “natural,” are similar to the environment in which the child learns to speak. The environment in which the child learns to speak, which is natural and usually does not include formal conversation lessons, is the same type of environment that the present invention presents in a sound-visual way. In a preferred linear embodiment of the invention, the listener is provided with an environment of storytelling or other presentation, the primary purpose of which is not literacy. When someone learns to speak, they are exposed to visual images or actions, respectively, demonstrated or implied by intermediaries (such as parents) in the student’s environment, which serves as links that provide links to parallel pronunciations. The environment of the present invention is an environment where visual images or actions, respectively, demonstrated or implied by intermediaries (such as parents) in the environment of the student (i.e. the child), serve as links that perform communication with parallel pronunciations. Such a setting includes expressive, consistent utterance, since mediators in the learner's environment generally do not give arbitrary utterances. Such a natural situation of learning a language is presented in a normal moving image, where natural situations of communication are presented and where the same word is shown again by natural repetition during the film. The natural surroundings and the emulation of a moving image provide connections between actions and objects and their corresponding descriptive pronunciations; the present invention expands the possibility of communication of the written word in a sound-imitative imitation of the natural environment.

 The present method makes it possible to teach reading by presenting the student with whole words, in contrast to syllables and letters of a word. Viewing and memorizing a whole word is similar to viewing and memorizing a symbol such as an image of a cat or a Chinese character, in which it is believed that such a whole word is processed in the same way by the human brain. Viewing each whole word (or reading from a worksheet) provides training based on the development of mental connections that are visually remembered or imprinted and restored through association, rather than analyzing the human brain, which is required for alphabetical, syllabic or phonetic reading.

 Where two records should be displayed, that is, one in written form and the other in graphic form, corresponding to one spoken word, two records can merge or separate from each other to show that they are interconnected or even are the same . For example, if a person in the video image says the word “cat”, the written word “to-o-t” can be changed into a cat icon.

 Regardless of whether the relationships created by the present invention are in the context of the current sound presentation or are to be created, the relationships created by the present invention arise in normal, natural, pictorial locations. By way of example, such connections may arise in the frames of a performance film, where a detective and a suspect are talking; in video images where the performer sings, or in the latest news broadcasts on television, where the meteorologist speaks and points to the map. In all the cases just mentioned, the goal does not necessarily include literacy.

 The present invention is also applicable to learning to read lips, where if pronunciation is made and if recording is displayed, then the movement of the lips is monitored as part of the visual portion of the presentation.

 One of the advantages of placing words on or near a contextual source within the area of the displayed television frame is that it makes it easier for the viewer to see the word when he or she hears the word, while maintaining focus on the action taking place on the presentation. Although the eye can see peripherally the words placed on the edges of the visible screen, or even outside the area with the image, he can only read them with difficulty when he holds the focus on the acting elements of the sound presentation. It is for this reason, along with others, that the present invention is superior to captioning on demand. In addition, the introduction of subtitles on demand represents more than one word at a time, which prevents the connection of one word with one sound. Moreover, the present invention represents the words in spatial relation with the speaker, which reduces the difficulty and minimizes the screen area occupied by the written word.

 When two people talk, regardless of whether they are facing each other or not, one plane between two people can serve as the plane on which written words will be displayed. This technique can also be used when one of the speakers is outside the camera, where the audience is aware of the relative position of the speaker outside the camera.

 The color, shape and other characteristics of the letters of each written word are made to be unobtrusive. For example, if the graphic background element on which the word is superimposed is dark blue, the letters of the word may be light blue or another shade of blue. In addition, the written word can be made translucent or semi-translucent, so that it will allow you to partially continuously see the background. In addition, the word may be a color, font, or other means encoded for its source.

Referring again to FIGS. 2 and 3, on which the head (H) of the speaker (S) is turned, the plane P ₁ , which is approximately perpendicular to the line passing through the speaker’s eyes, moves to three (3) additional positions P ₂ -P ₄ . When the word "AT" is pronounced, it appears in perspective in the P ₂ plane; when the word "SPOT" is pronounced, it also appears in perspective in the P ₃ plane, and finally, when the word "GO" is pronounced, it appears in the P ₄ plane. Each word is located at or near or even on the head, and preferably at or near the mouth (M) of the speaker when it is pronounced. Note that when the speaker’s head (H) turns (S), she also leans in to raise her chin (see plane P ₄ ). The orientation of the inscription preferably reflects the orientation of the head from side to side and up and down.

 In FIG. 3, all spoken words appear in the PP planes, which lie in the plane of the screen on which the sound representation is displayed, or parallel to it.

 Figure 4 shows a device designed to create a sound presentation, including the point of the operator; a video text generator designed to create the desired record (for example, the words “look”); an audio presentation input means for providing a presentation on which labels are not yet posted; means for providing a digital optical manipulator for combining text and such a sound presentation to provide coordination of pronunciation-recording in a proper orientation plane according to the present invention. This manipulation creates an invented audio presentation in which such coordination occurs throughout the presentation, and can be seen and heard without interruption in the presentation, and this embodiment is a linear embodiment of the present invention.

 Groups of letters are attached, imprinted, applied, or placed in some other way on that place of the image, which is most likely to be seen when pronouncing the word. When the speaker’s head is visible, the location should usually be at or near the mouth to suggest that the word comes out of the mouth. This sequence continues with respect to all or with respect to a substantial number of pronunciations for the entire presentation or, if necessary, for part of the presentation. The letters can be of any size, font, or color. In one preferred embodiment, the size, font, color, or any other graphic characteristic is chosen so as to reflect the background colors and the emotional and ideological essence of each utterance. Regarding the background, each written word should be implicitly translucent, semi-translucent, from wire frames or in a color that represents a shading of the background color that is significantly different from the background color in order to achieve visibility without leaving a retina or halo after the word has disappeared . Regarding emotions, intentions or meanings, angry words, for example, are red with sharply defined font patterns, while soothing lyrics will have a pastel tone with soft handwritten font patterns. Emotionally neutral words can be represented by default color. The purpose of the graphic properties is to provide the viewer-listener with dynamic graphics parallel to the pronunciation shades transmitted by changing the volume, tone, pitch or other vocal distinguishing features, and thereby strengthen the linking task, which can be restored in the future with the help of thought.

 In sound representations, natural situations of communication prevail. Such situations include, as mentioned above, the interrogation of a suspect by a detective. Placing words in frames that include natural communication situations provides a means of transportation to create the desired connection of sound and inscription, while the viewer-listener remains attentive to the natural communication of the presentation.

 Referring now to FIG. 5, we see that a linear embodiment of the invention is used to create an interactive embodiment by creating a computer program that allows the viewer-listener to delay the audio presentation in order to stop it from being viewed on a screen on which the audio presentation shows a menu for provision by selecting definitions of words, syntax, and contextual use of expressions or other information. Interactive interactive work is represented by the action of the viewer / listener, using a programmable training device designed to use such a program to display the presentation, stopping the presentation for reviewing the selected record and receiving additional information regarding such an inscription.

 Shown in FIG. 6, sound representations are created by the computer graphics designer at his or her workstation, where the video signal of the work (presentation) is presented on the screen for the developer (in analog or digital form). When working with frames (alternating images after 1 / 30th of a second), the developer creates a computer graphic image or text (i.e. a word) and superimposes it on the video signal of the frame or frames, depending on the length of the pronunciation time the word occupies. The length of time it takes to pronounce a word varies with a large number of words in everyday conversation in English (or another language), taking a period from 1 / 60th to 1/2 second. Using animation and using additional symbols of the coloring block software, you can get the font of the letters in the word and the orientation of the word in the selected plane.

 FIG. 7 illustrates a method for stretching a spoken word in time in a sound representation in order to provide a longer representation of the associated written word. This stretching or extension in time of the audible word is performed by digitizing the sound of the word on the hard disk in the form of a waveform, and then reconfiguring the waveform. This technique does not violate pitch or intonation.

The head (H ') in Fig. 8 is facing to the right of the viewer, as shown by the dashed line of the source (SL). The line (SL) lies in the reference plane (P ₅ ) of the speaker's conversation (not shown). The vertical reference plane (A) of the viewer is visible to the viewer as a line. This plane remains motionless. The line (SL) passes through the word "WOW" (delight) like a skewer for a fully fried carcass.

The distance at which the beginning of the word (WOW) is located from the speaker’s head (H ′) is preferably within the distance (d ₂ ), which is equal to twice the width (d ₁ ) of the speaker’s face (P) having a nose (N) (see Fig. 8). This arrangement of the word (WOW) in the range of 2d ₁ provides good results for scenes where the speaker’s head is in a close position (close-up). Where the head is at a distance, as when shooting in the background, the words may be larger than the head, but still adjacent to the head (H ') or move to the object of interest to the viewer and, in this case, the distance (d ₂ ) may be 3 or 4 times the distance (d ₁ ).

FIG. 9 is a schematic plan view of FIG. 8, showing a conversation plane (P ₅ ), plane A (at an angle of 180 ^° relative to the reference plane of the viewer), and B is a plane at an angle of 90 ^° . The conversation plane (P ₅ ), on which there is a source line (SL), includes the word "WOW" appearing with this orientation. Words that appear in other conversation planes (P ₆ ) and (P ₇ ) located at an angle of 25 ^o from the viewer's plane (A), which is at an angle of 180 ^o relative to the reference plane of the viewer, can be read, but since words placed in the conversation planes closer to the reference plane of the viewer (A) (the main plane of the viewer's vision), it is difficult to read such an arrangement (in this "blocking area"), rarely used in practice of the present invention.

In FIG. 10 shows the television screen 20 of the television receiver 21 with the control buttons 22, 23. The position of the head / face of the speaker is shown in several ways, as was shown in FIG. The left side of the screen 20 shows the head (H), or (F), the conversation plane (P ₁ ) with the source line (SL ₁ ) on this plane. The face plane (FP ₁ ) is perpendicular to the plane of conversation (P ₁ ). The source line (SL ₁ ) is perpendicular to the face plane (FP ₁ ). Face planes usually lie in planes perpendicular to the horizontal when the speaker is in a standing or sitting position. The source line (SL ₁ ) linearly bisects the word “look” (see). Other source lines (SL ₂ ), (SL ₃ ) and (SL ₄ ) are shown in their respective conversation planes (P ₂ ), (P ₃ ) and (P ₄ ), each of these lines linearly bisecting their respective word.

Finally, FIG. 11 shows a speaker (S ₂ ) with a head (H ₃ ), a face (F ₃ ), and a mouth (M). The face plane (FP ₂ ) is perpendicular to the plane of the conversation (not shown). The source line SL ₃ , which lies in the plane of conversation (not shown), halves the word "fast" (firmly). Since the head (H ₃ ) can move in any direction when the speaker (S ₂ ) tilts or rotates it back to the viewer, the words on the source line (SL ₃ ) when pronounced by their head (H ₃ ) in this orientation are in each case the conversation plane, except when the orientation of the conversation plane is such that the word located in it loses clarity for the viewer. For example, when the speaker (S ₂ ) is in a standing position and is facing away from the viewer, the word “fast”, if located in the plane of the conversation, will be at such an angle to the viewer when the word “fast” may be illegible. To avoid such illegibility, the word is placed in the plane as close as possible to the conversation plane, where the word "fast" is legible. In this case, the word “fast” will be shown in spatial orientation in such a chosen plane to give the impression that the word comes from the head (H ₃ ).

Where time permits, the word "fast" may initially appear in a position obscuring part of the mouth (M), and then quickly move along the line (SL ₅ ) of the conversation plane. Alternatively, for example, if a word should appear on the screen for 0.024 thousandths of a second, the word may appear for 0.008 thousandths of a second partially on the lips (M), and then move along the line (SL ₃ ) for 0.008 thousandths of a second and, finally stop on the line for another 0.008 thousandths of a second before disappearing.

 The purpose of placing words on the source line (SL) and in the planes in perspective, as presented here, is to make the word appear as if from the mouth, and then appear as an object in the three-dimensional space of the sound scene. As an object, the written word obeys the same physical laws as any other object. Thus, if someone walks in front of the speaker in a sound presentation using the present invention, the speaker’s speech may be muffled, and the look of his written word may be partially or completely blocked for an instant. The aim of this aspect of the invention is to make the appearing words turn out to be real objects, a concept very acceptable for young minds, and, in particular, for those who find the words "user friendly" rather than abstract.

 Words are arranged so that they appear in perspective with increasing or decreasing letter sizes in words (see Fig. 8, where the letter "W" on the left is less than the letter "O", which, in turn, is smaller than the letter "W" to her right). Words in perspective appear in such a way that their direction includes the appearance of movement in that direction. The word in perspective near the speaker’s mouth appears as if it was coming out of the mouth. Words are placed as close to the mouth as possible, without interfering with the speaker's facial expressions that are part of the message.

 Not all words spoken during the presentation must have a corresponding written word being displayed, since the selected periods of the presentation may present special difficulties in communication or for other reasons may not require training.

 A preferred use of the invention is to arrange words or other alphanumeric characters or other labels on magnetic tapes, film, computer diskettes, CD-ROMs or other media in an expressive sequence that provides communication with the oral element of a magnetic tape or film, or CD-ROM -drive, or computer diskette, as described above. This sequence can continue throughout the film or magnetic tape from start to finish. The sound representations of the present invention are preferably entertaining or other contextual expressive objects and contents. Studying by the viewer-listener takes place without much effort on his or her part, as he or she enjoys entertaining or other content.

 The present invention creates an impression on the area of the image of the presentation of the spoken word, as if it were visible in each such word, having size, color, font, movement and other characteristics. The word size represents the orientation of the word in the plane of the screen of the imaging device or in a plane lying at an angle to such a plane. Words with this orientation are three-dimensional, like other elements of the image.

 Labels may contain letters, words, pictures, or other characters.

 According to another embodiment of the present invention, the inscriptions are displayed in Braille, preferably on a separate device that a person (e.g., a person with visual impairment) can use while listening to a sound program. Similar to other embodiments, one simultaneous relationship is established between Braille entries and spoken expressions, so that the user is provided with the opportunity to associate such labels in a natural arrangement for the purpose of remembering that the labels are associated with these expressions.

 EXAMPLE.

 An entertaining video game is used, in which for the player there is an inventory of pictogram links (letter drawings). You can move the cursor slowly through the icons. One click of a mouse-type manipulator on any link object fades the link object ("morphing") the entire equivalent of the written word (logogram), while a voice-overs or talking head pronounces the word.

 The goal of the game is to create a series of pictogram links that create an expressive series. As soon as the player builds such a row, double-clicking the mouse will morph the link objects into written words (logograms), from left to right, one at a time at the same time and at the same time accompanied by the corresponding pronounced link object. Then the speaker loudly repeats the expressive series from left to right, with each word suddenly "accompanied in the background" by the link object.

 During the game, the delay created by the location of links that are not an expressive sequence does not give a result when two clicks are made, and the points do not make marks.

 The shades of color and graphic features of the font may vary in accordance with the natural course of the expressive speech turnover of the conversation. In itself, the general “organic look” of the invention will create a new, standardized “emotional graphic vocabulary”. For example, for the following emotions, the following colors and graphic features presented in Table 1 can be used.

 The nuances of font and color can also be used to communicate physical realities, for example, those that exist in nature (see table 2).

 Such associations are based on a general feeling and / or pre-existing studies of the association of associative graphic effects of color, structure, and so on, on human emotions and the preservation of the subject. In addition, for additional associative effects, it is possible to supplement the capabilities of existing computer graphics software, including visual phenomena such as "glow" and "radiation".

 Aesthetic subtitling in a narrative context in accordance with the present invention can be accomplished in a number of ways.

 In FIG. 12a-12b show steps for manually applying the aesthetic introduction of subtitles.

 FIG. 13a-13c present a direct vision system and a method for applying aesthetic subtitles.

 In FIG. 14 shows a system and method that slows down pronunciation without loss of pitch or intonation and without obvious distortion.

 In FIG. 15a-15c show images of normal and expanded waveforms, respectively, of the word “future” expanded by the system and method shown in FIG. The waveform in Fig. 15c has the same pitch as the waveform in Fig. 15a, since the wave amplitude has remained constant, although the wave has become wider.

 Another embodiment of the invention is useful when the waveform is expanded with a multiplier with some fractional multiplication factor, as opposed to a multiplier with an integer multiplication factor. For example, when you want to increase the wavelength by half (50% increase), as opposed to doubling the length (100% increase), known methods are randomly selected that extract part of the wave for expansion. In accordance with this aspect of the invention, the arbitrary selection of wave parts is limited only to vowels. This can be accomplished using means known to those skilled in the art.

 On Fig shows a digital system and method of applying aesthetic introduction of subtitles using well-known software for animating characters to place words.

 FIG. 17-21 relate to another embodiment of the present invention, which is a system and method for intuitively aesthetically introducing subtitles.

 FIG. 17 depicts the system and method shown in FIG. 16 using the aesthetic captioning software of the present embodiment.

 FIG. 18 is a block diagram showing a further detail of the computerized workstation used in FIG. 16 and 17 system and method.

 FIG. 19a-19d show details of the application of the aesthetic captioning of the present invention.

 FIG. 20a-20c show four quadrants of a virtual three-dimensional region where an aesthetically introduced subtitle word is born.

 FIG. 21 is a block diagram of a software module for positioning an aesthetically displayed word with an optimal orientation in any quadrant of a virtual three-dimensional region. The software is preferably run on a computerized workstation system. Although input devices known to those skilled in the art can be used, the user preferably determines the quadrant and draws the source line with the mouse and enters using the keyboard. By running the software module, the computer system automatically places the word on the source line, preferably so that the source line passes through the middle of the main body of the lower case letters (known to printers as “lower case letters without tall elements”), for example, through a horizontal strip in the letter "e". As soon as the system and program place the word, the source line is erased.

 FIG. 22 is a schematic representation of a multi-media platform integrating an interactive multi-media computer workstation to create interactive presentations of the present invention with aesthetic subtitle introductions. On Fig presents a block diagram of software designed to perform interactive features.

 FIG. 24 is a pronunciation block diagram of the interactive word shown in FIG. 23. In accordance with this aspect of the invention, when a user stops the aesthetic subtitled audio presentation on a particular word, he can learn a pronunciation lesson. The user can preferably speak into a microphone connected to a computer having a speech analysis program that compares the waveform generated from the user speaking the word with the standard waveform for the correct pronunciation stored in the computer file. The computer then provides user feedback that either confirms the correct pronunciation (for example, “good enough” or “excellent”), or prompts the user to try to say the word again.

 Other options shown in FIG. 23 can preferably be presented as a menu of interactive tasks that the user can select. For example, the user may choose a recording application that allows the user to simulate a word displayed by typing a word on a keyboard or writing a word on an electronic tablet that produces an output signal for handwriting recognition software. The interactive system preferably provides feedback to inform the user, regardless of whether the word was typed or spelled correctly or not.

 FIG. 25 represents word placement in the frontal “blocking zone” shown in FIG. 9. The word can optionally be placed in this zone (on a plane passing through line AA of Fig. 25), as one way of making it appear as coming out of the speaker's mouth.

 One word or more can also be placed by linking them to the hand or hands of a person using the sign language, so that there is a correspondence between such words and the only element of the sign language. An example of such an embodiment according to the present invention is shown in FIG. 26, which shows a speaker speaking on the screen of a television receiver and an inserted area delimited by a rectangular frame with the hand of a person who is simultaneously translating into sign language. When the speaker pronounces the word “Future” and the translator expresses the word with signs, the word “Future” is placed in the inserted area according to the element of the sign language for the word.

Claims (17)

 1. A way of familiarizing members of a population group with a group of recorded words from a language spoken in a given area, including creating a plurality of audiovisual presentations, each of which includes a plurality of visual segments containing a series of pronunciations performed by a person or other speakers in each segment ; superimposing on such segments of recorded words forming inscriptions corresponding to such pronunciations, one at a time, so that each pronunciation and each recorded word are related; providing at least one segment in the mentioned representations for each word in the group of written words, and broadcasting and otherwise introducing such representations into the territory to such an extent and for such a period of time that this group of people is familiarized with such recorded words of the specified language.  2. The method according to p. 1, characterized in that the created segments of the audio-visual representations include graphic scenes with natural situations of communication.  3. The method according to p. 1, characterized in that the speaker has ears, and the inscriptions appear in a plane that runs essentially perpendicular to the line going through the ears of the speaker.  4. The method according to p. 1, characterized in that the speakers who made the pronunciations have heads, each inscription being in close connection with the head of the speaker.  5. The method according to p. 1, characterized in that the audiovisual presentation has entertaining content.  6. The method according to p. 1, characterized in that the audiovisual presentation has instructive content.  7. The method according to p. 1, characterized in that at any given time for the viewer appears only one inscription.  8. A method of teaching learners, including the creation of one or more audiovisual presentations, including natural communication situations, each of which includes the presentation during such situations of many pronunciations simultaneously with the corresponding inscriptions, each audiovisual presentation includes an area which the learner’s attention is naturally directed to, and this area includes (i) the first section from which the outgoing utterances appear, or about which the gaze of the student is studied by establishing the meaning of the statements, and (ii) a second section that displays the mentioned inscriptions at the same time with each pronunciation, so that in the student’s mind there is a connection between the pronunciation of the word and the corresponding inscription; making such representations accessible to the learner and making it possible for such a learner to choose locations for observation and listening over time until each of the many pronunciations is heard by the learner together with the vision of the corresponding spellings a sufficient number of times to help learn these some pronunciations corresponding to such specific inscriptions.  9. The method according to p. 8, characterized in that the first and second sections partially coincide.  10. A method of placing a series of labels on images, frames or segments of an audiovisual presentation, and on these images, frames or other segments there is a person or inanimate speaker at the time the pronunciation is performed, including: selecting an operator-controlled unit of equipment that includes video text means ; displaying such a video text tool of a plurality of words, each word having different sizes, shapes and orientations for searching; the location of each of the series of images for observation by the block operator; activating the video text tool to select the displayed words and transfer the selected words to the selected image, frame or segment for their permanent placement on it, so that the word appears on the segment in local and temporary connection with the pronunciation of the word.  11. The method according to p. 10, characterized in that the words are placed next to the head of the speaker.  12. A computer-based data processing system for aesthetically introducing inscriptions on a plurality of fine segments, including pronunciations, comprising computer processor means for processing data, a memory means for storing data, means for applying to data corresponding to a plurality graphic segments, including pronunciations in each data segment corresponding to the written words, one at a time, respectively To such pronunciations, so that each utterance and each written word are associated.  13. The system of claim 12, wherein said means for applying to data comprises: means for introducing a first analog video signal; means for converting an analog video signal to digital video signal data; means for simultaneously displaying one frame of digital video signal data; means for user inputting word data and quadrant data; means for combining word data in accordance with quadrant data and other predefined criteria in digital video signal data; means for converting digital video signal data into a second analog video signal; means for outputting a second analog video signal.  14. The system of claim 12, wherein the means for user inputting word data and quadrant data comprises a means for enabling a user to select a quadrant; means designed to enable the user to draw a source line; means for calculating the angle for the source line; means for enabling a user to enter word data, and means for automatically spacing words along a source line.  15. A computer-based system designed to enable a user to interactively perform an audio-visual presentation with aesthetically entered subtitles, comprising means for displaying an audio-visual presentation; means for stopping the display of an audiovisual presentation; means designed to enable the user to define a word; means for processing additional data regarding a user-defined word.  16. The system of claim 15, wherein the means for enabling the user to determine the word allows the user to determine the word that is displayed when the display of the audiovisual presentation is stopped.  17. The system of claim 15, wherein the means for processing additional data regarding the word comprises means for storing standard data reflecting the proper pronunciation for a plurality of words; means for user input of speech data corresponding to the pronunciation of a word by the user; means for comparing speech data with standard word data; and means sensitive to means for comparing speech data intended to indicate to the user whether the pronunciation of the word was correct for the user or not.

Images