CN111935442A - Information display method and device and electronic equipment - Google Patents
Information display method and device and electronic equipment Download PDFInfo
- Publication number
- CN111935442A CN111935442A CN202010759690.4A CN202010759690A CN111935442A CN 111935442 A CN111935442 A CN 111935442A CN 202010759690 A CN202010759690 A CN 202010759690A CN 111935442 A CN111935442 A CN 111935442A
- Authority
- CN
- China
- Prior art keywords
- target
- user
- users
- participating user
- participating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Telephonic Communication Services (AREA)
Abstract
The embodiment of the disclosure discloses an information display method and device and electronic equipment. One embodiment of the method comprises: determining at least one target conference participating user from the multimedia conference participating users based on the liveness within the preset time period; and in response to the fact that a first target participating user of the at least one target participating user does not start the video currently exists, splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users. The users participating in the multimedia conference can browse the presentation image and/or video information streams of the active users in their terminals, which may improve the user experience.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of internet, in particular to an information display method and device and electronic equipment.
Background
The network multimedia conference can realize real-time transmission of information in the forms of moving images, voice, application data (electronic whiteboards, graphics) and the like between two points and multiple points. The method is suitable for teleconferencing, network discussion and the like. People at two or more locations can interact through a network multimedia conference.
In the network multimedia conference, information communication is mainly performed through voice and video, that is, when a user participates in the multimedia conference, the user can listen to audio information of the multimedia conference and/or browse video information of the multimedia conference through terminal equipment.
Disclosure of Invention
This disclosure presents concepts in a simplified form that are described in detail below in the detailed description. This disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the disclosure provide an information display method, an information display device and electronic equipment.
In a first aspect, some embodiments of the present disclosure provide an information display method, including: determining at least one target conference participating user from the multimedia conference participating users based on the liveness within the preset time period; and in response to the fact that a first target participating user of the at least one target participating user does not start the video currently exists, splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users.
In a second aspect, some embodiments of the present disclosure provide an information display apparatus including: the determining unit is used for determining at least one target conference user from the multimedia conference participant users based on the activity within the preset time period; and the display unit is used for splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users in response to the fact that the first target participating user which does not start the video currently exists in the at least one target participating user.
In a fourth aspect, some embodiments of the disclosure provide a computer readable medium having a computer program stored thereon, wherein the program when executed by a processor implements the method of the first aspect.
According to the information display method, the information display device and the electronic equipment, at least one target conference user is determined from multimedia conference participant users on the basis of the activity within the preset time period; and in response to the fact that a first target participating user of the at least one target participating user does not start the video currently exists, splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users. Compared with the scheme that only the information of the active users with the video information streams is displayed in the active user display area in the related art, the scheme provided by the disclosure not only shows the active users with the video information streams, but also shows the display images of the active users with only the audio information streams. The participating users of the multimedia conference can browse the presentation image and/or video information streams of the active users in their terminals. In addition, when the information of the active users is displayed in the active user display area, whether the active users turn on the camera or not is not considered, so that the display images and/or the video information streams of the real most active target participating users are displayed in the active user display area. Sudden changes of the target participating users corresponding to the display areas of the active users caused by the on and off of the video acquisition devices of the active users in the related art can be avoided, and the user experience can be improved.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
FIG. 1 is a flow diagram of one embodiment of an information display method according to the present disclosure;
FIG. 2 is a flow diagram of another embodiment of an information display method according to the present disclosure;
FIGS. 3A-3C are schematic diagrams of an application scenario of an information display method according to the present disclosure;
FIG. 4 is a schematic block diagram of one embodiment of an information display device according to the present disclosure;
fig. 5 is an exemplary system architecture to which the information display method or display apparatus of one embodiment of the present disclosure may be applied;
FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
Referring to fig. 1, a flow of some embodiments of an information display method according to the present disclosure is shown. The information display method comprises the following steps:
step 101, determining at least one target conference user from the multimedia conference participant users based on the liveness within a preset time period.
In this embodiment, the participating users of the multimedia conference can receive and transmit multimedia information streams through the terminal devices used by the participating users. The multimedia conference can be an audio conference or a video conference.
The terminal device may include, but is not limited to, a smart phone, a tablet computer, a laptop portable computer, a desktop computer, and the like.
In the process of the multimedia conference, the terminal equipment used by the participating users can collect the multimedia information flow of the participating users. The multimedia information stream may comprise an audio data stream and/or a video data stream. The terminal device may send the multimedia information stream to a server of a multimedia conference. The server can send the received multimedia information stream of the participating user to the terminal equipment of other participating users.
In addition, in the process of the multimedia conference, the terminal equipment used by the participating users can receive the multimedia information streams of other participating users sent by the server. The terminal equipment can display the multimedia information of the participating users. For example, the audio data stream of the participating user can be played, and the video information stream of the participating user can also be displayed.
In this embodiment, the preset time period may be any time period, such as a time period corresponding to the first 3 seconds, the first 5 seconds, or the first 10 seconds of the current time.
The activity of each participating user can be analyzed once every preset time period for the received multimedia information stream of the multimedia conference.
In some application scenarios, the preset time period may be a last preset time period of the preset time period at the current time.
In some application scenarios, the liveness of the participating users may be determined based on the audio data streams of the participating users.
In some optional implementation manners, the activity of each participating user in the preset time period may be determined according to the duration of the audio data of the speaking state of the participating user identified in the audio information stream of each participating user in the preset time period.
In some alternative implementations, the activity of the participating users may be determined based on the following steps:
firstly, the accumulated volumes of a plurality of multimedia conference participants in the preset time period are sequenced.
Secondly, according to the sorting result, determining the activity degree corresponding to each participating user.
The accumulated volume of each participating user in the preset time period can be counted. That is, for each participating user, the cumulative volume corresponding to a plurality of audio frames of the participating user in a preset time period is counted. And then sorting the accumulated volume corresponding to each of the plurality of participating users. The sorting may be sorting in order of the cumulative volume from large to small. It is also possible to sort the sound volumes in order of the cumulative volume from small to large.
In some application scenarios, the sequence corresponding to each of the participating users in the ranking result is inversely related to the activity of each of the participating users.
The cumulative volume of the plurality of participating users is arranged in descending order. In these application scenarios, the participant users with smaller sequence numbers in the above sequencing result have higher liveness in the multimedia conference.
In other application scenarios, the liveness of the M participating users may be divided into N levels, where N is less than or equal to M. M is a positive integer greater than or equal to 1, and N is a positive integer greater than or equal to 1.
The cumulative volume of the M participating users in the preset time period is arranged in order from large to small. In these application scenarios, the participating users with the corresponding ranking numbers smaller than the first preset threshold may be determined as participating users with the first activity, the participating users with the corresponding ranking numbers larger than or equal to the first preset threshold and smaller than or equal to the second preset threshold may be determined as participating users with the second activity, and so on until the nth level activity participating user is determined by using the nth-1 preset threshold. Where the N-1 th preset threshold is greater than the N-2 th preset threshold.
The activity degree represented by the N-1 th activity degree is larger than that represented by the N-1 th activity degree.
The first preset threshold, the second preset threshold, …, and the N-1 st preset threshold may be set according to a specific application scenario, and are not described herein again.
At least one target participant user which is currently displayed can be determined according to the activity degree corresponding to each participant user. For example, the participating users with the activity greater than the preset activity threshold are determined as the target participating users. Or selecting a preset number of target participating users according to the sequence of the liveness from large to small. The preset number here may be 1 or more.
And step 102, in response to the fact that the first target participating user of which the video is not started currently exists in the at least one target participating user, splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users.
After at least one target participant user is determined in step 101, a current video information stream of the target participant user may be obtained.
And if the target participating user which does not start the video exists in the at least one target participating user, taking the target participating user which does not start the video as the first target participating user. I.e. there is no video stream in the multimedia stream of the first target participant.
The currently presented image of the first target participating user may be displayed in mosaic with the video information streams of the other target participating users in the area for displaying active user information.
The currently presented image may be determined from an associated image associated with the first target participant user.
The current display image may be a partial image cut from the related image, or may be the related image. The associated images herein may include, but are not limited to: still images, and moving images. The associated image here may be preset by the user. The associated image may be a user avatar used by the user.
The area for displaying active user information (i.e. the active user display area) may be, for example, an area in the display screen of the terminal device for displaying active user information of a multimedia conference.
In some application scenarios, when the multimedia conference is in progress, a region for displaying information of participating users and a region for displaying information of active users may be set in a display interface of the multimedia conference.
The area for displaying the information of the participating users can display the information corresponding to a plurality of participating users of the multimedia conference. The information of the participating users can comprise video information of the participating users and also comprise associated images associated with the participating users. The area for displaying information of the participating users can display information of the participating users who are not active currently.
The splicing display of the current display image of the first target participating user and the video information streams of other target participating users comprises the steps of equally dividing the area for displaying active user information according to the number of the target participating users, distributing a display area with the same size for each target participating user, and then displaying the current display image or the video information stream of each target participating user in the corresponding display area.
In some optional implementation manners of this embodiment, the splicing and displaying the currently displayed image of the first target participating user and the video information streams of the other target participating users includes: and for each target participating user, determining the display proportion of the current display image or the current video information stream of the target participating user during splicing display according to the activity of the target participating user in the preset time period.
In these alternative implementations, the display duty of the target participating user with a high activity may be greater than the display duty of the active target participating user.
The display proportion refers to the ratio of the size of the display area of the current display image or video information stream of the target participant user in the display screen to the size of the spliced display area. The tiled display area here may be the area described above for displaying active user information.
When the splicing display is carried out, when the activity of the target joining users changes in the next preset time period, the display proportion of each target joining user can be adjusted in the splicing display area of the next preset time period.
The display proportion of good target participation is set through the activity, the activity degree of each target participation user can be intuitively known in the splicing display area, and the participation users can learn the dynamic state of the multimedia conference.
In the embodiment, at least one target conference participating user is determined from the multimedia conference participating users based on the liveness within the preset time period; and in response to the fact that a first target participating user of the at least one target participating user does not start the video currently exists, splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users. Compared with the scheme that only the information of the active user with the video information stream is displayed in the active user display area in the related art, the scheme provided by the disclosure not only displays the video information stream of the active user with the video information stream, but also can display the current display image of the active user with only the audio information stream. The participating users of the multimedia conference may browse the currently presented image and/or video information streams of the active users in their terminals. The participating users can intuitively browse the image information and/or the video information of the participating users in the active state. In addition, when the information of the active users is displayed in the active user display area, whether the active users turn on the camera or not is not considered, so that the display images and/or the video information streams of the real most active target participating users are displayed in the active user display area. The method and the device can avoid sudden change of the information of the target participating user displayed in the display area of the active user due to the on and off of the video acquisition equipment of the active user in the related technology, and can improve the user experience.
Continuing to refer to FIG. 2, a flow diagram of another embodiment of an information display method according to the present disclosure is shown. As shown in fig. 2, the information display method includes the following steps:
step 201, based on the activity within the preset time period, at least one target conference user is determined from the multimedia conference participant users.
In this embodiment, step 201 may be the same as or similar to step 101 in the embodiment shown in fig. 1, and is not described herein again.
Step 202, in response to determining that a first target participating user of which the video is not started currently exists in the at least one target participating user, splicing and displaying a currently displayed image of the first target participating user and video information streams of other target participating users; the video information streams of other target participating users are obtained based on the following steps: and intercepting and generating the initial video information flow of the target participating user by using the current video acquisition area corresponding to the other target participating users.
In this embodiment, the multimedia information stream of the other target participating user may include the initial video information stream. The initial video information stream is acquired based on the video acquisition device used by the other target participating users.
That is, the other target participating users may use the video capturing device to capture their own video information streams. And then sending the video information stream to other participating users through the server. The video capture device may be embedded in a terminal device (e.g., a mobile terminal, a multimedia conference terminal) used by the other target participating user for participating in the multimedia conference, or may be a separate video capture device.
The initial stream of video information, typically captured by a video capture device, is compressed and then uploaded to a server. The server can forward the compressed initial video information stream of the other target participating users to the terminal equipment used by each participating user for participating in the multimedia conference. The terminal device of the participating user can perform decompression operation on each compressed video frame to recover the initial video information stream of the other target user.
The frame height and the frame width of each initial video frame in the initial video information stream are relatively large, for example, the frame height of the initial video frame can be matched with the height of the display screen of the terminal equipment of the participating user, and the frame width of the initial video frame can be matched with the width of the display screen of the terminal equipment of the participating user.
The current display image needs to be spliced with the video information streams of other target participating users for display. The aspect ratio of the corresponding display area of the other target participant users in the tiled display may be different from the aspect ratio of the initial video frame. If the initial video frame is directly zoomed so that the size of the initial video frame matches the corresponding display area during the tiled display, the zoomed video frame may have distortion phenomena, such as deformation of five sense organs of other target participant users. In order to ensure that the image of the object in the video information stream for the splicing display is not distorted, the initial video information streams of other target participating users can be intercepted by using a video acquisition area. The aspect ratio of the video capture area corresponding to one other target participant user may be the same as the aspect ratio of the display area in the tiled display area corresponding to the other target participant user.
The current video capture area may be used to capture a portion of the image area from a current initial video frame corresponding to the initial video stream. And then splicing and displaying the current video frame obtained by intercepting the current initial video frame by using the current video acquisition area and the current display image.
Compared with the embodiment shown in fig. 1, the information display method provided in this embodiment highlights that the video information streams of other target participating users are generated by intercepting the initial video information streams of the other target participating users by using the current video acquisition regions corresponding to the other target participating users, so that the distortion degree is smaller in the image obtained by the mosaic display.
In the related art, for each participating user, the terminal device of the participating user may display information of an active user (that is, a target participating user) in a display interface of the multimedia conference, for example, the information of the target participating user is displayed in a splicing manner. When the terminal equipment of the user displays the video information stream of the target participating user, for each initial video frame in the video information stream of the target participating user, detecting a face area from the initial video frame of the target participating user, and determining a video acquisition area corresponding to the initial video frame according to the detected face area. Video frames for stitching are truncated from the initial video frame using the video capture area. That is, when displaying the video information stream of the target participating user, the face area in each initial video frame needs to be detected in real time, then the video acquisition area is set according to the face area, and finally the video frame for display is intercepted from each initial video frame by using the video acquisition area determined in real time. According to the scheme, when the video frames corresponding to the target participating users are displayed in a splicing mode, the face images of the target participating users can be usually displayed in the center of the display area corresponding to the target participating users. However, when the position of the target participating user moves relative to the video capture device used by the target participating user and the moving frequency is frequent, the terminal device for displaying the multimedia conference detects the face area in real time, and captures the initial video frame by using the video capture area determined according to the detected face area in real time to obtain the video frame for display, so that the background image of the video information stream of the target participating user continuously changes. The background image of the multimedia conference observed from the angle of the participant users of the multimedia conference has the phenomenon of continuous shaking, which brings bad use experience to the users.
In order to solve the above problem, in some optional implementations of this embodiment, the information display method further includes the following steps:
firstly, for one of other target participating users, judging whether the current video acquisition area of the other target participating user meets the updating condition.
In some application scenarios, for each of the other target participating users, it may be determined whether the current video capture area of the other target participating user satisfies the update condition.
In other application scenarios, for one of the other pre-designated target participating users, it is determined whether the current video capture area of the other target participating user satisfies the update condition.
The update condition here may be, for example, that the duration of use of the video capture area exceeds a preset duration threshold.
Further, the update condition includes that the duration of the video information stream of the other target participating users, which does not include the face, exceeds a preset duration threshold.
The duration of time during which no face is included may be the duration of time during which no face is included in the video stream.
The preset time threshold may be, for example, 0.5s, 1s, and the like. The preset time threshold may be set according to a specific application scenario, and is not limited herein.
And secondly, if the updating condition is met, updating the current video acquisition area, so that the video information streams of other target participating users comprise faces.
When the video acquisition area meets the updating condition, the current video acquisition area can be determined again, so that the video information stream which is intercepted by the current video acquisition area and has good target participation comprises a human face.
In these alternative implementations, the video capture area is not adjusted in real time, and is updated only when the video capture area meets the update condition. That is, for other target participating users, when capturing video information streams from the initial video information streams of the other target participating users, instead of performing face recognition on each initial video frame, determining a video acquisition area according to a face recognition result, and then capturing the initial video frame by using the determined video acquisition area to obtain a video frame for mosaic display corresponding to the initial video frame. Therefore, the phenomenon that the background image of the video information stream of the other target participating user is displayed and shaken due to the fact that the video acquisition area is determined according to the face area in real time in the related technology can be improved.
Please further refer to fig. 3A to 3C, which illustrate schematic diagrams of an application scenario of the information display method according to the present disclosure.
Referring to fig. 3A, the terminal device of the participating user C may receive the multimedia information stream of the multimedia conference sent by the multimedia conference server. The information stream may comprise multimedia information streams of a plurality of participating users.
The liveness of each of the participating users may be determined from the multimedia information streams of the participating users (e.g., A, B, C, D). The activity of each participating user can be determined according to the audio data in the preset time period. For example, the target participating users A and B are determined according to the activity. The target participant user B does not start the video function, and the target participant user B can be the first target participant user. The target participating user A starts the video function, and the target participating user A is used as other target participating users. The associated image of the first target participant user B (e.g., the avatar used by user B) may be taken as the current presentation image 304 of the first target participant user B. The terminal devices of participating user C may display a display interface 301 of the multimedia conference. An area 302 for displaying active user information may be included in the display interface 301. The currently presented image 304 of the first target participant B is stitched with the video information stream 303 of the other target participant a and displayed in the display area 302. The video information streams 303 of the other target participating users a are generated by intercepting the initial video information streams of the other target participating users a using the current video capture area.
In this way, not only active users with a video information stream are presented, but also presentation images of active users with only an audio information stream. The users participating in the multimedia conference can browse the presentation image and video information streams of the active users in their terminal devices. In addition, when the information of the active users is displayed in the active user display area, whether the active users turn on the camera or not is not considered, so that the display images and/or the video information streams of the real most active target participating users are displayed in the active user display area. Sudden changes of the target participating users corresponding to the display areas of the active users caused by the on and off of the video acquisition devices of the active users in the related art can be avoided, and the user experience can be improved.
Referring to fig. 3B and 3C, the other target user a moves relative to the video capturing device he uses. In the terminal device of the participating user C, the video information stream intercepted from the initial video information stream of the other target user a by using the current video acquisition area does not include the face image of the other target user a (as shown in fig. 3B). At this time, it may be determined whether the duration of the video information stream that does not include the face images of the other target users a is greater than a preset duration threshold, and if not, the video information stream 303' of the other target users a captured by using the current video capture area and the current display image 304 of the first target participating user B are displayed in the area 302. If yes, updating the current video acquisition area, so that the video information stream 303' of the other target participating users A intercepted by using the updated current video acquisition area comprises faces. The video information stream 303 "of the other target user a comprising a human face is presented in the area 302 with the currently presented image 304 of the first target participating user B. The phenomenon that the background image of the video information stream of the other target participating user displayed in the splicing display area shakes due to the fact that the video acquisition area is determined in real time according to the face area in the related technology can be improved.
In addition, the display duty of A and B in area 302 may be determined based on the liveness of target participating users A and B. As shown in fig. 3A and fig. 3B, the activity of the target participating user a is greater than that of the user B within the corresponding preset time period, and in the area 302, the display duty of the target participating user a is also greater than that of the user B. In FIG. 3C, the activity of target participant user B is greater than the activity of user A, and in region 302, the display duty of target participant user B may be adjusted to be greater than the display duty of user A.
As shown in fig. 3A to 3C, the display interface 301 may further include an area 305 for displaying information of each participating user. The area for displaying information about each of the participating users may display information about each of the participating users A, B, C, D. The information of each participant A, B, C, D may include video information, image information, text information, and the like. The method and the system can facilitate the participating users to know the detailed information of each participating user. The area 305 may be presented or not presented at the user's choice.
With further reference to fig. 4, as an implementation of the method illustrated in fig. 1 described above, the present disclosure provides one embodiment of an information display apparatus, which corresponds to those method embodiments illustrated in fig. 1, and which may be particularly applicable in various electronic devices.
As shown in fig. 4, the information display device of some embodiments includes: a determination unit 401 and a display unit 402. The determining unit 401 is configured to determine, based on the liveness within the preset time period, at least one target participant user from the multimedia conference participant users; a display unit 402, configured to, in response to determining that a first target participating user that does not start a video currently exists in the at least one target participating user, splice and display a currently displayed image of the first target participating user and video information streams of other target participating users.
In this embodiment, the detailed processing of the determining unit 401 and the displaying unit 402 of the information displaying apparatus and the technical effects thereof can refer to the related descriptions of step 101 and step 102 in the corresponding embodiment of fig. 1, which are not repeated herein.
In an optional implementation manner of some embodiments, the video information stream of the other target participating users is obtained based on the following steps: intercepting and generating initial video information flow of the target participating user by using a current video acquisition area corresponding to the other target participating users; and the initial video information stream is acquired based on the video acquisition equipment used by the other target participating users.
In an optional implementation manner of some embodiments, the information display device further includes a collection area update unit (not shown in the figure). The acquisition area updating unit is used for: for one of the other target participating users, judging whether the current video acquisition area of the other target participating user meets the updating condition; and if the updating condition is met, updating the current video acquisition area, so that the video information streams of the other target participating users comprise faces.
In an optional implementation of some embodiments, the update condition includes: the time length of the video information stream of the other target participating users, which does not include the human face, exceeds a preset time length threshold value.
In an alternative implementation of some embodiments, the liveness of the participating users is determined based on the audio data streams of the participating users.
In an alternative implementation of some embodiments, the information display device further comprises an activity determination unit (not shown in the figures). The activity of the participating users is determined by the activity determination unit based on the following steps: sequencing the accumulated volume of each multimedia conference participant in the preset time period; and determining the activity degree corresponding to each participating user according to the sorting result.
In an optional implementation of some embodiments, the display unit 402 is further configured to: and for each target participating user, determining the display proportion of the current display image or the current video information stream of the target participating user during the splicing display according to the activity of the target participating user in the preset time period.
The embodiment provides a pair of information display devices, wherein a determining unit is used for determining at least one target participating user from multimedia conference participating users based on the activity within a preset time period; and the display unit is used for splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users in response to the fact that the first target participating user which does not start the video currently exists in the at least one target participating user. Compared with the scheme that only the information of the active users with the video information streams is displayed in the active user display area in the related art, the scheme provided by the disclosure not only shows the active users with the video information streams, but also shows the display images of the active users with only the audio information streams. The participating users of the multimedia conference can browse the presentation image and/or video information streams of the active users in their terminals. The participating users can intuitively browse the image information or the video information of the participating users in an active state, and the user experience can be improved.
With further reference to fig. 5, fig. 5 illustrates an exemplary system architecture of an information display method or an information display apparatus to which some embodiments of the present disclosure may be applied.
As shown in fig. 5, the system architecture may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the terminal devices 501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal devices 501, 502, 503 may interact with a server 505 over a network 504 to receive or send messages or the like. The terminal devices 501, 502, 503 may have various client applications installed thereon, such as a web browser application, a search-type application, news information, a multimedia conference-type application, and the like. The client application in the terminal device 501, 502, 503 may receive the instruction of the user, and complete a corresponding function according to the instruction of the user, for example, receive the information stream of the multimedia conference according to the instruction of the user.
The terminal devices 501, 502, 503 may be hardware or software. When the terminal devices 501, 502, 503 are hardware, they may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture Experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like. When the terminal devices 501, 502, and 503 are software, they can be installed in the electronic devices listed above. It may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 505 may be a server providing various services, for example, receiving information streams of the multimedia conference sent by the terminal devices 501, 502, and 503, analyzing the information streams, and sending the analyzed information streams to the terminal devices 501, 502, and 503.
It should be noted that the information processing method provided by the embodiment of the present disclosure may be executed by a terminal device, and accordingly, the information processing apparatus may be provided in the terminal device 501, 502, 503. In addition, the information processing method provided by the embodiment of the present disclosure may also be executed by the server 505, and accordingly, an information processing apparatus may be provided in the server 505.
It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to fig. 6, a schematic diagram of an electronic device (e.g., a server or terminal device of fig. 5) suitable for use in implementing some embodiments of the present disclosure is shown. The terminal device in some embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The terminal device/server shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, the electronic device may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; a storage device 608 including, for example, a magnetic tape, a hard disk, a memory card, or the like; and a communication device 609. The communication means 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of some embodiments of the present disclosure.
It should be noted that the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: determining at least one target conference participating user from the multimedia conference participating users based on the liveness within the preset time period; and in response to the fact that a first target participating user of the at least one target participating user does not start the video currently exists, splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. Where the name of a unit does not in some cases constitute a limitation on the unit itself, for example, the determining unit may also be described as "determining at least one target participant user from among the multimedia conference participant users based on activity within a preset time period".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (10)
1. An information display method, comprising:
determining at least one target conference participating user from the multimedia conference participating users based on the liveness within the preset time period;
and in response to the fact that a first target participating user of the at least one target participating user does not start the video currently exists, splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users.
2. The method of claim 1, wherein the video stream of the other target participating user is obtained based on the following steps:
intercepting and generating initial video information flow of the target participating user by using a current video acquisition area corresponding to the other target participating users; and the initial video information stream is acquired based on the video acquisition equipment used by the other target participating users.
3. The method of claim 2, further comprising:
for one of the other target participating users, judging whether the current video acquisition area of the other target participating user meets the updating condition;
and if the updating condition is met, updating the current video acquisition area, so that the video information streams of the other target participating users comprise faces.
4. The method of claim 3, wherein the update condition comprises: the time length of the video information stream of the other target participating users, which does not include the human face, exceeds a preset time length threshold value.
5. The method of claim 1, wherein the activity level of the participating user is determined based on the audio data stream of the participating user.
6. The method of claim 5, wherein the activity level of the participating user is determined based on the steps of:
sequencing the accumulated volume of each multimedia conference participant in the preset time period;
and determining the activity degree corresponding to each participating user according to the sorting result.
7. The method of claim 1, wherein said displaying the currently presented image of the first target participating user in mosaic with the video information streams of the other target participating users comprises:
and for each target participating user, determining the display proportion of the current display image or the current video information stream of the target participating user during the splicing display according to the activity of the target participating user in the preset time period.
8. An information display device characterized by comprising:
the determining unit is used for determining at least one target conference user from the multimedia conference participant users based on the activity within the preset time period;
and the display unit is used for splicing and displaying the current display image of the first target participating user and the video information streams of other target participating users in response to the fact that the first target participating user which does not start the video currently exists in the at least one target participating user.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010759690.4A CN111935442A (en) | 2020-07-31 | 2020-07-31 | Information display method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010759690.4A CN111935442A (en) | 2020-07-31 | 2020-07-31 | Information display method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111935442A true CN111935442A (en) | 2020-11-13 |
Family
ID=73315037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010759690.4A Pending CN111935442A (en) | 2020-07-31 | 2020-07-31 | Information display method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111935442A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818303A (en) * | 2021-02-01 | 2021-05-18 | 北京字跳网络技术有限公司 | Interaction method and device and electronic equipment |
| CN113596384A (en) * | 2021-07-30 | 2021-11-02 | 北京字跳网络技术有限公司 | Automatic layout method, device and equipment for multimedia conference |
| CN113949891A (en) * | 2021-10-13 | 2022-01-18 | 咪咕文化科技有限公司 | A video processing method, device, server and client |
| CN114697149A (en) * | 2022-04-01 | 2022-07-01 | 北京字跳网络技术有限公司 | View refreshing method, device, equipment and medium for video conference |
| WO2025092790A1 (en) * | 2023-11-02 | 2025-05-08 | 广州视源电子科技股份有限公司 | Display method for video conference picture, and device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105357442A (en) * | 2015-11-27 | 2016-02-24 | 小米科技有限责任公司 | Shooting angle adjustment method and device for camera |
| US20160308920A1 (en) * | 2015-04-16 | 2016-10-20 | Microsoft Technology Licensing, Llc | Visual Configuration for Communication Session Participants |
| US20170280098A1 (en) * | 2014-09-26 | 2017-09-28 | Intel Corporation | Techniques for enhancing user experience in video conferencing |
| US20170332044A1 (en) * | 2016-05-12 | 2017-11-16 | Fuji Xerox Co., Ltd. | System and method for replacing user media streams with animated avatars in live videoconferences |
| CN109151309A (en) * | 2018-08-31 | 2019-01-04 | 北京小鱼在家科技有限公司 | A kind of method for controlling rotation of camera, device, equipment and storage medium |
-
2020
- 2020-07-31 CN CN202010759690.4A patent/CN111935442A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170280098A1 (en) * | 2014-09-26 | 2017-09-28 | Intel Corporation | Techniques for enhancing user experience in video conferencing |
| US20160308920A1 (en) * | 2015-04-16 | 2016-10-20 | Microsoft Technology Licensing, Llc | Visual Configuration for Communication Session Participants |
| CN105357442A (en) * | 2015-11-27 | 2016-02-24 | 小米科技有限责任公司 | Shooting angle adjustment method and device for camera |
| US20170332044A1 (en) * | 2016-05-12 | 2017-11-16 | Fuji Xerox Co., Ltd. | System and method for replacing user media streams with animated avatars in live videoconferences |
| CN109151309A (en) * | 2018-08-31 | 2019-01-04 | 北京小鱼在家科技有限公司 | A kind of method for controlling rotation of camera, device, equipment and storage medium |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818303A (en) * | 2021-02-01 | 2021-05-18 | 北京字跳网络技术有限公司 | Interaction method and device and electronic equipment |
| CN112818303B (en) * | 2021-02-01 | 2024-03-05 | 北京字跳网络技术有限公司 | Interactive methods, devices and electronic devices |
| CN113596384A (en) * | 2021-07-30 | 2021-11-02 | 北京字跳网络技术有限公司 | Automatic layout method, device and equipment for multimedia conference |
| CN113596384B (en) * | 2021-07-30 | 2023-05-26 | 北京字跳网络技术有限公司 | Automatic layout method, device and equipment for multimedia conference |
| CN113949891A (en) * | 2021-10-13 | 2022-01-18 | 咪咕文化科技有限公司 | A video processing method, device, server and client |
| CN113949891B (en) * | 2021-10-13 | 2023-12-08 | 咪咕文化科技有限公司 | Video processing method and device, server and client |
| CN114697149A (en) * | 2022-04-01 | 2022-07-01 | 北京字跳网络技术有限公司 | View refreshing method, device, equipment and medium for video conference |
| CN114697149B (en) * | 2022-04-01 | 2023-10-24 | 北京字跳网络技术有限公司 | View refreshing method, device, equipment and medium for video conference |
| WO2025092790A1 (en) * | 2023-11-02 | 2025-05-08 | 广州视源电子科技股份有限公司 | Display method for video conference picture, and device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111935442A (en) | Information display method and device and electronic equipment | |
| CN111899192B (en) | Interaction method, interaction device, electronic equipment and computer-readable storage medium | |
| CN112312225B (en) | Information display method and device, electronic equipment and readable medium | |
| CN112256223B (en) | Method and device for adjusting display fluency and electronic equipment | |
| CN111277893B (en) | Video processing method and device, readable medium and electronic equipment | |
| CN112312224A (en) | Information display method and device and electronic equipment | |
| CN110809189B (en) | Video playing method and device, electronic equipment and computer readable medium | |
| CN112203038B (en) | Processing method and device for online conference, electronic equipment and computer storage medium | |
| CN112312223A (en) | Information display method and device and electronic equipment | |
| US20250158845A1 (en) | Audio data pushing method, apparatus and system, and electronic device and storage medium | |
| CN112351222B (en) | Image special effect processing method and device, electronic equipment and computer readable storage medium | |
| CN112291502B (en) | Information interaction method, device and system and electronic equipment | |
| CN114371896B (en) | Prompting method, device, equipment and medium based on document sharing | |
| CN112261349B (en) | Image processing method and device and electronic equipment | |
| CN111459364B (en) | Icon updating method and device and electronic equipment | |
| CN110070496A (en) | Generation method, device and the hardware device of image special effect | |
| CN111147885B (en) | Live broadcast room interaction method and device, readable medium and electronic equipment | |
| CN116939130A (en) | Video generation method and device, electronic equipment and storage medium | |
| CN112218034A (en) | Video processing method, system, terminal and storage medium | |
| CN110809166A (en) | Video data processing method and device and electronic equipment | |
| CN111209432A (en) | Information acquisition method and device, electronic equipment and computer readable medium | |
| CN110457106B (en) | Information display method, device, equipment and storage medium | |
| CN112243157A (en) | Live broadcast control method and device, electronic equipment and computer readable medium | |
| CN112256221A (en) | Information display method and device and electronic equipment | |
| CN114339402B (en) | Video playing completion rate prediction method and device, medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201113 |