CN115002513B - Audio and video scheduling method and device, electronic equipment and computer readable storage medium - Google Patents

Abstract

The application provides an audio and video scheduling method, an audio and video scheduling device, electronic equipment and a computer readable storage medium, and relates to the technical field of audio and video. The method comprises the following steps: acquiring a first user experience index value corresponding to a target moment; acquiring a first link portrait corresponding to a first preset time period before a target moment under the condition that the first user experience index value is lower than a first threshold value; inputting the first link portrait into a user experience model to obtain a target user experience index value corresponding to a second time period; and adjusting the link and/or the code rate of the audio/video playing corresponding to the second time period according to the target user experience index value and the user experience model. According to the scheme provided by the embodiment of the application, the link and/or the code rate of the audio/video playing can be dynamically scheduled according to the user experience index value of the audio playing, the influence of the public network in a weak network environment on the audio/video playing is avoided, and the efficient operation of the network link is ensured.

Description

Audio and video scheduling methods, devices, electronic equipment and computer-readable storage media

Technical field

The present invention relates to the field of audio and video technology, and in particular, to an audio and video scheduling method, device, electronic equipment and computer-readable storage medium.

Background technique

As the concept of zero-contact and remote working becomes more and more popular across the world, how to provide customers with highly available, high-quality, low-latency real-time audio and video services in a public network environment and ensure the efficient operation of network links is becoming increasingly important. Currently, in order to ensure low latency and high quality of real-time audio and video in a public network environment, when it is detected that the network is in a weak network scenario, key frame retransmission, bit rate control, and audio and video forward error correction (Forward Error Correction) are used. , FEC) means to retry or reduce the bandwidth to ensure the quality of audio and video communication and reduce the impact of weak network scenarios on audio and video playback. However, if the network is always in a weak network scenario, unrestricted retransmission or bandwidth reduction will lead to retransmission storms or low bit rates, thus affecting user experience.

Contents of the invention

The purpose of the present invention is to provide an audio and video scheduling method, device, electronic equipment and computer-readable storage medium to solve the problem in the existing technology that if the network is always in a weak network scenario, unrestricted retransmission or reduced bandwidth will cause retransmission. Problems with transmission storm or low code rate.

In a first aspect, embodiments of the present invention provide an audio and video scheduling method, including:

Obtain the first user experience indicator value corresponding to the target moment;

When the first user experience index value is lower than the first threshold, obtain the first link portrait corresponding to the first preset time period before the target time; the link portrait is used to indicate the audio and video playback link Related Information;

Input the first link portrait into the user experience model to obtain the target user experience index value corresponding to the second time period; the second time period includes the second preset time period after the target time; the user experience model uses It is used to represent the correspondence between the user video playback experience index value and the link portrait under the target audio and video playback bit rate;

According to the target user experience indicator value and the user experience model, adjust the link and/or code rate of the audio and video playback corresponding to the second time period.

Optionally, the user experience indicator includes at least one of the following: video resolution, video frame rate, video freezing rate, audio freezing rate, frame rate per second, and packet loss rate.

Optionally, the obtaining the first user experience indicator value corresponding to the target moment includes:

Obtain the first link portrait corresponding to the target time, and determine the link portrait feature value based on the first link portrait;

The user experience index value is determined based on the link portrait characteristic value.

Optionally, the corresponding first link image at the target moment of acquisition includes:

Receive the audio and video link data reported by the user at the target time;

Analyze and process the audio and video link data to generate the first link portrait.

Optionally, the audio and video link data includes at least one of the following: timestamp, edge server, terminal model, operating system, network type, packet loss rate, video resolution, frame rate, sending code rate, receiving code Rate;

The first link portrait includes at least one of the following: time attributes, terminal attributes, network attributes, regional attributes, edge server attributes, and experience attributes;

The link portrait feature value includes at least one of the following: time element, terminal element, network element, regional element, edge server element, and experience element.

Optionally, the user experience model is a model obtained by performing model training on the first model:

The first model is:

Among them, f(V|θ) represents the user experience index value; V is the link portrait characteristic value; Mi is the i-th link portrait experience index value corresponding to the link portrait characteristic value, and δi is the i-th user experience The weight of the indicator, θ is the audio and video playback bit rate; n is an integer greater than or equal to 1;

The link portrait experience index value includes at least one of the following: video resolution value, video frame rate value, video freezing rate reciprocal value, audio freezing rate reciprocal value, and second opening rate value.

Optionally, perform model training on the first model to obtain the user experience model, including:

Obtain a historical link portrait set; the historical link portrait set includes link portraits before the current moment;

According to the historical link portrait set, perform model training on the first model through logistic regression algorithm and gradient descent method to obtain the weight of each user experience indicator;

The experience model is determined according to the first model and the weight of each user experience indicator.

Optionally, determining the user experience index value based on the link portrait characteristic value includes:

According to the link portrait characteristic value, the user experience index value is calculated through the user experience model.

Optionally, adjusting the link and/or code rate of audio and video playback corresponding to the second time period according to the target user experience indicator value and the user experience model includes:

When the target user experience index value is less than the second threshold, adjust the link and/or code rate corresponding to the user experience model until the target user experience index value corresponding to the user experience model is greater than or equal to the second threshold. threshold;

When the target user experience indicator value is greater than or equal to the second threshold, the link and code rate corresponding to the user experience model are used as the adjusted link and code rate for audio and video playback corresponding to the second time period.

In the second aspect, embodiments of the present invention provide an audio and video scheduling device, including:

The first acquisition module is used to acquire the first user experience indicator value corresponding to the target moment;

The second acquisition module is used to acquire the first link portrait corresponding to the first preset time period before the target time when the first user experience index value is lower than the first threshold; the link portrait is used to Indicates relevant information of audio and video playback links;

A processing module configured to input the first link portrait into the user experience model to obtain the target user experience index value corresponding to the second time period; the second time period includes a second preset time period after the target moment; The user experience model described above is used to characterize the correspondence between user video playback experience index values and link portraits under the target audio and video playback bit rate;

An adjustment module, configured to adjust the link and/or code rate of the audio and video playback corresponding to the second time period according to the target user experience indicator value and the user experience model.

In a third aspect, embodiments of the present invention provide an electronic device, including: a transceiver, a processor, a memory, and a program or instructions stored on the memory and executable on the processor; the processor When the program or instruction is executed, the audio and video scheduling method as described above is implemented.

In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium on which a program or instructions are stored. When the program or instructions are executed by a processor, the steps in the audio and video scheduling method as described above are implemented.

The audio and video scheduling method of the embodiment of the present invention can monitor the user experience of audio and video playback by obtaining the first user experience index value corresponding to the target time. When it is detected that the first user experience index value is lower than the first threshold, In this case, the target user experience index value is predicted based on the first link portrait and the user experience model; and the link corresponding to the audio and video playback in the second time period is predicted based on the predicted target user experience index value. and/or bitrate adjustment. The solution of the present invention can dynamically schedule the link and/or code rate of audio and video playback according to the user experience index value of audio playback, avoid the impact of audio and video playback due to the weak network environment of the public network, and ensure the high efficiency of the network link. Operation solves the problem in the existing technology that if the network is always in a weak network scenario, infinite retransmissions lead to retransmission storms or infinite bandwidth reduction leads to too low a code rate.

Description of the drawings

Figure 1 is a schematic flow chart of an audio and video scheduling method according to an embodiment of the present invention;

Figure 2 is a schematic diagram of the first link portrait elements according to the embodiment of the present invention;

Figure 3 is a schematic module diagram of an audio and video scheduling device according to an embodiment of the present invention;

Figure 4 is a schematic diagram of the physical device of the audio and video scheduling device according to the embodiment of the present invention;

FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments. In the following description, specific details, such as specific configurations and components, are provided solely to assist in a comprehensive understanding of embodiments of the invention. It will therefore be apparent to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Additionally, a description of inhibitory function and construction has been omitted for clarity and brevity.

It will be understood that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the invention. Thus, the appearances of "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments.

In various embodiments of the present invention, it should be understood that the size of the sequence numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its functions and internal logic, and should not be implemented in the present invention. The implementation of the examples does not constitute any limitations.

Additionally, the terms "system" and "network" are often used interchangeably in this article.

In the embodiments provided in this application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B only based on A. B can also be determined based on A and/or other information.

As shown in Figure 1, an audio and video scheduling method according to the embodiment of the present invention includes the following steps:

Step 101: Obtain the first user experience index value corresponding to the target time.

The audio and video scheduling method according to the embodiment of the present invention can monitor the user experience index value of audio playback in real time by obtaining the first user experience index value corresponding to the target time.

Step 102: When the first user experience index value is lower than the first threshold, obtain the first link portrait corresponding to the first preset time period before the target time; the link portrait is used to instruct audio and video playback Information about the link.

It should be noted that if the first user experience index value is lower than the first threshold, that is, the public network is in a weak network environment, and the environment of the public network is determined based on the first user experience index value.

In an embodiment of the present invention, the link portrait may include: time attributes, terminal attributes, network attributes, regional attributes, server attributes, and experience attributes. Representing the relevant information of audio and video playback links through link portraits can greatly reduce the magnitude of link data and increase the speed of calculating the target user experience index value.

Step 103: Enter the first link portrait into the user experience model to obtain the target user experience index value corresponding to the second time period; the second time period includes the second preset time period after the target time; the user The experience model is used to characterize the correspondence between user video playback experience indicator values and link portraits under the target audio and video playback bit rate.

It should be noted that the user experience model is a model after learning and training. According to the first link portrait, the target user experience index value corresponding to the second time period is predicted through the user experience model.

Step 104: Adjust the link and/or code rate of the audio and video playback corresponding to the second time period according to the target user experience indicator value and the user experience model.

In one embodiment of the present invention, link adjustment mainly refers to adjusting the edge server so that the user's audio and video experience is optimal during the second time period.

The solution of the present invention can trigger dynamic adjustment of the link and/or code rate when it is detected that the user experience index value of audio and video playback is lower than the first threshold. It avoids the impact of audio and video playback caused by the weak network environment of the public network, ensures the efficient operation of network links, and solves the problem in the existing technology that if the network is always in a weak network scenario, infinite retransmissions lead to retransmission storms or infinite bandwidth reduction. Causes the problem of too low code rate.

Optionally, the user experience indicator includes at least one of the following: video resolution, video frame rate, video freezing rate, audio freezing rate, frame rate per second, and packet loss rate.

In an embodiment of the present invention, the first threshold corresponding to the video resolution may be 800*480 or 1280*720; the first threshold corresponding to the video frame rate may be 30fps; the video freezes The first threshold corresponding to the audio freezing rate may be 6%; the first threshold corresponding to the audio freezing rate may be 2%.

In an embodiment of the present invention, the user experience index value is used to determine whether the user's public network is in a weak network state.

Optionally, the obtaining the first user experience indicator value corresponding to the target moment includes:

Obtain the first link portrait corresponding to the target time, and determine the link portrait feature value based on the first link portrait;

The user experience index value is determined based on the link portrait characteristic value.

In an embodiment of the present invention, determining the link portrait characteristic value based on the first link portrait includes:

The first link portrait is processed through a hash function to obtain the link portrait feature value.

Through the data format of the link portrait, the magnitude of the link data can be effectively reduced, thereby reducing the amount of calculation required to determine the user experience index value.

Optionally, the corresponding first link image at the target moment of acquisition includes:

Receive the audio and video link data reported by the user at the target time;

Analyze and process the audio and video link data to generate the first link portrait.

Link portrait is an important factor affecting the quality of link communication. In one embodiment of the present invention, the user reports audio and video link data every 5 seconds, and saves the received audio and video link data to obtain an audio and video link data set; The set is parsed and processed to generate a link portrait.

Optionally, the audio and video link data includes at least one of the following: timestamp, edge server, terminal model, operating system, network type, packet loss rate, video resolution, frame rate, sending code rate, receiving code Rate;

As shown in Figure 2, the first link portrait includes at least one of the following: time attributes, terminal attributes, network attributes, regional attributes, edge server attributes, and experience attributes;

The link portrait feature value includes at least one of the following: time element, terminal element, network element, regional element, edge server element, and experience element.

The solution of the embodiment of the present invention can more comprehensively describe and measure the link by establishing a link portrait based on the terminal model, audio and video service quality indicators, server elements, etc. It avoids adjusting the communication quality of the public network link only based on the server, without considering quality factors such as time period, terminal (model), server load, bit rate, packet loss rate, resolution, etc.

Optionally, the user experience model is a model obtained by performing model training on the first model:

The first model is:

Among them, f(V|θ) represents the user experience index value; V is the link portrait characteristic value; Mi is the i-th link portrait experience index value corresponding to the link portrait characteristic value, and δi is the i-th user experience The weight of the indicator, θ is the audio and video playback bit rate; n is an integer greater than or equal to 1;

The link portrait experience index value includes at least one of the following: video resolution value, video frame rate value, video freezing rate reciprocal value, audio freezing rate reciprocal value, and second opening rate value.

In an embodiment of the present invention, the link portrait characteristic value V is under the condition of code rate θ, and the user experience index value is calculated by the first model.

Optionally, perform model training on the first model to obtain the user experience model, including:

Obtain a historical link portrait set; the historical link portrait set includes link portraits before the current moment;

According to the historical link portrait set, perform model training on the first model through logistic regression algorithm and gradient descent method to obtain the weight of each user experience indicator;

The experience model is determined according to the first model and the weight of each user experience indicator.

The historical link portrait set is determined by parsing and storing the audio and video link data received at historical moments.

The first model is learned and trained through the historical link portrait set to determine the weight of each user experience indicator, and then obtain the user experience model, so that the user experience model meets the requirements of good model performance, good network status, and edge The server load is low, and under the condition of a certain code rate θ, the user experience score is expected to be higher.

According to the solution of the embodiment of the present invention, the first model is trained according to the historical link portrait set, and the first model is learned and trained through the historical link portrait set that stores a large amount of historical data, which can ensure training The accuracy of the user experience model is then obtained; and the calculation of the target user experience index value only uses the first link portrait corresponding to the first preset time period, which saves computing resources and improves computing efficiency. .

Optionally, determining the user experience index value based on the link portrait feature value includes:

According to the link portrait characteristic value, the user experience index value is calculated through the user experience model.

A practical solution of the present invention is to determine the network environment of the public network used by the user through the user experience index value. When the user experience index value is lower than the first threshold, it is considered that the network operating environment of the public network is poor and will affect The user's audio and video playback experience provides a trigger mechanism for dynamic adjustment of audio and video playback links and/or bit rates.

Optionally, adjusting the link and/or code rate of audio and video playback corresponding to the second time period according to the target user experience indicator value and the user experience model includes:

When the target user experience index value is less than the second threshold, adjust the link and/or code rate corresponding to the user experience model until the target user experience index value corresponding to the user experience model is greater than or equal to the second threshold. threshold;

When the target user experience indicator value is greater than or equal to the second threshold, the link and code rate corresponding to the user experience model are used as the adjusted link and code rate for audio and video playback corresponding to the second time period.

In an embodiment of the present invention, when calculating the target user experience index value, according to the user experience model, the first link portrait is the input, and the target user experience index value is the output; for all When the audio and video playback link and/or code rate corresponding to the second time period are adjusted, the target user experience indicator value is the input, for example, the second threshold is the input. According to the user experience model, the second The link and/or bitrate of the audio and video playback corresponding to the time period is output.

The practical solution of the present invention passes the second threshold and adjusts the link and code rate of the audio and video playback corresponding to the second time period according to the user experience model, so that the audio and video playback maintains good links and code rates. rate to obtain a better user audio and video experience.

As shown in Figure 3, an embodiment of the present invention provides an audio and video scheduling device 300, which includes:

The first acquisition module 301 is used to acquire the first user experience index value corresponding to the target time;

The second acquisition module 302 is configured to acquire the first link portrait corresponding to the first preset time period before the target time when the first user experience index value is lower than the first threshold; the link portrait is Information related to indicating audio and video playback links;

The processing module 303 is configured to input the first link portrait into the user experience model to obtain the target user experience index value corresponding to the second time period; the second time period includes a second preset time period after the target moment; The user experience model is used to characterize the correspondence between the user video playback experience index value and the link portrait under the target audio and video playback bit rate;

The adjustment module 304 is configured to adjust the link and/or code rate of the audio and video playback corresponding to the second time period according to the target user experience indicator value and the user experience model.

The audio and video scheduling device in the embodiment of the present invention can monitor the user experience of audio and video playback by obtaining the first user experience index value corresponding to the target time. When it is detected that the first user experience index value is lower than the first threshold, In this case, the target user experience index value is predicted based on the first link portrait and the user experience model; and the link corresponding to the audio and video playback in the second time period is predicted based on the predicted target user experience index value. and/or bitrate adjustment. The solution of the present invention can dynamically schedule the link and/or code rate of audio and video playback according to the user experience index value of audio playback, avoid the impact of audio and video playback due to the weak network environment of the public network, and ensure the high efficiency of the network link. Operation solves the problem in the existing technology that if the network is always in a weak network scenario, infinite retransmissions lead to retransmission storms or infinite bandwidth reduction leads to too low a code rate.

Optionally, the first acquisition module also includes:

The acquisition subunit is used to obtain the first link portrait corresponding to the target time, and determine the link portrait characteristic value according to the first link portrait;

Determining subunit, configured to determine the user experience index value according to the link portrait characteristic value.

Optionally, the acquisition subunit is also used to:

Receive the audio and video link data reported by the user at the target time;

Analyze and process the audio and video link data to generate the first link portrait.

Optionally, the determining subunit is also used to:

According to the link portrait characteristic value, the user experience index value is calculated through the user experience model.

Optionally, the adjustment module is also used to:

When the target user experience index value is less than the second threshold, adjust the link and/or code rate corresponding to the user experience model until the target user experience index value corresponding to the user experience model is greater than or equal to the second threshold. threshold;

When the target user experience indicator value is greater than or equal to the second threshold, the link and code rate corresponding to the user experience model are used as the adjusted link and code rate for audio and video playback corresponding to the second time period.

As shown in Figure 4, in one embodiment of the present invention, the physical device of the audio and video scheduling device may correspond to a data collection module, a quality data module, a link portrait module, a machine learning module, a computing module, an instruction module, and a dynamic scheduling module.

Optionally, the data collection module is used for:

Obtain the first user experience indicator value corresponding to the target moment; and,

When the first user experience index value is lower than the first threshold, the first link portrait corresponding to the first preset time period before the target time is obtained.

Optionally, the computing module is used for:

The first link portrait is input into the user experience model to obtain the target user experience index value corresponding to the second time period.

Optionally, the instruction module is configured to generate scheduling instructions according to the target user experience indicator value.

Optionally, the dynamic scheduling module is used for:

According to the target user experience indicator value and the user experience model, adjust the link and/or code rate of the audio and video playback corresponding to the second time period.

Optionally, the link portrait module is used for:

Obtain the first link portrait corresponding to the target time, and determine the link portrait feature value based on the first link portrait.

Optionally, the computing module is also used to:

The user experience index value is determined based on the link portrait characteristic value.

Optionally, the data collection module is also used to:

Receive the audio and video link data reported by the user at the target time;

Analyze and process the audio and video link data to generate the first link portrait.

Optionally, the data collection module is also used to:

Obtain a historical link portrait set; the historical link portrait set includes link portraits before the current moment.

Optionally, the machine learning module is used for:

According to the historical link portrait set, perform model training on the first model through logistic regression algorithm and gradient descent method to obtain the weight of each user experience indicator;

The experience model is determined according to the first model and the weight of each user experience indicator.

Optionally, the computing module is used for:

According to the link portrait characteristic value, the user experience index value is calculated through the user experience model.

Optionally, the dynamic scheduling module is also used to:

When the target user experience index value is less than the second threshold, adjust the link and/or code rate corresponding to the user experience model until the target user experience index value corresponding to the user experience model is greater than or equal to the second threshold. threshold;

When the target user experience indicator value is greater than or equal to the second threshold, the link and code rate corresponding to the user experience model are used as the adjusted link and code rate for audio and video playback corresponding to the second time period.

As shown in Figure 5, an embodiment of the present invention provides an electronic device, including a transceiver 510, a processor 500, a memory 520, and a program stored on the memory 520 and executable on the processor 500 or Instructions; when the processor 500 executes the program or instructions, the above audio and video scheduling method is applied to the above.

The transceiver 510 is used to receive and send data under the control of the processor 500.

In FIG. 5 , the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 500 and various circuits of the memory represented by memory 520 are linked together. The bus architecture can also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. The bus interface provides the interface. Transceiver 510 may be a plurality of elements, including a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 530 can also be an interface that can connect external and internal required equipment. The connected equipment includes but is not limited to a keypad, a display, a speaker, a microphone, a joystick, etc.

The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 can store data used by the processor 500 when performing operations.

Optionally, the transceiver 510 is used to obtain the first user experience indicator value corresponding to the target moment;

When the first user experience index value is lower than the first threshold, obtain the first link portrait corresponding to the first preset time period before the target time; the link portrait is used to indicate the audio and video playback link Related Information.

The processor 500 is configured to input the first link portrait into a user experience model to obtain a target user experience index value corresponding to a second time period; the second time period includes a second preset time period after the target time. ; The user experience model is used to characterize the correspondence between the user video playback experience index value and the link portrait under the target audio and video playback code rate;

According to the target user experience indicator value and the user experience model, adjust the link and/or code rate of the audio and video playback corresponding to the second time period.

Optionally, the transceiver 510 is also used for:

Obtain the first link portrait corresponding to the target time, and determine the link portrait feature value based on the first link portrait;

The user experience index value is determined based on the link portrait characteristic value.

Optionally, the transceiver 510 is also used to

Receive the audio and video link data reported by the user at the target time;

Analyze and process the audio and video link data to generate the first link portrait.

Optionally, the transceiver 510 is also used for:

Obtain a historical link portrait set; the historical link portrait set includes link portraits before the current moment;

Optionally, the processor 500 is also used to:

According to the historical link portrait set, perform model training on the first model through logistic regression algorithm and gradient descent method to obtain the weight of each user experience indicator;

The experience model is determined according to the first model and the weight of each user experience indicator.

Wherein, the first model is:

Among them, f(V|θ) represents the user experience index value; V is the link portrait characteristic value; Mi is the i-th link portrait experience index value corresponding to the link portrait characteristic value, and δi is the i-th user experience The weight of the indicator, θ is the audio and video playback bit rate; n is an integer greater than or equal to 1;

The link portrait experience index value includes at least one of the following: video resolution value, video frame rate value, video freezing rate reciprocal value, audio freezing rate reciprocal value, and second opening rate value.

Optionally, the processor 500 is also used to:

According to the link portrait characteristic value, the user experience index value is calculated through the user experience model.

Optionally, the processor 500 is also used to:

When the target user experience index value is less than the second threshold, adjust the link and/or code rate corresponding to the user experience model until the target user experience index value corresponding to the user experience model is greater than or equal to the second threshold. threshold;

When the target user experience indicator value is greater than or equal to the second threshold, the link and code rate corresponding to the user experience model are used as the adjusted link and code rate for audio and video playback corresponding to the second time period.

Embodiments of the present invention provide a computer-readable storage medium on which programs or instructions are stored. When the programs or instructions are executed by a processor, the steps in the audio and video scheduling method as described above are implemented.

Wherein, the processor is the processor in the electronic device described in the above embodiment. The readable storage media includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks.

It should be further noted that the terminals described in this specification include but are not limited to smartphones, tablet computers, etc., and many of the functional components described are called modules to more specifically emphasize the independence of their implementation.

In the embodiment of the present invention, the module can be implemented in software so as to be executed by various types of processors. For example, an identified module of executable code may include one or more physical or logical blocks of computer instructions, which may be structured, for example, as an object, procedure, or function. Nonetheless, the executable code of an identified module need not be physically located together, but may include different instructions stored on different bits that, when logically combined, constitute the module and implement the provisions of the module Purpose.

In fact, an executable code module can be a single instruction or many instructions, and can even be distributed over multiple different code segments, distributed among different programs, and distributed across multiple memory devices. Likewise, operational data may be identified within modules and may be implemented in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations (including on different storage devices), and may exist, at least in part, solely as electronic signals on a system or network.

When the module can be implemented using software, taking into account the level of existing hardware technology, those skilled in the art can build corresponding hardware circuits to implement the corresponding functions without considering the cost. The hardware circuits include conventional very large scale integration (VLSI) circuits or gate arrays as well as existing semiconductors such as logic chips, transistors, or other discrete components. Modules can also be implemented using programmable hardware devices, such as field programmable gate arrays, programmable array logic, programmable logic devices, etc.

The above exemplary embodiments have been described with reference to the accompanying drawings. Many different forms and embodiments are possible without departing from the spirit and teachings of the invention. Therefore, the invention should not be construed as the exemplary embodiments set forth herein. limits. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will convey the scope of the invention to those skilled in the art. In the drawings, component sizes and relative sizes may be exaggerated for clarity. The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will be further understood that the terms "comprising" and/or "including" when used in this specification indicate the presence of stated features, integers, steps, operations, components and/or components, but do not exclude the presence of one or more other The existence or addition of features, integers, steps, operations, components, components and/or families thereof. Unless otherwise indicated, when stated, a range of values includes the upper and lower limits of the range and any subranges therebetween.

The above is the preferred embodiment of the present invention. It should be pointed out that for those of ordinary skill in the art, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications can also be made. should be regarded as the protection scope of the present invention.

Claims (11)

1. An audio/video scheduling method is characterized by comprising the following steps:

acquiring a first user experience index value corresponding to a target moment;

acquiring a first link portrait corresponding to a first preset time period before a target moment under the condition that the first user experience index value is lower than a first threshold value; the link portrait is used for indicating related information of an audio/video playing link;

inputting the first link portrait into a user experience model to obtain a target user experience index value corresponding to a second time period; the second time period comprises a second preset time period after the target time; the user experience model is used for representing the corresponding relation between the user video playing experience index value and the link portrait under the target audio and video playing code rate;

according to the target user experience index value and the user experience model, adjusting a link and/or a code rate of audio and video playing corresponding to the second time period;

the user experience model is a model obtained by model training of the first model:

the first model is as follows:

wherein f (v|θ) represents a user experience index value; v is the link portrait characteristic value; mi is the ith link image experience index value corresponding to the link image characteristic value, δi is the weight of the ith user experience index, and θ is the audio/video playing code rate; n is an integer greater than or equal to 1;

the link image experience index value comprises at least one of the following: video resolution value, video frame value, video clip rate reciprocal value, audio clip rate reciprocal value, second opening value.

2. The audio-video scheduling method according to claim 1, wherein the user experience index comprises at least one of: video resolution, video frame rate, video clip rate, audio clip rate, second open rate, and packet loss rate.

3. The audio/video scheduling method according to claim 1, wherein the obtaining a first user experience index value corresponding to a target time includes:

acquiring a first link portrait corresponding to a target moment, and determining a link portrait characteristic value according to the first link portrait;

and determining the user experience index value according to the link portrait characteristic value.

4. The audio/video scheduling method according to claim 3, wherein the obtaining the corresponding first link representation at the target time includes:

receiving audio and video link data reported by a user at the target time;

and analyzing the audio and video link data to generate the first link portrait.

5. The audio-video scheduling method of claim 4, wherein the audio-video link data comprises at least one of: timestamp, edge server, terminal model, operating system, network type, packet loss rate, video resolution, frame rate, sending code rate, receiving code rate;

the first link representation includes at least one of: time attribute, terminal attribute, network attribute, region attribute, edge server attribute, experience attribute;

the link portrait characteristic value includes at least one of the following: time element, terminal element, network element, region element, edge server element, and experience element.

6. The audio-video scheduling method according to claim 1, wherein model training the first model to obtain the user experience model comprises:

acquiring a historical link portrait set; the historical link portrait set comprises link portraits before the current moment;

according to the historical link portrait set, performing model training on the first model through a logistic regression algorithm and a gradient descent method to obtain the weight of each user experience index;

and determining the experience model according to the first model and the weight of each user experience index.

7. The audio/video scheduling method according to claim 3, wherein the determining the user experience index value according to the link portrait characteristic value includes:

and calculating the user experience index value through the user experience model according to the link portrait characteristic value.

8. The audio/video scheduling method according to claim 1, wherein the adjusting the link and/or the code rate of the audio/video playing corresponding to the second time period according to the target user experience index value and the user experience model includes:

under the condition that the target user experience index value is smaller than a second threshold value, adjusting a link and/or a code rate corresponding to a user experience model until the target user experience index value corresponding to the user experience model is larger than or equal to the second threshold value;

and when the target user experience index value is greater than or equal to a second threshold value, the link and the code rate corresponding to the user experience model are used as the link and the code rate of the audio and video playing corresponding to the second time period after adjustment.

9. An audio/video scheduling apparatus, comprising:

the first acquisition module is used for acquiring a first user experience index value corresponding to the target moment;

the second acquisition module is used for acquiring a first link portrait corresponding to a first preset time period before the target moment under the condition that the first user experience index value is lower than a first threshold value; the link portrait is used for indicating related information of an audio/video playing link;

the processing module is used for inputting the first link portrait into the user experience model to obtain a target user experience index value corresponding to the second time period; the second time period comprises a second preset time period after the target time; the user experience model is used for representing the corresponding relation between the user video playing experience index value and the link portrait under the target audio and video playing code rate;

the adjusting module is used for adjusting the link and/or the code rate of the audio/video playing corresponding to the second time period according to the target user experience index value and the user experience model;

the user experience model is a model obtained by model training of the first model:

the first model is as follows:

wherein f (v|θ) represents a user experience index value; v is the link portrait characteristic value; mi is the ith link image experience index value corresponding to the link image characteristic value, δi is the weight of the ith user experience index, and θ is the audio/video playing code rate; n is an integer greater than or equal to 1;

the link image experience index value comprises at least one of the following: video resolution value, video frame value, video clip rate reciprocal value, audio clip rate reciprocal value, second opening value.

10. An electronic device, comprising: a transceiver, a processor, a memory, and a program or instructions stored on the memory and executable on the processor; an audio-video scheduling method according to any one of claims 1-8, wherein the program or instructions, when executed by the processor, implement the method.

11. A computer readable storage medium having stored thereon a program or instructions, which when executed by a processor, implements the steps of the audio video scheduling method of any one of claims 1-8.