WO2009039781A1 - Video data update method and system - Google Patents
Video data update method and system Download PDFInfo
- Publication number
- WO2009039781A1 WO2009039781A1 PCT/CN2008/072426 CN2008072426W WO2009039781A1 WO 2009039781 A1 WO2009039781 A1 WO 2009039781A1 CN 2008072426 W CN2008072426 W CN 2008072426W WO 2009039781 A1 WO2009039781 A1 WO 2009039781A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- video
- video data
- update request
- parameter
- update
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1083—In-session procedures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/752—Media network packet handling adapting media to network capabilities
Definitions
- the present invention relates to the field of communications, and in particular, to an update technique for video data. Background technique
- the media server device can be divided into two functional entities, MC (Media Controller) and MP (Media Processor), as shown in Figure 1.
- the MC is a media controller, and the corresponding physical device may be a Media Gateway Controller (MGC), which mainly performs conference control management, and implements resource management and access control functions for the MP device.
- the MP is a media processor, and the corresponding physical device may be a Media Gateway (MG), which mainly completes the collection and distribution of conference audio, video, and data media, and implements audio mixing, video picture switching, codec, and speed. Matching functions related to media processing.
- the interface protocol between MC and MP is the H.248 protocol.
- H.248 is a media gateway control protocol, a protocol used for physically separate media server control, which separates call control from media conversion.
- the media gateway controller controls and manages the media gateway and the H.248 endpoint through the protocol, and the media gateway also uses the protocol to transfer different types of access modes to RTP/RTCP suitable for transmission over the IP network (Real- Time Transport Protocol/Real-time Transport Control Protocol (Real-Time Transport Protocol), which implements service interworking between a packet network and a PSTN (Public Switched Telephony Network) network.
- IP network Real- Time Transport Protocol/Real-time Transport Control Protocol (Real-Time Transport Protocol)
- Real-Time Transport Protocol Real-Time Transport Protocol
- Termination There are two basic components in H.248: Termination and Context.
- the various resources on the MP are abstractly represented as Terminations, and the combination between the endpoints is abstracted as Context.
- a context can contain multiple endpoints, so Topology is used to describe the interrelationships between endpoints.
- connection of the call is actually the operation of the endpoint and context. Work. This operation is done by a command (Command) request and response between the MC and the MP.
- the parameters carried by the command also called descriptors (Descriptor) are classified into categories: Property, Signal, Event, and Statistic.
- the parameters with business relevance are logically aggregated into packages.
- the video coding compression standard mainly has two series of MPEG-x and H.26x, and the basic coding unit is Macro Block (MB).
- each macro block consists of four 8x8 luminance blocks. It consists of two 8x8 chroma blocks.
- One Group of Block (GOB) consists of 3x11 macroblocks.
- a QCIF (Quarter Common Intermediate Format) image consists of 3 GOBs.
- a CIF (Common Intermediate Format) image contains 12 GOBs.
- Motion estimation/motion compensation eliminates video time redundancy
- DCT Discrete Cosine Transform
- variable length code (VLC, Variable Length Code) eliminates statistical redundancy.
- VLC Variable Length Code
- the code stream is more susceptible to the error, and as a result, the synchronization with the encoding end will be lost at the decoding end, so that the VLC codeword cannot be correctly decoded before the next synchronization codeword is encountered;
- the predictive coding technique will spread the error into the entire video sequence, greatly reducing the quality of the reconstructed image. Therefore, in order to achieve good quality video transmission, it is necessary to take certain fault-tolerant measures in combination with the transmission characteristics of the actual application channel.
- the fault-tolerant algorithm can be mainly divided into encoder-based Wrong algorithm, decoder-based fault-tolerant algorithm and feedback channel-based fault-tolerant algorithm.
- the fault-tolerant algorithm based on the feedback channel refers to a method of obtaining error information by using a decoder and transmitting it to the encoder for error processing through a feedback channel, and the feedback information of the package is as follows: requesting video key frame refresh, requesting local Macro block data refresh, etc.
- the feedback channel of the video information generally has two modes: a channel attached to the bearer plane (intra-band channel) and a channel separated from the bearer plane (out-of-band channel). Both methods can pass the feedback information from the video decoder to the video encoder, thus implementing a feedback channel based fault tolerance algorithm.
- the existing transmission protocols can be divided into two types: out-of-band transmission protocol (separation from bearer plane) and in-band transport protocol (bound to bearer plane), mainly in the following types:
- the extended SIP-based method is an extended XML (extensible markup language) entity of the application/media_control+xml type in the SIP (Session Initiated Protocol) message body.
- the external data signaling method delivers the video data update request.
- This method only supports update requests for video key frames (full intra-coded frames), and requires the terminal to be a SIP terminal supporting the extension. Due to the SIP extension method, the out-of-band transmission method is generally adopted.
- the method of extending the RTCP packet is to use the RTCP extended packet method to provide video feedback information, and the provided video feedback information includes:
- SLI Slice Loss Indication
- the RTCP extended packet carries different parameters. Because of the RTCP extension mode, the in-band transmission mode is generally adopted.
- ITU-T H.245 provides a video data update request command for the signaling plane, the basic function of which is when When the decoder detects the error, it can request the encoder to send the key frame or the updated GOB and MB.
- the terminal is required to support H.245 signaling.
- the video update commands provided by the agreement include:
- videoFastUpdateMB used to request MB updates
- This protocol is generally adopted by H.323 terminals and uses out-of-band transmission.
- the above protocols can transmit video quality feedback information.
- different terminals may use different transmission channels and protocols to implement a fault-tolerant algorithm based on feedback channels.
- the H.324 terminal located in the softswitch network and the SIP terminal located in the IMS network possibly There are multiple terminals, only two parties are shown in the figure) for multimedia sessions.
- the H.324 terminal and the IMS terminal perform multimedia communication, and the IM-MGW/MGCF undertakes the tasks of interworking bearer control and call control.
- the MGCF also undertakes the termination task of the H.245 protocol.
- the H.324 terminal can usually support the processing of the video data fast update request in the H.245 protocol, but does not support the RTCP in-band request processing; the SIP terminal in the IMS network can support the in-band update request, and can also support the out-of-band SIP message. Order update request.
- the process of implementing the video data update request in the prior art may be as shown in FIG. 3.
- the two black circles in FIG. 3 identify the video quality detection points A and B, and initiate a quality report or a video I frame (ie, the entire frame).
- Update request When the IMS terminal supports the outband video data update request of the SIP + XML extension, as shown in FIG. 3, the MGW can receive the inband request initiated by the H.324 terminal of the softswitch network, and send the request to the MGCF through the H.245 mode. The MGCF can send a video data update request to the SIP terminal of the IMS network through the SIP+XML method, thereby implementing update of the video data.
- the MGCF can also receive the out-of-band request initiated by the SIP terminal through the SIP+XML method, and send the in-band video data update request to the H.324 terminal through the H.245 method, thereby realizing the update of the video data.
- the inventor found that the prior art has at least the following drawbacks: When the IM-MGW itself detects the video quality problem of the H.324 terminal or the SIP terminal, the IM-MGW cannot request the MGCF to update. Video data; When the SIP terminal feeds back the video data update request through the in-band RTP/RTCP, and the IM-MGW cannot provide the requested update data, the IM-MGW cannot report the update request to the MGCF through the H.248 protocol, so that the MGCF passes the H.245 message. Requesting video data updates to the H.324 terminal;
- the MGCF When the MGCF receives the H.245 video data update request from the H.324 terminal, and the SIP terminal does not support the SIP+XML video data update request, the MGCF cannot instruct the IM-MGW to send the in-band video data update request.
- the prior art does not provide an indication of how to update the video data that needs to be updated by the MGCF instructing the MGW in the above case.
- a media controller instructs a media processor to make an update request for video data that needs to be updated.
- the media processor detects a video quality of the video source, and requests the media controller to update the video data according to the detection result.
- the device provided by the embodiment of the present invention is a media controller, and the media controller includes:
- An update request receiving module configured to receive a request sent by the media processor, where the request is used to request video data that needs to be updated; and the request is that the media processor detects the video according to the indication sent by the indication sending module. The quality of the source video, and the request sent based on the detection result.
- the device provided by the embodiment of the present invention is a media processor, and the media processor includes:
- An indication receiving module configured to receive an indication sent by the media controller
- a detecting module configured to detect a video quality of the video source according to the indication
- An update request module configured to send to the media according to a video quality detection result of the detection module
- the controller requests video data that needs to be updated.
- the video data update system includes: a media controller and a media processor; the media processor is configured to detect a video quality of the video source according to the indication of the media controller, and according to the video quality detection result Sending a request to the media controller to request video data that needs to be updated;
- the media controller is configured to instruct the media processor to detect a video quality of a video source.
- the media controller sends an indication to the media processor to indicate the video quality of the video source, so that the media processor performs video quality detection, and can report to the media controller when the video quality problem is detected.
- the video processor performs the video data quality detection and the video data update request according to the indication of the media controller.
- the media processor Receiving, by the media processor, an indication sent by the media controller, and detecting, when detecting an in-band video data update request initiated by the video source, requesting, by the media controller, video data that needs to be updated; the indication is the media
- the processor receives an indication sent by the media controller, to instruct the media processor to detect an in-band video data update request initiated by the video source.
- Another device provided by the embodiment of the present invention is a media controller, where the media controller includes: an indication sending module, configured to send an indication to the media processor;
- An update request receiving module configured to receive a request sent by the media processor, requesting video data that needs to be updated; and the request is used by the media processor to detect a band initiated by the specified video source according to the indication sent by the indication sending module.
- Another device provided by the embodiment of the present invention is a media processor, where the media processor includes: an indication receiving module, configured to receive an indication sent by the media controller;
- a detecting module configured to detect, according to the indication, an in-band video data update request initiated by the specified video source;
- an update requesting module configured to request the media controller to update the video data after the detecting module detects the in-band video data update request.
- the video data update system includes: a media controller and a media processor; the media processor is configured to detect an in-band video update request of the video source according to the indication of the media controller, and After detecting the in-band video update request, sending a request to the media controller to request video data that needs to be updated;
- the media controller is configured to instruct the media processor to detect an in-band video update request initiated by a video source.
- the media controller sends an indication to the media processor to instruct the detection of the in-band video data update request of the video source, so that the media processor can detect the in-band video data update request of the video source.
- the media controller initiates a video data update request, which solves the deficiencies in the prior art.
- the media processor detects the in-band video data update request according to the indication of the media controller and reports the request to the media controller to request the video. Data Update.
- the media processor initiates an in-band video data update request to the video source specified by the indication according to the indication sent by the media controller, requesting to update the video data specified by the indication; the indication is sent by the media controller to the An indication of the media processor instructing the media processor to initiate an in-band video data update request to the video source to request to update the specified video data.
- Another device provided by the embodiment of the present invention is a media controller, where the media controller includes: an update request receiving module, configured to receive an out-of-band video data update request initiated by a video source; and an indication sending module, configured to send to the media processor Sending an indication, instructing the media processor to initiate an in-band video data update request, requesting video data that needs to be updated; and the video data requested to be updated by the in-band video data update request is an update indicated by the out-of-band video data update request Video data.
- an update request receiving module configured to receive an out-of-band video data update request initiated by a video source
- an indication sending module configured to send to the media processor Sending an indication, instructing the media processor to initiate an in-band video data update request, requesting video data that needs to be updated
- the video data requested to be updated by the in-band video data update request is an update indicated by the out-of-band video data update request Video data.
- Another device provided by the embodiment of the present invention is a media processor, where the media processor includes: an indication receiving module, configured to receive an indication sent by the media controller;
- An update request module configured to initiate an in-band video data update to the video source according to the indication Requesting, requesting to update the video data indicated by the indication.
- the video data update system includes: a media controller and a media processor; the media processor is configured to initiate an in-band video to the requested video source according to the indication of the media controller Update request, request to update video data;
- the media controller configured to send the indication to the media processor after receiving an out-of-band video update request of a video source, instructing the media processor to request the out-of-band video update request
- the video source initiates an in-band video data update request.
- the media controller sends an indication to the media processor to instruct the device to initiate an in-band video data update request, so that the media processor can initiate an in-band video data update request according to the indication after receiving the indication.
- FIG. 1 is a schematic diagram of a media processor in the prior art
- FIG. 2 is a schematic diagram of an architecture of a softswitch terminal and a SIP terminal in the prior art
- FIG. 3 is a schematic diagram of a scenario in which a softswitch terminal and a SIP terminal communicate with each other in the prior art
- FIG. 4A and FIG. 4B are schematic diagrams showing a scenario in which a softswitch terminal and a SIP terminal communicate with each other according to Embodiment 1 of the present invention
- FIG. 5A is a schematic flowchart of detecting video quality by an MGW according to Embodiment 1 of the present invention.
- FIG. 5B is a schematic diagram of a signaling flow of an MGW detecting a video quality according to Embodiment 1 of the present invention
- FIG. 6 is a schematic diagram of a scenario in which a softswitch terminal communicates with a SIP terminal according to Embodiment 2 of the present invention
- FIG. 7A is a schematic diagram of MGW detection according to Embodiment 2 of the present invention
- FIG. 7B is a schematic diagram of a signaling flow of an MGW detecting an in-band request packet according to Embodiment 2 of the present invention
- FIG. 8 is a schematic diagram of a scenario in which a softswitch terminal and a SIP terminal communicate with each other according to Embodiment 3 of the present invention
- FIG. 9B is a schematic flowchart of signaling flow of an MGCF instructing an MGW to send a video update request according to Embodiment 3 of the present invention
- FIG. 10 is a schematic structural diagram of a media controller according to an embodiment of the present invention.
- FIG. 11 is a schematic structural diagram of a media processor according to an embodiment of the present invention. detailed description
- the embodiment of the invention provides a method for detecting and reporting a video quality between a media controller and a media processor, or an in-band video data update request detection and reporting mode, or by a media controller, in a bearer control separation architecture.
- This embodiment describes a process in which the MGCF instructs the MGW to perform video quality detection and reporting.
- the MGW participates in the video codec, and can dynamically detect the video stream quality sent by the video encoding source, and report the quality information to the MGCF, so that the MGW is responsible for the quality tracking of the video encoding source during the session. Update request.
- the specific application scenario of this embodiment can be as shown in FIG. 4A and FIG. 4B.
- the video quality detection point can be set to the MGW.
- the MGW detects that there is a problem with the video quality of the H.324 terminal from the video encoding source, the video quality is fed back to the MGCF, and the MGCF requests the H.324 terminal to send the video update data through the H.245 mode.
- the video quality detection point can be set.
- MGW the MGW detects that there is a problem with the video quality of the SIP terminal from the video encoding source
- the video quality is fed back to the MGCF, and the MGCF requests the SIP terminal to send the video update data through the SIP+XML method.
- FIG. 5A is a schematic flowchart of the MGW detecting the video quality according to the first embodiment of the present invention.
- the flowchart describes the process of detecting the video quality by the MGW and reporting it to the MGCF according to the scenario shown in FIG. 4A and FIG. 4B, which specifically includes:
- Step 501 A multimedia session establishment process, in which the MGCF determines whether the MGW participates in video codec between different video sources, such as performing conference mixing or transcoding.
- Step 502 After confirming that the MGW participates in the video codec, the MGCF instructs the MGW to start detecting the video quality, and determines the detection standard, such as the video quality level requirement to be detected, and sends the determined detection standard to the MGW.
- the detection standard such as the video quality level requirement to be detected
- Step 503 During the decoding process, the MGW calculates or counts the received video data according to requirements according to the quality level specified by the MGCF.
- Steps 504 to 505 When the MGW calculates that the video quality falls below the quality level specified by the MGCF, and fails to restore the video quality level through the decoding process, the video data information that needs to be updated is reported to the MGCF.
- Step 506 The MGCF sends an out-of-band video data update request according to the video data information that needs to be updated by the MGW. If the H.324 video quality problem is detected, the MGCF sends an out-of-band video data update request to the H.324 terminal through the H.245 mode; if the SIP terminal video quality problem is detected, the MGCF sends the SIP terminal to the SIP terminal through the SIP+XML method. Send out-of-band video data update request.
- the setting and detection criteria of the video quality level are determined by the MGCF, and may be, but not limited to, a macroblock error rate, other detection standards of video quality, and the like.
- the MGW and the MGCF determine whether to request new video data according to the degree of quality degradation, thereby ensuring video data refresh between the video source and the MGW.
- This embodiment implements the above process by extending the H.248 protocol.
- this embodiment is in the H.248 Association.
- the video quality detection event is added to the conference.
- the MGCF may send the event to the MGW to instruct the MGW to detect and report the video quality event according to the specified standard.
- the H.248 protocol message reported by the MGW to the MGCF carries the video data update parameter. Used to indicate video data information that needs to be updated.
- the video quality detection event sent by the MGCF to the MGW carries video quality detection parameters, and the video quality detection parameters are used to indicate the standards followed by the MGW for video quality detection and reporting.
- the video quality detection parameters may include one or more of the following parameters:
- the minimum video macroblock error rate (Min Percent of Error, minpoe), the minimum video macroblock error rate can be used as a judgment data for whether video update is required, and the video macroblock error probability detected by the MGW. If the value of the minimum video macroblock error rate is less than the value of the minimum video macroblock error rate, the current video quality is considered acceptable, otherwise the current video quality is considered to be locally degraded, and a partial update request needs to be sent.
- the allowable range of the minimum video macroblock error rate can be set to 20-50.
- the maximum video macroblock error rate (Max Percent of Error, maxpoe), the maximum video macroblock error rate can be used as a judgment data for whether or not to perform video update, and the video macroblock error probability detected by the MGW If the value of the maximum video macroblock error rate is greater than the value of the maximum video macroblock error rate, the current video quality is considered to be seriously degraded, and the entire video frame update request, that is, the I frame request, needs to be sent.
- the allowable range of the maximum video macroblock error rate can be set to minpoe ⁇ maxpoe ⁇ 100.
- the minimum reporting interval which specifies the minimum interval for the MGW to report each time.
- the minimum reporting interval is set to prevent the MGW from reporting the event of detecting video quality too frequently when the network quality is poor and the video quality is always poor.
- the above parameters determine the standard and upper frequency of MGW for video quality detection.
- the MGW counts the actual number of received macroblocks and the number of macroblocks with errors.
- the MGW When the macroblock error rate reaches 30%, the MGW reports Video quality detection event, request to update local data; When the macroblock error rate reaches 60%, the MGW reports a video quality detection event, requesting to update the entire frame data; if for some reason, the macroblock error of the video data is for some reason
- the rate is always higher than the set threshold (minpoe or maxpoe), then when the MGW completes a thing After the report is submitted, it needs to wait for 5s to report the new video quality detection event.
- the MGW carries the video data update parameters based on the video quality detection event reported to the MGCF.
- the video data update parameters indicate the video data that needs to be updated.
- the video data update parameters may include one or more of the following parameters:
- this parameter indicates the first GOB identifier to be updated in the video image, and the GOB identifier is determined by a specific encoding algorithm
- this parameter indicates the number of GOBs that need to be updated
- this parameter indicates the first macroblock identifier to be updated, and the macroblock identifier is determined by a specific encoding algorithm
- this parameter indicates the number of macroblocks that need to be updated
- fastUpdatePicture this parameter indicates that the entire video image needs to be updated, that is, the video source is required to send I frame data.
- the above video data update parameter reported by the MGW may implement an update request for the local GOB, the local MB, and/or the entire I frame data.
- the video source is required to send new I frame data.
- the MGW detects the video quality of both parties in the session and reports the same process.
- the signaling process of the MGW detecting the video quality of the H.324 terminal and reporting the video data to be updated may be as shown in FIG. 5B.
- FIG. 5B is a schematic diagram of a signaling process for detecting video quality by an MGW according to Embodiment 1 of the present invention, which specifically includes:
- Step 511 Establish a multimedia session.
- Step 512 The MGCF sends a request message to the MGW through the H.248 protocol to the specified endpoint in the session, where the message indicates the video quality detection event and the corresponding event parameter.
- the MGCF sends the endpoint X corresponding to the H.324 terminal ( Termination X), the video quality detection event parameters include a minimum video macroblock error rate (minpoe) of 30, a maximum video macroblock error rate (maxpoe) of 60, and a reporting interval of 5 s.
- minpoe minimum video macroblock error rate
- maxpoe maximum video macroblock error rate
- Step 513 The MGW responds after receiving the request message.
- the MGCF is notified to perform local data update, which may include the first lost macroblock address, the number of lost macroblocks, and the time position (refer to AVPF (Audio-Visual Profile with Feedback)). SLI command).
- Step 516 The MGCF responds after receiving the notification message.
- an H.245 signal such as videoFastUpdatePicture
- the MGCF sends the endpoint X corresponding to the H.324 terminal, and the MGCF can also detect the endpoint corresponding to the SIP terminal. In this case, when the SIP endpoint detects the video quality problem. At the same time, the MGCF is also reported; and the MGCF will send a SIP + XML video data update request to the SIP terminal according to the reported parameters, requesting the SIP terminal to perform video data update.
- This embodiment describes that the MGCF instructs the MGW to perform in-band video data update signaling detection and The video data update request is reported, so that the MGW can detect the video data update request and report it to the MGCF, so that the MGCF delivers the video data update request to the requested video source, and implements the update of the video data during the session.
- a specific application scenario of this embodiment may be as shown in FIG. 6.
- the MGW detects the in-band video data update request initiated by the SIP terminal, and notifies the MGCF when the MGW detects the in-band video data update request from the SIP terminal, and the MGCF requests the H.324 terminal to send the video through the H.245 mode. update data.
- FIG. 7A is a schematic flowchart of detecting an in-band request message by an MGW according to Embodiment 2 of the present invention.
- the flowchart describes, for the scenario given in FIG. 6, the MGW detects a video data update request initiated by a SIP terminal, and reports it to the MGCF.
- the process specifically includes:
- Step 701 A multimedia session establishment process.
- Step 702 The MGCF determines an RTCP packet type of the in-band video data update request of the SIP terminal to be detected, and instructs the MGW to start detecting the RTCP packet.
- Steps 703 to 704 The MGW receives the RTCP packet sent by the terminal and analyzes the packet type to determine whether it is the packet type indicated by the MGCF.
- Step 705 When the MGW determines that the received RTCP packet is the type of the packet that is detected by the MGCF, the MGW reports the update parameter of the video data carried by the RTCP packet to the MGCF.
- Step 706 The MGCF sends an out-of-band video data update request to the H.324 terminal according to the video data reported by the MGW, and requests to update the video data indicated by the video data update parameter.
- the video data update parameter reported by the MGW is consistent with the video data update parameter carried by the inband RTCP packet sent by the SIP terminal.
- This embodiment implements the above process by extending the H.248 protocol.
- an event (a video update request detection event) for detecting an RTCP video quality feedback message is added to the H.248 protocol, and the MGCF may send the event to the MGW to indicate that the MGW has a payload type (payload type) of PSFB (payload).
- -specific feedback messages RTCP messages for load feedback messages, such as RTCP/AVPF request, detection and reporting.
- the event of the RTCP video quality feedback packet sent by the MGCF to the MGW carries an event parameter, where the event parameter is used to instruct the MGW to detect and report the RTCP video quality feedback message.
- the event parameters may include one or more of the following parameters. :
- the message type of the RTCP (Feedback Message Type, fmt) is used to determine the type of the packet to be detected.
- the type value is determined by the relevant standard of the RTCP video quality feedback extended message, wherein the message related to the video data update request includes Picture Loss Indication (PLI, Keyframe Update Request), Slice Loss Indication (SLI, Image Slice Update Request), Full Intra Request ( FIR, I frame update request), etc.
- PKI Picture Loss Indication
- SLI Slice Loss Indication
- FIR I frame update request
- the minimum reporting interval (nint) indicates the minimum interval for reporting twice.
- the minimum reporting interval is to prevent the MGW from being too frequent if the network quality is poor and the video quality is always poor. Report the incident.
- the above event parameters determine the standard and reporting frequency of the MGW for detecting the video data update request.
- the MGW reports the video update request to the MGCF based on the detection of the RTCP video quality feedback message.
- the detection event carries the following video data update parameters to indicate the video data to be updated.
- the video data update parameters may include one or more of the following parameters:
- this parameter indicates the first macroblock identifier to be updated, and the macroblock identifier is determined by a specific encoding algorithm
- this parameter indicates the number of macroblocks that need to be updated
- fastUpdatePicture this parameter indicates that the entire video image needs to be updated, that is, the video source is required to send I frame data.
- the video data update parameter reported by the MGW is related to the detected packet type, and different RTCP
- the packet carries different video data update parameters, and the video data update parameter reported by the MGW is determined by parameters carried in the RTCP packet.
- the correspondence between the MGW detected packet type, the reported video data update parameter, and the H.245 signaling sent by the MGCF is as follows:
- the H.245 signaling sent by the MGCF is videoFastUpdatePicture
- the reported video data update parameters are firstMB and numberOfMBs
- the firstMB and numberOfMBs parameter values are equal to the first and number parameter values in the SLI packet
- the H.245 signaling sent by the MGCF is videoFastupdateMB. .
- the above video data update parameter reported by the MGW can implement an update request for the local MB and the entire I frame data.
- the MGW detects the in-band video data update request sent by the SIP terminal, and reports the detected in-band update request to the MGCF through the event; the MGCF sends the H.245 out-of-band video data according to the received event.
- the signaling flow for updating the request to the H.324 terminal is as shown in FIG. 7B.
- FIG. 7B is a schematic diagram of a signaling flow of an MGW detecting an in-band request packet according to Embodiment 2 of the present invention, which specifically includes:
- Step 711 Establish a multimedia session.
- Step 712 The MGCF sends a request message to the MGW to the specified endpoint in the session, where the message indicates a video update request detection event, and a corresponding event parameter.
- the MGCF sends the endpoint X ( Termination X ) corresponding to the SIP terminal, and the video update request detection event sent by the MGCF to the MGW instructs the MGW to detect the RTCP/AVPF request, and only detects the payload type (Payload Type). ) is the RTCP message of PSFB.
- RFC 4585 and raft-ietf-avt-avpf-ccm-05 refer to RFC 4585 and raft-ietf-avt-avpf-ccm-05.
- Step 713 The MGW responds after receiving the request message.
- the video data update parameter parameters carried in the notification message are also different according to the type of the detected RTCP packet.
- the FIR message of the RTCP needs to be detected, and correspondingly, the updated video data update parameter parameter is a fastUpdatePicture parameter.
- Step 716 The MGCF responds after receiving the notification message.
- H.245 signaling such as videoFastUpdatePicture signaling
- This embodiment describes a process in which the MGCF instructs the MGW to transmit an in-band video data update request, thereby implementing video data update.
- the MGCF receives the relevant out-of-band request message, if the requested video source supports the in-band request mode, the MGCF sends an in-band request to the video source through the MGW.
- a specific application scenario of this embodiment may be as shown in FIG. 8.
- the MGW detects the out-of-band video data update request initiated by the H.324 terminal, and notifies the MGCF when the MGW detects the out-of-band video data update request from the H.324 terminal, and the MGCF requests through the in-band RTP/RTCP method.
- the SIP terminal sends video update data.
- a schematic flowchart of an MGCF instructing an MGW to send a video data update request according to Embodiment 3 of the present invention includes:
- Step 901 Establish a multimedia session.
- Step 902 The MGCF receives an out-of-band video data update request message, where the video data update parameter is carried.
- Step 903 If the video source supports the in-band request mode, the MGCF instructs the MGW to send the inband. The signaling is sent to the video source end, and the carried video data update parameter is consistent with the video data update parameter in the video data update request message received by the MGCF.
- Step 904 The MGW sends an in-band update request message to the video source end according to the indication of the MGCF, and the video data that needs to be updated indicated by the video data update parameter carried in the message is updated with the MGCF indicated by the video data update parameter.
- the video data is consistent.
- Step 905 After the MGW completes the request message transmission, it responds to the MGCF.
- the MGCF directly passes the H.245 message parameter to the MGW, and the MGW is responsible for converting between the H.245 message parameter and the RTCP extended message parameter, and sends the matched RTCP according to the H.245 message. Text.
- This embodiment implements the above process by extending the H.248 protocol.
- a signal indicating that the MGW sends an RTCP video quality extension message is added in the H.248 protocol, and the MGCF can indicate that the MGW sends an RTCP extension packet whose payload type (payload Type) is PSFB.
- the signal parameter is consistent with the H.245 signaling parameter and includes one or more of the following parameters:
- firstGOB indicating the first GOB identifier to be updated in the video image, and the GOB identification is determined by a specific encoding algorithm
- firstMB indicating the first macroblock identifier to be updated, and the macroblock identifier is determined by a specific encoding algorithm
- fastUpdatePicture indicating that the entire video image needs to be updated, that is, the video source is required to send I frame data.
- the MGW sends a corresponding RTCP packet according to the received signal parameter, and the corresponding relationship is: if the signal parameter carries the fastUpdatePicture parameter, the B' J MGW sends the FIR message, otherwise, the SLI message is sent;
- the conversion of out-of-band H.245 video update signaling to in-band RTCP extension can be realized.
- the above conversion can also be performed on the MGCF.
- the MGCF receives the H.245 video data update request of the H.324 terminal, and instructs the MGW to send the corresponding inband RTCP extension signaling to the SIP terminal.
- the signaling flow may be as shown in FIG. 9B. .
- FIG. 9B is a schematic diagram of a signaling flow of an MGCF instructing an MGW to send a video data update request according to Embodiment 3 of the present invention, which specifically includes:
- Step 911 first establish a multimedia session.
- Steps 912-913 After receiving the out-of-band H.245 video data update request, the MGCF sends a request message to the MGW, where the MGCF sends a signal indicating that the MGW sends the RTCP video quality extension message.
- Step 914 After receiving the MGCF video data update request indication, the MGW converts the signal parameter into an RTCP packet parameter, and sends a corresponding RTCP extended packet.
- Step 915 The MGW sends a response message to the MGCF.
- the embodiments of the present invention further provide the structures of the media controller, the media processor, and the video data update system for the processes of the foregoing first to third embodiments.
- the embodiment of the present invention provides a structure of a media controller.
- the media controller includes: an indication sending module and an update request receiving module, where: an indication sending module is used for Send an indication to the media processor;
- An update request receiving module configured to receive a request sent by a media processor, where the request is used for a request The video data to be updated; the request is a request that the media processor detects the video quality of the video source according to the indication sent by the sending module, and sends the request according to the detection result.
- the media controller may further include an update request sending module, configured to initiate an out-of-band video data update request according to the request received by the update request receiving module, and request the video source to update the video data that needs to be updated.
- an update request sending module configured to initiate an out-of-band video data update request according to the request received by the update request receiving module, and request the video source to update the video data that needs to be updated.
- the indication sent by the indication sending module of the media controller may specifically send a video quality detection event, where the video quality detection event carries a video quality detection parameter.
- the update request receiving module receives the video quality detection event reported by the media processor, and requests the media controller to update the video data; the reported video quality detection event is a video in the video quality detection event sent by the media processor according to the media controller.
- the quality detection parameter detects the video quality of the video source, and detects an event according to the video quality reported by the detection result, where the video data update parameter is carried, and the video data update parameter indicates the video data that needs to be updated.
- the embodiment of the present invention provides a structure of a media processor, as shown in the figure.
- the media processor includes: an indication receiving module, a detecting module, and an update requesting module, wherein: the receiving module is configured to receive an indication sent by the media controller;
- a detecting module configured to detect a video quality of the video source according to the received indication
- an update requesting module configured to send, to the media controller, a request for updating the video data according to the video quality detection result of the detecting module.
- the video quality detection event is sent by the media module, and the video quality detection event carries the video quality detection parameter.
- the detection module detects the video quality of the video source according to the video quality detection parameter.
- the update request module reports the video quality detection event to the media controller according to the video quality detection result of the detection module, and carries the video data update parameter, where the video data update parameter indicates the video data that needs to be updated.
- the video quality detection parameter includes a video macroblock error rate; the detection module detects a macroblock error rate of the video source according to the video quality detection parameter; and the update request module detects that the macroblock error rate of the video source is higher than the video. Reporting to the media controller when the video macroblock error rate in the quality detection parameter Frequency quality detection event.
- the video macroblock error rate may include a maximum macroblock error rate and a minimum macroblock error rate, if the detection module detects a video macroblock error rate of the video source at the maximum macroblock error rate and a minimum macroblock error rate.
- the update requesting module sends a request for updating the local video data; if the video macroblock error rate of the video source detected by the detecting module is greater than the maximum macroblock error rate, the update requesting module sends the updated full frame video data. request.
- the video quality detection parameter further includes a time interval parameter; the update request module reports the video quality detection event according to the time interval indicated by the time interval parameter.
- the embodiment of the present invention provides a video data update system, where the system includes the media controller and the media processor, which are applicable to the process described in the first embodiment, where: the media processor is used for Detecting a video quality of the video source according to the indication sent by the media controller, and requesting, from the media controller, the video data that needs to be updated according to the video quality detection result;
- a media controller configured to instruct the media processor to detect a video quality of the video source.
- the media controller is further configured to: after receiving the request sent by the media processor to request the updated video data, initiate a video data update request to the video source, requesting to update the video data that needs to be updated.
- the embodiment of the present invention provides a structure of a media controller, as shown in the figure.
- the media controller includes: an indication sending module and an update request receiving module, where: an indication sending module, configured to send an indication to the media processor;
- An update request receiving module configured to receive a request sent by the media processor, requesting video data that needs to be updated; the request is that the media processor detects an in-band video update request initiated by the specified video source according to the received indication, and detects the band A request sent when a video update request is made.
- the media controller may further include: an update request sending module, configured to send an out-of-band video data update request to the video source according to the update request to receive the request received by the module, to request the video source to update the video data that needs to be updated.
- an update request sending module configured to send an out-of-band video data update request to the video source according to the update request to receive the request received by the module, to request the video source to update the video data that needs to be updated.
- the indication sent by the indication sending module in the media controller is a video update request detection event, where the video update request detection event is used to instruct the media processor to detect the number of in-band videos initiated by the video source.
- the video update request detection event carries an event parameter, and the event parameter includes a real-time transmission control protocol message type parameter.
- the update request receiving module receives the video update request detection event reported by the media processor, and requests the media controller to update the video data; the reported video update request detection event is that the media processor detects the event according to the video update request sent by the media controller.
- the video data update parameter is a parameter for updating the local video data or a parameter for updating the entire frame video data, and what video data update parameter is carried in the video update request detection event is determined by the media processor according to the message type parameter.
- the embodiment of the present invention provides a structure of a media processor, as shown in the figure.
- the media processor includes: an indication receiving module, a detecting module, and an update requesting module, wherein: the receiving module is configured to receive an indication sent by the media controller;
- the detecting module is configured to detect, according to the indication received by the receiving module, the in-band video data update request initiated by the specified video source;
- the update request module is configured to request the media controller to update the video data after the detecting module detects the in-band video data update request.
- the instruction received by the indication receiving module in the media processor is a video update request detection event sent by the media controller, where the video update request detection event carries an event parameter, where the event parameter includes a real-time transmission control protocol message type parameter,
- the message type indicated by the real-time transmission control protocol packet type parameter is an image slice update request message, or a key frame update request message or an entire frame update request message, and the event parameter further includes an interval parameter.
- the detecting module detects the in-band video data update request according to the real-time transmission control protocol message type parameter.
- the update request module reports a video update request detection event to the media controller, and carries a video data update parameter, where the video data update parameter indicates the video data that needs to be updated, if the event parameter is further Including the interval parameter, the update request module reports on the time interval indicated by the time interval parameter.
- the frequency update requests a detection event.
- the update request module in the media processor includes a first update request submodule and a second update request submodule, where:
- a first update request sub-module configured to update the local video data according to the image slice update request message
- a second update request sub-module configured to update the request message according to the key frame update request message or the entire frame update request message, requesting to update the whole Frame video data.
- the embodiment of the present invention provides a video data update system, where the system includes the media controller and the media processor applicable to the process described in the second embodiment, where: the media processor is used for Detecting an in-band video update request of the video source according to the indication sent by the media controller, and requesting the media controller to update the video data after detecting the in-band video update request;
- a media controller configured to instruct the media processor to detect an in-band video update request initiated by the video source, and receive a request by the media processor to request the updated video data.
- the media controller is further configured to: after receiving the request for the video data that needs to be updated by the media processor, initiate a video data update request to the requested video source in the in-band video update request, requesting Update the video data that needs to be updated.
- the embodiment of the present invention provides a media controller, where the media controller includes an update request receiving module and an indication sending module, where:
- An update request receiving module configured to receive an out-of-band video data update request initiated by a video source
- an indication sending module configured to send an indication to the media processor, to instruct the media processor to initiate an in-band video data update request, and the in-band video data update A request to update the video data indicated by the received out-of-band video data update request.
- the indication sending module in the media controller sends the indication to the media processor after the update request receiving module receives the out-of-band video data update request.
- the embodiment of the present invention provides a media processor, where the media processor includes an indication receiving module and an update request module, where: An indication receiving module, configured to receive an indication sent by the media controller;
- an update requesting module configured to initiate an in-band video data update request to the video source according to the indication, requesting to update the video data indicated by the indication.
- the instruction received by the indication receiving module in the media processor carries a video data update parameter, where the video data update parameter is a parameter for updating the entire frame of video data or a parameter for updating the local video data; and the update request module in the media processor.
- the first update request submodule and the second update request submodule are included, where:
- the first update request sub-module is configured to send an in-band video data update request for updating the entire frame of video data according to the parameter for updating the entire frame of video data; and the second update request sub-module is configured to: convert the video data according to the parameter of the updated partial video data.
- the update parameters are converted to parameters of the corresponding out-of-band video data update request message, and an in-band video data update request for updating the partial video data is sent.
- the embodiment of the present invention provides a video data update system, where the system includes the media controller and the media processor that can be applied to the process described in the third embodiment, where: the media processor is used for And initiating an in-band video update request to the requested video source, requesting to update the video data according to the indication of the media controller; and the media controller is configured to: after receiving the out-of-band video update request of the video source, to the media processor The indication is sent to instruct the media processor to initiate an in-band video data update request to the requested video source in the out-of-band video update request.
- the present invention can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is a better implementation. the way.
- the essential software product of the present invention is stored in a readable storage medium, such as a floppy disk, a hard disk or an optical disk of a computer, and includes a plurality of instructions for causing a computer device (which may be a personal computer, The server, or network device, etc.) performs the methods described in various embodiments of the present invention.
- the IM-MGW when the IM-MGW detects the video quality problem of the H.324 terminal or the SIP terminal, the IM-MGW may request the MGCF according to the indication of the MGCF. Video data to be updated; When the SIP terminal feeds back the video data update request through the in-band RTP/RTCP, and the IM-MGW cannot provide the requested update data, the IM-MGW may update the request through the H.248 protocol according to the indication of the MGCF.
- the MGCF may instruct the IM-MGW to send a video data update request in the band.
- embodiments of the present invention may be in the above, -
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
视频数据的更新方法和系统 技术领域 Video data updating method and system
本发明涉及通信领域, 尤其涉及视频数据的更新技术。 背景技术 The present invention relates to the field of communications, and in particular, to an update technique for video data. Background technique
媒体服务器设备从功能上可以分为 MC ( Media Controller, 媒体控制器) 和 MP ( Media Processor, 媒体处理器)两个功能实体, 如图 1所示。 MC为媒 体控制器,对应的物理设备可以是媒体网关控制器(Media Gateway Controller, MGC ), 主要完成会议控制的管理、 以及实现对 MP设备的资源管理和接入控 制等功能。 MP 为媒体处理器, 对应的物理设备可以是媒体网关 (Media Gateway, MG ), 主要完成会议音频、 视频和数据媒体的汇集和分发, 以及实 现音频混音、 视频画面切换、 编解码和速率适配等与媒体处理相关的功能。 MC和 MP之间的接口协议是 H.248协议。 The media server device can be divided into two functional entities, MC (Media Controller) and MP (Media Processor), as shown in Figure 1. The MC is a media controller, and the corresponding physical device may be a Media Gateway Controller (MGC), which mainly performs conference control management, and implements resource management and access control functions for the MP device. The MP is a media processor, and the corresponding physical device may be a Media Gateway (MG), which mainly completes the collection and distribution of conference audio, video, and data media, and implements audio mixing, video picture switching, codec, and speed. Matching functions related to media processing. The interface protocol between MC and MP is the H.248 protocol.
H.248是媒体网关控制协议, 是用于物理分开的媒体服务器控制的协议, 能把呼叫控制从媒体转换中分离出来。媒体网关控制器通过该协议对媒体网关 及 H.248端点进行控制和管理,媒体网关也通过该协议将不同类型的接入方式 转 4匕为适合在 IP 网络上传送的 RTP/RTCP ( Real-time Transport Protocol/ Real-time Transport Control Protocol , 实时传输协议 /实时传输控制协议 )流, 实 现分组网络和 PSTN ( Public Switched Telephony Network, 公共交换电话网) 网络的业务互通。 H.248 is a media gateway control protocol, a protocol used for physically separate media server control, which separates call control from media conversion. The media gateway controller controls and manages the media gateway and the H.248 endpoint through the protocol, and the media gateway also uses the protocol to transfer different types of access modes to RTP/RTCP suitable for transmission over the IP network (Real- Time Transport Protocol/Real-time Transport Control Protocol (Real-Time Transport Protocol), which implements service interworking between a packet network and a PSTN (Public Switched Telephony Network) network.
H.248中有两个基本组成部分: 端点 (Termination )和上下文(Context ), MP上的各种资源被抽象表示为端点( Termination ), 端点之间的组合被抽象表 示为上下文(Context )。 上下文可以包含多个端点, 因而以拓朴 (Topology ) 来描述端点间的相互关系。 There are two basic components in H.248: Termination and Context. The various resources on the MP are abstractly represented as Terminations, and the combination between the endpoints is abstracted as Context. A context can contain multiple endpoints, so Topology is used to describe the interrelationships between endpoints.
基于协议的这种抽象模型, 呼叫的接续实际上就是对端点和上下文的操 作。 这种操作通过 MC和 MP之间的命令(Command )请求和响应来完成。 命 令所携带的参数, 也称为描述符(Descriptor ), 被划分为属性(Property ), 信 号 ( Signal ), 事件 ( Event ), 统计 ( Statistic )等类别。 具有业务相关性的参数 逻辑上聚合成为包( Package )。 Based on this abstract model of the protocol, the connection of the call is actually the operation of the endpoint and context. Work. This operation is done by a command (Command) request and response between the MC and the MP. The parameters carried by the command, also called descriptors (Descriptor), are classified into categories: Property, Signal, Event, and Statistic. The parameters with business relevance are logically aggregated into packages.
目前视频编码压缩标准主要有 MPEG-x和 H.26x两大系列,其基本编码单 位为宏块( Macro Block, MB ), 在 H.261标准中, 每个宏块由 4个 8x8的亮 度块和 2个 8x8的色度块组成, 一个块组( Group of Block, GOB ) 由 3x11个 宏块组成, 一个 QCIF ( Quarter Common Intermediate Format, 四分之一通用中 间格式) 图像由 3个 GOB组成, 一个 CIF ( Common Intermediate Format, 通 用中间格式) 图像则包含 12个 GOB。 这些压缩算法分别从三个方面改善编码 效率: At present, the video coding compression standard mainly has two series of MPEG-x and H.26x, and the basic coding unit is Macro Block (MB). In the H.261 standard, each macro block consists of four 8x8 luminance blocks. It consists of two 8x8 chroma blocks. One Group of Block (GOB) consists of 3x11 macroblocks. A QCIF (Quarter Common Intermediate Format) image consists of 3 GOBs. A CIF (Common Intermediate Format) image contains 12 GOBs. These compression algorithms improve coding efficiency in three ways:
运动估计 /运动补偿 ( ME/MC, Motion Estimation/ Motion Compensation ) 消除视频时间冗余; Motion estimation/motion compensation (ME/MC, Motion Estimation/ Motion Compensation) eliminates video time redundancy;
图像差值的离散余弦变换(DCT, Discrete Cosine Transform ) 消除空间冗 余; Discrete Cosine Transform (DCT) of image difference is used to eliminate spatial redundancy;
量化系数的可变长编码(VLC, Variable Length Code ) 消除统计冗余。 实践表明, 通过上述方法, 视频编码标准获得了极高的压缩效率。 但压缩 后的码流在传输过程中存在着一些棘手的问题, 其中比较突出的一点是: 一方 面, 这些压缩后的码流对信道比特误码非常敏感; 而另一方面, 信道由于衰减 等引入了大量的随机误码和突发误码, 影响了码流的正常传输。 尤其是当采用 了 VLC方案后, 码流更加容易受到误码的影响, 结果在解码端将失去与编码 端的同步,导致在遇到下一个同步码字之前无法对 VLC码字进行正确的解码; 同时预测编码技术会将错误扩散到整个视频序列中,极大地降低重建图像的质 量。 因此, 为了实现良好质量的视频传输,必须结合实际应用信道的传输特性, 采取一定的容错措施。 The variable length code (VLC, Variable Length Code) eliminates statistical redundancy. Practice has shown that the video coding standard achieves extremely high compression efficiency by the above method. However, there are some thorny problems in the transmission of the compressed code stream. The more prominent ones are: On the one hand, these compressed code streams are very sensitive to channel bit errors; on the other hand, the channel is attenuated, etc. A large number of random errors and burst errors are introduced, which affects the normal transmission of the code stream. Especially when the VLC scheme is adopted, the code stream is more susceptible to the error, and as a result, the synchronization with the encoding end will be lost at the decoding end, so that the VLC codeword cannot be correctly decoded before the next synchronization codeword is encountered; At the same time, the predictive coding technique will spread the error into the entire video sequence, greatly reducing the quality of the reconstructed image. Therefore, in order to achieve good quality video transmission, it is necessary to take certain fault-tolerant measures in combination with the transmission characteristics of the actual application channel.
根据在视频传输系统中位置的不同, 容错算法主要可分为基于编码器的容 错算法, 基于解码器的容错算法和基于反馈信道的容错算法。 其中基于反馈信 道的容错算法, 指利用解码器获得误码信息, 并通过反馈信道, 传送给编码器 进行误码处理的一种方式, 筒单的反馈信息如: 请求视频关键帧刷新、 请求局 部宏块数据刷新等。 Depending on the location in the video transmission system, the fault-tolerant algorithm can be mainly divided into encoder-based Wrong algorithm, decoder-based fault-tolerant algorithm and feedback channel-based fault-tolerant algorithm. The fault-tolerant algorithm based on the feedback channel refers to a method of obtaining error information by using a decoder and transmitting it to the encoder for error processing through a feedback channel, and the feedback information of the package is as follows: requesting video key frame refresh, requesting local Macro block data refresh, etc.
视频信息的反馈信道一般有附于承载面的信道(带内信道)和与承载面分 离的信道(带外信道)两种方式。 两种方式均可传递视频解码器到视频编码器 的反馈信息, 从而实现基于反馈信道的容错算法。 The feedback channel of the video information generally has two modes: a channel attached to the bearer plane (intra-band channel) and a channel separated from the bearer plane (out-of-band channel). Both methods can pass the feedback information from the video decoder to the video encoder, thus implementing a feedback channel based fault tolerance algorithm.
根据传输信道的不同, 现有的传输协议可分为带外传输协议(与承载面分 离)和带内传输协议(与承载面绑定) 两类, 主要有以下几种: According to different transmission channels, the existing transmission protocols can be divided into two types: out-of-band transmission protocol (separation from bearer plane) and in-band transport protocol (bound to bearer plane), mainly in the following types:
( 1 )基于扩展 SIP方式 (1) Based on extended SIP
基于扩展 SIP的方式是一种通过在 SIP ( Session Initiated Protocol, 会话初 始化协议) 消息体中增力口 application/media_control+xml 类型的扩展 XML ( extensible Markup Language, 扩展标识语言) 实体, 来实现通过带外信令的 方式传递视频数据更新请求。 The extended SIP-based method is an extended XML (extensible markup language) entity of the application/media_control+xml type in the SIP (Session Initiated Protocol) message body. The external data signaling method delivers the video data update request.
该方式仅支持对视频关键帧(全帧内编码帧)的更新请求, 且要求终端必 须为支持该扩展的 SIP终端。由于采用 SIP扩展方式,一般采用带外传输方式。 This method only supports update requests for video key frames (full intra-coded frames), and requires the terminal to be a SIP terminal supporting the extension. Due to the SIP extension method, the out-of-band transmission method is generally adopted.
( 2 )基于扩展 RTCP报文方式 (2) Based on extended RTCP message mode
基于扩展 RTCP报文的方式是一种利用 RTCP扩展报文方式来提供视频反 馈信息, 所提供的视频反馈信息包括: The method of extending the RTCP packet is to use the RTCP extended packet method to provide video feedback information, and the provided video feedback information includes:
关键帧更新请求 ( PLI, Picture Loss Indication ); Key frame update request (PLI, Picture Loss Indication);
图像片 (Slice ) 更新请求(SLI, Slice Loss Indication ); Slice Update Request (SLI, Slice Loss Indication);
参考图像选择请求(RPSI ); Reference image selection request (RPSI);
根据不同的反馈信息, RTCP扩展报文中携带不同参数, 由于采用 RTCP 扩展方式, 一般采用带内传输方式。 According to different feedback information, the RTCP extended packet carries different parameters. Because of the RTCP extension mode, the in-band transmission mode is generally adopted.
( 3 ) H.245信令 (3) H.245 signaling
ITU-T的 H.245提供了信令面的视频数据更新请求命令, 其基本功能是当 解码器端发现误码时, 可以请求编码器端发送关键帧或者更新的 GOB、 MB。 要求终端支持 H.245信令。 该协议提供的视频更新命令包括: ITU-T H.245 provides a video data update request command for the signaling plane, the basic function of which is when When the decoder detects the error, it can request the encoder to send the key frame or the updated GOB and MB. The terminal is required to support H.245 signaling. The video update commands provided by the agreement include:
videoFastUpdatePicture , 用来快速请求整帧更新; videoFastUpdatePicture , used to quickly request the entire frame update;
videoFastUpdateGOB, 用来请求 GOB更新; videoFastUpdateGOB, used to request GOB updates;
videoFastUpdateMB , 用来请求 MB更新; videoFastUpdateMB , used to request MB updates;
该协议一般被 H.323终端采用, 采用带外传输方式。 This protocol is generally adopted by H.323 terminals and uses out-of-band transmission.
上述协议均可传输视频质量反馈信息。 同时由于终端的多样性, 不同终端 可能采用不同的传输信道和协议来实现基于反馈信道的容错算法,如图 2所示, 位于软交换网络的 H.324终端与位于 IMS网络的 SIP终端 (可能有多方终端, 图中只示意两方 )进行多媒体会话。 其中, H.324终端与 IMS终端之间进行多 媒体通信, IM-MGW/MGCF承担互通承载控制和呼叫控制的任务, 根据 3GPP 协议 MGCF同时承担了 H.245协议的终结任务。 H.324终端通常可以支持 H.245 协议中视频数据快速更新请求的处理, 但不支持 RTCP带内请求的处理; IMS 网络下 SIP终端能够支持带内的更新请求, 也能够支持带外 SIP信令的更新请 求。 The above protocols can transmit video quality feedback information. At the same time, due to the diversity of terminals, different terminals may use different transmission channels and protocols to implement a fault-tolerant algorithm based on feedback channels. As shown in Figure 2, the H.324 terminal located in the softswitch network and the SIP terminal located in the IMS network (possibly There are multiple terminals, only two parties are shown in the figure) for multimedia sessions. Among them, the H.324 terminal and the IMS terminal perform multimedia communication, and the IM-MGW/MGCF undertakes the tasks of interworking bearer control and call control. According to the 3GPP protocol, the MGCF also undertakes the termination task of the H.245 protocol. The H.324 terminal can usually support the processing of the video data fast update request in the H.245 protocol, but does not support the RTCP in-band request processing; the SIP terminal in the IMS network can support the in-band update request, and can also support the out-of-band SIP message. Order update request.
现有技术中实现对视频数据更新请求的过程可如图 3所示, 图 3中的两个 黑圆点标识视频质量检测点 A和 B, 并发起质量上报或者视频 I帧 (即整帧) 更新请求。 当 IMS终端支持 SIP + XML扩展的带外视频数据更新请求时, 如 图 3所示, MGW能够接收软交换网络的 H.324终端发起的带内请求, 并通过 H.245方式发送给 MGCF, MGCF可通过 SIP + XML方式向 IMS网络的 SIP 终端发送视频数据更新请求, 从而实现对视频数据的更新。 MGCF还可以通过 SIP + XML方式接收 SIP终端发起的带外请求, 通过 H.245方式向 H.324终端 发送带内的视频数据更新请求, 从而实现对视频数据的更新。 The process of implementing the video data update request in the prior art may be as shown in FIG. 3. The two black circles in FIG. 3 identify the video quality detection points A and B, and initiate a quality report or a video I frame (ie, the entire frame). Update request. When the IMS terminal supports the outband video data update request of the SIP + XML extension, as shown in FIG. 3, the MGW can receive the inband request initiated by the H.324 terminal of the softswitch network, and send the request to the MGCF through the H.245 mode. The MGCF can send a video data update request to the SIP terminal of the IMS network through the SIP+XML method, thereby implementing update of the video data. The MGCF can also receive the out-of-band request initiated by the SIP terminal through the SIP+XML method, and send the in-band video data update request to the H.324 terminal through the H.245 method, thereby realizing the update of the video data.
发明人在实现本发明创造的过程中, 发现现有技术至少存在如下缺陷: 当 IM-MGW 自身检测到 H.324 终端或 SIP 终端的视频质量问题时, IM-MGW无法向 MGCF请求需要更新的视频数据; 当 SIP终端通过带内 RTP/RTCP反馈视频数据更新请求, 而 IM-MGW无 法提供所请求更新数据时, IM-MGW无法通过 H.248协议将更新请求上报给 MGCF, 以便 MGCF通过 H.245消息向 H.324终端请求视频数据更新; In the process of implementing the creation of the present invention, the inventor found that the prior art has at least the following drawbacks: When the IM-MGW itself detects the video quality problem of the H.324 terminal or the SIP terminal, the IM-MGW cannot request the MGCF to update. Video data; When the SIP terminal feeds back the video data update request through the in-band RTP/RTCP, and the IM-MGW cannot provide the requested update data, the IM-MGW cannot report the update request to the MGCF through the H.248 protocol, so that the MGCF passes the H.245 message. Requesting video data updates to the H.324 terminal;
当 MGCF收到来自 H.324终端的 H.245视频数据更新请求,而 SIP终端不 支持 SIP+XML的视频数据更新请求时, MGCF无法指示 IM-MGW发送带内 的视频数据更新请求。 When the MGCF receives the H.245 video data update request from the H.324 terminal, and the SIP terminal does not support the SIP+XML video data update request, the MGCF cannot instruct the IM-MGW to send the in-band video data update request.
综上, 现有技术没有提供在上述情况下, 如何通过 MGCF指示 MGW, 对 需要更新的视频数据进行更新请求。 发明内容 媒体控制器指示媒体处理器, 对需要更新的视频数据进行更新请求。 In summary, the prior art does not provide an indication of how to update the video data that needs to be updated by the MGCF instructing the MGW in the above case. SUMMARY OF THE INVENTION A media controller instructs a media processor to make an update request for video data that needs to be updated.
本发明实施例提供的视频数据的更新方法, 包括: The method for updating video data provided by the embodiment of the present invention includes:
媒体处理器接收媒体控制器发送的指示, 该指示用于指示所述媒体处理器 检测视频源的视频质量; Receiving, by the media processor, an indication sent by the media controller, the indication is used to instruct the media processor to detect a video quality of the video source;
所述媒体处理器检测视频源的视频质量, 并根据检测结果向所述媒体控制 器请求需要更新的视频数据。 The media processor detects a video quality of the video source, and requests the media controller to update the video data according to the detection result.
本发明实施例提供的装置为媒体控制器, 该媒体控制器包括: The device provided by the embodiment of the present invention is a media controller, and the media controller includes:
指示发送模块, 用于向媒体处理器发送指示; Instructing a sending module, configured to send an indication to the media processor;
更新请求接收模块, 用于接收所述媒体处理器发送的请求, 所述请求用于 请求需要更新的视频数据; 所述请求为所述媒体处理器根据所述指示发送模块 发送的指示, 检测视频源的视频质量, 并根据检测结果发送的请求。 An update request receiving module, configured to receive a request sent by the media processor, where the request is used to request video data that needs to be updated; and the request is that the media processor detects the video according to the indication sent by the indication sending module. The quality of the source video, and the request sent based on the detection result.
本发明实施例提供的装置为媒体处理器, 该媒体处理器包括: The device provided by the embodiment of the present invention is a media processor, and the media processor includes:
指示接收模块, 用于接收媒体控制器发送的指示; An indication receiving module, configured to receive an indication sent by the media controller;
检测模块, 用于根据所述指示检测视频源的视频质量; a detecting module, configured to detect a video quality of the video source according to the indication;
更新请求模块, 用于根据所述检测模块的视频质量检测结果, 向所述媒体 控制器请求需要更新的视频数据。 An update request module, configured to send to the media according to a video quality detection result of the detection module The controller requests video data that needs to be updated.
本发明实施例提供的视频数据更新系统,包括:媒体控制器和媒体处理器; 所述媒体处理器, 用于根据所述媒体控制器的指示检测视频源的视频质 量, 并根据视频质量检测结果向所述媒体控制器发送请求, 请求需要更新的视 频数据; The video data update system provided by the embodiment of the present invention includes: a media controller and a media processor; the media processor is configured to detect a video quality of the video source according to the indication of the media controller, and according to the video quality detection result Sending a request to the media controller to request video data that needs to be updated;
所述媒体控制器, 用于指示所述媒体处理器检测视频源的视频质量。 The media controller is configured to instruct the media processor to detect a video quality of a video source.
本发明的上述实施例, 通过媒体控制器向媒体处理器发送指示, 指示检测 视频源的视频质量, 使媒体处理器进行视频质量检测, 并在检测到视频质量问 题时, 能够上报给媒体控制器以进行视频数据更新, 解决了现有技术在这方面 的不足, 实现了由媒体处理器根据媒体控制器的指示进行视频数据质量检测和 对视频数据进行更新请求。 In the above embodiment of the present invention, the media controller sends an indication to the media processor to indicate the video quality of the video source, so that the media processor performs video quality detection, and can report to the media controller when the video quality problem is detected. In order to update the video data, the deficiencies of the prior art in this aspect are solved, and the video processor performs the video data quality detection and the video data update request according to the indication of the media controller.
本发明实施例提供的另一种视频数据的更新方法, 包括: Another method for updating video data according to an embodiment of the present invention includes:
媒体处理器接收媒体控制器发送的指示, 并进行检测, 在检测到视频源发 起的带内视频数据更新请求时, 向所述媒体控制器请求需要更新的视频数据; 所述指示为所述媒体处理器接收媒体控制器发送的指示, 用于指示所述媒 体处理器检测视频源发起的带内视频数据更新请求。 Receiving, by the media processor, an indication sent by the media controller, and detecting, when detecting an in-band video data update request initiated by the video source, requesting, by the media controller, video data that needs to be updated; the indication is the media The processor receives an indication sent by the media controller, to instruct the media processor to detect an in-band video data update request initiated by the video source.
本发明实施例提供的另一种装置为媒体控制器, 该媒体控制器包括: 指示发送模块, 用于向媒体处理器发送指示; Another device provided by the embodiment of the present invention is a media controller, where the media controller includes: an indication sending module, configured to send an indication to the media processor;
更新请求接收模块, 用于接收所述媒体处理器发送的请求, 请求需要更新 的视频数据; 所述请求为所述媒体处理器根据所述指示发送模块发送的指示, 检测指定视频源发起的带内视频更新请求, 并在检测到所述带内视频更新请求 时发送的请求。 An update request receiving module, configured to receive a request sent by the media processor, requesting video data that needs to be updated; and the request is used by the media processor to detect a band initiated by the specified video source according to the indication sent by the indication sending module. An internal video update request, and a request sent when the in-band video update request is detected.
本发明实施例提供的另一种装置为媒体处理器, 该媒体处理器包括: 指示接收模块, 用于接收媒体控制器发送的指示; Another device provided by the embodiment of the present invention is a media processor, where the media processor includes: an indication receiving module, configured to receive an indication sent by the media controller;
检测模块, 用于根据所述指示检测所述指示指定的视频源发起的带内视频 数据更新请求; 更新请求模块, 用于在所述检测模块检测到所述带内视频数据更新请求 后, 向所述媒体控制器请求需要更新的视频数据。 a detecting module, configured to detect, according to the indication, an in-band video data update request initiated by the specified video source; And an update requesting module, configured to request the media controller to update the video data after the detecting module detects the in-band video data update request.
本发明实施例提供的视频数据更新系统,包括:媒体控制器和媒体处理器; 所述媒体处理器, 用于根据所述媒体控制器的指示, 检测视频源的带内视 频更新请求, 并在检测到所述带内视频更新请求后向所述媒体控制器发送请 求, 请求需要更新的视频数据; The video data update system provided by the embodiment of the present invention includes: a media controller and a media processor; the media processor is configured to detect an in-band video update request of the video source according to the indication of the media controller, and After detecting the in-band video update request, sending a request to the media controller to request video data that needs to be updated;
所述媒体控制器,用于指示所述媒体处理器检测视频源发起的带内视频更 新请求。 The media controller is configured to instruct the media processor to detect an in-band video update request initiated by a video source.
本发明的上述实施例, 通过媒体控制器向媒体处理器发送指示, 指示检测 视频源的带内视频数据更新请求,使媒体处理器能够在检测到视频源的带内视 频数据更新请求后, 向媒体控制器发起视频数据更新请求, 解决了现有技术中 在这方面的不足, 实现了由媒体处理器根据媒体控制器的指示检测带内视频数 据更新请求并上报给媒体控制器, 以请求视频数据更新。 In the above embodiment of the present invention, the media controller sends an indication to the media processor to instruct the detection of the in-band video data update request of the video source, so that the media processor can detect the in-band video data update request of the video source. The media controller initiates a video data update request, which solves the deficiencies in the prior art. The media processor detects the in-band video data update request according to the indication of the media controller and reports the request to the media controller to request the video. Data Update.
本发明实施例提供的另一种视频数据的更新方法, 包括: Another method for updating video data according to an embodiment of the present invention includes:
媒体处理器根据媒体控制器发送的指示, 向所述指示指定的视频源发起带 内视频数据更新请求, 请求更新所述指示指定的视频数据; 所述指示为所述媒 体控制器发送给所述媒体处理器的指示, 指示所述媒体处理器向视频源发起带 内视频数据更新请求, 请求更新指定的视频数据。 And the media processor initiates an in-band video data update request to the video source specified by the indication according to the indication sent by the media controller, requesting to update the video data specified by the indication; the indication is sent by the media controller to the An indication of the media processor instructing the media processor to initiate an in-band video data update request to the video source to request to update the specified video data.
本发明实施例提供的另一种装置为媒体控制器, 该媒体控制器包括: 更新请求接收模块, 用于接收视频源发起的带外视频数据更新请求; 指示发送模块, 用于向媒体处理器发送指示, 指示所述媒体处理器发起带 内视频数据更新请求, 请求需要更新的视频数据; 所述带内视频数据更新请求 所请求更新的视频数据为所述带外视频数据更新请求所指示更新的视频数据。 Another device provided by the embodiment of the present invention is a media controller, where the media controller includes: an update request receiving module, configured to receive an out-of-band video data update request initiated by a video source; and an indication sending module, configured to send to the media processor Sending an indication, instructing the media processor to initiate an in-band video data update request, requesting video data that needs to be updated; and the video data requested to be updated by the in-band video data update request is an update indicated by the out-of-band video data update request Video data.
本发明实施例提供的另一种装置为媒体处理器, 该媒体处理器包括: 指示接收模块, 用于接收媒体控制器发送的指示; Another device provided by the embodiment of the present invention is a media processor, where the media processor includes: an indication receiving module, configured to receive an indication sent by the media controller;
更新请求模块, 用于根据所述指示, 向所述视频源发起带内视频数据更新 请求, 请求更新所述指示所指示的视频数据。 An update request module, configured to initiate an in-band video data update to the video source according to the indication Requesting, requesting to update the video data indicated by the indication.
本发明实施例提供的视频数据更新系统,包括:媒体控制器和媒体处理器; 所述媒体处理器, 用于根据所述媒体控制器的指示, 向所述被请求的视频 源发起带内视频更新请求, 请求更新视频数据; The video data update system provided by the embodiment of the present invention includes: a media controller and a media processor; the media processor is configured to initiate an in-band video to the requested video source according to the indication of the media controller Update request, request to update video data;
所述媒体控制器, 用于在接收到视频源的带外视频更新请求后, 向所述媒 体处理器发送所述指示,指示所述媒体处理器向所述带外视频更新请求中被请 求的视频源发起带内视频数据更新请求。 The media controller, configured to send the indication to the media processor after receiving an out-of-band video update request of a video source, instructing the media processor to request the out-of-band video update request The video source initiates an in-band video data update request.
本发明的上述实施例, 通过媒体控制器向媒体处理器发送指示, 指示其发 起带内视频数据更新请求,使媒体处理器能够在接收到该指示后按照该指示发 起带内视频数据更新请求, 解决了现有技术在这方面的不足, 实现了媒体处理 器根据媒体控制器的指示, 通过发起带内视频数据更新请求进行视频数据更 新。 In the foregoing embodiment of the present invention, the media controller sends an indication to the media processor to instruct the device to initiate an in-band video data update request, so that the media processor can initiate an in-band video data update request according to the indication after receiving the indication. The deficiencies of the prior art in this aspect are solved, and the media processor is implemented to perform video data update by initiating an in-band video data update request according to an instruction of the media controller.
附图说明 DRAWINGS
为了更清楚地说明本发明实施例或现有技术中的技术方案, 下面将对实施 例或现有技术描述中所需要使用的附图作筒单地介绍, 显而易见地, 下面描述 中的附图仅仅是本发明的一些实施例, 对于本领域普通技术人员来讲, 在不付 出创造性劳动的前提下, 还可以根据这些附图获得其他的附图。 In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description It is merely some embodiments of the present invention, and those skilled in the art can obtain other drawings according to the drawings without any creative work.
图 1为现有技术中媒体处理器的示意图; 1 is a schematic diagram of a media processor in the prior art;
图 2为现有技术中的软交换终端与 SIP终端互通的架构的示意图; 图 3为现有技术中的软交换终端与 SIP终端互通的场景示意图; 2 is a schematic diagram of an architecture of a softswitch terminal and a SIP terminal in the prior art; FIG. 3 is a schematic diagram of a scenario in which a softswitch terminal and a SIP terminal communicate with each other in the prior art;
图 4A和图 4B为本发明实施例一的软交换终端与 SIP终端互通的场景示 意图; 4A and FIG. 4B are schematic diagrams showing a scenario in which a softswitch terminal and a SIP terminal communicate with each other according to Embodiment 1 of the present invention;
图 5A为本发明实施例一的 MGW检测视频质量的流程示意图; 5A is a schematic flowchart of detecting video quality by an MGW according to Embodiment 1 of the present invention;
图 5B为本发明实施例一的 MGW检测视频质量的信令流程示意图; 图 6为本发明实施例二的软交换终端与 SIP终端互通的场景示意图; 图 7A为本发明实施例二的 MGW检测带内请求报文的流程示意图; 图 7B为本发明实施例二的 MGW检测带内请求报文的信令流程示意图; 图 8为本发明实施例三的软交换终端与 SIP终端互通的场景示意图; 图 9A为本发明实施例三的 MGCF指示 MGW发送视频更新请求的流程示 图 9B为本发明实施例三的 MGCF指示 MGW发送视频更新请求的信令流 程示意图; FIG. 5B is a schematic diagram of a signaling flow of an MGW detecting a video quality according to Embodiment 1 of the present invention; FIG. 6 is a schematic diagram of a scenario in which a softswitch terminal communicates with a SIP terminal according to Embodiment 2 of the present invention; FIG. 7A is a schematic diagram of MGW detection according to Embodiment 2 of the present invention; Schematic diagram of the process of in-band request messages; FIG. 7B is a schematic diagram of a signaling flow of an MGW detecting an in-band request packet according to Embodiment 2 of the present invention; FIG. 8 is a schematic diagram of a scenario in which a softswitch terminal and a SIP terminal communicate with each other according to Embodiment 3 of the present invention; FIG. 9B is a schematic flowchart of signaling flow of an MGCF instructing an MGW to send a video update request according to Embodiment 3 of the present invention;
图 10为本发明实施例的媒体控制器的结构示意图; FIG. 10 is a schematic structural diagram of a media controller according to an embodiment of the present invention;
图 11为本发明实施例的媒体处理器的结构示意图。 具体实施方式 FIG. 11 is a schematic structural diagram of a media processor according to an embodiment of the present invention. detailed description
下面结合附图对本发明实施例进行详细描述。 The embodiments of the present invention are described in detail below with reference to the accompanying drawings.
本发明实施例提出一种在承载控制分离架构下,媒体控制器和媒体处理器 之间通过视频质量检测与上报方式, 或者通过带内视频数据更新请求检测与上 报方式, 或者通过媒体控制器指示媒体处理器发起视频数据更新请求的方式, 通知视频编码源端发送更新数据, 从而提高视频通信质量的技术方案。 The embodiment of the invention provides a method for detecting and reporting a video quality between a media controller and a media processor, or an in-band video data update request detection and reporting mode, or by a media controller, in a bearer control separation architecture. The technical solution that the media processor initiates the video data update request, and notifies the video encoding source to send the update data, thereby improving the video communication quality.
实施例一 Embodiment 1
本实施例描述了 MGCF指示 MGW进行视频质量检测并上报的过程。 在 该过程中, MGW参与视频编解码, 并能动态检测视频编码源端发来的视频流 质量, 并将质量信息上报 MGCF, 从而实现会话过程中由 MGW来负责对视频 编码源的质量跟踪和更新请求。 This embodiment describes a process in which the MGCF instructs the MGW to perform video quality detection and reporting. In this process, the MGW participates in the video codec, and can dynamically detect the video stream quality sent by the video encoding source, and report the quality information to the MGCF, so that the MGW is responsible for the quality tracking of the video encoding source during the session. Update request.
本实施例的具体应用场景可以如图 4A和图 4B所示。 The specific application scenario of this embodiment can be as shown in FIG. 4A and FIG. 4B.
在图 4A中, 由于 MGW参与视频编解码, 因此可将视频质量检测点设置 于 MGW。 当 MGW检测到来自视频编码源端 H.324终端的视频质量存在问题 时, 向 MGCF反馈视频质量, MGCF通过 H.245方式请求 H.324终端发送视 频更新数据。 In Fig. 4A, since the MGW participates in video codec, the video quality detection point can be set to the MGW. When the MGW detects that there is a problem with the video quality of the H.324 terminal from the video encoding source, the video quality is fed back to the MGCF, and the MGCF requests the H.324 terminal to send the video update data through the H.245 mode.
在图 4B中, 由于 MGW参与视频编解码, 因此可将视频质量检测点设置 于 MGW。当 MGW检测到来自视频编码源端 SIP终端的视频质量存在问题时, 向 MGCF反馈视频质量, MGCF通过 SIP+XML方式请求 SIP终端发送视频更 新数据。 In FIG. 4B, since the MGW participates in the video codec, the video quality detection point can be set. At MGW. When the MGW detects that there is a problem with the video quality of the SIP terminal from the video encoding source, the video quality is fed back to the MGCF, and the MGCF requests the SIP terminal to send the video update data through the SIP+XML method.
参见图 5A, 为本发明实施例一的 MGW检测视频质量的流程示意图, 该 流程图针对图 4A和图 4B所给出的场景描述了由 MGW检测视频质量并上报 给 MGCF的过程, 具体包括: FIG. 5A is a schematic flowchart of the MGW detecting the video quality according to the first embodiment of the present invention. The flowchart describes the process of detecting the video quality by the MGW and reporting it to the MGCF according to the scenario shown in FIG. 4A and FIG. 4B, which specifically includes:
步骤 501、 多媒体会话建立过程, 在此过程中 MGCF确定 MGW是否参与 了在不同视频源之间的视频编解码, 如进行会议混屏或转码。 Step 501: A multimedia session establishment process, in which the MGCF determines whether the MGW participates in video codec between different video sources, such as performing conference mixing or transcoding.
步骤 502、 在确认 MGW参与视频编解码后, MGCF指示 MGW开始检测 视频质量, 并确定检测标准, 如需要检测的视频质量等级要求, 并将确定的检 测标准下发给该 MGW。 Step 502: After confirming that the MGW participates in the video codec, the MGCF instructs the MGW to start detecting the video quality, and determines the detection standard, such as the video quality level requirement to be detected, and sends the determined detection standard to the MGW.
步骤 503、 MGW在解码过程中, 根据 MGCF指定的质量等级要求, 对接 收到的视频数据按要求进行计算或统计。 Step 503: During the decoding process, the MGW calculates or counts the received video data according to requirements according to the quality level specified by the MGCF.
步骤 504 ~ 505、 当 MGW计算发现视频质量下降到 MGCF指定的质量等 级之下时, 同时无法通过解码过程恢复视频质量的等级时, 则向 MGCF上报 需要更新的视频数据信息。 Steps 504 to 505: When the MGW calculates that the video quality falls below the quality level specified by the MGCF, and fails to restore the video quality level through the decoding process, the video data information that needs to be updated is reported to the MGCF.
步骤 506、 MGCF根据 MGW上报的需要更新的视频数据信息, 发送带外 视频数据更新请求。 如果检测到 H.324端视频质量问题, 则 MGCF通过 H.245 方式向 H.324终端发送带外视频数据更新请求; 如果检测到 SIP终端视频质量 问题, 则 MGCF通过 SIP+XML方式向 SIP终端发送带外视频数据更新请求。 Step 506: The MGCF sends an out-of-band video data update request according to the video data information that needs to be updated by the MGW. If the H.324 video quality problem is detected, the MGCF sends an out-of-band video data update request to the H.324 terminal through the H.245 mode; if the SIP terminal video quality problem is detected, the MGCF sends the SIP terminal to the SIP terminal through the SIP+XML method. Send out-of-band video data update request.
图 5A所示的流程中, 视频质量等级的设定和检测标准由 MGCF决定, 可 能但不限于宏块误码率、 视频质量的其它检测标准等。 In the flow shown in Fig. 5A, the setting and detection criteria of the video quality level are determined by the MGCF, and may be, but not limited to, a macroblock error rate, other detection standards of video quality, and the like.
可以看出,当视频源由于网络传输或其它原因导致视频质量下降时, MGW 和 MGCF会根据质量下降程度, 决定是否请求新的视频数据, 从而保证视频 源到 MGW间的视频数据刷新。 It can be seen that when the video source is degraded due to network transmission or other reasons, the MGW and the MGCF determine whether to request new video data according to the degree of quality degradation, thereby ensuring video data refresh between the video source and the MGW.
本实施例通过扩展 H.248协议实现上述流程。 例如, 本实施例在 H.248协 议中增加视频质量检测事件, MGCF可以通过向 MGW下发该事件,指示 MGW 按照指定的标准对视频质量事件进行检测并上报; 在 MGW上报给 MGCF的 H.248协议消息中携带视频数据更新参数,用于表示需要更新的视频数据信息。 This embodiment implements the above process by extending the H.248 protocol. For example, this embodiment is in the H.248 Association. The video quality detection event is added to the conference. The MGCF may send the event to the MGW to instruct the MGW to detect and report the video quality event according to the specified standard. The H.248 protocol message reported by the MGW to the MGCF carries the video data update parameter. Used to indicate video data information that needs to be updated.
MGCF向 MGW下发的视频质量检测事件中,携带视频质量检测参数, 这 些视频质量检测参数指示 MGW进行视频质量检测与上报所遵循的标准,这些 视频质量检测参数可以包括如下一个或多个参数: The video quality detection event sent by the MGCF to the MGW carries video quality detection parameters, and the video quality detection parameters are used to indicate the standards followed by the MGW for video quality detection and reporting. The video quality detection parameters may include one or more of the following parameters:
1、 最小视频宏块误码率(Min Percent of Error, minpoe ), 可以将该最小 视频宏块误码率作为是否需要进行视频更新的一个判断数据, 当 MGW检测到 的视频宏块误码概率小于该最小视频宏块误码率的值时, 则认为当前的视频质 量可以接受,否则认为当前的视频质量出现局部下降,需要发送局部更新请求。 该最小视频宏块误码率的允许范围可设为 20-50。 1. The minimum video macroblock error rate (Min Percent of Error, minpoe), the minimum video macroblock error rate can be used as a judgment data for whether video update is required, and the video macroblock error probability detected by the MGW. If the value of the minimum video macroblock error rate is less than the value of the minimum video macroblock error rate, the current video quality is considered acceptable, otherwise the current video quality is considered to be locally degraded, and a partial update request needs to be sent. The allowable range of the minimum video macroblock error rate can be set to 20-50.
2、 最大视频宏块误码率 ( Max Percent of Error, maxpoe ), 可以将该最大 视频宏块误码率作为是否需要进行视频更新的一个判断数据, 当 MGW检测到 的视频宏块误码概率大于该最大视频宏块误码率的值时, 则认为当前的视频质 量严重下降, 需要发送整个视频帧更新请求, 即 I帧请求。 该最大视频宏块误 码率的允许范围可设为 minpoe< maxpoe<100。 2. The maximum video macroblock error rate (Max Percent of Error, maxpoe), the maximum video macroblock error rate can be used as a judgment data for whether or not to perform video update, and the video macroblock error probability detected by the MGW If the value of the maximum video macroblock error rate is greater than the value of the maximum video macroblock error rate, the current video quality is considered to be seriously degraded, and the entire video frame update request, that is, the I frame request, needs to be sent. The allowable range of the maximum video macroblock error rate can be set to minpoe < maxpoe < 100.
3、 最小上报时间间隔(notify interval, nint ), 该最小上报时间间隔规定了 MGW每次进行上报的最小时间间隔。 该最小上报时间间隔的设置是为了防止 在网络质量较差, 视频质量一直处于较差的情况下, MGW过于频繁的上报检 测视频质量的事件。 3. The minimum reporting interval (nint), which specifies the minimum interval for the MGW to report each time. The minimum reporting interval is set to prevent the MGW from reporting the event of detecting video quality too frequently when the network quality is poor and the video quality is always poor.
以上参数决定了 MGW对视频质量检测的标准和上 频率。 例如, 视频质 量检测事件参数为: minpoe=30, maxpoe=60, nint=5s; MGW统计实际接收的 宏块数量和出现误码的宏块数量, 当宏块误码率达到 30% , MGW上报视频质 量检测事件, 请求更新局部数据; 当宏块误码率达到 60% , MGW上报视频质 量检测事件, 请求更新整帧数据; 若在一段时间内, 由于某种原因视频数据的 宏块误码率始终高于设定的阈值( minpoe或 maxpoe ), 则当 MGW完成一次事 件上报后, 需要等待 5s再上报新的视频质量检测事件。 The above parameters determine the standard and upper frequency of MGW for video quality detection. For example, the video quality detection event parameters are: minpoe=30, maxpoe=60, nint=5s; The MGW counts the actual number of received macroblocks and the number of macroblocks with errors. When the macroblock error rate reaches 30%, the MGW reports Video quality detection event, request to update local data; When the macroblock error rate reaches 60%, the MGW reports a video quality detection event, requesting to update the entire frame data; if for some reason, the macroblock error of the video data is for some reason The rate is always higher than the set threshold (minpoe or maxpoe), then when the MGW completes a thing After the report is submitted, it needs to wait for 5s to report the new video quality detection event.
MGW基于视频质量检测事件向 MGCF上报的消息中,携带视频数据更新 参数, 这些视频数据更新参数指明了需要更新的视频数据, 这些视频数据更新 参数可以包括以下一个或多个参数: The MGW carries the video data update parameters based on the video quality detection event reported to the MGCF. The video data update parameters indicate the video data that needs to be updated. The video data update parameters may include one or more of the following parameters:
firstGOB ,该参数指明在视频图像中第 1个需要被更新的 GOB标识, GOB 标识由具体的编码算法决定; firstGOB, this parameter indicates the first GOB identifier to be updated in the video image, and the GOB identifier is determined by a specific encoding algorithm;
numberOfGOBs , 该参数指明需要更新的 GOB的数量; numberOfGOBs , this parameter indicates the number of GOBs that need to be updated;
firstMB , 该参数指明第 1个需要被更新的宏块标识,宏块标识由具体的编 码算法决定; firstMB, this parameter indicates the first macroblock identifier to be updated, and the macroblock identifier is determined by a specific encoding algorithm;
numberOfMBs , 该参数指明需要更新的宏块数量; numberOfMBs , this parameter indicates the number of macroblocks that need to be updated;
fastUpdatePicture, 该参数指明需要更新整个视频图像, 即需要视频源发送 I帧数据。 fastUpdatePicture, this parameter indicates that the entire video image needs to be updated, that is, the video source is required to send I frame data.
MGW上报的以上视频数据更新参数可以实现对局部 GOB、 局部 MB和 / 或整个 I帧数据的更新请求。 例如, 上报的视频数据更新参数为 firstGOB=3 , numberOfGOBs=5 , 表示需要更新第 3-7共 5个 GOB的视频数据; 上报的视频 数据更新参数为 fastUpdatePicture=l , 表示需要更新整个视频图像帧, 需要视 频源发送新的 I帧数据。 The above video data update parameter reported by the MGW may implement an update request for the local GOB, the local MB, and/or the entire I frame data. For example, the reported video data update parameter is firstGOB=3, numberOfGOBs=5, indicating that the video data of the 5th to 5th GOBs needs to be updated; the reported video data update parameter is fastUpdatePicture=l, indicating that the entire video image frame needs to be updated. , the video source is required to send new I frame data.
在图 2所示的网络结构下, MGW检测会话双方的视频质量并上报的过程 中, 对会话双方的检测过程是一致的。 其中 MGW检测 H.324终端视频质量, 并上报需要更新的视频数据的信令流程可如图 5B所示。 In the network structure shown in Figure 2, the MGW detects the video quality of both parties in the session and reports the same process. The signaling process of the MGW detecting the video quality of the H.324 terminal and reporting the video data to be updated may be as shown in FIG. 5B.
参见图 5B , 为本发明实施例一的 MGW检测视频质量的信令流程示意图, 具体包括: FIG. 5B is a schematic diagram of a signaling process for detecting video quality by an MGW according to Embodiment 1 of the present invention, which specifically includes:
步骤 511、 建立多媒体会话。 Step 511: Establish a multimedia session.
步骤 512、 MGCF 向会话中的指定端点通过 H.248 协议发送请求消息给 MGW, 消息中指示视频质量检测事件, 以及相应的事件参数。 Step 512: The MGCF sends a request message to the MGW through the H.248 protocol to the specified endpoint in the session, where the message indicates the video quality detection event and the corresponding event parameter.
本实施例中, MGCF下发检测的是 H.324终端所对应的端点 X( Termination X ), 视频质量检测事件参数包括最小视频宏块误码率(minpoe ) 为 30, 最大 视频宏块误码率 (maxpoe ) 为 60, 上报间隔时间为 5s。 In this embodiment, the MGCF sends the endpoint X corresponding to the H.324 terminal ( Termination X), the video quality detection event parameters include a minimum video macroblock error rate (minpoe) of 30, a maximum video macroblock error rate (maxpoe) of 60, and a reporting interval of 5 s.
步骤 513、 MGW收到请求消息后进行响应。 Step 513: The MGW responds after receiving the request message.
步骤 514 ~ 515、 MGW根据 MGCF指示视频质量检测事件及其参数, 开 始统计接收的宏块数量以及误码宏块的数量, 对视频流数据进行检测, 当检测 到视频宏块误码率大于 60%且与上次上报间隔大于 5s时, 向 MGCF发送通知 消息, 通过该通知消息向 MGCF上报视频质量检测事件, 其中携带视频数据 更新参数( fastUpdatePicture=l ), 指示出需要更新的视频数据。 Steps 514-515: The MGW starts to count the number of received macroblocks and the number of error macroblocks according to the MGCF indicating the video quality detection event and its parameters, and detects the video stream data, when detecting that the video macroblock error rate is greater than 60. And the notification message is sent to the MGCF, and the video quality detection event is reported to the MGCF by the notification message, which carries the video data update parameter (fastUpdatePicture=l), indicating the video data that needs to be updated.
当 MGW检测到宏块错误率高于最小视频宏块误码率 (如本实施例中的 minpoe=30 % )但低于最大视频宏块误码率 (如本实施例中的 maxpoe=60 % ) 时, 则通知 MGCF进行局部数据更新, 其中可包括第一个丢失的宏块地址、 丢失的宏块数量以及时间位置 (可以参考 AVPF ( Audio- Visual Profile with Feedback, 语音视频质量反馈) 中的 SLI命令)。 当检测到误码率高于最大视 频宏块误码率 (如本实施例中的 maxpoe=60 % ) 时, 则通过参数 fastUpdatePicture=l通知 MGCF进行整帧数据更新。 When the MGW detects that the macroblock error rate is higher than the minimum video macroblock error rate (such as minpoe=30% in this embodiment) but lower than the maximum video macroblock error rate (such as maxpoe=60% in this embodiment) When the MGCF is notified to perform local data update, which may include the first lost macroblock address, the number of lost macroblocks, and the time position (refer to AVPF (Audio-Visual Profile with Feedback)). SLI command). When it is detected that the bit error rate is higher than the maximum video macroblock error rate (such as maxpoe=60% in this embodiment), the MGCF is notified by the parameter fastUpdatePicture=1 to perform the entire frame data update.
步骤 516、 MGCF收到通知消息后进行响应。 Step 516: The MGCF responds after receiving the notification message.
步骤 517 、 MGCF 根据 MGW 上报的视频数据更新参数 ( fastUpdatePicture=l ) , 向 H.324 终 端 发 送 H.245 信 令 ( 如 videoFastUpdatePicture ), 请求 H.324终端更新整帧数据。 Step 517: The MGCF sends an H.245 signal (such as videoFastUpdatePicture) to the H.324 terminal according to the video data update parameter (fastUpdatePicture=l) reported by the MGW, and requests the H.324 terminal to update the entire frame data.
上述图 5B所示的流程中, MGCF下发检测的是 H.324终端所对应的端点 X, 此外 MGCF也可以检测 SIP终端对应的端点, 这种情况下, 当 SIP终端端 点检测到视频质量问题时, 同样要上报 MGCF; 而 MGCF将根据上报的参数 发送 SIP + XML方式的视频数据更新请求给 SIP终端,请求 SIP终端进行视频 数据更新。 In the foregoing process shown in FIG. 5B, the MGCF sends the endpoint X corresponding to the H.324 terminal, and the MGCF can also detect the endpoint corresponding to the SIP terminal. In this case, when the SIP endpoint detects the video quality problem. At the same time, the MGCF is also reported; and the MGCF will send a SIP + XML video data update request to the SIP terminal according to the reported parameters, requesting the SIP terminal to perform video data update.
实施例二 Embodiment 2
本实施例描述了 MGCF指示 MGW进行带内视频数据更新信令检测并上 报视频数据更新请求, 使 MGW能检测视频数据更新请求并上报给 MGCF, 从 而使 MGCF 向被请求的视频源传递视频数据更新请求, 实现会话过程中对视 频数据的更新。 This embodiment describes that the MGCF instructs the MGW to perform in-band video data update signaling detection and The video data update request is reported, so that the MGW can detect the video data update request and report it to the MGCF, so that the MGCF delivers the video data update request to the requested video source, and implements the update of the video data during the session.
本实施例的具体应用场景可以如图 6所示。 A specific application scenario of this embodiment may be as shown in FIG. 6.
在图 6中,由 MGW检测 SIP终端发起的带内视频数据更新请求,当 MGW 检测到来自 SIP终端的带内视频数据更新请求时通知 MGCF, MGCF通过 H.245 方式请求 H.324终端发送视频更新数据。 In FIG. 6, the MGW detects the in-band video data update request initiated by the SIP terminal, and notifies the MGCF when the MGW detects the in-band video data update request from the SIP terminal, and the MGCF requests the H.324 terminal to send the video through the H.245 mode. update data.
参见图 7A, 为本发明实施例二的 MGW检测带内请求报文的流程示意图, 该流程图针对图 6所给出的场景描述了由 MGW检测 SIP终端发起的视频数据 更新请求并上报给 MGCF的过程, 具体包括: FIG. 7A is a schematic flowchart of detecting an in-band request message by an MGW according to Embodiment 2 of the present invention. The flowchart describes, for the scenario given in FIG. 6, the MGW detects a video data update request initiated by a SIP terminal, and reports it to the MGCF. The process specifically includes:
步骤 701、 多媒体会话建立过程。 Step 701: A multimedia session establishment process.
步骤 702、MGCF决定需要检测的 SIP终端带内视频数据更新请求的 RTCP 报文类型, 并指示 MGW开始检测 RTCP报文。 Step 702: The MGCF determines an RTCP packet type of the in-band video data update request of the SIP terminal to be detected, and instructs the MGW to start detecting the RTCP packet.
步骤 703 ~ 704、MGW接收终端发来的 RTCP报文并对报文类型进行分析, 确定是否是 MGCF所指示检测的报文类型。 Steps 703 to 704: The MGW receives the RTCP packet sent by the terminal and analyzes the packet type to determine whether it is the packet type indicated by the MGCF.
步骤 705、 当 MGW确定接收到的 RTCP报文是 MGCF所指示检测的报文 类型时, 将该 RTCP报文所携带的视频数据更新参数上报给 MGCF。 Step 705: When the MGW determines that the received RTCP packet is the type of the packet that is detected by the MGCF, the MGW reports the update parameter of the video data carried by the RTCP packet to the MGCF.
步骤 706、 MGCF根据 MGW上报的视频数据更新参数, 通过 H.245方式 向 H.324终端发送带外视频数据更新请求, 请求更新该视频数据更新参数所指 示的视频数据。 Step 706: The MGCF sends an out-of-band video data update request to the H.324 terminal according to the video data reported by the MGW, and requests to update the video data indicated by the video data update parameter.
图 7A所示的上述流程中, MGW上报的视频数据更新参数与 SIP终端发 送的带内 RTCP报文所携带的视频数据更新参数一致。 In the above process shown in FIG. 7A, the video data update parameter reported by the MGW is consistent with the video data update parameter carried by the inband RTCP packet sent by the SIP terminal.
本实施例通过扩展 H.248协议实现上述流程。 本实施例在 H.248协议中增 加检测 RTCP视频质量反馈报文的事件(视频更新请求检测事件), MGCF可 以通过向 MGW下发该事件, 指示 MGW对负荷类型( Payload Type )为 PSFB ( payload-specific feedback messages, 对负荷的反馈消息) 的 RTCP报文, 如 RTCP/AVPF请求, 进行检测并上报。 This embodiment implements the above process by extending the H.248 protocol. In this embodiment, an event (a video update request detection event) for detecting an RTCP video quality feedback message is added to the H.248 protocol, and the MGCF may send the event to the MGW to indicate that the MGW has a payload type (payload type) of PSFB (payload). -specific feedback messages, RTCP messages for load feedback messages, such as RTCP/AVPF request, detection and reporting.
MGCF向 MGW下发的 RTCP视频质量反馈报文检测事件中,携带事件参 数, 该事件参数用于指示 MGW进行 RTCP视频质量反馈报文的检测和上报, 这些事件参数可以包括以下一个或多个参数: The event of the RTCP video quality feedback packet sent by the MGCF to the MGW carries an event parameter, where the event parameter is used to instruct the MGW to detect and report the RTCP video quality feedback message. The event parameters may include one or more of the following parameters. :
1、 RTCP的报文类型 (Feedback Message Type, fmt ), 用于确定需要检测 的报文类型。 类型值由 RTCP视频质量反馈扩展报文的相关标准确定, 其中与 视频数据更新请求相关的报文有 Picture Loss Indication ( PLI, 关键帧更新请 求)、 Slice Loss Indication ( SLI, 图像片更新请求)、 Full Intra Request ( FIR, I 帧更新请求)等。 1. The message type of the RTCP (Feedback Message Type, fmt) is used to determine the type of the packet to be detected. The type value is determined by the relevant standard of the RTCP video quality feedback extended message, wherein the message related to the video data update request includes Picture Loss Indication (PLI, Keyframe Update Request), Slice Loss Indication (SLI, Image Slice Update Request), Full Intra Request ( FIR, I frame update request), etc.
2、 最小上报时间间隔(notify interval, nint ), 表示两次上报的最小时间间 隔, 该最小上报时间间隔是为了防止在网络质量较差, 视频质量一直处于较差 的情况下, MGW过于频繁的上报事件。 2. The minimum reporting interval (nint) indicates the minimum interval for reporting twice. The minimum reporting interval is to prevent the MGW from being too frequent if the network quality is poor and the video quality is always poor. Report the incident.
以上事件参数决定了 MGW对视频数据更新请求检测的标准和上报频率。 例如, 视频更新请求检测事件参数为: fmt=FIR, nint=5s; MGW检测到 FIR 类型的 RTCP报文时, 上报视频更新事件, 请求更新视频数据; 若在一段时间 内, 由于某种原因 SIP终端不断发送 RTCP扩展 4艮文, 则当 MGW完成一次事 件检测和上报后, 需要等待 5s再上报新的视频更新请求检测事件。 The above event parameters determine the standard and reporting frequency of the MGW for detecting the video data update request. For example, the video update request detection event parameter is: fmt=FIR, nint=5s; when the MGW detects the FIR type RTCP message, the MGW reports a video update event and requests to update the video data; if for some time, for some reason SIP The terminal continuously sends the RTCP extension 4 message, and after the MGW completes an event detection and report, it needs to wait for 5 seconds to report a new video update request detection event.
MGW基于对 RTCP视频质量反馈报文的检测向 MGCF上报视频更新请求 检测事件中, 携带如下视频数据更新参数以指示需要更新的视频数据, 这些视 频数据更新参数可以包括以下一个或多个参数: The MGW reports the video update request to the MGCF based on the detection of the RTCP video quality feedback message. The detection event carries the following video data update parameters to indicate the video data to be updated. The video data update parameters may include one or more of the following parameters:
firstMB, 该参数指明第 1个需要被更新的宏块标识,宏块标识由具体的编 码算法决定; firstMB, this parameter indicates the first macroblock identifier to be updated, and the macroblock identifier is determined by a specific encoding algorithm;
numberOfMBs, 该参数指明需要更新的宏块数量; numberOfMBs, this parameter indicates the number of macroblocks that need to be updated;
fastUpdatePicture, 该参数指明需要更新整个视频图像, 即需要视频源发送 I帧数据。 fastUpdatePicture, this parameter indicates that the entire video image needs to be updated, that is, the video source is required to send I frame data.
MGW上报的视频数据更新参数与检测到的报文类型相关, 不同的 RTCP 报文携带不同的视频数据更新参数, MGW上报的视频数据更新参数由 RTCP 报文所携带的参数决定。 MGW检测到的报文类型、 上报的视频数据更新参数 以及 MGCF发送的 H.245信令对应关系如下: The video data update parameter reported by the MGW is related to the detected packet type, and different RTCP The packet carries different video data update parameters, and the video data update parameter reported by the MGW is determined by parameters carried in the RTCP packet. The correspondence between the MGW detected packet type, the reported video data update parameter, and the H.245 signaling sent by the MGCF is as follows:
当 MGW 检测到 FIR 与 PLI 报文时, 上报的视频数据更新参数为 fastUpdatePicture=l , MGCF发送的 H.245信令为 videoFastUpdatePicture; When the MGW detects the FIR and the PLI packet, the reported video data update parameter is fastUpdatePicture=l, and the H.245 signaling sent by the MGCF is videoFastUpdatePicture;
当 MGW检测到 SLI报文时, 上报的视频数据更新参数为 firstMB 与 numberOfMBs, 该 firstMB与 numberOfMBs的参数值分别等于 SLI报文中的 first与 number参数值, MGCF发送的 H.245信令为 videoFastupdateMB。 When the MGW detects the SLI packet, the reported video data update parameters are firstMB and numberOfMBs, the firstMB and numberOfMBs parameter values are equal to the first and number parameter values in the SLI packet, and the H.245 signaling sent by the MGCF is videoFastupdateMB. .
MGW上报的以上视频数据更新参数可以实现对局部 MB和整个 I帧数据 的更新请求。 例如, 上报的参数为 firstMB=3, numberOfMBs=5, 表示需要更 新第 3-7共 5个 MB的视频数据; 上报的参数为 fastUpdatePicture=l , 表示需 要更新整帧数据, 需要视频源发送新的 I帧数据。 The above video data update parameter reported by the MGW can implement an update request for the local MB and the entire I frame data. For example, the reported parameters are firstMB=3, numberOfMBs=5, indicating that it is necessary to update the 5-7 total 5 MB video data; the reported parameter is fastUpdatePicture=l, indicating that the entire frame data needs to be updated, and the video source is required to send a new one. I frame data.
在图 2所示的网络结构下, MGW检测 SIP终端发送的带内视频数据更新 请求, 将检测的带内更新请求通过事件上报给 MGCF; MGCF根据接收到的事 件发送 H.245带外视频数据更新请求给 H.324终端的信令流程如图 7B所示。 In the network structure shown in Figure 2, the MGW detects the in-band video data update request sent by the SIP terminal, and reports the detected in-band update request to the MGCF through the event; the MGCF sends the H.245 out-of-band video data according to the received event. The signaling flow for updating the request to the H.324 terminal is as shown in FIG. 7B.
参见图 7B, 为本发明实施例二的 MGW检测带内请求报文的信令流程示 意图, 具体包括: FIG. 7B is a schematic diagram of a signaling flow of an MGW detecting an in-band request packet according to Embodiment 2 of the present invention, which specifically includes:
步骤 711、 建立多媒体会话。 Step 711: Establish a multimedia session.
步骤 712、 MGCF向会话中指定端点发送请求消息给 MGW, 消息中指示 视频更新请求检测事件, 以及相应的事件参数。 Step 712: The MGCF sends a request message to the MGW to the specified endpoint in the session, where the message indicates a video update request detection event, and a corresponding event parameter.
本实施例中, MGCF下发检测的是 SIP终端所对应的端点 X ( Termination X ), MGCF向 MGW下发的视频更新请求检测事件指示 MGW检测 RTCP/AVPF 请求, 并只检测负荷类型( Payload Type )为 PSFB的 RTCP报文。 例如, 视频 更新请求检测事件参数为 fmt=FIR, 指示 MGW只检测 FIR扩展报文。 对于其 他类型的报文, 可参考 RFC 4585与 raft-ietf-avt-avpf-ccm-05。 In this embodiment, the MGCF sends the endpoint X ( Termination X ) corresponding to the SIP terminal, and the video update request detection event sent by the MGCF to the MGW instructs the MGW to detect the RTCP/AVPF request, and only detects the payload type (Payload Type). ) is the RTCP message of PSFB. For example, the video update request detection event parameter is fmt=FIR, indicating that the MGW only detects the FIR extension message. For other types of messages, refer to RFC 4585 and raft-ietf-avt-avpf-ccm-05.
步骤 713、 MGW收到请求消息后进行响应。 步骤 714 ~ 715、 MGW根据 MGCF指示的视频更新请求检测事件, 开始 对指定的 RTCP扩展报文进行检测。 当检测到 RTCP的 FIR报文时, 且与上次 发送通知消息的时间间隔大于 MGCF指定的时间间隔, 则通过发送通知消息 上报视频更新请求检测事件, 通知消息中携带参数 fastUpdatePicture=l , 指示 更新整帧视频。 Step 713: The MGW responds after receiving the request message. Steps 714 to 715: The MGW detects the event according to the video update request indicated by the MGCF, and starts to detect the specified RTCP extended packet. When the RTCP FIR message is detected, and the time interval from the last time the notification message is sent is greater than the time interval specified by the MGCF, the notification message is reported by sending a notification message, and the notification message carries the parameter fastUpdatePicture=l, indicating that the update is performed. Full frame video.
根据检测到的 RTCP报文类型的不同, 通知消息中携带的视频数据更新参 数参数也有所不同。 本实施例中, 需要检测的是 RTCP的 FIR报文, 相应地, 上报的视频数据更新参数参数为 fastUpdatePicture参数。 The video data update parameter parameters carried in the notification message are also different according to the type of the detected RTCP packet. In this embodiment, the FIR message of the RTCP needs to be detected, and correspondingly, the updated video data update parameter parameter is a fastUpdatePicture parameter.
步骤 716、 MGCF收到通知消息后进行响应。 Step 716: The MGCF responds after receiving the notification message.
步骤 717 、 MGCF 根据 MGW 上报的视频数据更新参数 ( fastUpdatePicture=l ) , 向 H.324 终 端 发 送 H.245 信 令 ( 如 videoFastUpdatePicture信令), 请求 H.324终端更新视频整帧数据。 Step 717: The MGCF sends an H.245 signaling (such as videoFastUpdatePicture signaling) to the H.324 terminal according to the video data update parameter reported by the MGW (fastUpdatePicture=l), and requests the H.324 terminal to update the video full frame data.
实施例三 Embodiment 3
本实施例描述了 MGCF指示 MGW发送带内视频数据更新请求, 从而实 现视频数据更新的过程。 当 MGCF接收到相关的带外请求报文时, 若被请求 的视频源支持带内请求方式, 则 MGCF通过 MGW向视频源发送带内请求。 This embodiment describes a process in which the MGCF instructs the MGW to transmit an in-band video data update request, thereby implementing video data update. When the MGCF receives the relevant out-of-band request message, if the requested video source supports the in-band request mode, the MGCF sends an in-band request to the video source through the MGW.
本实施例的具体应用场景可以如图 8所示。在图 8中,由 MGW检测 H.324 终端发起的带外视频数据更新请求, 当 MGW检测到来自 H.324终端的带外视 频数据更新请求时通知 MGCF, MGCF通过带内 RTP/RTCP方式请求 SIP终端 发送视频更新数据。 A specific application scenario of this embodiment may be as shown in FIG. 8. In FIG. 8, the MGW detects the out-of-band video data update request initiated by the H.324 terminal, and notifies the MGCF when the MGW detects the out-of-band video data update request from the H.324 terminal, and the MGCF requests through the in-band RTP/RTCP method. The SIP terminal sends video update data.
参见图 9A, 为本发明实施例三的 MGCF指示 MGW发送视频数据更新请 求的流程示意图, 具体包括: Referring to FIG. 9A, a schematic flowchart of an MGCF instructing an MGW to send a video data update request according to Embodiment 3 of the present invention includes:
步骤 901、 建立多媒体会话。 Step 901: Establish a multimedia session.
步骤 902、 MGCF收到带外方式的视频数据更新请求消息, 其中携带视频 数据更新参数。 Step 902: The MGCF receives an out-of-band video data update request message, where the video data update parameter is carried.
步骤 903、若视频源支持带内请求方式, 则 MGCF指示 MGW发送带内请 求信令给视频源端, 其中携带的视频数据更新参数与 MGCF接收到的视频数 据更新请求消息中的视频数据更新参数保持一致。 Step 903: If the video source supports the in-band request mode, the MGCF instructs the MGW to send the inband. The signaling is sent to the video source end, and the carried video data update parameter is consistent with the video data update parameter in the video data update request message received by the MGCF.
步骤 904、 MGW根据 MGCF的指示,向视频源端发送带内更新请求消息, 消息中携带的视频数据更新参数所指示的需要更新的视频数据, 与 MGCF通 过视频数据更新参数所指示的需要更新的视频数据保持一致。 Step 904: The MGW sends an in-band update request message to the video source end according to the indication of the MGCF, and the video data that needs to be updated indicated by the video data update parameter carried in the message is updated with the MGCF indicated by the video data update parameter. The video data is consistent.
步骤 905、 MGW完成请求消息发送后, 向 MGCF进行响应。 Step 905: After the MGW completes the request message transmission, it responds to the MGCF.
图 9A所示的流程中, MGCF直接将 H.245消息参数传递给 MGW, MGW 负责 H.245消息参数与 RTCP扩展报文参数之间的转换, 并根据 H.245消息发 送匹配的 RTCP 4艮文。 In the flow shown in FIG. 9A, the MGCF directly passes the H.245 message parameter to the MGW, and the MGW is responsible for converting between the H.245 message parameter and the RTCP extended message parameter, and sends the matched RTCP according to the H.245 message. Text.
本实施例通过扩展 H.248协议实现上述流程。 本实施例在 H.248协议中增 加指示 MGW发送 RTCP视频质量扩展报文的信号, MGCF可以通过该信号指 示 MGW发送负荷类型( Payload Type )为 PSFB的 RTCP扩展报文。 该信号参 数与 H.245信令参数一致, 包括以下一个或多个参数: This embodiment implements the above process by extending the H.248 protocol. In this embodiment, a signal indicating that the MGW sends an RTCP video quality extension message is added in the H.248 protocol, and the MGCF can indicate that the MGW sends an RTCP extension packet whose payload type (payload Type) is PSFB. The signal parameter is consistent with the H.245 signaling parameter and includes one or more of the following parameters:
1、 firstGOB, 指明在视频图像中第 1个需要被更新的 GOB标识, GOB标 识由具体的编码算法决定; 1. firstGOB, indicating the first GOB identifier to be updated in the video image, and the GOB identification is determined by a specific encoding algorithm;
2、 numberOfGOBs, 指明需要更新的 GOB数量; 2, numberOfGOBs, indicating the number of GOBs that need to be updated;
3、 firstMB, 指明第 1个需要被更新的宏块标识, 宏块标识由具体的编码 算法决定; 3. firstMB, indicating the first macroblock identifier to be updated, and the macroblock identifier is determined by a specific encoding algorithm;
4、 numberOfMBs, 指明需要更新的宏块数量; 4, numberOfMBs, indicating the number of macroblocks that need to be updated;
5、 fastUpdatePicture, 指明需要更新整个视频图像, 即需要视频源发送 I 帧数据。 5, fastUpdatePicture, indicating that the entire video image needs to be updated, that is, the video source is required to send I frame data.
MGW根据接收到的信号参数发送相应的 RTCP报文, 其对应关系为: 如果信号参数中携带 fastUpdatePicture参数, 贝' J MGW发送 FIR报文, 否 则发送 SLI报文; 其中 The MGW sends a corresponding RTCP packet according to the received signal parameter, and the corresponding relationship is: if the signal parameter carries the fastUpdatePicture parameter, the B' J MGW sends the FIR message, otherwise, the SLI message is sent;
SLI报文中的 first参数值计算为: first=firstMB+firstGOB*每个 GOB中宏 块的数量; SLI报文中的 number参数值计算为: number=numberOfGOBs*每个 GOB 中宏块的数量 +numberOfMBs; The value of the first parameter in the SLI message is calculated as: first=firstMB+firstGOB* the number of macroblocks in each GOB; The value of the number parameter in the SLI packet is calculated as: number=numberOfGOBs* the number of macroblocks in each GOB + numberOfMBs;
通过以上信号参数的转换,可以实现带外 H.245视频更新信令到带内 RTCP 扩展 文的转换。 以上转换也可以在 MGCF上进行。 Through the conversion of the above signal parameters, the conversion of out-of-band H.245 video update signaling to in-band RTCP extension can be realized. The above conversion can also be performed on the MGCF.
在图 2所示的网络结构下, MGCF接收 H.324终端的 H.245视频数据更新 请求, 并指示 MGW发送相应的带内 RTCP扩展信令至 SIP终端的信令流程可 如图 9B所示。 In the network structure shown in FIG. 2, the MGCF receives the H.245 video data update request of the H.324 terminal, and instructs the MGW to send the corresponding inband RTCP extension signaling to the SIP terminal. The signaling flow may be as shown in FIG. 9B. .
参见图 9B, 为本发明实施例三的 MGCF指示 MGW发送视频数据更新请 求的信令流程示意图, 具体包括: FIG. 9B is a schematic diagram of a signaling flow of an MGCF instructing an MGW to send a video data update request according to Embodiment 3 of the present invention, which specifically includes:
步骤 911、 首先建立多媒体会话。 Step 911, first establish a multimedia session.
步骤 912 ~ 913、 MGCF接收到带外 H.245视频数据更新请求后, 向 MGW 发送请求消息, 其中携带指示 MGW发送 RTCP视频质量扩展报文的信号。 Steps 912-913: After receiving the out-of-band H.245 video data update request, the MGCF sends a request message to the MGW, where the MGCF sends a signal indicating that the MGW sends the RTCP video quality extension message.
本实施例中, MGCF检测到的是 H.324终端所对应的端点 X ( Termination X ), 请求消息中的信号参数为 fastUpdatePicture= 1 , 指示 MGW向被请求的视 频源发送更新整帧视频的视频数据更新请求。 In this embodiment, the MGCF detects the endpoint X ( Termination X ) corresponding to the H.324 terminal, and the signal parameter in the request message is fastUpdatePicture= 1 , instructing the MGW to send a video for updating the entire frame video to the requested video source. Data update request.
步骤 914、 MGW接收到 MGCF视频数据更新请求指示后, 将信号参数转 换为 RTCP报文参数, 并发送相应的 RTCP扩展报文。 Step 914: After receiving the MGCF video data update request indication, the MGW converts the signal parameter into an RTCP packet parameter, and sends a corresponding RTCP extended packet.
本实施例中, MGW接收到的信号参数是 fastUpdatePicture=l ,依照前述的 信号参数与 RTCP扩展报文的对应关系, MGW发送 RTCP的 FIR报文。 In this embodiment, the signal parameter received by the MGW is fastUpdatePicture=l, and the MGW sends the RTCP FIR message according to the correspondence between the foregoing signal parameter and the RTCP extended message.
步骤 915、 MGW发送响应消息至 MGCF。 Step 915: The MGW sends a response message to the MGCF.
本发明实施例还分别针对上述实施例一至三的流程, 提供了媒体控制器、 媒体处理器和视频数据更新系统的结构。 The embodiments of the present invention further provide the structures of the media controller, the media processor, and the video data update system for the processes of the foregoing first to third embodiments.
针对实施例一的流程, 本发明实施例提供了一种媒体控制器的结构, 如图 10所示, 该媒体控制器包括: 指示发送模块和更新请求接收模块, 其中: 指示发送模块, 用于向媒体处理器发送指示; For the flow of the first embodiment, the embodiment of the present invention provides a structure of a media controller. As shown in FIG. 10, the media controller includes: an indication sending module and an update request receiving module, where: an indication sending module is used for Send an indication to the media processor;
更新请求接收模块, 用于接收媒体处理器发送的请求, 该请求用于请求需 要更新的视频数据; 该请求为媒体处理器根据指示发送模块发送的指示检测视 频源的视频质量, 并根据检测结果发送的请求。 An update request receiving module, configured to receive a request sent by a media processor, where the request is used for a request The video data to be updated; the request is a request that the media processor detects the video quality of the video source according to the indication sent by the sending module, and sends the request according to the detection result.
上述媒体控制器还可包括更新请求发送模块, 用于根据更新请求接收模块 接收到的请求, 发起带外视频数据更新请求, 向视频源请求更新该需要更新的 视频数据。 The media controller may further include an update request sending module, configured to initiate an out-of-band video data update request according to the request received by the update request receiving module, and request the video source to update the video data that needs to be updated.
上述媒体控制器的指示发送模块发送的指示具体可以发送视频质量检测 事件, 该视频质量检测事件中携带视频质量检测参数。 更新请求接收模块接收 媒体处理器上报的视频质量检测事件, 向媒体控制器请求需要更新的视频数 据; 该上报的视频质量检测事件是媒体处理器根据媒体控制器发送的视频质量 检测事件中的视频质量检测参数, 检测视频源的视频质量, 并根据检测结果上 报的视频质量检测事件, 其中携带视频数据更新参数, 该视频数据更新参数指 示出需要更新的视频数据。 The indication sent by the indication sending module of the media controller may specifically send a video quality detection event, where the video quality detection event carries a video quality detection parameter. The update request receiving module receives the video quality detection event reported by the media processor, and requests the media controller to update the video data; the reported video quality detection event is a video in the video quality detection event sent by the media processor according to the media controller. The quality detection parameter detects the video quality of the video source, and detects an event according to the video quality reported by the detection result, where the video data update parameter is carried, and the video data update parameter indicates the video data that needs to be updated.
针对实施例一的流程, 本发明实施例提供了一种媒体处理器的结构, 如图 For the process of the first embodiment, the embodiment of the present invention provides a structure of a media processor, as shown in the figure.
11所示,该媒体处理器包括: 指示接收模块、检测模块和更新请求模块,其中: 指示接收模块, 用于接收媒体控制器发送的指示; The media processor includes: an indication receiving module, a detecting module, and an update requesting module, wherein: the receiving module is configured to receive an indication sent by the media controller;
检测模块, 用于根据接收到的指示检测视频源的视频质量; a detecting module, configured to detect a video quality of the video source according to the received indication;
更新请求模块, 用于根据检测模块的视频质量检测结果, 向媒体控制器发 送请求需要更新视频数据的请求。 And an update requesting module, configured to send, to the media controller, a request for updating the video data according to the video quality detection result of the detecting module.
上述媒体处理器中的指示接收模块接收到的指示为媒体控制器发送的视 频质量检测事件, 视频质量检测事件中携带视频质量检测参数; 检测模块根据 该视频质量检测参数检测视频源的视频质量; 更新请求模块根据检测模块的视 频质量检测结果, 向媒体控制器上报视频质量检测事件, 携带视频数据更新参 数, 视频数据更新参数指示出需要更新的视频数据。 The video quality detection event is sent by the media module, and the video quality detection event carries the video quality detection parameter. The detection module detects the video quality of the video source according to the video quality detection parameter. The update request module reports the video quality detection event to the media controller according to the video quality detection result of the detection module, and carries the video data update parameter, where the video data update parameter indicates the video data that needs to be updated.
其中, 视频质量检测参数包括视频宏块误码率; 检测模块根据视频质量检 测参数检测视频源的宏块误码率; 更新请求模块在检测模块检测到视频源的宏 块误码率高于视频质量检测参数中的视频宏块误码率时, 向媒体控制器上报视 频质量检测事件。 视频宏块误码率可包括最大宏块误码率和最小宏块误码率, 如果检测模块检测到视频源的视频宏块误码率在该最大宏块误码率和最小宏 块误码率之间, 则更新请求模块发送更新局部视频数据的请求; 如果检测模块 检测到的视频源的视频宏块误码率大于最大宏块误码率, 则更新请求模块发送 更新整帧视频数据的请求。 The video quality detection parameter includes a video macroblock error rate; the detection module detects a macroblock error rate of the video source according to the video quality detection parameter; and the update request module detects that the macroblock error rate of the video source is higher than the video. Reporting to the media controller when the video macroblock error rate in the quality detection parameter Frequency quality detection event. The video macroblock error rate may include a maximum macroblock error rate and a minimum macroblock error rate, if the detection module detects a video macroblock error rate of the video source at the maximum macroblock error rate and a minimum macroblock error rate. Between the rates, the update requesting module sends a request for updating the local video data; if the video macroblock error rate of the video source detected by the detecting module is greater than the maximum macroblock error rate, the update requesting module sends the updated full frame video data. request.
视频质量检测参数还包括时间间隔参数; 更新请求模块按照该时间间隔参 数指示的时间间隔上报视频质量检测事件。 The video quality detection parameter further includes a time interval parameter; the update request module reports the video quality detection event according to the time interval indicated by the time interval parameter.
针对实施例一的流程, 本发明实施例提供了一种视频数据更新系统, 该系 统包括上述可应用于实施例一所述流程的媒体控制器和媒体处理器, 其中: 媒体处理器, 用于根据媒体控制器发送的指示检测视频源的视频质量, 并 根据视频质量检测结果向媒体控制器请求需要更新的视频数据; For the flow of the first embodiment, the embodiment of the present invention provides a video data update system, where the system includes the media controller and the media processor, which are applicable to the process described in the first embodiment, where: the media processor is used for Detecting a video quality of the video source according to the indication sent by the media controller, and requesting, from the media controller, the video data that needs to be updated according to the video quality detection result;
媒体控制器, 用于指示媒体处理器检测视频源的视频质量。 a media controller, configured to instruct the media processor to detect a video quality of the video source.
其中,媒体控制器还用于在接收到媒体处理器发送的请求需要更新的视频 数据的请求后, 向视频源发起视频数据更新请求, 请求更新所述需要更新的视 频数据。 The media controller is further configured to: after receiving the request sent by the media processor to request the updated video data, initiate a video data update request to the video source, requesting to update the video data that needs to be updated.
针对实施例二的流程, 本发明实施例提供了一种媒体控制器的结构, 如图 For the process of the second embodiment, the embodiment of the present invention provides a structure of a media controller, as shown in the figure.
10所示, 该媒体控制器包括: 指示发送模块和更新请求接收模块, 其中: 指示发送模块, 用于向媒体处理器发送指示; As shown in FIG. 10, the media controller includes: an indication sending module and an update request receiving module, where: an indication sending module, configured to send an indication to the media processor;
更新请求接收模块, 用于接收媒体处理器发送的请求, 请求需要更新的视 频数据; 该请求是媒体处理器根据接收到的指示检测指定视频源发起的带内视 频更新请求, 并在检测到带内视频更新请求时发送的请求。 An update request receiving module, configured to receive a request sent by the media processor, requesting video data that needs to be updated; the request is that the media processor detects an in-band video update request initiated by the specified video source according to the received indication, and detects the band A request sent when a video update request is made.
上述媒体控制器还可包括: 更新请求发送模块, 用于根据更新请求接收模 块接收到的请求, 向视频源发送带外视频数据更新请求, 以向视频源请求更新 需要更新的视频数据。 The media controller may further include: an update request sending module, configured to send an out-of-band video data update request to the video source according to the update request to receive the request received by the module, to request the video source to update the video data that needs to be updated.
上述媒体控制器中的指示发送模块发送的指示为视频更新请求检测事件, 该视频更新请求检测事件用于指示媒体处理器检测视频源发起的带内视频数 据更新请求。 该视频更新请求检测事件中携带事件参数, 事件参数包括实时传 输控制协议报文类型参数。 更新请求接收模块接收媒体处理器上报的视频更新 请求检测事件, 向媒体控制器请求需要更新的视频数据; 该上报的视频更新请 求检测事件是媒体处理器根据媒体控制器发送的视频更新请求检测事件,检测 视频源发起的带内视频数据更新请求, 并在检测到带内视频数据更新请求时上 报的视频更新请求检测事件,该视频更新请求检测事件中携带视频数据更新参 数, 该视频数据更新参数指示出需要更新的视频数据。 The indication sent by the indication sending module in the media controller is a video update request detection event, where the video update request detection event is used to instruct the media processor to detect the number of in-band videos initiated by the video source. According to the update request. The video update request detection event carries an event parameter, and the event parameter includes a real-time transmission control protocol message type parameter. The update request receiving module receives the video update request detection event reported by the media processor, and requests the media controller to update the video data; the reported video update request detection event is that the media processor detects the event according to the video update request sent by the media controller. And detecting, by the video source, an in-band video data update request, and detecting a video update request detection event that is reported when the in-band video data update request is detected, where the video update request detection event carries a video data update parameter, and the video data update parameter Indicates the video data that needs to be updated.
其中,视频数据更新参数为更新局部视频数据的参数或更新整帧视频数据 的参数, 视频更新请求检测事件中携带何种视频数据更新参数, 是由媒体处理 器根据报文类型参数确定的。 The video data update parameter is a parameter for updating the local video data or a parameter for updating the entire frame video data, and what video data update parameter is carried in the video update request detection event is determined by the media processor according to the message type parameter.
针对实施例二的流程, 本发明实施例提供了一种媒体处理器的结构, 如图 For the process of the second embodiment, the embodiment of the present invention provides a structure of a media processor, as shown in the figure.
11所示,该媒体处理器包括: 指示接收模块、检测模块和更新请求模块,其中: 指示接收模块, 用于接收媒体控制器发送的指示; The media processor includes: an indication receiving module, a detecting module, and an update requesting module, wherein: the receiving module is configured to receive an indication sent by the media controller;
检测模块, 用于根据指示接收模块接收到的指示, 检测该指示指定的视频 源发起的带内视频数据更新请求; The detecting module is configured to detect, according to the indication received by the receiving module, the in-band video data update request initiated by the specified video source;
更新请求模块, 用于在检测模块检测到带内视频数据更新请求后, 向媒体 控制器请求需要更新的视频数据。 The update request module is configured to request the media controller to update the video data after the detecting module detects the in-band video data update request.
上述媒体处理器中的指示接收模块接收到的指示为媒体控制器发送的视 频更新请求检测事件, 该视频更新请求检测事件中携带事件参数, 该事件参数 包括实时传输控制协议报文类型参数, 该实时传输控制协议报文类型参数所指 示的报文类型为图像片更新请求报文, 或者为关键帧更新请求报文或整帧更新 请求报文, 该事件参数还包括间间隔参数。 检测模块根据实时传输控制协议报 文类型参数检测带内视频数据更新请求。 更新请求模块在检测模块检测到带内 视频数据更新请求后, 向媒体控制器上报视频更新请求检测事件, 携带视频数 据更新参数, 该视频数据更新参数指示出需要更新的视频数据, 如果事件参数 还包括间间隔参数则更新请求模块按照时间间隔参数指示的时间间隔上报视 频更新请求检测事件。 The instruction received by the indication receiving module in the media processor is a video update request detection event sent by the media controller, where the video update request detection event carries an event parameter, where the event parameter includes a real-time transmission control protocol message type parameter, The message type indicated by the real-time transmission control protocol packet type parameter is an image slice update request message, or a key frame update request message or an entire frame update request message, and the event parameter further includes an interval parameter. The detecting module detects the in-band video data update request according to the real-time transmission control protocol message type parameter. After the detecting module detects the in-band video data update request, the update request module reports a video update request detection event to the media controller, and carries a video data update parameter, where the video data update parameter indicates the video data that needs to be updated, if the event parameter is further Including the interval parameter, the update request module reports on the time interval indicated by the time interval parameter. The frequency update requests a detection event.
上述媒体处理器中的更新请求模块包括第一更新请求子模块和第二更新 请求子模块, 其中: The update request module in the media processor includes a first update request submodule and a second update request submodule, where:
第一更新请求子模块, 用于根据图像片更新请求报文, 请求更新局部视频 数据; 第二更新请求子模块, 用于根据关键帧更新请求报文或整帧更新请求报 文, 请求更新整帧视频数据。 a first update request sub-module, configured to update the local video data according to the image slice update request message; and a second update request sub-module, configured to update the request message according to the key frame update request message or the entire frame update request message, requesting to update the whole Frame video data.
针对实施例二的流程, 本发明实施例提供了一种视频数据更新系统, 该系 统包括上述可应用于实施例二所述流程的媒体控制器和媒体处理器, 其中: 媒体处理器, 用于根据媒体控制器发送的指示, 检测视频源的带内视频更 新请求, 并在检测到带内视频更新请求后, 向所述媒体控制器请求需要更新的 视频数据; For the flow of the second embodiment, the embodiment of the present invention provides a video data update system, where the system includes the media controller and the media processor applicable to the process described in the second embodiment, where: the media processor is used for Detecting an in-band video update request of the video source according to the indication sent by the media controller, and requesting the media controller to update the video data after detecting the in-band video update request;
媒体控制器, 用于指示媒体处理器检测视频源发起的带内视频更新请求, 以及接收媒体处理器发送的所述请求需要更新的视频数据的请求。 And a media controller, configured to instruct the media processor to detect an in-band video update request initiated by the video source, and receive a request by the media processor to request the updated video data.
其中,媒体控制器还用于在接收到媒体处理器发送的所述请求需要更新的 视频数据的请求后, 向所述带内视频更新请求中的被请求的视频源发起视频数 据更新请求, 请求更新需要更新的视频数据。 The media controller is further configured to: after receiving the request for the video data that needs to be updated by the media processor, initiate a video data update request to the requested video source in the in-band video update request, requesting Update the video data that needs to be updated.
针对实施例三的流程, 本发明实施例提供了一种媒体控制器, 该媒体控制 器包括更新请求接收模块和指示发送模块, 其中: For the process of the third embodiment, the embodiment of the present invention provides a media controller, where the media controller includes an update request receiving module and an indication sending module, where:
更新请求接收模块, 用于接收视频源发起的带外视频数据更新请求; 指示发送模块, 用于向媒体处理器发送指示, 指示媒体处理器发起带内视 频数据更新请求, 该带内视频数据更新请求为更新接收到的带外视频数据更新 请求所指示的视频数据的请求。 An update request receiving module, configured to receive an out-of-band video data update request initiated by a video source, and an indication sending module, configured to send an indication to the media processor, to instruct the media processor to initiate an in-band video data update request, and the in-band video data update A request to update the video data indicated by the received out-of-band video data update request.
上述媒体控制器中的指示发送模块在更新请求接收模块接收到带外视频 数据更新请求后, 向媒体处理器发送该指示。 The indication sending module in the media controller sends the indication to the media processor after the update request receiving module receives the out-of-band video data update request.
针对实施例三的流程, 本发明实施例提供了一种媒体处理器, 该媒体处理 器包括指示接收模块和更新请求模块, 其中: 指示接收模块, 用于接收媒体控制器发送的指示; For the process of the third embodiment, the embodiment of the present invention provides a media processor, where the media processor includes an indication receiving module and an update request module, where: An indication receiving module, configured to receive an indication sent by the media controller;
更新请求模块, 用于根据该指示, 向视频源发起带内视频数据更新请求, 请求更新该指示所指示的视频数据。 And an update requesting module, configured to initiate an in-band video data update request to the video source according to the indication, requesting to update the video data indicated by the indication.
上述媒体处理器中的指示接收模块接收到的指示中携带视频数据更新参 数, 该视频数据更新参数为更新整帧视频数据的参数或更新局部视频数据的参 数; 上述媒体处理器中的更新请求模块包括第一更新请求子模块和第二更新请 求子模块, 其中: The instruction received by the indication receiving module in the media processor carries a video data update parameter, where the video data update parameter is a parameter for updating the entire frame of video data or a parameter for updating the local video data; and the update request module in the media processor. The first update request submodule and the second update request submodule are included, where:
第一更新请求子模块用于根据更新整帧视频数据的参数,发送更新整帧视 频数据的带内视频数据更新请求; 第二更新请求子模块用于根据更新局部视频 数据的参数,将视频数据更新参数转换为相应带外视频数据更新请求消息的参 数, 并发送更新局部视频数据的带内视频数据更新请求。 The first update request sub-module is configured to send an in-band video data update request for updating the entire frame of video data according to the parameter for updating the entire frame of video data; and the second update request sub-module is configured to: convert the video data according to the parameter of the updated partial video data. The update parameters are converted to parameters of the corresponding out-of-band video data update request message, and an in-band video data update request for updating the partial video data is sent.
针对实施例三的流程, 本发明实施例提供了一种视频数据更新系统, 该系 统包括上述可应用于实施例三所述流程的媒体控制器和媒体处理器, 其中: 媒体处理器, 用于根据媒体控制器的指示, 向被请求的视频源发起带内视 频更新请求, 请求更新视频数据; 所述媒体控制器, 用于在接收到视频源的带外视频更新请求后, 向媒体处 理器发送所述指示, 指示媒体处理器向带外视频更新请求中被请求的视频源发 起带内视频数据更新请求。 For the flow of the third embodiment, the embodiment of the present invention provides a video data update system, where the system includes the media controller and the media processor that can be applied to the process described in the third embodiment, where: the media processor is used for And initiating an in-band video update request to the requested video source, requesting to update the video data according to the indication of the media controller; and the media controller is configured to: after receiving the out-of-band video update request of the video source, to the media processor The indication is sent to instruct the media processor to initiate an in-band video data update request to the requested video source in the out-of-band video update request.
通过以上的实施方式的描述, 所属领域的技术人员可以清楚地了解到本发 明可借助软件加必需的通用硬件平台的方式来实现, 当然也可以通过硬件, 但 很多情况下前者是更佳的实施方式。 基于这样的理解, 本发明的技术方案本质 机软件产品存储在可读取的存储介质中, 如计算机的软盘, 硬盘或光盘等, 包 括若干指令用以使得一台计算机设备(可以是个人计算机, 服务器, 或者网络 设备等)执行本发明各个实施例所述的方法。 Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is a better implementation. the way. Based on such understanding, the essential software product of the present invention is stored in a readable storage medium, such as a floppy disk, a hard disk or an optical disk of a computer, and includes a plurality of instructions for causing a computer device (which may be a personal computer, The server, or network device, etc.) performs the methods described in various embodiments of the present invention.
综上所述, 通过本发明的实施例, 当 IM-MGW 自身检测到 H.324终端或 SIP终端的视频质量问题时, IM-MGW可根据 MGCF的指示向 MGCF请求需 要更新的视频数据; 当 SIP终端通过带内 RTP/RTCP反馈视频数据更新请求, 而 IM-MGW无法提供所请求更新数据时, IM-MGW可根据 MGCF的指示,通 过 H.248协议将更新请求上报给 MGCF,以便 MGCF通过 H.245消息向 H.324 终端请求视频数据更新;当 MGCF收到来自 H.324终端的 H.245视频数据更新 请求, 而 SIP终端不支持 SIP+XML 的视频数据更新请求时, MGCF可指示 IM-MGW发送带内的视频数据更新请求。 综上, 本发明的实施例可在上述情 、 - In summary, by using the embodiment of the present invention, when the IM-MGW detects the video quality problem of the H.324 terminal or the SIP terminal, the IM-MGW may request the MGCF according to the indication of the MGCF. Video data to be updated; When the SIP terminal feeds back the video data update request through the in-band RTP/RTCP, and the IM-MGW cannot provide the requested update data, the IM-MGW may update the request through the H.248 protocol according to the indication of the MGCF. Reported to the MGCF, so that the MGCF requests video data update from the H.324 terminal through the H.245 message; when the MGCF receives the H.245 video data update request from the H.324 terminal, and the SIP terminal does not support the SIP+XML video data. When the request is updated, the MGCF may instruct the IM-MGW to send a video data update request in the band. In summary, embodiments of the present invention may be in the above, -
明的精神和范围。 这样, 倘若本发明的这些修改和变型属于本发明权利要求及 其等同技术的 范围之内, 则本发明也意图包含这些改动和变型在内。 The spirit and scope of the Ming. Thus, it is intended that the present invention cover the modifications and variations of the inventions
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP08800920A EP2192745A4 (en) | 2007-09-20 | 2008-09-19 | METHOD AND SYSTEM FOR UPDATING VIDEO DATA |
US12/727,921 US20100220195A1 (en) | 2007-09-20 | 2010-03-19 | Method and system for updating video data |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN200710153049.0A CN101394568B (en) | 2007-09-20 | 2007-09-20 | Video data updating method, apparatus and method thereof |
CN200710153049.0 | 2007-09-20 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/727,921 Continuation US20100220195A1 (en) | 2007-09-20 | 2010-03-19 | Method and system for updating video data |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2009039781A1 true WO2009039781A1 (en) | 2009-04-02 |
Family
ID=40494598
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2008/072426 WO2009039781A1 (en) | 2007-09-20 | 2008-09-19 | Video data update method and system |
Country Status (4)
Country | Link |
---|---|
US (1) | US20100220195A1 (en) |
EP (1) | EP2192745A4 (en) |
CN (1) | CN101394568B (en) |
WO (1) | WO2009039781A1 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101442421A (en) * | 2007-11-19 | 2009-05-27 | 华为技术有限公司 | Method, apparatus and system for establishing conference |
US9860296B2 (en) | 2012-03-23 | 2018-01-02 | Avaya Inc. | System and method for end-to-end call quality indication |
US9356917B2 (en) | 2012-03-23 | 2016-05-31 | Avaya Inc. | System and method for end-to-end encryption and security indication at an endpoint |
US9178778B2 (en) | 2012-03-23 | 2015-11-03 | Avaya Inc. | System and method for end-to-end RTCP |
US10085029B2 (en) | 2015-07-21 | 2018-09-25 | Qualcomm Incorporated | Switching display devices in video telephony |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10125884A1 (en) * | 2001-05-28 | 2002-12-05 | Siemens Ag | Conferencing unit for multipoint communication has a multi-point control unit for controlling conferencing and for signaling and a multipoint processor unit for real-time processing of multimedia real data streams. |
WO2007045526A1 (en) * | 2005-10-21 | 2007-04-26 | Siemens Aktiengesellschaft | Signalling for establishing an h.324 videotelefony connection between a media gateway and a controller |
CN101127697A (en) * | 2006-08-18 | 2008-02-20 | 华为技术有限公司 | Flow control method and system under separated media control and processing architecture |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7777777B2 (en) * | 2002-04-30 | 2010-08-17 | Tandberg Telecom As | System and method for active call monitoring |
US20040252761A1 (en) * | 2003-06-16 | 2004-12-16 | Dilithium Networks Pty Limited (An Australian Corporation) | Method and apparatus for handling video communication errors |
US20050013309A1 (en) * | 2003-07-14 | 2005-01-20 | Channasandra Ravishankar | System and method for high quality video conferencing with heterogeneous end-points and networks |
US20050060411A1 (en) * | 2003-09-16 | 2005-03-17 | Stephane Coulombe | System and method for adaptation of peer-to-peer multimedia sessions |
FI115589B (en) * | 2003-10-14 | 2005-05-31 | Nokia Corp | Encoding and decoding redundant images |
KR100596945B1 (en) * | 2003-10-30 | 2006-07-04 | (주)씨앤에스 테크놀로지 | IP Video Terminal with Image Transceiver Bandwidth and Image Quality Control Function and Its Control Method |
WO2006002597A1 (en) * | 2004-06-30 | 2006-01-12 | Huawei Technologies Co., Ltd. | A method for acquiring the qos of the multimedia stream periodically |
KR100734859B1 (en) * | 2005-12-08 | 2007-07-03 | 한국전자통신연구원 | Apparatus and method for controlling Internet set-top box using Megcoa Protocol |
US20080115185A1 (en) * | 2006-10-31 | 2008-05-15 | Microsoft Corporation | Dynamic modification of video properties |
-
2007
- 2007-09-20 CN CN200710153049.0A patent/CN101394568B/en active Active
-
2008
- 2008-09-19 EP EP08800920A patent/EP2192745A4/en not_active Withdrawn
- 2008-09-19 WO PCT/CN2008/072426 patent/WO2009039781A1/en active Application Filing
-
2010
- 2010-03-19 US US12/727,921 patent/US20100220195A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10125884A1 (en) * | 2001-05-28 | 2002-12-05 | Siemens Ag | Conferencing unit for multipoint communication has a multi-point control unit for controlling conferencing and for signaling and a multipoint processor unit for real-time processing of multimedia real data streams. |
WO2007045526A1 (en) * | 2005-10-21 | 2007-04-26 | Siemens Aktiengesellschaft | Signalling for establishing an h.324 videotelefony connection between a media gateway and a controller |
CN101127697A (en) * | 2006-08-18 | 2008-02-20 | 华为技术有限公司 | Flow control method and system under separated media control and processing architecture |
Non-Patent Citations (2)
Title |
---|
"Extended RTP profile for Real-time transport Control Protocol(RTCP)-Based Feedback(RTP/AVPF", IETF STANDARDS, INTERNET ENGINEERING TASK FORCE, IETF, 1 July 2006 (2006-07-01) |
See also references of EP2192745A4 |
Also Published As
Publication number | Publication date |
---|---|
CN101394568B (en) | 2011-06-15 |
CN101394568A (en) | 2009-03-25 |
US20100220195A1 (en) | 2010-09-02 |
EP2192745A1 (en) | 2010-06-02 |
EP2192745A4 (en) | 2011-05-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7835346B2 (en) | Methods, systems, and computer program products for providing transcoder free operation (TrFO) and interworking between unlicensed mobile access (UMA) and universal mobile telecommunications system (UMTS) call legs using a media gateway | |
EP3335341B1 (en) | Sender side video telephony downgrade method | |
US8180915B2 (en) | Coding data streams | |
US20070297424A1 (en) | Method for IP-based service transport | |
CN100574467C (en) | A kind of band width control method and terminal equipment | |
CN101895569B (en) | Implementation method of video browsing, IMS video monitoring system and front monitoring front-end | |
WO2010037293A1 (en) | Method and system for improving quality of multimedia calls, media controlling equipment and media processing equipment | |
CN1668109A (en) | Adaptive video transcoding gateway with multiple conversion functions | |
CN1910926A (en) | Method and apparatus for handling video communication errors | |
CN103118238B (en) | The control method of video conference and video conferencing system | |
WO2009039781A1 (en) | Video data update method and system | |
US8179927B2 (en) | Method, system and gateway for negotiating the capability of data signal detector | |
WO2012003808A1 (en) | Method, device, and system for video refresh | |
WO2009121284A1 (en) | A method, system and gateway for supplying intelligent service | |
CN1988578A (en) | System and method for providing multimedia communication and household gateway | |
EP2264959B1 (en) | Media gateway, media gateway controller, method and system for communication of multimedia phones | |
WO2019228534A1 (en) | Media transmission method and h323-sip gateway | |
CN110324667B (en) | Novel video stream playing method and system | |
WO2009155773A1 (en) | A method, system, media gateway controller and media gateway for transmitting multi-media service in the next generation network | |
WO2006136070A1 (en) | A method for switching coding decoding format of media streams | |
CN102761526B (en) | VC 1 encodes the method that video and audio is transmitted in the terminal unit supporting H.323 protocol suite | |
WO2008071092A1 (en) | An ims terminal, a superposing server and an ims service processing method | |
CN119071589A (en) | Audio and video data processing method, system, electronic device and storage medium | |
Zeng et al. | Design of mobile video surveillance based on Android | |
KR100592876B1 (en) | User Information Management Method of High Definition Internet Broadcasting Service System |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 08800920 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2008800920 Country of ref document: EP |