+

WO2018176340A1 - Procédé de transmission vidéo, procédé de réception, système, et véhicule aérien sans pilote - Google Patents

Procédé de transmission vidéo, procédé de réception, système, et véhicule aérien sans pilote Download PDF

Info

Publication number
WO2018176340A1
WO2018176340A1 PCT/CN2017/078867 CN2017078867W WO2018176340A1 WO 2018176340 A1 WO2018176340 A1 WO 2018176340A1 CN 2017078867 W CN2017078867 W CN 2017078867W WO 2018176340 A1 WO2018176340 A1 WO 2018176340A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
video data
data units
image
data unit
Prior art date
Application number
PCT/CN2017/078867
Other languages
English (en)
Chinese (zh)
Inventor
朱磊
崔浩
龚明
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2017/078867 priority Critical patent/WO2018176340A1/fr
Priority to CN201780005004.0A priority patent/CN108513712A/zh
Publication of WO2018176340A1 publication Critical patent/WO2018176340A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2385—Channel allocation; Bandwidth allocation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H04N21/26216—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth

Definitions

  • the present invention relates to the field of image processing, and in particular, to a video transmitting method, a receiving method, a transmitting system, a receiving system, and an unmanned aerial vehicle suitable for a movable object.
  • multi-channel can Use multi-channel data transmission channel to achieve the purpose of expanding communication bandwidth; provide multi-channel wireless and unreliable channel conditions, provide matching source coding scheme, ensure timely reliability of video transmission, and improve video transmission quality. Key technology.
  • the video captured by the shooting device mounted on the UAV can be transmitted through the communication system of the UAV.
  • the data processor on the UAV acquires the video data captured by the shooting device, encodes the video data, and then encodes the video data.
  • the subsequent video data is sent to the receiving device on the ground.
  • the number of channels of the communication network between the drone and the receiving device may be multiple.
  • the drone may transmit the encoded video data to the receiving device by using multiple channels.
  • the multi-channel is mainly used for bandwidth expansion, the encoded video data is directly packetized, and the packetized data is distributed to different channels for transmission, that is, for one.
  • the code stream data obtained by encoding one frame of image is split, and the data packet obtained by splitting is distributed to different channels for transmission, however, when the receiving device receives the data packet transmitted on one of the channels.
  • the receiving device receives the data packet transmitted on one of the channels.
  • the invention provides a video transmitting method, a receiving method, a transmitting system, a receiving system and an unmanned aerial vehicle suitable for a movable object, which are used for improving the adaptability of the source and the channel and the fault tolerance of the video transmission.
  • a first aspect of the present invention is to provide a video transmitting method suitable for a movable object, including:
  • each sub-video data unit includes one or more sub-images
  • One or more of the channels are selected to transmit the encoded sub-video data unit.
  • a second aspect of the present invention is to provide a video receiving method suitable for a movable object, comprising:
  • the video data includes one or more image frames
  • the sub-video data unit includes at least one of a plurality of sub-images obtained by decomposing each of the image frames.
  • a third aspect of the present invention is to provide a video transmission system suitable for a movable object, comprising:
  • One or more imaging devices configured to acquire video data
  • One or more processors working alone or in concert, the one or more processors being configured to:
  • each sub-video data unit includes one or more sub-images
  • One or more of the channels are selected to transmit the encoded sub-video data unit.
  • a fourth aspect of the present invention is to provide a video receiving system suitable for a movable object, comprising:
  • a communication interface a plurality of encoded sub-video data units received from one or more channels;
  • One or more processors working alone or in concert, the one or more processors being configured to:
  • the video data includes one or more image frames
  • the sub-video data unit includes at least one of a plurality of sub-images obtained by decomposing each of the image frames.
  • a fifth aspect of the present invention is to provide an unmanned aerial vehicle comprising:
  • a power system mounted to the fuselage for providing flight power
  • a sixth aspect of the present invention is to provide a receiving device, including:
  • the present invention provides a video transmitting method, a receiving method, a transmitting system, a receiving system, and an unmanned aerial vehicle suitable for a movable object, by decomposing video data into a plurality of sub-video data units according to one or more characteristics of the evaluated channel, And encoding a plurality of sub-video data units separately, selecting at least one of the plurality of channels to transmit the encoded sub-video data unit, and selecting at least one of the plurality of channels to transmit one or more encoded ones
  • the sub-video data unit can be transmitted on the channel matched with the same, and the bandwidth of the video data is reduced while the bandwidth is extended, thereby ensuring the timeliness of video transmission.
  • the receiving system can reconstruct the video image according to the received one or more sub-video data, improve the fault tolerance of the video transmission, and enhance the robustness.
  • FIG. 1 is a schematic flowchart of a video sending method applicable to a movable object according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a sub video data unit according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a sub video data unit according to another embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a sub video data unit according to another embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of selecting one or more of the channels to send an encoded sub-video data unit according to an embodiment of the present disclosure
  • FIG. 6 is a coefficient image of a frame image after Hadamard transform according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of spatial transformation decomposition according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of spatial downsampling decomposition according to an embodiment of the present invention.
  • FIG. 9 is a schematic flowchart of decomposing video data into multiple sub-video data units according to one or more characteristics of the one or more channels according to an embodiment of the present disclosure
  • FIG. 10 is a schematic flowchart of selecting one or more of the channels to send an encoded sub-video data unit according to an embodiment of the present disclosure
  • FIG. 11 is a schematic flowchart diagram of a video receiving method applicable to a movable object according to an embodiment of the present invention.
  • FIG. 12 is a schematic diagram of a decoded sub-image according to an embodiment of the present invention.
  • FIG. 13 is a schematic diagram of a decoded sub-image according to another embodiment of the present invention.
  • FIG. 14 is a schematic diagram of reconstructing an original image according to an embodiment of the present invention.
  • FIG. 15 is a schematic diagram of reconstructing an original image according to another embodiment of the present invention.
  • 16 is a structural diagram of a video transmission system suitable for a movable object according to an embodiment of the present invention.
  • FIG. 17 is a structural diagram of a video receiving system suitable for a movable object according to an embodiment of the present invention.
  • a component when referred to as being "fixed” to another component, it can be directly on the other component or the component can be present. When a component is considered to "connect” another component, it can be directly connected to another component or possibly a central component.
  • the embodiment of the present invention provides a video transmission method suitable for a movable object.
  • the mobile platform can be an unmanned aerial vehicle, a ground mobile robot, a handheld cloud platform, etc., wherein FIG. 1 is an application according to an embodiment of the present invention.
  • a schematic flowchart of a video sending method for a movable object, as shown in FIG. 1, the method may include:
  • S101 evaluating one or more characteristics of one or more channels
  • one or more characteristics of the channel include at least a bandwidth; or one or more characteristics of the channel include at least one of: noise, interference, signal to noise ratio, bit error rate, fading rate, bandwidth; or, channel One or more characteristics including the number of available channels or similarities;
  • a specific implementation manner for evaluating one or more characteristics of one or more channels is not limited, and an achievable manner is: detecting a signal strength or position of a movable object; and then according to a signal strength of the movable object or The location evaluates one or more characteristics of the channel.
  • different application environments may affect the characteristics of the channel to varying degrees. For example, when a video transmission system is used in a city, there are dense or tall buildings in the application environment that affect the transmission of wireless signals carrying video data.
  • Wireless signals are transmitted between these buildings to form multipath effects, while the signal strength of the wireless signal is attenuated, and there is more electromagnetic noise in the urban environment, which may be characteristic of the channel of the video transmission system, such as making the channel.
  • the bandwidth is reduced, the bit error rate is increased, and the similarity between channels is reduced. Therefore, before evaluating the characteristics of the channel, the location information of the mobile platform and the signal strength of the wireless signal carrying the video data may be obtained from the positioning device on the mobile platform, and the channel is evaluated according to the position signal or the signal strength. characteristic.
  • the signal strength can also be the signal strength of the positioning signal.
  • the movable platform can detect the received positioning signal.
  • Signal strength by using the signal strength of the positioning, the application scenario of the system of the video transmission system can be known, thereby evaluating the characteristics of the current channel.
  • S102 Decompose video data into a plurality of sub-video data units according to one or more characteristics of one or more channels, where each sub-video data unit includes one or more sub-images;
  • the execution body of the embodiment may be a data processor, where the data processor may be a dedicated processor, for example, a processor for performing image processing, or a general-purpose processor, which is not specifically described in the present invention. Limited.
  • the decomposition mode of the video data is determined according to one or more characteristics of the foregoing partial evaluation, and the video data is decomposed into a plurality of sub-video data units according to the decomposition manner.
  • the video data is decomposed into a plurality of sub-video data in detail in a decomposition manner determined according to one or more characteristics of the channel.
  • the manner in which the video data is decomposed is determined based on one or more evaluated characteristics of the one or more channels, the decomposition comprising at least the number of sub-video data units after decomposition, and decomposing the video data into multiple
  • the data processor decomposes the video data according to the number of sub-video data units indicated in the decomposition mode, that is, the number of the plurality of sub-video data units obtained after the decomposition is the sub-inclusion included in the decomposition mode.
  • the number of video data units, wherein the number of sub-video data units is determined based on one or more characteristics of the evaluated channel.
  • the number of the decomposed sub-video data units included in the decomposition mode is determined to be 4 according to the characteristics of the one or more channels, and when the video data is decomposed, the data processor decomposes the video data into 4 sub-video data units.
  • the number of sub-video data units in the decomposition mode may be determined by available channel data. Specifically, before decomposing the video data, the number of available channels may be evaluated, and the number of available channels is used to determine the decomposed sub-distribution in the decomposition mode.
  • the number of video data units For example, the current number of available channels is four, and the data processor can decompose the video data into four sub-videos. Data unit.
  • the number of sub-video data units in the decomposition mode may be determined by the bandwidth of the channel. Specifically, before the video data is decomposed, the bandwidth of the available channel may be evaluated, and the number of channels in the channel whose bandwidth is greater than the preset bandwidth threshold is determined. The number of decomposed sub-video data units in the decomposition mode is determined by the number of channels satisfying the above conditions. For example, the number of channels in the current channel whose bandwidth is greater than the preset bandwidth threshold is four, and the video data can be decomposed into four sub-video data units.
  • the bit error rate of the available channel determines the number of channels in the channel whose bit error rate is greater than the preset bit error rate, and determine the decomposed sub-video data in the decomposition mode by using the number of channels satisfying the above conditions.
  • the number of units It should be noted that, in this embodiment, the number of sub-video data units in the decomposition mode is determined according to the number of available channels, the bandwidth of the channel, and the bit error rate of the channel, for illustrative purposes only, and those skilled in the art may also The other characteristics of the channel are used to determine the number of sub-video data units in the determined decomposition mode, which is not specifically limited herein.
  • one implementation manner for decomposing video data into a plurality of sub-video data units according to an decomposition manner is: decomposing the video data into a plurality of sub-video data according to a decomposition manner such that the sub-video data units have similarities. characteristic.
  • the decomposition mode is determined according to one or more characteristics of the evaluated channel, and the video data is decomposed into a plurality of sub-video data units having similar characteristics according to the decomposition manner.
  • the bandwidth or channel similarity (band similarity of the channel) of each of the multiple channels may be approximately the same, in this case, according to the bandwidth of the channel or the similarity of the channels (the bandwidth of the channel)
  • the similarity is determined by the decomposition mode of the video data, and the video data may be decomposed into a plurality of sub-video data units having similar characteristics according to the decomposition manner, wherein the similarity characteristic may be the size of the code stream data encoded by the sub-video data unit or The energy concentration of the sub-video data unit.
  • bit error rate or the similarity of the channel (the similarity of the bit error rate of the channel) of each of the plurality of channels, and the bit error rate of the plurality of channels may be substantially the same, in this case, according to the channel error
  • the bit rate or channel similarity determines the manner in which the video data is decomposed, and the video data can be decomposed into a plurality of sub-video data units having similar characteristics according to the decomposition method.
  • another achievable manner of decomposing video data into a plurality of sub-video data units according to an decomposition manner is: decomposing the video data into a plurality of sub-video data according to a decomposition manner such that the sub-video data unit has Different characteristics.
  • the decomposition mode is determined according to one or more characteristics of the evaluated channel, and the video data is decomposed into a plurality of sub-video data units having different characteristics according to the decomposition manner.
  • the sub-video data unit has different characteristics including: the code stream data size after the sub-video data unit is encoded, or the energy concentration of the sub-video data unit. For example, after evaluating the bandwidth of each channel of multiple channels, the bandwidth of each channel can be found to be different. To match different channel bandwidths, the decomposition mode is determined according to the bandwidth of the channel, and the video data can be decomposed according to the decomposition mode. Sub-video data units with different characteristics.
  • the size of the code stream after the decomposed sub-video data unit is encoded may be different (see the following for a detailed explanation).
  • the energy concentration of the decomposed sub-video data units may be different (see the following for a detailed explanation).
  • the number of channels in the plurality of channels whose bandwidth is greater than or equal to the bandwidth threshold may be determined, and the number of sub-video data units in the decomposition mode is determined according to the number of channels whose bandwidth is greater than or equal to the bandwidth threshold, according to the data.
  • the video data is decomposed into a specified number of sub-video data units, and the plurality of sub-video data units obtained after the decomposing have different energy concentration degrees, or the encoded sub-data data units obtained by the decomposing are different in size.
  • the decomposition manner may also be determined according to other characteristics of the channel, so that the plurality of sub-video data obtained by decomposing the video data according to the decomposition manner have different characteristics, for example, the bit error rate of the channel may be evaluated, and the bit error rate of each channel may be different.
  • the decomposition mode may be determined according to the bit error rate of the channel, and the video data is decomposed into multiple sub-video data units according to the decomposition mode, and the energy concentration of the plurality of sub-video data units is different, or the plurality of sub-video data units are encoded.
  • the stream data size varies.
  • the foregoing part determines the decomposition manner of the video data according to the bandwidth, the bit error rate, and the degree of similarity of the channel for illustrative purposes only, and those skilled in the art may also determine the decomposition mode according to other characteristics of the channel, where No specific restrictions are made.
  • This embodiment does not limit the number of image frames included in one video data.
  • the video data includes 6 image frames, that is, 6 frames, and further, in other embodiments, the video The number of image frames included in the data may also be other values.
  • the video data includes six image frames, which are frame 1, frame 2, frame 3, frame 4, frame 5, and frame 6, and frame 1, frame 2, frame 3, frame 4, and frame 5.
  • the frame 6 is separately decomposed.
  • the number of sub-images into which each image frame is decomposed is not limited.
  • each image frame is decomposed into four sub-images, which is only schematically illustrated herein. Subgraph obtained after each image frame is decomposed The number of images can also be other values.
  • Each of the sub-video data units includes at least one of the four sub-images corresponding to each of the image frames after each of the six image frames is decomposed.
  • each sub-video data unit includes one of the four sub-images corresponding to each of the six image frames.
  • the sub-video data unit 210 includes one sub-image 11 of the frame 1.
  • the sub-video data unit 310 includes two sub-images 11, 12 of the frame 1, and two sub-images 21, 22 of the frame 2.
  • sub-video data unit 320 includes one sub-image 13 of frame 1, frame 2
  • One sub-image 23 of frame 3 two sub-images 42, 43 of frame 4
  • sub-video data unit 330 includes One sub-image of each of the six image frames.
  • each sub-video data unit do not overlap.
  • the manner in which at least one of the plurality of sub-images corresponding to each of the plurality of image frames is combined to form a sub-video data unit may also have other combinations, which are not enumerated here.
  • the video data may also include only one image frame, that is, one frame image.
  • 40 indicates an image frame included in the video data, and the image frame 40 is decomposed. This embodiment does not limit one image frame.
  • Sub-picture 11, sub-image 12, sub-image 13, and sub-image 14 can be divided into the following achievable ways:
  • each sub-video data unit includes one sub-image, such as sub-video data unit 410, sub-video data unit 420, sub-video data unit 430, sub-video data unit 440, as shown in FIG.
  • each sub-video data unit includes two sub-images. This embodiment does not limit the combination of two sub-images included in one sub-video data unit.
  • the sub-image shown in FIG. Video data unit 450 and sub video data unit 460 wherein the sub video data The unit 450 includes a sub-image 11 and a sub-image 12, and the sub-video data unit 460 includes a sub-image 13 and a sub-image 14.
  • each sub-video data unit includes a different number of sub-images, such as the sub-video data unit 470 and the sub-video data unit 480 shown in FIG. 4, wherein the sub-video data unit 470 includes three sub-pictures.
  • the sub-video data unit 470 includes a sub-image 11, a sub-image 12, a sub-image 13, and the sub-video data unit 480 includes a sub-image 14.
  • the data processor encodes each sub-video data unit of the plurality of sub-video data units by using the decomposed sub-video data unit as a coding unit, and obtains a plurality of code stream data after encoding.
  • a sub-video data unit is encoded to obtain a code stream data, where the encoding includes source coding and/or channel coding, and the source coding manner may include H.263, H.264, and H.265.
  • the channel coding method may include error correction coding, and the types of the error correction code may include an RS code, that is, a Reed-Solomon code, a convolutional code, a Turbo code, a Polar code, an interleaving code, a pseudo random sequence scrambling code, and the like.
  • RS code that is, a Reed-Solomon code, a convolutional code, a Turbo code, a Polar code, an interleaving code, a pseudo random sequence scrambling code, and the like.
  • S104 Select one or more of the channels to transmit the encoded sub-video data unit.
  • one or more channels may be selected to transmit the encoded sub video data unit, and specifically, each encoded sub video data may be The unit selects a channel from which to transmit the encoded sub-video data unit.
  • the receiving device receives the sub-video data unit in one or more channels and reconstructs the video image based on the received sub-video data unit.
  • the video transmitting method applicable to the movable object provided by the embodiment by decomposing the video data into a plurality of sub-video data units according to one or more characteristics of the evaluated channel, and separately encoding the plurality of sub-video data units, selecting At least one of the plurality of channels transmits the encoded sub-video data unit, and when at least one of the plurality of channels is selected to transmit one or more of the encoded sub-video data units, the plurality of sub-video units are decomposed
  • the video data unit can match the characteristics of the channel, so that the sub-video data unit is sent to the receiving device on the channel matched thereto, thereby improving channel utilization and video data transmission efficiency, and at the same time, the receiving device can receive the encoding according to the receiving.
  • the sub-video data after the code reconstructs the video image, which improves the fault tolerance and robustness of the video transmission.
  • selecting one or more of the channels to transmit the encoded sub-video data unit comprises selecting one or more of the channels to transmit the code based on one and/or more characteristics of the evaluated channel. Sub-video data unit.
  • the decomposition mode is determined according to one or more characteristics of the channel, and the video data is decomposed according to the decomposition manner to obtain a plurality of sub-video data units, wherein the plurality of sub-video data units may have similar characteristics or multiple sub-video data. Units can have different characteristics.
  • the characteristics of the sub video data unit may include a code stream data size encoded by the sub video data unit or an energy concentration of the sub video data unit.
  • one or more characteristics of the channel are reviewed.
  • the video data can be decomposed into similar characteristics, for example:
  • the code stream data encoded by the plurality of sub video data units is similar in size, and the energy concentration of the plurality of sub video data units is substantially similar.
  • one channel may be randomly allocated to the sub video data unit.
  • the video data can be decomposed into different characteristics, for example: multiple sub- The code stream data encoded by the video data unit has different sizes, and the energy concentration of the plurality of sub video data units is different. Selecting one or more of the channels to transmit the encoded sub-video data unit includes: selecting a channel-transcoded sub-video data that matches a characteristic of the sub-video data unit from the plurality of channels according to one or more characteristics of the channel Units, specifically, can be implemented in several possible ways:
  • the first feasible manner is: selecting at least one channel to transmit the encoded sub-video data unit according to the code stream data size and channel bandwidth of the encoded sub-video data unit; specifically, the video data is decomposed to obtain multiple sub-videos.
  • the encoded sub-video data unit has different code stream data sizes and different channel bandwidths.
  • the code stream data size can be matched with the channel bandwidth, and the channel is selected to transmit the encoded sub-video data unit.
  • the data processor decomposes the video data to obtain four sub-video data units, the four sub-video data units are respectively recorded as a sub-video data unit A, a sub-video data unit B, a sub-video data unit C, and a sub-video data unit D.
  • the size of the code stream data is S0, S1, S2, and S3, and the size of the four code stream data may be different.
  • the size of the four code stream data is successively decremented, if the current wireless channel includes channel 1, channel 2, channel 3, and channel 4, and the bandwidth T of the above channel is T0, T1, T2, T3, and four, respectively.
  • the bandwidth of the channel is successively decremented.
  • one channel can be selected for each stream data according to the bandwidth of the current channel.
  • channel 1 with the largest channel bandwidth can be used to transmit the encoded sub-video data unit A
  • channel 2 with the second largest channel bandwidth can be used to transmit the encoded sub-video data unit B
  • the channel bandwidth is the third largest.
  • Channel 3 is used to transmit the encoded sub-video data unit C
  • channel 4 with the smallest channel bandwidth is used to transmit the encoded sub-video data unit D.
  • the channel with strong data transmission capability can transmit the sub-video data unit with larger coded stream data
  • the channel with weak data transmission capability can transmit the sub-video data unit with smaller code stream data.
  • the second feasible manner is: selecting at least one channel to transmit the encoded sub-video data unit according to the energy concentration degree and the channel bandwidth of the sub-video data unit, and further, according to the priority and channel of the encoded sub-video data unit.
  • Bandwidth at least one channel is selected to transmit the encoded sub-video data unit. Specifically, after the video data is decomposed, multiple sub-video data units are obtained. The energy concentration of the sub-video data units is different, and the bandwidth of the channel is also different. In this case, the energy concentration can be matched with the bandwidth of the channel, and the channel is selected for transmission.
  • the encoded sub-video data unit wherein the higher the energy concentration of the sub-video data unit is more important when the receiving device reconstructs the video data.
  • the data processor decomposes the video data to obtain four sub-video data units
  • the four sub-video data units are respectively recorded as a sub-video data unit A, a sub-video data unit B, a sub-video data unit C, and a sub-video data unit D.
  • the energy concentration of the four sub-video data units is different.
  • the energy concentration of the four sub-video data units is sequentially decremented, if the current radio channel includes channel 1, channel 2, and channel 3. Channel 4, and the bandwidth of the four channels is sequentially decremented.
  • the energy concentration of the sub-video data unit and the channel may be Bandwidth matching, that is, channel 1 with the largest channel bandwidth can be used to transmit the encoded sub-video data unit A with the highest energy concentration, and channel 2 with the second largest channel bandwidth can be used to transmit the encoded energy concentration second.
  • the high sub-video data unit B uses the channel 3 having the third largest channel bandwidth to transmit the encoded energy concentration third.
  • the number of channels in the foregoing embodiment is four for the purpose of illustration.
  • the number of channels may be other numbers.
  • the number of channels may be 5, 6, 7, 8, or the like.
  • a plurality of channels having the largest bandwidth in the channel may be selected to transmit a plurality of sub-video data units.
  • the video data may include one or more image frames.
  • Decomposing the video data into the plurality of sub-video data units specifically includes decomposing each of the one or more image frames in the video data into multiple The sub-images, at this time, each of the sub-video data units includes at least one of the plurality of sub-images decomposed in each of the image frames.
  • the foregoing sub-image is a part of an image frame, specifically, the sub-image may be one or more pixels of the image frame, or the sub-image may also be one or more conversion coefficients of the image frame, etc., below A schematic description will be made by decomposing one image frame into a plurality of sub-images.
  • one way to implement each of the one or more image frames in the video data into a plurality of sub-images is to decompose each of the one or more image frames in the video data into multiple sub-spaces. image.
  • Decomposing the image frame space into a plurality of sub-images in this embodiment may have multiple implementation manners, and one of the implementation manners is to decompose each of the one or more image frames in the video data into multiple sub-images.
  • the method is configured to: decompose each of the one or more image frames in the video data into a plurality of sub-images using a Fourier correlation transform or an orthogonal transform.
  • the Fourier correlation transform or the orthogonal transform is determined from a Hadamard transform, a discrete cosine transform, a discrete Fourier correlation transform, a Walsh-Hadamard transform, a Haar transform or an oblique transform.
  • a schematic diagram of a frame image is used.
  • This embodiment does not limit the number of pixels included in a frame image, and a frame image includes 16 pixels (4 ⁇ 4), for example, P1-P16.
  • the pixel value of 16 pixel points is represented, and the pixel values of each of the four adjacent pixel points of the 16 pixel points are spatially transformed and decomposed into four sub-images, and the following is schematically illustrated by Hadamard transform.
  • the spatial transformation decomposition process includes the following steps:
  • Step 1 Perform a Hadamard transform by using four adjacent pixels of the 16 pixels as a unit.
  • the conversion coefficients obtained by the Hadamard transform of P1, P2, P3, and P4 are H1, H2, and H3. H4, wherein the relationship between P1, P2, P3, P4 and H1, H2, H3, H4 satisfies the formulas (1), (2), (3), (4):
  • H1 (P1+P2+P3+P4+1)>>1 (1)
  • H2 (P1+P2–P3–P4+1)>>1 (2);
  • H3 (P1+P3–P2–P4+1)>>1 (3)
  • H4 (P1+P4–P2–P3+1)>>1 (4);
  • H1 contains the average energy of 4 pixels
  • H2 contains the average gradient of 4 pixels in the vertical direction
  • H3 contains 4 pixels.
  • H4 contains a cross gradient of 4 pixels, ie texture information. Therefore, when the receiving device reconstructs the image frame, H1 is the most important, the importance of H2 and H3 is second, and the importance of H4 is the smallest, that is, the importance of H1, H2, H3, and H4 is successively decreased.
  • Step 2 Decompose the conversion coefficients obtained by the Hadamard transform into different sub-images.
  • This embodiment does not limit the number of sub-images obtained by spatially transforming each frame of the image, for example, the sub-images obtained after the decomposition. The number is four.
  • H1 is assigned to the first sub-image
  • H2 is assigned to the second sub-image
  • H3 is assigned to the third sub-image
  • H4 is assigned to the fourth sub-image.
  • H5-H8 is decomposed into 4 sub-images in the same way
  • H9-H12 is decomposed into 4 sub-images in the same way
  • H13-H16 is decomposed into 4 sub-images in the same way, which is obtained as shown in Fig. 7.
  • each of the four sub-images after the spatial transformation decomposition is one quarter of the original image before the decomposition.
  • one image frame is decomposed into four sub-images for illustrative purposes, and one skilled in the art can decompose one image frame into other sub-images.
  • each of the one or more image frames in the video data can be decomposed into a plurality of sub-images, including: using spatial downsampling to image one or more image frames in the video data.
  • Each space in the space is decomposed into multiple sub-images.
  • the image frame is spatially downsampled and decomposed into 4 sub-images
  • the specific spatial downsampling decomposition process is:
  • Each of the 16 pixels adjacent to the 16 pixels is a unit, and 4 pixels in one unit are decomposed into different sub-images, for example, P1 is decomposed into the first sub-image, and P2 is decomposed into The second sub-image, which decomposes P3 into the third sub-image, decomposes P4 into the fourth sub-image.
  • the first sub-image may include a pixel with coordinates (2i, 2j) in the original image
  • the second sub-image may include a pixel with coordinates (2i+1, 2j) in the original image
  • the third sub-image may include pixels with coordinates (2i, 2j+1) in the original image
  • the fourth sub-image may include pixels with coordinates (2i+1, 2j+1) in the original image, where 2i+1 ⁇ W, 2j+1 ⁇ H.
  • an image frame having 16 pixels (4 ⁇ 4) is decomposed into 4 sub-images, wherein the number of pixels included in each sub-image is the same for illustrative purposes only, and in some implementations. In an example, depending on one or more characteristics of the channel, the number of pixels included in the sub-image may be different when the image frame is decomposed.
  • Each of the one or more image frames in the video data can be decomposed into a plurality of sub-images according to any of the above-described decomposition methods or spatial downsampling.
  • one image frame is one frame image
  • a plurality of image frames are multi-frame images.
  • the video data includes one or more image frames
  • the sub-video data unit includes at least one of a plurality of sub-images obtained by decomposing each of the image frames.
  • the sub-video data unit may include at least one of the plurality of sub-images obtained by decomposing the image frame.
  • a sub-video data unit includes a sub-image, and each sub-image obtained after the decomposition is encoded to obtain code stream data encoded by the sub-video unit.
  • each image frame that is, the image of each frame is decomposed as shown in FIG. 7 or FIG. 8.
  • the video data includes four image frames, and each image frame is decomposed into four.
  • each sub-image four consecutive image frames are decomposed to obtain 4*4 sub-images
  • each sub-video data unit may include a plurality of sub-images of 4*4 sub-images.
  • the number of sub-video data units may be four, and at least one sub-image is selected from a plurality of sub-images (four sub-images) decomposed in each image frame, and the selected sub-images are composed into sub-video data units.
  • each sub-image includes a portion of an image frame.
  • each sub-image includes one or more pixels of an image frame.
  • each sub-image includes one or more conversion coefficients of the image frame.
  • the energy concentration of the sub-image 1, the sub-image 2, the sub-image 3, and the sub-image 4 is the same or similar.
  • FIG. 8 the energy concentration of the sub-image 1, the sub-image 2, the sub-image 3, and the sub-image 4 is the same or similar.
  • the energy concentration of the sub-image 1 is the largest, the energy concentration of the sub-image 2 and the sub-image 3 is smaller than the energy concentration of the sub-image 1, and the energy concentration of the sub-image 4 is the smallest, which is further known, including Sub-picture data unit energy of sub-image 1
  • the concentration is the highest, and the energy concentration of the two sub-video data units including the sub-image 2 and the sub-image 3 respectively is second, and the sub-video data unit including the sub-image 4 has the lowest energy concentration.
  • the video data is decomposed by spatial downsampling or spatial transform, and the obtained plurality of sub video data units may have different code stream data sizes encoded by the plurality of video data units.
  • the method further includes:
  • S901 Determine a decomposition manner of the video data according to one or more evaluation characteristics of the one or more channels, and decompose the video data into multiple sub-video data units according to the decomposition manner;
  • S902 Send information for decomposing the video data into a plurality of sub-video data units.
  • the information that is decomposed into the decomposition mode of the plurality of sub-video data units may be sent to the receiving device, and the receiving device receives the information of the decomposition mode when receiving the information of the decomposition mode.
  • the decomposition method of the image frames in the video data for example, spatial down-sampling decomposition, Hadamard transform decomposition, etc.
  • the information for obtaining the decomposition method is more using the receiving device. Rebuild video data well.
  • the information of the decomposition mode may be included in one or more sub-video data units, that is, the information of the decomposition mode is transmitted while the sub-video data unit is transmitted; at this time, when the information of the decomposition mode is transmitted, the information of the decomposition mode may be encoded.
  • the information of the decomposition mode may be encoded at the beginning of the sub-video data unit, or the information of the decomposition mode may be encoded at the end of the sub-video data unit, specifically, this The special field may be known to the receiving device.
  • the receiving device may acquire the information of the decomposition mode from the sub-video data unit according to a preset rule.
  • the information of the decomposition mode may be transmitted by using a separate channel before transmitting the plurality of sub-video data units; wherein the channel for transmitting the information of the decomposition mode may be different from the channel for transmitting the sub-video data unit, that is, using a separate channel
  • the channel transmits the information of the decomposition mode, which can effectively ensure the stable reliability of the information transmission in the decomposition mode.
  • selecting one or more channels in the channel to transmit the encoded sub-video data unit is configured to include:
  • S1001 Divide the encoded multiple sub-video data units into one or more sub-video data unit groups according to one or more characteristics of one or more channels;
  • S1002 Select one or more of the channels to send a sub-video data unit group.
  • one or more characteristics of one or more channels may be used.
  • the plurality of sub-video data units are divided into one or more sub-video data unit groups, and the one sub-video data unit group may include one or more encoded sub-video data units, for example, according to available channels. Number, bandwidth, bit error rate, etc.
  • sub-video data unit groups for example, to decompose the video data unit into sub-video data unit A, sub-video data unit B, sub-video Data unit C, sub-video data unit D, if there are currently 2 available channels, the sub-video data unit A, the sub-video data unit B, the sub-video data unit C, and the sub-video data unit D may be divided into two sub-video data units.
  • the group after division, uses an available channel to transmit a sub-video data unit group.
  • the encoded plurality of sub-video data units may be divided into one or more sub-video data unit groups according to the channel wideband, for example, the video data unit is decomposed into the sub-video data unit A, the sub-video data unit B, and the sub-video data.
  • the unit C, the sub-video data unit D, the sub-video data unit E, the sub-video data unit F, and the size of the code stream data corresponding to the six sub-video data units are S0, S1, S2, S3, S4, and S5, respectively.
  • the sub-video data unit A, the sub-video data unit B, the sub-video data unit C, the sub-video data unit D, the sub-video data unit E, the sub-video may be provided for the transmission of the two sub-video data unit groups to meet the transmission delay requirement.
  • the data unit F is divided into two sub-video data unit groups, channel 1 transmits one sub-video data unit group, and channel 2 transmits another sub-video data unit group.
  • a sub-video data unit group can be transmitted by selecting a channel with a channel bandwidth matching the encoded code stream data size of the sub-video data unit group among the plurality of channels, that is, according to The sub-video data unit group encodes the code stream data size and the bandwidth of the channel to select a channel.
  • the sub-video data unit group encodes the code stream data size and the bandwidth of the channel to select a channel.
  • the size of the code stream data corresponding to the unit C, the sub-video data unit D, and the four sub-video data units is S0, S1, S2, and S3, respectively, and the size of the code stream data is successively decreased, if T0>S0+S1>T1 >S2+S3, sub-video data units A and B can be divided into one sub-video data unit group 1, and sub-video data units C and D are divided into one sub-video data.
  • Unit group 2 at this time, channel 1 matching sub-video data unit group 1 can be selected for transmitting sub-video data unit group 1, and channel 2 matching sub-video data unit group 2 can be selected for transmitting sub-video data unit Group 2.
  • the sub-video data unit group with large code stream data can be transmitted using a channel with a large bandwidth
  • the sub-video data unit group with small code stream data is transmitted using a channel with a small bandwidth.
  • one of the plurality of channels may be selected to transmit one sub-video data unit group according to the priority of the sub-video data unit group and the channel bandwidth.
  • the priority may be prioritized according to the energy concentration degree of the sub video data unit or the sub video data unit group, and the sub video data unit or the sub video data unit group with high energy concentration has high priority and low energy concentration.
  • the priority of the sub video data unit or the sub video data unit group is low.
  • the sub-video data unit group may be selected according to the channel bandwidth and the priority of the sub-video data unit group, for example, there are currently 2 channels.
  • the video data is decomposed into sub-video data unit A, sub-video data unit B, sub-video data unit C, sub-video data unit D, and the corresponding energy concentration degrees are sequentially Decreasing, the sub-video data units A and B are divided into one sub-video data unit group 1, and the energy concentration of the sub-video data unit group 1 is high, and the sub-video data units C and D are divided into one sub-video data unit group.
  • the energy concentration of the sub-video data unit group 1 is greater than the energy concentration of the sub-video data unit group 2
  • the priority of the sub-video data unit group 1 is greater than the priority of the sub-video data unit group 2, at this time,
  • channel 1 with a large channel bandwidth can be selected for transmission.
  • Video data unit group 1, the channel bandwidth may be selected small channel 2 to transmit video data unit sub-group 2. In this way, it is ensured that the sub-video data unit group 1 with high energy concentration/high priority is transmitted using a channel having a large bandwidth, and the sub-video data unit group 2 having low energy concentration/low priority is transmitted using a channel having a small bandwidth.
  • separately encoding the plurality of sub-video data units is configured to include:
  • a plurality of sub-video data units are encoded by a plurality of separate encoders.
  • multiple sub-video data units may be encoded in parallel by using multiple independent encoders; or, multiple sub-video data units may be encoded by using different video coding rules; or, the same video coding rule pair may be used.
  • a plurality of sub-video data units are encoded.
  • separately encoding the plurality of sub-video data units is configured to include:
  • Two or more of the plurality of sub-video data units are encoded by the same encoder.
  • separately encoding the plurality of sub-video data units is configured to include:
  • At least one of the plurality of sub-video data units is encoded using a motion compensation based video compression standard.
  • separately encoding the plurality of sub-video data units is configured to include:
  • Multiple sub-video data units are compressed according to different compression ratios.
  • the compression ratio may be determined according to one or more characteristics of the sub-video data unit; and for a plurality of sub-video data units, they may have the same characteristics or different characteristics.
  • sub-video data units are encoded or compressed by using different encoders or motion compensation-based video compression standards or different compression ratios, and the implementation manner is various, easy to operate, and the sub-video is effectively guaranteed.
  • the data unit performs coding stability and reliability.
  • the embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program instructions, and the program instructions may include some or all of the steps of the sending method in the foregoing embodiments.
  • FIG. 11 is a schematic flowchart of a video receiving method applicable to a movable object according to an embodiment of the present invention.
  • the embodiment provides a video receiving method suitable for a movable object, for example,
  • the video receiving method is configured to receive video image information transmitted by an unmanned aerial vehicle.
  • the video receiving method includes:
  • S1101 Receive a plurality of encoded sub-video data units transmitted by one or more channels;
  • the receiving device receives, by using one or more channels, a plurality of encoded sub-video data units transmitted by the video transmitting system, wherein each channel may transmit one or more encoded sub-video data units, and
  • the video data may include one or more image frames, and each image frame included in the video data may be decomposed into a plurality of sub-images before the video data is transmitted, and the sub-video data unit includes a plurality of sub-images obtained by decomposing each image frame. At least one of the sub-video data units includes at least one sub-image of each image frame.
  • four sub-video data units are received by the receiving device through one or more channels, and each sub-video data unit includes a sub-image obtained as shown in FIG. 7 or FIG. 8 for schematic description.
  • S1102 Decode the encoded multiple sub-video data units.
  • the receiving device may separately decode the four encoded sub-video data units, that is, separately perform code stream data corresponding to the four sub-images. Decode to obtain the decoded sub-video data unit.
  • the sub video data unit is transmitted in the wireless channel, due to noise interference, multipath effect, fading, etc., the sub video data unit obtained by the receiving device and the video transmission system of the UAV may be actually transmitted after being encoded. The sub video data unit is different, and the receiving device receives an error at this time.
  • the video transmission system system transmits the four encoded sub-images as shown in FIG. 7, the four sub-images obtained by the receiving device after decoding the encoded sub-images are as shown in FIG. 12, wherein if the sub-image 1 is transmitted Correct, H1 and h1 are the same, H2 and h2 are the same, H3 and h3 are the same, H4 and h4 are the same. If sub-image 1 transmission error, at least one of H1 and h1, H2 and h2, H3 and h3, H4 and h4 are not the same. Similarly, other sub-pictures are transmitted correctly or transmitted incorrectly, and the transform coefficients before transmission and the transform coefficients after transmission also have the same relationship.
  • the four sub-images obtained by the receiving device after decoding the encoded sub-images are as shown in FIG. 13, wherein if the sub-image 1 is transmitted correctly , P1 and p1 are the same, P2 and p2 are the same, P3 and p3 are the same, and P4 and p4 are the same. If sub-image 1 is transmitted incorrectly, at least one of P1 and p1, P2 and p2, P3 and p3, P4 and p4 are different. . Similarly, other sub-images are transmitted correctly or transmitted incorrectly, and the pixels before transmission and the pixels after transmission also have the same relationship.
  • S1103 reconstruct video data according to the decoded sub video data unit.
  • the receiving device at this time needs to know the decomposition mode information that decomposes the video data into a plurality of sub-video data units, so that the receiving device at this time can also receive Decomposition mode information sent by one or more channels for decomposing video data into a plurality of sub-video data units; further, after the receiving device decodes the encoded sub-images to obtain 4 sub-images, in order to improve the correctness of reconstructing the video data Rate, it is possible to detect that each sub-image is transmitted correctly or a transmission error, and reconstruct the original image according to receiving the correct sub-image, wherein the transmission of the wrong sub-image and the transmission of the correct sub-image are from the same image frame.
  • the sub-image sent by the video transmission system is the sub-image shown in FIG. 7, and the sub-image received by the receiving device is as shown in FIG. 12, as shown in FIG. 12, assuming that the sub-image 2 receives an error, the sub-image 1.
  • Sub-image 3 and sub-image 4 are all received correctly, then the receiving device according to the sub-picture shown in FIG. Image 1, sub-image 3, and sub-image 4 reconstruct the original image.
  • a value may be assigned to the sub-image transmitted in the decoded sub-video data unit.
  • a feasible implementation is: sub-video The sub-image assignment of the transmission error in the data unit has a value of zero.
  • h2, h6, h10, and h14 of the sub-image 2 can be set to 0.
  • H1, H2, H3, and H4 are obtained according to P1, P2, P3, and P4, and therefore, reconstruction is performed.
  • H1 and h1 are the same, H3 and h3 are the same, and H4 and h4 are the same. Therefore, p1, p2, p3, p4 obtained by inverse Hadamard transform and pixel values P1, P2, P3 in the original image, P4 may be different, but the original image is reconstructed according to receiving the correct sub-image, wherein the transmission of the erroneous sub-image and the transmission of the correct sub-image may be from the same image frame, ensuring that the reconstructed image is close to the original image.
  • h5, h6, h7, h8 are inversely transformed by Hadamard to obtain p5, p6, p7, p8, and h9, h10, h11, h12 are inversely transformed by Hadamama to obtain p9, p10, p11, p12, h13, H14, h15, h16 perform inverse Hadamard transform to obtain p13, p14, p15, p16, wherein h6, h10, h14 are all 0, and reconstruct the original image according to p1-p16 obtained by inverse Hadamard transform, as shown in Fig. 14. .
  • the sub-image sent by the video transmitting system is the sub-image shown in FIG. 8, and the sub-image received by the receiving device is as shown in FIG. 13.
  • the sub-image 3 receives an error.
  • the sub-image 1, the sub-image 2, and the sub-image 4 are all received correctly, and the receiving device reconstructs the original image according to the sub-image 1, the sub-image 2, the sub-image 3, and the sub-image 4 shown in FIG. 13, and when reconstructing the original image, the sub-image
  • the image 3 does not participate in the reconstruction process, that is, the receiving device reconstructs the original image only according to the sub-image 1, the sub-image 2, and the sub-image 4.
  • the specific process is as follows: since the original image includes 16 pixels, the sub-image 1, the sub-image 2, and the sub-image 4 A total of 12 pixels, according to FIG. 8, each of the 16 adjacent pixels in the original image is decomposed into 4 different sub-images. Therefore, according to the sub-image 1, the sub-image 2, and the sub-image When the image 4 reconstructs the original image, the first pixel p1 of the sub-image 1, the first pixel p2 of the sub-image 2, and the first pixel p4 of the sub-image 4 are the first group of the original image.
  • p5, p6, p8 are 3 pixels in the original image P5-P8, p9, p10, p12 are 3 pixels in the original image P9-P12, p13, P14 and p16 are three pixels in the original image P13-P16, and an image A as shown in FIG. 15 can be obtained from p1, p2, p4, p5, p6, p8, p9, p10, p12, p13, p14, p16.
  • the pixel remaining in the image A is the pixel included in the sub-image 3 that received the error, and since the sub-image 3 receives an error, the sub-image 3 cannot participate in the process of reconstructing the original image.
  • a sub-image 3 of the sub-video data unit may be assigned a value.
  • Another feasible implementation is: sub-video data. The value assigned to the sub-image 3 in which the error is transmitted in the unit is determined by interpolation.
  • the value assigned to the sub-picture transmitted in the sub-video data unit is determined based on the transmission of the correct sub-image.
  • the pixel value of the image A can be determined according to the interpolation method.
  • a feasible interpolation method is: p3 is equal to the arithmetic mean of p1, p2, and p4, and p7 is equal to the arithmetic mean of p5, p6, and p8.
  • P11 is equal to the arithmetic mean of p9, p10, p12, and p15 is equal to the arithmetic mean of p13, p14, p16, thereby obtaining the reconstructed original image B.
  • only one method of interpolating the pixel value of the image A is provided, and those skilled in the art may use other interpolation methods to determine the residual pixel value of the image A, which is not specifically limited herein.
  • the video receiving method for a movable object receives a plurality of encoded sub-video data units transmitted by one or more channels, and then separately decodes the plurality of sub-video data units to obtain a decoded sub- a video data unit, and reconstructing the original image according to the decoded sub-video data unit, thereby realizing that a reconstructed image without mosaic error can be obtained as long as one or more channel data is correctly received; the more correctly received channels, The final reconstructed image quality is higher; when all channels are correctly received, the maximum reconstructed image quality is obtained and the transmitted original image is consistent, thereby improving the fault tolerance of the receiving device in reconstructing the image and enhancing the robustness.
  • the embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program instructions, and the program instructions may include some or all of the steps of the receiving method in the foregoing embodiments.
  • FIG. 16 is a video transmission system suitable for a movable object according to an embodiment of the present invention.
  • FIG. 16 shows that the present embodiment provides a video transmission system suitable for a movable object, and the video transmission system 1600 can be disposed on a movable platform, for example, on an unmanned aerial vehicle, and used for The video sending method is performed.
  • the video sending system includes:
  • One or more imaging devices 1601 configured to acquire video data
  • One or more processors 1602 on the movable object, working alone or in concert, one or more processors 1602 are configured to:
  • each sub-video data unit includes one or more sub-images
  • One or more of the channels are selected to transmit the encoded sub-video data unit.
  • the processor 1602 may be configured to: when the video data is decomposed into a plurality of sub-video data units:
  • Each of the one or more image frames in the video data is decomposed into a plurality of sub-images, wherein each of the sub-video data units includes at least one of the plurality of sub-images of each of the image frames.
  • each of the sub-images described above includes a portion of an image frame, and specifically, each sub-image may include one or more pixels of the image frame, or each sub-image includes one or more conversion coefficients of the image frame.
  • the processor 1602 when the processor 1602 decomposes each of the one or more image frames in the video data into multiple sub-images, the processor 1602 may be configured to:
  • Each of the one or more image frames in the video data is decomposed into a plurality of sub-images.
  • the processor 1602 when the processor 1602 decomposes each of the one or more image frames in the video data into multiple sub-images, the processor 1602 may be configured to:
  • each of the one or more image frames in the video data into a plurality of sub-images using a Fourier correlation transform or an orthogonal transform; wherein the Fourier correlation transform or the orthogonal transform is a Hadamard transform Determined by discrete cosine transform, discrete Fourier correlation transform, Walsh-Hadamard transform, Haar transform or oblique transform.
  • the Fourier correlation transform or the orthogonal transform is a Hadamard transform Determined by discrete cosine transform, discrete Fourier correlation transform, Walsh-Hadamard transform, Haar transform or oblique transform.
  • processor 1602 decomposes each of the one or more image frames in the video data into a plurality of sub-images
  • the processor 1602 can be configured to:
  • Each of the one or more image frames in the video data is decomposed into a plurality of sub-images using spatial downsampling.
  • one or more characteristics of the foregoing channel include at least a bandwidth.
  • one or more characteristics of the channel may include at least one of the following: noise, interference, signal to noise ratio, bit error rate, and fading rate.
  • Bandwidth; alternatively, one or more characteristics of the channel may also be set to include the number of available channels or similarities.
  • the processor 1602 when the processor 1602 evaluates one or more characteristics of one or more channels, the processor 1602 can be configured to:
  • One or more characteristics of the channel are evaluated based on the signal strength or position of the movable object.
  • the processor 1602 may be configured to:
  • Decomposing the video data is determined according to one or more evaluation characteristics of the one or more channels, and the video data is decomposed into a plurality of sub-video data units according to the decomposition manner.
  • the foregoing decomposition manner includes at least the number of the sub-video data units after the decomposition.
  • an achievable manner is: processing The device 1602 is configured to:
  • the video data is decomposed into a plurality of sub-video data according to a decomposition manner such that the sub-video data units have similar characteristics.
  • processor 1602 is configured to:
  • the video data is decomposed into a plurality of sub-video data according to a decomposition manner such that the sub-video data units have different characteristics.
  • the different characteristics of the sub video data unit include: the size of the code stream data after the sub video data unit is encoded, or the energy concentration of the sub video data unit.
  • one implementation manner of implementing one or more of the selected channels to transmit the encoded sub-video data unit is: when the processor 1602 selects one or more of the channels to transmit the encoded sub-video data unit, Can be configured to:
  • One or more of the channels are selected to transmit the sub-video data unit group.
  • each sub-video data unit group may include one or more sub-video data units; one way of implementing the selection channel is: the channel is a code stream data size and channel that can be according to the data unit group Bandwidth selection; another way to implement channel selection may be: the channel is selected according to the priority of the sub-video data unit group and the channel bandwidth.
  • the processor 1602 may be further configured to: send information for decomposing the video data into a plurality of sub-video data units, wherein the information of the decomposition mode may be included in the information of the plurality of sub-video data units.
  • the information of the above decomposition mode may be encoded in a special field of the sub-picture data; at this time, the processor 1602 may be configured to encode the information of the decomposition mode in a special field of the sub-picture data in the sub-video data unit.
  • the processor 1602 may be configured to: when the video data is decomposed into information of a decomposition mode of the plurality of sub video data units, configured to transmit by using a separate channel before transmitting the plurality of sub video data units. Decomposition method information.
  • the processor 1602 is further configured to control the plurality of encoders to encode the plurality of sub-video data units when the information of the decomposition mode is encoded in a special field of the sub-picture data in the sub-video data unit.
  • the processor 1602 is specifically configured to control multiple encoders to perform parallel encoding on multiple sub-video data units;
  • the second achievable manner is that the processor 1602 is specifically configured to control multiple encoders to encode multiple sub-video data units by using different video coding rules.
  • a third achievable manner is that the processor 1602 is specifically configured to control multiple encoders to encode multiple sub-video data units by using the same video coding rule.
  • a fourth achievable manner is that the processor 1602 is further configured to control the encoder to encode two or more of the plurality of sub-video data units.
  • a fifth achievable manner is that the processor 1602 is further configured to control the encoder to encode at least one of the plurality of sub-video data units based on the motion compensated video compression standard.
  • the processor 1602 may be configured to compress the plurality of sub-video data units according to different compression ratios; wherein the compression ratio is based on one or a sub-video data unit Multiple characteristics are determined.
  • the movable object may be an unmanned aerial vehicle, and one or more imaging devices may be connected to the movable object through a carrier, wherein the carrier It can be a multi-axis universal joint.
  • the processor 1602 separately decomposes the video data into a plurality of sub-video data units, and separately encodes the plurality of sub-video data units according to channel characteristics and characteristics of the sub-video data units. Selecting at least one of the plurality of channels to transmit the encoded sub-video data unit, and when selecting at least one of the plurality of channels to transmit the one or more encoded sub-video data units, causing the sub-video data
  • the unit can transmit on the channel matched with it, and improve the transmission efficiency of the video data while expanding the bandwidth of the video transmission.
  • the video receiving device reconstructs the video data by using the sub-video data unit received from the multiple channels, Improve the fault tolerance and reliability of video transmission.
  • FIG. 17 is a structural diagram of a video receiving system suitable for a movable object according to an embodiment of the present invention.
  • the embodiment provides a video receiving system suitable for a movable object, where the video is provided.
  • the receiving system can be disposed on the receiving device, wherein the receiving device can be a remote controller, a smart phone, a wearable device (watch, wristband), a ground control station, a laptop, etc., and a combination, the video receiving system 1700 is configured to execute
  • the video receiving method specifically, the video receiving system includes:
  • a communication interface 1701 a plurality of encoded sub-video data units received from one or more channels;
  • One or more processors 1702 working alone or in concert, the one or more processors 1702 are configured to perform decoding of the received plurality of encoded sub-video data units, reconstructing from the decoded sub-video data units Video data; wherein the video data includes one or more image frames, and the sub-video data unit includes at least one of the plurality of sub-images obtained by decomposing each of the image frames.
  • the processor 1702 may be configured to separately decode the plurality of encoded sub-video data units.
  • the processor 1702 may be configured to: detect a transmission error of one or more sub-images of the decoded sub-video data unit, and reconstruct according to receiving the correct sub-image. Video data; in order to ensure the quality and the winning rate of the received image, the processor 1702 is further configured to: transmit the wrong sub-segment to the decoded sub-video data unit
  • the image is assigned a value, and specifically, a value assigned to the sub-picture transmitted with errors in the decoded sub-video data unit may be zero.
  • the processor 1702 when the processor 1702 assigns a value to the sub-picture data transmission error in the decoded sub-video data unit, the processor 1702 may be configured to: determine, by using an interpolation method, the sub-image allocated to the transmission error in the decoded sub-video data unit. Value.
  • the processor 1702 may be configured to: determine, according to the sub-image that is correctly transmitted, the decoded sub-video data unit. The value assigned to the erroneous sub-image is transmitted, wherein the erroneous sub-image is transmitted and the correct sub-image is transmitted from the same image frame.
  • the processor 1702 may be configured to reconstruct the video data by using an inverse transform.
  • the video receiving system for the movable object receives the plurality of encoded sub-video data units from the plurality of channels by the processor 1702, and then separately decodes the plurality of sub-video data units to obtain the decoded sub-s a video data unit, and reconstructing the original image according to the decoded sub-video data unit, thereby realizing that a reconstructed image without mosaic error can be obtained as long as one or more channel data is correctly received; the more correctly received channels, The final reconstructed image quality is higher; when all channels are correctly received, the maximum reconstructed image quality is obtained and the transmitted original image is consistent, thereby improving the fault tolerance of the receiving system in reconstructing the image and enhancing the robustness of the video receiving system. Sex.
  • This embodiment provides an unmanned aerial vehicle, including:
  • a power system installed in the fuselage to provide flight power
  • the unmanned aerial vehicle provided in this embodiment is configured to provide a video transmission system, wherein the processor in the video transmission system can decompose the video data into a plurality of sub-video data units, and separately encode the plurality of sub-video data units according to the channel. Characteristics and characteristics of the sub-video data unit, selecting at least one of the plurality of channels to transmit the encoded sub-video data unit, when selecting a plurality of signals When at least one channel of the channel transmits one or more encoded sub-video data units, the sub-video data unit can be transmitted on the channel matched thereto, and the transmission of the video data is improved while expanding the bandwidth of the video transmission. The efficiency, at the same time, the video receiving device reconstructs the video data by using the sub-video data units received from the plurality of channels, which can improve the fault tolerance and reliability of the video transmission.
  • the embodiment provides a receiving device, including: the above video receiving system, for example, the receiving device may be a remote controller, a smart phone, a tablet computer, a ground control station, a laptop computer, a watch, a wristband, etc., and combinations thereof.
  • the unmanned aerial vehicle can be controlled on the ground.
  • the related apparatus and method disclosed may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the modules or units is only a logical function division.
  • there may be another division manner for example, multiple units or components may be used. Combined or can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may contribute to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a software product.
  • the storage medium includes instructions for causing a computer processor 101 to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un procédé de transmission vidéo, un procédé de réception, et un système pour utilisation dans un objet mobile, et un véhicule aérien sans pilote. Le procédé de transmission de vidéo consiste à : évaluer une ou plusieurs caractéristiques d'un ou de plusieurs canaux ; segmenter des données vidéo en de multiples unités de données vidéo secondaires en fonction de l'une ou des plusieurs caractéristiques du ou des canaux, chaque unité de données de vidéo secondaire comprenant une ou plusieurs images secondaires ; effectuer respectivement un codage sur les multiples unités de données de vidéo secondaire ; et sélectionner un ou plusieurs des canaux pour transmettre les unités de données de vidéo secondaire codées. Dans le procédé de transmission vidéo, le procédé de réception, et le système pour utilisation dans un objet mobile, et le véhicule aérien sans pilote, au moins un canal est sélectionné parmi de multiples canaux pour transmettre une ou plusieurs unités de données de vidéo secondaire codées, de telle sorte que les unités de données de vidéo secondaire puissent être transmises sur des canaux correspondants, élargissant ainsi une bande passante pour transmission vidéo tout en améliorant l'efficacité de transmission de données vidéo, la tolérance aux défaillances et la fiabilité de transmission vidéo.
PCT/CN2017/078867 2017-03-30 2017-03-30 Procédé de transmission vidéo, procédé de réception, système, et véhicule aérien sans pilote WO2018176340A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2017/078867 WO2018176340A1 (fr) 2017-03-30 2017-03-30 Procédé de transmission vidéo, procédé de réception, système, et véhicule aérien sans pilote
CN201780005004.0A CN108513712A (zh) 2017-03-30 2017-03-30 视频发送方法、接收方法、系统以及无人飞行器

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/078867 WO2018176340A1 (fr) 2017-03-30 2017-03-30 Procédé de transmission vidéo, procédé de réception, système, et véhicule aérien sans pilote

Publications (1)

Publication Number Publication Date
WO2018176340A1 true WO2018176340A1 (fr) 2018-10-04

Family

ID=63375426

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/078867 WO2018176340A1 (fr) 2017-03-30 2017-03-30 Procédé de transmission vidéo, procédé de réception, système, et véhicule aérien sans pilote

Country Status (2)

Country Link
CN (1) CN108513712A (fr)
WO (1) WO2018176340A1 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110017575A (zh) * 2019-04-16 2019-07-16 珠海格力电器股份有限公司 信息传输装置、方法和系统、空调
CN114731422A (zh) * 2019-12-17 2022-07-08 华为技术有限公司 数据传输方法及装置
CN111865367B (zh) * 2020-06-12 2021-11-09 惠州拓邦电气技术有限公司 一种rfid数据传输方法、装置、电子设备及存储介质
CN112466083B (zh) * 2020-10-15 2023-01-31 中船重工远舟（北京）科技有限公司 一种船用火灾监测报警方法及系统
WO2022141121A1 (fr) * 2020-12-29 2022-07-07 深圳市大疆创新科技有限公司 Procédé de transmission d'image, plateforme mobile, dispositif de commande à distance, système, et support de stockage
CN114245181A (zh) * 2021-12-16 2022-03-25 北京数码视讯技术有限公司 混合信道下的多媒体数据传输方法和装置
CN114040226B (zh) * 2022-01-10 2022-03-11 北京小鸟科技股份有限公司 低带宽高分辨率视频传输的数据传输方法、系统及设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090161773A1 (en) * 2007-12-20 2009-06-25 Advanced Micro Devices, Inc. Channel Estimation Of Orthogonal Frequency Division Multiplexed Systems
US20130263202A1 (en) * 2010-06-22 2013-10-03 Gi Provision Limited Data stream rate adaptation mechanism
WO2013128010A9 (fr) * 2012-03-02 2014-07-03 Canon Kabushiki Kaisha Procédé et dispositifs pour coder une séquence d'images en un flux binaire vidéo hiérarchique, et décoder un flux binaire vidéo hiérarchique correspondant
CN104427302A (zh) * 2013-08-28 2015-03-18 北京计算机技术及应用研究所 一种超高清视频图像的多信道传输子系统及传输方法
CN105120230A (zh) * 2015-09-15 2015-12-02 成都时代星光科技有限公司 无人机图像监控和传输系统
CN106411838A (zh) * 2016-06-14 2017-02-15 青岛乾元通数码科技有限公司 一种多信道负载均衡音视频传输方法及系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008142150A (ja) * 2006-12-07 2008-06-26 Matsushita Electric Ind Co Ltd 医療端末および医療端末の制御方法
GB2501115B (en) * 2012-04-13 2015-02-18 Canon Kk Methods for segmenting and encoding an image, and corresponding devices

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090161773A1 (en) * 2007-12-20 2009-06-25 Advanced Micro Devices, Inc. Channel Estimation Of Orthogonal Frequency Division Multiplexed Systems
US20130263202A1 (en) * 2010-06-22 2013-10-03 Gi Provision Limited Data stream rate adaptation mechanism
WO2013128010A9 (fr) * 2012-03-02 2014-07-03 Canon Kabushiki Kaisha Procédé et dispositifs pour coder une séquence d'images en un flux binaire vidéo hiérarchique, et décoder un flux binaire vidéo hiérarchique correspondant
CN104427302A (zh) * 2013-08-28 2015-03-18 北京计算机技术及应用研究所 一种超高清视频图像的多信道传输子系统及传输方法
CN105120230A (zh) * 2015-09-15 2015-12-02 成都时代星光科技有限公司 无人机图像监控和传输系统
CN106411838A (zh) * 2016-06-14 2017-02-15 青岛乾元通数码科技有限公司 一种多信道负载均衡音视频传输方法及系统

Also Published As

Publication number Publication date
CN108513712A (zh) 2018-09-07

Similar Documents

Publication Publication Date Title
WO2018176340A1 (fr) Procédé de transmission vidéo, procédé de réception, système, et véhicule aérien sans pilote
CN111492660B (zh) 用于有损无线链路的视频编解码器数据恢复技术
US20220261616A1 (en) Clustering-based quantization for neural network compression
EP2622865B1 (fr) Procédé et appareil pour un codage vidéo à résolution arbitraire au moyen de mesures d'échantillonnage compressif
CN103873877A (zh) 远程桌面的图像传输方法及装置
WO2018176303A1 (fr) Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote
US20250119579A1 (en) Coordinate refinement and upsampling from quantized point cloud reconstruction
US20220360778A1 (en) Methods and apparatus for kernel tensor and tree partition based neural network compression framework
CN114884549A (zh) 基于深度学习的大规模mimo信道状态信息反馈方法
US10091518B2 (en) Systems and methods for transmission of compressive sensing data
CN102158312A (zh) 基于信号空间对齐的共信道多用户干扰抑制方法
WO2024220568A1 (fr) Codage prédictif génératif pour compression de nuage de points
WO2018176341A1 (fr) Procédé de transmission vidéo, procédé de réception, système et véhicule aérien sans pilote
KR101455553B1 (ko) 압축 측정을 사용하는 비디오 코딩
TWI669921B (zh) 基於深度學習作為通道狀態資訊之回饋方法
US10390025B2 (en) Method and apparatus for encoding and decoding video signal using improved prediction filter
WO2024102920A1 (fr) Autocodeurs maillés hétérogènes
WO2023155652A1 (fr) Procédé de codage, procédé de décodage et appareils associés
Yang et al. Adaptive error concealment for temporal–spatial multiple description video coding
RU2803766C2 (ru) Способ и устройство для кодирования/реконструкции атрибутов точек облака точек
EP4325754A1 (fr) Procédés de codage et de décodage, appareil de communication et système
WO2024216504A1 (fr) Procédé de compression de données, procédé de décompression de données, et dispositif associé
RU2316908C2 (ru) Способ совместного использования векторного квантования и фрактального кодирования изображений
CN104602027A (zh) 基于标量量化分割的视频多描述编码方法及系统
WO2025080594A1 (fr) Caractéristique d'arbre octal pour compression de nuage de points basée sur des caractéristiques profondes

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17903030

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17903030

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载