CN108174240A - Panoramic video playback method and system based on user location - Google Patents

Abstract

The invention provides a method and system for playing panoramic video based on user position. The method includes: acquiring the location of the user terminal; selecting one of multiple camera locations as a target camera location, wherein each of the multiple camera locations is provided with a panoramic camera, and the panoramic cameras are suitable for capturing images within their respective predetermined ranges. The panoramic video within; according to the location of the user terminal and the target camera position, determine the coordinates of the calibration points in the panoramic video shot by the panoramic camera at the target camera position; in the panoramic video shot by the panoramic camera at the target camera position, select A video picture including the calibration point and having a preset viewing area size is used as a video picture displayed by the user terminal. By using the above method and system, the user can quickly and accurately watch the video display picture at his location without manually adjusting the angle of view when watching the panoramic video, which overcomes the shortcomings of the prior art.

Description

Panoramic video playback method and system based on user location

technical field

The invention relates to the field of image processing, in particular to a method and system for playing panoramic video based on user positions.

Background technique

At present, most video playback terminals already have a panoramic video playback function. When the user watches the panoramic video, the viewing angle of the panoramic video may be adjusted as required. However, the traditional panoramic video is played from a unified viewing angle. Even with a free viewing angle, viewers are easy to get lost in the panoramic viewing angle and cannot find the north, and cannot find the target they really want to watch in the sliding screen viewing.

Contents of the invention

A brief overview of the invention is given below in order to provide a basic understanding of some aspects of the invention. It should be understood that this summary is not an exhaustive overview of the invention. It is not intended to identify key or critical parts of the invention nor to delineate the scope of the invention. Its purpose is merely to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.

In view of this, the present invention provides a method and system for playing panoramic video based on the user's location, to at least solve the problem that the user needs to manually adjust the viewing angle when watching the panoramic video and cannot quickly and accurately watch the video display screen at his location.

According to one aspect of the present invention, a panoramic video playback method based on user location is provided, the method comprising: obtaining the location of the user terminal; selecting one of multiple camera locations as the target camera location, wherein the multiple camera locations Each of them is provided with a panoramic camera respectively, and the panoramic camera is suitable for collecting panoramic video within a predetermined range; according to the position of the user terminal and the target camera position, determine the target in the panoramic video captured by the panoramic camera at the target camera position. The coordinates of the fixed point; in the panoramic video taken by the panoramic camera at the target camera position, select a video picture containing the marked point and having a preset field of view size as the video picture displayed by the user terminal.

According to another aspect of the present invention, there is also provided a panoramic video playback system based on the user's position, the system includes: a position acquisition module, adapted to acquire the position of the user terminal; a selection module, adapted to select one of multiple camera positions As the target shooting position, wherein, each of the plurality of shooting positions is provided with a panoramic camera, and the panoramic camera is suitable for collecting panoramic video within a predetermined range; position, determine the coordinates of the calibration point in the panoramic video shot by the panoramic camera at the target camera position; A video picture with a preset viewing area size is used as a video picture displayed by the user terminal.

When the user uses the panoramic video playback method and system based on the user's position of the present invention, he can quickly and accurately watch the video display screen at his location without manually adjusting the viewing angle when watching the panoramic video, which overcomes the shortcomings of the prior art .

Compared with the prior art, the user position-based panoramic video playback method and system of the present invention, when multiple multi-view panoramic cameras are placed in various positions of the scenic spot (or other scenes, such as stadiums), multiple users will The terminal can arbitrarily connect to a certain panoramic camera in the scenic area (or other scenes, such as a stadium) to enter the panoramic video screen, and quickly locate the video screen at its location at a certain moment when watching the panoramic video, that is, the video screen viewed contains the The video screen of the user terminal location. It should be understood that the system deployment location is not limited to the scenic area. In addition, the panoramic camera, the server, and the user are, for example, within the same local area network.

When multiple multi-view panoramic cameras are placed in various locations in the scenic area, multiple user terminals can connect to any panoramic camera at a certain location in the scenic area to watch the panoramic video, and quickly switch the screen to the scene that contains their location at a certain moment when watching the panoramic video. The video footage of the location.

These and other advantages of the present invention will be more apparent through the following detailed description of the preferred embodiments of the present invention with reference to the accompanying drawings.

Description of drawings

The present invention can be better understood by referring to the following description given in conjunction with the accompanying drawings, wherein the same or similar reference numerals are used throughout to designate the same or similar parts. The accompanying drawings, together with the following detailed description, are incorporated in and form a part of this specification, and serve to further illustrate preferred embodiments of the invention and explain the principles and advantages of the invention. In the attached picture:

FIG. 1 is a flow chart schematically showing an exemplary process of the user position-based panoramic video playback method of the present invention;

Fig. 2 is a structural diagram schematically showing an example of the user position-based panoramic video playback system of the present invention.

It will be appreciated by those skilled in the art that elements in the figures are illustrated for simplicity and clarity only and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of the embodiments of the present invention.

Detailed ways

Exemplary embodiments of the present invention will be described below with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in this specification. It should be understood, however, that in developing any such practical embodiment, many implementation-specific decisions must be made in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and those Restrictions may vary from implementation to implementation. Moreover, it should also be understood that development work, while potentially complex and time-consuming, would at least be a routine undertaking for those skilled in the art having the benefit of this disclosure.

Here, it should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only the device structure and/or processing steps closely related to the solution according to the present invention are shown in the drawings, and the Other details not relevant to the present invention are described.

An embodiment of the present invention provides a method for playing a panoramic video based on a user location, the method comprising: obtaining a location of a user terminal; selecting one of multiple camera locations as a target camera location, wherein one of the multiple camera locations Each is provided with panoramic camera respectively, and described panoramic camera is suitable for collecting the panoramic video in respective predetermined scope; According to user terminal position and target camera position, determine the mark point in the panoramic video shot by the panoramic camera at the target camera position Coordinates; in the panoramic video shot by the panoramic camera at the target camera position, select a video frame that includes the calibration point and has a preset field of view size as the video frame displayed by the user terminal.

FIG. 1 shows an exemplary processing flow of the user location-based panoramic video playback method of the present invention.

In the following, an embodiment of the present invention will be described by taking the process shown in FIG. 1 as an example. The method for playing a panoramic video based on the user's location may be executed, for example, on a server side.

As shown in FIG. 1, in step S110, the location of the user terminal is obtained.

According to an embodiment of the present invention, the location of the user terminal may be acquired in response to the user terminal accessing the panoramic video. In this way, at the initial moment when the user accesses the panoramic video, the viewing angle of the panoramic video starts to be adjusted.

Wherein, the moment when "the user terminal accesses the panoramic video" is, for example, when the server establishes a connection with the user terminal.

According to an embodiment of the present invention, the location of the user terminal may also be acquired in response to the find-yourself instruction sent by the user terminal. In this way, when the user sends an adjustment instruction at any other time, the position of the user terminal may be relocated to adjust the viewing angle of the panoramic video.

Wherein, the location of the user terminal is obtained, for example, through a positioning module (such as a GPS module) on the user terminal, such as when the user terminal accesses the panoramic video, or when the user terminal sends an instruction to find itself, the user terminal is requested to obtain its current position through positioning. Location.

According to an embodiment of the present invention, in step S110, video frame information may also be acquired.

According to an implementation manner, the video frame information may include a corresponding time stamp and/or frame sequence number when the user terminal accesses the panoramic video (that is, the sequence number of the displayed video frame when the user terminal accesses the panoramic video).

According to another implementation, the video frame information also includes the corresponding time stamp and/or frame sequence number when receiving the find-yourself instruction from the user terminal (that is, when receiving the find-yourself instruction from the user terminal, the The sequence number of the displayed video frame).

Wherein, the video frame information may be acquired from a user terminal, for example. For example, the user terminal may also send video frame information while sending the location.

In step S120, one of the multiple shooting positions is selected as the target shooting position, wherein each of the multiple shooting positions is provided with a panoramic camera, and the panoramic camera is suitable for collecting panoramic videos within a respective predetermined range.

According to an embodiment of the present invention, one of multiple camera locations can be selected as the target camera location according to the following process: all panoramic camera numbers are sent to the user terminal for the user terminal to select a panoramic camera number; The panoramic camera number selected by the terminal uses the shooting position corresponding to the panoramic camera number as the target shooting position. In this way, instead of displaying the panoramic camera distribution map on the user terminal, only the panoramic camera number is displayed, and the user terminal can independently select the panoramic camera to access and adjust the viewing angle of the panoramic video.

According to an embodiment of the present invention, it is also possible to select one of a plurality of camera positions as the target camera position according to the following process: send part or all of the camera positions to the user terminal to display the distribution of part or all of the panoramic cameras on the user terminal State: according to the camera location selected by the user terminal, the camera location is used as the target camera location. Wherein, the "part or all of the camera positions" in sending part or all of the camera positions to the user terminal is, for example, all of the camera positions, or it may be a part (such as a predetermined number, or a distance within a predetermined range) that is closer to the user terminal position. multiple) camera positions, etc. In this way, by displaying the distribution map of the panoramic cameras, the user terminal can independently select the panoramic cameras to access and adjust the viewing angle of the panoramic video.

In addition, according to an embodiment of the present invention, based on the location of the user terminal, the imaging location closest to the location of the user terminal among the plurality of imaging locations may be used as an initial default target imaging location. In this way, according to the user's position, the panoramic camera closest to the user's position can be automatically connected to enter the panoramic video, and the viewing angle of the panoramic video can be adjusted.

In step S130, according to the location of the user terminal and the target shooting position, the coordinates of the marked points in the panoramic video shot by the panoramic camera at the target shooting position are determined.

Wherein, each of the plurality of camera positions is respectively provided with a panoramic camera, and the panoramic camera is suitable for collecting panoramic videos within respective predetermined ranges.

In step S140 , from the panoramic video shot by the panoramic camera at the target shooting position, a video frame including a calibration point and having a preset viewing area size is selected as a video frame displayed by the user terminal.

According to an embodiment of the present invention, the selected video frame containing the calibration point and having the preset field of view size in the panoramic video taken by the panoramic camera at the target camera position is from the corresponding time stamp and/or frame sequence number. The video frame starts. That is to say, when the user terminal accesses the panoramic video, or when the user terminal sends an instruction to find itself, the angle of the panoramic video is adjusted through the above-mentioned processing of the present invention, and the video picture sent to the terminal after adjustment is different from the The last displayed frame before adjustment may be the same frame (or may be the next frame of this frame), so as to ensure the continuity of its viewing. It should be understood that, in other examples, the video picture sent to the user terminal after adjustment may also be discontinuous with the last frame of picture displayed by the user terminal before adjustment, for example, a real-time picture is displayed after adjustment.

According to an embodiment of the present invention, in step S130, for example, (X _w , Y _w , Z _w ) can be used to represent the coordinates of the user terminal position, so that the panoramic camera at the target shooting position can be calculated according to the following formula 1 The coordinates (X ₁ , Y ₁ ) of the calibration point in the corresponding panoramic image frame in the panoramic video:

Formula one:

Among them, Zc is a scale parameter, and f is the focal length of the panoramic camera at the target shooting position.

In addition, the above-mentioned "selecting a video frame including a marking point and having a preset viewing area size" is, for example, a video frame centered on the marking point and having a preset viewing area size.

According to an embodiment of the present invention, in step S140, for example, the calibration point (X ₁ , Y ₁ ) can be used as the center of the image, and the fixed dimensions W and H can be used as the length and width of the rectangular area, wherein the size of the rectangular area is preset Set the size of the field of view, and perform orthographic transformation on the rectangular area to obtain a spherical or cylindrical display area. The obtained spherical or cylindrical display area is subjected to back projection transformation to obtain a planar rectangular view corresponding to the spherical or cylindrical display area as a video image displayed by the user terminal.

According to an implementation manner, when the spherical display area is obtained through the orthographic transformation of the rectangular area, the projection transformation formula of the plane image coordinates and the spherical coordinates can refer to the following formula 2, for example:

Formula two:

Wherein, X' and Y' are spherical coordinates corresponding to plane coordinates (X ₁ , Y ₁ ).

Wherein, the aspect ratio of the rectangle corresponding to the preset viewing area size may, for example, be consistent with the aspect ratio of the video picture played by the user terminal.

According to an embodiment of the present invention, the method may further include: obtaining the ratio of the screen length and width of the user terminal as a reference ratio; The aspect ratio of the rectangular size corresponding to the viewing area size is consistent with the reference ratio. For example, assuming that the ratio of the length to width of the screen of the user terminal is 16:9, and the aspect ratio in the default default viewing area size is, for example, 4:3, a 16:9 screen can be cropped in the default viewing area size , the largest area, as the adjusted preset viewing area size. In this way, the adjusted preset viewing area size is also in the ratio of 16:9, which can make the user terminal in the "full-screen playback" panoramic video mode, without the user There are upper and lower black borders or left and right black borders on the terminal display screen to achieve true full-screen display. Alternatively, the preset viewing area size may also be scaled (such as by stretching or shrinking the long side or wide side, etc.) so that the aspect ratio of the adjusted preset viewing area size is equal to the above-mentioned reference ratio, so that the user terminal is in " In the panoramic video mode of "full-screen playback", there will be no black borders up and down or black borders left and right on the display screen of the user terminal.

According to an embodiment of the present invention, the method may further include: responding to the photographing request of the user terminal, intercepting the image part displayed by the user terminal in real time in the video frame image corresponding to the photographing request, and sending it to the user terminal as a photographing result. In this way, when the user terminal sends a photo-taking request, the image displayed on the device can be intercepted in real time for the user to save, so that framing and taking photos can be realized, or self-portraits can be realized.

Furthermore, according to an embodiment of the present invention, the plurality of camera positions may be fixed or movable. In this way, the panoramic cameras at multiple camera positions may all be fixed, or all may be movable, or some positions may be fixed, while others may be movable, etc. .

Wherein, the shooting position and/or posture corresponding to the fixed panoramic camera is, for example, pre-stored on the server side. The shooting position and/or posture corresponding to the panoramic camera with a movable position can be acquired in real time, for example, by a position sensor and/or posture sensor provided on the corresponding panoramic camera. The position sensor can be, for example, GPS and/or a barometric altimeter or the like. The attitude sensor may be, for example, an accelerometer and/or a gyroscope or the like.

In addition, the embodiment of the present invention also provides a panoramic video playback system based on the user's position, the system includes: a position acquisition module, adapted to acquire the position of the user terminal; a selection module, adapted to select one of multiple camera positions as the The target camera position, wherein each of the plurality of camera positions is provided with a panoramic camera, and the panoramic camera is suitable for collecting panoramic videos within a predetermined range; the determining module is adapted to determine the target camera position according to the user terminal position and the target camera position The coordinates of the marked points in the panoramic video shot by the panoramic camera at the position; the marking module is suitable for selecting the frame that contains the marked points and has a preset field of view size in the panoramic video shot by the panoramic camera at the target camera position The video picture is used as a video picture displayed by the user terminal.

FIG. 2 shows a schematic structural diagram of an example of a panoramic video playback system based on user positions. As shown in FIG. 2 , the system includes a position acquisition module 210 , a selection module 220 , a determination module 230 and a calibration module 240 .

The location obtaining module 210 is used to obtain the location of the user terminal.

The selection module 220 is used to select one of multiple camera locations as a target camera location, wherein each of the multiple camera locations is provided with a panoramic camera, and the panoramic cameras are suitable for collecting panoramic videos within respective predetermined ranges.

The determining module 230 is configured to determine the coordinates of the marked points in the panoramic video shot by the panoramic camera at the target shooting position according to the position of the user terminal and the target shooting position.

The marking module 240 is used to select, from the panoramic video shot by the panoramic camera at the target shooting position, a video frame including a calibration point and having a preset field of view size as a video frame displayed by the user terminal.

According to an embodiment of the present invention, the position acquisition module 210 , the selection module 220 , the determination module 230 and the calibration module 240 included in the user position-based panoramic video playback system may be set on the server side, for example. According to other embodiments, the above-mentioned user position-based panoramic video playback system may also include multiple panoramic cameras and/or one or more user terminals arranged at multiple camera positions, in addition to the above-mentioned modules on the server side.

According to an embodiment of the present invention, the determination module 230 may, for example, use (X _w , Y _w , Z _w ) to represent the coordinates of the user terminal location, and calculate the corresponding panoramic video in the panoramic video captured by the panoramic camera at the target shooting position according to Formula 1. The coordinates (X ₁ , Y ₁ ) of the calibration point in the image frame.

According to an embodiment of the present invention, the above-mentioned video frame including the marking point and having a preset viewing area size is, for example, a video frame centered on the marking point and having a preset viewing area size.

According to an embodiment of the present invention, the calibration module 240 may, for example, take the calibration point (X ₁ , Y ₁ ) as the center of the image, and use the fixed dimensions W and H as the length and width of the rectangular area, wherein the size of the rectangular area is the preset viewing angle. Domain size, transform the rectangular area by orthographic projection to obtain a spherical or cylindrical display area; transform the obtained spherical or cylindrical display area through back-projection transformation to obtain a planar rectangular view corresponding to the spherical or cylindrical display area, as the user The video screen displayed by the terminal.

Wherein, the projection transformation formula of the plane image coordinates and the spherical coordinates, for example, can adopt formula two.

According to an embodiment of the present invention, the aspect ratio of the rectangle corresponding to the preset viewing area size may, for example, be consistent with the aspect ratio of the video picture played by the user terminal.

According to an embodiment of the present invention, the calibration module 240 may obtain the ratio of the screen length and width of the user terminal as a reference ratio; adjust the size of the preset viewing area by zooming and/or cropping, so that the adjusted preset viewing area The aspect ratio of the rectangle size corresponding to the field size is consistent with the reference ratio.

According to an embodiment of the present invention, the position obtaining module 210 may obtain the position of the user terminal, for example, in response to the user terminal accessing the panoramic video.

According to an embodiment of the present invention, the location obtaining module 210 may, for example, obtain the location of the user terminal in response to a find-yourself instruction sent by the user terminal.

According to an embodiment of the present invention, the selection module 220 may, for example, send all the panoramic camera numbers to the user terminal for the user terminal to select a panoramic camera number; according to the panoramic camera number selected by the user terminal, the panoramic camera number The corresponding shooting position is taken as the target shooting position.

According to an embodiment of the present invention, the selection module 220 may, for example, send part or all of the camera locations to the user terminal to display the distribution status of some or all of the panoramic cameras on the user terminal; position as the target camera position.

According to an embodiment of the present invention, the selection module 220 may, for example, based on the location of the user terminal, use the imaging location closest to the location of the user terminal among the plurality of imaging locations as an initial default target imaging location.

According to an embodiment of the present invention, for example, the system may further include a photographing module (not shown in the figure), and the photographing module may, for example, respond to a photographing request of a user terminal, and capture real-time images of the user terminal in the video frame image corresponding to the photographing request. The displayed image part is sent to the user terminal as a photographing result.

According to an embodiment of the present invention, the plurality of camera positions may be fixed or movable.

According to an embodiment of the present invention, the shooting position and/or posture corresponding to the fixed panoramic camera may be pre-stored on the server side, for example.

According to an embodiment of the present invention, for example, a position sensor and/or an attitude sensor may be provided on the panoramic camera with a movable position, and the corresponding shooting position and/or attitude of the panoramic camera with a movable position may, for example, be determined by the position sensor disposed thereon. And/or attitude sensors are acquired in real time.

According to an embodiment of the present invention, the position sensor may include, for example, a GPS module and/or a barometric altimeter.

According to an embodiment of the present invention, the attitude sensor may include, for example, an accelerometer and/or a gyroscope.

According to an embodiment of the present invention, the system may also include an information acquisition module (not shown in the figure), for example, the information acquisition module may acquire video frame information (such as from the user terminal), and the video frame information includes, for example, the user terminal access panorama The time stamp and/or frame sequence number corresponding to the video, or including the time stamp and/or frame sequence number corresponding to when receiving an instruction to find yourself from the user terminal.

According to an embodiment of the present invention, in the panoramic video taken by the panoramic camera at the target camera position, the selected video frame containing the calibration point and having a preset field of view size is, for example, from a video with a time stamp and/or a frame sequence number The corresponding video frame starts.

It should be understood that the user location-based panoramic video playback system of the present invention can perform the processing of the user location-based panoramic video playback method described above, and can realize corresponding functions and effects, which will not be repeated here.

preferred embodiment

A preferred embodiment of adjusting the viewing angle of panoramic video in the above method and system of the present invention is described below.

In this embodiment, for example, the current video frame may be determined according to the timestamp when the user terminal accesses the panoramic video, and the viewing angle adjustment area may be determined at the current video frame, that is, at the current moment.

The position information of the panoramic camera at the target shooting position is known. In the calibrated mode, the external parameters and internal parameters of the panoramic camera are known, that is, the principal point of the image and the focal length f are known in the calibrated state.

Obtain the position information (X _w , Y _w , Z _w ) of the user terminal (for example, according to the GPS in the user equipment), and obtain any point in the three-dimensional space (X _w , Y _w , Z _w ) according to the calibrated external parameters and internal parameters The corresponding image coordinates (X ₁ , Y ₁ ), (X ₁ , Y ₁ ) are the image calibration points (distortion not considered), see Formula 1. Among them, _Zc is a scale parameter, _Zc is a direction vector formed by the connection between the user terminal and the panoramic camera, and the direction is from the panoramic camera to the user terminal, which can be calculated according to the location of the user terminal and the location of the panoramic camera at the target shooting position.

Take (X ₁ , Y ₁ ) as the center of the image, take the fixed size W*H as the length and width of the rectangular area, and set the length and width of the rectangular area to be consistent with the aspect ratio of the video screen of the panoramic video playback device; the rectangular area Orthographic transformation is performed to obtain a spherical or cylindrical display area; and then a back-projection transformation is performed to obtain a flat rectangular view. Formula 2 is the projection transformation of image coordinates and spherical coordinates.

Then, the video display screen of the user terminal is adjusted according to the planar rectangular view.

In summary, this application provides the following solutions:

Option 1. The panoramic video playback method based on user position, it is characterized in that, the method comprises:

Obtain the location of the user terminal;

Selecting one of the plurality of camera positions as the target camera position, wherein each of the plurality of camera positions is provided with a panoramic camera, and the panoramic camera is suitable for collecting panoramic videos within a respective predetermined range;

According to the position of the user terminal and the target camera position, determine the coordinates of the calibration points in the panoramic video shot by the panoramic camera at the target camera position;

From the panoramic video shot by the panoramic camera at the target shooting position, a video frame including the calibration point and having a preset field of view size is selected as the video frame displayed by the user terminal.

Scheme 2. according to the panoramic video playing method described in scheme 1, it is characterized in that, according to the user terminal position and the target photographing position, the step of determining the coordinates of the calibration point in the panoramic video taken by the panoramic camera at the target photographing position include:

Use (X _w , Y _w , Z _w ) to represent the coordinates of the user terminal position, and calculate the coordinates (X ₁ , _Y1 ):

Among them, Zc is a scale parameter, and f is the focal length of the panoramic camera at the target shooting position.

Scheme 3. According to the panoramic video playback method described in scheme 2, it is characterized in that the video picture containing the calibration point and having a preset viewing area size is centered on the calibration point and has a preset viewing area. domain-sized video frames.

Scheme 4. according to the panoramic video playing method described in scheme 2 or 3, it is characterized in that, in the panoramic video shot by the panoramic camera at the target camera position, select the one that includes the calibration point and has a preset field of view size of the video picture, as the video picture displayed by the user terminal, the steps include:

Taking the calibration point (X ₁ , Y ₁ ) as the center of the image, and taking the fixed dimensions W and H as the length and width of the rectangular area, wherein the size of the rectangular area is the size of the preset viewing area, the rectangular area is orthographically projected Transform to obtain a spherical or cylindrical display area;

The obtained spherical or cylindrical display area is subjected to back projection transformation to obtain a planar rectangular view corresponding to the spherical or cylindrical display area as a video image displayed by the user terminal.

Scheme 5. according to the panoramic video playing method described in scheme 4, it is characterized in that, the projection transformation formula of plane image coordinates and spherical coordinates is as follows:

Wherein, X' and Y' are spherical coordinates corresponding to plane coordinates (X ₁ , Y ₁ ).

Solution 6. The panoramic video playback method according to solution 5, wherein the aspect ratio of the rectangle corresponding to the preset viewing area size is consistent with the aspect ratio of the video screen played by the user terminal.

Scheme 7. according to the panoramic video playing method described in scheme 5 or 6, it is characterized in that, the method also includes:

Obtain the ratio of the screen length and width of the user terminal as a reference ratio;

The preset viewing area size is adjusted by scaling and/or clipping, so that the aspect ratio of the rectangular size corresponding to the adjusted preset viewing area size is consistent with the reference ratio.

Scheme 8. According to the panoramic video playback method described in any one of schemes 1-7, the step of obtaining the position of the user terminal comprises:

In response to the user terminal accessing the panoramic video, the location of the user terminal is acquired.

Scheme 9. According to the panoramic video playback method described in any one of schemes 1-7, the step of obtaining the position of the user terminal comprises:

In response to the find-yourself instruction sent by the user terminal, the location of the user terminal is acquired.

Scheme 10. according to the panoramic video playing method described in any one of schemes 1-9, it is characterized in that, the step of selecting one as the target photographing position in a plurality of photographing positions comprises:

Send all the panoramic camera numbers to the user terminal, so that the user terminal can select a panoramic camera number among them;

According to the panoramic camera number selected by the user terminal, the shooting position corresponding to the panoramic camera number is used as the target shooting position.

Scheme 11. according to the panoramic video playing method described in any one of schemes 1-9, it is characterized in that, the step of selecting one as the target photographing position in a plurality of photographing positions comprises:

Send part or all of the camera positions to the user terminal to display the distribution status of part or all of the panoramic cameras on the user terminal;

According to the shooting position selected by the user terminal, the shooting position is used as the target shooting position.

Scheme 12. according to the panoramic video playing method described in any one of schemes 1-11, it is characterized in that, the step of selecting one of the plurality of photographing positions as the target photographing position comprises:

Based on the location of the user terminal, the imaging location closest to the location of the user terminal among the plurality of imaging locations is used as an initial default target imaging location.

Scheme 13. According to the panoramic video playback method described in any one of schemes 1-12, it is characterized in that the method also includes:

In response to the photographing request of the user terminal, the image part displayed by the user terminal in real time is intercepted from the video frame image corresponding to the photographing request, and sent to the user terminal as a photographing result.

Solution 14. The panoramic video playback method according to any one of solutions 1-13, characterized in that the multiple camera positions are fixed or movable.

Scheme 15. The panoramic video playback method according to scheme 14, wherein the camera position and/or attitude corresponding to the fixed panoramic camera are pre-stored on the server side; the camera position and/or posture corresponding to the movable panoramic camera Or the attitude is acquired in real time by a position sensor and/or an attitude sensor arranged on the panoramic camera.

Solution 16. The panoramic video playback method according to any one of solutions 1-15, characterized in that the method also includes acquiring video frame information, the video frame information including the corresponding time stamp when the user terminal accesses the panoramic video and/or a frame sequence number, or include a time stamp and/or a frame sequence number corresponding to when an instruction to find oneself is received from a user terminal.

Scheme 17. The method for playing panoramic video according to scheme 16, wherein, in the panoramic video taken by the panoramic camera at the target camera position, the selected video that includes the calibration point and has a preset field of view size The picture starts from the video frame corresponding to the timestamp and/or frame sequence number.

Scheme 18. A panoramic video playback system based on user position, characterized in that the system includes:

A location acquisition module, adapted to acquire the location of the user terminal;

The selection module is adapted to select one of the multiple camera positions as the target camera position, wherein each of the multiple camera positions is provided with a panoramic camera, and the panoramic camera is suitable for collecting panoramic videos within a respective predetermined range ;

The determining module is adapted to determine the coordinates of the calibration points in the panoramic video shot by the panoramic camera at the target shooting position according to the position of the user terminal and the target shooting position;

The marking module is adapted to select, from the panoramic video captured by the panoramic camera at the target camera position, a video picture containing the calibration point and having a preset field of view size as the video picture displayed by the user terminal.

Scheme 19. The panoramic video playback system according to scheme 18, wherein the determination module is adapted to: use ( _Xw , _Yw , _Zw ) to represent the coordinates of the user terminal position, and calculate the target camera position according to the following formula The coordinates (X ₁ , Y ₁ ) of the calibration point in the corresponding panoramic image frame in the panoramic video captured by the panoramic camera at :

Among them, Zc is a scale parameter, and f is the focal length of the panoramic camera at the target shooting position.

Solution 20. The panoramic video playback system according to solution 19, characterized in that, the video frame containing the marking point and having a preset viewing area size is centered on the marking point and has a preset viewing area. domain-sized video frames.

Solution 21. The panoramic video playback system according to solution 19 or 20, wherein the calibration module is suitable for: taking the calibration point (X ₁ , Y ₁ ) as the center of the image, and taking the fixed dimensions W and H as the rectangular area The length and width of the rectangular area, wherein, the size of the rectangular area is the size of the preset viewing area, and the rectangular area is subjected to forward projection transformation to obtain a spherical or cylindrical display area; the obtained spherical or cylindrical display area is subjected to back projection transformation , to obtain a planar rectangular view corresponding to the spherical or cylindrical display area as a video picture displayed by the user terminal.

Scheme 22. According to the panoramic video playback system described in scheme 21, it is characterized in that the projection transformation formula of plane image coordinates and spherical coordinates is as follows:

Wherein, X' and Y' are spherical coordinates corresponding to plane coordinates (X ₁ , Y ₁ ).

Solution 23. The panoramic video playback system according to solution 22, wherein the aspect ratio of the rectangle corresponding to the preset viewing area size is consistent with the aspect ratio of the video screen played by the user terminal.

Solution 24. The panoramic video playback system according to solution 22 or 23, wherein the calibration module is further adapted to: acquire the ratio of the screen length and width of the user terminal as a reference ratio; The preset viewing area size is adjusted so that the aspect ratio of the rectangular size corresponding to the adjusted preset viewing area size is consistent with the reference ratio.

Solution 25. The panoramic video playback system according to any one of solutions 18-24, characterized in that, the position acquiring module is adapted to: acquire the position of the user terminal in response to the user terminal accessing the panoramic video.

Solution 26. The panoramic video playback system according to any one of solutions 18-24, characterized in that, the position acquisition module is adapted to: acquire the position of the user terminal in response to a find-yourself instruction sent by the user terminal.

Scheme 27. The panoramic video playback system according to any one of schemes 18-26, wherein the selection module is adapted to: send all panoramic camera numbers to the user terminal for the user terminal to select one of them Panoramic camera number: according to the panoramic camera number selected by the user terminal, the shooting position corresponding to the panoramic camera number is used as the target shooting position.

Scheme 28. The panoramic video playback system according to any one of schemes 18-26, wherein the selection module is adapted to: send part or all of the camera positions to the user terminal to display part or all of the camera positions on the user terminal The distribution state of all panoramic cameras; according to the shooting position selected by the user terminal, the shooting position is used as the target shooting position.

Scheme 29. The panoramic video playback system according to any one of schemes 18-28, wherein the selection module is adapted to: based on the position of the user terminal, select the nearest position of the plurality of camera positions from the position of the user terminal The camera position is used as the initial default target camera position.

Scheme 30. The panoramic video playback system according to any one of schemes 18-29, characterized in that the system also includes: a photographing module adapted to respond to the photographing request of the user terminal, and the video frame corresponding to the photographing request The part of the image displayed by the user terminal in real time is intercepted from the image, and sent to the user terminal as a photographing result.

Solution 31. The panoramic video playback system according to any one of solutions 18-30, characterized in that the multiple camera positions are fixed or movable.

Solution 32. The panoramic video playback system according to solution 31, characterized in that the shooting position and/or posture corresponding to the fixed panoramic camera is pre-stored on the server side.

Scheme 33. The panoramic video playback system according to scheme 31, wherein the panoramic camera with a movable position is provided with a position sensor and/or an attitude sensor, and the corresponding shooting position and/or attitude of the panoramic camera with a movable position is It is acquired in real time by a position sensor and/or an attitude sensor disposed thereon.

Solution 34. The panoramic video playback system according to solution 33, wherein the position sensor includes a GPS module and/or a barometric altimeter.

Solution 35. The panoramic video playback system according to solution 33 or 34, wherein the attitude sensor includes an accelerometer and/or a gyroscope.

Solution 36. The panoramic video playback system according to any one of solutions 18-35, characterized in that the system further includes: an information acquisition module, adapted to acquire video frame information, and the video frame information includes user terminal access The time stamp and/or frame sequence number corresponding to the panoramic video, or including the time stamp and/or frame sequence number corresponding to when receiving an instruction to find yourself from the user terminal.

Scheme 37. According to the panoramic video playback system described in scheme 36, it is characterized in that the video with the preset viewing area size that includes the calibration point is selected in the panoramic video captured by the panoramic camera at the target camera position. The picture starts from the video frame corresponding to the timestamp and/or frame sequence number.

Solution 38. The panoramic video playback system according to any one of solutions 18-37, characterized in that the system further includes a plurality of panoramic cameras arranged at the plurality of camera positions.

Solution 39. The panoramic video playback system according to any one of solutions 18-38, characterized in that the system further includes one or more user terminals.

While the invention has been described in terms of a limited number of embodiments, it will be apparent to a person skilled in the art having the benefit of the above description that other embodiments are conceivable within the scope of the invention thus described. In addition, it should be noted that the language used in the specification has been chosen primarily for the purpose of readability and instruction rather than to explain or define the inventive subject matter. Accordingly, many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. With respect to the scope of the present invention, the disclosure of the present invention is intended to be illustrative rather than restrictive, and the scope of the present invention is defined by the appended claims.

Claims (10)

1. The panoramic video playback method based on user position, is characterized in that, the method comprises: Obtain the location of the user terminal; Selecting one of the plurality of camera positions as the target camera position, wherein each of the plurality of camera positions is provided with a panoramic camera, and the panoramic camera is suitable for collecting panoramic videos within a respective predetermined range; According to the position of the user terminal and the target camera position, determine the coordinates of the calibration points in the panoramic video shot by the panoramic camera at the target camera position; From the panoramic video shot by the panoramic camera at the target shooting position, a video frame including the calibration point and having a preset field of view size is selected as the video frame displayed by the user terminal. 2. The panoramic video playback method according to claim 1, wherein, according to the user terminal position and the target camera position, the step of determining the coordinates of the marked points in the panoramic video taken by the panoramic camera at the target camera position include: Use (X _w , Y _w , Z _w ) to represent the coordinates of the user terminal position, and calculate the coordinates (X ₁ , _Y1 ): Among them, Zc is a scale parameter, and f is the focal length of the panoramic camera at the target shooting position. 3. The panoramic video playing method according to claim 2, characterized in that, in the panoramic video shot by the panoramic camera at the target camera position, select the one that includes the calibration point and has a preset field of view size The video picture, the steps as the video picture displayed by the user terminal include: Taking the calibration point (X ₁ , Y ₁ ) as the center of the image, and taking the fixed dimensions W and H as the length and width of the rectangular area, wherein the size of the rectangular area is the size of the preset viewing area, the rectangular area is orthographically projected Transform to obtain a spherical or cylindrical display area; The obtained spherical or cylindrical display area is subjected to back projection transformation to obtain a planar rectangular view corresponding to the spherical or cylindrical display area as a video image displayed by the user terminal. 4. panoramic video playing method according to claim 3, is characterized in that, the projection transformation formula of plane image coordinates and spherical coordinates is as follows: Wherein, X' and Y' are spherical coordinates corresponding to plane coordinates (X ₁ , Y ₁ ). 5. The panoramic video playback method according to claim 4, characterized in that, the aspect ratio of the rectangle corresponding to the preset viewing area size is consistent with the aspect ratio of the video picture played by the user terminal. 6. according to claim 4 or 5 described panoramic video playing methods, it is characterized in that, this method also comprises: Obtain the ratio of the screen length and width of the user terminal as a reference ratio; The preset viewing area size is adjusted by scaling and/or clipping, so that the aspect ratio of the rectangular size corresponding to the adjusted preset viewing area size is consistent with the reference ratio. 7. according to the panoramic video playing method described in any one in claim 1-6, it is characterized in that, the described step of selecting one as target camera position in a plurality of camera positions comprises: Send part or all of the camera positions to the user terminal to display the distribution status of part or all of the panoramic cameras on the user terminal; According to the shooting position selected by the user terminal, the shooting position is used as the target shooting position. 8. according to the panoramic video playback method described in any one in claim 1-7, it is characterized in that, the described step of selecting one as target camera position in a plurality of camera positions comprises: Based on the location of the user terminal, the imaging location closest to the location of the user terminal among the plurality of imaging locations is used as an initial default target imaging location. 9. The panoramic video playing method according to any one of claims 1-8, characterized in that, a plurality of camera positions are fixed or movable. 10. The panoramic video playback system based on the user's position is characterized in that the system comprises: A location acquisition module, adapted to acquire the location of the user terminal; The selection module is adapted to select one of the multiple camera positions as the target camera position, wherein each of the multiple camera positions is provided with a panoramic camera, and the panoramic camera is suitable for collecting panoramic videos within a respective predetermined range ; The determining module is adapted to determine the coordinates of the calibration points in the panoramic video shot by the panoramic camera at the target shooting position according to the position of the user terminal and the target shooting position; The marking module is adapted to select, from the panoramic video shot by the panoramic camera at the target shooting position, a video picture containing the calibration point and having a preset field of view size as the video picture displayed by the user terminal.