CN117911597A - A GPU method for efficient colorization of multi-field dense point clouds - Google Patents

Abstract

The present invention discloses a GPU method for efficient coloring of multi-field dense point clouds, comprising a CPU-side preprocessing module and a GPU-side parallel coloring module. The CPU-side preprocessing module comprises: classifying field types into binary fields and non-binary fields; counting the value range of each non-binary field except the color field; performing multi-grouping on the fields; numbering and sorting the fields; encoding a predefined color bar sequence into a one-dimensional image sequence; performing piecewise linear interpolation upsampling on the one-dimensional image sequence; creating a one-dimensional texture sequence corresponding to the one-dimensional image sequence; and passing the above processing results into a graphics rendering pipeline. The GPU-side parallel coloring module comprises: converting vertex coordinates from a global coordinate system to a clipping coordinate system; coloring vertices according to the active field type; performing post-processing on the colored vertices; and coloring the rasterized fragments. The present invention can efficiently present the coloring effects of dense point clouds under different fields and different color bars.

Description

A GPU method for efficient colorization of multi-field dense point clouds

Technical Field

The present invention relates to the visualization and expression of digital twins based on dense point clouds in the fields of computer vision and computer graphics, and in particular to a method for efficiently coloring multi-field dense point clouds by utilizing a GPU (Graphics Processing Unit) parallel processing architecture.

Background technique

In recent years, LiDAR (Light Detection and Ranging) technology has developed rapidly and has been widely used in emerging fields such as digital twins, smart cities, and unmanned driving. Compared with structured light technology and multi-view dense matching technology, the point cloud obtained by LiDAR technology has higher accuracy, higher resolution, and stronger robustness to environmental factors such as lighting and materials. With the rapid upgrade and iteration of software and hardware such as LiDAR transmitters, receivers, inertial navigation technology, positioning technology, data storage devices, and real-time communication modules, the point cloud space collected by various LiDAR platforms (handheld, vehicle-mounted, and airborne, etc.) has a wider range, higher density, and more field attributes.

Real-time visualization of point clouds in different fields is an important supporting technology for deep application analysis such as ground feature inversion, structural analysis, and change monitoring. However, the existing mainstream technology uses the central processing unit (CPU) to update the rendering effect of large scene dense point clouds in different fields point by point, which has low real-time performance and is difficult to meet the timeliness requirements of deep application analysis.

Therefore, in response to the above technical bottlenecks, the present invention proposes a GPU method for efficient coloring of multi-field dense point clouds. The most significant advantage of the present invention is that each point in the dense point cloud is regarded as a vertex in three-dimensional visualization, and the parallel processing architecture of the GPU graphics rendering pipeline is used to quickly present the visualization effect of multi-field dense point clouds in different fields and different color bars. The key technology of the present invention can provide an efficient and fast visualization solution for deep application analysis such as object classification, semantic extraction, and target detection based on dense point clouds.

Summary of the invention

In order to overcome the shortcomings of the prior art, the present invention proposes a GPU method for efficient coloring of multi-field dense point clouds, which uses the parallel processing capability of the GPU to quickly present the visualization effects of multi-field dense point clouds in different fields and different color bars, and provides visualization technical support for subsequent application analysis. The features of the present invention include two aspects: a CPU-side preprocessing module and a GPU-side parallel coloring module.

The CPU-side preprocessing module comprises:

Step 1, classify the field types into binary fields and non-binary fields;

Step 2, counting the value range of each non-binary field except the color field;

Step 3, perform multi-grouping on the fields;

Step 4, sort the fields by number;

Step 5, encoding the predefined color bar sequence into a one-dimensional image sequence;

Step 6, performing piecewise linear interpolation upsampling on the one-dimensional image sequence;

Step 7, creating a one-dimensional texture sequence corresponding to the one-dimensional image sequence;

Step 8, passing the above processing results into the vertex shader of the graphics rendering pipeline;

The GPU-side parallel shading module includes:

Step 9, convert the vertex coordinates from the global coordinate system to the clipping coordinate system;

Step 10, color the vertices according to the active field type;

Step 11, performing post-processing on the colored vertices;

Step 12: coloring the rasterized fragments.

Step 3: Multi-group the fields to reduce the number of vertex attribute association points used in the graphics rendering pipeline, improve the efficiency of transmitting field values from the CPU to the GPU and the scalability of the application, wherein the multi-grouping method includes:

(1) Group the coordinate fields (x, y, z) and the color fields (r, g, b) into two triplets;

(2) Iteratively group the remaining non-binary fields into four-tuples; when the number m of the remaining non-binary fields is less than 4, group the m fields into m-tuples;

(3) Iteratively grouping the binary fields into triplets; when the number n of remaining binary fields is less than 3, grouping the n fields into an n-tuple;

(4) Use vertex array objects VAO and vertex buffer objects VBO to transfer a set of tuple sequences.

Step 6, performing piecewise linear interpolation upsampling on the one-dimensional image sequence to improve the byte alignment and access efficiency of texture data in the video memory and enhance the compatibility of the GPU method on different hardware device platforms, said step 6 comprising:

(1) Rapidly calculate the target pixel width after upsampling based on the bitwise AND operation and the logarithmic base conversion formula;

(2) Determine the number of pixels that need to be interpolated between two adjacent pixels;

(3) performing piecewise linear interpolation using pixel coordinate distance as weight;

(4) Create an active texture of the one-dimensional image sequence and bind it to the vertex shader as a consistent one-dimensional sampler.

Step 10: Color the vertices according to the active field type, that is, use different coloring mechanisms for binary fields and non-binary fields:

(1) If the field type corresponding to the active field number is a color field, the color of the vertex is directly taken from the vertex color attribute array as the shading result;

(2) If the field type corresponding to the active field number is a non-binary field other than a color field, the linear texture coordinates are calculated according to the value range of the non-binary field and the value of the vertex on the non-binary field, and a naturally connected shading result is obtained in the active texture through the linear texture coordinates;

(3) If the field type corresponding to the active field number is a binary field, the pixels at the first and last positions of the active texture are used as the shading results when the binary field takes the values of 0 and 1, respectively.

In step 5, assume that the number of predefined color bars is n, each color bar contains m pixels, the number of pixels in each color bar is different, and each pixel contains three color channels (r, g, b). The specific steps for generating a one-dimensional image sequence are:

(1) Take a color bar ^Ci from the predefined color bar set in turn;

(2) Allocate 3M bytes of memory ^Mi for ^Ci ;

(3) Write each pixel of ^Ci into ^Mi in sequence;

(4) Construct a one-dimensional image I ⁱ according to the number of pixels m of C ⁱ and the corresponding memory M ⁱ ;

(5) Record I ⁱ in a one-dimensional image sequence L = {I ⁱ }.

In step 8, according to the data characteristics of the processing result of the CPU-side preprocessing module, different transmission mechanisms are used to respectively transmit the value range of the non-binary field, the active field number currently selected by the user, the multi-tuple sequence set, and the active texture currently selected by the user to the vertex shader, and the step 8 includes:

(1) Use uniform consistency variables to pass the value range of non-binary fields and the number of the active field currently selected by the user;

(2) Use VAO and VBO to transmit a set of multi-tuple sequences;

(3) Bind the active texture currently selected by the user to the vertex shader as a uniform sampler1D consistent one-dimensional sampler.

The field types currently switched by the user are divided into binary fields and non-binary fields. The two types of fields use different coloring mechanisms, including:

(1) If the field type corresponding to the active field number is a color field, the color of the vertex is directly taken from the vertex color attribute array as the shading result;

(2) If the field type corresponding to the active field number is a binary field, the pixels at the first and last positions of the active texture are used as the coloring results when the binary field takes 0 and 1 respectively;

(3) If the field type corresponding to the active field number is a non-binary field other than a color field, the linear texture coordinates are calculated based on the value range of the non-binary field and the value of the vertex on the non-binary field, and a naturally connected shading result is obtained in the active texture through the linear texture coordinates.

In step 11, the vertices shaded by the vertex shader are first subjected to a post-processing stage and then rasterized. The vertex post-processing stage includes five steps: transformation feedback, primitive assembly, viewing frustum clipping, perspective division, and viewport transformation.

The primitives processed in steps 1 to 11 are all vector points. Vector points can be presented on the screen only after being rasterized. Rasterization rasterizes vector points into fragments according to window coordinates. Fragments are pixel units, and then the fragments are colored in the fragment shader.

In the rasterization stage, anti-aliasing is enabled. Under the condition of enabling anti-aliasing, the graphics rendering pipeline constructs a circular area with a radius of _ps with the vector point to be rasterized as the center. The fragments whose center coordinates are located in the circular area constitute the area after the vector point is rasterized. In the rasterization process, the size of the pixel area occupied by the vector point after being rasterized is called the point size _ps .

The discrete fragments generated by rasterization are sent to the fragment shader for final shading. The discretized fragments formed after rasterization of the vector points have the same attribute values. The fragment shader uses the output results of the vertex shader to shade the discretized fragments in parallel; finally, after the clipping test, template test and depth test in the fragment-by-fragment operation stage, the visible fragments are sent to the frame buffer and drawn to the screen to complete the shading process in one frame.

The advantages of the present invention are: using the parallel processing capability of GPU to quickly present the visualization effect of multi-field dense point cloud in different fields and different color bars, and providing visualization technology support for subsequent application analysis. Providing an efficient and fast visualization solution for deep application analysis such as object classification, semantic extraction, target detection, etc. based on dense point cloud. Treating each point in the dense point cloud as a vertex in three-dimensional visualization, the parallel processing architecture of GPU graphics rendering pipeline is used to quickly present the visualization effect of multi-field dense point cloud in different fields and different color bars.

BRIEF DESCRIPTION OF THE DRAWINGS

The following is a brief description of the contents expressed in the drawings of the present invention and the marks in the drawings:

FIG1 is an overall process of efficient coloring of multi-field dense point clouds according to an embodiment of the present invention;

FIG2 is a statistical overview of field information of a standard point cloud file according to an embodiment of the present invention;

FIG3 is a schematic diagram of piecewise linear interpolation upsampling according to an embodiment of the present invention;

FIG4 is a schematic diagram of the organizational structure relationship between VAO and VBO according to an embodiment of the present invention;

FIG5 is a flow chart of a graphics rendering pipeline according to an embodiment of the present invention;

FIG6 is a schematic diagram of texture coordinate distribution according to an embodiment of the present invention;

FIG7 is a flow chart of the vertex post-processing stage of an embodiment of the present invention;

FIG8 is a perspective projection flat-head viewing frustum of an embodiment of the present invention;

FIG9 is a diagram showing the adaptive adjustment of near and far cross sections according to the viewpoint position and the minimum outer cover of the dense point cloud according to an embodiment of the present invention;

FIG10 is a diagram showing adaptive adjustment of upper, lower, left, and right cross sections according to the viewport aspect ratio according to an embodiment of the present invention;

FIG11 is a normalized device coordinate system according to an embodiment of the present invention;

FIG. 12 is a diagram showing a rasterization method when anti-aliasing is enabled according to an embodiment of the present invention.

Detailed ways

The specific implementation of the present invention will be further explained in detail below by describing the optimal embodiment with reference to the accompanying drawings.

The present embodiment provides a GPU method for efficient coloring of multi-field dense point clouds, and the visualization of multi-field dense point clouds is a key technology for presenting deep analysis results such as object classification, semantic extraction, and target detection. The existing mainstream technical means color each point in the point cloud point by point on the CPU (Central Processing Unit) side according to a given field, so it is inefficient and difficult to meet the real-time needs of deep application analysis. The present invention proposes a GPU (Graphics Processing Unit) method for efficient coloring of multi-field dense point clouds, which includes two major processing modules: a CPU-side preprocessing module and a GPU-side parallel coloring module. The CPU-side preprocessing module includes the following links: classifying the field type into binary fields and non-binary fields according to the data type and attributes of the field; counting the value range of each non-binary field except the color field; multi-grouping the fields; numbering and sorting the fields; encoding the predefined color bar sequence into a one-dimensional image sequence; performing piecewise linear interpolation upsampling on the one-dimensional image sequence; creating a one-dimensional texture sequence corresponding to the one-dimensional image sequence; and passing the above processing results into the graphics rendering pipeline. The parallel shading module on the GPU side includes the following steps: converting vertex coordinates from the global coordinate system to the clipping coordinate system; shading vertices according to the active field type; performing post-processing on the vertices after shading; and shading the rasterized fragments. The technical means of the present invention not only takes into account all the field information of the standard point cloud file, but also expands the field types according to the application scenario, and uses the parallel processing capabilities of the GPU graphics rendering pipeline to efficiently present the shading effects of dense point clouds under different fields and different color bars, which can serve popular industries such as road testing, unmanned driving, and industrial parts inspection based on point cloud data.

A GPU (Graphics Processing Unit) method for efficient coloring of a multi-field dense point cloud in this embodiment mainly includes a CPU-side preprocessing module and a GPU-side parallel coloring module, each of which mainly includes the following processing steps:

1. CPU-side preprocessing module

Step 1, classify the field types into binary fields and non-binary fields;

Step 2, counting the value range of each non-binary field except the color field;

Step 3, perform multi-grouping on the fields;

Step 4: Sort the fields by number (the number that the user switches to during the interaction is the active field);

Step 5, encoding the predefined color bar sequence into a one-dimensional image sequence;

Step 6, performing piecewise linear interpolation upsampling on the one-dimensional image sequence;

Step 7, creating a one-dimensional texture sequence corresponding to the one-dimensional image sequence (the color bar selected by the user during the interaction corresponds to an active texture);

Step 8: pass the above processing results to the vertex shader of the graphics rendering pipeline.

2. GPU-side parallel shading module

Step 9, convert the vertex coordinates from the global coordinate system to the clipping coordinate system;

Step 10, color the vertices according to the active field type;

Step 11, performing post-processing on the colored vertices;

Step 12: coloring the rasterized fragments.

The following is a detailed description of the technical details of the CPU-side preprocessing module and the GPU-side parallel shading module:

1. CPU-side preprocessing

Step 1: Classify field types into binary fields and non-binary fields

The three-dimensional point cloud of the target scene can be obtained through multi-view dense matching technology, structured light camera and LiDAR acquisition equipment. The three-dimensional point cloud is composed of dense points, and each point contains a lot of field information that records the properties of the point cloud. According to the data type and characteristics of the field, the field can be classified into binary fields and non-binary fields. Figure 2 counts all the field information contained in the standard point cloud file. Each field has a different data type and data size. Among them, GPS time: the data length of this non-binary field is the largest, which is 8 bytes; direction identification and edge identification: the data length of these two binary fields is the smallest, which is 1 bit. The definition of binary fields and non-binary values in the present invention is as follows: if the value of the field is 0 or 1, the field is a binary field (the data length of the binary field is only 1 bit. Taking the binary field of the direction identification as an example, when it takes 1, it represents the scanning direction from left to right; when it takes 0, it represents the scanning direction from right to left); otherwise, the field is a non-binary field. The category information is a composite field with a data length of 1 byte (8 bits): the first 4 bits (0 to 4) represent 32 classification results and are non-binary fields; the 5th, 6th, and 7th bits are binary fields, representing synthetic point identifiers, key point identifiers, and reserved identifiers, respectively. The extended field is a user-defined field, which can be either a binary field or a non-binary field. For a more detailed description of each field, please refer to the document "LAS Specification Version 1.2".

According to the field information of Figure 2, when reading the file header of the las/laz file (las file is loaded using the liblas open source library; laz file is loaded using the PDAL open source library), the present invention marks the read fields as binary fields or non-binary fields in turn.

Step 2: Count the value range of each non-binary field except the color field

The present invention performs linear smoothing based on spatial range on non-binary fields to achieve a natural and seamless coloring effect. Therefore, it is necessary to count the value range of each non-binary field except the color field (the value of the color field itself can be used as the coloring result, so there is no need to count). When the point cloud file is loaded for the first time, the present invention will count the interval range of each field point by point. Suppose the number of non-binary fields ^Fi in the dense point cloud is N, then there are N binary groups Respectively represent the value ranges of the N non-binary fields (/> represents the minimum value of F ⁱ , /> represents the maximum value of F ⁱ ). For the convenience of statistics, the initial time is / > (FLT_MAX represents the maximum positive value that a single-precision floating-point number can obtain), the process of counting the value range of each non-binary field is as follows:

(1) Determine whether the record file that stores the value range of the non-binary field exists on the disk: if it does, no statistics are required and the record file is directly loaded; otherwise, go to (2);

(2) Read each point P in the point cloud file point by point;

(3) Traverse each field F ⁱ of P;

(4) Determine the field type of F ⁱ : If F ⁱ is a binary field, no statistics are required; otherwise, go to (5);

(5) Determine whether F ⁱ is a color information field: if so, no statistics are required; otherwise, go to (6);

(6) Update ^Gi based on the value of F ⁱ That is

Among them, min and max represent the minimum and maximum values between the two respectively.

(7) Save the value range of each non-binary field as a record file on the disk.

Step 3: Multi-group the fields

The purpose of multi-grouping fields is to reduce the number of vertex attribute association points used in the graphics rendering pipeline, improve the efficiency of transmitting field values from the CPU to the GPU, and improve the scalability of the application. In the present invention, it is necessary to use the vertex attribute association points on the vertex shader to transfer all field values from the CPU side to the GPU side. In the graphics rendering pipeline of a general rendering engine such as OpenGL (Open Graphics Library), the upper limit of the number of vertex attribute association points is 16. If each field is associated with the vertex attribute association point as a channel, on the one hand, the transmission efficiency will be reduced and the GPU load will be increased; on the other hand, the scalability of the shader will be reduced (in the multi-field dense point cloud shading process based on the graphics rendering pipeline, in addition to the basic field information, additional fields such as deletion flags and selection flags will be introduced according to the needs of the application scenario. These additional fields also need to be associated with the vertex attribute association point). In view of the above factors, the present invention performs multi-grouping according to the field type, that is, multiple fields are combined to form a multi-group, and the multi-group is associated with a vertex attribute association point as a whole and passed to the vertex shader. The specific multi-grouping scheme is as follows:

(1) The coordinate field (x, y, z) and the color field (r, g, b) are two inherent non-binary fields. These two fields are grouped into two triplets.

(2) iteratively grouping the remaining non-binary fields (such as echo intensity, echo number, echo quantity, and scanning angle) into four-tuples; when the number m of the remaining non-binary fields is less than 4, grouping the m fields into an m-tuple;

(3) Iteratively group the binary fields (such as synthetic point identifier ss, key point identifier kk, and reserved identifier rr) into triplets (ss, kk, rr); when the number of remaining binary fields n is less than 3, group these n fields into an n-tuple.

Through grouping, each field belongs to a tuple. Given x points, each tuple corresponds to x records, and these x records form a tuple sequence, and multiple tuple sequences form a tuple sequence set. In subsequent processing, each tuple sequence is associated with a vertex attribute association point on the vertex shader in turn and passed to the GPU's graphics rendering pipeline. With the help of tuple grouping, the number of vertex attribute association points used on the vertex shader can be reduced by 60%, which not only improves transmission efficiency; it also enhances scalability, and the saved vertex attribute association points can be used to express fields in other application scenarios.

Step 4: Sort the fields by number

When rendering multi-field dense point clouds, it is necessary to quickly present the corresponding visualization effects according to the fields specified by the user. When the user selects a field, the number of the field is returned, and the number is sent to the shader in the graphics rendering pipeline. Therefore, it is necessary to establish a one-to-one correspondence between "field name-field number". The present invention establishes a number for each field as shown in Figure 2 based on the field arrangement order in the file header of the standard point cloud file. The data type of the number is an unsigned integer variable. When the user switches to select a field, the field is activated and becomes the active field. The corresponding number value is sent to the vertex shader, and the vertex shader performs parallel shading processing on the corresponding field according to the number.

Step 5: Encode the predefined color bar sequence into a one-dimensional image sequence

In the multi-field dense point cloud coloring based on the GPU graphics rendering pipeline, it is necessary to consider not only the active field selected by the user, but also the color bar selected by the user, that is, the coloring result is presented according to the color bar. As shown in Figure 3, the color bar is a sequence composed of a group of pixels of different colors, which can be customized according to the user's preferences. The user selects different color bars to present the coloring effect of the point cloud under the current color bar. The color bar information needs to be processed in two aspects before it can be accessed by the graphics rendering pipeline: first, it is encoded and converted into a one-dimensional image; then a one-dimensional texture is created. Assume that the number of predefined color bars is d, each color bar contains c pixels (the number of pixels in each color bar is not the same), and each pixel contains three color channels (r, g, b). The specific steps to generate a one-dimensional image sequence are:

(1) Take a color bar ^Ci from the predefined color bar set in turn;

(2) Allocate 3*c bytes of memory ^Mi for ^Ci ;

(3) Write each pixel of ^Ci into ^Mi in sequence;

(4) The number of pixels c of ^Ci and the corresponding memory ^Mi construct a one-dimensional image ^Ii , that is, the pixel width of ^Ii is equal to c, the pixel height is equal to 1, and the pixel data is ^Mi

(5) Record I ⁱ in a one-dimensional image sequence L = {I ⁱ }.

Step 6: Perform piecewise linear interpolation upsampling on the one-dimensional image sequence to obtain a one-dimensional image sequence with a pixel width of n ⁱ .

The one-dimensional image sequence stored in the memory cannot be directly accessed by the graphics rendering pipeline and needs to be converted into a texture before it can be read by the shader. Texture is a special data structure in the rendering engine, which is used to exchange color information between the memory and the video memory. When the texture size is a power of 2, not only can the alignment degree and access efficiency of the texture in the video memory be improved, but also the compatibility across platforms can be improved. Therefore, before transmitting the color information to the graphics rendering pipeline, the present invention performs piecewise linear interpolation upsampling on the one-dimensional image sequence generated in step 5, so that the pixel width of each one-dimensional image is adaptively adjusted to the positive integer n ⁱ (n ⁱ is equal to the power of 2 and n ⁱ ≥ w ⁱ ) closest to w ⁱ (w ⁱ is the pixel width of the one-dimensional image I ⁱ ). Now take out the one-dimensional image I ⁱ from the one-dimensional image sequence L = {I ⁱ }, and the technical details of the piecewise linear interpolation upsampling of I ⁱ are as follows:

(1) Quickly determine whether w ⁱ is equal to a power of 2 through a bitwise AND operation;

boolisPowerOfTwo＝ ^wi &(wi ^- 1)

If isPowerOfTwo is true, it means that w ⁱ is equal to the power of 2, and there is no need to perform interpolation upsampling on I ⁱ ; otherwise, go to (2);

(2) Calculate the positive integer n ⁱ that is closest to w ⁱ (n ⁱ is equal to the power of 2 and n ⁱ > w ⁱ )

In the above formula, first, the logarithm a of ^wi with base 2 is calculated using the logarithmic base conversion formula; then an offset of 0.5 is added to a and rounded down to obtain the positive integer b closest to a; then 2 is raised to the power of b to obtain c; finally, the size of c and ^wi is determined: if c< ^wi , then ⁿⁱ is equal to 2c; otherwise, ⁿⁱ is equal to c; the floor function in the formula represents the rounding down calculation.

(3) Perform piecewise linear interpolation upsampling according to the target pixel width

In order to ensure that the color of the colored point cloud can be smoothly and seamlessly connected, the present invention adopts a piecewise linear interpolation upsampling strategy as shown in Figure 3. The general idea of piecewise linear interpolation upsampling is to linearly interpolate a specified number of pixels between two adjacent pixels in I ⁱ . The specific technical details are:

① Determine the number of pixels that need to be interpolated between two adjacent pixels in I ⁱ

Since the pixel width of I ⁱ before interpolation is w ⁱ and the pixel width of I ⁱ after interpolation is n ⁱ , n ⁱ - ^{w i} pixels need to be interpolated. The present invention uniformly performs linear interpolation between adjacent pixels to reduce the color bar difference before and after interpolation, that is, the average number of interpolation times between two adjacent pixels in I ⁱ is If n ⁱ - ^{w i} cannot be divided by w ⁱ - 1, the remainder is added between the last two adjacent pixels, that is, the number of pixels to be inserted between the last two adjacent pixels is

Among them, mod(a,b) represents the The remainder of .

②Perform piecewise linear interpolation with pixel coordinate distance as weight

After determining the number of pixels t that need to be interpolated between two adjacent pixels, piecewise linear interpolation can be performed between two adjacent pixels in sequence according to the following formula:

In the above formula, j (0≤j≤t+2) represents the position of the current interpolation pixel, t+2 represents the total number of pixels after segment interpolation, p, q are the linear weights during interpolation, _p0 , _p1 represent the pixel values of two adjacent pixels in ^Ii , and _pj represents the interpolation result. Therefore, when j=0, _pj = _p0 ; when j=t+2, _pj = _p1 .

Step 7: Create a one-dimensional texture sequence corresponding to the one-dimensional image sequence. The one-dimensional image sequence refers to the one-dimensional image sequence after piecewise linear interpolation upsampling.

The elements in the one-dimensional image sequence need to be converted into a one-dimensional texture before they can be accessed by the graphics rendering pipeline. In the classic general-purpose three-dimensional rendering engine OpenGL/OpenGL ES (OpenGL ES is the embedded mobile version of OpenGL), the basic process of creating a texture based on a given image is as follows:

(1) Call glGenTextures to request a texture handle from the rendering engine;

(2) Call glBindTexture to switch the texture handle to the currently activated resource. All subsequent processing is for the operation of the activated texture.

(3) Call glTexParameteri to set the filtering mode (GL_TEXTURE_MIN_FILTER/TEXTURE_MAG_FILTER) when the texture is reduced/enlarged. In the present invention, in order to achieve a natural and smooth shading effect of dense point clouds, the linear filter GL_LINEAR is used as the parameter of glTexParameteri;

(4) Call glTexImage1D to load the one-dimensional image data into the texture.

Step 8: pass the preprocessing result (the preprocessing result is the result of processing in steps 1, 2, and 3) to the vertex shader of the graphics rendering pipeline

In order to reduce the number of transmissions from the CPU to the GPU and improve the access and drawing efficiency of the graphics rendering pipeline, the present invention adopts different transmission mechanisms to transfer the value range of the non-binary field, the active field number currently selected by the user, the multi-tuple sequence set and the active texture currently selected by the user to the vertex shader according to the data characteristics of each of the above processing results. The specific implementation steps are as follows:

(1) Use uniform consistency variables to pass the value range of non-binary fields and the number of the active field currently selected by the user

A consistency variable is a special variable in the shader, which cannot be modified in the processing flow of the graphics rendering pipeline and can only be transferred and updated from the CPU to the GPU. A consistency variable can transfer various data types, including basic data types such as bool, int, uint, float, double, etc., as well as multi-tuple data types organized in the form of vectors such as vec2, vec3, vec4, etc., and can also transfer advanced data types such as matrices such as mat2, mat3, mat4, etc. In the present invention, the value range of a non-binary field is transferred using vec2, that is, the minimum and maximum values of the value range of a non-binary field are both represented by a floating point type; the active field number currently selected by the user is transferred using uint, that is, the active field number currently selected by the user is represented by an unsigned integer.

(2) Using VAO and VBO to transfer a set of tuple sequences

In order to reduce the number of state switching times during three-dimensional rendering and improve the transmission efficiency between the CPU and the GPU, the present invention uses a combination of vertex array objects VAO (Vertex Array Object) and vertex buffer objects VBO (Vertex Buffer Object) to further improve the shading rendering efficiency of multi-field dense point clouds. The organizational structure relationship between VAO and VBO is shown in Figure 4: VAO contains 16 vertex attribute association points, each vertex attribute association point corresponds to a VBO, and each VBO can be bound to a set of vertex attribute arrays. Therefore, the multi-tuple sequence formed after multi-tuple grouping in step 3 can be converted into a vertex attribute array on the VBO. By pre-setting the vertex attribute array bound to the VBO on the VAO, there is no need to call the drawing command in the rendering of each frame. Only the VAO needs to be activated to efficiently complete the rendering of the dense point cloud.

In practical applications, there are two ways to use VBO: "one-to-one" and "many-to-one". In the "one-to-one" usage mode, each vertex attribute array is bound to a VBO; in the "many-to-one" usage mode, multiple vertex attribute arrays can be bound to a VBO. According to the rendering characteristics of VAO, both "one-to-one" and "many-to-one" can improve rendering efficiency, but "many-to-one" can reduce the number of VBOs created and reduce the consumption of hardware resources. Therefore, the present invention adopts the "many-to-one" solution. In summary, the key steps of the present invention to improve rendering performance based on VAO and VBO are as follows:

①Call glGenVertexArrays to create a VAO handle;

②Call glBindVertexArray to activate VAO;

③Call glGenBuffers to create a VBO handle;

④Call glBindBuffer to activate VBO;

⑤Map the tuple sequence set to a VBO vertex attribute array;

Assume that the tuple sequence set contains k _x tuple sequences, each of which stores data records of n _x points on m _x fields. The steps to convert the tuple sequence set into the vertex attribute array on the VBO are as follows:

a. Calculate the total number of bytes required for a set of tuple sequences

In order to take byte alignment into consideration to improve reading efficiency, the data type of each field in the present invention is set to float floating point type during the conversion process (the byte length of the float data type is 4, and 4 bytes is the default byte alignment length of each hardware platform). The total number of bytes required for the tuple sequence set is

b. Call glBufferData to allocate VBO memory space according to the total number of bytes

c. Starting from the first tuple sequence, map each tuple sequence D in the tuple sequence set L to the vertex attribute array on the VBO. The pseudo code of the mapping process is as follows:

Among them, target represents the vertex array object and takes GL_ARRAY_BUFFER, offset represents the byte offset of D in VBO, size represents the byte size of D, data represents the starting memory address of D, so the function glBufferSubData copies the data stored in D to the memory pre-allocated by VBO; location represents the position index where D is mapped to VBO, GL_FLOAT represents that the data type is floating point type, GL_FALSE represents that the mapped vertex attribute is a non-normalized vertex attribute, stride represents the step interval between adjacent elements in the vertex attribute array, and the default value is 0, which means that it is arranged in a compact format, so the function glVertexAttribPointer specifies the mapping position and data organization format of D on VBO.

The vertex attribute array mapped to the VBO is associated with the vertex attribute association point of the vertex shader during the graphics rendering pipeline. The vertex shader accesses the vertex attributes at the corresponding association point through the position index recorded in the VBO. For more detailed technical details about VAO and VBO, please refer to the document "The Graphics System: A Specification.

(3) Bind the active texture currently selected by the user as a uniform sampler1D consistent one-dimensional sampler to the vertex shader

Sampler (including sampler1D, sampler2D and sampler3D) samplers are opaque data types (Opaque Types) predefined in GLSL (Graphics Library Shading Language). The so-called opaque data type means that this data type cannot be accessed directly through the reading and writing of variables, and can only be read through special functions built into GLSL (such as texture1D, texture2D, texture3D). The general graphics rendering pipeline generally comes with 8 sampler binding points, and the shader can access 8 textures at the same time to achieve multiple texture effects. The texture is bound to the corresponding position and becomes a sampler that can be accessed by the shader. Through the sampler, the shader can access pixel information at a specific position based on the texture coordinates. For a more detailed technical introduction to samplers, please refer to the document "The Shading Language.

2. GPU-side parallel shading

Parallel rendering is an inherent characteristic of the GPU graphics rendering pipeline. When performing rendering tasks for complex three-dimensional models, each graphics processing unit on the GPU can simultaneously execute independent graphics rendering pipelines (each graphics processing unit corresponds to a graphics rendering pipeline, and each graphics rendering pipeline processes a certain primitive; in the present invention, the vertex shader in each graphics rendering pipeline corresponds to a point in a dense point cloud, and the vertex shader can access all field information of the point), thereby achieving the purpose of parallel acceleration. Figure 5 is the rendering process of the OpenGL graphics rendering pipeline, which includes two types of modules: programmable modules (modules corresponding to the dotted boxes in Figure 5) and non-programmable modules (modules corresponding to the solid boxes in Figure 5). For detailed technical details about each module, please refer to the document "The Shading Language.

The present invention mainly relates to two major programmable modules (vertex shader and fragment shader) and two major non-programmable modules (vertex post-processing and rasterization) of a graphics rendering pipeline.

Step 9: Convert vertex coordinates from the global coordinate system to the clip coordinate system

The spatial perspective transformation is an operation that the vertex shader must perform. Its purpose is to transform the input vertex position information from the global coordinate system to the clip coordinate system to provide a basis for subsequent processing. The vertex shader performs the spatial perspective transformation to transform the input vertex position information from the global coordinate system to the clip coordinate system. The spatial perspective transformation can be represented by matrix operations, that is,

Among them, M represents the model view matrix, (x,y,z,1) ^T represents the coordinates of the vertex in the global coordinate system, (x _e ,y _e ,ze _e , _we ) ^T represents the coordinates of the vertex in the camera coordinate system, P represents the projection matrix, and (x _c ,y _c ,z _c ,w _c ) ^T represents the coordinates of the vertex in the clip coordinate system.

Step 10: Color the vertices according to the active field type

Coloring vertices is the core task of the vertex shader. Since the field types currently selected by the user are divided into binary fields and non-binary fields, the two types of fields require different coloring mechanisms. The specific coloring technology details are described as follows:

(1) If the field type corresponding to the active field number is a color field, the color of the vertex is directly taken from the vertex color attribute array as the shading result;

(2) If the field type corresponding to the active field number is a binary field, the pixels at the first and last positions of the active texture are used as the coloring results when the binary field takes 0 and 1 respectively;

(3) If the field type corresponding to the active field number is a non-binary field other than a color field, the linear texture coordinates are calculated based on the value range of the non-binary field and the value of the vertex on the non-binary field, and a naturally connected shading result is obtained in the active texture through the linear texture coordinates.

In 3D visualization, texture coordinates are used to access pixels at specific locations of the texture. As shown in Figure 6, given a 2D texture, the origin (0,0) of the texture coordinate (tx,ty) is located at the lower left corner, and the horizontal coordinate tx increases from 0 to 1 from left to right; the vertical coordinate ty increases from 0 to 1 from bottom to top. Therefore, the texture coordinate value range is 0≤tx,ty≤1. Let the pixel width of the 2D texture be w and the pixel height be h, then the texture coordinate of the pixel at the i-th row and j-th column (0≤i≤w,0≤j≤h) is

If the sampler corresponding to the active texture passed into the vertex shading is cb, the pixel value pixelValue at the texture coordinate can be obtained through the predefined function texture2D in GLSL, that is,

vec4 pixelValue = texture2D(cb,vec2(tx,ty))

In the present invention, the texture corresponding to cb is a one-dimensional texture, so the above code can be simplified to

vec4 pixelValue = texture1D(cb,vec2(tx,0))

Among them, vec4 is the built-in data type of GLSL, which represents the value of the pixel on the four channels (r, g, b, w) (0≤r, g, b, w≤1); vec2 is a two-dimensional vector, representing the texture coordinates.

Therefore, if the user switches the selected active field to a binary field, when the value of the current vertex on this field is 0, the corresponding shading result is

vec4 pixelValue = texture1D(cb,vec2(0,0))

Otherwise, the coloring result is

vec4 pixelValue = texture1D(cb,vec2(1,0))

If the active field selected by the user is a non-binary field, the linear texture coordinate (lx, 0) is calculated, and the naturally connected pixel is taken from the active texture through (lx, 0) as the shading result. The following takes the active field as the elevation field as an example to explain the specific process of calculating lx. Let the value range of the elevation field be [h _min ,h _max ], the elevation of the current vertex be h _v (h _v is the z component of the vertex coordinates), and the pixel width of the active texture be w _t , then

In the above formula, _rn represents the spatial range of elevation, _su represents the elevation range corresponding to a single pixel in the active texture, then _pi represents the pixel position of the current point mapped to the active texture, and _lx is the calculated linear texture coordinate.

Step 11: Post-process the shaded vertices

After being colored by the vertex shader, the vertices need to go through the post-processing stage before they can be rasterized (in the present invention, the object to be colored is each point in the dense point cloud, so there is no need for the participation of the surface shader and the geometry shader). The process of the vertex post-processing stage is shown in Figure 7, which specifically includes five links: transform feedback, primitive assembly, viewing volume clipping, perspective division, and viewport transform. Among them, the last three links are closely related to the present invention.

(1) Visual cone clipping

The purpose of frustum clipping is to remove outliers and reduce extra drawing overhead (as shown in Figure 8, outliers are defined as points outside the frustum). The frustum under perspective projection consists of six sections, which are established by the perspective projection matrix P. P can be constructed by specifying the positions of the six sections l _p , r _p , t _p , b _p , n _p , and f _p in the camera coordinate system e-xyz, that is,

Where _lp represents the left section, _rp represents the right section, _tp represents the upper section, _bp represents the lower section, _np represents the near section, and _fp represents the far section. P transforms the camera coordinates into clipping coordinates. In the clipping coordinate system c-xyz, the relationship between the components of the clipping coordinates ( _xc , _yc , _zc , _wc ) is as follows

Where ze _is the z component of the camera coordinates. If a point does not satisfy the above inequality after perspective projection transformation, it means that the point is an external point and should be removed.

The cross-sectional positions of the perspective projection matrix P are closely related to perspective deformation and depth conflict (z-fighting). Therefore, the present invention starts from the following two aspects to adaptively adjust the positions of the six cross sections, establish the optimal projection matrix parameters, and ensure the visualization effect of the dense point cloud.

① Adaptively adjust the near and far sections according to the viewpoint pose and the minimum outer cover of the dense point cloud

Since vertex depth information is nonlinearly distributed in the depth buffer (i.e., the depth accuracy near the near section is higher, and the depth accuracy near the far section is lower), if the near section is very close to the viewpoint and the far section is very far from the viewpoint, it will cause depth conflict when rendering a large scene dense point cloud (flickering will occur when interacting with the scene), affecting the visualization effect. Therefore, the present invention designs a mechanism for adaptively adjusting the position of the near and far sections, automatically calculating the position of the near and far sections in each frame according to the viewpoint posture and the minimum outsourcing of the scene, and optimizing the accuracy distribution of the depth buffer.

As shown in Figure 9, the spatial range of the minimum outer bounding box BB of the dense point cloud is established by its eight corner points in the global coordinate system o-xyz, and the viewpoint pose is established by the camera coordinate system e-xyz. Get the pair of corner points p and q closest to and farthest from the viewpoint position e, then the near section _np and the far section _fp are equal to/> and/> In/> The projection length on

The above mechanism can ensure that the near and far sections just lock the overall depth range of the dense point cloud in the z-axis direction of the camera coordinate system, make full use of the precision distribution of the depth buffer, and prevent depth conflicts.

②Adaptively adjust the upper, lower, left, and right sections according to the proportion of the viewport size

The positions of the upper, lower, left, and right sections should be coordinated with the size of the rendering viewport to prevent perspective distortion. As shown in FIG10 , the present invention establishes the positions of the upper, lower, left, and right sections according to the viewport aspect ratio λ and the near section _np and the far section _fp , that is,

In the above formula, a represents the field of view FOV (Field Of View Angle); r represents The tangent value of is equal to the ratio of the viewport height h to 2n _p ; λ represents the viewport aspect ratio, w is the viewport width, and h is the viewport height. The bold lines in Figure 10 are the four adjusted cross-section positions.

In practical applications, when the viewport size changes, the upper, lower, left, and right sections need to be adjusted; when the viewpoint posture is changed or a dense point cloud is overloaded, the near and far sections need to be adjusted.

(2) Perspective division

Perspective division converts the clipping coordinates into normalized device coordinates NDC (Normalized Device Coordinate) as the basis for subsequent viewport transformation. As shown in Figure 11, the normalized device coordinates are in the left-hand coordinate system n-xyz, which is obtained by the perspective division of the following formula, that is,

Therefore, the value range of _xn , _yn , and _zn is all between [-1,1].

(3) Viewport Transformation

The viewport transformation converts the normalized device coordinates into window coordinates, providing a basis for subsequent rasterization. The starting point w ( _xo , _yo , _zo ) of the window coordinate system w-xyz is located at the lower left corner of the viewport in the default configuration (it can also be specified as the upper left corner or customized by the user), the x-axis is to the right, the y-axis is upward, and the z-axis points to the inside of the screen. Let the width of the viewport be w and the height be h, then the normalized device coordinates ( _xn , _yn , _zn ) can be converted to window coordinates ( _xw , _yw , _zw ) by the following formula, that is,

Where _df represents the depth value at the far section, _dn represents the depth value at the near section, and in the default configuration, _df = 1, _dn = 0. Before viewport conversion, you can call the OpenGL interface glViewport to specify the start point, width and height of the viewport, and call glDepthRange to specify the depth range.

Step 12: Color the rasterized fragments

In the above processing, the processed primitives are all vector points, which need to be rasterized before they can be displayed on the screen. Rasterization rasterizes vector points into fragments (fragments can be understood as pixel units) according to window coordinates, and then shades the fragments in the fragment shader. In the rasterization process, the size of the pixel area occupied by the vector point after rasterization is called the point size _ps , _which can be calculated by the following formula

Among them, u _s is the size specified by glPointSize, a _x , _by , c _z is the distance attenuation coefficient, d _w represents the distance from the vector vertex to the viewpoint, max _s , min _s represent the upper and lower limits of the scale size after rasterization, and clamp limits the value of p _s to the range of [min _s , max _s ]; the values of a _x , _by , c _z , max _s , min _s are related to the specific hardware implementation.

In order to improve the visualization effect of dense point clouds, the present invention turns on anti-aliasing in the rasterization stage. As shown in Figure 12, when anti-aliasing is turned on, the graphics rendering pipeline will construct a circular area with a radius of _ps with the vector point to be rasterized as the center. The fragments whose center coordinates are located in the circular area constitute the area after the vector point is rasterized.

The discrete fragments generated by rasterization are sent to the fragment shader for final shading. The discretized fragments formed by rasterization of vector points have the same attribute values (the attribute values here are the shader results output by the vertex shader). The fragment shader uses the output results of the vertex shader to shade the discretized fragments in parallel. Finally, after the clipping test (Scissor Test), stencil test (Stencil Test) and depth test (DepthTest) in the fragment-by-fragment operation stage, the visible fragments are sent to the frame buffer and drawn to the screen, completing the shading process in one frame.

The technical advantages, effects or strengths of this solution include:

(1) Compared with the traditional CPU point-by-point serial shading technology route, the present invention uses the parallel processing capability of the GPU graphics rendering pipeline to quickly perform customized shading on multi-field dense point clouds to meet the effectiveness requirements of subsequent application analysis;

(2) The technical means of the present invention not only considers all the field information of the standard point cloud file, but also can expand the field types according to the application scenario, and efficiently present the coloring effect of the dense point cloud under different fields and different color bars;

(3) Since the present invention is developed based on the graphics rendering pipeline of a general graphics card and C++ code, it is compatible with different hardware devices and has reliable cross-platform performance.

Obviously, the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, they are all within the protection scope of the present invention.

Claims (10)

1. A GPU method for efficiently coloring multi-field dense point cloud is characterized in that: the method comprises a CPU end preprocessing module and a GPU end parallel coloring module, wherein the CPU end preprocessing step preprocesses point cloud data and user predefined color bars to generate a preprocessing result which can be loaded by the GPU; the GPU side parallel coloring module is responsible for parallel coloring of the point cloud data.

2. The GPU method of claim 1, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud:

The CPU end preprocessing module comprises:

step1, classifying field types into binary fields and non-binary fields;

Step 2, counting the value range of each non-binary field except the color field;

Step3, performing multi-element grouping on the fields;

step 4, numbering and sorting the fields;

Step 5, coding the predefined color bar sequence into a one-dimensional image sequence;

step 6, performing piecewise linear interpolation up-sampling on the one-dimensional image sequence;

Step 7, creating a one-dimensional texture sequence corresponding to the one-dimensional image sequence;

step 8, transmitting the processing result into a vertex shader of the graphics rendering pipeline;

The GPU side parallel coloring module comprises:

step 9, converting the vertex coordinates from a global coordinate system to a clipping coordinate system;

step 10, coloring the vertex according to the type of the activity field;

step 11, performing post-processing on the colored vertexes;

and step 12, coloring the rasterized fragments.

3. The GPU method of claim 2, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud:

Step3, performing multi-element grouping on the fields, so as to reduce the number of vertex attribute association points in a graphics rendering pipeline, improve the efficiency of transmitting field values to the GPU by the CPU and improve the expandability of application, wherein the multi-element grouping method comprises the following steps:

(1) Grouping the coordinate fields (x, y, z) and the color fields (r, g, b) in two triples;

(2) Iteratively grouping the remaining non-binary fields into quaternions; when the number m of the remaining non-binary fields is less than 4, then grouping the m fields into m tuples;

(3) Iteratively grouping binary fields into triples; when the number n of the remaining binary fields is less than 3, grouping the n fields into an n-tuple;

(4) The set of sequences of tuples is transmitted using vertex array objects VAO and vertex buffer objects VBO.

4. The GPU method of claim 2, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud:

Step 6, performing piecewise linear interpolation up-sampling on a one-dimensional image sequence to improve byte alignment degree and access efficiency of texture data in a video memory and enhance compatibility of a GPU method on different hardware equipment platforms, wherein the step 6 comprises the following steps:

(1) Rapidly calculating the width of the up-sampled target pixel based on bitwise and operation and a logarithmic bottom formula;

(2) Establishing the number of pixels needing interpolation between two adjacent pixels;

(3) Performing piecewise linear interpolation with the pixel coordinate distance as a weight;

(4) An active texture of a one-dimensional image sequence is created and bound as a consistent one-dimensional sampler to a vertex shader.

5. The GPU method of claim 2, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud:

Step 10, coloring the vertex according to the type of the active field, namely adopting different coloring mechanisms for the binary field and the non-binary field:

(1) If the field type corresponding to the active field number is a color field, directly taking out the color of the vertex from the vertex color attribute array as a coloring result;

(2) If the field type corresponding to the active field number is a non-binary field except a color field, calculating linear texture coordinates according to the value range of the non-binary field and the value of the vertex on the non-binary field, and obtaining a natural coloring result in the active texture through the linear texture coordinates;

(3) If the field type corresponding to the active field number is a binary field, the pixels at the first and last positions of the active texture are respectively used as coloring results when the binary field is 0 and 1.

6. The GPU method of claim 2, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud:

In step 5, let the number of predefined color bars be n, each color bar contains m pixels, the number of pixels of each color bar is different, each pixel contains (r, g, b) three color channels, and the specific steps for generating the one-dimensional image sequence are:

(1) Sequentially extracting one color bar C ⁱ from the predefined color bar set;

(2) 3M-byte size memory M ⁱ is allocated for C ⁱ;

(3) Sequentially writing each pixel of C ⁱ to M ⁱ;

(4) Constructing a one-dimensional image I ⁱ according to the pixel number M of the C ⁱ and the corresponding memory M ⁱ;

(5) I ⁱ is recorded in a one-dimensional image sequence l= { I ⁱ }.

7. The GPU method of claim 2, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud: in step 8, according to the data characteristics of the processing result of the CPU side preprocessing module, different transmission mechanisms are adopted to respectively transmit the value range of the non-binary field, the number of the active field currently selected by the user, the multi-group sequence set and the active texture currently selected by the user into the vertex shader, and step 8 includes:

(1) Transmitting the value range of the non-binary field and the number of the active field currently selected by the user by using the unitorm consistency variable;

(2) Transmitting the set of multi-tuple sequences using VAO and VBO;

(3) And binding the active texture currently selected by the user as a uniform sample 1D consistency one-dimensional sampler to the top-point colorizer.

8. The GPU method of claim 2, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud: the field types selected by the current switching of the user are divided into binary fields and non-binary fields, and the two types of fields adopt different coloring mechanisms, wherein the coloring mechanism comprises:

(1) If the field type corresponding to the active field number is a color field, directly taking out the color of the vertex from the vertex color attribute array as a coloring result;

(2) If the field type corresponding to the active field number is a binary field, respectively taking pixels at the first and last positions of the active texture as coloring results when the binary field is 0 and 1;

(3) If the field type corresponding to the active field number is a non-binary field except a color field, calculating linear texture coordinates according to the value range of the non-binary field and the value of the vertex on the non-binary field, and obtaining a natural coloring result in the active texture through the linear texture coordinates.

9. The GPU method of claim 2, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud:

in step 11, the vertex colored by the vertex shader is rasterized after a post-processing stage, wherein the post-processing stage of the vertex comprises 5 links of transformation feedback, primitive assembly, cone clipping, perspective division and viewport transformation.

10. The GPU method of claim 2, wherein the GPU method is configured to efficiently paint a multi-field dense point cloud:

the graphic elements processed in the steps 1 to 11 are vector points, the vector points can be displayed on a screen after being subjected to rasterization, the rasterization is used for rasterizing the vector points into the pixel elements according to window coordinates, the pixel elements are pixel units, and then the pixel elements are colored in a pixel element shader;

Starting antialiasing in a rasterization stage, and under the condition of starting antialiasing, constructing a circular area with the radius of p _s by the graphics rendering pipeline by taking vector points to be rasterized as circle centers; the cells with the central coordinates of the cells in the circular area form a vector point rasterized area; in the rasterization process, the size of the pixel area occupied by the vector point after being rasterized is called the size p _s of the point;

The discrete patch generated by rasterization is sent to a patch shader for final coloring, the discrete patch formed by rasterizing vector points has the same attribute value, and the patch shader uses the output result of the vertex shader to color the discrete patch in parallel; finally, after the clipping test, the template test and the depth test in the fragment-by-fragment operation stage, the visible fragments are sent into a frame buffer area and drawn on a screen, and the coloring treatment in one frame is completed.