CN101710429A

CN101710429A - Illumination algorithm of augmented reality system based on dynamic light map

Info

Publication number: CN101710429A
Application number: CN200910044517A
Authority: CN
Inventors: 陈浩; 孙建华; 谭茗; 曾平
Original assignee: Hunan University
Current assignee: Hunan University
Priority date: 2009-10-12
Filing date: 2009-10-12
Publication date: 2010-05-19
Anticipated expiration: 2029-10-12
Also published as: CN101710429B

Abstract

In order to realize the real-time accuracy of lighting in augmented reality system, a fast lighting algorithm based on dynamic light map is proposed. This method calculates the signal amplification and signal offset of the camera by collecting the pixel brightness of the calibration object in the video image, and uses the ambient light information recorded by the dynamically updated light map to achieve consistent lighting between virtual and real scenes, and Scenes that can adapt to changes in lighting.

Description

A lighting algorithm for augmented reality system based on dynamic light map

技术领域technical field

本发明属于计算机图形学和图像处理领域，具体涉及通过图形图像处理对增强现实场景进行光照增强的方法。The invention belongs to the field of computer graphics and image processing, and in particular relates to a method for enhancing illumination of an augmented reality scene through graphic image processing.

背景技术Background technique

增强现实(Augmented Reality，简称AR)技术是虚拟现实技术的一个分支，是一种将真实环境同虚拟物体相融合的技术。在国外，AR技术已被广泛运用于医学可视化、维护与修理、理解与提示、机器人路径规划、娱乐、军用飞机导航和攻击瞄准等领域。近年来，AR也引起了国内研究者的关注，相关的应用也越来越多。Augmented Reality (AR) technology is a branch of virtual reality technology, which is a technology that integrates the real environment with virtual objects. In foreign countries, AR technology has been widely used in medical visualization, maintenance and repair, understanding and prompting, robot path planning, entertainment, military aircraft navigation and attack targeting and other fields. In recent years, AR has also attracted the attention of domestic researchers, and there are more and more related applications.

增强现实技术需要解决的三个关键的问题是：1.如何将虚拟物体和真实场景进行校准，使它们之间能够无缝地结合，这就是增强现实的注册技术所要解决的问题；2.怎样在真实场景中显示虚拟物体，使其与真实场景协调一致，是立体显示技术所要解决的问题；3.实时性问题。随着AR注册技术的逐步完善，人们已逐渐把注意力转移到了如何增强AR场景的真实感上。The three key problems that augmented reality technology needs to solve are: 1. How to calibrate virtual objects and real scenes so that they can be seamlessly combined, which is the problem to be solved by augmented reality registration technology; 2. How to It is a problem to be solved by the stereoscopic display technology to display the virtual object in the real scene so that it is consistent with the real scene; 3. Real-time problem. With the gradual improvement of AR registration technology, people have gradually shifted their attention to how to enhance the realism of AR scenes.

为了使增强现实场景看起来更加真实，更容易让人们接受，我们需要解决虚拟物体与真实场景之间的光照一致性问题。为了得到令人信服的光照效果，首先要对真实场景的光照情况进行分析，然后根据分析的结果建立虚拟物体的光照模型，从而达到虚、实物体相融合的效果。近年来的很多方法都是使用辅助测光设备——鱼眼透镜摄像机来记录环境中的光照信息，并以此确定各个光源的位置，然后用已知的光源对增强现实场景进行照明。这类方法能够准确测量环境光照，实现逼真的光照效果，但是需要昂贵的测光设备，这对于普通的实验条件来说是无法实现的。In order to make augmented reality scenes look more real and easier for people to accept, we need to solve the problem of lighting consistency between virtual objects and real scenes. In order to obtain a convincing lighting effect, it is necessary to analyze the lighting conditions of the real scene first, and then establish a lighting model of the virtual object according to the analysis results, so as to achieve the fusion effect of virtual and real objects. Many methods in recent years use auxiliary photometry equipment - fisheye lens camera to record the lighting information in the environment, and use this to determine the position of each light source, and then use the known light source to illuminate the augmented reality scene. Such methods can accurately measure ambient lighting and achieve realistic lighting effects, but require expensive photometry equipment, which is impossible for ordinary experimental conditions.

发明内容Contents of the invention

本发明针对现有的增强现实光照技术对硬件条件要求较高的情况，提出了一种基于动态light map(光照图，用于记录真实场景中各个法线方向上的光照情况)的增强光照算法。该算法避免使用HDR(High Dynamic Range)摄像机，而是用light map记录环境中的光照情况，并且对light map进行动态更新，能够达到实时交互的帧率，产生正确的明暗和阴影，且实现起来较为简单，通用性较强。The present invention proposes an enhanced lighting algorithm based on a dynamic light map (illumination map, used to record the lighting conditions in each normal direction in a real scene) in view of the fact that the existing augmented reality lighting technology has high requirements on hardware conditions. . This algorithm avoids the use of HDR (High Dynamic Range) cameras, but uses the light map to record the lighting conditions in the environment, and dynamically updates the light map, which can achieve the frame rate of real-time interaction, produce correct light and shade and shadows, and realize it Simpler and more versatile.

本发明的光照算法是基于自然特征的，而不是使用传统的人工标识。用于校准的物体是具有清晰图案的平面物体。算法中使用动态更新的light map记录环境光照，并用虚拟光源模拟真实光源来对虚拟物体进行照明，以产生与真实环境一致的光照。图1所示为AR系统光照生成过程。The illumination algorithm of the present invention is based on natural features, rather than using traditional artificial markers. The object used for calibration is a flat object with a clear pattern. In the algorithm, the dynamically updated light map is used to record the ambient light, and the virtual light source is used to simulate the real light source to illuminate the virtual object, so as to produce the light consistent with the real environment. Figure 1 shows the AR system lighting generation process.

1.场景的光照模型1. The lighting model of the scene

本发明所选择的校准图案是具有漫反射表面的，它向各个方向上反射的光线是相同的。在时刻t，校准图案上的某个点所反射的光线数量取决于该点的表面法线方向n_t。为了能够准确的计算法线方向，我们通常用以某个点为中心的一个小面片π来代替该点，则面片π在时刻t所接收的光能为：The calibration pattern selected in the present invention has a diffuse reflective surface, and the light reflected by it in all directions is the same. At time t, the amount of light reflected by a point on the calibration pattern depends on the direction n _t of the surface normal at that point. In order to calculate the normal direction accurately, we usually replace the point with a small patch π centered on a certain point, then the light energy received by the patch π at time t is:

${x x}_{π π,, t t} = = {Σ Σ}_{l l = = 00}^{L L} max max (({n no}_{t t} \cdot &Center Dot; {d d}_{l l},, 00)) {Ω Ω}_{l l} - - - - - - ((11))$

其中，Ω_l表示光源L的能量，d_l表示它的方向。在已经计算得到校准图案表面各点的反射系数的情况下，可以用下面的公式来表示面片π在t时刻的象素亮度I(π，t)：Among them, Ω _l represents the energy of the light source L, and d _l represents its direction. In the case that the reflection coefficient of each point on the surface of the calibration pattern has been calculated, the following formula can be used to represent the pixel brightness I(π,t) of the patch π at time t:

I(π，t)＝gρ_πx_t+b (2)I(π,t)=gρπ _x _t +b (2)

式中的ρ_π表示面片π上各点的反射系数的平均值，用π上的平均亮度来计算。g和b分别表示摄像机的信号放大量和信号偏置。ρ _π in the formula represents the average value of the reflection coefficient of each point on the surface π, which is calculated by the average brightness on π. g and b represent the signal amplification and signal bias of the camera respectively.

2.动态light map的生成2. Generation of dynamic light map

从上面的光照模型分析可知，要计算出面片π的亮度，就必须先计算光能Ω。我们也可认为空间中的某个位置(如校准图案所在的位置)，它的各个朝向(也就是以它为中心的某个球面上的每个点的法线方向)上所接收到的光能与法线方向n有关(如图2所示)，因此式(1)中的x_π，t可看成是n的函数。这样就可以用一个light map M(n)来表示该位置的光照情况。在这种情况下，虚拟物体表面的象素亮度可以表示成：From the above illumination model analysis, it can be seen that to calculate the brightness of the facet π, the light energy Ω must be calculated first. We can also think that at a certain position in space (such as the position of the calibration pattern), the light received on its various orientations (that is, the normal direction of each point on a certain spherical surface centered on it) can be related to the normal direction n (as shown in Figure 2), so x _{π, t} in formula (1) can be regarded as a function of n. In this way, a light map M(n) can be used to represent the lighting situation of the location. In this case, the pixel brightness on the surface of the virtual object can be expressed as:

I(v，t)＝gρ_vM_t(n_v)+b (3)I(v,t)=gρ _v M _t (n _v )+b (3)

式中的ρ_v表示虚拟物体表面的反射系数，在虚拟物体建模时定义。ρ _v in the formula represents the reflection coefficient of the virtual object surface, which is defined when the virtual object is modeled.

由(3)式可知，要得到虚拟物体表面的亮度，就必须先求出摄像机的g和b，可通过式(2)求得。式(2)中有三个未知量，即g、b和x_t。令g′＝1/g，b′＝1/b，则式(2)可表示为It can be seen from formula (3) that to obtain the brightness of the virtual object surface, the g and b of the camera must be obtained first, which can be obtained through formula (2). There are three unknowns in formula (2), namely g, b and x _t . Let g'=1/g, b'=1/b, then formula (2) can be expressed as

-I(π，t)g′+ρ_πx_t+b′＝0 (4)-I(π,t)g'+ρ _π x _t +b'＝0 (4)

将不同时刻所得到的式(4)放在一起就形成了一个方程组，如下：Putting the formula (4) obtained at different times together forms a system of equations, as follows:

$\{\begin{matrix} - - I I ((π π,, {t t}_{11})) {g g}^{' '} + + {ρ ρ}_{π π} {x x}_{t t 11} + + {b b}^{' '} = = 00 \\ - - I I ((π π,, {t t}_{22})) {g g}^{' '} + + {ρ ρ}_{π π} {x x}_{t t 22} + + {b b}^{' '} = = 00 \\ \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; \\ - - I I ((π π,, {t t}_{n no})) {g g}^{' '} + + {ρ ρ}_{π π} {x x}_{tn tn} + + {b b}^{' '} = = 00 \end{matrix} - - - - - - ((55))$

方程组(5)也可以表述成式(6)。为简化表达式，式中用I_tn表示式(5)中的I(π，t_n)：Equation (5) can also be expressed as formula (6). To simplify the expression, I(π,t _n ) in formula (5) is represented by I _tn in the formula:

$\{\begin{matrix} - - {I I}_{{t t}_{11}} {g g}^{' '} + + {b b}^{' '} + + {ρ ρ}_{π π} {x x}_{{t t}_{11}} + + 00 \cdot &Center Dot; {x x}_{{t t}_{22}} + + \cdot &Center Dot; \cdot \cdot \cdot \cdot + + 00 \cdot \cdot {x x}_{{t t}_{n no}} = = 00 \\ - - {I I}_{{t t}_{22}} {g g}^{' '} + + {b b}^{' '} + + 00 \cdot &Center Dot; {x x}_{{t t}_{11}} + + {ρ ρ}_{π π} {x x}_{{t t}_{22}} + + \cdot &Center Dot; \cdot \cdot \cdot &Center Dot; + + 00 \cdot &Center Dot; {x x}_{{t t}_{n no}} = = 00 \\ \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; \cdot \cdot \cdot &Center Dot; \\ - - {I I}_{{t t}_{n no}} {g g}^{' '} + + {b b}^{' '} + + 00 \cdot \cdot {x x}_{{t t}_{11}} + + 00 \cdot &Center Dot; {x x}_{{t t}_{22}} + + \cdot \cdot \cdot &Center Dot; \cdot \cdot + + {ρ ρ}_{π π} {x x}_{{t t}_{n no}} = = 00 \end{matrix} - - - - - - ((66))$

将方程组(6)转化成矩阵形式如下：Transform the equation group (6) into a matrix form as follows:

$[\begin{matrix} - - {I I}_{{t t}_{11}} & 11 & {ρ ρ}_{π π} & 00 \cdot \cdot \cdot \cdot \cdot \cdot 00 \\ - - {I I}_{{t t}_{22}} & 11 & 00 & {ρ ρ}_{π π} \cdot &Center Dot; \cdot \cdot \cdot \cdot 00 \\ \cdot \cdot \cdot \cdot \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; \\ - - {I I}_{{t t}_{n no}} & 11 & 00 & 00 \cdot &Center Dot; \cdot &Center Dot; \cdot &Center Dot; {ρ ρ}_{π π} \end{matrix}] [\begin{matrix} {g g}^{' '} \\ {b b}^{' '} \\ {x x}_{{t t}_{11}} \\ \cdot &Center Dot; \\ \cdot &Center Dot; \\ \cdot &Center Dot; \\ {x x}_{{t t}_{n no}} \end{matrix}] = = 00 - - - - - - ((77))$

解式(7)可得到多个特征值和多个特征向量，而与最小特征值相对应的特征向量就是我们要求的值。解向量中的(x_t1，x_t2，…，x_tn)即为形成M(n)所需的各个法线方向的光能值。Solving formula (7) can get multiple eigenvalues and multiple eigenvectors, and the eigenvector corresponding to the smallest eigenvalue is the value we require. (x _t1 , x _t2 ,..., x _tn ) in the solution vector are the light energy values in each normal direction required to form M(n).

以上的计算结果是在光照不变的情况下得到的。当校准物体的空间位置变化或者光源发生改变的时候，虚拟物体的光照情况也会随之改变。因此，必须对light map进行及时的更新，以适应环境光照的变化。The above calculation results are obtained under the condition of constant illumination. When the spatial position of the calibration object changes or the light source changes, the lighting conditions of the virtual object will also change accordingly. Therefore, the light map must be updated in time to adapt to changes in ambient lighting.

令M_t(n)为t时刻的light map，则t+1时刻的light map可以用下面的等式来表示：Let M _t (n) be the light map at time t, then the light map at time t+1 can be expressed by the following equation:

M_t+1(n)＝(1-f(n))M_t(n)+f(n)x_t+1 (8)M _t+1 (n)=(1-f(n))M _t (n)+f(n)x _t+1 (8)

其中，

Figure DEST_PATH_GA20191945200910044517X01D00043

in,

式(8)中的x_t+1是t+1时刻所观察到的法线方向为n_t+1的面片所接收的光能。式(9)中的s是一个模糊常量。事实上，当光照突然发生变化时，该方法不能够及时地更新整个light map，而是每一帧都只能采样到一个法线方向并更新该法线方向上的光能值，而其它法线方向上的光能值保持不变。因此，我们在建立了light map以后，对每一个新的帧都要进行更新，从而最大程度的达到及时更新的效果。x _t+1 in formula (8) is the light energy received by the facet with normal direction n _{t+1 observed at time t+1} . s in formula (9) is a fuzzy constant. In fact, when the light changes suddenly, this method cannot update the entire light map in time, but can only sample one normal direction in each frame and update the light energy value in the normal direction, while other methods The light energy value in the direction of the line remains constant. Therefore, after we have established the light map, we must update each new frame, so as to achieve the effect of timely update to the greatest extent.

在绘制虚拟物体时，light map可作为纹理贴图来表现虚拟物体表面的明暗状况。When drawing a virtual object, the light map can be used as a texture map to express the light and dark conditions of the virtual object surface.

3.光源位置分析3. Light source location analysis

为了确定AR场景中的阴影状况，我们需要计算真实环境中的光源位置。当真实环境中只有一个光源的时候，能够很容易模拟出光源的位置信息：扫描整个light map，找出与最大光能值相对应的法线方向n_s，则可以确定该方向就是光源所在的方向。当真实环境中存在不只一个光源的时候，可以用多个虚拟光源来模拟真实光源。我们给定一个较大的亮度值I_c作为临界亮度值，当某个法线方向上的光能值大于(或等于)I_c时，就认为该方向上可能存在一个光源；反之，则认为该方向上不存在光源。对于每个可能存在光源的法线方向n，我们进行如下操作以确定该方向上是否真的存在光源：首先，定义一个较小的角度ε来表示临界角度。然后，逐个扫描light map中的法线方向，当某个法线方向n′与法线方向n的夹角小于临界角度ε时，将n′看作是n的邻近法线方向。最后，判断各个邻近法线方向上的光强度是否小于n上的光强度，如果所有邻近法线方向上的光强度都小于n上的光强度，则法线方向n上必定存在光源，否则，不存在光源。这个过程可以用式(10)来表示。所有的虚拟光源都分布在以虚拟物体为中心的球面上，如图3所示。In order to determine the shadow situation in the AR scene, we need to calculate the position of the light source in the real environment. When there is only one light source in the real environment, it is easy to simulate the position information of the light source: scan the entire light map and find out the normal direction n _s corresponding to the maximum light energy value, then it can be determined that this direction is where the light source is located direction. When there is more than one light source in the real environment, multiple virtual light sources can be used to simulate the real light source. We give a large brightness value I _c as the critical brightness value, when the light energy value in a certain normal direction is greater than (or equal to) I _c , it is considered that there may be a light source in this direction; otherwise, it is considered that There are no light sources in this direction. For each normal direction n where there may be a light source, we perform the following operations to determine whether there is really a light source in this direction: First, define a small angle ε to represent the critical angle. Then, scan the normal direction in the light map one by one, when the angle between a certain normal direction n' and the normal direction n When it is less than the critical angle ε, n' is regarded as the adjacent normal direction of n. Finally, judge whether the light intensity in each adjacent normal direction is less than the light intensity on n, if the light intensity in all adjacent normal directions is less than the light intensity on n, then there must be a light source in the normal direction n, otherwise, There is no light source. This process can be represented by formula (10). All virtual light sources are distributed on a spherical surface centered on the virtual object, as shown in Figure 3.

Figure DEST_PATH_GA20191945200910044517X01D00052

式(10)中的I、I_n、I_n′分别表示环境中的光能、法线方向n上的光能和法线方向n′上的光能。I, _In , and In _' in formula (10) represent the light energy in the environment, the light energy in the normal direction n, and the light energy in the normal direction n', respectively.

在确定了所有虚拟光源的位置后，就可以利用Phong光照模型(一个可接受的“标准”模型，它反映了漫射、镜面反射和环境光对物体表面作用的结合)来计算增强现实场景中的阴影。After determining the positions of all virtual light sources, the Phong lighting model (an accepted "standard" model that reflects the combined effects of diffuse, specular, and ambient light on surfaces) can be used to calculate the shadows.

4.阴影生成4. Shadow generation

本发明中所用的虚拟物体是由许多三角形面片组成的，每一个面片对应着三个顶点v₁(x₁，y₁，z₁)、v₂(x₂，y₂，z₂)、v₃(x₃，y₃，z₃)。这三个顶点可以确定空间中一个唯一的平面：The virtual object used in the present invention is composed of many triangular faces, each face corresponds to three vertices v ₁ (x ₁ , y ₁ , z ₁ ), v ₂ (x ₂ , y ₂ , z ₂ ) , v ₃ (x ₃ , y ₃ , z ₃ ). These three vertices define a unique plane in space:

ax+by+cz+d＝0 (11)ax+by+cz+d＝0 (11)

其中，in,

a＝y₁*(z₂-z₃)+y₂*(z₃-z₁)+y₃*(z₁-z₂)；a=y ₁ *(z ₂ -z ₃ )+y ₂ *(z ₃ -z ₁ )+y ₃ *(z ₁ -z ₂ );

b＝z₁*(x₂-x₃)+z₂*(x₃-x₁)+z₃*(x₁-x₂)；b=z ₁ *(x ₂ -x ₃ )+z ₂ *(x ₃ -x ₁ )+z ₃ *(x ₁ -x ₂ );

c＝x₁*(y₂-y₃)+x₂*(y₃-y₁)+x₃*(y₁-y₂)；c＝x ₁ *(y ₂ -y ₃ )+x ₂ *(y ₃ -y ₁ )+x ₃ *(y ₁ -y ₂ );

d＝-(x₁*(y₂*z₃-y₃*z₂)+x₂*(y₃*z₁-y₁*z₃)+x₃*(y₁*z₂-y₂*z₁))。d＝-(x ₁ *(y ₂ *z ₃ -y ₃ *z ₂ )+x ₂ *(y ₃ *z ₁ -y ₁ *z ₃ )+x ₃ *(y ₁ *z ₂ -y ₂ *z ₁ )).

假设光源的位置为(x_p，y_p，z_p)，则在确定了某个面片的平面参数后，就可以利用下面的公式来判断该面片是否正对着光源：Assuming that the position of the light source is (x _p , y _p , z _p ), after determining the plane parameters of a patch, the following formula can be used to determine whether the patch is facing the light source:

S＝a*x_p+b*y_p+c*z_p+d (12)S＝a*x _p +b*y _p +c*z _p +d (12)

若计算出的S值大于0，则该面片正对着光源；否则，该面片背对着光源。If the calculated S value is greater than 0, the patch is facing the light source; otherwise, the patch is facing away from the light source.

逐个扫描虚拟物体中的三角形面片，当某个三角形面片正对着光源时，依次判断它的三条边是否为临界边：如果与某条边相邻的面片不存在，或者背对着光源，那么这个边就是临界边；否则，该边不为临界边。Scan the triangular faces in the virtual object one by one. When a triangular face is facing the light source, judge whether its three sides are critical edges in turn: if the face adjacent to a certain edge does not exist, or faces away from the light source, then this edge is a critical edge; otherwise, this edge is not a critical edge.

所有的临界边构成了虚拟物体上能够投射阴影的区域的边界。在已知场景中其它物体(包括虚拟物体和真实物体)的3D结构后，就可以利用计算机图形学中的阴影体方法生成该物体投射到其它物体上的阴影。All critical edges constitute the boundaries of the area on the virtual object that can cast shadows. After the 3D structure of other objects (including virtual objects and real objects) in the scene is known, the shadow volume method in computer graphics can be used to generate shadows cast by the object on other objects.

阴影体方法产生阴影有两个步骤：首先连接光源与各个临界边的两个端点，并延长至较远处，这样就可以形成由临界边的两个端点和延长线的末端所确定的半无限四边形，所有这样的四边形构成了物体的阴影体；然后根据阴影体与场景中其它物体的交线确定物体投射到其它物体上的阴影区域。The shadow volume method has two steps to generate shadows: first connect the light source and the two endpoints of each critical edge, and extend to a farther distance, so that a semi-infinity determined by the two endpoints of the critical edge and the end of the extension line can be formed Quadrilaterals, all such quadrilaterals constitute the shadow volume of the object; then determine the shadow area cast by the object on other objects based on the intersection of the shadow volume and other objects in the scene.

附图说明Description of drawings

图1：AR系统光照生成过程。Figure 1: AR system lighting generation process.

图2：物体表面光照示意图。Figure 2: Schematic diagram of object surface illumination.

图3：虚拟光源分布图Figure 3: Virtual light source distribution diagram

图4：光照结果图Figure 4: Lighting result graph

具体实施过程Specific implementation process

1.程序框架1. Program framework

输入：系统的输入为视频图像(VImage[i])和虚拟物体3D模型(obj)。Input: The input of the system is a video image (VImage[i]) and a virtual object 3D model (obj).

Figure DEST_PATH_GA20191945200910044517X01D00071

2.实验环境2. Experimental environment

本发明的硬件环境由三部分组成：一个USB摄像头、一台PC主机和一台LCD显示器。其中，PC主机的CPU为Core 2 Duo E6550，2.33GHz，内存为1GBRAM，显卡为NVIDIA GeForce 8600 GT，其OpenGL版本为2.1.1。The hardware environment of the present invention consists of three parts: a USB camera, a PC host and an LCD display. Among them, the CPU of the PC host is Core 2 Duo E6550, 2.33GHz, the memory is 1GB RAM, the graphics card is NVIDIA GeForce 8600 GT, and its OpenGL version is 2.1.1.

本发明的软件实现以Windows XP为平台，在Visual studio 2005环境下，使用OpenCV、OpenGL等库开发。The software realization of the present invention takes Windows XP as platform, under Visual studio 2005 environment, uses library developments such as OpenCV, OpenGL.

实验中首先利用USB摄像头拍摄真实场景的视频，在程序采集到一定的光照信息后，就将虚拟物体绘制到真实场景中，并对AR场景进行明暗增强，如图4所示。图中的平面物体为校准物体，茶壶为计算机生成的虚拟物体。In the experiment, the USB camera is used to shoot the video of the real scene. After the program collects certain lighting information, the virtual object is drawn into the real scene, and the AR scene is brightened and darkened, as shown in Figure 4. The planar object in the figure is a calibration object, and the teapot is a computer-generated virtual object.

在对虚拟茶壶进行明暗绘制的同时，程序计算环境中光源的位置。然后根据光源的位置以及虚、实物体的几何结构与空间位置，计算并绘制出“茶壶”投射到校准平面上的阴影，如图4所示。While shading the virtual teapot, the program calculates the position of the light source in the environment. Then, according to the position of the light source and the geometric structure and spatial position of the virtual and solid objects, the shadow projected by the "teapot" on the calibration plane is calculated and drawn, as shown in Figure 4.

Claims

1. An augmented reality system illumination algorithm based on a dynamic light map.

The method comprises the steps of acquiring pixel brightness of a calibration object in a video image based on a dynamic light map, calculating signal amplification amount and signal bias of a camera by a consistent illumination algorithm of the dynamic light map, generating a surface light and shade map of a virtual object according to environment illumination conditions in each normal direction recorded by the light map, and then calculating the direction of a light source to generate a shadow of an Augmented Reality (AR) scene. This enables consistent illumination of the scene and the virtual object in the video image. And the light map is updated according to each frame image, so that the scene with changed illumination can be adapted.

2. The illumination algorithm of the augmented reality system according to claim 1, wherein the relative illumination information in each normal direction in the environment is used to replace the environment absolute illumination condition, so as to illuminate the augmented reality scene, and achieve a more realistic illumination effect under simple hardware conditions.

3. The augmented reality system illumination algorithm of claim 1 or 2, wherein:

(1) instead of simulating a real light source with a virtual light source to illuminate the virtual object to realize the shading of the surface of the virtual object, the light map is used to generate the surface shading map of the virtual object according to the environment lighting conditions in the normal directions recorded by the light map.

(2) According to the brightness distribution relation in the light map, the direction of the light source in the real environment can be accurately judged.

(3) Various shadows in the scene are generated using the shadow volume method based on the orientation of the light source and the geometry of all objects in the augmented reality scene, including virtual objects and real objects.