CN106204431A - The display packing of intelligent glasses and device - Google Patents

Abstract

The present invention provides a display method and device for smart glasses. The method includes: adjusting and calibrating the display area on the display screen of the smart glasses so that the display area is consistent with what the user sees through the smart glasses. The external area is consistent; adjust the fixed position of the optical instrument on the smart glasses, so that the size of the first image data acquired by the optical instrument is consistent with the size of the second image data that the user sees through the smart glasses; according to the The second image data adjusts the display position of the first image data in the display area, so that the first image data overlaps with the second image data and displays the first image data.

Description

Display method and device for smart glasses

technical field

The invention relates to the field of smart wear, in particular to a display method and device for smart glasses.

Background technique

The current smart glasses can be divided into occlusion type and transmission type according to the display principle. The typical representative of occlusion type is the Oculus series. A device that blocks natural vision and overlays it with displayed images, this device completely blocks normal vision in the absence of power. Typical representatives of the see-through type are googleglass, Epson Moverio BT-200 smart glasses and Microsoft's hololens. This device itself does not block the natural field of view, and the image will only overlap with part of the natural field of view. And in the absence of energy It will not block the normal field of view. The technical means in this paper are mainly aimed at the transmission-type smart glasses, and the display of this type of smart glasses cannot completely cover the natural field of vision. The current mainstream method is to completely display the images collected by the camera on the lenticular display. As shown in Figure 1, the large solid line image is the image seen by the human eye through the lens in the real scene, and the small dotted line image is the image on the lens display screen. The image on the lenticular display is only the image collected by the camera, so the virtual image in the field of view will interfere with the real image, affecting user experience.

The disadvantage of the prior art is that visually, the displayed image overlaps with the actual image, as shown in Figure 1, and there is no correspondence between the overlap. The human eye sees the image as a picture-in-picture with repetitive content. In this way, the image information will be mixed, thereby affecting people's acquisition of normal information. At the same time, because the display window only occupies a small part of the visual field, the amount and content of displayed information are limited and the content is not clear enough.

Contents of the invention

The purpose of the present invention is to generate a corresponding relationship between the displayed content and the content of the actual field of view, so that the content of the natural field of view is not blocked and the corresponding information in the natural field of view can be supplemented. As shown in Figure 2, the displayed content It can completely coincide with the real image actually observed, so that the displayed information is more intuitive and can accurately correspond to the real object. At the same time, the addition of information on the virtual object can also correspond to the real object more accurately, improving the accuracy of the displayed information. Since the ratio of the virtual image to the real image is 1:1, it can be better The details of the image are guaranteed.

In order to solve the above problems, the present invention specifically provides a display method for smart glasses, the method includes: adjusting and calibrating the display area on the display screen of the smart glasses, so that the display area and the user see through the smart glasses The external area seen is consistent; adjust the fixed position of the optical instrument on the smart glasses, so that the size of the first image data acquired by the optical instrument is consistent with the size of the second image data that the user sees through the smart glasses; Adjusting the display position of the first image data in the display area according to the second image data, so that the first image data overlaps with the second image data and displays the first image data.

In the display method of the above-mentioned smart glasses, preferably, the adjusting and calibrating the display area on the display screen of the smart glasses includes: by observing a calibration board whose distance from the display screen is a predetermined threshold, using a checkerboard calibration method or a target A shape calibration method is used to calibrate the display area on the display screen.

In the above display method for smart glasses, preferably, after displaying the first image data, it further includes: monitoring the observation distance between the image acquisition end of the optical instrument and the collected real object, when the observation distance is less than When the predetermined threshold is reached, the first image data displayed on the display screen is enlarged or reduced according to a predetermined ratio.

In the above display method for smart glasses, preferably, the overlapping of the first image data and the second image data includes: the user observes and adjusts the display position of the first image data so that the first image The data is overlapped with the second image data; or, the vision of the human eye is simulated through a pre-set sampling optical device, and the second image data obtained by the sampling optical device is matched and overlapped with the first image data.

In the above display method for smart glasses, preferably, simulating human vision through the preset sampling optical device and matching and overlapping the second image data obtained by the sampling optical device with the first image data includes: fixing the second image data, moving the first image data horizontally or vertically according to a predetermined pixel unit, and recording the similarity between the moved first image data and the second image data obtained by comparing the two degree value; by moving the first image data horizontally or vertically multiple times, the similarity value is obtained to form a similarity matrix; according to the similarity matrix, the method for determining the horizontal movement or vertical movement of the first image data is obtained moving distance; moving the first image data according to the moving distance.

In the above display method for smart glasses, preferably, after displaying the first image data, it further includes: obtaining the observation distance between the user's glasses and the display screen, and adjusting the viewing distance according to predetermined parameters according to the observation distance. output and display on the display screen after determining the size and/or display position of the first image data.

In the above-mentioned display method for smart glasses, preferably, after displaying the first image data, it further includes: monitoring a control instruction input by the user, adjusting the first image data according to the control instruction and then displaying the first image data on the display screen The above output is displayed.

In the above-mentioned display method for smart glasses, preferably, the monitoring the control instruction input by the user, adjusting the first image data according to the control instruction and then outputting and displaying on the display screen includes: according to the control instruction, monitoring the displacement of the smart glasses, and outputting and displaying the first image data on the display screen after enlarging or reducing the first image data according to the displacement.

In the above display method for smart glasses, preferably, the monitoring the control command input by the user, outputting and displaying the first image data on the display screen after adjusting the first image data according to the control command further includes: monitoring the first Whether calibration image data is included in the image data, if calibration image data is included, the monitoring picture containing calibration image data in the first image data is intercepted; according to the control instruction, the monitoring picture is compared with the model picture in the database , obtaining the calibration model corresponding to the calibration image data; adding the calibration model to the first image data, and adjusting the size and display position of the calibration model; superimposing the first calibration model with the calibration model The image data is output and displayed on the display screen.

In the above display method for smart glasses, preferably, the distance between the user's glasses and the display screen is greater than or equal to 0.5 cm.

The present invention also provides a display device for smart glasses, the display device includes a calibration module, a model adjustment module and a displacement adjustment module; the calibration module is used to adjust and calibrate the display area on the display screen of the smart glasses, so that The display area is consistent with the external area seen by the user through the smart glasses; the model adjustment module is used to correspondingly adjust the first obtained by the optical instrument according to different fixed positions of the optical instrument on the smart glasses. image data, so that the size of the first image data is consistent with the size of the second image data that the user sees through the smart glasses; the displacement adjustment module is used to adjust the first image data according to the second image data The display position in the display area is such that the first image data and the second image data are overlapped and the first image data is displayed.

In the above-mentioned display device for smart glasses, preferably, the display device further includes a distance determination module, and the distance determination module is used to obtain the observation distance between the user's glasses and the display screen, and collect the distance from the optical instrument The observation distance between the image collection end of the optical instrument and the collected real object is determined, or the observation distance between the image collection end of the optical instrument and the collected real object is judged according to the first image data.

The beneficial technical effects of the present invention are: the display method of the smart glasses can display the information corresponding to the actual scene on the smart glasses in equal proportions and corresponding positions, so that it is not easy to block the field of view, no information confusion occurs, and more details can be displayed. Ensure the correspondence and accuracy of information display; at the same time, with the cooperation of the ranging function, it can provide better compensation to ensure the use effect at close range, and according to the corresponding relationship, users can obtain more real scenes and illusions through smart glasses A whole new experience with collections.

In order to make the above and other objects, features and advantages of the present invention more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic diagram of a display screen of smart glasses in the prior art;

FIG. 2 is a schematic diagram of a screen displayed by the display method of smart glasses provided by the present invention;

Fig. 3 is a schematic diagram of the effect of the display method of the smart glasses provided by the present invention;

FIG. 4 is a schematic flowchart of a display method for smart glasses provided by the present invention;

FIG. 5 is a schematic diagram of the principle of a display method for smart glasses provided in a preferred embodiment of the present invention.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Please refer to Fig. 2 and Fig. 4, the present invention provides a display mode of smart glasses. This display mode allows the human eye to observe the natural field of view and the ratio of objects seen through virtual imaging to be 1:1. The position of the image also coincides with the position of the image in the natural field of view to ensure accurate display of detailed information; the display method mainly includes the following implementation steps: S401 Adjust and calibrate the display area on the display screen of the smart glasses so that the display area It is consistent with the external area seen by the user through the smart glasses; S402 adjusts the fixed position of the optical instrument on the smart glasses, so that the first image data acquired by the optical instrument is consistent with the area seen by the user through the smart glasses. The received second image data are of the same size; S403 Adjust the display position of the first image data in the display area according to the second image data, so that the first image data and the second image data overlap and display the the first image data.

The main purpose in the above step S401 is to calibrate the size of the display area of the display screen. Specifically, by observing the checkerboard calibration board with a predetermined threshold outside the display screen, the area on the display screen can be calibrated by the checkerboard calibration method or the target calibration method. Display area; for example: when performing the calibration of the display area, the calibration method adopts a checkerboard calibration board, where the size of the checkerboard calibration board should be determined according to the display window size of the display, and the size can be less than 20 times the size of the display window, as close to 20 times as possible , more than 20 times may cause the calibration plate to be incompletely displayed, and the calibration plate is too small to increase the calibration error. Observe the checkerboard calibration board through the display screen, and pull the calibration board away at the same time. The farther the distance, the better, at least 20 times the distance e from the human eye to the display screen. In general, we think that e=1cm, then the distance should be more than 20cm , so that the error is guaranteed to be below 5%. At the same time, if the calibration board is too far away, the calibration board cannot fill the display screen, and the calibration purpose cannot be achieved. It is better to display the entire calibration board in the transparent display screen, so as to effectively reduce Error rate.

In the above step S402, the size of the displayed content is mainly calibrated, so that the size ratio of the displayed content is consistent with the ratio of human observation, so as to facilitate the overlapping of later images, so that the size of the displayed object seen by the user through the display screen is the same as that of the human eye. The actual size of the normal observation is the same; in this process, the human eye can be used to calibrate directly, or the data collected by the camera can be used instead of the human eye for calibration; the former method is the direct calibration of the human eye. On the display screen, while being observed by the user, the image display ratio and size are adjusted independently. This process is mainly judged by the human eye and then adjusted by an external control device or other means to the first image data displayed on the display screen. ; Regarding the latter method of calibrating data collected by a camera instead of the human eye, it is mainly to set up a camera through the display screen to simulate the position of the human eye for image collection. The collected image can be considered as the image seen by the human eye, and the size of the displayed content is adjusted The method of consistent proportion is to use the existing camera for sampling without displaying virtual content. This camera is called a sampling camera. It is used for calibration and records the size of the sampled content (you can use the specific number of pixels included) to indicate), and then block the display screen to ensure that the sampling camera can only collect the content of the display screen (this content does not include any transmitted real world) and display the virtual image collected by the camera (device camera) on the display screen, At the same time, the image is gradually enlarged or reduced, the sampling camera records the transformation process, and the process data is compared with the previously collected physical content, and the zoom-in and zoom-out parameters corresponding to the image with the closest number of pixels are recorded, and considered at this time The image is consistent with the actual size of the human eye, and the range of pixel variation within plus or minus 20% is reserved for subsequent fine-tuning by users. The above two methods can be selected and used according to the actual situation. Of course, adjusting the size ratio of the displayed content is not only the above two methods. The above method is only a clearer illustration. The process plan adopted does not impose any restrictions on its specific implementation process.

In a preferred embodiment of the present invention, the above step S403 may also include: allowing the user to observe and adjust the display position of the first image data so that the first image data overlaps the second image data; or by presetting The sampling optical device simulates human vision and matches and overlaps the second image data obtained by the sampling optical device with the first image data. The method of simulating the human eye with an optical device further includes: fixing the second image data, moving the first image data horizontally or vertically according to predetermined pixel units, and recording the moved first image data and The similarity value between the two obtained by comparing the second image data; by moving the first image data multiple times horizontally or vertically, the similarity value is obtained to form a similarity matrix; according to the similarity matrix , obtaining a moving distance for determining the lateral movement or vertical movement of the first image data; and moving the first image data according to the moving distance. Specifically, in actual use, firstly, the similarity matrix is composed of the similarity values obtained by comparing the second image with the first image multiple times. For example, only one value is obtained for each movement, so through multiple movements in the horizontal and vertical directions, the Multiple values can be obtained, and then a matrix is formed from the values, and finally the normal distribution (one-dimensional) and Gaussian distribution (two-dimensional) are satisfied as constraints, and the extreme value of the Gaussian distribution function of the matrix is solved as the final extreme value result , and determine the horizontal or vertical movement distance of the first image data based on the extreme value result.

In the above-mentioned embodiment, the post-stage image overlapping is the adjustment of the image display position. The process adopted is similar to the calibration process in step S402. The camera can be set up through the display screen to simulate the position of the human eye for image acquisition. The acquired image It can be thought of as the image seen by the human eye. At this time, the projected image is compared with the sampled real image. The coverage similarity value of the projected image on the real image satisfies the Gaussian distribution (the similarity here can be the value of area overlap or pixel value interpolation, etc. There are many algorithms. , the present invention will not give examples one by one here), at this time, the corresponding extremum of the Gaussian distribution function can be obtained, and the extremum point is considered to be a point overlapping with the actual position, and the overall display image horizontally around the point is 10% of the width , the vertical direction is 10% of the height as the range for users to fine-tune.

In a preferred embodiment of the present invention, the first image data can be further calibrated after the display of the first image data, so that when the user uses smart glasses to observe close-up objects, the displayed data can be more accurate. Clearly, the present invention also provides the following processing method: monitor the observation distance between the image acquisition end of the optical instrument and the collected real object, and when the observation distance is less than a predetermined threshold, zoom in or out on the display screen according to a predetermined ratio The first image data displayed on the screen; wherein, the predetermined threshold can be set according to the actual situation of the smart glasses device, and can usually be set to 20 times the distance between the human eye and the display screen. When the human eye or an optical instrument observes a Scenes are often accompanied by out-of-focus situations. At this time, adjusting the display data of optical instruments according to the preset threshold range can effectively help users observe close-up objects more clearly; for example: when actually observing close-up objects, when the smart glasses detect the object When the distance from the acquisition end of the optical instrument is lower than a threshold, the image can be compensated according to the formula c=a(1+e/b), where the distance from the human eye to the imaging is a, the distance from the human eye to the real object is b, and the camera The distance to the imaging is c, the distance from the camera to the real object is d, and the distance from the human eye to the display is e; e is a constant by default, that is to say, the size of the displayed image is only related to b, then we can measure the distance b to The size of the displayed picture is adjusted, and when b shrinks, the image is automatically enlarged so as to ensure the accuracy of the image display. For example, the device itself has a ranging function, that is, the b value can be obtained here. If the device is a distance measuring device on the human eye side, that is, the value of e is also known. Then (1+e/b) in c=a(1+e/b) can get the specific value m, that is, c=am at this time means that the ratio of the image on the display screen to the actual image is m, assuming If the ratio value obtained by the previous calibration is n, the required scaling value is m/n, that is, the existing image is reduced by n times and then enlarged by m times. The actual operation is often performed directly using the specific value of m/n. If the distance e on the side of the human eye is unknown, the default e is a constant (this value does not change under normal circumstances), and the standard calibration n=(1+e/b) can get the e value. The core application of the distance measurement of b is that the scale of zooming in and out can change with the distance of the observed object, so as to avoid the occurrence of distortion.

It should be noted that in general, the virtual image completely coincides with the image seen by the human eye through the lens. However, according to the principle based on the formula c=a(1+e/b), the image matching can only be realized when b is large enough and e is small enough; therefore, if the smart glasses themselves do not have the distance measuring function, e needs to be limited A certain effective range (0.5cm-2cm), that is to say, the distance between the human eye and the display screen needs to be at least 0.5 cm to ensure the display effect.

In the above embodiments, to display the image data captured by the optical instrument as consistent with the scene observed by the human eye, it is necessary to process the captured image data, which involves the distance between the human eye and the display screen, the distance between the camera and the real object. Parameters such as distance, in order to more clearly illustrate the process and principle of the present invention, as shown in Figure 3 below, the principle of the present invention is further explained; in the process of using smart glasses, the human eye sees the physical image through the chip, if To achieve an image size of 1:1 and coincident positions, the displayed image must be consistent with the lens imaging image. That is to say, the image of the object observed by the human eye through the lens should be consistent with the image collected by the camera and displayed on the monitor. Of course, the image collected by the camera is an inverted real image, and here it is equivalent to the upright virtual image in front; In most cases, the image collected by the camera is not consistent with the real object seen by the human eye. Therefore, in order to ensure that the displayed image is consistent with the actual object ratio observed by the human eye, it is necessary to further correct the displayed image; in this process, first It is necessary to set the distance from the human eye 302 to the imaging 3021 as a, the distance from the human eye 302 to the real object 301 as b, the distance from the camera 303 to the imaging 3022 as c, and the distance from the camera 303 to the real object 301 as d; The size of the lens image observed by the human eye must be consistent with the formula a/b=c/d. Here c is variable through focal length and zooming in and out of the image. b and d satisfy the relationship d=b+e, assuming that the camera 303 is on a straight line with the human eye 302 and the real object 301, then e is the distance between b and d. So the formula can be changed to a/b=c/(b+e). The formula c=a(1+e/b) is obtained. In this formula, a is a constant, because the example of imaging between the human eye and the lens can be considered constant; e is also a constant, because the distance between the camera 303 and the human eye 301 can also be considered constant during wearing smart glasses , of course, the general camera should be in front of the human eye, that is, the example of d should be smaller than the example of b, that is to say, e<0 here. In this way we know that the variable c only has a certain relationship with b to satisfy the formula c=a(1+e/b), and a and e are constant and known. This formula also shows that if the observed object is infinitely far away, e/b is close to 0, c=a, so the image captured by the camera should be consistent with the lens image observed by the human eye. That is to say, if e is small enough and b is large enough, we think c=a, that is, the size of the camera image can be considered without considering the distance between the human eye and the object. In general, the distance of e is less than or equal to 1cm, that is to say, if the object is 20cm, the error should be about 5%. If the object is within 2% of the error at 50cm, the error is less than 1% above 100cm. Of course, we can use the formula to get the accurate size of the generated virtual image. In this way, through the above method, after knowing the corresponding parameter b value through the distance measurement or other distance measurement methods that come with the smart glasses in actual work, we can use the above formula to determine the size of the virtual image. The image is adaptively adjusted to ensure the accuracy of the display of c; the distance measurement function can also be limited to when the distance of the object is close to a certain extent, and then the formula algorithm is used for compensation. In other cases, it is considered that c=a.

Of course, although the value of c in the above-mentioned embodiment can be changed by adjusting the parameters of the camera, the value of c cannot be directly set in actual work, which is equivalent to the inability to directly set the size of the displayed image. Therefore, the present invention can The above purpose can be achieved by changing the parameters of the camera or the size of the displayed image. Based on this purpose, it is necessary to use a certain calibration method to initialize the relationship between the displayed content and the content captured by the camera; in order to ensure the accuracy of the calibration, the target can be The object (calibration board, the calibration board here is generally a black and white grid calibration board similar to a chess board) is placed 1 meter away, the number of grids on the calibration board is observed through the display chip, and the calibration board is moved so that it is in the direction obtained by the display window. The number of grids is exactly an integer (because the image itself does not have the problem of deformation, so choose horizontal or vertical to be an integer); keep the positions of the smart glasses and the calibration board unchanged, turn on the camera, display the shooting content, and adjust the display content to With the same number of squares, we can think that the ratio of the image displayed at this time is 1 to 1 with the image observed by the human eye; on this basis, the displayed image is translated or moved horizontally to align its position, or The target calibration board can be re-calibrated so that the displayed content coincides with the content observed by the human eye.

Due to the difference of the user itself, the distance of a is likely to change. At the same time, due to the wearing relationship, the relative position of the human eye and the display chip is not the same as the calibration position. Therefore, if necessary, the user needs to wear it. Carry out secondary adjustment, that is, further image calibration; the display method of smart glasses provided by the present invention further includes: acquiring the observation distance between the user's glasses and the display screen, and adjusting the first viewing distance according to predetermined parameters according to the observation distance. The size and/or display position of an image data is output and displayed on the display screen. In this way, the same observation experience can be effectively given to different users through this solution. Based on the different distances between different users and the display screen when wearing smart glasses, the first image data can be purposely adjusted so that it can be accurate and real. The image data is overlaid.

In the above-mentioned embodiment, the predetermined parameters can be pre-tested and recorded for different observation distances, and the parameters that need to be adjusted for the first image data, so that when the user actually wears the smart glasses, the first image data can be correspondingly adjusted according to the observation distance. image data, the present invention will not describe it in detail here; of course, when adjusting the first image data, it can also be adjusted by the user independently. In a preferred embodiment of the present invention, it is also provided: monitor the control command input by the user, according to the The control instruction adjusts the first image data and then outputs and displays them on the display screen; that is to say, the user can adjust the first image data independently according to the specific feeling when wearing it or the parameters known in advance; for example: when the smart glasses are set , set the distance e between the human eye and the display screen as 1cm, and the distance e between the human eye and the display screen of a certain user is 2cm, then the first image data can be adjusted by the parameter amount corresponding to 2cm, so as to achieve the second The purpose of overlapping the first image data with the second image data; if the user cannot know the above parameters and cannot accurately judge the distance between the human eye and the display screen, at this time, the present invention also provides monitoring of the smart glasses according to the control instructions According to the displacement situation, the first image data is enlarged or reduced according to the displacement situation, and then output and displayed on the display screen. For example, a gyroscope is used as an input device for translation. The gyroscope is generally integrated in smart glasses to collect the posture of the human head. When calibration is required, that is, when the control command is received, the current gyroscope is first recorded. After the user observes the real image and the virtual image in the display chip, the user turns the smart glasses along a certain direction; for example, if the virtual image does not overlap with the real image, and the virtual image is above the real image, then the The virtual image moves down as a whole. At this time, the operation that the user needs to do is to lower the head, so that the gyroscope will record that the direction of the head deviates from the horizontal downward, and feed back this information to move the overall display content downward. The operations in other directions are similar ; After calibration, the image captured by the camera and displayed on the monitor can achieve the effect of Figure 2, the information display is more accurate, and the details of the displayed content are preserved. Those skilled in the art should know that the gyroscope is only a better technical solution adopted by the present invention to explain the user's adjustment of the first image data more clearly. The present invention does not limit the user to adjust the first image data independently. Equipment and methods.

On the basis of the corresponding display of the above-mentioned real objects and images, the present invention also provides a preferred embodiment to simply explain the application scene and real-time process of the display method. After the calibration of the first image data and the second image data is completed, at this time The image displayed by the smart glasses is consistent with the real situation. At this time, the smart glasses further monitor whether the first image data contains the calibration image data. If the calibration image data is included, the first image data contains the calibration image. A monitoring picture of the data; comparing the monitoring picture with the model picture in the database according to the control instruction to obtain a calibration model corresponding to the calibration image data; adding the calibration model to the first image data, and Adjusting the size and display position of the calibration model; outputting and displaying the first image data superimposed with the calibration model on the display screen. For example: the user can write down or mark a predetermined identifier on the external object, such as text, symbol or pattern, etc.; when the smart glasses detect that the first image data contains the identification information, the image is intercepted and shared with the The cloud database is compared to determine the type of object marked by the user. The method for image recognition and judgment can be implemented using existing technologies, and the present invention will not be described in detail here; The model data such as the model of the object is for the user to confirm. After the user confirms, the model data is superimposed on the first image data for user reference; The internal structure of the object can also be constructed by rendering on a computer or other equipment according to specific drawings or instructions. Through this method, the user can superimpose the model data that needs to be referenced on the real display interface for reference and judgment, for example: in the doctor's execution During the operation, the specified symbol can be calibrated on the patient's body. When the smart glasses detect the symbol, it can obtain various models of the corresponding human body from the database. A doctor chooses images such as the human artery perspective model or the bone perspective model. At this time, the smart glasses The glasses adjust the model according to the proportion of the real object and superimpose the display on the first image data. The doctor can accurately judge the user's physical condition through the displayed content and provide more accurate operations such as surgery; When the house is being decorated and other operations, in order to ensure that the construction will not affect the safety of the house, damage to the load-bearing walls, internal wiring, etc., it is necessary to study the relevant drawings of the house; The internal model of the house can accurately and real-time understand the situation of the house during construction, which greatly saves the time of the construction personnel and ensures the personal safety of the construction personnel during the construction process; of course, the present invention can be used in many scenes in life through the above method. The present invention will not give examples one by one here.

Please refer to Figure 5, in a preferred embodiment of the present invention, a schematic diagram of the principle presented to the user when actually using the smart glasses display method provided by the present invention is given. In Figure 5, the user is in the initial stage, normally The observed scene is the palm, and then the smart glasses are worn to further understand the skeletal conditions of the palm. At this time, the smart glasses superimpose the perspective view of the palm bones obtained through CT shooting on the palm for display. What can be seen is the perspective view of the palm skeleton taken by CT, and the original palm is covered and no longer displayed; On the original object, other objects are not changed or intervened; for example, when the object in front of the user is the whole arm, and the user only needs to understand the skeleton condition of the palm, the display content will be overlaid on the original palm position to display the skeleton condition diagram , without making any changes to the arm; or in other words, when the house is being renovated, in order to more clearly display the wiring in the wall, what the user sees through the smart glasses at this moment is that the position where the wall contains the wiring shows the wiring The trend situation, while other parts, such as the place outside the wall line, remain in their original state and are displayed normally.

Those skilled in the art will know that the image observed by one eye is a plane image, and the depth of field can only be obtained by observation with two eyes. Therefore, when the user wears smart glasses, in order to ensure that the image observed by the user is consistent with the real image, the present invention uses at least two Different image data are acquired and processed, and then displayed on the left and right display screens of the smart glasses respectively, so that the user can obtain a more realistic experience when viewing the smart glasses later. At the same time, the display screen of the smart glasses provided by the present invention is a transparent display panel capable of displaying pictures, so that the user can observe the actual external situation through the smart glasses when using or not using the smart glasses.

The display method of the smart glasses provided by the present invention can display the information corresponding to the actual scene on the smart glasses at the corresponding positions in equal proportions, so that it is not easy to block the field of vision, does not cause information confusion, and can display more details to ensure the correspondence of information display and accurate; at the same time, with the cooperation of the ranging function, it can provide better compensation to ensure the use effect at close range, and according to the corresponding relationship, users can obtain more real-world and virtual experience through smart glasses.

The present invention also provides a display device for smart glasses, the display device includes a calibration module, a model adjustment module and a displacement adjustment module; the calibration module is used to adjust and calibrate the display area on the display screen of the smart glasses, so that The display area is consistent with the external area seen by the user through the smart glasses; the model adjustment module is used to correspondingly adjust the first obtained by the optical instrument according to different fixed positions of the optical instrument on the smart glasses. image data, so that the size of the first image data is consistent with the size of the second image data that the user sees through the smart glasses; the displacement adjustment module is used to adjust the first image data according to the second image data The display position in the display area is such that the first image data and the second image data are overlapped and the first image data is displayed.

In the above-mentioned embodiments, the display device provided by the present invention may further include a distance judging module, the distance judging module is used to obtain the observation distance between the user's glasses and the display screen, and collect the image of the optical instrument The observation distance between the acquisition end and the collected objects, or the observation distance between the image acquisition end of the optical instrument and the collected objects is judged according to the first image data; the distance determination module accurately obtains various distance parameters , so that the smart glasses can adjust the first image data more accurately and effectively according to these parameters later.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

In the present invention, specific examples have been applied to explain the principles and implementation methods of the present invention, and the descriptions of the above examples are only used to help understand the method of the present invention and its core idea; meanwhile, for those of ordinary skill in the art, according to this The idea of the invention will have changes in the specific implementation and scope of application. To sum up, the contents of this specification should not be construed as limiting the present invention.

Claims (12)

1. A display method for smart glasses, characterized in that the method comprises: Adjusting and calibrating the display area on the display screen of the smart glasses, so that the display area is consistent with the external area seen by the user through the smart glasses; Adjusting the fixed position of the optical instrument on the smart glasses, so that the size of the first image data acquired by the optical instrument is consistent with the size of the second image data seen by the user through the smart glasses; The display position of the first image data in the display area is adjusted according to the second image data, so that the first image data and the second image data overlap and display the first image data. 2. The display method of smart glasses according to claim 1, wherein the adjusting and calibrating the display area on the display screen of the smart glasses comprises: observing a calibration plate with a predetermined threshold outside the display screen , using a checkerboard calibration method or a target calibration method to calibrate the display area on the display screen. 3. The display method of smart glasses according to claim 1, characterized in that, after displaying the first image data, it also includes: monitoring the observation distance between the image collection end of the optical instrument and the collected real objects , when the observation distance is smaller than a predetermined threshold, enlarge or reduce the first image data displayed on the display screen according to a predetermined ratio. 4. The display method of smart glasses according to claim 1, wherein the overlapping of the first image data and the second image data comprises: Observing and adjusting the display position of the first image data by the user, so that the first image data overlaps with the second image data; Or, the vision of the human eye is simulated through a pre-set sampling optical device, and the second image data obtained by the sampling optical device is matched and overlapped with the first image data. 5. The display method of smart glasses according to claim 4, characterized in that simulating human vision through the preset sampling optical device and combining the second image data obtained by the sampling optical device with the first image data Matching overlaps include: Fixing the second image data, moving the first image data horizontally or vertically according to predetermined pixel units, and recording the difference between the moved first image data and the second image data obtained by comparing The similarity value between; Obtaining the similarity values to form a similarity matrix by moving the first image data horizontally or vertically multiple times; According to the similarity matrix, obtain and determine the movement distance of the first image data moving horizontally or vertically; The first image data is moved according to the moving distance. 6. The display method of smart glasses according to claim 1, characterized in that, after displaying the first image data, it further comprises: acquiring the viewing distance between the user's glasses and the display screen, according to the After adjusting the size and/or display position of the first image data according to predetermined parameters, the observation distance is output and displayed on the display screen. 7. The display method of smart glasses according to claim 1, characterized in that, after displaying the first image data, it further comprises: monitoring a control instruction input by a user, and adjusting the first image according to the control instruction The data is then output and displayed on the display screen. 8. The display method for smart glasses according to claim 7, wherein the monitoring of the control command input by the user, and outputting and displaying on the display screen after adjusting the first image data according to the control command includes : According to the control instruction, monitor the displacement of the smart glasses, enlarge or reduce the first image data according to the displacement, and then output and display on the display screen. 9. The display method of smart glasses according to claim 7, characterized in that, after monitoring the control command input by the user, adjusting the first image data according to the control command and then outputting and displaying the first image data on the display screen Include: Monitoring whether the first image data contains calibration image data, if the calibration image data is included, intercepting the monitoring picture containing calibration image data in the first image data; comparing the monitoring picture with the model picture in the database according to the control instruction to obtain the calibration model corresponding to the calibration image data; adding the calibration model to the first image data, and adjusting the size and display position of the calibration model; Outputting and displaying the first image data superimposed with the calibration model on the display screen. 10. The display method for smart glasses according to claim 1, wherein the distance between the user's glasses and the display screen is greater than or equal to 0.5 cm. 11. A display device for smart glasses, characterized in that the display device comprises a calibration module, a model adjustment module and a displacement adjustment module; The calibration module is used to adjust and calibrate the display area on the display screen of the smart glasses, so that the display area is consistent with the external area seen by the user through the smart glasses; The model adjustment module is used to correspondingly adjust the first image data obtained by the optical instrument according to different fixed positions of the optical instrument on the smart glasses, so that the first image data and the user see through the smart glasses The size of the second image data received is the same; The displacement adjustment module is used to adjust the display position of the first image data in the display area according to the second image data, so that the first image data and the second image data overlap and display the first image data. image data. 12. The display device of smart glasses according to claim 11, characterized in that, the display device further comprises a distance determination module, the distance determination module is used to obtain the observation distance between the user's glasses and the display screen , and collect the observation distance between the image acquisition end of the optical instrument and the collected real object, or judge the observation distance between the image acquisition end of the optical instrument and the collected real object according to the first image data.