CN115047979B - Head-mounted display equipment control system and interaction method - Google Patents
Head-mounted display equipment control system and interaction method Download PDFInfo
- Publication number
- CN115047979B CN115047979B CN202210971651.XA CN202210971651A CN115047979B CN 115047979 B CN115047979 B CN 115047979B CN 202210971651 A CN202210971651 A CN 202210971651A CN 115047979 B CN115047979 B CN 115047979B
- Authority
- CN
- China
- Prior art keywords
- information
- head
- acquisition module
- limb
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
- Position Input By Displaying (AREA)
Abstract
The application discloses a head-mounted display device control system and an interaction method, wherein the system comprises a head-mounted display device and a limb information acquisition module, the limb information acquisition module is in wireless communication connection with the head-mounted display device, the head-mounted display device comprises a camera, a head action sensor and a host control module, the camera is used for acquiring user body shape information and environment light information, and the head action sensor is used for detecting head movement information; the limb information acquisition module is used for acquiring limb movement information of different limb parts and sending the limb movement information to the host control module; the host control module is used for establishing a holographic image based on the head movement information, the limb movement information, the user body shape information and the environment light information so as to send and display the holographic image on the opposite terminal device and receive and display the holographic image sent by the opposite terminal device. The method and the device solve the technical problem that the holographic image is poor in presentation effect due to the fact that the accuracy of the collected human body information is low.
Description
Technical Field
The application relates to the technical field of AR/VR, in particular to a head-mounted display device control system and an interaction method.
Background
At present, a lot of AR/VR products appear in the market, this type of product can be shot through the camera in real time and detect means and realize holographic image's transmission, however, because the camera on the head-mounted device is located user's head position, it is not good enough to shoot human holistic image angle, additionally, because the camera needs ambient light when shooting the image, the data that the camera was gathered under the too strong or too weak circumstances of light are not good, lead to the human information accuracy of gathering lower, and then lead to holographic image's the effect that presents not good.
Disclosure of Invention
The main purpose of the application is to provide a head-mounted display device control system and an interaction method, and the system and the method aim at solving the technical problem that in the prior art, because the accuracy of collected human body information is low, the presentation effect of a holographic image is poor.
In order to achieve the above object, the present application provides a head-mounted display device control system, control system includes head-mounted display device and limbs information acquisition module, limbs information acquisition module with head-mounted display device wireless communication connects, head-mounted display device includes host control module, wherein:
the camera is used for collecting user body shape information and environment light information;
the head motion sensor is used for detecting head movement information of a target user;
the body information acquisition module is used for acquiring body movement information of different body parts of the target user and sending the body movement information to the host control module;
the host control module is used for receiving the limb movement information sent by the limb information acquisition module, establishing a holographic image of the target user based on the head movement information, the limb movement information, the user body shape information and the ambient light information, sending and displaying the holographic image on an opposite terminal device which is in communication connection with the head-mounted display device, and receiving and displaying the holographic image sent by the opposite terminal device.
The application also provides a head-mounted display device interaction method, which comprises the following steps:
acquiring user body shape information and environment light information through a camera, and detecting head movement information of a target user through a head action sensor;
the limb movement information of different limb parts of a target user is acquired through a limb information acquisition module, and is sent to a host control module;
the method comprises the steps that limb movement information sent by a limb information acquisition module is received through a host control module, holographic images of a target user are established based on head movement information, the limb movement information, user body shape information and ambient light information, the holographic images are sent and displayed on opposite-end equipment which is in communication connection with head-mounted display equipment, and the holographic images sent by the opposite-end equipment are received and displayed.
The present application further provides a storage medium, which is a computer-readable storage medium, on which an interaction control program is stored, where the interaction control program is executed by a processor to implement the steps of the interaction method as described above.
The application provides a wear display device control system and mutual method, control system includes wear display device and limbs information acquisition module, limbs information acquisition module with wear display device wireless communication connection, wear display device includes camera, head motion sensor and host computer control module, wherein: the camera is used for collecting body shape information of a user and ambient light information; the head motion sensor is used for detecting head movement information of a target user; the limb information acquisition module is used for acquiring limb movement information of different limb parts of the target user and sending the limb movement information to the host control module; the host control module is used for receiving the limb movement information sent by the limb information acquisition module, establishing a holographic image of the target user based on the head movement information, the limb movement information, the user body shape information and the environment light information, sending and displaying the holographic image on opposite-end equipment which is in communication connection with the head-mounted display equipment, and receiving and displaying the holographic image sent by the opposite-end equipment, so that the aim that the limb movement information of different limb parts of the target user is accurately acquired through the limb information acquisition module is achieved, the requirements on the use environment light are low, human body action capture is not distorted under the conditions of strong light and weak light, an accurate holographic image is established based on the limb movement information, the holographic image is sent and displayed on the opposite-end equipment which is in communication connection with the head-mounted display equipment, the three-dimensional display effect of the holographic image is improved, and the use experience of the user is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a schematic diagram of a control system of the present application;
FIG. 2 is a schematic structural diagram of a wrist information collection module or an ankle information collection module according to the present application;
FIG. 3 is a schematic structural diagram of a waist information acquisition module according to the present application;
FIG. 4 is a schematic structural diagram of a head mounted display device according to the present application;
FIG. 5 is a schematic flow chart diagram illustrating a first embodiment of an interaction method according to the present application;
fig. 6 is a flowchart illustrating a second embodiment of the interaction method of the present application.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The embodiment of the application provides a wear display device control system, refer to fig. 1, fig. 1 is this application control system's structural schematic diagram, wherein, control system includes that head mounted display device 01 and limbs information acquisition module, limbs information acquisition module with head mounted display device wireless communication connects, wherein, head mounted display device includes equipment such as AR, VR and MR, limbs information acquisition module includes one or more in wrist information acquisition module 02, 03, ankle information acquisition module 05, 06 and waist information acquisition module 03.
Wherein, referring to fig. 2, fig. 2 is a schematic structural diagram of a wrist information collection module or an ankle information collection module in the present application, the wrist information collection module includes a wireless communication unit, a power supply unit, a control unit, a motion position sensor, a myoelectricity collection and analysis unit and a vibration motor unit, the wrist information collection module is worn on the wrist of a target user, the ankle information collection module includes a wireless communication unit, a power supply unit, a control unit, a motion position sensor, a myoelectricity collection and analysis unit and a vibration motor unit, the ankle information collection module is worn on the ankle of the target user, further, referring to fig. 3, fig. 3 is a schematic structural diagram of a waist information collection module in the present application, the waist information collection module includes a wireless communication unit, a power supply unit, a control unit, a motion position sensor and a vibration motor unit, the waist information acquisition module is worn on the waist of the target user, and it should be noted that the power supply unit is used for supplying power to equipment, the motion sensor is used for acquiring the movement track and the movement height of the target user corresponding to the limb part, the myoelectricity acquisition and analysis module is used for acquiring the myoelectricity signal of the hand and/or the foot and analyzing the myoelectricity signal to obtain the gesture information and/or the foot motion information of the target user, such as the change of the acquired skin bioelectricity, so as to accurately judge the activity state of the fingers, toes, palms and soles, and the wireless communication unit is used for transmitting the limb movement information, such as the movement track, the movement height, the gesture information and/or the foot motion information, to a host control module in the head-mounted display equipment, such as wifi, and the like, the control module is used for receiving commands issued by the head-mounted display device, such as motor vibration commands, and additionally, the vibration motor unit is used for vibrating during interaction and increasing interaction experience, such as shaking hands, hugging, obstacle avoidance and the like, wherein it needs to be explained that an additional information acquisition module can be added to the head-mounted display device control system to acquire data, such as additionally adding the information acquisition module to be worn on the forearm and/or the shank of a target user, so that more detailed limb action information is obtained, and human body information is fed back more accurately.
Additionally, referring to fig. 4, fig. 4 is a schematic structural diagram of the head-mounted display device in the present application, where the head-mounted display device includes a power supply unit, a host control module, a head motion sensor, an audio module, an optical machine imaging module, a camera, and a vibration motor unit, where the audio module is divided into an audio acquisition module, an audio transmission module, an audio reception module, and an audio playing module, it should be noted that the opposite-end device includes head-mounted devices such as AR, VR, and MR, and may further include home-furnishing-type display devices, terminal devices, for example, smart televisions, mobile phones, and flat panels, and a communication connection is established between the opposite-end device and the head-mounted display device.
The system comprises a power supply unit, a head motion sensor, a camera, a host control module, a wrist information acquisition module, an ankle information acquisition module and a waist information acquisition module, wherein the power supply unit is used for supplying power to equipment, the head motion sensor is used for acquiring a moving track and a moving height of the head of a target user, the vibration motor unit is used for vibrating during interaction to increase interaction experience, such as shaking hands, hugs and avoiding obstacles, the motion sensor is used for acquiring head moving information of the target user, the camera is used for acquiring body shape information and ambient light information of the user, for example, color information such as ambient light change and clothes color and skin color of the user is acquired, the host control module receives body moving information sent by the wrist information acquisition module, the ankle information acquisition module and the waist information acquisition module, and establishes a holographic image of the target user based on the head moving information, the body moving information, the user body shape information and the ambient light information so as to send and display the holographic image on an opposite terminal device which is in communication connection with the head-mounted display equipment.
In order to more clearly describe the working principle of the control system, the following steps a1 to a4, b1 to b4 and c1 to c2 are used for detailed description.
Step a1, the camera is used for collecting user body shape information and ambient light information;
step a2, the head motion sensor is used for detecting head movement information of a target user;
in this embodiment, it should be noted that, after the head-mounted display device is started, the head-mounted display device is automatically paired with each module in the limb information acquisition module, and after the pairing is successful, it is proved that normal communication activities can be performed, and then the camera in the head-mounted display device can perform real-time shooting and detection to acquire user shape information such as clothing color and skin color of a target user and acquire environment light information such as environment light intensity, and additionally, the head movement information of the target user is detected in real time through a head movement sensor in the head-mounted display device.
A3, the limb information acquisition module is used for acquiring limb movement information of different limb parts of the target user and sending the limb movement information to the host control module;
in this embodiment, it should be noted that the limb information acquisition module at least includes one or more of a wrist information acquisition module, an ankle information acquisition module and a waist information acquisition module, where the number of the information acquisition modules is not specifically limited, preferably, the number of the wrist information acquisition modules is 2, and the wrist information acquisition modules are respectively worn on the wrist of the target user, the number of the ankle information acquisition modules is 2, and the wrist information acquisition modules are respectively worn on the ankle of the target user, and the number of the waist information acquisition modules is 1, and the wrist information acquisition module is worn on the waist of the target user.
Specifically, the movement track and the movement height of the corresponding limb part of the target user are respectively acquired through a wrist information acquisition module, an ankle information acquisition module and a motion sensor in a waist information acquisition module, additionally, myoelectric signals of hands and/or feet are acquired through myoelectric acquisition and analysis units in the wrist information acquisition module and the ankle information acquisition module, gesture information and/or foot motion information of the target user are obtained through analysis of the myoelectric signals, and tiny motion changes of fingers and tiny movement tracks of feet can be accurately captured through changes of skin bioelectricity acquired by the myoelectric acquisition and analysis units, so that the acquired limb motion information is more accurate, and further, the limb motion information is sent to a host control module of the head-mounted display device through a wireless communication unit.
And a4, the host control module is used for receiving the limb movement information sent by the limb information acquisition module, establishing a holographic image of the target user based on the head movement information, the limb movement information, the user body shape information and the ambient light information, sending and displaying the holographic image on an opposite terminal device which is in communication connection with the head-mounted display device, and receiving and displaying the holographic image sent by the opposite terminal device.
In this embodiment, specifically, the body movement information sent by each module in the body information collection module is received by the host control module, further, based on the head movement information, the body movement information of different body parts, the body shape information of the user, and the ambient light information, a hologram of the target user is accurately constructed, further, after establishing a food call connection with an opposite-end device, the hologram is encoded and sent to an opposite-end device, so that the opposite-end device decodes the hologram, and the user views the decoded hologram, wherein the opposite-end device includes head-mounted devices such as AR, VR, and MR, and may further include home-type display devices, terminal devices, and the like, such as smart televisions, mobile phones, and tablets, and a communication connection is established between the opposite-end device and the head-mounted display device, for example, the opposite-end device and the head-mounted display device establish a video call connection. Additionally, the host control module of the head-mounted display device also receives the holographic image sent by the opposite terminal device, and then projects the received holographic image onto the retina of the target user through the optical machine imaging module, so that the three-dimensional display of the image is realized, the augmented reality effect is improved, and the image interaction can be performed in a space-time spanning manner.
Additionally, the head-mounted display device further includes an audio acquisition module, an audio transmission module, an audio reception module, and an audio playback module, wherein:
step b1, the audio acquisition module is used for acquiring the audio information of the target user;
step b2, the audio sending module is used for sending the audio information of the target user to the opposite terminal equipment for playing;
step b3, the audio receiving module is used for receiving the audio information sent by the opposite terminal equipment;
and b4, the audio playing module is used for playing the audio information of the opposite terminal equipment.
In this embodiment, specifically, an audio acquisition module in the head-mounted display device is used for acquiring audio information of the target user, and then the audio transmission module encodes the audio information, and transmits the encoded audio information to the peer device for playing, and the audio reception module receives the audio information transmitted by the peer device, and the audio playback module plays the audio information of the peer device, so that voice and image interaction in time and space crossing is realized, and the control system can be applied to the fields of real-time communication, cross-regional conferences, cross-regional course training and the like.
Additionally, wear display device still includes the camera, wrist information acquisition module ankle information acquisition module and waist information acquisition module all includes motor vibrations unit, wherein:
step c1, the camera and/or the limb information acquisition module is further used for detecting whether the target user has limb contact with the received holographic image or not based on the imaging proportion and the preset imaging distance of the received holographic image;
and c2, the host control module is used for issuing a vibration command to the motor vibration unit when the limb contact is detected to exist so as to control the motor vibration unit to vibrate.
In this embodiment, specifically, after receiving a holographic image sent by an opposite-end device, or after sending the holographic image to the opposite-end device by a head-mounted display device, the holographic image is presented to a user in real time according to a preset imaging ratio, in this embodiment of the present application, an imaging distance between the holographic image and the user is preset, for example, the imaging distance is set to 2m, and then, based on the imaging ratio of the received holographic image and the preset imaging distance, whether there is limb contact between the target user and the received holographic image is detected by a limb information acquisition module, and/or whether there is limb contact is determined by a camera of the head-mounted display device, when detecting that there is limb contact, a vibration command is issued to the motor vibration unit by the host control module to control the motor vibration unit in the information acquisition module corresponding to the limb contact to vibrate, and in another implementable manner, characteristic information such as speed, strength, amplitude, and the like of a handshake is detected and recorded by an opposite-side acquisition and analysis unit, so that vibration strength of the motor vibration unit is used for feedback, user experience is improved, and space-time-space interaction between users is achieved.
In a first embodiment of the present application, with reference to fig. 5, an interaction method is provided, where the interaction method includes:
step S10, acquiring user body shape information and ambient light information through a camera, and detecting head movement information of a target user through a head action sensor;
in this embodiment, it should be noted that, after the head-mounted display device is started up, the head-mounted display device is automatically paired with each module in the limb information acquisition module, and after the pairing is successful, it is proved that normal communication activities can be performed. Specifically, after the pairing operation is completed, the user needs to perform posture calibration according to a preset calibration posture requirement, and in order to accurately calculate user shape information of the target user, such as height, thickness and the like, the preset calibration posture can select a posture set by standing, squatting, unfolding two arms and the like, additionally, a camera in the head-mounted display device can perform real-time shooting detection to acquire user shape information of the target user, such as clothing color, skin color and the like, and acquire ambient light information, such as ambient light intensity and the like, and additionally, head movement information of the target user is detected in real time through a head movement sensor in the head-mounted display device.
Step S20, collecting the limb movement information of different limb parts of a target user through a limb information collection module, and sending the limb movement information to a host control module;
in this embodiment, it should be noted that the limb movement information includes one or more of a movement track, a movement height, gesture information, and foot motion information. Specifically, the movement track and the movement height of the corresponding limb part of the target user are respectively acquired through the wrist information acquisition module, the ankle information acquisition module and the motion sensor in the waist information acquisition module, myoelectric signals are acquired through myoelectric acquisition and analysis units in the wrist information acquisition module and the ankle information acquisition module, gesture information and/or foot motion information of the target user are obtained based on myoelectric signal analysis, and the acquired movement track, movement height, gesture information and foot motion information of the corresponding limb part are sent to the host control module in the head-mounted display device.
And step S30, receiving the limb movement information sent by the limb information acquisition module through the host control module, establishing a holographic image of the target user based on the head movement information, the limb movement information, the user body shape information and the ambient light information, sending and displaying the holographic image on an opposite terminal device which is in communication connection with the head-mounted display device, and receiving and displaying the holographic image sent by the opposite terminal device.
In this embodiment, specifically, the body movement information sent by the body information collection module is received by a host control module in the head-mounted display device, and then, based on the head movement information, the body movement information, the user body shape information, and the ambient light information, a body information model of the target user, that is, a hologram of the target user is constructed. Furthermore, the host control module in the peer device decodes the received hologram, restores the received hologram to obtain the hologram of the target user, and projects the hologram onto the retina of the user through the optical machine imaging unit in the peer device, so that the hologram is displayed in front of the peer in a 3D stereoscopic manner.
After the steps of sending and displaying the hologram on the peer device that establishes a communication connection with the head-mounted display device, and receiving and displaying the hologram sent by the peer device, the method further includes:
and S40, acquiring the audio information of the target user through the head-mounted display equipment, sending the audio information of the target user to the opposite terminal equipment for playing, and receiving and playing the audio information sent by the opposite terminal equipment.
In this embodiment, specifically, the audio information of the target user is collected through an audio collection module in the head-mounted display device, and then the audio information is encoded, and the encoded audio information is sent to the peer device through an audio sending module to be played.
Through the scheme, the limb movement information of different limb parts of a target user is accurately acquired through the limb information acquisition module, the requirement on the light of a use environment is low, the human body motion capture cannot be distorted under the conditions of strong light and weak light, so that accurate holographic images are constructed based on the limb movement information, the holographic images are sent and displayed on the opposite terminal equipment which is connected with the head-mounted display equipment in a communication mode, the three-dimensional display effect of the holographic images is improved, and the use experience of the user is improved.
Further, referring to fig. 6, in another embodiment of the present application, after the steps of transmitting and displaying the hologram on a peer device that establishes a communication connection with the head-mounted display device, and receiving and displaying the hologram transmitted by the peer device, the method further includes:
step A10, determining the imaging proportion of the received holographic image;
step A20, detecting whether the target user and the received holographic image have limb contact or not based on the imaging proportion and a preset imaging distance;
and step A30, when the contact of limbs is detected, a vibration command is issued to a motor vibration unit in the limb information acquisition module through the host control module so as to control the motor vibration unit to vibrate.
In this embodiment, specifically, before the hologram is created, based on the user body shape information, the imaging ratio of the hologram may be determined, and whether or not the limb is in contact with the user body and the contact degree may be accurately determined. And then based on the imaging proportion and the preset imaging distance of the received holographic image, detecting whether the target user and the received holographic image have limb contact through a limb information acquisition module, and/or judging whether the limb contact exists through a camera of the head-mounted display device, when detecting that the limb contact exists, sending a vibration command to the motor vibration unit through the host control module so as to control the motor vibration unit in the information acquisition module corresponding to the limb contact to vibrate, for example, when the opposite side stretches out the hand of the user to shake the hand, the user stretches out the hand and moves forwards, comprehensively judging whether the imaging is successful through the camera and/or the wrist information acquisition module, if so, sending the motor vibration command to the motor vibration unit in the wrist information acquisition module through the host control module so as to control the motor vibration unit to vibrate, and in another implementation mode, detecting and recording characteristic information such as speed, strength and amplitude of the shaking of the opposite side through the acquisition and analysis unit, so that the vibration force of the motor vibration unit is fed back, the experience of the user is perfected, and the cross-time-space interaction between the users is realized.
Through the scheme, the space-time crossing interaction is achieved, the touch feedback is given through the vibrating motor, and the use experience of a user is improved.
The present application provides a storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores one or more programs, which can be further executed by one or more processors to implement the steps of the interaction method described in any one of the above.
The specific implementation of the computer-readable storage medium of the present application is substantially the same as the embodiments of the interaction method described above, and is not described herein again.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.
Claims (7)
1. The utility model provides a wear display device control system, its characterized in that, control system includes wear display device and limbs information acquisition module, limbs information acquisition module with wear display device wireless communication connection, wear display device includes camera, head motion sensor and host computer control module, wherein:
the camera is used for collecting user body shape information and environment light information;
the head motion sensor is used for detecting head movement information of a target user;
the body information acquisition module is used for acquiring body movement information of different body parts of the target user and sending the body movement information to the host control module;
the host control module is used for receiving the limb movement information sent by the limb information acquisition module, establishing a holographic image of the target user based on the head movement information, the limb movement information, the user body shape information and the environment light information, sending and displaying the holographic image on opposite-end equipment which is in communication connection with the head-mounted display equipment, and receiving and displaying the holographic image sent by the opposite-end equipment;
the limb information acquisition module comprises one or more of a wrist information acquisition module, an ankle information acquisition module and a waist information acquisition module;
wherein the wrist information acquisition module, the ankle information acquisition module and the waist information acquisition module comprise motor vibration units, wherein,
the camera and/or the limb information acquisition module are/is also used for detecting whether the target user is in limb contact with the received holographic image or not based on the imaging proportion and the preset imaging distance of the received holographic image;
the main machine control module is further used for issuing a vibration command to the motor vibration unit when detecting that limb contact exists so as to control the motor vibration unit to vibrate.
2. The head-mounted display device control system of claim 1, wherein the head-mounted display device further comprises an audio acquisition module, an audio transmission module, an audio reception module, and an audio playback module, wherein:
the audio acquisition module is used for acquiring the audio information of the target user;
the audio sending module is used for sending the audio information of the target user to the opposite terminal equipment for playing;
the audio receiving module is used for receiving the audio information sent by the opposite terminal equipment;
and the audio playing module is used for playing the audio information of the opposite terminal equipment.
3. The head-mounted display device control system of claim 1, wherein the limb movement information comprises one or more of a movement trajectory, a movement height, gesture information, and foot motion information;
the wrist information acquisition module, ankle information acquisition module and waist information acquisition module all include motion sensor, wrist information acquisition module with ankle information acquisition module still includes flesh electricity collection analysis unit, wherein:
the motion sensor is used for acquiring the movement track and the movement height of the corresponding limb part of the target user;
the myoelectricity acquisition and analysis unit is used for acquiring myoelectricity signals of hands and/or feet and analyzing the myoelectricity signals to obtain gesture information and/or foot action information of a target user.
4. The head-mounted display device control system of claim 1, wherein the head-mounted display device further comprises an optical-mechanical imaging module to project the received hologram onto the retina of the target user.
5. A head-mounted display device interaction method is applied to the head-mounted display device control system according to any one of claims 1 to 4, and the interaction method comprises the following steps:
acquiring user body shape information and environment light information through a camera, and detecting head movement information of a target user through a head action sensor;
the limb movement information of different limb parts of a target user is acquired through a limb information acquisition module, and is sent to a host control module;
receiving limb movement information sent by the limb information acquisition module through the host control module, establishing a holographic image of the target user based on the head movement information, the limb movement information, the user body shape information and the ambient light information, sending and displaying the holographic image on an opposite terminal device which is in communication connection with the head-mounted display device, and receiving and displaying the holographic image sent by the opposite terminal device;
after the steps of sending and displaying the hologram on the peer device that establishes a communication connection with the head-mounted display device, and receiving and displaying the hologram sent by the peer device, the method further includes:
determining the imaging proportion of the received holographic image;
detecting whether the target user is in limb contact with the received holographic image or not based on the imaging proportion and a preset imaging distance;
when detecting that the limbs contact, the main machine control module issues a vibration command to a motor vibration unit in the limb information acquisition module to control the motor vibration unit to vibrate.
6. The head-mounted display device interaction method as claimed in claim 5, wherein the step of collecting the body movement information of the target user through the body information collection module comprises:
respectively acquiring the movement track and the movement height of the corresponding limb part of the target user through the action sensors in the wrist information acquisition module, the ankle information acquisition module and the waist information acquisition module;
myoelectric signals are collected through myoelectric collecting and analyzing units in the wrist information collecting module and the ankle information collecting module, and gesture information and/or foot action information of a target user are obtained based on myoelectric signal analysis.
7. The head-mounted display device interaction method according to claim 5, further comprising, after the steps of transmitting and displaying the hologram on a peer device that establishes a communication connection with the head-mounted display device, and receiving and displaying the hologram transmitted by the peer device, the steps of:
and acquiring the audio information of the target user through the head-mounted display equipment, sending the audio information of the target user to the opposite terminal equipment for playing, and receiving and playing the audio information sent by the opposite terminal equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210971651.XA CN115047979B (en) | 2022-08-15 | 2022-08-15 | Head-mounted display equipment control system and interaction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210971651.XA CN115047979B (en) | 2022-08-15 | 2022-08-15 | Head-mounted display equipment control system and interaction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115047979A CN115047979A (en) | 2022-09-13 |
| CN115047979B true CN115047979B (en) | 2022-11-01 |
Family
ID=83166894
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210971651.XA Active CN115047979B (en) | 2022-08-15 | 2022-08-15 | Head-mounted display equipment control system and interaction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115047979B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104011788A (en) * | 2011-10-28 | 2014-08-27 | 奇跃公司 | Systems and methods for augmented and virtual reality |
| CN105068248A (en) * | 2015-08-03 | 2015-11-18 | 众景视界（北京）科技有限公司 | Head-mounted holographic intelligent glasses |
| CN105183147A (en) * | 2015-08-03 | 2015-12-23 | 众景视界（北京）科技有限公司 | Head-mounted smart device and method thereof for modeling three-dimensional virtual limb |
| CN108815804A (en) * | 2018-06-12 | 2018-11-16 | 王晶 | VR rehabilitation training of upper limbs platform and method based on MYO armlet and mobile terminal |
-
2022
- 2022-08-15 CN CN202210971651.XA patent/CN115047979B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104011788A (en) * | 2011-10-28 | 2014-08-27 | 奇跃公司 | Systems and methods for augmented and virtual reality |
| CN105068248A (en) * | 2015-08-03 | 2015-11-18 | 众景视界（北京）科技有限公司 | Head-mounted holographic intelligent glasses |
| CN105183147A (en) * | 2015-08-03 | 2015-12-23 | 众景视界（北京）科技有限公司 | Head-mounted smart device and method thereof for modeling three-dimensional virtual limb |
| CN108815804A (en) * | 2018-06-12 | 2018-11-16 | 王晶 | VR rehabilitation training of upper limbs platform and method based on MYO armlet and mobile terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115047979A (en) | 2022-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11262841B2 (en) | Wireless wrist computing and control device and method for 3D imaging, mapping, networking and interfacing | |
| EP3459061A1 (en) | Multi -joint tracking combining embedded sensors and an external | |
| US20210132681A1 (en) | Natural human-computer interaction system based on multi-sensing data fusion | |
| CN102221887A (en) | Interactive projection system and method | |
| CN115768532A (en) | Augmented reality interactive movement device using laser radar sensor | |
| CN113694343B (en) | An immersive anti-stress psychological training system and method based on VR technology | |
| CN109675289B (en) | Motion assessment system based on VR and motion capture | |
| JP2022043264A (en) | Exercise evaluation system | |
| KR20180057543A (en) | Image processing apparatus, analysis system, method for processing images, and program | |
| KR20050082559A (en) | Dance learning system, internet community service system and internet community service method using the same, dance learning method, and computer executable recording media on which programs implement said methods are recorded | |
| CN113395294A (en) | Large-space multi-person interactive integrated system based on cloud rendering and use method thereof | |
| CN206819290U (en) | A kind of system of virtual reality multi-person interactive | |
| CN115047979B (en) | Head-mounted display equipment control system and interaction method | |
| US20230256327A1 (en) | Visual guidance-based mobile game system and mobile game response method | |
| CN216313145U (en) | Large-space multi-person interactive integrated system based on cloud rendering | |
| JP7300436B2 (en) | Information processing device, system, information processing method and information processing program | |
| CN211180839U (en) | A kind of sports teaching equipment and sports teaching system | |
| CN115116141A (en) | Physical training monitoring system | |
| US11990219B1 (en) | Augmented therapy | |
| CN113633962A (en) | Large-space multi-person interactive integrated system | |
| CN113298013A (en) | Motion correction method, motion correction device, storage medium and electronic equipment | |
| CN207270687U (en) | A kind of binocular handle system, device and AR devices | |
| CN106774935B (en) | Display device | |
| CN116808567B (en) | A simulated walking system based on virtual reality | |
| US20250131655A1 (en) | Method, apparatus, device, medium and program for displaying a virtual character |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |