US20100069780A1 - Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems - Google Patents
Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems Download PDFInfo
- Publication number
- US20100069780A1 US20100069780A1 US12/406,405 US40640509A US2010069780A1 US 20100069780 A1 US20100069780 A1 US 20100069780A1 US 40640509 A US40640509 A US 40640509A US 2010069780 A1 US2010069780 A1 US 2010069780A1
- Authority
- US
- United States
- Prior art keywords
- computer
- individual
- parallel control
- zones
- biosignal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
Definitions
- the present invention generally relates to computer-related technology, and more particularly to the use of biosignals of a user wishing to control a computer-controllable activity or operation, including computer games.
- BCI Brain-computer interface
- NI Neural Interface
- BI Biosignal Interface
- Applications include biomedical appliances such as wheelchair and sailboat controls, as well as communication devices allowing, for example, conversion of eye positions to keystrokes of a word processing device.
- Other applications include biofeedback devices aimed at the control of emotional states, and NI devices to control computer games. In the broadest sense, even voice recognition can be considered as a biosignal interface.
- Biopotentials generally result from the activity-dependent change of ionic composition of any cell's cytoplasm.
- all living cells are at a resting potential, typically ⁇ 20 to ⁇ 80 mV across their membranes versus the extracellular space.
- Excitation of any cell results in opening of selective ion channels, starting with fast sodium channels and calcium channels, allowing extracellular Na+ to enter the cell's cytoplasm and thereby depolarize the cell to a typical range of about +100 to about +150 mV compared to the extracellular fluid. If this type of excitation happens in multiple cells simultaneously, extracellular electrodes can sense the difference in charge and the resulting electrode output signals can be recorded.
- biopotential and changes thereof are the basis for a variety of diagnostic tools, such as electrokardiogram (EKG), electromyogram (EMG) and electroencephalogram (EMG).
- EKG electrokardiogram
- EMG electromyogram
- EMG electroencephalogram
- biofeedback has been used for the purpose of facilitating meditation or preparing athletes for sporting events.
- a relatively new use of biosignals includes their use in computer games as a novel contribution to virtual reality sensation.
- Invasive sensors have been used to alleviate the lack of functionality in individuals that suffer from some type of disability, for example, as described by Hochberg et al., “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature 442: 164-171(13 Jul. 2006). Most invasive sensors are derivatives of the “Utah Array” developed by Richard A.
- Non-invasive electrodes generally need greater spatial separation for de-convoluting spatial properties of recorded signals as described in U.S. Pat. No. 6,014,582 or using near-field and far-field signals as described in U.S. Pat. No. 6,032,072.
- U.S. Pat. No. 6,950,698 discloses a five or seven electrode array and the positioning of the array on the forehead of a patient to optimally separate EOG, EEG and EMG signals.
- U.S. Pat. No. 7,206,625 to Kutz et al. discloses a compact measuring apparatus wherein the amplifier is directly adjacent to the sensors to reduce antenna effects and improve the signal to noise ratio.
- 6,728,564 discloses a system configurable to use a classical one-channel approach or else to alternately switch between predefined parts of the sensor array to simulate a two-channel system for EEG and EMG measurements.
- the Emotive EPOC system employs a sensor array integrated into a helmet-like structure to convert the amplitudes of EEG signals into levitation of given objects in computer games and rotating the objects using rotational signals created by a gyroscope built into the headset.
- a recurring issue associated with the use of biosignals is that it can be relatively difficult for a given user to control his or her brain activity.
- Alpha, beta and gamma brain waves are readily accessible for sensing with EEG sensors or related devices and can be separated into subgroups based on frequency properties.
- EEG sensors or related devices can be separated into subgroups based on frequency properties.
- Timing of signals however is critical for most control functions, regardless of whether they are used for navigation systems or within another computer-related application.
- a case in point is the use of biosignals in gaming applications to trigger, for example, shooting or jumping in first person shooter (FPS) games.
- muscle signals can be readily and arbitrarily triggered, regardless of whether they relate to facial movements or, for example, eye movements.
- electrical muscle signals are difficult to separate into different channels, and tend to propagate across the body making it difficult to distinguish their precise origin. Even if accomplished, the user is posed with a somewhat difficult task of acquiring the necessary skills to master the exercise of different muscles without crossing over between groups.
- the present invention describes an apparatus and method for using biosignals of an individual to control a computer-related technology, by which multiple instances of a single or a group of substantially identical signals are able to be converted into multiple, complex command functions using signal duplication into multiple parallel channels operable as individual computer input/control devices.
- a biosignal-computer-interface apparatus includes means for generating biosignals based on at least one physiological parameter of an individual, and computer-interface means for performing multiple tasks, including converting the biosignals into at least one input signal, establishing a scale encompassing different levels of the input signal, multiplying the input signal into parallel control channels, dividing the scale into multiple zones for each of the parallel control channels, assigning computer commands to individual zones of the multiple zones, and generating the computer command assigned to one or more of the individual zones if the level of the input signal is within that individual zone.
- the individual zones can be the same or different among the parallel control channels in terms of the number of individual zones and ranges of the scale covered by the individual zones.
- the method includes converting biosignals into at least one input signal, assigning multiple computer commands to multiple individual zones of multiple parallel control channels, generating at least one of the computer commands if the input signal exceeds a threshold of at least one of the individual zones of the parallel control channels, and simultaneously generating the computer commands assigned to two or more of the individual zones of two or more of the parallel control channels if the input signal is within the two or more individual zones.
- the computer-interface means may be any of a variety of equipment well known in the computer-related art, including a general-purpose or special-purpose computer on which specialized software is running to perform the multiple tasks, or peripheral computer hardware, specialized hardware, or any other computing/processing equipment that can be manufactured or modified to be programmed and configured for performing the multiple tasks through or with a computer or any other computer-related technology.
- a particular aspect of the invention is the ability to use biosignals generated by non-invasive types of electrodes adapted for monitoring a variety of physiological parameters, including biopotentials associated with muscle activity, to generate output signals capable of controlling electronic systems, nonlimiting examples of which include gaming and other applications running on computers, communication devices, vehicles, weapon systems, etc.
- the invention achieves more differentiated controls over a given electronic system based on assigning multiple different commands to multiple individual zones of multiple parallel control channels whose individual zones may overlap. In this manner, it is possible to use a single biosignal as an input to produce simple individual commands as well as complex commands corresponding to combinations of individual commands.
- the apparatus and method are capable of generating a complex control signal from the single biosignal as a result of the biosignal being the basis for the input to both parallel control channels and then generating a command that is a combination of the individual commands assigned to the overlapping zones.
- FIG. 1 is a screen shot of a configuration panel generated by software adapted for controlling a gaming application through multiple parallel control channels on the basis of a single biological-generated signal, wherein the source of the signal has been selected as the biopotential of a muscle or group of muscles.
- FIG. 2 is another screen shot of a configuration panel generated by the software, and shows the manner in which a first of the parallel control channels of FIG. 1 is configured into four individual zones: no action for inputs below the threshold of a first zone (Z 1 ) assigned to the keyboard character “W”, and actuation of the keyboard characters “W,” “SpaceBar” and “S” for inputs within first, second and third zones (Z 1 , Z 2 , and Z 3 ), respectively.
- FIGS. 3 and 4 are additional screen shots of configuration panels generated by the software, which show the assignment of keyboard characters for the second and third control channels of FIG. 1 , respectively, wherein “A” and “D” of the second and third control channels, respectively, are assigned to the same zone (Z 1 ) and “spacebar” of both channels is assigned to another zone (Z 2 ).
- FIG. 5 is a screen shot of three tiled configuration panels generated by the software, and shows a summary of the overlap of the individual zones and identifies the various commands (actions) that will be input into the gaming application as a result of the simultaneous action of multiples of the keystrokes of the keyboard characters assigned in FIGS. 2 through 4 .
- FIG. 6 is a block diagram representing a biosignal-computer-interface apparatus in accordance with an embodiment of the present invention.
- the present invention provides a method and apparatus that can be used to convert multiple instances of a single biosignal or a group of substantially the same biosignals into multiple, complex command functions using signal duplication into multiple parallel channels that effectively serve as separate computer controller devices, each divided into several operational zones.
- the zones of one control channel can overlap with zones in other control channels.
- simultaneous commands can be created by binding, for example, different keyboard characters to overlapping zones of two or more control channels.
- biopotentials that is, electrical discharges resulting from excitation or relaxation of nerve, muscle or skin cells.
- the dynamic range of electrical potentials can be assigned to a tension scale of, for example, 0 to 100 where 0 corresponds to substantially complete muscle relaxation and 100 corresponds to a high excitation of the muscles.
- This scale of 1 to 100 can be referred to as a biosignal input joystick, though it should be understood that the muscle-based biosignal can be broadly utilized as, in effect, a variety of different types of computer input/controller devices.
- the scale of the biosignal input joystick can be divided into different input zones, and each input zone can be bound to a particular control function so that if the level of muscle tension is within a given zone, a particular command signal is generated that is associated with that control function.
- the control function can be a keystroke that specifies a specific action in a computer game, common examples of which include pressing the “W” key to move forward (for example, the user's computer graphic representation (avatar) of himself or herself), pressing the “S” key to move backward, pressing the spacebar to jump, and similar typical key bindings used to control computer games through a computer keyboard.
- the control signal changes to another key binding corresponding to another specific action, which may be a different keystroke or the same keystroke with a different mode of use, for example, a single actuation (press and release), a dwell, a hold time duration, a repeat interval, etc., as evidenced by the nonlimiting variety of modalities included in the pull-down list in FIG. 2 .
- another specific action which may be a different keystroke or the same keystroke with a different mode of use, for example, a single actuation (press and release), a dwell, a hold time duration, a repeat interval, etc.
- the invention will be described in the context of its implementation in computer games and gaming applications.
- keystrokes will be assumed to be bound to the following specific actions: the “W” key for moving forward, the “S” key for moving backward, the “A” key for moving to the left, the “D” key for moving to the right, and the spacebar for jumping.
- the present invention is well suited for gaming using keyboard inputs, it is foreseeable that the invention can be implemented in a variety of other computer-related and computer-controlled activities and operations that may be used for entertainment, diagnostic, or control-related purposes. Notable examples are the control of communication devices (e.g., word processors), vehicles (e.g., wheelchairs), and weapon systems.
- communication devices e.g., word processors
- vehicles e.g., wheelchairs
- weapon systems e.g., weapon systems.
- FIG. 1 shows a configuration panel generated by software adapted for controlling a gaming application capable of using up to four control channels, identified as “joystick controllers,” on the basis of a single biosignal (while the term “joystick” will be used, it should be understood that the controllers could be used to simulate other computer input/controller devices).
- the panel shows the biosignal in the process of being selected as a muscle source from a list of possible sources that include alpha and beta brain waves, by which brain activity could be monitored as an input.
- FIG. 1 shows a configuration panel generated by software adapted for controlling a gaming application capable of using up to four control channels, identified as “joystick controllers,” on the basis of a single biosignal (while the term “joystick” will be used, it should be understood that the controllers could be used to simulate other computer input/controller devices).
- the panel shows the biosignal in the process of being selected as a muscle source from a list of possible sources that include alpha and beta brain waves, by which brain activity could
- a first of the joystick controllers is in the process of being configured so that the “W” keystroke (input) for the game will be activated with a “Hold” modality when the level of the signal is within a first zone (Z 1 ) of the tension scale that has been associated with the biosignal obtained from the chosen muscle group.
- the “spacebar” key input for the game will be activated with a “Repeat-Hold” modality if the level of the signal exceeds the upper limit of Z 1 , coinciding with a lower threshold for the next higher second zone (Z 2 ) of the tension scale associated with the same muscle group, and the “S” key input for the game will be activated with a “Dwell-Repeat-Hold” modality when the level of the signal exceeds the lower threshold for the next higher third zone (Z 3 ) corresponding to the highest level of the tension scale.
- FIGS. 2 and 3 show additional configuration panels by which additional keystrokes and/or modalities are bound to tension zones that lie within or overlap the zones assigned to the first controller.
- the second controller has been configured so that its first zone (Z 1 ) lies entirely within the first zone (Z 1 ) of the first controller, but for a different keystroke and modality: the “A” keyboard character and a “Dwell-Repeat-Hold” modality.
- the second zone (Z 2 ) of the second controller has been configured so that the excitation level associated with Z 2 overlaps the first, second and third zones (Z 1 , Z 2 and Z 3 ) of the first controller.
- the keystroke associated with Z 2 of the second controller is the same as Z 2 of the first controller (the “spacebar” signal) and the same modality (“Repeat-Hold”), but is different than the keystrokes associated with Z 1 (the “W” key) and Z 3 (the “S” key) of the first controller.
- this keystroke is associated with a forward walking command. If the muscle tension level within the muscle group exceeds the lower threshold of Z 1 of the second controller, not only does the actuation of the “W” input occur in accordance with the first controller, but also the actuation of the “A” input occurs in accordance with the programming for the second controller. In the present example, this combination of keystrokes is associated with a leftward-forward walking command.
- the actuation of the “spacebar” input of the second controller is combined with the “W” input of the first controller, the combination of which results in a forward jump command.
- the muscle tension level rises sufficiently to exceed the threshold of Z 2 of the first controller, only the spacebar is actuated in accordance with the programming for the first and second controllers, resulting in only a vertical jump command.
- the actuation of the “S” input of the first controller is added to the “spacebar” input of the second controller, the combination of which results in a rearward jump command.
- the only limitation of the transfer rate is from the input device to the computer.
- the third parallel joystick controller has also been assigned to generate the same “spacebar” signal within its Z 2 level for muscle tension.
- its Z 1 level has been bound to a different keystroke: the “D” keyboard character.
- a muscle tension level that simultaneously lies within Z 1 of the first, second and controllers a muscle tension level that simultaneously lies within Z 1 of the first controller and Z 2 of the second and third controllers, a muscle tension level that simultaneously lies within Z 2 of the first controller and Z 2 of the second and third controllers, and a muscle tension level that simultaneously lies within Z 3 of the first controller and Z 2 of the second and third controllers.
- level thresholds different for the different control channels and assigning different modalities, for example, setting “Delay,” “Hold” and/or “Repeat” modes for each keystroke, a combination of keystrokes equivalent to a macro function can be emulated even in applications that do not support macros.
- the same command button can be used in multiple instances on the same or on parallel controllers.
- the “spacebar” can be assigned to a zone of the first controller corresponding to a muscle tension level from 40% to 60% on the scale, and another zone corresponding to a muscle tension level from 80% to 100% on the first controller.
- a desired action sequence can be easily created, for example, walk forward—jump forward—walk backward—jump backward, by overlapping the two spacebar zones of the first controller with a forward input command zone (e.g., from 20% to 60%) and a backward input command (e.g., 60% to 100%) zone of a different controller.
- a forward input command zone e.g., from 20% to 60%
- a backward input command e.g., 60% to 100%
- FIG. 5 One such example is represented in FIG. 5 to include an actuation sequence of no action, run (or walk) forward (Z 1 of the first controller only), run zigzag (alternating left and right) forward (Z 1 of the first, second and third controllers), jump forward (Z 1 of the first controller and Z 2 of the second and third controllers), jump still (Z 2 of the first, second and third controllers), and jump backward (Z 3 of the first controller and Z 2 of the second and third controllers).
- FIG. 6 a block diagram representing an embodiment of a biosignal-computer-interface apparatus 10 capable of using a substantially uniform biosignal input to generate an electrical signal corresponding to physiological parameters of a user 12 , for example, biopotentials generated by the muscles, nerves and/or skin of the user 12 .
- the biosignal input can be isolated from noise and other signals using standard methods, for example, as described in U.S. Pat. Nos. 5,474,082, 5,692,517 and 6,636,763, the contents of which are incorporated herein by reference.
- the biosignal input can be sensed by one or more non-invasive electrodes 14 of a well-known type, though the use of other types of electrodes are also within the scope of the invention.
- the outputs of the electrodes 14 will typically produce analog signals that can be digitized and sent to a computer 16 , which as used herein includes general-purpose computers (for example, personal computers (PCs)), special-purpose computers, peripheral computer hardware, specialized hardware, or any other computing/processing equipment that can be manufactured or modified to be programmed and configured for performing the multiple tasks through or with a computer or any other computer-related technology.
- the outputs of the electrodes 14 can be transmitted through a serial interface or any other suitable interface, including but not limited to USB, Bluetooth, and IEEE1394 Firewire interfaces.
- Software 18 (for example, gaming software) is represented as running on the computer 16 to transform the individual electrode output signals into a single input signal 20 , for example, corresponding to the muscle joystick assigned in FIG.
- the software 18 can also be used to calibrate the signal 20 to reflect or adjust for properties of the individual user, such as the maximum muscle tension that the user can generate for the purpose of establishing the upper end of the 0 to 100 scale, as well as environmental parameters like relative humidity and temperature that can impact the electrical properties of the skin.
- the 0 to 100 scale range of the signal 20 can then be subdivided into individual signal levels, typically in a linear or logarithmic scale.
- muscle tension is shown in the form of a sliding bar scale, though a dial or any other suitable visual representation could be used.
- the software 18 is then used to multiply the signal 20 into any desired number of multiple parallel control channels 22 corresponding to the virtual joystick controllers of FIGS. 1 through 5 .
- the control channels 22 can be defined by the software 18 and preferably use substantially the same scale based on the 0 to 100 scale range of the signal 20 .
- the muscle tension level used as the input to the parallel channels 22 is preferably always the same.
- Each individual channel 22 can then divide the scale into multiple zones spanning from a no/low tension level to a high tension level, with each zone for each channel 22 being assigned to a keyboard character (or, in the case of a game application that can or requires use of a different controller device, some other type of control button, trigger, etc.) that when actuated produces a command output.
- command outputs of the channels 22 can be used to control a game 24 running on the computer 16 or possibly another computer device in communication with the computer 16 .
- command outputs could be used to control other types of devices and equipment, including but not limited to communication devices, vehicles, and weapon systems.
- FIG. 2 indicates the first controller as being configured to entail a “Hold” parameter assigned to the “W” key, a “Repeat-Hold” parameter assigned to the “spacebar” key, and a “Dwell-Repeat-Hold” parameter assigned to the “S” key.
- the result would be that the game's graphical representation (avatar) constantly walks forward until the signal reaches the threshold for Z 2 , followed by initiating the “spacebar” key for jumping at a repeat frequency with a specified hold down duration, both of which are preferably defined by the user.
- the “S” key for walking backward is initiated.
- the actual transmission of the command is delayed until the dwell interval (preferably defined by the user) is satisfied.
- the signal is repeated at the repeat frequency and hold down duration defined by the user.
- the player can define a delayed, slow retreat, in combination with any other controls defined on the second controller using a single input signal. Any other combination of modalities in combination with different keystrokes across a plurality of controllers is possible as long as the application supports it.
- a variation of the scheme outlined above could be to assign the same key to multiple zones within one controller, but setting different repeat intervals and hold durations for the individual zones.
- a controller using the “W” key can easily be configured to work like an accelerator in a racing game where power-slides and spin-outs can be triggered by assigning “S” or break commands on a parallel control channel.
- Another example would be gear-shift commands in combination with acceleration and breaking on a parallel control channel.
- the present invention provides a number of advantages, including: ease of use of a hands-free interface between biosignals and a computer; arbitrary triggering of response based on voluntary muscle tension, precise timing of the trigger events, multi-functionality of the same trigger zone through overlapping command signal assignment in parallel control channels, and flexible configuration of the command structure through arbitrary assignment of command signals and command modes.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Dermatology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
- This application claims the benefit of U.S. Provisional Application No. 61/037,723, filed Mar. 19, 2008, the contents of which are incorporated herein by reference.
- The present invention generally relates to computer-related technology, and more particularly to the use of biosignals of a user wishing to control a computer-controllable activity or operation, including computer games.
- Brain-computer interface (BCI) or Neural Interface (NI) devices that fall into the general category of Biosignal Interface (BI) technology are gaining increasing importance for controlling electronic systems, a notable example of which is computers. Applications include biomedical appliances such as wheelchair and sailboat controls, as well as communication devices allowing, for example, conversion of eye positions to keystrokes of a word processing device. Other applications include biofeedback devices aimed at the control of emotional states, and NI devices to control computer games. In the broadest sense, even voice recognition can be considered as a biosignal interface.
- Biopotentials generally result from the activity-dependent change of ionic composition of any cell's cytoplasm. In an idle state, all living cells are at a resting potential, typically −20 to −80 mV across their membranes versus the extracellular space. Excitation of any cell results in opening of selective ion channels, starting with fast sodium channels and calcium channels, allowing extracellular Na+ to enter the cell's cytoplasm and thereby depolarize the cell to a typical range of about +100 to about +150 mV compared to the extracellular fluid. If this type of excitation happens in multiple cells simultaneously, extracellular electrodes can sense the difference in charge and the resulting electrode output signals can be recorded. This type of biopotential and changes thereof are the basis for a variety of diagnostic tools, such as electrokardiogram (EKG), electromyogram (EMG) and electroencephalogram (EMG). The exploitation of biopotentials beyond the diagnostic applications is emerging in prosthetic limbs, where nerve signals can be measured and converted into control signals for governing mechanical movement of artificial limbs. In addition, biofeedback has been used for the purpose of facilitating meditation or preparing athletes for sporting events. A relatively new use of biosignals includes their use in computer games as a novel contribution to virtual reality sensation.
- In the general field of using brain-based measurements as the source of biopotentials for diagnostic purposes, three different principles have emerged based on the type of sensor used, namely, sensors or sensor arrays adapted for implantation into the brain (invasive sensors), implantation into the skull and against the gray matter of the brain (partially invasive), or non-invasive placement meaning that the electrodes are simply placed on the skin. Invasive sensors have been used to alleviate the lack of functionality in individuals that suffer from some type of disability, for example, as described by Hochberg et al., “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature 442: 164-171(13 Jul. 2006). Most invasive sensors are derivatives of the “Utah Array” developed by Richard A. Norman at the University of Utah, using approximately one hundred hair-thin electrodes to record extracellular potentials. In commercial applications, the Cyberkinetics “Braingate” is a device that uses invasively implanted electrodes to control wheelchairs and other devices. Likewise, partially invasive systems have already proven functional to play video games. In contrast, non-invasive electrodes have typically been limited to use for therapeutic purposes. As taught in U.S. Pat. Nos. 6,795,724 and 7,035,686, biofeedback using color-based neurofeedback has been employed based on the assignment of different colors on a computer screen to different states of neuronal activity.
- Non-invasive electrodes generally need greater spatial separation for de-convoluting spatial properties of recorded signals as described in U.S. Pat. No. 6,014,582 or using near-field and far-field signals as described in U.S. Pat. No. 6,032,072. U.S. Pat. No. 6,950,698 discloses a five or seven electrode array and the positioning of the array on the forehead of a patient to optimally separate EOG, EEG and EMG signals. U.S. Pat. No. 7,206,625 to Kutz et al. discloses a compact measuring apparatus wherein the amplifier is directly adjacent to the sensors to reduce antenna effects and improve the signal to noise ratio. U.S. Pat. No. 6,728,564 discloses a system configurable to use a classical one-channel approach or else to alternately switch between predefined parts of the sensor array to simulate a two-channel system for EEG and EMG measurements. The Emotive EPOC system employs a sensor array integrated into a helmet-like structure to convert the amplitudes of EEG signals into levitation of given objects in computer games and rotating the objects using rotational signals created by a gyroscope built into the headset.
- A recurring issue associated with the use of biosignals is that it can be relatively difficult for a given user to control his or her brain activity. Alpha, beta and gamma brain waves are readily accessible for sensing with EEG sensors or related devices and can be separated into subgroups based on frequency properties. However, for most individuals it is very difficult to arbitrarily influence activity of selected subgroups of brain waves, especially in a time-controlled fashion. Timing of signals however is critical for most control functions, regardless of whether they are used for navigation systems or within another computer-related application. A case in point is the use of biosignals in gaming applications to trigger, for example, shooting or jumping in first person shooter (FPS) games.
- In contrast to true brain waves, muscle signals can be readily and arbitrarily triggered, regardless of whether they relate to facial movements or, for example, eye movements. On the other hand, electrical muscle signals are difficult to separate into different channels, and tend to propagate across the body making it difficult to distinguish their precise origin. Even if accomplished, the user is posed with a somewhat difficult task of acquiring the necessary skills to master the exercise of different muscles without crossing over between groups.
- The present invention describes an apparatus and method for using biosignals of an individual to control a computer-related technology, by which multiple instances of a single or a group of substantially identical signals are able to be converted into multiple, complex command functions using signal duplication into multiple parallel channels operable as individual computer input/control devices.
- According to a first aspect of the invention, a biosignal-computer-interface apparatus is provided that includes means for generating biosignals based on at least one physiological parameter of an individual, and computer-interface means for performing multiple tasks, including converting the biosignals into at least one input signal, establishing a scale encompassing different levels of the input signal, multiplying the input signal into parallel control channels, dividing the scale into multiple zones for each of the parallel control channels, assigning computer commands to individual zones of the multiple zones, and generating the computer command assigned to one or more of the individual zones if the level of the input signal is within that individual zone. The individual zones can be the same or different among the parallel control channels in terms of the number of individual zones and ranges of the scale covered by the individual zones.
- According to a second aspect of the invention, the method includes converting biosignals into at least one input signal, assigning multiple computer commands to multiple individual zones of multiple parallel control channels, generating at least one of the computer commands if the input signal exceeds a threshold of at least one of the individual zones of the parallel control channels, and simultaneously generating the computer commands assigned to two or more of the individual zones of two or more of the parallel control channels if the input signal is within the two or more individual zones.
- The computer-interface means may be any of a variety of equipment well known in the computer-related art, including a general-purpose or special-purpose computer on which specialized software is running to perform the multiple tasks, or peripheral computer hardware, specialized hardware, or any other computing/processing equipment that can be manufactured or modified to be programmed and configured for performing the multiple tasks through or with a computer or any other computer-related technology. Though it is foreseeable that invasive and partially-invasive electrodes could be employed by the invention, a particular aspect of the invention is the ability to use biosignals generated by non-invasive types of electrodes adapted for monitoring a variety of physiological parameters, including biopotentials associated with muscle activity, to generate output signals capable of controlling electronic systems, nonlimiting examples of which include gaming and other applications running on computers, communication devices, vehicles, weapon systems, etc. The invention achieves more differentiated controls over a given electronic system based on assigning multiple different commands to multiple individual zones of multiple parallel control channels whose individual zones may overlap. In this manner, it is possible to use a single biosignal as an input to produce simple individual commands as well as complex commands corresponding to combinations of individual commands. In particular, if the biosignal is at a level coinciding with two overlapping zones of two parallel control channels, the apparatus and method are capable of generating a complex control signal from the single biosignal as a result of the biosignal being the basis for the input to both parallel control channels and then generating a command that is a combination of the individual commands assigned to the overlapping zones.
- Other aspects and advantages of this invention will be better appreciated from the following detailed description.
-
FIG. 1 is a screen shot of a configuration panel generated by software adapted for controlling a gaming application through multiple parallel control channels on the basis of a single biological-generated signal, wherein the source of the signal has been selected as the biopotential of a muscle or group of muscles. -
FIG. 2 is another screen shot of a configuration panel generated by the software, and shows the manner in which a first of the parallel control channels ofFIG. 1 is configured into four individual zones: no action for inputs below the threshold of a first zone (Z1) assigned to the keyboard character “W”, and actuation of the keyboard characters “W,” “SpaceBar” and “S” for inputs within first, second and third zones (Z1, Z2, and Z3), respectively. -
FIGS. 3 and 4 are additional screen shots of configuration panels generated by the software, which show the assignment of keyboard characters for the second and third control channels ofFIG. 1 , respectively, wherein “A” and “D” of the second and third control channels, respectively, are assigned to the same zone (Z1) and “spacebar” of both channels is assigned to another zone (Z2). -
FIG. 5 is a screen shot of three tiled configuration panels generated by the software, and shows a summary of the overlap of the individual zones and identifies the various commands (actions) that will be input into the gaming application as a result of the simultaneous action of multiples of the keystrokes of the keyboard characters assigned inFIGS. 2 through 4 . -
FIG. 6 is a block diagram representing a biosignal-computer-interface apparatus in accordance with an embodiment of the present invention. - The present invention provides a method and apparatus that can be used to convert multiple instances of a single biosignal or a group of substantially the same biosignals into multiple, complex command functions using signal duplication into multiple parallel channels that effectively serve as separate computer controller devices, each divided into several operational zones. The zones of one control channel can overlap with zones in other control channels. In this manner, simultaneous commands can be created by binding, for example, different keyboard characters to overlapping zones of two or more control channels. A variety of sources are contemplated for the biosignals, though of particular interest are biopotentials, that is, electrical discharges resulting from excitation or relaxation of nerve, muscle or skin cells.
- An example of implementing the present invention will be described in reference to the dynamic range of electrical potentials that can be obtained by sensing tension in one or more groups of the user's muscles, for example, the facial muscles of a human, using a single electrode or more preferably an array of electrodes. The dynamic range of electrical potentials can be assigned to a tension scale of, for example, 0 to 100 where 0 corresponds to substantially complete muscle relaxation and 100 corresponds to a high excitation of the muscles. This scale of 1 to 100 can be referred to as a biosignal input joystick, though it should be understood that the muscle-based biosignal can be broadly utilized as, in effect, a variety of different types of computer input/controller devices. In a simple example, the scale of the biosignal input joystick can be divided into different input zones, and each input zone can be bound to a particular control function so that if the level of muscle tension is within a given zone, a particular command signal is generated that is associated with that control function. For example, the control function can be a keystroke that specifies a specific action in a computer game, common examples of which include pressing the “W” key to move forward (for example, the user's computer graphic representation (avatar) of himself or herself), pressing the “S” key to move backward, pressing the spacebar to jump, and similar typical key bindings used to control computer games through a computer keyboard. Whenever the signal transcends from one input zone to another, the control signal changes to another key binding corresponding to another specific action, which may be a different keystroke or the same keystroke with a different mode of use, for example, a single actuation (press and release), a dwell, a hold time duration, a repeat interval, etc., as evidenced by the nonlimiting variety of modalities included in the pull-down list in
FIG. 2 . After leaving an input zone, the corresponding control signal is terminated by the subsequent control signal associated with the new input zone, resulting in a single control signal being transmitted. - In computer gaming applications, many actions require combinations of different key strokes to achieve desired actions. For example, in order to jump forward, it is necessary to press the jump (spacebar) key and the forward (“W”) key simultaneously. Likewise, jumping backwards requires simultaneous pressing of the “spacebar” and “S” key. These actions can be achieved with the present invention by multiplying a single biosignal input joystick (for example, tension in a single group of muscles) into several control channels, each with multiple input zones. The input zones of the control channels can be defined and actuated in parallel, and different keystrokes and modalities (e.g., single, dwell, hold, repeat, etc.) can be assigned to the input zones independently of each other and with different level thresholds.
- In the following description, the invention will be described in the context of its implementation in computer games and gaming applications. For the convenience of the discussion, the following keystrokes will be assumed to be bound to the following specific actions: the “W” key for moving forward, the “S” key for moving backward, the “A” key for moving to the left, the “D” key for moving to the right, and the spacebar for jumping. While the present invention is well suited for gaming using keyboard inputs, it is foreseeable that the invention can be implemented in a variety of other computer-related and computer-controlled activities and operations that may be used for entertainment, diagnostic, or control-related purposes. Notable examples are the control of communication devices (e.g., word processors), vehicles (e.g., wheelchairs), and weapon systems.
- As an illustrative example,
FIG. 1 shows a configuration panel generated by software adapted for controlling a gaming application capable of using up to four control channels, identified as “joystick controllers,” on the basis of a single biosignal (while the term “joystick” will be used, it should be understood that the controllers could be used to simulate other computer input/controller devices). The panel shows the biosignal in the process of being selected as a muscle source from a list of possible sources that include alpha and beta brain waves, by which brain activity could be monitored as an input. InFIG. 2 , a first of the joystick controllers is in the process of being configured so that the “W” keystroke (input) for the game will be activated with a “Hold” modality when the level of the signal is within a first zone (Z1) of the tension scale that has been associated with the biosignal obtained from the chosen muscle group. Furthermore, the “spacebar” key input for the game will be activated with a “Repeat-Hold” modality if the level of the signal exceeds the upper limit of Z1, coinciding with a lower threshold for the next higher second zone (Z2) of the tension scale associated with the same muscle group, and the “S” key input for the game will be activated with a “Dwell-Repeat-Hold” modality when the level of the signal exceeds the lower threshold for the next higher third zone (Z3) corresponding to the highest level of the tension scale.FIGS. 2 and 3 show additional configuration panels by which additional keystrokes and/or modalities are bound to tension zones that lie within or overlap the zones assigned to the first controller. For example, the second controller has been configured so that its first zone (Z1) lies entirely within the first zone (Z1) of the first controller, but for a different keystroke and modality: the “A” keyboard character and a “Dwell-Repeat-Hold” modality. Furthermore, the second zone (Z2) of the second controller has been configured so that the excitation level associated with Z2 overlaps the first, second and third zones (Z1, Z2 and Z3) of the first controller. The keystroke associated with Z2 of the second controller is the same as Z2 of the first controller (the “spacebar” signal) and the same modality (“Repeat-Hold”), but is different than the keystrokes associated with Z1 (the “W” key) and Z3 (the “S” key) of the first controller. - Based on the programming of the first and second controllers described above and shown in
FIGS. 2 and 3 , if the muscle tension level within the muscle group is within Z1 of the first controller but below Z1 of the second controller, only the W key is actuated in accordance with the programming for the first controller. In the present example, this keystroke is associated with a forward walking command. If the muscle tension level within the muscle group exceeds the lower threshold of Z1 of the second controller, not only does the actuation of the “W” input occur in accordance with the first controller, but also the actuation of the “A” input occurs in accordance with the programming for the second controller. In the present example, this combination of keystrokes is associated with a leftward-forward walking command. As the muscle tension level continues to rise into Z2 of the second controller, the actuation of the “spacebar” input of the second controller is combined with the “W” input of the first controller, the combination of which results in a forward jump command. However, if the muscle tension level rises sufficiently to exceed the threshold of Z2 of the first controller, only the spacebar is actuated in accordance with the programming for the first and second controllers, resulting in only a vertical jump command. Finally, if the muscle tension level continues to rise into Z3 of the first controller, the actuation of the “S” input of the first controller is added to the “spacebar” input of the second controller, the combination of which results in a rearward jump command. In these substantially simultaneous modes of operation, the only limitation of the transfer rate is from the input device to the computer. - In the present example of
FIGS. 2 through 4 , the third parallel joystick controller has also been assigned to generate the same “spacebar” signal within its Z2 level for muscle tension. However, its Z1 level has been bound to a different keystroke: the “D” keyboard character. As a result, four combinations of three quasi-simultaneous key strokes are made available based on a single muscle tension input: a muscle tension level that simultaneously lies within Z1 of the first, second and controllers, a muscle tension level that simultaneously lies within Z1 of the first controller and Z2 of the second and third controllers, a muscle tension level that simultaneously lies within Z2 of the first controller and Z2 of the second and third controllers, and a muscle tension level that simultaneously lies within Z3 of the first controller and Z2 of the second and third controllers. By setting the level thresholds different for the different control channels and assigning different modalities, for example, setting “Delay,” “Hold” and/or “Repeat” modes for each keystroke, a combination of keystrokes equivalent to a macro function can be emulated even in applications that do not support macros. - The same command button can be used in multiple instances on the same or on parallel controllers. For example, the “spacebar” can be assigned to a zone of the first controller corresponding to a muscle tension level from 40% to 60% on the scale, and another zone corresponding to a muscle tension level from 80% to 100% on the first controller. In this manner, a desired action sequence can be easily created, for example, walk forward—jump forward—walk backward—jump backward, by overlapping the two spacebar zones of the first controller with a forward input command zone (e.g., from 20% to 60%) and a backward input command (e.g., 60% to 100%) zone of a different controller. Any other combination of keystrokes supported by the application is possible and can be implemented at the user's discretion. One such example is represented in
FIG. 5 to include an actuation sequence of no action, run (or walk) forward (Z1 of the first controller only), run zigzag (alternating left and right) forward (Z1 of the first, second and third controllers), jump forward (Z1 of the first controller and Z2 of the second and third controllers), jump still (Z2 of the first, second and third controllers), and jump backward (Z3 of the first controller and Z2 of the second and third controllers). - In view of the foregoing,
FIG. 6 a block diagram representing an embodiment of a biosignal-computer-interface apparatus 10 capable of using a substantially uniform biosignal input to generate an electrical signal corresponding to physiological parameters of a user 12, for example, biopotentials generated by the muscles, nerves and/or skin of the user 12. The biosignal input can be isolated from noise and other signals using standard methods, for example, as described in U.S. Pat. Nos. 5,474,082, 5,692,517 and 6,636,763, the contents of which are incorporated herein by reference. The biosignal input can be sensed by one or morenon-invasive electrodes 14 of a well-known type, though the use of other types of electrodes are also within the scope of the invention. The outputs of theelectrodes 14 will typically produce analog signals that can be digitized and sent to acomputer 16, which as used herein includes general-purpose computers (for example, personal computers (PCs)), special-purpose computers, peripheral computer hardware, specialized hardware, or any other computing/processing equipment that can be manufactured or modified to be programmed and configured for performing the multiple tasks through or with a computer or any other computer-related technology. The outputs of theelectrodes 14 can be transmitted through a serial interface or any other suitable interface, including but not limited to USB, Bluetooth, and IEEE1394 Firewire interfaces. Software 18 (for example, gaming software) is represented as running on thecomputer 16 to transform the individual electrode output signals into asingle input signal 20, for example, corresponding to the muscle joystick assigned inFIG. 1 , that reflects signal strength. Thesoftware 18 can also be used to calibrate thesignal 20 to reflect or adjust for properties of the individual user, such as the maximum muscle tension that the user can generate for the purpose of establishing the upper end of the 0 to 100 scale, as well as environmental parameters like relative humidity and temperature that can impact the electrical properties of the skin. The 0 to 100 scale range of thesignal 20 can then be subdivided into individual signal levels, typically in a linear or logarithmic scale. InFIGS. 2 through 4 , muscle tension is shown in the form of a sliding bar scale, though a dial or any other suitable visual representation could be used. - The
software 18 is then used to multiply thesignal 20 into any desired number of multipleparallel control channels 22 corresponding to the virtual joystick controllers ofFIGS. 1 through 5 . Thecontrol channels 22 can be defined by thesoftware 18 and preferably use substantially the same scale based on the 0 to 100 scale range of thesignal 20. As a result, the muscle tension level used as the input to theparallel channels 22 is preferably always the same. Eachindividual channel 22 can then divide the scale into multiple zones spanning from a no/low tension level to a high tension level, with each zone for eachchannel 22 being assigned to a keyboard character (or, in the case of a game application that can or requires use of a different controller device, some other type of control button, trigger, etc.) that when actuated produces a command output. Finally, the command outputs of thechannels 22 can be used to control agame 24 running on thecomputer 16 or possibly another computer device in communication with thecomputer 16. Alternatively and as previously noted, the command outputs could be used to control other types of devices and equipment, including but not limited to communication devices, vehicles, and weapon systems. - The ability to add “Dwell,” “Repeat” and “Hold” modalities to the keys provides an extension to the versatility of the invention. For example,
FIG. 2 indicates the first controller as being configured to entail a “Hold” parameter assigned to the “W” key, a “Repeat-Hold” parameter assigned to the “spacebar” key, and a “Dwell-Repeat-Hold” parameter assigned to the “S” key. The result would be that the game's graphical representation (avatar) constantly walks forward until the signal reaches the threshold for Z2, followed by initiating the “spacebar” key for jumping at a repeat frequency with a specified hold down duration, both of which are preferably defined by the user. Once the signal reaches the threshold for Z3, the “S” key for walking backward is initiated. However, because of the dwell function, the actual transmission of the command is delayed until the dwell interval (preferably defined by the user) is satisfied. After the initial dwell delay, the signal is repeated at the repeat frequency and hold down duration defined by the user. As a result, the player can define a delayed, slow retreat, in combination with any other controls defined on the second controller using a single input signal. Any other combination of modalities in combination with different keystrokes across a plurality of controllers is possible as long as the application supports it. - A variation of the scheme outlined above could be to assign the same key to multiple zones within one controller, but setting different repeat intervals and hold durations for the individual zones. Using a gradual increase in keystroke frequency, a controller using the “W” key can easily be configured to work like an accelerator in a racing game where power-slides and spin-outs can be triggered by assigning “S” or break commands on a parallel control channel. Another example would be gear-shift commands in combination with acceleration and breaking on a parallel control channel.
- In view of the above, the present invention provides a number of advantages, including: ease of use of a hands-free interface between biosignals and a computer; arbitrary triggering of response based on voluntary muscle tension, precise timing of the trigger events, multi-functionality of the same trigger zone through overlapping command signal assignment in parallel control channels, and flexible configuration of the command structure through arbitrary assignment of command signals and command modes.
- While the invention has been described in terms of particular embodiments, it is apparent that other forms could be adopted by one skilled in the art. Therefore, the scope of the invention is to be limited only by the following claims.
Claims (27)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/406,405 US20100069780A1 (en) | 2008-03-19 | 2009-03-18 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
US12/957,585 US8376965B2 (en) | 2008-03-19 | 2010-12-01 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
US13/739,410 US20130120246A1 (en) | 2008-03-19 | 2013-01-11 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US3772308P | 2008-03-19 | 2008-03-19 | |
US12/406,405 US20100069780A1 (en) | 2008-03-19 | 2009-03-18 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/957,585 Division US8376965B2 (en) | 2008-03-19 | 2010-12-01 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
Publications (1)
Publication Number | Publication Date |
---|---|
US20100069780A1 true US20100069780A1 (en) | 2010-03-18 |
Family
ID=42007826
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/406,405 Abandoned US20100069780A1 (en) | 2008-03-19 | 2009-03-18 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
US12/957,585 Expired - Fee Related US8376965B2 (en) | 2008-03-19 | 2010-12-01 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
US13/739,410 Abandoned US20130120246A1 (en) | 2008-03-19 | 2013-01-11 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/957,585 Expired - Fee Related US8376965B2 (en) | 2008-03-19 | 2010-12-01 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
US13/739,410 Abandoned US20130120246A1 (en) | 2008-03-19 | 2013-01-11 | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems |
Country Status (1)
Country | Link |
---|---|
US (3) | US20100069780A1 (en) |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110183757A1 (en) * | 2010-01-22 | 2011-07-28 | Nintendo Co., Ltd. | Storage medium having game program stored therein, game apparatus, control method, and game system |
US20110207100A1 (en) * | 2008-10-20 | 2011-08-25 | Koninklijke Philips Electronics N.V. | Controlling an influence on a user in a rendering environment |
US20120226330A1 (en) * | 2009-11-05 | 2012-09-06 | Koninklijke Philips Electronics N.V. | Electrical muscle stimulation |
US20120296476A1 (en) * | 2009-10-30 | 2012-11-22 | Richard John Cale | Environmental control method and system |
US20160066836A1 (en) * | 2014-09-09 | 2016-03-10 | Dynofit, Inc. | Gamified electromyographic neuromuscular reeducation therapy system |
US9468541B2 (en) | 2010-05-05 | 2016-10-18 | University Of Maryland College Park | Time domain-based methods for noninvasive brain-machine interfaces |
CN109508094A (en) * | 2018-12-11 | 2019-03-22 | 西安交通大学 | A kind of vision inducting brain-machine interface method of the asynchronous eye movement switch of combination |
CN111094941A (en) * | 2019-11-29 | 2020-05-01 | 深圳市汇顶科技股份有限公司 | PPG circuit, biological feature detection device and biological feature detection method |
US20210330242A1 (en) * | 2014-05-30 | 2021-10-28 | The Regents Of The University Of Michigan | Brain-computer interface for facilitating direct selection of multiple-choice answers and the identification of state changes |
US11273283B2 (en) | 2017-12-31 | 2022-03-15 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to enhance emotional response |
US11364361B2 (en) | 2018-04-20 | 2022-06-21 | Neuroenhancement Lab, LLC | System and method for inducing sleep by transplanting mental states |
US11452839B2 (en) | 2018-09-14 | 2022-09-27 | Neuroenhancement Lab, LLC | System and method of improving sleep |
US11717686B2 (en) | 2017-12-04 | 2023-08-08 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to facilitate learning and performance |
US11723579B2 (en) | 2017-09-19 | 2023-08-15 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement |
US11786694B2 (en) | 2019-05-24 | 2023-10-17 | NeuroLight, Inc. | Device, method, and app for facilitating sleep |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2241246A1 (en) * | 2009-04-14 | 2010-10-20 | Koninklijke Philips Electronics N.V. | Using muscle tension sensing to locate an analyte measurement site on the skin |
US9814426B2 (en) | 2012-06-14 | 2017-11-14 | Medibotics Llc | Mobile wearable electromagnetic brain activity monitor |
US10130277B2 (en) | 2014-01-28 | 2018-11-20 | Medibotics Llc | Willpower glasses (TM)—a wearable food consumption monitor |
US20160240098A1 (en) * | 2015-02-12 | 2016-08-18 | Seoul National University R&Db Foundation | Smart tablet-based neurofeedback device combined with cognitive training, method and computer-readable medium thereof |
US20170259167A1 (en) * | 2016-03-14 | 2017-09-14 | Nathan Sterling Cook | Brainwave virtual reality apparatus and method |
US10070799B2 (en) | 2016-12-02 | 2018-09-11 | Pison Technology, Inc. | Detecting and using body tissue electrical signals |
CN108303970A (en) * | 2018-02-27 | 2018-07-20 | 吉旗(成都)科技有限公司 | A kind of method of automatic identification vehicle diagnosis interface channel |
CN109308118B (en) * | 2018-09-04 | 2021-12-14 | 安徽大学 | EOG-based Chinese eye-writing signal recognition system and its recognition method |
US10817058B1 (en) * | 2019-06-19 | 2020-10-27 | Eagle Technology, Llc | Electronic device with EMG sensor based scaling control output and related methods |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5692517A (en) * | 1993-01-06 | 1997-12-02 | Junker; Andrew | Brain-body actuated system |
US6014582A (en) * | 1996-10-23 | 2000-01-11 | He; Bin | Method and apparatus of biosignal spatial analysis |
US6032072A (en) * | 1998-01-30 | 2000-02-29 | Aspect Medical Systems, Inc. | Method for enhancing and separating biopotential signals |
US6085115A (en) * | 1997-05-22 | 2000-07-04 | Massachusetts Institite Of Technology | Biopotential measurement including electroporation of tissue surface |
US6636763B1 (en) * | 1998-12-10 | 2003-10-21 | Andrew Junker | Brain-body actuated system |
US6728564B2 (en) * | 2001-07-03 | 2004-04-27 | Instrumentarium Corp. | Configurable sensor system for measuring biopotentials |
US6795724B2 (en) * | 2002-02-19 | 2004-09-21 | Mark Bradford Hogan | Color-based neurofeedback |
US6950698B2 (en) * | 2003-07-02 | 2005-09-27 | Instrumentarium Corp. | Method of positioning electrodes for central nervous system monitoring |
US7206625B2 (en) * | 2003-10-23 | 2007-04-17 | Vivosonic Inc. | Method and apparatus for the collection of physiological electrical potentials |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060155576A1 (en) * | 2004-06-14 | 2006-07-13 | Ryan Marshall Deluz | Configurable particle system representation for biofeedback applications |
-
2009
- 2009-03-18 US US12/406,405 patent/US20100069780A1/en not_active Abandoned
-
2010
- 2010-12-01 US US12/957,585 patent/US8376965B2/en not_active Expired - Fee Related
-
2013
- 2013-01-11 US US13/739,410 patent/US20130120246A1/en not_active Abandoned
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5692517A (en) * | 1993-01-06 | 1997-12-02 | Junker; Andrew | Brain-body actuated system |
US6014582A (en) * | 1996-10-23 | 2000-01-11 | He; Bin | Method and apparatus of biosignal spatial analysis |
US6085115A (en) * | 1997-05-22 | 2000-07-04 | Massachusetts Institite Of Technology | Biopotential measurement including electroporation of tissue surface |
US6032072A (en) * | 1998-01-30 | 2000-02-29 | Aspect Medical Systems, Inc. | Method for enhancing and separating biopotential signals |
US6636763B1 (en) * | 1998-12-10 | 2003-10-21 | Andrew Junker | Brain-body actuated system |
US6728564B2 (en) * | 2001-07-03 | 2004-04-27 | Instrumentarium Corp. | Configurable sensor system for measuring biopotentials |
US6795724B2 (en) * | 2002-02-19 | 2004-09-21 | Mark Bradford Hogan | Color-based neurofeedback |
US7035686B2 (en) * | 2002-02-19 | 2006-04-25 | Mark Bradford Hogan | Color-based neurofeedback |
US6950698B2 (en) * | 2003-07-02 | 2005-09-27 | Instrumentarium Corp. | Method of positioning electrodes for central nervous system monitoring |
US7206625B2 (en) * | 2003-10-23 | 2007-04-17 | Vivosonic Inc. | Method and apparatus for the collection of physiological electrical potentials |
Cited By (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9612654B2 (en) * | 2008-10-20 | 2017-04-04 | Koninklijke Philips N.V. | Controlling an influence on a user in a rendering environment |
US20110207100A1 (en) * | 2008-10-20 | 2011-08-25 | Koninklijke Philips Electronics N.V. | Controlling an influence on a user in a rendering environment |
US20120296476A1 (en) * | 2009-10-30 | 2012-11-22 | Richard John Cale | Environmental control method and system |
US20120226330A1 (en) * | 2009-11-05 | 2012-09-06 | Koninklijke Philips Electronics N.V. | Electrical muscle stimulation |
US9878152B2 (en) * | 2009-11-05 | 2018-01-30 | Koninklijke Philips N.V. | Electrical muscle stimulation |
US8449392B2 (en) * | 2010-01-22 | 2013-05-28 | Nintendo Co., Ltd. | Storage medium having game program stored therein, game apparatus, control method, and game system using a heartbeat for performing a game process in a virtual game world |
US20110183757A1 (en) * | 2010-01-22 | 2011-07-28 | Nintendo Co., Ltd. | Storage medium having game program stored therein, game apparatus, control method, and game system |
US9468541B2 (en) | 2010-05-05 | 2016-10-18 | University Of Maryland College Park | Time domain-based methods for noninvasive brain-machine interfaces |
US20210330242A1 (en) * | 2014-05-30 | 2021-10-28 | The Regents Of The University Of Michigan | Brain-computer interface for facilitating direct selection of multiple-choice answers and the identification of state changes |
US12121360B2 (en) * | 2014-05-30 | 2024-10-22 | The Regents Of The University Of Michigan | Brain-computer interface for facilitating direct selection of multiple-choice answers and the identification of state changes |
US20160066836A1 (en) * | 2014-09-09 | 2016-03-10 | Dynofit, Inc. | Gamified electromyographic neuromuscular reeducation therapy system |
US11723579B2 (en) | 2017-09-19 | 2023-08-15 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement |
US11717686B2 (en) | 2017-12-04 | 2023-08-08 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to facilitate learning and performance |
US11478603B2 (en) | 2017-12-31 | 2022-10-25 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to enhance emotional response |
US11273283B2 (en) | 2017-12-31 | 2022-03-15 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to enhance emotional response |
US11318277B2 (en) | 2017-12-31 | 2022-05-03 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement to enhance emotional response |
US11364361B2 (en) | 2018-04-20 | 2022-06-21 | Neuroenhancement Lab, LLC | System and method for inducing sleep by transplanting mental states |
US11452839B2 (en) | 2018-09-14 | 2022-09-27 | Neuroenhancement Lab, LLC | System and method of improving sleep |
CN109508094A (en) * | 2018-12-11 | 2019-03-22 | 西安交通大学 | A kind of vision inducting brain-machine interface method of the asynchronous eye movement switch of combination |
US11786694B2 (en) | 2019-05-24 | 2023-10-17 | NeuroLight, Inc. | Device, method, and app for facilitating sleep |
KR20210068327A (en) | 2019-11-29 | 2021-06-09 | 선전 구딕스 테크놀로지 컴퍼니, 리미티드 | Photovolume fluctuation waveform circuit, biological characteristic detection device and biological characteristic detection method |
WO2021102892A1 (en) * | 2019-11-29 | 2021-06-03 | 深圳市汇顶科技股份有限公司 | Ppg circuit, biological feature detection device, and biological feature detection method |
CN111094941A (en) * | 2019-11-29 | 2020-05-01 | 深圳市汇顶科技股份有限公司 | PPG circuit, biological feature detection device and biological feature detection method |
US12268485B2 (en) | 2019-11-29 | 2025-04-08 | Shenzhen GOODIX Technology Co., Ltd. | Photoplethysmogram circuit, biological characteristics detection device and biological characteristics detection method |
Also Published As
Publication number | Publication date |
---|---|
US20110301488A1 (en) | 2011-12-08 |
US20130120246A1 (en) | 2013-05-16 |
US8376965B2 (en) | 2013-02-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8376965B2 (en) | Method and apparatus for using biopotentials for simultaneous multiple control functions in computer systems | |
Vourvopoulos et al. | Evaluation of commercial brain–computer interfaces in real and virtual world environment: A pilot study | |
Nijholt et al. | Turning shortcomings into challenges: Brain–computer interfaces for games | |
Blankertz et al. | The Berlin brain–computer interface: non-medical uses of BCI technology | |
Baykara et al. | Effects of training and motivation on auditory P300 brain–computer interface performance | |
Tangermann et al. | Playing pinball with non-invasive BCI. | |
US20170259167A1 (en) | Brainwave virtual reality apparatus and method | |
CN100366215C (en) | Control method and system based on EEG homeostasis evoked response and sensory testing method and system | |
Mueller-Putz et al. | Temporal coding of brain patterns for direct limb control in humans | |
CN104287747A (en) | Exercise rehabilitation robot interactive control method based on emotion perception | |
Chan et al. | An overview of brain computer interfaces | |
Bakaoukas et al. | Examining brain activity while playing computer games | |
Gonzalez-Vargas et al. | Human-machine interface for the control of multi-function systems based on electrocutaneous menu: application to multi-grasp prosthetic hands | |
Rezeika et al. | 30-targets hybrid BNCI speller based on SSVEP and EMG | |
CN111529888A (en) | VR psychological relaxation system | |
Becker et al. | Physiologically interactive gaming with the 3D agent Max | |
Edmonds et al. | On physiological computing with an application in interactive art | |
Daniel et al. | Brain controlled lego NXT mindstorms 2.0 platform | |
Krauledat et al. | Playing Pinball with non-invasive BCI | |
Al-Negheimish et al. | Brainwave typing: Comparative study of p300 and motor imagery for typing using dry-electrode EEG devices | |
Müller et al. | Flex your muscles: EMG-based serious game controls | |
Pinegger et al. | Sheet music by mind: Towards a brain-computer interface for composing | |
Duprès et al. | Toward a hybrid brain-machine interface for palliating motor handicap with Duchenne muscular dystrophy: A case report | |
Bansal et al. | EEG Based BCI—Control Applications | |
Vi et al. | Quantifying EEG measured task engagement for use in gaming applications |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: OCZ TECHNOLOGY GROUP, INC.,CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SCHUETTE, FRANZ MICHAEL;REEL/FRAME:022603/0373 Effective date: 20090330 Owner name: BRAIN ACTUAGED TECHNOLOGIES,OHIO Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:JUNKER, ANDREW (NMN);REEL/FRAME:022603/0382 Effective date: 20090420 |
|
AS | Assignment |
Owner name: BRAIN ACTUATED TECHNOLOGIES,OHIO Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME FROM BRAIN ACTUAGED TECHNOLOGIES TO BRAIN ACTUATED TECHNOLOGIES DUE TO TYPOGRAPHICAL ERROR PREVIOUSLY RECORDED ON REEL 022603 FRAME 0382. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF THE ENTIRE RIGHT, TITLE AND GOOD WILL;ASSIGNOR:JUNKER, ANDREW (NMN);REEL/FRAME:022716/0134 Effective date: 20090420 |
|
AS | Assignment |
Owner name: BCINET, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:OCZ TECHNOLOGY GROUP, INC.;REEL/FRAME:024793/0177 Effective date: 20100804 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |