Main

Brain–computer interfaces have shown promise as systems for restoring, replacing and augmenting lost or impaired neurological function in a variety of contexts, including paralysis from stroke and spinal cord injury, blindness and some forms of cognitive impairment1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21. Multiple innovations over the past several decades have contributed to the potential of these neural interfaces, including advances in the areas of applied neuroscience and multichannel electrophysiology22,23,24,25,26,27,28,29,30,31, mathematical and computational approaches to neural decoding27,32,33,34,35,36,37,38, power-efficient custom electronics and the development of application-specific integrated circuits39,40,41,42,43,44,45,46,47,48,49,50,51, as well as materials science and device packaging52,53,54,55,56,57,58,59. Nevertheless, the practical impact of such systems remains limited, with only a small number of patients worldwide having received highly customized interfaces through clinical trials60,61,62.

To achieve meaningful clinical impact on the large populations of patients who stand to benefit from brain–computer interface technologies63,64,65,66,67,68, surgical procedures involved in implanting neural interfaces should be minimally invasive, reversible and avoid damaging neural tissue. Advanced brain–computer interfaces require collection and processing of large amounts of neural data, potentially spanning multiple brain regions. As a result, high-density microelectrode arrays have been replacing more traditional macroelectrode arrays, offering smaller features and improved spatial resolution69,70,71,72,73,74. Systems for clinical use should also demonstrate a high degree of scalability in terms of channel count and speed of implantation. Tissue damage and total procedural time ideally should not increase in proportion to channel number, as contemporary channel counts, already reaching the many thousands, will likely increase by orders of magnitude with further progress in this field.

Microelectrode arrays that penetrate the brain have facilitated high-spatial-resolution recordings for brain–computer interfaces1,2,4,8,9,10,11, but at the cost of invasiveness and tissue damage that scale with the number of implanted electrodes75,76,77. Such systems are also difficult to remove or replace without causing damage to surrounding brain tissue75. In addition, in the context of brain–computer interfaces, neural decoding performance from penetrating electrodes has been shown to be less stable over time than neural decoding based on surface electrodes19,20. It is not yet clear whether approaches involving softer penetrating electrodes offer a substantially different trade-off30,78. For this reason, non-penetrating cortical surface microelectrodes represent a potentially attractive alternative69,79,80,81,82,83. In practice, electrocorticography (ECoG) has already facilitated capture of high-quality signals for effective use in brain–computer interfaces in several applications, including motor and speech neural prostheses7,19,32,34,72,84,85,86,87,88,89,90. Higher-spatial-resolution micro-electrocorticography (µECoG) therefore represents a promising combination of atraumatic insertion and improved signal quality. However, the limits of information that can be extracted at high resolution have not been fully characterized.

Here we demonstrate a modular and highly scalable system of conformable, thin-film microelectrodes designed for rapid, minimally invasive deployment on the cortical surface. We demonstrate the practicality and in vivo performance of the system in Göttingen minipigs in anaesthetized states and during awake locomotor behaviour. We demonstrate feasibility of using the microelectrodes in the context of clinical neurosurgery for high-resolution cortical mapping during epilepsy surgery or for the removal of tumours near eloquent regions of the human brain associated with language and sensorimotor function. We describe implantation of thousands of electrodes simultaneously in multiple regions of the neocortex in both hemispheres, including areas related to vision as well as limb and facial somatosensory and motor function. We further demonstrate feasibility of using these arrays for electrophysiologic functions required of contemporary brain–computer interfaces, including neural recording, cortical stimulation (‘neuromodulation’) and neural decoding. We characterize the spatial scales over which electrophysiologic information is represented at the cortical surface, and show that accurate neural decoding can be achieved and that accuracy improves as a function of both area coverage and spatial density. The method is intended to facilitate human clinical use of brain–computer interface technology by delivering the microelectrode numbers and spatial densities required for advanced, high-performance brain–computer interface applications in a safe and time-efficient manner that is compatible with proven and reliable neurosurgical techniques.

System overview

The system as configured for in vivo neural recording and stimulation comprises a modular set of thin-film microelectrode arrays designed for subdural implantation using a ‘cranial micro-slit’ technique. Two versions of the microelectrode array were fabricated for this study (Supplementary Figs. 1–3). The first comprises 529 electrodes of multiple sizes, with groups of electrodes measuring 20 µm, 50 µm, 100 µm and 200 µm in diameter; a uniform inter-electrode pitch of 300 µm was used for the entire array (Supplementary Fig. 2). The second version comprises 1,024 channels, 977 of which are 50 µm recording electrodes, 42 of which are 380 µm electrodes optimized for stimulation, and 5 of which are 500 µm reference electrodes; a uniform inter-electrode pitch of 400 µm was used for the entire array (Supplementary Fig. 3). We designed the 529-channel array with multiple electrode sizes and 300 μm pitch to help determine the optimal inter-electrode spacing and recording electrode size for the larger 1,024-channel array, based on a combination of manufacturing reliability, electrode impedance and analysis of inter-electrode correlation. As expected, smaller electrodes captured more unique high-frequency information across adjacent channels, but at the cost of higher impedance and greater manufacturing variability; and tighter pitch results in increased inter-electrode correlation across all spectral bands, but in a frequency-dependent manner. The choices of 400 μm pitch and 50 μm recording electrode diameter for the 1,024-channel array were made after weighing the results of these analyses.

Microelectrode arrays can be inserted individually or in modular assemblies (Supplementary Fig. 1), with each array connected to a customized hardware interface. After subdural array implantation, the interconnecting cable of each microelectrode array module passes through a dural incision and a cranial micro-slit incision, is tunnelled under the scalp as needed and is connected to an individual head stage. The head stage contains electronics for analogue-to-digital conversion and signal conditioning, and streams data to a custom software system for real-time data visualization, processing and storage. The overall system configuration is illustrated in Fig. 1.

Fig. 1: Overview of the minimally invasive thin-film neural interface system.
figure 1

This schematic illustration shows how neural activity is acquired from and stimulated by the thin-film cortical interface. a–d, A Göttingen minipig undergoes cranial micro-slit implantation (a) of a set of subdural µECoG arrays comprising a total of >2,000 microelectrodes (b,c), in modules containing 529 (Supplementary Fig. 2a) or 1,024 channels (d) (Supplementary Fig. 1) each. In the anatomic schematics (b,c), the subdural space is shown in blue, dura in purple and skull in beige. The outermost layer shown represents the scalp. e, A representative 380 μm electrode is shown schematically together with example stimulation waveforms. f, A group of 50 μm microelectrodes is shown in detail together with example traces from recorded neural activity. g,h, The microelectrode arrays provide (g) multichannel data that is used in a variety of electrophysiologic paradigms to perform neural recording of both spontaneous and stimulus-evoked neural activity as well as (h) decoding and focal stimulation of neural activity across a variety of functional brain regions. Panels e–g are conceptual illustrations and do not contain physiological data.

Full size image

Electrode array characterization

All microelectrode arrays were thoroughly characterized before insertion testing through direct inspection and both in vitro and in vivo electrode impedance mapping. The process yields were >93% and 91% for the 529-channel and 1,024-channel arrays, respectively. Electrode impedance exhibits a predictable dependence on electrode surface area, ranging from an average of 802 ± 30 kΩ for 20 µm electrodes to 8.25 ± 0.65 kΩ for 380 µm electrodes, and is robust to implantation, confirmed by the ratio of impedance before and after implantation showing little change across the array (Supplementary Fig. 4). The intrinsic filtering, signal-to-noise and noise floor properties of the µECoG array and recording system were characterized as well, using established analytic techniques91. The power spectral densities calculated across multiple recording epochs under physiologic conditions are shown in Supplementary Fig. 5 for each of the principal electrode types in the array.

Feasibility of insertion technique

We have demonstrated the feasibility of inserting our high-density microelectrode arrays using a minimally invasive ‘cranial micro-slit’ technique (Fig. 2 and Supplementary Video 1). The procedure uses precision sagittal saw blades to make 500- to 900-μm-wide incisions in the skull at approach angles approximately tangential to the cortical surface, facilitating subdural insertion of our thin-film arrays without requiring a burr hole or craniotomy. Trajectory planning and insertion were performed using fluoroscopy or computed tomographic image guidance, and electrode insertion was monitored using neuroendoscopy. To validate the procedure, we have performed 22 cranial micro-slit insertions (between 1 and 4 insertions per animal) in 8 Göttingen minipigs (an additional 61 arrays were implanted through small craniotomies in 21 Göttingen minipigs for electrophysiologic and biocompatibility studies). In addition, we have performed multiple cranial micro-slit insertions in 23 fresh cadaveric human heads, targeting the precentral gyrus in the expected region of the upper extremity primary motor cortex, and verifying placement with a combination of endoscopy, fluoroscopy and computed tomography. We have demonstrated that the entire surgical procedure for cranial micro-slit insertion, from initial skin incision to endoscope-guided array placement and final securing of the array positions, can be safely performed in under 20 min.

Fig. 2: Safety and reversibility of neural interface implantation in animal models.
figure 2

a–c, The ‘cranial micro-slit’ insertion technique is illustrated. Cranial and dural incisions are made tangent to the cortical surface (a). The electrode array is mounted on a stylet for implantation into the subdural space through the cranial slit (b). The stylet inserts into a polyimide pocket on the back of the array. The stylet is removed, leaving the array in place (c). d–f, Endoscopic views of these surgical steps are shown in a cadaveric human head, showing the dura observed through the micro-slit osteotomy (d), the array with a radiopaque gold marker being inserted into the subdural space (e) and the microelectrodes in situ with the cortical surface seen through the thin-film array (f) (black arrow head, cut surface of skull seen from within the cranial micro-slit; white arrow head, outer surface of dura; yellow arrow head, undersurface of dura; open circle, subdural space; orange arrow head, pia of the cortical surface; the largest electrode shown measures 200 µm in diameter). g,h, Computed tomography scans in the coronal plane show arrays placed via a cranial micro-slit technique on the human motor cortex (g) and pig somatosensory cortex (h), with array edges delineated by radiopaque gold markers (blue arrow heads, radiopaque array markers; yellow arrow, cranial micro-slit). i,j, A modular configuration of two 529-channel arrays is shown in situ on the cortical surface following a frontal craniotomy in a Göttingen minipig (i); the same region of the cortical surface immediately following array removal (j), demonstrating an intact pial layer and no damage to the brain. k,l, Safety was also assessed using standard, semi-quantitative histology techniques following 42-day chronic array implantation. Histologic analysis demonstrated preservation of the cortical surface architecture and no systematic differences between cortical regions implanted with the thin-film microelectrode array (k) and standard 4-contact subdural electrode array (l) (HE, haematoxylin and eosin; D, dura mater; dotted blue line, region of the cortical surface in contact with array before explantation; green arrows, cortical surface; yellow arrows, subcortical white matter). Representative micrographs are shown from 1 of 8 replicates for each device.

Full size image

Safety and reversibility

To assess the translational safety and reversibility of microelectrode array placement, we conducted a formal implantation study in 16 Göttingen minipigs implanted with either bilateral thin-film 1,024-channel microelectrode arrays of the form described here (‘Test’) or standard 4-contact subdural strip electrodes (platinum contacts with silastic encapsulation, ‘Control’). The cohorts were further split into two time points to assess the subacute (7 days) and chronic (42 days) responses to device implantation. All animals were clinically assessed throughout the duration of the implant, and following euthanasia, the calvaria and brains of each animal were sent to an independent, board-certified veterinary neuropathologist who grossed each specimen while maintaining complete photographic records, and reviewed multiple histologic sections stained with traditional haematoxylin and eosin and immunohistochemical stains (Iba1, GFAP). No neurologic impairment was noted in any animal in the study. Following euthanasia, gross examination of the cortical surface demonstrated no apparent tissue disruption owing to conformal electrode array implantation; an example image of the cortical surface after array removal and postmortem craniectomy is shown in Fig. 2j. Histologic evaluation of cortical sections immediately beneath implantation sites following acute (7-day) and chronic (42-day) implants revealed no evidence of acute or chronic tissue injury owing to subdural electrode implantation (Fig. 2k,l). The overall tissue response to implantation was similar for the thin-film and control devices, with slightly less inflammation noted in response to chronic implantation of the more conformal thin-film device relative to that observed in implantation of the control device. Array removal at chronic time points was feasible using gentle traction from the epidural portion of the array, demonstrating reversibility of the implant at least out to 42 days.

Modularity and scalability

The fabricated arrays facilitate alignment and modular assembly, as shown in Supplementary Fig. 1, so that multiple electrode modules can be joined to yield larger constructs covering larger portions of the cortical surface, without substantially increasing the complexity, risk or time required for array insertion. It is also possible to insert multiple arrays through a single slit. We performed in vivo insertions of doubly connected 529-channel modules (1,058 channels over 0.96 cm2 of the cortical surface area).

We have also demonstrated the ability to interface with multiple anatomic and functional areas of the neocortex simultaneously in vivo. We have performed simultaneous bilateral insertions over somatosensory and motor cortex, and have recorded simultaneously from multiple sensory regions, including portions of somatosensory, visual and auditory cortex (Supplementary Fig. 6). Functional localization for each region is confirmed using evoked potentials (Fig. 3). The maximum number of devices that we have placed in a single animal using the cranial micro-slit insertion technique so far is four, two 529-channel devices in each of two cranial slit osteotomies, one per hemisphere, for a total of 2,116 channels. The electrode array positions correspond schematically to the light blue regions in Supplementary Fig. 6.

Fig. 3: Neural recordings from multiple functional brain regions resolve electrographic features from the cortical surface at high spatial resolution in both anaesthetized and awake-ambulatory states.
figure 3

a, Example voltage recording traces along one column of 50 μm electrodes. b, Example voltage traces obtained during spontaneous walking from one of two 1,024-channel thin-film subdural microelectrode arrays placed over the sensorimotor cortex (one over each hemisphere) in a Göttingen minipig. c, Schematic of the Göttingen minipig brain showing colour-coded areas corresponding to placement of thin-film subdural microelectrode arrays in different functional regions in the same animal. d, An overlay of sub-recordings in a (green box) to highlight the micro-differences observed across electrodes. e, Example spectrograms corresponding to two selected channels from an array over the right somatosensory cortex. f, Schematic representation of the experimental set-up, in which an adult Göttingen minipig was permitted to walk ad lib on a treadmill shortly after emerging from anaesthesia following array placement. Accelerometers and motion-capture fiducials were placed on each limb in order to facilitate motion tracking time-synchronized with cortical activity. g, SSEPs corresponding to electrical stimulation of the tibial nerve. The inset highlights reversal of phase over a sub-millimetre scale, and the ellipsoid highlights the two-dimensional nature of the isoelectric contour. Traces span 1 s and the maximum peak-to-peak signal is approximately 5 μV. h, VEPs from photostimulation of the left eye. Same voltage and time scale bar as g. i, Representation of cortical surface activity from the left hemisphere sensorimotor cortex, as recorded from a 1,024-channel electrode array at one frame of a 20 kHz recording, displayed as a colour map of voltage, superimposed on and aligned to an image of the underlying cortical surface (each dot corresponds to one microelectrode in the array).

Full size image

Neural recording

We sought to leverage the high resolution and data bandwidth of our integrated system to map electrocortical activity on a fine-grained scale (Fig. 3). In preclinical studies involving Göttingen minipigs, implanted arrays were used for high-bandwidth and high-spatial-resolution neural recording of both spontaneous cortical electrographic activity and evoked potentials from multiple functional regions (Fig. 3). During free recording of spontaneous cortical activity, our software enables real-time visualization of raw voltages or spectral power of 1,024 channels simultaneously (Fig. 3 and Supplementary Videos 2–4). Recorded electrocortical activity from individual channels can be viewed in both time and frequency domains, revealing the presence of electrocortical activity at frequencies up to 500 Hz. The degree of correlation across electrodes decreases with distance and with increasing frequency (Fig. 4f and Supplementary Fig. 7). Importantly, even at 300 μm spacing, adjacent electrodes exhibit incompletely correlated activity, particularly at higher frequencies, and independent of the timescales investigated in this analysis (Fig. 4g). For example, beta-band (12–30 Hz) r2 is in the range of 0.67 ± 0.10, low gamma (30–50 Hz) r2 is in the range of 0.56 ± 0.10, mid-gamma (70–110 Hz) r2 is in the range of 0.35 ± 0.08, and high gamma (130–170 Hz) r2 is in the range of 0.12 ± 0.06, all at 300 μm spacing for 50 μm electrodes, suggesting that even at this spatial scale the total amount of electrophysiologic information available at the cortical surface has not been completely extracted.

Fig. 4: Neural decoding accuracy benefits from both greater extent of cortical surface area and higher density of microelectrode coverage.
figure 4

a, Tactile stimulation locations (hollow circles) on the rostrum of the Göttingen minipig used for neural decoding. The green dot indicates the current stimulation location. b, Placement of one 1,024-channel electrode array on the cortical surface overlying the rostrum somatosensory cortex on the contralateral hemisphere (A, anterior; M, medial). c, Illustration of electrode subsampling to simulate a lower density array. Bright dots indicate the selected electrode, while the dark spots indicate excluded electrodes. d, Architecture of the preprocessing and CRNN used for decoding stimulated location. Raw 20 kHz signal is downsampled to 1 kHz and then preprocessed for low-frequency and high-gamma neural features (2,048 total). The neural features are then fed into a CRNN consisting of one 1D convolutional layer, two layers of bidirectional, gated recurrent units (BiGRU), and a fully connected layer. e, Decoding accuracy (left) as a function of selected electrodes for one animal, and (right) maximum decoding accuracy for three animals (n = 50 decoder models per animal for each number of channels). Each animal was decoded for a different number of stimulation locations owing to differences in tactile stimulation quality from experiment to experiment. The electrode yields for these arrays are 96%, 97% and 98%, respectively. f, The Pearson correlation coefficient r2 computed for signals recorded during spontaneous cortical activity, as a function of electrode separation and computed for various EEG bands; coloured bands indicate plus or minus one standard deviation. g, Pearson correlation coefficient r2 computed over different timescales for single-electrode separation (400 μm) (n = 11,856 electrode pairs). ch, channels.

Full size image

To further explore the utility of high-spatial-density neural recording, evoked potentials were obtained across multiple arrays and multiple functional regions. Robust somatosensory evoked potentials (SSEPs) were obtained in arrays positioned over the somatosensory cortex following electrical or tactile stimulation (Fig. 3) of all four limbs. When the arrays span both motor and sensory cortex, the SSEPs demonstrate clear phase reversal at the motor–sensory junction (Fig. 3g, inset); in contrast to traditional macroelectrode strips, which enable only coarse localization of the boundary to within a fraction of a centimetre, and typically in one dimension, we are able to identify this boundary with 300 μm resolution, and as an isoelectric contour line in two dimensions, providing precise mapping of functional boundaries on the cortical surface. Robust visual evoked potentials (VEPs) were similarly obtained in arrays positioned over the visual cortex following time-synchronized photostimulation of the retina (Fig. 3h). We also recorded electrocorticographic activity in awake, freely ambulating animals (Fig. 3b,f,i). Time-synchronized neural data from 2,048 channels (1,024 per hemisphere in the region of the primary sensorimotor cortex) were acquired together with three-dimensional motion-capture data using multiple fiducial markers on each limb (Fig. 3f) as well as accelerometers mounted on all four limbs and the head.

Multimodal neural decoding

We next sought to evaluate the ability of the system to use high-resolution electrocortical activity to perform neural decoding in a variety of paradigms related to somatic sensation (multi-point discrimination), vision and volitional walking during conscious awake behaviour (Fig. 4 and Supplementary Figs. 8–11).

In somatosensory decoding experiments designed to assess multi-point discrimination, one 1,024-channel array was placed over the somatosensory cortex and electrocortical activity was recorded during semi-automated stimulation of up to 30 predefined rostrum locations separated by <1 cm (Fig. 4a,b). The observed evoked potentials from different stimulation locations exhibited spatial and temporal variation on multiple scales (Supplementary Figs. 7 and 8). Multi-scale spatial variation was also observed in free recording, especially when comparing across frequency bands (Fig. 4f and Supplementary Fig. 7). We therefore developed a deep neural network architecture for decoding that incorporates both spectral and spatial features. Raw neural data from each channel are first preprocessed into low-frequency features and high-gamma band amplitude (Methods), and these features are then passed into a neural network consisting of single convolutional layers, several bidirectional gated recurrent units (GRUs) and a final fully connected layer (Fig. 4d). The network takes as input the frequency features extracted from 600 consecutive time samples surrounding a stimulation event, and outputs one of N decoding outputs. Across 4 independent animals, we trained a custom network to predict 1 of 8 rostrum stimulation locations using 10–20 min of training data and then evaluated the accuracy of the resulting model in real time on a prospectively collected validation set. We obtained overall test-set accuracies of 85–98% (mean 92.75% ± 6%), with most of the residual errors being near diagonal (Supplementary Fig. 9). Similar model architectures were able to obtain high accuracy (82.5–100%) in discriminating individual limb SSEPs and on–off VEPs (Supplementary Fig. 10).

We next investigated the relative importance of spatial resolution by successively downsampling the rostrum stimulation electrode array data from each animal (Fig. 4c). As expected, decoding accuracy increased monotonically with increasing channel count. However, for the 8-class decoding problem, >95% accuracy was achieved even with 4-fold downsampling, making it difficult to demonstrate increased benefit with higher-resolution data. Therefore, in 3 consecutive animals, we increased the difficulty of the decoding problem by training models on 14, 24 and 30 different rostrum positions, respectively (Fig. 4e). With harder discrimination problems, the value of higher-resolution data becomes more apparent, with the highest accuracies being obtained when all 1,024 channels are used for decoding, and no evidence of saturation for the 24- and 30-class classifications. The impact of electrode number and spatial density on neural decoding performance is shown in Fig. 4e and highlighted in Supplementary Fig. 12.

We also sought to decode volitional motor activity in awake, consciously behaving animals. To this end, we performed bilateral craniotomies and placed one 1,024-channel array over each motor cortex (for a total of 2,048 electrodes per animal) and awakened the animal in a harness suspended over a treadmill designed to permit walking ad lib (Methods). The spontaneous motor activity of the animal was recorded using motion-tracking cameras and a 3-axis accelerometer on each limb. We observed characteristic and reproducible electrocortical spectral changes in periods immediately preceding voluntary limb or head movement, distinguishable from rest (Supplementary Fig. 9). We were able to resolve gross features of motor behaviour consistently across different animals, demonstrating three-class decoding for head, limb movement and rest with a simple convolutional neural network (CNN) architecture (Methods) obtaining accuracies of 53–69% (k-fold cross validation; k = 5, 10; chance corresponds to 33%). Furthermore, depending on the behaviours exhibited by each animal, we also applied several individualized behaviour decoding paradigms, showing that versatile decoding of sensory and motor modalities from multiple body regions could be achieved with the same array and network structure, with accuracy of 60–80% (Supplementary Fig. 9) (k-fold cross validation; k = 5).

Human clinical pilot study

In a pilot clinical study involving five neurosurgical patients undergoing intraoperative electrophysiologic mapping, we evaluated the feasibility of using our integrated system to acquire, process and display high-spatiotemporal-resolution electrocortical data in real time. Spontaneous electrocortical recordings were obtained in two patients under general anaesthesia, on conventional 4-electrode strips and in higher resolution on the 1,024-electrode devices. Where a reversal of phase in the SSEP, corresponding to the functional central sulcus, was demonstrated using the conventional 4-electrode strip, this was recapitulated in higher resolution using the 1,024-electrode device, revealing a high-resolution phase reversal ‘contour line’ in two dimensions over the cortical surface. Three patients underwent awake language mapping, and 1,024-channel electrocortical recordings were obtained from the left superior temporal gyrus, left angular gyrus and left frontal operculum, respectively, during speech time-synchronized to auditory or single-word visual cues. We were able to decode speech events from the electrocortical activity of these awake neurosurgical patients (Fig. 5a–h). As expected in the context of intraoperative language mapping in awake patients, the spectral properties of the electrophysiologic and speech signals were strongly correlated. Compared with intervals without speech attempts, we observed reproducible modulation of low-frequency (0–40 Hz) bands in the intervals immediately surrounding actual speech production events (Fig. 5c–g and Supplementary Video 4). These strong correlations provided a basis for binary detection of speech onset, achieving an accuracy of 79.8% (95% confidence interval, 77.6–82.0%) from even the short training windows provided during awake craniotomies. Using only 4 min of training data containing 54 patient utterances, we demonstrate 79% accuracy in distinguishing speech events from silence on the basis of the electrocortical activity from the high-resolution array (Fig. 5h).

Fig. 5: Human intraoperative ECoG.
figure 5

a, Axial gadolinium-contrast-enhanced T1-weighted magnetic resonance imaging (MRI) of the brain of one patient involved in the pilot study, demonstrating a tumour in the left temporal lobe. The red bar indicates placement of a 1,024-channel electrode array (99% electrode yield) during awake language mapping. b,c, Cortical surface of the left superior temporal gyrus before (b) and following (c) electrode array placement. d, Overlay representation of electrocortical activity from the 1,024-channel electrode array at the time point indicated by the dashed line in e and f, immediately before speech onset. The colour map represents normalized raw voltage as obtained from the digital steps of the analogue-to-digital converter (dark blue, low; yellow, high). e, Audio amplitude recorded during patient speech. f, Time–frequency spectrogram of audio recording during the same time interval shown in e. g, Time–frequency spectrogram of the voltage waveform from a representative electrode (star). h, Accuracy (Acc) of decoding speech onset on the basis of a 4 min training set of spoken words, in offline decoding, as assessed in 100 randomly shuffled train–test samples.

Full size image

Neural stimulation

The electrode arrays are capable of bidirectional function, with every electrode able to perform either recording or stimulation. To characterize the ability of our system to modulate cortical activity in vivo, 16 electrodes per 529-channel array were designated for use in cortical stimulation. Safe stimulation thresholds were determined in vitro (50–100 µA, 200 µs pulse width, as shown in Fig. 6a; Methods), and cortical stimulation using these parameters was performed in vivo. The 200 µm electrodes were used for stimulation in each trial, and the remaining sites on the same array, as well as all sites on adjacent arrays, were used for recording.

Fig. 6: Cortical microstimulation modulates cortical activity in ways that can be characterized in high spatial and temporal detail.
figure 6

a, Stimulation waveform used for in vitro confirmation of safe polarization potential, with 100 μA overlaid on the waveform for reference. In vivo applied current waveforms used the same applied current but without the interphase delay used for identifying polarization potential. b, Example traces for an electrode (blue arrow in g) 1 mm from the stimulation electrode (red ring in g) for 8 stimulation trial recordings. The Hjorth ‘activity’ of each trial is computed as the variance of the signal from 200 ms to 2,000 ms post-stimulation (green box), and the average activity is taken over 40 trials. c, Corresponding traces for the animal under heavier anaesthesia. d, Corresponding traces for the electrode without stimulation under light anaesthesia. e, Stimulated activity is plotted against control activity. Each point represents an individual microelectrode; the highlighted points are located within 5 electrode spacings (2 mm) from the stimulating electrode. The histograms show the distributions of activity with (side panel) and without (top panel) stimulation. The blue arrow shows the same electrode as indicated in g. f, Stimulated activity under light versus heavy anaesthesia, plotted using a scheme analogous to that used for e. The histograms show the distributions of stimulation-induced activity under different levels of anaesthesia. g, Activity across the two adjacently placed arrays with stimulation applied at the highlighted electrode. The electrode yields of these arrays are 90% (left) and 86% (right). h, Activity across the arrays without stimulation, using the same colour scale as in g. i, Differential activity across the arrays, calculated as the difference between g and h, revealing a region of suppressed activity surrounding the stimulating electrode. j, Map of differential stimulated activity between light and heavy anaesthesia.

Full size image

We performed focal stimulation of the visual cortex using the paradigm described while recording stimulus-evoked cortical activity (Fig. 6b–j). Cortical stimulation modulated high-gamma band power in a focal region (approximately 2 mm in diameter) surrounding the stimulated electrode over timescales of <2 s (Fig. 6e–i). Importantly, this effect was reduced by increasing the depth of anaesthesia (Fig. 6f,j), and the effect of stimulation was observed to spread across array boundaries (Fig. 6g–j), indicating that the observed effects are physiological and not artefacts of stimulation. Within the range of parameters tested, no induced saccades were observed in these experiments.

Discussion

Here we have described a modular and highly scalable brain–computer interface system capable of rapid, minimally invasive surgical deployment over multiple areas of the cortical surface in a reversible and atraumatic manner92. Through a series of live animal and human cadaveric surgeries, we demonstrate the safety and feasibility of delivering >2,000 microelectrodes into the subdural space of multiple functional areas of the brain at a rapid rate (>1,000 electrodes per minute) through 400- to 900-μm-wide skull incisions. Using thin-film microelectrode arrays that achieve 200- to 1,000-fold higher electrode density than standard cortical grids, we demonstrate that electrophysiologic information is available at the cortical surface at scales as small as <300 µm. We also show that high density is required to achieve accurate neural decoding from electrocortical activity, in the context of multi-class somatosensory decoding, as the number of distinct states grows. Finally, we demonstrate the ability of these bidirectional electrodes to modulate cortical activity through focal stimulation of individual electrodes. The system is designed to deliver the large microelectrode numbers and high spatial densities required to facilitate confident adoption and routine clinical use of advanced, high-performance brain–computer interface technologies. Our early experience in human patients substantiates the clinical usability and modularity of the system93.

Clinically useful neural interfaces should balance the need for improved function, achieved through increasing channel counts and spatial resolution, against the invasiveness of the surgical procedure and damage to brain tissue associated with penetrating intracortical electrode arrays. In designing our system, we have chosen to prioritize safety and scalability over other design criteria with an eye towards rapid and efficient use in human clinical applications. One potential benefit of a non-penetrating, cortical interface is the ability to reliably explant the device after chronic implantation in humans without causing tissue damage. Our chronic implantation study demonstrated straightforward and atraumatic explantation of the device after 42-day implants, supporting potential for long-term interfaces that may eventually be removed or upgraded. However, human clinical studies with long-term implantation will be required to fully substantiate this claim.

While it is unlikely that any single system or electrode type will be ideally suited to the full breadth of future clinical applications of neural interfaces, safety, scalability and reversibility are key features to maximize the addressable clinical populations that could benefit from this technology in the near future. Specifically, we believe that it is important that tissue damage and total implantation time for neural interfaces not increase linearly in proportion to electrode count, as scaling such systems by several orders of magnitude would then lead to prohibitive levels of tissue damage94,95,96,97 or impractically long implantation times. The thin-film microelectrode array technology described here compares favourably on these measures, enabling thousands of electrodes to be rapidly deployed to multiple functional areas without damaging cortical tissue. Indeed, it becomes conceivable to envision deploying a thin-film-based neural interface over the majority of the accessible human neocortex.

Several groups have used advanced surface array techniques to correlate neural activity with motor function for the control of neural prostheses in paralysed patients7,87,88,98,99,100,101, to achieve speech decoding in anarthric patients6,19,20,80,89,102, or for other applications of high-resolution ECoG72,80,103,104,105,106,107,108,109,110,111,112,113. Using the considerably increased resolution of the microelectrode array described here, we demonstrate neural decoding across multiple functional modalities, including gross and fine somatosensation, vision and volitional motor function during awake, spontaneous, untrained behaviour, as well as for predicting and detecting onset of human speech, with a machine learning framework that suggests that work in the field so far has not yet fully capitalized on the electrical information present at the cortical surface. Maximal decoding performance is achieved by incorporating information at multiple spatial scales, requiring systems and techniques that can combine wide coverage and high spatial resolution. In addition, we have demonstrated the feasibility of using these electrode arrays for dynamic, sub-millimetre-scale mapping of the cortical surface in clinical practice, in support of high-precision neurosurgery in eloquent brain regions.

Cortical stimulation is also a key capability of closed-loop brain–computer interfaces, as well as future neural prostheses for restoring functions such as somatic sensation, proprioceptive feedback and vision114. Here we characterize an operating regime for cortical stimulation from thin-film cortical surface microelectrodes, stimulate the cortical surface and monitor stimulus-evoked cortical activity in a manner that facilitates visualization and analysis of cortical surface activity at spatial and temporal resolutions and total spatial extents not previously possible.

This work has a number of limitations. In particular, we have not demonstrated the performance of the thin-film electrode interface in the context of a fully integrated package designed for chronic implantation and wireless data transfer. A full system of this nature is under development and will be described in future studies. We have also focused on demonstrating the decodability of neural data obtained from the thin-film arrays, but given constraints imposed by our animal models (untrained, freely behaving Göttingen minipigs) and limited duration of our intraoperative recordings have not optimized decoding performance for any specific clinical indication. While the present work demonstrates the feasibility of delivering focal cortical stimulation and modulating physiologic activity through a high-density cortical surface array, while also observing the effects of the stimulation at high spatiotemporal resolution, further work will be required to determine the physiological significance of the modulation demonstrated here in different experimental and clinical contexts. Finally, the clinical data reported in this study were gathered as part of a feasibility study to demonstrate the viability of our system for short-term, intraoperative recordings; future studies will be needed to demonstrate the safety and clinical utility of our thin-film microelectrode array for specific indications, including intraoperative mapping and restoring motor or speech function in paralysed patients.

The thin-film electrode array that we describe here may form the basis for a modular, scalable, minimally invasive brain–computer interface system. The approach is designed to deliver the benefits of high-density, high-channel-count, high-data-rate neural interfaces to the millions of patients with neurologic disorders who stand to benefit from this technology.

Methods

Array fabrication and characterization

Initial 529-channel microelectrode arrays were fabricated on 150–200 mm wafers using a spin-on polyimide. The fabrication process briefly comprised spin-coating, soft-bake and vacuum cure of an approximately 10 μm layer of polyimide; photolithographic patterning, deposition and liftoff of 20 nm/210 nm/20 nm Ti/Pt/Ti trace metal; O2 plasma treatment of the polyimide surface; spin-coating, soft-bake and vacuum cure of an approximately 10 μm layer of polyimide; hard mask deposition and patterning for polyimide outline and electrode site opening; polyimide etch and electrode surface exposure in O2/CF4 plasma; hard mask strip; photolithographic patterning, deposition and liftoff of 20 nm/20 nm/500 nm of Ti/Pt/Au bond pad metallization; and O2 plasma post-clean of the polyimide surface. Following microfabrication, devices were released in deionized water, optically inspected for trace, electrode and pad defects, dehydration baked and thermocompression bonded to an organic interposer using a flip-chip tool. The fabrication process is similar for the 1,024-channel array, except that the first metal stack is adjusted to include gold in the trace metal stack for reduction of trace impedance (with platinum remaining as the tissue contacting material), and designs are adjusted as pictured in Fig. 1 and Supplementary Fig. 3. Key design changes include 50 μm and 380 μm recording and stimulation electrodes, respectively, as well as 500 μm on-array reference electrodes. Additional microfabrication details have been described in our previous work71.

Each array pocket was laser cut from adhesive-backed polyimide film, then aligned with the 800 µm alignment holes and markers at the distal tip of the microelectrode arrays and compressed to form the final structure used for insertion.

The microelectrode arrays are designed to be assembled into larger connected modules in a scalable fashion to achieve greater cortical coverage. Spacing and orientation were controlled during modular assembly with the assistance of alignment holes. The arrays were bonded by applying ISO 10993 biologically tested ultraviolet-curing cyanoacrylate to the overlapping regions of adjacent array modules.

Before assembly, bonded microelectrode array-interposer assemblies were optically inspected in bond, cable and electrode areas, and a sampling of electrodes were characterized electrochemically. Electrochemical characterization was performed on a potentiostat (Wavedriver 100, Pine Research) in a 3-electrode configuration (with Ag/AgCl reference electrode and Pt coil counter-electrode) and comprised cyclic voltammetry and electrochemical impedance spectroscopy on at least one electrode per size in phosphate-buffered saline (PBS) at pH 7.4. The cyclic voltammetry measurements were performed (from 0 V to 1.2 V to −0.65 V to 0 V relative to the reference electrode) to confirm electrode surface identity using platinum oxidation and Pt–O reduction peaks, hydrogen adsorption and H2 oxidative desorption. In addition, cyclic voltammetry measurements provide information on charge storage capacity and real surface area and identify the water window. Electrochemical impedance spectroscopy measurements were performed from 10 Hz to 10 kHz (on each electrode size) to confirm that 1 kHz impedance and cut-off frequency are within expected ranges, and to provide references for later in vitro impedance mapping performed using the Intan chips in a two-electrode configuration. In vitro impedance mapping was performed in PBS on fully assembled devices (across all electrodes) at 100 Hz, 200 Hz, 500 Hz, 1,000 Hz, 2,000 Hz and 5,000 Hz using the Intan chips in our custom 529- and 1,024-channel head stages. Post-implantation, electrodes with impedance below 4 MOhm at 1 kHz were considered accepted, and rejected electrodes were excluded from neural decoding tasks.

Surgical implantation

Surgical technique

In vivo testing of the surgical insertion technique and electrode array performance were performed in adult female Göttingen minipigs. The breed was selected for well-characterized functional neuroanatomy as well as skull thickness comparable to that of adult humans. The study protocol was approved by the IACUC of DaVinci Biomedical Research Products. Local anaesthesia was achieved in the region of the skin incisions using intradermal lidocaine. General anaesthesia was maintained with isoflurane at levels sufficient to produce analgesia without suppressing electrocorticographic activity, a balance that was facilitated by the minimally invasive nature of the procedure.

We developed a ‘cranial micro-slit’ technique for array implantation. To insert each electrode array, a cranial incision was made using a modified 400-μm-thick sagittal saw blade (or an 800-μm-thick pair of such blades), at an entry angle tangential to the cortical surface. A 350 μm fibrescope was then inserted through the cranial incision and used to visualize the dura, which was coagulated and cut under direct endoscopic vision. Endoscopy was similarly used to guide insertion of each electrode array into the subdural space. In some instances, a 1.6 mm semi-rigid endoscope was used through a separate pilot hole to facilitate improved image quality for photography or videography of the procedure.

Electrode arrays were positioned subdurally on the cortical surface under simultaneous endoscopic and fluoroscopic guidance. Manipulation of each thin-film array was performed using a radiopaque stylet. The stylet tip was designed to fit within a polyimide ‘pocket’ on the reverse side of each array. Placement, depth and angulation of cranial incisions and electrode arrays were also guided by fluoroscopy or computed tomography. Each stylet was removed following fluoroscopic confirmation of array position, leaving only the thin-film subdural microelectrode arrays in position on the cortical surface.

To decouple assessment of the surgical technique from characterization of surface microelectrode array recordings, additional procedures were performed in which electrode arrays were placed on the cortical surface through small, traditional craniectomies. In these procedures, the craniectomy was performed with a high-speed burr, the dura was separately incised and elevated to expose the cortical surface in the region of interest, haemostasis was meticulously achieved, and the microelectrode was placed on the cortical surface under direct vision.

Human intraoperative array implantation

On the basis of the reversibility of the electrode array deployment and existing safety and biocompatibility data, the Layer 7 Cortical Interface was designated a ‘Non-Significant Risk’ device in the context of limited intraoperative use, and Institutional Review Board (IRB) approval was obtained for short-duration cortical surface recordings alongside standard electrophysiologic mapping performed according to the neurosurgical standard of care, with informed consent obtained preoperatively (West Virginia University Medical Center IRB protocol number 2207618749). The human electrophysiologic data reported here were obtained under total intravenous anaesthesia with propofol and fentanyl or propofol and remifentanil, with the addition of dexmedetomidine in patients undergoing awake language mapping, with a 1,024-channel microelectrode array placed alongside a standard subdural electrode strip for up to 15 min. In these patients, the subdural electrodes were placed after traditional craniotomies were performed to expose the regions of surgical interest.

Safety and reversibility in animal model

Implantation test

To fully characterize the biocompatibility and reversibility of implantation of the Layer 7 device, we designed a formal implantation study (conducted in accordance with the US Food and Drug Administration (FDA)’s Good Laboratory Practices as outlined in 21 CFR Part 58) under a protocol approved by the IACUC of DaVinci Biomedical Research Products. Sixteen adult, female Göttingen minipigs were implanted bilaterally with either 2 Layer 7 electrode arrays (‘Test’) or 2 AdTech subdural electrodes (‘Control’) made through small burr-hole incisions. The cohorts were further split into two time points to assess the subacute (7 days) and chronic (42 days) responses to device implantation. All animals were clinically assessed with daily neurologic exams throughout the duration of the implant. Following euthanasia, the brains were pressure-perfused with 10% neutral-buffered formalin, following which the calvaria and brains of each animal were sent to an independent, board-certified veterinary neuropathologist. Each specimen was grossed while maintaining complete photographic records. Histologic sections from both implanted and non-implanted regions of the cortex from each animal were processed and stained with traditional haematoxylin and eosin and immunohistochemical stains (Iba1, GFAP). All pathologic findings were scored by the independent veterinary neuropathologist on a semi-quantitative scoring scale: 0 indicates that the finding was not present, 1 indicates that the finding was minimal, 2 indicates that the finding was mild, 3 indicates that the finding was moderate, 4 indicates that the finding was marked, and 5 indicates that the finding was severe.

Electrophysiology

System configuration and recording hardware

The 529-channel customized neural recording and stimulation system is based on chips and controllers made by Intan Technologies. The custom amplifier printed circuit boards used to interface with the implanted electrode arrays each contained 8 of the RHD2164 64-channel amplifier chips and 1 of the RHS2116 16-channel stimulator/amplifier chips, allowing for simultaneous recording from up to 528 channels and stimulation from up to 16 channels. In addition, each board allows for a hardware reference from 1 of 16 sites distributed across the array. The digitized data is transferred from the amplifier boards to an associated Intan Technologies 1,024-channel RHD controller or 126-channel RHS controller using low-voltage differential signalling, where it is then stored on a USB-connected computer.

The amplifier boards are designed to allow each board to be easily coupled to any array-interposer assembly through the inclusion of an array of pogo pins that make contact with an associated pad on the array-interposer assembly, connecting each electrode site with an amplifier input. These two boards are aligned and held together by two plates with integrated alignment features placed on the outward-facing sides of the boards and screwed together. Additional protection of these electronics is provided by a custom, 3D-printed casing with strain-relief features for the electrode array and optional mounting braces to fix the entire assembly to the skull.

The 1,024-channel configuration was similar to the above, but with 16 64-channel amplifier chips required for all of the recording electrodes, and with references and stimulation electrodes wired externally as needed. These recording boards were attached using mezzanine connectors rather than pogo pins, for a more miniaturized interface.

Recording software and data preprocessing

The recording computers interface with either controller via a custom configuration of the Intan Technologies RHX Data Acquisition Software, which allows for real-time event-triggered averaging in addition to base functionality. The sampling rate for recording is set at 20 kHz per channel, generating data at a rate around 2 GB per minute for each set of 1,024 channels. A 60 Hz notch filter is applied online during recording. For post hoc analysis of local field potentials, unless otherwise specified, data is first downsampled to 5 kHz using a Fourier method and then processed with a 5th-order Butterworth low-pass filter at 250 Hz.

Software methods

Data processing is performed in C/C++, Python and MATLAB, using community standard frameworks, including but not limited to Qt, CUDA, NumPy, SciPy, PyTorch and Matplotlib.

Machine learning model training is performed using PyTorch, accelerated by an NVIDIA RTX 4090. Machine learning model inference is performed using TorchScript, utilizing 12th generation Intel Core i7 processors.

Data is parsed in real time by configuring Intan RHX to serialize data using the Intan DAT format. DAT format serialization reduces the latency between data acquisition on-chip, and serialization to disk, when compared with the default RHD format, which buffers and writes to disk in 128-sample chunks. This enables smooth and consistent playback, regardless of sampling frequency.

Real-time visualizations (Figs. 1g and 3i) are implemented using the Qt framework. Visualization of amplifier data is either performed using raw data, with no post-processing applied, or with one or more filters applied. Most commonly used for real-time analysis was a simple kernel smoothing technique applied to minimize the visual effects of lower-quality channels, or reduce the amount of low amplitude, per-channel noise, which can lead to flickering of channels. While a number of kernel smoothing techniques can be applied, the most commonly used is the 3 × 3 Gaussian kernel.

Trials are generated by reading and combining amplifier samples and digital input signals, sampled simultaneously. Amplifier data is buffered in memory and aligned with the corresponding digital input signals. When a TTL high signal is detected on a predefined channel of the digital input, a number of amplifier samples are selected preceding and following the digital signal, and emitted as a trial. Additional processing of trial data may be performed, depending on the experimental set-up or analytical techniques being used downstream, although the methods vary depending on the experimental set-up and the downstream analytical techniques being applied.

Trials are combined with external metadata to classify each trial as associated with a particular stimulus site. For example, in the case of SSEPs in the Göttingen minipig, these stimuli may correspond to sites on the rostrum, or to the null case when no stimulus was applied. Model training follows the standard procedure for model training and evaluation in PyTorch or MATLAB. Sampled data is shuffled and partitioned into ‘testing’ or ‘training’ collections for each epoch of training. Following training, model inference is performed in line with subsequent trial generation by evaluating the model on each trial, as the trial is emitted.

Free recording of spontaneous cortical activity

Example spectrograms are generated from data obtained at 20 kHz per channel, where spectral density is computed for a temporal resolution of 45 ms and a frequency resolution of 19.5 Hz using a Hann window.

To demonstrate spatial correlation between pairs of electrodes, a total of 100 s of raw 20 kHz stimulus-free neural data, recorded using 1,024-channel arrays in 6 different animals, was separated into 50 continuous, non-overlapping, 2 s segments. Within each segment, the squared Pearson correlation coefficient r2 is computed for every pair of electrodes and associated to the corresponding physical distance separating those electrodes. The r2 values across all electrode pairs were subsequently binned into 50 different distance ranges, and the average and standard deviation of the r2 values was computed for each distance range. The same analysis was also conducted for 0.25 s, 0.5 s and 1 s time segments (Fig. 4f,g).

Evoked potentials

SSEPs were evoked by applying periodic pressure on the rostrum or peripheral nerve electrical stimulation. SSEPs caused by rostrum stimulation (rostrum SSEP) are measured by either (1) manually applying pressure at six different locations on the rostrum using a conical tip or (2) applying tactile pressure at up to 30 different locations on the rostrum using a programmed pneumatic piston array. The onset of a stimulus is defined as the instance when 0.1 lbf of force is applied to the rostrum. For peripheral nerve SSEP, electrical stimulation was applied to median and tibial nerves of each side of the animal by placing twisted subdermal needle electrodes (13 mm–27 Gauge, Cadwell) near the location of the nerves. Repetitive stimuli (300 µs pulse at 2.79 Hz, more than 300 times) were presented using a Cadwell Cascade IOMax System with a limb module at intensities 1.5–2 times the threshold required to visualize twitching in the muscle distal to the stimulated nerve. Each nerve was stimulated separately while cortical responses were recorded by the electrode array. Neural response waveforms were temporally aligned to the stimulus onset. SSEPs were then computed as the averaged time-aligned signals over 250 stimuli for peripheral nerve SSEP and 140 stimuli for rostrum SSEP.

To elicit VEPs, the eyelid corresponding to the stimulated retina was retracted temporarily while periodic 50 ms flashes were generated at 1 Hz from an array of white light-emitting diodes (LEDs). Neural response waveforms were temporally aligned to the stimulus onset. VEPs were calculated as the time-aligned averaged signals over 150 trials.

Cortical stimulation

Electrical stimulation at the cortical surface was applied at one of the 200 µm electrodes, controlled by the Intan Technologies RHS controller and RHX software. Charge-balanced, biphasic, cathodic-first, 200 µs pulses of 100 µA peak current were delivered at 0.25 Hz. The evoked potentials were recorded over a series of trials. During analysis, for each trial and electrode, the Hjorth ‘activity’ of each trial was computed as the variance of the signal from 200 ms to 2,000 ms post-stimulation, and the average activity was taken over 40 trials.

Electrophysiologic recording and motion capture during awake locomotor activity

A 1,024-channel array was placed over the sensorimotor cortex on each hemisphere following carefully sized bilateral craniectomies. Two Intan 1,024-channel RHD controllers were used to record from both arrays simultaneously.

A harness (Ruffwear) was placed on the animal while it was on the operating table before being transported to the treadmill (Firepaw).

To capture motions of the animals, a pair of OAK-1 cameras (Luxonis) was used, with one camera placed on each side of the animal. One camera was to cover anterior view of the animal to capture head and forelimb movements and the other was used to capture a posterior view, including hindlimb movements. Videos were recorded at 60 frames per second using a modified set of codes from DepthAI SDK provided by the camera manufacturer. Each pair of cameras was synchronized frame-by-frame. To synchronize videos and neural recordings, 4 LEDs were placed on the treadmill and were controlled by an Arduino microcontroller that controlled the LEDs with a pulse whose ‘on’ state duration was 100 ms at 0.25 Hz. At least one LED is captured in each of the videos on each side, and the pulse was recorded as a digital input at 20 kHz by the Intan controller.

After recording sessions, each video was annotated using Premier Pro 2023 (Adobe) to classify the animal behaviours into one of the following three states: resting, limb movements and head movements. Limb movements included locomotion forward or backward, discrete one-limb movements, non-locomotive multiple-limb movements or a sequence of those limb movements without any rest. A movement onset frame was defined to be a frame where one or more limbs started to move. During resting, none of the body parts visible in any of the videos moves. A beginning and an end of each class are annotated.

Human intraoperative recording

Human intraoperative recording, real-time analysis and visualization were performed using 1,024-electrode arrays configured with customized head stages and software configured as described in earlier sections, but packaged in a manner designed to facilitate ethylene oxide sterilization and secure fixation in the surgical field.

Spontaneous cortical activity and upper-limb SSEPs were obtained as described in earlier sections.

During awake language mapping, auditory cues provided by the examiner or visual cues (single words) presented on a screen instructed the patient to speak individual words. The cues and the full auditory output of the patient were recorded and time-synchronized with the electrophysiologic data for offline analysis.

Neural decoding

Sensory decoding

Multi-class single-shot decoding efficacy is demonstrated by classifying array-wide neural recordings of pneumatics-based rostrum SSEPs using a convolutional-recurrent neural network (CRNN). For each stimulation location, the stimulus was localized in space (within 3 mm radius from target), controlled in duration (50 ms) and stable in applied force (40 psi peak). Recordings were first downsampled from 20 kHz to 1 kHz. Two sets of neural features, low frequency (<10 Hz) signals and high gamma amplitude (70–150 Hz), were then extracted and concatenated. High gamma amplitude was extracted by applying a 70–150 Hz band-pass filter to the signal and computed for the absolute value of the Hilbert transform. This resulted in 2 neural features per electrode per time step, totalling 2,048 neural features per time step. The features were further downsampled in time to 100 Hz. Recording segments of 300 ms duration (125 ms pre-onset and 175 ms post-onset) were each associated with 1 of 13 to 29 stimulation locations or spontaneous activity, yielding 14 to 30 classes. Each location was stimulated in 150–200 trials with an 80%–20% split into training and testing sets. The model was trained over 1,500 iterations using cross-entropy loss and gradient-descent ADAM optimizer. L1 regularization was used for all weights.

The CRNN (Fig. 4d) consists of one 1D convolutional block, two bidirectional gated recurrent units (BiGRU), followed by a fully connected layer. The convolutional block has 128 filters, and a 10% dropout at training time. The GRU layers each have only one internal layer, and an additional 10% dropout at training time.

Decoding with reduced electrode density was simulated by using only data collected from a subset of evenly spaced electrodes. When the number of selected electrodes is greater than 512 channels, the evenly spaced electrodes were excluded instead. A total of 50 models were trained for each electrode configuration. The average decoding accuracy and standard deviation for each electrode configuration is presented.

Motor decoding in consciously behaving large animals

For motor decoding, three classes of behaviour were considered: head movements, limb movements and rest. Movement events preceded by at least 750 ms of rest and the rest events that lasted at least 2 s were chosen. For one of the sessions, we included sensory stimulation of the rostrum and aligned the data to the onset of the stimulation. µECoG recording data was downsampled from 20 kHz to 1 kHz and was aligned to the movement onset of limb and head movements or to the middle of resting periods to contain the data segment (−500 ms, 500 ms) around the alignment point. We then normalized the data for each μECoG channel across all trial classes to have zero mean and equal variance. To train on a balanced number of samples per behaviour class, we used undersampling by matching the number of samples in the minority behaviour class. The first samples were selected to keep the effect of the anaesthesia and the activity level of the animal consistent. For validation of decoding performance, k-fold cross validation was used (k = 5 or 10) to compute accuracy statistics and confusion matrices.

Owing to the smaller number of motor events compared with sensory stimulation events, we used a simpler convolutional neural network (CNN) architecture to decode the behavioural state of an animal in each trial to avoid overfitting. The input layer of the network receives per-μECoG-channel normalized neural signals from each trial as a data array having dimensions equal to the number of ECoG channels (2,048) by the chosen number of time subintervals (40, corresponding to 40 25 ms averaging intervals per 1 s of data collected). The architecture comprises four consecutive 2D convolutional layers, each with increasing numbers of filters (8, 16, 32, 64) and a kernel size of 3 × 3 with ‘same’ padding. After each convolutional layer, batch normalization and a ReLU activation function are applied, followed by a max-pooling layer with a pool size of 2 × 2 and a stride of 2. For the fourth convolutional layer, the max-pooling layer was omitted. The architecture concludes with a fully connected layer, a softmax layer and a classification layer for each behaviour state type. A cross-entropy loss function and a stochastic gradient-descent algorithm with momentum optimization were used to train the network.

Speech decoding from patients undergoing awake craniotomies for language mapping

We used a logistic regression model on 4 min of neural data during which a patient spoke from a limited vocabulary of single-syllable words. The model was trained and tested on 61 trials with speech and 61 trials without speech. The 1,024-channel neural data was selected in the time window of 1.5 s around the speech onset (0.5 s pre-speech and 1.0 s post-speech onset) and 1.5 s without speech during which the patient was resting. We then evaluated the model performance by running the prediction 100 times by randomly resampling the training and test data by a 9:1 ratio without replacement.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.