Drone Control

This project explains how a drone can be controlled using brain signals (EEG), specifically by imagining a hand movement or a simple object to take the drone off, move it forward and land it as well as more complicated control setup options. This project was created in 2013 and was since deployed off-the-lab and tested by more than 4000 people over the past 5 years.

Click Here For More Videos

As an emerging and new interaction technology, BCIs offer a good prospect of inspiring people’s imaginations and of providing a new and enjoyable way of interaction.

For example, personal drones are a popular phenomenon that takes an increasing role in our daily lives. There are “selfie drones” that take off from a user’s wrist to take a “selfie” photo, delivery drones, drones for rescue operations, and all these drones can be controlled by BCIs.

We have been working on the drone controlling system since 2013 and as of 2020, we have published 7+ scientific papers, performed 400 live demos with 4000+ active participants over 3 continents, 5 countries and 57 cities.

We initially presented our first BCI system for drone control at major HCI conferences (UBICOMP and CHI) that allowed visitors to try BCIs in a few minutes (Figure 2) [5,6]. Certainly it is not possible to achieve full 3D control with high accuracy within a few minutes; however, participants can learn just enough to perform some limited 2D control and have fun. We proposed that visitors use imagined hand and feet movements (you have to imagine and intend to move your hands or feet until right before they actually start moving) to toggle between the landed and flying state and to make the drone fly forward until it reached a destination target. The task was very simple and the control was somewhat imperfect, but most visitors were thrilled about the experience and eager for more. We even had a visit by the award-winning science fiction author Margaret Atwood, who enjoyed the experience (see a photo below!). This goes to show that despite their imperfections, BCIs can already be used in HCI applications, including those for entertainment and art.

We then ran several experiments focusing on signal processing for this type of application. We proposed a “Co- learning BCI” (CLBCI) that reduces the initial amount of training and makes BCIs more suitable for recreational applications. We replicated an existing experiment where the BCI controls a drone and compare CLBCI to the Operant Conditioning protocol (OC) over three durations of practice (1 day, 1 week, 1 month). We find that OC works at 80% after a month practice, but the performance is between 60 and 70% any earlier. In a week of practice, CLBCI reaches a performance of around 75%. We conclude that our CLBCI is better suited for recreational use. OC should be reserved for users for whom performance is the main concern. Given our observations, it is likely that CLBCI (but more generally co-adaptive asynchronous BCIs) and OC eventually converge to the same performance where users have learned to modulate their signals correctly. We have identified promising directions and actionable ideas (shorter initial training, co-learning) for researchers in this field.

Finally, we presented data from detailed observations of a controlled in-the-wild drone piloting Brain-Computer Interface (BCI) system. During 10 days of demonstration at seven non- specialized public events, 1563 people learned about the system in various social configurations. Observations of audience behavior revealed recurring behavioral patterns. From these observations a framework of interaction with BCI systems was deduced. It describes the phases of passing by an installation, viewing and reacting, passive and active interaction, group interactions, and follow-up actions. We also conducted semi- structured interviews with the people who interacted with the drone piloting system. The interviews revealed the barriers and several directions for further research on BCIs. Our findings can be useful for designing the BCIs for everyday adoption by a wide range of people.

Click Here To View Prizes and Awards