Ferreira et al.’s “Understanding and Proposing a Design Rationale of Digital Games based on Brain-Computer Interface: Results of the AdmiralMind Battleship Study” (2014) presents a valuable foundational study on brain-computer interface (BCI) game design. This article begins with a review of BCI gaming research, including previous reviews/theoretical articles and an overview of BCI game examples. The game examples are organized in terms of three major BCI gaming control categories including: (1)neurofeedback-based controls, which the authors describe as including methods that use alpha and beta waves for estimating cognitive/affective states such as focus, relaxing/meditation, and concentration of the user; (2) visual stimuli based controls, such as elicitation of the steady state visually evoked potential (SSVEP) and P300 event related potential (ERP); and (3) imaged movement, often measured via synchronization/desynchronization of mu rhythms from corresponding motor regions. The authors use the term hybrid detection to specify BCI control strategies that utilize more than one of these three methods, which is not to be confused with multimodal control, a term the authors use to describe BCI games that use traditional game control mechanisms, such the keyboard, mouse, and a joystick, in conjunction with a BCI.
Beyond the informative review of previous BCI game development that the article provides, the authors present a design rationale that addresses many key decisions to be made early-on in the BCI game development process. As key considerations in the BCI game development process, the authors identify the questions:
1. Which game genre?
Here, the authors describe the potentials and limitations that current BCI technology offers for a range of genres, including first-person shooter (FPS) games, role-playing games (RPGs), puzzle games, sports games, etc. For each genre identified, the design rationale offers recommendations for BCI control, such as taking advantage of less responsive BCI control features in slower, turned-based style games such as RPGs and puzzle games, whereas in action, sports, and FPS games, more responsive (and immersive) controls such as imagined movement would be more appropriate.
2. Real time or turn-based?
As mentioned in the previous design rationale consideration, particular BCI game control strategies can be approximately divided into controls that could potentially be responsive enough to control real time action versus those suited better for turn-based control. While turn-based controls in general are better suited for BCI, the more engaging interaction offered by real time control provides incentive for BCI researchers and game developers to overcome the challenges of associated with their effective implementation.
3. Which type of EEG headset?
In the article, this question is framed in terms of commercial (i.e. consumer) headsets vs. professional headsets. Professional BCI detection devices (utilizing not only EEG, but also fNIRS, fMRI, MEG, ECoG, etc.) offer greater precision and the potentially to more dramatically expand the frontiers of BCI implementations. However, the expanding availability of consumer headsets, including those offered by NeuroSky,Muse, Emotiv, etc., has made the home BCI gaming market a reality. Indeed, Emotiv’s EPOC has been shown to be capable of capturing adequate motor imagery control in combination with the open source software package OpenViBe, expanding the possibilities of BCI control strategies available with consumer-grade devices.
4. Use multimodal integration?
As mentioned in the first paragraph, Ferreira et al. define multimodal control as the use of other control devices with BCI, such as a keyboard and mouse. While this allows for more complex game mechanics given the current limitations of BCI technologies, the authors point out this may disappoint players expecting a pure BCI experience, as well as prevent use by the key BCI demographic of users with low-to-no mobility. The authors also suggest virtual reality (VR) can be combined with BCI, offering enhanced game immersion, although consumer VR technology is also at an early, very limited stage of availability.
5. Single or multiple detection approach?
This consideration relates to what the authors define as hybrid control, whereby multiple BCI control strategies are utilized. While this has the potential to expand the diversity of purely BCI controls in a game, the authors point out it is easier on both the developer and player to utilize a single BCI control strategy. Indeed, as pointed out in the next consideration, certain BCI control strategies can have a substantial training time associated with them in order to be effective, and the combined use of multiple strategies, if not thoughtfully implemented, may become too frustrating for players to effectively master.
6. Which detection approach?
As described in the first paragraph above, the authors describe three major BCI control strategies; “neurofeedback” (e.g. concentration), visual stimuli responses (e.g. P300, SSVEP), and imaged movement (e.g. mu rhythm synchronization/desynchronization). Control based on states such as concentration are the most easily implementable on consumer BCI devices, as in the case of NeuroSky’s eSense meters. Imagined movement is perhaps most closely related to how BCI gaming would ideally function, whereas players can simply control movement of game characters and objects by thought alone, and in a responsive fashion. However, due to the anatomic origin of motor signals, it is likely only more expensive devices like Emotiv’s EPOC are capable of capturing this. Likewise, capturing imagined movement can require substantial training. Visual stimulus response-based control offers a middle ground in terms of more complex control, although presents further considerations of its own…
7. Which stimulus?
This consideration relates specifically to visual stimulus response-based control, which can be further subdivided into oscillatory (e.g. SSVEP) and transient (e.g. P300) control mechanisms. SSVEP offers faster detection than P300, although the authors point out it can be tiresome for the players eyes and a potential risk to epilepsy sensitive people. P300, while slower paced, can be an effective control in games well-suited for grid-based selection controls.
In summary, the proposed design rationale described in this paper helps organize the major considerations that must be taken when designing a BCI game and provides numerous helpful suggestions along the way. The paper also provides an example implementation of the design rationale in order to demonstrate its use when planning a BCI game based on the classic Battleship two-player guessing game. Overall, Ferreira et al. provide a thoughtful and engaging meditation on BCI game design that developers interested in BCI can learn much from.
Reference: Ferreira, A. L. S., Marciano, J. N., de Miranda, L. C., & de Miranda, E. E. C. (2014). Understanding and Proposing a Design Rationale of Digital Games based on Brain-Computer Interface: Results of the AdmiralMind Battleship Study. SBC Journal on Interactive Systems, 5(1), 3-15. (Link)