Email this Article Email   

CHIPS Articles: Army experiments with human experience-driven mixed reality to enhance Soldier performance in multidomain operations

Army experiments with human experience-driven mixed reality to enhance Soldier performance in multidomain operations
By CCDC Soldier Center Public Affairs - June 30, 2020
NATICK, MA – U.S. Army researchers launched a new working group to streamline human performance research efforts on Soldier mixed reality display technologies aimed at enhancing cognitive performance, user experience, and human-systems integration during operations and training.

Mixed Reality, or XR, is a collection of technologies that spans a continuum extending from the completely real to the completely virtual environment. XR includes augmented reality (AR), when real environments are enhanced by digital elements, augmented virtuality (AV), when virtual environments are enhanced by real elements, and entirely virtual reality (VR). While XR refers to a collection of similar technologies, it does not necessarily consider the human using the technology, like human experience-driven mixed reality, or HEXR, does.

Department of Defense researchers spanning joint services and agencies as well as academia and industry are working to understand how XR impacts cognition, user experience, and human systems integration in order to improve the technology with a focus on enhancing Soldier performance.

Early results indicate that XR capabilities and integrated display systems hold great potential to enhance warfighter lethality in combat operations and increase effectiveness in training – key tenets of the Soldier Lethality and Synthetic Training Environment Cross-functional Teams underpinning the human performance element of the Army modernization enterprise.

XR promises to deliver improved battlespace visualization and mission planning capabilities, intuitive presentation of relevant and timely mission information, and new communication capabilities for small teams operating in multi-domain operations (MDO). In training, XR offers engaging and immersive multimodal simulations of tasks and environments, accelerated learning and training, and timely practice and rehearsal of cognitive and motor skills.

Army leaders want to advance these pivotal capabilities, and researchers are leveraging the collective efforts of the XR community by sharing results and data in a more efficient way.

The Human Experience Mixed Reality Working Group, or HEXR-WG, was established by Army civilian XR researchers to provide a centralized collaboration platform that more efficiently links stakeholders to researchers and engineers across the DoD’s mixed reality human experience community.

This grassroots effort is being spearheaded by the U.S. Army Combat Capabilities Development Command Soldier Center and supported by partners at CCDC’s Command, Control, Computers, Communications, Cyber, Intelligence, Surveillance and Reconnaissance, or C5ISR Center; and the CCDC Army Research Laboratory (CCDC ARL).

“If we neglect to consider the human as an integration platform for XR technology, this could lead to the development of XR systems that are underutilized or negatively impact human performance,” said Dr. Aaron Gardony, a research psychologist from CCDC Soldier Center who is serving as the center’s lead representative to the HEXR-WG.

“The HEXR-WG’s main goal is to ensure XR technologies developed and fielded by the Army fully consider and truly enhance human experience and human performance by collectively working to ensure these technologies are intuitive and easy to use, enhance components of Soldier lethality like situational awareness and decision making, and avoid unintended cognitive and physical impairment.”

The result will be a collaborative approach across the CCDC, Army, and joint services to ensure that appropriate knowledge and expertise are applied, duplication of effort is avoided, and successes and lessons learned are adequately leveraged and retained, according to Gardony.

As a member of Soldier Center’s Cognitive Science & Applications Team and lead for the Measuring and Advancing Soldier Tactical Readiness and Effectiveness, or MASTR-E Program’s “Soldier Optimized Augmented Reality for Enhanced Tactical Decision Making” work package, Gardony assumed the lead role in standing up the group by preparing a charter, recruiting membership, and co-organizing a virtual kickoff meeting held on May 21.

CCDC Soldier Center will share results from its MASTR-E program in addition to efforts underway within its Simulation Technology & Training Center and the Mission Information, Human Factors, and Cognitive Science & Applications teams, which all have various HEXR research and development efforts falling under the group’s three major research components: cognitive performance, user experience, and human systems integration.

ARL will also share results from their XR-focused work.

"Our research aims to improve total system performance between Soldiers and intelligent systems,” said Dr. Michael Guess, researcher at the CCDC Army Research Laboratory's Human Research and Engineering Directorate. “Mixed reality display devices offer unique ways to communicate situational awareness information to Soldiers. Additionally, sensors on mixed reality devices can be leveraged to provide Soldiers with novel capabilities.”

“Our work seeks to effectively integrate Soldiers and systems in unique ways that will allow for improved decision-making,” said Guess. “We are excited to collaborate with Dr. Gardony, and the broader CCDC mixed reality community, through the HEXR-WG.”

Emerging XR capabilities, such as the Integrated Visual Augmentation System (IVAS), will be rapidly fielded to Soldiers in the next few years, and the group will work closely with XR stakeholders like the U.S. Army Program Executive Office (PEO) Soldier’s Program Manager (PM-IVAS) to share results and analysis. By rapidly responding to requests for input and communicating relevant scientific findings, subject matter experts will be able to inform IVAS development and ultimately improve its human experience for Soldiers.

"My focus at CCDC C5ISR Center has been on developing mixed reality capabilities for PM-IVAS in support of the warfighter,” said Dr. Robert H. Luke III, a computer engineer and the Heads-Up Display (HUD) Experience Lead for PM-IVAS. “It was brought to my attention that the capabilities we are creating could be better optimized for Soldier interaction by collaborating with the HEXR-WG.”

“Together, our teams will work to bridge the divide between technology creation and Soldier acceptance," said Luke.

Looking forward, the group will continue to meet regularly to foster collaborations, share research outcomes, and stay informed of XR technological developments.

“We are excited to execute the working group’s vision by advancing the state-of-the-art in human experience-driven XR and to enhance Soldier performance,” said Gardony.

For more information, visit:
Army Futures Command
Army Research Lab
Network Enterprise Technology CommandARCYBER
Army Cyber Center of Excellence
Army Cyber Institute

Human Experience-driven Mixed Reality (HEXR) research is shaping how Soldiers will operate and train. (CCDC Soldier Center)
Related CHIPS Articles
Related DON CIO News
Related DON CIO Policy

CHIPS is an official U.S. Navy website sponsored by the Department of the Navy (DON) Chief Information Officer, the Department of Defense Enterprise Software Initiative (ESI) and the DON's ESI Software Product Manager Team at Space and Naval Warfare Systems Center Pacific.

Online ISSN 2154-1779; Print ISSN 1047-9988
Hyperlink Disclaimer