Skip to main content
Skip to article control options
No Access

A Visual Analytical System for Mission Modeling and Execution

AIAA 2021-4094
Session: Designing Human-Machine Teaming
Published Online:

View Video Presentation:

A multi-robot swarm depends on a behavior control system that translates behaviors to observable and coordinated patterns of activity of an agent in a swarm. These agents process input signals from sensors that quantify characteristics of the environment, and/or other agents [1]. Solutions such as reactive feedback control loops were used in early multi-robot systems. State-based techniques have been adopted to represent complex behavioral systems efficiently through modularizing [2]. The behavior control system becomes an implementation of a finite state machine or is implemented by state-based approaches [3]. Each agent in a swarm can have different attributes, such as size, capability, sensory, and mobility. To support each agent to function in a different environment the behavior control system should provide communications with other software programs and available resources on demand based on behaviors it receives. The behavior control system invokes predefined routines with different parameters an agent generates or captures in a geometrical space. Then, can coordinate activities between different agents in the swarm, such as swarm movement, collision avoidance, and object pickup. Hierarchical state machines (HSMs) have proven to be very well suited for these robot behaviors for several years in NASA applications because of its expressiveness with compact behavior descriptions. HSMs can then do formal analysis and verification such as deciding reachability of certain states or model checking [4] against properties of specific behaviors. The automated process exceeds manual analysis in systems with very large and complex behaviors [5]. A visualization tool for HSM can aid system engineers and operators to build and monitor an HSM and reduce the complexity of behavior construction that ultimately improve the scalability of an HSM-based system. The visualization can help system engineers and operators to trace transitions from a source state to a destination state with trigger, guard, and action through profound formal approaches. The hierarchy created in an HSM allows engineers and operators to validate transitions across HSM boundaries between behaviors by checking an appropriate hierarchical relationship between behaviors and allow the agent to transition. The communication between different agents in a robotic swarm team can be treated as additional inputs and outputs to the HSM. They are used to coordinate task planning, motion, and operations of multiple robots. The research builds a visual analytical system that provides human operators the status of a multi-agent autonomous robotics system and allows the user to take actions to the system. The system follows the mission model defined by the autonomous system and visualizes mission progress across multiple agents. The system uses innovative information visualization and human-machine teaming techniques to communicate high dimensions of information provided by the execution engine and behaviors of the multi-agent autonomous robotics, especially, information that are hard to be explained in a large flowchart using hierarchical state machines.