By Mr. Mark Rothgeb and Dr. John Sustersic


With the advent and successes of the MQ-1 Predator, RQ-4 Global Hawk, MQ-9 Reaper, and other Unmanned Aerial Vehicles (UAVs), expectations for vehicles with extended reach capabilities have been growing within the warfighter community for Autonomous Underwater Vehicles (AUVs). The Predator's significant impact in the Middle East operational theatres beginning with Afghanistan and Pakistan is largely due to the ability to achieve safe standoff for the warfighters. It provided reliable operation while executing relevant precision strike missions. The Chief of Naval Operations in a congressional report in February of 2016 stated:
Autonomous Undersea Vehicles are a key component of the Navy's effort to improve and expand undersea superiority. These unmanned vehicles will be able to operate independently from or in cooperation with manned vehicles..."


The U.S. Navy is working to make significant advances in unmanned vehicle autonomy, which traditionally has been limited to scripted operations with little to no intelligent decision-making. Aggressively pursuing solutions to extend the capability of the Navy using unmanned systems that are truly autonomous will increasingly remove humans from the loop and decrease the burden on human cognitive resources.

The challenges of the maritime undersea environment are unique and particularly severe as noted in the Defense Sciences Board's task force report.

" cannot overcome certain physical limitations of the marine environment, essentially mandating greater autonomy"

While AUVs can perform waypoint operations, avoid basic obstacles, keep from grounding, and activate payloads, their autonomous decision-making is limited and often non-existent. To be operationally relevant for most missions, AUVs must be able to perform multi-dimensional decision-making in an environment that simultaneously considers all factors including weather, sea state, water column characteristics, fishing areas, merchant lanes, geopolitical boundaries (territorial waters and economic exclusion zones), shipping lanes, threat active and passive detection, countermeasures, degraded self-health, information assurance, GPS denial, mission timeline constraints, water space boundaries, and energy management.

These real-world challenges are routinely resolved by submarine crews, who have 100 years of corporate experience in dealing with the manifold unpredictable realities of the ocean environment and underwater operations. The coming generation of AUV systems will need to leverage this expertise and knowledge and embed it in intelligent autonomy that will enable platforms to perform relevant missions. These systems will require intelligent multi-dimensional decision-making abilities akin to the human watch team on a submarine.

Simultaneous with the increasing mission need for intelligent AUVs is the rapid acceleration of Artificial Intelligence (AI) technologies, spear-headed by industry and technology leaders within government laboratories and academia. An anecdotal example of this is the race by auto makers to produce autonomonously operating vehicles. This has resulted in significant industry investment as they anticipate the market potential. Similarly, Apple (Siri), Microsoft (Cortana), Google (Google Now), Facebook (Fasttext) , and many others are working toward semantic understanding for improving human interaction with their systems.

Because of the convergence of increasing AUV mission needs and rapidly advancing AI technologies, the Navy community is on the verge of creating AUVs with greatly increased operational capability and usefulness. Deep belief networks, genetic algorithms, learning systems, cognitive architectures, and other AI techniques will provide critical capabilities in leading this advance.


Figure 1. MANTA incorporates a software agent for the primary roles of the Submarine Watch Team.


A promising next-generation autonomy system has been developed through funding by the Office of Naval Research SwampWorks to develop a framework in which cognitive technologies and crew knowledge and expertise from Subject Matter Experts (SMEs) can be naturally combined. As a starting point, the team determined that incremental improvement in existing fielded autonomy technologies, many of which date back to the 1980s and 1990s, would be insufficient to handle the decision complexity and ambiguity of real environments that future missions require. A clean sheet approach was taken to incorporate cognitive decision-making and the capability to enable learning mechanisms as an intrinsic part of the architecture from the ground up. The team created the "Multi-agent Architecture for Natural and Trusted Autonomy" (MANTA) system, which directly incorporated the submarine watch stander crew as its model for autonomy as depicted in Figure 1.

Using the submarine watch team as a model for autonomy provided significant benefits.

  • Functional decomposition and modularity of the system is natural in that each member of the watch stander team is well defined and coherent, which is fundamental to software architecture.
  • Knowledge of the crew and its experience naturally fit into the appropriate correlated agent.
  • Interactions between watch team members (software agents) is well defined in the submarine "Interior Communications Manual" (ICM), and these natural language litanies form the basis for the messaging interfaces and agent interaction.
  • Delegated authority provides back-up and overwatch as the Commanding Officer (CO) delegates to the Officer of the Deck (OOD) and on down. High-level monitoring and correcting occurs when needed. This adds to resilient and safe operation when incorporated within autonomy.
  • Levels of constraints are inherent in the team hierarchy, providing layered operating margins for mission execution. For example, water space is allocated from fleet to the submarine. Within that allocated waterspace the CO directs the navigator to define the ships operating envelope (where in the Ocean the submarine will operate) to provide safe transit and operating constraints, and within that the OOD works to stay near the Path of Intended Motion. Using this same approach in software provides layered safety and makes it robust with regard to the dynamic environment.
  • Collaborative operations between manned and unmanned systems is natural in that the vehicle is tasked with mission objectives, timelines, and priorities within its capabilities just as a human or another manned system. This means that no specialized interface is required between manned and unmanned systems.
  • Trust in autonomy (an often overlooked but critical attribute for autonomy acceptance) is attained as tasking and interactions are all natural language. Explanation of decisions is provided by each agent as it would be in the watch team providing transparency.
  • The approach is open-ended (not brittle) in that submarine crews are the most capable autonomous underwater system (albeit manned). As we evolve to capture their decision-making and expertise in cognitive and learning systems, this architecture provides a natural landing pad within the relevant correlated watch team agent.
  • As a longer term goal, algorithms for submarine operations may be naturally ported to autonomy, and autonomy capabilities can be used to up-level the submarine crew over time and be used for training. For example, an evolved capable navigator agent may be used to train navigators or could even augment the submarine crew as an intelligent advisory agent.


Figure 2. The layered approach in each agent allows naturally appropriated technologies to be
applied in each layer.


A crucial feature in MANTA's approach is the ability to apply the correct technology to the agents within the system and to enable incorporation of future technologies without breaking the model. Figure 2 shows that each agent is composed of a cognitive layer, a computational layer, and an interactive layer. The cognitive (thinking/deciding) layer, a piece missing in operational AUVs today, enables multi-dimensional decision making to take place using cognitive technologies that are well suited to simultaneously handle the variety of considerations mentioned earlier, accomplishing the mission within environmental, threat, time, and capability constraints.

Architecture layers can be replaced as new technologies evolve. For example, MANTA started using Robot Operating System (ROS) for the intercommunications layers. Mid-stream in development ROS was replaced with ZeroMQ, an alternative messaging method. This was accomplished without impacting the computation and cognitive layers. Soar is currently used for the cognitive layer, but the layering allows replacement with a different cognitive engine such as Adaptive Control of Thought-Rational (ACT-R). An individual agent may also use a different cognitive engine than the rest of the agents. In addition to the intra-agent modularity, the inter-agent modularity in MANTA allows any agent to be replaced so long as the new agent conforms to the agent intercommunications.

To incorporate cognitive technology with the SME knowledge base, a cross disciplinary team composed of computer scientists (engineering and AI), cognitive psychologists, active duty Submariners, and mathematicians was used to develop the prototype system that was regularly tested throughout the summer of 2017 on a small IVER-3 AUV.

The resulting autonomy architecture breaks the paradigm currently used in today's AUV systems and is indicative of the coming revolution in intelligent autonomy.

While the cognitive layer is new with regard to fielded systems, leveraging the vast submarine operational SME knowledge base provides a jumpstart toward advancing development of knowledge-based systems that can be effectively used in smarter, fully autonomous AUVs. The rapid pace of innovation of cognitive systems and the capability to inject advancements into an architectural framework will continue to further enable better handling of the dynamic and unconstrained environments typical of the modern battlespace. Moreover, using a submarine tactical center as the fundamental design model for the AUV is a game changer. It enables a natural inclusion of cognitive decision-making functions at all levels (every software "role") and enables the evolution of more robust AUV systems.

Funding Source via Naval Sea Systems Command Omnibus contract N00024-12D-6404

Mr. Mark Rothgeb and Dr. John Sustersic Applied Research Laboratory at the Pennsylvania State University