MOCU was created to provide its users the ability to control and monitor multiple existing or newly developed heterogeneous unmanned systems simultaneously. The term heterogeneous is used to describe vehicles which are dissimilar in both modality (land, air, sea, or undersea) and communications protocol. The goal is to not just minimally control, but to have full access to the vehicle/sensor and its payloads. To achieve this goal, a modular, scalable approach was developed.
The technical approach will be to leverage, to every extent possible, previous and on-going work in the three primary technology areas (sensors, 3D modeling/reconstruction software, and visualization tools). By following the Technology Transfer collaboration model of harvesting technology from all sources, applying seed money where needed, and partnering with other programs with similar goals, an integrated solution will be developed. This project will not have enough funding to develop and integrate all of these technologies from scratch.
The primary technology advancement envisioned under this effort will be the maturization of existing component technologies, and, even more so, the integration of those technologies into a single cohesive system.
The primary advancements achieved under this project will be the maturity of and extension to existing 3D modeling, registration, and viewing technologies. Currently, the 3D modeling technologies are capable of providing medium resolution models of one perspective of an object. This project will focus on maturing and developing technologies that provide a full 360 degree model at a high resolution within a reasonably short amount of time such that the capability will be useful in an operational scenario.
The approach taken will include a survey and evaluation of the existing technologies looking at how well they meet the requirements and/or how easily they can be matured to meet the requirements. After the best technologies have been identified they will be matured, integrated, tested, and demonstrated in incremental levels until the combined system is at a TRL 6.
A preliminary list of tasks planned under this project includes the following:
- Technology Survey/Investigation/Evaluation – conduct a thorough survey, investigation, and evaluation of all sensor, modeling, and visualization technologies that could apply to this project.
- Technology Development Roadmap – the development of a roadmap that will take selected technologies from their current state to a maturity level required for integration into the final system as well as the development path for the integration of those components.
- Software and Hardware Technical Development – the execution of the technology development roadmap. This will likely include work to mature sensor systems, modeling software, the registration algorithms, and the visualization tools.
- Integration – the integration of the sensor systems, model-reconstruction software, registration algorithms, and visualization tools into a cohesive system. This phase is less about overcoming technical hurdles and more about working out the interface issues.
- System Demonstration – this includes the testing and demonstration of the completed, integrated system in a relevant environment.
MOCU is inherently scalable, due to its modular nature. As stated previously, most of the functionality within MOCU resides in its modules. Depending on the system requirements, the capabilities of MOCU are adjusted by adding or removing modules. The memory footprint and computational requirements are determined by which modules are loaded. It is important to note the core does not need to be modified to add or remove modules.
Figure 3. Handheld OCU for PackBot Scout
If the application requires a minimal installation, only the required modules to control or monitor a specific vehicle or sensor are installed on the target machine. For instance, the user may require small handheld hardware to control a teleoperated iRobot PackBot Scout (Figure 3). In this example, where only video and status are necessary, the OCU would consist of the MOCU core, PackBot protocol module and PackBot media module. However, a larger control structure where multiple semi-autonomous vehicles communicating in different protocols and modalities may require more functionality including maps, vehicle status, planner and scheduler. MOCU can be reconfigured to run on a broad range of hardware, including small embedded processors, all using the same core and a subset of available modules.
FLEXIBLE USER INTERFACE
The user interface (display as defined by XML configuration files. The user interface definition includes the display and user input. The display configuration specifies the location and types of maps, video, gauges, etc. The user input includes the mapping of the joystick, keyboard, and mouse to commands in MOCU.
MOCU has two general modes of operation; monitor and control modes. In both modes, the user interface is defined by the configuration files discussed in section 2.4. One operator can use MOCU to monitor multiple vehicles and sensors. At any given time, the operator can only control one vehicle but monitor all of the devices in the system. Control is requested of the vehicle and if granted, MOCU enters into control mode.
In monitor mode, all the vehicles in communication with MOCU are shown geographically on all displayed maps (Figure 2). Vehicle status information is also presented for each vehicle along with the option to display video if the status gauge is available. No control is available to the vehicle or sensors while in monitor mode. At any point, the user can switch to control mode by selecting a vehicle.
While in control mode, MOCU has complete control of a vehicle or sensor and its payloads. More status information is available for display and the vehicle or sensor can be controlled by teleoperation or by sending waypoints to the vehicle, depending on what the vehicle or sensor supports.
The configuration files, like the modules, are loaded at runtime. Because MOCU configures itself via the configuration files at runtime there is no need to compile the core software to make drastic changes. For instance, changing the type of map displayed or adding a compass gauge to an existing display for a particular vehicle can be done by modifying an XML file with a standard text editor.
MOCU uses configuration files to determine how the display is arranged and where user input is mapped. A special configuration file specifies the display layout in monitor mode. In control mode, the current configuration is determined by the currently selected vehicle’s identity. MOCU attempts find the configuration file that is most specific to that vehicle, starting with the vehicle name and progressing through model, class, manufacturer, protocol and finally, modality (UAV, USV, etc.). This allows one configuration to be used for all vehicles or a different user interface for each vehicle or the same interface for vehicles of the same modality, or any variation of these. For example, one configuration file can be used for all iRobot vehicles by specifying “iRobot” in a configuration file. Alternately, two separate configuration files can be created, one for the PackBot Scout and one for the Gator. MOCU will select the most specific configuration file for a particular vehicle. If one file is general (iRobot) and another is specific (iRobot Gator), MOCU will select the specific configuration file over the more general file.
Figure 4. Example of a configuration for an unmanned sea vehicle. DNC and video are displayed along with various status in both graphical gauges and text.
Figure 5. Example of a configuration for an unmanned ground vehicle. No map is displayed but video is prominent.
The importance of having a flexible interface is derived from controlling and monitoring all types of vehicles and sensors. For sea vehicles, the operator will require digital nautical charts, detailing features in the waterways (Figure 4). However, for ground vehicles, detailed terrain information is required. In some applications, map data is not needed and only video and status information is required (Figure 5). In this instance, the display configuration would not include maps. The ability to easily modify the user interface is an important attribute to being a truly common OCU.
The layout of the display and mapping of the user input is generally customer specific. Each customer has their own requirements which data is to be displayed and how it is arranged. By using runtime loaded configuration files, each customer can see a different display but use the same software. For example, some customers require the display be configurable by the user and have a Microsoft Windows application feel with docking, resizable windows. Others require the application run full screen, with no dynamic window configuration. Another example is the mapping of joystick buttons. Each vehicle provides different functionality. The mapping of a joystick often reflects these special capabilities. With the configuration files, button 1 can be mapped to the IR filter toggle function for one vehicle and laser pointer toggle for another. Buttons in the display are accessed by the mouse or touch screen and are similarly configurable.
The configuration files specify how the information from the currently selected vehicle is presented on the display. Graphical gauges are used to give quick feedback while textual gauges provide absolute values where appropriate. The style of a gauge, including dashboard, bar graph, textual or indicator, may be chosen to present the in the format that is best suited to the information that is being conveyed.
The military is moving towards network centric warfare where systems will be required to communicate with one another to share information. MOCU has the capability to communicate with existing command and control (C2) systems to provide data from robotic systems to enrich the battle space awareness. The communication with a C2 system may be unidirectional (in either direction) or bidirectional. The operator may receive a greater perspective with knowledge of surrounding systems, but also the systems up the chain receive the information collected by the vehicles being monitored by MOCU.
In addition to receiving the route that a vehicle is executing, the C2 system can also send routes to the vehicle via MOCU. Depending on how MOCU is set up, these routes can either be passed on unchanged to the vehicle immediately, or they can be displayed to the MOCU operator as a C2 Link suggested route. In the latter case, the MOCU operator can accept the route as is, modify the route before sending it down to the vehicle, or reject the route outright. Whichever option the operator chooses, the action is reported to the C2 Link so that the higher level C2 system is aware of what the vehicle is currently doing.
Track data is one of the most common bits of information to be exchanged between a higher level C2 system and MOCU. For example, a vehicle or sensor is tracking radar contacts that information is sent up to the C2 system to increase situational awareness for the area around each of the unmanned systems. Similarly, the C2 system can relay contact or track information down to MOCU. Since the devices that generate the contacts for a higher level C2 system are much more capable and accurate than those residing on typical unmanned vehicles this is a tremendous opportunity to increase the situational awareness of both the MOCU operator as well as any vehicles that are communicating with MOCU. This contact information can be used by the unmanned vehicle for path planning and obstacle avoidance purposes.
Originally created due to the need for a single Operator Control Unit (OCU) that can be used to control a wide variety of unmanned systems. Before MOCU new vehicles required a new custom OCU to be created specifically for each one, while at the same time large parts of the OCUs overlapped for most vehicles. The first version of MOCU took advantage of those similarities by introducing the dynamic loading of modules on top of a “core” module common to all vehicles. MOCU 1 had modules of different types – map display module, vehicle control module, video module. A few limitations became apparent with the design of MOCU 1. There was a different API for each module, which made it difficult to add new module types. Furthermore, all existing modules had to communicate directly only with the “core” and could not communicate with each other. The display of MOCU was determined primarily by configuration files, which made customizing the display simply a matter of editing a text file.
The improvements made in the second version of MOCU allowed for direct communication between different modules. This was accomplished via the Data Server module, which uses a publish/subscribe interface in order to establish a platform for data exchange between all MOCU 2 modules. A “core” module still provided the base functionality and tied all other modules together. But modules could publish their objects, properties, and methods and could subscribe to those of other modules via the Data Server. That way, when one module published a change, all subscribed modules received a notification of the new data. A module could also be notified when an object subscribed to one of its published properties or objects. Another benefit from the Data Server is the independence that it brought to each module. Thanks to the Data Server new modules for new functionalities could be added without affecting the functionality of MOCU 2 or that of previous projects. As a result, MOCU 2 could run with only a few required modules, and so it could be loaded on small embedded systems with limited space, but it could also be used as a multi-module mega-control-unit since it had the capability to support such a vast range of modules.