As with any unmanned vehicle attempting to navigate in a complex environment, good sensor data is critical, and getting good data is often the most difficult part of the project. The marine environment poses many challenges including waves, spray, and a chaotic obstacle setting. There are, however, some advantages to the marine environment including well charted operating areas, absence of negative obstacles (holes or cliffs), a mostly planar surface (except for the waves), no vegetation, etc. It’s important that the sensors are selected to make the most of the environmental advantages and to provide the best data possible in the challenging realms. For the purposes of this paper, the sensors have been divided into two categories based on which OA component they support.
DELIBERATIVE OA SENSORS
The sensors for the deliberative OA component need to provide data about obstacles in the far-field (e.g., >200-300 yards) and provide state information (position, course, and speed) for the moving obstacles. These are the sensors that are most commonly used in the commercial marine industry and include nautical charts and X-band radar.
Digital Nautical Charts
The National Geospatial-Intelligence Agency’s (NGA) Digital Nautical Charts (DNC) are, according to the NGA’s website, “an unclassified, vector-based, digital database containing maritime significant features essential for safe marine navigation”.11 A chart server was created to parse through the hierarchy of thousands of files and convert all the locations of permanent stationary obstacles (coastlines, piers, buoys, shallow water, etc.) from latitude/longitude points to occupancy grid pixels in the obstacle map. The chart server sorts the DNC data into four main categories of obstacles: above water, on the water’s surface, below water, and land. Only those features with attributes of vertical clearance, horizontal clearance, or depth clearance that constitute a threat to the USV are added to the obstacle map. The data in the DNCs is encoded in a Vector Product Format (VPF), signifying that the data can be represented at any resolution without losing quality. This will become very important as the USV tackles missions ranging in hundreds of nautical miles but still avoiding obstacles in channels no wider than 50 yards.
Once the obstacle map has been created at a resolution beneficial for planning an obstacle free route, it is delivered to the deliberative OA component. This map is dynamically updated whenever the USV moves close to the edge of the map, when it is close to the edge of the DNC libraries’ boundaries, or when a new route extends beyond the edge of the map. Currently the maps are created at a scale of 1000 pixels wide by 1000 pixels high, with each pixel representing 10 meters by 10 meters. This resolution was chosen based on the size of the USV, the size of most relevant obstacles, and the intent to keep a safe zone around the USV of at least 50 meters.
The radar system on the SSC Pacific USV is a standard marine radar (Furuno) with a third-party PC controller. The controller, developed by Xenex Innovations Ltd., provides a digital networked interface for the radar. The Xenex system provides an API to access the radar data and controls as well as an Advanced Radar Plotting Aid (ARPA) Software Development Kit (SDK), which provides algorithms to automatically acquire and track up to 100 contacts.
SSC Pacific has invested a significant amount of effort in characterizing the performance of the radar’s ARPA function as this is the primary means of detecting uncharted obstacles in the far-field. As with any sensor in the real world there are many idiosyncrasies with the ARPA function. One significant problem with the radar is that it tends to classify noise from the shoreline return as contacts which are often shown to be moving at a significant velocity and in the direction of the USV. These false contacts are obviously detrimental to the successful operation of the path planner. To mitigate this problem, the on-board nautical chart server can be used to calculated polygons that follow the shoreline and structures along the shoreline. The radar contacts are compared with these polygons and those that fall inside a polygon are rejected and deleted from the radar’s list. An example of this is shown in Figure 12, where the blue circles and green leader lines indicate the current list of ARPA contacts.
Another challenge with the radar is that for a small, highly maneuverable boat, the turn-rate can approximate that of the radar itself. During high turn-rate maneuvers, the radar is either turning much faster or slower that normal, relative to earth, and the data is therefore skewed. When this occurs most often the contacts are lost until the USV returns to a relatively straight trajectory. Multiple approaches to mitigate these effects are currently under investigation.
Figure 12. Radar contacts with no filtering.
Figure 13. Radar contacts with shoreline filtering.
Automatic Identification System (AIS)
AIS is a system used by ships and vessel traffic systems (VTS) principally for identification of vessels at sea. AIS helps to resolve the difficulty of identifying ships when not in sight (e.g. at night, in fog, in radar blind arcs or shadows or at distance) by providing a means for ships to exchange ID, position, course, speed and other ship data with all other nearby ships and VTS stations. It works by integrating a standardized VHF transceiver system with a GPS receiver and other navigational equipment on board ship (Gyro compass, Rate of turn indicator, etc.).
REACTIVE OA SENSORS
The sensors for the reactive OA component need to provide high-resolution data about obstacles in close proximity to the USV (e.g., <200-300 yards) and at a much higher rate than the deliberative sensors. Some of these sensors are typically not found in the commercial marine industry but many have been used extensively in UGV programs.
SSC Pacific has recently acquired a Velodyne HDL-64E lidar with 360° by 26.8° field of view. Initial tests demonstrate that there are little to no returns for an unchurned sea surface which simplifies obstacle detection. The lidar can see obstacles as small as a kayak in the water at 50 meters. Larger obstacles with high reflectivity can be detected up to 100-120 meters.
Stereo vision sensors have been commonplace on autonomous unmanned ground vehicles for many years and are the primary obstacle detection sensors on the NASA Mars Rovers. SSC Pacific has been working with the NASA Jet Propulsion Laboratory (JPL) for a number of years to transition technology to its UGV programs. That work is now being extended to the USV domain with very promising initial results. The stereo vision system provides high-resolution 3D data about the near-field environment, which can be converted into a 2D obstacle map and fused with data from the other reactive sensors.
Stereo Vision Image
Stereo vision is capable of providing very high quality 3D data but also has the disadvantage of requiring precise calibration every time the cameras are mounted. There is also the risk that the cameras may move relative to one another slightly which will affect the calibration and result in erroneous data. Because of some of the potential pitfalls of stereo vision, SSC Pacific is also investigating a monocular vision solution. The concept is that in any given frame of video, the horizon line is detected and obstacles are located on the water portion of the image. Since the distance to the horizon is known (from the nautical charts), and the height and properties of the camera are known, a rough estimate of range can be calculated for each obstacle given the number of pixels below the horizon line an obstacle appears in an image.
SSC Pacific has developed the basic algorithms for detecting the horizon line under varying conditions (in the presence of city skyline, for example) as well as the algorithms for segmenting obstacles on the water. The initial results of this work are very promising and will be published in a future paper.
Digital Nautical Charts
The chart server also provides the reactive obstacle avoidance component a small portion of the larger deliberative planning map. The larger map is sectioned off, rotated so the heading of the robot is up, and delivered to the reactive obstacle avoidance component at a rate of 10 hertz. Although the usefulness of the chart data depends on the accuracy of the USV position estimate, it still makes sense to take every advantage of the nautical charts as much as possible.
Like the nautical charts, the radar is used again in the reactive component, but this time it’s the raw radar return data that is used. The radar return is, in essence, a ready-made obstacle map. In an ideal radar image, only obstacles on the water or the shoreline show returns. The radar data is converted from the polar scan line format to the Cartesian obstacle grid representation and fused with the other sensor data. Of course, the data is never ideal and often contains noise. At first inspection, it also appears that the radar returns no useful data within approximately 100 yards of the USV as it is a solid disk of noise. Upon further investigation, however, SSC Pacific has determined that both the image noise and the center disk noise can be filtered out, yielding useful obstacle data within the 100 yard radius, which is critical for the reactive OA component.