UW Applied Physics Laboratory (APL)

A Low Cost, Open Source Visual Odometry Solution For Coastal ROV

In situ information is key to guide management, conservation, and restoration of coastal ecosystems. For population or fishery management, this equates to information about species abundance and distribution. For marine debris mitigation, one requires information about the spatial distribution of, e.g., “ghost-pots,” derelict crab pots that require manual removal. While deep-water-rated Remotely Operated Vehicles (ROVs) have historically been used to explore benthic locations, divers have typically been used to answer questions in relatively shallow (5–50m depth) ecosystems. However, dive operations are labor intensive, hazardous, and constrained by short working times, particularly at greater depths. As the field has matured, low-cost ROVs have emerged on the market that are highly capable, customizable, and apt for standardized shallow-water deployment. Alongside partners and collaborators, the Seattle Aquarium is pioneering the use of small, customized ROVs to conduct rigorous, standardized surveys of shallow ecosystems such as kelp forests. As a non-profit seeking to maximize the collective capacity for data collection, the Aquarium seeks to develop a framework for ROV operations that can be adopted by other entities. Towards that end, the Aquarium is making their approach, specific methods, and code as accessible as possible, such that state, federal, Tribal, academic, and non-profit partners can leverage low-cost ROVs for data collection. Unfortunately, some of the sensors required to implement rigorous ROV survey protocols remain relatively expensive, e.g., an acoustic Doppler Velocity Log (DVL) is $8k–the single most expensive item of an otherwise low-cost (~$12k) framework. However, it is highly likely that stereo cameras, on-board computing, and computer vision methods (e.g., Visual Odometry) could replace – and potentially even improve upon – the performance of the DVL for ROV navigation and localization near the seafloor. Doing so would enable a greater degree of ROV positioning and control for a much lower cost, thus enabling a larger number of entities to adopt the methodology and advance their own coastal questions of interest. This project is supported by Dr. Aaron Marburg at UW’s Applied Physics Lab (APL) and Dr. Zachary Randell and Clyde McQueen at the Seattle Aquarium. Dr. Marburg’s experience and knowledge about robotics, perception, and situational awareness makes him ideally suited to co-advise students on this project. Dr. Randell leads the Aquarium’s ROV research program and will enable students to experience and understand the real-world field requirements of shallow-water ROV operations along coastal Washington. Additionally, the team’s extensive experience with robotics and software development will ensure the students have access to the technical expertise required to successfully advance this project. This student team will work to develop a downward-looking stereo camera payload to aid ROV localization and navigation. A preliminary goal this student team will work toward is to use computer vision to measure vehicle speed over the seafloor (through visual odometry) and altitude (through stereopsis) as a supplemental input to the vehicle’s integrated navigation system. A stretch goal this student team will work to achieve is to perform vehicle localization through the identification and mapping of visual landmarks (i.e. visual SLAM). The core challenges will be in both selecting and implementing an appropriate algorithm, but also evaluating its performance in real-world subsea conditions, and mitigating areas which require improvement. The payload will be electrically and mechanically integrated into a Blue Robotics BlueROV2 ROV (provided by the project mentors). This student team will work to integrate the outputs from the sensor into the existing software control system for the ROV, based on the ArduPilot / ArduSub firmware. This will include identifying and handling conditions where their visual approach is inaccurate or failed due to occlusion, lack of features, or turbidity; and propagating system status both to ArduPilot and to the human operator. A stretch goal this student team will work to achieve is to perform closed-loop control (e.g. station keeping or track-following) using information from the student’s navigation sensor. This student team will work to design experiments to quantify the performance of their algorithm. Metrics will include accuracy of velocity and altitude measurements for the visual odometry, and position estimates for visual SLAM. The ROV will be equipped with a Water Linked DVL which will provide a direct comparison on vehicle velocity and altitude. This student team will also work to use a prepared environment with visual fiducials to allow measurement of ground truth vehicle position. The system will also be evaluated on size, weight, power and cost (SWAP-C) although these parameters will not be prioritized for this initial prototype. Students will be responsible for all elements of the project, however, the mentors will provide strong guidance on critical-path decisions, e.g., the selection of a computing-camera platform. The team mentors have extensive experience integrating with ArduPilot and can guide that software development. Further, APL staff engineer and machine shop resources are available to assist with design of mission-critical pressure vessel(s) and other components. The student team will be required to complete preliminary and critical design reviews for core engineering elements of the project (electrical and mechanical integration into the BlueROV, software integration into ArduPilot) with team sponsors. Depending on the needs of the team, testing may occur in the UW School of Oceanography test tank, from shore near campus (ship canal, Lake Union), near the Aquarium or from an Aquarium vessel. The Aquarium will provide access to Piers 59, 60, and 62 to facilitate field testing. If it would be useful to the students, this will include the deployment of a fixed grid pattern along the seafloor to enable video calibrations and testing. Additionally, if useful to the students, the Aquarium will make its vessel available thus enabling a wider array of test conditions. This student team will work to design and build a downward-facing stereo camera payload for the BlueROV suitable for shallow water experimentations (<30m). They will work to implement a visual odometry algorithm and integrate output from this algorithm into the existing BlueOS/ArduPilot navigation and control system for the BlueROV. The team will work to design experiments to quantify the performance of their algorithm in comparison to existing solutions (e.g. navigation without a velocity source and with the existing ROV DVL). As a stretch goal, students will work to implement a visual mapping / vSLAM algorithm and demonstrate autonomous navigation of the BlueROV along tracklines, including evaluation of its performance. Deliverables this student team will work to achieve include: -A working prototype of the student camera system, integrated into a BlueROV2. -All mechanical and electrical integration details will be documented and shared publicly under an appropriate open source hardware license through a project website / Github repos. -Within the limits imposed by third-party tools used, all software will be documented and published in an open source manner through a project web site / Github repos. -Students will work to complete all required publications for CoE capstone program -Ideally, students will work to participate in or produce additional media for program supporters including APL, Seattle Aquarium and Blue Robotics.