Amazon

Multi-Robot 3D Scanner

This student team will work to design and develop a multi-robot 3D scanner which can handle and scan a variety of objects of different shapes and sizes and produce even and detailed scans of the entire part with little to no obfuscation or unscanned areas. Students will use an off-the-shelf or in-house developed 3D scanner which can produce detailed, full-color 3D scans of the parts, and the resulting 3D models should be noise-free and watertight. The solution the students work to develop should implement multiple robots which work together to scan all areas of the part without obfuscation, which may require adjusting grip one or multiple times. The solution the students work to develop should also be robust and adaptable to a wide variety of parts and should be accurate in terms of dimensions and colors. This project has some hard constraints, but is otherwise relatively open-ended. The solution the students work to develop should be flexible and adaptable, and should be able to handle and scan a variety of parts ranging in shape and size. The solution should involve multiple robots, all interacting together. At a minimum, there should be two robots interacting together, but students are allowed and encouraged to use more than two robots. How the student group chooses to implement this requirement is up to them. Students could approach this with one robot holding the scanner and the other manipulating the part, or use a stationary scanner and two arms that manipulate the part together. Students can either use an off-the-shelf scanner or develop their own, but it should utilize two or more of the following scanning technologies, including but no limited to: photogrammetry, depth cameras, LiDAR, laser, and/or another advanced scanning technology that can capture geometry and color. Finally, students should implement detection of obfuscation by robot end effector and implement a repositioning algorithm where the robot adjusts its grip on the part or hands it to another robot to allow all areas of the part to be scanned. The scanner the students work to develop should produce a high-resolution, full-color 3D model. There is no time limit, but shorter scan times (or less number of moves) are preferred (while maintaining accuracy and coverage). Students will be evaluated on accuracy, coverage, repeatability, time to complete/number of moves, and flexibility/adaptability to different objects and shapes. Performance metrics will be discussed and established with the student group and will be used for objective evaluation. Recommended metrics are accuracy (±0.5mm) and coverage (95%), but different metrics and/or values can be used. The overall scope and goals/deliverables of this project shall be discussed with the student group and adjusted if necessary. The main deliverables this student team will work to achieve are: - Multi-robot interaction to handle and scan a variety of parts with different shapes and sizes. At a minimum, students should use two robots, but they are allowed and encouraged to use more. This may require implementation of collision avoidance in the path planning. - Use of an off-the-shelf or homebuilt 3D scanner that utilizes photogrammetry, depth cameras, LiDAR, laser, and/or another advanced scanning technology that can capture geometry and color. - Detection of obfuscation by robot end effector and repositioning (adjusting grip) to allow all areas of the part to be scanned. - Post-processing algorithm (off-the-shelf or custom-developed) that removes background and noise, producing a high-res, watertight 3D model with full-color textures. - Design of a user interface which easily allows users to scan any object and retrieve the data. The stretch goals this student team will work to achieve are: - Even coverage/resolution across the entire model (all areas have the same LoD present). - Use of generative AI to generate/recover details (color, geometry) in areas which may be obfuscated or unscannable. - In the case of non-rigid objects, compensate for deflection or deformations in the shape of the object caused by the robot gripping. - Error detection and recovery: in the event of a crash or movement error, recognize the fault and recover.