YAK: 3D Reconstruction in ROS2

So why YAK?

It’s difficult for robots to perceive objects in the real world, especially when those objects are shiny, previously unseen, or (gasp!) both! We are excited to introduce Yak, an open-source GPU-accelerated ROS2 package which addresses some of these challenges using Truncated Signed Distance Fields. The Southwest Research Institute booth demo from the Automate 2019 conference serves as a case study for a ROS2 system that integrates Yak into its perception and motion planning pipeline.

A bit of technical background

A Truncated Signed Distance Field (TSDF) is a 3D voxel array representing objects within a volume of space in which each voxel is labeled with the distance to the nearest surface. The TSDF algorithm can be efficiently parallelized on a general-purpose graphics processor, which allows data from RGB-D cameras to be integrated into the volume in real time.

Numerous observations of an object from different perspectives average out noise and errors due to specular highlights and interreflections, producing a smooth continuous surface. This is a key advantage over equivalent point-cloud-centric strategies, which require additional processing to distinguish between engineered features and erroneous artifacts in the scan data. The volume can be converted to a triangular mesh using the Marching Cubes algorithm and then handed off to application-specific processes.

Machined Al Test Part

Machined Al Test Part

YAK Reconstruction of Machined Al Part

YAK Reconstruction of Machined Al Part

Reconstructing in the real world

My group at Southwest Research Institute has been working with TSDFs since Spring 2017. This year we started work on our first commercial projects leveraging TSDFs in industrial applications. We developed our booth demo at the 2019 Automate conference to provide an open-source example of this type of system.

The set of libraries we use in these applications is called Yak (which stands for Yet Another KinectFusion, in reference to the substantial prior history of TSDF algorithms). Yak consists of two repositories: a ROS-agnostic set of core libraries implementing the TSDF algorithm, and a repository containing ROS packages wrapping the core libraries in a node with subscribers for image data and services to handle meshing and resetting the volume. Both ROS and ROS2 versions of this node are provided. A unique feature of Yak compared to previous TSDF libraries is that the pose of the sensor origin can be provided through the ROS tf system from an outside source such as robot forward kinematics or external tracking, which is advantageous for robotic applications since it leverages information that is generally already known to the system.

The test system, which is in the pipeline for open-source release as a practical demonstration, uses a Realsense D435 RGB-D camera mounted on a Kuka iiwa7 to collect 3D images of a shiny metal tube bent into a previously-unseen configuration. The scans are integrated into a TSDF volume as the robot moves the camera around the tube.

The resulting mesh is processed by a ROS node using surface analysis functions from VTK to extract waypoints and tangential vectors along the length of the tube. These waypoints constitute the seed trajectory for a motion plan generated by our Descartes and Tesseract libraries which sweeps a ring along the tube while avoiding collision. Camera and turntable extrinsic calibration was performed using a nonlinear optimization function from the robot_cal_tools package.

A picture of the reconstructed mesh used to generate a robot trajectory is below. Additional information about the development of this demo (plus some neat video!) is available here. As ROS2 training is integrated into the ROS-I Consortium Americas curriculum, we look forward to sharing more lessons learned about applying Yak towards similar applications.

Screenshot from 2019-03-19 20-10-22.png

What next?

Yak is open-source as of July 2019. While development is ongoing and we anticipate that the library APIs will continue to evolve, we encourage interested parties to check it out at github.com/ros-industrial/yak and github.com/ros-industrial/yak_ros .

ROS Industrial Conference #RICEU2018 (Session 4)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the fourth instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT (watch all but 1 talks in this YouTube playlist)
  4. Hardware and application highlights (watch all but 1 talks in this YouTube playlist)

Day 3 - Session “Hardware and Application Highlights“

Georg Heppner (FZI) and Fabian Fürst (Opel) At ROS-Industrial Conference 2018

Georg Heppner (FZI) and Fabian Fürst (Opel) At ROS-Industrial Conference 2018

In the fourth and final session of the ROS-Industrial Conference 2018, the focus was on hardware developments and applications implemented in industrial use cases. Fabian Fuerst, Opel, and Georg Heppner, FZI, delivered the session keynote. They presented their solution for flexible automotive assembly with industrial robotic co-workers. The application was developed as part of the EU EuRoC project. In this four-year competition, more than 100 participants initially worked on new robotic solutions for the manufacturing industry. In the course of several evaluation rounds, the team from FZI, Opel and MRK Systeme GmbH was able to assert itself successfully to the end.

During the course of the project, the FZI developed an automated robotic assembly for flexible polymer door sealings on car doors. The sealing is a closed ring, which has to be fixed with up to 40 plastic pins depending on the model, an ergonomically unfavourable task that could not be automated until now. The developed assembly cell is very flexible and open, so that the robot can be used without a safety fence. For this purpose, an external force control was developed that can be used easily and directly also for numerous other robots as a package of ROS-Industrial. The CAD-2-PATH software is used for the simple path creation for the robot. This enables a quick adjustment to other door models and does not require any expert knowledge. This is important because there are different door models and sealing types and the automation solution must be adaptable accordingly and quickly. It is notable that the application received positive assessment from Opel with regards to safety, typically a sensitive topic when applying novel tools such as ROS in automotive applications.

Paul Evans (Southwest Research Institute / ROS-Industrial North America) at ROS-Industrial Conference 2018

Paul Evans (Southwest Research Institute / ROS-Industrial North America) at ROS-Industrial Conference 2018

The presentation by Paul Evans, Southwest Research Institute and ROS-Industrial Consortium North Americas, provided current information on the activities of the North America Consortium such as strategic initiatives, trainings, and networking activities. These also focus on voices of members and include activities for the strategy alignment, for more robustness and flexibility and agility. There are also collaborations with OEMs who support ROS or develop their own drivers. At the ROS-I Consortium Americas Annual Meeting 2018, different applications were presented, for example an order batch picking robot from Bastian Solutions and a robotic system for agile aerospace applications like sanding, blending, drilling etc. for the U.S. Air Force. A last highlight that Evans presented was the ROS-I collaboration with BMW and Microsoft. While RIC-North Americas supported the evaluation of simulation environments that included physics engines the RIC-EU partners provided additional navigation support and training for mobile robots at the BMW plant to support assembly logistics. The solution is deployed on Microsoft Azure.

Mobile robots was also the topic of the lecture by Karsten Bohlmann, E&K Automation. He presented solutions for ROS on AGVs and perception-driven load handling and PLC interfaces.

Arun Damodaran (Denso) at ROS-Inudstrial Conference 2018

Arun Damodaran (Denso) at ROS-Inudstrial Conference 2018

Denso Robotics Europe was present at the conference with Arun Damodaran, who talked about Cobotta, the ROS-enabled collaborative robot. This is a six-axis arm with a reach of 342 mm, a repeatability of 0,05 mm and a payload of 500 g. It has an inherently safe design, meets all requirements for safety-standards corresponding to the ISO norms and is compliant thanks to safety-rated monitored function. Another advantage is its easy set-up and use. This is realized by the usage of the robot programming software drag&bot. Developed by the spin-off of the same name of Fraunhofer IPA, the software enables the programming of robots like Cobotta with the drag and drop principle. No expert knowledge is needed. The software is also based on ROS, works independently from any robot manufacturer and can be reused as well as shared via the cloud. Denso has been engaged in the development of ROS components and packages (simulation, control, path creating) for its robots since 2012 and now uses an open platform for controlling the Cobotta.

Felipe Garcia Lopez from Fraunhofer IPA focused on a networked navigation solution for mobile robots in industrial applications. This is particularly useful for changing environments in which mobile robots should independently select free routes. Fraunhofer IPA and Bär Automation, for example, have implemented a navigation solution for agile assembly in automobile production. With this, AGVs can locate themselves robustly and precisely based on sensor data, even without special infrastructure. This makes it possible to easily adapt existing paths or integrate new ones even after commissioning. Since the software's sensor fusion module can process data from almost any sensor, very customer-specific solutions can be implemented.

Another example is the networked navigation for smart transport robots at BMW. Here as well there were few static landmarks, a lot of dynamic obstacles and sparse sensor data in large-scale environments. A process reliability of more than 99% had to be fulfilled. The presented navigation as well as the vehicle control are ROS-based. At the end of the presentation, an outlook into Cloud-Navigation was given: Mobile robots and stationary sensors are then connected using a Cloud-based IT-infrastructure. The environment is cooperatively modelled and SLAM is used. This enables also solutions for “Navigation-as-a-service” meaning map updates and cooperative path planning for each robot. With Cloud-Navigation, local hardware and computational resources can be reduced and the quality and flexibility of the overall navigation system is enhanced.

Thomas Pilz (Pilz GmbH & Co. KG) at ROS-Inudstrial Conference 2018

Thomas Pilz (Pilz GmbH & Co. KG) at ROS-Inudstrial Conference 2018

ROS as an appropriate solution both inside and outside of industry – this was the starting point for Thomas Pilz, Managing Partner of family owned company Pilz. Combined with his own career and his experience with the first service robots, lightweight robots and robots outside production environments, he first described how the question of safety standards has changed in recent years. The definition and understanding of a robot is currently in the process of changing significantly. For Pilz, systems such as the Care-O-bot® from Fraunhofer IPA are the new upcoming robots. They operate outside of cages, are mobile and users can easily interact with them and program them using ROS. He sees ROS as a success factor for service robots because of its modular design, its standardization, additional flexibility through programming languages and its networked, interoperable system in line with Industry 4.0.

Robots that are to interact with humans are also changing the required safety technology at Pilz in the long term because all previous infrastructure such as fences is no longer required. This led Pilz to develop its own robot arm with appropriate safety technology. They use ROS modules developed by Pilz because they are breaking new ground with the development of the robot arm and can thus fall back on a broad programming knowledge base. They had nothing to lose with the new product. However, in order for them to meet the safety standards, the modules must no longer be changed in an uncontrolled manner. To improve this, Pilz recommends changing the safety standards so that they are also amenable to Open Source. Finally yet importantly, he believes that the term robot manufacturer will also change, because this role will increasingly be fulfilled by those who implement the application and no longer by those who produce the robot or components for it. In the lively discussion after the presentation, Pilz once again emphasized two arguments in favour of ROS. First: When it is said that ROS is tedious, one should bear in mind that the development of proprietary software is also difficult. Second: ROS is tedious, but fun. Pilz also sees ROS as a decisive factor for employee satisfaction and as an argument for staying with Pilz.

At the end of the conference, Gaël Blondel from the Eclipse Foundation presented the Eclipse Foundation and its Robotics Activities. The platform with around 280 corporate members, half of them from Europe, provides a mature, scalable, and business-friendly environment for open source software collaboration and innovation. Eclipse is vendor-neutral and offers a business-friendly ecosystem based on extensible platforms. They offer their own IP management and licensing but also accept other business-friendly licenses. Several working groups are particularly engaged in development processes for robotics. One example for a robotic project managed with Eclipse is the EU project RobMoSys that aims to coordinate the whole community’s best and consorted efforts to realise a step-change towards a European ecosystem for open and sustainable industry-grade software development.

At the end of the event, Mirko Bordignon and Thilo Zimmermann thanked the participants for another great and record breaking ROS-Industrial Conference. Presentations and videos of the event have been made available on the event website: https://rosindustrial.org/events/2018/12/11/ros-industrial-conference-2018

ROS Industrial Conference #RICEU2018 (Session 3)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the third instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT (watch all but 1 talks in this YouTube playlist)
  4. Hardware and application highlights

Day 2 - Session “Robotics meets IT“

Henrik Christensen (UC San Diego) at ROS-Industrial Conference 2018

Henrik Christensen (UC San Diego) at ROS-Industrial Conference 2018

The third session testified the growing importance of ROS to support the development and deployment of robotic solutions from companies outside the traditional boundaries of this industry. Predominantly software players such as Amazon or Google now offer platforms leveraging ROS, which they described during the session.

Henrik Christensen, from UC San Diego and ROBO Global, gave a very inspiring keynote speech on why robotics is increasingly using cloud technologies and how it will benefit from them. He outlined three factors as current business drivers for this development: the increasing demand for flexibility in production, the aging world population and the associated increasing demand for service robots at home, and finally the trend that more and more people live in cities, posing great challenges for logistics. All robot solutions must be cost-efficient and robust at the same time in order to offer the required reliability. If computer performance always had to be on board, the hardware would often be inadequate (e.g. for slim service robots for private use) or the costs for suitable hardware would be too high (e.g. for autonomous cars).

Technologies from or in the cloud can be a solution for this. Christensen presented the value of these ecosystems using extensive market examples and explained how they differ in agility and size. Many successful companies, primarily in the USA and Asia, have shifted their business model from owning things or technologies to orchestrating them and offering services. For robotics, ROS 2.0 can be a decisive door opener here, offering the standardization required for platforms.

Milad Geravand (Bosch Engineering) at ROS-Industrial Conference 2018

Milad Geravand (Bosch Engineering) at ROS-Industrial Conference 2018

The next presentations in the session took up these and similar ideas and presented existing solutions. Milad Geravand from Bosch Engineering presented a modular software platform for mobile systems such as cleaning, off-road and intralogistics robots and how they can be developed more efficiently. In his experience, the difficulties in the development process are similar in many companies: The applications are usually very different, the software is becoming increasingly complex, a structured deployment and integration process is lacking. ROS is not yet ready for the products and the leap from prototype to series production is still too big. With the software platform presented, which is based on ROS, Bosch would therefore like to address precisely these challenges and enable uses cases to be developed quickly and reliably.

Eric Jensen, working for Canonical, the company well known for the Ubuntu Linux distribution, presented the advantages of Ubuntu Core especially with regard to security that is still an open issue for ROS. The mentioned advantages are: A minimal, transactional Ubuntu for appliances, safe and reliable updates with tests and rollbacks, app containment and isolation with managed access to resources, a unique development environment familiar for Linux developers and the possibility to easily create app stores for all devices needed. Furthermore, Ubuntu has one of the biggest developer communities in the world and is backed by Canonical itself, an important plus for security. Last but not least, the system offers automatic security warnings for the „snaps“, the special package format in Ubuntu, system audits through package verification and compliance management – all are important features for an improved security.

Roger Barga (Amazon AWS) at ROS-Inudstrial Conference 2018

Roger Barga (Amazon AWS) at ROS-Inudstrial Conference 2018

Only a few weeks before the ROS-Industrial-Conference, Amazon, for a long time far more than an e-commerce store, had introduced its new platform AWS RoboMaker, which caused a sensation beyond the ROS-Community. Roger Barga, General Manager at AWS Robotics & Autonomous Services, kindly presented this novel development at the conference. Amazon's commitment to robotics is based on discussions with around 100 companies, during which they were able to identify two main problems in robot development. On the one hand, this is a very high demand for automation solutions with simultaneous difficulties with ROS such as security or performance. On the other hand, the development process is usually very inefficient.

The RoboMaker platform addresses these requirements with its four main components. It offers a browser-based development environment, which in turn has integrated cloud extensions for ROS as well as a simulation environment. The cloud extensions range from machine learning tools to monitoring and analytics. Concrete capabilities for robots include speech recognition and output, video streaming, image and video analysis, as well as logging and monitoring with Amazon CloudWatch. The simulation environment allows thousands of simulations to be run in parallel. The fourth component is fleet management, so that robot applications can be deployed over the air. The presentation ended with a short introduction to the learning environment of RoboMaker, with which Amazon applies reinforcement learning to robots. The robots then learn according to the principle "trial and error". By merging all errors within a fleet in the cloud, a large knowledge base is quickly available and not every single robot has to make a specific error to learn from, but it benefits from the learning experiences of other robots in the fleet.

The topic of robotics in the cloud was also the focus of the lecture by Christian Henkel from Fraunhofer IPA. In his experience, the deployment of ROS-based applications on distributed systems such as mobile robots is still too great a challenge, which he would like to address in his work with docker containers (dockeROS). With his solution, it is possible to simply run ros nodes in docker containers on remote robots.

Martin Hägele (Fraunhofer IPA) moderates a panel discussion with Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google).

Martin Hägele (Fraunhofer IPA) moderates a panel discussion with Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google).

With Damon Kohler, Google Robotics and its recently presented cloud solution were also represented at the conference. In his introductory remarks, Kohler mentioned several challenges related to cloud robotics, including security, connectivity and latency, and distributing work, e.g. partitioning problems. In contrast, he sees advantages such as scalability, collaborative perception and behaviour and a robust change management and monitoring. He sees cloud robotics as a further development of the well-known principle "sense -> plan -> act" around the component "sense -> share -> plan -> act" and as an interplay of edge and cloud processing.

The aims of cloud robotics are an increased launch cadence, more data and more users and a better resource utilization. This shall be reached by infrastructure as a service, design for small and decoupled components as well as tools for automation and orchestration. The ROS nodes correspond to the Google micro-services: They are stateless and replicable, which means horizontally scalable. The container orchestration engine Kubernetes helps to deploy and release these micro-services. Several mature and robust logging and monitoring tools like Stackdriver help managing the system. The heart of the whole is the Cloud Robotics Core, being available from beginning of 2019 that enables to integrate Kubernetes on robots. Overall, Google’s vision is an open platform and a thriving ecosystem where integrators, developers, hardware developers and operators can collaborate with customers efficiently.

The second day of the conference ended with a panel discussion. The panellists were Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google). Moderated by Martin Hägele (Fraunhofer IPA), they summed up some advantages from their respective company perspectives, but also existing challenges of ROS and the role of open source software and robotics for their corporate strategy.

ROS Industrial Conference #RICEU2018 (Session 2)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the second instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT
  4. Hardware and application highlights

Day 2 - Session “Software and System Integration Topics“

Dave Coleman (PickNik) at ROS-Industrial Conference 2018

Dave Coleman (PickNik) at ROS-Industrial Conference 2018

The second day of the conference started with the session "Software and System Integration Topics". Dave Coleman, founder of Picknik Consulting and lead maintainer of MoveIt!, opened the session with a very personal keynote about his commitment to open source software, from his student days to his role as an entrepreneur. He reported how he got in touch with the beginnings of ROS at Willow Garage and highlighted the unique spirit with which the project was incubated. He introduced the successful MoveIt! library, shared his lessons learned and the challenges which many open source projects face. As a proof of how Open Source and business can successfully coexist, he described the founding of PickNik and how the company is profitable without investors.

The following presentations were more technical and started with Víctor Mayoral Vilches, CEO of Acutronic Robotics. He talked about his company's solutions for system integration in modular systems, through the device H-ROS SoM (System on Module), used as example. In his opinion, ROS already addresses many programming needs, but system integration goes far beyond programming and requires extensive resources for each new project. He therefore sees modularity as an essential improvement. Combining the features of a real-time capable link layer made of RTOS and the Linux Network stack, and ROS 2.0, he presented the challenges and developed solutions to achieve easier system integration. He also gave insights into the use of AI to further reduce programming efforts and to train the robot instead, a technology that is still in its infancy. As part of a Focused Technical Project with ROSIN, the company also worked on the interoperability of modules.

Jon Tjerngren (ABB) at ROS-Industrial Conference 2018

Jon Tjerngren (ABB) at ROS-Industrial Conference 2018

Jon Tjerngren presented how ABB robots can be used with ROS. For this purpose, the company developed various ease-of-use packages with ROS that simplify and accelerate the setup of ABB robots. All of them are already freely available online: abb_librws can be used to off-load of computational heavy tasks, e.g. image processing. abb_libegm can be used for motion correction and as an StateMachine add-in for remote control.

ROS2 Embedded tailored to real-time operating systems was the topic of Ingo Lütkebohle’s presentation from Bosch Corporate Research. He emphasized the importance that ROS must also be integrated into the firmware. This would better address four challenges: hardware access, latency, power savings, and safety. To this end, he presented a solution developed in the OFERA project with which ROS2 can be used in microcontrollers.

André Santos from INESC TEC and University of Minho, focused on software quality. More and more robot systems are safety-critical systems, which places very high demands on the quality of the software. Finding errors in the code early on reduces costs and development time. Although there are various static analysis tools, none offers ROS-specific analysis. This is why the HAROS (High Assurance ROS) framework was developed, which is capable of extracting and, to some extent, reverse-engineering the computation graph. It also provides a visualization of the extracted graph and enables property-based testing for ROS.

Anders Billise Beck (UR) at ROS-Inudstrial Conference 2018

Anders Billise Beck (UR) at ROS-Inudstrial Conference 2018

Anders Billersoe Beck from Universal Robots was the last speaker in the second session. He introduced the new UR e-series (with integrated force/torque sensor, 500 Hz controller frequency and more new features) and how ROS supports it. For this, a new driver is developed in a Focused Technical Project of ROSIN together with the FZI, which will also remain open-source. The goal is to make a UR robot easy to use and enable plug-and-play with ROS. The driver should make two modes of operation possible: remote control and ROS URcap embedding. More supported features are calibration, a new safety system and easier programming. Beck concluded the presentation with some points that he believes are in need of improvement to make ROS ready for industrial applications. These are easier general use, proper handling of hard and soft real-time boundaries and supporting more control in edge devices.

ROS-Industrial Americas 2018 Annual Meeting Review

The ROS-Industrial Consortium Americas (RICA) held its 2018 Annual Meeting in San Antonio, on the campus of Southwest Research Institute (SwRI) on March 7th and 8th, 2018. This was a two-day event, with the 7th open to the public, including tours and demonstrations, followed by Consortium Members meeting on the 8th with a road-mapping exercise and project idea brainstorming.

This was the first time that RICA held the event over two full days. Also, this was the most well attended event, topping out over 80 people on the 7th. There were talks spanning from the more strategic/visionary to the technical with regards to open-source robotics application development. This provides an excellent cross-section of the technical development community and organization decision makers to share ideas and cross-pollinate taking back what they learned to their organizations.

The morning of the 7th featured:

  • SwRI Introduction - Paul Evans - SwRI
  • ROS-I Consortium/Introduction - Matt Robinson - SwRI
  • Manufacturing in Mixed Reality - Dr. Aditya Das - UTARI
  • Discussion on the Design of a Multiuse Workcell and Incorporation of the Descartes Package - Christina Petlowany - UT Austin Nuclear Robotics Group
  • Integrating ROS into NASA Space Exploration Missions - Dustin Gooding - NASA

The talks touched on a mix of how humans can interact with the technological solutions and also the need for solutions that can work within environments originally designed for people. The common thread is enabling humans and robots to work more efficiently within the same spaces, and leveraging the same tools.

Rick Meyers of the ARM Institute & Air Force Research Laboratory, during the lunchtime keynote, discussed the vision and motivations of Air Force ManTech to drive advancements in automation and robotics in the manufacturing environment. This tied into the motivation of the Advanced Automation for Agile Aerospace Applications (A5) program, and how ROS ties into the realization of the Air Force ManTech vision.

The tours and demonstrations included many different applications, all with either ROS/ROS-Industrial element, though in some cases complimentary. ADLINK Neuron focused on coordinated mobile robots and a means to assist their industrial partners to easily transition to the ROS2 environment and provide consulting services for DDS implementation and ROS-related algorithm development.

KEBA demonstrated their new ROS RMI interface integrated into their controller, while UTARI demonstrated Manufacturing in Mixed Reality implemented through the Microsoft HoloLens, allowing users to fuse process guidelines, real-time inspection data, and cross reference information to determine adaptive measures and project outcomes.

SwRI and the ROS-I team demonstrated an example of merging SwRI’s Human Performance Initiative’s Markerless Motion Capture combined with path planning to retrieve an object from an open grasp. SwRI’s Applied Sensing Department showcased their Class 8 truck enabling all attendees to go for a ride, while gaining insights to the vehicle’s capabilities. The ROS-I team at SwRI also presented Robotic Blending Milestone 4, Intelligent Part Reconstruction, with TSDF implementation, and Trajopt, a newly fully-integrated into ROS sequential convex optimizer. The UT Austin Nuclear Robotics Group demonstrated their improved situational awareness for mobile manipulation on their Husky platform where users could “drive” the system to pick up a presented object.

Finally, the SwRI team presented and demonstrated the A5 platform, which is a mobile manipulation platform designed to perform numerous processes on large aircraft in an unstructured setting. The process demonstrated was sanding of a test panel overhead. Overviews of the localization and planning on the visualization were included.

Talks for the afternoon centered around OEM and Integration service providers, and included:

  • ADLINK Neuron: An industrial oriented ROS2-based platform - Hao-Chih Lin - ADLINK
  • Unique ROS Combination with Safety and PLC - Thomas Linde - KEBA
  • Leveraging ROS-Industrial to Deliver Customer Value - Joe Zoghzoghy - Bastian Solutions

This set of talks brought home innovations by the OEM and service provider communities. Bastian Solutions’ story of concept via working with the ROS-Industrial team, through pilot and into production, demonstrated a real value proposition for mobile solution, and broader ROS-enabled, development for the integrator community.

The morning of the 8th featured:

  • RIC-Americas Highlights and Upcoming Events - Matt Robinson & Levi Armstrong - SwRI
  • RIC-Europe Highlights & ROSiN Update - Mirko Bordignon - Fraunhofer IPA
  • ROS-Industrial Lessons from Bootstrapping in Asia Pacific - Min Ling Chan - ARTC
  • ROS2 is Here - Dirk Thomas - Open Robotics
  • ARM Institute Introduction & Update - Bob Grabowski - ARM Institute
  • Windows IoT & Robotics - Lou Amadio - Microsoft

Matt Robinson covered strategic initiatives for the Consortium followed by Levi Armstrong covering RICA technical developments, including TrajOpt and Intelligent Part Reconstruction, Noether, PCL Afront Mesher, and Qt Creator updates and upcoming release.

Mirko Bordignon highlighted for the Americas audience what is happening around the ROSIN initiative, driving awareness, and furthering the global nature of ROS-I. Min Ling Chan shared progress within the Asia-Pacific region and the progress and status of the Pack ML Focused Technical Project, which has a Phase 2 launch coming soon.

Dirk Thomas of Open Robotics presented the latest on ROS2, and for the first time we were happy to welcome Bob Grabowski of the ARM Institute. The ARM Institute is the newest DoD Manufacturing Innovation Institute, and this is the first Annual Meeting since the Institute’s launch. Synergies between the ARM Institute and ROS-I will be important to monitor moving forward.

The morning session concluded when the Windows IoT and Azure teams were represented respectively by Lou Amadio and Ryan Pedersen, presenting their current strategy for ROS support and their plans moving forward, particularly for ROS2.

The featured keynote was presented by Dr. Phil Freeman of Boeing, “Why Boeing is Using ROS-Industrial.” Phil offered great insights to the value of ROS-Industrial for Boeing, and what it has enabled for their operations in the context of the challenges Boeing faces. The talk featured example applications and conveyed the message that within the robotics space we truly are at a tipping point with regards to capability and accessibility.

A road-mapping session was then conducted, focusing on problems to solve. The idea is to tie problems to projects and then identify the capabilities that need to be developed to meet certain prioritized problems. The problem focus areas were Human Capability, Quality Processes and Execution, Flexibility/Agility, and Strategy/Alignment. Common themes were: standard interfaces, documentation, ROS2 for Industrial applications, ownership and community engagement, simpler recovery means, and real-time diagnostics.

The afternoon speaker session touched on technologies that seek to enable richer and more reliable networking and data sharing/management through the application development/implementation process, and across the value stream:

Now that the dust has settled, these are some observations from this seat:

  1. ROS-Industrial is a big tent, and is truly global. Each Consortium needs to optimize how it works within their region to meet their member needs and optimally leverage resources available to them.
  2. As regional resources are optimized, the other consortia need to monitor developments, share information and ensure that all within the broader ROS-I organization are aware what is in-flight, what development activities are happening where, to reduce/eliminate redundant efforts.
  3. ROS2 is here, but there is work to do. It will be important to monitor developments and foster awareness to enable developers, solution providers, and end users to leverage ROS2 capability to complement their end solutions when and where appropriate.
  4. There are a number of innovators, solution providers, and end users realizing value proposition on ROS/ROS-Industrial deployments TODAY, and in some cases for some time. Let’s socialize and share their success stories.
  5. Foster both membership engagement and community engagement in the vision and execution of the vision for ROS-Industrial. We are excited to both enable start-ups to engage, but also improve how we leverage our University partners. Through effective projects, sponsorships, or roles within the ROS-I organizational structure, these all help foster a sense of community and subsequent ownership.
  6. There is an inflection point or tipping point, and for advanced robotics this seems to be an appropriate time. The idea also, that ROS can span beyond just the robotic processes, but do more to enable more intelligent processing via leveraging IoT, enable leverage of advanced technologies for further end user value seems to be gaining steam.
  7. We advance ROS-Industrial together. Engage, participate, communicate, and we succeed together.

As always, we are looking forward to feedback on the event and how to improve this event and events moving forward. We are looking forward to bringing back the online quarterly membership meetings, so keep an eye on that, as coordination and the invites are hosted on a rotational basis by the three Consortium managers. ROS-Industrial is an open-source project, and with that we seek to be open, and a be that forum for sharing ideas, and solving problems for industry in the 21st century.

Public day presentations can be found on the Event Page within the agenda after each speaker line item. Member day presentations are included behind the member portal, and are available for download.

Thanks for your support of open-source automation for industry!