Building Out a ROS2 Mobile Scan-N-Plan Demonstration

As part of the ROS Industrial Consortium Americas 2020 Annual meeting, SwRI demonstrated a mobile robot application bridging ROS 2 with ROS 1 drivers for a mobile application. The exercise refined our capabilities with ROS2 systems, and many lessons learned along the way will inform later work in collaboration between mobile bases and industrial manipulators.

ROS2 Mobile Scan-N-Plan Node Diagram

Our demonstration leveraged the Clearpath Ridgeback and a Kuka Iiwa. Based on previous work with the Iiwa, we chose to use the ROS1 driver available for the robot. To integrate this into the system, both the input and output of the robot had to be bridged to the ROS2 system. This was achieved with a modified version of the ROS action bridge, which enabled joint command actions to be streamed to the robot, and a standard implementation of the ROS message bridge, to retrieve telemetry information from the Iiwa. With the support of ClearPath, we updated the OS of the Ridgeback to support ROS Melodic, an important stepping stone towards future application bridging the mobile base to a greater ROS2 system.

ROS-I Americas Annual Meeting attendees ask questions after observing demonstration

The demonstration itself was an expansion on the normal Scan’N’Plan framework. The mobile platform was driven to a position of interest; in this case, an aircraft fuselage segment. A scan path, generated in the robot frame, was run and fed into YAK to create a model of the target. Once the scan was complete, alignment and tool path generation was done using a hybrid perception tool to remove masked features. Hybrid perception leverages machine learning classification on 2D data sets overlayed on 3D data, where both are available from the perception sensor set up, in this case an Intel® RealSense™ D435.

These tool paths were then streamed to the robot in the same manner as the scan path. However, our end effector (in this case, an array of lasers) would toggle on and off using segmentation described above, to protect regions of interest.

We hope all present enjoyed this demonstration. We look forward to applying the lessons learned here to future work with mobile robots!

Recapping the 2020 ROS-I Americas Annual Meeting

The 8th ROS-Industrial Consortium Americas Annual Meeting was held March 4-5, 2020, at Southwest Research Institute in San Antonio, Texas. It reminded us of both how far we have come, yet how much there is to be done. As is the tradition with this event, the first day was open to the public, while day two focused on the membership and the mission of the consortium, and what we can do as a body to further leverage open-source for industry.

For those who could not attend due to COVID-19 travel restrictions, we offered video conferencing. In some cases, we had to adjust the speaking schedule to accommodate for remote presentations. All of the Day 1 presentations and videos can be found on the event page, while all the Day 2 content is at the ROS-I Consortium Member Portal.

Day 1 – Overview, Panel & Tours

Day one kicked off with an introduction by SwRI’s Paul Evans followed by a Consortium Overview, by Matt Robinson, and a more technically focused talk on deploying ROS for industrial automation by SwRI’s ROS-Industrial Americas Technical Lead Levi Armstrong. This highlighted recent production deployments, addressed recent challenges, and discussed development of novel new capability for the end-user site.

The morning continued with a talk by Roger Barga, GM of AWS Robotics, on the Role of the Cloud in the Future of Robotics. Here Roger highlighted practical applications of cloud computing with regards to robotics application development, testing and deployment support. Currently AWS is supporting ROS, ROS2, and Gazebo within their services, and provide additional tools to facilitate adding features to applications such as text or speech recognition and security. A case study featuring member Bastian Solutions was shared and a call for multi-robot simulation use cases was put out to attendees.

Alex Shikany, VP of Membership & Business Intelligence of A3, spoke on the role of robotics on employment. Here the point was made that in times of investment in automation, unemployment also declines, indicated that for the U.S. that investment in automation coincides with increased hiring. Essentially, though there is an increase in the amount of work being automated, this doesn’t manifest in fewer jobs; the jobs evolve.

A panel discussion featuring Joshua Heppner of Intel, Lou Amadio from Microsoft, and Roger Barga from AWS fielded questions relating to cloud, sustainability, education, and areas for growth. The Americas Consortium has evolved in recent years with more engagement from the tech industry, as they seek to understand industry needs and how ROS and related applications can scale in robust, sustained ways.

To wrap up day one, Shaun Edwards and Paul Hvass of Plus One Robotics shared a perspective from the founders of ROS-Industrial. They challenged the community to think about breaking down barriers with a simple question: “Where is the Download Button?”

The afternoon featured tours and demonstrations within the SwRI organization and in collaboration with partners. SwRI demonstrations included a ROS2 Mobile Scan-N-Plan that featured localization, reconstruction, machine learning-based segmentation, tool trajectory planning, and optimization-based free space motion planning. Other demonstrations included SLED, and SwRI’s Ranger for autonomous vehicle localization, as well as work in single camera markerless motion capture by SwRI Human Performance Initiative, which has broad applicability in manufacturing robotics with regards to capability. Mathworks demonstrated some of the latest features related to ROS and Simulink, including safety, and UT Austin demonstrated improved means for ease of use in complex systems for inspection/handling tasks in hazardous environments.

The tour and demonstrations included off-campus visits to Xyrec and Plus One Robotics. Shaun Edwards and Paul Hvass offered tours and a behind-the-scenes perspective for how their business has grown and the need to provide their customers a means to try and test before deciding to implement. Xyrec visitors were treated to a firsthand overview the largest mobile robot produced, designed to apply laser ablation surface treatments to commercial aircraft.

Day one concluded over dinner and networking. It was an excellent opportunity to chat with key partners, stakeholders, and new faces about the landscape of advanced robotics, and how we are seeing more rapid progression from academia to factory floor.

Day 2 – Membership Highlights

Day two, the member-only portion, is a forum for membership to share what they are working on, challenges, and seek out areas and partners for collaboration. The day started off with Consortium updates by region, starting off with Americas, then progressed through the EU and Asia-Pacific. Each region has shown progress to delivering more content and training related to ROS2. The EU Consortium shared updates and outcomes of the ROSin initiative, and Asia-Pacific discussed the Singapore-led healthcare initiative Robotics Middleware Framework (RMF), which seeks to disrupt how healthcare adopts both IoT and robotics.

A panel discussed why end-users are looking to advanced robotics solutions via a ROS-based approach, and what are some of the motivations, and challenges that these approaches face today. David Leeson of Glidewell Labs, Greg Balandran of Spirit AeroSystems, and Mitch Pryor of UT Austin Nuclear Robotics Group shared anecdotes and practical examples where there are no solutions in the traditional industrial world. They also discussed challenges their organizations face and shared how an open-source model enables more efficient spread of emerging capability, hopefully preventing silo formation amongst a select few solution providers.

From here, Dave Coleman of PickNik shared recent developments around MoveIt2, including capabilities around real-time path planning. Open Robotics Developer Advocate Katherine Scott then shared developments and opportunities to collaborate around ROS2 and the curation of the ROS ecosystem.

The keynote for the member day of the annual meeting featured Greg Balandran, R&T Engineered Factory Automation Manager ad Spirit AeroSystems, who discussed ROS-I’s Influence on Automation Strategy. Greg shared challenges that his organization has faced and how leveraging ROS and working within ROS-I helped accelerate their automation development and roadmap execution.

The day two afternoon kicked off with a workshop where members broke into groups to discuss elements of the vision for ROS-I, needs, and the areas where membership can focus for advancement. The key takeaways from this workshop were:

• There is a huge desire for richer industrial-centric training and education resources, beyond the current instructor-led model. These could include cloud-based exercises of the current topics, as well as new topics such as navigation for industrial-centric applications.
• There was a great discussion around additional vehicles to drive investment into ROS-I and the capabilities to accelerate the capabilities of interest. This included collaboration with other organizations that fund mid-TRL research as well as other funding vehicles that foster small business engagement/participation.
• Numerous topics focused on moving ROS-I into a product or something tangible that an end-user can develop an application with tools they have existing familiarity with, such as CAD environments and the physics tools that they currently leverage.
• Detailed listing of capabilities that are desired to enable member applications that include real-time process planning, high-res SLAM, dynamic models, error handling and recover at the system level, and many others.

This feedback enables the Consortium to update and prioritize collaborations, training topics, project topics that are put forward and more to ensure the focus is appropriate relative to the needs of the membership and the broader ROS-Industrial community.

The remainder of the day focused on member presentations that shared recent developments and contributions. Lou Amadio of Microsoft kicked it off with a presentation on recent developments relative to ROS and Windows, Azure, and Visual Studio. Arnie Kravitz, CTO of the ARM Institute, discussed how ROS and ROS-I play a key role in the ARM Institute’s vision in fostering advanced robot capability from the research domain, to be ready for production.

YJ Lim, Sr. Technical Product Manager of MathWorks, shared what the latest developments by MathWorks have meant for developing functional safety in ROS-enabled systems. NOV’s Justin Kinney, Technical Manager Robotics and Mechatronics, shared the end-user ROS journey, from “What is a ROS” to their own functional full-scale demonstration on a test bed, a powerful story that lays out what is possible by leveraging the resources available in the community and working with the other members to further your vision.

Wrapping up the day, John Wen of Rensselaer Polytechnic Institute, and John Wason of Wason Technologies, shared their collaborative ARM Institute project, on Robot Raconteur, which is a middleware that enables easy integration of numerous types of hardware components, which now has interoperability with ROS, and how they are seeking to progress the work to enable richer industrial applications. Eugen Solowjow, Research Scientist at Siemens Corporate Technology, shared recent developments by Siemens that further the tighter connection between industry standard technology, that has the safety and reliability that is expected, but offers both interoperability with ROS-enabled components, as well as the latest AI on the edge capability that is currently emerging for advanced capabilities.

Moving Forward

It was a full two days, and though there remains uncertainty relative to how many in person or physical development we can do, the benefit of collaborative software development, and as evidenced in all the activity in the open source community, though physical testing and developments may have slowed, show there is a lot of work that can be done, even during a challenging time. We hope that those who attended, both in person or remote, also feel as motivated to see how we can further ROS-I into the project and product that we feel it can be. We look forward to stepping up to the challenges put forth by our membership and partners, and hope you also look forward to joining in that journey.

ROS-Industrial Training hosted by Americas Member Glidewell Laboratories

SwRI ROS-I trainers, Levi Armstrong & Jorge Nicho, went on the road for a Glidewell Laboratories hosted session of ROS-Industrial training at Irvine, California offices during the days of February 12th to 14th. This training featured basic and advanced tracks where the basic teaches the fundamentals of ROS and builds up to a motion planning application with a robotic arm.

The advanced track focuses on making use of the powerful tools in the Point Cloud Library in order to build advanced perception applications whose result can then be used to command robot arms to act based on the sensed information.

We would like to thank all those that came out to the training and of course, our hosts Glidewell Laboratories, not just for the space, but for also the use of their lab equipment, which included some different configurations than what have been commonly used for lab exerices.

Keep an eye out on the events listing for the next ROS-I training slated for summer 2020. We look forward to offering new content, with focus on ROS2 in the coming months!

ROS Bootcamp 2020 - a learning initiative in collaboration with Singapore Polytechnic

This year’s edition of the ROS Bootcamp was held from the 16th to 20th March, in collaboration with Singapore Polytechnic. The bootcamp stretched over a 5-day intensive hands-on experience on topics such as perception and autonomous navigation with the turtlebot3.

Timelapse of the students coming to class!

The students were first introduced to the basics of linux commands and writing a ROS subscriber and publisher. Students were then tasked to write up their own scripts to incorporate the use of open-sourced ROS packages such as AR Tracking & the ROS Navigation Stack. They had also learnt the importance of testing their work in simulation, using powerful tools on ROS such as RViz & Gazebo, before trying the actual hardware.

On the last day of the bootcamp, a mini navigation competition took place to see who’s turtlebot would be able to navigate through a maze autonomously using what they have learnt.

A successful run of the student’s work!

 The ROS-Industrial team is happy to see a handful of very enthusiastic and strong-willed future engineers growing interest more in the development of robotic applications! We look forward to next year’s edition!

This was ROS-Industrial Conference #RICEU2019 (Day 3)

ROS-Industrial Conference 2019

December 10-12, 2019

Seven years after the very first public event entirely devoted to discussing the potential benefits of a shared, open-source framework for industrial robotics and automation, Fraunhofer IPA hosted the 7th edition of the ROS-Industrial Conference on December 10 to 12, 2019 (all slides and videos are linked under the program) .

This is the third instalment of a series of three consecutive blog posts, presenting content and discussions according to the event days including the follwing sessions:

1. Day 1 with “EU ROS Updates” (watch all talks in this YouTube playlist of Day 1)

2. Day 2 with “Software and System Integration” & “Robotics meets IT” (watch all talks in this YouTube Playlist of Day 2)

3. Day 3 with “Hardware and Application highlights” & “Platforms and Community” (watch all talks in this YouTube Playlist of Day 3)

Day 3: Hardware and Application Highlights

On the third and last conference day, speakers came from Universal Robot, KUKA, and Pilz just to name a few examples. The opening presentation held Paul Evans talking about ROS-Industrial North Americas updates and highlights from Southwest Research Institute (SwRI). He summed up ROS-Industrial activities including membership growth, ROS2 demo on booth at Automate 2019, meetings, trainings (focusing on ROS2), initiatives, community engagement… Some technical highlights are for example the Scan-N-Plan tools that enable real-time robot trajectory planning from 3D scan data. Within the project A5, robotic applications for the aerospace industry were developed, e.g. collaborative and adaptive solutions.

Aerospace manufacturing was also the key domain Rik Tonnaer from the research center “Smart Advanced Manufacturing XL“, cooperating with TU Delft, talked about. This industry is challenging for automation solutions because of the variety of processes, which almost all require human dexterity, craftsmanship, and adaptation to variations. In addition, there are large part sizes, similar but not identical products and processes and a long legacy of approved and certified processes. Using the example of a drilling process, Tonnaer demonstrated how an automation solution can be developed despite the challenges mentioned. The research center aims at deploying the first drilling systems beginning of 2021. Technology transfer to other industries is also on the agenda. ROS is their platform of choice to maximize reuse, collaboration, and separation info functional components.

Paul Evans (Southwest Research Institute / ROS-Industrial North America) presenting ROS-Industrial North America updates & SwRI application highlights

The third contribution came from robot manufacturer KUKA, a new prominent contributor to ROS. Thomas Ruehr started his presentation explaining how to interface KUKA robots with ROS via KUKA Robot Controller and KUKA System Software (KSS) as well as via KUKA Sunrise/RoboticsAPI. A product specifically benefiting from ROS is the navigation for the mobile manipulation platform KMR iiwa. Apart from that, there are already many KUKA robots running on ROS. The company is working on a more canonical offering with respect to a catalog of URDF models and official drivers for KSS, sunrise robots and the mobile platform. KUKA is actively asking the community for KUKA specific needs to help strengthen the company’s engagement in open-source software.

Another prominent robot manufacturer is heavily working on ROS support: Universal Robots. Anders Billesø Beck presented the new UR ROS driver addressing pre-existing hurdles like a clouded landscape with more than 200 variants of a ROS driver for UR and instability towards API changes. The new driver launched in October 2019 is the result of a Focused Technical Project with FZI Research Center for Information Technology that was funded by the EU project ROSIN (see presentation on day 1 for more information). Main advantages of the new driver are its ease of use and better performance and stability. The driver will remain open source and rely on future community contributions. It offers a stable control interface, teach-pendant integration, factory calibration in ROS, safety compliance speed scaling, eSeries tool communication and full safety compliance. UR is now working towards industrial-grade performance and stability.

In addition to the first lecture by Paul Evans, Erik Unemyr presented activities and application highlights from the ROS-Industrial Asia Pacific consortium managed by the Advanced Remanufacturing and Technology Centre (ARTC) of A*STAR (Agency for Science, Technology and Research) in Singapore. As is the case for Europe and North Americas, the Asia Pacific Consortium continues to support members, provide ROS trainings and other successful ROS events on a regular basis, and plans to offer ROS2 based trainings going forward. Unemyr covered a few of ARTC’s core robotics technology focus areas, including the mission to lower the technology barrier for adopting robotics for various applications. One way to achieve this is using a model-based teaching approach of robots using 3D computer vision. Another concept is using Augmented Reality teaching to enable simplified robotic use cases. The second highlight topic relate to providing more flexibility in automating high-mix applications that require toolpath generation. Unemyr also highlighted a key Singapore-sponsored project currently in development intended for large scale interoperable robotic deployments using ROS-enabled robots, called the Robotics Middleware Framework (RMF).

Erik Unemyr (Advanced Remanufacturing and Technology Centre (ARTC)) presenting ROS-Industrial Asia Pacific updates & ARTC application highlights

Stefan Doerr from Fraunhofer IPA presented the topic “Towards Plug & Play solutions for autonomous navigation of mobile robots and AGV”. The versatile navigation stack from his team does not need any markers or additional infrastructure, is mostly platform-, hardware- and sensor-independent and based on ROS. With its three core components long-term SLAM, zone-based global route planning and dynamic, local path and trajectory optimization the software has already been deployed on a variety of systems starting from vacuum cleaners and ending at autonomously driving trucks. Its plug & play technologies are the key to the widespread of mobile robots and AGVs in industry. The latest deployment of the software was successfully realized for Smart Transport Robots at BMW production plants. Challenges there were sparse sensor data, a highly variable environment with hardly any static structure, interaction with forklifts, trugger trains etc., large environments of more than 100,000 square meters and limited maneuverability. Ongoing research work at IPA is about machine learning for the navigation stack.

The company Pilz, well-known for its safety technologies, talked about its safety certified ROS-native industrial manipulator PRBT 6. Manuel Schoen presented the different modules for service robotics applications and how they benefit from ROS. The robot does not need any proprietary controller or teach-pendant but can be used directly with ROS. The safety controller for safety functionality is partially implemented in ROS as well. Since the robot itself in combination with the safety controller is certifiable under ISO 10218-1, the system integrator can focus on the application. Applications can be realized with the robot running automatically, running with manually reduced speed, or with manually high speed. In 2019, Pilz also presented results of a Focused Technical Project funded by the ROSIN project. The FTP implemented a trajectory generator with a MoveIt-interface for easy planning and execution of Cartesian standard-paths (LIN, PTP, CIRC) according to industrial requirements. In addition, the blending of multiple sequential motion commands was realized.

ROS is also entering the retail domain. This is done within the framework of the “REFILLs” project (Robotics Enabling Fully-Integrated Logistics Lines for Supermarkets). Jonathan Cacace from PRISMA Lab of University of Naples "Federico II" talked about the project’s aim: Novel robotics systems in close and smart collaboration with humans will allow addressing the main in-store logistics processes for retail stores, leading to a smarter self-refilling in supermarkets. Challenges like the changing environment, variety of objects, or tight work spaces have to be considered. To automatize for example the depalletizing of heterogeneous and cluttered objects, technologies like image processing, perception, grasping, and manipulation must be realized. Therefore, the ROS-based motion planner MoveIt plays a key role in the software development.

Manuel Schön (Pilz GmbH & Co. KG) presenting Safety Certified ROS-native Industrial Manipulator

Day 3: Platforms and Community

Coming to the end of the conference, Penny Scully from Jungle gave an overview about the offerings of the company. It aims at connecting industrial partners and  robotics developers who offer “robofacturing”, a software defined manufacturing process of using robotics and AI technologies within the production of goods. Flexibility, time to market, and autonomy are main criteria of their offering. Jungle is developing an online platform to resell resolved challenges to a wider portfolio of industrial partners, like an app store for robotics, to extend their developers reach into the manufacturing industry. Processes like quality inspection, pressing, and trimming have already been implemented. Labelling, battery module assembly, and (un-)packing are in the future scope of the robofacturing process.

Benjamin Goldschmidt of Silexica highlighted its new SLX Analytics product. The German startup called attention to sporadic errors that occur in nowaday's complex software systems (e.g. an Autonomous Driving System). Because the complexity of such systems is so high, the developers are facing the challenge of understanding the overall system behaviour and knowing, in case of an error, which software component failed and why. With SLX Analytics, Silexica addresses these needs by providing automated testing of system metrics, enabling customers to uncover system-level defects early on and thereby reducing the risk of dangerous defects in the product. The first product release took place in January 2020. 

The conference concluded with a contribution of the Eclipse Foundation summarizing the importance of open-source communities for robotics. Philippe Krief presented two company stories from Bosch and MQTT. Their lessons learned: Communities do not only help being competitive against “bigger fishes” but they are also a great vehicle to gain visibility and recognition. That is why Eclipse is launching an Industrial Robotics Working Group collaborating with different ROS-related research projects like RobMoSys, ROSIN, or SeRoNet. Krief’s motto “Open-Source is a journey, not a destination” does not only  describe the work of the Eclipse Foundation very well, but is a nice closing word for the ROS-Industrial Conference 2019.

All three consortia, their members and managing institutes like Fraunhofer IPA are happy and proud to be part of this journey. Let us continue it for example with #RICEU2020! For some more impressions of the whole event please watch the event video.

This was ROS-Industrial Conference #RICEU2019 (Day 2)

ROS-Industrial Conference 2019

December 10-12, 2019

Seven years after the very first public event entirely devoted to discussing the potential benefits of a shared, open-source framework for industrial robotics and automation, Fraunhofer IPA hosted the 7th edition of the ROS-Industrial Conference on December 10 to 12, 2019.

This is the second instalment of a series of three consecutive blog posts, presenting content and discussions according to the event days including the follwing sessions:

1. Day 1 with “EU ROS Updates” (watch all talks in this YouTube playlist of Day 1)

2. Day 2 with “Software and System Integration” & “Robotics meets IT” (YouTube Playlist of Day 2)

3. Day 3 with “Hardware and Application highlights” & “Platforms and Community”

Day 2 Part 1: Software and System Integration

The second day of the ROS-Industrial Conference featured speakers from several well-known companies that have been using ROS for a long time or started using it recently and are developing it further together with the community. While some companies already spoke at the conference in earlier years, there were also speakers from new contributors. This shows that the interest in ROS from the industrial side is still unbroken and increasing.

Matt Hansen from Intel opened the second day. He presented the ROS developments of his company, which mainly include a Robot Development Kit (RDK) and navigation software. Intel especially focuses on the adoption of ROS2. The RDK supports the development and implementation of software components for mapping and planning, machine vision (point cloud generation, object detection, face and gesture detection and more) and intelligent handling or grasp detection. The comprehensive project navigation2 pursues the goals of providing a customizable (thanks to behavior trees), modular and extensible software. To ensure quality and maintainability, an automated system test was created. It uses Gazebo and a Turtlebot 3 model to test localization, transition into the ‘active’ lifecycle state and navigation.

Matt Hansen, Intel, giving a overview on ROS2 Robot Dev Kit and Navigation2

ROS2 tracing was the topic of Ingo Lütkebohle from Bosch. Tracing is important because there are currently various problems in performance analysis and execution monitoring, for example: How long does my system take to react? How much resource is it consuming? Factors like distributed systems or repetitive periodic processing complicate performance analyses. Lütkebohle explained, which kind of information is recordable, which tracepoints exist in ROS2, and how static and dynamic tracing differ. In the end, he gave an example of a tracing installation and implementation process.

ROS as the basis for a framework with which industrial robots do no longer need to be programmed but can be intuitively instructed: Pablo Quilez from the startup drag&bot, whose first research activities took place at Fraunhofer IPA, spoke about this framework and first industrial projects. With the help of a graphical user interface, the user creates a robot program via drag&drop of function blocks. drag&bot is manufacturer independent, offers the same user interface for different robots, and does not require any expert knowledge in robotics. It is an open platform that third parties can extend, e.g. with ROS packages

Arne Roennau and his colleagues from FZI Research Center for Information Technology developed ROS based Cartesian controllers that enable motion, force, and compliance control for robotic manipulators. Cartesian control in task space is necessary for closed loop force control, direct teaching, contact-rich manipulation, manual guidance. That is why FZI worked on active Cartesian compliance using three controller modules and implemented them for a car door sealing assembly, a satellite assembly and many other applications. The goals here were to give the robot error correcting contact skills for autonomous execution that are transferable to different robots.

More contributions to ROS presented Steve Peters from Open Robotics, an institution that has been supporting and evolving ROS and Gazebo since many years. In this context, Open Robotics offers several developer tools to facilitate the use and integration of open source software and supports upcoming ROS releases like Melodic (May 2023) and Noetic (2025), the latter probably being the last ROS1 distribution. Together with other companies in the ROS2 Technical Steering Committee, Open Robotics helps managing the roadmap, contributes development efforts and sets developer policies. Finally yet importantly, interfacing ROS and Gazebo has been on the agenda of Open Robotics for quite some years.

Gazebo simulations were also the main topic in the presentation of Musa Morena Marcusso Manhães. She works for Bosch where she develops a Python library for scripting and rapid-prototyping of Gazebo simulations. So far, there are several application-dependent difficulties with respect to simulations (e.g. generation variations of worlds and models, scripting of world layouts and event-based actions). Marcusso’s approach is the procedural generation, a technique from gaming development. It enables rapid-prototyping of simulation scenarios and abstractions to simulation entities, allows scripting of Gazebo simulations, extends templating options for robot descriptions and improves the conversion between URDF and SDF.

Musa Morena Marcusso Manhães, BOSCH, presenting A Python library for scripting and rapid-prototyping of simulated Gazebo models and worlds called pcg_gazebo_pkgs

Nadia Hammoudeh Garcia from Fraunhofer IPA highlighted the potentials of the combination of model-driven engineering (MDE) and ROS. Among others, ROS developers can benefit from the definition of common design patterns and specifications, model checker techniques and automated code generators.Since there are already about 4000 hand-written open-source ROS packages, Hammoudeh not only develops tools to generate code but also to automatically extract ROS models from the existing code. She released an eclipse-based tooling with a graphical interface and model editors as well as domain-specific languages to the models. All her contributions are also part of the German research project “Service Robotic Network” (see presentation from day 1 for more information).

Not only MDE can facilitate the deployment of ROS but also the tools from MathWorks, MATLAB and Simulink. Shashank Sharma presented how this works, which advantages the tools offer, and how they address common challenges of autonomous systems, e.g. multi-domain expertise, complexity, and performance evaluation. He also discussed the key advantages of Model-Based Design and how it can be applied to the robotics and autonomous systems. Therefore, MATLAB helps analyzing and visualizing ROS data and prototype algorithms. A Simulink model allows automated code (C and C++) and ROS node generation as well as prototyping new algorithms through the ROS interface. Finally, the tools from the company allow incorporate ROS in Model-Based Design workflows

Day 2 Part 2: Robotics meets IT

Robotics and IT are becoming more and more interlinked – this was already evident at the ROS-Industrial Conference 2018, when Roger Barga from Amazon Web Services (AWS) was invited to present a new service for robot applications “Amazon Robomaker“. Now, one year later, he presented the status of the service. It offers comprehensive functionalities for each of the three stages “design and develop“ (storage, logging, metrics, image and video recognition etc.), “test and verify“ (simulation tools for thousands of concurrent simulations and model training), and “deploy and update“ (control and multiple deployments, manage robots across multiple brands, deployment over-the-air). In addition, AWS contributed source code to ROS2 core, along with tools for ROS2 to improve functionality and code quality. The company will continue these efforts.

Roger Barga, AWS Amazon web services, giving a keynote on The Robotic Edge

Andrei Kholodnyi from Wind River addressed the problem that there is an increasing amount of robots using ROS but a decreasing amount of embedded software engineers. He stated that people do not want to program bit and bytes anymore. That is why Wind River provides the downloadable SDK VxWorks for non-commercial usage. ROS2 is built on top of the VxWorks SDK and developers can deploy and run it on ARM and Intel.

Canonical, as the company who publishes Ubuntu, gives developers support for simple, secure, and scalable robotic deployment in the field of ROS. Rhys Davies, a product manager for Canonical's robotics initiatives presented tools like Snaps, containerized software packages for all Linux distributions, the company’s efforts for continued support for Python 2, and Extended Security Maintenance (EMS) for ROS, to maintain a robot past its usual lifetime. With these offerings, Canonical aims at supporting users to move to ROS2 and to facilitate the transition of robot systems already in practice.

The company eProsima, represented by Jaime Martin Losa, offers the open-source DDS for ROS2 “eProsima Fast RTPS”. This DDS offers real-time behavior (static allocations, non-blocking calls, sync and async publishing), intra-process communication and discovery server. Martin Losa detailed performance features using the example of an iRobot framework and gave numbers with respect to benchmarking criteria like latency and throughput. More features are currently being developed with the help of project funding from ROSIN (see presentation from day 1 for more details).

Besides Amazon, Microsoft was one of the most prominent companies presenting at ROS-Industrial Conference. Gunter Logemann talked about ROS applications with Visual Studio Code and Azure. Since June 2018, the IT giant has been contributing to the ROS development and since then, 279 ROS packages have been enabled on Windows. New features are for example the Azure IoT_Hub connector, Azure Kinect ROS Node, and Windows ML node. The developer environment Visual Studio Code offers several ROS extensions like automatic workspace activation, starting, stopping, and viewing the ROS system status, and automatically discover build tasks. ROS2 support is also included.

Both offensive and defensive security aspects were presented by Endika Gil Uriarte and Victor Mayoral from Alias Robotics. They explained that the network and transport layers are the most vulnerable points in an OS and not ROS, ROS2 or the application itself. For them, security is a process and cannot be “finished” with a certain technology. The company offers a variety of security tools and has profound knowledge concerning vendors as well as ROS and ROS2 thanks to its public robot vulnerability database and to a robot security survey done in conjunction with Joanneum Research Robotics. One of the solutions to enhance security is the company’s toolbox for robot security “alurity”. Soon, there will also be RIS, the Robot Immune System for UR robots offering defensive robot security.

audience at ROS-Industrial Conference 2019 at Fraunhofer IPA in Stuttgart, gErmany

Analytics for autonomous driving and large-scale sensor data processing was the topic of Jan Wiegelmann working for Autovia. The challenge here are the petabytes of data that are generated daily both from simulation and real-world sensor data. The Autovia Analytics Platform enables large-scale data processing for these use cases. Autovia IO is a data access layer that enables analytics apps to read ROS bags from local and cloud storage. It works for diverse platforms and does not require a ROS installation. Additionally, Autovia FS is a virtual file system with which all applications can read ROS bags from local and cloud storage. Autovia IDE completes the offer: It is an analytic toolbox running as managed service and addresses R&D engineers in automotive and aviation industry.

Christoph Hellmann Santos closed the second conference day and presented the EU initiative agROBOfood, which aims at bringing ROS to the agro-food sector. This domain will have to cope with major changes like ageing population and less working population, climate change, less productiveness of ecological food… New methods like vertical or urban farming and the key topic sustainability are on the rise. Already now, we see quite many robot platforms on the fields that might help mastering these changes. In this context, agROBOfood has three offers: Firstly, ROS will be pushed to use it in agricultural applications. Main topics are functional safety for mobile agricultural robots, software architecture and development and organizational structures like working sponsors and adding groups. Second, the project addresses SMEs with an open call for funding ROS software developments. Third, service providers can join the agROBOfood network and benefit from access to technology maps, market knowledge, and standardization activities. Finally, they can get in contact with people from all areas of agrofood robotics.

In the evening of this conference day, the social dinner took place in the restaurant “Leonhardts” in direct neighborhood to Stuttgart’s iconic TV tower. For some more impressions of the whole event please watch the event video.

This was ROS-Industrial Conference #RICEU2019 (Day 1)

ROS-Industrial Conference 2019

December 10-12, 2019

Seven years after the very first public event entirely devoted to discussing the potential benefits of a shared, open-source framework for industrial robotics and automation, Fraunhofer IPA hosted the 7th edition of the ROS-Industrial Conference on December 10 to 12, 2019.

This is the first instalment of a series of three consecutive blog posts, presenting content and discussions according to the event days including the follwing sessions:

1. Day 1 with “EU ROS Updates” (watch all talks of Day 1 in this YouTube playlist)

2. Day 2 with “Software and System Integration” & “Robotics meets IT”

3. Day 3 with “Hardware and Application highlights” & “Platforms and Community”

Day 1: Updates from ongoing activities in the EU

Already for the seventh time, Fraunhofer IPA invited to a big ROS-Industrial-Event. Meanwhile, the conference – for several years now scheduled shortly before the Christmas break – has established itself as the European event on the topic of ROS, where developers, researchers, companies and all interested parties can learn about the current status of the free Robot Operating System ROS and have the chance to network and exchange information. Last December, some 150 participants came to Stuttgart and benefited from around 40 presentations from academia and industry.

The first day was dedicated to "EU Updates". There is a lot to report here because extensive national and EU funding is flowing into research projects that directly serve further developments of ROS and, in particular, work towards industrial suitability. The projects cooperate closely with the existing community and are strongly networked with each other because they pursue a common goal despite their different approaches: the creation of an EU Digital Industrial Platform for Robotics.

The lectures started with an opening by Thilo Zimmermann, Manager of the ROS Industrial Consortium Europe, as well as by Christoph Hellmann Santos, Group Leader at Fraunhofer IPA, and Werner Kraus, Head of the department Robot and Assistive Systems, also at Fraunhofer IPA. Kraus gave an overview of the current robotics market and presented growing market figures from the statistics of the "International Federation of Robotics" on both industrial and service robotics. The latter in particular often relies on ROS, because with utilizing existing open source software like ROS, the first prototypes can be created much quicker without having to reinvent the wheel and expend too many own resources.

Werner Kraus, head of department of robotics and assistive systems at Fraunhofer IPA

Updates on EU project ROSIN

The largest funding project for ROS is currently ROSIN, which will come to an end in December 2020. First, Carlos Hernandez Corbato from TU Delft gave an introduction to the project, which has two objectives: 1) assuring the availability of high-quality robot software tools and components, and 2) creating a sufficiently large European user- and developer base. To this end, the project is particularly active in three areas: quality assurance of software development and components, continuing education for students and professionals, and financial support for ROS components in the context of "focused technical projects" (FTPs). In more than 50 FTPs, more than three million Euros of funding have already been granted.

In order to improve software quality, ROSIN continues to develop technologies and methods like continuous integration, model-driven development and model-in-the-loop, automated test generation and code scanning. In this context, Andrzej Wasowski presented the topic "[Reactive] Programming with [Rx]ROS". Reactive programming raises the abstraction level comparing to the standard ROS API, by making the flow of information explicitly stand out in the source code.

As far as further education measures are concerned, 530 students and 436 professionals have already been trained in ROS-I schools and ROS-I academies, as Stephan Kallweit from FH Aachen reported. The project goal of 1000 participants has thus already been almost reached and ROSIN has been very successful as a multiplier of ROS knowledge. In addition, in "educational projects" (EPs), formats such as web-based interactive tutorials, ROS training centers or applied training events by 3rd parties are financially supported. The last project year is to be used in particular to create ROS2 training content.

Carlos Hernandez Corbato, professor at TU Delft and coordinator of EU project ROSIN

Insights in „Focused Technical Projects“

As mentioned above, ROSIN has already promoted more than 50 FTPs addressing specific industry needs. Possible FTP contents were algorithms, e.g. a SLAM algorithm, application templates, improvement of existing components, process-related work, e.g. code security audits, improvement of documentation or the integration with other software frameworks. Five FTPs presented their work at the conference:

• Rafael Arrais from the research institute INESCTEC talked about „ROBIN: The ROS-CODESYS Bridge“. It focusses on developing and releasing a bidirectional, reliable and structured communication bridge between ROS and CODESYS, a softPLC that can run on embedded devices and that supports a variety of fieldbuses, and even OPC UA. The developed software will allow the parametrization of ROS modules through IEC61131-3 programming languages and also streamline the interoperability between ROS and robotic hardware or automation equipment, fully empowering the Industry 4.0 paradigm of Plug’n’Produce.

• The Norwegian company PPM, represented by Trygve Thommesen, developed BLACKBOX, an automated trigger-based reporting, data recording and playback unit, collecting data from robot and manufacturing systems. It takes error reporting and recovery of industrial robot systems to a new level, by developing and utilizing the innovative ROS based framework. The framework is built upon components from the project partner’s previous research and existing ROS modules.

• The Spanish CATEC worked on a robust and reliable GPS-free localization algorithm for aerial robots applied to industrial applications. As Paloma Carrasco told, it focuses on safety and computational efficiency. This ROS library helps to foster the development of drone-based solutions for industrial inspections.

• Olivier Michel, CEO at Cyberbotics, presented cross-platform ROS simulation for mobile manipulators. This FTP aims at developing a simulation framework for training pilots of robots used for intervention in case of a nuclear incident. These robots include industrial arms and will be controlled using ROS. The project will contribute to the ROS-Industrial community with a new open-source, cross-platform, high-performance simulator compatible with ROS for industrial robots.

• Finally, Luca Muratore from IIT in Italy talked about ROS End-Effector. It provides a ROS-based set of standard interfaces to command robotics end-effectors in an agnostic fashion. Nowadays, end-effectors are controlled using customized software wrappers and specific frameworks: ROS End-Effectors aims to design and implement a universal ROS package capable to communicate with different end-effectors. The project will be ROS2 compatible and will work both in simulation (Gazebo environment) and in the real hardware.

The session ended with a lecture by Olivier Stasse, LAAS-CNRS. He reported on the use of ROS in robots that are to be used for partial automation in aircraft construction. This domain requires lightweight, safe, mobile, and versatile manufacturing cells that enable human-robot collaboration. Stasse is developing this within the framework of the “Rob4Fam” lab (Robots For the Future of Aircraft Manufacturing). Implemented technologies are real-time/interactive planning, torque control, SLAM algorithm and balance so that robots could climb stairs, screw, or drill. All will be ROS controlled. Also, ROS facilitates the integration between lab and industry.

More projects boosting ROS

Besides the research project ROSIN, there are other national or European research efforts building on or improving ROS components. One of them is the EU project RobMoSys, represented at the conference by Dennis Stampfer from Ulm University of Applied Science. It aims at developing composable models and software for robotic systems and uses a model-driven engineering approach as key enabler for complex software and system integration and for integrating existing technologies. A key component is MROS: Metracontrol for ROS2 systems, a RobMoSys-integrated technical project. Carlos Hernandez Corbato from TU Delft talked about this project that enables models@runtime to drive and architectural adaption for reliable autonomy.

On the national level, there is the German project “Service Robotic Network” (SeRoNet) that aims at building a community-driven platform for a more efficient development of service robots. With the technology behind, the design, development, and deployment of service robots in a variety of areas, from logistics, care, and healthcare to assembly support in manufacturing operations shall be made much easier than today. Through the online platform “robot.one“, users, system integrators and component manufacturers of service robot solutions will be able to collaborate efficiently and jointly support solutions from requirements analysis to deployment. In 2020, two open calls offer funding opportunities: The first call, open until late March, adresses companies that aim to make their software or hardware components compatible to the SeRoNet platform. The second call, in summer 2020, addresses end users and system integrators to propose and implement novel service robot solutions using SeRoNet technology.

Last but not least there is the project RoboPORT. Within this project, an online development platform for service robotics with an extensive library of open source robotics hardware is being created. New collaborative processes such as open innovation, crowd engineering and similar methods will be mapped on the platform to support a continuous and distributed development process. Also, the project supports ideation processes, hackathons and makeathons and offers a network of highly motivated domain experts that help realizing specific project ideas.

ROS is leaving shop floors

The EU activities around ROS are not only focused on the use of free software in production environments. The presentation by Gonzalo Casas from ETH Zurich showed how ROS can be used for architecture and digital fabrication in the construction industry. For example, a robot can use the path planning software MoveIt to build a wall with bricks.

Gonzalo Casas of ETH Zurich presented how ROS for architecture and digital fabrication in the construction industry

Finally, it is the agricultural and food sector where robotic applications are increasingly in demand, including those using ROS. Fraunhofer IPA is also very active in this area, for example in the lead project at Fraunhofer "Cognitive Agriculture" (COGNAC) for sustainable and at the same time profitable agriculture. At EU level, the agROBOfood project is currently gaining momentum with the aim of building the European ecosystem for the effective adoption of robotics technologies in the European agrifood sector. These agricultural activities are not the first ones related to ROS. As Andreas Linz from Hochschule Osnabrück presented, there were projects like BoniRob and elwoRob that already built on ROS. The advantages are manifold: ROS is modular, expandable, reusable, has a huge and active Community and offers over 3000 nodes for nearly all use cases and a large collection of helpful tools, i.e. Rviz. Also, many companies provide ROS compatible drivers and software tools for their hardware, and the integration with other open source libraries like PCL (Point Cloud Library), OpenCV or Gazebo is possible.

The BoniRob system is a modular Robot Platform for agricultural applications using an app concept. Depending on the task, different hardware and software modules can be installed. So far, there is a phenotyping app and a soil2data app measuring the nutrients in the soil. Furthermore, there is the robot elWobot for maintenance in orchards and vineyards and there are robots for education and demonstration. For all these, ROS plays a crucial role.

The first conference day ended with a welcome reception and several ROS demonstrators at the Stuttgart research campus “ARENA2036” for future car manufacturing.

For some more impressions of the whole event please watch the event video.

Hands on with FRAMOS D435e camera featuring Intel® RealSense™ technology for industrial robotics

It has been nearly 10 years since the release of the XBOX Kinect camera, and things have certainly changed. Gone are the days of using reverse-engineered software drivers or soldering on USB cables. We have arrived in a time of 3D perception plenty. Multiple vendor-supported 3D camera options exist utilizing different depth camera technologies, to such an extent that picking one can often be difficult. The ever-popular 3D camera survey has grown to over 20 cameras, and it is still growing. Nevertheless, the ROS-I team is committed to testing as many of these sensors as possible and putting them through their paces.

One option that I have recently become fond of is the Intel RealSense. In my mind it is a sort of industry standard. Released and actively supported by ROS-I consortium member Intel, the RealSense is inexpensive, available off the shelf, and works reliably across a wide variety of operating conditions. Combine that with a stellar ROS (and ROS 2) driver, and you have a winner. The convenience of being able to just plug in the camera, bring up RealSense-viewer, and then tune or debug the camera cannot be overstated. It is a camera for the robot masses. However, it does have a problem. The Intel RealSense is USB 3.

For industrial automation tasks USB cameras have long been an issue. We often set up robotic wrist cameras at Southwest Research Institute. With a properly calibrated depth camera on a robot wrist we are able to reconstruct large parts and automate complex industrial tasks. USB complicates this setup. Flaky connections can plague a setup like this, leading developers to ritualistically disconnect and reconnect USB cables whenever something goes wrong. Further, cable runs quickly exceed the length that USB can handle leading to the use of sometimes-suspect USB-ethernet extenders. If only there was a camera option that had the software backing of the Intel RealSense with the industrial readiness of POE. Meet the FRAMOS Industrial Depth Camera D435e. Based on the Intel RealSense technology, FRAMOS has packaged this D435e 3D GigE camera in an IP66 enclosure and replaced the USB-C connector with an industrial-ready M12 GigE connection. While it certainly isn’t going to replace the Intel RealSense as the camera for the robot masses, it might be the camera for the industrial robot masses.

The ROS-I team recently got the chance to work with the camera hands-on, deploying it on an industrial scan-and-plan project. The setup for the FRAMOS RealSense is straightforward if a little quirky. On Linux one must simply install the FRAMOS CameraSuite Software and then install their custom version of the librealsense SDK. Then upon configuring the network settings for the camera, it behaves just like the USB RealSense. ROS doesn’t know the difference, and the RealSense-viewer behaves in the same way. However, using it has not been without its difficulties. ROS support for this camera was released October 29, 2019. We began integrating it on our system on November 4, 2019. As such, there were some problems. The software suite only allowed a limited set of the parameters to be set and required a custom version of libglf3w. This caused an issue where the camera mysteriously stopped working after an unrelated package was updated, and the only apparent solution was to reinstall the software suite. However, a few weeks later a new version of the software was released that fixed both issues.

Overall, using this camera has been troublesome but to an ever-decreasing degree. To some extent, the convenience and maturity of the Intel RealSense makes it easy to complain about any small thing that goes wrong on new hardware, but I personally have high hopes for the FRAMOS Realsense. When debugging, their software support was quick to respond, and the software seems to be in a state of continued improvement. We still run into occasional crashes, but similar issues plagued the Intel RealSense a mere year ago that have since been resolved. The necessity of the custom version of librealsense SDK is its biggest oddity, but with continued collaboration between FRAMOS and Intel this may be resolved someday.

Ultimately, when I think about the ideal camera for a ROS-I system, I imagine a POE camera with easy to use software, rock-solid vendor support, and an active ROS community. While there will always be cameras that excel at one application or another due to depth of field, resolution, or sensing technology, for medium to large scan and plan applications, the FRAMOS RealSense is well on its way to achieve that goal. Others will undoubtedly join the market, but regardless it is an exciting time for 3D perception in industrial applications!

Observations from CRAV.AI 2019 Conference on Collaborative Robotics and AI

I had the opportunity to attend CRAV.ai 2019 in San Jose CA and present SwRI's work on collaborative robots; this work described how we built a sophisticated collaborative robotics application where various tasks were taught to a robot arm by way of human demonstration. One highlight of this application is that it leveraged low-cost sensors and open-source software frameworks such as ROS, MoveIt, AruCo, Ceres, etc.

The presentation was very well received, and we hope it'll lead to opportunities to further that work and continue to do research in that space.

Furthermore, there were a great deal of interesting presentations that explored innovative ways to make robots collaborate and empower human workers in the various industries where the inherent adaptability and dexterity of humans remains irreplaceable.

Human Augmentation of Robots for the Automation Age (SARCOS):

• Described the following challenges in manufacturing:

• Projected labor shortages in the US and many other industrialized nations in the next decade will have a negative impact on the economy in the trillions of dollars.

• Occupational injuries incur an annual cost of \$100 billion in the US

• Complete automation isn’t the solution to the labor shortage challenges ahead

  • Humans will continue to play a role in manufacturing due to the unstructured and unpredictable nature of very many tasks for which automation falls off short.

  • SARCOS showed its GUARDIAN XO powered exoskeleton which a human can operate in unstructured environments in order to carry out a diverse set of tasks.

  • The suit provides the human worker with added strength and endurance, reduces the risk of injuries and enhances productivity.

How Robot Motion Planning is setting Robots Free (Collaborative Robotics):

• Interesting approach to multi robot motion planning using precomputed swept volumes

How AI, Robotics, Vision, and Industry 4.0 Will Revolutionize manufacturing (Canon USA)

• Great summary of past technological revolutions

  • Mechanization, steam power, weaving loom (Industry 1.0)

  • Mass production, assembly line, electrical energy (Industry 2.0)

  • Automation, computers, electronics (Industry 3.0)

• Industry 4.0 forecast

  • Increased efficiency through robot/human collaboration

  • The concept of the Smart Factory (highly digitized and interconnected production)

  • Large scale data anaylysis

• Lights Out Manufacturing

  • Advanced sensing technologies that allows robots and machines to operate in the dark

  • If implemented correctly could maximize efficiency and profitability.

  • Not a common approach in factories but its viable given current technologies.

• The Role of AI

  • Predictive maintenance would eliminate the need for predetermined schedules

  • Through machine learning and data collection, systems could adapt to changes or function with fewer interruptions

  • Increase the remaining useful life of machinery

Volumetric Technologies for Future Sports Experiences (Intel Sports)

• The role of advanced technologies in today’s sports

  • An array of cameras placed around the stadium allows creating a virtual camera view at any desired location

  • Broadcast Enhancements allow creating a narrative or facilitate advertisement

  • Automated virtual camera movement allows following players, follow ball, predict best camera position, etc.

  • Used by various professional tournaments and leagues

• Volumetric video

  • Can create rich, compelling and immersive media experience.

  • Combined with VR headsets would provide a fundamentally new way to experience sports

It was a pleasure to be part of such an interesting event, and we look forward to both contributing and working with others to advance robotics, in particular where collaboration is a key element.

What Went Down at ROSCon 2019

ROS-Industrial representatives from the three global regions attended and presented at ROSCon Macau 2019, held from Oct. 31 to Nov 1.

The booth was supported by all the three consortia from Asia Pacific, Americas and Europe. Some initial changes were the introduction of workshops ahead of the formal agenda, including a very good workshop on “Day 0” on the ecosystem (ROS is getting bigger and bigger, how do we get those people more into an active role?).

This year, the ROS-Industrial team presented the demonstration titled “Robotic Pick & Place with Augmented Reality” which was made to showcase the interoperability of ROS by allowing robots to perform tasks by learning from an operator input using an intuitive augmented reality interface – removing the need for programming of robots.

As far as exhibitors along with ROS-I, it was noticed that there was a noticeable increase in exhibitors (~40) with more companies attending/supporting ROSCon, so this speaks to the growth and evolution of the community and for the event itself.

Obvious recruitment, seeking of ROS-skilled professionals, part of "we are hiring”, even more so than in past years. While another hot topic, or maybe the red ribbon of the event, was "we need more and complete documentation of ROS2."

As such with such full in content events, or how long the coffee breaks, they are never sufficient to meet and greet everybody.

Photo : Ng Wei Kien & Dejanira Araiza at the ROS-Industrial Booth showcasing the demo.

The ROS-I teams were selected and gave three talks:

Levi Armstrong and Chris Lewis presented on “Industrial Manufacturing Automation Leveraging ROS” and Dejanira Araiza-Illan on the topic “PackML2: State Machine Based System Programming, Monitoring and Control in ROS2” with Michael Ripperger presenting on “Flexible Framework for Quantitative Reachability Analysis.”

Photo : Levi Armstrong, ROS-I Americas/SwRI presenting Industrial Manufacturing Automation Leveraging ROS .

There was also one of the major keynote presentations on the Robotics Middleware Framework (RMF) about roadmap to adopt robotic solutions and smart systems in Singapore’s public healthcare sectors.

A brief summary of presentations that stood out to the teams present:

• ROS Real Time Workshop
  • Both hardware and software requirements
    • Real-time kernel, messaging, etc.
  • Lots of effort required to setup/run real-time Linux kernel
  • Security and real-time conflict of interest?
  • Timing and determinism
    • TCP/IP can meet determinism
    • UDP can meet timing
    • Neither meet both
  • Large effort by various companies to benchmark timing, processing speed, memory usage, latency, etc. of ROS2 with various DDS implementations
  • Very important for mobile robotics and autonomous applications
  • Benefits of this effort will likely be available by the time most industrial manipulation applications we work on care about this functionality
• ROS2 on VxWorks
  • ROS2 dependencies installed
  • Run ROS2 on an RTOS
• ROS2 migration of Navigation Stack
  • Re-design of the architecture to leverage ROS2 features
    • Component and life-cycle nodes
  • Run-time definable behavior trees for decision-making in fault scenario
• Reactive programming
  • Paradigm for handling asynchronous data streams
  • RxROS (ROSin funded): https://github.com/rosin-project/rxros
• ROS in Jupyter Notebook
  • Essentially an online IDE that can compile and run code
  • Alternative for industrial training (Python)
  • C++ support?
• High Assurance ROS
  • Static analysis for ROS system
  • Looks at unconnected topics, etc.
  • https://github.com/git-afsantos/haros
• Reactive Jogger
  • UT Austin Robotics Lab
  • Teleoperation robot jogger with singularity avoidance, signal filtering, reactive control
• Cartesian Controllers
  • Cartesian motion, compliance and force controllers
  • Position and velocity control
  • _https://github.com/fzi-forschungszentrum-informatik/cartesian_controllers_
• OMPL constrained planning
  • Requires equality constraint function and gradient
  • Performs search on constraint manifold instead of full configuration space
  • https://ompl.kavrakilab.org/constrainedPlanning.html
• MoveIt Task Constructor
  • Framework for defining tasks with multiple valid success paths
  • _https://github.com/ros-planning/moveit_task_constructor_
• Pilz Industrial Motion Control
  • Deterministic motion planners for industrial move types (MoveL, MoveJ, MoveC)
  • Include blending parameters

Photo : Michael Ripperger, ROS-I Americas/SwRI, presents processes planning framework at the MoveIt Workshop.

Things to take away…

• High level observations:
  • Majority of attendance/presentations related to mobile robots in logistics space and autonomous driving
  • Lots of people care about real-time capability
  • Lots of people diving deep into the RMW and DDS layers to fix bugs and improve performance
  • Lots of people care about scaling robot operations
    • Discussion around how ROS fits into fleet deployment framework
    • System of systems architecture approach
  • Not much about manipulation
    • One half of one day
    • Not a lot of manufacturing-centric content outside of ROS-I

• Interesting Information
  • TRAC_IK has mutliple objectives for driving gradient
    • Speed: default (joint speed?)
    • Distance: minimizes joint distance from seed
      • Should probably be preferred to minimize configuration changes
    • Manipulability
  • BioIK
    • https://github.com/TAMS-Group/bio_ik
  • ROS2 Life-cycle nodes
    • State-machine for nodes
  • ROS2 component nodes
    • Similar to nodelets
    • Can share memory with other component nodes in the same container
  • Use MoveIt planning objects in favor of the MoveGroup
  • Microsoft is releasing HoloLens v2.0 that are “industrial grade”

Thoughts relative to ROS-I...

• We should become more familiar with ros_control capability
  • Trajectory replacement
  • Cartesian and force control capability
  • Create some compelling demos
  • Real-time interaction with external devices
• We need to be more active/supportive of core ROS repositories
• Creation of ROS Calibration GitHub organization
  • We should move our calibration libraries here to make them more discoverable
  • We can still brand them thoroughly as ROS-I developed

Finally, it is clear that ROS2 development, including for navigation, and optimization of the DDS/transport layer and discussion of real-time capabilities are making great progress. However, there are challenges, particularly related to areas that ROS-I seeks to address. Capabilities for manipulation/path planning for manipulators, documentation, and significant progress on ease of use, to enable manufacturing-centric organizations to really jump in. In the interim there is enough tech industry engagement to fill the void, to provide additional tools and offer solutions to meet industry needs. Now is the time to create compelling ROS2 demonstrations/reference applications that drive further end-user engagement. ROSCon always lights a fire of inspiration, now to just set to the work of getting that fire to spread. Looking forward to 2020!

Content provided by Sheila Suppiah, ROS-I AP, Thilo Zimmerman, ROS-I EU, and Michael Ripperger, ROS-I Americas.

Alliance with The Singapore Industrial Automation Association

ROS-Industrial Consortium Asia Pacific signs alliance with Singapore Industrial Automation Association (SIAA)

On the third day of the Industrial Transformation Asia Pacific (ITAP) 2019, a Memorandum of Understanding was signed between the Singapore Industrial Automation Association (SIAA) , ROS-Industrial Consortium Asia Pacific and the National Robotics R&D Programme Office (NR2PO), The guest-of-honour to witness this event was the Senior Minister of State for Trade and Industry, Singapore, Dr. Koh Poh Koon.

The signing ceremony taking place at ITAP 2019, Sandbox 2 on 24th October 2019.

This partnership will enable collaboration between the three organizations to accelerate the adoption of robotics using the Robot Operating System (ROS) in the automation industry and by system integrators in Singapore. ROS-Industrial Consortium will continue to work with organizations and associations to promote open innovation and collaborations, as well as the use of democratic robotics.

ROS-Industrial Asia Pacific at ITAP 2019

The Industrial Transformation Asia Pacific (ITAP) is the regional version of the iconic Hannover Messe, organized by Singex Exhibitions & Deutsche Messe. This event is one of the leading trade events in relation with Industry 4.0.

ITAP was aimed at bringing in key stakeholders and companies to encourage collaboration and deepen the understanding of advanced manufacturing and adoption of Industry 4.0 solutions.

ROS-Industrial Asia Pacific Consortium participated as an exhibitor this year, showcasing diverse demonstrations powered by the capabilities of ROS, and also featured with our consortium members ADLINK & Pepperl+Fuchs. The exhibition was stretched over a course of three days, from the 22nd to the 24th of October, 2019.

ROS-Industrial Asia Pacific Team at ITAP 2019, Day 1, before it’s doors opened to visitors!

One of our teammates Bey Hao Yun had the opportunity to conduct a few sessions hands-on session over the three-day period, which aimed to demonstrate to the audience the simplicity of creating a robotics solution using off-the-shelf open source software modules. His presentation excellently illustrated the entire programming flow in the usage of ROS for such applications.

Bey Hao Yun, from ROS-Industrial Consortium Asia Pacific, during the sandbox workshop.

Our consortium manager, Erik Unemyr, had also conducted a sandbox presentation on Accelerating Automation & Robotics Solutions with Open Source Software.

Erik presenting at Hall 2 Sandbox during ITAP 2019.

It was great to see users of ROS as well as people who are interested in learning and adopting ROS solutions come forward with questions for us throughout the entire exhibition.

A big thank you from the ROS-Industrial Consortium Asia Pacific team and we hope to see you in our future events!

Contributors:

Erik Unemyr

Sheila Devi Suppiah

ROS-I Training Day introduced ROS2 as advanced topic

SwRI hosted a session for ROS-Industrial training onsite in San Antonio on October 8-10. Of special interest was an introduction to ROS2 as a new advanced topic. This was the first inclusion of ROS2 material at a ROS-Industrial Americas training event and drew significant interest, with over a dozen developers attending.

ROS2 is a new iteration of ROS with many aspects undergoing complete redesigns, including core components such as the middleware layer and the build system. Along with these architectural redesigns, Open Robotics is taking the opportunity of breaking compatibility to make extensive improvements to the ROS experience in all aspects. As a consequence, many concepts, tools, and techniques we grew familiar with in ROS1 are no longer present and some upfront time learning about the changes is needed. The new training material developed at SwRI aims to ease this transition by providing some practice working in ROS2 for developers already familiar with ROS1 systems. There are currently three exercises developed which have been made available on the public ROS-Industrial training website. These exercises run through (1) the basics of working with ROS2 systems with a focus on the changes to the command-line tools, (2) the steps needed to port existing ROS1 packages with C++ code to functionally equivalent ROS2 packages, and (3) how to use the ROS1-ROS2 bridge to enable communication between two systems when porting is not yet feasible. Some additional details were also presented to the group, especially focused around SwRI’s ongoing experiences in porting a very large ROS1 codebase to ROS2.

Overall reception at the training event was quite positive, with a lot of interest to closely monitor the ongoing ROS2 development and see when the best time to start focusing on ROS2 will be. We expect that ROS2 training will continue to be a core topic, and plan to continue developing more material that will cover additional pieces of ROS2. Of course, eventually we expect to be training everyone in ROS2 from the start and have ROS1 relegated to maintenance mode. Be sure to check back frequently, as we’re right in the middle of this transition and updates could happen any time!

Tech Workshop on MoveIt, security & skill oriented programming with ROS

The Fall edition of ROS-Industrial EU Tech Workshop took place at Fraunhofer IPA on October 09th and 10th, 2019.

We were glad to host two European MoveIt maintainers, namely Henning Kayser of ROS-Industrial Consortium member PickNik Robotics and Michael Görner from University of Hamburg. They gave us an insight into the latest developments of MoveIt (incorporating motion planning, manipulation, 3D perception, kinematics, control & navigation), current and planned developments for ROS2 (MoveIt2), and a hands-on on ROS(1)-based 'bare-metal to product'. First they presented an inside-view of the manipulation framework. Providing complementary academic and industrial perspectives, they shared their views and experiences on MoveIt's overall structure, practical deployment of planning-based pipelines, complex manipulation planning using the MoveIt Task Constructor, and upcoming future projects and ideas for a ROS2 migration. The workshop concluded with a practical session that guided the participants to setup a functional Pick&Place pipeline from a custom bare robot description. Slides and code examples are available at https://github.com/henningkayser/ROS-Industrial_EU_Fall19_MoveIt .

The first session on day 2 of the ROS-Industrial EU Fall'19 Workshop was about security in ROS where Sebastian Taurer from JOANNEUM RESEARCH presented his work on a penetration testing tool for ROS1, called 'ROSPenTo', and gave an introduction on how to use SROS2 to secure communications in ROS2. In the first part of the session ROSPenTo was introduced to provide basic information on how it works and what a user can do with it. During the hands-on section the participants were guided through a step-by-step manual showing how to analyse, penetrate and modify a running ROS1 system using ROSPenTo. In the second part of the session ROS2's security tools (a.k.a. SROS2) were explained and used to setup and configure a security infrastructure. The provided examples demonstrated the creation of all necessary security artefacts (e.g. keys, certificates, etc.) and also the procedure to securely distribute the artefacts to different machines. All the related information as well as the workshop tutorial can be found here: https://github.com/jr-robotics/ROS-Industrial_EU_Fall19_Workshop

The ScalABLE4.0 session at the ROS Industrial EU Fall'19 Workshop focused on presenting the set of technologies which are enabling flexibility in production lines in two industrial pilots of the automotive sector: PSA Peugeot Citroën and Simoldes Plásticos. Within the project, a complete digital manufacturing software stack is being developed, entitled 'Open Scalable Production System' (OSPS). The OSPS aims to be applied to efficiently and effectively visualize, virtualize, construct, control, maintain and optimize production lines through a tight integration of the enterprise information systems with transformable automation equipment paired up with the necessary open interfaces for optimized solutions on all hierarchy levels (slides).

During the workshop, attendees were introduced and got a chance to test, interact and develop with the set of components that compose the OSPS, namely: (i) The Advanced Plant Model, which is responsible for virtually integrating data from the industrial shop floor in a centralized digital twin; (ii) The Production Manager, which is a cloud-based software module that issues and supervises the execution of manufacturing tasks; (iii) SkiROS and Task Manager, which are distinct ROS-based approaches to orchestrating the behaviour of robotic systems; (iv) The Skill-based Robot Programming methodology, which enables the reutilization and adaptation of ROS-based robotic applications to different purposes, platforms, and environments; (v) and, finally, the ROS-CODESYS bridge (ROBIN - https://github.com/ScalABLE40/robin), which enables horizontal integration between robots and automation equipment.

As part of the Scalable project, Bjarne Grossmann from AAU and cofounder of RiACT presented their skill-based robot control software SkiROS v2 (slides). Their technology is based on extended behavior trees that allows the definition of reactive behavior for highly flexible manufacturing environments. The framework is backed by a semantic database for inference and support of task planning to automatically generate complex tasks. In the hands-on session, Bjarne demonstrated the system with a SkiROS-implementation of the classical ROS turtlesim demo. He showed that SkiROS can be easily used to create complex behavior (and not only for turtles). The demo can be found on the git repository https://github.com/Bjarne-AAU/skiros-demo. Soon, there will be an official open source release of the software. Stay tuned on www.riact.eu!

Next European expert workshops will be organized in Spring and Fall 2020. We will keep you posted!

PS: Some links to upcoming events in this respect:

• Oct. 30- Nov 01, 2019, ROSCon 2019 in Macao: roscon.ros.org/2019/
• Nov. 02, 2019, MoveIt Workshop Macao: moveit.ros.org/moveit/ros/2019/09/19/moveit-workshop-macau.html
• Nov. 20, 2019, World MoveIt Day (WMD2019): moveit.ros.org/moveit/ros/2019/07/31/world-moveit-day-2019.html
• Dec. 10-12, 2019, ROS-Industrial Conference 2019 in Stuttgart: rosindustrial.org/riceu2019

Documentation updates improve ROS utilization and functionality

Lessons from UR driver updates reinforce importance of documentation

A key strength of the open-source community is the capacity to build on the knowledge of other developers who enable future advancements. Documentation plays a critical role in advancing understanding, which improves ROS utilization globally. This will be increasingly important as we move from ROS to ROS2 and document the various steps, including driver updates, necessary to execute projects.

Recent driver updates for a Universal Robots project helped demonstrate the importance of documentation to our team. In July 2019, Universal Robots updated their software for e-series and CB-series to 5.4 and 3.10. We were using the 5.4 software on UR 10e robots for a few different projects, and the ur_modern_driver [1] for ROS kinetic and melodic was no longer compatible due to this update. I updated the driver by investigating the release notes for 5.4.x.x [2] and the client_interface document [3].

View fullsize

UR10 E-series in the SwRI collaborative lab

To update the ROS driver for compatibility with software updates, I first identified what changes occurred and located appropriate software documentation for the hardware. The software documentation defined modules and variable types for the changes, which allowed for comparison with equivalent variables and modules in the current driver code. Without proper documentation from Universal Robots this would have been a much more difficult endeavor.

The ur_modern_driver specifically interacts with the client interface. Two variables were added to the 5.4 software client interface: a reserved byte in the Masterboard data sub package of Robot State Message to be used for internal UR use and a safety status value to the real time interface. The client interface document gave the types and sizes of these variables along with variables used in previous software versions.

The client interface for 5.3 and earlier had an internal UR int in Robot Mode Data I could compare to, and the safety status value could be compared to any of the other double variables in the previous driver’s RealTime interface. I used the search feature in QTCreator to find instances of these variables and added equivalent lines for the new ones. Then I used QTCreator’s debugger to track where new if statements and functions needed to be added to allow the driver to detect the new software version being used and access these new variables.

Working on this upgrade reinforced the necessity of good documentation. In general, the ur_modern_driver had more detailed documentation than many other ROS repos; however, it could still be improved. The README had no mention of the purpose of the use_lowbandwidth_trajectory_follower parameter in the launch files or the urXXe_bringup_joint_limited.launch file; both of these are useful when simulations are being overenthusiastic in their trajectory planning. I added documentation to the README to help others use these features to troubleshoot.

To update drivers, you will need to know what has been changed in the software, and you will ideally have access to a previous version of the driver. Because industrial hardware is intended to be reliable and accessible for multiple clients, there is often plenty of useful documentation if you can search with the correct terminology. Using the known changes and the documentation to compare with the previous driver code allowed me to update the driver fairly quickly so projects could move forward.

1https://github.com/ros-industrial/ur_modern_driver

2https://www.universal-robots.com/how-tos-and-faqs/faq/ur-faq/release-note-software-version-54xx/

3https://www.universal-robots.com/how-tos-and-faqs/how-to/ur-how-tos/remote-control-via-tcpip-16496/

A Look Back at RIA's Autonomous Mobile Robot Conference

This is a guest post by Southwest Research Institute Intelligent Machine’s Group Lead Cody Porter, as he was on-site at the RIA AMR Conference in Louisville, KY.

The Robotics Industries Association (RIA) and the Association for Advancing Automation (A3) hosted the inaugural Autonomous Mobile Robot Conference September 17th in Louisville, Kentucky. The event was a vast success with over 400 attendees ranging from end users, integrators, OEMs, researchers, and academics discussing the current AMR technologies, tips for integration, and future growth.

The opening speaker, Melonee Wise of Fetch Robotics, delivered a fantastic talk about the current technologies used in most AMR applications. The basics operations for how AMRs perceive and sense their environment, how to navigate dynamic environments, and what safety considerations are relevant to AMRs.

Matthew Rendall of OTTO Motors elegantly explained the most prevalent value proposition for AMRs. The labor force is shrinking in the United States while customer expectations for shorter lead times increase. The efficiency and effectiveness of AMRs are major benefits to most material handling applications and will see a continued acceptance in industry at large. Several other speakers, such as Denise Ebenhoech of Kuka and Norm Williams of OMRON, showed some additional applications of how AMRs have been used in other applications such as machine tending, machine feeding, lab automation, and several manufacturing applications.

Other topics of interest included the safety standards that govern the use of AMRs. A NIST report in 2013 highlighted the current gaps in published safety standards of the combination of automated ground vehicles (AGVs) and manipulators. The new standard, RIA R15.08, is in development. Michael Gerstenberger, chair of the R15.08 Drafting Subcommittee, gave a preview. A decision tree of the scope of this standard was presented as shown below.

The talks concluded with Aaron Prather, Senior Technical Advisor of FedEx, who gave an overview of the logistics industry and what lessons FedEx has learned in AMR implementation. The talk gave a great perspective of the end user wish list and current shortcomings of AMRs. The first was a technical challenge; AMRs must be able to handle outdoor environments. Sun-blinding sensors, ground conditions disrupting odometry, platforms not able to withstand weather, and the dead reckoning of wide-open spaces must be addressed to open a vast number of use cases. The second shortcoming is the lack of interoperability of multiple AMR applications. End users are looking to fit AMRs within legacy systems as well as allowing multiple AMR OEMs to provide solutions for the best use cases. All presentation materials are publicly released on the RIA website.

The accelerated adoption of AMRs into warehousing and material handling applications is fertile ground for advanced software solutions that continue to leverage ROS. The future of interoperability is one of the most obvious solutions as the map data could be seamlessly shared between all platforms if OEMs are willing to expose, as it appears industry is expecting. In addition, the path planning and manipulation capabilities demonstrated in ROS-based systems could continue to expand the use cases far beyond simple A-to-B material handling applications. The AMRs of the future will need to seamlessly switch between manufacturing processes and reallocate manipulators to where they are needed on large parts or large factories. Let’s get beyond where we are today, and even the successes that have been demonstrated, and support the innovation that is possible, while satisfying end user demand, with richer collaboration-enabling frameworks such as ROS.

Overseas Internship at ROS-Industrial Consortium Asia Pacific

Hello, my name is Willem de Graaf and I’m a former graduate intern at ROS-Industrial Asia Pacific, located in Singapore. I am majoring in Mechanical Engineering at Delft University of Technology in The Netherlands. During one of my Master’s courses I got acquainted with the ROS framework which got me interested to investigate further. When I got the opportunity to go abroad for my internship, I applied to ROS-Industrial Asia Pacific and got an intern position!

I would like to elaborate on how amazing this time was by guiding you through a typical day at the ROS Industrial Asia-Pacific office.

My day starts with traveling to the office. Owning a car in Singapore is quite expensive, so the majority of people travels with public transport. After 4 months in Singapore, I still don’t know if I can blame this behaviour on the expensive cars or the high humidity (as a Dutchy I thought it would be a good idea to have a 30 minute walk to the office on my first day. They considered me crazy). It is approximately a 10-minute walk to the MRT station where a shuttle bus will take me to the office. The whole journey will take around 25 minutes, which is quite fast for the average person working in the office. I arrive at the office between 8.15 and 8.30, where my day will start with a nice cup of coffee.

Before elaborating about the daily activities, it is helpful to give a little background information about the internship assignment. The assignment involved the design of an open-source library that will evaluate a gripper configuration against an object that has to be picked up by that gripper. The goal was to build a foundation of capabilities that will enhance the pick and place pipeline for manufacturing and warehouse environments. With this assignment, I got a lot of freedom to experiment with the existing ROS-Industrial software stack and the available hardware in the office.

Collaborative robot setup at the ROS-Industrial Consortium Asia Pacific office

The team within the organization is young, diverse and highly motivated to achieve the best in collaboration with companies. As said, I got my own project where I was given the opportunity to explore and express my own ideas. The team is working on a lot of projects at the same time and because we had a daily stand-up where we had to elaborate on our progress, we kept focus and challenged each other on a daily basis. From here onwards, I ran my own project and planned my daily activities. Most of the time I was developing software, but when it was time for testing the fun part started. I was able to use UR robots, KUKA robots, multiple grippers, conveyor belts and a lot of more fun stuff. You are free to explore the possibilities with different hardware and electronics.

Pick and place flexible gripper and conveyor belt setup - ROS-Industrial Asia Pacific Workshop 2019

A typical Asian thing is to get lunch together at a food court. Because the office location was not situated around food courts, we had to take a bus every day to lunch. For a European, this is the best part of the day. The variety of food is enormous and the amount is tremendous, especially taking the price into account. Comparing to the European prices a standard lunch is quite cheap. Due to this I have to confess that my cooking skills did not improve during my time in Singapore :p.

In the last week of my internship, our office hosted a yearly conference – the ROS-Industrial Asia Pacific Workshop 2019, where the capabilities of the developed library were showcased at a demonstration setup.

Willem de Graaf (Intern) and Arunava Nag (ROS-Industrial team) at the ROS-Industrial Asia Pacific Workshop 2019

I would like to conclude with saying that working at ROS-Industrial Asia Pacific could not have turned out better for me than it did. I learned a lot about the diverse Singaporean culture and people, I learned a lot about the Asian way of getting things done and above all, I learned a lot about ROS, which was my main goal.

If you’re seeking for an internship, please do not hesitate to contact them as they’re always open for enthusiastic and motivated students that will contribute to get open source robotics to companies!

Willem de Graaf

YAK: 3D Reconstruction in ROS2

So why YAK?

It’s difficult for robots to perceive objects in the real world, especially when those objects are shiny, previously unseen, or (gasp!) both! We are excited to introduce Yak, an open-source GPU-accelerated ROS2 package which addresses some of these challenges using Truncated Signed Distance Fields. The Southwest Research Institute booth demo from the Automate 2019 conference serves as a case study for a ROS2 system that integrates Yak into its perception and motion planning pipeline.

A bit of technical background

A Truncated Signed Distance Field (TSDF) is a 3D voxel array representing objects within a volume of space in which each voxel is labeled with the distance to the nearest surface. The TSDF algorithm can be efficiently parallelized on a general-purpose graphics processor, which allows data from RGB-D cameras to be integrated into the volume in real time.

Numerous observations of an object from different perspectives average out noise and errors due to specular highlights and interreflections, producing a smooth continuous surface. This is a key advantage over equivalent point-cloud-centric strategies, which require additional processing to distinguish between engineered features and erroneous artifacts in the scan data. The volume can be converted to a triangular mesh using the Marching Cubes algorithm and then handed off to application-specific processes.

Machined Al Test Part

YAK Reconstruction of Machined Al Part

Reconstructing in the real world

My group at Southwest Research Institute has been working with TSDFs since Spring 2017. This year we started work on our first commercial projects leveraging TSDFs in industrial applications. We developed our booth demo at the 2019 Automate conference to provide an open-source example of this type of system.

The set of libraries we use in these applications is called Yak (which stands for Yet Another KinectFusion, in reference to the substantial prior history of TSDF algorithms). Yak consists of two repositories: a ROS-agnostic set of core libraries implementing the TSDF algorithm, and a repository containing ROS packages wrapping the core libraries in a node with subscribers for image data and services to handle meshing and resetting the volume. Both ROS and ROS2 versions of this node are provided. A unique feature of Yak compared to previous TSDF libraries is that the pose of the sensor origin can be provided through the ROS tf system from an outside source such as robot forward kinematics or external tracking, which is advantageous for robotic applications since it leverages information that is generally already known to the system.

The test system, which is in the pipeline for open-source release as a practical demonstration, uses a Realsense D435 RGB-D camera mounted on a Kuka iiwa7 to collect 3D images of a shiny metal tube bent into a previously-unseen configuration. The scans are integrated into a TSDF volume as the robot moves the camera around the tube.

The resulting mesh is processed by a ROS node using surface analysis functions from VTK to extract waypoints and tangential vectors along the length of the tube. These waypoints constitute the seed trajectory for a motion plan generated by our Descartes and Tesseract libraries which sweeps a ring along the tube while avoiding collision. Camera and turntable extrinsic calibration was performed using a nonlinear optimization function from the robot_cal_tools package.

A picture of the reconstructed mesh used to generate a robot trajectory is below. Additional information about the development of this demo (plus some neat video!) is available here. As ROS2 training is integrated into the ROS-I Consortium Americas curriculum, we look forward to sharing more lessons learned about applying Yak towards similar applications.

What next?

Yak is open-source as of July 2019. While development is ongoing and we anticipate that the library APIs will continue to evolve, we encourage interested parties to check it out at github.com/ros-industrial/yak and github.com/ros-industrial/yak_ros .