ROS Industrial Conference #RICEU2018 (Session 4)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the fourth instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT (watch all but 1 talks in this YouTube playlist)
  4. Hardware and application highlights (watch all but 1 talks in this YouTube playlist)

Day 3 - Session “Hardware and Application Highlights“

Georg Heppner (FZI) and Fabian Fürst (Opel) At ROS-Industrial Conference 2018

Georg Heppner (FZI) and Fabian Fürst (Opel) At ROS-Industrial Conference 2018

In the fourth and final session of the ROS-Industrial Conference 2018, the focus was on hardware developments and applications implemented in industrial use cases. Fabian Fuerst, Opel, and Georg Heppner, FZI, delivered the session keynote. They presented their solution for flexible automotive assembly with industrial robotic co-workers. The application was developed as part of the EU EuRoC project. In this four-year competition, more than 100 participants initially worked on new robotic solutions for the manufacturing industry. In the course of several evaluation rounds, the team from FZI, Opel and MRK Systeme GmbH was able to assert itself successfully to the end.

During the course of the project, the FZI developed an automated robotic assembly for flexible polymer door sealings on car doors. The sealing is a closed ring, which has to be fixed with up to 40 plastic pins depending on the model, an ergonomically unfavourable task that could not be automated until now. The developed assembly cell is very flexible and open, so that the robot can be used without a safety fence. For this purpose, an external force control was developed that can be used easily and directly also for numerous other robots as a package of ROS-Industrial. The CAD-2-PATH software is used for the simple path creation for the robot. This enables a quick adjustment to other door models and does not require any expert knowledge. This is important because there are different door models and sealing types and the automation solution must be adaptable accordingly and quickly. It is notable that the application received positive assessment from Opel with regards to safety, typically a sensitive topic when applying novel tools such as ROS in automotive applications.

Paul Evans (Southwest Research Institute / ROS-Industrial North America) at ROS-Industrial Conference 2018

Paul Evans (Southwest Research Institute / ROS-Industrial North America) at ROS-Industrial Conference 2018

The presentation by Paul Evans, Southwest Research Institute and ROS-Industrial Consortium North Americas, provided current information on the activities of the North America Consortium such as strategic initiatives, trainings, and networking activities. These also focus on voices of members and include activities for the strategy alignment, for more robustness and flexibility and agility. There are also collaborations with OEMs who support ROS or develop their own drivers. At the ROS-I Consortium Americas Annual Meeting 2018, different applications were presented, for example an order batch picking robot from Bastian Solutions and a robotic system for agile aerospace applications like sanding, blending, drilling etc. for the U.S. Air Force. A last highlight that Evans presented was the ROS-I collaboration with BMW and Microsoft. While RIC-North Americas supported the evaluation of simulation environments that included physics engines the RIC-EU partners provided additional navigation support and training for mobile robots at the BMW plant to support assembly logistics. The solution is deployed on Microsoft Azure.

Mobile robots was also the topic of the lecture by Karsten Bohlmann, E&K Automation. He presented solutions for ROS on AGVs and perception-driven load handling and PLC interfaces.

Arun Damodaran (Denso) at ROS-Inudstrial Conference 2018

Arun Damodaran (Denso) at ROS-Inudstrial Conference 2018

Denso Robotics Europe was present at the conference with Arun Damodaran, who talked about Cobotta, the ROS-enabled collaborative robot. This is a six-axis arm with a reach of 342 mm, a repeatability of 0,05 mm and a payload of 500 g. It has an inherently safe design, meets all requirements for safety-standards corresponding to the ISO norms and is compliant thanks to safety-rated monitored function. Another advantage is its easy set-up and use. This is realized by the usage of the robot programming software drag&bot. Developed by the spin-off of the same name of Fraunhofer IPA, the software enables the programming of robots like Cobotta with the drag and drop principle. No expert knowledge is needed. The software is also based on ROS, works independently from any robot manufacturer and can be reused as well as shared via the cloud. Denso has been engaged in the development of ROS components and packages (simulation, control, path creating) for its robots since 2012 and now uses an open platform for controlling the Cobotta.

Felipe Garcia Lopez from Fraunhofer IPA focused on a networked navigation solution for mobile robots in industrial applications. This is particularly useful for changing environments in which mobile robots should independently select free routes. Fraunhofer IPA and Bär Automation, for example, have implemented a navigation solution for agile assembly in automobile production. With this, AGVs can locate themselves robustly and precisely based on sensor data, even without special infrastructure. This makes it possible to easily adapt existing paths or integrate new ones even after commissioning. Since the software's sensor fusion module can process data from almost any sensor, very customer-specific solutions can be implemented.

Another example is the networked navigation for smart transport robots at BMW. Here as well there were few static landmarks, a lot of dynamic obstacles and sparse sensor data in large-scale environments. A process reliability of more than 99% had to be fulfilled. The presented navigation as well as the vehicle control are ROS-based. At the end of the presentation, an outlook into Cloud-Navigation was given: Mobile robots and stationary sensors are then connected using a Cloud-based IT-infrastructure. The environment is cooperatively modelled and SLAM is used. This enables also solutions for “Navigation-as-a-service” meaning map updates and cooperative path planning for each robot. With Cloud-Navigation, local hardware and computational resources can be reduced and the quality and flexibility of the overall navigation system is enhanced.

Thomas Pilz (Pilz GmbH & Co. KG) at ROS-Inudstrial Conference 2018

Thomas Pilz (Pilz GmbH & Co. KG) at ROS-Inudstrial Conference 2018

ROS as an appropriate solution both inside and outside of industry – this was the starting point for Thomas Pilz, Managing Partner of family owned company Pilz. Combined with his own career and his experience with the first service robots, lightweight robots and robots outside production environments, he first described how the question of safety standards has changed in recent years. The definition and understanding of a robot is currently in the process of changing significantly. For Pilz, systems such as the Care-O-bot® from Fraunhofer IPA are the new upcoming robots. They operate outside of cages, are mobile and users can easily interact with them and program them using ROS. He sees ROS as a success factor for service robots because of its modular design, its standardization, additional flexibility through programming languages and its networked, interoperable system in line with Industry 4.0.

Robots that are to interact with humans are also changing the required safety technology at Pilz in the long term because all previous infrastructure such as fences is no longer required. This led Pilz to develop its own robot arm with appropriate safety technology. They use ROS modules developed by Pilz because they are breaking new ground with the development of the robot arm and can thus fall back on a broad programming knowledge base. They had nothing to lose with the new product. However, in order for them to meet the safety standards, the modules must no longer be changed in an uncontrolled manner. To improve this, Pilz recommends changing the safety standards so that they are also amenable to Open Source. Finally yet importantly, he believes that the term robot manufacturer will also change, because this role will increasingly be fulfilled by those who implement the application and no longer by those who produce the robot or components for it. In the lively discussion after the presentation, Pilz once again emphasized two arguments in favour of ROS. First: When it is said that ROS is tedious, one should bear in mind that the development of proprietary software is also difficult. Second: ROS is tedious, but fun. Pilz also sees ROS as a decisive factor for employee satisfaction and as an argument for staying with Pilz.

At the end of the conference, Gaël Blondel from the Eclipse Foundation presented the Eclipse Foundation and its Robotics Activities. The platform with around 280 corporate members, half of them from Europe, provides a mature, scalable, and business-friendly environment for open source software collaboration and innovation. Eclipse is vendor-neutral and offers a business-friendly ecosystem based on extensible platforms. They offer their own IP management and licensing but also accept other business-friendly licenses. Several working groups are particularly engaged in development processes for robotics. One example for a robotic project managed with Eclipse is the EU project RobMoSys that aims to coordinate the whole community’s best and consorted efforts to realise a step-change towards a European ecosystem for open and sustainable industry-grade software development.

At the end of the event, Mirko Bordignon and Thilo Zimmermann thanked the participants for another great and record breaking ROS-Industrial Conference. Presentations and videos of the event have been made available on the event website: https://rosindustrial.org/events/2018/12/11/ros-industrial-conference-2018

ROS Industrial Conference #RICEU2018 (Session 3)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the third instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT (watch all but 1 talks in this YouTube playlist)
  4. Hardware and application highlights

Day 2 - Session “Robotics meets IT“

Henrik Christensen (UC San Diego) at ROS-Industrial Conference 2018

Henrik Christensen (UC San Diego) at ROS-Industrial Conference 2018

The third session testified the growing importance of ROS to support the development and deployment of robotic solutions from companies outside the traditional boundaries of this industry. Predominantly software players such as Amazon or Google now offer platforms leveraging ROS, which they described during the session.

Henrik Christensen, from UC San Diego and ROBO Global, gave a very inspiring keynote speech on why robotics is increasingly using cloud technologies and how it will benefit from them. He outlined three factors as current business drivers for this development: the increasing demand for flexibility in production, the aging world population and the associated increasing demand for service robots at home, and finally the trend that more and more people live in cities, posing great challenges for logistics. All robot solutions must be cost-efficient and robust at the same time in order to offer the required reliability. If computer performance always had to be on board, the hardware would often be inadequate (e.g. for slim service robots for private use) or the costs for suitable hardware would be too high (e.g. for autonomous cars).

Technologies from or in the cloud can be a solution for this. Christensen presented the value of these ecosystems using extensive market examples and explained how they differ in agility and size. Many successful companies, primarily in the USA and Asia, have shifted their business model from owning things or technologies to orchestrating them and offering services. For robotics, ROS 2.0 can be a decisive door opener here, offering the standardization required for platforms.

Milad Geravand (Bosch Engineering) at ROS-Industrial Conference 2018

Milad Geravand (Bosch Engineering) at ROS-Industrial Conference 2018

The next presentations in the session took up these and similar ideas and presented existing solutions. Milad Geravand from Bosch Engineering presented a modular software platform for mobile systems such as cleaning, off-road and intralogistics robots and how they can be developed more efficiently. In his experience, the difficulties in the development process are similar in many companies: The applications are usually very different, the software is becoming increasingly complex, a structured deployment and integration process is lacking. ROS is not yet ready for the products and the leap from prototype to series production is still too big. With the software platform presented, which is based on ROS, Bosch would therefore like to address precisely these challenges and enable uses cases to be developed quickly and reliably.

Eric Jensen, working for Canonical, the company well known for the Ubuntu Linux distribution, presented the advantages of Ubuntu Core especially with regard to security that is still an open issue for ROS. The mentioned advantages are: A minimal, transactional Ubuntu for appliances, safe and reliable updates with tests and rollbacks, app containment and isolation with managed access to resources, a unique development environment familiar for Linux developers and the possibility to easily create app stores for all devices needed. Furthermore, Ubuntu has one of the biggest developer communities in the world and is backed by Canonical itself, an important plus for security. Last but not least, the system offers automatic security warnings for the „snaps“, the special package format in Ubuntu, system audits through package verification and compliance management – all are important features for an improved security.

Roger Barga (Amazon AWS) at ROS-Inudstrial Conference 2018

Roger Barga (Amazon AWS) at ROS-Inudstrial Conference 2018

Only a few weeks before the ROS-Industrial-Conference, Amazon, for a long time far more than an e-commerce store, had introduced its new platform AWS RoboMaker, which caused a sensation beyond the ROS-Community. Roger Barga, General Manager at AWS Robotics & Autonomous Services, kindly presented this novel development at the conference. Amazon's commitment to robotics is based on discussions with around 100 companies, during which they were able to identify two main problems in robot development. On the one hand, this is a very high demand for automation solutions with simultaneous difficulties with ROS such as security or performance. On the other hand, the development process is usually very inefficient.

The RoboMaker platform addresses these requirements with its four main components. It offers a browser-based development environment, which in turn has integrated cloud extensions for ROS as well as a simulation environment. The cloud extensions range from machine learning tools to monitoring and analytics. Concrete capabilities for robots include speech recognition and output, video streaming, image and video analysis, as well as logging and monitoring with Amazon CloudWatch. The simulation environment allows thousands of simulations to be run in parallel. The fourth component is fleet management, so that robot applications can be deployed over the air. The presentation ended with a short introduction to the learning environment of RoboMaker, with which Amazon applies reinforcement learning to robots. The robots then learn according to the principle "trial and error". By merging all errors within a fleet in the cloud, a large knowledge base is quickly available and not every single robot has to make a specific error to learn from, but it benefits from the learning experiences of other robots in the fleet.

The topic of robotics in the cloud was also the focus of the lecture by Christian Henkel from Fraunhofer IPA. In his experience, the deployment of ROS-based applications on distributed systems such as mobile robots is still too great a challenge, which he would like to address in his work with docker containers (dockeROS). With his solution, it is possible to simply run ros nodes in docker containers on remote robots.

Martin Hägele (Fraunhofer IPA) moderates a panel discussion with Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google).

Martin Hägele (Fraunhofer IPA) moderates a panel discussion with Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google).

With Damon Kohler, Google Robotics and its recently presented cloud solution were also represented at the conference. In his introductory remarks, Kohler mentioned several challenges related to cloud robotics, including security, connectivity and latency, and distributing work, e.g. partitioning problems. In contrast, he sees advantages such as scalability, collaborative perception and behaviour and a robust change management and monitoring. He sees cloud robotics as a further development of the well-known principle "sense -> plan -> act" around the component "sense -> share -> plan -> act" and as an interplay of edge and cloud processing.

The aims of cloud robotics are an increased launch cadence, more data and more users and a better resource utilization. This shall be reached by infrastructure as a service, design for small and decoupled components as well as tools for automation and orchestration. The ROS nodes correspond to the Google micro-services: They are stateless and replicable, which means horizontally scalable. The container orchestration engine Kubernetes helps to deploy and release these micro-services. Several mature and robust logging and monitoring tools like Stackdriver help managing the system. The heart of the whole is the Cloud Robotics Core, being available from beginning of 2019 that enables to integrate Kubernetes on robots. Overall, Google’s vision is an open platform and a thriving ecosystem where integrators, developers, hardware developers and operators can collaborate with customers efficiently.

The second day of the conference ended with a panel discussion. The panellists were Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google). Moderated by Martin Hägele (Fraunhofer IPA), they summed up some advantages from their respective company perspectives, but also existing challenges of ROS and the role of open source software and robotics for their corporate strategy.

ROS Industrial Conference #RICEU2018 (Session 2)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the second instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT
  4. Hardware and application highlights

Day 2 - Session “Software and System Integration Topics“

Dave Coleman (PickNik) at ROS-Industrial Conference 2018

Dave Coleman (PickNik) at ROS-Industrial Conference 2018

The second day of the conference started with the session "Software and System Integration Topics". Dave Coleman, founder of Picknik Consulting and lead maintainer of MoveIt!, opened the session with a very personal keynote about his commitment to open source software, from his student days to his role as an entrepreneur. He reported how he got in touch with the beginnings of ROS at Willow Garage and highlighted the unique spirit with which the project was incubated. He introduced the successful MoveIt! library, shared his lessons learned and the challenges which many open source projects face. As a proof of how Open Source and business can successfully coexist, he described the founding of PickNik and how the company is profitable without investors.

The following presentations were more technical and started with Víctor Mayoral Vilches, CEO of Acutronic Robotics. He talked about his company's solutions for system integration in modular systems, through the device H-ROS SoM (System on Module), used as example. In his opinion, ROS already addresses many programming needs, but system integration goes far beyond programming and requires extensive resources for each new project. He therefore sees modularity as an essential improvement. Combining the features of a real-time capable link layer made of RTOS and the Linux Network stack, and ROS 2.0, he presented the challenges and developed solutions to achieve easier system integration. He also gave insights into the use of AI to further reduce programming efforts and to train the robot instead, a technology that is still in its infancy. As part of a Focused Technical Project with ROSIN, the company also worked on the interoperability of modules.

Jon Tjerngren (ABB) at ROS-Industrial Conference 2018

Jon Tjerngren (ABB) at ROS-Industrial Conference 2018

Jon Tjerngren presented how ABB robots can be used with ROS. For this purpose, the company developed various ease-of-use packages with ROS that simplify and accelerate the setup of ABB robots. All of them are already freely available online: abb_librws can be used to off-load of computational heavy tasks, e.g. image processing. abb_libegm can be used for motion correction and as an StateMachine add-in for remote control.

ROS2 Embedded tailored to real-time operating systems was the topic of Ingo Lütkebohle’s presentation from Bosch Corporate Research. He emphasized the importance that ROS must also be integrated into the firmware. This would better address four challenges: hardware access, latency, power savings, and safety. To this end, he presented a solution developed in the OFERA project with which ROS2 can be used in microcontrollers.

André Santos from INESC TEC and University of Minho, focused on software quality. More and more robot systems are safety-critical systems, which places very high demands on the quality of the software. Finding errors in the code early on reduces costs and development time. Although there are various static analysis tools, none offers ROS-specific analysis. This is why the HAROS (High Assurance ROS) framework was developed, which is capable of extracting and, to some extent, reverse-engineering the computation graph. It also provides a visualization of the extracted graph and enables property-based testing for ROS.

Anders Billise Beck (UR) at ROS-Inudstrial Conference 2018

Anders Billise Beck (UR) at ROS-Inudstrial Conference 2018

Anders Billersoe Beck from Universal Robots was the last speaker in the second session. He introduced the new UR e-series (with integrated force/torque sensor, 500 Hz controller frequency and more new features) and how ROS supports it. For this, a new driver is developed in a Focused Technical Project of ROSIN together with the FZI, which will also remain open-source. The goal is to make a UR robot easy to use and enable plug-and-play with ROS. The driver should make two modes of operation possible: remote control and ROS URcap embedding. More supported features are calibration, a new safety system and easier programming. Beck concluded the presentation with some points that he believes are in need of improvement to make ROS ready for industrial applications. These are easier general use, proper handling of hard and soft real-time boundaries and supporting more control in edge devices.

ROS-Industrial Consortium Asia Pacific Training Milestone

ROS-Industrial Consortium Asia Pacific, supported by Southwest Research Institute, has now trained more than 100 participants on ROS since its start in 2017!

On 11-13th December ROS-Industrial Consortium Asia Pacific conducted a Basic course on ROS, at the Advanced Remanufacturing and Technology Centre, Singapore. It was followed by an Advanced topic course on the 14th December.

ROS-Industrial Consortium Asia Pacific December 2018 Training - Basic Course

ROS-Industrial Consortium Asia Pacific December 2018 Training - Basic Course

The workshop was led by Dr. Joseph Polden, and the full class of 15 participants was first introduced to the fundamental concepts of the ROS architecture and package ecosystem, followed by hands-on exercises in motion planning and perception.

ROS-Industrial Consortium Asia Pacific December 2018 Training - Group Photo

ROS-Industrial Consortium Asia Pacific December 2018 Training - Group Photo

Thank you again for those of you who participated in this round of training! Please reach out to ROS-Industrial Consortium Asia Pacific (ros-i_asia@artc.a-star.edu.sg) if you have interest in signing up for our upcoming ROS training events in 2019!

ROS Industrial Conference #RICEU2018 (Session 1)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: This year's ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the first instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration
  3. Robotics meets IT
  4. Hardware and application highlights

Day 1 - Session "EU ROS Updates"

Mirko Bordignon (Fraunhofer IPA) opening ROS-Industrial Conference 2018

Mirko Bordignon (Fraunhofer IPA) opening ROS-Industrial Conference 2018

The topic of open source software for robotics was present in the media throughout the year, and announcements that companies such as Google, Amazon and Microsoft would rely on ROS made waves outside the community, too. In addition, there is a booming robotics market. Martin Hägele (Fraunhofer IPA) highlighted this in his opening talk based on current market figures and areas of application for industrial and service robotics. In this respect, it is not surprising that politics and research funding on a national and international level are becoming increasingly aware of ROS. The speakers on the first day of the conference presented the projects and activities currently underway here.

ROSIN project overview

Bringing ROS into industrial application in Europe is one of the main activities of the EU project ROSIN (ROS-Industrial Quality Assured Robot Software Components) that Carlos Hernandez Corbato of TU Delft presented. This runs from 2017 to 2020 and is mainly involved in three fields:

  1. Further development of ROS components within the framework of so-called Focused Technical Projects (FTPs)
  2. Tools for software quality assurance
  3. Education activities

Applications for FTPs can still be submitted until 2020. The next cut-off date is April 5th 2019. All information on the short application process can be found here. A decisive criterion: The project provides funding for developments for which there are concrete market requirements. For this reason, the project finances one third of the software development and the applicant takes over the other two thirds.

Carlos Hernandez Corbato (TU Delft) at ROS-Industrial Conference 2018

Carlos Hernandez Corbato (TU Delft) at ROS-Industrial Conference 2018

Successful FTP examples Pilz, Nobleo, PPM and Roboception

ROSIN granted already 20 applications for FTPs and 21 more are under review. Here are four examples of successful FTPs:

For Pilz, “Industrial Trajectory Generation for MoveIt!” was granted: Most industrial robot manipulators supported in ROS come with a MoveIt! configuration. The Motion Planning plugin for RViz allows simple and visualized planning and execution of free-space motion. Planning and obstacle avoidance work mostly out-of-the-box. This FTP addresses Cartesian motion: existing libraries for Cartesian trajectory generation lacked a user-friendly interface. The FTP implements a trajectory generator with a MoveIt!-interface for easy planning and execution of Cartesian standard-paths. In addition, the blending of multiple sequential motion commands is realized.

For Nobleo Projects, “Full Coverage Path Planning and Control“ was granted: Many robotic applications need to plan a path that passes over all points of an area or volume of interest while avoiding obstacles. As soon as a path is planned, the next challenge is to control it. As neither ROS, nor ROS Industrial are currently providing needed packages incorporating this (complete) coverage path planning or trajectory tracking functionality, this FTP proposes to develop, verify and validate these packages.

For PPM, the FTP “ROSweld” was granted: It develops an innovative ROS based framework for planning, monitoring and control of multi-pass robot applications with an intuitive, user-friendly GUI. The framework is built upon components from the project partners’ previous research and existing ROS modules. ROSWELD is demonstrated by the case study in heavy, multi-pass welding.

For Roboception, the FTP “Visard4ROS” was granted: Visard4ROS will provide a ROS interface to fully exploit the capabilities of the rc_visard sensor and to easily integrate it into robotic products or research platforms. As part of the process, Visard4ROS will also provide documentation for integration of sensors with standard industrial interfaces such as GigE Vision and GenICam, plus examples and good practices for using separate libraries to build ROS-I hardware drivers.

Education activities

Stephan Kallweit (FH Aachen) and Jonathan Hechtbauer (Fraunhofer IPA) at ROS-Industrial Conference 2018

Stephan Kallweit (FH Aachen) and Jonathan Hechtbauer (Fraunhofer IPA) at ROS-Industrial Conference 2018

The second goal of ROSIN are education activities. Stephan Kallweit (FH Aachen) and Jonathan Hechtbauer (Fraunhofer IPA) presented the two formats with which the project conveys ROS knowledge. One of them is the ROS-I School. It addresses university students and young professionals to get an entry to the ROS-Industrial eco-system. Its teaching concept consists of seminars, tutorials and workshops. In addition, ROSIN has founded the ROS-I Academy. It consists of a ROS-I certified engineer program to assess certain skills within the ROS-Industrial software engineering eco-system. The certified skills comprise basic knowledge in ROS-Industrial, skills in code review and specialised ROS-Industrial topics. Check out the website for upcoming events.

Quality Assurance Tools

The third main activity of the ROSIN project are measures and technologies to improve the quality of software. Adam Alami and Zhoulai Fu (IT University of Copenhagen) presented the ongoing steps. On the one hand, a process and supporting tools are developed for quality assurance, where the quality of packages can be measured, assigned and displayed. Furthermore, ownerships for QA practices, tools and infrastructure will be appointed. Furthermore, code review practices are going to be reinstituted and a code scanning method and tool will be implemented. A quality hub website is already online in order to create a source of knowledge for quality assurance. A source of collaboration for quality assurance offers this page.

Another quality improvement measure is the automated code testing. Traditional platforms are not effective enough to provide the reliability that ROS needs today as they run the same and very few test harness for many times. However, a ROS package is reliable, when it works as expected for all run-time scenarios. That is why ROSIN aims at developing a reliability-oriented testing framework that will be integrated to the ROS ecosystem.

Outlook: Further research activities thanks to RobMoSys and SeRoNet

The conference day ended with two contributions on other research projects that also rely on ROS. Dennis Stampfer (University of Applied Sciences Ulm) presented RobMoSys. It aims at coordinating the community’s efforts to realize an industry-grade software development European ecosystem that is open, sustainable and ensures industrial quality. This shall increase the scalability and quality of robotics software development, help to commoditize base functionality, such as motion control, navigation, software components of certifiable quality and achieve predictable system integration. It addresses user requirements like, among others, reduction of development time and costs, shorter time to market and safety via a model-driven approach.

Björn Kahl (Fraunhofer IPA) presented SeRoNet. This project intends to significantly simplify the design, development, and deployment of service robots in a variety of areas, from logistics, care, and healthcare to assembly support in manufacturing operations. Through an online platform, users, system integrators and component manufacturers of service robot solutions will be able to collaborate efficiently and jointly support solutions from requirements analysis to deployment. The SeRoNet platform (available from summer 2019) will bring together users and producers of robotic solutions and will create a market for service robot solutions, services and hardware as well as software components for application solutions. Both projects will publish Open Calls in 2019, for which companies involved in robotics can apply for funding.

Observations from the first RIA Robotic Grinding and Finishing Conference hosted by 3M

A recent conference brought together end-users, solution providers, OEMs and researchers to discuss the latest in robotic applications around grinding and surface finishing (https://www.robotics.org/robotic-grinding-and-finishing-conference). It was an eye opening event and the first of its kind to focus on automation for these types of processes. While there are many conferences on automation, the topics of surface grinding and finishing are rarely at the top of the topic areas. This event also underscored how there are few specialists in this area.

Day 1

The first panel of note discussed whether to automate these operations and how to understand if there was benefit in attacking what is typically considered manual work. There were a number of assumptions that went into the approach, but we learned quickly there was a lot of interest as this was an audience that has historically struggled with legacy approaches and hardware in automating such operations and processes.

Charles Gales of Weldon Solutions Presents on Automating of Complimentary Processes as a means to introduce surface finishing to your operations.

Charles Gales of Weldon Solutions Presents on Automating of Complimentary Processes as a means to introduce surface finishing to your operations.

The sessions’ two main points were: automate as you can, such as complimentary processes (capture more work with that automation investment), understand your process and burden, and be cognizant of the fact of how you do it a certain way manually today doesn’t mean that will be the most optimal way to execute your process robotically.

Force control was the next panel that gained a lot of attention. Obviously, force/torque sensing in a number of areas has become an area of active robotic interest in traditional automation applications and of course applicable in surface processing. There were introductory conversations about the types of force control, such as the pros and cons of passive versus active. A number of compelling applications leveraging active force control were featured. ATI, PushCorp and FerRobotics all offer approaches to meet a wide array of client needs and applications.

ATI, PushCorp, and Fer Robotics participate on a panel discussing force control in surface finishing and grinding applications.

ATI, PushCorp, and Fer Robotics participate on a panel discussing force control in surface finishing and grinding applications.

Near the end of the day was an additional panel around DIY Integration, which was incredibly eye opening and generated a lot of conversation. The panelists – Brandon Berth, from Kohler, Matt Morrison, from Marshalltown, and Scott Harms, from MetalQuest – shared interesting stories about making progress on their automation journey through trial and error and developing their own in-house skills to manage deployments. These were stories that relied on a commitment and a process; starting small, developing the skills and gradually moving to more complex applications.

Day 2

The second day started off with Kuka presenting on enabling small and medium enterprises to take on grinding application development. The key tool they demonstrated was the ability in their simulation ecosystem to empirically model the sanding process and highlight the path and planned surface contact and subsequent material removal. This provided a very compelling visual assessment for the grinding process as it is being developed in their off-line environment.

The next two presentations were very interesting as they showcased two competent integrators in the space. What was evident, even though their core expertise varied, was that they relied heavily on the teach pendant and skilled online programmers to really bring the process over the finish line. Much like the prior presentations, or the presentations by the DIY’ers, here again we were relying on skilled technicians and industrial robot programmers to complete the implementation of the automation. The level of variation management is limited due to the nature of the deployments. Also, for some of these applications, it may not be realistic to enable too much flexibility as the broad array of complex surface finishing applications was impressive.

I had the opportunity to talk about ROS-Industrial and the work that has been going on in surface finishing using perception and advanced path planning and process planning techniques. This space has talked about path planners (Trajopt!) and applications (Robotic Blending, A5 and undergrads using these tools), so I won’t get into those details here, but two thing are evident. There is a need to change how we approach these processes if we want to truly attack these types of applications. We can’t rely on skilled robot techs to have a teach pendant in-hand. Not every company can grow that expertise in-house over years. Even if they do, how do they scale beyond their core facilities?

Panel on advances in surface finishing moderated by Jay Douglass of the ARM Institute

Panel on advances in surface finishing moderated by Jay Douglass of the ARM Institute

I was asked about the capabilities in the blending and A5 videos and how ready they are to go into anyone’s plant. That is a great question, and I would be asking that, too, if I was in that seat, as I did when I was on the industry side. We are working on that, hence we were there with the ARM Institute and all the interest in the surface finishing topic calls, and continued development of A5, and an additional milestone upcoming for Robotic Blending. That is why there is a ROS-I Consortium and a ROSIN initiative in the EU. We will continue to make the modules more robust and build out application examples that can be leveraged and molded into end-user capability. In the meantime we hope to entice integrator and solution providers to embrace a ROS-based software approach along with our OEM partners. If we work together we can build out reusable components that meet this need, in this greatly underserved area. The exciting part is there is a lot we can do, and it doesn’t have to that far away! It was great to see 3M, our hosts, showcase a ROS-based demonstration and do their part to introduce the concepts and how they could lend a hand in solution development. We are excited to have 3M as such an engaged partner and hope together we can keep growing the open-source tool revolution! #GoROS

New Release of ROS Qt Creator 4.8 RC on Xenial and Bionic

We are pleased to announce the release of the ROS Qt Creator Plug-in for Qt Creator 4.8 RC on Xenial and Bionic. The ROS Qt Creator Plug-in creates a centralized location for ROS tools to increase efficiency and simplify tasks.

Picture obtained from  Qt Blog

Picture obtained from Qt Blog

Highlights:

  • Qt Creator 4.8 introduces several new rich features and improvements to existing capabilities.
    • Generic Programming Language Support (Python Support!)
      • To use this feature, you must enable the Language Support Plugin
    • C++
      • Compilation Database Projects!
      • Clang Format Based Indentation!
      • Cppcheck Diagnostics!
      • Simultaneously debugging one or more executables!
  • ROS Plug-in introducint a few new features and bug fixes
    • Upgraded to Qt Creator 4.8
    • Added catkin_test_results run step
    • Added ROS Settings Page to configure default settings
    • Bug Fixes
      • Issue #284 Package Wizard caused Qt Creator to crash if using when a ROS project is not loaded.
      • Issue #289 Clicking Help caused Qt Creator to crash
qt-creator-ros-settings-page.png

A ROS-Industrial Collaboration with Microsoft and BMW

ROS-Industrial recently had the opportunity to collaborate with Microsoft, BMW and Open Robotics on an automation solution that was featured in Season 3 of the Decoded show on YouTube. This enabled the ROS-I Consortium to realize sustainable gains on the team’s vision for greater efficiency and visibility with respect to logistics and material management challenges in assembly plants.

Mobile Robot.png

BMW has set forth a vision where they break down the barriers between the historical automation paradigms and the challenges with interacting with the largely manual operations of their ever-increasing high-mix assembly operations. Historically, materials are delivered to the line for human operators to consume and exact quantities and status are lost at that point of consumption. The idea is to leverage intelligent autonomous operation to give better visibility to what is where, while leveraging cloud technologies to create a tighter loop and connection to the order delivery systems. The goal is to enable a leaner operational buffer within the workflow, reducing carried inventory and driving greater efficiency.

This is where Microsoft came into the picture, with their Azure solutions and client support team to do rapid development sprints to enable tighter coupling between their SAP work order environment to the “to-be deployed” autonomous robotic fleet.

BMW has built a home-grown start transport robot that runs on the same battery used in the i3 model car they produce. However, there needed to be coordination of these assets over the long term with a richer simulation environment as the platform’s capability increased. This is where the Microsoft team came to deliver.

Close Up.png

Originally, the Southwest Research Institute (SwRI) ROS-I team support was around navigation and evaluation of Gazebo in manufacturing environments supporting many mobile robots. It became clear as the Microsoft team got to work that Gazebo would not support spinning up multiple mobile robots in one instance. In fact, due to how Gazebo is structured, even a handful of robots brought Gazebo processing to a crawl. This led to the implementation of Argos, an open-source simulator that also has the capability to include physics as the scale BMW was interested in. A container strategy was developed and, in the end, the ROS-I team ended up learning quite a bit from the Microsoft team through the week-long development hack on SwRI’s campus. This development week really furthered the understanding of capability with regard to richer simulation capabilities, including the physics, which supports the ROS-I vision of tighter process performance and management of non-rigid bodies within the planning cycle.

As things got going in Germany, as the Decoded episode shows, the Microsoft team worked closely with BMW’s ROS and Manufacturing Execution System (MES) developers, tightly coupling the SAP functionality along with the fleet management and navigation tuning functionality that was required along with the Argos implementation. In the end, this led to a functional, if not sustainable, solution for the BMW team as they continued to refine the performance of the specific ROS-based robot, leveraging the Azure environment to assure SAP to simulation, to robot action within the Azure platform.

VPLogistics.png

We hope those who watch this Decoded episode agree this demonstrates that collaborations between for-profit entities such as Microsoft, and nonprofits – such as SwRI, Open Robotics and Fraunhofer IPA, as well as open-source projects such as ROS-I – can enable end-users such as BMW to create their own sustainable, high performing solutions. We believe the open-source contributions will enable others to leverage the development and hopefully expand the capability. This idea of a pre-competitive foundation that enables interoperability and flexibility without generating silos is key if we are to move the ball forward with respect to operational efficiency gains at the scale we need.

As we have seen in recent months, robotics development and IT are entering a new phase, where more teams and individuals can grasp their own destiny and ROS-Industrial is excited to be a part of that journey!

Thanks to Microsoft and BMW as well as the Decoded team for making this project possible.

Notes from the ROS-Industrial Consortium-Americas Training Workshop at SwRI

The ROS-I Americas consortium hosted its third training event of 2018 at Southwest Research Institute on November 13-15. The three-day workshop was led by Dr. Josh Langsfeld, Joseph Schornak, and Michael Ripperger and attended by 16 participants hailing from diverse backgrounds across academia and industry.

The curriculum was split into Basic and Advanced tracks, and alternated between slide-driven lectures and hands-on coding exercises. On Day 1, students in the Basic class were introduced to core ROS concepts like nodes, messages, services, and parameters, while the Advanced class developed a Python-enabled perception pipeline and explored introspective debugging tools.

Dinner following the Day 1 session was hosted at La Gloria in San Antonio’s Pearl district, where attendees contended with six courses of tacos and tortas.

The two classes reunited on Day 2 to develop a perception-driven planning application for a simulated robotic workcell. On Day 3 students were free to independently pursue a variety of open-ended lab exercises, including developing a more advanced pick-and-place application and adapting their programs from previous exercises to run on an actual UR5 robot.

1fb2c62c-9083-43d9-a9f8-ef61694208b2-original.jpeg

The ROS-I initiative is currently working on a migration to Melodic that will include the training exercises and curriculum for an initial unveiling in early 2019. Also under consideration for early 2019 is an introductory ROS2 advanced topic.

Please reach out to us if you have any suggestions for improvements or additions to the lectures or exercise. We really enjoyed working with you and we hope to see you again!

ROS-Industrial Training (UK) (1st edition)

The first UK edition of the ROS-Industrial Training took place last week hosted by re.je at UCL Here East 5 - 9 November 2018. The target is to boost participants from industry with little or no ROS knowledge to a level with broad comprehension of the robotic framework. The theoretical content is consolidated with workshops and is applied on simulated and physical mobile and industrial robots.

Agenda:

  • Day 1: Introduction to Linux
  • Day 2-4: ROS basic training (introduction, manipulation, navigation)
  • Day 5: Advanced ROS topics (statemachine-based programming tools; build, test, release workflow and tools)
The next edition of the training will take place in Q1 2019. Please watch the re.je, ROS-Industrial and ROSIN websites for more details.

The inaugural ROSCon JP (Japan) in 2018

The first ROSCon JP opened in Tokyo on Friday, September 14, 2018 with much anticipation. It was an amazing sell-out crowd with 200 participants from the community. ROS-Industrial Consortium Asia Pacific was honored to participate and present our efforts in the industrial space to the conference attendees.


The one-day event was not dampened by the early morning rain and the room was quickly filled up with excited participants to hear the keynote speech from Brian Gerkey (CEO Open Robotics) and various prestigious speakers from industry, such as TORK, JAXA, Toyota Research Institute, Honda Research Institute, Hitachi and various universities.

ROSCONJP Logo-s.jpg
Fig 1.  Presentations by various speakers in the fully packed seminar room

Fig 1. Presentations by various speakers in the fully packed seminar room

Fig 2. Brian Gerkey (CEO, Open Robotics) presents on ‘Future ROS’

Fig 2. Brian Gerkey (CEO, Open Robotics) presents on ‘Future ROS’

Brian’s speech envisioned how ROS2.0 will be designed to address the future needs of industry and commercial users. This is extremely important to us as an anchor for our work in the ROS-Industrial Consortium and will soon prove its value in our development work.

fig 3. Presentation from Open RoboticS on ROS 2.0

fig 3. Presentation from Open RoboticS on ROS 2.0

Our presentation looks through the lens of Asia Pacific as we started our journey for ROS-Industrial and what challenges were faced in developing the communities in the region and the need to gain their confidence and trust on using open source software.

Fig 4.  Nicholas Yeo representing ROS-Industrial Consortium - Asia Pacific provided the insights on ‘ The Journey of building ROS-Industrial initiatives in Asia Pacific’

Fig 4. Nicholas Yeo representing ROS-Industrial Consortium - Asia Pacific provided the insights on ‘ The Journey of building ROS-Industrial initiatives in Asia Pacific’

The most highly anticipated talk was nevertheless from Sony Corporation and their ‘kawaii’ robot dog – Aibo. The generation 2 Aibo is running on ROS with Amazon Web Services. Sony has made a significant stride to shift from internal proprietary solution to embrace open source. The crowd was amazed by the demonstration by Sony Corporation.

Fig 5.  Sony Corporation presented on the ‘Use of ROS in Aibo and Sony's efforts

Fig 5. Sony Corporation presented on the ‘Use of ROS in Aibo and Sony's efforts

FIg 6.  The adorable Aibo on the demonstration booth by Sony Corporation

FIg 6. The adorable Aibo on the demonstration booth by Sony Corporation

The event has demonstrated the passion and interests from the Japanese communities. It is very exciting to feel from the community that ROS will continue to evolve and adopted in the coming years. I like to thank the executive committee team (Jeff Biggs, Egashira Hirokazu, Yukiko Nakagawa, Yutaka Kondo) for organizing this wonderful event. I will be looking forward to the development of ROS in this Community. The ROS-Industrial Consortium - Asia Pacific will continue to explore how we can strengthen and develop the ecosystem with our Japan partners.

FiG 7. Group photo with the ROSCON Japan committee and participants

FiG 7. Group photo with the ROSCON Japan committee and participants

For more information about the event and presentations please visit: http://roscon.jp/

IMTS 2018 – Leveraging Open Standards and Technologies to Re-Imagine Interoperability within Factories

The International Manufacturing Technology Show (IMTS) held in Chicago brings together technology solution providers, innovators, thought leaders and interested parties seeking to leverage innovations, and improved capabilities to impact their operations and bottom line. The IMTS 2018 Emerging Technology Center (ETC), hosted by the Association for Manufacturing Technology (AMT), a team consisting of Southwest Research Institute, AMT, and Vimana presented a practical demonstration showing a NIST grant-funded initiative leveraging open technologies to facilitate interoperability between manufacturing equipment team members. The demonstration was supported by Hurco Companies, Hexagon Manufacturing Intelligence, and Universal Robots. The intent was to show how a new version of the previously developed ROS-to-MTConnect bridge can be extended along with an extended MTConnect architecture, to enable a many-to-many interoperability that seeks to enable more intelligent interoperability.

Figure 1. Demo at the ETC at IMTS

Figure 1. Demo at the ETC at IMTS

MTConnect is an open, royalty-free standard intended to foster greater interoperability between controls, devices and software applications, by publishing structured data over networks, using the TPC/IP. The initial project, mentioned above, demonstrated the ability to implement ROS-Industrial to enable the execution of robot paths and use the MTConnect protocol for communications between the robot and a CNC machine tool. Similar to the previous effort, this new solution is primarily software based and leveraged the open standard application level protocol, MTConnect, and the open source Robot Operating System (ROS) Industrial to enable facility-level interoperability between robot teams and machine-cell devices.

The expansion of the previous ROS/MTConnect solution, further enhances the viability of using industry supported open source software for smart manufacturing applications. Open source software permits a continuation of free development, over a very large development workspace that ultimately solves complex problems where the solution is free to the end user. The output from this project is intended to enable industry-wide adoption of open source technologies, by providing a use-case and testbed showcasing lower cost solutions for comprehensive factory floor integration for the small- and medium-sized manufacturer. In parallel, it is anticipated that this work will foster and/or inspire other solution providers to incorporate this approach to leverage and incentivize both the leverage of open standards/open source as well as further refine their capabilities to align with the vision further moving the ball forward, enabling a future state where dynamic agile execution may be realized.

At IMTS

demonstration at IMTS was intended to show the type of operation or intelligence that may be deployed by leveraging this new approach to interoperability. As seen in Figure 2, the intent is to enable a robot, leveraging ROS/ROS-Industrial, to communicate with other types of manufacturing equipment that already take advantage of the MTConnect standard, as far as communicating what they are doing.

Figure 2. Robot able to “understand” what the other team members need and/or are doing.

Figure 2. Robot able to “understand” what the other team members need and/or are doing.

We are also demonstrating the ability of a robot to perform more than material handling tasks. The robot can also tend the machines’ need for a replacement tool or a coolant change. The MTConnect standard is providing the language to allow the equipment to express its needs, from the movement of material to the maintenance tasks required to keep the cell at top performance. The architecture for device orchestration and collaboration provides the framework for allowing multiple manufacturing processes to coordinate their activities to complete a task. The task-based models and the coordinator models can be seen in Figure 2. This architecture enables the ROS-I platform to provide the ability for the robot to find optimal ways to dynamically move material and other assets where they need to be.

This framework is inherently extensible. For instance, industrial AI will be an essential addition to future capability, enabling the notion of autonomous, continuous improvement or dynamic optimization through learning. Plans can be previewed as conditions change and subject matter experts that choose to intervene to ensure consistency in value stream performance can also be additional input into this Industrial AI capability.

Figure 3. State Model Architecture

Figure 3. State Model Architecture

Along with this software and supporting architecture will be a simulation test environment. This will enable testing of various scenarios. Within the scope of the current project is testing the scalability of the current developed state that was developed at IMTS 2018. This will include multi-robot scenarios to ensure the software and architecture support these use cases and of course support future iterations as we seek to extend the capability beyond one robot servicing to, say, one or two fixed assets.

Figure 4. Simulation Environment

Figure 4. Simulation Environment

Both the software and the virtual test environment will be made fully open source with documentation. In parallel, the team is excited to see test beds assembled to enable further testing with hardware in the loop supported by additional research, non-profit, and even for-profit entities. The goal is to enable a future state where dynamism may be managed on-the-fly by enabling intelligent devices to effectively share information and act on it, both leveraging information from order to deliver systems, best practice rules, along with developments around industrial AI noted above. As we move forward towards smart, agile manufacturing, and we reduce the risk of inventory and static supply chains. Our ability to rapidly deploy equipment and repurpose it will be vital to expanding our industry and allowing for more customization and productivity.

Figure 5. Multi-robot agility and self-optimization and organization

Figure 5. Multi-robot agility and self-optimization and organization

What was the reception at IMTS 2018?

Overall, there were a lot of questions regarding the scope of the work: what it means, when is it available, and how do average end-users take advantage?

The ETC was covered by team members from all organizations that contributed, and each brought a different perspective to what may be realized, what we would like to see next, and what specifically they heard from those they spoke with at the ETC.

Shaurabh Singh of AMT provided his insights:

> “The ETC cell was received very well. Many were appreciative of the non PLC distributed cell concept. A couple of them had a centralized master control in their cells and were stuck as their engineer who configured it had left. The peer-to-peer network-based interaction between different cell equipment was an attractive solution to them. For a lot of them it was about getting aware of different capabilities of MTConnect and ROS. Engineers asked a lot of technical questions including about MTConnect Interfaces Pub-Sub implementation, flexible collaboration models and edge computing/intelligence. On one hand, people were interested in the machine intelligence and task priority abstraction; on the other hand, they asked questions about low-level ROS dynamic path planning. Most of them were interested in where the project was headed and when it would get into the market. A lot of discussions were also on the ERP-machine level integration, process planning, part specifications and machine capabilities which are already ongoing parallel development in the MTConnect Standard Committee.”

Figure 6. Crowd Visiting the ETC demonstration

Figure 6. Crowd Visiting the ETC demonstration

Josh Langsfeld of SwRI, Lead ROS Software Developer:

> “I thought the response to the project and the technology was quite positive. It was interesting to see the widely varying perspectives different people had when seeing our demo. Some were just interested in the idea of a robot doing machine tending at all while others were more interested in how the CMM could automatically get its output to change the CNC program. People who were aware of MTConnect and had used it before were excited about its potential for defining tasks between multiple devices. I think the idea of having mobile robots servicing and tending to a whole factory floor of various machines on demand was an especially compelling vision for how this technology can be put to practical use. We have a long way to go before reaching that, especially on the robot side since we'll have to really make use of and scale up some of the advanced planning and sensing capabilities of ROS-I. The potential is definitely there though, and I'm excited to see how this work continues to develop in the future.”

Matthew Powelson of SwRI, ROS Developer and Integration/Debug:

> “I think the reception on the whole was good. Several people were excited about it, and said things like "can you help me do this?" or "my customers really want this." People really seemed to like this idea of doing cool demos like they saw in the Fanuc booth, but doing it without having to buy all yellow robots because we are using open standards and open code. However, there were some outliers that I think are important. First, I remember one representative from a robot OEM say that this kind of open code would commoditize machine tools, and that was their biggest fear. Another man didn't like the idea of decentralized intelligence. He described some of the projects he had worked on over his 25-year career and just said that "one centralized controller isn't really that bad." Shaurabh Singh, of AMT, make a good point about how this manufacturing space is going through the same sort of transformation that the IT space did 10 years ago, where you see established companies like Microsoft now releasing code open source because it actually makes things> more > accessible and safe – not less. Open software and open standards have obvious advantages to the end user, but we still need to keep making the case to the OEMs.”

I think the team’s insights are interesting and my interactions with those that came by the ETC were very similar. It was exciting to interact with such a diverse audience, in the context of what their business is, what they sell or are looking to buy, and/or where they operate or are based. This diversity is part of the challenge when we talk about simpler interoperability, the simple “plug it in and it works”. I believe this is a simple, compelling, yet “lot to do” vision, and I hope you will stay engaged as this work moves forward.

We will seek to have all the software open source by the end of October 2018 in the MTConnect GitHub repository. There are plans to have a physical test bed established so that both industry interested parties as well as NIST and other research organizations can continue to further the capability. Please let us know if you have any questions, or would like to learn more. A detailed final report as well as follow-on presentations will be upcoming, and we will announce those via our typical communication vehicles. In the meantime, please keep the dialog going. We always look forward to questions and feedback!

UPDATE!

The software and the demonstration simulation are now available over at:

World ROS-I Day – What went down and what it means

The ROS-Industrial global community organized and pulled together the first World ROS-I Day, inspired by the successes of World MoveIt! Day, but focusing on the repositories relevant to ROS-I. We had five sites signed up to host, and we meet up in virtual collaboration rooms as well. The event kicked off from Asia, led by our friends at ARTC in Singapore, ROS-I Asia-Pacific, handing off to the EU, hosted by Fraunhofer IPA, then finally to the Americas, hosted by SwRI’s ROS-I Team located at LiftOff, thanks to PlusOne Robotics, in San Antonio, Texas.

The Asia-Pacific team was joined by a team based in Delhi, India, and the U.S. team was also joined by the Open Robotics team in Mountain View, California. Though these were the meet ups that we were aware of, there were a number of additional teams and individuals that contributed, and the ROS-I developing teams have been happy to see the level of engagement, and hope we can maintain this level of participation moving forward.

ROS-I Asia-Pacific Meet Up

ROS-I Asia-Pacific Meet Up

Monitor View of U.S. Location (LiftOff, San Antonio, Texas) and EU Location  (Fraunhofer IPA)

Monitor View of U.S. Location (LiftOff, San Antonio, Texas) and EU Location

(Fraunhofer IPA)

Developers Working on Issues

Developers Working on Issues

We were excited to get to work and do some clean up with the ROS-I teams on the various repositories and packages, working on a predefined list of issues. The ROS-I development teams started with a list of repositories as the focus of the inaugural World ROS-I Day on July 11. This enabled for making of a manageable scope and allowed for tracking of progress throughout the day.

The repositories included for the event were:

On the ros-industrial organisation:

  • ros_industrial_issues

  • industrial_training

  • industrial_ci

  • industrial_moveit

  • industrial_core

  • ros_qtc_plugin

  • robotiq

  • universal_robot (except driver infrastructure)

  • ros_canopen

Also over at the ros-industrial-consortium organization:

  • Descartes

  • Descartes tutorials

The maintainers worked up reviewing the repositories and classifying the issues and including them in an issue board. This really enabled the organization and workflow for the event. We had designated assigners for issues and leveraged an IRC chat room to do assignment of issues and enable efficient communication for remote teams. Within the rooms we had to be careful managing that we did not do redundant work. Though a few times issues were worked by multiple developers, for the most part, the event went off well considering the logistical challenges that can be encountered when working across multiple time zones around the world.

Over the 22 hours that World ROS-I Day was in-flight, 31 issues were closed and there are, at this time, 16 Pull Requests to be reviewed, with a handful of assigned issues still being worked, per the issue board.

WRID18 Issue Board

WRID18 Issue Board

Overall, the ROS-I team, and we hope all those that leverage ROS-Industrial repositories, feel this was a worthwhile and a quite successful event. Traffic statistics indicate that the repositories of interest saw sustained activity even after World ROS-I Day.

Views and Uniques-WRID.jpg
Unique Visitors WRID.jpg

Moving forward, we are working to continue the momentum. This will manifest itself in an effort to improve the monthly ROS-I Developers’ Meeting, which we have been working to socialize via the ROS-I Discourse Category at https://discourse.ros.org/t/ros-i-developers-meeting/5047/18. Furthermore, we are excited to continue our support of World MoveIt! Day this October. And, of course, we are excited to begin the planning for a follow World ROS-I Day, and are looking forward to getting the feedback from the developing community for how to make this event more meaningful and engaging.

It is an exciting time in open-source software as an interest is advancing in industrial and manufacturing automation capability. We look forward to continue engaging the development community and to provide avenues for collaboration. ROS-I set out to be an open-source project to bring the power of ROS to industrial applications. Along the way, it has at times, been difficult to bring the developing community to these same applications. We hope through engagement and meaningful collaboration events, we can build that community, and we hope you look forward to laying those foundational pieces with us.

Keep an eye out for updates on planning/details for World MoveIt! Day, and of course the second annual World ROS-I Day, potentially in the first quarter of 2019, but of course, we are open to timing recommendations. Thanks for your continued interest in open-source for industrial automation!

ROS-I Consortium Americas Hosts Training Event in Seattle, Washington

The ROS-Industrial Americas consortium hosted its second training event of 2018 in Seattle on July 17-19, attended by 15 students from companies across various industries. The three-day event, hosted by Levi Armstrong and Michael Ripperger from Southwest Research Institute, featured a basic and advanced track where participants were able to explore ROS-related topics from ROS architecture and communication to motion planning, perception, and code testing.

New to this particular training session was the inclusion of a Python node within the Perception Pipeline to enable the understanding of C++ and Python node interaction. Additional content around RVIZ GUI creation and debugging tools were also featured. To explore these new modules and the rest of the training content, check out the ROS-I training wiki here.

The training event was a combination of lecture and hands-on coding and hardware demos. At the end of the event, students were able to interact with provided UR5 robots and test the code they created on the robots. Overall the training was a great opportunity to learn more about ROS and network with ROS-I Consortium members, or their partners, in the robotics field leveraging ROS in their own applications.

Thank you to our hosts in the Emerald City and to all of our attendees for making this class a great success.

A third ROS training event for this year is currently in the works for the fall, so stay tuned for more details! Also, don't forget, full Consortium members are able to host ROS-I Consortium Americas training events, such as this event in Seattle. As always, do not hesitate to offer feedback relative to how training can be improved to meet your needs. We are always interested in member and community feedback!

Optimization Motion Planning with Tesseract and TrajOpt for Industrial Applications

Summary

Southwest Research Institute launched an internal R&D project to integrate the existing motion planner TrajOpt (Trajectory Optimization for Motion Planning) into ROS. TrajOpt was created at UC Berkeley as a software framework for generating robot trajectories by local optimization. The integration of TrajOpt necessitated new capabilities that spawned the creation of several new packages: tesseract and trajopt_ros. The tesseract package contains a new lightweight motion planning environment designed for industrial application, while the trajopt_ros package contains packages specific to TrajOpt. We will demonstrate how these tools complement existing planners, like OMPL or Descartes, to solve complex problems quickly and robustly.

Description

The original implementation of TrajOpt was developed using OpenRave for Kinematics and Bullet for contact checking. The first step was to replace OpenRave with MoveIt! Kinematics and second replace the collision environment with MoveIt!’s Collision environment. Early on in the process several limitations were found in both MoveIt!’s Kinematics and Collision environment.

TrajOpt requires the ability to calculate specific information about the robot not provided by MoveIt!, but it turns out KDL, which is one of the kinematics libraries used by MoveIt!, provides methods for obtaining the required information. This resulted in the development of a custom kinematics library built on KDL for both kinematic chains and non-kinematic chains. Secondly TrajOpt leverages specific characteristics of convex to convex contact checking to provide the minimum translation vector to move two objects out of collision. In the process of integrating with MoveIt!, it was determined that it did not provide detailed distance information. Also, after further evaluation it was found that MoveIt! does not support convex to convex collision checking requiring significant API changes across multiple repositories. Since the IR&D was time-sensitive, it was determined to not use MoveIt! and create a light-weight Motion Planning Environment (Tesseract).

The Tesseract environment was designed to support SwRI’s work in complex industrial motion planning applications where flexibility and modularity are key to adapting to new applications. Packages include:

  • tesseract_core – Contains platform agnostic interfaces and data structures to be used. tesseract_ros –ROS implementation of the interfaces identified in the tesseract_core package, currently leverages Orocos/KDL libraries.
  • tesseract_collision – ROS implementation of a Bullet collision library. It includes both continuous and discrete collision checking for convex-convex and convex-concave shapes.
  • tesseract_msgs – ROS message types used by Tesseract.
  • tesseract_rviz –ROS visualization plugins for Rviz for both the environment state and trajectories.
  • tesseract_monitoring – Provides tools for monitoring the active environment state and publishing contact information. This is useful if the robot is being controlled outside of ROS, but you want to make sure it does not collide with objects in the environment. Also includes the environment monitor, which is the main environment facilitating requests to add, remove, disable and enable collision objects, while publishing its current state to keep other ROS nodes updated.
  • tesseract_planning – Contains interface bridges between Tesseract Environment and motion planners OMPL and TrajOpt.

After the creation of Tesseract all necessary capabilities were available to finish the integration of TrajOpt into ROS. The new motion planner was evaluated against the following use case while minimizing joint velocity, acceleration and jerk cost along with collision avoidance cost:

  • Fully Constrained Cartesian Path
  • Semi-Constrained Cartesian Path
  • Free Space Path
  • Semi-Constrained Free Space Path
  • Free Space + Constrained Cartesian Path

Only a few of the above case will be discussed below. The first is a Semi-Constrained Free Space Path where a KUKA iiwa needs to plan around a sphere while maintaining tool orientation with the z-axis rotation free to rotate. Each step of the optimization is shown in Figure 1. Note that once the robot is out of collision the remaining iteration are spent minimize joint velocity, acceleration and jerk.

Figure       SEQ
Figure \* ARABIC      1         - KUKA iiwa (7 DOF) TrajOpt free space planning round sphere

Figure 1 - KUKA iiwa (7 DOF) TrajOpt free space planning round sphere

The next use case was a complex industrial application. It is an 8 DOF problem, where the robot picks up a seat off of a conveyor and loads the seat into a car. This is a very challenging task, given the amount of manipulation required to pass through the doorway and set the seat without colliding with the car structure. The final trajectory, found in 0.482 seconds using TrajOpt with continuous collision checking enabled, is shown in Figure 2.

Figure       SEQ
Figure \* ARABIC      2         - TrajOpt Car Seat Installation Example

Figure 2 - TrajOpt Car Seat Installation Example

One significant advantage of the Tesseracts implementation of the Bullet Collision Library is the ability to perform continuous collision checking. An example demonstrating the use of continuous collision with TrajOpt is shown in Figure 3. Each red box represents a state in the trajectory with the green box being the collision object to avoid during planning. Under discrete collision checking each state would be found to be collision free even if the motion between states was not. With continuous collision checking enabled, it can be seen that a collision is detected when transitioning between state 2 and state 3 resulting in a collision-free trajectory.

Figure       SEQ
Figure \* ARABIC      3         - Planar Box (2 DOF) TrajOpt free space planning with continuous collision checking

Figure 3 - Planar Box (2 DOF) TrajOpt free space planning with continuous collision checking

The remaining application to discuss is a complex Semi-Constrained Cartesian path with 437 poses each with 5 Degrees Of Freedom (DOF) fixed and the tool z-axis free to rotate. The application is performing a deburr operation on a complex puzzle piece shown in Figure 4. Also the problem includes a 7 DOF robot and a 2 DOF positioner for the spindle making it a non-fixed base kinematic chain. This requires the use of Tesseract’s joint kinematic model developed for this particular use case. The TrajOpt motion planner problem contains roughly 3,000 constraints and was able to solve in 4.4 seconds.

Figure       SEQ
Figure \* ARABIC      4         – Semi-Constrained Cartesian Planning Problem

Figure 4 – Semi-Constrained Cartesian Planning Problem

TrajOpt has shown itself capable of solving very difficult problems but, as an optimization, it can be sensitive to its initial conditions. This presentation will explore several strategies for seeding the solver, including the integration of sampling planners like OMPL and Descartes. These planners can coarsely and quickly sample the space of a problem to generate candidate solutions that can be refined by TrajOpt. If that should fail, a new robot configuration is selected from the sampled problem-space and the process repeated. For some problems this results in a planner finding a true global optimum.

References

  1. Tesseract Repository
  2. TrajOpt ROS Repository
  3. Examples (Tesseract and TrajOpt ROS)
  4. Videos

Driving Intelligent Inspection Processes for NDE/NDI

Nondestructive evaluation (NDE) techniques for parts and structures that are created from either forming processes or additively manufactured processes typically have had to be manually performed. Recent advances in scanning technologies and intelligent path planning tools, such as those available within ROS -Industrial, present a platform capable of performing multiple NDE processes in an intelligent fashion leveraging the Scan-N-Plan framework.

The Sensors Systems & Nondestructive Evaluation team within Southwest Research®, along with ROS-Industrial developers have conceived of a concept to enable a Scan-N-Plan approach to Nondestructive Inspection (NDI) thereby reducing the amount of inspection time by only performing detailed surface or volumetric inspections where mandated by an initial higher level screening.

The Sensors Systems & Nondestructive Evaluation team has developed a unique technology within their portfolio that performs full volumetric inspection using a guided wave technique that leverages an omni-directional probe. The concept would be to perform a macro-level scan leveraging the MsT360 Guided Wave Sensor, analyzing the output and for indications, or areas of interest, drive low level inspection. A sample output from this stationary sensor can be seen below. This output can then be leveraged to generate process paths for follow on inspections such as Eddy Current (EC) or Ultrasonic Testing (UT), only in the areas of interest. This provides the full volumetric inspection output, but reduces the time spent doing a 100 percent surface scan with higher resolution traditional UT. This improves solution velocity as well as reduces post-processing clean up.

Based on experience with the MsT360 follow-on low level inspection techniques can be selected based on the characterized potential defect. The system would then Plan trajectories based on this scan for each process, including the stand-off or contact forces and trajectories mandated by the process. A map could be visualized and colored to aid the operator in understanding which regions will be inspected by which NDE process. In the below example for instance, a process would be planned for the area identified with the 35% wall area, or where weld 1 and 2 are located, as opposed to UT or EC of the entire volume or surface respectively.

Scan output.JPG

Sample output from MsT360

A sample workflow for this type of operation would be as such: • Assembly or Piece within a robotic system is ready for NDE Assessment • Locate MsT360 Guided Wave Sensor (Manually or Robotically) • Perform macro volumetric inspection • Analyze output • For areas of concern plan trajectories for each process type (EC or UT) and execute inspection in the areas of interest • Tool Change • Perform inspection • Tool Change if required • Clean • Generate Report; either unload or launch a repair process

The flexibility inherent to this design and the capability included enable processing of piece-parts, such as forms, as well as fabricated structures. The macro scan and detailed path planning approach is more efficient for when it is not inherently value added to inspect an entire volume or surface area.

Planned Region.JPG

As can be seen in the above image, specific regions can be highlighted based on an output such as described earlier, or these can be user adjusted based on a response from a developed GUI. These trajectories can be planned for all the processes of interest, in this case UT and EC, with their requirements relative to process execution included. Leveraging the flexibility of the Scan-N-Plan framework it is easy to see how NDE processes can either be included into a multi-process cell, or just automated in a manner that effectively applies them where required, or where that detailed scrutiny is most beneficial. Furthermore, this can be augmented with visual imaging to allow for assessment of human readable details , or human generated markings to drive further process planning and inspection. For future consideration as well are several visual NDE techniques for surface inspection that could be incorporated, such as dye penetrant, magnetic particle, pulsed thermography, etc.

Human Markings.JPG

Example of Identification and Tool Path Plans for a Human Generated Indication

As advances in both inspection technologies and automation move forward there lie the opportunities to more tightly couple these processes, and optimize their effectiveness in the areas of greatest need. This enables more efficient utilization while optimizing cost and throughput, as well as reducing non-value-added steps that currently come with many of the NDE techniques. Moving forward we hope to see more opportunities for integrated quality evaluation within automation applications, enabling further optimization of manufacturing value streams. For more information about SwRI’s NDE solutions please visit: https://www.swri.org/industries/sensors-systems-nondestructive-evaluation-nde

A brief report from the ROS-Industrial EU Spring workshop

Thanks to the participation of several members of the ROS-Industrial community in Europe and of new participants interested in knowing more on the topic, we are happy to report on a successful ROS-Industrial EU Spring’18 Workshop at Fraunhofer IPA.

During the two days of May 28-29, organizations within the ROS-Industrial consortium had the opportunity to present their current development projects to an audience of technical experts.

Erle Robotics, represented by Irati Zamalloa, presented the concept of a common interface for the description of robot modules and hardware sub-components allowing their reusability, composability and interoperability. The project is already publicly available as a ROS2.0 implementation. During the ensuing Q&A, feedback from participants pointed to the possibility of integrating such effort with a MDE approach, where the information could be semantically annotated and verified. Further feedback is welcome, with a github repository setup for the purpose.

DSC_0002.JPG

The second workshop session focused on ways to make ROS-based systems and PLCs interoperable. Sebastian Friedl from the University of Stuttgart gave an introduction to the OPC-UA architecture and the improvements that his institution is making on this open source implementation of the protocol. He also presented the ROS <-> OPC-UA gateway that is under development within the SeRoNet project.

On a similar topic, Tiago Pinto from INESC TEC continued the session showing how his team approaches communication between CODESYS and ROS systems. The implemented wrapper is already being used for practical application within the project ScalABLE 4.0, with source code scheduled for public release in August.

DSC_0004.JPG

Wrapping up the topic, Ludovic Delval from Faunhofer IPA presented a survey of the existing ROS drivers implementations supporting different fieldbus protocols. The detailed overview was appreciated by the participants and is available as reference here.

As part of the efforts of the ROSIN project to make ROS-Industrial better and business-friendlier, partners of the consortia joined for the second day of the workshop to show the tools developed for the quality assurance on the ROS software development. Jonathan Hechtbauer from Fraunhofer IPA and Anthony Remazeilles from Tecnalia gave an update of the software efforts like the improvements on the quality badge, the rosinstall time machine tool, and a generator for ROS packages. Preliminary code is available at the github organization of the project.

Concluding the event, the ROSIN coordinator Carlos Hernández Corbato informed the audience about the opportunities for ROSIN Focused Technical Projects. A grant up to 100K to fund your software development is available to institutions with a legal seat in the EU and associated countries. The application process is explained in detail at the project page.

Given the request from interested parties who could not join us in Stuttgart, we made the content and slides of this workshop publicly available under the following link.

We look forward to a second edition of the workshop in the fall!

New Release of the ROS Qt Creator Plug-in

We are pleased to announce the release of the ROS Qt Creator Plug-in for Qt Creator 4.5.1 on Trusty and Xenial. The ROS Qt Creator Plug-in creates a centralized location for ROS tools to increase efficiency and simplify tasks.

Highlights:

  • The installation has changed from using a debian installation method to using the Qt Installer framework. This change is to facilitate tighter integration with existing ROS capabilities and libraries within Qt Creator.
image1.png
  • A set of new video tutorials were contributed by Nathan George broken down into five parts:
    • Installation
    • Import, Build, and Run Settings
    • Create Hello World C++
    • Building Hello World
    • Indexing, Auto Complete and Code Style
  • Updated wiki using Sphinx and GitHub Pages to provide a richer wiki.
  • In an effort to make it simpler when using the dugger within Qt Creator for ROS an “Attach to unstarted process” run step was created as shown below.
image2a.png
  • A set of existing ROS templates were added to simplify adding ROS specific files within Qt Creator.
image3.png
  • Additional changes
    • Show hidden files/folder like .clang-format and .rosinstall.
    • Support for catkin tools partial build capabilities.

First ROS-Industrial Consortium Americas Training for 2018

The ROS-Industrial Consortium Americas recently completed another successful ROS-I training event April 10-12 at Southwest Research Institute. We had 11 people from a variety of industries complete the Basic and Advanced training material, which covers a variety of topics from creating a ROS node, publishers, subscribers, and using the ROS parameter server. The Advanced training material covered motion planning using Stomp and Descartes, as well as creating a computer vision application using the Point Cloud Library (PCL).

On Day 1, the Basic and Advance track groups met separately, but were merged together the remaining days. The training material was cumulative so that by the end of Day 2, participants had a working ROS project that could move a robot to a simulated box location. On Day 3, everyone got to try their hand at a more challenging vision or path planning project. Many were able to complete the challenge and execute it on one of the available robots.

20180412_085406.jpg

Attendees also got a short tour of SwRI's lab space that included hardware for application testing and development. By request, a subset of attendees requested a breakout session where they were able to see a side-by-side comparison of some of the 3D depth sensors available, as well as a new surface reconstruction technique, taking advantage of the proximity of the lab space and the development hardware available.

20180412_084254.jpg
20180412_084818.jpg

Thanks to everyone who came and helped make this event great! If you have questions about ROS-Industrial training, please feel free to contact us. Keep an eye out for the next ROS-Industrial Consortium Americas event at the Events page!

Leveraging Scan-N-Plan for Additive Manufacturing

Implementing Additive Manufacturing (AM) principles with a collaborative twist is still an unexplored area of manufacturing. Traditional AM typically involves step motors moving a printing head, or a laser bonding a material to itself. More advanced solutions have even gone to mounting print heads as robotic end effectors to gain a larger print volume. A ROS framework has already been developed in this space, though additional features are always of interest.

Often when a part is damaged it is either thrown out or repaired with direct labor. What if there was a way for autonomous blending part repair with AM? Assuming a known CAD model, a Scan-N-Plan AM solution could bring new life to previously scrapped parts. The method proposed here involves bringing in a damaged part, doing a laser scan and determine which elements of the part need repair.

Laser scans produce good resolution of parts and is largely insensitive to material and surface quality though some cases may require several scans to achieve high resolution. A pre-scan process can easily assume this role to reduce print downtime. The scan in the image below was done on a FARO arm. As can be seen, in the image below, the quality is excellent. The file exports a point cloud that can be imported into the ROS framework. Similarly, a non-contact structured light sensor could provide the output as seen below, whereby subsequent process planning could be driven.

Crack.jpg

Once the file is scanned, the part is then checked against a master CAD file. The deviation of the point clouds indicates where the flaw exists. The point cloud is then converted to a YAML file and the path is generated. Locating the part between scan and print head is done using 3 known touch off points or a non-contact form of localization. The material deposition process is then free to take place. For cases such as the above example where prep-work is required to enable material deposition, a cutting or grinding tool can be process planned and executed as well to create a suitable prepared region for material deposition to occur.

Additive Value Stream Reman.png

In remanufacturing use cases, specifically where the variation is high due to the unknown state of the provided material, a more efficient workflow with higher return and repurposing of field return material is possible. The leverage of a ROS-based framework in a distributed manner could enable common intelligence being applied over a range of assets residing at multiple sites, making process control more uniform wherever the operation takes place. Furthermore, as in the case in remanufacturing, as opposed to additive processing (building up) a complete part, leveraging the Scan-N-Plan framework enables optimization of applying only where the additive process delivers the most value in the broader context of the value stream and cost of both incoming materials and the processing itself. An example of this would be the replacement of a linkage forging on a structure with an additive deposit, where the remaining structure is fabricated plate material, the additive can be optimized to both manage load (print direction and properties), while optimizing costs where commodities do the job and are readily available at favorable costs.

Issues that could arise with this process include interference with the print head with the part (oddly shaped flange interfering with a robot wrist independent of the print head) and cable management for certain types of deposition processes. Though material waste is minimal with advances in additive near-net processing, there is the cost and maintenance of the media and encompassing all of the tools that need to be accessible to the manipulator, from the QA assessment, material prep/finish/surface treatments, and of course the AM process itself. Underside printing would be a second print unless an auxiliary positioner is implemented, such as in some of the more recent hybrid additive/subtractive platforms.

Physical attributes of a repair part are untested based on metallurgical properties. Part repair is still a fledging industry of research, though there have been recent advances in aerospace remanufacturing. Cohen has done some work with tissue engineering and still uses a 3-axis step motor . Great advances still stand to be made in this research area, however a number of tools exist to create an agile process leveraging the benefits of additive and approaches like Scan-N-Plan.