IMAGINE-B5G

The proposed use case involves the collaborative assembly of a ball valve by a human operator and a UR10e robot Arm. The 5G sensor platform is mounted between the flange and the gripper of the robot, with a camera connected to it to provide a real-time video feed. This camera stream is transmitted to an edge computing device located near the robot cell, where machine learning-based object detection inference is performed. The system uses a predefined assembly graph of the ball valve and the object detection results to determine the position of the next workpiece to be assembled.  This information is then sent to a Hololens 2 augmented reality (AR) device worn by the operator. Through an AR interface, the operator can see a 3D visual overlay (arrow or bounding box) of the next workpiece to be assembled, guiding them step by step through the assembly process where they have to move and screw together the parts of the ball valve with the help of the robot. This precise visual assistance improves the accuracy and efficiency of the operator’s actions, significantly reducing the likelihood of errors. The AR assistant is designed to robustly support the
operator by providing real-time, context-aware guidance  throughout the assembly process. The camera feed, transmitted over the 5G network, ensures that the AR assistant operates with minimal latency, allowing the system to identify the current state of the assembly and virtually highlight the
next part to be handled. By projecting 3D holograms and visual cues directly onto the operator's field of view via the Hololens 2, the system reduces the need for the operator to constantly refer to external instructions, computer screens, and reducing the mental workload of having to recall it from
memory. The AR assistant not only guides the operator to the correct workpiece but also enhances situational awareness.