Online weakly paired similarity learning for surface material retrieval

Purpose: For robots to more actively interact with the surrounding environment in object manipulation tasks or walking, they must understand the physical attributes of objects and surface materials they encounter. Dynamic tactile sensing can effectively capture rich information about material properties. Hence, methods that convey and interpret this tactile information to the user can improve the quality of human–machine interaction. This paper aims to propose a visual-tactile cross-modal retrieval framework to convey tactile information of surface material for perceptual estimation. Design/methodology/approach: The tactile information of a new unknown surface material can be used to retrieve perceptually similar surface from an available surface visual sample set by associating tactile information to visual information of material surfaces. For the proposed framework, the authors propose an online low-rank similarity learning method, which can effectively and efficiently capture the cross-modal relative similarity between visual and tactile modalities. Findings: Experimental results conducted on the Technischen Universität München Haptic Texture Database demonstrate the effectiveness of the proposed framework and the method. Originality/value: This paper provides a visual-tactile cross-modal perception method for recognizing material surface. By the method, a robot can communicate and interpret the conveyed information about the surface material properties to the user; it will further improve the quality of robot interaction.