RFID-based and real-time lip movement recognition system.

• [TMC 2022] “Hearme: Accurate and real-time lip reading based on commercial rfid devices”, Shigeng Zhang, Zijing Ma, Kaixuan Lu, Xuan Liu, Jia Liu, Song Guo, Albert Y Zomaya, Jian Zhang, Jianxin Wang

In this project, we design and implement HearMe, an accurate and real-time lip-reading system built on commercial RFID devices. HearMe can be used to accurately recognize different words in a pre-defined vocabulary without limitations in light conditions and can be used in multiple user scenarios by leveraging RFID’s ability in identifying different users. Experimental results show that HearMe detects speaking actions of the user with an accuracy higher than 0.95 and recognizes different words in a 20-words vocabulary with an average accuracy higher than 0.88.