V2R: FMCW radar data synthesis from videos for long range gesture recognition

The increasing popularity of wireless sensing applications has led to a growing demand for large datasets of realistic wireless data. However, collecting such data is often time-consuming and expensive. To address this challenge, we present a novel Video-to-Radar (V2R) framework that generates synthetic mmWave radar data for human gestures using unstructured videos. The V2R framework combines a mesh fitting model to extract 3D spatial features of human subjects from videos, with an FMCW radar physics model to simulate realistic radar signals. This approach enables the generation of diverse synthetic data, which can be used to train and validate gesture recognition models. By incorporating the V2R-generated data, we demonstrate a significant improvement in the classification accuracy of our LSTM-based gesture recognition model, with the overall accuracy improvement of 12%.