Generating semantically valid adversarial questions for TableQA
Adversarial attack on question answering systems over tabular data (TableQA) can help evaluate to what extent TableQA systems can understand natural language questions and reason with tables. However, generating natural language adversarial questions is difficult, because even a single character swap could lead to huge semantic difference in human perception. In this paper, we propose SAGE (Semantically valid Adversarial GEnerator), a Wasserstein sequence-to-sequence model for white-box adversarial attack on TableQA systems. To preserve meaning of original questions, we apply minimum risk training with SIMILE and entity delexicalization. We use GumbelSoftmax to incorporate adversarial loss for end-to-end training. Our experiments show that SAGE outperforms existing local attack models on semantic validity and fluency while achieving a good attack success rate. Finally, we demonstrate that adversarial training with SAGE-augmented data can improve performance and robustness of TableQA systems.