Natural Language to SQL: State of the Art and Open Problems

Translating users’ natural language queries (nl) into sql queries (i.e., nl2sql) can significantly reduce barriers to accessing relational databases and support various commercial applications. The performance of nl2sql has been greatly improved with the emergence of large language models (LLMs). In this context, it is crucial to assess our current position, determine the nl2sql solutions that should be adopted for specific scenarios by practitioners, and identify the research topics that researchers should explore next. In this tutorial, we will provide a comprehensive overview of nl2sql techniques, covering every aspect of its lifecycle, from the collection and synthesis of training data, recent advancements in nl2sql translation techniques using LLMs and agents, debugging nl2sql processes, to multi-angle and scenario-based evaluation of nl2sql methods. We conclude by highlighting the research challenges and open problems in nl2sql.