Toward Understanding FPGA Synthesis Tool Bugs

Field Programmable Gate Array (FPGA) synthesis tools are crucial for hardware development and AI acceleration, and their bugs could compromise hardware reliability and risk downstream applications. However, it remains unknown in understanding the characteristics of these bugs. What are the root causes that trigger bugs in FPGA synthesis tools? What are the characteristics of these bugs? What are the challenges in detecting and addressing them? This article takes the first step toward answering these questions by conducting a comprehensive study of FPGA synthesis tool bugs. We analyze 551 confirmed bugs in both commercial and open-source FPGA synthesis tools, i.e., Vivado, Quartus Prime, and Yosys, covering root causes, symptoms, bug-prone components, fix characteristics, and achieve 17 valuable findings. We find that, on average, around 46.2% of bugs result from Hardware Description Language (HDL) standard non-compliance across the three tools. However, it is hard for current formal validations to fully test HDL standards compliance. Additionally, on average, over 25.8% bugs show domain-specific optimization traits due to inappropriate optimization and mapping. Meanwhile, beyond 28% of bugs trigger unexpected behavior without clear signs, making the formulation of effective test oracles challenging. These findings help addressing FPGA synthesis tool bugs and guide further research.