Analysing the Analysers: An investigation of source code analysis tools

Context: The primary expectation from a software system revolves around its functionality. However, as the software development process advances, equal emphasis is placed on the quality of the software system for non-functional attributes like maintainability and performance. Tools are available to aid in this endeavour, assessing the quality of a software system from multiple perspectives. Objective: This study aims to perform a comprehensive analysis of a particular set of source code analytical tools by examining diverse perspectives found in the literature and documentations. Given the vast array of programming languages available today, selecting appropriate source-code analytical tools presents a significant challenge. Therefore, this analysis aims to provide general insights to aid in selecting a more suitable analytical tool tailored to specific requirements. Method: Seven prominent static analysis tools, namely SonarQube, Coverty, CodeSonar, Snyk Code, ESLint, Klocwork, and PMD, were chosen based on their prevalence in the literature and recognition in the software development community. To systematically categorise and organise their distinctive features and capabilities, a taxonomy was developed. This taxonomy covers crucial dimensions, including input support, technology employed, extensibility, user experience, rules, configurability, and supported languages. Results: The comparative analysis highlights the distinctive strengths of each tool. SonarQube stands out as a comprehensive solution with a hybrid approach supporting static and dynamic code evaluations, accommodating multiple languages and integrating with popular Integrated Development Environments (IDEs). Coverity excels in identifying security vulnerabilities and defects, making it an excellent choice for security-focused development. CodeSonar prioritises code security and safety, offering a robust analysis. Snyk Code and ESLint, focusing on JavaScript, emphasise code quality and standards adherence. Klocwork is exceptional in defect detection and security analysis for C, C++, and Java. Lastly, PMD specialises in Java, emphasising code style and best practices.