OverOps’ new SonarQube integration brings together the best in static and dynamic analysis to help your team shift left and ensure code is always production-ready.
Most of us are very familiar with the famous Mark Zuckerberg declaration to “move fast and break things.” That was a very bold statement back in 2014, and, as Zuckerberg quickly realized, probably too bold. He later retracted it, stating his new desire to move fast, yet stay stable.
Today, while speed is certainly vital to remaining competitive in the digital business landscape, quality holds equal weight. Organizations not only need to be first to market with innovative new products and features, but their applications need to work seamlessly 100% of the time. Any impact on the customer experience can mean diminished brand reputation, lost sales and decreased revenue.
As the move towards increased development velocity continues to stress software quality, OverOps is more dedicated than ever to providing our customers with features and integrations that make it easy to deliver quality software at speed.
Today we’re excited to announce our new integration with SonarQube, arming our users with the combined power of static and dynamic code analysis to shift left, preventing poor quality code from ever reaching production. Below, we share a bit more about our integration and the market conditions driving the need for both tools.
The Rule of Ten and the Importance of Shift Left Testing
There is a notion in the software development industry known as “The Rule of Ten” that states the cost of finding and fixing software defects increases by 10X, as compared to an issue detected in the prior stage of quality assurance. This means that the implications of code errors that make it all the way to production are enormous, especially if the organization isn’t able to detect and resolve issues quickly.
To better address code quality (without compromising speed) many organizations are investing heavily in “shift left” or “shift left testing” initiatives, emphasizing the use of automation for earlier detection and resolution of code defects in pre-production. Static code analysis tools like SonarQube and dynamic analysis solutions like OverOps have emerged as critical components of this approach.
Next, we’ll dig into the differences between these two categories of tools and how they work together.
Static Analysis vs. Dynamic Analysis
To quote one of OverOps’ solution engineers, let’s use a familiar analogy to articulate the differences between static and dynamic analysis tools:
“Static code analysis is analogous to practicing your baseball swing with a practice net and a pitching machine. There are minimal surprises. After a few swings, you know exactly where the ball is going to be every time. This helps to work on fundamentals and to make sure that you have good form. While this helps with improving your game, it can only get you so far.
“Dynamic code analysis is more like practicing your swing against a live pitcher with variation in the types and locations of each pitch. It tests not only your fundamentals, but your ability to react to different, unexpected situations. When done in production, it’s like perfecting your swing at the bottom of the 9th with the bases loaded. Did I mention that the score is tied with 2 outs? The stakes are high.”
Now to put this in software terms, static analysis tools like SonarQube examine an application’s source code against a given set of rules or coding standards before a program is run. SonarQube users are able to detect code vulnerabilities and code smells (i.e., any characteristic in the source code that could indicate a deeper problem), and ensure adherence to commonly accepted coding standards.
It also provides “Test Coverage” reports that describe the degree to which the code has been exercised. SonarQube then enforces these rules with quality gates. For example, if test coverage is below 80% on new code, SonarQube will mark the build as failing.
While static analysis does a good job of catching many issues, these tools are restricted by their reliance on foresight. You can only detect what you build a test case for, missing out on all the runtime activity that happens in the background. This means that even if testing indicates 100% code coverage, that doesn’t mean 100% of critical issues were identified.
Static code analysis is great for providing an automated pair of eyes for code review to detect bad coding practices and security issues, but it doesn’t address what happens when code actually executes. This means it lacks critical context related to developer intent and production failure scenarios that are difficult to anticipate.
Dynamic analysis helps fill this gap. Tools like OverOps analyze the code as it executes to detect runtime errors that would otherwise be missed by static analysis tools – all without relying on any foresight.
Shift Left with SonarQube + OverOps
The OverOps plugin for SonarQube helps users leverage the combined power of these two methodologies to ensure code quality. By feeding OverOps data directly into the SonarQube platform, users are able to enhance their existing quality gates with insight into runtime errors.
So how does it work?
1. Identify Critical Errors
When SonarQube users install the OverOps plugin, it automatically creates an OverOps event rule for Java code based on the following quality gate conditions:
- New – what are the new errors that didn’t exist in previous releases?
- Critical – what are the most urgent errors to address? (e.g. NullPointer, IndexOutOfBounds, etc.)
- Resurfaced – which previously resolved errors are happening again?
- Unique – how many unique errors are happening?
The thresholds for these metrics can be tailored to which issues are most significant to the user’s application. As users run their tests, OverOps simultaneously analyzes the code as it executes to identify runtime errors.
2. Block a Release When Critical Errors Are Detected
When any of the thresholds for the above OverOps quality gates are exceeded, SonarQube will fail the release. Users can then immediately view runtime issues in their SonarQube dashboard and see which quality gates failed and the severity of the issue, as well as access a link directly to the OverOps event analysis.
3. Get the Complete Context Required to Resolve
OverOps event analysis then arms developers with full error context including the stack trace,, variable state, system state and more – without requiring foresight or code changes. With this rich data, developers can quickly reproduce runtime issues, resolve them and promote the code without significant impact to release schedules.
The OverOps plugin for SonarQube is simple to set up and comes with out of the box settings that can be easily customized to best meet a user or team’s needs. The metrics can be added into existing quality profiles and can be configured at any level – global, project, command line, etc. – to accommodate varying requirements and fit into the user’s larger CI/CD pipeline. Critical exception types can be easily customized based on implications within a user’s environment.
By combining dynamic analysis and static analysis, developers can continue to increase velocity while simultaneously increasing code quality. Organizations are empowered to leverage shift left testing, reducing cost while producing a higher quality product. The addition of OverOps data in your SonarQube instance allows you to identify, prevent and resolve critical issues before your customer is ever impacted.
To see the OverOps SonarQube integration and our other plugins in action, sign-up for a free trial or join us for a webinar where we’ll present a demo and discuss static vs. dynamic analysis in more detail.