[Video] Scala in Production: 5 Advanced Techniques for Debugging Distributed Systems

 ● 01st Jul 2015

3 min read

Scala Days: 5 Techniques to Improve How You Debug Scala and Akka in Production

Unlike debugging during development stages, production debugging is focused on tracing back your application’s steps. If your system is not prepared for it in advance, there’s little you can do to understand what might have gone wrong. In development you have your IDE debugger, you can step into, step back, and pretty much travel in time with your application. When your code is deployed to production, these capabilities are lost and a new skill set is needed to achieve this level of insight into your application.

In the following talk from Scala Days Amsterdam, Tal Weiss, OverOps’s CEO, tackles exactly this issue and guides us through 5 techniques to improve how you debug servers in production.

How production grade applications solve the server debugging problem?

Method #1: Distributed logging – Transaction IDs, common pitfalls and using Thread names to your advantage.
Method #2: Preemptive jstacks – Setting triggers to call jstack programmatically and extract the data you need to solve issues that only happen when your application reaches a certain pressure.
Method #3: Extracting state with BTrace – How to hook up to a live JVM and use scripting to probe variable state and other metrics.
Method #4: Custom Java Agents – An advanced technique for instrumenting code dynamically, written in Java.
Method #5: Native Java Agents – The foundation for powerful JVM profiling and monitoring tools, platform dependent, written in C++ and provide a set of highly sought after capabilities.

Check out the video to dive in deeper into each method:

Production debugging is hard, and it’s getting harder. With architectures becoming more distributed and code more asynchronous and reactive, pinpointing and resolving errors that happen in production is no child’s game. This talk covers some essential tools and more advanced techniques Scala developers can use to debug live applications and resolve errors quickly. It explores crucial techniques for distributed debugging – and some of the pitfalls that make resolution much harder, and can lead to downtime. The talk also touches on some little-known JVM tools and capabilities that give you super-deep visibility at high scale without making you restart it or attach debuggers.
For the full list of available videos, check out the Scala Days 2015 Amsterdam Parleys channel.


The current state of production grade applications requires refining the methods for working with systems in high scale. What this means in practice is that you need to make sure development teams weave this mindset into your production environment, making it resilient and able to supply all information needed to solve any type of error in the minimum time possible.

Alex is the Director of Product Marketing at OverOps. As an engineer-turned-marketer, he is passionate about transforming complex topics into simple narratives and using his experience to help software engineering navigate their way through the crowded DevOps landscape.

Troubleshooting Apache Spark Applications with OverOps OverOps’ ability to detect precisely why something broke and to see variable state is invaluable in a distributed compute environment.
Troubleshooting Apache Spark Applications with OverOps

Next Article

The Fastest Way to Why.

Eliminate the detective work of searching logs for the Cause of critical issues. Resolve issues in minutes.
Learn More