What I learned from Artur Bergman in less than 18 minutes

Thanks to @lounibos retweet from OreillyMedia on Twitter, I recently came across this presentation from Artur Bergman given at Velocity EU Berlin last month.  To be totally honest, I had previously never heard of Artur Bergman, but I had heard of his company Fastly (www.fastly.com).
Artur Bergman presents “Full Stack Awareness” at Velocity EU
In his presentation (above) you learn how to consider that good system performance engineering requires you to have a comprehensive understanding of the complete system, end-to-end, from “kernel to continents” as he says. Watch the video to glean a whole set of tips and tricks on Linux performance...and you will also hear a nearly flawless use of vernacular language which I found quite refreshing. Some of my reflections:
  • People still don’t understand how computers work. This is great news for me as a performance engineer and tester, because that means I will continue to be employed.  But this is bad news because we aren’t making any headway on improving the state of the industry’s maturity around system performance.
  • There are a few new cool tools to use for Linux performance work, like latencytop, which might be even better than powertop, and “sar –B” or “sar –r” for memory diagnosis -- and also don’t forget that you might need to dig into the source code to find the root cause of a performance limitation.
  • Don’t forget that Linux kernels were created first as desktop operating systems and there is still a tremendous amount of code that still isn’t optimized for massively scalable server optimization – look for NUMA limitations, single-threaded I/O drivers and virtual memory management assumptions.
  • Correlation is not causation: you might not be aware that just because you correlate activities between system performance metrics or measures graphed on a mutual timeline, doesn’t absolutely mean that they are actually linked causally. This includes misleading bottlenecks:  like increased disk I/O is not a disk bottleneck, if there’s a swap condition due to limited memory.
Thank you Artur for your contributions to performance, your honesty, integrity and f’ing serious commitment to f’ing performance!  You can follow him at @crucially and find the rest of the crew at Fastly.