Ubuntu Summit 2023 Day 3 (05 November 2023)
bpftrace recipes: 5 real problems solved by Trent Lloyd
Let's walk through how you, as an operations engineer, can identify and solve 5 real problems using bpftrace to kickstart you on the way to using this technology to solve your next mystery while also learning a ""fun"" level of detail about how traditional and bpftrace based tools work at a low level and how their performance impact compares as a result.
Perhaps even more than you, as a Support Engineer at Canonical I often do not have the luxury of modifying the software under observation, additionally, many difficult to diagnose problems manifest as outlier cases requiring us to statistically measure and correlate requests in ways that don't already exist.
Historically this can be challenging as sampling those outliers requires specific debug/analysis code to be added and systems to be restarted. Instead, dynamic runtime tracing combined with BPF in the form of bpftrace allows us to load very small and fast programs into the kernel that run in the hotpath and summarise or analyse exactly the events we need, transmitting only a very small amount of data out of the kernel to be analysed in userspace. This instrumentation is installed at runtime with no changes to the system or a substantial impact on system latency or performance.
While such programs can be written as more complex and clunky C+Python scripts, 'bpftrace' allows us to write these in a nice Domain Specific Language (DSL) that combines the in-kernel data collection and userspace analysis components into a single coherent script that is writable and understandable even by those who may not be software engineers or kernel experts.
Link to talk details and author bio: events.canonical.com/event/31...
Remember to Subscribe and watch more videos on our KZread channels:
Ubuntu OnAir: / ubuntuonair
Ubuntu: / ubuntuos
And follow our other social accounts:
LinkedIn - bit.ly/3Jw6jGN
Twitter - bit.ly/3OXSIJE
Facebook - bit.ly/3Q15Yyn
Instagram - bit.ly/3vE7Kxk