We're here to talk about Continuous Profiling in Production: What, Why, and How. We're going to talk a little bit about why performance tools matter, why you might need tools to understand performance problems in your application stack. We're going to talk about the differences between a development environment and a production environment, and why just looking at things in development might not be the right way to understand performance problems. Then look at a few different approaches at the high level as to how we can solve these things.
Firstly, at a very high level, why do performance tools matter? Well, firstly, when you think about software development problems, there are three broad categories of things that you get, and this applies to performance problems and performance aspects of your software as much as others. The first one's Known Knowns. So these are theoretically the easy aspects of your software. You know that reading data out of your main memory is going to be faster than reading data off a hard disc. You can have that in your mental model; that's something that you can confidently bank on.
Then you've got your Known Unknowns. These are things, aspects of your software system, where you know that there could be a potential for performance problems there, but you don't know how big of an issue it is. For example, every time you've got an unbounded size of Q in a production system, you don't know whether it'll be a performance problem, but you know it could be an issue there. You know that's there's this main memory disk speed ratio difference, but you don't know how big it is until you actually look at it in some particular systems. So these are things that you may not know all the details on, but you can plan ahead for. You can mentally model, you can think through.
Then there are the Unknown Unknowns. They're real spanner-in-the-works problems. These don't necessarily have to be hard or complicated, but they could be something like, you've spent months building a beautiful, well-thought-through scalable parallel algorithm, and then you've got a sequential logger in the middle of this system that's slowing everything down to the speed of one thread, and you didn't think about that. You wouldn't necessarily think about those kind of issues ahead of time until you hit them in production, until they're real problems for your system. The things that you, maybe with hindsight, you would think, "Oh yes, I'd get that next time." But until you hit it, it's an Unknown Unknown.
A lot of the other aspects of software performance can be planned ahead. You can architect and design for Known Knowns and for Known Unknowns, but Unknown Unknowns, you're going to need to have something that measures how that system actually works to understand how you can solve those performance problems.
Read more at:
/Service Ventures Team
Comments