Buggy #Wordpress site? I start deactivating plugins. If the website breaks further, I reactivate the plugin temporarily, but so far I've always been able to remove at least several plugins without it impacting the website at all. Usually the plugins that do impact site functionality can easily be replaced with some vanilla webcode.
Buggy non-Wordpress site? I start replacing #frameworks with vanilla webcode. It becomes easier to read, less convoluted, and it's easier to integrate the external code that really does matter (like Google Analytics code).
I call this process Subtraction. To me, it's an all-around win: the code is easier to read and faster to edit for me and future developers, and the website runs faster. Often Subtraction will solve the problems in and of themselves, or simplify the problems far enough that a solution is easy to implement (rather than trying to work around existing, conflicting webcode, which is often a problem).
Now, my first instinct on any project is to see what I can delete, and move forward from there. And I've always been glad that I took that step.