Drupal - How do you improve Drupal performance?

Caching, caching, and caching.

Some suggestions I have previously given to a similar question on d.o.

1. Putting Varnish or another reverse-proxy in front of your http-deamon is probably the single best thing you can do.
2. During DrupalCon Copehagen, Rasmus stated that using a php opcode cache, such as APC, is one of the best things you can do to speed up PHP in general. Performance improves with newer versions of PHP. There is also additional benefits to upgrading PHP when you upgrade Drupal. From 6 to 8, Drupal will go through a major shift towards object orientation, which is also were most performance improvements happens in the newer PHP versions.
3. Memcache is a popular choice for speeding up cache, by putting the cache into memory instead of disc.
4. Panels + Caching combined with Cache actions can increase performance significantly, even for logged in users, as it supports quite complex logic.
5. The Entity Cache is a nice and zero-conf speed boost for anyone using Drupal 7.
6. Write-heavy sites have fewer "well established" solutions. Some options include.
   1. Moving frequent writes entirely, for example statistics, somewhere else, such as google analytics.
   2. Caching frequent write operations with a custom solution in something like NodeJS that will write to DB once every Xth second.
   3. Sacrifice the sacred ACID, and use a database like MongoDB. (See Berdir's comment below)
   4. Cluster your SQL-database. Do reads from one database, writes to another. This is native to D7 and Pressflow can help with that in D6.

These are notes from my experiences and might vary from what others experience. I predominantly use LAMP stack and have considered the same in my suggestions.

Thumb rules for caching that I generally follow.

1. Process Once Use Multiple Times.
2. Live with stale data when possible
3. Clear Caches infrequently and keep it very specific.
4. When possible do the changes at the lowest level in the stack. LAMP - DCCc : Linux, Apache, Mysql, PHP, Drupal Core, Contrib and custom module.

Improve Performance of a Drupal Site (In the increasing order of complexity)

5. Keep the core updated, contrib module and themes updated. Yes it matters.

6. Install APC on your server. (Moved to top based on suggestion from Letharion)

7. Page Caching : admin/config/development/performance Difference between Minimum cache lifetime and Expiration of cached pages

8. Block Caching https://drupal.org/project/blockcache_alter Caching options for all the blocks.
9. Aggregate javascript and css files - Front End Improvements https://www.drupal.org/project/advagg
10. Disable Unnecessary modules. Every module adds to the amount of code that needs to be available for a page load. And it also increases the number of lookups. Wherver possible use a generic module in place of multiple module that does specific functionalities.
11. Cache Views content - Content aware caching for Views https://www.drupal.org/project/views_content_cache
12. Disable DB logging - Use https://drupal.org/project/syslog_ng
13. Reduce 404 Errors - http://www.brokenlinkcheck.com/
14. Fast 404 Responses - https://drupal.org/project/fast_404 - Try handling at server level.
15. Client Side Validations - https://www.drupal.org/project/clientside_validation
16. Compress Image - https://www.drupal.org/project/imageapi_optimize
17. Lazy Loading of Images - Don’t load unnecessary images - https://www.drupal.org/project/lazyloader

18. Use Sprite Sheets - https://www.drupal.org/project/spritesheets

19. Set Minimum Cache Life Time Value to a higher number and use cache clearing modules to clear the caches for specific pages - Whenever I edit/update a node all the page caches for anonymous user are lost

20. Use Devel Module to watch queries.
21. Rewrite Views Queries / avoid Views if its a overkill.
22. XHProf - https://www.drupal.org/project/XHProf
23. FPM, HHVM.
24. DB Profiling and Tuning - https://www.drupal.org/project/dbtuner
25. Use Boost, don't Bootstrap DB if not required. https://drupal.org/project/boost For most of the small to medium sites Boost is good enough and you may not need Reverse Proxies or so.
26. Use CDNs - https://www.drupal.org/project/cdn Its easy to set up.
27. If your cache tables are huge use Memcached - If you can install memcached and set up RAM for it, it is not as complex as it sounds.
28. Etags - Configure Etags properly. https://developer.yahoo.com/blogs/ydnfiveblog/high-performance-sites-rule-13-configure-etags-7211.html
29. Use Reverse Proxy Server - Varnish(at-least for assets). Helps a lot if most of your users are anonymous.
30. Compressed transfer - Enable gzip compression
31. Keep Alive - Use Persistent Connections where possible.
32. Progressive JPEGS -
33. CACHING IN CODE - Eaton’s blog is awesome. http://www.lullabot.com/blog/article/beginners-guide-caching-data-drupal-7
34. Implement Cache Warming - https://www.drupal.org/project/cache_warmer - Cache Warm the pages before the end user hits them.
35. Master Slave DB Config - https://www.drupal.org/project/autoslave makes it easier for you to set up one.
36. Database Clusters - https://stackoverflow.com/questions/1163216/database-cluster-and-load-balancing
37. Load Balancers -http://en.wikipedia.org/wiki/Load_balancing_(computing)
38. Use heuristic Cache Warming - https://www.drupal.org/project/cache_graceful
39. Authenticated User Caching - https://www.drupal.org/project/authcache

Boost module is also quite useful, it creates static file caches from your webpages. It's mostly for websites with lots of Anonymous user traffic.

Boost provides static page caching for Drupal enabling a very significant performance and scalability boost for sites that receive mostly anonymous traffic. For shared hosting this is your best option in terms of improving performance. On dedicated servers, you may want to consider Varnish instead.

Apache is fully supported, with Nginx, Lighttpd and IIS 7 semi-supported. Boost will cache & gzip compress html, xml, ajax, css, & javascript. Boosts cache expiration logic is very advanced; it's fairly simple to have different cache lifetimes for different parts of your site. The built in crawler makes sure expired content is quickly regenerated for fast page loading.