Better front end performance in Drupal
Update 2012-10-21:
My site is now powered by Jekyll and located in this repository.
This is how I handle things on this site:
Apart from using aggregation (CSS and JavaScript) and minification (CSS) in the bandwidth optimization settings (admin/config/development/performance) here are a few more things that you can do.
Exclude unnecessary CSS files
Inspired by the Tao theme I've excluded a couple CSS files from core and contrib. They're only causing damage by increasing the amount of kb served. And not just be simply existing but also forcing me to override them in my theme if they weren't excluded.
Use minified core JavaScript files
Speedy is an easy to use module that provides minified versions of core JavaScript files that are not already minified.
Minify theme JavaScript files with UglifyJS
Make sure all the JavaScript files in your theme are minified using UglifyJS.
Decrease the number of HTTP requests
By using hook_css_alter() and hook_js_alter() I'm combining all the CSS and JavaScript files into just one CSS and one JavaScript file. This results in four less HTTP requests on every page load for me on this site.
See source code for CSS and JavaScript.