Wordpress - Stop WP from creating "Sample Page" and "Hello World!" post

Create a file install.php in your wp-content directory. In that file you declare the wp_install_defaults() function. If you just leave that function empty no default categories, links, posts or pages will be created.

The default (pluggable) wp_install_defaults() function can be found in wp-admin/includes/upgrade.php.

Also see: https://stackoverflow.com/questions/550086/how-to-delete-the-default-hello-world-post-before-wordpress-install

If you're using Multisite

The accepted answer is destructive in that it cancels all other set-up items in the overridden function. A less destructive way to do it for multisite installs is to delete the default content during new blog creation by hooking in to wpmu_new_blog

add_action( 'wpmu_new_blog', 'delete_wordpress_defaults', 100, 1 );

function delete_wordpress_defaults(){

    // 'Hello World!' post
    wp_delete_post( 1, true );

    // 'Sample page' page
    wp_delete_post( 2, true );
}