Wordpress - What's the purpose of the paged.php file?

If we look in template-loader.php, we can see the conditions under which paged.php will be loaded:

if ( defined('WP_USE_THEMES') && WP_USE_THEMES ) :
    $template = false;
    if     ( is_404()            && $template = get_404_template()            ) :
    elseif ( is_search()         && $template = get_search_template()         ) :
    elseif ( is_tax()            && $template = get_taxonomy_template()       ) :
    elseif ( is_front_page()     && $template = get_front_page_template()     ) :
    elseif ( is_home()           && $template = get_home_template()           ) :
    elseif ( is_attachment()     && $template = get_attachment_template()     ) :
        remove_filter('the_content', 'prepend_attachment');
    elseif ( is_single()         && $template = get_single_template()         ) :
    elseif ( is_page()           && $template = get_page_template()           ) :
    elseif ( is_category()       && $template = get_category_template()       ) :
    elseif ( is_tag()            && $template = get_tag_template()            ) :
    elseif ( is_author()         && $template = get_author_template()         ) :
    elseif ( is_date()           && $template = get_date_template()           ) :
    elseif ( is_archive()        && $template = get_archive_template()        ) :
    elseif ( is_comments_popup() && $template = get_comments_popup_template() ) :
    elseif ( is_paged()          && $template = get_paged_template()          ) :
    else :
        $template = get_index_template();
    endif;
    if ( $template = apply_filters( 'template_include', $template ) )
        include( $template );
    return;
endif;

The last elseif is where the paged template is loaded if it exists:

elseif ( is_paged()          && $template = get_paged_template()          ) :

Which means that all of the checks above have to return false for the paged.php template to be loaded, the query is_paged and no other more content-specific template was found.

Yup, if you have paged.php present in your theme, that template will be used for all but the first page of the archive. This is in case the styling/markup for your archive is drastically different between the first page, and subsequent pages.