Wordpress - Sticky Posts exceed posts per page limit

Here is an approach to account for sticky posts by getting the number of sticky posts (if any) and include that in the calculation posts_per_page parameter:

add_action('pre_get_posts', 'ad_custom_query');
function ad_custom_query($query) {

    if ($query->is_main_query() && is_home()) {

        // set the number of posts per page
        $posts_per_page = 12;
        // get sticky posts array
        $sticky_posts = get_option( 'sticky_posts' );

        // if we have any sticky posts and we are at the first page
        if (is_array($sticky_posts) && !$query->is_paged()) {

            // counnt the number of sticky posts
            $sticky_count = count($sticky_posts);

            // and if the number of sticky posts is less than
            // the number we want to set:
            if ($sticky_count < $posts_per_page) {
                $query->set('posts_per_page', $posts_per_page - $sticky_count);

            // if the number of sticky posts is greater than or equal
            // the number of pages we want to set:
            } else {
                $query->set('posts_per_page', 1);
            }

        // fallback in case we have no sticky posts
        // and we are not on the first page
        } else {
            $query->set('posts_per_page', $posts_per_page);
        }
    }
}

Edit

In the case where the number of posts per page we wish to set is less than or equal to the number of sticky posts, I have set the posts_per_page to one and that will result in 13 or more posts $sticky_count + 1 (in this case) only on the first page (subsequent pages will have 12 posts). Maybe that is OK since this case is rare and +1 post on the first page may not be that significant.

This is because Wordpress will display all sticky posts first and on one page (the first page) even if their count is greater than the posts_per_page parameter, so we set the posts_per_page in this case to the minimal amount possible which is 1, because 0 and negative values will disable the posts_per_page parameter and that will make Wordpress to display all posts on the first page.

There an issue if the sticky posts are in the first page.

The solution is to decrement the sticky post count for the sticky posts that are part of the first page.

function fix_posts_per_page_with_sticky_posts( $query ) {

    if ( $query->is_main_query() ) {

        // set the number of posts per page
        $posts_per_page = 12;

        // get sticky posts array
        $sticky_posts = get_option( 'sticky_posts' );

        // get queried post ids array
        $ids = array();
        $args = array(
            'post_type' => 'post',
            'post_per_page' => $posts_per_page,
            'paged' => 1
        );

        $posts = get_posts( $args );

        foreach ( $posts as $post ) {
            $ids[] = $post->ID;
        }

        // if we have any sticky posts and we are at the first page
        if ( is_array( $sticky_posts ) && ! $query->is_paged() ) {

            // count the number of sticky posts
            $sticky_count = count( $sticky_posts );

            foreach ( $sticky_posts as $sticky_post ) {
                if ( in_array( $sticky_post, $ids ) ) {
                    // decrement sticky posts count if the sticky post in on the page
                    $sticky_count--;
                }
            }

            // and if the number of sticky posts is less than
            // the number we want to set:
            if ( $sticky_count < $posts_per_page ) {
                $query->set( 'posts_per_page', $posts_per_page - $sticky_count );

            // if the number of sticky posts is greater than or equal
            // the number of pages we want to set:
            } else {
                $query->set( 'posts_per_page', 1 );
            }

        // fallback in case we have no sticky posts
        // and we are not on the first page
        } else {
            $query->set( 'posts_per_page', $posts_per_page );
        }
    }
}
add_action( 'pre_get_posts', 'fix_posts_per_page_with_sticky_posts'  );

I hope it will help