How to find out when next() reaches the end, then go to the first item

Determine the .next() ahead of time by checking its length property.

$('.nextSingle').click( function() {
       // Cache the ancestor
    var $ancestor = $(this).parent().parent().parent();
       // Get the next .newsSingle
    var $next = $ancestor.next('.newsSingle');
       // If there wasn't a next one, go back to the first.
    if( $next.length == 0 ) {
        $next = $ancestor.prevAll('.newsSingle').last();;
    }

    //Get the height of the next element
    var thisHeight = $next.attr('rel');

    //Hide the current element
    $ancestor.animate({
            paddingBottom:'0px',
            top:'48px',
            height: '491px'
        }, 300);

        //Get the next element and slide it in      
    $next.animate({
            top:'539px',
            height: thisHeight,
            paddingBottom:'100px'
        }, 300);
});

By the way, you could replace .parent().parent().parent() with .closest('.newsSingle') (if your markup allows it).

EDIT: I corrected the thisHeight to use the $next element that we referenced.

according to the jquery documentation, an empty jquery object will return of .length 0.

so what you need to do is check for the return when you call .next, and then call :first

http://api.jquery.com/next/

As a useful reference, the following is a function you can write and include:

$.fn.nextOrFirst = function(selector)
{
  var next = this.next(selector);
  return (next.length) ? next : this.prevAll(selector).last();
};

$.fn.prevOrLast = function(selector)
{
  var prev = this.prev(selector);
  return (prev.length) ? prev : this.nextAll(selector).last();
};

Instead of:

var $next = $ancestor.next('.newsSingle');
   // If there wasn't a next one, go back to the first.
if( $next.length == 0 ) {
    $next = $ancestor.prevAll('.newsSingle').last();;
}

It would be:

$next = $ancestor.nextOrFirst('.newsSingle');

Reference: http://www.mattvanandel.com/999/jquery-nextorfirst-function-guarantees-a-selection/