Jquery fadeOut/fadeIn callback not working

The problem is that the .fadeOut() callback is called once per animated element, not once at the end. You could modify your code to keep a counter of how many times it has been called, but far easier - assuming at least jQuery 1.6 - is to use .promise(), which will resolve after all the associated animations complete:

$(document).ready(function () {
    var $lis = $("li.slider"),
        start = 0;

    $lis.hide()
        .slice(start, start + 4)
        .show();

    $("button").click(function () {
        $lis.slice(start, start + 4)
            .fadeOut(500)
            .promise()
            .done(function () {
                start += 4;
                if (start > $lis.length) {
                    start = 0;
                }
                $lis.slice(start, start + 4).fadeIn();
            });
    });
});

Demo: http://jsfiddle.net/w7Yuk

I made a couple of other changes to your code, e.g., caching the jQuery object with the li elements, and removing the "end" variable.

I created a nice little jsFiddle demo that modifies what you had and gets you a nice smooth transition:

HTML:

Give button an id of "next" so that you can target it specifically, in case there are other buttons on page.

<ul>
   <li class="slider"> Item-1 </li>
   <li class="slider"> Item-2 </li>
   <li class="slider"> Item-3 </li>
   <li class="slider"> Item-4 </li>
   <li class="slider"> Item-5 </li>
   <li class="slider"> Item-6 </li>
   <li class="slider"> Item-7 </li>
   <li class="slider"> Item-8 </li>
   <li class="slider"> Item-9 </li>
   <li class="slider"> Item-10 </li>
   <li class="slider"> Item-11 </li>
   <li class="slider"> Item-12 </li>
   <li class="slider"> Item-13 </li>
   <li class="slider"> Item-14 </li>
   <li class="slider"> Item-15 </li>
   <li class="slider"> Item-16 </li>
</ul>

<button id="next"> Next </button>

CSS:

Start both off with display none so we can fade them in nicely on load.

.slider { display: none; }
#next { display: none; }

jQuery:

I like to cache elements, so I started off by doing that. I then fade in both the first 4 LI elements and the next button. I use the recommended handler of .on() to bind the click event of the next button. After we set start and end we call .fadeOut() on the next button and the current 4 LI elements. Now, the reason your callback is screwy is due to the fact that their is a callback for every element in your selector ( so 4 times ). Instead, we need to use .promise() to wait for all of them to complete as a whole and then we can call the .fadeIn() method on both the next button and the next 4 LI elements. Just a side note, I use .stop(true,true) to eliminate any animation queuing that there might be.

var $list = $("ul li");
var $next = $("#next");
var start = 0;
var end = 4;

$next.fadeIn(500);
$list.slice(start,end).fadeIn(500);

$next.on("click", function() {

  start += 4;
  end += 4;

  if( start >= $list.length ){
    start = 0;
    end = 4;
  }

  $next.stop(true,true).fadeOut(500);
  $list.stop(true,true).fadeOut(500);

  $list.promise().done(function() {
    $list.slice(start,end).stop(true,true).fadeIn(500);
    $next.stop(true,true).fadeIn(500);
  });

});