Jquery change CSS class on clicking

Wrong usage of $.each()

Use this way:

$(".btn").click(function () {
    $(".btn").removeClass("course-btn-tab-selected");
    $(this).addClass("course-btn-tab-selected");        
});

Should do the trick:

$(".btn").click(function () { 
        $(".course-btn-tab-selected").removeClass("course-btn-tab-selected").addClass('course-btn-tab');                 
        $(this).removeClass('course-btn-tab').addClass("course-btn-tab-selected");        

    });

You do not need to use each here, on click of element with class btn remove class for all elements with class btn and assign the desired class to current element (referred by $(this)) which is event source. Also I assume you want to remove selected class from previous elements.

$(".btn").click(function () { 
     if($(this).hasClass("course-btn-tab-selected"))
          $(".btn").removeClass("course-btn-tab-selected").addClass("course-btn-tab");               
     $(this).addClass("course-btn-tab-selected");        
});

Edit: You can improve this by hold the last selected element and changing it class if it suits you.

previouslyClicked = $(".btn").eq(0); //Assuming first tab is selected by default
$(".btn").click(function () {       
     previouslyClicked.removeClass("course-btn-tab-selected").addClass("course-btn-tab");                 
     $(this).addClass("course-btn-tab-selected");
     previouslyClicked = $(this);           
});