Setting OnClickListener on RecyclerView itself not on item

mRecyclerView.addOnItemTouchListener(new RecyclerView.SimpleOnItemTouchListener() {
    @Override
    public boolean onInterceptTouchEvent(RecyclerView rv, MotionEvent e) {
        return true;
    }
});

Then just handle the type of MotionEvent you want. Return true if you don't want the RecylerView doing anything else with it.