How to make RecyclerView scroll smoothly?

The typical source of "jerky" scrolling in Android is the app taking too much time on the main application thread updating the UI. In the case of RecyclerView, this would mean taking too much time in onBindViewHolder() or possibly in onCreateViewHolder(). Those each need to return in sub-millisecond times, meaning you cannot do disk I/O or network I/O in them.

think I found the problem. I am using holder.photo.setImageBitmap(BitmapFactory.decodeFile(itemList.get(position).get‌​AbsolutePath())); on onBindViewHolder() and the image file is around 100kb each with about a dozen images in the list

Yes, that will be doing disk I/O and image decoding on the main application thread. That will be slow enough to cause jank in the UI (i.e., "jerky" scrolling).

Consider using an image loading library, like Picasso or Universal Image Loader, as they can populate your ImageView from the bitmap asynchronously.

This sample app uses Universal Image Loader to help populate the contents of a RecyclerView (with GridLayoutManager), with the data source being the available videos on the device.

Add this wherever you have declared RecyclerView in your Activity or Fragment

RecyclerView mRecyclerview = (RecyclerView) findViewById(...);
mRecyclerview.setNestedScrollingEnabled(false);

setNestedScrollview(false) does the work for you.