RecyclerView updates only on scroll

try specifying the position of the item inserted or removed instead of notifyDataSetChanged();. you can use

notifyItemInserted(int pos);
notifyItemRangeChanged(int start, int end);`

Most likely you are calling notifyDataSetChanged from background thread.

Simply (and ugly) fix you can do is to use:

recyclerView.post(new Runnable(){ @Override public void run(){ adapter.notifyDataSetChanged(); } });

If notifyDataSetChanged is called correctly from main thread then the list is updated without touching it.