when is the particle speeding up and when is it slowing down

You've got the right idea, but not the right execution. Let's call the direction of positive $x$ "up" and the opposed direction down.

It is moving upward--has a positive velocity--on $(0,1)$ and $(3,4)$. It is moving downward--has a negative velocity--on $(1,3).$ Put another way, the graph's height is increasing on $(0,1)$ as we move to the right--likewise in $(3,4)$--and decreasing on $(1,3)$ as we move to the right.

It is accelerating upward--has a positive acceleration--on $(2,4)$. It is accelerating downward--has a negative acceleration--on $(0,2)$. Put another way, the graph's slope is increasing on $(2,4)$ as we move to the right, and the graph's slope is decreasing on $(0,2)$ as we move to the right.

Now check where the signs match.