Big oh of $(n+1)\sqrt{n-1}$
Yes, $O((n+1)\sqrt{n-1}) = O(n^{3/2})$.
It should be noted that technically $(n+1)\sqrt{n-1}$ is Big Oh of any function that grows at least as fast as $n^{3/2}$. Thus $(n+1)\sqrt{n-1}$ is $O(n^{1000})$ but not $O(n)$. However, I've seen some CS people use Big Oh notation as if it were Big Theta.
More strongly, $(n + 1)\sqrt{n - 1} \sim n\sqrt{n};$ to wit, the limit of the ratio is 1.