4-digit password with unique digits not in ascending or descending order

As you have already worked out, there are $^{10}P_4=5040$ passwords that repeat no digit.

From this number we are to subtract those passwords whose digits are all increasing or all decreasing. All such passwords can be generated by picking four digits out of the ten without regards to order – there are $\binom{10}4=210$ ways to do so – and then arranging them in increasing or decreasing order as required. Since we have two choices of order, we subtract $210\cdot2=420$ passwords.

Hence there are $5040-420=4620$ passwords with unique digits that are not all increasing or all decreasing.

There are $\binom{10}4$ ways to select $4$ distinct digits.

Under the condition that there is no increasing and no decreasing they can be arranged in $4!-2$ ways.

So that gives a total of:$$\binom{10}4\left(4!-2\right)$$ possibilities.