Why does it seem everything I push moves at a constant velocity?

It's not easy pushing something with by hand with a constant force greater than kinetic friction.

Try using a rubber band and a ruler to pull something across the table with a constant force. I think if you focus on keeping the rubber band stretched a constant amount while you pull you will notice the object will accelerate.

I am aware that a constant force causes a constant acceleration but friction can counteract this.

True. Although any non-zero net force acting on an object causes it to accelerate. The net force does not have to be constant in time for acceleration to happen.

However, if I push something across a table, for example, it seems no matter how hard I push, the object travels at a constant velocity, even if I apply more force than the kinetic friction.

Well, the object was at rest on the table. Then you pushed it and it started moving. Therefore the velocity of the object changed, and you caused it to accelerate. The object is obviously not traveling at a constant velocity.

The object seems to always travel at the same velocity as my hand, does this mean I am not actually applying a constant force?

You probably are not applying a constant force (or maybe you are. I cannot say without being there and actually measuring the force you apply). But you are for sure applying some force which is causing the object to accelerate. Your hand would then also be accelerating while it is in contact with the object. This doesn't mean the object has a constant velocity though.

I don't know if you've learned about energy and power yet, but if you have, that leads to a pretty plausible explanation. As you push this object with force $F$ at speed $v$, the power you expend (like the horsepower of a car) is given by $P=Fv$. As you start the object in motion, it's accelerating. But this process of acceleration is limited by the power your body is comfortable supplying. To keep accelerating, you need to supply a force greater than the force of friction $F_f$, which means that $P>F_fv$. You run into a limit at speed $v=P/F_f$.

When it's not physically difficult, like moving a coin across your desk, then I don't think it's true that it moves at constant velocity, unless you choose to make it so. You could choose to flick the coin or something. This makes sense because you're not running into your power limit.