2D Spaceship movement math

Well, lets consider the realistic problem first and see why this doesn't work and how we have to differ from it. In space as long as your engines are firing, you will be accelerating. Your speed is only limited by your fuel (and in fact you can accelerate faster once you've spent some fuel because your moving less mass).

To give this model an effective maximum speed, you can consider particles in space slowing you down and causing friction. The faster you go, the more particles you're hitting and the faster you're hitting them, so eventually at some fast enough speed, you will be hitting enough particles the amount of decelerating they do exactly cancels out the amount of accelerating your engine is doing.

This realistic model does NOT sound like what you want. The reason being: You have to introduce friction. This means if you cut your engines, you will automatically start to slow down. You can probably count this as one of the unintended forces you do not want.

This leaves us with reducing the effective force of your engine to 0 upon reaching a certain speed. Now keep in mind if your going max speed in the north direction, you still want force to be able to push you in the east direction, so your engines shouldn't be cut out by raw velocity alone, but instead based on the velocity your going in the direction your engines are pointing.

So, for the math:

You want to do a cross dot product between your engine pointing vector and your velocity vector to get the effective velocity in the direction your engines are pointing. Once you have this velocity, say, 125 mph (with a max speed of 150) you can then scale back the force of your engines is exerting to (150-125)/150*(Force of Engines).

This will drastically change the velocity graph of how long it will take you to accelerate to full speed. As you approach the full speed your engines become less and less powerful. Test this out and see if it is what you want. Another approach is to just say Force of Engines = 0 if the dot product is >=150, otherwise it is full force. This will allow you to accelerate linearly to your max speed, but no further.

Now that I think about it, this model isn't perfect, because you could accelerate to 150 mph in the north direction, and then turn east and accelerate to 150 mph going in that direction for a total of 212 mph in the north east direction, so not a perfect solution.

I really do like Wongsungi's answer (with the Lorentz factor), but I wanted to note that the code can be simplified to have fewer floating-point operations.

Instead of calculating the Lorentz factor (which itself is a reciprocal) and then dividing by it, like this:

        double lorentz_factor = 1/sqrt(b);
        dv /= lorentz_factor;

simply multiply by the reciprocal of the Lorentz factor, like this:

        double reciprocal_lorentz_factor = sqrt(b);
        dv *= reciprocal_lorentz_factor;

This eliminates one floating-point operation from the code, and also eliminates the need to clamp b to DBL_MIN (it can now be clamped to 0 because we're not dividing anymore). Why divide by the reciprocal of x when you can just multiply by x?

Additionally, if you can guarantee that the magnitude of v will never exceed c, then you can eliminate the testing of b being less than zero.

Finally, you can eliminate two additional sqrt() operations by using length_squared() instead of length() in the outer if statement:

    if (new_v.length_squared() > v.length_squared())
    {
        const float c = 4;

        float b = 1 - v.length_squared()/(c*c);
        if (b < 0) b = 0;

        double reciprocal_lorentz_factor = sqrt(b);
        dv *= reciprocal_lorentz_factor;
    }

This may only make a 0.1% difference in speed, but I think the code is simpler this way.

Take a page from relative physics, where objects cannot exceed the speed of light:

(See below for my working C++ code snippet and running demo [Windows only].)

1. Set the constant c to the maximum speed an object can reach (the "speed of light" in your game).
2. If applying a force will increase the speed of the object, divide the acceleration (change in velocity) by the Lorentz factor. The if condition is not realistic in terms of special relativity, but it keeps the ship more "controllable" at high speeds.
3. Update: Normally, the ship will be hard to maneuver when going at speeds near c because changing direction requires an acceleration that pushes velocity past c (The Lorentz factor will end up scaling acceleration in the new direction to nearly nothing.) To regain maneuverability, use the direction that the velocity vector would have been without Lorentz scaling with the magnitude of the scaled velocity vector.

Explanation:

Definition of Lorentz factor, where v is velocity and c is the speed of light:

This works because the Lorentz factor approaches infinity as velocity increases. Objects would need an infinite amount of force applied to cross the speed of light. At lower velocities, the Lorentz factor is very close to 1, approximating classical Newtonian physics.

Graph of Lorentz factor as velocity increases:

Note: I previously tried to solve a similar problem in my asteroids game by playing with friction settings. I just came up with this solution as I read your question^^

Update: I tried implementing this and found one potential flaw: acceleration in all directions is limited as the speed of light c is approached, including deceleration! (Counter-intuitive, but does this happen with special relativity in the real world?) I guess this algorithm could be modified to account for the directions of the velocity and force vectors... The algorithm has been modified to account for directions of vectors so the ship does not "lose controllability" at high speeds.

Update: Here is a code snippet from my asteroids game, which uses the Lorentz factor to limit the speed of game objects. It works pretty well!

update:* added downloadable demo (Windows only; build from source code for other platforms) of this algorithm in action. I'm not sure if all the dependencies were included in the zip; please let me know if something's missing. And have fun^^

void CObject::applyForces()
{
    // acceleration: change in velocity due to force f on object with mass m
    vector2f dv = f/m;

    // new velocity if acceleration dv applied
    vector2f new_v = v + dv;

    // only apply Lorentz factor if acceleration increases speed
    if (new_v.length() > v.length())
    {
        // maximum speed objects may reach (the "speed of light")
        const float c = 4;

        float b = 1 - v.length_squared()/(c*c);
        if (b <= 0) b = DBL_MIN;

        double lorentz_factor = 1/sqrt(b);
        dv /= lorentz_factor;
    }

    // apply acceleration to object's velocity
    v += dv;

    // Update:
    // Allow acceleration in the forward direction to change the direction
    // of v by using the direction of new_v (without the Lorentz factor)
    // with the magnitude of v (that applies the Lorentz factor).
    if (v.length() > 0)
    {
        v = new_v.normalized() * v.length();
    }
}