C : How do you simulate an 'exception'?

Exception-like behavior in C is accomplished via setjmp/longjmp. However, what you really want here is an error code. If all values are potentially returnable, then you may want to take in an out-parameter as a pointer, and use that to return the value, like so:

int pop(double* outval)
{
        if(outval == 0) return -1;
        if(sp > 0)
                *outval = val[--sp];
        else {
                printf("error: stack empty\n");
                return -1;
        }
        return 0;
}

Not ideal, obviously, but such are the limitations of C.

Also, if you go this road, you may want to define symbolic constants for your error codes (or use some of the standard ones), so that a user can distinguish between "stack empty" and "you gave me a null pointer, dumbass".

You could build an exception system on top of longjmp/setjmp: Exceptions in C with Longjmp and Setjmp. It actually works quite well, and the article is a good read as well. Here's how your code could look like if you used the exception system from the linked article:

  TRY {
    ...
    THROW(MY_EXCEPTION);
    /* Unreachable */
  } CATCH(MY_EXCEPTION) {
    ...
  } CATCH(OTHER_EXCEPTION) {
    ...
  } FINALLY {
    ...
  }

It's amazing what you can do with a little macros, right? It's equally amazing how hard it is to figure out what the heck is going on if you don't already know what the macros do.

longjmp/setjmp are portable: C89, C99, and POSIX.1-2001 specify setjmp().

Note, however, that exceptions implemented in this way will still have some limitations compared to "real" exceptions in C# or C++. A major problem is that only your code will be compatible with this exception system. As there is no established standard for exceptions in C, system and third party libraries just won't interoperate optimally with your homegrown exception system. Still, this can sometimes turn out to be a useful hack.

I don't recommend using this in serious code which programmers other than yourself are supposed to work with. It's just too easy to shoot yourself in the foot with this if you don't know exactly what is going on. Threading, resource management, and signal handling are problem areas which non-toy programs will encounter if you attempt to use longjmp "exceptions".

One approach is to specify that pop() has undefined behaviour if the stack is empty. You then have to provide an is_empty() function that can be called to check the stack.

Another approach is to use C++, which does have exceptions :-)

You have a few options:

1) Magic error value. Not always good enough, for the reason you describe. I guess in theory for this case you could return a NaN, but I don't recommend it.

2) Define that it is not valid to pop when the stack is empty. Then your code either just assumes it's non-empty (and goes undefined if it is), or asserts.

3) Change the signature of the function so that you can indicate success or failure:

int pop(double *dptr)
{
    if(sp > 0) {
            *dptr = val[--sp];
            return 0;
    } else {
            return 1;
    }
}

Document it as "If successful, returns 0 and writes the value to the location pointed to by dptr. On failure, returns a non-zero value."

Optionally, you could use the return value or errno to indicate the reason for failure, although for this particular example there is only one reason.

4) Pass an "exception" object into every function by pointer, and write a value to it on failure. Caller then checks it or not according to how they use the return value. This is a lot like using "errno", but without it being a thread-wide value.

5) As others have said, implement exceptions with setjmp/longjmp. It's doable, but requires either passing an extra parameter everywhere (the target of the longjmp to perform on failure), or else hiding it in globals. It also makes typical C-style resource handling a nightmare, because you can't call anything that might jump out past your stack level if you're holding a resource which you're responsible for freeing.