Should I bother detecting OOM (out of memory) errors in my C code?

Look at the other side of the question: if you malloc memory, it fails, and you don't detect it at the malloc, when will you detect it?

Obviously, when you attempt to dereference the pointer.

How will you detect it? By getting a Bus error or something similar, somewhere after the malloc that you'll have to track down with a core dump and the debugger.

On the other hand, you can write

  #define OOM 42 /* just some number */

  /* ... */

  if((ptr=malloc(size))==NULL){
      /* a well-behaved fprintf should NOT malloc, so it can be used
       * in this sort of context
       */
      fprintf(stderr,"OOM at %s: %s\n", __FILE__, __LINE__);
      exit(OOM);
   }

and get "OOM at parser.c:447".

You pick.

Update

Good question about graceful return. The difficulty with assuring a graceful return is that in general you really can't set up a paradigm or pattern of how you do that, especially in C, which is after all a fancy assembly language. In a garbage-collected environment, you could force a GC; in a language with exceptions, you can throw an exception and unwind things. In C you have to do it yourself and so you have to decide how much effort you want to put into it.

In most programs, abnormally terminating is about the best you can do. In this scheme you (hopefully) get a useful message on stderr -- of course it could also be to a logger or something like that -- and a known value as the return code.

HIgh reliability programs with short recovery times push you into something like recovery blocks, where you write code that attempts to get a system back into a survivable state. These are great, but complicated; the paper I linked to talks about them in detail.

In the middle, you can come up with a more complicated memory management scheme, say managing your own pool of dynamic memory -- after all, if someone else can write malloc, so can you.

But there's just no general pattern (of which I'm aware anyway) for cleaning up enough to be able to return reliably and let the surrounding program continue.

Out of memory conditions can happen even on modern computers with lots of memory, if the user or system administrator restricts (see ulimit) the memory space for a process, or the operating system supports memory allocation limits per user. In pathological cases, fragmentation makes this fairly likely, even.

However, since use of dynamically allocated memory is prevalent in modern programs, for good reasons, it becomes very hairy to handle out-of-memory errors. Checking and handling errors of this kind would have to be done everywhere, at high cost of complexity.

I find that it is better to design the program so that it can crash at any time. For example, make sure data the user has created gets saved on disk all the time, even if the user does not explicitly save it. (See vi -r, for example.) This way, you can create a function to allocate memory that terminates the program if there is an error. Since your application is designed to handle crashes at any time, it's OK to crash. The user will be surprised, but won't lose (much) work.

The never-failing allocation function might be something like this (untested, uncompiled code, for demonstration purposes only):

/* Callback function so application can do some emergency saving if it wants to. */
static void (*safe_malloc_callback)(int error_number, size_t requested);

void safe_malloc_set_callback(void (*callback)(int, size_t))
{
    safe_malloc_callback = callback;
}

void *safe_malloc(size_t n)
{
    void *p;

    if (n == 0)
        n = 1; /* malloc(0) is not well defined. */
    p = malloc(n);
    if (p == NULL) {
        if (safe_malloc_callback)
            safe_malloc_callback(errno, n);
        exit(EXIT_FAILURE);
    }
    return p;
}

Valerie Aurora's article Crash-only software might be illuminating.

Regardless of the platform (except maybe embedded systems) it's a good idea to check for NULL and then just exit without doing any (or much) cleanup by hand.

Out of memory isn't a simple error. It is a catastrophe on todays systems.

The book The Practice of Programming (Brian W. Kernighan and Rob Pike, 1999) defines functions like emalloc() which just exits with an error message if there's no memory left.