Is malloc/free a syscall or a library routine provided by libc?

Very often, malloc and free are using lower-level virtual memory allocation services and allocating several pages (or even megabytes) at once, using system calls like mmap and munmap (and perhaps sbrk). Often malloc prefers to reuse previously freed memory space when relevant. Most malloc implementations use various and different strategies for "large" and "small" allocations, etc...

Notice that virtual address space can be limited, e.g. with setrlimit(2). Use on Linux pmap(1) and proc(5) to learn more about the virtual address space of some process (e.g. /proc/self/maps for your own one or /proc/1234/maps - also the pmap 1234 command - for process of pid 1234).

You could look at your GNU libc source code, look into the source code of other C standard libraries (such as musl-libc), read about malloc implementations, choose some other ones or implement your own, or use strace to find out experimentally.

Read the syscalls man page (i.e. syscalls(2)) and the file <asm/unistd.h> for a list of system calls.

---

a very fast malloc

^{(I believe that this could be the fastest implementation of malloc; however it is not very useful; it is conforming to the standards, e.g. n1570 or better)}

I strongly believe that the C standard is very vague about malloc and free. I'm pretty sure that the following functions are respecting the letter (but not the spirit) of the standard:

 /* politically incorrect, but very probably standard conforming */
 void *malloc (size_t sz) { if (sz>0) errno = ENOMEM; return NULL; }
 void free(void*ptr) { }

Of course you'll code calloc and realloc accordingly.

^{(BTW every code using malloc should test against its failure, but some -incorrectly- don't; malloc can return NULL on failure and people should test against that case)}

---

The GNU libc gives you hooks for your own malloc functions (and you could even probably use Boehm's Garbage Collector transparently thru them). These hooks could become deprecated and are non-standard.

If using GNU libc, look also into mallinfo(3) and malloc_stat(3) and related functions.

malloc and free are standard C library functions which are to be implemented by each C implementation.

The C standard only defines the way in which these functions behave and the behavior expected from them. How they are to be implemented in left to each implementation.

In short they are implementation detail of the implementation you use.

(An "implementation" consists of the compiler, the linker, the runtime library, and probably a few other things.)