What exactly is POSIX?

POSIX first was a standard in 1988 long before the Single UNIX Specification. It was one of the attempts at unifying all the various UNIX forks and UNIX-like systems. POSIX is an IEEE Standard, but as the IEEE does not own the UNIX® trademark, the standard is not UNIX® though it is based on the existing UNIX API at that time. The first standard POSIX.1 is formally known as IEEE std 1003.1-1988.[1] IEEE charged a substantial fee to obtain a copy of the standard.

The Open Group released the Single UNIX Specification (SUSv2) in 1997 based on IEEE's work of the POSIX standard. SUSv3 was released in 2001 from a joint working group between IEEE and The Open Group known as the Austin Group. SUSv3 is also known as POSIX:2001[2]. There is now also POSIX:2004 and POSIX:2008 which is the core of SUSv4. As for what UNIX® is, UNIX® is whatever the current registered trademark holder says it is. Since 1994, that is The Open Group.

Novell acquired the UNIX® systems business from AT&T/USL which is where UNIX® was born. In 1994, they sold the right to the UNIX® trademark to X/Open[3] now know as The Open Group. They then sold the UNIX® source code to SCO as UNIXWARE®.[3] UNIX® itself has forked many times[4][5] partly due to AT&T's licensing model. Purchasing UNIX® gave you the complete source of the operating system and the full tool-chain to build it. Modifications to the source can be distributed and used by anyone who owned a license to UNIX® from AT&T. The license fee was in the thousands.

BSD was a project at Berkeley which added a number of enhancements to the UNIX® operating system. BSD code was released under a much more liberal license than AT&T's source and did not require a license fee or even a requirement to be distributed with source, unlike the GPL that the GNU Project and Linux use. This has caused a good part of the BSD code to be included with various commercial UNIX forks. By around 4.3BSD, they had nearly replaced any need for the original AT&T UNIX® source code. FreeBSD/NetBSD/OpenBSD are all forks of 4.3BSD that are a complete operating system and have none of the original AT&T source code. Nor do they have right to the UNIX® trademark, but much of their code is used by commercial UNIX operating systems. The Socket API used on UNIX was developed on BSD and the Unix Fast Filesystem code was borrowed and used on various UNIX Operating Systems like Solaris with their own enhancements.

Linux was developed in 1991, but was developed from scratch unlike BSD and uses the existing GNU Project which is a clean-room implementation of much of the UNIX user-space. It implements much of POSIX for compatibility and is UNIX-like in design, but it does not have the close connection to AT&T or UNIX® that the BSDs have.


Most important things POSIX 7 defines

  1. C API

    Greatly extends ANSI C with things like:

    • more file operations: mkdir, dirname, symlink, readlink, link (hardlinks), poll(), stat, sync, nftw()
    • process and threads: fork, execl, wait, pipe, semaphors sem_*, shared memory (shm_*), kill, scheduling parameters (nice, sched_*), sleep, mkfifo, setpgid()
    • networking: socket()
    • memory management: mmap, mlock, mprotect, madvise, brk()
    • utilities: regular expressions (reg*)

    Those APIs also determine underlying system concepts on which they depend, e.g. fork requires a concept of a process.

    Many Linux system calls exist to implement a specific POSIX C API function and make Linux compliant, e.g. sys_write, sys_read, ... Many of those syscalls also have Linux-specific extensions however.

    Major Linux desktop implementation: glibc, which in many cases just provides a shallow wrapper to system calls.

  2. CLI utilities

    E.g.: cd, ls, echo, ...

    Many utilities are direct shell front ends for a corresponding C API function, e.g. mkdir.

    Major Linux desktop implementation: GNU Coreutils for the small ones, separate GNU projects for the big ones: sed, grep, awk, ... Some CLI utilities are implemented by Bash as built-ins.

  3. Shell language

    E.g., a=b; echo "$a"

    Major Linux desktop implementation: GNU Bash.

  4. Environment variables

    E.g.: HOME, PATH.

    PATH search semantics are specified, including how slashes prevent PATH search.

  5. Program exit status

    ANSI C says 0 or EXIT_SUCCESS for success, EXIT_FAILURE for failure, and leaves the rest implementation defined.

    POSIX adds:

    • 126: command found but not executable.

    • 127: command not found.

    • > 128: terminated by a signal.

      But POSIX does not seem to specify the 128 + SIGNAL_ID rule used by Bash: Default exit code when process is terminated?

  6. Regular expression

    There are two types: BRE (Basic) and ERE (Extended). Basic is deprecated and only kept to not break APIs.

    Those are implemented by C API functions, and used throughout CLI utilities, e.g. grep accepts BREs by default, and EREs with -E.

    E.g.: echo 'a.1' | grep -E 'a.[[:digit:]]'

    Major Linux implementation: glibc implements the functions under regex.h which programs like grep can use as backend.

  7. Directory struture

    E.g.: /dev/null, /tmp

    The Linux FHS greatly extends POSIX.

  8. Filenames

    • / is the path separator
    • NUL cannot be used
    • . is cwd, .. parent
    • portable filenames
      • use at most max 14 chars and 256 for the full path
      • can only contain: a-zA-Z0-9._-

    See also: https://stackoverflow.com/questions/18550253/what-is-posix-compliance-for-filesystem

  9. Command line utility API conventions

    Not mandatory, used by POSIX, but almost nowhere else, notably not in GNU. But true, it is too restrictive, e.g. single letter flags only (e.g. -a), no double hyphen long versions (e.g. --all).

    A few widely used conventions:

    • - means stdin where a file is expected
    • -- terminates flags, e.g. ls -- -l to list a directory named -l

    See also: https://stackoverflow.com/questions/8957222/are-there-standards-for-linux-command-line-switches-and-arguments

  10. "POSIX ACLs" (Access Control Lists), e.g. as used as backend for setfacl.

    This was withdrawn but it was implemented in several OSes, including in Linux with setxattr.

Who conforms to POSIX?

Many systems follow POSIX closely, but few are actually certified by the Open Group which maintains the standard. Notable certified ones include:

  • OS X (Apple) X stands for both 10 and UNIX. Was the first Apple POSIX system, released circa 2001. See also: https://stackoverflow.com/questions/5785516/is-osx-a-posix-os
  • AIX (IBM)
  • HP-UX (HP)
  • Solaris (Oracle)

Most Linux distros are very compliant, but not certified because they don't want to pay the compliance check. Inspur's K-UX and Huawei's EulerOS are two certified examples.

The official list of certified systems be found at: https://www.opengroup.org/openbrand/register/ and also at the wiki page.

Windows

Windows implemented POSIX on some of its professional distributions.

Since it was an optional feature, programmers could not rely on it for most end user applications.

Support was deprecated in Windows 8:

  • https://stackoverflow.com/questions/4746043/where-does-microsoft-windows-7-posix-implementation-currently-stand
  • https://superuser.com/questions/495360/does-windows-8-still-implement-posix
  • Feature request: https://windows.uservoice.com/forums/265757-windows-feature-suggestions/suggestions/6573649-full-posix-support

In 2016 a new official Linux-like API called "Windows Subsystem for Linux" was announced. It includes Linux system calls, ELF running, parts of the /proc filesystem, Bash, GCC, (TODO likely glibc?), apt-get and more: https://channel9.msdn.com/Events/Build/2016/P488 so I believe that it will allow Windows to run much, if not all, of POSIX. However, it is focused on developers / deployment instead of end users. In particular, there were no plans to allow access to the Windows GUI.

Historical overview of the official Microsoft POSIX compatibility: http://brianreiter.org/2010/08/24/the-sad-history-of-the-microsoft-posix-subsystem/

Cygwin is a well known GPL third-party project for that "provides substantial POSIX API functionality" for Windows, but requires that you "rebuild your application from source if you want it to run on Windows". MSYS2 is a related project that seems to add more functionality on top of Cygwin.

Android

Android has its own C library (Bionic) which does not fully support POSIX as of Android O: https://stackoverflow.com/questions/27604455/is-android-posix-compatible

Bonus level

The Linux Standard Base further extends POSIX.

Use the non-frames indexes, they are much more readable and searchable: http://pubs.opengroup.org/onlinepubs/9699919799/nfindex.html

Get a full zipped version of the HTML pages for grepping: https://stackoverflow.com/questions/453993/is-there-a-listing-of-the-posix-api-functions/45832939#45832939


POSIX is the Portable Operating System standard. It describes certain utilities, APIs, and services a compliant operating system must provide to software (for example sockets, file I/O and threading) along with conventions on how these should be called from a program.

The idea is that a program written for one POSIX-Compliant OS would be easier to port to another POSIX-compliant OS than porting between non-POSIX-compliant OSes. This is why it is much easier to port an application from, say, FreeBSD to Linux than it is to port it from FreeBSD to Windows (though Windows ostensibly supports a subset of POSIX.)

Tags:

Posix

Standard

Recent Posts