What are the pros/cons of Upstart and systemd?

2016 Update

Most answers here are five years old so it's time for some updates.

Ubuntu used to use upstart by default but they abandoned it last year in favor of systemd - see:

Grab your pitchforks: Ubuntu to switch to systemd on Monday (The Register)

Because of that there is a nice article Systemd for Upstart Users on Ubuntu wiki - very detailed comparison between upstart and systemd and a transition guide from upstart to systemd.

(Note that according to the Ubuntu wiki you can still run upstart on current versions of Ubuntu by default by installing the upstart-sysv and running sudo update-initramfs -u but considering the scope of the systemd project I don't know how it works in practice, or whether or not systemd is possible to uninstall.)

Most of the info in the Commands and Scripts sections below is adapted from some of the examples used in that article (that is conveniently licensed just like Stack Exchange user contributions under the Creative Commons Attribution-ShareAlike 3.0 License).

Here is a quick comparison of common commands and simple scripts, see sections below for detailed explanation. This answer is comparing the old behavior of Upstart-based systems with the new behavior of systemd-based systems, as asked in the question, but note that the commands tagged as "Upstart" are not necessarily Upstart-specific - they are often commands that are common to every non-systemd Linux and Unix system.

Commands

Running su:

upstart: su
systemd: machinectl shell

(see "su command replacement" section below)

Running screen:

upstart: screen
systemd: systemd-run --user --scope screen

(see "Unexpected killing of background processes" section below)

Running tmux:

upstart: tmux
systemd: systemd-run --user --scope tmux

(see "Unexpected killing of background processes" section below)

Starting job foo:

upstart: start foo
systemd: systemctl start foo

Stopping job foo:

upstart: stop foo
systemd: systemctl stop foo

Restarting job foo:

upstart: restart foo
systemd: systemctl restart foo

Listing jobs:

upstart: initctl list
systemd: systemctl status

Checking configuration of job foo:

upstart: init-checkconf /etc/init/foo.conf
systemd: systemd-analyze verify /lib/systemd/system/foo.service

Listing job's environement variables:

upstart: initctl list-env
systemd: systemctl show-environment

Setting job's environment variable:

upstart: initctl set-env foo=bar
systemd: systemctl set-environment foo=bar

Removing job's environment variable:

upstart: initctl unset-env foo
systemd: systemctl unset-environment foo

Logs

In upstart, the logs are normal text files in the /var/log/upstart directory, so you can process them as usual:

cat /var/log/upstart/foo.log
tail -f /var/log/upstart/foo.log

In systemd logs are stored in an internal binary format (not as text files) so you need to use journalctl command to access them:

sudo journalctl -u foo
sudo journalctl -u foo -f

Scripts

Example upstart script written in /etc/init/foo.conf:

description "Job that runs the foo daemon"
start on runlevel [2345]
stop on runlevel [016]
env statedir=/var/cache/foo
pre-start exec mkdir -p $statedir
exec /usr/bin/foo-daemon --arg1 "hello world" --statedir $statedir

Example systemd script written in /lib/systemd/system/foo.service:

[Unit]
Description=Job that runs the foo daemon
Documentation=man:foo(1)
[Service]
Type=forking
Environment=statedir=/var/cache/foo
ExecStartPre=/usr/bin/mkdir -p ${statedir}
ExecStart=/usr/bin/foo-daemon --arg1 "hello world" --statedir ${statedir}
[Install]
WantedBy=multi-user.target

su command replacement

A su command replacement was merged into systemd in pull request #1022:

Add new "machinectl shell" command for su(1)-like behaviour

because, according to Lennart Poettering, "su is really a broken concept".

He explains that "you can use su and sudo as before, but don't expect that it will work in full".

The official way to achieve a su-like behavior is now:

machinectl shell

It has been further explained by Lennart Poettering in the discussion to issue #825:

"Well, there have been long discussions about this, but the problem is that what su is supposed to do is very unclear. [...] Long story short: su is really a broken concept. It will given you kind of a shell, and it’s fine to use it for that, but it’s not a full login, and shouldn’t be mistaken for one." - Lennart Poettering

Unexpected killing of background processes

Commands like:

screen
tmux
nohup

no longer work as expected. For example, nohup is a POSIX command to make sure that the process keeps running after you log out from your session. It no longer works on systemd. Also programs like screen and tmux need to be invoked in a special way or otherwise the processes that you run with them will get killed (while not getting those processes killed is usually the main reason of running screen or tmux in the first place).

This is not a mistake, it is a deliberate decision, so it is not likely to get fixed in the future. This is what Lennart Poettering has said about this issue:

In my view it was actually quite strange of UNIX that it by default let arbitrary user code stay around unrestricted after logout. It has been discussed for ages now among many OS people, that this should possible but certainly not be the default, but nobody dared so far to flip the switch to turn it from a default to an option. Not cleaning up user sessions after logout is not only ugly and somewhat hackish but also a security problem. systemd 230 now finally flipped the switch and finally by default cleans everything up correctly when the user logs out.

For more info see:

Systemd Starts Killing Your Background Processes By Default
Systemd v230 kills background processes after user logs out, breaks screen, tmux
Debian Bug #825394: systemd kill background processes after user logs out

High-level startup concept

In a way systemd works backwards - in upstart jobs start as soon as they can and in systemd jobs start when they have to. At the end of the day the same jobs can be started by both systems and in pretty much the same order, but you think about it looking from an opposite direction so to speak.

Here is how Systemd for Upstart Users explains it:

Upstart's model for starting processes (jobs) is "greedy event-based", i. e. all available jobs whose startup events happen are started as early as possible. During boot, upstart synthesizes some initial events like startup or rcS as the "tree root", the early services start on those, and later services start when the former are running. A new job merely needs to install its configuration file into /etc/init/ to become active.

systemd's model for starting processes (units) is "lazy dependency-based", i. e. a unit will only start if and when some other starting unit depends on it. During boot, systemd starts a "root unit" (default.target, can be overridden in grub), which then transitively expands and starts its dependencies. A new unit needs to add itself as a dependency of a unit of the boot sequence (commonly multi-user.target) in order to become active.

Usage in distributions

Now some recent data according to Wikipedia:

Distributions using upstart by default:

Ubuntu (from 9.10 to 14.10)
Chrome OS
Chromium OS

Distributions using systemd by default:

Arch Linux - since October 2012
CentOS - since April 2014 (7.14.04)
CoreOS - sice October 2013 (v94.0.0)
Debian - since April 2015 (v8)
Fedora - since May 2011 (v15)
Mageia - since May 2012 (v2.0)
openSUSE - since September 2012 (v12.2)
Red Hat Enterprise Linux - since June 2014 (v7.0)
SUSE Linux Enterprise Server - since October 2014 (v12)
Ubuntu - since April 2015 (v15.04)

(See Wikipedia for up to date info)

Distributions using neither Upstart nor systemd:

Devuan (Debian fork created that resulted from the systemd controversies in the Debian community that led to a resignation of Ian Jackson) - specifically promotes Init Freedom with the following init systems considered for inclusion: sinit, OpenRC, runit, s6 and shepherd.
Void Linux - uses runit as the init system and service supervisor
Gentoo - uses OpenRC
OS X - uses launchd
FreeBSD uses a a traditional BSD-style init (not SysV init)
NetBSD uses rc.d
DragonFly uses traditional init
OpenBSD uses the rc system startup script described here
Alpine Linux (relatively new and little known distribution, with strong emphasis on security is getting more popular - e.g. Docker is moving its official images from Ubuntu to Alpine) uses the OpenRC init system

Controversy

In the past A fork of Debian has been proposed to avoid systemd. The Devuan GNU+Linux was created - a fork of Debian without systemd (thanks to fpmurphy1 for pointing it out in the comments).

For more info about this controversy, see:

The official Debian position on systemd
The systemd controversy
Debian Exodus declaration in 2014:

As many of you might know already, the Init GR Debian vote promoted by Ian Jackson wasn't useful to protect Debian's legacy and its users from the systemd avalanche.

This situation prospects a lock in systemd dependencies which is de-facto threatening freedom of development and has serious consequences for Debian, its upstream and its downstream.

The CTTE managed to swap a dependency and gain us time over a subtle install of systemd over sysvinit, but even this process was exhausting and full of drama. Ultimately, a week ago, Ian Jackson resigned. [...]

Ian Jackson's resignation:

I am resigning from the Technical Committee with immediate effect.

While it is important that the views of the 30-40% of the project who agree with me should continue to be represented on the TC, I myself am clearly too controversial a figure at this point to do so. I should step aside to try to reduce the extent to which conversations about the project's governance are personalised. [...]

The Init Freedom:

Devuan was born out of a controversy over the decision to use as the default init system for Debian. The official Debian position on systemd is full of claims that others have debunked. Interested readers can continue discussing this hot topic in The systemd controversy. However we encourage you to keep your head cool and your voice civil. At Devuan we’re more interested in programming them wrong than looking back. [...]

Some websites and articles dedicated to the systemd controversy has been created:

Without-Systemd.org
Systemd-Free.org
The Init Freedom
Systemd on Suckless

There is a lot of interesting discussion on Hacker News:

https://news.ycombinator.com/item?id=7728692
https://news.ycombinator.com/item?id=13387845
https://news.ycombinator.com/item?id=11797075
https://news.ycombinator.com/item?id=12600413
https://news.ycombinator.com/item?id=11845051
https://news.ycombinator.com/item?id=11782364
https://news.ycombinator.com/item?id=12877378
https://news.ycombinator.com/item?id=10483780
https://news.ycombinator.com/item?id=13469935

Similar tendencies in other distros can be observed as well:

The Church of Suckless NixOS is looking for followers

Philosophy

upstart follows the Unix philosophy of DOTADIW - "Do One Thing and Do It Well." It is a replacement for the traditional init daemon. It doesn't do anything other than starting and stopping services. Other tasks are delegated to other specialized subsystems.

systemd does much more than that. In addition to starting and stopping services it also manages passwords, logins, terminals, power management, factory resets, log processing, file system mount points, networking and much more - see the NEWS file for some of the features.

Plans of expansion

According to A Perspective for systemd What Has Been Achieved, and What Lies Ahead presentation by Lennart Poettering in 2014 at GNOME.asia, here are the main objectives of systemd, areas that were already covered and those that were still in progress:

systemd objectives:

Our objectives

Turning Linux from a bag of bits into a competitive General Purpose Operating System.

Building the Internet’s Next Generation OS Unifying pointless differences between distributions

Bringing innovation back to the core OS

Desktop, Server, Container, Embedded, Mobile, Cloud, Cluster, . . . These areas are closer together than you might think

Reducing administrator complexity, reliability without supervision

Everything introspectable

Auto discovery, plug and play is key

We fix things where they are broken, never tape over them

Areas already covered:

What we already cover:

init system, journal logging, login management, device management, temporary and volatile file management, binary format registration, backlight save/restore, rfkill save/restore, bootchart, readahead, encrypted storage setup, EFI/GPT partition discovery, virtual machine/container registration, minimal container management, hostname management, locale management, time management, random seed management, sysctl variable management, console managment, . . .

Work in progress:

What we are working on:

network management

systemd-networkd

Local DNS cache, mDNS responder, LLMNR responder, DNSSEC verification

IPC in the kernel

kdbus, sd-bus

Time synchronisation with NTP

systemd-timesyncd

More integration with containers

Sandboxing of Services

Sandboxing of Apps

OS Image format

Container image format

App image format

GPT with auto-discovery

Stateless systems, instantiatable systems, factory reset

/usr is the OS

/etc is (optional) configuration

/var is (optional) state

Atomic node initialisation and updates

Integration with the cloud

Service management across nodes

Verifiable OS images

All the way to the firmware

Boot Loading

Scope of this answer

As fpmurphy1 noted in the comments, "It should be pointed out that systemd has expanded its scope of work over the years far beyond simply that of system startup."

I tried to include most of the relevant info here. Here I am comparing the common features of Upstart and systemd when used as init systems as asked in the question and I only mention features of systemd that go beyond the scope of an init system because those cannot be compared to Startup, but their presence is important to understand the difference between those two projects. The relevant documentation should be checked for more info.

More info

More info can be found at:

upstart website
systemd website
Upstart on Wikipedia
Systemd on Wikipedia
The architecture of systemd on Wikipedia
Linus Torvalds and others on Linux's systemd (ZDNet)
About the systemd controversy by Robert Graham
Init Freedom Campaign
Rationale for switching from upstart to systemd?

Extras

The LinOxide Team has created a Systemd vs SysV Init Linux Cheatsheet.

Both upstart and systemd are attempts to solve some of the problems with the limitations of the traditional SysV init system. For example, some services need to start after other services (for example, you can't mount NFS filesystems until the network is running), but the only way in SysV to handle that is to set the links in the rc#.d directory such that one is before the other. Add to that, you might need to re-number everything later when dependencies are added or changed. Upstart and Systemd have more intelligent settings for defining requirements. Also, there's the issue with the fact that everything is a shell script of some sort, and not everyone writes the best init scripts. That also impacts the speed of the startup.

Some of the advantages of systemd I can see:

Every process started gets its own cgroup or a particular cgroup.
Pre-creation of sockets and file handles for services, similar to how xinetd does for it's services, allowing dependent services to start faster. For example, systemd will hold open the filehandle for /dev/log for syslog, and subsequent services that send to /dev/log will have their messages buffered until syslogd is ready to take over.
Fewer processes run to actually start a service. This means you aren't writing a shell script to start up your service. This can be a speed improvement, and (IMO) something easier to set up in the first place.

One disadvantage I know of is that to take advantage of systemd's socket/FH preallocation, many daemons will have to be patched to have the FH passed to them by systemd.

Saw systemd mentioned on Arch General ML today. So read up on it. The H Online as ever is a great source for Linux Technology and is where I found my place to start researching Systemd as SysV Init and Upstart alternative. However the H Online article (in this case) isn't a very useful read, the real use behind it is it gives links to the useful reads.

The real answer is in the announcement of systemd. Which gives some crucial points of what's wrong with SysV initd, and what new systems need to do

To start less.
And to start more in parallel.

Its major plan to do this seems to be to start services only as they're needed, and to start a socket for that service, so that the service that needs it can connect to the created socket long before the daemon is fully online. Apparently a socket will retain a small amount of buffered data meaning that no data will be lost during the lag, it will be handled as soon as the daemon is online.

Another part of the plan seems to be to not serialize filesystems, but instead mount those on demand as well, that way you're not waiting on your /home/, etc (not to be confused with /etc) to mount, and/or fsck when you could be starting daemons as / and /var/ etc, are already mounted. It said it was going to use autofs to this end.

It also has the goal of creating .desktop style init descriptors as a replacement for scripts. This will prevent tons of slow sh processes and even more forks of processes from things like sed and grep that are often used in shell scripts.

They also plan not to start some services until they are asked for, and perhaps even shut them off if they are no longer needed, bluetooth module, and daemon are only needed when you're using a bluetooth device for example. Another example given is the ssh daemon. This is the kind of thing that inetd is capable of. Personally I'm not sure I like this, as it might mean latency when I do need them, and in the case of ssh I think it means a possible security vulnerability, if my inetd were compromised the whole system would be. However, I've been informed that using this to breach this system is infeasible and that if I want to I can disable this feature per service and in other ways.

Another feature is apparently going to be the capability to start based on time events, either at a regularly scheduled interval or at a certain time. This is similar to what crond and atd do now. Though I was told it will not support user "cron". Personally this sounds like the most pointless thing. I think this was written/thought up by people who don't work in multiuser environments, there isn't much purpose to user cron if you're the only user on the system, other than not running as root. I work on multiuser systems daily, and the rule is always run user scripts as the user. But maybe I don't have the foresight they do, and it will in no way make it so that I can't run crond or atd, so it doesn't hurt anyone but the developers I suppose.

The big disadvantage of systemd is that some daemons will have to be modified in order to take full advantage of it. They'll work now, but they'd work better if they were written specifically for its socket model.

It seems for the most part the systemd's peoples problem with upstart is the event system, and that they believe it to not make sense or be unnecessary. Perhaps their words put it best.

Or to put it simpler: the fact that the user just started D-Bus is in no way an indication that NetworkManager should be started too (but this is what Upstart would do). It's right the other way round: when the user asks for NetworkManager, that is definitely an indication that D-Bus should be started too (which is certainly what most users would expect, right?).

A good init system should start only what is needed, and that on-demand. Either lazily or parallelized and in advance. However it should not start more than necessary, particularly not everything installed that could use that service.

As I've already said this is discussed much more comprehensively in the announcement of systemd.