Marathon vs Kubernetes vs Docker Swarm on DC/OS with Docker containers

Though it's a bit outdated, it may be helpful to read What's the difference between Apache's Mesos and Google's Kubernetes, to get some of the basics right. Also, note that Mesos operates on a different level than Kubernetes/Marathon/Chronos. Last but not least, see Docker Swarm + Mesos by Timothy Chen, keeping in mind that Marathon and Swarm can operate simultaneously on the same Mesos cluster.

I'll try to break down the unique aspects of each container orchestration framework on Mesos.

Use Docker Swarm if:

• You want to use the familiar Docker API to launch Docker containers on Mesos.
• Swarm may eventually provide an API to talk to Kubernetes (even K8s-Mesos) too.
• See: http://www.techrepublic.com/article/docker-and-mesos-like-peanut-butter-and-jelly/

Use Kubernetes-Mesos if:

• You want to launch K8s Pods, which are groups of containers co-scheduled and co-located together, sharing resources.
• You want to launch a service alongside one or more sidekick containers (e.g. log archiver, metrics monitor) that live next to the parent container.
• You want to use the K8s label-based service-discovery, load-balancing, and replication control.
• See http://kubernetesio.blogspot.com/2015/04/kubernetes-and-mesosphere-dcos.html

Use Marathon if:

• You want to launch Docker or non-Docker long-running apps/services.
• You want to use Mesos attributes for constraint-based scheduling.
• You want to use Application Groups and Dependencies to launch, scale, or upgrade related services.
• You want to use health checks to automatically restart unhealthy services or rollback unhealthy deployments/upgrades.
• You want to integrate HAProxy or Consul for service discovery.
• You want to launch and monitor apps through a web UI or REST API.
• You want to use a framework built from the start with Mesos in mind.

Use Chronos if:

• You want to launch Docker or non-Docker tasks that are expected to exit.
• You want to schedule a task to run at a specific time/schedule (a la cron).
• You want to schedule a DAG workflow of dependent tasks.
• You want to launch and monitor jobs through a web UI or REST API.
• You want to use a framework built from the start with Mesos in mind.