Should I use Vagrant or Docker for creating an isolated environment?
I'm the author of Docker.
The short answer is that if you want to manage machines, you should use Vagrant. And if you want to build and run applications environments, you should use Docker.
Vagrant is a tool for managing virtual machines. Docker is a tool for building and deploying applications by packaging them into lightweight containers. A container can hold pretty much any software component along with its dependencies (executables, libraries, configuration files, etc.), and execute it in a guaranteed and repeatable runtime environment. This makes it very easy to build your app once and deploy it anywhere - on your laptop for testing, then on different servers for live deployment, etc.
It's a common misconception that you can only use Docker on Linux. That's incorrect; you can also install Docker on Mac, and Windows. When installed on Mac, Docker bundles a tiny Linux VM (25 MB on disk!) which acts as a wrapper for your container. Once installed this is completely transparent; you can use the Docker command-line in exactly the same way. This gives you the best of both worlds: you can test and develop your application using containers, which are very lightweight, easy to test and easy to move around (see for example https://hub.docker.com for sharing reusable containers with the Docker community), and you don't need to worry about the nitty-gritty details of managing virtual machines, which are just a means to an end anyway.
In theory it's possible to use Vagrant as an abstraction layer for Docker. I recommend against this for two reasons:
First, Vagrant is not a good abstraction for Docker. Vagrant was designed to manage virtual machines. Docker was designed to manage an application runtime. This means that Docker, by design, can interact with an application in richer ways, and has more information about the application runtime. The primitives in Docker are processes, log streams, environment variables, and network links between components. The primitives in Vagrant are machines, block devices, and ssh keys. Vagrant simply sits lower in the stack, and the only way it can interact with a container is by pretending it's just another kind of machine, that you can "boot" and "log into". So, sure, you can type "vagrant up" with a Docker plugin and something pretty will happen. Is it a substitute for the full breadth of what Docker can do? Try native Docker for a couple days and see for yourself :)
Second, the lock-in argument. "If you use Vagrant as an abstraction, you will not be locked into Docker!". From the point of view of Vagrant, which is designed to manage machines, this makes perfect sense: aren't containers just another kind of machine? Just like Amazon EC2 and VMware, we must be careful not to tie our provisioning tools to any particular vendor! This would create lock-in - better to abstract it all away with Vagrant. Except this misses the point of Docker entirely. Docker doesn't provision machines; it wraps your application in a lightweight portable runtime which can be dropped anywhere.
What runtime you choose for your application has nothing to do with how you provision your machines! For example it's pretty frequent to deploy applications to machines which are provisioned by someone else (for example an EC2 instance deployed by your system administrator, perhaps using Vagrant), or to bare metal machines which Vagrant can't provision at all. Conversely, you may use Vagrant to provision machines which have nothing to do with developing your application - for example a ready-to-use Windows IIS box or something. Or you may use Vagrant to provision machines for projects which don't use Docker - perhaps they use a combination of rubygems and rvm for dependency management and sandboxing for example.
In summary: Vagrant is for managing machines, and Docker is for building and running application environments.
Disclaimer: I wrote Vagrant! But because I wrote Vagrant, I spend most of my time living in the DevOps world which includes software like Docker. I work with a lot of companies using Vagrant and many use Docker, and I see how the two interplay.
Before I talk too much, a direct answer: in your specific scenario (yourself working alone, working on Linux, using Docker in production), you can stick with Docker alone and simplify things. In many other scenarios (I discuss further), it isn't so easy.
It isn't correct to directly compare Vagrant to Docker. In some scenarios, they do overlap, and in the vast majority, they don't. Actually, the more apt comparison would be Vagrant versus something like Boot2Docker (minimal OS that can run Docker). Vagrant is a level above Docker in terms of abstractions, so it isn't a fair comparison in most cases.
Vagrant launches things to run apps/services for the purpose of development. This can be on VirtualBox, VMware. It can be remote like AWS, OpenStack. Within those, if you use containers, Vagrant doesn't care, and embraces that: it can automatically install, pull down, build, and run Docker containers, for example. With Vagrant 1.6, Vagrant has docker-based development environments, and supports using Docker with the same workflow as Vagrant across Linux, Mac, and Windows. Vagrant doesn't try to replace Docker here, it embraces Docker practices.
Docker specifically runs Docker containers. If you're comparing directly to Vagrant: it is specifically a more specific (can only run Docker containers), less flexible (requires Linux or Linux host somewhere) solution. Of course if you're talking about production or CI, there is no comparison to Vagrant! Vagrant doesn't live in these environments, and so Docker should be used.
If your organization runs only Docker containers for all their projects and only has developers running on Linux, then okay, Docker could definitely work for you!
Otherwise, I don't see a benefit to attempting to use Docker alone, since you lose a lot of what Vagrant has to offer, which have real business/productivity benefits:
Vagrant can launch VirtualBox, VMware, AWS, OpenStack, etc. machines. It doesn't matter what you need, Vagrant can launch it. If you are using Docker, Vagrant can install Docker on any of these so you can use them for that purpose.
Vagrant is a single workflow for all your projects. Or to put another way, it is just one thing people have to learn to run a project whether it is in a Docker container or not. If, for example, in the future, a competitor arises to compete directly with Docker, Vagrant will be able to run that too.
Vagrant works on Windows (back to XP), Mac (back to 10.5), and Linux (back to kernel 2.6). In all three cases, the workflow is the same. If you use Docker, Vagrant can launch a machine (VM or remote) that can run Docker on all three of these systems.
Vagrant knows how to configure some advanced or non-trivial things like networking and syncing folders. For example: Vagrant knows how to attach a static IP to a machine or forward ports, and the configuration is the same no matter what system you use (VirtualBox, VMware, etc.) For synced folders, Vagrant provides multiple mechanisms to get your local files over to the remote machine (VirtualBox shared folders, NFS, rsync, Samba [plugin], etc.). If you're using Docker, even Docker with a VM without Vagrant, you would have to manually do this or they would have to reinvent Vagrant in this case.
Vagrant 1.6 has first-class support for docker-based development environments. This will not launch a virtual machine on Linux, and will automatically launch a virtual machine on Mac and Windows. The end result is that working with Docker is uniform across all platforms, while Vagrant still handles the tedious details of things such as networking, synced folders, etc.
To address specific counter arguments that I've heard in favor of using Docker instead of Vagrant:
"It is less moving parts" - Yes, it can be, if you use Docker exclusively for every project. Even then, it is sacrificing flexibility for Docker lock-in. If you ever decide to not use Docker for any project, past, present, or future, then you'll have more moving parts. If you had used Vagrant, you have that one moving part that supports the rest.
"It is faster!" - Once you have the host that can run Linux containers, Docker is definitely faster at running a container than any virtual machine would be to launch. But launching a virtual machine (or remote machine) is a one-time cost. Over the course of the day, most Vagrant users never actually destroy their VM. It is a strange optimization for development environments. In production, where Docker really shines, I understand the need to quickly spin up/down containers.
I hope now its clear to see that it is very difficult, and I believe not correct, to compare Docker to Vagrant. For dev environments, Vagrant is more abstract, more general. Docker (and the various ways you can make it behave like Vagrant) is a specific use case of Vagrant, ignoring everything else Vagrant has to offer.
In conclusion: in highly specific use cases, Docker is certainly a possible replacement for Vagrant. In most use cases, it is not. Vagrant doesn't hinder your usage of Docker; it actually does what it can to make that experience smoother. If you find this isn't true, I'm happy to take suggestions to improve things, since a goal of Vagrant is to work equally well with any system.
Hope this clears things up!