Experimenting Locally with Terraform

Terraform supports a bunch of providers, but the vast majority of them are public cloud based.

However, you could set up a local VMware vSphere cluster and use the vSphere provider to interact with that to get you going. There's also a provider for OpenStack if you want to set up an OpenStack cluster.

Alternatively you could try using something like HPE's Eucalyptus which provides API compatibility with AWS, but on-premises.

That said, unless you already have a datacenter running VMware, all of those options are pretty awful and will take a lot of effort to get setup so you may be best waiting for your firewall to be opened up instead.

There isn't unfortunately a nice frictionless first party implementation of a VirtualBox provider but you could try this third-party VirtualBox provider.

While we wait for these network issues to be resolved, is there any way that I can experiment with Terraform locally without needing to connect to Azure or AWS? Perhaps with Virtual Box?

I use KVM to have as many virtual machines as I need to play around with anything, and I am now learning Terraform to, but with the specific intention of automating the creation of this VMs.

So if you are looking for a localhost playground to just learn Terraform, then KVM is a nice and more lightweight approach then using other heavy virtualization technologies, like the one you mentioned, Virtualbox.

If you want to learn Terraform while at same time you play with the AWS and Azure providers, then this is not a good solution for you, otherwise just give it a try.

Install KVM

First you need to install KVM on your Linux machine.

sudo apt update &&  sudo apt upgrade

Check for if the system support hardware virtualization:

$ egrep -c ‘(svm|vmx)’ /proc/cpuinfo
4

The output needs to be greater then 0, otherwise we need to enter into the Bios and enable VT technology.

Check if support KVM acelaration:

Installing the tool:

sudo apt install cpu-checker

Checking:

$ sudo kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

Installing KVM

sudo apt install qemu qemu-kvm libvirt-bin  bridge-utils  virt-manager

Check is running:

service libvirtd status

If not running:

sudo service libvirtd start && sudo update-rc.d libvirtd enable

Another check:

sudo virsh -c qemu:///system list

How to Create VMs in KVM with Terraform

We will create the VMs by using this KVM Libvirt Provider.

mkdir playground && cd playground

Terraform main file

Create the file main.tf:

################################################################################
# ENV VARS
################################################################################

# https://www.terraform.io/docs/commands/environment-variables.html

variable "VM_COUNT" {
  default = 3
  type = number
}

variable "VM_USER" {
  default = "developer"
  type = string
}

variable "VM_HOSTNAME" {
  default = "vm"
  type = string
}

variable "VM_IMG_URL" {
  default = "https://cloud-images.ubuntu.com/releases/bionic/release/ubuntu-18.04-server-cloudimg-amd64.img"
  type = string
}

variable "VM_IMG_FORMAT" {
  default = "qcow2"
  type = string
}

variable "VM_CIDR_RANGE" {
  default = "10.10.10.10/24"
  type = string
}


################################################################################
# PROVIDERS
################################################################################

# instance the provider
provider "libvirt" {
  uri = "qemu:///system"
}


################################################################################
# DATA TEMPLATES
################################################################################

# https://www.terraform.io/docs/providers/template/d/file.html

# https://www.terraform.io/docs/providers/template/d/cloudinit_config.html
data "template_file" "user_data" {
  template = file("${path.module}/cloud_init.cfg")
  vars = {
    VM_USER = var.VM_USER
  }
}

data "template_file" "network_config" {
  template = file("${path.module}/network_config.cfg")
}


################################################################################
# RESOURCES
################################################################################

resource "libvirt_pool" "vm" {
  name = "${var.VM_HOSTNAME}_pool"
  type = "dir"
  path = "/tmp/terraform-provider-libvirt-pool-ubuntu"
}

# We fetch the latest ubuntu release image from their mirrors
resource "libvirt_volume" "vm" {
  count  = var.VM_COUNT
  name   = "${var.VM_HOSTNAME}-${count.index}_volume.${var.VM_IMG_FORMAT}"
  pool   = libvirt_pool.vm.name
  source = var.VM_IMG_URL
  format = var.VM_IMG_FORMAT
}

# Create a public network for the VMs
resource "libvirt_network" "vm_public_network" {
   name = "${var.VM_HOSTNAME}_network"
   mode = "nat"
   domain = "${var.VM_HOSTNAME}.local"
   addresses = ["${var.VM_CIDR_RANGE}"]
   dhcp {
    enabled = true
   }
   dns {
    enabled = true
   }
}

# for more info about paramater check this out
# https://github.com/dmacvicar/terraform-provider-libvirt/blob/master/website/docs/r/cloudinit.html.markdown
# Use CloudInit to add our ssh-key to the instance
# you can add also meta_data field
resource "libvirt_cloudinit_disk" "cloudinit" {
  name           = "${var.VM_HOSTNAME}_cloudinit.iso"
  user_data      = data.template_file.user_data.rendered
  network_config = data.template_file.network_config.rendered
  pool           = libvirt_pool.vm.name
}

# Create the machine
resource "libvirt_domain" "vm" {
  count  = var.VM_COUNT
  name   = "${var.VM_HOSTNAME}-${count.index}"
  memory = "1024"
  vcpu   = 1

  cloudinit = "${libvirt_cloudinit_disk.cloudinit.id}"

  # TODO: Automate the creation of public network
  network_interface {
    network_id = "${libvirt_network.vm_public_network.id}"
    #network_id = "6d8e2494-835d-4baf-a14f-3a5c705febcc"
    #network_name = "vm_docker_network"
    network_name = "${libvirt_network.vm_public_network.name}"
  }

  # IMPORTANT
  # Ubuntu can hang is a isa-serial is not present at boot time.
  # If you find your CPU 100% and never is available this is why.
  #
  # This is a known bug on cloud images, since they expect a console
  # we need to pass it:
  # https://bugs.launchpad.net/cloud-images/+bug/1573095
  console {
    type        = "pty"
    target_port = "0"
    target_type = "serial"
  }

  console {
    type        = "pty"
    target_type = "virtio"
    target_port = "1"
  }

  disk {
    volume_id = "${libvirt_volume.vm[count.index].id}"
  }

  graphics {
    type        = "spice"
    listen_type = "address"
    autoport    = true
  }
}


################################################################################
# TERRAFORM CONFIG
################################################################################

terraform {
  required_version = ">= 0.12"
}

Env Vars

You can set env vars to override any default value for the variables defined in the ENV VARS section, but each env var name must be prefixed with TF_VAR:

export TF_VAR_VM_COUNT=5

Cloud Init

Create the file cloud_init.cfg:

#cloud-config
users:
  - name: ${VM_USER}
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: users, admin
    home: /home/${VM_USER}
    shell: /bin/bash
    ssh-authorized-keys:
      - ssh-rsa your-public-key-here
ssh_pwauth: True
disable_root: false
chpasswd:
  list: |
     ${VM_USER}:linux
  expire: False
package_update: true
package_upgrade: true
packages:
    - qemu-guest-agent
    - apt-transport-https
    - ca-certificates
    - curl
    - gnupg-agent
    - software-properties-common
    - zsh
growpart:
  mode: auto
  devices: ['/']
runcmd:
  - [ sh, -c, 'curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -' ]
  - [ sh, -c, 'sudo apt-key fingerprint 0EBFCD88']
  - [ sh, -c, 'sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"' ]
  - [ sh, -c, 'sudo apt update' ]
  - [ sh, -c, 'sudo apt install -y docker-ce docker-ce-cli containerd.io' ]
  - [ sh, -c, 'printf "\nalias dk=\"sudo docker\"\n" >> /home/${VM_USER}/.bashrc' ]
  - [ sh, -c, 'printf "\nalias dkc=\"sudo docker container\"\n" >> /home/${VM_USER}/.bashrc' ]
  - [ sh, -c, 'printf "\nalias dki=\"sudo docker image\"\n" >> /home/${VM_USER}/.bashrc' ]
  - [ sh, -c, 'printf "\nalias dks=\"sudo docker service\"\n" >> /home/${VM_USER}/.bashrc' ]
  - [ sh, -c, 'printf "\nalias dkn=\"sudo docker node\"\n" >> /home/${VM_USER}/.bashrc' ]

NOTE: Add your public key into the file

Network Config

Create the file network_config.cfg:

version: 2
ethernets:
  ens3:
     dhcp4: true

Run Terraform

To init Terraform:

terraform init

Create the VMs:

terraform apply

VMSs IP Addresses

Get the ips for each VM:

virsh net-dhcp-leases vm_network

NOTE: The VMs we created will take some seconds until they get their ips, thus you may need to repeat this command several times until you get the ips.

Now just SSH into one of the VMs with the ip address you got from virsh:

ssh developer@vm-ip-address

Have fun and enjoy your new playground.