What's the easiest way to set up a LAMP stack?

sudo apt-get update
sudo apt-get install tasksel
sudo tasksel install lamp-server

It will install all the basic LAMP stack for you, prompt for MySQL root password, etc.

More specifically it will install the following packages, and their dependencies.

mysql-client-core-5.1 libwrap0 apache2  
libaprutil1-dbd-sqlite3 tcpd  
libapache2-mod-php5 apache2.2-common  
apache2-utils php5-common  
libaprutil1-ldap libaprutil1  
php5-mysql mysql-server-core-5.1  
libdbi-perl libplrpc-perl mysql-server  
apache2.2-bin libdbd-mysql-perl  
libhtml-template-perl  
libnet-daemon-perl libapr1  
mysql-server-5.1 libmysqlclient16  
ssl-cert apache2-mpm-prefork  
mysql-common mysql-client-5.1  

You might also want to take a peek at the Ubuntu Server Guide.

Install Apache

sudo apt-get install apache2

Install PHP

sudo apt-get install php5 libapache2-mod-php5

As fo 16.04, the number is dropped:

sudo apt-get install php libapache2-mod-php

Install MySQL

sudo apt-get install mysql-server

Install phpMyAdmin

sudo apt-get install libapache2-mod-auth-mysql php5-mysql phpmyadmin

As with the PHP installation, in 16.04, the number is dropped:

sudo apt-get install libapache2-mod-auth-mysql php-mysql phpmyadmin

Combined installation

16.04:

sudo apt-get install apache2 php libapache2-mod-php mysql-server libapache2-mod-auth-mysql php-mysql phpmyadmin

Before 16.04:

sudo apt-get install apache2 php5 libapache2-mod-php5 mysql-server libapache2-mod-auth-mysql php5-mysql phpmyadmin

The easiest way to install LAMP with PHPMyAdmin is using:

sudo apt-get  install lamp-server^ phpmyadmin 

You don't even need to install taskel. More details can be found here, which gives this:

To access PHPMyAdmin, open terminal & type:

sudo -H gedit /etc/apache2/apache2.conf

Add this line somewhere in that file:

Include /etc/phpmyadmin/apache.conf 

Finally restart Apache using:

/etc/init.d/apache2 restart

or

sudo service apache2 restart