If you are not getting a package on Ubuntu, please check to see if your /etc/apt/sources.list file is updated. This is common with most VPS providers

On VPS, especially with challenging small memory limit (e.g. 64MB or 128MB) lighttpd is the best option instead of Apache. In this article I will show you how to do that. On a fresh box (which has no apache installed) type this to get lighttpd webserver installed

apt-get install lighttpd

Now install PHP5 CGI package and PHP5 command line package

apt-get install php5-cgi php5-cli

Edit the following file

nano /etc/php5/cgi/php.ini

and uncomment this line

cgi.fix_pathinfo = 1

Now enable fastcgi module

lighty-enable-mod fastcgi

Restart Lighttpd

/etc/init.d/lighttpd restart

You get the welcome page

Lets install MySQL 5.1 server and client

apt-get install mysql-server mysql-client

and optionally phpMyAdmin

apt-get install phpmyadmin

Now create a test page to see if phpinfo is all Ok

nano /var/www/info.php

Type in the following

<?php
phpinfo();

Installing phpMyAdmin on Ubuntu server is even easier, you just need to

sudo apt-get install phpmyadmin

but what if we accidently skip the configuration screen where phpmyadmin makes modifications to apache?

Dont worry just do the following

Edit Apache’s configuration file (assuming you are using apache2)

sudo nano /etc/apache2/apache2.conf
# better to use the following command instead of the one above
#gksudo gedit /etc/apache2/apache2.conf

Add the following line of code inside apache2.conf:

Include /etc/phpmyadmin/apache.conf

Now restart Apache:

sudo /etc/init.d/apache2 restart

Go to /phpmyadmin/ and login with your mysql username and password.