Resize AWS EC2 EBS drives on the fly with zero downtime

Running out of hard drive space is a bane of every system administrator. Shutting down the server to add another drive then relaunching, or maybe you are lucky enough to have hot swappable hard drives making the process less painful. Well in the cloud on Amazon AWS you no longer need to shutdown or reimage your instance to expand your hard drives. If you are already using EBS backed instances, you now have the ability to modify your volumes on the fly.

This is how you resize your EBS drives on the fly on a EC2 instance. If you are using instance storage you cannot do this obviously, it only works on EBS backed instances.
Read the rest of this entry »

Shell script to get all WordPress databases and URL’s

Below you will find a simple script that you can use to find all the databases and URL’s associated to wordpress sites in a shared database. This script requires a SQL user with the ability to ‘show databases’ and access to the DB’s you want to query.

The script grabs a list of all the databases on the server then loops through them all retrieving the URL from the wp_options table. The script also takes into consideration the fact the the table might not actually be wp_options (could contain some random characters) so we use a wildcard there.

Read the rest of this entry »

Useful commands to run in a terminal

With the 1000’s of commands used over the years I sometimes forget what I have run. This post is a collection of commands that have been useful for me in the past and might be useful to others too. All commands are set to run from local(.).

CREATE AN SSH TUNNEL (remember -i if using pem):

ssh -C2qTnN -D 8080 user@location

FIND ALL FILES/DIRECTORIES WITH XXX PERMS AND CHANGE IT TO XXX:
Files:

find . -type d -perm 0XXX -exec sudo chmod XXX {} \;

Directories:

find . -type f -perm 0XXX -exec sudo chmod XXX {} \;

Read the rest of this entry »

Merging 100+K Gzipped Files Together

In order to open a large volume of cloudfront logs in a spreadsheet, we needed to merge over 100K .gz files on OS X Maverick into a single file. Trying to use the following command resulted in a /usr/bin/cat: Argument list too long error:

cat logs/*.gz >> combined_logs.gz 

The reason this error occurs is because bash expands the asterisk to all matching files, producing a very long command line. To circumvent this problem the xargs command needs to be used to split up the list:




Read the rest of this entry »

Blocking IP’s while behind an Amazon ELB with Apache

So Amazon does not allow accept/deny rules on their firewall system which is beyond silly in this day and age, but I digress. If you want to limit access to an apache server behind Amazon’s Elastic Load Balancer, your options are fairly sparse (although there are options).

One such way to accomplish this is to first allow logging of the IP from the ELB using the X-Forwarded-For header. You can see how to do this here.

So once you have setup logging of the IP’s in the apache logs, you now want to adjust the site configuration files. We have individual files in /etc/apache2/sites-available, so we add the following to the specific site. You may do something similar or you may have it all in a httpd.conf, or you can even put this into a .htaccess file.
Read the rest of this entry »