• Resize AWS EC2 EBS drives on the fly with zero downtime

    Running out of hard drive space is a bane of every system administrator. Shutting down the server to add another drive then relaunching, or maybe you are lucky enough to have hot swappable hard drives making the process less painful. Well in the cloud on Amazon AWS you no longer need to shutdown or reimage your instance to expand your hard drives. If you are already using EBS backed instances, you now have the ability to modify your volumes on the fly.




    This is how you resize your EBS drives on the fly on a EC2 instance. If you are using instance storage you cannot do this obviously, it only works on EBS backed instances.
    Read the rest of this entry »

    Share
  • Amazon AWS costs have improved 55-100% since 2009

    I previously wrote an article in 2009 about how much you might expect to pay for hosting your server/services using Amazon AWS EC2 and S3. The conclusion was that it would cost you approximately $920/year up to over $10K/year depending on the instance type you chose to run to host your server on Amazon.

    Over the course of time the cost of hosting on Amazon Web Services has come down dramatically and their service offerings have increased 5 fold from what they had. Let’s take a look at what you will pay now versus 2009 for the comparable setups. For the sake of simplicity I am going to use current 4th generation on demand pricing but there are a few cases where it might be cheaper to use 3rd generation over 4th. There are very few good reasons to still be using first generation servers though. It should also be pointed out that Amazon also launched spot instances which are an even cheaper way to run an ec2 instance albeit not necessarily as reliable.
    Read the rest of this entry »

    Share
  • Collection of useful Apache redirects, rewrites, directives and reminders

    These are a slew of rewrite and redirection commands for Apache that can be setup in .htaccess or other site configuration files. In some cases there are multiple ways of doing things that might work in one situation but not another. I have a love/hate relationship with mod_rewrite, as I think many do, but in the end it is a powerful tool that can do so much that it is indispensable.

    ** IMPORTANT – Anywhere you see RewriteXXXXX, make sure you have “RewriteEngine On” to enable it! **

    Custom error pages:

    
    ErrorDocument 400 /errors/400.html
    ErrorDocument 401 /errors/401.html
    ErrorDocument 403 /errors/403.html
    ErrorDocument 404 /errors/404.html
    ErrorDocument 500 /errors/500.html

    Read the rest of this entry »

    Share
  • Merging 100+K Gzipped Files Together

    In order to open a large volume of cloudfront logs in a spreadsheet, we needed to merge over 100K .gz files on OS X Maverick into a single file. Trying to use the following command resulted in a /usr/bin/cat: Argument list too long error:

    cat logs/*.gz >> combined_logs.gz 

    The reason this error occurs is because bash expands the asterisk to all matching files, producing a very long command line. To circumvent this problem the xargs command needs to be used to split up the list:




    Read the rest of this entry »

    Share
  • Blocking IP’s while behind an Amazon ELB with Apache

    So Amazon does not allow accept/deny rules on their firewall system which is beyond silly in this day and age, but I digress. If you want to limit access to an apache server behind Amazon’s Elastic Load Balancer, your options are fairly sparse (although there are options).

    One such way to accomplish this is to first allow logging of the IP from the ELB using the X-Forwarded-For header. You can see how to do this here.

    So once you have setup logging of the IP’s in the apache logs, you now want to adjust the site configuration files. We have individual files in /etc/apache2/sites-available, so we add the following to the specific site. You may do something similar or you may have it all in a httpd.conf, or you can even put this into a .htaccess file.
    Read the rest of this entry »

    Share