About this blog

'Going Spatial' is my personal blog, the views on this site are entirely my own and should in no way be attributed to anyone else or as the opinion of any organisation.

My tweets on GIS, Humanitarian, Tech, Games and Randomness

Thursday 24 June 2010

Amazon Cloud is Ubuntu cloud - now shall we set up our own private cloud for GIS?

It was with some interest that I discovered that Amazon Web Services (AWS) was built on top of Ubuntu. I thought it was using XenServer or something but no, it is Ubuntu.

The details of their enterprise cloud offering is here.

What is of great interest is the ability to create your own cloud from the same installation. Am very tempted as our work on AWS, while fruitful has met with some odd issues. Also, the AWS isn't cheap - there has been a lot of testing over the past month and every hour that we have the AMIs running, it is costing us. Yes, the large AMIs may only cost $0.48 per hour but it soon adds up over time especially when one is only testing.

As part of our testing, we noticed that the AMIs were a bit sluggish in terms of CPU performance when compared to a standalone server. We decided to investigate this a bit further and discovered that Amazon rates their AMIs in Amazon EC2 Compute Units in order to provide everyone with a consistent measure of CPU capacity. Since Amazon purchase commodity servers all with a different rating, this approach makes perfect sense. However, how much horsepower is a since Amazon EC2 Compute Unit?

According to their website, it One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.

Not one of the fastest is it? Over time, Amazon will add more processors but this unit will remain. Please note that these are also single-core.

Here's a selection of Instances that Amazon now make available to your CPU hungry GIS application. We've been using the following:

Small Instance

1.7 GB memory
1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit)
32-bit platform
I/O Performance: Moderate
API name: m1.small
Cost: $0.12 per hour

Large Instance
7.5 GB memory
4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
64-bit platform
I/O Performance: High
API name: m1.large
Cost: $0.48 per hour

Thursday 10 June 2010

Datahub now live

Very pleased that now, we have our first venture into the brave new world of the cloud. Despite Larry Ellison's disdain for the term, I like it.

The OS OpenData was made available to the general public and we've been busy uploading the data and rendering it through ArcGIS Server. Version 1.0 of the service went live in time for the ESRI (UK) 2010 User Conference.
We have now released version 2.0 of the our service, using the free data and it includes separate mapservices for Meridian2 data, Panorama and Strategi. Release 3.0 will see OS VectorMap District being added to the list. All will be cached and highly available.

Overall, we are very pleased with the speed of the service and look forward to expanding the work more.

To connect:

1. Javascript API : http://datahub.esriuk.com/ArcGIS/rest/services/
2. Through ArcGIS Desktop http://datahub.esriuk.com/ArcGIS/services

Security Groups in AWS

Need to group them!
Just discovered that it isn't possible to move a created AMI from one security group to another security group after its initial creation. This is not too helpful as many organisations probably have a fluid and organic idea over the concept of AWS Security groups and will wish for some flexibility once they have been created.

Losing an AMI is also very easy to do!

Naming and identifying AMI
We've also decided to refer to our AMI using the last three digits of their unique instance id (example: '27b' or 'f4f') as a quick and easy way to identify them. There is room for confusing still between 'instance' and 'ami' but we'll sort this out when we get to it.

Thursday 3 June 2010

Small AMI

Always wondered why the 'smallest' available AMI was called 'M1.LARGE' but I have discovered that if you create an AMI from Amazon's own template, use the basic 32-bit Windows web server and voila, you have the 'M1.SMALL' AMI available to you. This AMI costs $0.12 per hour as compared to the $0.48 per hour for the M1.LARGE AMI. The small AMI also has 1.7GB of RAM available and is optimised for light weight web servers, probably all running behind the Network Load Balancer.