About this blog

'Going Spatial' is my personal blog, the views on this site are entirely my own and should in no way be attributed to anyone else or as the opinion of any organisation.

My tweets on GIS, Humanitarian, Tech, Games and Randomness

Wednesday 25 September 2013

Public Cloud Provider going bust...so the sky must be falling right?

Run Forrest, run!
Recently, a large public cloud storage provider, Nirvanix, informed customers that they were going out of business and that anyone with data in the Nirvanix public cloud, should pull it down as soon as possible. Am sure this would have sent massive shock waves around the cloud computing community; both users and suppliers.
There's no word of the impending doom on the website....

While I have no worries about the biggest players in the field (Amazon Web Services, Microsoft Azure, Google to name three) - I think Nirvanix will be the first of a few mid-range suppliers who will struggle to make it long-term.

Now, this makes it an interesting predicament: while most people at my company (and my peers) have embraced the cloud (public/private/hybrid) there's definitely a small but vocal minority who are now crowing 'I told you so'. Reading between the lines from the various press releases, it looks like there's a few Nirvanix customers who kept all their data with the supplier. More worryingly, there are other customers who are using Nirvanix resources, without even knowing it as they might be contracted through a Nirvanix Managed Service Provider. Either way, the situation is not looking too rosey for them at the moment.

One of our disaster recovery scenarios involves AWS going offline (an EMP strike? a malicious insider-led hack of the DNS servers? a slow-burning virus/malware corrupting the data?) or AWS going out of business. The likelihood of this scenario coming to pass for AWS is low, but then again, am sure Nirvanix customers thought of the same thing.

Of course, hindsight is always 20/20 and the view from the rear-view mirror is always clearer than the windscreen (took this off Warren Buffet) but I hope this doesn't take the wind out of the cloud-computing sails. For all our cloud services, we have our data in at least two places. One of them is not with AWS and we're seriously considering a third option of mirroring our data onto either Azure or Rackspace for redundancy. In fact, we're now starting to regard each cloud provider as a 'commodity' and this relevation pulls us neatly into the orbit of multi-cloud management solution providers.

Hmmm which cloud shall I use today?
In my view, what has happened with Nirvanix is an exception rather than the rule. However, it is interesting that a a fair number of businesses do not want to work with the bigger cloud providers citing reasons such as 'too complicated', 'not flexible enough' and the customer requiring very bespoke arrangements, for example complying with EU Safe Harbour regulations.

I wonder what happened with Nirvanix? I know that the main cloud providers have been engaging in a price war that is making some providers very attractive options for those looking for alternatives. It looks like Nirvanix ran out of cash. I think the price will continue to drop which is great for consumers but this could lead to the last man standing syndrome with only the biggest providers remaining. Am sure this would fall foul of one competition commission in one region or another! The mid-range cloud providers offer some very niche, very cool options that the larger providers may not have in their repertoire. For the sake of variety I hope the mid-range providers survive and thrive. For customers; best to spread your risk and use more than one cloud provider and include your own on-premise resources. It might mean more spend but it is the cost of doing business; consider it an insurance policy. For me? Am going to start a new backup job right now and migrate some of the crucial business data off to another provider.....

Friday 13 September 2013

Resizing CentOS 6.4 Partitions in VirtualBox 4.2.16

As part of my day to day messing around, I run a lot of VMs on virtualbox. I play with new stuff and install new software, reinstall old software and do all sorts of crazy things. This experimentation takes time and disk space and sometimes, I just don't have enough space.

I had CentOS6.4 installed on VirtualBox 4.2.16 and I was running out of space. I had 8.0GB total and thought it would be enough. Nope. I wanted to install the latest version of Portal for ArcGIS 10.2 and ArcGIS Server 10.2 and other bits and well, 8.0Gb wasn't going to cut it. I could have spent some time removing a lot of 'test' software on the CentOS 6.4 VM but already had a skinny version of the OS. Anyway, disk is cheap.

So, I wanted to increase the 8.0GB to 20.0GB WITHOUT having to reinstall the whole OS! It shouldn't be too difficult to resize the linux partition now would it?

Ordinarily, if this was a physical machine; it would be quite easy but with it already running inside Virtual Machine; I reckon it would be a little bit difficult....and it was.

For all those who have struggled; here's how I did it. Ironically, it is easier in Windows but it's all part of the learning experience. 

What I had and what you (probably) need:

  • VirtualBox 4.2.16 (you should be running this already)
  • Your CentOS 6.4 OS (already installed - you want to expand this right?)
  • Clonezilla 1.2.8
  • Gparted (or SystemRescueCD 3.2)

The steps broadly are:

You have a VM called 'CentOS' - with an 8.0GB VDI disk attached.

1. Create a new virtual disk of the desired size in VirtualBox. I created a new 20.0GB disk.
2. Using Clonezilla, attach the ISO to the VM and boot into it. Clone the current CentOS 6.4 disk to the new 20.0Gb disk. 
3. Boot into Gparted or SystemRescueCD and resize the existing partitions to fit into the new space.
4. Issue a few commands that extends the file system to fill the new size.
5. Mount the new virtual hard disk and boot into it.
6. New space should be there.

Now in detail.....

==Create your new disk in VirtualBox==

In VirtualBox select your VM in the left hand table of contents and choose settings. In the settings window that pops up, scroll down to the 'storage' option on the left-hand side table of contents. This is where you can can add extra (or different) hard drives and change your DVD rom etc.

Make sure the controller is selected and choose the 'add hard disk' option.

You will be prompted with a new pop up box asking to either 'create new disk', 'choose existing disk', 'cancel'. The option you should choose is 'create new disk'.

Choose 'VDI (VirtualBox Disk Image)' as the disk option and select 'next'.

Now it is time for the storage options - you have a choice of two and I usually go for Dynamically allocated to save disk space and time. Now click 'next'.

The next window specifies the file location and size of the new VDI disk. This usually confuses most people. First thing is to choose a name for the disk, then the size of the disk (i.e 20GB) and then an appropriate location for your new virtual hard drive. Best place would be the same place as your existing VDI disk for the CentOS distribution.

Once done, the drive has been created it will be attached automatically to the CentOS OS's virtual hard drive controller. It will be now ready to used; this new disk will be the target of the next step in which the smaller drive will be cloned onto this new drive.

==Boot into Clonezilla and clone your current disk to the new disk==

Right, time to clone - if you don't have a copy of Clonezilla, download the ISO.

Now attach it to the CentOS VM, you know - the same one with the new disk on it? If you can't remember how to do it in VirtualBox - click on the existing DVD/CD drive and click on the disk icon to the right of the CD/DVD Drive option and select the ISO. Click on the 'Live CD/DVD' check box just to ensure that when the VM boots; that it will do so from the attached ISO  (aka Live CD/DVD) and not the virtual hard drives.

Once this is done, click on the 'ok' option and then start the VM. Now with a new virtual hard drive attached and a bootable Clonezilla ISO. All looking good!

Start up the VM, and you should boot into Clonezilla, a very cool tool. Using Clonezilla - clone the existing disk image to the new image. The time it takes is dependent on the size of disk. Once completed. Shutdown the VM, detach the Clonezilla ISO and detach the original VM disk. Switch the new VM disk to be the primary, bootable drive.

Try starting it up - it should boot.

Once booted, you will notice that, despite the extra disk space - your VM doesn't see it. Don't worry. The next steps will take care of this and resize the partitions without trashing your OS.

And it is also time to use another tool!

==Use GParted to resize the partitions without destroying your OS==
Now, time for another iso download! Get Gparted (http://sourceforge.net/project/downloading.php?group_id=115843&filename=gparted-livecd-0.3.4-11.iso&7005223) and mount it as a bootable disk alongside your new VM disk.

Boot into Gparted and using the GUI, resize your current partition to fill the entirety of the disk. What partitions I hear you ask? Well depending on how you have built the OS and mounted the drive, I would extend your / directory - this should mean everything below it will expand as well. Basically, most of the new applications you install will go into /usr or /etc/bin or something similar. Personal files and stuff, should go into /home. Extend them how you wish - just use the extra space you now have available to you!

Once done - there is one more step to do!

Shutdown your VM and disconnect GParted. Leave your new VM disk where it is.

==Using some command line magic, extend the current file system to fill this new space==
You have cloned the original drive to a newer and bigger drive. You then extended the existing partition(s) to use the new space. One final step is to extend the FILE SYSTEM to fill this new space. Unfortunately, you can't do this (easily) when the machine is booted, so attach either SystemRescueCD or some other LiveCD (preferably CentOS) and boot into this. You will need access to the command line and since this is all linux, this is easy enough.

Let's see what we have available in the Volume Group.

(note the free space now in the Volume Group which can now be assigned to a Logical Volume)

Alloc PE / Size 1596 / 8.00 GB
Free PE / Size 960 / 22.00 GB

Now we have to add the same into our logical volume
/usr/sbin/lvextend -l +100%FREE /dev/VolGroup00/LogVol00
Note: Yes, the attribute does say +100%FREE
Note2: VolGroup00 on my machine = vg_centos and LogVol00 = vg_root

So once the lvextend has completed, tap in:

Alloc PE / Size 1596 / 30.00 GB
Free PE / Size 960 / 00.00 GB

Extending the partition means now there's more space BUT your OS still can't see the new space. Using the 'resize2fs' command; it increases the size of the file system to use the new space. Trigger online resizing of the live and mounted filesystem so the new disk space can be utilized immediately:

resize2fs -p /dev/mapper/VolGroup00-LogVol00

Now, the system has a bigger diskspace to play around with.

df -h

Filesystem Size Used Avail Use% Mounted on
45G 3.2G 40G 8% /
/dev/sda1 99M 19M 76M 20% /boot
tmpfs 1014M 0 1014M 0% /dev/shm
Filesystem Size Used Avail Use% Mounted on
30G 8.2G 22.7G 18% / 
/dev/sda1 99M 19M 76M 20% /boot
tmpfs 1014M 0 1014M 0% /dev/shm

Now that is so much better!   

We're almost done. Shutdown the VM and detach the ISO, leaving just your new VM Disk. Make this the primary disk and boot. With luck, the VM will start up, you will be able to log in and it will have the extra space you allocated.

Well done. Definitely a bit involved but immensely satisfying once you've done it!

Now go and enjoy all that new space.

Wednesday 11 September 2013

World's Biggest Data Breaches & Hacks (infographic)

World's Biggest Data Breaches and Hacks

I have spent a good part of my morning and afternoon exploring this simple but powerful infographic. Right now, I am tempted to renew all my credit cards and passwords (again).

'via Blog this'