About this blog

'Going Spatial' is my personal blog, the views on this site are entirely my own and should in no way be attributed to anyone else or as the opinion of any organisation.

My tweets on GIS, Humanitarian, Tech, Games and Randomness

Thursday, 28 November 2013

Velocity 2013 - speed me up!

Go faster baby! Velocity 2013

'Velocity 2013 is the conference where people talk about how to get things done (fast) in the real world - if you want to know how the best in the world handle their Operations, Velocity is the place to learn.'

Three days of concentrated focus on the key aspects of web performance, operations, and mobile performance, Velocity is the place to learn.

There you go; that was the strap line.

This is the second time that I have attended in as many years and I find the material and the speakers all very compelling and interesting. Certainly, in the work that we do in GIS, the lessons and processes are not that much different from many other IT companies. We have the same issues over configuration management

The first day of this conference clashed with the last day of the Esri European Developer Summit, so I had to miss the Esri one and the tea and scones they were serving that last day!


Copyright: Velocity2013

Bring the Noise: Making effective user of quarter of a million metrics. Jon Cowle from Etsy

Definitely interesting talk about Etsy.com's capture and use of 250,000 metrics on a regular basis. Due to their need to practise continuous deployment, they have too many metrics. So they practise anomaly detection instead and stream the metrics to memcache, developed an 'anomaly description alphabet' and used this to detect anomalies as well as providing insight into future events / searches. Also used GRAPHITE, GANGLIA, MESSAGEPACK, STATSD, Carbon Relay, Skyline and Occulus (collectively known as KALE). All on Github.

The highlight of the talk was Jon's 'Four Horseman of the Apocalypse' (when it comes to monitoring...): Too Many Parameters, Normality, Spike Influence, and Seasonality.

Useful Links:

Graphite (a scalable realtime Graphing tool) - http://graphite.wikidot.com/
Ganglia (scalable distributed monitoring system) - http://ganglia.sourceforge.net/

Responsive Images Techniques and Beyond. Yoav Weiss (WM Square)

A little bit beyond me but the gist of the talk was in responsive and responsible image creation in order to speed the internet up. Some interesting tidbits: 61% of internet traffic is imagery and larger imagery costs more in terms of transimission times and bandwidth. While the idea that bandwidth will always increase (and get cheaper) - the idea to resize the images according to the device calling them is a good one. Especially when sometime in the near future (within 12 months) mobile smart phones will outsell PCs AND probably generate more traffic. The speaker had application web page called 'Sizersoze' that can be used to estimate the savings when it comes to responsive imagery.

The application: http://sizersoze.org

​Be mean to your code with Gauntlt and the rugged way. James Wickett (Mentor Graphics)

A very enjoyable talk on incorporating security into the devops flow of continous delivery. While configuration management and infrastructure as code has been looked at as a new 'religion'; with joint tools being developed; security has been left out but should be incorporated as an essential part of devops. With continuous delivery of software and applications, security testing should also be completed on the same basis. Part of the talk included a live-hands-on-follow-me-demo using a downloaded vagrant box with the Gauntlt tools already installed. The tool carries out 'rugged opinonated testing'.
I liked his breakdown of how Devs, Ops and Security maintained CM:
  • Devs - Github/Syn
  • Ops - Wiki and text
  • Sec - Sourceforge
The Demo: https://gist.github.com/wickett/25d90a462706639446cc

Gaunlt: http://gauntlt.org/

Business bits​ of web operations. Theo Schlossnagle (OmnIT/Ciconus)

All businesses are their to make money and the ability for everyone in the organisation to understand the basic business drivers for the organisation is a very useful. I found the talk clear and concise. The speaker is quite straightforward. Terms covered include EBITDA, TCO, RIO and RISK. Nothing new but put together very well.

Here's a copy of the slides: https://speakerdeck.com/postwait/the-business-bits-of-web-operations

Getting 100B metrics onto disk. Jonathan Thurman (New Relic)

​New Relic collects lots of data (LOL) and their database is currently 216TB (and growing) spread over 9 MySQL databases. Some issues: they can't move their accounts database due to massive primary key collisions! Sharding is a requirement but with multiple points of failure. RAM is cheaper than I/O so they try keep everything in buffer. Reading is fast while writing is slow. They are completely on-premise in Chicago and no cloud based resources. Some interesting stats: kill all queries over an hour and all user UI queries over 75 seconds.

Slides are here: http://www.slideshare.net/slideshow/embed_code/28275612

Other notes from the keynotes:
Changing DevOps Culture at the BBC (recommended): http://www.youtube.com/embed/wFo9HgbDwXQ?autoplay=1 (video)
Making Government Digital Services Fast (recommended): http://www.youtube.com/embed/fo1VbYQB39E?autoplay=1 (video)

Slowing down to go faster: responsive web design at the BBC News: http://velocityconf.com/velocityeu2013/public/schedule/detail/31766 (video)

Other slides I found useful:

Yes they had lightsabers. Picture from Velocity2013