About this blog

'Going Spatial' is my personal blog, the views on this site are entirely my own and should in no way be attributed to anyone else or as the opinion of any organisation.

My tweets on GIS, Humanitarian, Tech, Games and Randomness

Sunday, 10 October 2010

A good resource for ESRI ArcGIS Server Map Cache Creation

I found a good resource for ESRI ArcGIS server map cache creation - advanced concepts at this year's 2010 ESRI Developer Summit. 

The video is interesting and recommended for those working with large map caches. Here it is

Video:

There's also a useful presentation that was used in the video:

From this video and presentation - some useful points: 

  1. Get the number of SOC instances correct. The guidance is for the server to be running at about 95% CPU. When we do our test areas of caching, we should look at the CPU used, and increase/decrease the SOCS to reach this level. If the server runs at 100% cache creation time gets worse. The advice is to start at ‘Number of Cores + 1’, and work up or down as necessary. 
  2. Do NOT try to create large caches in one go. Instead use the Feature Class option. Create a feature class which divides the area into manageable chunks. E.g Each chunk being about 4 hours of cache creation time. It is then possible for the progress to be monitored, and also to start off again, in case of server failure just doing the uncached areas.  [Script: http://resources.esri.com/geoprocessing/index.cfm?fa=codeGalleryDetails&scriptID=15896 ] 
  3. Probably do NOT try to use cache on demand. The reason the performance is bad, is that even though the request is only for a tile, ArcGIS server will generate a ‘super tile’. A super tile will typically contain lots of tiles. This means that the map request is taking a lot longer than is necessary. The reason for for creating a super tile, is so that duplicate labels do not appear in the map cache. We could, only as a final resort,  consider caching areas of map based on what they contain (e.g.  urban areas), and leave the cache on demand for large areas of nothing. E.g. the sea. Cache on demand will be quick here, as there is nothing to draw!. 
  4. In addition to these tips, we can consider using the power of Amazon. 
  5. Create several VMs for Caching. Each VM will cache a different region (e.g. England, Scotland, Wales) , using the Feature Class Technique to track each servers progress 
  6. Once caching is complete. Dismount the drives, and remount them all onto one server. [Avoids copy to S3]. Use the ‘Import Cache Toolbox tool, to merge the caches into one.  Allows us to use the Compact Caches all the time.  [http://help.arcgis.com/en/arcgisserver/10.0/help/arcgis_server_dotnet_help/index.html#//009300000078000000.htm
    Image Formats
a. We should use JPEG 50% compression for Imagery.
b. We should use PNG 32 for Vector and >256 colours.
c. We could consider using ‘Mixed Format’ Caches, for the transparency problem.
The slide show discusses a different reason for using this format, but the
transparency problems we have been having may be a good other reason for using
this format. For data updates consider using ‘Compare Feature Classes’ script. This
can generate a list of affected tiles, and therefore determine which tiles need to be
updated in the cache. This can be used in conjunction with the ‘Feature Class’
caching option. [http://arcscripts.esri.com/details.asp?dbid=16866]
3)      
Investigate ‘Keep Alive’ IIS setting. Need to investigate if this will work through Amazon Load
Balancer