Dynamic scaling of distributed cache

Topics: Developer Forum, User Forum
Jan 2, 2012 at 4:35 AM

I haven't found any guidance or discussion on the best practice for dynamically scaling a distributed cache configuration without restarting all clients.

I'd like to deploy IIS web server instances on EC2, each with an install of SharedCache server. In a typical cloud app, I plan on spinning up new web server instances behind a load balancer when the load requires it, and shut down instances when the load decreases.

In such operation, I obviously won't know all of the server IPs ahead of time and specify them in the client config file. Can I simply add a new IndexusServerSetting object to the client's IndexusDistributionCache.SharedCache.ServersList collection after caching operations have begun, and the client will immediately and automatically partition key hashes onto the new node? All clients should do this at the same time, ideally, so I plan on using a messaging service like RabbitMQ to notify each web server instance when a node appears or disappears.

I'd obviously like a solution where I don't have to reconfigure all IIS web.config files and restart them just because I added a new SharedCache node.

Mar 7, 2012 at 12:35 PM

One major issue with this is that the shared cache implementation of ketama is not correct.  Adding one extra node has a very big impact of the position of each object in the cluster.  Effect is that as soon as you add a new node it pretty much invalidates the entire cache, causing lots of cache misses.