Create your own mcrouter cluster

Standard

Recently I am trying to help a fellow worker to resolve memcached hotkey issue. Yeah, memcached can be a bottleneck, if there are too many request bombarding for the same key. For that, I recommend to him using Facebook mcrouter.

As we all know there is no such thing as a memcached cluster. They operate as a silo, and they don’t talk to each other. Most of the time you need another server to operate like a puppet master to the memcached, whether is it for sharding or replication.

Memcached router allows users of memcached to sharded AND replicated, which beats Redis. So the following are the steps to set up a memcached cluster

1) Set up 3 Instances. One for mcrouter, the other 2 for memcached

2) SSH into the mcrouter instance

3) do a git pull

git clone git@github.com:facebook/mcrouter.git

4) Install mcrouter

./mcrouter/mcrouter/scripts/install_ubuntu_14.04.sh /home/$USER/mcrouter-install/ -j4

5) Test whether mcrouter is working

webapp@ip-10-11-13-50:~$ ~/mcrouter-install/install/bin/mcrouter --help
mcrouter 1.0
usage: /home/webapp/mcrouter-install/install/bin/mcrouter [options] -p port(s) -f config

libmcrouter options:

And it will list below all the options you can use

6) Create a config file that contains the IP of the other 2 memcached servers

{
   "pools": {
      "A": {
         "servers": [
           "<ip_address_1>:11211",
           "<ip_address_2>:11211"
         ]
      }
   },
   "route": {
     "type": "OperationSelectorRoute",
     "operation_policies": {
       "add": "AllSyncRoute|Pool|A",
       "delete": "AllSyncRoute|Pool|A",
       "get": "RandomRoute|Pool|A",
       "set": "AllSyncRoute|Pool|A"
     }
   }
 }

from there you have a pool called A. There are 2 IPs and use the 11211 to talk to the servers

7) Prep up the other 2 memcached servers. Run “sudo apt-get install memcached” or “sudo yum install memcached”

8) Run the command

sudo ~/mcrouter-install/install/bin/mcrouter --config-file=mcrouter.json -p 11211 &

And you will see the memcached router talk to the 2 node memcached

Also sharing with you the result of a 2000 clients per second load test on this architecture. The result looks very good.

Screen Shot 2015-03-14 at 4.28.51 PM

For more information on MC Router, check this link

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s