Amazon
ElastiCache internally uses Memcached 1.4.5 engine. We are going to
explore the memory allocation and eviction policies of Amazon ElastiCache
through our understanding of memcached internals in this article. If you are a
complex and extensive user of Amazon ElastiCache it is better to have detailed
understanding of how the internals work, to avoid leakages and maintain overall
optimum performance.
How
is it organized internally?
Amazon
ElastiCache node usually breaks the allocated memory into smaller parts called
pages. Each page is usually 1 megabyte in size. Each page is then assigned to a
slab class when necessary. Each slab class is in turn divided into chunks of a
specific size. The chunks in each slab have the same size.
Following
diagram illustrates the above divisions:
If you are
using memcached on EC2 you can view the slab, chunk and byte details by using
the following command $. /memcached -vv
$ ./memcached -vv
slab class 1: chunk size 80 perslab 13107
slab class 2: chunk size 104 perslab 10082
slab class 3: chunk size 136 perslab 7710
slab class 4: chunk size 176 perslab 5957
slab class 5: chunk size 224 perslab 4681
slab class 6: chunk size 280 perslab 3744
slab class 7: chunk size 352 perslab 2978
slab class 8: chunk size 440 perslab 2383
slab class 9: chunk size 552 perslab 1899
slab class 10: chunk size 696 perslab 1506
[...etc...]
slab class 1: chunk size 80 perslab 13107
slab class 2: chunk size 104 perslab 10082
slab class 3: chunk size 136 perslab 7710
slab class 4: chunk size 176 perslab 5957
slab class 5: chunk size 224 perslab 4681
slab class 6: chunk size 280 perslab 3744
slab class 7: chunk size 352 perslab 2978
slab class 8: chunk size 440 perslab 2383
slab class 9: chunk size 552 perslab 1899
slab class 10: chunk size 696 perslab 1506
[...etc...]
From the
output of the above command you can observe that there is page with 80 byte
chunks (slab class 1), a page with 104 byte chunks (slab class 2). In slab class
1, each chunk is 80 bytes and each page can then contain 13,107 chunks (1 MB /
80 bytes). This continues all the way up the slab chain till 1 megabyte (with
growth factor of ~1.25). Note: There can
be multiple pages assigned for each slab-class, but as soon as a page is
assigned to a slab-class, it is permanent.
When you are
storing items in Amazon ElastiCache, they are pushed into the slab class of the
nearest fit. If your key + miscellaneous data + value is 70 bytes total, it
will go into slab class 1, with an overhead loss of 10 bytes per item. If your
data is 90 bytes total, it will go into Slab class 2, with an overhead of 14
bytes. If your cache access pattern ends up putting 90% of your pages in slab
class 2, there will be less memory available for slab class 3. This simple model is followed by
memcached engine to avoid memory defragmentation and get better speed in performance
versus memory usage tradeoff.
When a page is full (meaning all chunks in the page are
filled) and we still need to add another item, engine will fetch a new free
page from the pool, assign it to the specified slab-class (depending upon item
size), partitions it into chunks and gets the first available chunk to store
the new item. Note: These 1024 byte pages are assigned
on a FCFS basis (first come-first served), to the slab classes.
When there are no more pages left to assign to slab
class, it will use LRU algorithm to evict items to reclaim memory. In the following
section you can see in detail about the eviction mechanism of Amazon
ElastiCache works.
When
items are evicted?
Memory for an item is not actively reclaimed in Amazon
ElastiCache. The memcached engine does not have in-built background threads
that explicitly expires item and reclaims chunks. For instance, if you store an
item and it expires, it still sits in the LRU cache at its position, until it
falls to the end of the cache and is reused. However, if you fetch an expired
item, memcached will find that the item is expired and free its memory for
reuse.
To explain eviction mechanism in detail; Items are
evicted from Amazon ElastiCache if they are expired or the slab class is
completely out of free chunks and there are no free pages to assign to a slab
class. In case there are no free chunks, or
no free pages in the appropriate slab class, Amazon ElastiCache will look at
the LRU in tail to reclaim an item. Basically, it will search the last few
items in the “end” and identifies the ones that are already expired, makes it
free for reuse. If it cannot find an expired item on the end, it will
"evict" one which has not yet expired. Actually you could end up with one slab class
constantly evicting recently used items, on the other hand another slab having
a bunch of old items that just sit around. For example: When we need a 104 byte
chunk, it will evict a 104 byte chunk, even though there might be a 280 byte
chunk that is even older. This explains
the internal workings that “Each slab-class has its own LRU and statistical
counter updates, it behaves like a separate cache itself, it is not global LRU,
but slab class LRU in short”.
How
to reduce memory overheads?
If you are
running Amazon ElastiCache clusters of few MB/GB overheads will not matter
much. But if you are running Amazon ElastiCache clusters spanning in Hundreds
of GB- > TB, you will end up losing lots of allocated memory as overheads if
the chunk size and growth factor is not planned properly.
Since
memcached engine of Amazon ElastiCache does defrag, you need to plan for better
compaction and avoid overheads to keep the chunk sizes closer to your item
sizes. This is not a one-time activity, but a periodical activity aligned to
your cache growth.
A simple approach that can be followed is:
·
Identify
the item size usage and cache access patterns periodically
·
Set
the initial chunk size and growth factors accordingly in AWS console or API's for Amazon ElastiCache. You can also set
the factor using (-f) and the initial chunk-size (-s) options if its memcached
on EC2.
·
If item sizes are big and predictable it is
recommended to have bigger chunks and growth factors, if the item sizes are
varied it is better to have smaller initial size and growth factor. This will
keep the wastage minimal.
Disclaimer: The article is entirely
based on the premise that Amazon ElastiCache uses memcached 1.4.5 as the
engine. If AWS team has customized some sections of memcached engine, the
author will not be liable for misinforming readers.
Related Articles
Part 1: Understanding Amazon ElastiCache Internals : Connection overhead
Part 2: Understanding Amazon ElastiCache Internals : Elasticity Implication and Solutions
Part 3: Understanding Amazon ElastiCache Internals : Auto Discovery
Part 4: Understanding Amazon ElastiCache Internals : Economics of Choosing Cache Node Type
Launching Amazon ElastiCache in 3 Easy Steps
Caching architectures using Memcached & Amazon ElastiCache
Web Session Synchronization patterns in AWS
References:
No comments:
Post a Comment