The test was :
•Key : test
•Data : 90 bytes
•Flags : 0
:(
Le lundi 16 novembre 2015 19:01:50 UTC+1, Nicolas Martinez a écrit :
>
> Ok... i think i have understood what you say:
>
> Key (Characters Number) + 1 + Data (Characters Number) + Header + Chunk
> Size + CAS Size
>
> Header = Flags (Characters Number) + Key (Characters Numbers)
>
> + 2 bytes ( \r\n )
>
> + 4 bytes (2 spaces and 1 \r)
>
> Chunk Size = 48 bytes (default)
>
> CAS Size = 8 bytes (64 bits platform)
>
> Seems to be good:
>
> 4 + 1 + 90 + (1+4+2+4) + 48 + 8 = 162 Bytes => Slab 4 (192 Bytes)
>
> But, if i start Memcached with -C, it's wrong.
>
> 4 + 1 + 90 + (1+4+2+4) + 48 = 154 Bytes => *Slab 3 (152 Bytes)*
>
> It must be in Slab4 (192 Bytes) No?
>
>
> Le lundi 16 novembre 2015 15:37:59 UTC+1, Nicolas Martinez a écrit :
>>
>> Thank you very much for yours answers.
>> Ok for CAS... i don't use -C so i have to add 8 bytes
>>
>> i still don"t understand these lines :
>> >> 2b appended to data_bytes for an extra "\r\n" +
>> So, \r\n == 2b ?
>>
>> >> + 4 bytes for spaces and \r\n
>> which spaces?
>> What is this \r\n ? Isn't already counted before?
>> There is 1 "\r\n" and 1 "\"
>>
>> echo -e 'set 30 0 3600 30\r\n'$data'\r'| nc ${memcached_server} 11211
>>
>>
>> With your example:
>> * Key : 2c
>> * Val : 28 bytes
>> * Flg : 0 (1bytes)
>>
>> turns into:
>>
>>> * Key : 3b
>>
>> => key number characters + 1
>>
>>> * Val : 30b
>>
>> => 28 bytes + 2 bytes ("\r\n")
>>
>>> * Hdr : 4b + 3b == 7b
>>
>> => What are 4b?
>> => 3b are: flags (1b) + ??
>>
>>> * Itm : 56b
>>
>> => 48b + 8b (CAS)
>>
>>> => 96b. which is the cap for slab 1 in a default setup.
>>
>>
>>
>> Thank you again.
>>
>> Le lundi 16 novembre 2015 00:31:07 UTC+1, Dormando a écrit :
>>>
>>> Read carefully:
>>>
>>> item_size_ok(const size_t nkey, const int flags, const int nbytes) {
>>>
>>> passes:
>>>
>>> size_t ntotal = item_make_header(nkey + 1, flags, nbytes,
>>> prefix, &nsuffix);
>>>
>>> Then conditionally:
>>>
>>> if (settings.use_cas) {
>>> ntotal += sizeof(uint64_t);
>>> }
>>>
>>> item_make_header is doing:
>>> nsuffix = (uint8_t) snprintf(suffix, 40, " %d %d\r\n", flags, nbytes -
>>> 2);
>>>
>>> Then:
>>>
>>> return sizeof(item) + nkey + *nsuffix + nbytes;
>>>
>>> It's convoluted but shirt.
>>>
>>> the lengths are:
>>> key +
>>> 1 +
>>> data_bytes +
>>> 2b appended to data_bytes for an extra "\r\n" +
>>> stringified rep of the flags + data length
>>> + 4 bytes for spaces and \r\n (these are carriage returns, one byte
>>> each)
>>> + 8b for CAS if enabled
>>>
>>> CAS can be turned off via the -C starttime arg. it takes up 8 bytes.
>>>
>>> Example:
>>> * Key : 2c
>>> * Val : 28b
>>> * Flg : 0 (1b)
>>>
>>> turns into:
>>> * Key : 3b
>>> * Val : 30b
>>> * Hdr : 4b + 3b == 7b
>>> * Itm : 56b
>>> => 96b. which is the cap for slab 1 in a default setup.
>>>
>>> It's tough to get it exact for small chunks due to the way the header is
>>> added. You should ballpark or tune the -f value to align with your
>>> observed data.
>>>
>>> On Sun, 15 Nov 2015, Nicolas Martinez wrote:
>>>
>>> > Hi,
>>> > Is CAS always used?
>>> > If yes, we have to always add 56 bytes to the KEY and VALUE ?
>>> > you don't count FLAGS characters??
>>> >
>>> > I've found that Flags's size (number of characters) impact the
>>> storage.
>>> >
>>> > Example:
>>> > * Key : 2 characters = 2 bytes
>>> > * Value : 28 characters = 28 bytes
>>> > * FLAGS : 1 characters = 1 bytes
>>> > => 31 bytes
>>> >
>>> > seems to take the same storage as
>>> > * Key : 1 characters = 1 bytes
>>> > * Value : 28 characters = 28 bytes
>>> > * FLAGS : 2 characters = 2 bytes
>>> > => 31 bytes ... wich is the limit to be stored in Slab1
>>> >
>>> > ok for the /r/n ... should take 4 bytes no?
>>> >
>>> > So, if we count 56 bytes for CAS :
>>> 56(cas)+31(key+value+flags)+4(/r/n)= 91
>>> >
>>> > Not good... :(
>>> >
>>> > where I'm wrong ??
>>> >
>>> > Le samedi 14 novembre 2015 23:55:12 UTC+1, Dormando a écrit :
>>> > The mysql docs don't speak for the main tree... that's their own
>>> thing.
>>> >
>>> > the "sizes" binary that comes with the source tree tells you how
>>> many
>>> > bytes an item will use (though I intend to add this output to
>>> the 'stats'
>>> > output somewhere). With CAS this is 56 bytes.
>>> >
>>> > 56 + 2 + 30 == 88. Class 1 by default (in 1.4.24) is 96 bytes,
>>> but the
>>> > item still ends up in class 2.
>>> >
>>> > Why is this? (unfortunately?) because memcached pre-renders part
>>> of the
>>> > text protocol into the item header:
>>> >
>>> > *nsuffix = (uint8_t) snprintf(suffix, 40, " %d %d\r\n", flags,
>>> nbytes -
>>> > 2);
>>> > return sizeof(item) + nkey + *nsuffix + nbytes;
>>> >
>>> > so the flags + length are getting flattened + \r\n added to the
>>> end.
>>> > Together that's just enough to push it over the edge. It'd also
>>> be nice to
>>> > add a highly optimized numerics printf so I could twiddle
>>> options to save
>>> > a few bytes of memory in objects, but don't get your hopes up
>>> for that
>>> > happening soon :)
>>> >
>>> > On Sat, 14 Nov 2015, Nicolas Martinez wrote:
>>> >
>>> > > Add: Memcached version : 1.4.4 (RedHat)
>>> > >
>>> > > Le samedi 14 novembre 2015 14:49:37 UTC+1, Nicolas Martinez a
>>> écrit :
>>> > > Hi, few days i'm reading Memcached documentation and
>>> blogs... and i don't understand how objects are stored.
>>> > >
>>> > > My test
>>> > >
>>> > > 3 slabs :
>>> > >
>>> > > * 96.0 Bytes
>>> > > * 120.0 Bytes
>>> > > * 240.0 Bytes
>>> > > Everywhere, it's told :
>>> > > * if data is < 96 Bytes, it will be stored in Slabs1 (96B)
>>> > > * if data > 96B and < 120B, it will be stored in Slabs2
>>> (120B)
>>> > > * if data > 120B, it will be stored in Slabs3 (240B)
>>> > > * etc.
>>> > > BUT, for example, when i'm creating an 30B object, it's stored
>>> in Slab2 (120B), and NOT in Slab1 (96B).
>>> > >
>>> > > External sources:
>>> > > For example, the default size for the smallest block is
>>> 88 bytes (40 bytes of value, and the default 48 bytes for the key and flag
>>> data). If the size of the first item you store into the cache is less than
>>> 40 bytes, then a slab with a block size of 88 bytes is created and the
>>> value stored.
>>> > > =>
>>> https://dev.mysql.com/doc/mysql-ha-scalability/en/ha-memcached-using-memory.html
>>>
>>> > >
>>> > >
>>> > > WRONG
>>> > >
>>> > > A slab class is a collection of pages divided into same
>>> sized chunks. Each slab class is referenced to by its chunk size. So we’ll
>>> have Slab class 80kb, Slab class 100kb and so on. When an object needs to
>>> be stored, its size determines where it gets stored. So if the object is
>>> larger than 80kb but less than 100kb, it gets stored into
>>> > Slab
>>> > > class 100kb.
>>> > > =>
>>> http://returnfoo.com/2012/02/memcached-memory-allocation-and-optimization-2/
>>>
>>> > >
>>> > >
>>> > > WRONG
>>> > >
>>> > > How i create an object:
>>> > >
>>> > > data=$(pwgen 30 -c 1)
>>> > > echo -e 'set 30 0 3600 30\r\n'$data'\r'| nc
>>> ${memcached_server} 11211
>>> > >
>>> > >
>>> > > So, when 30B object is creating :
>>> > > * key name : 30 = 2 bytes
>>> > > * value: 30 characters = 30 bytes
>>> > > * tags : 0 = 1 bytes
>>> > > => All = 33 bytes
>>> > > If i add the default 48b as explained on Mysql website : 33 +
>>> 48 = 81B ... so < Slab1 (91B)... but always stored in Slab2 (120B)
>>> > >
>>> > > So, the size used to store object in the good Slab is not:
>>> > > * object value size
>>> > > * sum of KEY, VALUE and TAGS in bytes
>>> > > KEY size : 1 character = 1 B
>>> > > VALUE size : 1 character = 1 B
>>> > > TAGS size : 1 character = 1 B
>>> > >
>>> > > ... as read everywhere
>>> > >
>>> > > So, It seems that: (SUM of KEY+VALUE+TAGS )
>>> > > * For slab1 96.0 Bytes, data stored if <= 31 B (SUM of
>>> 2+28+1 )
>>> > > * For slab2 120.0 Bytes, data stored if <= 55 B (SUM of
>>> 2+52+1 )
>>> > > * For slab3 152.0 Bytes, data stored if <= 87 B (SUM of
>>> 2+84+1 )
>>> > > * For slab4 192.0 Bytes, data stored if <= 126 B (SUM of
>>> 3+122+1 )
>>> > > * For slab5 240.0 Bytes, data stored if <= 174 B (SUM of
>>> 3+170+1 )
>>> > > * etc.
>>> > >
>>> > > My configuration :
>>> > > * Chunk Size : 48
>>> > > * Chunk Growth Factore: 1,25
>>> > > * Max Bytes: 64.0 MBytes
>>> > >
>>> > > So, someone could explain me how the data is stored in the
>>> right Slabs???
>>> > > How calculate it???
>>> > >
>>> > > Thank you
>>> > >
>>> > >
>>> > > --
>>> > >
>>> > > ---
>>> > > You received this message because you are subscribed to the
>>> Google Groups "memcached" group.
>>> > > To unsubscribe from this group and stop receiving emails from
>>> it, send an email to [email protected].
>>> > > For more options, visit https://groups.google.com/d/optout.
>>> > >
>>> > >
>>> >
>>> > --
>>> >
>>> > ---
>>> > You received this message because you are subscribed to the Google
>>> Groups "memcached" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> > For more options, visit https://groups.google.com/d/optout.
>>> >
>>> >
>>
>>
--
---
You received this message because you are subscribed to the Google Groups
"memcached" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
