[
https://issues.apache.org/jira/browse/IGNITE-28473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Abashev updated IGNITE-28473:
----------------------------------
Description:
When sending the same message to multiple nodes via
`GridIoManager.sendToGridTopic()`, the same `Message` object is passed in a
loop for each recipient. Serialization does not happen in the sender thread —
it happens later, in NIO worker threads
(`DirectNioClientWorker.writeToBuffer()` →
`GridIoMessageMarshallableSerializer.writeTo()`).
This results in the same message being serialized **N times concurrently** —
once per target node, each time in a separate NIO worker thread — even though
the byte representation is identical for all recipients. The result is a
performance drop due to redundant serialization work.
**Call stack:**
GridIoMessageMarshallableSerializer.writeTo()
→ GridNioServer$DirectNioClientWorker.writeToBuffer()
→ processWrite0() → processWrite()
→ AbstractNioClientWorker.processSelectedKeysOptimized()
→ bodyInternal() → body()
**Expected behavior:** the message should be serialized **once** in the thread
where it was constructed, before being enqueued to the NIO layer. NIO workers
should operate on a pre-built byte buffer rather than re-serializing the
mutable message object independently.
**Proposed approach:** use the existing serialization mechanism to produce the
byte representation of the message once, eagerly, before handing it off to the
send queue. NIO workers then write the cached bytes directly to the socket
buffer without invoking the serializer.
was:
Code-generated serialization mechanism uses the same buffer size checks on
every field write just like the previous serialization mechanism.
Right now fields of a message are written to the output stream in the same
order they are declared in the message class. That's why message serializer has
to perform buffer size checks for each individual field as it doesn't have any
information about the size of the next field.
But if one reorders the fields so that all fields of the same primitive type
are written in one batch, it is possible to perform buffer size check only
once. If number of fields and their type is known, serializer could ensure the
space in the buffer in one check. This would allow to save some CPU cycles.
> Concurrent message serialization in NIO workers causes performance degradation
> ------------------------------------------------------------------------------
>
> Key: IGNITE-28473
> URL: https://issues.apache.org/jira/browse/IGNITE-28473
> Project: Ignite
> Issue Type: Task
> Reporter: Alex Abashev
> Assignee: Alex Abashev
> Priority: Minor
> Labels: IEP-132, ise
>
> When sending the same message to multiple nodes via
> `GridIoManager.sendToGridTopic()`, the same `Message` object is passed in a
> loop for each recipient. Serialization does not happen in the sender thread —
> it happens later, in NIO worker threads
> (`DirectNioClientWorker.writeToBuffer()` →
> `GridIoMessageMarshallableSerializer.writeTo()`).
> This results in the same message being serialized **N times concurrently** —
> once per target node, each time in a separate NIO worker thread — even though
> the byte representation is identical for all recipients. The result is a
> performance drop due to redundant serialization work.
> **Call stack:**
> GridIoMessageMarshallableSerializer.writeTo()
> → GridNioServer$DirectNioClientWorker.writeToBuffer()
> → processWrite0() → processWrite()
> → AbstractNioClientWorker.processSelectedKeysOptimized()
> → bodyInternal() → body()
> **Expected behavior:** the message should be serialized **once** in the
> thread where it was constructed, before being enqueued to the NIO layer. NIO
> workers should operate on a pre-built byte buffer rather than re-serializing
> the mutable message object independently.
> **Proposed approach:** use the existing serialization mechanism to produce
> the byte representation of the message once, eagerly, before handing it off
> to the send queue. NIO workers then write the cached bytes directly to the
> socket buffer without invoking the serializer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
