Dmytro Shteflyuk created THRIFT-6153:
----------------------------------------
Summary: Reduce native Ruby Compact Protocol varint write overhead
Key: THRIFT-6153
URL: https://issues.apache.org/jira/browse/THRIFT-6153
Project: Thrift
Issue Type: Bug
Components: Ruby - Library
Reporter: Dmytro Shteflyuk
Assignee: Dmytro Shteflyuk
h3. Problem
The native Ruby Compact Protocol encoder writes each byte of a multibyte varint
through the transport separately. Encoding boundary i32 and i64 values
therefore creates five or ten one-byte Ruby strings and performs five or ten
dynamic transport calls instead of one buffered call.
h3. Client impact
Ruby clients that serialize integer-heavy Compact payloads pay avoidable
object-allocation and method-dispatch costs. The overhead is most visible for
small payloads, where protocol framing and scalar writes dominate. The encoded
wire bytes remain correct.
h3. Reproduction
Run the following from an Apache Thrift checkout after building the Ruby native
extension:
{code:ruby}
require "thrift"
class RecordingTransport < Thrift::BaseTransport
attr_reader :writes
def initialize
@writes = []
end
def write(data)
@writes << data.dup
end
end
def encode(type, value)
transport = RecordingTransport.new
protocol = Thrift::CompactProtocol.new(transport)
protocol.public_send(type, value)
name = type.to_s.delete_prefix("write_")
bytes = transport.writes.join.unpack1("H*")
puts "#{name} writes=#{transport.writes.length} bytes=#{bytes}"
end
puts "native=#{Thrift::CompactProtocol.new(RecordingTransport.new).native?}"
encode(:write_i32, -(2**31))
encode(:write_i64, -(2**63))
{code}
Testing on master commit {{c8f876181c4fa132b5a6b42fdbfe04db06b4eee9}} produces:
{noformat}
native=true
i32 writes=5 bytes=ffffffff0f
i64 writes=10 bytes=ffffffffffffffffff01
{noformat}
h3. Expected behavior
The native Compact Protocol encoder should accumulate each varint in a bounded
stack buffer and call the transport's write method once. This should preserve
the exact wire bytes and generic transport behavior while reducing per-scalar
overhead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
