[
https://issues.apache.org/jira/browse/HBASE-30413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18132964#comment-18132964
]
mazhengxuan commented on HBASE-30413:
-------------------------------------
PR is up: https://github.com/apache/hbase-connectors/pull/171. It fixes the
Spark 3 and Spark 4 composite row-key writes and the Spark 3 read path, with
round-trip tests in both modules. I left existing undelimited keys untouched
since their field boundaries can be ambiguous. Please let me know if you’d
prefer a different compatibility approach.
> Fix concatenation inconsistency between read/write path on composite row keys
> -----------------------------------------------------------------------------
>
> Key: HBASE-30413
> URL: https://issues.apache.org/jira/browse/HBASE-30413
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-connectors
> Reporter: Wellington Chevreuil
> Assignee: mazhengxuan
> Priority: Major
> Labels: pull-request-available
>
> This affects both spark3 and spark4 modules, currently, and was caught during
> the the implementation of sparkSQL write functions to the spark 4 module.
> In HBaseDataWriter.buildRowKey, composite row key components are concatenated
> without the delimiter that the reader expects for variable-length string
> fields (see HBasePartitionReader.parseRowKey). A catalog such as key1:key2
> with an unbounded string key1 cannot be read back reliably if written through
> this write path.
> This also needs to be addressed in the spark3 module. In
> DefaultSource.convertToPut, lines 229-243, it also concatenates row key
> component bytes with no delimiter.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
