goel-skd commented on code in PR #760:
URL: https://github.com/apache/iceberg-cpp/pull/760#discussion_r3486230320
##########
src/iceberg/util/string_util.h:
##########
@@ -41,22 +40,36 @@ concept FromChars = requires(const char* p, T& v) {
std::from_chars(p, p, v); };
class ICEBERG_EXPORT StringUtils {
public:
- // NOTE: These convert ASCII letters only; all other bytes, including
non-ASCII
- // (multibyte UTF-8) bytes, are passed through unchanged.
- // See https://github.com/apache/iceberg-cpp/issues/613.
- static std::string ToLower(std::string_view str) {
- return str | std::ranges::views::transform(ToLowerAscii) |
- std::ranges::to<std::string>();
- }
-
+ /// \brief Lower-case a UTF-8 string using Unicode simple (1:1) case mapping.
+ ///
+ /// Intended for case-insensitive name matching, similar to Iceberg Java's
+ /// toLowerCase(Locale.ROOT). The mapping is locale-independent, matching
the intent
+ /// of Locale.ROOT. It uses simple (1:1) case mapping rather than Java's
full case
+ /// mapping, so results differ for a few code points; e.g. U+0130 (capital I
with dot
+ /// above) maps to U+0069 ("i") here, but to U+0069 U+0307 ("i" + combining
dot above)
+ /// in Java. For ASCII and the large majority of letters the two agree.
+ ///
+ /// Invalid UTF-8 input is returned unchanged.
+ /// See https://github.com/apache/iceberg-cpp/issues/613.
+ static std::string ToLower(std::string_view str);
+
+ /// \brief Upper-case the ASCII letters (a-z) in a string; all other bytes,
including
+ /// multi-byte UTF-8 sequences, are left unchanged.
+ ///
+ /// Deliberately ASCII-only and, unlike ToLower, not Unicode-aware. It is
only used to
+ /// normalize ASCII enum/codec strings (e.g. "gzip" -> "GZIP", "all" ->
"ALL") for
+ /// case-insensitive comparison. A Unicode upper-case is intentionally not
provided:
+ /// simple case mapping would be wrong for some letters (e.g. "ß" (U+00DF)
would stay
+ /// unchanged instead of becoming "SS"), and no caller needs it.
static std::string ToUpper(std::string_view str) {
return str | std::ranges::views::transform(ToUpperAscii) |
std::ranges::to<std::string>();
}
+ /// \brief Case-insensitive equality; compares the ToLower forms of both
operands and
+ /// therefore inherits ToLower's Unicode simple-mapping behavior.
static bool EqualsIgnoreCase(std::string_view lhs, std::string_view rhs) {
- return std::ranges::equal(
- lhs, rhs, [](char lc, char rc) { return ToLowerAscii(lc) ==
ToLowerAscii(rc); });
+ return ToLower(lhs) == ToLower(rhs);
}
static bool StartsWithIgnoreCase(std::string_view str, std::string_view
prefix) {
Review Comment:
Thanks for pointing this sneaky bug out @wgtmac. It will bite when num bytes
change across the case toggle.
Let me push a fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
