merlimat opened a new pull request, #25632:
URL: https://github.com/apache/pulsar/pull/25632
## Summary
Adds the ability to look up scalable topics in a namespace by an exact
`(propertyKey, propertyValue)` pair, backed by the `MetadataStore`
secondary-index API. Native-index stores (Oxia) serve the lookup directly
in O(matches); other stores transparently fall back to a children scan.
### MetadataCache
- New overloads on `MetadataCache`:
- `create(path, value, indexExtractor)`
- `readModifyUpdate(path, modifyFn, indexExtractor)`
where `indexExtractor` is `Function<T, Map<String, String>>` returning the
`indexName -> secondaryKey` map associated with the record.
- `MetadataCacheImpl` overrides both: when an extractor is provided and the
underlying store is `MetadataStoreExtended`, the write goes through
`storeExtended.put(path, bytes, version, options, indexes)`. Stores
without index support transparently fall back to the plain `put`.
### ScalableTopicResources
- Index name = `topic-prop-` + property key, secondary key = property value.
Each entry of the topic's `properties` map becomes one index entry.
- `createScalableTopicAsync` / `updateScalableTopicAsync` thread the
extractor through on every write so updates refresh the index.
- New `findScalableTopicsByPropertyAsync(namespace, propertyKey,
propertyValue)`
calls `store.findByIndex` with scan prefix `/topics/<namespace>`. The
fallback predicate re-deserialises the record and re-checks the property.
### Admin API
- `GET /admin/v2/scalable/{tenant}/{ns}` accepts optional `?propertyKey` +
`?propertyValue` query params. When both are set the listing is served by
the index; otherwise the existing children-listing path runs unchanged.
- `ScalableTopics` admin client gains `listScalableTopicsByProperty` (sync +
async).
- `pulsar-admin scalable-topics list <ns> --property key=value` filters the
CLI listing the same way.
### Scope
- Scalable topics only — partitioned and non-partitioned managed-ledger
topics are not touched.
- No backward compat / reindex: only newly-created or updated scalable
topics are queryable.
## Test plan
- `MetadataCacheSecondaryIndexTest` (pulsar-metadata) — runs the new cache
overloads against every store implementation (Memory, RocksDB, ZooKeeper,
MockZooKeeper, **Oxia**). Asserts index queryability, refresh on update,
and empty-extractor behaviour.
- `ScalableTopicPropertyIndexTest` (pulsar-broker) — focused unit coverage
for `findScalableTopicsByPropertyAsync`: exact/multi match, namespace
scoping, empty results, update refresh, and topics with no properties not
being matched.
- `ScalableTopicsListByPropertyTest` (pulsar-broker) — end-to-end through
`PulsarAdmin` against a real shared broker.
### Matching PR(s) in forked repositories
- area/broker
- area/client-tools
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
