lburgazzoli opened a new issue #1779:
URL: https://github.com/apache/camel-k/issues/1779
To improve performances, we should be able to partition a source.
We should have support for static partitioning and optionally a dynamic one.
**Static partitions**
we could add the relevant information in the form of properties to a
KameletBinding, something like:
```yaml
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: jdbc-source
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: jdbc-source
properties:
partition-keys: <1>
- table1
- table2
- table3
partition-property: <2>
- tables
sink:
ref:
kind: KafkaTopic
apiVersion: kafka.strimzi.io/v1beta1
name: my-topic
```
1. list of values used to logically partition the source
2. the property used by the underlying kamelet to get a reference to the
partition keys it has to use
The operator then detect the presence of the special properties
`partition-keys` and `partition-property` and could create as example a
`StatefulSet` of integrations where each integration receives only a subset of
the `partition-keys`
**Dynamic partitions**
Instead of hardcoding the values of the partitions, the operator could use a
meta service provided by the kamelet to determine how a source can be paritioned
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
