Rajat Khandelwal created LENS-656:
-------------------------------------
Summary: make Storage class more important
Key: LENS-656
URL: https://issues.apache.org/jira/browse/LENS-656
Project: Apache Lens
Issue Type: Improvement
Reporter: Rajat Khandelwal
Current usage of Stogage in lens:
* Storages are crated with xml, where only class name is required. It has to be
a subclass of org.apache.lens.cube.metadata.Storage. Two subclasses are already
provided: org.apache.lens.cube.metadata.HDFSStorage, and
org.apache.lens.storage.db.DBStorage.
* Current requirement of storage subclass is to implement methods of
org.apache.lens.cube.metadata.PartitionMetahook (will be shown below). The two
provided subclasses provide a no-op implementations of this.
Issues:
* DBStorage has a dbUrl variable supposed to capture DB url. But that is not
used anywhere and is redundant. In the current design, the db url is required
in configuration of JDBCDriver. This means that drivers and storages are deeply
coupled.
* For querying partitions, Storage is not queried. Instead a map of storage
name to partitions is maintained elsewhere. Another example of deep coupling
Proposed changes:
* Storage as an independent entity. Basically removing the deep coupling
* Facts will have Storage instances instead of storage names. With loose
coupling, one storage will have only one instance.
* Driver conf has a property "supported storages". That is currently used in
the resolvers. Instead the property will become an attribute of the driver.
{noformat}
public interface PartitionMetahook {
void preAddPartitions(List<StoragePartitionDesc> storagePartitionDesc) throws
HiveException;
void commitAddPartitions(List<StoragePartitionDesc> storagePartitionDesc)
throws HiveException;
void rollbackAddPartitions(List<StoragePartitionDesc> storagePartitionDesc)
throws HiveException;
void preDropPartition(String storageTableName, List<String> partVals) throws
HiveException;
void commitDropPartition(String storageTableName, List<String> partVals)
throws HiveException;
void rollbackDropPartition(String storageTableName, List<String> partVals)
throws HiveException;
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
