Great. Thanks so much Hendrik for your experience! We will might have high volume levels to deal with but probably not high commit rates.
On Thu, Sep 12, 2019 at 1:45 AM Hendrik Haddorp <hendrik.hadd...@gmx.net> wrote: > Hi, > > we have some setups that use an encryption zone in HDFS. Once you have > the hdfs config setup the rest is transparent to the client and thus > Solr works just fine like that. Said that, we have some general issues > with Solr and HDFS. The main problem seems to be around the transaction > log files. We have a quite high commit rate and these short lived files > don't seem to play well with HDFS and triple replication of the blocks > in HDFS. But encryption did not add anything issues for us. > > regards, > Hendrik > > On 11.09.19 22:53, John Thorhauer wrote: > > Hi, > > > > I am interested in encrypting/protecting my solr indices. I am wondering > > if Solr can work the an encrypted HDFS. I see that these instructions ( > > > https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.0/configuring-hdfs-encryption/content/configuring_and_using_hdfs_data_at_rest_encryption.html > ) > > explain that: > > > > "After permissions are set, Java API clients and HDFS applications with > > sufficient HDFS and Ranger KMS access privileges can write and read > to/from > > files in the encryption zone" > > > > > > So I am wondering if the solr/java api that uses HDFS would work with > this > > as well and also, has anyone had experience running this? Either good > > or bad? > > > > Thanks, > > John > > > >
