Dear Jack, Hi, One more thing to mention: I dont want to use solr or lucence for indexing accumulo or full text search inside that. I am looking for have both in a sync mode. I mean import some parts of data to solr for indexing. For this purpose probably I need something like trigger in RDBMS, I have to define something (probably with accumulo iterator) to import to solr on inserting new data. Regards.
On Fri, Jul 25, 2014 at 12:59 PM, Ali Nazemian <alinazem...@gmail.com> wrote: > Dear Jack, > Actually I am going to do benefit-cost analysis for in-house developement > or going for sqrrl support. > Best regards. > > > On Thu, Jul 24, 2014 at 11:48 PM, Jack Krupansky <j...@basetechnology.com> > wrote: > >> Like I said, you're going to have to be a real, hard-core gunslinger to >> do that well. Sqrrl uses Lucene directly, BTW: >> >> "Full-Text Search: Utilizing open-source Lucene and custom indexing >> methods, Sqrrl Enterprise users can conduct real-time, full-text search >> across data in Sqrrl Enterprise." >> >> See: >> http://sqrrl.com/product/search/ >> >> Out of curiosity, why are you not using that integrated Lucene support of >> Sqrrl Enterprise? >> >> >> -- Jack Krupansky >> >> -----Original Message----- From: Ali Nazemian >> Sent: Thursday, July 24, 2014 3:07 PM >> >> To: solr-user@lucene.apache.org >> Subject: Re: integrating Accumulo with solr >> >> Dear Jack, >> Thank you. I am aware of datastax but I am looking for integrating >> accumulo >> with solr. This is something like what sqrrl guys offer. >> Regards. >> >> >> On Thu, Jul 24, 2014 at 7:27 PM, Jack Krupansky <j...@basetechnology.com> >> wrote: >> >> If you are not a "true hard-core gunslinger" who is willing to dive in >>> and >>> integrate the code yourself, instead you should give serious >>> consideration >>> to a product such as DataStax Enterprise that fully integrates and >>> packages >>> a NoSQL database (Cassandra) and Solr for search. The security aspects >>> are >>> still a work in progress, but certainly headed in the right direction. >>> And >>> it has Hadoop and Spark integration as well. >>> >>> See: >>> http://www.datastax.com/what-we-offer/products-services/ >>> datastax-enterprise >>> >>> -- Jack Krupansky >>> >>> -----Original Message----- From: Ali Nazemian >>> Sent: Thursday, July 24, 2014 10:30 AM >>> To: solr-user@lucene.apache.org >>> Subject: Re: integrating Accumulo with solr >>> >>> >>> Thank you very much. Nice Idea but how can Solr and Accumulo can be >>> synchronized in this way? >>> I know that Solr can be integrated with HDFS and also Accumulo works on >>> the >>> top of HDFS. So can I use HDFS as integration point? I mean set Solr to >>> use >>> HDFS as a source of documents as well as the destination of documents. >>> Regards. >>> >>> >>> On Thu, Jul 24, 2014 at 4:33 PM, Joe Gresock <jgres...@gmail.com> wrote: >>> >>> Ali, >>> >>>> >>>> Sounds like a good choice. It's pretty standard to store the primary >>>> storage id as a field in Solr so that you can search the full text in >>>> Solr >>>> and then retrieve the full document elsewhere. >>>> >>>> I would recommend creating a document structure in Solr with whatever >>>> fields you want indexed (most likely as text_en, etc.), and then store a >>>> "string" field named "content_id", which would be the Accumulo row id >>>> that >>>> you look up with a scan. >>>> >>>> One caveat -- Accumulo will be protected at the cell level, but if you >>>> need >>>> your Solr search results to be protected by complex authorization >>>> strings >>>> similar to Accumulo, you will need to write your own QParserPlugin and >>>> use >>>> post filtering: >>>> http://java.dzone.com/articles/custom-security-filtering-solr >>>> >>>> The code you see in that article is written for an earlier version of >>>> Solr, >>>> but it's not too difficult to adjust it for the latest (we've done so in >>>> our project). Once you've implemented this, you would store an >>>> "authorizations" string field in each Solr document, and pass in the >>>> authorizations that the user has access to in the fq parameter of every >>>> query. It's also not too bad to write something that parses the >>>> Accumulo >>>> authorizations string (like A&B&(C|D|E|F)) and interpret it accordingly >>>> in >>>> the QParserPlugin. >>>> >>>> This will give you true row level security in Solr and Accumulo, and it >>>> performs quite well in Solr. >>>> >>>> Let me know if you have any other questions. >>>> >>>> Joe >>>> >>>> >>>> On Thu, Jul 24, 2014 at 4:07 AM, Ali Nazemian <alinazem...@gmail.com> >>>> wrote: >>>> >>>> > Dear Joe, >>>> > Hi, >>>> > I am going to store the crawl web pages in accumulo as the main >>>> storage >>>> > part of my project and I need to give these data to solr for indexing >>>> > >>>> and >>>> > user searches. I need to do some social and web analysis on my data as >>>> well >>>> > as having some security features. Therefore accumulo is my choice for >>>> > >>>> the >>>> > database part and for index and search I am going to use Solr. Would >>>> > you >>>> > please guide me through that? >>>> > >>>> > >>>> > >>>> > On Thu, Jul 24, 2014 at 1:28 AM, Joe Gresock <jgres...@gmail.com> >>>> wrote: >>>> > >>>> > > We store data in both Solr and Accumulo -- do you have more details >>>> about >>>> > > what kind of data and indexing you want? Is there a reason you're >>>> > thinking >>>> > > of using both databases in particular? >>>> > > >>>> > > >>>> > > On Wed, Jul 23, 2014 at 5:17 AM, Ali Nazemian < >>>> alinazem...@gmail.com> >>>> > > wrote: >>>> > > >>>> > > > Dear All, >>>> > > > Hi, >>>> > > > I was wondering is there anybody out there that tried to integrate >>>> Solr >>>> > > > with Accumulo? I was thinking about using Accumulo on top of HDFS >>>> > >>>> > > and >>>> > > using >>>> > > > Solr to index data inside Accumulo? Do you have any idea how can I >>>> > > > do >>>> > > such >>>> > > > integration? >>>> > > > >>>> > > > Best regards. >>>> > > > >>>> > > > -- >>>> > > > A.Nazemian >>>> > > > >>>> > > >>>> > > >>>> > > >>>> > > -- >>>> > > I know what it is to be in need, and I know what it is to have > > >>>> plenty. >>>> I >>>> > > have learned the secret of being content in any and every situation, >>>> > > whether well fed or hungry, whether living in plenty or in want. I >>>> > >>>> > can >>>> > do >>>> > > all this through him who gives me strength. *-Philippians >>>> 4:12-13* >>>> > > >>>> > >>>> > >>>> > >>>> > -- >>>> > A.Nazemian >>>> > >>>> >>>> >>>> >>>> -- >>>> I know what it is to be in need, and I know what it is to have plenty. >>>> I >>>> have learned the secret of being content in any and every situation, >>>> whether well fed or hungry, whether living in plenty or in want. I can >>>> do >>>> all this through him who gives me strength. *-Philippians 4:12-13* >>>> >>>> >>>> >>> >>> -- >>> A.Nazemian >>> >>> >> >> >> -- >> A.Nazemian >> > > > > -- > A.Nazemian > -- A.Nazemian
