Re: Solr for Content Management

On 6/7/2018 12:10 PM, Moenieb Davids wrote: > Challenges: > When performing full text searches without concurrently executing updates, > solr seems to be doing well. Running updates also does okish given the > nature of the transaction. However, when I run search and updates > simultaneously, perfo

Re: Solr for Content Management

Hi, It is also likely that your indexing is using resources and that there are not enough resources for queries to process. Indexing can put stress on heap and GCs might be slowing Solr down resulting in observed latency. Can you tell us a bit more on size of your index, server configs, heap siz

Re: Solr for Content Management

And in solrconfig.xml, it is possible to configure the searches to warm the index up before the users see it. Regards, Alex On Thu, Jun 7, 2018, 21:27 David Hastings, wrote: > When you are sending updates you are adjusting the segments which take them > out of memory and the index becomes "

Re: Solr for Content Management

When you are sending updates you are adjusting the segments which take them out of memory and the index becomes "cold" until it gets enough searches to cache the various aspects of the index. On Thu, Jun 7, 2018 at 2:10 PM, Moenieb Davids wrote: > Hi All, > > Background: > I am currently testing

Re: Solr for Multi Tenant architecture

Dear all, Thank you for all your advices. This comment says: "SolrCloud starts to have serious problems when you create a lot of collections. We are aware of the scalability issues, but they are not easy to fix." http://lucene.472066.n3.nabble.com/Fwd-Solr-Cloud-6-0-0-hangs-when-creating-large-

Re: Solr for Multi Tenant architecture

HI Chamil, One thing to consider is relevancy, especially in case tenants' domains are different (e.g. one is tech and other pharmacy). If you go with one collection and use same field (e.g. desc) for all tenants, you will get one field stats and could skew results ordering if you order by sco

Re: Solr for Multi Tenant architecture

Apple did a preso on massive multi-tenancy. I haven’t watched it yet, but it might help. https://www.youtube.com/watch?v=_Erkln5WWLw wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Aug 27, 2016, at 10:

Re: Solr for Multi Tenant architecture

Thank you everyone for your great support. I will update you with our final approach. Best regards, Chamil On Aug 28, 2016 01:34, "John Bickerstaff" wrote: > In my own work, the risk to the business if every single client cannot > access search is so great, we would never consider putting ever

Re: Solr for Multi Tenant architecture

In my own work, the risk to the business if every single client cannot access search is so great, we would never consider putting everything in one. You should certainly ask that question of the business stakeholders before you decide. For that reason, I might recommend that each of the multiple

Re: Solr for Multi Tenant architecture

On 8/26/2016 1:13 PM, Chamil Jeewantha wrote: > We are using SolrCloud as the search provider of a multi-tenant cloud based > application. We have one schema for all the tenants. The indexes will have > large number(millions) of documents. > > As of our research, we have two options, > >- One l

Re: Solr for Multi Tenant architecture

There's no one right answer here. I've also seen a hybrid approach where there are multiple collections each of which has some number of tenants resident. Eventually, you need to think of some kind of partitioning, my rough number of documents for a single core is 50M (NOTE: I've seen between 10M a

Re: Solr for real time analytics system

Hi Rohit, Please take a loot at Streaming expressions & Parallel SQL Interface. That should meet many of your analytics requirement (aggregation queries like sum/average/groupby etc). https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions https://cwiki.apache.org/confluence/displa

Re: Solr for real time analytics system

Thanks Bhimavarapu for the information. We are creating our own dashboard, so probably wont need kibana/banana. I was more curious about Solr support for fast aggregation query over very large data set. As suggested, I guess elasticsearch has this capability. Is there any published metrics or dat

Re: Solr for real time analytics system

A few people did a real time analytics system with solr and talked about it at conferences. Maybe you'll find their presentations useful: https://www.youtube.com/results?search_query=solr%20real%20time%20analytics&oq=&gs_l= (esp. the first one: https://www.youtube.com/watch?v=PkoyCxBXAiA ) On Thu,

Re: Solr for real time analytics system

Hello Rohit, You can use the Banana project which was forked from Kibana , and works with all kinds of time series (and non-time series) data stored in Apache Solr . It uses Kibana's powerful dashboard configuration capabilities,

Re: Solr for Pictures

I was playing with exiftool (written in perl) and a custom java class built using metadata-extrator project (https://github.com/drewnoakes/metadata-extractor) and wondering if there is anything built into Solr or are there any best practices (general practices) to index pictures. On 10/29/15

Re: Solr for Pictures

Some extra googling yield this Wiki from a integration between Tika and a EXIFTool https://wiki.apache.org/tika/EXIFToolParser > On Oct 29, 2015, at 1:48 PM, Daniel Valdivia wrote: > > I think you can look into Tika for this https://tika.apache.or

RE: Solr for Pictures

Hi - Solr does integrate with Apache Tika, which happily accepts images and other media formats. I am not sure if EXIF is exposed though but you might want to try. Otherwise patch it up or use Tika in your own process that indexes data to Solr. https://cwiki.apache.org/confluence/display/solr

Re: Solr for Pictures

I think you can look into Tika for this https://tika.apache.org/ There’s handlers to integrate Tika and Solr, some context: https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Solr+Cell+using+Apache+Tika

Re: Solr For

You're describing two different entities: Job and Employee. Since they are clearly different in any way you will need two different cores with two different schemas. - Thanks, Michael -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-For-tp4097928p4098025.html Sent fr

Re: Solr For

Depends One core one schema file ... One solrconfig.xml . So if you want only one core then put all required fields of both search in one schema file and carry out your searches Otherwise make two cores having two schema file and perform searches accordingly ... On 27-Oct-2013 7:22 AM, "

Re: Solr For

On 27 October 2013 07:22, Baskar Sikkayan wrote: > > Hi, >Looking for solr config for Job Site. In a job site there are 2 main > searches. > > 1) Employee can search for job ( based on skill set, job location, title, > salary ) > 2) Employer can search for employees ( based on skill set, exp,

Re: Solr for routing a webapp

Why not pass the parameters using "?parameter1=value1¶meter2=value2" ? mvg, Jasper On Thu, Apr 26, 2012 at 9:03 PM, Paul Libbrecht wrote: > Or write your own query component mapping /solr/* in the web.xml, exposing > the request by a thread-local through a filter, and reading this setting the

Re: Solr for routing a webapp

Or write your own query component mapping /solr/* in the web.xml, exposing the request by a thread-local through a filter, and reading this setting the appropriate query parameters... Performance-wise, this seems quite reasonable I think. paul Le 26 avr. 2012 à 16:58, Paul Libbrecht a écrit :

Re: Solr for routing a webapp

Have you tried using mod_rewrite for this? paul Le 26 avr. 2012 à 15:16, Björn Zapadlo a écrit : > Hello, > > I'm thinking about using a Solr index for routing a webapp. > > I have pregenerated base urls in my index. E.g. > /foo/bar1 > /foo/bar2 > /foo/bar3 > /foo/bar4 > /bar/foo1 > /bar/foo2

Re: SolR for time-series data

Hi Alan, Solr can do this fast and easy, but I wonder if a simple key-value-store won't fit better for your suits. Do you really only need to query be chart_id, or do you also need to query by time range? In either case, as long as your data fits into an in-memory database, I would suggest

Re: SolR for time-series data

Sax is attractive, but I have found it lacking in practice. My primary issue is that in order to get sufficient recall for practical matching problems, I had to do enough query expansion that the speed advantage of inverted indexes went away. The OP was asking for blob storage, however, and I thi

Re: SolR for time-series data

Definitely should be possible. As an aside, I've also thought one could do more time series stuff. Have a look at the iSax stuff by Shieh and Koegh: http://www.cs.ucr.edu/~eamonn/iSAX/iSAX.html On Dec 3, 2011, at 12:10 PM, Alan Miller wrote: > Hi, > > I have a webapp that plots a bunch of t

Re: SolR for time-series data

Hi Alan, at my job we had a really succesful implementation similar to what you are proposing. With a classic RDBM, we hit serious performance issues so, we moved to solr to display time series of data. The 'trick' was to facet on a date field, to get 'counts' of data for a time series on a specifi

Re: Solr for finding similar word between two documents

xt.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message > From: rohan rai > To: solr-user@lucene.apache.org > Sent: Thu, February 3, 2011 2:35:39 PM > Subject: Re: Solr for finding similar word between two documents > &

Re: Solr for finding similar word between two documents

Lets say 1 have document(file) which is large and contains word inside it. And the 2nd document also is a text file. Problem is to find all those words in 2nd document which is present in first document when both of the files are large enough. Regards Rohan On Fri, Feb 4, 2011 at 1:01 AM, openv

Re: Solr for finding similar word between two documents

Rohan : what you want to do can be done with quite little effort if your document has a limited size (up to some Mo) with common and basic structures like Hasmap. Do you have any additional information on your problem so that we can give you more useful inputs ? 2011/2/3 Gora Mohanty > On Thu,

Re: Solr for finding similar word between two documents

On Thu, Feb 3, 2011 at 11:32 PM, rohan rai wrote: > Is there a way to use solr and get similar words between two document > (files). [...] This is *way* too vague t make any sense out of. Could you elaborate, as I could have sworn that what you seem to want is the essential function of a search e

Re: Solr for noSQL

Hi All I don't know if it answers any of your question but if you are interested by that check out : Lucandra ( Cassandra + Lucene) 2011/2/1 Steven Noels > On Tue, Feb 1, 2011 at 11:52 AM, Upayavira wrote: > > > > > > Apologies if my "nothing funky" sounded like you weren't doing cool > > st

Re: Solr for noSQL

On Tue, Feb 1, 2011 at 11:52 AM, Upayavira wrote: > > Apologies if my "nothing funky" sounded like you weren't doing cool > stuff. No offense whatsoever. I think my longer reply paints a more accurate light on what Lily means in terms of "SOLR for NoSQL", and it was your reaction who triggered

Re: Solr for noSQL

On Tue, 01 Feb 2011 07:22 +0100, "Steven Noels" wrote: > On Mon, Jan 31, 2011 at 9:38 PM, Upayavira wrote: > > > > > > > On Mon, 31 Jan 2011 08:40 -0500, "Estrada Groups" > > wrote: > > > What are the advantages of using something like HBase over your standard > > > Lucene index with Solr? It

Re: Solr for noSQL

On Mon, Jan 31, 2011 at 9:38 PM, Upayavira wrote: > > > On Mon, 31 Jan 2011 08:40 -0500, "Estrada Groups" > wrote: > > What are the advantages of using something like HBase over your standard > > Lucene index with Solr? It would seem to me like you'd be losing a lot of > > what Lucene has to off

Re: Solr for noSQL

On Mon, 31 Jan 2011 08:40 -0500, "Estrada Groups" wrote: > What are the advantages of using something like HBase over your standard > Lucene index with Solr? It would seem to me like you'd be losing a lot of > what Lucene has to offer!?! I think Steven is saying that he has an indexer app that

Re: Solr for noSQL

What are the advantages of using something like HBase over your standard Lucene index with Solr? It would seem to me like you'd be losing a lot of what Lucene has to offer!?! Adam On Jan 31, 2011, at 5:34 AM, Steven Noels wrote: > On Fri, Jan 28, 2011 at 1:30 AM, Jianbin Dai wrote: > >> Hi,

Re: Solr for noSQL

On Fri, Jan 28, 2011 at 1:30 AM, Jianbin Dai wrote: > Hi, > > > > Do we have data import handler to fast read in data from noSQL database, > specifically, MongoDB I am thinking to use? > > Or a more general question, how does Solr work with noSQL database? > Can't say anything about MongoDB, bu

Re: Solr for noSQL

501&tag=nl.e036' EARTH has a Right To Life, otherwise we all die. - Original Message From: Upayavira To: solr-user@lucene.apache.org Sent: Fri, January 28, 2011 1:41:42 AM Subject: Re: Solr for noSQL On Thu, 27 Jan 2011 21:38 -0800, "Dennis Gearon" wrote: &g

Re: Solr for noSQL

wn mistakes. It is usually a >> better >> idea to learn from others¹ mistakes, so you do not have to make them >> yourself. >> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036' >> >> >> EARTH has a Right To Life, >> o

Re: Solr for noSQL

rity/?p=4501&tag=nl.e036' > > > EARTH has a Right To Life, > otherwise we all die. > > > > - Original Message ---- > From: Lance Norskog > To: solr-user@lucene.apache.org > Sent: Thu, January 27, 2011 9:33:25 PM > Subject: Re: Solr for noSQL > > The

Re: Solr for noSQL

On Thu, 27 Jan 2011 21:38 -0800, "Dennis Gearon" wrote: > Why not make one's own DIH handler, Lance? Personally, I don't like that approach. Solr is best related to as something of a black box that you configure, then push content to. Having Solr know about your data sources, and pull content i

Re: Solr for noSQL

On Fri, Jan 28, 2011 at 6:00 AM, Jianbin Dai wrote: [...] > Do we have data import handler to fast read in data from noSQL database, > specifically, MongoDB I am thinking to use? [...] Have you tried the links that a Google search turns up? Some of them look like pretty good prospects. Regards,

Re: Solr for noSQL

Do we have performance measurement? Would it be much slower compared to other DIH? > There no special connectors available to read from the key-value > stores like memcache/cassandra/mongodb. You would have to get a Java > client library for the DB and code your own dataimporthandler > datasourc

Re: Solr for noSQL

ic.com.com/security/?p=4501&tag=nl.e036' EARTH has a Right To Life, otherwise we all die. - Original Message From: Lance Norskog To: solr-user@lucene.apache.org Sent: Thu, January 27, 2011 9:33:25 PM Subject: Re: Solr for noSQL There no special connectors available to read

Re: Solr for noSQL

There no special connectors available to read from the key-value stores like memcache/cassandra/mongodb. You would have to get a Java client library for the DB and code your own dataimporthandler datasource. I cannot recommend this; you should make your own program to read data and upload to Solr

Re: SOLR for Log analysis feasibility

my thoughts exactly that it may seem fairly straightforward but i fear for when a client wants a perfectly reasonable new feature to be added to their report and SOLR simply cannot support this feature. i am hoping we wont have any real issues with scalability as Loggly because we dont index and

Re: SOLR for Log analysis feasibility

We do a lot of precisely this sort of thing. Ours is a commercial product (Honeycomb Lexicon) that extracts behavioural information from logs, events and network data (don't worry, I'm not pushing this on you!) - only to say that there are a lot of considerations beyond base Solr when it comes to h

Re: SOLR for Log analysis feasibility

i know, it's not solr .. but perhaps you should have a look at it: http://www.cloudera.com/blog/2010/09/using-flume-to-collect-apache-2-web-server-logs/ On Tue, Nov 30, 2010 at 12:58 PM, Peter Karich wrote: > take a look into this: > http://vimeo.com/16102543 > > for that amount of data it isn'

Re: SOLR for Log analysis feasibility

take a look into this: http://vimeo.com/16102543 for that amount of data it isn't that easy :-) We are looking into building a reporting feature and investigating solutions which will allow us to search though our logs for downloads, searches and view history. Each log item is relatively smal

Re: Solr for statistical data

Does this do what you want? http://wiki.apache.org/solr/StatsComponent I can see that "group by" is a possible enhancement to this component. Kjetil Ødegaard wrote: Hi all, we're currently using Solr 1.4.0 in a project for statistical data, where we group and sum a number of "double" values.

Re: Solr for statistical data

Set up your JVM to produce the heap dumps in case of OOM and try to analyze them with a profiler like YourKit. This could give you some ideas on what takes memory and what potentially could be reduced. Sometimes the cache settings could be adjusted without significant performance toll etc. See what

Re: Solr for statistical data

I don't know if this thread might help with your problems any, but it might give some pointers: http://lucene.472066.n3.nabble.com/Tuning-Solr-caches-with-high-commit-rates-NRT-td1461275.html --Thoma

Re: Solr for statistical data

On Thu, Sep 16, 2010 at 11:48 AM, Peter Karich wrote: > Hi Kjetil, > > is this custom component (which performes groub by + calcs stats) > somewhere available? > I would like to do something similar. Would you mind to share if it > isn't already available? > > The grouping stuff sounds similar to

Re: Solr for statistical data

Hi Kjetil, is this custom component (which performes groub by + calcs stats) somewhere available? I would like to do something similar. Would you mind to share if it isn't already available? The grouping stuff sounds similar to https://issues.apache.org/jira/browse/SOLR-236 where you can have me

Re: Solr for multiple websites

; Also, how can we setup load balancing between multiple solrs > > Thanks > > Hitendra > > -Original Message- > From: Grijesh.singh [mailto:pintu.grij...@gmail.com] > Sent: Thursday, August 19, 2010 10:25 AM > To: solr-user@lucene.apache.org > Subject: Re: Sol

RE: Solr for multiple websites

.   -Original message- From: Hitendra Molleti Sent: Thu 19-08-2010 14:38 To: solr-user@lucene.apache.org; CC: 'Jonathan DeMello' ; amer.mahf...@itp.com; 'Nishchint Yogishwar' ; Subject: RE: Solr for multiple websites Thanks Girjesh. Can you please let me know what are the p

RE: Solr for multiple websites

: solr-user@lucene.apache.org Subject: Re: Solr for multiple websites Using multicore is the right approach -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-for-multiple-websites-tp1173220p1219 772.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr for multiple websites

Using multicore is the right approach -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-for-multiple-websites-tp1173220p1219772.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr for reporting purposes

this right? - Original Message - From: "adeelmahmood" To: solr-user@lucene.apache.org Sent: Monday, 1 March, 2010 2:05:44 PM Subject: Re: solr for reporting purposes well thanks for ur reply .. as far as the load goes again I think most of the reports will be for 1000-4000

Re: solr for reporting purposes

t; as such, but a problem generally of unrestricted adhoc reporting > > > > > - Original Message - > From: "adeelmahmood" > To: solr-user@lucene.apache.org > Sent: Saturday, 27 February, 2010 5:57:00 AM > Subject: Re: solr for reporting purposes

Re: solr for reporting purposes

To: solr-user@lucene.apache.org Sent: Saturday, 27 February, 2010 5:57:00 AM Subject: Re: solr for reporting purposes I just want to clarify if its not obvious .. that the reason I am concerned about the performance of solr is becaues for reporting requests I will probably have to request all

Re: solr for reporting purposes

I just want to clarify if its not obvious .. that the reason I am concerned about the performance of solr is becaues for reporting requests I will probably have to request all result rows at the same time .. instead of 10 or 20 adeelmahmood wrote: > > we are trying to use solr for somewhat of a

Re: Solr for large volume data processing with minimal full-text serach

If you need anything close to realtime (~ few seconds) hadoop and its ilk is not a choice. Solr is fine. But be prepared to dedicate a lot of hardware for that On Fri, Nov 7, 2008 at 10:53 PM, souravm <[EMAIL PROTECTED]> wrote: > Hi Shalin, > > Thanks for your input. > > Yes I agree that my applic

Re: Solr for Whole Web Search

If that is the case you should look @ the DataImportHandler examples as they can already index RSS, im doing it now for ~ a dozen feeds on an hourly basis. (This is also for any XML-based feed for XHTML, XML, etc). I find Nutch more useful for plain vanilla HTML (something that was built

Re: Solr for Whole Web Search

Grant thanks for the response. A couple of other people have recommended trying the Nutch + Solr approach, but I am not sure what the real benefit of doing that is. Since Nutch provides most of the same features as Solr and Solr has some nice additional features (like spell checking, incre

Re: Solr for Whole Web Search

On Oct 22, 2008, at 7:57 AM, John Martyniak wrote: I am very new to Solr, but I have played with Nutch and Lucene. Has anybody used Solr for a whole web indexing application? Which Spider did you use? How does it compare to Nutch? There is a patch that combines Nutch + Solr. Nutch is used

Re: solr for corpus?

-- Original Message From: Chris Hostetter <[EMAIL PROTECTED]> To: solr-user@lucene.apache.org Sent: Monday, May 14, 2007 6:40:32 PM Subject: Re: solr for corpus? : 3. Lemma and PoS-tag (Part-of-Speech tag) or generally additional keys : on the word-level, : ie for every word we also have it

Re: solr for corpus?

: 3. Lemma and PoS-tag (Part-of-Speech tag) or generally additional keys : on the word-level, : ie for every word we also have its lemma-value and its PoS. : or in newer systems with xml-attributes: : trees : : Important is, that it has to be possible to mix this various layers in : one query, eg

RE: solr for corpus?

Regarding the Lemma and PoS-tag requirement: you might handle this by inserting each word as its own document, with "lemma", "pos", and "word" fields, thereby allowing you lots of search flexibility. You could also include ID fields for the item and (if necessary) part (chapter etc.) and use these

Re: solr for corpus?

Hi matej, since I didn't see anyone answering your question yet, I'll have a go at it, but I'm not one of the Solr developers, I've just used it so far and am very happy with it. I use it for searching literary texts, storing information from a SQL database in the Solr documents as metadata fo