Solrj possible deadlock
Hello, I was running a batch index the other day using the Solrj EmbeddedSolrServer when the process abruptly froze in it's tracks after running for about 4-5 hours and indexing ~400K documents. There were no document locks so it would seem likely that there was some kind of thread deadlock. I was hoping someone might be able to tell me some information about the following thread dump taken at the time: Full thread dump OpenJDK Client VM (1.6.0-b09 mixed mode): "DestroyJavaVM" prio=10 tid=0x9322a800 nid=0xcef waiting on condition [0x..0x0018a044] java.lang.Thread.State: RUNNABLE "Java2D Disposer" daemon prio=10 tid=0x0a28cc00 nid=0xf1c in Object.wait() [0x0311d000..0x0311def4] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x97a96840> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133) - locked <0x97a96840> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) at sun.java2d.Disposer.run(Disposer.java:143) at java.lang.Thread.run(Thread.java:636) "pool-1-thread-1" prio=10 tid=0x93a26c00 nid=0xcf7 waiting on condition [0x08a6a000..0x08a6b074] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x967acfd0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1978) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) "Low Memory Detector" daemon prio=10 tid=0x93a00c00 nid=0xcf5 runnable [0x..0x] java.lang.Thread.State: RUNNABLE "CompilerThread0" daemon prio=10 tid=0x09fe9800 nid=0xcf4 waiting on condition [0x..0x096a7af4] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" daemon prio=10 tid=0x09fe8800 nid=0xcf3 waiting on condition [0x..0x] java.lang.Thread.State: RUNNABLE "Finalizer" daemon prio=10 tid=0x09fd7000 nid=0xcf2 in Object.wait() [0x005ca000..0x005caef4] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x966e6d40> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133) - locked <0x966e6d40> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177) "Reference Handler" daemon prio=10 tid=0x09fd2c00 nid=0xcf1 in Object.wait() [0x00579000..0x00579d74] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x966e6dc8> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) - locked <0x966e6dc8> (a java.lang.ref.Reference$Lock) "VM Thread" prio=10 tid=0x09fcf800 nid=0xcf0 runnable "VM Periodic Task Thread" prio=10 tid=0x93a02400 nid=0xcf6 waiting on condition JNI global references: 1072 Heap def new generation total 36288K, used 23695K [0x93f1, 0x9667, 0x9667) eden space 32256K, 73% used [0x93f1, 0x95633f60, 0x95e9) from space 4032K, 0% used [0x95e9, 0x95e9, 0x9628) to space 4032K, 0% used [0x9628, 0x9628, 0x9667) tenured generation total 483968K, used 72129K [0x9667, 0xb3f1, 0xb3f1) the space 483968K, 14% used [0x9667, 0x9ace04b8, 0x9ace0600, 0xb3f1) compacting perm gen total 23040K, used 22983K [0xb3f1, 0xb559, 0xb7f1) the space 23040K, 99% used [0xb3f1, 0xb5581ff8, 0xb5582000, 0xb559) No shared spaces configured. Cheers. Brett. -- View this message in context: http://www.nabble.com/Solrj-possible-deadlock-tp25530146p25530146.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solrj possible deadlock
I had the same problem again yesterday except the process halted after about 20mins this time. pof wrote: > > Hello, I was running a batch index the other day using the Solrj > EmbeddedSolrServer when the process abruptly froze in it's tracks after > running for about 4-5 hours and indexing ~400K documents. There were no > document locks so it would seem likely that there was some kind of thread > deadlock. I was hoping someone might be able to tell me some information > about the following thread dump taken at the time: > > Full thread dump OpenJDK Client VM (1.6.0-b09 mixed mode): > > "DestroyJavaVM" prio=10 tid=0x9322a800 nid=0xcef waiting on condition > [0x..0x0018a044] >java.lang.Thread.State: RUNNABLE > > "Java2D Disposer" daemon prio=10 tid=0x0a28cc00 nid=0xf1c in Object.wait() > [0x0311d000..0x0311def4] >java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x97a96840> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133) > - locked <0x97a96840> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) > at sun.java2d.Disposer.run(Disposer.java:143) > at java.lang.Thread.run(Thread.java:636) > > "pool-1-thread-1" prio=10 tid=0x93a26c00 nid=0xcf7 waiting on condition > [0x08a6a000..0x08a6b074] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x967acfd0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1978) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:636) > > "Low Memory Detector" daemon prio=10 tid=0x93a00c00 nid=0xcf5 runnable > [0x..0x] >java.lang.Thread.State: RUNNABLE > > "CompilerThread0" daemon prio=10 tid=0x09fe9800 nid=0xcf4 waiting on > condition [0x..0x096a7af4] >java.lang.Thread.State: RUNNABLE > > "Signal Dispatcher" daemon prio=10 tid=0x09fe8800 nid=0xcf3 waiting on > condition [0x..0x] >java.lang.Thread.State: RUNNABLE > > "Finalizer" daemon prio=10 tid=0x09fd7000 nid=0xcf2 in Object.wait() > [0x005ca000..0x005caef4] >java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x966e6d40> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133) > - locked <0x966e6d40> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) > at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177) > > "Reference Handler" daemon prio=10 tid=0x09fd2c00 nid=0xcf1 in > Object.wait() [0x00579000..0x00579d74] >java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x966e6dc8> (a java.lang.ref.Reference$Lock) > at java.lang.Object.wait(Object.java:502) > at > java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) > - locked <0x966e6dc8> (a java.lang.ref.Reference$Lock) > > "VM Thread" prio=10 tid=0x09fcf800 nid=0xcf0 runnable > > "VM Periodic Task Thread" prio=10 tid=0x93a02400 nid=0xcf6 waiting on > condition > > JNI global references: 1072 > > Heap > def new generation total 36288K, used 23695K [0x93f1, 0x9667, > 0x9667) > eden space 32256K, 73% used [0x93f1, 0x95633f60, 0x95e9) > from space 4032K, 0% used [0x95e9, 0x95e9, 0x9628) > to space 4032K, 0% used [0x9628, 0x9628, 0x9667) > tenured generation total 483968K, used 72129K [0x9667, 0xb3f1, > 0xb3f1) >the space 483968K, 14% used [0x9667, 0x9ace04b8, 0x9ace0600, > 0xb3f1) > compacting perm gen total 23040K, used 22983K [0xb3f1, 0xb559, > 0xb7f1) >the space 23040K, 99% used [0xb3f1, 0xb5581ff8, 0xb5582000, > 0xb559) > No shared spaces configured. > > Cheers. Brett. > -- View this message in context: http://www.nabble.com/Solrj-possible-deadlock-tp25530146p25531321.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solrj possible deadlock
Well, in the same processes I am using a jdbc connection to get all the relative paths to the documents I want to index, then I parse the documents to plain text using tones of open source libraries like POI, PFDBox etc.(which might account for java2d) then I add them to the index and commit every 2000 documents. I write a db row for each row I index so I can resume where I left off after a crash or exception. My app will happily index for hours then after it hangs, the resumed indexing run will only last a few additional minutes! The thread dumps look the same. Cheers. ryantxu wrote: > > do you have anything custom going on? > > The fact that the lock is in java2d seems suspicious... > > > On Sep 23, 2009, at 7:01 PM, pof wrote: > >> >> I had the same problem again yesterday except the process halted >> after about >> 20mins this time. >> >> >> pof wrote: >>> >>> Hello, I was running a batch index the other day using the Solrj >>> EmbeddedSolrServer when the process abruptly froze in it's tracks >>> after >>> running for about 4-5 hours and indexing ~400K documents. There >>> were no >>> document locks so it would seem likely that there was some kind of >>> thread >>> deadlock. I was hoping someone might be able to tell me some >>> information >>> about the following thread dump taken at the time: >>> >>> Full thread dump OpenJDK Client VM (1.6.0-b09 mixed mode): >>> >>> "DestroyJavaVM" prio=10 tid=0x9322a800 nid=0xcef waiting on condition >>> [0x..0x0018a044] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Java2D Disposer" daemon prio=10 tid=0x0a28cc00 nid=0xf1c in >>> Object.wait() >>> [0x0311d000..0x0311def4] >>> java.lang.Thread.State: WAITING (on object monitor) >>>at java.lang.Object.wait(Native Method) >>>- waiting on <0x97a96840> (a java.lang.ref.ReferenceQueue >>> $Lock) >>>at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: >>> 133) >>>- locked <0x97a96840> (a java.lang.ref.ReferenceQueue$Lock) >>>at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: >>> 149) >>>at sun.java2d.Disposer.run(Disposer.java:143) >>>at java.lang.Thread.run(Thread.java:636) >>> >>> "pool-1-thread-1" prio=10 tid=0x93a26c00 nid=0xcf7 waiting on >>> condition >>> [0x08a6a000..0x08a6b074] >>> java.lang.Thread.State: WAITING (parking) >>>at sun.misc.Unsafe.park(Native Method) >>>- parking to wait for <0x967acfd0> (a >>> java.util.concurrent.locks.AbstractQueuedSynchronizer >>> $ConditionObject) >>>at >>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) >>>at >>> java.util.concurrent.locks.AbstractQueuedSynchronizer >>> $ConditionObject.await(AbstractQueuedSynchronizer.java:1978) >>>at >>> java >>> .util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java: >>> 386) >>>at >>> java >>> .util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java: >>> 1043) >>>at >>> java >>> .util >>> .concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: >>> 1103) >>>at >>> java.util.concurrent.ThreadPoolExecutor >>> $Worker.run(ThreadPoolExecutor.java:603) >>>at java.lang.Thread.run(Thread.java:636) >>> >>> "Low Memory Detector" daemon prio=10 tid=0x93a00c00 nid=0xcf5 >>> runnable >>> [0x..0x] >>> java.lang.Thread.State: RUNNABLE >>> >>> "CompilerThread0" daemon prio=10 tid=0x09fe9800 nid=0xcf4 waiting on >>> condition [0x..0x096a7af4] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Signal Dispatcher" daemon prio=10 tid=0x09fe8800 nid=0xcf3 waiting >>> on >>> condition [0x..0x] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Finalizer" daemon prio=10 tid=0x09fd7000 nid=0xcf2 in Object.wait() >>> [0x005ca000..0x005caef4] >>> java.lang.Thread.State: WAITING (on object monitor) >>>at java.lang.Object.wait(Native Method) >>>- waiting on <0x966e6d40> (a java.lang.ref.ReferenceQueue >>> $Lock) >>>at java.lang.ref.Referen
EmbeddedSolrServer seperate process
Hi, one question: Will EmbeddedSolrServer work on a seperate java process than the solr servlet(start.jar) as long as it is run on the same machine? Thanks. -- View this message in context: http://www.nabble.com/EmbeddedSolrServer-seperate-process-tp24046680p24046680.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: EmbeddedSolrServer seperate process
It certain does, thank you. Shalin Shekhar Mangar wrote: > > On Tue, Jun 16, 2009 at 8:15 AM, pof wrote: > >> >> Hi, one question: Will EmbeddedSolrServer work on a seperate java process >> than the solr servlet(start.jar) as long as it is run on the same >> machine? >> >> > EmbeddedSolrServer is run in the same process as the application which > uses > it. CommonsHttpSolrServer is used by an application to communicate with > Solr > running in a separate host (or JVM) using HTTP. The start.jar is the jetty > servlet container which can be used to host solr. > > Does that answer your question? > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://www.nabble.com/EmbeddedSolrServer-seperate-process-tp24046680p24047849.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Jetty confusion
Hi, I am currently trying to write a Jetty embedded java app that implements
SOLR and uses SOLRJ by excepting posts telling it to do a batch index, or a
deletion or what have you. At this point I am completely lost trying to
follow http://wiki.apache.org/solr/SolrJetty . In my constructor I am doing
the following call:
Server server = new Server();
XmlConfiguration configuration = new XmlConfiguration(new
FileInputStream("solrjetty.xml"));
My xml has two calls, an addConnector to configure the port etc. and the
addWebApplication as specified on the solr wiki. When running the app I get
this:
Exception in thread "main" java.lang.IllegalStateException: No Method: /solr/*/webapps/solr.wartrueorg/mortbay/jetty/servlet/webdefault.xml/solr/home/solr/home on class
org.mortbay.jetty.Server
Can anyone point me in the right direction? Thanks.
--
View this message in context:
http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24087264.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Jetty confusion
My problem is that my project doesn't compile and I have know way of knowing
if I'm on the right track code wise. There just isn't any comprehensive
guide out there for having a solr/jetty app.
Development Team wrote:
>
> Hey,
> So... I'm assuming your problem is that you're having trouble
> deploying
> Solr in Jetty? Or is your problem that it's deploying just fine but your
> code throws an exception when you try to run it?
> I am running Solr in Jetty, and I just copied the war into the
> webapps
> directory and it worked. It was accessible under /solr, and it was
> accessible under the port that Jetty has as its HTTP listener (which is
> probably 8080 by default, but probably won't be 8983). To specify the
> solr-home I use a Java system property (instead of the JNDI way) since I
> already have other necessary system properties for my apps. So if your
> problem turns out to be with the JNDI, sorry I won't be of much help.
> Hope that helps...
>
> - Daryl.
>
>
> On Thu, Jun 18, 2009 at 2:44 AM, pof wrote:
>
>>
>> Hi, I am currently trying to write a Jetty embedded java app that
>> implements
>> SOLR and uses SOLRJ by excepting posts telling it to do a batch index, or
>> a
>> deletion or what have you. At this point I am completely lost trying to
>> follow http://wiki.apache.org/solr/SolrJetty . In my constructor I am
>> doing
>> the following call:
>>
>> Server server = new Server();
>> XmlConfiguration configuration = new XmlConfiguration(new
>> FileInputStream("solrjetty.xml"));
>>
>> My xml has two calls, an addConnector to configure the port etc. and the
>> addWebApplication as specified on the solr wiki. When running the app I
>> get
>> this:
>>
>> Exception in thread "main" java.lang.IllegalStateException: No Method:
>> > name="addWebApplication">/solr/*/webapps/solr.war> name="extractWAR">true>
>> name="defaultsDescriptor">org/mortbay/jetty/servlet/webdefault.xml> name="addEnvEntry">/solr/home> type="String">/solr/home on class
>> org.mortbay.jetty.Server
>>
>> Can anyone point me in the right direction? Thanks.
>> --
>> View this message in context:
>> http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24087264.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context:
http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24099696.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Jetty confusion
Development Team wrote: > > To specify the > solr-home I use a Java system property (instead of the JNDI way) since I > already have other necessary system properties for my apps. > Could you please give me a concrete example of how you did this? There is no example code or commandline examples to be found. Cheers, Brett. -- View this message in context: http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24104378.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Jetty confusion
Hi, the problem was my xml configurations. I am now using a java process (seperate from the webserver) using the EmbeddedSolrServer method and post a commit to the webserver at the end to sync the two cores which seems to work well for me. Thanks for the help. Development Team wrote: > > Hi Brett, > Well, I'm running Solr in Jetty with JBoss, so I used the JBoss > method > of specifying properties (properties-service.xml). However, you can supply > the solr-home to the command-line when you start Jetty by using a > parameter > like, "-Dsolr.solr.home=C:\solr". You can do it like how they do it for > Tomcat: http://wiki.apache.org/solr/SolrTomcat?highlight=(solr.home) > > You mention your code is not compiling... the code should be able to > compile whether or not you can actually start solr with the right > solr-home. > It should also compile regardless of how what container you deploy Solr > into. What exactly are you trying to do besides getting Solr to start in > Jetty? > > - Daryl. > > > > On Thu, Jun 18, 2009 at 9:58 PM, pof wrote: > >> >> >> Development Team wrote: >> > >> > To specify the >> > solr-home I use a Java system property (instead of the JNDI way) since >> I >> > already have other necessary system properties for my apps. >> > >> >> Could you please give me a concrete example of how you did this? There is >> no >> example code or commandline examples to be found. >> >> Cheers, Brett. >> >> -- >> View this message in context: >> http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24104378.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24142282.html Sent from the Solr - User mailing list archive at Nabble.com.
Sorlj when to commit?
Hi, I am doing a large batch (thousands) of insertions to my index using an EmbeddedSolrServer. I was wondering how often should I use server.commit() as I am trying to avoid unecessary bottlenecks. Thanks, Brett. -- View this message in context: http://www.nabble.com/Sorlj-when-to-commit--tp24142326p24142326.html Sent from the Solr - User mailing list archive at Nabble.com.
Solrj no search results
Hi, I'm using an EmbeddedSolrServer. Adding documents with the example jetty server example using this method worked fine: doc1.addField( "id", "id1"); doc1.addField( "name", "doc1"); doc1.addField( "price", 10); server.add(doc1) However now I have changed the schema.xml so I can use my own fields and the documents will add to index (no compilation errors), however I will not get any search results back what so ever. Any Ideas? Thanks. -- View this message in context: http://www.nabble.com/Solrj-no-search-results-tp24179484p24179484.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr document security
Hi, I am wanting to add document-level security that works as following: An external process makes a query to the index, depending on their security allowences based of a login id a list of hits are returned minus any the user are meant to know even exist. I was thinking maybe a custom filter with a JDBC connection to check security of the user vs. the document. I'm not sure how I would add the filter or how to write the filter or how to get the login id from a GET parameter. Any suggestions, comments etc.? Thanks. Brett. -- View this message in context: http://www.nabble.com/Solr-document-security-tp24197620p24197620.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr document security
Thats what I was going to do originally, however what is stopping a user from
simply running a search through http://localhost:8983/solr/admin/ of the
index server?
Norberto Meijome-6 wrote:
>
> On Wed, 24 Jun 2009 23:20:26 -0700 (PDT)
> pof wrote:
>
>>
>> Hi, I am wanting to add document-level security that works as following:
>> An
>> external process makes a query to the index, depending on their security
>> allowences based of a login id a list of hits are returned minus any the
>> user are meant to know even exist. I was thinking maybe a custom filter
>> with
>> a JDBC connection to check security of the user vs. the document. I'm not
>> sure how I would add the filter or how to write the filter or how to get
>> the
>> login id from a GET parameter. Any suggestions, comments etc.?
>
> Hi Brett,
> (keeping in mind that i've been away from SOLR for 8 months, but i
> dont think this was added of late)
>
> standard approach is to manage security @ your
> application layer, not @ SOLR. ie, search, return documents (which should
> contain some kind of data to identify their ACL ) and then you can decide
> whether to show it or not.
>
> HIH
> _
> {Beto|Norberto|Numard} Meijome
>
> "They never open their mouths without subtracting from the sum of human
> knowledge." Thomas Brackett Reed
>
> I speak for myself, not my employer. Contents may be hot. Slippery when
> wet.
> Reading disclaimers makes you go blind. Writing them is worse. You have
> been
> Warned.
>
>
--
View this message in context:
http://www.nabble.com/Solr-document-security-tp24197620p24212752.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr document security
I want to use filtering or similar, any help?
Otis Gospodnetic wrote:
>
>
> That URL to your Solr Admin page should never be exposed to the outside
> world. You can play with network, routing, DNS and other similar things
> to make sure one can't get to this from the outside even if the URL is
> know.
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
> - Original Message
>> From: pof
>> To: solr-user@lucene.apache.org
>> Sent: Thursday, June 25, 2009 7:40:12 PM
>> Subject: Re: Solr document security
>>
>>
>> Thats what I was going to do originally, however what is stopping a user
>> from
>> simply running a search through http://localhost:8983/solr/admin/ of the
>> index server?
>>
>>
>> Norberto Meijome-6 wrote:
>> >
>> > On Wed, 24 Jun 2009 23:20:26 -0700 (PDT)
>> > pof wrote:
>> >
>> >>
>> >> Hi, I am wanting to add document-level security that works as
>> following:
>> >> An
>> >> external process makes a query to the index, depending on their
>> security
>> >> allowences based of a login id a list of hits are returned minus any
>> the
>> >> user are meant to know even exist. I was thinking maybe a custom
>> filter
>> >> with
>> >> a JDBC connection to check security of the user vs. the document. I'm
>> not
>> >> sure how I would add the filter or how to write the filter or how to
>> get
>> >> the
>> >> login id from a GET parameter. Any suggestions, comments etc.?
>> >
>> > Hi Brett,
>> > (keeping in mind that i've been away from SOLR for 8 months, but i
>> > dont think this was added of late)
>> >
>> > standard approach is to manage security @ your
>> > application layer, not @ SOLR. ie, search, return documents (which
>> should
>> > contain some kind of data to identify their ACL ) and then you can
>> decide
>> > whether to show it or not.
>> >
>> > HIH
>> > _
>> > {Beto|Norberto|Numard} Meijome
>> >
>> > "They never open their mouths without subtracting from the sum of human
>> > knowledge." Thomas Brackett Reed
>> >
>> > I speak for myself, not my employer. Contents may be hot. Slippery when
>> > wet.
>> > Reading disclaimers makes you go blind. Writing them is worse. You have
>> > been
>> > Warned.
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Solr-document-security-tp24197620p24212752.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
--
View this message in context:
http://www.nabble.com/Solr-document-security-tp24197620p24246624.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr document security
The custom dispatcher filter example on
http://wiki.apache.org/solr/SolrSecurity seems to be what I need however I
don't understand the code. Could someone break it down for me? Thanks.
pof wrote:
>
> I want to use filtering or similar, any help?
>
>
> Otis Gospodnetic wrote:
>>
>>
>> That URL to your Solr Admin page should never be exposed to the outside
>> world. You can play with network, routing, DNS and other similar things
>> to make sure one can't get to this from the outside even if the URL is
>> know.
>>
>> Otis
>> --
>> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>>
>>
>>
>> - Original Message
>>> From: pof
>>> To: solr-user@lucene.apache.org
>>> Sent: Thursday, June 25, 2009 7:40:12 PM
>>> Subject: Re: Solr document security
>>>
>>>
>>> Thats what I was going to do originally, however what is stopping a user
>>> from
>>> simply running a search through http://localhost:8983/solr/admin/ of the
>>> index server?
>>>
>>>
>>> Norberto Meijome-6 wrote:
>>> >
>>> > On Wed, 24 Jun 2009 23:20:26 -0700 (PDT)
>>> > pof wrote:
>>> >
>>> >>
>>> >> Hi, I am wanting to add document-level security that works as
>>> following:
>>> >> An
>>> >> external process makes a query to the index, depending on their
>>> security
>>> >> allowences based of a login id a list of hits are returned minus any
>>> the
>>> >> user are meant to know even exist. I was thinking maybe a custom
>>> filter
>>> >> with
>>> >> a JDBC connection to check security of the user vs. the document. I'm
>>> not
>>> >> sure how I would add the filter or how to write the filter or how to
>>> get
>>> >> the
>>> >> login id from a GET parameter. Any suggestions, comments etc.?
>>> >
>>> > Hi Brett,
>>> > (keeping in mind that i've been away from SOLR for 8 months, but i
>>> > dont think this was added of late)
>>> >
>>> > standard approach is to manage security @ your
>>> > application layer, not @ SOLR. ie, search, return documents (which
>>> should
>>> > contain some kind of data to identify their ACL ) and then you can
>>> decide
>>> > whether to show it or not.
>>> >
>>> > HIH
>>> > _
>>> > {Beto|Norberto|Numard} Meijome
>>> >
>>> > "They never open their mouths without subtracting from the sum of
>>> human
>>> > knowledge." Thomas Brackett Reed
>>> >
>>> > I speak for myself, not my employer. Contents may be hot. Slippery
>>> when
>>> > wet.
>>> > Reading disclaimers makes you go blind. Writing them is worse. You
>>> have
>>> > been
>>> > Warned.
>>> >
>>> >
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Solr-document-security-tp24197620p24212752.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>>
>
>
--
View this message in context:
http://www.nabble.com/Solr-document-security-tp24197620p24246685.html
Sent from the Solr - User mailing list archive at Nabble.com.
Getting hits in RequestHandler
Hi, I am writing my own request handler and I was wondering how I go about get a list of hits back. Thanks. -- View this message in context: http://www.nabble.com/Getting-hits-in-RequestHandler-tp24248810p24248810.html Sent from the Solr - User mailing list archive at Nabble.com.
Deleting from SolrQueryResponse
Hi, I was wondering if any has had luck deleting added documents to SolrQueryResponse? I am subclassing StandardRequestHandler and after I run the handle request body method (super.handleRequestBody(req, rsp);) I won't to filter out some of the hits. Thanks. Brett -- View this message in context: http://www.nabble.com/Deleting-from-SolrQueryResponse-tp24266686p24266686.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Deleting from SolrQueryResponse
This thread got buried last night, so... bump. pof wrote: > > Hi, I was wondering if any has had luck deleting added documents from > SolrQueryResponse? I am subclassing StandardRequestHandler and after I run > the handle request body method (super.handleRequestBody(req, rsp);) I want > to filter out some of the hits. > > Thanks. Brett > -- View this message in context: http://www.nabble.com/Deleting-from-SolrQueryResponse-tp24266686p24281972.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Deleting from SolrQueryResponse
hossman wrote: > > one thing to keep in mind however is that post-processing a DocList to > filter stuff out is almost never a good idea -- things get really > convoluted when you think about dealing with pagination and except for > some really trivial use cases you can never know what your upper bound > should be when deciding how many hits to request from underlying > IndexSearcher. You're usually better off restructuring your problem so > that you can construct a Query/Filter/DocSet that you want to filter by > first and *then* executing the search to generate the DocList in a single > pass. > I was wanting to edit the DocList in a custom SearchComponent to be executed after the QueryComponent. I do not require facetting etc. If do not want facetted results, will I still need to take any special steps not to break the doclist? hossman wrote: > > PS: replying to your own message (or reposting it) to "bump" it up > generally doesn't encourage replies any faster -- it just increases the > volume of traffic on the list, and if anything antagonizes people and > makes them less interested in responding. > Okay, sorry I wasn't certain on the protocol on that. Thanks, Brett. -- View this message in context: http://www.nabble.com/Deleting-from-SolrQueryResponse-tp24266686p24315607.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Deleting from SolrQueryResponse
Okay. So still, how would I go about creating a new DocList and Docset as they cannot be instantiated? Thanks, Brett. hossman wrote: > > > : > one thing to keep in mind however is that post-processing a DocList to > : > filter stuff out is almost never a good idea -- things get really > : > convoluted when you think about dealing with pagination and except for > : > some really trivial use cases you can never know what your upper bound > : > should be when deciding how many hits to request from underlying > : > IndexSearcher. You're usually better off restructuring your problem > so > : > that you can construct a Query/Filter/DocSet that you want to filter > by > : > first and *then* executing the search to generate the DocList in a > single > : > pass. > : > > : I was wanting to edit the DocList in a custom SearchComponent to be > executed > : after the QueryComponent. I do not require facetting etc. If do not want > : facetted results, will I still need to take any special steps not to > break > : the doclist? > > i didn't say anything about faceting ... i was talking about pagination > (ie: making sure you return the X number of rows the user asked for, and > making sure that when they *next* X rows you start at the right spot. > > > > > -Hoss > > > -- View this message in context: http://www.nabble.com/Deleting-from-SolrQueryResponse-tp24266686p24456659.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 1.4 Release Date
Any updates on this? Cheers. Gurjot Singh wrote: > > Hi, I am curious to know when is the scheduled/tentative release date of > Solr 1.4. > > Thanks, > Gurjot > > -- View this message in context: http://www.nabble.com/Solr-1.4-Release-Date-tp23260381p24473570.html Sent from the Solr - User mailing list archive at Nabble.com.
DocList Pagination
Hi, I am try to get the next DocList "page" in my custom search component. Could I get a code example of this? Cheers. -- View this message in context: http://www.nabble.com/DocList-Pagination-tp24581850p24581850.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Regarding Response Builder
I would just do something like this:
String myParam = req.getParams().get("xparam");
where "xparam" is:
http://localhost:8983/solr/select/?q=dog&xparam=something&start=0&rows=10&indent=on
Kartik1 wrote:
>
> The responsebuiilder class has SolrQueryRequest as public type. Using
> SolrQueryRequest we can get a list of SolrParams like
>
> SolrParams params = req.getParams();
>
> Now I want to get the values of those params. What should be the approach
> as SolrParams is an abstract class and its get(String) method is abstract?
>
> Best regards,
> Amandeep Singh
>
>
>
--
View this message in context:
http://www.nabble.com/Regarding-Response-Builder-tp24456722p24600481.html
Sent from the Solr - User mailing list archive at Nabble.com.
