Stopping Solr results in broken Solr - killing me! CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:371) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221) at org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) ... 18 more Thanks, Russell Jurney @rjurney <http://twitter.com/rjurney> russell.jur...@gmail.com LI <http://linkedin.com/in/russelljurney> FB <http://facebook.com/jurney> datasyndrome.com
How do I use dismax or edismax to rank using 60% tf-idf and 40% a numeric field?
Hello Solarians, I know how to boost a query and I see the methods for tf and idf in streaming scripting. What I don’t know is how to incorporate these things together at a specific percentage of the ranking function. How do I write a query to use dismax or edismax to rank using 60% tf-idf score and 40% the value of a numeric field? Thanks, Russ -- Thanks, Russell Jurney @rjurney <http://twitter.com/rjurney> russell.jur...@gmail.com LI <http://linkedin.com/in/russelljurney> FB <http://facebook.com/jurney> datasyndrome.com
Re: Solr + Parquets
There are ways to load data directly from Spark to Solr but I didn't find any of them satisfactory so I just create enough Spark partitions with reparition() (increase partition count)/coalesce() (decrease partition count) that I get as many Parquet files as I want and then I use a bash script to iterate and load the files via the post command. Thanks, Russell Jurney @rjurney <http://twitter.com/rjurney> russell.jur...@gmail.com LI <http://linkedin.com/in/russelljurney> FB <http://facebook.com/jurney> datasyndrome.com On Fri, Aug 7, 2020 at 9:48 AM Jörn Franke wrote: > DIH is deprecated and it will be removed from Solr. You may though still > be able to install it as a plug-in. However, AFAIK nobody maintains it. Do > not use it anymore > > You can write a custom Spark data source that writes to Solr or does it in > a spark Map step using SolrJ . > In both cases do not create 100s of executors to avoid overloading. > > > > Am 07.08.2020 um 18:39 schrieb Kevin Van Lieshout < > kevin.vanl...@gmail.com>: > > > > Hi, > > > > Is there any assistance around writing parquets from spark to solr shards > > or is it possible to customize a DIH to import a parquet to a solr shard. > > Let me know if this is possible, or the best work around for this. Much > > appreciated, thanks > > > > > > Kevin VL >
Re: Solr + Parquets
Sorry, I'm a goofball. I use Parquet but use bzip2 json format for the last
hop.
Thanks,
Russell Jurney @rjurney <http://twitter.com/rjurney>
russell.jur...@gmail.com LI <http://linkedin.com/in/russelljurney> FB
<http://facebook.com/jurney> datasyndrome.com
On Mon, Aug 10, 2020 at 7:56 PM Aroop Ganguly
wrote:
>
> > script to iterate and load the files via the post command.
> You mean load parquet filed over post? That sounds unbelievable …
> Do u mean you created Solr doc for each parquet record in a partition and
> used solrJ or some other java lib to post the docs to Solr?
>
> df.mapPatitions(p => { ///batch the parquet records, convert batch to a
> solr-doc-batch, then send to Solr via Solr request})
>
>
> If you are sending raw parquet to Solr I would love to learn more :) !
>
> > On Aug 10, 2020, at 7:50 PM, Russell Jurney
> wrote:
> >
> > There are ways to load data directly from Spark to Solr but I didn't find
> > any of them satisfactory so I just create enough Spark partitions with
> > reparition() (increase partition count)/coalesce() (decrease partition
> > count) that I get as many Parquet files as I want and then I use a bash
> > script to iterate and load the files via the post command.
> >
> > Thanks,
> > Russell Jurney @rjurney <http://twitter.com/rjurney>
> > russell.jur...@gmail.com LI <http://linkedin.com/in/russelljurney> FB
> > <http://facebook.com/jurney> datasyndrome.com
> >
> >
> > On Fri, Aug 7, 2020 at 9:48 AM Jörn Franke wrote:
> >
> >> DIH is deprecated and it will be removed from Solr. You may though still
> >> be able to install it as a plug-in. However, AFAIK nobody maintains it.
> Do
> >> not use it anymore
> >>
> >> You can write a custom Spark data source that writes to Solr or does it
> in
> >> a spark Map step using SolrJ .
> >> In both cases do not create 100s of executors to avoid overloading.
> >>
> >>
> >>> Am 07.08.2020 um 18:39 schrieb Kevin Van Lieshout <
> >> kevin.vanl...@gmail.com>:
> >>>
> >>> Hi,
> >>>
> >>> Is there any assistance around writing parquets from spark to solr
> shards
> >>> or is it possible to customize a DIH to import a parquet to a solr
> shard.
> >>> Let me know if this is possible, or the best work around for this. Much
> >>> appreciated, thanks
> >>>
> >>>
> >>> Kevin VL
> >>
>
>
