Data Import Handler - Concurrent Entity Importing

[James Greene]

Hello, I'm new to the group here so please excuse me if I do not have the
etiquette down yet.

Is it possible to have multiple entities (customer configurable, up to 40
atm) in a DIH configuration to be imported at once?  Right now I have
multiple root entities in my configuration but they get indexes
sequentially and this means the entities that are last are always delayed
hitting the index.

I'm trying to migrate an existing setup (solr 6.6) that utilizes a
different collection for each "entity type" into a single collection (solr
8.4) to get around some of the hurdles faced when needing to have searches
that require multiple block joins and currently does not work going cross
core.

I'm also wondering if it is better to fully qualify a field name or use two
different fields for performing the "same" search.  i.e:


{
type_A_status; Active
type_A_value: Test
}
vs
{
type: A
status: Active
value: Test
}

Solr 8.X _nest_path_ and DIH

[James Greene]

I've been trying to get the _nest_path_ queries working with no success.
Does anyone have a link to an example of the configurations needed to get
this to work?

I'm using DIH to index my data and the child entities are getting indexed
(i can query them directly). But using the [child] ChildDocTransformer
returns none of the child entities.

Is there a way to debug what _nest_path_ has actually indexed (see if its
even being set by solr?).

Re: How to determine why solr stops running?

[James Greene]

Check the log for for an OOM crash.  Fatal exceptions will be in the main
solr log and out of memory errors will be in their own -oom log.

I've encountered quite a few solr crashes and usually it's when there's a
threshold of concurrent users and/or indexing happening.



On Thu, May 14, 2020, 9:23 AM Ryan W  wrote:

> Hi all,
>
> I manage a site where solr has stopped running a couple times in the past
> week. The server hasn't been rebooted, so that's not the reason.  What else
> causes solr to stop running?  How can I investigate why this is happening?
>
> Thank you,
> Ryan
>

Re: How to determine why solr stops running?

[James Greene]

I usually do a combination of grepping for ERROR in solr logs and checking
journalctl to see if an external program may have killed the process.


Cheers,

/
*   James Austin Greene
*  www.jamesaustingreene.com
*  336-lol-nerd
/


On Mon, May 18, 2020 at 1:39 PM Erick Erickson 
wrote:

> ps aux | grep solr
>
> on a *.nix system will show you all the runtime parameters.
>
> > On May 18, 2020, at 12:46 PM, Ryan W  wrote:
> >
> > Is there a config file containing the start params?  I run solr like...
> >
> > bin/solr start
> >
> > I have not seen anything in the logs that seems informative. When I grep
> in
> > the logs directory for 'memory', I see nothing besides a couple entries
> > like...
> >
> > 2020-05-14 13:05:56.155 INFO  (main) [   ]
> o.a.s.h.a.MetricsHistoryHandler
> > No .system collection, keeping metrics history in memory.
> >
> > I don't know what that entry means, though the date does roughly coincide
> > with the last time solr stopped running.
> >
> > Thank you.
> >
> >
> > On Mon, May 18, 2020 at 12:00 PM Erick Erickson  >
> > wrote:
> >
> >> Probably, but check that you are running with the oom-killer, it'll be
> in
> >> your start params.
> >>
> >> But absent that, something external will be the culprit, Solr doesn't
> stop
> >> by itself. Do look at the Solr log once things stop, it should show if
> >> someone or something stopped it.
> >>
> >> On Mon, May 18, 2020, 10:43 Ryan W  wrote:
> >>
> >>> I don't see any log file with "oom" in the file name.  Does that mean
> >> there
> >>> hasn't been an out-of-memory issue?  Thanks.
> >>>
> >>> On Thu, May 14, 2020 at 10:05 AM James Greene <
> >> ja...@jamesaustingreene.com
> >>>>
> >>> wrote:
> >>>
> >>>> Check the log for for an OOM crash.  Fatal exceptions will be in the
> >> main
> >>>> solr log and out of memory errors will be in their own -oom log.
> >>>>
> >>>> I've encountered quite a few solr crashes and usually it's when
> >> there's a
> >>>> threshold of concurrent users and/or indexing happening.
> >>>>
> >>>>
> >>>>
> >>>> On Thu, May 14, 2020, 9:23 AM Ryan W  wrote:
> >>>>
> >>>>> Hi all,
> >>>>>
> >>>>> I manage a site where solr has stopped running a couple times in the
> >>> past
> >>>>> week. The server hasn't been rebooted, so that's not the reason.
> >> What
> >>>> else
> >>>>> causes solr to stop running?  How can I investigate why this is
> >>>> happening?
> >>>>>
> >>>>> Thank you,
> >>>>> Ryan
> >>>>>
> >>>>
> >>>
> >>
>
>