mmfr_exact is a string field. key_phrases is a multivalued string field.
On Thu, Apr 28, 2016 at 11:47 AM, Yonik Seeley <ysee...@gmail.com> wrote:
> What about the field types though... are they single valued or multi
> valued, string, text, numeric?
>
> -Yonik
>
>
> On Thu, Apr 28, 2016 at 11:43 AM, Nick Vasilyev
> <nick.vasily...@gmail.com> wrote:
> > Hi Yonik,
> >
> > I forgot to mention that the index is approximately 50 million docs split
> > across 4 shards (replication factor 2) on 2 solr replicas.
> >
> > This particular script will filter items based on a category
> (10-~1,000,000
> > items in each) and run facets on top X terms for particular fields. Query
> > looks like this:
> >
> > {
> > q => "cat:$code",
> > rows => 0,
> > facet => 'true',
> > 'facet.field' => [ 'key_phrases', 'mmfr_exact' ],
> > 'f.key_phrases.facet.limit' => 100,
> > 'f.mmfr_exact.facet.limit' => 20,
> > 'facet.mincount' => 5,
> > distrib => 'false',
> > }
> >
> > I know it can be re-worked some, especially considering there are
> thousands
> > of similar requests going out. However we didn't have this issue before
> and
> > I am worried that it may be a symptom of a larger underlying problem.
> >
> > On Thu, Apr 28, 2016 at 11:34 AM, Yonik Seeley <ysee...@gmail.com>
> wrote:
> >
> >> On Thu, Apr 28, 2016 at 11:29 AM, Nick Vasilyev
> >> <nick.vasily...@gmail.com> wrote:
> >> > Hello,
> >> >
> >> > We recently upgraded to Solr 5.2.1 with jre1.8.0_74 and are seeing
> long
> >> GC
> >> > pauses when running jobs that do some hairy faceting. The same jobs
> >> worked
> >> > fine with our previous 4.6 Solr.
> >>
> >> What does a typical request look like, and what are the field types
> >> that faceting is done on?
> >>
> >> -Yonik
> >>
> >>
> >> > The JVM is configured with 32GB heap with default GC settings, however
> >> I've
> >> > been tweaking the GC settings to no avail. The latest version had the
> >> > following differences from the default config:
> >> >
> >> > XX:ConcGCThreads and XX:ParallelGCThreads are increased from 4 to 7
> >> >
> >> > XX:CMSInitiatingOccupancyFraction increased from 50 to 70
> >> >
> >> >
> >> > Here is a sample output from the gc_log
> >> >
> >> > 2016-04-28T04:36:47.240-0400: 27905.535: Total time for which
> application
> >> > threads were stopped: 0.1667520 seconds, Stopping threads took:
> 0.0171900
> >> > seconds
> >> > {Heap before GC invocations=2051 (full 59):
> >> > par new generation total 6990528K, used 2626705K
> [0x00002b16c0000000,
> >> > 0x00002b18c0000000, 0x00002b18c0000000)
> >> > eden space 5592448K, 44% used [0x00002b16c0000000,
> 0x00002b17571b9948,
> >> > 0x00002b1815560000)
> >> > from space 1398080K, 10% used [0x00002b1815560000,
> 0x00002b181e8cac28,
> >> > 0x00002b186aab0000)
> >> > to space 1398080K, 0% used [0x00002b186aab0000,
> 0x00002b186aab0000,
> >> > 0x00002b18c0000000)
> >> > concurrent mark-sweep generation total 25165824K, used 25122205K
> >> > [0x00002b18c0000000, 0x00002b1ec0000000, 0x00002b1ec0000000)
> >> > Metaspace used 41840K, capacity 42284K, committed 42680K,
> reserved
> >> > 43008K
> >> > 2016-04-28T04:36:49.828-0400: 27908.123: [GC (Allocation Failure)
> >> > 2016-04-28T04:36:49.828-0400: 27908.124:
> >> [CMS2016-04-28T04:36:49.912-0400:
> >> > 27908.207: [CMS-concurr
> >> > ent-abortable-preclean: 5.615/5.862 secs] [Times: user=17.70 sys=2.77,
> >> > real=5.86 secs]
> >> > (concurrent mode failure): 25122205K->15103706K(25165824K), 8.5567560
> >> > secs] 27748910K->15103706K(32156352K), [Metaspace:
> >> 41840K->41840K(43008K)],
> >> > 8.5657830 secs] [
> >> > Times: user=8.56 sys=0.01, real=8.57 secs]
> >> > Heap after GC invocations=2052 (full 60):
> >> > par new generation total 6990528K, used 0K [0x00002b16c0000000,
> >> > 0x00002b18c0000000, 0x00002b18c0000000)
> >> > eden space 5592448K, 0% used [0x00002b16c0000000,
> 0x00002b16c0000000,
> >> > 0x00002b1815560000)
> >> > from space 1398080K, 0% used [0x00002b1815560000,
> 0x00002b1815560000,
> >> > 0x00002b186aab0000)
> >> > to space 1398080K, 0% used [0x00002b186aab0000,
> 0x00002b186aab0000,
> >> > 0x00002b18c0000000)
> >> > concurrent mark-sweep generation total 25165824K, used 15103706K
> >> > [0x00002b18c0000000, 0x00002b1ec0000000, 0x00002b1ec0000000)
> >> > Metaspace used 41840K, capacity 42284K, committed 42680K,
> reserved
> >> > 43008K
> >> > }
> >> > 2016-04-28T04:36:58.395-0400: 27916.690: Total time for which
> application
> >> > threads were stopped: 8.5676090 seconds, Stopping threads took:
> 0.0003930
> >> > seconds
> >> >
> >> > I read the instructions here,
> https://wiki.apache.org/solr/ShawnHeisey,
> >> but
> >> > they seem to be specific to Java 7. Are there any updated
> recommendations
> >> > for Java 8?
> >>
>
