Hi Erik,
Yes, i deleted the index and re-indexed after increasing the value (i have
restarted tomcat as well)
but still no luck. but i was just wondering the field that i am trying to
index has the complete document text in it as i am storing that. but not
getting the complete terms/tokens into the index to perform the search.
What would be the suggestible Analyzers, filters that I should check with?
Currently I am using the following:
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.StandardFilterFactory" />
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.StandardFilterFactory" />
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
Please suggest
Best Regards,
Kranti K K Parisa
On Tue, Jan 19, 2010 at 9:03 PM, Erick Erickson <erickerick...@gmail.com>wrote:
> Did you reindex the documents you examined? That limit
> is applied when you index.
>
> Try searching the user list for maxfieldlength, this topic
> has been discussed many times and you should find a
> solution.
>
> HTH
> Erick
>
> 2010/1/19 Kranti™ K K Parisa <kranti.par...@gmail.com>
>
> > Can anyone suggest/guide me on this.
> >
> > Best Regards,
> > Kranti K K Parisa
> >
> >
> >
> > 2010/1/19 Kranti™ K K Parisa <kranti.par...@gmail.com>
> >
> > > Hi Mark,
> > >
> > > I changed the value to 1,000,000,000 to just test my luck.
> > >
> > > But unfortunately I am still not getting the index for all Token.
> > >
> > > Please suggest.
> > >
> > > Best Regards,
> > > Kranti K K Parisa
> > >
> > >
> > >
> > > 2010/1/19 Kranti™ K K Parisa <kranti.par...@gmail.com>
> > >
> > > Hi Mark,
> > >>
> > >> As you see my config file contains the value as 10,000
> > >> <maxFieldLength>10000</maxFieldLength>
> > >>
> > >> But when I check thru Lukeall jar file I can see the Term count around
> > >> 3,000.
> > >>
> > >> Please suggest.
> > >>
> > >> Best Regards,
> > >> Kranti K K Parisa
> > >>
> > >>
> > >>
> > >> 2010/1/19 Mark Miller <markrmil...@gmail.com>
> > >>
> > >> It limits the number of tokens that will be indexed.
> > >>>
> > >>> Kranti™ K K Parisa wrote:
> > >>> > Hi Mark,
> > >>> >
> > >>> > I really appreciate the quick reply.
> > >>> >
> > >>> > here is what I have in the config xml
> > >>> >
> > >>> > <ramBufferSizeMB>32</ramBufferSizeMB>
> > >>> > <maxMergeDocs>2147483647</maxMergeDocs>
> > >>> > * <maxFieldLength>10000</maxFieldLength>*
> > >>> > <writeLockTimeout>1000</writeLockTimeout>
> > >>> > <commitLockTimeout>10000</commitLockTimeout>
> > >>> >
> > >>> > Does this matter with Tokens?? Because the field I am using is
> having
> > >>> > the full content of the file ( I checked that using Lukeall jar
> > file),
> > >>> > how ever Tokens are not getting generated completely because of
> which
> > >>> > my search not working for the full content.
> > >>> >
> > >>> > Please suggest.
> > >>> >
> > >>> > Best Regards,
> > >>> > Kranti K K Parisa
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Tue, Jan 19, 2010 at 8:27 PM, Mark Miller <
> markrmil...@gmail.com
> > >>> > <mailto:markrmil...@gmail.com>> wrote:
> > >>> >
> > >>> > Kranti™ K K Parisa wrote:
> > >>> > > Hi All,
> > >>> > >
> > >>> > > I have a problem using SOLR indexing. I am trying to index 96
> > >>> > pages PDF file
> > >>> > > (using PDFBox for extracting the file contents into String).
> > But
> > >>> > > surprisingly SOLR Indexing is not done for the full document.
> > >>> > Means I can't
> > >>> > > get all the token how ever the field contains the full text
> of
> > >>> > the PDF as i
> > >>> > > am storing the field along with indexing.
> > >>> > >
> > >>> > > Is there any such limitations with SOLR indexing, please let
> me
> > >>> > know at the
> > >>> > > earliest.
> > >>> > >
> > >>> > > Thanks in advance!
> > >>> > >
> > >>> > > Best Regards,
> > >>> > > Kranti K K Parisa
> > >>> > >
> > >>> > >
> > >>> > Take a look at maxFieldLength in solrconfig.xml
> > >>> >
> > >>> > --
> > >>> > - Mark
> > >>> >
> > >>> > http://www.lucidimagination.com
> > >>> >
> > >>> >
> > >>> >
> > >>> >
> > >>>
> > >>>
> > >>> --
> > >>> - Mark
> > >>>
> > >>> http://www.lucidimagination.com
> > >>>
> > >>>
> > >>>
> > >>>
> > >>
> > >
> >
>
