Please see below the dir listing and relevant part of schema file (I have
removed the name part from fields for obvious reasons).
Also regarding .frq file why exactly is it needed? Is it required in phrase
searching (I am not using highlighting or MoreLikeThis on this index file)
too? and this is not made if all fields are using omitTF?
Thanks alot!
--------------Dir Listing------------------
01/16/2011 06:05 AM <DIR> .
01/16/2011 06:05 AM <DIR> ..
01/15/2011 03:58 PM <DIR> log
04/22/2010 12:42 AM 549 luke.jnlp
01/16/2011 04:58 AM 20 segments.gen
01/16/2011 04:58 AM 287 segments_5hl
01/16/2011 02:17 AM 4,760,716,827 _36w.fdt
01/16/2011 02:17 AM 107,732,836 _36w.fdx
01/16/2011 02:15 AM 4,032 _36w.fnm
01/16/2011 04:36 AM 25,221,109,245 _36w.frq
01/16/2011 04:38 AM 4,457,445,928 _36w.nrm
01/16/2011 04:36 AM 126,866,227,056 _36w.prx
01/16/2011 04:36 AM 22,510,915 _36w.tii
01/16/2011 04:36 AM 1,635,096,862 _36w.tis
01/16/2011 04:58 AM 18,341,750 _36w.tvd
01/16/2011 04:58 AM 78,450,397,739 _36w.tvf
01/16/2011 04:58 AM 215,465,668 _36w.tvx
14 File(s) 241,755,049,714 bytes
3 Dir(s) 1,072,112,025,600 bytes free
-----------------Schema File----------------------
F:\IndexingAppsRealTime\index>
<?xml version="1.0" encoding="UTF-8" ?>
<schema name="example" version="1.2">
<types>
<fieldType name="string" class="solr.StrField" sortMissingLast="true"
omitNorms="true"/>
<fieldtype name="text" class="solr.TextField">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"
luceneMatchVersion="LUCENE_29"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"/>-->
</analyzer>
</fieldtype>
</types>
<fields>
<field name="text" type="text" indexed="true" stored="false" />
</fields>
<fields>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true" />
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="false" />
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true" />
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="string" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<field name="" type="text" indexed="true" stored="true"/>
<dynamicField name="" type="text" indexed="true" stored="false"/>
</schema>
On Sun, Jan 16, 2011 at 6:52 PM, Otis Gospodnetic <
otis_gospodne...@yahoo.com> wrote:
> Hm, want to email the index dir listing (ls -lah) + the field type and
> field
> definitions from your schema.xml?
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
>
>
>
> ----- Original Message ----
> > From: Salman Akram <salman.ak...@northbaysolutions.net>
> > To: solr-user@lucene.apache.org
> > Sent: Sun, January 16, 2011 7:51:15 AM
> > Subject: Re: TVF file
> >
> > Nops. I optimized it with Standard File Format and cleaned up Index dir
> > through Luke. It adds upto to the total size when I optimized it with
> > Compound File Format.
> >
> > On Sun, Jan 16, 2011 at 5:46 PM, Otis Gospodnetic <
> > otis_gospodne...@yahoo.com> wrote:
> >
> > > Is it possible that the tvf file you are looking at is old (i.e. not
> part
> > > of
> > > your active index)?
> > >
> > > Otis
> > > ----
> > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> > > Lucene ecosystem search :: http://search-lucene.com/
> > >
> > >
> > >
> > > ----- Original Message ----
> > > > From: Salman Akram <salman.ak...@northbaysolutions.net>
> > > > To: solr-user@lucene.apache.org
> > > > Sent: Sun, January 16, 2011 6:17:23 AM
> > > > Subject: Re: TVF file
> > > >
> > > > Some more info I copied it from Luke and below is what it says
> for...
> > > >
> > > > Text Fields --> stored/uncompressed,indexed,tokenized
> > > > String Fields -->
> stored/uncompressed,indexed,omitTermFreqAndPositions
> > > >
> > > > The main contents field is not stored so it doesn't show up on Luke
> but
> > > that
> > > > is Analyzed and Tokenized for searching.
> > > >
> > > > On Sun, Jan 16, 2011 at 3:50 PM, Salman Akram <
> > > > salman.ak...@northbaysolutions.net> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > From my understanding TVF file stores the Term Vectors
> > > (Positions/Offset)
> > > > > so if no field has Field.TermVector set (default is NO) so it
> > > shouldn't be
> > > > > created, right?
> > > > >
> > > > > I have an index created through SOLR on which no field had any
> value
> > > for
> > > > > TermVectors so by default it shouldn't be saved. All the fields
> are
> > > either
> > > > > String or Text. All fields have just indexed and stored
> attributes set
> > > to
> > > > > True. String fields have omitNorms = true as well.
> > > > >
> > > > > Even in Luke it doesn't show V (Term Vector) flag but I have a
> big
> TVF
> > > file
> > > > > in my index. Its almost 30% of the total index (around 60% is the
> PRX
> > > > > positions file).
> > > > >
> > > > > Also in Luke it shows 'f' (omitTF) flag for strings but not for
> text
> > > > > fields.
> > > > >
> > > > > Any ideas what's going on? Thanks!
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Salman Akram
> > > > > Senior Software Engineer - Tech Lead
> > > > > 80-A, Abu Bakar Block, Garden Town, Pakistan
> > > > > Cell: +92-321-4391210
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Salman Akram
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Salman Akram
> >
>
--
Regards,
Salman Akram
