Anyone has a clue?
> List, > > > I somehow fail to index certain pdf files using the > ExtractingRequestHandler in Solr 1.4 with default solrconfig.xml but > modified schema. I have a very simple schema for this case using only > and ID field, a timestamp field and two dynamic fields; ignored_* and > attr_* both indexed, stored and multivalued strings. They are > multivalued simple because some HTML files fail when storing multiple > hyperlinks. > > I have posted multiple files to > http://.../update/extract?literal.id=doc1 including: > 1. the whitepaper at > http://www.lucidimagination.com/whitepaper/whats-new-in-lucene-2-9?sc=AP > 2. the html file of the frontpage of http://nu.nl/ > 3. another pdf at > http://www.google.nl/url?sa=t&source=web&ct=res&cd=1&ved=0CAcQFjAA&url=http%3A%2F%2Fcsl.stanford.edu%2F~christos%2Fpublications%2F2007.cmp_mapreduce.hpca.pdf&rct=j&q=2007.cmp_mapreduce.hpca.pdf&ei=PPz7SpiiOM6l4QbZjKjRAw&usg=AFQjCNHs-olxbUQrGCXpNMHfcZvY8aMk8A > > For each document i have a corresponding select/?q=*:*: > > > 1. No text? Should i see something? > > <doc><str name="id">doc1</str> > <arr name="ignored_content_type"> > <str>application/octet-stream</str> > </arr> > <arr name="ignored_stream_content_type"> > <str> > text/xml; charset=UTF-8; > boundary=----------------------------cf57b4ad644d > </str> > </arr> > <arr name="ignored_stream_size"> > <str>491238</str> > </arr> > <arr name="ignored_text"> > <str> </str> > </arr> > <date name="timestamp">2009-11-12T12:17:23.016Z</date> > </doc> > > > 2. Plenty of data, this seems to be ok > > <doc> > <str name="id">doc1</str> > <arr name="ignored_content_type"> > <str>application/xhtml+xml</str> > </arr> > <arr name="ignored_links"> > <str>http://www.nu.nl/</str> > <str>http://www.nu.nl/</str> > <str>http://www.nu.nl/algemeen/</str> > <str>http://www.nu.nl/economie/</str> > .... > <arr name="ignored_stream_content_type"> > <str> > text/xml; charset=UTF-8; > boundary=----------------------------b6e44d087bdd > </str> > </arr> > <arr name="ignored_stream_size"> > <str>36991</str> > </arr> > <arr name="ignored_text"> > <str> > A LOT OF TEXT HERE > </str> > </arr> > <date name="timestamp">2009-11-12T12:19:15.415Z</date> > </doc> > > > 3. a lot of garbage > > <doc> > <str name="id">doc1</str> > <arr name="ignored_content_encoding"> > <str>windows-1252</str> > </arr> > <arr name="ignored_content_language"> > <str>fr</str> > </arr> > <arr name="ignored_content_type"> > <str>text/plain</str> > </arr> > <arr name="ignored_language"> > <str>fr</str> > </arr> > <arr name="ignored_stream_content_type"> > <str> > text/xml; charset=UTF-8; > boundary=----------------------------83df0fd4d358 > </str> > </arr> > <arr name="ignored_stream_size"> > <str>361458</str> > </arr> > <arr name="ignored_text"> > <str> > A LOT OF GARBAGE HERE including > > ió½·Þp™ó 40› > š©xÓ ^CøùI3람š³î¨V ÚÜ¡yS4 ¹£ ² ›H 6õɨ5¤ ÅÜ磩bädÒøŸ\ �s%OîÐÙIÑYRäŠ ;4 > ¢9"r "—!rEôˆÌ {SìûD²à £©ïœ«{‘ínÆ N÷ô¥F»�™ ±¡Ë'ú\³=·m„Þ »ý)³Å=j¶B¢)` Ñ > „Ï™hjCu{£É5{¢¯ç6½Ñhr¢ºÃ=J M- AqsøtÜì ÿ^Rl S?¿óšM‰—lv‘Ø›Qüãý´ þžŽ > $S;¾¦wze³Ù)qÉú§ ‰› ãqó…Ó ‰ª"U:šBÝ‘GuŠ"ë > MM±Òv �~ ‚N‹t¢ä§~Ì ÞŒS—Êòö¼ÊÄQaº¸¿7tñ ¾Áç œãØŒ58$O 3Å~�8¿L ‡ëŽó©pk_ > Ša Â=u×; (ä<�...@.œ÷ä ù° µk+ÿ PP~ ¨*ݤ¿Œ™¡D» @fI$0°�Î Ù·p“Œ,Øâ †¶v > ¤v1#8¼0 › èð€-†šZ 6¾ ! ñb ˆbˆ¤v)LS)T X² ¬ l...@€ 6E$Q > endstream > endobj > 137 0 > obj<</Type/Encoding/BaseEncoding/WinAnsiEncoding/Differences[1/W/o/r/d/C/u/n/t/M/a/i/x/l/S/g/c/h/K/m/e/s/R/v/I/P/A/H/L/space/p]>> > endobj > 138 0 obj<</Type/FontDescriptor/FontFile2 136 0 R/FontBBox[0 -210 942 > 728]/FontName/WQHWKD+TTE31911E0t00/Flags 4/MissingWidth 750/StemV > 141/CapHeight 728/Ascent 728/Descent -210/ItalicAngle 0>> > endobj > 139 0 obj<</Count 12/Kids[140 0 R 141 0 R]/Type/Pages>> > endobj > 140 0 obj<</Count 6/Kids[147 0 R 1 0 R 4 0 R 7 0 R 22 0 R 25 0 > R]/Type/Pages/Parent 139 0 R>> > endobj > 141 0 obj<</Count 6/Kids[39 0 R 42 0 R 45 0 R 82 0 R 92 0 R 122 0 > R]/Type/Pages/Parent > > .... > > </str> > </arr> > <date name="timestamp">2009-11-12T12:21:28.306Z</date> > </doc> > > > Any ideas? Why doesn't the whitepaper produce any results and why is the > next whitepaper full of garbage? At least i'm happy that HTML works > fine. > > > > Regards, > > - > Markus Jelsma Buyways B.V. > Technisch Architect Friesestraatweg 215c > http://www.buyways.nl 9743 AD Groningen > > > Alg. 050-853 6600 KvK 01074105 > Tel. 050-853 6620 Fax. 050-3118124 > Mob. 06-5025 8350 In: http://www.linkedin.com/in/markus17 >
