Nope, I got this on the dovecot side. But I think the mailbox got
indexed anyway - I did a search and it returned results from that
mailbox after a few seconds. This is just amazing - have no idea why it
took me so long to get it.
Best,
Francis
On 2020-08-27 22:25, Alexandre Rafalovitch wrote:
Is this a Solr-side message? Looks like dovecot doing proactive
trimming of some crazy long header.
You can lookup the record by UID in the Admin UI (UID=153535 instead
of *:*) to check what is being indexed. Check that dovecot does not do
any prefixing of field names (any record from first generic query will
show that)
Most likely (if it is a header) you did not want to search on it
anyway. And if the field type it indexes too is a string (therefore
not tokenized), you probably will not be able to search on it
meaningfully anyway.
Regards,
Alex.
On Thu, 27 Aug 2020 at 16:14, Francis Augusto Medeiros-Logeay
<r...@med-lo.eu> wrote:
It works now! You were right - the files were on a different place. It
seems to be working now.
One last question:
I got this error:
400/49727doveadm(fran...@francisaugusto.com): Warning:
fts-solr(fran...@francisaugusto.com): Mailbox All Mail UID=153535
header
size is huge, truncating
22400/49727
How can I index this?
Best,
Francis
On 2020-08-27 18:58, Francis Augusto Medeiros-Logeay wrote:
> Hi,
>
> I have - for a long time now - hoped to use an fts engine with
> dovecot. My dovecot version is 2.3.7.2 under Ubuntu 20.04.
>
> I installed solr 7.7.3 and then 8.6.0 to see if this was a
> version-related error. I copied the schema from 7.7.0 as many people
> said this was fine.
>
> I get the following error when trying to reindex a user's mailbox:
>
> doveadm(fran...@francisaugusto.com): Error: fts_solr: Indexing failed:
> 400 Bad Request
> doveadm(fran...@francisaugusto.com): Error: Mailbox INBOX: Transaction
> commit failed: FTS transaction commit failed: backend deinit
> doveadm(fran...@francisaugusto.com): Debug: auth-master: conn
> unix:/var/run/dovecot/auth-userdb: Disconnected: Connection closed
> (fd=10)
>
> On Solr I get this error:
>
> org.apache.solr.common.SolrException: Exception writing document id
> 210/9fd7941e8297d25d91600000c3fdd3da/fran...@francisaugusto.com to the
> index; possible analysis error: cannot change field "box" from index
> options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index
> options=DOCS
> at
>
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:246)
>
> Parallel to this, I got some log messages on Solr before attempting to
> reindex the user (sorry for the garbadged text:
>
> Time (Local) Level Core Logger Message
> 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader
Solr
> loaded a deprecated plugin/analysis class [solr.TrieLongField]. Please
> consult documentation how to replace it accordingly.
> 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader
Solr
> loaded a deprecated plugin/analysis class [solr.SynonymFilterFactory].
> Please consult documentation how to replace it accordingly.
> 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader
Solr
> loaded a deprecated plugin/analysis class
> [solr.WordDelimiterFilterFactory]. Please consult documentation how to
> replace it accordingly.
> 7/22/2020, 6:43:46 PM ERROR
> false x:dovecot RequestHandlerBase java.lang.IllegalStateException:
> Type mismatch: uid was indexed as SORTED_NUMERIC
> 7/22/2020, 6:43:46 PM ERROR
> false x:dovecot HttpSolrCall null:java.lang.IllegalStateException:
> Type mismatch: uid was indexed as SORTED_NUMERIC
> 7/22/2020, 6:43:49 PM ERROR
> false x:dovecot RequestHandlerBase java.lang.IllegalStateException:
> Type mismatch: uid was indexed as SORTED_NUMERIC
> 7/22/2020, 6:43:49 PM ERROR
> false x:dovecot HttpSolrCall null:java.lang.IllegalStateException:
> Type mismatch: uid was indexed as SORTED_NUMERIC
> 7/22/2020, 6:43:56 PM ERROR
> false x:dovecot RequestHandlerBase java.lang.IllegalStateException:
> Type mismatch: uid was indexed as SORTED_NUMERIC
> 7/22/2020, 6:43:56 PM ERROR
> false x:dovecot HttpSolrCall null:java.lang.IllegalSta
>
> I am tried again. I get no errors when doing a `doveadm fts rescan`,
> but get errors when trying this:
>
> doveadm index -u myu...@mydomain.com INBOX
> doveadm(myu...@mydomain.com): Error: fts_solr: Indexing failed: 400 Bad
> Request
> doveadm(myu...@mydomain.com): Error: Mailbox INBOX: Transaction commit
> failed: FTS transaction commit failed: backend deinit
>
> I guess this is a matter of waiting the reindex to be over?
>
> I get so many "Type mismatch" errors in Solr, except for this one that
> looks different and showed up after trying the doveadm index command
> above:
>
>
> ERROR true
> x:dovecot
> RequestHandlerBase
> org.apache.solr.common.SolrException: Exception writing document id
> 210/9fd7941e8297d25d91600000c3fdd3da/myu...@mydomain.com to the index;
> possible analysis error: cannot change field "box" from index
> options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index
> options=DOCS
> org.apache.solr.common.SolrException: Exception writing document id
> 210/9fd7941e8297d25d91600000c3fdd3da/myu...@mydomain.com to the index;
> possible analysis error: cannot change field "box" from index
> options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index
> options=DOCS
> at
>
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:246)
> at
>
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
> at
>
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
> at
>
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:259)
> at
>
org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:489)
> at
>
org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpdateProcessor.java:339)
> at
> org.apache.solr.update.VersionBucket.runWithLock(VersionBucket.java:50)
> at
>
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:339)
> at
>
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:225)
> at
>
org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
> at
> org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:261)
> at
> org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:188)
> at
>
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
> at
>
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
> at
>
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:198)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2576)
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:803)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:582)
> at
>
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
> at
>
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
> at
>
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
>
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
>
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at
>
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
> at
>
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at
>
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
> at
>
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
> at
>
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
> at
>
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at
>
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at
>
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
> at
>
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
> at
>
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
>
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at
>
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:505)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
> at
>
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
> at
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at
>
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at
>
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at
>
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
>
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
>
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
>
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
> at
>
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.lang.IllegalArgumentException: cannot change field
> "box" from index options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent
> index options=DOCS
> at
>
org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:375)
> at
>
org.apache.lucene.index.FieldInfos$FieldNumbers.setIndexOptions(FieldInfos.java:441)
> at
>
org.apache.lucene.index.DefaultIndexingChain.initIndexOptions(DefaultIndexingChain.java:701)
> at
>
org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:664)
> at
>
org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:440)
> at
>
org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
> at
>
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
> at
>
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
> at
>
org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:971)
> at
>
org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:344)
> at
>
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:291)
> at
>
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:238)
> ... 53 more
> Last Check: 8/2/2020, 1:51:12 PM
> Show dates in UTC
> Auto-Refresh
>
> Any clue on what I could do here? I have hoped to use Solr but fail to
> see instructions that just work. I've been told things must be
> reindexed. and I tried that - although I don't know if I did it right.
> Could anyone offer some hints/tips on what to do to make it work?
>
> Best,
>
> Francis
>
> PS - I can show the full log of one of the type mismatch error as
> well...
--
Francis Augusto Medeiros-Logeay
Oslo, Norway
--
Francis Augusto Medeiros-Logeay
Oslo, Norway
