Is this a Solr-side message? Looks like dovecot doing proactive trimming of some crazy long header.
You can lookup the record by UID in the Admin UI (UID=153535 instead of *:*) to check what is being indexed. Check that dovecot does not do any prefixing of field names (any record from first generic query will show that) Most likely (if it is a header) you did not want to search on it anyway. And if the field type it indexes too is a string (therefore not tokenized), you probably will not be able to search on it meaningfully anyway. Regards, Alex. On Thu, 27 Aug 2020 at 16:14, Francis Augusto Medeiros-Logeay <r...@med-lo.eu> wrote: > > It works now! You were right - the files were on a different place. It > seems to be working now. > > One last question: > > I got this error: > > 400/49727doveadm(fran...@francisaugusto.com): Warning: > fts-solr(fran...@francisaugusto.com): Mailbox All Mail UID=153535 header > size is huge, truncating > 22400/49727 > > How can I index this? > > Best, > > Francis > > > On 2020-08-27 18:58, Francis Augusto Medeiros-Logeay wrote: > > Hi, > > > > I have - for a long time now - hoped to use an fts engine with > > dovecot. My dovecot version is 2.3.7.2 under Ubuntu 20.04. > > > > I installed solr 7.7.3 and then 8.6.0 to see if this was a > > version-related error. I copied the schema from 7.7.0 as many people > > said this was fine. > > > > I get the following error when trying to reindex a user's mailbox: > > > > doveadm(fran...@francisaugusto.com): Error: fts_solr: Indexing failed: > > 400 Bad Request > > doveadm(fran...@francisaugusto.com): Error: Mailbox INBOX: Transaction > > commit failed: FTS transaction commit failed: backend deinit > > doveadm(fran...@francisaugusto.com): Debug: auth-master: conn > > unix:/var/run/dovecot/auth-userdb: Disconnected: Connection closed > > (fd=10) > > > > On Solr I get this error: > > > > org.apache.solr.common.SolrException: Exception writing document id > > 210/9fd7941e8297d25d91600000c3fdd3da/fran...@francisaugusto.com to the > > index; possible analysis error: cannot change field "box" from index > > options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index > > options=DOCS > > at > > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:246) > > > > Parallel to this, I got some log messages on Solr before attempting to > > reindex the user (sorry for the garbadged text: > > > > Time (Local) Level Core Logger Message > > 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader > > Solr > > loaded a deprecated plugin/analysis class [solr.TrieLongField]. Please > > consult documentation how to replace it accordingly. > > 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader > > Solr > > loaded a deprecated plugin/analysis class [solr.SynonymFilterFactory]. > > Please consult documentation how to replace it accordingly. > > 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader > > Solr > > loaded a deprecated plugin/analysis class > > [solr.WordDelimiterFilterFactory]. Please consult documentation how to > > replace it accordingly. > > 7/22/2020, 6:43:46 PM ERROR > > false x:dovecot RequestHandlerBase > > java.lang.IllegalStateException: > > Type mismatch: uid was indexed as SORTED_NUMERIC > > 7/22/2020, 6:43:46 PM ERROR > > false x:dovecot HttpSolrCall null:java.lang.IllegalStateException: > > Type mismatch: uid was indexed as SORTED_NUMERIC > > 7/22/2020, 6:43:49 PM ERROR > > false x:dovecot RequestHandlerBase > > java.lang.IllegalStateException: > > Type mismatch: uid was indexed as SORTED_NUMERIC > > 7/22/2020, 6:43:49 PM ERROR > > false x:dovecot HttpSolrCall null:java.lang.IllegalStateException: > > Type mismatch: uid was indexed as SORTED_NUMERIC > > 7/22/2020, 6:43:56 PM ERROR > > false x:dovecot RequestHandlerBase > > java.lang.IllegalStateException: > > Type mismatch: uid was indexed as SORTED_NUMERIC > > 7/22/2020, 6:43:56 PM ERROR > > false x:dovecot HttpSolrCall null:java.lang.IllegalSta > > > > I am tried again. I get no errors when doing a `doveadm fts rescan`, > > but get errors when trying this: > > > > doveadm index -u myu...@mydomain.com INBOX > > doveadm(myu...@mydomain.com): Error: fts_solr: Indexing failed: 400 Bad > > Request > > doveadm(myu...@mydomain.com): Error: Mailbox INBOX: Transaction commit > > failed: FTS transaction commit failed: backend deinit > > > > I guess this is a matter of waiting the reindex to be over? > > > > I get so many "Type mismatch" errors in Solr, except for this one that > > looks different and showed up after trying the doveadm index command > > above: > > > > > > ERROR true > > x:dovecot > > RequestHandlerBase > > org.apache.solr.common.SolrException: Exception writing document id > > 210/9fd7941e8297d25d91600000c3fdd3da/myu...@mydomain.com to the index; > > possible analysis error: cannot change field "box" from index > > options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index > > options=DOCS > > org.apache.solr.common.SolrException: Exception writing document id > > 210/9fd7941e8297d25d91600000c3fdd3da/myu...@mydomain.com to the index; > > possible analysis error: cannot change field "box" from index > > options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index > > options=DOCS > > at > > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:246) > > at > > org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76) > > at > > org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) > > at > > org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:259) > > at > > org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:489) > > at > > org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpdateProcessor.java:339) > > at > > org.apache.solr.update.VersionBucket.runWithLock(VersionBucket.java:50) > > at > > org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:339) > > at > > org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:225) > > at > > org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103) > > at > > org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:261) > > at > > org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:188) > > at > > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) > > at > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68) > > at > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:198) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2576) > > at > > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:803) > > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:582) > > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424) > > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351) > > at > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) > > at > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > > at > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > > at > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > > at > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > > at > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > > at > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > > at > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249) > > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > > at > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) > > at > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > at > > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > at org.eclipse.jetty.server.Server.handle(Server.java:505) > > at > > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > > at > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > > at > > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > > at > > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > at > > org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > > at > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917) > > at java.base/java.lang.Thread.run(Thread.java:834) > > Caused by: java.lang.IllegalArgumentException: cannot change field > > "box" from index options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent > > index options=DOCS > > at > > org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:375) > > at > > org.apache.lucene.index.FieldInfos$FieldNumbers.setIndexOptions(FieldInfos.java:441) > > at > > org.apache.lucene.index.DefaultIndexingChain.initIndexOptions(DefaultIndexingChain.java:701) > > at > > org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:664) > > at > > org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:440) > > at > > org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406) > > at > > org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250) > > at > > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495) > > at > > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594) > > at > > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586) > > at > > org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:971) > > at > > org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:344) > > at > > org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:291) > > at > > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:238) > > ... 53 more > > Last Check: 8/2/2020, 1:51:12 PM > > Show dates in UTC > > Auto-Refresh > > > > Any clue on what I could do here? I have hoped to use Solr but fail to > > see instructions that just work. I've been told things must be > > reindexed. and I tried that - although I don't know if I did it right. > > Could anyone offer some hints/tips on what to do to make it work? > > > > Best, > > > > Francis > > > > PS - I can show the full log of one of the type mismatch error as > > well... > > -- > Francis Augusto Medeiros-Logeay > Oslo, Norway
