bq: but is there any possibility that the values stick around until there is a segment merge for some strange reason
There better not be or it's a bug. Things will stick around until you issue a commit, is there any chance that's the problem? If you can document the exact steps, maybe we can reproduce the issue and raise a JIRA. Best, Erick On Thu, Apr 27, 2017 at 6:03 AM, Chris Ulicny <culicny@iq.media> wrote: > Yeah, something's not quite right somewhere. We never even considered > in-place updates an option since it requires the fields to be non-indexed > and non-stored. Our schemas never have any field that satisfies those two > conditions let alone the other necessary ones. > > I went ahead and tested the atomic updates on different textfields, and I > still can't get the indexed but not-stored othertext_field to disappear. So > far set, add, and remove updates do not change it regardless of what the > fields are in the atomic update. > > It would be extraordinarily useful if this update behavior is now expected > (but not currently documented) functionality. > > I'm not too familiar with the nitty-gritty details of merging segment > files, but is there any possibility that the values stick around until > there is a segment merge for some strange reason? > > On Thu, Apr 27, 2017 at 12:59 AM Dorian Hoxha <dorian.ho...@gmail.com> > wrote: > >> @Chris, >> According to doc-link-above, only INC,SET are in-place-updates. And only >> when they're not indexed/stored, while your 'integer-field' is. So still >> shenanigans in there somewhere (docs,your-code,your-test,solr-code). >> >> On Thu, Apr 27, 2017 at 2:04 AM, Chris Ulicny <culicny@iq.media> wrote: >> >> > That's probably it then. None of the atomic updates that I've tried have >> > been on TextFields. I'll give the TextField atomic update to verify that >> it >> > will clear the other field. >> > >> > Has this functionality been consistent since atomic updates were >> > introduced, or is this a side effect of some other change? It'd be very >> > convenient for us to use this functionality as it currently works, but if >> > it's something that prevents us from upgrading versions in the future, we >> > should probably avoid expecting it to work. >> > >> > On Wed, Apr 26, 2017 at 7:36 PM Ishan Chattopadhyaya < >> > ichattopadhy...@gmail.com> wrote: >> > >> > > > Hmm, interesting. I can imagine that as long as you're updating >> > > > docValues fields, the other_text field would be there. But the >> instant >> > > > you updated a non-docValues field (text_field in your example) the >> > > > other_text field would disappear >> > > >> > > I can confirm this. When in-place updates to DV fields are done, the >> rest >> > > of the fields remain as they were. >> > > >> > > On Thu, Apr 27, 2017 at 4:33 AM, Erick Erickson < >> erickerick...@gmail.com >> > > >> > > wrote: >> > > >> > > > Hmm, interesting. I can imagine that as long as you're updating >> > > > docValues fields, the other_text field would be there. But the >> instant >> > > > you updated a non-docValues field (text_field in your example) the >> > > > other_text field would disappear. >> > > > >> > > > I DO NOT KNOW this for a fact, but I'm asking people who do. >> > > > >> > > > On Wed, Apr 26, 2017 at 2:13 PM, Dorian Hoxha < >> dorian.ho...@gmail.com> >> > > > wrote: >> > > > > There are In Place Updates, but according to docs they stll >> shouldn't >> > > > work >> > > > > in your case: >> > > > > https://cwiki.apache.org/confluence/display/solr/ >> > > > Updating+Parts+of+Documents >> > > > > >> > > > > On Wed, Apr 26, 2017 at 10:36 PM, Chris Ulicny <culicny@iq.media> >> > > wrote: >> > > > > >> > > > >> That's the thing I'm curious about though. As I mentioned in the >> > first >> > > > >> post, I've already tried a few tests, and the value seems to still >> > be >> > > > >> present after an atomic update. >> > > > >> >> > > > >> I haven't exhausted all possible atomic updates, but 'set' and >> 'add' >> > > > seem >> > > > >> to preserve the non-stored text field. >> > > > >> >> > > > >> Thanks, >> > > > >> Chris >> > > > >> >> > > > >> On Wed, Apr 26, 2017 at 4:07 PM Dorian Hoxha < >> > dorian.ho...@gmail.com> >> > > > >> wrote: >> > > > >> >> > > > >> > You'll lose the data in that field. Try doing a commit and it >> > should >> > > > >> > happen. >> > > > >> > >> > > > >> > On Wed, Apr 26, 2017 at 9:50 PM, Chris Ulicny <culicny@iq.media >> > >> > > > wrote: >> > > > >> > >> > > > >> > > Thanks Shawn, I didn't realize docValues were enabled by >> default >> > > > now. >> > > > >> > > That's very convenient and probably makes a lot of the schemas >> > > we've >> > > > >> been >> > > > >> > > making excessively verbose. >> > > > >> > > >> > > > >> > > This is on 6.3.0. Do you know what the first version was that >> > they >> > > > >> added >> > > > >> > > the docValues by default for non-Text field? >> > > > >> > > >> > > > >> > > However, that shouldn't apply to this since I'm concerned >> with a >> > > > >> > non-stored >> > > > >> > > TextField without docValues enabled. >> > > > >> > > >> > > > >> > > Best, >> > > > >> > > Chris >> > > > >> > > >> > > > >> > > On Wed, Apr 26, 2017 at 3:36 PM Shawn Heisey < >> > apa...@elyograg.org >> > > > >> > > > >> > wrote: >> > > > >> > > >> > > > >> > > > On 4/25/2017 1:40 PM, Chris Ulicny wrote: >> > > > >> > > > > Hello all, >> > > > >> > > > > >> > > > >> > > > > Suppose I have the following fields in a document and >> > populate >> > > > all >> > > > >> 4 >> > > > >> > > > fields >> > > > >> > > > > for every document. >> > > > >> > > > > >> > > > >> > > > > id: uniqueKey, indexed and stored >> > > > >> > > > > integer_field: indexed and stored >> > > > >> > > > > text_field: indexed and stored >> > > > >> > > > > othertext_field: indexed but not stored >> > > > >> > > > > >> > > > >> > > > > No default values, multivalues, docvalues, copyfields, or >> > any >> > > > other >> > > > >> > > > > properties set. >> > > > >> > > > >> > > > >> > > > You didn't indicate the Solr version. In recent Solr >> > versions, >> > > > most >> > > > >> > > > field classes other than TextField have docValues enabled by >> > > > default, >> > > > >> > > > even if the config is not mentioned on the field, and in >> those >> > > > >> > versions, >> > > > >> > > > docValues will take the place of stored if stored is false. >> > > > >> > > > >> > > > >> > > > Thanks, >> > > > >> > > > Shawn >> > > > >> > > > >> > > > >> > > > >> > > > >> > > >> > > > >> > >> > > > >> >> > > > >> > > >> > >>
