Hi all, at last I've tried to write my own payload function, and it has worked (pretty well) immediately! Well, to be honest I'm a little puzzled, so I have applied immediately the first rule: always beware when your code works at first run.
Now I'm testing it. On Mon, Oct 21, 2019 at 4:37 PM Erick Erickson <erickerick...@gmail.com> wrote: > Don’t go to thousands of fields, it’s usually a bad idea for that many > fields. > > If all you’re doing is returning the data for display, have you considered > indexing, perhaps in a separate field, a string token for each? Something > like > store1_USD_125.00 > ? > > Then have the UI split that apart in a pleasing manner. > > Doesn’t work if you want to, say, sort by price though. > > Erick > > > On Oct 21, 2019, at 10:19 AM, Alexandre Rafalovitch <arafa...@gmail.com> > wrote: > > > > I remember several years ago a discussion/blog post about a similar > > problem. The author went through a lot of thinking and decided that > > the best way to deal with a similar problem was to have Solr documents > > represent different level of abstraction, more granular. > > > > IIRC, the equivalent for your example would be to represent 'pricing' > > (or even pricing/availability) as the document, not a 'product'. You > > may need to duplicate product field values for that to work. But that > > - apparently - allowed them to represent a lot of concepts (like > > upcoming discounts, etc) easier by just adding availability dates, etc > > into that final lower-level record. And, of course, allowed to update > > individual store/product price by changing one record per time without > > having to invalidate the cache for all other stores. > > > > Regards, > > Alex. > > > > > > On Mon, 21 Oct 2019 at 09:59, Vincenzo D'Amore <v.dam...@gmail.com> > wrote: > >> > >> Hi Erick, > >> > >> thanks for getting back to me. We started to use payloads because we > have > >> the classical per-store pricing problem. > >> Thousands of stores across and different prices. > >> Then we found the payloads very useful started to use it for many > reasons, > >> like enabling/disabling the product for such store, save the stock > >> availability, or save the other info like buy/sell price, discount > rates, > >> and so on. > >> All those information are numbers, but stores can also be in different > >> countries, I mean would be useful also have the currency and other > >> attributes related to the store. > >> > >> Thinking about an alternative for payloads maybe I could use the dynamic > >> fields, well, I know it is ugly. > >> > >> Consider this hypothetical case where I have two field payload : > >> > >> payloadPrice: [ > >> "store1|125.0", > >> "store2|220.0", > >> "store3|225.0" > >> ] > >> > >> payloadCurrency: [ > >> "store1|USD", > >> "store2|EUR", > >> "store3|GBP" > >> ] > >> > >> with dynamic fields I could have different fields for each document. > >> > >> currency_store1_s: "USD" > >> currency_store2_s: "EUR" > >> currency_store3_s: "GBP" > >> > >> But how many dynamic fields like this can I have? more than thousands? > >> > >> Again, I've just started to look at solr-ocrhighlighting github project > you > >> suggested. > >> Those seems have written their own payload object type where store ocr > >> highlighting information. > >> It seems interesting, I'll take a look immediately. > >> > >> Thanks again for your time. > >> > >> Best regards, > >> Vincenzo > >> > >> > >> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <erickerick...@gmail.com > > > >> wrote: > >> > >>> This is one of those situations where I know a client did it, but > didn’t > >>> see the code myself. > >>> > >>> So I can’t help much. > >>> > >>> Perhaps a good question at this point, though, is “why do you want to > add > >>> string payloads anyway”? > >>> > >>> This isn’t the client, but it might give you some pointers: > >>> > >>> > >>> > https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java > >>> > >>> Best, > >>> Erick > >>> > >>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <v.dam...@gmail.com> > >>> wrote: > >>>> > >>>> Hi Erick, > >>>> > >>>> It seems I've reached a dead-point, or at least it seems looking at > the > >>>> code, it seems I can't easily add a custom decoder: > >>>> > >>>> Looking at PayloadUtils class there is getPayloadDecoder method > invoked > >>> to > >>>> return the PayloadDecoder : > >>>> > >>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) { > >>>> PayloadDecoder decoder = null; > >>>> > >>>> String encoder = getPayloadEncoder(fieldType); > >>>> > >>>> if ("integer".equals(encoder)) { > >>>> decoder = (BytesRef payload) -> payload == null ? 1 : > >>>> PayloadHelper.decodeInt(payload.bytes, payload.offset); > >>>> } > >>>> if ("float".equals(encoder)) { > >>>> decoder = (BytesRef payload) -> payload == null ? 1 : > >>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset); > >>>> } > >>>> // encoder could be "identity" at this point, in the case of > >>>> DelimitedTokenFilterFactory encoder="identity" > >>>> > >>>> // TODO: support pluggable payload decoders? > >>>> > >>>> return decoder; > >>>> } > >>>> > >>>> Any advice to work around this situation? > >>>> > >>>> > >>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson < > erickerick...@gmail.com> > >>>> wrote: > >>>> > >>>>> You’d need to write one. Payloads are generally intended to hold > >>> numerics > >>>>> you can then use in a function query to factor into the score… > >>>>> > >>>>> Best, > >>>>> Erick > >>>>> > >>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <v.dam...@gmail.com> > >>>>> wrote: > >>>>>> > >>>>>> Sorry, I just realized that I was wrong in how I'm using the payload > >>>>>> function. > >>>>>> Give that the payload function only handles a numeric (integer or > >>> float) > >>>>>> payload, could you suggest me an alternative function that handles > >>>>> strings? > >>>>>> If not, should I write one? > >>>>>> > >>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore < > v.dam...@gmail.com> > >>>>>> wrote: > >>>>>> > >>>>>>> Hi all, > >>>>>>> > >>>>>>> I'm trying to understand what I did wrong with a payload query that > >>>>>>> returns > >>>>>>> > >>>>>>> error: { > >>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException", > >>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ], > >>>>>>> msg: "No payload decoder found for field: colorCode", > >>>>>>> code: 400 > >>>>>>> } > >>>>>>> > >>>>>>> I have reduced my problem in a little sample to show what happens > to > >>> me. > >>>>>>> Basically I have a document with a couple of payload fields one > >>>>>>> delimited_payloads_string and one delimited_payloads_integer > >>>>>>> > >>>>>>> { > >>>>>>> field_dps: "key|data", > >>>>>>> field_dpi: "key|1", > >>>>>>> } > >>>>>>> > >>>>>>> When I execute this query solr returns as expected the payload for > the > >>>>> key > >>>>>>> > >>>>>>> q=*:*&fl=payload(field_dpi,key) > >>>>>>> > >>>>>>> { > >>>>>>> payload(field_dpi,key): 1 > >>>>>>> } > >>>>>>> > >>>>>>> But for the strings there have to be something of different to do, > >>>>> because > >>>>>>> I'm unable receive the payload value back. Executing this query, > as in > >>>>> the > >>>>>>> short introduction of this post, I receive an error. > >>>>>>> > >>>>>>> ?q=*:*&fl=payload(field_dps,key) > >>>>>>> > >>>>>>> error: { > >>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException", > >>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ], > >>>>>>> msg: "No payload decoder found for field: colorCode", > >>>>>>> code: 400 > >>>>>>> } > >>>>>>> > >>>>>>> Am I doing something wrong? How can I read strings payload data? > >>>>>>> > >>>>>>> Thanks in advance for your time, > >>>>>>> Vincenzo > >>>>>>> > >>>>>>> -- > >>>>>>> Vincenzo D'Amore > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> -- > >>>>>> Vincenzo D'Amore > >>>>> > >>>>> > >>>> > >>>> -- > >>>> Vincenzo D'Amore > >>> > >>> > >> > >> -- > >> Vincenzo D'Amore > > -- Vincenzo D'Amore
