Re: Error casting to PointField
Thanks Shawn and Erick. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
using uuid for documents
I have two questions about using uuid:
In Solr refs, it is stated that the unique id could be of uuid type. I
found we can generate uuid when the doc id with uuid field type is null or
we it is generated by using
doc.addField("id", UUID.randomUUID().toString()) in solrj.
1- Suppose my unique id type is string. Can I generate a uuid value from my
unique id string and insert it as a distinct uuid field in Solr?
1- Suppose my unique id type is uuid. If I try to insert a random string
which is not in uuid format, I get the "invalid uuid" error. Is there anyway
to generate a correct uuid value in solr GUI similar to what solrj
UUID.randomUUID().toString() generates?
Best,
Zahra
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Setting of TMP in solr.cmd (for Windows) causes invisibility of the Solr to JDK monitoring tools
Hi Erick, thanks for your feedback, so I've created a corresponding issue in here: https://issues.apache.org/jira/browse/SOLR-12776 Hopefully that will suffice :) Regards Petr __ > Od: "Erick Erickson" > Komu: "solr-user" > Datum: 03.09.2018 01:38 > Předmět: Re: Setting of TMP in solr.cmd (for Windows) causes invisibility of > the Solr to JDK monitoring tools > >Hmmm, please raise a JIRA and, if possible, attach a patch that works >for you. Most of us don't have Windows machines readily available >which hobbles testing, so it's very helpful of someone can test in a >real environment. > >Best, >Erick >On Sun, Sep 2, 2018 at 1:47 PM wrote: >> >> Hi, >> >> please notice the following lines added (among others) to "solr.cmd" by >> commit >> https://github.com/apache/lucene-solr/commit/b36c68b16e67ae701cefce052a4fdbaac88fb65c >> for https://issues.apache.org/jira/browse/SOLR-6833 about 4 years ago: >> >> set TMP=!SOLR_HOME:%EXAMPLE_DIR%=! >> IF NOT "%TMP%"=="%SOLR_HOME%" ( >> set "SOLR_LOGS_DIR=%SOLR_HOME%\..\logs" >> set "LOG4J_CONFIG=file:%EXAMPLE_DIR%\resources\log4j.properties" >> ) >> >> Apparently, the new variable "TMP" is just a temporary one, but by >> coincidence, this variable is also important for JVM! As this system >> variable tells where the "hsperfdata_" directory for storing >> applications' monitoring data should be located. And if this is changed, JDK >> tools like JVisualVM and others won't locally see the given Java >> application, because they search in a different default location. Tested >> with Java 8u152 and Solr 6.3.0. >> >> So Solr authors, could you please rename that "TMP" variable to something >> else, or maybe remove it completely (not sure about the latter alternative)? >> Hopefully it is as easy as described above and I haven't overlooked some >> special meaning of that problematic lines... >> >> Best regards >> >> Petr B. > >
Re: using uuid for documents
Hi Zahra,
We had a similar issue in our collections and we solved this by generating a
UUID value using the content of other fields, in reply to your first question.
Have a look at the section
https://lucene.apache.org/solr/guide/7_4/update-request-processors.html#custom-update-request-processor-chain,
specifically at how SignatureUpdateProcessorFactory is used.
In our case, we create a UUID in field “id” (signatureField) from the fields
bioentity_identifier, property_name and property_value (fields).
true
id
false
bioentity_identifier,property_name,property_value
solr.processor.Lookup3Signature
As for your second question, how is that different to automatic UUID generation?
HTH,
Alfonso
> On 16 Sep 2018, at 11:20, Zahra Aminolroaya wrote:
>
> I have two questions about using uuid:
>
> In Solr refs, it is stated that the unique id could be of uuid type. I
> found we can generate uuid when the doc id with uuid field type is null or
> we it is generated by using
> doc.addField("id", UUID.randomUUID().toString()) in solrj.
>
> 1- Suppose my unique id type is string. Can I generate a uuid value from my
> unique id string and insert it as a distinct uuid field in Solr?
>
>
> 1- Suppose my unique id type is uuid. If I try to insert a random string
> which is not in uuid format, I get the "invalid uuid" error. Is there anyway
> to generate a correct uuid value in solr GUI similar to what solrj
> UUID.randomUUID().toString() generates?
>
> Best,
> Zahra
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
--
Alfonso Muñoz-Pomer Fuentes
Senior Lead Software Engineer @ Gene Expression Team
European Bioinformatics Institute (EMBL-EBI)
European Molecular Biology Laboratory
Tel:+ 44 (0) 1223 49 2633
Skype: amunozpomer
Re: Is that a mistake or bug?
I think we both understand you well :) So once again, to explain it to you,
please have a look at the aforementioned
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/component/ResponseBuilder.java,
these lines:
final Boolean segmentTerminatedEarly = result.getSegmentTerminatedEarly();
if (segmentTerminatedEarly != null) {
rsp.getResponseHeader().add(SolrQueryResponse.RESPONSE_HEADER_SEGMENT_TERMINATED_EARLY_KEY,
segmentTerminatedEarly);
}
Got it now? :)
Petr
__
> Od: "zhenyuan wei"
> Komu: solr-user@lucene.apache.org
> Datum: 03.09.2018 11:21
> Předmět: Re: Is that a mistake or bug?
>
>Oh ~ I feel embarrassed to explaining it again, maybe my english not so
>well~
>my actually mean is: IF QueryResult.segmentTerminatedEarly is boolean
>,not Boolean , declared in QueryResult.
>public class QueryResult{
> private boolean partialResults
> * private Boolean segmentTerminatedEarly; > private boolean
>segmentTerminatedEarly;*
> ..
>}
>
>then in QueryComponent.process() method, like follow :
>
>QueryResult result = new QueryResult();
>cmd.setSegmentTerminateEarly(params.getBool(CommonParams.SEGMENT_TERMINATE_EARLY,
>CommonParams.SEGMENT_TERMINATE_EARLY_DEFAULT));
>
>
>*if (cmd.getSegmentTerminateEarly()) { // this if block code can be
>deleted .result.setSegmentTerminatedEarly(Boolean.FALSE); } *
>
>
>
>
>
>
>
> 于2018年9月3日周一 下午4:52写道：
>
>> Hi, really nope :) Because as MK writes below,
>> result.segmentTerminatedEarly is used as a 3-state variable.
>>
>> The only line that could be improved, is probably replacing
>> "Boolean.FALSE" by simply "false", but that is really a minor thing...
>>
>> Regards
>>
>> PB
>> __
>> > Od: "zhenyuan wei"
>> > Komu: solr-user@lucene.apache.org
>> > Datum: 03.09.2018 10:24
>> > Předmět: Re: Is that a mistake or bug?
>> >
>> >I mean, use terminatedEarly as basic boolean type, then no need to
>> explicitly
>> >assign it as Boolean.FALSE, because basic boolean's default value is
>> false.
>> >
>> >Mikhail Khludnev 于2018年9月3日周一 下午4:13写道：
>> >
>> >> Nope. In this case, it will respond terminatedEarly=false even if noone
>> >> request it.
>> >>
>> >> On Mon, Sep 3, 2018 at 9:09 AM zhenyuan wei wrote:
>> >>
>> >> > Yeah，got it~. So the QueryResult.segmentTerminatedEarly maybe a
>> boolean,
>> >> > instead of Boolean, is better， right?
>> >> >
>> >> > Mikhail Khludnev 于2018年9月3日周一 下午1:36写道：
>> >> >
>> >> > > It's neither, it's on purpose. By default
>> >> result.segmentTerminatedEarly
>> >> > is
>> >> > > null, hence it doesn't appear in result output. see
>> >> > > ResponseBuilder.setResult(QueryResult).
>> >> > > So, if cmd requests early termination, it sets false by default,
>> >> enabling
>> >> > > "false" output even it won't be the case. And later it might be
>> flipped
>> >> > to
>> >> > > true.
>> >> > >
>> >> > >
>> >> > > On Mon, Sep 3, 2018 at 5:57 AM zhenyuan wei
>> wrote:
>> >> > >
>> >> > > > Hi all,
>> >> > > > I saw the code like following:
>> >> > > >
>> >> > > > QueryResult result = new QueryResult();
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> cmd.setSegmentTerminateEarly(params.getBool(CommonParams.SEGMENT_TERMINATE_EARLY,
>> >> > > > CommonParams.SEGMENT_TERMINATE_EARLY_DEFAULT));
>> >> > > > if (cmd.getSegmentTerminateEarly()) {
>> >> > > > result.setSegmentTerminatedEarly(Boolean.FALSE);
>> >> > > > }
>> >> > > >
>> >> > > > It says if request's param segmentTerminateEarly=true, which means
>> >> > search
>> >> > > > maybe terminated early within a segment, then set
>> >> > > > result.setSegmentTerminatedEarly as false , this code is of a
>> little
>> >> > > > confusion
>> >> > > > .
>> >> > > >
>> >> > >
>> >> > >
>> >> > > --
>> >> > > Sincerely yours
>> >> > > Mikhail Khludnev
>> >> > >
>> >> >
>> >>
>> >>
>> >> --
>> >> Sincerely yours
>> >> Mikhail Khludnev
>> >>
>> >
>> >
>>
>
>
Unsubscribe me
How secure is Zookeeper digest auth?
Hi, We plan to enable (digest) authentication and ACL with Zookeeper to improve security. However, we have not been able to answer the question of how secure such a setup will be, given that ZK 3.4.x TCP communication is unencrypted. So, do anyone know if ZK sends the password in cleartext over the network, so that anyone who can sniff the network can also pick up the password, and connect and read/write nodes in ZK? We'll of course add all the firewall and IP filtering we can. Do you have any other tricks you use to increase ZK security? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com
Re: How secure is Zookeeper digest auth?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Jan, On 9/16/18 16:22, Jan Høydahl wrote: > We plan to enable (digest) authentication and ACL with Zookeeper to > improve security. Can you be more explicit? There is HTTP DIGEST auth and then there are "digested" (hashed) passwords for the user-database. The former is secure on the wire and the other one is wire-agnostic. > However, we have not been able to answer the question of how secure > such a setup will be, given that ZK 3.4.x TCP communication is > unencrypted. > > So, do anyone know if ZK sends the password in cleartext over the > network, so that anyone who can sniff the network can also pick up > the password, and connect and read/write nodes in ZK? > > We'll of course add all the firewall and IP filtering we can. Do > you have any other tricks you use to increase ZK security? I'm not using ZK (yet) so this may be supremely ignorant since I don't know what protocol it uses to communicate: I would recommend using mutual-TLS authentication everywhere. I have just deployed such a system (single-node, no cluster/ZK) and all of the communication for both admin and querying are over client-authenticated TLS. Even if an attacker gets onto the box where Solr is running, they cannot attack it without also breaking filesystem privileges or exploiting the users who have access to the Solr client key stores. (I just did a little Googling and it looks like only ZK 3.5+ has TLS available. At any rate, that should be your target for the future if you really want a secure environment. - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAluewOgACgkQHPApP6U8 pFiE1g/8CiRxFySxCPZRU+OdGaw5JjtMNGs3oBDaf75LIQYDnsXAU9wJFjaEKymD snceusjikN85XyPIFBWLhbWvrdjKhJxm29q8xqqnwTkY1WmGis53Es9NHyT/I1UX dY3UGAbf148+ZR6NtCFDQPVQtKKfHqE/VAl2bJzMARTC1nPS3v3mtgKEbrAC5ZqX WMMkb6pOFH58Yj7jeEdHi/y8MKEOeXV3MynWrsSRqGsJsG4Ms55pdBvWtZmIZR+c 0sM4d7zUl18/JjP82YvhhHvHW0IQL+TGKLE1s22p6JRrMU9fzcxNoD9b1r9WORGl UixQETpBPkKw+VWXBesTxTNkprddMH6oGzm2KkWb9zOH0BehF/ChjB1W0vnC7RXB lEKWdNkwbLfrP1r+plpy2aVc3PV0lw3jsJdxLf3tMTEPgzeU6wweiJR+YMW6J0iS 4TWFouuL6yGSY7jT99lW+CmBfKHGEXoUlrxS2WSM9BvYuV8pJvzVuEkb1PmXUQdI rgQIW30Vk0jDwS6SMxdOy/TkbCDAV9dFqsqmYFTSN9W8jBdSx9RevOPnJyVnvCvI qq96sTqhPa0iSHYWWK5PAzZAvfbcRmohcut/1ZWml1pNZlZzBT0QGQUJm9CzXfS7 v6FNf7PrpIiqOlai1Js67Fm6QrWzjGPVhDl474Q1tAG1rFU2cSM= =U0Fj -END PGP SIGNATURE-
Re: How secure is Zookeeper digest auth?
Thanks. Solr uses Zk 3.4.x with no support for TLS. Zk communication is over low level binary TCP. I’m aware that if/when we get 3.5.x support we should deploy SSL. My question is, how do folks secure their current zk and is there a way to circumvent the lack of SSL? I don’t care if people can read the traffic but I do care if the auth credentials can easily be sniffed and replayed, thus enable write acces to zk, which lets you disable all Solr security. Jan > 16. sep. 2018 kl. 22:45 skrev Christopher Schultz > : > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Jan, > >> On 9/16/18 16:22, Jan Høydahl wrote: >> We plan to enable (digest) authentication and ACL with Zookeeper to >> improve security. > > Can you be more explicit? There is HTTP DIGEST auth and then there are > "digested" (hashed) passwords for the user-database. The former is > secure on the wire and the other one is wire-agnostic. > >> However, we have not been able to answer the question of how secure >> such a setup will be, given that ZK 3.4.x TCP communication is >> unencrypted. >> >> So, do anyone know if ZK sends the password in cleartext over the >> network, so that anyone who can sniff the network can also pick up >> the password, and connect and read/write nodes in ZK? >> >> We'll of course add all the firewall and IP filtering we can. Do >> you have any other tricks you use to increase ZK security? > > I'm not using ZK (yet) so this may be supremely ignorant since I don't > know what protocol it uses to communicate: I would recommend using > mutual-TLS authentication everywhere. I have just deployed such a > system (single-node, no cluster/ZK) and all of the communication for > both admin and querying are over client-authenticated TLS. > > Even if an attacker gets onto the box where Solr is running, they > cannot attack it without also breaking filesystem privileges or > exploiting the users who have access to the Solr client key stores. > > (I just did a little Googling and it looks like only ZK 3.5+ has TLS > available. At any rate, that should be your target for the future if > you really want a secure environment. > > - -chris > -BEGIN PGP SIGNATURE- > Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ > > iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAluewOgACgkQHPApP6U8 > pFiE1g/8CiRxFySxCPZRU+OdGaw5JjtMNGs3oBDaf75LIQYDnsXAU9wJFjaEKymD > snceusjikN85XyPIFBWLhbWvrdjKhJxm29q8xqqnwTkY1WmGis53Es9NHyT/I1UX > dY3UGAbf148+ZR6NtCFDQPVQtKKfHqE/VAl2bJzMARTC1nPS3v3mtgKEbrAC5ZqX > WMMkb6pOFH58Yj7jeEdHi/y8MKEOeXV3MynWrsSRqGsJsG4Ms55pdBvWtZmIZR+c > 0sM4d7zUl18/JjP82YvhhHvHW0IQL+TGKLE1s22p6JRrMU9fzcxNoD9b1r9WORGl > UixQETpBPkKw+VWXBesTxTNkprddMH6oGzm2KkWb9zOH0BehF/ChjB1W0vnC7RXB > lEKWdNkwbLfrP1r+plpy2aVc3PV0lw3jsJdxLf3tMTEPgzeU6wweiJR+YMW6J0iS > 4TWFouuL6yGSY7jT99lW+CmBfKHGEXoUlrxS2WSM9BvYuV8pJvzVuEkb1PmXUQdI > rgQIW30Vk0jDwS6SMxdOy/TkbCDAV9dFqsqmYFTSN9W8jBdSx9RevOPnJyVnvCvI > qq96sTqhPa0iSHYWWK5PAzZAvfbcRmohcut/1ZWml1pNZlZzBT0QGQUJm9CzXfS7 > v6FNf7PrpIiqOlai1Js67Fm6QrWzjGPVhDl474Q1tAG1rFU2cSM= > =U0Fj > -END PGP SIGNATURE-
Re: Unsubscribe me
Please follow the instructions here to unsubscribe: http://lucene.apache.org/solr/community.html#mailing-lists-irc You must use the same email address that you used to subscribe. Regards, Edwin On Mon, 17 Sep 2018 at 00:49, Srinu Lella wrote: > >
Re: Logging fails when starting Solr in Windows using solr.cmd
On 2018/07/03 08:53:20, ja...@jafurrer.ch wrote: > Hi, > > I was intending to open an Issue in Jira when I read that I'm supposed > to first contact this mailinglist. > > Problem description > == > > System: Microsoft Windows 10 Enterprise Version 10.0.16299 Build 16299 > > Steps to reproduce the problem: > 1) Download solr-7.4.0.tgz > 2) Unzip to C:\solr-7.4.0 > 3) No changes (configuration or otherwise) whatsoever > 4) Open cmd.exe > 5) Execute the following command: cd c:\solr-7.4.0\bin > 6) Execute the following command: solr.cmd start -p 8983 > 7) The following console output appears: > > > c:\solr-7.4.0\bin>solr.cmd start -p 8983 > ERROR StatusLogger Unable to access > file:/c:/solr-7.4.0/server/file:c:/solr-7.4.0/server/scripts/cloud-scripts/log4j2.xml > java.io.FileNotFoundException: > c:\solr-7.4.0\server\file:c:\solr-7.4.0\server\scripts\cloud-scripts\log4j2.xml > > (Die Syntax für den Dateinamen, Verzeichnisnamen oder die > Datenträgerbezeichnung ist falsch) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at java.io.FileInputStream.(FileInputStream.java:93) > at > sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90) > at > sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188) > at java.net.URL.openStream(URL.java:1045) > at > org.apache.logging.log4j.core.config.ConfigurationSource.fromUri(ConfigurationSource.java:247) > at > org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:404) > at > org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:346) > at > org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:260) > at > org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:615) > at > org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:636) > at > org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:231) > at > org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:153) > at > org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45) > at > org.apache.logging.log4j.LogManager.getContext(LogManager.java:194) > at > org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:121) > at > org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43) > at > org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46) > at > org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29) > at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358) > at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383) > at org.apache.solr.util.SolrCLI.(SolrCLI.java:228) > ERROR StatusLogger Unable to access > file:/c:/solr-7.4.0/server/file:c:/solr-7.4.0/server/resources/log4j2.xml > java.io.FileNotFoundException: > c:\solr-7.4.0\server\file:c:\solr-7.4.0\server\resources\log4j2.xml (Die > Syntax für den Dateinamen, Verzeichnisnamen oder die > Datenträgerbezeichnung ist falsch) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at java.io.FileInputStream.(FileInputStream.java:93) > at > sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90) > at > sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188) > at java.net.URL.openStream(URL.java:1045) > at > org.apache.logging.log4j.core.config.ConfigurationSource.fromUri(ConfigurationSource.java:247) > at > org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:404) > at > org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:346) > at > org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:260) > at > org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:615) > at > org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:636) > at > org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:231) > at > org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:153) > at > org.apache.l
