On Fri, 6 Jan 2017 10:45:02 -0600
Webster Homer wrote:
> I was seeing something like this, and it turned out to be a problem with
> our autoCommit and autoSoftCommit settings. We had overly aggressive
> settings that eventually started failing with errors around too many
> warming searchers etc..
For NOT NULL, I had some success using:
WHERE field_name <> '' (greater or less than empty quotes)
Best regards,
Gethin.
From: Joel Bernstein
Sent: 05 January 2017 20:12:19
To: solr-user@lucene.apache.org
Subject: Re: Regarding /sql -- WHERE <> IS NULL and IS
Hi All,
Background:
I have a mainframe file that I want to upload and the data is pipe delimited.
Some of the records however have a few fields less that others within the same
file and when I try to import the file, Solr has an issue with the amount of
columns vs the amount of values, which is
Hi All,
My solr server has a few large index files (say ~10G). I am looking
for some help on breaking them it into smaller ones (each < 4G) to satisfy
my application requirements. Are there any such tools available?
Appreciate your help.
Thanks
NRC
Hi All,
My solr server has a few large index files (say ~10G). I am looking
for some help on breaking them it into smaller ones (each < 4G) to satisfy
my application requirements. Basically, I am not looking for any
optimization of index here (ex: optimize, expungeDeletes etc.).
Are there a
Hi,
After some reading into the documentation, supposedly the Lucene FieldCache
is the only one that it's not possible to disable.
Fetching the config for a collection through the REST API I found an entry
like this:
"query": {
"useFilterForSortedQuery": true,
"queryResultWindowSize": 1,
"queryR
Hello, Yago.
"size": "1", "showItems": "-1", "initialSize": "10", "name":
"fieldValueCache"
These are Solr's UnInvertedFields, not Lucene's FieldCache.
That 1 is for all fields of the collection schema.
Collection reload or commit drop all entries from this cache.
On Mon, Jan 9, 2017 at
Thanks for re reply Mikhail,
Do you know if the 1 value is configurable? My insert rate is so high
(5000 docs/s) that the cache it's quite useless.
In the case of the Lucene field cache, it's possible "clean" it in some way?
Some cache is eating my memory heap.
-
Best regards
/Yago
-
Hi All,
I have a problem simillar to this one, where the indexes in multiple solr
shards has created large index files (~10 GB each) and wanted to split this
large file on each shard into smaller files.
Please provide some guidelines.
Thanks,
Manan Sheth
___
On Mon, Jan 9, 2017 at 2:17 PM, Yago Riveiro wrote:
> Thanks for re reply Mikhail,
>
> Do you know if the 1 value is configurable?
yes. in solrconfig.xml
https://cwiki.apache.org/confluence/display/solr/Query+Settings+in+SolrConfig#QuerySettingsinSolrConfig-Caches
iirc you cant' fully disabl
Hi,
Can anybody help me, I need to get term frequency for a specific filed, I
use the techproduct example and I use this code:
//
import java.util.List;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.so
How we can integrate SOLR in IBM filenet 5.2?
--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-integrate-SOLR-in-ibm-filenet-5-2-1-tp4313090.html
Sent from the Solr - User mailing list archive at Nabble.com.
The documentation says that the only caches configurable are:
- filterCache
- queryResultCache
- documentCache
- user defined caches
There is no entry for fieldValueCache and in my case all of list in the
documentation are disable ...
--
/Yago Riveiro
On 9 Jan 2017 13:20 +, Mikhail Khludn
Apache ManifolCF is probably your friend here:
http://manifoldcf.apache.org/en_US/index.html
-Original message-
> From:puneetmishra2555
> Sent: Monday 9th January 2017 14:37
> To: solr-user@lucene.apache.org
> Subject: How to integrate SOLR in ibm filenet 5.2.1?
>
> How we can integra
Hi,
Try split on linux or unix
split -l 100 originalfile.csv
this will split a file into 100 lines each
see other options for how to split like size
-Original Message-
From: Narsimha Reddy CHALLA [mailto:chnredd...@gmail.com]
Sent: 09 January 2017 12:12 PM
To: solr-user@lucene.apache.
Hi All,
While doing index merging through IndexWriter.forceMerge method in solr 6.2.1,
I am passing the argument as 30, but it is still merging all the data (earlier
collection use to have 10 segments) into single segment. Please provide some
information in understading the behaviour.
Thanks
Is this really works for lucene index files?
Thanks,
Manan Sheth
From: Moenieb Davids
Sent: Monday, January 9, 2017 7:36 PM
To: solr-user@lucene.apache.org
Subject: RE: Help needed in breaking large index file into smaller ones
Hi,
Try split on linux or
No, it does not work by splitting. First of all lucene index files are not
text files. There is a segment_NN file which will refer index files in a
commit. So, when we split a large index file into smaller ones, the
corresponding segment_NN file also needs to be updated with new index files
OR a ne
You can try to reindex your data to another collection with more shards
--
/Yago Riveiro
On 9 Jan 2017 14:15 +, Narsimha Reddy CHALLA , wrote:
> No, it does not work by splitting. First of all lucene index files are not
> text files. There is a segment_NN file which will refer index files in
That's good to hear. I didn't think there would be any reason that using
lvm would impact solr's performance but wanted to see if there was anything
I've missed.
As far as other performance goes, we use pcie and sata solid state drives
since the indexes are mostly too large to cache entirely in me
Hi,
Using the CloudSolrStream, is it possible define the setZkConnectTimeout and
setZkClientTimeout of internal CloudSolrClient?
The default negotiation timeout is set to 10 seconds.
Regards,
/Yago
-
Best regards
/Yago
--
View this message in context:
http://lucene.472066.n3.nabble.com
The root cause was the aggressive logging filling up the file system. Our
admins have the logs on the same file system with the data, so when the
filesystem got full it couldn't write to the transaction logs which
corrupted them
Thank you for the tips on recovery, I will forward them to our admins
Can you set Solr config segments to a higher number, don't optimize and you
will get smaller files after a new index is created.
Can you reindex ?
Bill Bell
Sent from mobile
> On Jan 9, 2017, at 7:15 AM, Narsimha Reddy CHALLA
> wrote:
>
> No, it does not work by splitting. First of all luce
Try disabling and perf may get better
Bill Bell
Sent from mobile
> On Jan 9, 2017, at 6:41 AM, Yago Riveiro wrote:
>
> The documentation says that the only caches configurable are:
>
> - filterCache
> - queryResultCache
> - documentCache
> - user defined caches
>
> There is no entry for fie
I am available for consulting projects if your project needs help.
Been doing Solr work for 6 years...
Bill Bell
Sent from mobile
Hi All,
Is it possible to have facet date range without specifying start and and of the
range.
Otherwise, is it possible to put in the same request start to min value and end
to max value.
Thank you.
Regards,NKI.
Yeah we normally take the number of GB on a machine for the index size on disk
and then double it for memory...
For example we have 28gb on disk and we see great perf at 64gb ram.
If you can do that you will probably get good results. Remember to not give
Java much memory. We set it at 12gb. We
can we customize SOLR search for IBM Filenet 5.2?
--
View this message in context:
http://lucene.472066.n3.nabble.com/can-we-customize-SOLR-search-for-IBM-Filenet-5-2-tp4313091.html
Sent from the Solr - User mailing list archive at Nabble.com.
Hi,
Aplogies for my response, did not read the question properly.
I was speaking about splitting files for import
-Original Message-
From: billnb...@gmail.com [mailto:billnb...@gmail.com]
Sent: 09 January 2017 05:45 PM
To: solr-user@lucene.apache.org
Subject: Re: Help needed in breaking
This probably says why
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/core/SolrConfig.java#L258
On Mon, Jan 9, 2017 at 4:41 PM, Yago Riveiro wrote:
> The documentation says that the only caches configurable are:
>
> - filterCache
> - queryResultCache
> - doc
Ok, then I need to configure to reduce the size of the cache.
Thanks for the help Mikhail.
--
/Yago Riveiro
On 9 Jan 2017 17:01 +, Mikhail Khludnev , wrote:
> This probably says why
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/core/SolrConfig.java#L
Hi, i made a soir project with multiple entity.
I want to launch one entity index with an URL.
How i can choose the entity that i want in my url?
Thank to your help
--
View this message in context:
http://lucene.472066.n3.nabble.com/Soir-Ulr-entity-tp4313172.html
Sent from the Solr - User mail
Hi,
I’m Shashank. I’m new to Solr and was trying to use amazon-aws sdk along with
Solr. I added amazon-aws.jar and its third party dependencies under
/solr-6.3.0/server/solr/lib folder. Even after I add all required dependencies,
I keep getting NoClassDefinitionError and NoSuchMethod Errors. I
Perhaps you can copy this index into a separate location. Remove odd and
even docs into former and later indexes consequently, and then force merge
to single segment in both locations separately.
Perhaps shard splitting in SolrCloud does something like that.
On Mon, Jan 9, 2017 at 1:12 PM, Narsimh
Hello Huda,
Try to check this
https://github.com/apache/lucene-solr/blob/master/solr/solrj/src/test/org/apache/solr/client/solrj/response/TermsResponseTest.java
On Mon, Jan 9, 2017 at 4:31 PM, huda barakat
wrote:
> Hi,
> Can anybody help me, I need to get term frequency for a specific filed,
Can you provide more information about:
- Are you using Solr in standalone or SolrCloud mode? What version of Solr?
- Why do you want this? Lack of disk space? Uneven distribution of data on
shards?
- Do you want this data together i.e. as part of a single collection?
You can check out the followi
Why do you have a requirement that the indexes be < 4G? If it's
arbitrarily imposed why bother?
Or is it a non-negotiable requirement imposed by the platform you're on?
Because just splitting the files into a smaller set won't help you if
you then start to index into it, the merge process will ju
Why? What do you think this will accomplish? I'm wondering if this is
an XY problem.
Best,
Erick
On Mon, Jan 9, 2017 at 7:48 AM, Manan Sheth wrote:
> Hi All,
>
> I have a problem simillar to this one, where the indexes in multiple solr
> shards has created large index files (~10 GB each) and wa
Currently these are not settable.It's easy enough to add a setter for this
values. What types of behaviors have you run into when CloudSolrClient is
having timeouts issues?
Joel Bernstein
http://joelsolr.blogspot.com/
On Mon, Jan 9, 2017 at 10:06 AM, Yago Riveiro
wrote:
> Hi,
>
> Using the Clou
On 12/22/2016 8:10 AM, Webster Homer wrote:
> While testing CDCR I found that it is writing tons of log messages per
> second. Example:
> 2016-12-21 23:24:41.652 INFO (qtp110456297-13) [c:sial-catalog-material
> s:shard1 r:core_node1 x:sial-catalog-material_shard1_replica1]
> o.a.s.c.S.Request [si
On 1/8/2017 11:21 PM, Manan Sheth wrote:
> Currently, We are in process of upgrading existing Solr indexes from Solr 4.x
> to Solr 6.2.1. In order to upgrade existing indexes we are planning to use
> IndexUpgrader class in sequential manner from Solr 4.x to Solr 5.x and Solr
> 5.x to Solr 6.2.1.
On 11/28/2016 11:06 AM, Walter Underwood wrote:
> Worst case:
> 1. Disable merging.
> 2. Delete all the documents.
> 3. Add all the documents.
> 4. Enable merging.
>
> After step 3, you have two copies of everything, one deleted copy and one new
> copy.
> The merge makes a third copy.
Just gettin
On 1/9/2017 11:35 AM, Shashank Pedamallu wrote:
> I’m Shashank. I’m new to Solr and was trying to use amazon-aws sdk
> along with Solr. I added amazon-aws.jar and its third party
> dependencies under /solr-6.3.0/server/solr/lib folder. Even after I
> add all required dependencies, I keep getting No
On 1/9/2017 6:31 AM, huda barakat wrote:
> Can anybody help me, I need to get term frequency for a specific
> filed, I use the techproduct example and I use this code:
The variable "terms" is null on line 29, which is why you are getting
NullPointerException.
> query.setRequestHandler("terms");
Hi Erick,
Its due to some past issues observed with Joins on Solr 4, which got OOM on
joining to large indexes after optimization/compaction, if those are stored as
smaller files those gets fit into memory and operations are performed
appropriately. Also, there are slow write/commit/updates are
Additionally to answer Anshum's queries,
We are currently using Solr 4.10 and planning to upgrade to Solr 6.2.1 and
upgradation process in creating the current problem.
We are using it in SolrCloud with 8-10 shards split on different nodes each
having segment size ~30 GB for some collection and
Hi,
I wasn't happy with how our current solr configuration handled diacritics (like
'é') in the text and in search queries, since it simply considered the letter
with a diacritic as a distinct letter. Ie 'é' didn't match 'e', and vice versa.
Except for a handful rare words where the diacritical
47 matches
Mail list logo
