Try not committing so often (perhaps until you are done).
Don't use post.sh, or modify it to remove the commit.
OK, I modified it to not commit after and I also realized I had
SOLR-126 (autocommit) on, which I disabled. Is there a rule of thumb
on when to commit / optimize?
Part of the problem might be repeatedly inserting the same doc over
and over again-- that is an odd pattern of deletes, which might be
triggering a bad performance case on the lucene or solr side (I'm
assuming the doc has a unique key).
Could be, but the same issue occurs on 400K unique docs. I made the
post.sh test case to see what exactly the issue is.
I've discovered something wonky with commits and open files...
If either autoCommit is on, or I commit after every add, the number
of open file descriptors from Lucene goes up way high and does not go
back down. I just ran
# lsof | grep solr | wc -l
after adding 1000 docs and got 125265 open fdescs. If I stop adding
docs this count does not go down -- it does not go down until I
restart solr. This would be the cause of my too many files open
problem. Turning off autocommit / not commiting after every add keeps
this count steady at 100-200. The files are all of type:
java 32254 bwhitman 3654r REG 8,1 12
15417767 /home/bwhitman/solr/working/data/index/_86u.nrm (deleted)
java 32254 bwhitman 3655r REG 8,1 42024
15417813 /home/bwhitman/solr/working/data/index/_86t.fdt (deleted)
java 32254 bwhitman 3656r REG 8,1 16
15417814 /home/bwhitman/solr/working/data/index/_86t.fdx (deleted)
java 32254 bwhitman 3657r REG 8,1 27420
15417817 /home/bwhitman/solr/working/data/index/_86t.tis (deleted)
java 32254 bwhitman 3658r REG 8,1 368
15417818 /home/bwhitman/solr/working/data/index/_86t.tii (deleted)
java 32254 bwhitman 3659r REG 8,1 7652
15417815 /home/bwhitman/solr/working/data/index/_86t.frq (deleted)
java 32254 bwhitman 3660r REG 8,1 24860
15417816 /home/bwhitman/solr/working/data/index/_86t.prx (deleted)
java 32254 bwhitman 3661r REG 8,1 20
15417819 /home/bwhitman/solr/working/data/index/_86t.nrm (deleted)
java 32254 bwhitman 3662r REG 8,1 20
15417819 /home/bwhitman/solr/working/data/index/_86t.nrm (deleted)
java 32254 bwhitman 3663r REG 8,1 20
15417819 /home/bwhitman/solr/working/data/index/_86t.nrm (deleted)
java 32254 bwhitman 3664r REG 8,1 20
15417819 /home/bwhitman/solr/working/data/index/_86t.nrm (deleted)
java 32254 bwhitman 3665r REG 8,1 20
15417819 /home/bwhitman/solr/working/data/index/_86t.nrm (deleted)
java 32254 bwhitman 3666r REG 8,1 20
15417819 /home/bwhitman/solr/working/data/index/_86t.nrm (deleted)
java 32254 bwhitman 3667r REG 8,1 20
15417819 /home/bwhitman/solr/working/data/index/_86t.nrm (deleted)
java 32254 bwhitman 3668r REG 8,1 20
15417819 /home/bwhitman/solr/working/data/index/_86t.nrm (deleted)
java 32254 bwhitman 3669r REG 8,1 210120000
15417669 /home/bwhitman/solr/working/data/index/_85y.fdt
java 32254 bwhitman 3670r REG 8,1 80000
15417670 /home/bwhitman/solr/working/data/index/_85y.fdx
java 32254 bwhitman 3671r REG 8,1 46736
15417673 /home/bwhitman/solr/working/data/index/_85y.tis
java 32254 bwhitman 3672r REG 8,1 503
15417674 /home/bwhitman/solr/working/data/index/_85y.tii
java 32254 bwhitman 3673r REG 8,1 43936224
15417671 /home/bwhitman/solr/working/data/index/_85y.frq
java 32254 bwhitman 3674r REG 8,1 124300000
15417672 /home/bwhitman/solr/working/data/index/_85y.prx
java 32254 bwhitman 3675r REG 8,1 80004
15417675 /home/bwhitman/solr/working/data/index/_85y.nrm
java 32254 bwhitman 3676r REG 8,1 80004
15417675 /home/bwhitman/solr/working/data/index/_85y.nrm
java 32254 bwhitman 3677r REG 8,1 80004
15417675 /home/bwhitman/solr/working/data/index/_85y.nrm
java 32254 bwhitman 3678r REG 8,1 80004
15417675 /home/bwhitman/solr/working/data/index/_85y.nrm
java 32254 bwhitman 3679r REG 8,1 80004
15417675 /home/bwhitman/solr/working/data/index/_85y.nrm
java 32254 bwhitman 3680r REG 8,1 80004
15417675 /home/bwhitman/solr/working/data/index/_85y.nrm
java 32254 bwhitman 3681r REG 8,1 80004
15417675 /home/bwhitman/solr/working/data/index/_85y.nrm
java 32254 bwhitman 3682r REG 8,1 80004
15417675 /home/bwhitman/solr/working/data/index/_85y.nrm
Bug or feature?
