Hi All,
I built a server with Lucene/Solr (branch_8_0,
<https://github.com/apache/lucene-solr/tree/branch_8_0>
https://github.com/apache/lucene-solr/tree/branch_8_0) and used a client to
test the server. However, when the client sent requests, I found the server
outputs lots of seemingly error information. Meanwhile, the client didn't
save any results to the output file. I am very appreciated if you can give
some suggestions about how to build a simple and correct client/server
benchmark on lucene/solr with luceneutil (
https://github.com/mikemccand/luceneutil.git). Thanks much in advance! All
the detailed information about my test is as follows.
*The server-side outputs
----------------------> output seemingly error information without any
information about whether the query from the client is successful or not.*
===================
$ ./run_server.sh
[...]
Thread-1: ignoring exc:
java.util.IllegalFormatConversionException: d != java.lang.String
at
java.base/java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4426)
at
java.base/java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2938)
at java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:2892)
at java.base/java.util.Formatter.format(Formatter.java:2673)
at java.base/java.util.Formatter.format(Formatter.java:2609)
at java.base/java.lang.String.format(String.java:2938)
at perf.RemoteTaskSource.taskDone(RemoteTaskSource.java:148)
at perf.TaskThreads$TaskThread.run(TaskThreads.java:96)
Thread-2: ignoring exc:
java.util.IllegalFormatConversionException: d != java.lang.String
at
java.base/java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4426)
at
java.base/java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2938)
at java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:2892)
at java.base/java.util.Formatter.format(Formatter.java:2673)
at java.base/java.util.Formatter.format(Formatter.java:2609)
at java.base/java.lang.String.format(String.java:2938)
at perf.RemoteTaskSource.taskDone(RemoteTaskSource.java:148)
at perf.TaskThreads$TaskThread.run(TaskThreads.java:96)
[...]
*The client-side outputs
--------------------------------> no results are saved after the client
finishes, such that I don't know how many queries are successful.*
==================
$ ./run_client.sh
Mean QPS 10.0
10521 tasks
265 tasks after prune
Sent all tasks 3 times.
79.9 sec: Done sending tasks...
^C 348.9 sec: Done...
*The steps to build the server and client for the test*
=========================================
*Step #1*: download luceneutil and rollback to commit
``Lucene70->Lucene80''.
cd /home/harry/test/lucene80; git clone
https://github.com/mikemccand/luceneutil.git ./util
cd util; git reset --hard fd45176d449e70cfda57097df0f4d3302df0e5dc
*Step #2*: setup luceneutil according to README at
https://github.com/mikemccand/luceneutil.git.
python src/python/setup.py -download; cd ../data; unlzma
enwiki-20120502-lines-1k.txt.lzma.
*Step #3*: ``running a first benchmark" according to README at
https://github.com/mikemccand/luceneutil.git.
cd ..; git clone https://github.com/apache/lucene-solr.git; cd lucene_solr;
git checkout branch_8_0
cd ..; cp lucene_solr trunk -rf; cp lucene_solr patch -rf; cd util; python
src/python/localrun.py -source wikimedium10k
*Step #4*: Run the server.
cd scripts; ./run_server.sh
$ cat ./run_server.sh
#!/bin/bash
SERVER_IP="server:127.0.0.1:7777"
LOG_FILE="./log"
java -server -Xms2g -Xmx2g -XX:-TieredCompilation
-XX:+HeapDumpOnOutOfMemoryError -Xbatch -classpath
/home/harry/test/lucene80/patch/lucene/build/core/lucene-core-8.0.0-SNAPSHOT.jar:/home/harry/test/lucene80/patch/lucene/build/core/classes/test:/home/harry/test/lucene80/patch/lucene/build/sandbox/classes/java:/home/harry/test/lucene80/patch/lucene/build/misc/classes/java:/home/harry/test/lucene80/patch/lucene/build/facet/classes/java:/home/mike/src/lucene-c-boost/dist/luceneCBoost-SNAPSHOT.jar:/home/harry/test/lucene80/patch/lucene/build/analysis/common/classes/java:/home/harry/test/lucene80/patch/lucene/build/analysis/icu/classes/java:/home/harry/test/lucene80/patch/lucene/build/queryparser/classes/java:/home/harry/test/lucene80/patch/lucene/build/grouping/classes/java:/home/harry/test/lucene80/patch/lucene/build/suggest/classes/java:/home/harry/test/lucene80/patch/lucene/build/highlighter/classes/java:/home/harry/test/lucene80/patch/lucene/build/codecs/classes/java:/home/harry/test/lucene80/patch/lucene/build/queries/classes/java:/home/harry/test/lucene80/patch/lucene/facet/lib/hppc-0.8.1.jar:/home/harry/test/lucene80/util/lib/HdrHistogram.jar:/home/harry/test/lucene80/util/build
perf.SearchPerfTest -dirImpl MMapDirectory -indexPath
/home/harry/test/lucene80/indices/wikimedium10k.trunk.facets.taxonomy:Date.taxonomy:Month.taxonomy:DayOfYear.sortedset:Month.sortedset:DayOfYear.Lucene80.Lucene50.nd0.01M
-analyzer StandardAnalyzer -taskSource $SERVER_IP -searchThreadCount 2
-field body -staticSeed 0 -seed 0 -similarity BM25Similarity -commit multi
-hiliteImpl FastVectorHighlighter -log $LOG_FILE -topN 10 -pk
*Step #5*: Run the client.
./run_client.sh
$cat ./run_client.sh
#!/bin/bash
python -u /home/harry/test/lucene80/util/src/python/sendTasks.py
/home/harry/test/lucene80/util/tasks/wikimedium.1M.nostopwords.tasks
localhost 7777 10 10 70 results.pk
Thanks,
harry
