GitHub user twilmes opened a pull request:
https://github.com/apache/incubator-tinkerpop/pull/307
TINKERPOP-1295 Precompile ScriptInputFormat scripts once during
initialization of ScriptRecordReader
This update precompiles an input script and then reads the input file using
the compiled script instead of repeatedly calling the engine.eval(). This
should cut down on the time spent repeatedly eval-ing the input script.
I ran a quick and dirty benchmark on my measly macbook with
SparkGraphComputer, 2 workers. `g.V().count()` on a test file with 250,000
vertices and a simple `.groovy` script to read it in.
Average of 10 runs
-------------------------
before (TP 3.2.0 - engine.eval): 14975.7 ms
after (TP-1295 - w/ compiled script): 10163.6 ms
`mvn clean install` success
VOTE: +1
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-tinkerpop TINKERPOP-1295
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-tinkerpop/pull/307.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #307
----
commit be6f8d79f606cc0a39dd45457e26328dd31e5636
Author: Ted Wilmes <[email protected]>
Date: 2016-05-06T19:54:54Z
Precompile scripts during ScriptRecordReader initialization.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
