Staffan opened up this feature request for ooSQLite and I thought I would
discuss it with him on this list so that other interested people would be
aware of it.
I'm not going into a lot of detail about the SQLite feature itself, but it
essentially allows you to define your own way of sorting strings.
The original title of the request was: Registration of additional functions
in ooSQLite and I wanted to explain why I changed the title to Staffan.
Basically SQLite allows you to register user defined collation, but you
can also register user defined functions, user defined aggregates, and user
defined modules.
The handling of user defined collations is completely separate from user
defined functions. I have always intended to add these 2 features to
ooSQLite. In the body of the request Staffan talked about his desire to
try a user defined collation, so I narrowed the scope of the feature
request to collations. I will add the user defined functions next in a
separate RFE. The user defined function feature will use the method
createFunction().
Two methods support user defined collations: createCollation() and
collationNeeded()
createCollation(), and createFunction() when it is implemented, allow you
to implement the collation or function in Rexx. However SQLite allows the
user collations and functions to be written in C / C++ in an external
shared library. There is a mechanism to load an entire library of
collations and functions. This is now supported in ooSQLite with the new
loadExtension() method. And the ooSQLite distribution has an example of
how to write the extension library and use it.
createCollation() and createFunction() will also support adding an external
collation or function, in addition to supporting the definition in Rexx.
This is important because, while many Rexx programmers have little or no C
/ C++ experience, there are many who do. In addition Rexx programmers with
no C experience who want or need a collation written in C / C++ may be able
to talk someone else into writing one for them.
>From Staffan:
"I've compared this way of sorting to the sort callback functions available
for list and tree views and as expected the overhead seems to be more or
less the same. Using test data with around 11,000 rows both methods of
sorting increase the response times about 25 times, so it's a fairly costly
operation. This I believe is caused by the constant transfer of operation
between the rexx and non-rexx environments, in which case I guess not much
can be done to improve it."
This is true to a degree. In order to sort, the SQLite database repeatedly
invokes the C callback function in the ooSQLite framework with 2 strings to
compare. This C function needs to get access to the interpreter kernel,
convert the C strings to Rexx strings, invoke the Rexx method that does
the comparison and return the result to SQLite. Acquiring access to the
interpreter kernel is probably the most costly part of that.
But there is one thing that might help. In addition to gaining access to
the kernel there is the problem of threading. There can be
no guarantee that SQLite will invoke the callback on the same thread as the
Rexx programmer has used to invoke createCollation(). If it is not the
same thread there is the additional overhead of acquiring a thread context
and releasing it each time the callback is invoked.
It seems to me, with no testing to back this up, that you can ensure, (or
at least greatly increase the likelihood,) that everything stays on the
same thread by using the collationNeeded() method.
Rather than use createCollation() first, you would use collationNeeded()
first. This registers a callback function that SQLited invokes when it
encounters an undefined collation. Then in your collation needed callback,
you use createCollation(). It is highly probable then that the collation
callback will run on the same thread as it was created on. This eliminates
the need to acquire a new thread context and release it each time the
callback is invoked.
Of course this would only hold true for the first use of the collation if
your Rexx program is using a lot of threads. So, it should help if you can
make sure the collation is always used from the same thread after its first
use.
There is the collationCustom.rex example that shows how to use
collationNeeded().
In addition Staffan, for the external library example I just defined a
collation that is essentially a reverse sort. If you were to share your
Rexx collation definition with me, it is highly likely that I would add a C
/ C++ definition of that collation to the example. Which you could of
course use in your programs.
And I would highly encourage anyone implementing C / C++ user defined
extensions to contribute them to the ooSQLite project to make them publicly
available.
--
Mark Miesfeld
------------------------------------------------------------------------------
Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1
_______________________________________________
Oorexx-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/oorexx-users
