On 01/04/2016 09:04 PM, Alvaro Soliverez wrote:
Hello Joe,
You are right about the performance of KMyMoney using the database.
End of November we had a chat with another developer and it was one of
our main items. Basically, the problem is that the database engine
uses the same logic as the plain text file, instead of taking
advantage of the SQL engine for calculating balances.
In this case, I can guess it's recalculating the balances for every
imported transaction. That's probably something that can be fixed, so
that it takes a more sensible. Not in the range of milliseconds, but
still much better than 30 minutes.
Overall, the main issue is calculating balances, as it goes through
every transaction since the opening of the account. It uses a cache,
but any change in transactions invalidates the cache and it has to
recalculate it.
Thank you for your feedback.
Regards,
Alvaro
On 2016-01-04 20:12, Joe W. Byers wrote:
All,
I think the development list is the best for this issue. Yesterday, I
executed a QIF file import of 109 transactions with 34 new payees to
be added to Kmymoney that is using a mysqlDB. This process took over
18 hours to complete. I think that this is a problem. I have
analyzed thousands of transactions as an energy risk manager using
other software both downloading and uploading them. 109 should take
only a few milliseconds.
I am using Fedora 23 with Kmymoney 4.7.1 and KDE Dev platform 4.14.14.
I will provided all information to assist with resolving this
database performance issue.
Happy New Year.
--
JOE W. BYERS
Alvaro,
I would be happy to work with you on this enhancement. The performance
is not just with this import, but with basic loading and updates. If I
understand your explanation, the logic refreshes all balance if the
cache is invalidated even if only on account needs refreshing. I agree
that utilizing database functionality would be beneficial, but I caution
on some of this. I work with ETRM systems and off ETRM enhancements,
the debate between quantitative functions and systems is always what
should be on the db server side and what can be on the client with ETL
layers to move results back and forth. There is a trade off. In my
experience the client has always had higher performance for the user on
some things while the db server is a workhorse for transactions (in the
hundreds of thousands). If you move all the accounting to the db
server, then Kmymoney could basically be forked between a client based
and DB server version.
Thank you for all of this work and again, I have a test db version
configured that we can use to test.
--
*Joe W. Byers*
