Hi Sebastian,
Looking at the Virtuoso configuration file params you provided I not
you are running in LiteMode, with only the following set:
[Parameters]
LiteMode=1
ServerPort=1111
DisableTcpSocket=1
PrefixResultNames=0
ServerThreads=5
CheckpointInterval=10
MaxDirtyBuffers=50
SchedulerInterval=5
FreeTextBatchSize=1000
Which means the Virtuoso NumberOfBuffers which is not set and
controls the amount of RAM used by the server will be the default of
2000 * 8K buffers = 16K . So you entire Virtuoso Server is running
within 16K of memory and you are attempting to insert a 1MB plus
triple, I think it will have problems as the server will then having
to be swapping between memory and disk like crazy. I would suggest you
set the NumberOfBuffers to something like 200000 to give the Virtuoso
server a reasonable amount of RAM to perform such inserts ...
Best Regards
Hugh Williams
Professional Services
OpenLink Software
Web: http://www.openlinksw.com
Support: http://support.openlinksw.com
Forums: http://boards.openlinksw.com/support
Twitter: http://twitter.com/OpenLink
On 8 Oct 2009, at 21:07, Sebastian Trueg wrote:
I cannot give you the odbc.log from the original situation. That
file is
more than 1GB in size.
But I recreated the situation with some dummy data using an empty
database which I started with the attached config file. This time the
query even shows up in the iodbc.log file.
The virtuoso log is empty.
I did not wait overnight this time. ;)
Hope this helps.
Cheers,
Sebastian
Hugh Williams wrote:
Hi Sebastian,
Can you please provide a complete iODBC trace so we can see the query
and sequence of call be made. It might also we worth checking the
Virtuoso Server log (virtuoso.log) to see if any errors are being
reported on that side.
Best Regards
Hugh Williams
Professional Services
OpenLink Software
Web: http://www.openlinksw.com
Support: http://support.openlinksw.com
Forums: http://boards.openlinksw.com/support
Twitter: http://twitter.com/OpenLink
On 8 Oct 2009, at 10:59, Sebastian Trueg wrote:
I have a problem with adding very large literal objects to
Virtuoso. In
fact, Virtuoso sat there during the whole night using nearly all
CPU and
not finishing the insert query.
In the log it looks something like this:
[000006.264449]
nepomukservices 7F1307359950 ENTER SQLExecDirect
SQLHSTMT 0x179f1e0
SQLCHAR * 0x19c5928
SQLINTEGER 10303672
So we have a query of size > 1MB. Is there a known maximum size on
the
query length? And if so what is the correct way to get such a
value in,
if it is at all possible.
BTW: I am using libiodbc to perform the queries on Virtuoso
5.0.11.3039,
a snapshot of the upcoming 5.0.12.
Thanks a lot,
Sebastian Trueg
Mandriva
(Soprano Virtuoso integration development)
http://soprano.sourceforge.net
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year.
Jumpstart your
developing skills, take BlackBerry mobile applications to market
and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register
now!
http://p.sf.net/sfu/devconference
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
[Database]
DatabaseFile=/tmp/trueg/test/soprano-virtuoso.db
ErrorLogFile=/tmp/trueg/test/soprano-virtuoso.log
TransactionFile=/tmp/trueg/test/soprano-virtuoso.trx
xa_persistent_file=/tmp/trueg/test/soprano-virtuoso.pxa
[TempDatabase]
DatabaseFile=/tmp/trueg/test/soprano-virtuoso-temp.db
TransactionFile=/tmp/trueg/test/soprano-virtuoso-temp.trx
MaxCheckpointRemap=1000
[Parameters]
LiteMode=1
ServerPort=1111
DisableTcpSocket=1
PrefixResultNames=0
ServerThreads=5
CheckpointInterval=10
MaxDirtyBuffers=50
SchedulerInterval=5
FreeTextBatchSize=1000
** iODBC Trace file
** Trace started on Thu Oct 08 21:55:47 2009
** Driver Manager: 03.52.0607.1008
[000000.000917]
sopranocmd 7F14336D8700 ENTER SQLAllocHandle
SQLSMALLINT 1 (SQL_HANDLE_ENV)
SQLHANDLE 0x0 (SQL_NULL_HANDLE)
SQLHANDLE * 0x7fff3c152e40
[000000.000964]
sopranocmd 7F14336D8700 EXIT SQLAllocHandle with return code 0
(SQL_SUCCESS)
SQLSMALLINT 1 (SQL_HANDLE_ENV)
SQLHANDLE 0x0 (SQL_NULL_HANDLE)
SQLHANDLE * 0x7fff3c152e40 (0x121ea60)
[000000.001000]
sopranocmd 7F14336D8700 ENTER SQLSetEnvAttr
SQLHENV 0x121ea60
SQLINTEGER 200 (SQL_ATTR_ODBC_VERSION)
SQLPOINTER 0x3
SQLINTEGER * 4294967291 (SQL_IS_UINTEGER)
[000000.001038]
sopranocmd 7F14336D8700 EXIT SQLSetEnvAttr with return code 0
(SQL_SUCCESS)
SQLHENV 0x121ea60
SQLINTEGER 200 (SQL_ATTR_ODBC_VERSION)
SQLPOINTER 0x3
SQLINTEGER * 4294967291 (SQL_IS_UINTEGER)
[000003.355982]
sopranocmd 7F14336D8700 ENTER SQLAllocConnect
SQLHENV 0x121ea60
SQLHDBC * 0x7fff3c152e08
[000003.356041]
sopranocmd 7F14336D8700 EXIT SQLAllocConnect with return code
0 (SQL_SUCCESS)
SQLHENV 0x121ea60
SQLHDBC * 0x7fff3c152e08 (0x1212a40)
[000003.356071]
sopranocmd 7F14336D8700 ENTER SQLSetConnectOption
SQLHDBC 0x1212a40
SQLUSMALLINT 102 (SQL_AUTOCOMMIT)
SQLLEN 1
[000003.356102]
sopranocmd 7F14336D8700 EXIT SQLSetConnectOption with return
code 0 (SQL_SUCCESS)
SQLHDBC 0x1212a40
SQLUSMALLINT 102 (SQL_AUTOCOMMIT)
SQLLEN 1
[000003.356139]
sopranocmd 7F14336D8700 ENTER SQLDriverConnect
SQLHDBC 0x1212a40
SQLPOINTER 0x0
SQLCHAR * 0x121b2e8
| host=localhost:1111;uid=dba;pwd=***;driv |
| er=/usr/local/lib/virtodbc_r.so |
SQLSMALLINT -3 (SQL_NTS)
SQLCHAR * 0x7fff3c151df0
SQLSMALLINT 4096
SQLSMALLINT * 0x7fff3c152e06
SQLUSMALLINT 1 (SQL_DRIVER_COMPLETE)
[000003.361949]
sopranocmd 7F14336D8700 EXIT SQLDriverConnect with return code
0 (SQL_SUCCESS)
SQLHDBC 0x1212a40
SQLPOINTER 0x0
SQLCHAR * 0x121b2e8
SQLSMALLINT -3 (SQL_NTS)
SQLCHAR * 0x7fff3c151df0
| DRIVER=/usr/local/lib/virtodbc_r.so;SERV |
| ER=OpenLink;UID=dba;PWD=***;HOST=localho |
| st:1111 |
SQLSMALLINT 4096
SQLSMALLINT * 0x7fff3c152e06 (87)
SQLUSMALLINT 1 (SQL_DRIVER_COMPLETE)
[000003.477108]
sopranocmd 7F14336D8700 ENTER SQLAllocHandle
SQLSMALLINT 3 (SQL_HANDLE_STMT)
SQLHANDLE 0x1212a40
SQLHANDLE * 0x7fff3c152cf8
[000003.477217]
sopranocmd 7F14336D8700 EXIT SQLAllocHandle with return code 0
(SQL_SUCCESS)
SQLSMALLINT 3 (SQL_HANDLE_STMT)
SQLHANDLE 0x1212a40
SQLHANDLE * 0x7fff3c152cf8 (0x12c30a0)
[000003.480321]
sopranocmd 7F14336D8700 ENTER SQLExecDirect
SQLHSTMT 0x12c30a0
SQLCHAR * 0x7f142c5f1028
| sparql insert into graph <sopranofakes:/ |
| DEFAULTGRAPH> { <urn:hello:world> <urn:h |
| ello:super> "kdeinit4:Shuttingdownrunnin |
| gclient. kdeinit4:preparingtolaunch/home |
| /kde4dev/kde/dev/kde/inst/kde4/lib64/lib |
| kdeinit4_klauncher.so kdeinit4:LaunchedK |
| Launcher,pid=19033,result=0 kdeinit4:ope |
| nedconnectionto:0.0 kdeinit4:preparingto |
| launch/home/kde4dev/kde/dev/kde/inst/kde |
| 4/lib64/libkdeinit4_kded4.so kdeinit4:La |
| unchedKDED,pid=19034result=0 KDEDaemon(k |
| ded)alreadyrunning. kdeinit4:GotEXT_EXEC |
| /home/kde4dev/kde/dev/kde/inst/kde4/bin/ |
| kbuildsycoca4fromlauncher. kdeinit4:prep |
| aringtolaunch/home/kde4dev/kde/dev/kde/i |
| nst/kde4/lib/libkdeinit4_kbuildsycoca4.s |
| o kbuildsycoca4running... kbuildsycoca4( |
| 19036)kdemain:Reusingexistingksycoca kbu |
| ildsycoca4(19036)KBuildSycoca::recreate: |
| Recreatingksycocafile(/var/tmp/kdecache- |
| trueg/ksycoca4,version161) kbuildsycoca4 |
| (19036)KBuildSycoca::createEntry:new:scr |
| eensaver.desktop kbuildsycoca4(19036)VFo |
| lderMenu::mergeFile:VFolderMenu::mergeFi |
| le:/home/trueg/.config/menus/application |
| s-merged/wine-Programs-CatanGmbH-CatanOn |
| lineWelt-Catan-DasKartenspiel-Optionenve |
| rändern.menu kbuildsycoca4(19036)VFolde |
| rMenu::mergeFile:VFolderMenu::mergeFile: |
| /home/trueg/.config/menus/applications-m |
| erged/wine-Programs-CatanGmbH-CatanOnlin |
| eWelt-Catan-DasKartenspiel-Deckeditorsta |
| rten.menu kbuildsycoca4(19036)VFolderMen |
| u::mergeFile:VFolderMenu::mergeFile:/hom |
| e/trueg/.config/menus/applications-merge |
| d/wine-Programs-CatanGmbH-CatanOnlineWel |
| t-Uninstall.menu kbuildsycoca4(19036)VFo |
| lderMenu::mergeFile:VFolderMenu::mergeFi |
| le:/home/trueg/.config/menus/application |
| s-merged/wine-Programs-CatanGmbH-CatanOn |
| lineWelt-Catan-DasKartenspiel-Tutorialst |
| arten.menu kbuildsycoca4(19036)VFolderMe |
| nu::mergeFile:VFolderMenu::mergeFile:/ho |
| me/trueg/.config/menus/applications-merg |
| ed/wine-Programs-CatanGmbH-CatanOnlineWe |
| lt-CatanOnlineWelt.menu kbuildsycoca4(19 |
| 036)VFolderMenu::mergeFile:VFolderMenu:: |
| mergeFile:/etc/xdg/menus/applications-me |
| rged/ggz.merge.menu kbuildsycoca4(19036) |
| VFolderMenu::mergeFile:VFolderMenu::merg |
| eFile:/etc/xdg/menus/ggz.menu kbuildsyco |
| ca4(19036)foldNode:Directoryandggz.direc |
| toryrequirescombining! kbuildsycoca4(190 |
| 36)VFolderMenu::pushDocInfo:Menuapplicat |
| ions-kmenuedit.menunotfound. kbuildsycoc |
| a4(19036)VFolderMenu::processMenu:Proces |
| singKDELegacydirsfor<KDE> kbuildsycoca4( |
| 19036)VFolderMenu::processKDELegacyDirs: |
| kbuildsycoca4(19036)VFolderMenu::loadAp |
| plications:Lookingupapplicationsunder/ho |
| me/kde4dev/kde/dev/kde/inst/kde4/share/a |
| pplications/ kbuildsycoca4(19036)VFolder |
| Menu::loadApplications:Lookingupapplicat |
| ionsunder/home/kde4dev/kde/dev/kde/inst/ |
| kde4/share/applications/kde4 kbuildsycoc |
| a4(19036)VFolderMenu::loadApplications:L |
| ookingupapplicationsunder/usr/share/appl |
| ications/ kbuildsycoca4(19036)VFolderMen |
| u::loadApplications:Lookingupapplication |
| sunder/usr/share/applications/kde kbuild |
| sycoca4(19036)VFolderMenu::loadApplicati |
| ons:Lookingupapplicationsunder/usr/share |
| /applications/kde4 kbuildsycoca4(19036)/ |
| kdecore(services)KServicePrivate::init:T |
| hedesktopentryfile/usr/share/application |
| s/kde4/koffice.desktophasType=Applicatio |
| nbutnoExecline kbuildsycoca4(19036)KBuil |
| dServiceFactory::createEntry:InvalidServ |
| ice:/usr/share/applications/kde4/koffice |
| .desktop kbuildsycoca4(19036)VFolderMenu |
| ::loadApplications:Lookingupapplications |
| under/home/trueg/.local/share/applicatio |
| ns/ kbuildsycoca4(19036)VFolderMenu::loa |
| dApplications:Lookingupapplicationsunder |
| /home/trueg/.local/share/applications/wi |
| ne kbuildsycoca4(19036)VFolderMenu::load |
| Applications:Lookingupapplicationsunder/ |
| home/trueg/.local/share/applications/win |
| e/Programs kbuildsycoca4(19036)VFolderMe |
| nu::loadApplications:Lookingupapplicatio |
| nsunder/home/trueg/.local/share/applicat |
| ions/wine/Programs/DVD-labPRO2 kbuildsyc |
| oca4(19036)VFolderMenu::loadApplications |
| :Lookingupapplicationsunder/home/trueg/. |
| local/share/applications/wine/Programs/C |
| atanGmbH kbuildsycoca4(19036)VFolderMenu |
| ::loadApplications:Lookingupapplications |
| under/home/trueg/.local/share/applicatio |
| ns/wine/Programs/CatanGmbH/CatanOnlineWe |
| lt kbuildsycoca4(19036)VFolderMenu::load |
| Applications:Lookingupapplicationsunder/ |
| home/trueg/.local/share/applications/win |
| e/Programs/CatanGmbH/CatanOnlineWelt/Cat |
| an-DasKartenspiel kbuildsycoca4(19036)VF |
| olderMenu::loadApplications:Lookingupapp |
| licationsunder/home/trueg/.local/share/a |
| pplications/wine/Programs/TMPGEnc kbuild |
| sycoca4(19036)KBuildServiceFactory::popu |
| lateServiceTypes:/usr/share/applications |
| /comix.desktopspecifiesundefinedmimetype |
| /servicetypeimage/svg kbuildsycoca4(1903 |
| 6)KBuildServiceFactory::populateServiceT |
| ypes:/usr/share/applications/comix.deskt |
| opspecifiesundefinedmimetype/servicetype |
| image/svg-xml kbuildsycoca4(19036)KBuild |
| ServiceFactory::populateServiceTypes:/us |
| r/share/applications/comix.desktopspecif |
| iesundefinedmimetype/servicetypeimage/vn |
| d.adobe.svg+xml kbuildsycoca4(19036)KBui |
| ldServiceFactory::populateServiceTypes:/ |
| usr/share/applications/comix.desktopspec |
| ifiesundefinedmimetype/servicetypetext/x |
| ml-svg kbuildsycoca4(19036)KBuildService |
| Factory::populateServiceTypes:mailodypar |
| t.desktopspecifiesundefinedmimetype/serv |
| icetypetext/english kbuildsycoca4(19036) |
| KBuildServiceFactory::populateServiceTyp |
| es:mailodypart.desktopspecifiesundefined |
| mimetype/servicetypetext/x-c++ kbuildsyc |
| oca4(19036)KBuildServiceFactory::populat |
| eServiceTypes:/usr/share/applications/kd |
| e4/okularApplication_mobi.desktopspecifi |
| esundefinedmimetype/servicetypeapplicati |
| on/x-mobipocket-ebook kbuildsycoca4(1903 |
| 6)KBuildServiceFactory::populateServiceT |
| ypes:libokularGenerator_mobi.desktopspec |
| ifiesundefinedmimetype/servicetypeapplic |
| ation/x-mobipocket-ebook kbuildsycoca4(1 |
| 9036)KBuildServiceFactory::populateServi |
| ceTypes:okularMobi.desktopspecifiesundef |
| inedmimetype/servicetypeapplication/x-mo |
| bipocket-ebook kbuildsycoca4(19036)KBuil |
| dServiceFactory::populateServiceTypes:Se |
| rviceMenus/amarok_append.desktopspecifie |
| sundefinedmimetype/servicetypeaudio/* kb |
| uildsycoca4(19036)KBuildServiceFactory:: |
| populateServiceTypes:kleopatra_decryptve |
| rifyfiles.desktopspecifiesundefinedmimet |
| ype/servicetypeapplicaton/pkcs7-signatur |
| e kbuildsycoca4(19036)KBuildServiceFacto |
| ry::populateServiceTypes:/usr/share/appl |
| ications/gimp.desktopspecifiesundefinedm |
| imetype/servicetypeimage/pcx kbuildsycoc |
| a4(19036)KBuildServiceFactory::populateS |
| erviceTypes:/usr/share/applications/kde/ |
| knowit.desktopspecifiesundefinedmimetype |
| /servicetypeapplication/x-knowit kbuilds |
| ycoca4(19036)KBuildServiceFactory::popul |
| ateServiceTypes:ServiceMenus/ark_service |
| menu.desktopspecifiesundefinedmimetype/s |
| ervicetypeapplication/x-servicepack kbui |
| ldsycoca4(19036)KBuildServiceFactory::po |
| pulateServiceTypes:/home/kde4dev/kde/dev |
| /kde/inst/kde4/share/applications/kde4/k |
| 3b.desktopspecifiesundefinedmimetype/ser |
| vicetypeapplication/x-k3b kbuildsycoca4( |
| 19036)KBuildServiceFactory::populateServ |
| iceTypes:windowsexethumbnail.desktopspec |
| ifiesundefinedmimetype/servicetypeapplic |
| ation/x-msdownload kbuildsycoca4(19036)K |
| BuildServiceFactory::populateServiceType |
| s:ServiceMenus/nepomuk-peopletag.desktop |
| specifiesundefinedmimetype/servicetypeim |
| age/* kbuildsycoca4(19036)KBuildServiceF |
| actory::populateServiceTypes:ServiceMenu |
| s/k3b_create_audio_cd.desktopspecifiesun |
| definedmimetype/servicetypeaudio/* kbuil |
| dsycoca4(19036)KBuildServiceFactory::pop |
| ulateServiceTypes:/home/kde4dev/kde/dev/ |
| kde/inst/kde4/share/applications/kde4/kn |
| ob_part.desktopspecifiesundefinedmimetyp |
| e/servicetypetext/english kbuildsycoca4( |
| 19036)KBuildServiceFactory::populateServ |
| iceTypes:/home/kde4dev/kde/dev/kde/inst/ |
| kde4/share/applications/kde4/knob_part.d |
| esktopspecifiesundefinedmimetype/service |
| typetext/x-c++ kbuildsycoca4(19036)KBuil |
| dServiceFactory::populateServiceTypes:/u |
| sr/share/applications/openoffice.org-bas |
| e.desktopspecifiesundefinedmimetype/serv |
| icetypeapplication/vnd.sun.xml.base kbui |
| ldsycoca4(19036)KBuildServiceFactory::po |
| pulateServiceTypes:ServiceMenus/k3b_writ |
| e_iso_image.desktopspecifiesundefinedmim |
| etype/servicetypeapplication/x-iso kbuil |
| dsycoca4(19036)KBuildServiceFactory::pop |
| ulateServiceTypes:ServiceMenus/k3b_write |
| _iso_image.desktopspecifiesundefinedmime |
| type/servicetypeinode/ISO-image kbuildsy |
| coca4(19036)KBuildServiceFactory::popula |
| teServiceTypes:/usr/share/applications/o |
| penoffice.org-impress.desktopspecifiesun |
| definedmimetype/servicetypeapplication/v |
| nd.ms-powerpoint.presentation.macroenabl |
| ed.12 kbuildsycoca4(19036)KBuildServiceF |
| actory::populateServiceTypes:/usr/share/ |
| applications/openoffice.org-impress.desk |
| topspecifiesundefinedmimetype/servicetyp |
| eapplication/vnd.openxmlformats-officedo |
| cument.presentationml.template kbuildsyc |
| oca4(19036)KBuildServiceFactory::populat |
| eServiceTypes:/usr/share/applications/op |
| enoffice.org-impress.desktopspecifiesund |
| efinedmimetype/servicetypeapplication/vn |
| d.ms-powerpoint.template.macroenabled.12 |
| kbuildsycoca4(19036)KBuildServiceFactor |
| y::populateServiceTypes:/usr/share/appli |
| cations/kde4/ktorrent.desktopspecifiesun |
| definedmimetype/servicetypeapplication/x |
| -torrent kbuildsycoca4(19036)KBuildServi |
| ceFactory::populateServiceTypes:kerfuffl |
| e_libxz.desktopspecifiesundefinedmimetyp |
| e/servicetypeapplication/x-xz kbuildsyco |
| ca4(19036)KBuildServiceFactory::populate |
| ServiceTypes:phononbackends/xine.desktop |
| specifiesundefinedmimetype/servicetypevi |
| deo/x-quicktime kbuildsycoca4(19036)KBui |
| ldServiceFactory::populateServiceTypes:p |
| hononbackends/xine.desktopspecifiesundef |
| inedmimetype/servicetypevideo/mkv kbuild |
| sycoca4(19036)KBuildServiceFactory::popu |
| lateServiceTypes:phononbackends/xine.des |
| ktopspecifiesundefinedmimetype/servicety |
| pevideo/msvideo kbuildsycoca4(19036)KBui |
| ldServiceFactory::populateServiceTypes:p |
| hononbackends/xine.desktopspecifiesundef |
| inedmimetype/servicetypeaudio/aiff kbuil |
| dsycoca4(19036)KBuildServiceFactory::pop |
| ulateServiceTypes:phononbackends/xine.de |
| sktopspecifiesundefinedmimetype/servicet |
| ypeaudio/x-pn-aiff kbuildsycoca4(19036)K |
| BuildServiceFactory::populateServiceType |
| s:phononbackends/xine. |
SQLINTEGER 1123814
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart
your
developing skills, take BlackBerry mobile applications to market and
stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
