Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
On Wed, Sep 22, 2010 at 4:01 AM, Sheet Spotter wrote: > I would love to see Robin's test harness. I would also love to see the > images that failed with v2. For the record I cannot give you (or anyone else) the images that failed, I was testing against a random selection of images pulled from the texture cache (and yes i know how to get a complete image out the cache) i wanted it to be a fair cross section of real SL textures not contrived test cases. I can probably people on an individual basis UUIDs of failed images but i'm afraid you will have to obtain the images yourselves I can't just go around distributing SL textures that are not mine. Maybe we can make a collection of test textures that we (the collective we) own, if an individual copyright holder has textures they can donate we can build a collection of test textures that can be used for metrics now and in the future. > > I started diving into the v2 branch of OpenJPEG a few days ago, after > research JPEG 2000 for a week. Tackling the bugs from the failed images > might be a good way to become more familiar with the source code. I can get full back traces to the offending lines which will be some help. > > I posted a very minor patch correcting some simple build problems with the > DLL version of OpenJPEG V2. > http://code.google.com/p/openjpeg/issues/detail?id=40 Yes i had to fix a few issues to get the dll to build, nothing major. > Thank you for publishing your results, Robin. They are encouraging me to > continue looking at OpenJPEG v2. Thanks OJP V2 looks like its worth the effort, so far using it with the viewer is failing badly. Encode now i've stopped it crashing is failing causing a "bake loop of hell" which consumes massive amounts of CPU time and spams the LL servers with bakes, so bad all around. As for the test harness, Yes let me tidy it slightly and I'll publish it, its really simple but i'm lazily working around one issue with error reporting currently. I also hopefully will have some new data tonight myself, looks like my results are artificially slow, BUT there remains a massive difference between KDU and openjpg, possibly worse than the x4 data i obtained, maybe in the order of x16-x20 slower but i need to rerun all the tests. This could also explain why SSE was showing odd results. Robin ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
How many textures do you need? I bet I have a bunch that are perfectly OK for this purpose as well as for commercial use. And if you need more I can generate them using filters. I.e.; this is not a block. From: Robin Cornelius To: Sheet Spotter Cc: opensource-dev Sent: Wed, September 22, 2010 3:55:20 AM Subject: Re: [opensource-dev] Openjpeg/KDU the cold hard metrics On Wed, Sep 22, 2010 at 4:01 AM, Sheet Spotter wrote: > I would love to see Robin's test harness. I would also love to see the > images that failed with v2. For the record I cannot give you (or anyone else) the images that failed, I was testing against a random selection of images pulled from the texture cache (and yes i know how to get a complete image out the cache) i wanted it to be a fair cross section of real SL textures not contrived test cases. I can probably people on an individual basis UUIDs of failed images but i'm afraid you will have to obtain the images yourselves I can't just go around distributing SL textures that are not mine. Maybe we can make a collection of test textures that we (the collective we) own, if an individual copyright holder has textures they can donate we can build a collection of test textures that can be used for metrics now and in the future. > > I started diving into the v2 branch of OpenJPEG a few days ago, after > research JPEG 2000 for a week. Tackling the bugs from the failed images > might be a good way to become more familiar with the source code. I can get full back traces to the offending lines which will be some help. > > I posted a very minor patch correcting some simple build problems with the > DLL version of OpenJPEG V2. >http://code.google.com/p/openjpeg/issues/detail?id=40 Yes i had to fix a few issues to get the dll to build, nothing major. > Thank you for publishing your results, Robin. They are encouraging me to > continue looking at OpenJPEG v2. Thanks OJP V2 looks like its worth the effort, so far using it with the viewer is failing badly. Encode now i've stopped it crashing is failing causing a "bake loop of hell" which consumes massive amounts of CPU time and spams the LL servers with bakes, so bad all around. As for the test harness, Yes let me tidy it slightly and I'll publish it, its really simple but i'm lazily working around one issue with error reporting currently. I also hopefully will have some new data tonight myself, looks like my results are artificially slow, BUT there remains a massive difference between KDU and openjpg, possibly worse than the x4 data i obtained, maybe in the order of x16-x20 slower but i need to rerun all the tests. This could also explain why SSE was showing odd results. Robin ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
On Wed, Sep 22, 2010 at 9:29 AM, Ann Otoole wrote: > How many textures do you need? I bet I have a bunch that are perfectly OK > for this purpose as well as for commercial use. And if you need more I can > generate them using filters. > > I.e.; this is not a block. > Thanks, I was running just under 300 for my tests, but we could do with a mix of RGB and RGBA textures + some that have lossless encoding and also some encoded on KDU and some on Openjpeg as there are differences. So any you want to donate to the cause would be most welcome, i can easily add a text file with a list of textures and copyright by who etc so its clear who actually owns them. Also i can encode if necessary but esp with the KDU ones i need to be 100% i've made them the same as the viewer does. We ideally want to get more that 300 and get a good test bed of say 1000 in total and then that will be very useful with a common set of textures for everyone to compare results with as many things in common as possible, this will then show differences on optimisations between different CPUs, operating systems etc and make it very easy for changes to openjpeg or any other decoder to be compared on mass. Regards Robin ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
On 22/09/2010 5:55 PM, Robin Cornelius wrote: > On Wed, Sep 22, 2010 at 4:01 AM, Sheet Spotter > wrote: >> I would love to see Robin's test harness. I would also love to see the >> images that failed with v2. > For the record I cannot give you (or anyone else) the images that > failed, I was testing against a random selection of images pulled from > the texture cache (and yes i know how to get a complete image out the > cache) It's not hard to do. I've got this little HTTP texture-proxy thing that queries the local computers here to share local viewer caches and save on bandwidth. And yes, the contents aren't the sort of thing you want to pass around willy-nilly. -- Tateru Nino http://dwellonit.taterunino.net/ ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
[opensource-dev] Larger UI Question - Skinning Progress
Just some thoughts about Skinning - love the Furry bottombar btw Miss. :-) Currently I skin for Viewer 2 - see http://wiki.secondlife.com/wiki/Viewer_Skins/Starlight, and also for the Viewer 2-based TPV Kirstens Viewer (as do Ginger M and Niran). I have carried out a number of xml changes and incorporated many made by other residents in StarLight. First, a reality check - you can change only so much, even if you are good at deciphering and twisting the xml. Remember that all of the functionality is carried out in the code which is in the world of TPV changes. StarLight employs little UI changes (or big in the case of the profile panels) and the use of 'functions/controls' provided to the interface, which are normally buried in the debug settings. What you can do is give the interface a slightly different look and feel, but the more different you want it, other than new background files or colours, the harder it gets - but sometimes some clever colouring or textures can make a big difference to user impression. You can also make all elements aero (i.e. see-through) if you want - with the exception of the menu bar, which even KirstenLee hasn't been able to sort out. Recently I have changed a number of the xml files to make a version that can be used with both light (i.e. dark on light) and the normal (i.e. light on dark) skins by just changing the textures and colors.xml file, and expect to launch a first 'light' skin soon as this is a big demand. An example of a pink light on dark skin I am working on is at https://wiki.secondlife.com/w/images/0/02/StarLight_Piggy_Pink.jpg - and don't worry, I am making more plain versions. My own 'Nostalgia Blue' version incorporating a touch of Viewer 1 is at https://wiki.secondlife.com/w/images/3/3e/Nostalgia_Blue.png - out as an option in the next StarLight version. If anyone has any questions about what can or can't be done (or at least my understanding FWIW) or wants help I am always willing to help, whether you want to use StarLight tweaks, add a skin over StarLight, or just colour the interface. ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Larger UI Question - Skinning Progress
I love your Nostalgia one, great job. From: Hitomi Tiponi To: Opensource_dev Sent: Wed, September 22, 2010 7:30:26 AM Subject: [opensource-dev] Larger UI Question - Skinning Progress Just some thoughts about Skinning - love the Furry bottombar btw Miss. :-) Currently I skin for Viewer 2 - see http://wiki.secondlife.com/wiki/Viewer_Skins/Starlight, and also for the Viewer 2-based TPV Kirstens Viewer (as do Ginger M and Niran). I have carried out a number of xml changes and incorporated many made by other residents in StarLight. First, a reality check - you can change only so much, even if you are good at deciphering and twisting the xml. Remember that all of the functionality is carried out in the code which is in the world of TPV changes. StarLight employs little UI changes (or big in the case of the profile panels) and the use of 'functions/controls' provided to the interface, which are normally buried in the debug settings. What you can do is give the interface a slightly different look and feel, but the more different you want it, other than new background files or colours, the harder it gets - but sometimes some clever colouring or textures can make a big difference to user impression. You can also make all elements aero (i.e. see-through) if you want - with the exception of the menu bar, which even KirstenLee hasn't been able to sort out. Recently I have changed a number of the xml files to make a version that can be used with both light (i.e. dark on light) and the normal (i.e. light on dark) skins by just changing the textures and colors.xml file, and expect to launch a first 'light' skin soon as this is a big demand. An example of a pink light on dark skin I am working on is at https://wiki.secondlife.com/w/images/0/02/StarLight_Piggy_Pink.jpg - and don't worry, I am making more plain versions. My own 'Nostalgia Blue' version incorporating a touch of Viewer 1 is at https://wiki.secondlife.com/w/images/3/3e/Nostalgia_Blue.png - out as an option in the next StarLight version. If anyone has any questions about what can or can't be done (or at least my understanding FWIW) or wants help I am always willing to help, whether you want to use StarLight tweaks, add a skin over StarLight, or just colour the interface. ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
Doing this out of a local cache would make it clearer what the time for the library is. I suspect these numbers understate the kdu advantage. On Wed, Sep 22, 2010 at 7:30 AM, Tateru Nino wrote: > > > On 22/09/2010 5:55 PM, Robin Cornelius wrote: >> On Wed, Sep 22, 2010 at 4:01 AM, Sheet Spotter >> wrote: >>> I would love to see Robin's test harness. I would also love to see the >>> images that failed with v2. >> For the record I cannot give you (or anyone else) the images that >> failed, I was testing against a random selection of images pulled from >> the texture cache (and yes i know how to get a complete image out the >> cache) > It's not hard to do. I've got this little HTTP texture-proxy thing that > queries the local computers here to share local viewer caches and save > on bandwidth. And yes, the contents aren't the sort of thing you want to > pass around willy-nilly. > > -- > Tateru Nino > http://dwellonit.taterunino.net/ > > ___ > Policies and (un)subscribe information available here: > http://wiki.secondlife.com/wiki/OpenSource-Dev > Please read the policies before posting to keep unmoderated posting privileges > ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
Is it possible for you to release your test harness for these comparisons, including the texture set? I noticed there isn't data for linux or mac platforms, which I am curious about. My eyeball tells me KDU is more optimized for windows than other platforms, but I'd like to test this empirically, as well as see how well my hardware compares to these results using the same test. ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
On Wed, Sep 22, 2010 at 7:35 PM, Arrehn Oberlander wrote: > Is it possible for you to release your test harness for these comparisons, > including the texture set? I can't give out the texture set, hopefully as per a previous discussion on this topic, we can assemble a good texture set that we can all share and compare metrics. the code can be found at http://bitbucket.org/robincornelius/viewer-development-j2c-metrics The code although based on 2.X should still work on 1.X (my original tests were a 1.X build) but i wanted to clean this up a bit before release and to be honest we are concerned with openjpeg/kdu not the viewer for these tests, my test code itsself is painfully simple. Load a texture into an LLImage, run decode time it. repeat and rince. The code forms a seperate project "j2k_metric" which depends on llcommon/llimage/llimagej2coj and llvfs (cmake rules included to make it build out of the box. But to make it run you need to set the working directory to be the sharedlibs folder (or copy dlls/so's around) as i've not bothered to stage it yet (patches welcome) The exe accepts one command line argument and that is the path to a folder of textures, if no path is given it assumes a default path of "C:\Textures" to make things a tiny bit easier on windows. Hope this is enough to get people started Robin ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
In theory, KDU 6.4 is supposed to be platform neutral - or at least that's one of the claims the mfr. is making about the new version. On Sep 22, 2010, at 11:35 AM, Arrehn Oberlander wrote: > Is it possible for you to release your test harness for these comparisons, > including the texture set? > > I noticed there isn't data for linux or mac platforms, which I am curious > about. My eyeball tells me KDU is more optimized for windows than other > platforms, but I'd like to test this empirically, as well as see how well my > hardware compares to these results using the same test. > ___ > Policies and (un)subscribe information available here: > http://wiki.secondlife.com/wiki/OpenSource-Dev > Please read the policies before posting to keep unmoderated posting privileges ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
On 09/22/2010 02:12 PM, Lillian Yiyuan wrote: > Doing this out of a local cache would make it clearer what the time > for the library is. I suspect these numbers understate the kdu > advantage. > I'm pretty sure Robin's test harness is very minimal, if you are thinking that there's a bloated client attached to it, muddying the numbers. ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Looking for a PJIRA entry about build grid UI bug with deferred rendering enabled
On Mon, 20 Sep 2010 10:39:51 +0100, Tofu Linden wrote: > Opensource Obscure wrote: >> When Deferred Rendering is enabled and you're in >> building mode, the RGB arrows that you use to move >> prims don't "stop" at the prim surface anymore. >> >> Deferred Rendering OFF: >> http://m.friendfeed-media.com/948b6fc14d7ec63ec01a1f8f253a48869c6cbcd0 >> >> Deferred Rendering ON: >> http://m.friendfeed-media.com/206e6a7e4a14518e27f084d30c9cec0d58873520 >> >> If I recall correctly, a PJIRA entry already exists >> for this bug - but unfortunately I can't find it. >> >> Can you help me to find it please? I'd like to update it. >> >> Opensource Obscure > > I don't think I've run across a pjira for this, but I'd be interested > to hear. If it takes too long to find an existing one, please do file > a new one, IMHO it's better to have a few dupes than fail to have a bug > filed at all. :) https://jira.secondlife.com/browse/VWR-19419 - good news is that this now seems to be fixed in the latest Development viewer releases. Opensource Obscure ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
There may be another option for obtaining a common set of J2C files for comparison. Viewer 2 installs 784 JPEG-2000 files (*.j2c) in the "local_assets" folder under the main install folder (e.g., "C:\Program Files\SecondLifeViewer2"). The "j2k_metric" test harness decoded all 784 files from the "local_assets" folder without errors. The OpenJPEG 1.3.0 library was 3.7 times slower than the KDU library that comes with Viewer 2.1.1.208043 (the current release version). I have been unsuccessful at implementing the additional functions to make the OpenJPEG v2 interface compatible with the earlier v1.3.0 interface. The test application aborted with buffer overrun errors. Oopsie! The tests were run under Windoze 7 on a Pentium i7-950 (3 GHz Hyperthreaded) using a RelWithDebInfo build run from the command line. (Running the same build from within VisualStudio 2005 was noticeably slower. There was one small surprise. Only two threads were in use for both the OpenJPEG v1.3.0 and KDU tests. I expected KDU to use all eight available threads. Sheet Spotter -Original Message- From: opensource-dev-boun...@lists.secondlife.com [mailto:opensource-dev-boun...@lists.secondlife.com] On Behalf Of Robin Cornelius Sent: September 22, 2010 2:00 PM To: Arrehn Oberlander Cc: opensource-dev Subject: Re: [opensource-dev] Openjpeg/KDU the cold hard metrics On Wed, Sep 22, 2010 at 7:35 PM, Arrehn Oberlander wrote: > Is it possible for you to release your test harness for these comparisons, > including the texture set? I can't give out the texture set, hopefully as per a previous discussion on this topic, we can assemble a good texture set that we can all share and compare metrics. the code can be found at http://bitbucket.org/robincornelius/viewer-development-j2c-metrics The code although based on 2.X should still work on 1.X (my original tests were a 1.X build) but i wanted to clean this up a bit before release and to be honest we are concerned with openjpeg/kdu not the viewer for these tests, my test code itsself is painfully simple. Load a texture into an LLImage, run decode time it. repeat and rince. The code forms a seperate project "j2k_metric" which depends on llcommon/llimage/llimagej2coj and llvfs (cmake rules included to make it build out of the box. But to make it run you need to set the working directory to be the sharedlibs folder (or copy dlls/so's around) as i've not bothered to stage it yet (patches welcome) The exe accepts one command line argument and that is the path to a folder of textures, if no path is given it assumes a default path of "C:\Textures" to make things a tiny bit easier on windows. Hope this is enough to get people started Robin ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
Re: [opensource-dev] Openjpeg/KDU the cold hard metrics
> There was one small surprise. Only two threads were in use for both the > OpenJPEG v1.3.0 and KDU tests. I expected KDU to use all eight available > threads. > Remember that LL is still shipping a downright ancient KDU 4.* based llkdu.dll with the viewer. If they successfully update to 6.4.* or later, you'll probably see all cores used. ___ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/OpenSource-Dev Please read the policies before posting to keep unmoderated posting privileges
