Re: Direct3D Game test framework

2007-08-08 Thread Tom Wickline
Can you try this, set Wine to run in a virtual desktop of 1024x768 and set X at 16bpp and run 3DMark2000 in win98 mode. then cd into the install directory and see if that works. Tom On 8/8/07, martin pilka <[EMAIL PROTECTED]> wrote: > > Hello Martin > > > > If you cd into the 3DMark2000 install d

Re: Direct3D Game test framework

2007-08-08 Thread martin pilka
> Hello Martin > > If you cd into the 3DMark2000 install directory and run it from there > does it work? > > Tom Thanks Tom, that was it. I got a bit further, the message I got this time is: 3DMark could not initialize your 3D Accelerator. Try to downgrade your display settings. I played with

Re: Direct3D Game test framework

2007-08-07 Thread martin pilka
> is wined3d built? That was it - I was missing some dev libs. Thanks! I fixed that, now I am receiving this error: Not all of the E2 drivers were found. Check that you have 3DMark installed properly. Attached are the log files from installation and run. Martin 3dmark2000log.tar.gz Descriptio

Re: Direct3D Game test framework

2007-08-06 Thread Stefan Dösinger
Am Montag, 6. August 2007 16:49 schrieb martin pilka: > > Do you have HKCU/Software/Wine/Direct3D/DirectDrawRenderer set to "gdi" > > perhaps? This disables D3D support altogether. You could try with a > > fresh .wine, it should run 3dmark2000 and 2001 out of the box. > > I tried latest WineHq, it

Re: Direct3D Game test framework

2007-08-06 Thread martin pilka
> Do you have HKCU/Software/Wine/Direct3D/DirectDrawRenderer set to "gdi" > perhaps? This disables D3D support altogether. You could try with a > fresh .wine, it should run 3dmark2000 and 2001 out of the box. I tried latest WineHq, it created ~/.wine directory and installed 3DMark2000 correctly,

Re: Direct3D Game test framework

2007-08-02 Thread Stefan Dösinger
> To make this useful, we need to solve 'DirectX detection failed' bug, so we > can run real tests nightly. Could you send me ~/.wine directory you are > using with latest WineHq? Do you have HKCU/Software/Wine/Direct3D/DirectDrawRenderer set to "gdi" perhaps? This disables D3D support altogether.

Re: Direct3D Game test framework

2007-08-02 Thread martin pilka
Hello Stefan! > It is without CxTest and without VNC. I do have HW acceleration and OpenGL > (GLX) enabled. Attached is output of xdpyinfo. > > I am going to try whether 3DMark2001 works. 3DMark2001 behaves the same way. Note this is long lasting bug, I remember on WineConf 2006 I was able to r

Re: Direct3D Game test framework

2007-08-01 Thread martin pilka
Stefan Dösinger wrote: > Am Dienstag, 31. Juli 2007 16:42 schrieb martin pilka: >> Hello Stefan, >> >> link works, I was able to run your test. Installation is ok, however test >> itself ends with '3DMark Error': '3DMark needs DirectX 7 and proper drivers >> installed in order to run'. This happens

Re: Direct3D Game test framework

2007-07-31 Thread Stefan Dösinger
Am Dienstag, 31. Juli 2007 16:42 schrieb martin pilka: > Hello Stefan, > > link works, I was able to run your test. Installation is ok, however test > itself ends with '3DMark Error': '3DMark needs DirectX 7 and proper drivers > installed in order to run'. This happens without CxTest as well, with

Re: Direct3D Game test framework

2007-07-31 Thread martin pilka
Hello Stefan, link works, I was able to run your test. Installation is ok, however test itself ends with '3DMark Error': '3DMark needs DirectX 7 and proper drivers installed in order to run'. This happens without CxTest as well, with latest WineHq. Note that DirectX7 or newer is correctly detecte

Re: Direct3D Game test framework

2007-07-31 Thread Stefan Dösinger
Am Dienstag, 31. Juli 2007 12:28 schrieb martin pilka: > > With Francois' help I could build a test package for 3DMark2000 now: > > > > http://stud4.tuwien.ac.at/~e0526822/3dmark2000.tar.bz2 > > > > It runs two tests, one at 640x480, another one at 1024x768, writes the > > results to a file and rea

Re: Direct3D Game test framework

2007-07-31 Thread martin pilka
> With Francois' help I could build a test package for 3DMark2000 now: > > http://stud4.tuwien.ac.at/~e0526822/3dmark2000.tar.bz2 > > It runs two tests, one at 640x480, another one at 1024x768, writes the > results > to a file and reads the file back. No benchmark results are sent to the > ser

Re: Direct3D Game test framework

2007-07-24 Thread martin pilka
Stefan Dösinger wrote: > The roadmap looks good to me. I am missing some clarification how to deal > with > different hardware configurations. This is how CxTest supports it now: * There is 1:1 association between submitter and his configuration, i.e. "Stefan - Desktop" --> "AMD 1.8 GHz 1 GB

Re: Direct3D Game test framework

2007-07-24 Thread Stefan Dösinger
Am Dienstag, 24. Juli 2007 12:41 schrieb martin pilka: > Hello Stefan, > > following is specification proposal for first few iterations. It is more > focused on functionality than technical details. Feel free to comment. The roadmap looks good to me. I am missing some clarification how to deal with

Re: Direct3D Game test framework

2007-07-24 Thread H. Verbeet
On 24/07/07, martin pilka <[EMAIL PROTECTED]> wrote: Hello Stefan, following is specification proposal for first few iterations. It is more focused on functionality than technical details. Feel free to comment. 1st iteration = * We take your 3dMark test, adjust it a bit and make i

Re: Direct3D Game test framework

2007-07-24 Thread martin pilka
Hello Stefan, following is specification proposal for first few iterations. It is more focused on functionality than technical details. Feel free to comment. 1st iteration = * We take your 3dMark test, adjust it a bit and make it run and submit results regularly on one of our testin

Re: Direct3D Game test framework

2007-07-23 Thread Stefan Dösinger
Am Montag, 23. Juli 2007 15:16 schrieb martin pilka: > Hello Stefan! > > > If the pages are important I can go to Vienna and boot the server, but it > > would be a 6 hour travel just to press a power button. > > Not really important, I just wanted to have closer look what values you put > into your

Re: Direct3D Game test framework

2007-07-23 Thread martin pilka
Hello Stefan! > If the pages are important I can go to Vienna and boot the server, but it > would be a > 6 hour travel just to press a power button. Not really important, I just wanted to have closer look what values you put into your graphs, however I come with something and then we tweak it

Re: Direct3D Game test framework

2007-07-19 Thread Mirek Slugeň
I tried some Direct3D tests under Linux with CVS wine and under Windows XP. Configuration: Core 2 Duo [EMAIL PROTECTED], 2GB RAM, GF 6800 GS Options in wine registry: GLSL, FBO, VideoMemorySize 256, PS, VS, SoftwareEmulation Patches: Thread_prio patch, 3DMark hack results: wine/windows 3DMar

Re: Direct3D Game test framework

2007-07-19 Thread martin pilka
Stefan Dösinger wrote: > Yes, thats essentially what we need. I've been setting up a quick and > dirty test site myself, with a few lines of php and gnuplot: > > http://84.112.174.163/~stefan/laptop/3dmark2000/results.php Stefan, could you put the link life again? Or send me screenshots of it. I a

Re: Direct3D Game test framework

2007-07-12 Thread Paul Millar
On Thursday 12 July 2007 16:19:05 Detlef Riekenberg wrote: > You can see actual results for wine here: > http://www.astro.gla.ac.uk/users/paulm/WRT/wrt.php Sadly, WRT is very much in the past-tense. It was based on a Heath Robinson arrangement involving CVS email notifications bouncing around ou

Re: Direct3D Game test framework

2007-07-12 Thread Detlef Riekenberg
On Do, 2007-07-05 at 15:16 +0200, martin pilka wrote: > Detlef Riekenberg wrote: > > > > We can also reuse an existing Framework: > > winetest.exe and http://test.winehq.org/data > http://test.winehq.org/data/200707031000/ > > I am wondering, why is the Wine column always empty? Nobody send th

Re: Direct3D Game test framework

2007-07-11 Thread Dan Kegel
Martin wrote: YAWT looks interesting Thanks! If you need any changes in it, ask Lei (or ask me to ask Lei). I also had a look to AutoHotKey, and its language seems to solve some of the problems we have with CxTest scripts (double click, better support for menus, better conditioning). It is

Re: Direct3D Game test framework

2007-07-10 Thread martin pilka
Hello Dan! > I'm sure there will always be some tests that are > better written directly in CxTest, but I'm going to > try to push yawt forward, too. With > luck we'll have the two test systems working > well together in a month or two. YAWT looks interesting, I also had a look to AutoHotKey, a

re: Direct3D Game test framework

2007-07-10 Thread Dan Kegel
Michal Okresa wrote: We (people around CxTest) think that extending CxTest is better approach to achieve your goals rather than writing new application or modifying existing. It could be done with smaller effort (in opposite of AutoHotkey, AutoIt, ...) because CxTest is specialised for testing W

Re: Direct3D Game test framework

2007-07-10 Thread Michal Okresa
Stefan Dösinger wrote: We shall discuss the requirements in more exact manner. 1. Define demo packages I think for a start we don't need the ability for users to add new demo packages. A fixed set of test packages which is modifyable by raw database access would be enough. Otherwi

re: Direct3D Game test framework

2007-07-05 Thread Dan Kegel
Martin wrote: on YAWT homepage, I read: "YAWT can be used either standalone, or as part of the cxtest.org automated regression test suite for Wine" You mean, it can generate result emails which can be parsed by CxTest server? I mean that it should be trivial to wrap a Yawt test into a cxtest s

Re: Direct3D Game test framework

2007-07-05 Thread Stefan Dösinger
> * Ability to regularly benchmark certain parts of game (and thus Wine, > which is involved) in clean environment and submit raw data results to > server > > * On the server, process submitted data and present them via web in user > friendly form; support for pointing to regressions (and finding r

Re: Direct3D Game test framework

2007-07-05 Thread martin pilka
Dan, on YAWT homepage, I read: "YAWT can be used either standalone, or as part of the cxtest.org automated regression test suite for Wine" You mean, it can generate result emails which can be parsed by CxTest server? Or how does it work? Thanks, Martin Dan Kegel wrote: > Vit Hrachovy wrote:

Re: Direct3D Game test framework

2007-07-05 Thread martin pilka
Francois Gouget wrote: > You could also write some software to automatically analyze this data on > the server but that seems overkill. Yes, we tried this while ago and it was really overkill. I believe results should be analyzed by man, but he needs user friendly formatted information for this.

Re: Direct3D Game test framework

2007-07-05 Thread martin pilka
Detlef Riekenberg wrote: > On Di, 2007-06-26 at 13:06 +0200, Stefan Dösinger wrote: > >> Appart of controlling the apps we'll need some programs to start the >> application scripts, extract the results and do something with them. > >> We also need some database to collect the results. > > We c

Re: Direct3D Game test framework

2007-07-05 Thread martin pilka
>> Running applications with different log levels, tracing output, >> comparing screenshots with different levels of fuzzy comparaion >> - it's not faraway. > How would fuzzy comparison work? See projects findimagedupes and gqview (duplicate window, hit 'D' in gqview interface), both presented as

Re: Direct3D Game test framework

2007-07-05 Thread martin pilka
Stefan Dösinger wrote: > --no-vnc works fine for me for playing tests, but it does not work for > recording tests. VNC was never hard requirement of CxTest, we are using it to guarantee same testing environment everywhere. However, it is been a while I ran it without VNC, let me check. Martin

Re: Direct3D Game test framework

2007-07-05 Thread martin pilka
Hello Stefan! > We don't have to transfer all the rendering screenshots over VNC though. > I am CCing Martin Pilka on this, maybe he can give us some more insight. I apologize for late answer - I was (and still am) ill. However, now I read whole thread, let me summarize the goals you would expec

Re: Direct3D Game test framework

2007-06-28 Thread Stefan Dösinger
Am Donnerstag, 28. Juni 2007 18:05 schrieb Stefan Dösinger: > So whats left? > *) Make the Email system send custom test return data, not just success / > failure I found a simple solution for that: I just dump the results onto the log channel, which is sent to the server which can then grep the n

Re: Direct3D Game test framework

2007-06-28 Thread Stefan Dösinger
Hi, With Francois' help I could build a test package for 3DMark2000 now: http://stud4.tuwien.ac.at/~e0526822/3dmark2000.tar.bz2 It runs two tests, one at 640x480, another one at 1024x768, writes the results to a file and reads the file back. No benchmark results are sent to the server yet(just

Re: Direct3D Game test framework

2007-06-28 Thread Stefan Dösinger
> > The problem with vnc is recording a cxtest test(ie button clicks, > > keystrokes). That doesn't have to be fast, just beeing able to init glx > > is ok. But I agree on the non-trivial point. So far I've used manually > > coded keystrokes instead of button clicks, and it works, although it a > >

Re: Direct3D Game test framework

2007-06-28 Thread H. Verbeet
On 28/06/07, Stefan Dösinger <[EMAIL PROTECTED]> wrote: Am Donnerstag, 28. Juni 2007 09:09 schrieb H. Verbeet: > On 28/06/07, Steven Edwards <[EMAIL PROTECTED]> wrote: > > Could we add an extension to vnc to use Mesa? > > I suspect it would be non-trivial, but more importantly I think it > would

Re: Direct3D Game test framework

2007-06-28 Thread Stefan Dösinger
Am Donnerstag, 28. Juni 2007 09:09 schrieb H. Verbeet: > On 28/06/07, Steven Edwards <[EMAIL PROTECTED]> wrote: > > Could we add an extension to vnc to use Mesa? > > I suspect it would be non-trivial, but more importantly I think it > would defeat the purpose of measuring performance on different >

Re: Direct3D Game test framework

2007-06-28 Thread H. Verbeet
On 28/06/07, Steven Edwards <[EMAIL PROTECTED]> wrote: Could we add an extension to vnc to use Mesa? I suspect it would be non-trivial, but more importantly I think it would defeat the purpose of measuring performance on different hardware configurations.

Re: Direct3D Game test framework

2007-06-27 Thread Steven Edwards
On 6/27/07, Stefan Dösinger <[EMAIL PROTECTED]> wrote: --no-vnc works fine for me for playing tests, but it does not work for recording tests. Thats pretty bad because the vnc server does not even have opengl, not even a software renderer. Because of that the benchmarks refuse Could we add an e

Re: Direct3D Game test framework

2007-06-27 Thread Stefan Dösinger
> cxtest's run.sh script has a --no-vnc option. I know I have once run a > test outside of VNC so VNC is not a hard requirement. cxtest will > probably complain about it though, and you might lose some functionality > like automatic screenshots on failures. The best would be to try it out. --no-vnc

Re: Direct3D Game test framework

2007-06-27 Thread Francois Gouget
On Mon, 25 Jun 2007, Stefan Dösinger wrote: [...] > > One issue I see is actually interpreting the results. When is a > > performance drop large enough to be a problem? Sometimes a change will > > slightly reduce performance for some applications, but significantly > > improve it for others. > That

Re: Direct3D Game test framework

2007-06-27 Thread Francois Gouget
On Mon, 25 Jun 2007, Stefan Dösinger wrote: > > cxtest has already impressive infratructure (installation, creation of > > clean environment) and nice scripting. However, it requires Metacity WM > > and VNC. > The Metacity WM doesn't really concern me, the VNC thing however does. This > makes gam

Re: Direct3D Game test framework

2007-06-26 Thread Stefan Dösinger
> We shall discuss the requirements in more exact manner. > > 1. Define demo packages I think for a start we don't need the ability for users to add new demo packages. A fixed set of test packages which is modifyable by raw database access would be enough. Otherwise we end up with everyone

Re: Direct3D Game test framework

2007-06-26 Thread Detlef Riekenberg
On Di, 2007-06-26 at 13:06 +0200, Stefan Dösinger wrote: > Appart of controlling the apps we'll need some programs to start the > application scripts, extract the results and do something with them. > We also need some database to collect the results. We can also reuse an existing Framework: w

Re: Direct3D Game test framework

2007-06-26 Thread Vit Hrachovy
On Tue, Jun 26, 2007 at 01:06:02PM +0200, Stefan Dösinger wrote: > > I second Vit's recommendation. We're using Autohotkey > > scripts in our test framework > > http://code.google.com/p/yawt/ > > and I've been meaning to work with Vit to get his scripts > > incorporated into our framework. > App

Re: Direct3D Game test framework

2007-06-26 Thread Stefan Dösinger
> I second Vit's recommendation. We're using Autohotkey > scripts in our test framework > http://code.google.com/p/yawt/ > and I've been meaning to work with Vit to get his scripts > incorporated into our framework. (We've been distracted > getting the next release of Picasa out, sorry.) I had

re: Direct3D Game test framework

2007-06-25 Thread Dan Kegel
Vit Hrachovy wrote: Stefan Dösinger wrote: > [Any suggestions for a good app test framework?] I'm personally using AutoHotkey(http://www.autohotkey.com) as an automatic testing framework backend. It's a Windows application, so it can be run through Wine itself and it's sandboxed from X11. I se

Re: Direct3D Game test framework

2007-06-25 Thread H. Verbeet
On 25/06/07, Stefan Dösinger <[EMAIL PROTECTED]> wrote: > Afaik 3DMark can also generate performance graphs. I prefer to have the raw data myself, to do my own processing, so I'm not sure if that helps us. But graphcs are just a nice to have, we don't need them. I think it has the ability to ex

Re: Direct3D Game test framework

2007-06-25 Thread Stefan Dösinger
> cxtest has already impressive infratructure (installation, creation of > clean environment) and nice scripting. However, it requires Metacity WM > and VNC. The Metacity WM doesn't really concern me, the VNC thing however does. This makes game testing harder since VNC isn't a part of the average

Re: Direct3D Game test framework

2007-06-25 Thread Stefan Dösinger
> Afaik 3DMark can also generate performance graphs. I prefer to have the raw data myself, to do my own processing, so I'm not sure if that helps us. But graphcs are just a nice to have, we don't need them. > We could always try writing something ourselves, I guess :-) > The problem there is goin

Re: Direct3D Game test framework

2007-06-25 Thread Vit Hrachovy
On Mon, Jun 25, 2007 at 03:40:42PM +0200, Stefan Dösinger wrote: > Hi, > Now that we're facing Direct3D 9 feature completion soon - VTF support coming > in from Henri, High order patches and thread safety from me - the d3d work > will change a bit soon. Instead of adding new features we'll debug

Re: Direct3D Game test framework

2007-06-25 Thread H. Verbeet
On 25/06/07, Stefan Dösinger <[EMAIL PROTECTED]> wrote: 3DMark: Popular benchmark for testing the overall performance, doesn't give very detailed results though. Controlling: Standard windows controls, keyboard events should do the job Result readback: Can write results to a file, but must take c