On Tue, Apr 23, 2013 at 09:50:16AM +0200, Christian Hoene wrote: > Hi, > > currently, the codec comparison tests are running. Because of the request of > many codec developers, we plan to extend those tests: We might add audio > tests in which the content is varied to a large extend. For that, we need > sample that cannot be compressed well by Opus or AAC-eLD. For me, it is easy > to get those difficult samples for Opus. It is much challenging to get those > for AAC-eLD. Thus, if somebody had to time to study the weaknesses of > AAC-eLD, please forward me the samples.
Uhm, so ... while I'm certain that the codec developers will be delighted if you can point out any new killer samples that they aren't yet aware of (since significant work has already been made to improve the encoder for the known ones, and that work is still ongoing) -- I'm also pretty certain that going out of your way to deliberately select such samples immediately disqualifies this from being characterised as a "comparison test", or at least claiming that it's even remotely representative of what people will observe over a general corpus of their own audio, given the degree to which such samples really are outliers. > I cannot start fair tests if I do not have challenging samples for both > codecs. While such a test might have some novelty value to show "here are some non-exhaustive results for the worst samples that we could find in a few days of searching", I'm pretty sure words like "fair" and "scientific rigour" don't really belong in the same sentence. Not in the least when you also say "we have the established list for one codec, but the known killers for the other is at present entirely unknown to us". If you want to spend your time doing that, that's fine, and the results may well be 'interesting'. But mischaracterising them as a "comparison" test would just be somewhere on the spectrum from "mildly amusing" to "a sad day for Modern Science". It's your reputation though, and I can't tell you how to spend it. But you might want to think this through a little better if you are going to paint this with the brush of Being Science. This isn't the cosmetics industry, other people can measure these things too, and will continue to for some time to come. Cheers, Ron _______________________________________________ codec mailing list [email protected] https://www.ietf.org/mailman/listinfo/codec
