Dear Nicola,
a simple, trivial, non qualitative way to compare output beetween
platforms, could be a sample by sample subtraction. The resultant signal
should give the deviation error (in the case of identity, this should
give a sequence of zeroes). One can be the comparation both in float
samples and in integer samples. 
Any other idea?

Gab

Nicola Bernardini wrote:
> 
> What about the quality of the output? What I mean by that is:
> 
> 1) is there a reference output (perhaps done with some other precision
>    tool that is trustable at least more than csound, or more than one)
>    so that deviation from the output can be measured?
> 
> 2) these measurements would probably differ from platform to platform...
> 
> 3) there could be numerical bugs in the algorithms which could be
>    uncovered by the reference implementation of the algorithm in some
>    other language (say Matlab/Octave, or Mathematica, or whatever you like)
> 
> Sinewaves oscillators can be easy to test, but when you get to some
> nifty filters or reverbs, then it gets harder, does'nt it?
> 
> Have you already discussed this aspect?
> 
> ciao
> 
> Nicola
> 
> ------------------------------------------------------------------------
> Nicola Bernardini
> E-mail: nicb@axnet.it
> 
> Re graphics: A picture is worth 10K words -- but only those to describe
> the picture.  Hardly any sets of 10K words can be adequately described
> with pictures.

-- 
Gabriel Maldonado

http://web.tiscalinet.it/G-Maldonado


--

Gabriel Maldonado wrote:
  a simple, trivial, non qualitative way to compare output beetween
  platforms, could be a sample by sample subtraction. The resultant signal
  should give the deviation error (in the case of identity, this should
  give a sequence of zeroes). One can be the comparation both in float
  samples and in integer samples. 

Unfortunately, this sometimes doesn't work so well due to subtle
differences in arithmetic between platforms.  For example, Intel
processors store intermediate floating-point results in an 80-bit
format even when the results are 64-bit or 32-bit.  Depending upon
how a compiler decides to store intermediate results, the outcome
can vary in the last bit.  And in any case, the result can vary
slightly from that produced on a PPC or Alpha.  Sometimes the
differences are entirely below rounding error, and 16-bit or even
float samples will be identical.  In other cases (e.g. physical
modeling or high-index FM) error accumulation can make results
diverge over time and even fall out of phase with each other on
very long notes.

Sample comparison is worth a try, however.  I'd suggest using the
standard deviation or variance of the sample differencess as a metric.
Just be prepared for those cases when it doesn't work even though
nothing is broken...

		-Ed


--

Right. What I was hinting at is: how do you produce the so-called
'correct' samples? with mathlab/octave, or what? Do these exist
somewhere as references?

ciao

Nicola

Yesterday, Gabriel Maldonado mi scrisse cio` che segue:

> Dear Nicola,
> a simple, trivial, non qualitative way to compare output beetween
> platforms, could be a sample by sample subtraction. The resultant signal
> should give the deviation error (in the case of identity, this should
> give a sequence of zeroes). One can be the comparation both in float
> samples and in integer samples. 
> Any other idea?

[snip]

Yesterday, Ed Hall mi scrisse cio` che segue:

[snip]
> Unfortunately, this sometimes doesn't work so well due to subtle
> differences in arithmetic between platforms.  For example, Intel
> processors store intermediate floating-point results in an 80-bit
> format even when the results are 64-bit or 32-bit.  Depending upon
[etc. - snip]

------------------------------------------------------------------------
Nicola Bernardini
E-mail: nicb@axnet.it
 
Re graphics: A picture is worth 10K words -- but only those to describe
the picture.  Hardly any sets of 10K words can be adequately described
with pictures.

--