Most people doh't need this facility, and won't run into its sharp edges.
Those who do need it, need it bad, and will work with the problems. So I
think the size should be increased to the maximum possible on the machine.

Original Message:
-----------------
From: Richard Dobson richarddobson@blueyonder.co.uk
Date: Wed, 15 Sep 2004 21:18:37 +0100
To: csound-dev@eartha.mills.edu
Subject: [CSOUND-DEV:5315] Re: [Csnd] maximum ftable size cs5


At least under VC++ 6, Win32, the limit is a table size just a tad below
that:

1073741816 floats

(based on a maximum alloc of 0xFFFFFFE0 bytes).

There is always a little margin retained by C for housekeeping the buffer,
so 
the theoretical maximum under the OS is not available in practice. One
would 
have to obtain the defined max alloc value (e.g. from malloc.h).

And yes, other internal changes would be required (e.g. PHMASK at the top
of 
cs.h). These values are used in all ftable lookup calculations. Note that
when 
ftable_size = MAXLEN, the code reduces to truncated lookup. the smaller the 
table size is relative to MAXLEN, the more accurate the interpolation can
be.

Do we really need tables of this size? In the great majority of cases the
OS 
will have no choice but to use virtual memory on disk for such accesses
(and 
this can take the machine some significant time to set up), so the 
advantages 
over explicit disk access may be  more imagined than real. It might also be 
expected to cause problems when Csound is running as a plugin.



Richard Dobson





Anthony Kozar wrote:

> I am moving this thread over to the development list.
> 
> Is changing he following line in cs.h all that is necessary to increase
the
> max table length?:
> 
> #define MAXLEN     0x1000000L
> 
> Or are other internal changes also needed?  If not, I propose changing to:
> 
> #define MAXLEN  0x40000000L
> 
> That would allow table lengths up to 1073741824 which is 64 times the
> current maximum.  (Which should allow 24,347 seconds of mono audio @
> 44,100Hz sr -- and one table will require 4GB of RAM !!).
> 
> (Actually, by my calculations, the current maximum is ~380 seconds, not
47).
> 


--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .

Well OK, if it does not break or audibly affect existing lookup table code. The 
"maximum possible" will in any case need to be masked to a power of two, so 
probably we can reliably take this to be 0x80000000 on 32bit platforms.

There is no standard/portable mechanism in ANSI C to get the maximum alloc size. 
In C++ of course, we have std::allocator.max_size() available.

I remain interested in just what needs are met by this, but not by disk-based 
access.

Richard Dobson

gogins@pipeline.com wrote:

> Most people doh't need this facility, and won't run into its sharp edges.
> Those who do need it, need it bad, and will work with the problems. So I
> think the size should be increased to the maximum possible on the machine.
> 
> Original Message:
> -----------------
> From: Richard Dobson richarddobson@blueyonder.co.uk
> Date: Wed, 15 Sep 2004 21:18:37 +0100
> To: csound-dev@eartha.mills.edu
> Subject: [CSOUND-DEV:5315] Re: [Csnd] maximum ftable size cs5
> 
> 
> At least under VC++ 6, Win32, the limit is a table size just a tad below
> that:
> 
...

When I looked at this issue previously, I thought that table sizes were
stored as signed longs.  That would make 0x80000000 a negative number.  That
is why I suggested 0x40000000.  But as you pointed out, the maximum alloc
size on a 32-bit platform has to be less than 4GB, and 0x40000000 makes a
table size of exactly 4GB.  So perhaps 0x20000000 would be good enough for
now.

Also, (I am not very familiar with the code, but) does a MAXLEN of X still
allow X+1 to be used as a table size?

Anthony

PS.  For the record, on my PPC G3 with Metrowerks Std Lib, I get
1,073,741,823 from the following code:

    std::allocator a;
    cout    << a.max_size();


On 9/15/04 5:59 PM, Richard Dobson etched in
stone:

> Well OK, if it does not break or audibly affect existing lookup table code.
> The 
> "maximum possible" will in any case need to be masked to a power of two, so
> probably we can reliably take this to be 0x80000000 on 32bit platforms.
> 
> There is no standard/portable mechanism in ANSI C to get the maximum alloc
> size. 
> In C++ of course, we have std::allocator.max_size() available.

Should be easy enough to change it to unsigned long, unless a negative value is 
used to indicate some error condition. size_t is always an unsigned integral 
type. I seem to recall Gabriel used a negative value to indicate a 
non-power-of-two size, so perhaps he can comment on this. The issue was not only 
the size itself, but the special indication of the guard point, which is defined 
differently according to whether the size is a power of two or power-of-two plus 
1. 0x80000000 is 2GB in an unsigned long.

If we really are going to demand huge array sizes, etc, Csound will really have 
to get disciplined over the specification of unsigned variables. Much of the 
time, variables are signed simply because it saves typing! It is illogical for 
any size variable to be signed, in the absence of other requirements such as an 
error value.

Funnily enough, in the AIFF spec, the size of a chunk is defined to be contained 
in a signed long, so you will never get a "correct" AIFF or AIFF-C file > 2GB 
anyway. WAVE, being an altogether more sane format ( :-) ) uses unsigned longs 
for the size field of a chunk, so if you want to use the largest possible 
soundfiles to fill tables, it has to be WAVE. Similarly, if you really want the 
maximum possible size for an ftable, an unsigned type will be required.


Amazing how the seemingly simplest thing can get so complicated!

Richard Dobson

Anthony Kozar wrote:

> When I looked at this issue previously, I thought that table sizes were
> stored as signed longs.  That would make 0x80000000 a negative number.  That
> is why I suggested 0x40000000.  But as you pointed out, the maximum alloc
> size on a 32-bit platform has to be less than 4GB, and 0x40000000 makes a
> table size of exactly 4GB.  So perhaps 0x20000000 would be good enough for
> now.
> 
> Also, (I am not very familiar with the code, but) does a MAXLEN of X still
> allow X+1 to be used as a table size?
> 
> Anthony
> 
> PS.  For the record, on my PPC G3 with Metrowerks Std Lib, I get
> 1,073,741,823 from the following code:
> 
>     std::allocator a;
>     cout    << a.max_size();
> 
> 
> On 9/15/04 5:59 PM, Richard Dobson etched in
> stone:
> 
> 
>>Well OK, if it does not break or audibly affect existing lookup table code.
>>The 
>>"maximum possible" will in any case need to be masked to a power of two, so
>>probably we can reliably take this to be 0x80000000 on 32bit platforms.
>>
>>There is no standard/portable mechanism in ANSI C to get the maximum alloc
>>size. 
>>In C++ of course, we have std::allocator.max_size() available.
> 
> 
> 
>