Ok, drop SD2. What formats do you think a cross-platform Csound should support?

Regards,
Mike

-----Original Message-----
>From: Erik de Castro Lopo 
>Sent: Jan 17, 2006 12:33 PM
>To: csound-devel@lists.sourceforge.net
>Subject: Re: [Cs-dev] File formats
>
>Michael Gogins wrote:
>
>> WAVEX and SD2, at least.
>
>Why SD2? SD2 only really works well on MacOSX with file systems
>that support resource forks.
>
>Erik
>-- 
>+-----------------------------------------------------------+
>  Erik de Castro Lopo
>+-----------------------------------------------------------+
>"We must not forget that Allah's rules have to be established in all lands."
>-- Imam Muzammil H. Siddiqi of the Islamic Society of North America
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
>for problems?  Stop!  Download the new AJAX search engine that makes
>searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
>_______________________________________________
>Csound-devel mailing list
>Csound-devel@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/csound-devel





-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

So, SD2 is Sound Designer 2 then?  If so, I think that it should be
supported because it WAS supported on MacOS under Csound 4.  SD2 was in fact
the default output format on the Mac under Csound 4 (replacing Ircam).

I never brought up the fact that this support disappeared because I assumed
that libsndfile had no way of writing a resource fork.  If I understand it
correctly, SD2 is essentially just a raw, headerless format with header
information stored in the Mac resource fork.

While we are on this subject of special Mac support, if libsndfile can write
resource forks for SD2, can we also make it so that it sets the creator and
file type metadata on the Mac file system.  As it is now, all of my Csound 5
output appears as text files instead of sound files in the Finder.

I would be happy to contribute code to accomplish this if it is OK.

Anthony Kozar
anthonykozar AT sbcglobal DOT net

    
Michael Gogins wrote on 1/17/06 1:15 PM:

> Ok, drop SD2. What formats do you think a cross-platform Csound should
> support?
> 
> Regards,
> Mike
> 
> -----Original Message-----
>> From: Erik de Castro Lopo 
>> Sent: Jan 17, 2006 12:33 PM
>> To: csound-devel@lists.sourceforge.net
>> Subject: Re: [Cs-dev] File formats
>> 
>> Michael Gogins wrote:
>> 
>>> WAVEX and SD2, at least.
>> 
>> Why SD2? SD2 only really works well on MacOSX with file systems
>> that support resource forks.



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Michael Gogins wrote:

> Ok, drop SD2. What formats do you think a cross-platform Csound should
> support?

WAV, AIFF defintely.

AU is also nice because it supports both endian-nesses and most reasonable
PCM formats. AU is especially good for temporary files.

Recent versions of libsndfile support FLAC which is a lossless codec 
which is useful for storing large files. Compiling FLAC for windows
is still a bit of a PITA.

Ogg Vorbis support in libsndfile is still under development (by someone
else) and I have no idea when that will be done.

MP3 is unlikely to be added any time soon because of the patent and
licensing issues.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
"Every time you get a windows programmer asking you to write some
ass-backward workaround, think of it as a crack junkie asking you
to help stuff his pipe because his hands are too shaky."
-- Conrad Parker


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Erik de Castro Lopo wrote:

> Michael Gogins wrote:
> 
> 
>>Ok, drop SD2. What formats do you think a cross-platform Csound should
>>support?
> 
> 
> WAV, AIFF defintely.
> 
> AU is also nice because it supports both endian-nesses and most reasonable
> PCM formats. AU is especially good for temporary files.
..


And CAF (the new Apple multi-channel file format for 64bit architectures). 
Despite the fact that the 64bit G5  now appears to be a product-line dead-end as 
far as Apple is concerned (the new Dual-Core Intel machines revert to the old 
32bit architecture), this is clearly an important format for the future, and 
unless I am mistaken, did I not see a first level of support for it in libsndfile?


Richard Dobson




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Richard Dobson wrote:

> And CAF (the new Apple multi-channel file format for 64bit architectures). 
> Despite the fact that the 64bit G5  now appears to be a product-line dead-end as 
> far as Apple is concerned (the new Dual-Core Intel machines revert to the old 
> 32bit architecture), this is clearly an important format for the future, and 
> unless I am mistaken, did I not see a first level of support for it in libsndfile?

libsndfile supports it. See http://www.mega-nerd.com/libsndfile/

However, I'm not so sure that this will be an important future format.

Obviously this format was designed before Steve Jobs decided to ditch PPC.
The big problem with CAF is that although the audio data can be in either
big or little endian format, the header data is always big endian. Once
Apple goes to intel that makes very little sense and retrofitting little
endian headers to CAF will likely be a mess.

I think Apple blew this one.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
I have now spent two days debugging C# on dotnot and all I can say
is OMFG!!!! The MS tools are completely fucking abysmal. Its the
most developer hostile environment I've ever worked on. The fact 
that people are actually able to release software that sort of 
works using these tools completely boggles my mind.


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Erik de Castro Lopo wrote:

..
> 
> Obviously this format was designed before Steve Jobs decided to ditch PPC.
> The big problem with CAF is that although the audio data can be in either
> big or little endian format, the header data is always big endian. Once
> Apple goes to intel that makes very little sense and retrofitting little
> endian headers to CAF will likely be a mess.
> 
> I think Apple blew this one.
> 

They have been plotting the Intel route for some considerable time (Darwin has 
been available on Intel for many years); I very much doubt the Intel path was a 
last-ditch decision. I guess time will tell! But the industry clout of Quicktime 
on both platforms is substantial. Big-endian is also network order, which may be 
relevant for real-time streaming what are expected to be huge files. Swapping 
bytes in headers is of little consequence; swapping the audio data is much more 
the thing to avoid. And given the number of Mac users still on OS X 10.1 and 
10.2 (to say nothing of the diehards on OS 9.x!), big-endian file formats will 
be relevant for quite a few years yet.

Richard Dobson





-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Anthony Kozar wrote:

> So, SD2 is Sound Designer 2 then?

Yes.

> I never brought up the fact that this support disappeared because I assumed
> that libsndfile had no way of writing a resource fork.

Poor assumption :-).

>  If I understand it
> correctly, SD2 is essentially just a raw, headerless format with header
> information stored in the Mac resource fork.

Yep.

The main problem with SD2 on OSX is transporting those files to Linux
and windows systems. The vast majority of naive users simply copy over
the data fork and then bitch about not being able open them because
they don't know the sample rate, channel count, bit width etc.

I also seem to remember that there is at least one OSX supported file
system (UFS??) that doesn't support resource forks so the you end up
with a pair file files like:

    file.sd2
    .file.sd2

where the first is the data fork and the second is the resource fork.
This is the solution libsndfile uses on Linux and windows and I'm
not real keen on because it is so inherantly fragile.

Looking back I wish that I made SD2 a read only format.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
Question #10342: Ruling on a Muslim eating with a kaafir
http://islamqa.com/index.php?ln=eng&ds=qa&lv=browse&QR=10342&dgn=4


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Hi Erik,

I am looking at the support for .sd2 files now in libsndfile-1.0.13pre6 (I
had been looking at 1.0.10 at that was not yet complete and so was
confusing), and it appears that the code is using some kind of hack in the
Apple Posix implementation to read/write the resource fork.  This will not
work on MacOS 9 though.

I have also noticed that the MacOS9 folder in the libsndfile distribution
has disappeared.  I have been compiling it for OS 9 now for awhile in order
to continue supporting Csound 5 on that platform.  Besides removing those
files, were any other changes made to drop support of OS 9?  Can the OS 9
support be added back?  I have some updates to the OS 9 specific files as
well so that they compile with more recent versions.

Regarding the resource fork support,

Erik de Castro Lopo wrote on 1/18/06 2:18 AM:

> I also seem to remember that there is at least one OSX supported file
> system (UFS??) that doesn't support resource forks so the you end up
> with a pair file files like:
> 
> file.sd2
> .file.sd2
> 
> where the first is the data fork and the second is the resource fork.

(._file.sd2 actually)

This solution is also used by Apple in packaging .tgz files for their
installer.  But it seems to be different than the convention used by this
code in libsndfile which I have never seen before:


    /* Test for MacOSX style resource fork on HPFS or HPFS+ filesystems. */
    LSF_SNPRINTF (psf->rsrcpath, sizeof (psf->rsrcpath), "%s/rsrc",
psf->filepath) ;
    psf->error = SFE_NO_ERROR ;
    if ((psf->rsrcdes = psf_open_fd (psf->rsrcpath, open_mode)) >= 0)
    {    psf->rsrclength = psf_get_filelen_fd (psf->rsrcdes) ;
        if (psf->rsrclength > 0 || (open_mode & SFM_WRITE))
            return SFE_NO_ERROR ;
        psf_close_fd (psf->rsrcdes) ;
        psf->rsrcdes = -1 ;
        } ;

    if (psf->rsrcdes == - SFE_BAD_OPEN_MODE)
    {    psf->error = SFE_BAD_OPEN_MODE ;
        return psf->error ;
        } ;

And Apple has also used the convention on DOS disks of storing resource
forks in a subdirectory called RESOURCE.FRK.  Basically, Apple is very
inconsistent and can't make up its mind (and not just on this issue).

In order to support SD2 files on MacOS 9 though, another method would have
to be used to access the resource forks.  It would likely involve making Mac
System calls in libsndfile.

The Mac file system also supports setting two pieces of metadata called file
type and creator type.  These are somewhat supported on OS X but are very
important for files to be usable on OS 9.  If you would accept changes to
libsndfile to support setting these, I would appreciate it.

Thanks.

Anthony Kozar
anthonykozar AT sbcglobal DOT net



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

On Jan 18, 2006, at 2:18 AM, Erik de Castro Lopo wrote:

> I also seem to remember that there is at least one OSX supported file
> system (UFS??) that doesn't support resource forks so the you end up
> with a pair file files like:
>
>     file.sd2
>     .file.sd2
>
> where the first is the data fork and the second is the resource fork.
> This is the solution libsndfile uses on Linux and windows and I'm
> not real keen on because it is so inherantly fragile.

Actually the resource forks end up like

	__MACOSX   (A folder)
	file.sd2	  (Raw data)
	._file.sd2 (A folder)
		|
		+-> information contained in the "file.rsrc", usually something like 
._file.rsc

In other words, a mess.

I would spend my 2 cents in this discussion advocating FLAC and AU 
format audio files for the linear PCM side of things, and when 
Libsndfile supports it, OGG and MP2 on the compressed end.

Will adding filetypes like MP2 add an additional dependency of 
libquicktime to build libsndfile ? If so, then I take back the support 
for MP2 and will just advocate Ogg Vorbis format instead, provided no 
additional dependencies are introduced.


-David



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Anthony Kozar wrote:

> Hi Erik,
> 
> I am looking at the support for .sd2 files now in libsndfile-1.0.13pre6 (I
> had been looking at 1.0.10 at that was not yet complete and so was
> confusing), and it appears that the code is using some kind of hack in the
> Apple Posix implementation to read/write the resource fork.  This will not
> work on MacOS 9 though.
> 
> I have also noticed that the MacOS9 folder in the libsndfile distribution
> has disappeared. 

I dropped support for OS9 because I had no way of testing it.

> Can the OS 9 support be added back?

Not unless I can test on that platform. Basically I don't like
shipping code that I haven't tested.

In the past I have also experimented with letting other people
maintain certain aspects that I was having difficulty supporting
(ie project files for the microsoft development environment) and
it was a disaster. I was not able to keep these files up to date
and every time I shipped a new release I got emails about it
not working.


  I have some updates to the OS 9 specific files as
> well so that they compile with more recent versions.
> 
> Regarding the resource fork support,
> 
> Erik de Castro Lopo wrote on 1/18/06 2:18 AM:
> 
> > I also seem to remember that there is at least one OSX supported file
> > system (UFS??) that doesn't support resource forks so the you end up
> > with a pair file files like:
> > 
> > file.sd2
> > .file.sd2
> > 
> > where the first is the data fork and the second is the resource fork.
> 
> (._file.sd2 actually)

Yes, you are right.

> This solution is also used by Apple in packaging .tgz files for their
> installer.  But it seems to be different than the convention used by this
> code in libsndfile which I have never seen before:
> 
> 
>     /* Test for MacOSX style resource fork on HPFS or HPFS+ filesystems. */
>     LSF_SNPRINTF (psf->rsrcpath, sizeof (psf->rsrcpath), "%s/rsrc",
> psf->filepath) ;

This code works correctly on OSX. On OSX the file named:

    /path/to/file

has a resource fork that can be accessed using:

    /path/to/file/rsrc

> In order to support SD2 files on MacOS 9 though, another method would have
> to be used to access the resource forks.  It would likely involve making Mac
> System calls in libsndfile.

I currently have two file I/O backends, a Unix/Posix one for Linux,
MacOSX and others and a Win32 one usined microsofts API. 

Each release of libsndfile is currently teste on:

    Linux x86
    Linux PowerPC
    MacOSX (PowerPC)
    Win32 (MinGW)
    Win32 (Cygwin)
    Win32 (MSVC)

Since my ibook is on its very last legs (the battery is shot and
so is the screen) testing on MacOSX is going to get increasingly
difficult as the screen degrades. I should still be able to run
Linux on it and just treat it as a headless box I ssh into.

> The Mac file system also supports setting two pieces of metadata called file
> type and creator type.

I thought that even Apple considers this deprecated. I'm not
keen on supporting a legacy OS which a huge number of quirks
that almost nobody uses. It was my understanding that the
vast majority of serious Apple users havemoved from OS9 to
OSX some time in the last three years.

> If you would accept changes to
> libsndfile to support setting these, I would appreciate it.

The problem is not whether libsndfile can support it, but 
whether I can support it.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
"Incompetence, like misery, seeks company." -- Erik Naggum


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net