I noticed tonight that the code in csoundLoadExternal() removes all leading
directory components from the name it is passed.  (And amazingly would do
this correctly on MacOS 9 which, in general, has different rules for
pathnames :)   This includes the case when trying to open a plugin specified
on the command-line with --opcode-lib= .

Does this make sense?  Seems to me that that is one of the uses of
--opcode-lib, to be able load a plugin from _anywhere_.  And if
csoundLoadExternal() IS supposed to just remove the path and search in
OPCODEDIR when all plugins are automatically loaded from that directory
already, then what is the purpose of the --opcode-lib flag now?

:-)

Anthony Kozar
anthonykozar AT sbcglobal DOT net
http://akozar.spymac.net/



-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Anthony Kozar wrote:

> I noticed tonight that the code in csoundLoadExternal() removes all leading
> directory components from the name it is passed.  (And amazingly would do
> this correctly on MacOS 9 which, in general, has different rules for
> pathnames :)   This includes the case when trying to open a plugin specified
> on the command-line with --opcode-lib= .
> 
> Does this make sense?  Seems to me that that is one of the uses of
> --opcode-lib, to be able load a plugin from _anywhere_.  And if
> csoundLoadExternal() IS supposed to just remove the path and search in
> OPCODEDIR when all plugins are automatically loaded from that directory
> already, then what is the purpose of the --opcode-lib flag now?

The name without directory components is only used when printing
messages. The actual library loading still uses the full path name.


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Ah! OK.  Sorry, then ... got confused.

Another question though:  I have plugin loading working here now on MacOS 9
via the --opcode-lib flag, but when I specify, for example,

--opcode-lib=ugens7lib

it loads this library twice! (It prints the name twice, and I confirmed by
stepping thru the code that it is loaded twice and the opcodes are appended
twice).

Somehow, the opcode-lib string becomes "ugens7lib,ugens7lib".

Another issue: the code for checking whether a library has already been
loaded makes a bad assumption.  It compares the two pointers returned by
csoundOpenLibrary(), but the MacOS is perfectly happy to open a library
twice and returns two _different_ reference numbers!

Anthony

Istvan Varga wrote on 10/14/05 4:34 AM:

> The name without directory components is only used when printing
> messages. The actual library loading still uses the full path name.



-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Anthony Kozar wrote:

> Another question though:  I have plugin loading working here now on MacOS 9
> via the --opcode-lib flag, but when I specify, for example,
> 
> --opcode-lib=ugens7lib
> 
> it loads this library twice! (It prints the name twice, and I confirmed by
> stepping thru the code that it is loaded twice and the opcodes are appended
> twice).
> 
> Somehow, the opcode-lib string becomes "ugens7lib,ugens7lib".
> 
> Another issue: the code for checking whether a library has already been
> loaded makes a bad assumption.  It compares the two pointers returned by
> csoundOpenLibrary(), but the MacOS is perfectly happy to open a library
> twice and returns two _different_ reference numbers!

Loading the library twice is a side-effect of how the command line
parser works. I can add code for the Mac that checks for duplicate
libraries by comparing the file names, but this does not always
detect multiple loading of the same file (e.g. full vs. relative
paths, symbolic links, etc.). At least the above error would be
fixed, though.
I did expect that comparing the pointers may not work on some
platforms, but where it does, it is more reliable than checking
file names, and is also simple to implement.
By the way, how does the MacOS behave exactly when a library is
loaded multiple times ? Does it maintain a reference count requiring
an equal number of closing the library, like the other systems ?
Are multiple copies of the library loaded into memory (I hope not) ?


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

For now, I added code to csoundLoadExternals() that sorts the list
of plugin libraries loaded with --opcode-lib, and filters out duplicate
entries.

Anthony Kozar wrote:

> Ah! OK.  Sorry, then ... got confused.
> 
> Another question though:  I have plugin loading working here now on MacOS 9
> via the --opcode-lib flag, but when I specify, for example,
> 
> --opcode-lib=ugens7lib
> 
> it loads this library twice! (It prints the name twice, and I confirmed by
> stepping thru the code that it is loaded twice and the opcodes are appended
> twice).
> 
> Somehow, the opcode-lib string becomes "ugens7lib,ugens7lib".
> 
> Another issue: the code for checking whether a library has already been
> loaded makes a bad assumption.  It compares the two pointers returned by
> csoundOpenLibrary(), but the MacOS is perfectly happy to open a library
> twice and returns two _different_ reference numbers!


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Istvan Varga wrote on 10/14/05 5:15 AM:

> Loading the library twice is a side-effect of how the command line
> parser works. I can add code for the Mac that checks for duplicate
> libraries by comparing the file names, but this does not always
> detect multiple loading of the same file (e.g. full vs. relative
> paths, symbolic links, etc.).

I do not think that you should add code just for the Mac -- it would be nice
if all platforms did not try to open the same libraries twice.

BTW, MacOS 7-9 do not use pathnames for opening libraries at all.  They do
not even use filenames.  Libraries are referenced via an internal logical
name so that they can still be found even if the user renames them.  I have
been working around this issue for now by always keeping the filename and
logical name the same.  But when I go to add auto-loading support for OS 9,
the function csoundLoadModules() will need to have a completely different
implementation because of this fact.

> I did expect that comparing the pointers may not work on some
> platforms, but where it does, it is more reliable than checking
> file names, and is also simple to implement.

Comparing the logical names would be the best method on OS 9 but is not
strictly necessary.  I have found a way today to have the OS 9
implementation of csoundOpenLibrary() check to see if a library has already
been loaded previously before it loads the requested lib.  If it has, then
it returns NULL.  The annoying thing about this though is with the
duplication of lib names, I end up with two messages for each library, one
confirming loading and one saying it failed.  Hopefully, your new code will
fix this.

BTW, it would be desirable to me to be able to output platform-specific
error messages from the functions csoundOpenLibrary(), csoundCloseLibrary(),
and csoundGetLibrarySymbol().  Would it be possible to have these functions
take a CSOUND* pointer so that csoundMessage, etc. may be called?

Also, will csoundGetLibrarySymbol() only be used to retrieve code symbols,
or is it conceivable that a data symbol might be asked for someday?  MacOS 9
allows checking to make sure that the symbol is of the correct type, but I
will have to remove this check if data symbols are to be imported.

> By the way, how does the MacOS behave exactly when a library is
> loaded multiple times ? Does it maintain a reference count requiring
> an equal number of closing the library, like the other systems ?
> Are multiple copies of the library loaded into memory (I hope not) ?

I think that the Mac actually had a very sophisticated shared library
implementation when it was introduced in System 7 (1993?).  It seems to even
be better today than the linking mechanisms on some other platforms from
what I can tell.  (eg.  linking together multiple code fragments with the
same symbol names is not a problem).

AFAIK, no library is ever completely loaded into memory twice.  There are
options to either duplicate or share the global data in the library and a
single application can manually open a library multiple times to get
multiple copies of its static data.  (This is how MacCsound was able to call
Csound repeatedly without quitting long before Csound was
multiply-instantiable).

Each time a library is opened by an app (including the same one) it creates
a new "connection" and the app receives a unique connection ID.  This is the
number that I return to Csound from the open library call and that is why it
does not detect the duplicate opening. **  Every connection is reference
counted, and libraries are unloaded automatically when the count reaches
zero.  Connections are closed automatically when an app quits but can be
closed manually before that.

I will have my code in CVS shortly, so you can examine it if you wish.

** The value returned from csoundOpenLibrary(), BTW, has to be a number on
OS 9, not a pointer to some opaque structure.  Casting the connection ID to
(void*) does seem to work, but makes me feel a little bit "dirty."


Thanks again for the help!

Anthony Kozar
anthonykozar AT sbcglobal DOT net
http://akozar.spymac.net/




-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Sorry but I haven't had working email all day, so I could not respond
earlier to tell you that I partially solved this problem.  But, removing the
duplicates is probably a good idea anyways.

Anthony

Istvan Varga wrote on 10/14/05 3:33 PM:

> For now, I added code to csoundLoadExternals() that sorts the list
> of plugin libraries loaded with --opcode-lib, and filters out duplicate
> entries.



-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Anthony Kozar wrote:

> I do not think that you should add code just for the Mac -- it would be nice
> if all platforms did not try to open the same libraries twice.

The filtering code was already added for all platforms, although it
only makes a useful difference where multiple loading of the same
file does not return the same handle.

> BTW, MacOS 7-9 do not use pathnames for opening libraries at all.  They do
> not even use filenames.  Libraries are referenced via an internal logical
> name so that they can still be found even if the user renames them.  I have
> been working around this issue for now by always keeping the filename and
> logical name the same.  But when I go to add auto-loading support for OS 9,
> the function csoundLoadModules() will need to have a completely different
> implementation because of this fact.

Well, if there is no dirent.h interface on the Mac then a separate
full implementation of csoundLoadModules() is needed anyway, and there
is already an #ifdef that wraps the entire function.

> BTW, it would be desirable to me to be able to output platform-specific
> error messages from the functions csoundOpenLibrary(), csoundCloseLibrary(),
> and csoundGetLibrarySymbol().  Would it be possible to have these functions
> take a CSOUND* pointer so that csoundMessage, etc. may be called?

These functions are intended to be generic wrappers that can be used
by the host application without being associated with an instance pointer,
so returning error values rather than printing messages may be preferred.
csoundCloseLibrary() already has an integer return value, and the most
likely failure in csoundGetLibrarySymbol() is not finding the symbol, so
that function may be fine as it is, too. In the case of csoundOpenLibrary(),
there may be multiple reasons for not being able to open a file (typically
either because the file does not exist, or it is not a dynamic library), so
it could be changed to return an integer error value and store the handle
in a void** argument.
Alternatively, if the functions are removed from the host API and made
interal, there should be no problem with adding a CSOUND* argument (this
may also allow for moving any statics - there are some in the OS X code -
to CSOUND, and using functions that may possibly call longjmp). However,
printing messages may still be undesired, particularly in the case of
csoundGetLibrarySymbol(): you would always get a long stream of error
messages, as many of the plugin interface functions are optional and
often not defined.

> Also, will csoundGetLibrarySymbol() only be used to retrieve code symbols,
> or is it conceivable that a data symbol might be asked for someday?  MacOS 9
> allows checking to make sure that the symbol is of the correct type, but I
> will have to remove this check if data symbols are to be imported.

In fact, on most non-Mac platforms there are no separate ways for getting
the address of a code or data symbol in a shared library. On Windows there
is
   FARPROC GetProcAddress(HMODULE hModule, LPCSTR lpProcName);
while POSIX systems have
   void *dlsym(void *handle, const char *symbol);
Note that other than the names, the two functions are basically identical,
and both can return a handle to any symbol in the library, regardless of
whether it is a function or a variable. You are assumed to know the type
and cast accordingly.

> ** The value returned from csoundOpenLibrary(), BTW, has to be a number on
> OS 9, not a pointer to some opaque structure.  Casting the connection ID to
> (void*) does seem to work, but makes me feel a little bit "dirty."

Well, if you think casting to void* is not reliable for some reason, then
you can allocate a CFragConnectionID with malloc() and return the address
of that, with calling free() in csoundCloseLibrary(). However, if a simple
cast does work, then it might be kept, even if it is ugly; writing portable
wrappers to platform dependent interfaces does involve such tricks sometimes.
Of course, if the return type is changed to some integer, then the other
plastforms would have the ugly casts, or maintain a database of loaded
libraries to which an integer index is returned.


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Istvan Varga wrote on 10/15/05 8:04 AM:

> The filtering code was already added for all platforms, although it
> only makes a useful difference where multiple loading of the same
> file does not return the same handle.

Yes, I tried it last night, and it works well.  Thanks!

> Well, if there is no dirent.h interface on the Mac then a separate
> full implementation of csoundLoadModules() is needed anyway, and there
> is already an #ifdef that wraps the entire function.

That is my plan as dirent.h does not return the information really needed to
open a library on OS 9.  Should be no problem to implement what I need.

>> BTW, it would be desirable to me to be able to output platform-specific
>> error messages from the functions csoundOpenLibrary(), csoundCloseLibrary(),
>> and csoundGetLibrarySymbol().  Would it be possible to have these functions
>> take a CSOUND* pointer so that csoundMessage, etc. may be called?
> 
> These functions are intended to be generic wrappers that can be used
> by the host application without being associated with an instance pointer,
> so returning error values rather than printing messages may be preferred.

This would probably be an OK solution.

> In the case of csoundOpenLibrary(),
> there may be multiple reasons for not being able to open a file (typically
> either because the file does not exist, or it is not a dynamic library), so
> it could be changed to return an integer error value and store the handle
> in a void** argument.

I wanted to print more descriptive errors on this one because of the variety
of reasons for failure as you mention.  In some circumstances, it would be
possible for a Csound plugin to be found and for everything to be OK except
that some other library that the plugin depends on is not available.  On
MacOS 9, it is possible to identify exactly which library failed to load and
report that to the user.  (A good example of this scenario would be trying
to load the python opcodes without a python library in the system).

But, either way -- being able to return an error code or being able to print
a custom message would be OK with me.  Doing nothing would make debugging
slightly more time-consuming for me (and possibly annoying to users), but if
no one else cares about these issues, then I can live with that option as
well.

> printing messages may still be undesired, particularly in the case of
> csoundGetLibrarySymbol(): you would always get a long stream of error
> messages, as many of the plugin interface functions are optional and
> often not defined.

You are correct, of course!  I did consider that.

> In fact, on most non-Mac platforms there are no separate ways for getting
> the address of a code or data symbol in a shared library. On Windows there

Encountering a symbol of the wrong type is the only error I can see wanting
to report from csoundGetLibrarySymbol().  But, this _should_ never happen in
practice.  Therefore, I will remove the type check from the OS 9 code so
that it works the same as the other platforms :)

>> ** The value returned from csoundOpenLibrary(), BTW, has to be a number on
>> OS 9, not a pointer to some opaque structure.  Casting the connection ID to
>> (void*) does seem to work, but makes me feel a little bit "dirty."
> 
> Well, if you think casting to void* is not reliable for some reason, then
> you can allocate a CFragConnectionID with malloc() and return the address
> of that, with calling free() in csoundCloseLibrary().

I considered this as well.  I will check if the cast is safe, and if not I
will take the malloc approach.

> Of course, if the return type is changed to some integer, then the other
> plastforms would have the ugly casts, or maintain a database of loaded
> libraries to which an integer index is returned.

I wasn't suggesting changing it -- I was really just pointing out the
difference for informational purposes.

Anthony Kozar
anthonykozar AT sbcglobal DOT net
http://akozar.spymac.net/



-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net