dlopening from a SWIG library

[John Pye-2]

Hi there

I've got some code here for which I've been building some python hooks
as a first step towards implementing new interface for the old code.

This code is an equation solver program (called ascend), and part of
what it does is to compile C-code for mathematical equations so that
they can quickly evaluated at native machine speeds. This means that a C
source file is generated and then compiled and dlopened then using
dlsym, then an 'initialisation' routine is run to load the symbols for
functions which evaluate the equations in the source code.

My problem -- and I'm not sure if it's a linking problem, or a SWIG
problem, or a python problem -- is that when I compile my C++ 'hook
classes' and call them from a C++ main() function, they work as expected
(the main() program creates some equations, compiles them into a
temporary shared library (.so file), dlopens it and loads a symbol for
the 'initialisation' routine using dlsym), but when I compile my hooks
into a shared object and load it from python, the temporary shared
library won't load.

The temporary shared library contains a symbol which is present in the
main executable (ie from my main() function). This symbol is also
present in my _ascend.so python library. I think that what is happening
is that when my dlopen call is made, and there is found to be an
undefined symbol (a function contained in the main ascend code, being
called from the 'initialisation' routine), dlopen is looking in the
symbol table for the currently-running executable to try to resolve the
undefined symbol. In the case of the main() program, that symbol is
found. In the case using using the SWIG library from python,  the symbol
is not found, because *python* is the running executable, and the symbol
is in my _ascend.so, not in the python executable.

Is this a limitation of dlopen? Can I not perform 'cascading' dlopens?
How can I work around it so that my library compiled for python using
SWIG can perform dlopens of its own, which resolve undefined symbols
back in my python/SWIG library?

Any suggestions much appreciated,

Cheers
JP

--
John Pye
School of Mechanical and Manufacturing Engineering
The University of New South Wales
Sydney  NSW 2052  Australia
t [phone]
f [phone]
mailto:john.pye_AT_student_DOT_unsw.edu.au
http://pye.dyndns.org/



-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Swig-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/swig-user

[Bill Clarke]

John Pye wrote, On 11/07/05 16:22:
[...]

> My problem -- and I'm not sure if it's a linking problem, or a SWIG
> problem, or a python problem -- is that when I compile my C++ 'hook
> classes' and call them from a C++ main() function, they work as expected
> (the main() program creates some equations, compiles them into a
> temporary shared library (.so file), dlopens it and loads a symbol for
> the 'initialisation' routine using dlsym), but when I compile my hooks
> into a shared object and load it from python, the temporary shared
> library won't load.
>
> The temporary shared library contains a symbol which is present in the
> main executable (ie from my main() function). This symbol is also
> present in my _ascend.so python library. I think that what is happening
> is that when my dlopen call is made, and there is found to be an
> undefined symbol (a function contained in the main ascend code, being
> called from the 'initialisation' routine), dlopen is looking in the
> symbol table for the currently-running executable to try to resolve the
> undefined symbol. In the case of the main() program, that symbol is
> found. In the case using using the SWIG library from python,  the symbol
> is not found, because *python* is the running executable, and the symbol
> is in my _ascend.so, not in the python executable.
>
> Is this a limitation of dlopen? Can I not perform 'cascading' dlopens?
> How can I work around it so that my library compiled for python using
> SWIG can perform dlopens of its own, which resolve undefined symbols
> back in my python/SWIG library?

i suspect this is a linking problem.
what platform (OS version) are you running this on?
perhaps you need to check the dlopen flags to both your wrapper library
(python sys.setdlopenflags?) and the dlopened templorary library (have
you tried RTLD_GLOBAL?).

cheers,
/lib
--
/lib BillClarke PostdoctoralFellow CompSci ANU cs.anu.edu.au/CC-NUMA
http://llib.cjb.net [hidden email]  tel:[phone]x5687 fax:x0010
PGPid:B381EE7DB7D3E58F17248C672E2DA124ADADF444 GNU unix LaTeX XPilot
Buffy DrWho Goodies StarTrek XFiles Origami SML SMP MPI mozilla tcsh
Asimov Bear Clarke Donaldson Volleyball Ultimate Cricket emacs C++ X
Jordan Kay Lackey Martin Stasheff DeepPurple H&C KLF Queen PinkFloyd


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Swig-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/swig-user

[John Pye-2]

Hi Bill,

Thanks very much. "sys.setdlopenflags(dl.RTLD_GLOBAL)" was just what I
needed. I was setting these flags from within my C code however it seems
that the python setting might have been overriding somehow.

Cheers,
JP

Bill Clarke wrote:

>John Pye wrote, On 11/07/05 16:22:
>[...]
>  
>
>>My problem -- and I'm not sure if it's a linking problem, or a SWIG
>>problem, or a python problem -- is that when I compile my C++ 'hook
>>classes' and call them from a C++ main() function, they work as expected
>>(the main() program creates some equations, compiles them into a
>>temporary shared library (.so file), dlopens it and loads a symbol for
>>the 'initialisation' routine using dlsym), but when I compile my hooks
>>into a shared object and load it from python, the temporary shared
>>library won't load.
>>
>>The temporary shared library contains a symbol which is present in the
>>main executable (ie from my main() function). This symbol is also
>>present in my _ascend.so python library. I think that what is happening
>>is that when my dlopen call is made, and there is found to be an
>>undefined symbol (a function contained in the main ascend code, being
>>called from the 'initialisation' routine), dlopen is looking in the
>>symbol table for the currently-running executable to try to resolve the
>>undefined symbol. In the case of the main() program, that symbol is
>>found. In the case using using the SWIG library from python,  the symbol
>>is not found, because *python* is the running executable, and the symbol
>>is in my _ascend.so, not in the python executable.
>>
>>Is this a limitation of dlopen? Can I not perform 'cascading' dlopens?
>>How can I work around it so that my library compiled for python using
>>SWIG can perform dlopens of its own, which resolve undefined symbols
>>back in my python/SWIG library?
>>    
>>
>
>i suspect this is a linking problem.
>what platform (OS version) are you running this on?
>perhaps you need to check the dlopen flags to both your wrapper library
>(python sys.setdlopenflags?) and the dlopened templorary library (have
>you tried RTLD_GLOBAL?).
>
>cheers,
>/lib
>  
>

--
John Pye
School of Mechanical and Manufacturing Engineering
The University of New South Wales
Sydney  NSW 2052  Australia
t [phone]
f [phone]
mailto:john.pye_AT_student_DOT_unsw.edu.au
http://pye.dyndns.org/



-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Swig-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/swig-user