Libtool, SWIG and testing

[Guillaume Marcais]

Hello all,

I am improving/writing a C++ project (called MUMmer) which creates a shared
library with libtool. In the same project, I also create a SWIG binding to
access in Perl some of the functionality of this shared library.

Libtool creates the shared library in a hidden directory:

.libs/libumdmummer.so

The SWIG compilation creates a couple a files, a perl module file and a
shared library:

swig/perl5/mummer.pm
swig/perl5/.libs/mummer.so

The problem comes in when when I write tests in Perl. I need to add
directories to the Perl lib path so that it finds the module and shared
library, like this:

perl -I ./swig/perl5 -I ./swig/perl5/.libs test.pl

Passing the path './swig/perl5/.libs' seems very hackish and not portable.
This path was not specified anywhere, libtools decided to hide files there
on its own, and it feels like I should not use these files directly.

What is the proper way for testing programs to find (and use, via dlopen)
not yet installed shared libraries compiled by automake/libtool?

Thanks,
Guillaume.

[Gavin Smith]

Post by Guillaume Marcais
I am improving/writing a C++ project (called MUMmer) which creates a shared
library with libtool. In the same project, I also create a SWIG binding to
access in Perl some of the functionality of this shared library.
.libs/libumdmummer.so
The SWIG compilation creates a couple a files, a perl module file and a
swig/perl5/mummer.pm
swig/perl5/.libs/mummer.so
The problem comes in when when I write tests in Perl. I need to add
directories to the Perl lib path so that it finds the module and shared
perl -I ./swig/perl5 -I ./swig/perl5/.libs test.pl
Passing the path './swig/perl5/.libs' seems very hackish and not portable.
This path was not specified anywhere, libtools decided to hide files there
on its own, and it feels like I should not use these files directly.

I'd suggest not hard-coding the name of the shared library in the Perl
code. It could be extracted from the *.la file. That would likely
improve portability. I don't see any way to avoid looking in .libs.

There are other ways to affect Perl's module search path: by setting
the PERL5LIB environmental variable, or by adding directories to @INC
when the Perl program is run.

I have a similar problem at the moment on a project. It doesn't use
SWIG, but XS. (I know nothing about SWIG.) You can see the code at
http://svn.savannah.gnu.org/viewvc/trunk/tp/Texinfo/Convert/XSParagraph/XSParagraph.pm?revision=6463&root=texinfo&view=markup
. It uses the DynaLoader module to load a dynamic library file.

See this recent discussion:
https://lists.gnu.org/archive/html/automake/2015-06/msg00018.html and
https://lists.gnu.org/archive/html/automake/2015-07/msg00002.html.
Apparently this is a road not very well travelled, so there is no
"proper way to do it".

[Guillaume Marcais]

Post by Gavin Smith
I'd suggest not hard-coding the name of the shared library in the Perl
code. It could be extracted from the *.la file. That would likely
improve portability. I don't see any way to avoid looking in .libs.

Thank you for your answer.

In this case, the Perl code that loads the shared library is generated by
SWIG (mummer.pm). It does use DynaLoader and calls "bootstrap mummer;". The
Makefile.am part that builds this library reads:

perlextdir = $(PERL_EXT_LIB)
perlext_SCRIPTS = perl5/mummer.pm
perlext_LTLIBRARIES = perl5/mummer.la
perl5_mummer_la_SOURCES = perl5/swig_wrap.cpp $(SWIG_SRC)

plus some other flags set in _CPPFLAGS, _LDFLAGS etc. Could really LIBTOOL
build a shared object in the .libs directory that could not be loaded by
"bootstrap mummer;"?

In other words, for testing, I would rather use the Perl code that is
generated by SWIG and that will ultimately will be installed, rather than
having to write my own loader. So far it seems to work by only tweaking the
environment with only the name ".libs" being really hard coded here. The
name of the library is coming from the Makefile.am (perl5/mummer.la).

Guillaume.

Post by Gavin Smith
There are other ways to affect Perl's module search path: by setting
when the Perl program is run.
I have a similar problem at the moment on a project. It doesn't use
SWIG, but XS. (I know nothing about SWIG.) You can see the code at
http://svn.savannah.gnu.org/viewvc/trunk/tp/Texinfo/Convert/XSParagraph/XSParagraph.pm?revision=6463&root=texinfo&view=markup
. It uses the DynaLoader module to load a dynamic library file.
https://lists.gnu.org/archive/html/automake/2015-06/msg00018.html and
https://lists.gnu.org/archive/html/automake/2015-07/msg00002.html.
Apparently this is a road not very well travelled, so there is no
"proper way to do it".

[Gavin Smith]

Post by Guillaume Marcais

Post by Gavin Smith
I'd suggest not hard-coding the name of the shared library in the Perl
code. It could be extracted from the *.la file. That would likely
improve portability. I don't see any way to avoid looking in .libs.

Thank you for your answer.
In this case, the Perl code that loads the shared library is generated by
SWIG (mummer.pm). It does use DynaLoader and calls "bootstrap mummer;". The
perlextdir = $(PERL_EXT_LIB)
perlext_SCRIPTS = perl5/mummer.pm
perlext_LTLIBRARIES = perl5/mummer.la
perl5_mummer_la_SOURCES = perl5/swig_wrap.cpp $(SWIG_SRC)
plus some other flags set in _CPPFLAGS, _LDFLAGS etc. Could really LIBTOOL
build a shared object in the .libs directory that could not be loaded by
"bootstrap mummer;"?

From DynaLoader manpage:

dl_findfile()
Syntax:

@filepaths = dl_findfile(@names)

Determine the full paths (including file suffix) of one or more
loadable files given their generic names and optionally one or more
directories. Searches directories in @dl_library_path by default
and returns an empty list if no files were found.

Names can be specified in a variety of platform independent forms.
Any names in the form -lname are converted into libname.*, where .*
is an appropriate suffix for the platform.

bootstrap()
Syntax:

bootstrap($module [...])

This is the normal entry point for automatic dynamic loading in
Perl.

It performs the following actions:

· locates an auto/$module directory by searching @INC

· uses dl_findfile() to determine the filename to load


so this relies on libtool creating the file and calling it
"mummer.so.0" or similar. That's reasonably likely; I was being safe
because I figured that libtool added the "dlname" line to the *.la
file for a reason, for example:

# The name that we can dlopen(3).
dlname='mummer.so.0'

Post by Guillaume Marcais
In other words, for testing, I would rather use the Perl code that is
generated by SWIG and that will ultimately will be installed, rather than
having to write my own loader. So far it seems to work by only tweaking the
environment with only the name ".libs" being really hard coded here. The
name of the library is coming from the Makefile.am (perl5/mummer.la).

There are two library files, the *.la file and the actual file that is
dlopen'd, which could be a *.so file.

Does SWIG know anything about libtool?

Maybe you could ask on the libtool mailing list whether you can rely
on the library file being placed in .libs.