Makefile.in, LIBTOOL and shared/static builds.

[o***@fredslev.dk]

Hi,

I am using the GNU libtool alternative slibtool which has some benefits
such as a smaller code base, actively maintained and a huge performance
boost.

The problem is that because slibtool doesn't work in the top directory
of a project it doesn't know if it should build with --enable-static,
--enable-shared or etc. To work around this slibtool ships with
symlinks, slibtool-shared and slibtool-static which alter the behavior
and works well enough if the user knows if it should be a static or
shared build. Unfortunately this also means that slibtool is not
capable of respecting ./configure flags such as --enable-shared or
--enable-static and will fall apart with more complicated builds like
libreoffice which has many internal dependencies where they mix and
match shared and static builds.

From talking with the slibtool developer I am informed this would be
best fixed in automake.

Currently it seems automake will add to Makefile.in.

$(LIBTOOL)

For slibtool and maybe some other projects (?) it would be much better
if automake instead added to Makefile.in.

$(LIBTOOL) $(LIBTOOL_LINKAGE)

Or something to that effect which I understand should be safe, this way
slibtool could know exactly what it should use with much less
complications for the user or failures with more complicated builds.

For reference the slibtool git repo can be found in the following link.

https://git.midipix.org/cgit.cgi/slibtool/

Thanks for any help,
Hunter

[John Calcote]

Post by o***@fredslev.dk
Hi,
I am using the GNU libtool alternative slibtool which has some benefits
such as a smaller code base, actively maintained and a huge performance
boost.

I’m curious - it’s neat that slibtool exists, but if you need functionality
offered by libtool then why not just use libtool?

John

[o***@fredslev.dk]

Post by John Calcote
I’m curious - it’s neat that slibtool exists, but if you need
functionality offered by libtool then why not just use libtool?

Frankly libtool is 12,000+ line shell script which is hard to
understand, maintain, fix and is rather slow.

A while ago with my 6 core AMD FX-6350 cpu I tested building
libtomcrypt with both libtool and slibtool using six make jobs. This
program does heavily use libtool, but does not use autotools.

With libtool:

real 0m43.242s
user 2m9.735s
sys 0m38.435s

With slibtool:

real 0m12.180s
user 0m42.789s
sys 0m10.406s

Another way to represent the performance difference is by compiling a
single file and measuring the system calls with 'strace -fc'.

libtool --mode=compile --tag=CC /bin/true -c foo.c

or

slibtool --mode=compile --tag=CC /bin/true -c foo.c

Here with a simple hello world I am seeing 1682 system calls with
libtool and 345 with slibtool.

==================
$ cat /tmp/hello.c
#include <stdio.h>
int main()
{
// printf() displays the string inside quotation
printf("Hello, World!");
return 0;
}
==================

These performance boosts can make a significant difference with
projects like mesa which are updated often. The smaller and actively
maintained code base is also encouraging, but another thing I would
like to point out is that one of libtool's "features" is that it will
ignore bugs from various configure.ac or Makefile.am files and will
successfully build a program with potentially unintended behavior.

One example I recall is where I found a configure.ac which was
incorrectly using sed on the output of some internal commands and
feeding the compiler bogus flags. Libtool silently swallowed the bug
and moved on where slibtool helpfully printed the error, allowed the
bug to be found and then fixed.

Additional benefits are that installing the .la files is optional (Most
programs will not break if they are missing) and that there is only one
slibtool binary for all the supported targets and target flavors.

My understanding is the problem here is that slibtool works at a lower
level than libtool and as a result is not able to rely on what is not
in files like Makefile.in as libtool can. This should be relatively
easy to fix in automake (?) with making it possible to obtain this
information easier.

Alternative suggestions on how to improve this would be of course
welcome.

Note that this may of been sent twice, I'm not sure if the first e-mail
was correctly sent...

[Bob Friesenhahn]

Post by o***@fredslev.dk

Post by John Calcote
I’m curious - it’s neat that slibtool exists, but if you need
functionality offered by libtool then why not just use libtool?

Frankly libtool is 12,000+ line shell script which is hard to
understand, maintain, fix and is rather slow.

Agreed.

Post by o***@fredslev.dk
A while ago with my 6 core AMD FX-6350 cpu I tested building
libtomcrypt with both libtool and slibtool using six make jobs. This
program does heavily use libtool, but does not use autotools.

[ results stuff removed ]

The overhead attributed to libtool seems rather high. Is there
something about your execution environment or your libtool usage which
causes more overhead than usual?

Post by o***@fredslev.dk
Here with a simple hello world I am seeing 1682 system calls with
libtool and 345 with slibtool.

Agreed. A shell script will always use more system calls and
overhead than a simple utility since shell scripts are comprised of
many calls to simple (or complex) utilities.

Post by o***@fredslev.dk
One example I recall is where I found a configure.ac which was
incorrectly using sed on the output of some internal commands and
feeding the compiler bogus flags. Libtool silently swallowed the bug
and moved on where slibtool helpfully printed the error, allowed the
bug to be found and then fixed.

How does slibtool validate arguments? Does it understand the specific
set of arguments allowed by the compiler/linker being used?

Post by o***@fredslev.dk
Additional benefits are that installing the .la files is optional (Most
programs will not break if they are missing) and that there is only one
slibtool binary for all the supported targets and target flavors.

What are the supported targets and target flavors?

Does slibtool discard most of the portabilty (host OSs and toolchains)
offered by the libtool script?

Post by o***@fredslev.dk
My understanding is the problem here is that slibtool works at a lower
level than libtool and as a result is not able to rely on what is not
in files like Makefile.in as libtool can. This should be relatively
easy to fix in automake (?) with making it possible to obtain this
information easier.

Usually it is possible to substitute Makefile text to replace the
default rules used by Automake. This should allow changing how
Automake invokes the tool in order to pass additional arguments. Have
you tried that?

Bob

[o***@fredslev.dk]

The overhead attributed to libtool seems rather high. Is there
something about your execution environment or your libtool usage which
causes more overhead than usual?

Not that I am aware of, its the standard GNU libtool in
Slackware64-current.

How does slibtool validate arguments? Does it understand the specific
set of arguments allowed by the compiler/linker being used?

Slibtool handles all arguments that ere required for correct operation
and then passes everything that is not related to libtool to the
compiler. In my example I recall it was a compiler error and libtool
was swallowing the bogus argument before it got the compiler.

What are the supported targets and target flavors?
Does slibtool discard most of the portabilty (host OSs and toolchains)
offered by the libtool script?

The slibtool developer offered me this reply to post:

===============

Regarding slibtool's portability: the tool is written in C and has no
external dependencies (other than the system's libc). The official
requirement for building slibtool is a c99 compiler and a system
supporting -D_XOPEN_SOURCE=600, however with just a few CFLAGS you
could build slibtool with a c89 compiler for a system that only
supports -D_XOPEN_SOURCE=500.

With respect to targets and target flavors, slibtool's approach (as
well as the approach of slibtool's own configure+Makefile) is to
*support any target by default* while making support of "special"
targets, as well as non-default target flavors, both possible and easy.
For additional context, here are some relevant bits from slibtool's
driver and API:

struct slbt_host_params {
const char * host;
const char * flavor;
const char * ar;
const char * ranlib;
const char * dlltool;
const char * mdso;
const char * ldrpath;
};

struct slbt_flavor_settings {
const char * imagefmt;
const char * arprefix;
const char * arsuffix;
const char * dsoprefix;
const char * dsosuffix;
const char * exeprefix;
const char * exesuffix;
const char * impprefix;
const char * impsuffix;
const char * ldpathenv;
};

#define SLBT_FLAVOR_SETTINGS(flavor,bfmt,arp,ars,dsop,dsos,exep,exes,impp,imps,ldenv) \
static const struct slbt_flavor_settings flavor = { \
bfmt,arp,ars,dsop,dsos,exep,exes,impp,imps,ldenv}

SLBT_FLAVOR_SETTINGS(host_flavor_default, "elf", "lib",".a", "lib",".so", "","", "", "", "LD_LIBRARY_PATH");
SLBT_FLAVOR_SETTINGS(host_flavor_midipix, "pe", "lib",".a", "lib",".so", "","", "lib",".lib.a", "LD_LIBRARY_PATH");
SLBT_FLAVOR_SETTINGS(host_flavor_mingw, "pe", "lib",".a", "lib",".dll", "",".exe", "lib",".dll.a", "PATH");
SLBT_FLAVOR_SETTINGS(host_flavor_cygwin, "pe", "lib",".a", "lib",".dll", "",".exe", "lib",".dll.a", "PATH");
SLBT_FLAVOR_SETTINGS(host_flavor_darwin, "macho","lib",".a",
"lib",".dylib", "","", "", "", "DYLD_LIBRARY_PATH");

===============

Usually it is possible to substitute Makefile text to replace the
default rules used by Automake. This should allow changing how
Automake invokes the tool in order to pass additional arguments. Have
you tried that?

No, I did not consider that and I am not sure I really understand the
suggestion?