Discussion:
Portable $addprefix
Kip Warner
2017-08-25 02:02:48 UTC
Permalink
Hey list,

I'd like to transform the following variable in my Makefile.am from...

files_only = a.foo b.foo c.foo d.foo ...

Into...

files_with_path = dir/a.foo dir/b.foo dir/c.foo dir/d.foo ...

I'm aware of GNU Make's $addprefix magic. I'm also aware it's naughty
to use where portability is a concern and a different implementation of
make may be used behind Automake. I was wondering if anyone has come up
with a recipe for scenarios like this?
--
Kip Warner | Senior Software Engineer
OpenPGP signed/encrypted mail preferred
http://www.thevertigo.com
Nick Bowler
2017-08-25 15:33:19 UTC
Permalink
Hello,
Post by Kip Warner
I'd like to transform the following variable in my Makefile.am from...
files_only = a.foo b.foo c.foo d.foo ...
Into...
files_with_path = dir/a.foo dir/b.foo dir/c.foo dir/d.foo ...
I'm not aware of any truly portable way to do this directly in make.

But your example looks like a pretty static list (i.e., this list won't
be changed by the user after the package is generated), so the portable
way is to just generate both lists in advance, at the same time you run
automake (perhaps with a perl script that postprocesses Makefile.in).

If the list depends on configure results then another possibility is
to have configure generate both lists.

Finally, while not portable to all make implementations, expansions
like this:

$(files_only:%:dir/%)

do work in multiple implementations other than GNU make.

Cheers,
Nick
Kip Warner
2017-08-26 00:27:48 UTC
Permalink
Post by Nick Bowler
I'm not aware of any truly portable way to do this directly in make.
Hey Nick. Thanks for getting back to me. 

Something that I guess I'm still confused about is why after more than
a decade does Automake not have a portable way to do these kinds of
routine things?

I realize that GNU Make's extensions aren't found in all (or even most)
implementations of make(1), but you'd think the most common extensions
in it would have some way for Automake to detect where they are
available in the underlying implementation, and if not GNU Make or
another supported implementation, to emit kind of Perl logic / shell
magic as you suggest below.
Post by Nick Bowler
But your example looks like a pretty static list (i.e., this list
won't be changed by the user after the package is generated), so the
portable way is to just generate both lists in advance, at the same
time you run automake (perhaps with a perl script that postprocesses
Makefile.in).
Agreed. The list is definitely static. Do you have a perl recipe? I'm
not really familiar with the language or how to integrate it with
Automake.
Post by Nick Bowler
Finally, while not portable to all make implementations, expansions
  $(files_only:%:dir/%)
do work in multiple implementations other than GNU make.
I like this, but unfortunately I need to target for portability.
--
Kip Warner | Senior Software Engineer
OpenPGP signed/encrypted mail preferred
http://www.thevertigo.com
Mathieu Lirzin
2017-08-27 17:44:59 UTC
Permalink
Hello,
Post by Kip Warner
Hey list,
I'd like to transform the following variable in my Makefile.am from...
files_only = a.foo b.foo c.foo d.foo ...
Into...
files_with_path = dir/a.foo dir/b.foo dir/c.foo dir/d.foo ...
I'm aware of GNU Make's $addprefix magic. I'm also aware it's naughty
to use where portability is a concern and a different implementation of
make may be used behind Automake. I was wondering if anyone has come up
with a recipe for scenarios like this?
Would something like this work for you?

files_with_path = `for f in $(files_only); do echo "dir/$$f"; done`
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37
Kip Warner
2017-08-28 01:59:15 UTC
Permalink
Post by Mathieu Lirzin
Would something like this work for you?
  files_with_path = `for f in $(files_only); do echo "dir/$$f"; done`
Hey Mathieu,

Thank you for the suggestion. I'm only hesitant to use that because I'm
not sure if Automake needs to initialize the variable prior to emitting
Makefile from Makefile.am. Do you know if this is the case?

I went to test it, but for some reason I keep getting these errors on
that line:

Makefile:836: *** missing separator. Stop.
--
Kip Warner | Senior Software Engineer
OpenPGP signed/encrypted mail preferred
http://www.thevertigo.com
Mathieu Lirzin
2017-08-28 09:31:34 UTC
Permalink
Post by Kip Warner
Post by Mathieu Lirzin
Would something like this work for you?
  files_with_path = `for f in $(files_only); do echo "dir/$$f"; done`
Hey Mathieu,
Thank you for the suggestion. I'm only hesitant to use that because I'm
not sure if Automake needs to initialize the variable prior to emitting
Makefile from Makefile.am. Do you know if this is the case?
'files_with_path' will not be interpreted by Automake, it will just copy
the line in the generated Makefile. Since Make variables are macros the
content of the variable will be simply be copied by Make where
$(files_with_path) is used. As a consequence it should only be used in
the recipe of a rule, not as a target or prerequisite.
Post by Kip Warner
I went to test it, but for some reason I keep getting these errors on
Makefile:836: *** missing separator. Stop.
There is a bug in my suggestion. I guess this error is related to that.

o--8<---------------cut here---------------start------------->8---
files_only = foo.x bar.x baz.x
files_with_path = `for f in $(files_only); do echo "dir/$$f"; done`

all:
@echo "$(files_with_path)"
--8<---------------cut here---------------end--------------->8---

$ make
dir/foo.x
dir/bar.x
dir/baz.x

with that version it seems to work better:

--8<---------------cut here---------------start------------->8---
files_only = foo.x bar.x baz.x
files_with_path = `for f in $(files_only); do printf "dir/%s " $$f; done`

all:
@echo "$(files_with_path)"
--8<---------------cut here---------------end--------------->8---

$ make
dir/foo.x dir/bar.x dir/baz.x

HTH.
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37
Kip Warner
2017-09-02 00:25:19 UTC
Permalink
Post by Mathieu Lirzin
'files_with_path' will not be interpreted by Automake, it will just
copy the line in the generated Makefile. Since Make variables are
macros the content of the variable will be simply be copied by Make
where $(files_with_path) is used.  As a consequence it should only be
used in the recipe of a rule, not as a target or prerequisite.
Thanks Matt, but unfortunately I'll need to be able to use it as at
least a target, and possibly later as a prerequisite as well.
--
Kip Warner | Senior Software Engineer
OpenPGP signed/encrypted mail preferred
http://www.thevertigo.com
Quinn Grier
2017-08-28 18:03:31 UTC
Permalink
Post by Kip Warner
Hey list,
I'd like to transform the following variable in my Makefile.am from...
files_only = a.foo b.foo c.foo d.foo ...
Into...
files_with_path = dir/a.foo dir/b.foo dir/c.foo dir/d.foo ...
I'm aware of GNU Make's $addprefix magic. I'm also aware it's naughty
to use where portability is a concern and a different implementation of
make may be used behind Automake. I was wondering if anyone has come up
with a recipe for scenarios like this?
A portable way to do this is to do the work in configure.ac and transfer
it to the Makefile with AC_SUBST. For example, with an empty Makefile.am
and the following configure.ac:

AC_INIT([Example], [1.0])
AM_INIT_AUTOMAKE([foreign])

m4_define([m_files_only], [a.foo b.foo c.foo])
files_only='m_files_only'
files_with_path='m4_map_args_w(m_files_only, [dir/], [], [ ])'
AC_SUBST([files_only])
AC_SUBST([files_with_path])

AC_CONFIG_FILES([Makefile])
AC_OUTPUT

The desired macros will appear in the resulting Makefile:

files_only = a.foo b.foo c.foo
files_with_path = dir/a.foo dir/b.foo dir/c.foo
Kip Warner
2017-09-02 00:34:59 UTC
Permalink
Post by Quinn Grier
A portable way to do this is to do the work in configure.ac and transfer
it to the Makefile with AC_SUBST. For example, with an empty
Makefile.am
      AC_INIT([Example], [1.0])
      AM_INIT_AUTOMAKE([foreign])
      m4_define([m_files_only], [a.foo b.foo c.foo])
      files_only='m_files_only'
      files_with_path='m4_map_args_w(m_files_only, [dir/], [], [ ])'
      AC_SUBST([files_only])
      AC_SUBST([files_with_path])
      AC_CONFIG_FILES([Makefile])
      AC_OUTPUT
      files_only = a.foo b.foo c.foo
      files_with_path = dir/a.foo dir/b.foo dir/c.foo
Hey Quinn. This would have worked perfectly, except automake bails when
I run autoreconf:

autoreconf: running: automake --add-missing --copy --no-force
configure.ac:310: error: 'parser_clobbered_source_full_paths' includes configure substitution '@parser _clobbered_source_full_paths@'
configure.ac:310: and is referred to from 'nodist_narayan_designer_SOURCES';
configure.ac:310: configure substitutions are not allowed in _SOURCES variables
autoreconf: automake failed with exit status: 1

This is likely because as the message says, I am attempting to use the
substitution within a *_SOURCES variable.

Is this a dead end or is there a workaround? Assume a reference within
a _SOURCES variable is probably necessary.
--
Kip Warner | Senior Software Engineer
OpenPGP signed/encrypted mail preferred
http://www.thevertigo.com
Quinn Grier
2017-09-02 03:27:02 UTC
Permalink
Post by Kip Warner
Post by Quinn Grier
A portable way to do this is to do the work in configure.ac and transfer
it to the Makefile with AC_SUBST. For example, with an empty
Makefile.am
      AC_INIT([Example], [1.0])
      AM_INIT_AUTOMAKE([foreign])
      m4_define([m_files_only], [a.foo b.foo c.foo])
      files_only='m_files_only'
      files_with_path='m4_map_args_w(m_files_only, [dir/], [], [ ])'
      AC_SUBST([files_only])
      AC_SUBST([files_with_path])
      AC_CONFIG_FILES([Makefile])
      AC_OUTPUT
      files_only = a.foo b.foo c.foo
      files_with_path = dir/a.foo dir/b.foo dir/c.foo
Hey Quinn. This would have worked perfectly, except automake bails when
autoreconf: running: automake --add-missing --copy --no-force
configure.ac:310: and is referred to from 'nodist_narayan_designer_SOURCES';
configure.ac:310: configure substitutions are not allowed in _SOURCES variables
autoreconf: automake failed with exit status: 1
This is likely because as the message says, I am attempting to use the
substitution within a *_SOURCES variable.
Is this a dead end or is there a workaround? Assume a reference within
a _SOURCES variable is probably necessary.
You can fix this by generating the complete macro definitions before
Automake runs and including them in Makefile.am.

One way you can do this is with a shell script that you must run before
autoreconf. For example:

files_only='a.foo b.foo c.foo'
files_path='dir/'
files_with_path=
for f in $files_only; do
files_with_path=$files_with_path\ $files_path$f
done
echo files_only = \
$files_only >files_only.am || exit 1
echo files_with_path = \
$files_with_path >files_with_path.am || exit 1

And, inside Makefile.am, you add these lines:

include files_only.am
include files_with_path.am

This is a natural fit if you're already using an autogen.sh script, as
you can put this code in there. If you're not using one, then this would
essentially force you to start using one, as you need to remember to run
this code before autoreconf.

Another idea is to generate the .am files in configure.ac at M4 time,
which should cause autoreconf to generate them before it runs Automake.
In this case, an autogen.sh script is unnecessary.

Here is an example configure.ac that does this:

AC_INIT([Example], [1.0])
AM_INIT_AUTOMAKE([foreign])
AC_PROG_CC

m4_define([m_files_only], [[a.foo b.foo c.foo]])
m4_define([m_files_path], [[[dir/]]])
m4_define([m_files_with_path],
m4_dquote(m4_map_args_w(m_files_only,
m_files_path, [], [ ])))
m4_syscmd([echo files_only = ]dnl
m_files_only[ >files_only.am])
m4_assert(m4_sysval[ == 0])
m4_syscmd([echo files_with_path = ]dnl
m_files_with_path[ >files_with_path.am])
m4_assert(m4_sysval[ == 0])

AC_CONFIG_FILES([Makefile])
AC_OUTPUT

And here is the accompanying example Makefile.am:

include files_only.am
include files_with_path.am
bin_PROGRAMS = foo
foo_SOURCES = foo.c $(files_with_path)
Peter Johansson
2017-09-02 10:16:01 UTC
Permalink
Post by Quinn Grier
You can fix this by generating the complete macro definitions before
Automake runs and including them in Makefile.am.
Another way to do this is to use

macro `AX_AM_MACROS_STATIC' in Autoconf Macro Archive

https://www.gnu.org/software/autoconf-archive/ax_am_macros_static.html#ax_am_macros_static

but it can be a bit quirky to get correct

Cheers,
Peter
Kip Warner
2017-09-03 01:56:10 UTC
Permalink
Post by Peter Johansson
Post by Quinn Grier
You can fix this by generating the complete macro definitions before
Automake runs and including them in Makefile.am.
Another way to do this is to use
macro `AX_AM_MACROS_STATIC' in Autoconf Macro Archive
Thanks Peter and Quinn for your suggestions. Overall the variable I am
attempting to initialize is only suppose to contain four files in it,
so I reckon this is getting far more convoluted than is necessary at
this time. If it later changes to dozens or more, it will probably be a
good idea to revisit your combined suggestions.
--
Kip Warner | Senior Software Engineer
OpenPGP signed/encrypted mail preferred
http://www.thevertigo.com
Thomas Jahns
2017-09-04 08:56:44 UTC
Permalink
Post by Kip Warner
I'd like to transform the following variable in my Makefile.am from...
files_only = a.foo b.foo c.foo d.foo ...
Into...
files_with_path = dir/a.foo dir/b.foo dir/c.foo dir/d.foo ...
Can you give more context why you need to substitute on the left-hand-side here?
It's after all simply a Makefile-Variable, so I don't exactly see what's the
purpose.

Regards, Thomas

Loading...