Problem with VPATH builds and SCRIPTS primary

[Raphaël Halimi]

Hi,

I have a problem with parallel build trees (a.k.a. VPATH Builds).

I'm trying to use GNU autotools to distribute a shell script that need
to be installed with some other files (sample configuration, man pages,
data, etc etc).

Nothing in this project needs to be compiled, but the script is passed
through sed to replace some variables (as showed through examples in
section 9.1 of automake documentation, and section 4.8.2 of autoconf
documentation).

The provided examples work with recursive make, but I only want to
manage a single Makefile.am, and thus I want to use non-recursive make;
that's where my problem appears.

I'm trying to have the final script (the file where sed output is
redirected) in the same subdirectory as the script template, but I can't
find a way to do this that would let "make distcheck" work.

According to section 2.2.6 of GNU automake documentation:

"The build tree usually has the same subdirectory layout as the source
tree; its subdirectories are created automatically by the build system."

Unfortunately, it doesn't seem to be true for the SCRIPTS primary (I
assume it's true and works as expected for compiled binaries and
libraries, I can't know, I'm new to autotools and I'm a sysadmin, I
develop only scripts, nothing compiled).

So, at first, I did this :

-----%<-----

#
# Main script
#

pkgsysconfdir = $(sysconfdir)/$(PACKAGE)

bin_SCRIPTS = scripts/my_script

# Command to substitute variables
do_subst = $(SED) \
-e 's,[@]PACKAGE[@],$(PACKAGE),g' \
-e 's,[@]VERSION[@],$(VERSION),g' \
-e 's,[@]pkgsysconfdir[@],$(pkgsysconfdir),g' \
-e 's,[@]pkgdatadir[@],$(pkgdatadir),g'

# Generic recipe to substitute variables in scripts
$(bin_SCRIPTS): Makefile
$(do_subst) $(srcdir)/$@.in > $@
chmod +x $@

# Per-script prerequisites
scripts/my_script: $(srcdir)/scripts/my_script.in

----->%-----

This works for a normal build, but not with "make distcheck"; it fails
with the error message (translated from French):

/bin/bash: scripts/my_script: No such file or directory

I guess make doesn't create the "scripts" subdirectory in "_build/sub"
as it's supposed to (according to the documentation quoted above), so
the shell can't redirect sed's output to this file.

So for now I'm doing this:

-----%<-----

#
# Main script
#

pkgsysconfdir = $(sysconfdir)/$(PACKAGE)

bin_SCRIPTS = my_script

# Command to substitute variables
do_subst = $(SED) \
-e 's,[@]PACKAGE[@],$(PACKAGE),g' \
-e 's,[@]VERSION[@],$(VERSION),g' \
-e 's,[@]pkgsysconfdir[@],$(pkgsysconfdir),g' \
-e 's,[@]pkgdatadir[@],$(pkgdatadir),g'

# Generic recipe to substitute variables in scripts
$(bin_SCRIPTS): Makefile
$(do_subst) $(srcdir)/scripts/$@.in > $@
chmod +x $@

# Per-script prerequisites
my_script: scripts/my_script.in

----->%-----

It works with both a normal build and "make distcheck", but the script
ends up in $(srcdir), instead of the "scripts" subdirectory, which is
not my initial goal.

Is there a way to create the script in the "scripts" subdirectory, *and*
make it work with "make distcheck" ?

Thanks in advance.

Regards,

[Russ Allbery]

Post by Raphaël Halimi
"The build tree usually has the same subdirectory layout as the source
tree; its subdirectories are created automatically by the build system."
Unfortunately, it doesn't seem to be true for the SCRIPTS primary (I
assume it's true and works as expected for compiled binaries and
libraries, I can't know, I'm new to autotools and I'm a sysadmin, I
develop only scripts, nothing compiled).

I've run into this problem before, and just had Autoconf create the
directory by adding:

AC_CONFIG_COMMANDS([server], [test -d server || mkdir server])

(in my case, the name of the directory containing the scripts was server).
It's kind of a hack, but it works.

[Raphaël Halimi]

Post by Russ Allbery

Post by Raphaël Halimi
"The build tree usually has the same subdirectory layout as the source
tree; its subdirectories are created automatically by the build system."
Unfortunately, it doesn't seem to be true for the SCRIPTS primary (I
assume it's true and works as expected for compiled binaries and
libraries, I can't know, I'm new to autotools and I'm a sysadmin, I
develop only scripts, nothing compiled).

I've run into this problem before, and just had Autoconf create the
AC_CONFIG_COMMANDS([server], [test -d server || mkdir server])
(in my case, the name of the directory containing the scripts was server).
It's kind of a hack, but it works.

Thanks, this trick worked.

On a more general note, I don't understand why the SCRIPTS primary is so
much "inferior" to the PROGRAMS one; the latter automatically takes care
of files to be distributed or cleaned, and of subdirectories creation,
whereas the former needs those files to be explicitly included in
EXTRA_DIST and CLEANFILES, and the subdirectories to be manually created.

How comes ? Is there a technical limitation that makes those things
impossible (or too hard) to guess for scripts ?

Regards,

[Peter Johansson]

Hi Raphael,

I have the following in my Makefile.am


CLEANFILES += data/.dirstamp

data/.dirstamp:
@$(MKDIR_P) data
@: > $@


data/foo.txt: data/.dirstamp
<some rule>

Cheers,
Peter

Post by Raphaël Halimi
Hi,
I have a problem with parallel build trees (a.k.a. VPATH Builds).
I'm trying to use GNU autotools to distribute a shell script that need
to be installed with some other files (sample configuration, man pages,
data, etc etc).
Nothing in this project needs to be compiled, but the script is passed
through sed to replace some variables (as showed through examples in
section 9.1 of automake documentation, and section 4.8.2 of autoconf
documentation).
The provided examples work with recursive make, but I only want to
manage a single Makefile.am, and thus I want to use non-recursive make;
that's where my problem appears.
I'm trying to have the final script (the file where sed output is
redirected) in the same subdirectory as the script template, but I can't
find a way to do this that would let "make distcheck" work.
"The build tree usually has the same subdirectory layout as the source
tree; its subdirectories are created automatically by the build system."
Unfortunately, it doesn't seem to be true for the SCRIPTS primary (I
assume it's true and works as expected for compiled binaries and
libraries, I can't know, I'm new to autotools and I'm a sysadmin, I
develop only scripts, nothing compiled).
-----%<-----
#
# Main script
#
pkgsysconfdir = $(sysconfdir)/$(PACKAGE)
bin_SCRIPTS = scripts/my_script
# Command to substitute variables
do_subst = $(SED) \
# Generic recipe to substitute variables in scripts
$(bin_SCRIPTS): Makefile
# Per-script prerequisites
scripts/my_script: $(srcdir)/scripts/my_script.in
----->%-----
This works for a normal build, but not with "make distcheck"; it fails
/bin/bash: scripts/my_script: No such file or directory
I guess make doesn't create the "scripts" subdirectory in "_build/sub"
as it's supposed to (according to the documentation quoted above), so
the shell can't redirect sed's output to this file.
-----%<-----
#
# Main script
#
pkgsysconfdir = $(sysconfdir)/$(PACKAGE)
bin_SCRIPTS = my_script
# Command to substitute variables
do_subst = $(SED) \
# Generic recipe to substitute variables in scripts
$(bin_SCRIPTS): Makefile
# Per-script prerequisites
my_script: scripts/my_script.in
----->%-----
It works with both a normal build and "make distcheck", but the script
ends up in $(srcdir), instead of the "scripts" subdirectory, which is
not my initial goal.
Is there a way to create the script in the "scripts" subdirectory, *and*
make it work with "make distcheck" ?
Thanks in advance.
Regards,

[Raphaël Halimi]

Hi Peter,

Thanks for your answer.

Post by Peter Johansson
CLEANFILES += data/.dirstamp
@$(MKDIR_P) data
@: > $@

This creates a directory "data", and then a file ".dirstamp" in this
directory by redirecting the (empty) result of the "true" command to
this file.

Post by Peter Johansson
data/foo.txt: data/.dirstamp
<some rule>

This creates a file "foo.txt" (in my case it would be a shell script) in
the "data" directory according to a some rule.

Unlike the other solution, this doesn't involve configure.ac, and so
it's usable with a "pure" automake setup (without autoconf). This is not
the case of my current setup but still, I see this as an advantage, to
clearly separate the "configure" phase and the "make" phase (I'm still
new to autotools).

Are there other pros/cons I didn't see, compared to the creation of the
directory with AC_CONFIG_COMMANDS ?

Regards,

[Peter Johansson]

Post by Raphaël Halimi
Hi Peter,
Thanks for your answer.

Post by Peter Johansson
CLEANFILES += data/.dirstamp
@$(MKDIR_P) data
@: > $@

This creates a directory "data", and then a file ".dirstamp" in this
directory by redirecting the (empty) result of the "true" command to
this file.

Yes, I stole this rule from the rule generated by Automake when it
creates files in subdirs.

Post by Raphaël Halimi

Post by Peter Johansson
data/foo.txt: data/.dirstamp
<some rule>

This creates a file "foo.txt" (in my case it would be a shell script) in
the "data" directory according to a some rule.

Correct

Post by Raphaël Halimi
Unlike the other solution, this doesn't involve configure.ac, and so
it's usable with a "pure" automake setup (without autoconf). This is not
the case of my current setup but still, I see this as an advantage, to
clearly separate the "configure" phase and the "make" phase (I'm still
new to autotools).

Yes, I prefer to have the code for this in Makefile.am (rather than
configure.ac); it's the natural place to look when 'make' doesn't work
and easier for new developers to understand, imho.

Cheers,
Peter