Discussion:
copyright problem with install-sh, request for clean-room rewrite
Eric Blake
2018-09-03 14:13:13 UTC
Permalink
There has been concern expressed in another (non-public) list that the
AC_PROG_INSTALL macro in autoconf may be causing unnecessary grief to
downstream packagers. This macro requires the existence of an auxiliary
program named 'install-sh' or the older spelling 'install.sh', although
autoconf itself does not install it, and merely recommends that a user
either grab the copy from Autoconf's sources or from Automake (where
Autoconf's copy came from Automake).

Furthermore, when using 'automake --install' (which can be triggered by
Autoconf's 'autoreconf -i'), Automake will install a copy of
'install-sh' that was originally written by the X Consortium. While the
license on that file is permissive enough to be included in the GPL,
there is concern that it might be at odds with projects that use the
Autotools but are relying on the GPLv3 exception that allows the use of
Autotool-provided output in order to license a project under a different
license than GPL.

I am not a lawyer, and do not have an opinion on whether the concern is
valid or spurious - however, I _do_ want to make life as easy for
downstream projects to use Autoconf in such a way that they have a
minimum of legal hurdles to clear, so anything we can do to alleviate
the concern, whether or not the concern has any legal standing, is still
a wise course of action.

On the other hand, note that install-sh is only ever copied verbatim
into a project, and does NOT form the basis of any of the source code
compiled into a final binary, but is ONLY used as a 'make install'
helper script to make installing the final binary easier to do without
relying on the existence of programs that the GNU Coding Standards does
not state to be universally portable. On that grounds, its presence in
a tarball does not change the license of the final binaries created by
compiling and installing the tarball, but MAY be grounds to list that
distinct license/copyright in a manifest describing the copyright of all
files included in the tarball. So on that front alone, we are indeed
making life more complicated for someone trying to generate such a
package manifest, when we recommend the copying of a non-FSF file.

Note that the most recent version of 'install-sh' as installed by
Automake states:
# Copyright (C) 1994 X Consortium

Meanwhile, the autoconf manual states:
Of the other files that might be used with @command{configure},
@file{config.h.in} is under whatever copyright you use for your
@file{configure.ac}. @file{config.sub} and @file{config.guess} have an
exception to the GPL when they are used with an Autoconf-generated
@command{configure} script, which permits you to distribute them under the
same terms as the rest of your package. @file{install-sh} is from the X
Consortium and is not copyrighted.

This is contradictory, and needs to be fixed in the Autoconf manual.
However, the question on the floor is whether the fix is merely
rewording that sentence to describe the actual copyright and permissive
license, or if a better fix might be possible, such as rewriting
'install-sh' from scratch so that we can ship a version of the file that
is completely copyright FSF and therefore uses the identical
GPLv3+exception as the rest of the files we care about, rather than
having to play the legal games of whether the X Consortium copy
introduces any wrinkles in the first place.

Is anyone willing to take on a clean-room reimplementation of the
functionality of install-sh? If so, I can write up a list of
requirements for what your implementation must provide; you must write a
shell script that has that functionality, and without using any
reference to the existing install-sh file, and where your work can have
copyright assigned to FSF; but you can use GNU Coreutils' install
program for a reference on functionality questions (I consider myself
ineligible for such a clean-room reimplementation, since I am one of the
people that has already made edits to the 'install-sh' belonging to
Automake, but I have no qualms in reviewing your work for accuracy and
portability).

For further information, we can look at how the file appears now
compared to the version we originally lifted from X11R5 back in Nov
1995, and ignoring whitespace changes (the original file used TAB, the
current version uses space):

$ cd automake.git
$ git log 042e80a7 -1
commit 042e80a71210ad691473fd7f37ff71c9c376a129
Author: Tom Tromey <***@redhat.com>
Date: Sun Nov 12 23:01:53 1995 +0000

Initial revision
$ git rev-parse --short=8 HEAD
a348d830
$ git diff -w -b 042e80a712..HEAD lib/install-sh | grep @@
@@ -1,238 +1,518 @@

So right away, we see that nearly the entire file has been rewritten in
some form or another over time by contributors that have all assigned
copyright to the FSF, as the differences form a single hunk. What's
more, we can now review which lines remain unchanged from our original
copy into the current age, and see that it is mostly comments or bare
control flow, which on its own doesn't do much:

$ git diff -w -b 042e80a712..HEAD lib/install-sh | sed 's/^[^ ].*//' |
cat -s

#!/bin/sh

# install - install a program, script, or datafile

#
# Calling this script install-sh is preferred over install.sh, to prevent

# when there is no Makefile.
#
# This script is compatible with the BSD install script, but was written
# from scratch.

rmcmd="$rmprog -f"

case $1 in

-g) chgrpcmd="$chgrpprog $2"

shift

fi

esac
done

exit 1

else

fi

dst=$src

else

# might cause directories to be created, which would be especially bad
# if $src (and thus $dsttmp) contains '*'.

exit 1
fi

exit 1

else

fi

else

fi

shift

else

fi

done

fi

fi

# If any of these fail, we abort the whole thing. If we want to
# ignore errors from any of these, just make sure not to ignore

# Now rename the file to the real destination.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
Thomas Dickey
2018-09-03 15:35:03 UTC
Permalink
On Mon, Sep 03, 2018 at 09:13:13AM -0500, Eric Blake wrote:
...
Note that the most recent version of 'install-sh' as installed by Automake
# Copyright (C) 1994 X Consortium
looking at my collection of untarred X sources, there's an issue with that:

The comment just before reads something like

# This originates from X11R5 (mit/util/scripts/install.sh), which was
# later released in X11R6 (xc/config/util/install.sh) with the
# following copyright and license.

however -

the first mention of copyright for any version of that file in X
sources was in 2003 (in xterm's sources), citing 1991. xterm has
the 1994 date from autoconf (i.e.,. a secondhand report).

X's sources don't have that file...

The X11R5 file has this comment (no copyright notice):

#
# install - install a program, script, or datafile
#
# $XConsortium: install.sh,v 1.2 89/12/18 14:47:22 jim Exp $
#
# This script is compatible with the BSD install script, but was written
# from scratch.
#

There's an additional twist: Thomas Roell made a script
mit/server/ddx/x386/etc/install.sh derived from the X11R5
mit/util/scripts/install.sh, adding a copyright notice:

# Copyright 1990,91 by Thomas Roell, Dinkelscherben, Germany.

With all that, the available information shows that someone "fixed" things
in the early 2000s by adding a copyright date to address the derived work.

Perhaps autoconf's source repo has additional information.
As is, the copyright was apparently not applied by the owner of the file.
--
Thomas E. Dickey <***@invisible-island.net>
https://invisible-island.net
ftp://ftp.invisible-island.net
Eric Blake
2018-09-03 20:20:14 UTC
Permalink
Post by Thomas Dickey
...
Note that the most recent version of 'install-sh' as installed by Automake
# Copyright (C) 1994 X Consortium
Thanks for digging up more history.
Post by Thomas Dickey
With all that, the available information shows that someone "fixed" things
in the early 2000s by adding a copyright date to address the derived work.
Git blame points to Automake commit 7bb3bbe6e:

commit 7bb3bbe6e2560604a2f925fec6ff7b4afd74e180
Author: Alexandre Duret-Lutz <***@gnu.org>
Date: Fri May 9 17:58:21 2003 +0000

* lib/install-sh: Update copyright notice and license to that of
X11R6. This removes an advertising clause reported as Debian bug
#191717.

http://bugs.debian.org/191717 is an interesting read, pointing out that
versions of automake prior to that point had:

-# Copyright 1991 by the Massachusetts Institute of Technology
-# (FSF changes in the public domain.)
[with an advertising clause]

before what now reads as:

+# Copyright (C) 1994 X Consortium
[with no advertising clause]

In turn, that copyright line first appeared in commit 4314bddf in Jul
1996, prior to which point it was indeed shipped without a copyright line.
Post by Thomas Dickey
Perhaps autoconf's source repo has additional information.
Looking there, it has moved over time (from the top level, to
config/install-sh, to now build-aux/config.sh). The initial commit
there in 1994 indeed had no copyright (see commit 430e0b6), and the MIT
copyright notice was added in Jul 1996 (commit ce6c1f20) - looks like
autoconf and automake were kept relatively close in sync at that time,
and only later with commit 383913b4 in Sep 1998 did autoconf start
syncing install-sh from a different location rather than maintaining it
in parallel with automake.
Post by Thomas Dickey
As is, the copyright was apparently not applied by the owner of the file.
So the sentence in the autoconf manual was, at least at one point in
history, correct about the file not carrying a copyright; but is now out
of date based on what subsequent modifications have pulled in (although
tracking actual copyright may be a lot harder than just doing a
clean-room rewrite).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
Mathieu Lirzin
2018-09-03 16:03:58 UTC
Permalink
Hello Eric,
Post by Eric Blake
This is contradictory, and needs to be fixed in the Autoconf
manual. However, the question on the floor is whether the fix is
merely rewording that sentence to describe the actual copyright and
permissive license, or if a better fix might be possible, such as
rewriting 'install-sh' from scratch so that we can ship a version of
the file that is completely copyright FSF and therefore uses the
identical GPLv3+exception as the rest of the files we care about,
rather than having to play the legal games of whether the X Consortium
copy introduces any wrinkles in the first place.
Given the small size of this script it seems a reasonable option to
rewrite it from scratch, to avoid the license mess.
Post by Eric Blake
Is anyone willing to take on a clean-room reimplementation of the
functionality of install-sh? If so, I can write up a list of
requirements for what your implementation must provide; you must write
a shell script that has that functionality, and without using any
reference to the existing install-sh file, and where your work can
have copyright assigned to FSF; but you can use GNU Coreutils' install
program for a reference on functionality questions (I consider myself
ineligible for such a clean-room reimplementation, since I am one of
the people that has already made edits to the 'install-sh' belonging
to Automake, but I have no qualms in reviewing your work for accuracy
and portability).
According to ‘git blame’ I appear to not have touch this file, so if you
think that I am eligible, I am volunteering on this rewriting task.
Your help regarding ‘install-sh’ requirements analysis and code review
would indeed be welcome. Regarding the requirements I guess Automake
test suite already contains some tests validating some of them.

WDYT?
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37
Paul Eggert
2018-09-03 18:24:32 UTC
Permalink
Post by Mathieu Lirzin
According to ‘git blame’ I appear to not have touch this file, so if you
think that I am eligible, I am volunteering on this rewriting task.
Thanks for volunteering. Eric, do you think it would save time overall if you
sent him the part of install-sh that you are sure is *not* problematic, i.e.,
the part that is already copyright by the FSF? That's typical procedure in a
cleanroom rewrite of anything large; dunno if it's overkill here.
Thomas Dickey
2018-09-03 21:11:43 UTC
Permalink
----- Original Message -----
| From: "Paul Eggert" <***@cs.ucla.edu>
| To: "Mathieu Lirzin" <***@gnu.org>, "Eric Blake" <***@redhat.com>
| Cc: bug-***@gnu.org, "automake" <***@gnu.org>
| Sent: Monday, September 3, 2018 2:24:32 PM
| Subject: Re: copyright problem with install-sh, request for clean-room rewrite

| Mathieu Lirzin wrote:
|> According to ‘git blame’ I appear to not have touch this file, so if you
|> think that I am eligible, I am volunteering on this rewriting task.
|
| Thanks for volunteering. Eric, do you think it would save time overall if you
| sent him the part of install-sh that you are sure is *not* problematic, i.e.,
| the part that is already copyright by the FSF? That's typical procedure in a
| cleanroom rewrite of anything large; dunno if it's overkill here.

If it was an issue for me, I'd rewrite it.

fwiw, the copyright was added here -

http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=commit;h=ce6c1f203a5052cda0fdf642e9b462950d954a1d
--
Thomas E. Dickey <***@invisible-island.net>
http://invisible-island.net
ftp://ftp.invisible-island.net
Thomas Dickey
2018-09-03 21:21:41 UTC
Permalink
----- Original Message -----
| From: "Thomas Dickey" <***@his.com>
| To: "Paul Eggert" <***@cs.ucla.edu>
| Cc: "Mathieu Lirzin" <***@gnu.org>, "Eric Blake" <***@redhat.com>, "bug-autoconf" <bug-***@gnu.org>, "automake"
| <***@gnu.org>
| Sent: Monday, September 3, 2018 5:11:43 PM
| Subject: Re: copyright problem with install-sh, request for clean-room rewrite

| ----- Original Message -----
|| From: "Paul Eggert" <***@cs.ucla.edu>
|| To: "Mathieu Lirzin" <***@gnu.org>, "Eric Blake" <***@redhat.com>
|| Cc: bug-***@gnu.org, "automake" <***@gnu.org>
|| Sent: Monday, September 3, 2018 2:24:32 PM
|| Subject: Re: copyright problem with install-sh, request for clean-room rewrite
|
|| Mathieu Lirzin wrote:
||> According to ‘git blame’ I appear to not have touch this file, so if you
||> think that I am eligible, I am volunteering on this rewriting task.
||
|| Thanks for volunteering. Eric, do you think it would save time overall if you
|| sent him the part of install-sh that you are sure is *not* problematic, i.e.,
|| the part that is already copyright by the FSF? That's typical procedure in a
|| cleanroom rewrite of anything large; dunno if it's overkill here.
|
| If it was an issue for me, I'd rewrite it.
|
| fwiw, the copyright was added here -
|
| http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=commit;h=ce6c1f203a5052cda0fdf642e9b462950d954a1d

(for 1991). 1994 came in here:

http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=blobdiff;f=config/install-sh;h=e4f003959e284d64b8cddff996f3248a38edcb6a;hp=36f96f3e033cfaeefc092147d99d92f78e030d12;hb=18ca9ab3ec1a9268c70e1d65644e7db0cac7dab9;hpb=f16c8d206e87a41e74f11da93165e95022cbc9eb

(either way, the copyright date is bogus).
--
Thomas E. Dickey <***@invisible-island.net>
http://invisible-island.net
ftp://ftp.invisible-island.net
Eric Blake
2018-09-03 20:41:33 UTC
Permalink
Post by Paul Eggert
Post by Mathieu Lirzin
According to ‘git blame’ I appear to not have touch this file, so if you
think that I am eligible, I am volunteering on this rewriting task.
Thanks for volunteering.
Indeed, and I think you are reasonably safe for the task.
Post by Paul Eggert
Eric, do you think it would save time overall
if you sent him the part of install-sh that you are sure is *not*
problematic, i.e., the part that is already copyright by the FSF? That's
typical procedure in a cleanroom rewrite of anything large; dunno if
it's overkill here.
One thing that is for certain copyright by the FSF is the fact that
'install-sh --help' outputs a useful summary (as the original install-sh
merely reports "install: --help does not exist"). So here's a nice
starting point:

$ lib/install-sh --help
Usage: lib/install-sh [OPTION]... [-T] SRCFILE DSTFILE
or: lib/install-sh [OPTION]... SRCFILES... DIRECTORY
or: lib/install-sh [OPTION]... -t DIRECTORY SRCFILES...
or: lib/install-sh [OPTION]... -d DIRECTORIES...

In the 1st form, copy SRCFILE to DSTFILE.
In the 2nd and 3rd, copy all SRCFILES to DIRECTORY.
In the 4th, create DIRECTORIES.

Options:
--help display this help and exit.
--version display version info and exit.

-c (ignored)
-C install only if different (preserve the last data
modification time)
-d create directories instead of installing files.
-g GROUP chgrp installed files to GROUP.
-m MODE chmod installed files to MODE.
-o USER chown installed files to USER.
-s strip installed files.
-t DIRECTORY install into DIRECTORY.
-T report an error if DSTFILE is a directory.

Environment variables override the default commands:
CHGRPPROG CHMODPROG CHOWNPROG CMPPROG CPPROG MKDIRPROG MVPROG
RMPROG STRIPPROG


Other black-box testing that I can readily describe is based on how
automake uses the script (as those are the command line invocations that
must work; it may be easier to start with a rewritten script that is
less full-featured than what the current one does, as long as it
provides everything that automake actually uses it for):

install_sh_DATA = $(install_sh) -c -m 644
install_sh_PROGRAM = $(install_sh) -c
install_sh_SCRIPT = $(install_sh) -c
...
mkinstalldirs = $(install_sh) -d
...
INSTALL_STRIP_PROGRAM = $(install_sh) -c -s
...
-test -n "$(am__skip_mode_fix)" \
|| find "$(distdir)" -type d ! -perm -755 \
-exec chmod u+rwx,go+rx {} \; -o \
! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
! -type d ! -perm -400 -exec chmod a+r {} \; -o \
! -type d ! -perm -444 -exec $(install_sh) -c -m a+r {} {} \; \
|| chmod -R a+r "$(distdir)"

...
install-strip:
if test -z '$(STRIP)'; then \
$(MAKE) $(AM_MAKEFLAGS)
INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \
install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)"
INSTALL_STRIP_FLAG=-s \
install; \
else \
$(MAKE) $(AM_MAKEFLAGS)
INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \
install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)"
INSTALL_STRIP_FLAG=-s \
"INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \
fi
...


Comparing to GNU coreutils, which also ignores -c, I have no idea why
Automake's snippets that use $(install_sh) pass -c; and while I see a
definite use of -d, -m, -s, and $STRIPPROG, I don't see an obvious use
of -g, -o, -C, or some of the other environment variables, at least for
what automake itself expects to exist (although it's harder to predict
what functionality other projects have come to rely on in their
Makefile.am usage of install-sh).

I also note that 'lib/mkinstalldirs' has a different copyright/license
(namely, "Public domain"), where I don't know if we need to rewrite that
one as well; and find it odd that we also have a line in generated
Makefiles that uses 'install-sh -d' in place of mkinstalldirs. It may
make sense to have automake quit shipping mkinstalldirs, and merely rely
on $(MKDIR_P) and/or 'install-sh -d', for one less helper script
installed by automake.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
Loading...