Discussion:
[PATCH] port elisp-compilation support to emacs-23.1 and newer
Jim Meyering
2017-11-23 15:38:57 UTC
Permalink
I wanted to make a new idutils release, but was blocked because
its "make distcheck" would fail. That was because it distributes
and builds from an elisp file, and automake's elisp-compilation
rule uses a function that was marked obsolete back in 2009.
Upstream Emacs finally removed support for that function in May,
so anyone using emacs built since then will see the same failure
I saw. It also strikes whenever building from a read-only source
directory.

This change switches the build command to use the "new" way.

I started discussion on emacs-devel last night:
https://lists.gnu.org/archive/html/emacs-devel/2017-11/msg00551.html


From ecad5844100d5193ecd58f66f31f6bbf0ef04e23 Mon Sep 17 00:00:00 2001
From: Jim Meyering <***@fb.com>
Date: Wed, 22 Nov 2017 21:07:29 -0800
Subject: [PATCH] port elisp-compilation support to emacs-23.1 and newer

In May of 2017, support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory. In emacs-23.1 (July 2009) byte-compile-dest-file-function
became the recommended way to adjust the byte-compiler's destination.
* lib/am/lisp.am (.el.elc): Use byte-compile-dest-file-function,
rather than byte-compile-dest-file.
* t/lisp-readonly-srcdir.sh: New file, to test for the above.
* t/list-of-tests.mk (handwritten_TESTS): Add it.
---
lib/am/lisp.am | 2 +-
t/lisp-readonly-srcdir.sh | 46 ++++++++++++++++++++++++++++++++++++++++++++++
t/list-of-tests.mk | 1 +
3 files changed, 48 insertions(+), 1 deletion(-)
create mode 100644 t/lisp-readonly-srcdir.sh

diff --git a/lib/am/lisp.am b/lib/am/lisp.am
index 881bf3457..cacbc6feb 100644
--- a/lib/am/lisp.am
+++ b/lib/am/lisp.am
@@ -41,7 +41,7 @@ endif %?INSTALL%
$(EMACS) --batch \
$(AM_ELCFLAGS) $(ELCFLAGS) \
$$am__subdir_includes -L $(builddir) -L $(srcdir) \
- --eval "(defun byte-compile-dest-file (f) \"$@\")" \
+ --eval "(setq byte-compile-dest-file-function (lambda (_) \"$@\"))" \
--eval "(unless (byte-compile-file \"$<\") (kill-emacs 1))"; \
else :; fi

diff --git a/t/lisp-readonly-srcdir.sh b/t/lisp-readonly-srcdir.sh
new file mode 100644
index 000000000..38b866404
--- /dev/null
+++ b/t/lisp-readonly-srcdir.sh
@@ -0,0 +1,46 @@
+#! /bin/sh
+# Copyright (C) 2017 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+# Ensure that building elisp from a read-only srcdir works.
+
+required=emacs
+. test-init.sh
+
+cat > Makefile.am << 'EOF'
+lisp_LISP = am-one.el
+EOF
+
+cat >> configure.ac << 'EOF'
+AM_PATH_LISPDIR
+AC_OUTPUT
+EOF
+
+echo > am-one.el
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+
+mkdir sub
+chmod a=rx .
+
+cd sub
+../configure
+$MAKE
+
+test -f am-one.elc
+
+:
diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk
index d234aef48..3dab63d32 100644
--- a/t/list-of-tests.mk
+++ b/t/list-of-tests.mk
@@ -656,6 +656,7 @@ t/lisp5.sh \
t/lisp6.sh \
t/lisp7.sh \
t/lisp8.sh \
+t/lisp-readonly-srcdir.sh \
t/lisp-loadpath.sh \
t/lisp-subdir.sh \
t/lisp-subdir2.sh \
--
2.14.1.729.g59c0ea183
Jim Meyering
2017-11-23 21:57:58 UTC
Permalink
Post by Jim Meyering
I wanted to make a new idutils release, but was blocked because
its "make distcheck" would fail. That was because it distributes
and builds from an elisp file, and automake's elisp-compilation
rule uses a function that was marked obsolete back in 2009.
Upstream Emacs finally removed support for that function in May,
so anyone using emacs built since then will see the same failure
I saw. It also strikes whenever building from a read-only source
directory.
This change switches the build command to use the "new" way.
https://lists.gnu.org/archive/html/emacs-devel/2017-11/msg00551.html
From ecad5844100d5193ecd58f66f31f6bbf0ef04e23 Mon Sep 17 00:00:00 2001
Date: Wed, 22 Nov 2017 21:07:29 -0800
Subject: [PATCH] port elisp-compilation support to emacs-23.1 and newer
In May of 2017, support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory. In emacs-23.1 (July 2009) byte-compile-dest-file-function
became the recommended way to adjust the byte-compiler's destination.
* lib/am/lisp.am (.el.elc): Use byte-compile-dest-file-function,
rather than byte-compile-dest-file.
* t/lisp-readonly-srcdir.sh: New file, to test for the above.
* t/list-of-tests.mk (handwritten_TESTS): Add it.
Pushed to the micro branch:

https://git.savannah.gnu.org/cgit/automake.git/commit/?h=micro&id=9182df7e4810a411147d523de8cd141e749c5e39
Mathieu Lirzin
2017-11-23 23:57:35 UTC
Permalink
Hello Jim,
Post by Jim Meyering
Post by Jim Meyering
I wanted to make a new idutils release, but was blocked because
its "make distcheck" would fail. That was because it distributes
and builds from an elisp file, and automake's elisp-compilation
rule uses a function that was marked obsolete back in 2009.
Upstream Emacs finally removed support for that function in May,
so anyone using emacs built since then will see the same failure
I saw. It also strikes whenever building from a read-only source
directory.
This change switches the build command to use the "new" way.
https://lists.gnu.org/archive/html/emacs-devel/2017-11/msg00551.html
From ecad5844100d5193ecd58f66f31f6bbf0ef04e23 Mon Sep 17 00:00:00 2001
Date: Wed, 22 Nov 2017 21:07:29 -0800
Subject: [PATCH] port elisp-compilation support to emacs-23.1 and newer
In May of 2017, support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory. In emacs-23.1 (July 2009) byte-compile-dest-file-function
became the recommended way to adjust the byte-compiler's destination.
* lib/am/lisp.am (.el.elc): Use byte-compile-dest-file-function,
rather than byte-compile-dest-file.
* t/lisp-readonly-srcdir.sh: New file, to test for the above.
* t/list-of-tests.mk (handwritten_TESTS): Add it.
https://git.savannah.gnu.org/cgit/automake.git/commit/?h=micro&id=9182df7e4810a411147d523de8cd141e749c5e39
With the "recent" change in Automake branch naming scheme, 'master'
seems a better fit for this:

https://lists.gnu.org/archive/html/bug-automake/2017-09/msg00015.html

Thanks.
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37
Jim Meyering
2017-11-24 06:51:45 UTC
Permalink
Post by Mathieu Lirzin
Hello Jim,
Post by Jim Meyering
Post by Jim Meyering
I wanted to make a new idutils release, but was blocked because
its "make distcheck" would fail. That was because it distributes
and builds from an elisp file, and automake's elisp-compilation
rule uses a function that was marked obsolete back in 2009.
Upstream Emacs finally removed support for that function in May,
so anyone using emacs built since then will see the same failure
I saw. It also strikes whenever building from a read-only source
directory.
This change switches the build command to use the "new" way.
https://lists.gnu.org/archive/html/emacs-devel/2017-11/msg00551.html
From ecad5844100d5193ecd58f66f31f6bbf0ef04e23 Mon Sep 17 00:00:00 2001
Date: Wed, 22 Nov 2017 21:07:29 -0800
Subject: [PATCH] port elisp-compilation support to emacs-23.1 and newer
In May of 2017, support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory. In emacs-23.1 (July 2009) byte-compile-dest-file-function
became the recommended way to adjust the byte-compiler's destination.
* lib/am/lisp.am (.el.elc): Use byte-compile-dest-file-function,
rather than byte-compile-dest-file.
* t/lisp-readonly-srcdir.sh: New file, to test for the above.
* t/list-of-tests.mk (handwritten_TESTS): Add it.
https://git.savannah.gnu.org/cgit/automake.git/commit/?h=micro&id=9182df7e4810a411147d523de8cd141e749c5e39
With the "recent" change in Automake branch naming scheme, 'master'
https://lists.gnu.org/archive/html/bug-automake/2017-09/msg00015.html
Thanks.
Hi Mathieu,
Happy to adjust. Would you prefer that I merge micro into master,
then... or something else? Then delete micro? When I noticed that I'd
created that branch (after reading the description in HACKING), I
figured I'd missed something.
Mathieu Lirzin
2017-11-24 12:03:17 UTC
Permalink
Post by Jim Meyering
Post by Mathieu Lirzin
Post by Jim Meyering
https://git.savannah.gnu.org/cgit/automake.git/commit/?h=micro&id=9182df7e4810a411147d523de8cd141e749c5e39
With the "recent" change in Automake branch naming scheme, 'master'
https://lists.gnu.org/archive/html/bug-automake/2017-09/msg00015.html
Thanks.
Hi Mathieu,
Happy to adjust. Would you prefer that I merge micro into master,
then... or something else? Then delete micro? When I noticed that I'd
created that branch (after reading the description in HACKING), I
figured I'd missed something.
Indeed HACKING is not up-to-date, I will fix that.

Cherry-picking the commit and deleting 'micro' would be nice. Moreover
I think it worths adding a NEWS entry for this bug fix, if you agree
please add it. :-)

The fix will be released in 1.16. I was planning to make this release
by the end of the year, but given that I am quite busy at the university
I think i won't be able to make it before January/February.

Thanks for fixing this issue.
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37
Mathieu Lirzin
2017-11-24 12:42:33 UTC
Permalink
Post by Mathieu Lirzin
Indeed HACKING is not up-to-date, I will fix that.
Here is a patch that should help describing the new branching model more
accurately. If you see further improvements or would prefer different
wording, tell me.
Jim Meyering
2017-11-27 01:53:03 UTC
Permalink
Post by Mathieu Lirzin
Post by Mathieu Lirzin
Indeed HACKING is not up-to-date, I will fix that.
Here is a patch that should help describing the new branching model more
accurately. If you see further improvements or would prefer different
wording, tell me.
Thanks.
That looks fine.

By the way, I'm waiting to hear from Emacs folks to determine how to
word the NEWS entry for my master-destined elisp-compilation-fix
commit.
Glenn Morris
2017-11-27 18:27:25 UTC
Permalink
Post by Jim Meyering
In May of 2017, support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory.
In general, Emacs expects .el and .elc to be found in the same
directory. Not adhering to this convention will likely break various
Emacs features. Is this really something automake needs to enable at all?
Jim Meyering
2017-11-27 20:52:48 UTC
Permalink
Post by Glenn Morris
Post by Jim Meyering
In May of 2017, support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory.
In general, Emacs expects .el and .elc to be found in the same
directory. Not adhering to this convention will likely break various
Emacs features. Is this really something automake needs to enable at all?
An alternative would be to copy-or-link the .el file into the
destination directory. Indeed. That would work without breaking pre-23
emacs, so I will adjust my automake patch before pushing it to master.

Thanks.

However, please do consider undoing that breaking change before the
next emacs release, so we have a chance to release a fixed version of
automake before you remove the functionality being used in all
existing Makefile.in files.
Jim Meyering
2017-11-28 04:12:52 UTC
Permalink
Post by Jim Meyering
Post by Glenn Morris
Post by Jim Meyering
In May of 2017, support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory.
In general, Emacs expects .el and .elc to be found in the same
directory. Not adhering to this convention will likely break various
Emacs features. Is this really something automake needs to enable at all?
An alternative would be to copy-or-link the .el file into the
destination directory. Indeed. That would work without breaking pre-23
emacs, so I will adjust my automake patch before pushing it to master.
Hi Glenn,

I've thought about this some more and do not like the idea of
requiring automake's elisp-compilation rule to make a copy of the
source file in the destination directory in this slightly contrived
case. Remember: this arises only in a non-srcdir build. That means
build artifacts end up being written into the mostly-empty current
directory hierarchy, which does not have copies of the sources.
Installation processes will continue to copy both .el and .elc files
into place.
Jim Meyering
2017-11-28 04:18:13 UTC
Permalink
Post by Jim Meyering
Post by Jim Meyering
Post by Glenn Morris
Post by Jim Meyering
In May of 2017, support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory.
In general, Emacs expects .el and .elc to be found in the same
directory. Not adhering to this convention will likely break various
Emacs features. Is this really something automake needs to enable at all?
An alternative would be to copy-or-link the .el file into the
destination directory. Indeed. That would work without breaking pre-23
emacs, so I will adjust my automake patch before pushing it to master.
Hi Glenn,
I've thought about this some more and do not like the idea of
requiring automake's elisp-compilation rule to make a copy of the
source file in the destination directory in this slightly contrived
case. Remember: this arises only in a non-srcdir build. That means
build artifacts end up being written into the mostly-empty current
directory hierarchy, which does not have copies of the sources.
Installation processes will continue to copy both .el and .elc files
into place.
Here is the updated (NEWS addition) patch that I expect to push to
master tomorrow. Feedback welcome. I will also delete the "micro"
branch I created.
Mathieu Lirzin
2017-11-28 21:20:01 UTC
Permalink
Post by Jim Meyering
Post by Jim Meyering
Post by Jim Meyering
Post by Glenn Morris
In general, Emacs expects .el and .elc to be found in the same
directory. Not adhering to this convention will likely break various
Emacs features. Is this really something automake needs to enable at all?
An alternative would be to copy-or-link the .el file into the
destination directory. Indeed. That would work without breaking pre-23
emacs, so I will adjust my automake patch before pushing it to master.
Hi Glenn,
I've thought about this some more and do not like the idea of
requiring automake's elisp-compilation rule to make a copy of the
source file in the destination directory in this slightly contrived
case. Remember: this arises only in a non-srcdir build. That means
build artifacts end up being written into the mostly-empty current
directory hierarchy, which does not have copies of the sources.
Installation processes will continue to copy both .el and .elc files
into place.
I don't like this idea neither.
Post by Jim Meyering
Here is the updated (NEWS addition) patch that I expect to push to
master tomorrow. Feedback welcome. I will also delete the "micro"
branch I created.
From 7558bddcc9cf5ee14441304c2cfc7cffb566daba Mon Sep 17 00:00:00 2001
Date: Wed, 22 Nov 2017 21:07:29 -0800
Subject: [PATCH] port elisp-compilation support to emacs-23.1 and newer
In May of 2017, Emacs' support for using the long-deprecated
byte-compile-dest-file function was removed, and that removal broke
automake's elisp-compiling rule for any .el file not in the current
directory. In emacs-23.1 (July 2009) byte-compile-dest-file-function
became the recommended way to adjust the byte-compiler's destination.
We expect the removed functionality to be restored for Emacs-26,
albeit with dissuasive diagnostics warning about the imminent removal
of this functionality. It may be removed in Emacs-27.
* lib/am/lisp.am (.el.elc): Use byte-compile-dest-file-function,
rather than byte-compile-dest-file.
* t/lisp-readonly-srcdir.sh: New file, to test for the above.
* t/list-of-tests.mk (handwritten_TESTS): Add it.
* NEWS (Bugs fixed): Mention this problem.
OK to push.

Thanks.
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37
Glenn Morris
2017-11-28 22:23:44 UTC
Permalink
Remember: this arises only in a non-srcdir build. That means build
artifacts end up being written into the mostly-empty current directory
hierarchy, which does not have copies of the sources. Installation
processes will continue to copy both .el and .elc files into place.
Oh, so you only split .el and .elc up while building, but they get
installed into the same place? That's not so bad, though it is, eg, not
something that Emacs supports in its own build process (in non-srcdir
builds, .elc still get written into srcdir).

BTW:

ELCFLAGS=-lbytecomp

is presumably another way to solve your issue, for any version of Emacs
and automake (once bytecomp is loaded you can redefine functions as you wish).
Jim Meyering
2017-11-29 03:41:02 UTC
Permalink
Post by Glenn Morris
Remember: this arises only in a non-srcdir build. That means build
artifacts end up being written into the mostly-empty current directory
hierarchy, which does not have copies of the sources. Installation
processes will continue to copy both .el and .elc files into place.
Oh, so you only split .el and .elc up while building, but they get
installed into the same place? That's not so bad, though it is, eg, not
something that Emacs supports in its own build process (in non-srcdir
builds, .elc still get written into srcdir).
ELCFLAGS=-lbytecomp
is presumably another way to solve your issue, for any version of Emacs
and automake (once bytecomp is loaded you can redefine functions as you wish).
Thanks for the suggestion, Glenn.
However, I've hit a new snag: subdirs.
I posted details and a smaller stand-alone demonstrator on the
emacs-devel thread: TL;DR this automake test fails regardless:

make check TESTS='t/lisp-subdir.sh'
Glenn Morris
2017-11-29 20:17:51 UTC
Permalink
The obsolete bytecomp feature is back as of Emacs 9964db4.

BTW, why doesn't lisp.am use the standard "-f batch-byte-compile"
method of producing .elc files?

Your two issues that affected only automake illustrate that the way
automake generates .elc files is different to the vast majority of
Emacs projects.
Jim Meyering
2017-12-11 01:01:38 UTC
Permalink
Post by Glenn Morris
The obsolete bytecomp feature is back as of Emacs 9964db4.
Thanks, I noticed when that was restored, but have been a way for a while.
Post by Glenn Morris
BTW, why doesn't lisp.am use the standard "-f batch-byte-compile"
method of producing .elc files?
Your two issues that affected only automake illustrate that the way
automake generates .elc files is different to the vast majority of
Emacs projects.
Thanks for the suggestion, Glenn. However, I don't see how "-f
batch-byte-compile" can be used when the .elc file must be created in
a directory separate from the one containing the .el file. I think
automake generates code the way it does because GNU coding standards
mandate that one be able to build from a read-only hierarchy of
sources (think read-only media).
Glenn Morris
2017-12-11 17:56:08 UTC
Permalink
However, I don't see how "-f batch-byte-compile" can be used when
the .elc file must be created in a directory separate from the one
containing the .el file.
I meant, instead of reinventing the wheel with this part:

--eval "(unless (byte-compile-file \"$<\") (kill-emacs 1))"

Example:

mkdir /tmp/foo /tmp/bar
echo '(message "hi")' > /tmp/foo/foo.el
emacs --batch \
--eval '(setq byte-compile-dest-file-function (lambda (x) "/tmp/bar/foo.elc"))' \
-f batch-byte-compile /tmp/foo/foo.el

-> generates /tmp/bar/foo.elc

batch-byte-compile exists since forever.
I think automake generates code the way it does because GNU coding
standards mandate that one be able to build from a read-only
hierarchy of sources (think read-only media).
Again, not something Emacs itself supports for its own Lisp files.
Jim Meyering
2017-12-11 19:03:52 UTC
Permalink
Post by Jim Meyering
However, I don't see how "-f batch-byte-compile" can be used when
the .elc file must be created in a directory separate from the one
containing the .el file.
--eval "(unless (byte-compile-file \"$<\") (kill-emacs 1))"
mkdir /tmp/foo /tmp/bar
echo '(message "hi")' > /tmp/foo/foo.el
emacs --batch \
--eval '(setq byte-compile-dest-file-function (lambda (x) "/tmp/bar/foo.elc"))' \
-f batch-byte-compile /tmp/foo/foo.el
-> generates /tmp/bar/foo.elc
batch-byte-compile exists since forever.
Thank you. That looks better, indeed. I will see if I can adapt the
automake patch accordingly.
Jim Meyering
2017-12-16 21:34:42 UTC
Permalink
Post by Jim Meyering
Post by Jim Meyering
However, I don't see how "-f batch-byte-compile" can be used when
the .elc file must be created in a directory separate from the one
containing the .el file.
--eval "(unless (byte-compile-file \"$<\") (kill-emacs 1))"
mkdir /tmp/foo /tmp/bar
echo '(message "hi")' > /tmp/foo/foo.el
emacs --batch \
--eval '(setq byte-compile-dest-file-function (lambda (x) "/tmp/bar/foo.elc"))' \
-f batch-byte-compile /tmp/foo/foo.el
-> generates /tmp/bar/foo.elc
batch-byte-compile exists since forever.
Thank you. That looks better, indeed. I will see if I can adapt the
automake patch accordingly.
I've done that and will push the attached to master later today.
Thanks,
Jim

Loading...