[gtk+/gtk-3-22] configure: Don't declare functions in AC_TRY_COMPILE and AC_LANG_PROGRAM

[Ting-Wei Lan <lantw src gnome org>]

commit 31f7b55306a92f97dbc3a88f42b4a8f8661ad689
Author: Ting-Wei Lan <lantw src gnome org>
Date:   Wed Jan 3 01:25:49 2018 +0800

    configure: Don't declare functions in AC_TRY_COMPILE and AC_LANG_PROGRAM
    
    Both AC_TRY_COMPILE and AC_LANG_PROGRAM put code passed to their second
    arguments to the body of the main function. This means that we cannot
    and should not declare functions there, or we end up checking whether
    the compiler support nested functions instead of whether a compiler or
    linker flag is supported.
    
    GCC supports nested functions and tests succeed. Clang doesn't support
    nested functions, so tests fail and -fvisibility=hidden won't be used.
    This means that functions which are not intended to be used by other
    programs, such as gtk_menu_tracker*, gtk_action_observ*,
    gtk_menu_muxer_*, become global symbols with default visibility.
    
    GNOME Shell has a private library libgnome-shell-menu.so, which also has
    symbols gtk_menu_tracker*, gtk_action_observ*, gtk_menu_muxer_* that are
    intended to be used by GNOME Shell itself. When GNOME Shell still used
    Autotools build system, the executable gnome-shell explicitly linked to
    libgnome-shell-menu.so, so the linker loaded libgnome-shell-menu.so
    before libgtk-3.so.0 and GNOME Shell used correct symbols from its
    private library.
    
    However, after GNOME Shell switched to Meson build system, gnome-shell
    executable no longer lists libgnome-shell-menu.so as its dependency.
    Even if we adds it to the build file, it won't be listed in DT_NEEDED of
    gnome-shell because Meson uses -Wl,--as-needed by default. This causes
    the runtime linker to load libgtk-3.so.0 before libgnome-shell-menu.so
    and symbols gtk_menu_tracker*, gtk_action_observ*, gtk_menu_muxer_* are
    bound to libgtk-3.so.0 instead of libgnome-shell-menu.so. GNOME Shell
    hangs when opening more than one window because it uses functions from
    the wrong library.
    
    This problem is already fixed in OpenBSD ports. The article describing
    it can be found on OpenBSD Journal with this link:
    https://undeadly.org/cgi?action=article;sid=20170930133438
    
    https://bugzilla.gnome.org/show_bug.cgi?id=791943

 configure.ac |    5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)
---
diff --git a/configure.ac b/configure.ac
index 904b2ba..16e011d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1820,7 +1820,7 @@ case "$host" in
     SAVED_CFLAGS="${CFLAGS}"
     CFLAGS="-fvisibility=hidden"
     AC_MSG_CHECKING([for -fvisibility=hidden compiler flag])
-    AC_TRY_COMPILE([], [int main (void) { return 0; }],
+    AC_TRY_COMPILE([], [return 0],
                    AC_MSG_RESULT(yes)
                    enable_fvisibility_hidden=yes,
                    AC_MSG_RESULT(no)
@@ -1901,8 +1901,7 @@ AC_ARG_ENABLE(Bsymbolic,
               [SAVED_LDFLAGS="${LDFLAGS}"
                AC_MSG_CHECKING([for -Bsymbolic-functions linker flag])
                LDFLAGS=-Wl,-Bsymbolic-functions
-               AC_LINK_IFELSE([AC_LANG_PROGRAM([[]],
-                                               [[int main (void) { return 0; }]])],
+               AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[return 0]])],
                               [AC_MSG_RESULT(yes)
                                enable_Bsymbolic=yes],
                               [AC_MSG_RESULT(no)