[gnome-continuous-yocto/gnomeostree-3.28-rocko: 4424/8267] verify-bashisms: fix problems with tinfoil2

[Emmanuele Bassi <ebassi src gnome org>]

commit 010f9fa86a993554e2b9134e6baf96899649a2ef
Author: Patrick Ohly <patrick ohly intel com>
Date:   Tue Jan 31 13:50:29 2017 +0100

    verify-bashisms: fix problems with tinfoil2
    
    tinfoil2 is based on a client/server architecture, which broke the
    verify-bashisms script:
    
    - The tinfoil instance and its data proxies can't be pickled, so
      all interaction with the bitbake server has to run in the main
      script process and only processing of the plain scripts can
      be done with multiprocessing:
    
      _pickle.PicklingError: Can't pickle <class 
'bb.tinfoil.TinfoilCookerAdapter.TinfoilRecipeCacheAdapter'>: attribute lookup TinfoilRecipeCacheAdapter on 
bb.tinfoil failed
    
    - The multiprocessing pool has to be created before initializing
      tinfoil, otherwise the pool workers end up trying to communicate
      with the bitbake server during shutdown:
    
      ERROR: UI received SIGTERM
      Process ForkPoolWorker-2:
      Traceback (most recent call last):
        File "/usr/lib/python3.4/multiprocessing/process.py", line 257, in _bootstrap
          util._exit_function()
        File "/usr/lib/python3.4/multiprocessing/util.py", line 286, in _exit_function
          _run_finalizers(0)
        ...
        File "/usr/lib/python3.4/multiprocessing/process.py", line 131, in is_alive
          assert self._parent_pid == os.getpid(), 'can only test a child process'
       AssertionError: can only test a child process
    
    - func() needs to defined before creating the pool to avoid:
    
      AttributeError: Can't get attribute 'func' on <module '__main__' from 
'/work/openembedded-core/scripts/verify-bashisms'>
    
    (From OE-Core rev: aa439f11c7f414774843720d68ebe0a6d3375ea6)
    
    Signed-off-by: Patrick Ohly <patrick ohly intel com>
    Signed-off-by: Richard Purdie <richard purdie linuxfoundation org>

 scripts/verify-bashisms |   44 +++++++++++++++++++++++++-------------------
 1 files changed, 25 insertions(+), 19 deletions(-)
---
diff --git a/scripts/verify-bashisms b/scripts/verify-bashisms
index ed0a563..613f174 100755
--- a/scripts/verify-bashisms
+++ b/scripts/verify-bashisms
@@ -71,6 +71,20 @@ if __name__=='__main__':
         print("Cannot find checkbashisms.pl on $PATH, get it from 
https://anonscm.debian.org/cgit/collab-maint/devscripts.git/plain/scripts/checkbashisms.pl";)
         sys.exit(1)
 
+    # The order of defining the worker function,
+    # initializing the pool and connecting to the
+    # bitbake server is crucial, don't change it.
+    def func(item):
+        fn, scripts = item
+        result = []
+        for key, script in scripts:
+            r = process(fn, key, script)
+            if r: result.extend(r)
+        return fn, result
+
+    import multiprocessing
+    pool = multiprocessing.Pool()
+
     tinfoil = get_tinfoil()
 
     # This is only the default configuration and should iterate over
@@ -83,32 +97,24 @@ if __name__=='__main__':
     else:
         initial_pns = sorted(pkg_pn)
 
-    pns = []
-    print("Generating file list...")
+    pns = {}
+    print("Generating scripts...")
     for pn in initial_pns:
         for fn in pkg_pn[pn]:
             # There's no point checking multiple BBCLASSEXTENDed variants of the same recipe
             realfn, _, _ = bb.cache.virtualfn2realfn(fn)
             if realfn not in pns:
-                pns.append(realfn)
-
-
-    def func(fn):
-        result = []
-        data = tinfoil.parse_recipe_file(fn)
-        for key in data.keys():
-            if data.getVarFlag(key, "func") and not data.getVarFlag(key, "python"):
-                script = data.getVar(key, False)
-                if not script: continue
-                #print ("%s:%s" % (fn, key))
-                r = process(fn, key, script)
-                if r: result.extend(r)
-        return fn, result
+                data = tinfoil.parse_recipe_file(realfn)
+                scripts = []
+                for key in data.keys():
+                    if data.getVarFlag(key, "func") and not data.getVarFlag(key, "python"):
+                        script = data.getVar(key, False)
+                        if script:
+                            scripts.append((key, script))
+                pns[realfn] = scripts
 
     print("Scanning scripts...\n")
-    import multiprocessing
-    pool = multiprocessing.Pool()
-    for pn,results in pool.imap(func, pns):
+    for pn, results in pool.imap(func, pns.items()):
         if results:
             print(pn)
             for message,source in results: