[gnome-continuous-yocto/gnomeostree-3.28-rocko: 3561/8267] bitbake: runqueue.py: monitor disk space at regular time intervals

[Emmanuele Bassi <ebassi src gnome org>]

commit ad20ee9febb6bb9beca084b72e21aaddda24bcb9
Author: Patrick Ohly <patrick ohly intel com>
Date:   Tue Nov 29 17:47:43 2016 +0100

    bitbake: runqueue.py: monitor disk space at regular time intervals
    
    Hooking the disk monitor into the regular heatbeat event instead
    of the runqueue solves two problems:
    
    - When there is just one long running task which fills up the disk,
      the previous approach did not notice that until after the completion
      of the task because _execute_runqueue() only gets called on task
      state changes. As a result, aborting a build did not work in this
      case.
    
    - When there are many short-lived tasks, disk space was getting
      checked very frequently. When the storage that is getting checked
      is on an NFS server, that can lead to noticable traffic to the
      server.
    
    (Bitbake rev: 4547eea26803a9cd355d8b045197bcbdbb36a9ad)
    
    Signed-off-by: Patrick Ohly <patrick ohly intel com>
    Signed-off-by: Richard Purdie <richard purdie linuxfoundation org>

 bitbake/lib/bb/runqueue.py |   24 +++++++++++++++++++-----
 1 files changed, 19 insertions(+), 5 deletions(-)
---
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index 51d68a5..3d8ae1f 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -984,8 +984,14 @@ class RunQueue:
         self.state = runQueuePrepare
 
         # For disk space monitor
+        # Invoked at regular time intervals via the bitbake heartbeat event
+        # while the build is running. We generate a unique name for the handler
+        # here, just in case that there ever is more than one RunQueue instance,
+        # start the handler when reaching runQueueSceneRun, and stop it when
+        # done with the build.
         self.dm = monitordisk.diskMonitor(cfgData)
-
+        self.dm_event_handler_name = '_bb_diskmonitor_' + str(id(self))
+        self.dm_event_handler_registered = False
         self.rqexe = None
         self.worker = {}
         self.fakeworker = {}
@@ -1208,10 +1214,12 @@ class RunQueue:
                 self.rqdata.init_progress_reporter.next_stage()
                 self.rqexe = RunQueueExecuteScenequeue(self)
 
-        if self.state in [runQueueSceneRun, runQueueRunning, runQueueCleanUp]:
-            self.dm.check(self)
-
         if self.state is runQueueSceneRun:
+            if not self.dm_event_handler_registered:
+                 res = bb.event.register(self.dm_event_handler_name,
+                                         lambda x: self.dm.check(self) if self.state in [runQueueSceneRun, 
runQueueRunning, runQueueCleanUp] else False,
+                                         ('bb.event.HeartbeatEvent',))
+                 self.dm_event_handler_registered = True
             retval = self.rqexe.execute()
 
         if self.state is runQueueRunInit:
@@ -1230,7 +1238,13 @@ class RunQueue:
         if self.state is runQueueCleanUp:
             retval = self.rqexe.finish()
 
-        if (self.state is runQueueComplete or self.state is runQueueFailed) and self.rqexe:
+        build_done = self.state is runQueueComplete or self.state is runQueueFailed
+
+        if build_done and self.dm_event_handler_registered:
+            bb.event.remove(self.dm_event_handler_name, None)
+            self.dm_event_handler_registered = False
+
+        if build_done and self.rqexe:
             self.teardown_workers()
             if self.rqexe.stats.failed:
                 logger.info("Tasks Summary: Attempted %d tasks of which %d didn't need to be rerun and %d 
failed.", self.rqexe.stats.completed + self.rqexe.stats.failed, self.rqexe.stats.skipped, 
self.rqexe.stats.failed)