Re: Parallel building

[Frederic Crozat <fcrozat mandrakesoft com>]

Le lun 24/03/2003 à 14:35, Michael Meeks a écrit :
> Hi Frederic,
> 
> On Thu, 2003-03-20 at 15:15, Frederic Crozat wrote:
> > and in Makefile.shared:
> > %.h %-stubs.c  %-common.c %-imodule.c %-skelimpl.c: $(IDL_DIR)%.idl
> > $(IDL_COMPILER)
> > ...
> > 
> > is causing parallel build to fail because make thinks all those
> > generated files can be build in parallel, which is completely false..
> 
> 	Heh ;-) It's strange isn't it. It seems to clearly say to me that all
> those files are generated from the IDL.

You are right of course.. But currently, idl compiler generates the 5
files at the same time.. So, there is no need for run it 5 times..

> 	Is the problem that it runs 5 copies of the IDL compiler ? it's really
> good that you're looking at this though - it's annoyed me for some time.

Well, it wouldn't be a problem if we weren't removing files before and
modifying them after that :: 
-(rm -f $(*).h $(*)-stubs.c $(*)-skels.c $(*)-common.c $(*)-imodule.c
$(*)-skelimpl.c || true) > /dev/null

and 

for I in $(*).h $(*)-stubs.c $(*)-skels.c $(*)-common.c $(*)-imodule.c
$(*)-skelimpl.c; do \
                if test -f $$I; then \
                        sed -e 's,OObject,Object,g' -e
's,TTypeCode,TypeCode,g'
$$I > $$I.out; \
                        mv $$I.out $$I ; \
                fi; \
        done

This is causing trouble because if you run this code 5 times in
parallel, you can be sure you'll be removing some files while some other
parallel make is trying to use them to compile other files (and it
happens :((

> > I've been able to workaround the problem in orb-core, by using a simple
> > dependency on corba-ops-stubs.c (instead of $(OPS_IDLOUT)), which
> > serialize the build (since all the files are generated at the same time
> > by the IDL compiler..)
> 
> 	What line did you add to create the dependency ?

This is for src/orb/orb-core/Makefile.am
--- Makefile.am 19 Mar 2003 12:12:02 -0000      1.45
+++ Makefile.am 26 Mar 2003 08:52:54 -0000
@@ -49,7 +49,9 @@
  
 IDLOUT=corba-defs-common.c
 DEFS_IDLOUT=iop-defs-common.c
-OPS_IDLOUT= corba-ops.h corba-ops-stubs.c corba-ops-common.c
corba-ops-skels.c
+OPS_STUBS_IDLOUT = corba-ops-stubs.c
+OPS_OTHER_IDLOUT = corba-ops.h corba-ops-common.c corba-ops-skels.c
+OPS_IDLOUT= $(OPS_STUBS_IDLOUT) $(OPS_OTHER_IDLOUT)
 IFACE_IDLOUT=orbit-interface-common.c
  
 $(liborb_core_la_OBJECTS) : $(BUILT_SOURCES)
@@ -66,7 +68,8 @@
 include $(top_srcdir)/Makefile.shared
  
 IDL_FLAGS_CORBA_OPS = --showcpperrors
-$(OPS_IDLOUT): corba-ops.idl $(IDL_COMPILER)
+
+$(OPS_STUBS_IDLOUT): corba-ops.idl $(IDL_COMPILER)
        -(rm -f $(OPS_IDLOUT) || true) > /dev/null
        $(IDL_COMPILER) $(IDL_FLAGS_CORBA_OPS) --deps
.deps/corba-ops.idl.P $<
        for I in $(OPS_IDLOUT); do


> > However, I haven't been able yet to fix the problem in Makefile.shared,
> > because it is using pattern matching.. 
> 
> 	I'm not sure; clearly we want to be able to build the generated source
> in parallel; but block on the IDL compilation process; it's not
> altogether clear to me why that doesn't work now, although it clearly
> doesn't.

I hope my explanations are more clear now..

-- 
Frederic Crozat <fcrozat@mandrakesoft.com>
MandrakeSoft