Re: WIN32: Spawning a command-line process

[Tor Lillqvist <tml iki fi>]

> If command-line program is linked with -mswindows

You mean -mwindows (just a typo, I am sure, but in case somebody else
doesn't know).

> then everything works fine,

You mean everything works as expected when your "main" (GUI) app runs
the other program and reads its output? Good, that is as expected.

but I can not execute "prog" in command line.

You mean you can't run the command-line program interactively from a
cmd.exe (or whatever shell) window? True, that's how programs marked
as "GUI" executables work. You have to explicitly redirect its output
to a file, or a pipe, to see it. Run for instance:

  prog | more

or if you are using a Unix-style environment:

  prog | less

If you don't want to do that, your simplest option is to just build
two separate versions of the "prog" program: One linked with
-mwindows, one without. Note that you don't have to change a single
line of code to make it into a "GUI" application, just link it with
-mwindows.

If you read somewhere how "GUI" applications should have WinMain() and
not main(), etc, take that with a large pinch of salt. That is just a
*convention* normally in use (enforced?) in Visual Studio, and just
partly, as far as I can recall, supported by mingw.

> From Windows's point of view, the "GUI-ness" or "console-ness" of an
.exe is just a single field in the .exe header. One can even change it
in an existing .exe file. I only know of the editbin.exe tool that
comes with MSVC to do that, but as what needs to be done is just to
change one byte, it wouldn't be hard to write a dozen-line program to
do it.

Especially also note that the "GUI" vs. "console" field in the .exe
header doesn't have any effect on whether the application can have a
GUI or not.

It's perfectly possible to use editbin to change the executable of,
for instance, Notepad, into a console executable. It will work as
before, but just open a console window (to which it won't print
anything). Note that you have to copy notepad.exe from its normal
location first so that Windows's automatic security mechanism won't
restore it to its original state thinking some tampering is going on.

> Does anybody have an idea how to solve this problem?

If you don't want to build two otherwise identical copies of the
"prog" program (the only difference being that one is linked with
-mwindows and the other one not), one trick could be to use the new
AttachConsole() API that is present in Windows XP and later. Compile
this sample program and check how it works when run from a shell in a
console, or from Explorer. Borrow ideas from this:

#include <stdio.h>

#define _WIN32_WINNT 0x0500
#include <windows.h>

int
main (int argc, char **argv)
{
  int allocated_new_console = FALSE;

  if (!AttachConsole (ATTACH_PARENT_PROCESS))
    {
      if (!AllocConsole ())
        {
          MessageBox (NULL,
                      "Could not either attach to parent's console or
allocate a new console.",
                      "Error",
                      MB_ICONEXCLAMATION|MB_OK);
          exit (1);
        }
      allocated_new_console = TRUE;
    }
  freopen ("CONOUT$", "w", stdout);

  if (allocated_new_console)
    printf ("Could not attach to parent's console, had to allocate a
new one, sorry.\n");

  printf ("Hello there!\n");

  Sleep (10000);

  return 0;
}

In your case you would not have the branch where AllocConsole() is
called. Just try AttachConsole(ATTACH_PARENT_PROCESS) and if that
works, reopen stdout to the console that the process now has. Should
work like a dream.

--tml