Package SCons :: Package compat :: Module _scons_subprocess
[hide private]
[frames] | no frames]

Source Code for Module SCons.compat._scons_subprocess

   1  # subprocess - Subprocesses with accessible I/O streams 
   2  # 
   3  # For more information about this module, see PEP 324. 
   4  # 
   5  # This module should remain compatible with Python 2.2, see PEP 291. 
   6  # 
   7  # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se> 
   8  # 
   9  # Licensed to PSF under a Contributor Agreement. 
  10  # See http://www.python.org/2.4/license for licensing details. 
  11   
  12  r"""subprocess - Subprocesses with accessible I/O streams 
  13   
  14  This module allows you to spawn processes, connect to their 
  15  input/output/error pipes, and obtain their return codes.  This module 
  16  intends to replace several other, older modules and functions, like: 
  17   
  18  os.system 
  19  os.spawn* 
  20  os.popen* 
  21  popen2.* 
  22  commands.* 
  23   
  24  Information about how the subprocess module can be used to replace these 
  25  modules and functions can be found below. 
  26   
  27   
  28   
  29  Using the subprocess module 
  30  =========================== 
  31  This module defines one class called Popen: 
  32   
  33  class Popen(args, bufsize=0, executable=None, 
  34              stdin=None, stdout=None, stderr=None, 
  35              preexec_fn=None, close_fds=False, shell=False, 
  36              cwd=None, env=None, universal_newlines=False, 
  37              startupinfo=None, creationflags=0): 
  38   
  39   
  40  Arguments are: 
  41   
  42  args should be a string, or a sequence of program arguments.  The 
  43  program to execute is normally the first item in the args sequence or 
  44  string, but can be explicitly set by using the executable argument. 
  45   
  46  On UNIX, with shell=False (default): In this case, the Popen class 
  47  uses os.execvp() to execute the child program.  args should normally 
  48  be a sequence.  A string will be treated as a sequence with the string 
  49  as the only item (the program to execute). 
  50   
  51  On UNIX, with shell=True: If args is a string, it specifies the 
  52  command string to execute through the shell.  If args is a sequence, 
  53  the first item specifies the command string, and any additional items 
  54  will be treated as additional shell arguments. 
  55   
  56  On Windows: the Popen class uses CreateProcess() to execute the child 
  57  program, which operates on strings.  If args is a sequence, it will be 
  58  converted to a string using the list2cmdline method.  Please note that 
  59  not all MS Windows applications interpret the command line the same 
  60  way: The list2cmdline is designed for applications using the same 
  61  rules as the MS C runtime. 
  62   
  63  bufsize, if given, has the same meaning as the corresponding argument 
  64  to the built-in open() function: 0 means unbuffered, 1 means line 
  65  buffered, any other positive value means use a buffer of 
  66  (approximately) that size.  A negative bufsize means to use the system 
  67  default, which usually means fully buffered.  The default value for 
  68  bufsize is 0 (unbuffered). 
  69   
  70  stdin, stdout and stderr specify the executed programs' standard 
  71  input, standard output and standard error file handles, respectively. 
  72  Valid values are PIPE, an existing file descriptor (a positive 
  73  integer), an existing file object, and None.  PIPE indicates that a 
  74  new pipe to the child should be created.  With None, no redirection 
  75  will occur; the child's file handles will be inherited from the 
  76  parent.  Additionally, stderr can be STDOUT, which indicates that the 
  77  stderr data from the applications should be captured into the same 
  78  file handle as for stdout. 
  79   
  80  If preexec_fn is set to a callable object, this object will be called 
  81  in the child process just before the child is executed. 
  82   
  83  If close_fds is true, all file descriptors except 0, 1 and 2 will be 
  84  closed before the child process is executed. 
  85   
  86  if shell is true, the specified command will be executed through the 
  87  shell. 
  88   
  89  If cwd is not None, the current directory will be changed to cwd 
  90  before the child is executed. 
  91   
  92  If env is not None, it defines the environment variables for the new 
  93  process. 
  94   
  95  If universal_newlines is true, the file objects stdout and stderr are 
  96  opened as a text files, but lines may be terminated by any of '\n', 
  97  the Unix end-of-line convention, '\r', the Macintosh convention or 
  98  '\r\n', the Windows convention.  All of these external representations 
  99  are seen as '\n' by the Python program.  Note: This feature is only 
 100  available if Python is built with universal newline support (the 
 101  default).  Also, the newlines attribute of the file objects stdout, 
 102  stdin and stderr are not updated by the communicate() method. 
 103   
 104  The startupinfo and creationflags, if given, will be passed to the 
 105  underlying CreateProcess() function.  They can specify things such as 
 106  appearance of the main window and priority for the new process. 
 107  (Windows only) 
 108   
 109   
 110  This module also defines two shortcut functions: 
 111   
 112  call(*popenargs, **kwargs): 
 113      Run command with arguments.  Wait for command to complete, then 
 114      return the returncode attribute. 
 115   
 116      The arguments are the same as for the Popen constructor.  Example: 
 117   
 118      retcode = call(["ls", "-l"]) 
 119   
 120  check_call(*popenargs, **kwargs): 
 121      Run command with arguments.  Wait for command to complete.  If the 
 122      exit code was zero then return, otherwise raise 
 123      CalledProcessError.  The CalledProcessError object will have the 
 124      return code in the returncode attribute. 
 125   
 126      The arguments are the same as for the Popen constructor.  Example: 
 127   
 128      check_call(["ls", "-l"]) 
 129   
 130  Exceptions 
 131  ---------- 
 132  Exceptions raised in the child process, before the new program has 
 133  started to execute, will be re-raised in the parent.  Additionally, 
 134  the exception object will have one extra attribute called 
 135  'child_traceback', which is a string containing traceback information 
 136  from the childs point of view. 
 137   
 138  The most common exception raised is OSError.  This occurs, for 
 139  example, when trying to execute a non-existent file.  Applications 
 140  should prepare for OSErrors. 
 141   
 142  A ValueError will be raised if Popen is called with invalid arguments. 
 143   
 144  check_call() will raise CalledProcessError, if the called process 
 145  returns a non-zero return code. 
 146   
 147   
 148  Security 
 149  -------- 
 150  Unlike some other popen functions, this implementation will never call 
 151  /bin/sh implicitly.  This means that all characters, including shell 
 152  metacharacters, can safely be passed to child processes. 
 153   
 154   
 155  Popen objects 
 156  ============= 
 157  Instances of the Popen class have the following methods: 
 158   
 159  poll() 
 160      Check if child process has terminated.  Returns returncode 
 161      attribute. 
 162   
 163  wait() 
 164      Wait for child process to terminate.  Returns returncode attribute. 
 165   
 166  communicate(input=None) 
 167      Interact with process: Send data to stdin.  Read data from stdout 
 168      and stderr, until end-of-file is reached.  Wait for process to 
 169      terminate.  The optional stdin argument should be a string to be 
 170      sent to the child process, or None, if no data should be sent to 
 171      the child. 
 172   
 173      communicate() returns a tuple (stdout, stderr). 
 174   
 175      Note: The data read is buffered in memory, so do not use this 
 176      method if the data size is large or unlimited. 
 177   
 178  The following attributes are also available: 
 179   
 180  stdin 
 181      If the stdin argument is PIPE, this attribute is a file object 
 182      that provides input to the child process.  Otherwise, it is None. 
 183   
 184  stdout 
 185      If the stdout argument is PIPE, this attribute is a file object 
 186      that provides output from the child process.  Otherwise, it is 
 187      None. 
 188   
 189  stderr 
 190      If the stderr argument is PIPE, this attribute is file object that 
 191      provides error output from the child process.  Otherwise, it is 
 192      None. 
 193   
 194  pid 
 195      The process ID of the child process. 
 196   
 197  returncode 
 198      The child return code.  A None value indicates that the process 
 199      hasn't terminated yet.  A negative value -N indicates that the 
 200      child was terminated by signal N (UNIX only). 
 201   
 202   
 203  Replacing older functions with the subprocess module 
 204  ==================================================== 
 205  In this section, "a ==> b" means that b can be used as a replacement 
 206  for a. 
 207   
 208  Note: All functions in this section fail (more or less) silently if 
 209  the executed program cannot be found; this module raises an OSError 
 210  exception. 
 211   
 212  In the following examples, we assume that the subprocess module is 
 213  imported with "from subprocess import *". 
 214   
 215   
 216  Replacing /bin/sh shell backquote 
 217  --------------------------------- 
 218  output=`mycmd myarg` 
 219  ==> 
 220  output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0] 
 221   
 222   
 223  Replacing shell pipe line 
 224  ------------------------- 
 225  output=`dmesg | grep hda` 
 226  ==> 
 227  p1 = Popen(["dmesg"], stdout=PIPE) 
 228  p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) 
 229  output = p2.communicate()[0] 
 230   
 231   
 232  Replacing os.system() 
 233  --------------------- 
 234  sts = os.system("mycmd" + " myarg") 
 235  ==> 
 236  p = Popen("mycmd" + " myarg", shell=True) 
 237  pid, sts = os.waitpid(p.pid, 0) 
 238   
 239  Note: 
 240   
 241  * Calling the program through the shell is usually not required. 
 242   
 243  * It's easier to look at the returncode attribute than the 
 244    exitstatus. 
 245   
 246  A more real-world example would look like this: 
 247   
 248  try: 
 249      retcode = call("mycmd" + " myarg", shell=True) 
 250      if retcode < 0: 
 251          print >>sys.stderr, "Child was terminated by signal", -retcode 
 252      else: 
 253          print >>sys.stderr, "Child returned", retcode 
 254  except OSError, e: 
 255      print >>sys.stderr, "Execution failed:", e 
 256   
 257   
 258  Replacing os.spawn* 
 259  ------------------- 
 260  P_NOWAIT example: 
 261   
 262  pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg") 
 263  ==> 
 264  pid = Popen(["/bin/mycmd", "myarg"]).pid 
 265   
 266   
 267  P_WAIT example: 
 268   
 269  retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg") 
 270  ==> 
 271  retcode = call(["/bin/mycmd", "myarg"]) 
 272   
 273   
 274  Vector example: 
 275   
 276  os.spawnvp(os.P_NOWAIT, path, args) 
 277  ==> 
 278  Popen([path] + args[1:]) 
 279   
 280   
 281  Environment example: 
 282   
 283  os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env) 
 284  ==> 
 285  Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"}) 
 286   
 287   
 288  Replacing os.popen* 
 289  ------------------- 
 290  pipe = os.popen(cmd, mode='r', bufsize) 
 291  ==> 
 292  pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout 
 293   
 294  pipe = os.popen(cmd, mode='w', bufsize) 
 295  ==> 
 296  pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin 
 297   
 298   
 299  (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize) 
 300  ==> 
 301  p = Popen(cmd, shell=True, bufsize=bufsize, 
 302            stdin=PIPE, stdout=PIPE, close_fds=True) 
 303  (child_stdin, child_stdout) = (p.stdin, p.stdout) 
 304   
 305   
 306  (child_stdin, 
 307   child_stdout, 
 308   child_stderr) = os.popen3(cmd, mode, bufsize) 
 309  ==> 
 310  p = Popen(cmd, shell=True, bufsize=bufsize, 
 311            stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) 
 312  (child_stdin, 
 313   child_stdout, 
 314   child_stderr) = (p.stdin, p.stdout, p.stderr) 
 315   
 316   
 317  (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize) 
 318  ==> 
 319  p = Popen(cmd, shell=True, bufsize=bufsize, 
 320            stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) 
 321  (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout) 
 322   
 323   
 324  Replacing popen2.* 
 325  ------------------ 
 326  Note: If the cmd argument to popen2 functions is a string, the command 
 327  is executed through /bin/sh.  If it is a list, the command is directly 
 328  executed. 
 329   
 330  (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) 
 331  ==> 
 332  p = Popen(["somestring"], shell=True, bufsize=bufsize 
 333            stdin=PIPE, stdout=PIPE, close_fds=True) 
 334  (child_stdout, child_stdin) = (p.stdout, p.stdin) 
 335   
 336   
 337  (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode) 
 338  ==> 
 339  p = Popen(["mycmd", "myarg"], bufsize=bufsize, 
 340            stdin=PIPE, stdout=PIPE, close_fds=True) 
 341  (child_stdout, child_stdin) = (p.stdout, p.stdin) 
 342   
 343  The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen, 
 344  except that: 
 345   
 346  * subprocess.Popen raises an exception if the execution fails 
 347  * the capturestderr argument is replaced with the stderr argument. 
 348  * stdin=PIPE and stdout=PIPE must be specified. 
 349  * popen2 closes all filedescriptors by default, but you have to specify 
 350    close_fds=True with subprocess.Popen. 
 351   
 352   
 353  """ 
 354   
 355  import sys 
 356  mswindows = (sys.platform == "win32") 
 357   
 358  import os 
 359  import string 
 360  import types 
 361  import traceback 
 362   
 363  # Exception classes used by this module. 
364 -class CalledProcessError(Exception):
365 """This exception is raised when a process run by check_call() returns 366 a non-zero exit status. The exit status will be stored in the 367 returncode attribute."""
368 - def __init__(self, returncode, cmd):
369 self.returncode = returncode 370 self.cmd = cmd
371 - def __str__(self):
372 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
373 374 375 if mswindows: 376 try: 377 import threading 378 except ImportError: 379 # SCons: the threading module is only used by the communicate() 380 # method, which we don't actually use, so don't worry if we 381 # can't import it. 382 pass 383 import msvcrt 384 if 0: # <-- change this to use pywin32 instead of the _subprocess driver 385 import pywintypes 386 from win32api import GetStdHandle, STD_INPUT_HANDLE, \ 387 STD_OUTPUT_HANDLE, STD_ERROR_HANDLE 388 from win32api import GetCurrentProcess, DuplicateHandle, \ 389 GetModuleFileName, GetVersion 390 from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE 391 from win32pipe import CreatePipe 392 from win32process import CreateProcess, STARTUPINFO, \ 393 GetExitCodeProcess, STARTF_USESTDHANDLES, \ 394 STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE 395 from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0 396 else: 397 from _subprocess import *
398 - class STARTUPINFO:
399 dwFlags = 0 400 hStdInput = None 401 hStdOutput = None 402 hStdError = None 403 wShowWindow = 0
404 - class pywintypes:
405 error = IOError
406 else: 407 import select 408 import errno 409 import fcntl 410 import pickle 411 412 try: 413 fcntl.F_GETFD 414 except AttributeError: 415 fcntl.F_GETFD = 1 416 417 try: 418 fcntl.F_SETFD 419 except AttributeError: 420 fcntl.F_SETFD = 2 421 422 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "CalledProcessError"] 423 424 try: 425 MAXFD = os.sysconf("SC_OPEN_MAX") 426 except KeyboardInterrupt: 427 raise # SCons: don't swallow keyboard interrupts 428 except: 429 MAXFD = 256 430 431 # True/False does not exist on 2.2.0 432 try: 433 False 434 except NameError: 435 False = 0 436 True = 1 437 438 try: 439 isinstance(1, int) 440 except TypeError:
441 - def is_int(obj):
442 return type(obj) == type(1)
443 - def is_int_or_long(obj):
444 return type(obj) in (type(1), type(1L))
445 else:
446 - def is_int(obj):
447 return isinstance(obj, int)
448 - def is_int_or_long(obj):
449 return isinstance(obj, (int, long))
450 451 try: 452 types.StringTypes 453 except AttributeError: 454 try: 455 types.StringTypes = (types.StringType, types.UnicodeType) 456 except AttributeError: 457 types.StringTypes = (types.StringType,)
458 - def is_string(obj):
459 return type(obj) in types.StringTypes
460 else:
461 - def is_string(obj):
462 return isinstance(obj, types.StringTypes)
463 464 _active = [] 465
466 -def _cleanup():
467 for inst in _active[:]: 468 if inst.poll(_deadstate=sys.maxint) >= 0: 469 try: 470 _active.remove(inst) 471 except ValueError: 472 # This can happen if two threads create a new Popen instance. 473 # It's harmless that it was already removed, so ignore. 474 pass
475 476 PIPE = -1 477 STDOUT = -2 478 479
480 -def call(*popenargs, **kwargs):
481 """Run command with arguments. Wait for command to complete, then 482 return the returncode attribute. 483 484 The arguments are the same as for the Popen constructor. Example: 485 486 retcode = call(["ls", "-l"]) 487 """ 488 return apply(Popen, popenargs, kwargs).wait()
489 490
491 -def check_call(*popenargs, **kwargs):
492 """Run command with arguments. Wait for command to complete. If 493 the exit code was zero then return, otherwise raise 494 CalledProcessError. The CalledProcessError object will have the 495 return code in the returncode attribute. 496 497 The arguments are the same as for the Popen constructor. Example: 498 499 check_call(["ls", "-l"]) 500 """ 501 retcode = apply(call, popenargs, kwargs) 502 cmd = kwargs.get("args") 503 if cmd is None: 504 cmd = popenargs[0] 505 if retcode: 506 raise CalledProcessError(retcode, cmd) 507 return retcode
508 509
510 -def list2cmdline(seq):
511 """ 512 Translate a sequence of arguments into a command line 513 string, using the same rules as the MS C runtime: 514 515 1) Arguments are delimited by white space, which is either a 516 space or a tab. 517 518 2) A string surrounded by double quotation marks is 519 interpreted as a single argument, regardless of white space 520 contained within. A quoted string can be embedded in an 521 argument. 522 523 3) A double quotation mark preceded by a backslash is 524 interpreted as a literal double quotation mark. 525 526 4) Backslashes are interpreted literally, unless they 527 immediately precede a double quotation mark. 528 529 5) If backslashes immediately precede a double quotation mark, 530 every pair of backslashes is interpreted as a literal 531 backslash. If the number of backslashes is odd, the last 532 backslash escapes the next double quotation mark as 533 described in rule 3. 534 """ 535 536 # See 537 # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp 538 result = [] 539 needquote = False 540 for arg in seq: 541 bs_buf = [] 542 543 # Add a space to separate this argument from the others 544 if result: 545 result.append(' ') 546 547 needquote = (" " in arg) or ("\t" in arg) 548 if needquote: 549 result.append('"') 550 551 for c in arg: 552 if c == '\\': 553 # Don't know if we need to double yet. 554 bs_buf.append(c) 555 elif c == '"': 556 # Double backspaces. 557 result.append('\\' * len(bs_buf)*2) 558 bs_buf = [] 559 result.append('\\"') 560 else: 561 # Normal char 562 if bs_buf: 563 result.extend(bs_buf) 564 bs_buf = [] 565 result.append(c) 566 567 # Add remaining backspaces, if any. 568 if bs_buf: 569 result.extend(bs_buf) 570 571 if needquote: 572 result.extend(bs_buf) 573 result.append('"') 574 575 return string.join(result, '')
576 577 578 try: 579 object 580 except NameError:
581 - class object:
582 pass
583
584 -class Popen(object):
585 - def __init__(self, args, bufsize=0, executable=None, 586 stdin=None, stdout=None, stderr=None, 587 preexec_fn=None, close_fds=False, shell=False, 588 cwd=None, env=None, universal_newlines=False, 589 startupinfo=None, creationflags=0):
590 """Create new Popen instance.""" 591 _cleanup() 592 593 self._child_created = False 594 if not is_int_or_long(bufsize): 595 raise TypeError("bufsize must be an integer") 596 597 if mswindows: 598 if preexec_fn is not None: 599 raise ValueError("preexec_fn is not supported on Windows " 600 "platforms") 601 if close_fds: 602 raise ValueError("close_fds is not supported on Windows " 603 "platforms") 604 else: 605 # POSIX 606 if startupinfo is not None: 607 raise ValueError("startupinfo is only supported on Windows " 608 "platforms") 609 if creationflags != 0: 610 raise ValueError("creationflags is only supported on Windows " 611 "platforms") 612 613 self.stdin = None 614 self.stdout = None 615 self.stderr = None 616 self.pid = None 617 self.returncode = None 618 self.universal_newlines = universal_newlines 619 620 # Input and output objects. The general principle is like 621 # this: 622 # 623 # Parent Child 624 # ------ ----- 625 # p2cwrite ---stdin---> p2cread 626 # c2pread <--stdout--- c2pwrite 627 # errread <--stderr--- errwrite 628 # 629 # On POSIX, the child objects are file descriptors. On 630 # Windows, these are Windows file handles. The parent objects 631 # are file descriptors on both platforms. The parent objects 632 # are None when not using PIPEs. The child objects are None 633 # when not redirecting. 634 635 (p2cread, p2cwrite, 636 c2pread, c2pwrite, 637 errread, errwrite) = self._get_handles(stdin, stdout, stderr) 638 639 self._execute_child(args, executable, preexec_fn, close_fds, 640 cwd, env, universal_newlines, 641 startupinfo, creationflags, shell, 642 p2cread, p2cwrite, 643 c2pread, c2pwrite, 644 errread, errwrite) 645 646 if p2cwrite: 647 self.stdin = os.fdopen(p2cwrite, 'wb', bufsize) 648 if c2pread: 649 if universal_newlines: 650 self.stdout = os.fdopen(c2pread, 'rU', bufsize) 651 else: 652 self.stdout = os.fdopen(c2pread, 'rb', bufsize) 653 if errread: 654 if universal_newlines: 655 self.stderr = os.fdopen(errread, 'rU', bufsize) 656 else: 657 self.stderr = os.fdopen(errread, 'rb', bufsize)
658 659
660 - def _translate_newlines(self, data):
661 data = data.replace("\r\n", "\n") 662 data = data.replace("\r", "\n") 663 return data
664 665
666 - def __del__(self):
667 if not self._child_created: 668 # We didn't get to successfully create a child process. 669 return 670 # In case the child hasn't been waited on, check if it's done. 671 self.poll(_deadstate=sys.maxint) 672 if self.returncode is None and _active is not None: 673 # Child is still running, keep us alive until we can wait on it. 674 _active.append(self)
675 676
677 - def communicate(self, input=None):
678 """Interact with process: Send data to stdin. Read data from 679 stdout and stderr, until end-of-file is reached. Wait for 680 process to terminate. The optional input argument should be a 681 string to be sent to the child process, or None, if no data 682 should be sent to the child. 683 684 communicate() returns a tuple (stdout, stderr).""" 685 686 # Optimization: If we are only using one pipe, or no pipe at 687 # all, using select() or threads is unnecessary. 688 if [self.stdin, self.stdout, self.stderr].count(None) >= 2: 689 stdout = None 690 stderr = None 691 if self.stdin: 692 if input: 693 self.stdin.write(input) 694 self.stdin.close() 695 elif self.stdout: 696 stdout = self.stdout.read() 697 elif self.stderr: 698 stderr = self.stderr.read() 699 self.wait() 700 return (stdout, stderr) 701 702 return self._communicate(input)
703 704 705 if mswindows: 706 # 707 # Windows methods 708 #
709 - def _get_handles(self, stdin, stdout, stderr):
710 """Construct and return tupel with IO objects: 711 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 712 """ 713 if stdin is None and stdout is None and stderr is None: 714 return (None, None, None, None, None, None) 715 716 p2cread, p2cwrite = None, None 717 c2pread, c2pwrite = None, None 718 errread, errwrite = None, None 719 720 if stdin is None: 721 p2cread = GetStdHandle(STD_INPUT_HANDLE) 722 elif stdin == PIPE: 723 p2cread, p2cwrite = CreatePipe(None, 0) 724 # Detach and turn into fd 725 p2cwrite = p2cwrite.Detach() 726 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0) 727 elif is_int(stdin): 728 p2cread = msvcrt.get_osfhandle(stdin) 729 else: 730 # Assuming file-like object 731 p2cread = msvcrt.get_osfhandle(stdin.fileno()) 732 p2cread = self._make_inheritable(p2cread) 733 734 if stdout is None: 735 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE) 736 elif stdout == PIPE: 737 c2pread, c2pwrite = CreatePipe(None, 0) 738 # Detach and turn into fd 739 c2pread = c2pread.Detach() 740 c2pread = msvcrt.open_osfhandle(c2pread, 0) 741 elif is_int(stdout): 742 c2pwrite = msvcrt.get_osfhandle(stdout) 743 else: 744 # Assuming file-like object 745 c2pwrite = msvcrt.get_osfhandle(stdout.fileno()) 746 c2pwrite = self._make_inheritable(c2pwrite) 747 748 if stderr is None: 749 errwrite = GetStdHandle(STD_ERROR_HANDLE) 750 elif stderr == PIPE: 751 errread, errwrite = CreatePipe(None, 0) 752 # Detach and turn into fd 753 errread = errread.Detach() 754 errread = msvcrt.open_osfhandle(errread, 0) 755 elif stderr == STDOUT: 756 errwrite = c2pwrite 757 elif is_int(stderr): 758 errwrite = msvcrt.get_osfhandle(stderr) 759 else: 760 # Assuming file-like object 761 errwrite = msvcrt.get_osfhandle(stderr.fileno()) 762 errwrite = self._make_inheritable(errwrite) 763 764 return (p2cread, p2cwrite, 765 c2pread, c2pwrite, 766 errread, errwrite)
767 768
769 - def _make_inheritable(self, handle):
770 """Return a duplicate of handle, which is inheritable""" 771 return DuplicateHandle(GetCurrentProcess(), handle, 772 GetCurrentProcess(), 0, 1, 773 DUPLICATE_SAME_ACCESS)
774 775
776 - def _find_w9xpopen(self):
777 """Find and return absolut path to w9xpopen.exe""" 778 w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)), 779 "w9xpopen.exe") 780 if not os.path.exists(w9xpopen): 781 # Eeek - file-not-found - possibly an embedding 782 # situation - see if we can locate it in sys.exec_prefix 783 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix), 784 "w9xpopen.exe") 785 if not os.path.exists(w9xpopen): 786 raise RuntimeError("Cannot locate w9xpopen.exe, which is " 787 "needed for Popen to work with your " 788 "shell or platform.") 789 return w9xpopen
790 791
792 - def _execute_child(self, args, executable, preexec_fn, close_fds, 793 cwd, env, universal_newlines, 794 startupinfo, creationflags, shell, 795 p2cread, p2cwrite, 796 c2pread, c2pwrite, 797 errread, errwrite):
798 """Execute program (MS Windows version)""" 799 800 if not isinstance(args, types.StringTypes): 801 args = list2cmdline(args) 802 803 # Process startup details 804 if startupinfo is None: 805 startupinfo = STARTUPINFO() 806 if None not in (p2cread, c2pwrite, errwrite): 807 startupinfo.dwFlags = startupinfo.dwFlags | STARTF_USESTDHANDLES 808 startupinfo.hStdInput = p2cread 809 startupinfo.hStdOutput = c2pwrite 810 startupinfo.hStdError = errwrite 811 812 if shell: 813 startupinfo.dwFlags = startupinfo.dwFlags | STARTF_USESHOWWINDOW 814 startupinfo.wShowWindow = SW_HIDE 815 comspec = os.environ.get("COMSPEC", "cmd.exe") 816 args = comspec + " /c " + args 817 if (GetVersion() >= 0x80000000L or 818 os.path.basename(comspec).lower() == "command.com"): 819 # Win9x, or using command.com on NT. We need to 820 # use the w9xpopen intermediate program. For more 821 # information, see KB Q150956 822 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp) 823 w9xpopen = self._find_w9xpopen() 824 args = '"%s" %s' % (w9xpopen, args) 825 # Not passing CREATE_NEW_CONSOLE has been known to 826 # cause random failures on win9x. Specifically a 827 # dialog: "Your program accessed mem currently in 828 # use at xxx" and a hopeful warning about the 829 # stability of your system. Cost is Ctrl+C wont 830 # kill children. 831 creationflags = creationflags | CREATE_NEW_CONSOLE 832 833 # Start the process 834 try: 835 hp, ht, pid, tid = CreateProcess(executable, args, 836 # no special security 837 None, None, 838 # must inherit handles to pass std 839 # handles 840 1, 841 creationflags, 842 env, 843 cwd, 844 startupinfo) 845 except pywintypes.error, e: 846 # Translate pywintypes.error to WindowsError, which is 847 # a subclass of OSError. FIXME: We should really 848 # translate errno using _sys_errlist (or simliar), but 849 # how can this be done from Python? 850 raise apply(WindowsError, e.args) 851 852 # Retain the process handle, but close the thread handle 853 self._child_created = True 854 self._handle = hp 855 self.pid = pid 856 ht.Close() 857 858 # Child is launched. Close the parent's copy of those pipe 859 # handles that only the child should have open. You need 860 # to make sure that no handles to the write end of the 861 # output pipe are maintained in this process or else the 862 # pipe will not close when the child process exits and the 863 # ReadFile will hang. 864 if p2cread is not None: 865 p2cread.Close() 866 if c2pwrite is not None: 867 c2pwrite.Close() 868 if errwrite is not None: 869 errwrite.Close()
870 871
872 - def poll(self, _deadstate=None):
873 """Check if child process has terminated. Returns returncode 874 attribute.""" 875 if self.returncode is None: 876 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0: 877 self.returncode = GetExitCodeProcess(self._handle) 878 return self.returncode
879 880
881 - def wait(self):
882 """Wait for child process to terminate. Returns returncode 883 attribute.""" 884 if self.returncode is None: 885 obj = WaitForSingleObject(self._handle, INFINITE) 886 self.returncode = GetExitCodeProcess(self._handle) 887 return self.returncode
888 889
890 - def _readerthread(self, fh, buffer):
891 buffer.append(fh.read())
892 893
894 - def _communicate(self, input):
895 stdout = None # Return 896 stderr = None # Return 897 898 if self.stdout: 899 stdout = [] 900 stdout_thread = threading.Thread(target=self._readerthread, 901 args=(self.stdout, stdout)) 902 stdout_thread.setDaemon(True) 903 stdout_thread.start() 904 if self.stderr: 905 stderr = [] 906 stderr_thread = threading.Thread(target=self._readerthread, 907 args=(self.stderr, stderr)) 908 stderr_thread.setDaemon(True) 909 stderr_thread.start() 910 911 if self.stdin: 912 if input is not None: 913 self.stdin.write(input) 914 self.stdin.close() 915 916 if self.stdout: 917 stdout_thread.join() 918 if self.stderr: 919 stderr_thread.join() 920 921 # All data exchanged. Translate lists into strings. 922 if stdout is not None: 923 stdout = stdout[0] 924 if stderr is not None: 925 stderr = stderr[0] 926 927 # Translate newlines, if requested. We cannot let the file 928 # object do the translation: It is based on stdio, which is 929 # impossible to combine with select (unless forcing no 930 # buffering). 931 if self.universal_newlines and hasattr(file, 'newlines'): 932 if stdout: 933 stdout = self._translate_newlines(stdout) 934 if stderr: 935 stderr = self._translate_newlines(stderr) 936 937 self.wait() 938 return (stdout, stderr)
939 940 else: 941 # 942 # POSIX methods 943 #
944 - def _get_handles(self, stdin, stdout, stderr):
945 """Construct and return tupel with IO objects: 946 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 947 """ 948 p2cread, p2cwrite = None, None 949 c2pread, c2pwrite = None, None 950 errread, errwrite = None, None 951 952 if stdin is None: 953 pass 954 elif stdin == PIPE: 955 p2cread, p2cwrite = os.pipe() 956 elif is_int(stdin): 957 p2cread = stdin 958 else: 959 # Assuming file-like object 960 p2cread = stdin.fileno() 961 962 if stdout is None: 963 pass 964 elif stdout == PIPE: 965 c2pread, c2pwrite = os.pipe() 966 elif is_int(stdout): 967 c2pwrite = stdout 968 else: 969 # Assuming file-like object 970 c2pwrite = stdout.fileno() 971 972 if stderr is None: 973 pass 974 elif stderr == PIPE: 975 errread, errwrite = os.pipe() 976 elif stderr == STDOUT: 977 errwrite = c2pwrite 978 elif is_int(stderr): 979 errwrite = stderr 980 else: 981 # Assuming file-like object 982 errwrite = stderr.fileno() 983 984 return (p2cread, p2cwrite, 985 c2pread, c2pwrite, 986 errread, errwrite)
987 988
989 - def _set_cloexec_flag(self, fd):
990 try: 991 cloexec_flag = fcntl.FD_CLOEXEC 992 except AttributeError: 993 cloexec_flag = 1 994 995 old = fcntl.fcntl(fd, fcntl.F_GETFD) 996 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
997 998
999 - def _close_fds(self, but):
1000 for i in xrange(3, MAXFD): 1001 if i == but: 1002 continue 1003 try: 1004 os.close(i) 1005 except KeyboardInterrupt: 1006 raise # SCons: don't swallow keyboard interrupts 1007 except: 1008 pass
1009 1010
1011 - def _execute_child(self, args, executable, preexec_fn, close_fds, 1012 cwd, env, universal_newlines, 1013 startupinfo, creationflags, shell, 1014 p2cread, p2cwrite, 1015 c2pread, c2pwrite, 1016 errread, errwrite):
1017 """Execute program (POSIX version)""" 1018 1019 if is_string(args): 1020 args = [args] 1021 1022 if shell: 1023 args = ["/bin/sh", "-c"] + args 1024 1025 if executable is None: 1026 executable = args[0] 1027 1028 # For transferring possible exec failure from child to parent 1029 # The first char specifies the exception type: 0 means 1030 # OSError, 1 means some other error. 1031 errpipe_read, errpipe_write = os.pipe() 1032 self._set_cloexec_flag(errpipe_write) 1033 1034 self.pid = os.fork() 1035 self._child_created = True 1036 if self.pid == 0: 1037 # Child 1038 try: 1039 # Close parent's pipe ends 1040 if p2cwrite: 1041 os.close(p2cwrite) 1042 if c2pread: 1043 os.close(c2pread) 1044 if errread: 1045 os.close(errread) 1046 os.close(errpipe_read) 1047 1048 # Dup fds for child 1049 if p2cread: 1050 os.dup2(p2cread, 0) 1051 if c2pwrite: 1052 os.dup2(c2pwrite, 1) 1053 if errwrite: 1054 os.dup2(errwrite, 2) 1055 1056 # Close pipe fds. Make sure we don't close the same 1057 # fd more than once, or standard fds. 1058 try: 1059 set 1060 except NameError: 1061 # Fall-back for earlier Python versions, so epydoc 1062 # can use this module directly to execute things. 1063 if p2cread: 1064 os.close(p2cread) 1065 if c2pwrite and c2pwrite not in (p2cread,): 1066 os.close(c2pwrite) 1067 if errwrite and errwrite not in (p2cread, c2pwrite): 1068 os.close(errwrite) 1069 else: 1070 for fd in set((p2cread, c2pwrite, errwrite))-set((0,1,2)): 1071 if fd: os.close(fd) 1072 1073 # Close all other fds, if asked for 1074 if close_fds: 1075 self._close_fds(but=errpipe_write) 1076 1077 if cwd is not None: 1078 os.chdir(cwd) 1079 1080 if preexec_fn: 1081 apply(preexec_fn) 1082 1083 if env is None: 1084 os.execvp(executable, args) 1085 else: 1086 os.execvpe(executable, args, env) 1087 1088 except KeyboardInterrupt: 1089 raise # SCons: don't swallow keyboard interrupts 1090 1091 except: 1092 exc_type, exc_value, tb = sys.exc_info() 1093 # Save the traceback and attach it to the exception object 1094 exc_lines = traceback.format_exception(exc_type, 1095 exc_value, 1096 tb) 1097 exc_value.child_traceback = string.join(exc_lines, '') 1098 os.write(errpipe_write, pickle.dumps(exc_value)) 1099 1100 # This exitcode won't be reported to applications, so it 1101 # really doesn't matter what we return. 1102 os._exit(255) 1103 1104 # Parent 1105 os.close(errpipe_write) 1106 if p2cread and p2cwrite: 1107 os.close(p2cread) 1108 if c2pwrite and c2pread: 1109 os.close(c2pwrite) 1110 if errwrite and errread: 1111 os.close(errwrite) 1112 1113 # Wait for exec to fail or succeed; possibly raising exception 1114 data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB 1115 os.close(errpipe_read) 1116 if data != "": 1117 os.waitpid(self.pid, 0) 1118 child_exception = pickle.loads(data) 1119 raise child_exception
1120 1121
1122 - def _handle_exitstatus(self, sts):
1123 if os.WIFSIGNALED(sts): 1124 self.returncode = -os.WTERMSIG(sts) 1125 elif os.WIFEXITED(sts): 1126 self.returncode = os.WEXITSTATUS(sts) 1127 else: 1128 # Should never happen 1129 raise RuntimeError("Unknown child exit status!")
1130 1131
1132 - def poll(self, _deadstate=None):
1133 """Check if child process has terminated. Returns returncode 1134 attribute.""" 1135 if self.returncode is None: 1136 try: 1137 pid, sts = os.waitpid(self.pid, os.WNOHANG) 1138 if pid == self.pid: 1139 self._handle_exitstatus(sts) 1140 except os.error: 1141 if _deadstate is not None: 1142 self.returncode = _deadstate 1143 return self.returncode
1144 1145
1146 - def wait(self):
1147 """Wait for child process to terminate. Returns returncode 1148 attribute.""" 1149 if self.returncode is None: 1150 pid, sts = os.waitpid(self.pid, 0) 1151 self._handle_exitstatus(sts) 1152 return self.returncode
1153 1154
1155 - def _communicate(self, input):
1156 read_set = [] 1157 write_set = [] 1158 stdout = None # Return 1159 stderr = None # Return 1160 1161 if self.stdin: 1162 # Flush stdio buffer. This might block, if the user has 1163 # been writing to .stdin in an uncontrolled fashion. 1164 self.stdin.flush() 1165 if input: 1166 write_set.append(self.stdin) 1167 else: 1168 self.stdin.close() 1169 if self.stdout: 1170 read_set.append(self.stdout) 1171 stdout = [] 1172 if self.stderr: 1173 read_set.append(self.stderr) 1174 stderr = [] 1175 1176 input_offset = 0 1177 while read_set or write_set: 1178 rlist, wlist, xlist = select.select(read_set, write_set, []) 1179 1180 if self.stdin in wlist: 1181 # When select has indicated that the file is writable, 1182 # we can write up to PIPE_BUF bytes without risk 1183 # blocking. POSIX defines PIPE_BUF >= 512 1184 bytes_written = os.write(self.stdin.fileno(), buffer(input, input_offset, 512)) 1185 input_offset = input_offset + bytes_written 1186 if input_offset >= len(input): 1187 self.stdin.close() 1188 write_set.remove(self.stdin) 1189 1190 if self.stdout in rlist: 1191 data = os.read(self.stdout.fileno(), 1024) 1192 if data == "": 1193 self.stdout.close() 1194 read_set.remove(self.stdout) 1195 stdout.append(data) 1196 1197 if self.stderr in rlist: 1198 data = os.read(self.stderr.fileno(), 1024) 1199 if data == "": 1200 self.stderr.close() 1201 read_set.remove(self.stderr) 1202 stderr.append(data) 1203 1204 # All data exchanged. Translate lists into strings. 1205 if stdout is not None: 1206 stdout = string.join(stdout, '') 1207 if stderr is not None: 1208 stderr = string.join(stderr, '') 1209 1210 # Translate newlines, if requested. We cannot let the file 1211 # object do the translation: It is based on stdio, which is 1212 # impossible to combine with select (unless forcing no 1213 # buffering). 1214 if self.universal_newlines and hasattr(file, 'newlines'): 1215 if stdout: 1216 stdout = self._translate_newlines(stdout) 1217 if stderr: 1218 stderr = self._translate_newlines(stderr) 1219 1220 self.wait() 1221 return (stdout, stderr)
1222 1223
1224 -def _demo_posix():
1225 # 1226 # Example 1: Simple redirection: Get process list 1227 # 1228 plist = Popen(["ps"], stdout=PIPE).communicate()[0] 1229 print "Process list:" 1230 print plist 1231 1232 # 1233 # Example 2: Change uid before executing child 1234 # 1235 if os.getuid() == 0: 1236 p = Popen(["id"], preexec_fn=lambda: os.setuid(100)) 1237 p.wait() 1238 1239 # 1240 # Example 3: Connecting several subprocesses 1241 # 1242 print "Looking for 'hda'..." 1243 p1 = Popen(["dmesg"], stdout=PIPE) 1244 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) 1245 print repr(p2.communicate()[0]) 1246 1247 # 1248 # Example 4: Catch execution error 1249 # 1250 print 1251 print "Trying a weird file..." 1252 try: 1253 print Popen(["/this/path/does/not/exist"]).communicate() 1254 except OSError, e: 1255 if e.errno == errno.ENOENT: 1256 print "The file didn't exist. I thought so..." 1257 print "Child traceback:" 1258 print e.child_traceback 1259 else: 1260 print "Error", e.errno 1261 else: 1262 sys.stderr.write( "Gosh. No error.\n" )
1263 1264
1265 -def _demo_windows():
1266 # 1267 # Example 1: Connecting several subprocesses 1268 # 1269 print "Looking for 'PROMPT' in set output..." 1270 p1 = Popen("set", stdout=PIPE, shell=True) 1271 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE) 1272 print repr(p2.communicate()[0]) 1273 1274 # 1275 # Example 2: Simple execution of program 1276 # 1277 print "Executing calc..." 1278 p = Popen("calc") 1279 p.wait()
1280 1281 1282 if __name__ == "__main__": 1283 if mswindows: 1284 _demo_windows() 1285 else: 1286 _demo_posix() 1287