1
2
3
4
5
6
7
8
9
10
11
12 r"""subprocess - Subprocesses with accessible I/O streams
13
14 This module allows you to spawn processes, connect to their
15 input/output/error pipes, and obtain their return codes. This module
16 intends to replace several other, older modules and functions, like:
17
18 os.system
19 os.spawn*
20 os.popen*
21 popen2.*
22 commands.*
23
24 Information about how the subprocess module can be used to replace these
25 modules and functions can be found below.
26
27
28
29 Using the subprocess module
30 ===========================
31 This module defines one class called Popen:
32
33 class Popen(args, bufsize=0, executable=None,
34 stdin=None, stdout=None, stderr=None,
35 preexec_fn=None, close_fds=False, shell=False,
36 cwd=None, env=None, universal_newlines=False,
37 startupinfo=None, creationflags=0):
38
39
40 Arguments are:
41
42 args should be a string, or a sequence of program arguments. The
43 program to execute is normally the first item in the args sequence or
44 string, but can be explicitly set by using the executable argument.
45
46 On UNIX, with shell=False (default): In this case, the Popen class
47 uses os.execvp() to execute the child program. args should normally
48 be a sequence. A string will be treated as a sequence with the string
49 as the only item (the program to execute).
50
51 On UNIX, with shell=True: If args is a string, it specifies the
52 command string to execute through the shell. If args is a sequence,
53 the first item specifies the command string, and any additional items
54 will be treated as additional shell arguments.
55
56 On Windows: the Popen class uses CreateProcess() to execute the child
57 program, which operates on strings. If args is a sequence, it will be
58 converted to a string using the list2cmdline method. Please note that
59 not all MS Windows applications interpret the command line the same
60 way: The list2cmdline is designed for applications using the same
61 rules as the MS C runtime.
62
63 bufsize, if given, has the same meaning as the corresponding argument
64 to the built-in open() function: 0 means unbuffered, 1 means line
65 buffered, any other positive value means use a buffer of
66 (approximately) that size. A negative bufsize means to use the system
67 default, which usually means fully buffered. The default value for
68 bufsize is 0 (unbuffered).
69
70 stdin, stdout and stderr specify the executed programs' standard
71 input, standard output and standard error file handles, respectively.
72 Valid values are PIPE, an existing file descriptor (a positive
73 integer), an existing file object, and None. PIPE indicates that a
74 new pipe to the child should be created. With None, no redirection
75 will occur; the child's file handles will be inherited from the
76 parent. Additionally, stderr can be STDOUT, which indicates that the
77 stderr data from the applications should be captured into the same
78 file handle as for stdout.
79
80 If preexec_fn is set to a callable object, this object will be called
81 in the child process just before the child is executed.
82
83 If close_fds is true, all file descriptors except 0, 1 and 2 will be
84 closed before the child process is executed.
85
86 if shell is true, the specified command will be executed through the
87 shell.
88
89 If cwd is not None, the current directory will be changed to cwd
90 before the child is executed.
91
92 If env is not None, it defines the environment variables for the new
93 process.
94
95 If universal_newlines is true, the file objects stdout and stderr are
96 opened as a text files, but lines may be terminated by any of '\n',
97 the Unix end-of-line convention, '\r', the Macintosh convention or
98 '\r\n', the Windows convention. All of these external representations
99 are seen as '\n' by the Python program. Note: This feature is only
100 available if Python is built with universal newline support (the
101 default). Also, the newlines attribute of the file objects stdout,
102 stdin and stderr are not updated by the communicate() method.
103
104 The startupinfo and creationflags, if given, will be passed to the
105 underlying CreateProcess() function. They can specify things such as
106 appearance of the main window and priority for the new process.
107 (Windows only)
108
109
110 This module also defines two shortcut functions:
111
112 call(*popenargs, **kwargs):
113 Run command with arguments. Wait for command to complete, then
114 return the returncode attribute.
115
116 The arguments are the same as for the Popen constructor. Example:
117
118 retcode = call(["ls", "-l"])
119
120 check_call(*popenargs, **kwargs):
121 Run command with arguments. Wait for command to complete. If the
122 exit code was zero then return, otherwise raise
123 CalledProcessError. The CalledProcessError object will have the
124 return code in the returncode attribute.
125
126 The arguments are the same as for the Popen constructor. Example:
127
128 check_call(["ls", "-l"])
129
130 Exceptions
131 ----------
132 Exceptions raised in the child process, before the new program has
133 started to execute, will be re-raised in the parent. Additionally,
134 the exception object will have one extra attribute called
135 'child_traceback', which is a string containing traceback information
136 from the childs point of view.
137
138 The most common exception raised is OSError. This occurs, for
139 example, when trying to execute a non-existent file. Applications
140 should prepare for OSErrors.
141
142 A ValueError will be raised if Popen is called with invalid arguments.
143
144 check_call() will raise CalledProcessError, if the called process
145 returns a non-zero return code.
146
147
148 Security
149 --------
150 Unlike some other popen functions, this implementation will never call
151 /bin/sh implicitly. This means that all characters, including shell
152 metacharacters, can safely be passed to child processes.
153
154
155 Popen objects
156 =============
157 Instances of the Popen class have the following methods:
158
159 poll()
160 Check if child process has terminated. Returns returncode
161 attribute.
162
163 wait()
164 Wait for child process to terminate. Returns returncode attribute.
165
166 communicate(input=None)
167 Interact with process: Send data to stdin. Read data from stdout
168 and stderr, until end-of-file is reached. Wait for process to
169 terminate. The optional stdin argument should be a string to be
170 sent to the child process, or None, if no data should be sent to
171 the child.
172
173 communicate() returns a tuple (stdout, stderr).
174
175 Note: The data read is buffered in memory, so do not use this
176 method if the data size is large or unlimited.
177
178 The following attributes are also available:
179
180 stdin
181 If the stdin argument is PIPE, this attribute is a file object
182 that provides input to the child process. Otherwise, it is None.
183
184 stdout
185 If the stdout argument is PIPE, this attribute is a file object
186 that provides output from the child process. Otherwise, it is
187 None.
188
189 stderr
190 If the stderr argument is PIPE, this attribute is file object that
191 provides error output from the child process. Otherwise, it is
192 None.
193
194 pid
195 The process ID of the child process.
196
197 returncode
198 The child return code. A None value indicates that the process
199 hasn't terminated yet. A negative value -N indicates that the
200 child was terminated by signal N (UNIX only).
201
202
203 Replacing older functions with the subprocess module
204 ====================================================
205 In this section, "a ==> b" means that b can be used as a replacement
206 for a.
207
208 Note: All functions in this section fail (more or less) silently if
209 the executed program cannot be found; this module raises an OSError
210 exception.
211
212 In the following examples, we assume that the subprocess module is
213 imported with "from subprocess import *".
214
215
216 Replacing /bin/sh shell backquote
217 ---------------------------------
218 output=`mycmd myarg`
219 ==>
220 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
221
222
223 Replacing shell pipe line
224 -------------------------
225 output=`dmesg | grep hda`
226 ==>
227 p1 = Popen(["dmesg"], stdout=PIPE)
228 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
229 output = p2.communicate()[0]
230
231
232 Replacing os.system()
233 ---------------------
234 sts = os.system("mycmd" + " myarg")
235 ==>
236 p = Popen("mycmd" + " myarg", shell=True)
237 pid, sts = os.waitpid(p.pid, 0)
238
239 Note:
240
241 * Calling the program through the shell is usually not required.
242
243 * It's easier to look at the returncode attribute than the
244 exitstatus.
245
246 A more real-world example would look like this:
247
248 try:
249 retcode = call("mycmd" + " myarg", shell=True)
250 if retcode < 0:
251 print >>sys.stderr, "Child was terminated by signal", -retcode
252 else:
253 print >>sys.stderr, "Child returned", retcode
254 except OSError, e:
255 print >>sys.stderr, "Execution failed:", e
256
257
258 Replacing os.spawn*
259 -------------------
260 P_NOWAIT example:
261
262 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
263 ==>
264 pid = Popen(["/bin/mycmd", "myarg"]).pid
265
266
267 P_WAIT example:
268
269 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
270 ==>
271 retcode = call(["/bin/mycmd", "myarg"])
272
273
274 Vector example:
275
276 os.spawnvp(os.P_NOWAIT, path, args)
277 ==>
278 Popen([path] + args[1:])
279
280
281 Environment example:
282
283 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
284 ==>
285 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
286
287
288 Replacing os.popen*
289 -------------------
290 pipe = os.popen(cmd, mode='r', bufsize)
291 ==>
292 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout
293
294 pipe = os.popen(cmd, mode='w', bufsize)
295 ==>
296 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
297
298
299 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
300 ==>
301 p = Popen(cmd, shell=True, bufsize=bufsize,
302 stdin=PIPE, stdout=PIPE, close_fds=True)
303 (child_stdin, child_stdout) = (p.stdin, p.stdout)
304
305
306 (child_stdin,
307 child_stdout,
308 child_stderr) = os.popen3(cmd, mode, bufsize)
309 ==>
310 p = Popen(cmd, shell=True, bufsize=bufsize,
311 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
312 (child_stdin,
313 child_stdout,
314 child_stderr) = (p.stdin, p.stdout, p.stderr)
315
316
317 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
318 ==>
319 p = Popen(cmd, shell=True, bufsize=bufsize,
320 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
321 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
322
323
324 Replacing popen2.*
325 ------------------
326 Note: If the cmd argument to popen2 functions is a string, the command
327 is executed through /bin/sh. If it is a list, the command is directly
328 executed.
329
330 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
331 ==>
332 p = Popen(["somestring"], shell=True, bufsize=bufsize
333 stdin=PIPE, stdout=PIPE, close_fds=True)
334 (child_stdout, child_stdin) = (p.stdout, p.stdin)
335
336
337 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
338 ==>
339 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
340 stdin=PIPE, stdout=PIPE, close_fds=True)
341 (child_stdout, child_stdin) = (p.stdout, p.stdin)
342
343 The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen,
344 except that:
345
346 * subprocess.Popen raises an exception if the execution fails
347 * the capturestderr argument is replaced with the stderr argument.
348 * stdin=PIPE and stdout=PIPE must be specified.
349 * popen2 closes all filedescriptors by default, but you have to specify
350 close_fds=True with subprocess.Popen.
351
352
353 """
354
355 import sys
356 mswindows = (sys.platform == "win32")
357
358 import os
359 import string
360 import types
361 import traceback
362
363
365 """This exception is raised when a process run by check_call() returns
366 a non-zero exit status. The exit status will be stored in the
367 returncode attribute."""
369 self.returncode = returncode
370 self.cmd = cmd
372 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
373
374
375 if mswindows:
376 try:
377 import threading
378 except ImportError:
379
380
381
382 pass
383 import msvcrt
384 if 0:
385 import pywintypes
386 from win32api import GetStdHandle, STD_INPUT_HANDLE, \
387 STD_OUTPUT_HANDLE, STD_ERROR_HANDLE
388 from win32api import GetCurrentProcess, DuplicateHandle, \
389 GetModuleFileName, GetVersion
390 from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE
391 from win32pipe import CreatePipe
392 from win32process import CreateProcess, STARTUPINFO, \
393 GetExitCodeProcess, STARTF_USESTDHANDLES, \
394 STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE
395 from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0
396 else:
397 from _subprocess import *
406 else:
407 import select
408 import errno
409 import fcntl
410 import pickle
411
412 try:
413 fcntl.F_GETFD
414 except AttributeError:
415 fcntl.F_GETFD = 1
416
417 try:
418 fcntl.F_SETFD
419 except AttributeError:
420 fcntl.F_SETFD = 2
421
422 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "CalledProcessError"]
423
424 try:
425 MAXFD = os.sysconf("SC_OPEN_MAX")
426 except KeyboardInterrupt:
427 raise
428 except:
429 MAXFD = 256
430
431
432 try:
433 False
434 except NameError:
435 False = 0
436 True = 1
437
438 try:
439 isinstance(1, int)
440 except TypeError:
442 return type(obj) == type(1)
444 return type(obj) in (type(1), type(1L))
445 else:
447 return isinstance(obj, int)
449 return isinstance(obj, (int, long))
450
451 try:
452 types.StringTypes
453 except AttributeError:
454 try:
455 types.StringTypes = (types.StringType, types.UnicodeType)
456 except AttributeError:
457 types.StringTypes = (types.StringType,)
460 else:
463
464 _active = []
465
467 for inst in _active[:]:
468 if inst.poll(_deadstate=sys.maxint) >= 0:
469 try:
470 _active.remove(inst)
471 except ValueError:
472
473
474 pass
475
476 PIPE = -1
477 STDOUT = -2
478
479
480 -def call(*popenargs, **kwargs):
481 """Run command with arguments. Wait for command to complete, then
482 return the returncode attribute.
483
484 The arguments are the same as for the Popen constructor. Example:
485
486 retcode = call(["ls", "-l"])
487 """
488 return apply(Popen, popenargs, kwargs).wait()
489
490
492 """Run command with arguments. Wait for command to complete. If
493 the exit code was zero then return, otherwise raise
494 CalledProcessError. The CalledProcessError object will have the
495 return code in the returncode attribute.
496
497 The arguments are the same as for the Popen constructor. Example:
498
499 check_call(["ls", "-l"])
500 """
501 retcode = apply(call, popenargs, kwargs)
502 cmd = kwargs.get("args")
503 if cmd is None:
504 cmd = popenargs[0]
505 if retcode:
506 raise CalledProcessError(retcode, cmd)
507 return retcode
508
509
511 """
512 Translate a sequence of arguments into a command line
513 string, using the same rules as the MS C runtime:
514
515 1) Arguments are delimited by white space, which is either a
516 space or a tab.
517
518 2) A string surrounded by double quotation marks is
519 interpreted as a single argument, regardless of white space
520 contained within. A quoted string can be embedded in an
521 argument.
522
523 3) A double quotation mark preceded by a backslash is
524 interpreted as a literal double quotation mark.
525
526 4) Backslashes are interpreted literally, unless they
527 immediately precede a double quotation mark.
528
529 5) If backslashes immediately precede a double quotation mark,
530 every pair of backslashes is interpreted as a literal
531 backslash. If the number of backslashes is odd, the last
532 backslash escapes the next double quotation mark as
533 described in rule 3.
534 """
535
536
537
538 result = []
539 needquote = False
540 for arg in seq:
541 bs_buf = []
542
543
544 if result:
545 result.append(' ')
546
547 needquote = (" " in arg) or ("\t" in arg)
548 if needquote:
549 result.append('"')
550
551 for c in arg:
552 if c == '\\':
553
554 bs_buf.append(c)
555 elif c == '"':
556
557 result.append('\\' * len(bs_buf)*2)
558 bs_buf = []
559 result.append('\\"')
560 else:
561
562 if bs_buf:
563 result.extend(bs_buf)
564 bs_buf = []
565 result.append(c)
566
567
568 if bs_buf:
569 result.extend(bs_buf)
570
571 if needquote:
572 result.extend(bs_buf)
573 result.append('"')
574
575 return string.join(result, '')
576
577
578 try:
579 object
580 except NameError:
583
585 - def __init__(self, args, bufsize=0, executable=None,
586 stdin=None, stdout=None, stderr=None,
587 preexec_fn=None, close_fds=False, shell=False,
588 cwd=None, env=None, universal_newlines=False,
589 startupinfo=None, creationflags=0):
590 """Create new Popen instance."""
591 _cleanup()
592
593 self._child_created = False
594 if not is_int_or_long(bufsize):
595 raise TypeError("bufsize must be an integer")
596
597 if mswindows:
598 if preexec_fn is not None:
599 raise ValueError("preexec_fn is not supported on Windows "
600 "platforms")
601 if close_fds:
602 raise ValueError("close_fds is not supported on Windows "
603 "platforms")
604 else:
605
606 if startupinfo is not None:
607 raise ValueError("startupinfo is only supported on Windows "
608 "platforms")
609 if creationflags != 0:
610 raise ValueError("creationflags is only supported on Windows "
611 "platforms")
612
613 self.stdin = None
614 self.stdout = None
615 self.stderr = None
616 self.pid = None
617 self.returncode = None
618 self.universal_newlines = universal_newlines
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635 (p2cread, p2cwrite,
636 c2pread, c2pwrite,
637 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
638
639 self._execute_child(args, executable, preexec_fn, close_fds,
640 cwd, env, universal_newlines,
641