o bt@sUdZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z m Z m Z ddl mZddlmZmZmZddlmZddlmZddlmZmZdd lmZmZdd lmZdd lm Z e!rme"d zddl#Z$Wn e"y~dZ#Ynwe$Z#z ddl%Z&ddl'Z'Wn e"ydZ%Ynwe&Z%ej(Z(iZ)e e*e fe+d <ddZ,ddZ-ddZ.Gdddej/Z0Gdddej/Z1GdddeZ2GdddZ3e3Z4ddZ5eeGddde2Z6eeGd d!d!ej/e2Z7dS)"z UNIX Process management. Do NOT use this module directly - use reactor.spawnProcess() instead. Maintainer: Itamar Shtull-Trauring N)CallableDictOptional) implementer)abstracterrorfdesc) BaseProcess)IProcessTransport)CONNECTION_DONECONNECTION_LOST)failurelog)platform) switchUIDz^twisted.internet.process does not work on Windows. Use the reactor.spawnProcess() API instead.reapProcessHandlerscCsttD]}|qdS)z( Reap all registered processes. N)listrvalues reapProcess)processr:/usr/lib/python3/dist-packages/twisted/internet/process.pyreapAllProcesses=s rcCs|tvrtdz t|tj\}}Wnty2td|dt|dur.YdSd}Ynw|r<| |dS|t|<dS)z Register a process handler for the given pid, in case L{reapAllProcesses} is called. @param pid: the pid of the process. @param process: a process handler. z.Try to register an already registered process.Failed to reap :N) r RuntimeErroroswaitpidWNOHANG BaseExceptionrmsgerr processEnded)pidrauxPIDstatusrrrregisterReapProcessHandlerGs  r&cCs&|tvr t||kstdt|=dS)zd Unregister a process handler previously registered with L{registerReapProcessHandler}. z+Try to unregister a process not registered.N)rr)r#rrrrunregisterReapProcessHandlerbs r'c@sNeZdZdZdZdZdZdddZddZd d Z d d Z d dZ ddZ dS) ProcessWritera (Internal) Helper class to write into a Process's input pipe. I am a helper which describes a selectable asynchronous writer to a process's input pipe, including stdin. @ivar enableReadHack: A flag which determines how readability on this write descriptor will be handled. If C{True}, then readability may indicate the reader for this write descriptor has been closed (ie, the connection has been lost). If C{False}, then readability events are ignored. rFcCstj||t|||_||_||_t t | j s$d|_n|r*d|_nz t | dWn ty@d|_Ynw|jrJ|dSdS)zJ Initialize, specifying a Process instance to connect to. FTrN)rFileDescriptor__init__rsetNonBlockingprocnamefdstatS_ISFIFOrfstatfilenost_modeenableReadHackreadOSError startReading)selfreactorr-r.r3 forceReadHackrrrr+~s"    zProcessWriter.__init__cC|jS)z< Return the fileno() of my process's stdin. r/r9rrrr3zProcessWriter.filenocCs,t|j|}|t|kr|jr||Sz6 Write some data to the open process. )r writeToFDr/lenr5r8)r9datarvrrr writeSomeDataszProcessWriter.writeSomeDatacCs|tj||dSN) stopReadingrr*writer9rCrrrrHszProcessWriter.writecCs|jrtS|dS)a The only way a write pipe can become "readable" is at EOF, because the child has closed it, and we're using a reactor which doesn't distinguish between readable and closed (such as the select reactor). Except that's not true on linux < 2.6.11. It has the following characteristics: write pipe is completely empty => POLLOUT (writable in select), write pipe is not completely empty => POLLIN (readable in select), write pipe's reader closed => POLLIN|POLLERR (readable and writable in select) That's what this funky code is for. If linux was not broken, this function could be simply "return CONNECTION_LOST". N)r5r rGr>rrrdoReads zProcessWriter.doReadcCs.t|jtj|||j|j|dS)z= See abstract.FileDescriptor.connectionLost. N) r setBlockingr/rr*connectionLostr-childConnectionLostr.r9reasonrrrrLs zProcessWriter.connectionLostN)F) __name__ __module__ __qualname____doc__ connectedicr5r+r3rErHrJrLrrrrr(ls    r(c@sLeZdZdZdZddZddZddZd d Zd d Z d dZ ddZ dS) ProcessReaderzx ProcessReader I am a selectable representation of a process's output pipe, such as stdout and stderr. TcCs6tj||t|||_||_||_|dS)zA Initialize, specifying a process to connect to. N) rr*r+rr,r-r.r/r8)r9r:r-r.r3rrrr+s   zProcessReader.__init__cCr<)z= Return the fileno() of my process's stderr. r=r>rrrr3r?zProcessReader.filenocCs|dksJtS)N)r rIrrrrEs zProcessReader.writeSomeDatacCst|j|jS)z@ This is called when the pipe becomes readable. )r readFromFDr/ dataReceivedr>rrrrJszProcessReader.doReadcCs|j|j|dSrF)r-childDataReceivedr.rIrrrrYzProcessReader.dataReceivedcCs>|jr|jsd|_||jd|jttdSdSdS)Nr)r) rT disconnectingrGr: callLaterrLr Failurer r>rrrloseConnections zProcessReader.loseConnectioncCs"tj|||j|j|dS)zk Close my end of the pipe, signal the Process (which signals the ProcessProtocol). N)rr*rLr-rMr.rNrrrrLzProcessReader.connectionLostN) rPrQrRrSrTr+r3rErJrYr_rLrrrrrVs  rVc@sleZdZUdZdZeeed<dZddZ ddZ dd Z d d Z d d Z ddZddZdefddZdS) _BaseProcessz0 Base class for Process and PTYProcess. Nr%c Csz+z t|jtj\}}Wnty)}z|jtjkrd}nWYd}~nd}~wwWntyDt d|jdt d}Ynw|rSt ||| |dSdS)a Try to reap a process (without blocking) via waitpid. This is called when sigchild is caught or a Process object loses its "connection" (stdout is closed) This ought to result in reaping all zombie processes, since it will be called twice as often as it needs to be. (Unfortunately, this is a slightly experimental approach, since UNIX has no way to be really sure that your process is going to go away w/o blocking. I don't want to block.) Nrr) rrr#rr7errnoECHILDrrr r!r'r")r9r#r%errrrs(     z_BaseProcess.reapProcesscCsHd}}t|rt|}nt|}|s|rt|||St|SrF)r WIFEXITED WEXITSTATUSWTERMSIGrProcessTerminated ProcessDone)r9r%exitCodesigrrr _getReason6s    z_BaseProcess._getReasonc Csj|dvr ttd|}|jdurtz t|j|WdSty4}z |jtjkr/td}~ww)a* Send the given signal C{signalID} to the process. It'll translate a few signals ('HUP', 'STOP', 'INT', 'KILL', 'TERM') from a string representation to its int value, otherwise it'll pass directly the value provided @type signalID: C{str} or C{int} )HUPSTOPINTKILLTERMSIGN) getattrsignalr#ProcessExitedAlreadyrkillr7rbESRCH)r9signalIDrdrrr signalProcess@s   z_BaseProcess.signalProcesscCs4tdtjD]}t|tjkrt|tjqdSNr))rangertNSIG getsignalSIG_IGNSIG_DFL)r9 signalnumrrr_resetSignalDispositionUs z$_BaseProcess._resetSignalDispositionc Ks&t}tzt|_Wnty|rtw|jdkrzt d|j d i|| ||||||WnFtyz3t j tdddd} d|t|t|} | | tj| d| td D]} t| qmWn tyYnwYnwtd |rtd |_dS) a Fork and then exec sub-process. @param path: the path where to run the new process. @type path: L{bytes} or L{unicode} @param uid: if defined, the uid used to run the new process. @type uid: L{int} @param gid: if defined, the gid used to run the new process. @type gid: L{int} @param executable: the executable to run in a new process. @type executable: L{str} @param args: arguments used to create the new process. @type args: L{list}. @param environment: environment used for the new process. @type environment: L{dict}. @param kwargs: keyword arguments to L{_setupChild} method. rNwbzutf-8)encodingz)Upon execvpe {} {} in environment id {} :filer)r)gc isenableddisablerforkr#renablesyssettrace _setupChild _execChildio TextIOWrapperfdopenformatstridrH traceback print_excflushr{close_exitr%) r9pathuidgid executableargs environmentkwargscollectorEnabledstderrr r/rrr_fork`sF          * z_BaseProcess._forkcOt)zB Setup the child process. Override in subclasses. NotImplementedError)r9rrrrrrr?z_BaseProcess._setupChildcCsn|rt||dus|dur.|durt}|durt}tdtdt||t|||dS)z? The exec() which is done in the forked child. Nr)rchdirgeteuidgetegidsetuidsetgidrexecvpe)r9rrrrrrrrrrs    z_BaseProcess._execChildreturncCsd|jj|j|jS)z5 String representation of a process. z<{} pid={} status={}>)r __class__rPr#r%r>rrr__repr__s z_BaseProcess.__repr__)rPrQrRrSr%rint__annotations__r#rrlryrrrrrrrrrrras   _rac@sPeZdZdZejZejZeZddZ ddZ ddZ dd Z d d Z d d ZdS) _FDDetectorav This class contains the logic necessary to decide which of the available system techniques should be used to detect the open file descriptors for the current process. The chosen technique gets monkey-patched into the _listOpenFDs method of this class so that the detection only needs to occur once. @ivar listdir: The implementation of listdir to use. This gets overwritten by the test cases. @ivar getpid: The implementation of getpid to use, returns the PID of the running process. @ivar openfile: The implementation of open() to use, by default the Python builtin. cCs|j|j|jg|_dSrF)_procFDImplementation_devFDImplementation_fallbackFDImplementation_implementationsr>rrrr+s z_FDDetector.__init__cCs||_|S)z Return an iterable of file descriptors which I{may} be open in this process. This will try to return the fewest possible descriptors without missing any. )_getImplementation _listOpenFDsr>rrrrs z_FDDetector._listOpenFDsc Csp|jD]2}z|}Wn tyYqw|dd |}Wdn1s(wY||kr5|Sq|S)a Pick a method which gives correct results for C{_listOpenFDs} in this runtime environment. This involves a lot of very platform-specific checks, some of which may be relatively expensive. Therefore the returned method should be saved and re-used, rather than always calling this method to determine what it is. See the implementation for the details of how a method is selected. z /dev/nullrN)rropenfile)r9implbeforeafterrrrrs   z_FDDetector._getImplementationcCsd}dd||D}|S)z Simple implementation for systems where /dev/fd actually works. See: http://www.freebsd.org/cgi/man.cgi?fdescfs z/dev/fdcSg|]}t|qSrr.0r/rrr +z4_FDDetector._devFDImplementation..)listdir)r9dnameresultrrrr%sz _FDDetector._devFDImplementationcCs"d|f}dd||DS)zk Simple implementation for systems where /proc/pid/fd exists (we assume it works). z /proc/%d/fdcSrrrrrrrr4rz5_FDDetector._procFDImplementation..)getpidr)r9rrrrr.r`z!_FDDetector._procFDImplementationcCsHzddl}Wntyd}Yt|Swtd||jd}t|S)au Fallback implementation where either the resource module can inform us about the upper bound of how many FDs to expect, or where we just guess a constant maximum if there is no resource module. All possible file descriptors from 0 to that upper bound are returned with no attempt to exclude invalid file descriptor values. rNir))resource ImportErrormin getrlimit RLIMIT_NOFILEr{)r9rmaxfdsrrrr6s  z%_FDDetector._fallbackFDImplementationN)rPrQrRrSrrropenrr+rrrrrrrrrrs  rcCstS)zZ Use the global detector object to figure out which FD implementation to use. )detectorrrrrrrNsrc@seZdZdZdZdZdZdZeZ e Z   d+ddZ ddZ d d Zd d Zd dZddZddZddZddZddZddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*ZdS),Processa An operating-system Process. This represents an operating-system process with arbitrary input/output pipes connected to it. Those pipes may represent standard input, standard output, and standard error, or any other file descriptor. On UNIX, this is implemented using fork(), exec(), pipe() and fcntl(). These calls may not exist elsewhere so this code is not cross-platform. (also, windows can only select on sockets...) FrNc  sJ|sd| vs Jd| vsJt||i|_i} | dur'dddd} |j} | r1td| gfdd} i} zw| D]V\}}| rMtd|||dkri| \}}| r`td ||f|| |<|| |<q@|dkr| \}}| r|td ||f|| |<|| |<q@t|tksJ|d || |<q@| rtd | | rtd | |j ||||||| d Wnt yD]} t | qw||_ | D]3\}}t | || |dkr|||||}||j|<| |dkr|j||||dd}||j|<qz|j dur |j |Wnt ytYnwt|j|dS)  Spawn an operating-system process. This is where the hard work of disconnecting all currently open files / forking / executing the new process happens. (This is executed automatically when a Process is instantiated.) This will also run the subprocess as a given user ID and group ID, if specified. (Implementation Note: this doesn't support all the arcane nuances of setXXuid on UNIX: it will assume that either your effective or real UID is 0.) rwN)rr)rchildFDscs"t\}}||g||fSrF)rpipeextend)rr _openedPipesrrrs zProcess.__init__..pipez[%d]zreadFD=%d, writeFD=%dz should be an intfdmaphelpers)rT)r;)rrar+pipesdebugprintitemstyperrrrrprotoprocessReaderFactoryprocessWriterFactorymakeConnectionrr!r&r#)r9r:rrrrrrrrrrrrchildFDtargetreadFDwriteFDparentFDreaderwriterrrrr+ns                    zProcess.__init__c Cs|j}|r tj}|d|}tD] }||vrq|r$||kr$qzt|Wqt y4Yqw|r>t d||dt | D][}||}||kr^|rXt d||dt |qD||vrt|}|rut d||f|dt|t|D] \} } | |kr||| <q|rt d||f|dt||qDg} |D]}|| vr|| vr| |q|rt d| |d| D]}t|q|dS) a fdmap[childFD] = parentFD The child wants to end up with 'childFD' attached to what used to be the parent's parentFD. As an example, a bash command run like 'command 2>&1' would correspond to an fdmap of {0:0, 1:1, 2:1}. 'command >foo.txt' would be {0:0, 1:os.open('foo.txt'), 2:2}. This is accomplished in two steps:: 1. close all file descriptors that aren't values of fdmap. This means 0 .. maxfds (or just the open fds within that range, if the platform supports '/proc//fd'). 2. for each childFD:: - if fdmap[childFD] == childFD, the descriptor is already in place. Make sure the CLOEXEC flag is not set, then delete the entry from fdmap. - if childFD is in fdmap.values(), then the target descriptor is busy. Use os.dup() to move it elsewhere, update all fdmap[childFD] items that point to it, then close the original. Then fall through to the next case. - now fdmap[childFD] is not in fdmap.values(), and is free. Use os.dup2() to move it to the right place, then close the original. zstarting _setupChild rrz%d already in placezos.dup(%d) -> %dzos.dup2(%d,%d)oldN) debug_childrrrHrrr3rrrrsortedkeysr_unsetCloseOnExecduprrdup2appendr) r9rrerrfddestListr/childr newtargetcprrrrrs\            zProcess._setupChildcCs|j||dSrFrrHr9rrCrrr writeToChildEr[zProcess.writeToChildcCs ||jvr|j|dSdSrF)rr_)r9rrrr closeChildFDHs zProcess.closeChildFDcC&|jD] }t|tr|qdSrF)rr isinstancerVrGr9rrrrpauseProducingP  zProcess.pauseProducingcCrrF)rrrrVr8rrrrresumeProducingUrzProcess.resumeProducingcCs|ddS)zD Call this to close standard input on this process. rNrr>rrr closeStdin[zProcess.closeStdincC|ddSrzrr>rrr closeStdoutazProcess.closeStdoutcCrNrrr>rrr closeStderrdr zProcess.closeStderrcCs|||dSrF)rr r r>rrrr_gs zProcess.loseConnectioncC"d|jvr|jd|dSdSz Call this to write to standard input on this process. NOTE: This will silently lose data if there is no standard input. rNrrIrrrrHl z Process.writecCs,d|jvr|jd||dS|dS)z Call this to register producer for standard input. If there is no standard input producer.stopProducing() will be called immediately. rN)rregisterProducer stopProducing)r9producer streamingrrrrus  zProcess.registerProducercCs d|jvr|jddSdS)z= Call this to unregister producer for standard input.rN)runregisterProducerr>rrrrs zProcess.unregisterProducercCr r)r writeSequence)r9seqrrrrrzProcess.writeSequencecCs|j||dSrFrrZ)r9r.rCrrrrZszProcess.childDataReceivedcCsTt|j||j|=z|j|Wn ty#tYnw| dSrF) rrrr3rrMrrr!maybeCallProcessEnded)r9rrOrrrrMs   zProcess.childConnectionLostcCs*|jrdS|js|dSt|dSrF)r lostProcessrrarr>rrrrs zProcess.maybeCallProcessEndedcCrrFrr>rrrgetHostzProcess.getHostcCrrFrr>rrrgetPeerrzProcess.getPeerNNN)rPrQrRrSrrr%r#r(rrVrr+rrrrrrr r r_rHrrrrZrMrrrrrrrrVs<   w`     rc@seZdZdZdZdZ   dddZddZdd Zd d Z d d Z ddZ ddZ ddZ ddZddZddZddZdS) PTYProcessz< An operating-system Process that uses PTY support. rNc Cstdurt| ttfstdtj||t||t| ttfr)| \} } } nt \} } z|j ||||||| | dWnt yWt| ttfsVt | t | wt | t| | |_|d|_d|_z|j|Wn t ytYnwt|j|dS)rNz:cannot use PTYProcess on platforms without the pty module.)masterfdslavefdr)r)ptyrtuplerrrr*r+raopenptyrrrrrr,r/r8rTr%rrrr!r&r#) r9r:rrrrrrrusePTYrr _rrrr+sL          zPTYProcess.__init__c Cst|tt|tjdtdD] }||kr t|qt|dt|dt|dt D]}|dkrMzt|Wq6t yLYq6wq6| dS)a9 Set up child process after C{fork()} but before C{exec()}. This involves: - closing C{masterfd}, since it is not used in the subprocess - creating a new session with C{os.setsid} - changing the controlling terminal of the process (and the new session) to point at C{slavefd} - duplicating C{slavefd} to standard input, output, and error - closing all other open file descriptors (according to L{_listOpenFDs}) - re-setting all signal handlers to C{SIG_DFL} @param masterfd: The master end of a PTY file descriptors opened with C{openpty}. @type masterfd: L{int} @param slavefd: The slave end of a PTY opened with C{openpty}. @type slavefd: L{int} rrr)rN) rrsetsidfcntlioctltermios TIOCSCTTYr{rrrr)r9rr r/rrrrs&         zPTYProcess._setupChildcCdSrFrr>rrrr0szPTYProcess.closeStdincCr,rFrr>rrrr 5zPTYProcess.closeStdoutcCr,rFrr>rrrr 8r-zPTYProcess.closeStderrcstjfddS)zM Called when my standard output stream is ready for reading. csjd|Srzr)rCr>rr@sz#PTYProcess.doRead..)rrXr/r>rr>rrJ;szPTYProcess.doReadcCr<)zR This returns the file number of standard output on this process. r=r>rrrr3Cr?zPTYProcess.filenocCs|jdkr t|dSdSr )rrarr>rrrrIs z PTYProcess.maybeCallProcessEndedcCs4tj||t|j|jd7_|dS)zU I call this to clean up when one or all of my connections has died. r)N)rr*rLrrr/rrrNrrrrLSs  zPTYProcess.connectionLostcCst|j|Sr@)rrAr/rIrrrrE\rzPTYProcess.writeSomeDatacCrrFr)r9 descriptorrrrrbrzPTYProcess.closeChildFDcCrrFrrrrrrfrzPTYProcess.writeToChildr)rPrQrRrSr%r#r+rrr r rJr3rrLrErrrrrrrs&  C0   r)8rSrbrrrrtr0rrtypingrrrzope.interfacertwisted.internetrrrtwisted.internet._baseprocessr twisted.internet.interfacesr twisted.internet.mainr r twisted.pythonr rtwisted.python.runtimertwisted.python.utilr isWindowsrr!_ptyr(_fcntlr*rurrrrr&r'r*r(rVrarrrrrrrrrsj           h<Rj_