o bY@sdZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZmZddlmZddlmZdd lmZmZmZdd lmZmZefZd Z d Z!d Z"dZ#dZ$dZ%dZ&dZ'dZ(dZ)dZ*dZ+dZ,ee ddddddddZ-ia.ia/ddZ0d d!Z1d"d#Z2d$d%Z3d&d'Z4dBd(d)Z5d*d+Z6d,d-Z7Gd.d/d/Z8e eGd0d1d1Z9e eGd2d3d3Z:Gd4d5d5Z;Gd6d7d7ZGd:d;d;Z?Gdd?ZCe?ddfd@dAZDdS)CaM S-expression-based persistence of python objects. It does something very much like L{Pickle}; however, pickle's main goal seems to be efficiency (both in space and time); jelly's main goals are security, human readability, and portability to other environments. This is how Jelly converts various objects to s-expressions. Boolean:: True --> ['boolean', 'true'] Integer:: 1 --> 1 List:: [1, 2] --> ['list', 1, 2] String:: "hello" --> "hello" Float:: 2.3 --> 2.3 Dictionary:: {'a': 1, 'b': 'c'} --> ['dictionary', ['b', 'c'], ['a', 1]] Module:: UserString --> ['module', 'UserString'] Class:: UserString.UserString --> ['class', ['module', 'UserString'], 'UserString'] Function:: string.join --> ['function', 'join', ['module', 'string']] Instance: s is an instance of UserString.UserString, with a __dict__ {'data': 'hello'}:: ["UserString.UserString", ['dictionary', ['data', 'hello']]] Class Method: UserString.UserString.center:: ['method', 'center', ['None'], ['class', ['module', 'UserString'], 'UserString']] Instance Method: s.center, where s is an instance of UserString.UserString:: ['method', 'center', ['instance', ['reference', 1, ['class', ['module', 'UserString'], 'UserString']], ['dictionary', ['data', 'd']]], ['dereference', 1]] The Python 2.x C{sets.Set} and C{sets.ImmutableSet} classes are serialized to the same thing as the builtin C{set} and C{frozenset} classes. (This is only relevant if you are communicating with a version of jelly running on an older version of Python.) @author: Glyph Lefkowitz N)reduce) implementer)Version)NotKnown _Container _Dereference_DictKeyAndValue_InstanceMethod_Tuple) nativeString)deprecatedModuleAttribute)namedAny namedObjectqual) IJellyable IUnjellyableNoneclassmodulefunctions dereferences persistents references dictionaryslistsetstuplesinstance frozensetTwistedz'instance_atom is unused within Twisted.ztwisted.spread.jelly instance_atoms unpersistablecCst|tr ||SdS)a Given an object, if that object is a type, return a new, blank instance of that type which has not had C{__init__} called on it. If the object is not a type, return L{None}. @param cls: The type (or class) to create an instance of. @type cls: L{type} or something else that cannot be instantiated. @return: a new blank instance or L{None} if C{cls} is not a class or type. N) isinstancetype__new__)clsr6/usr/lib/python3/dist-packages/twisted/spread/jelly.py _createBlanks r!cs,t|fdd}td|}||S)z Make a new instance of a class without calling its __init__ method. @param state: A C{dict} used to update C{inst.__dict__} either directly or via C{__setstate__}, if available. @return: A new instance of C{cls}. cs |_dSN__dict__)stateinstancerr defaultSetter z#_newInstance..defaultSetter __setstate__)r!getattr)rr%r(setterrr&r _newInstances   r-cCs.t|t}|r t|}t|ts|d}|S)Nutf-8)rrrbytesencode) classnamepisObjectrrr _maybeClasss   r3cCt|}|t|<t|dS)at Set which local class will represent a remote type. If you have written a Copyable class that you expect your client to be receiving, write a local "copy" class to represent it, then call:: jellier.setUnjellyableForClass('module.package.Class', MyCopier). Call this at the module level immediately after its class definition. MyCopier should be a subclass of RemoteCopy. The classname may be a special tag returned by 'Copyable.getTypeToCopyFor' rather than an actual classname. This call is also for cached classes, since there will be no overlap. The rules are the same. N)r3unjellyableRegistryglobalSecurity allowTypes) classname unjellyablerrr setUnjellyableForClasssr:cCr4)a Set the factory to construct a remote instance of a type:: jellier.setUnjellyableFactoryForClass('module.package.Class', MyFactory) Call this at the module level immediately after its class definition. C{copyFactory} should return an instance or subclass of L{RemoteCopy}. Similar to L{setUnjellyableForClass} except it uses a factory instead of creating an instance. N)r3unjellyableFactoryRegistryr6r7)r8 copyFactoryrrr setUnjellyableFactoryForClasssr=c Csn|dur|j}|r d|}t|D]#}t||}zt||}Wn ty(Yqw|r4t|||qdS)a Set all classes in a module derived from C{baseClass} as copiers for a corresponding remote class. When you have a hierarchy of Copyable (or Cacheable) classes on one side, and a mirror structure of Copied (or RemoteCache) classes on the other, use this to setUnjellyableForClass all your Copieds for the Copyables. Each copyTag (the "classname" argument to getTypeToCopyFor, and what the Copyable's getTypeToCopyFor returns) is formed from adding a prefix to the Copied's class name. The prefix defaults to module.__name__. If you wish the copy tag to consist of solely the classname, pass the empty string ''. @param module: a module object from which to pull the Copied classes. (passing sys.modules[__name__] might be useful) @param baseClass: the base class from which all your Copied classes derive. @param prefix: the string prefixed to classnames to form the unjellyableRegistry. Nz%s.)__name__dirr+ issubclass TypeErrorr:)module baseClassprefixnameloadedyesrrr setUnjellyableForClassTrees   rHcCsPt|dr |}n|j}||}|t|jd||g| ||S)zM Utility method to default to 'normal' state rules in serialization. __getstate__r.) hasattrrIr$prepareextendr __class__r0jellypreserve)instjellierr%sxprrr getInstanceStates     rScCs0||d}t|dr|||S||_|S)zO Utility method to default to 'normal' state rules in unserialization. r*)unjellyrJr*r$)rP unjellier jellyListr%rrr setInstanceStates   rXc@s&eZdZdZddZdefddZdS) Unpersistablezd This is an instance of a class that comes back when something couldn't be unpersisted. cCs ||_dS)zY Initialize an unpersistable object with a descriptive C{reason} string. N)reason)selfrZrrr __init__%s zUnpersistable.__init__returncCsdt|jS)NzUnpersistable(%s))reprrZr[rrr __repr__+zUnpersistable.__repr__N)r> __module__ __qualname____doc__r\strr`rrrr rYsrYc@ eZdZdZddZddZdS) Jellyablezc Inherit from me to Jelly yourself directly with the `getStateFor' convenience method. cCs|jSr"r#)r[rQrrr getStateFor6szJellyable.getStateForcCs<||}|t|jd|||g|||S)zH @see: L{twisted.spread.interfaces.IJellyable.jellyFor} r.)rKrLrrMr0rNrhrO)r[rQrRrrr jellyFor9s  zJellyable.jellyForN)r>rbrcrdrhrirrrr rg/ rgc@rf) Unjellyablezf Inherit from me to Unjelly yourself directly with the C{setStateFor} convenience method. cCs ||_dSr"r#)r[rVr%rrr setStateForNr)zUnjellyable.setStateForcCs||d}||||S)z Perform the inverse operation of L{Jellyable.jellyFor}. @see: L{twisted.spread.interfaces.IUnjellyable.unjellyFor} rT)rUrl)r[rVrWr%rrr unjellyForQs zUnjellyable.unjellyForN)r>rbrcrdrlrmrrrr rkGrjrkc@sZeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ dddZ dS)_JellierzC (Internal) This class manages state for a call to jelly() cCs.||_i|_i|_i|_d|_||_||_dS)z Initialize. rTN)taster preservedcookedcooker_ref_idpersistentStoreinvoker)r[rortrurrr r\as z_Jellier.__init__cCsR|jt|}t|}|j}|jd|_t||g|dd<t|g|jt|<|S)a (internal) Backreference an object. Notes on this method for the hapless future maintainer: If I've already gone through the prepare/preserve cycle on the specified object (it is being referenced after the serializer is "done with" it, e.g. this reference is NOT circular), the copy-in-place of aList is relevant, since the list being modified is the actual, pre-existing jelly expression that was returned for that object. If not, it's technically superfluous, since the value in self.preserved didn't need to be set, but the invariant that self.preserved[id(object)] is a list is convenient because that means we don't have to test and create it or not create it here, creating fewer code-paths. that's why self.preserved is always set to a list. Sorry that this code is so hard to follow, but Python objects are tricky to persist correctly. -glyph rTN)rpidcopyrsreference_atomdereference_atomrq)r[objectaListnewListrefidrrr _cookps  z_Jellier._cookcCs g|jt|<||jt|<gS)a] (internal) Create a list for persisting an object to. This will allow backreferences to be made internal to the object. (circular references). The reason this needs to happen is that we don't generate an ID for every object, so we won't necessarily know which ID the object will have in the future. When it is 'cooked' ( see _cook ), it will be assigned an ID, and the temporary placeholder list created here will be modified in-place to create an expression that gives this object an ID: [reference id# [object-jelly]]. )rprvrr)r[rzrrr rKsz_Jellier.preparecCsDt||jvr||jt|d<|jt|}|S||jt|<|S)zQ (internal) Mark an object's persistent list for later referral. )rvrqrp)r[rzsexprrr rOs z_Jellier.preservecCs>t|}||jvr|j|S||jvr|||j|SdSr")rvrqrpr~)r[objobjIdrrr _checkMutables     z_Jellier._checkMutablec Cst|tr||}|r|S||St|}|jt|dr|t t t fvr,|St|t j rH|j}|j}|j}d|j||||gS|turSd|dgSt|tdr]dgSt|t jrmd|jd|jgSt|t jrxd|jgS|turd |rd pd gS|tjur|jrtd d ddd|j|j|j|j |j!|j"|j#fDdgS|tj$ur|jrtd dddd|j |j!|j"|j#fDdgS|tj%urdddd|j|j|jfDdgS|tj&urdddd|j'|j(|j)fDdgSt*|trdt|dgS|t+j,ur|-|S||}|r)|S|.|}|t/ur=|0|1t2|n|t3urL|0|1t4|n|t5vrn|6t7|8D]\}} |6|||| gqZnq|t9ur}|0|1t:|nb|t;ur|0|1t<|nSt|jd} d} |j=r|=||} | dur|6t>|6| n/|j?|jr|6| t@|dr|A} n|jB} |6|| n |Cdt|j||D||StEd|d|)Nr.smethodunicodeUTF-8rr.rbooleantruefalsez2Currently can't jelly datetime objects with tzinfodatetime cSg|]}t|qSrre.0xrrr z"_Jellier.jelly..timecSrrrrrrr rrdatecSrrrrrrr r timedeltacSrrrrrrr r rrrIz$instance of class %s deemed insecurezType not allowed for object: )Frrgrrirro isTypeAllowedrr0r/intfloattypes MethodType__self____func__rMr>rNre FunctionTyperbrc ModuleTypebooldatetimetzinfoNotImplementedErrorjoinyearmonthdayhourminutesecond microsecondtimedate timedeltadaysseconds microsecondsr@decimalDecimal jelly_decimalrKlistrL_jellyIterable list_atomtuple tuple_atom DictTypesappenddictionary_atomitemssetset_atom frozensetfrozenset_atomrtpersistent_atomisClassAllowedrJrIr$ unpersistablerO InsecureJelly) r[rpreRefobjTypeaSelfaFuncaClassrRkeyval className persistentr%rrr rNs                                z_Jellier.jellyccs"|V|D]}||VqdS)a Jelly an iterable object. @param atom: the identifier atom of the object. @type atom: C{str} @param obj: any iterable object. @type obj: C{iterable} @return: a generator of jellied data. @rtype: C{generator} N)rN)r[atomritemrrr r<s  z_Jellier._jellyIterablecCs0|\}}}tdd|}|r| }d||gS)z Jelly a decimal object. @param d: a decimal object to serialize. @type d: C{decimal.Decimal} @return: jelly for the decimal object. @rtype: C{list} cSs |d|S)N r)leftrightrrr Xs z(_Jellier.jelly_decimal..decimal)as_tupler)r[dsigngutsexponentvaluerrr rMs   z_Jellier.jelly_decimalNcCs8|durg}|tt|tr|d}|||S)z (internal) Returns an sexp: (unpersistable "reason"). Utility method for making note that a particular object could not be serialized. Nr.)runpersistable_atomrrer0)r[rZrRrrr r]s    z_Jellier.unpersistabler") r>rbrcrdr\r~rKrOrrNrrrrrrr rn\srnc@seZdZddZddZddZddZd d Zd d Zd dZ ddZ ddZ ddZ ddZ ddZddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zd/d0Zd1d2Zd3d4Zd5d6Zd7d8Zd9d:Zd;S)< _UnjelliercCs"||_||_i|_g|_||_dSr")ropersistentLoad references postCallbacksru)r[rorrurrr r\ls  z_Unjellier.__init__cCs ||}|jD]}|q|Sr")rUr)r[romrrr unjellyFullss  z_Unjellier.unjellyFullcCst|dr |j|j|S)z If the given object has support for the C{postUnjelly} hook, set it up to be called at the end of deserialization. @param unjellied: an object that has already been unjellied. @return: C{unjellied} postUnjelly)rJrrr)r[ unjelliedrrr _maybePostUnjellyys z_Unjellier._maybePostUnjellyc Cs&t|tur|S|d}|j|st|t|}|dur/tt|d|}| |||St |}|durD| || |dSt |}t|d|d}|dur\||ddS| d}d|dd} |j| s{td| d|d t|} |j| std ||| |dS) NrrmrTz _unjelly_%srzModule z not allowed (in type z).zClass %s not allowed.)rrrorrr5getr+r!rr;rUr splitrisModuleAllowedrr_genericUnjelly) r[r jelTypeBytesregClassmethod regFactory jelTypeTextthunk nameSplitmodNameclzrrr rUs4        z_Unjellier.unjellycCs|t|||S)a Unjelly a type for which no specific unjellier is registered, but which is nonetheless allowed. @param cls: the class of the instance we are unjellying. @type cls: L{type} @param state: The jellied representation of the object's state; its C{__dict__} unless it has a C{__setstate__} that takes something else. @type state: L{list} @return: the new, unjellied instance. )rr-rU)r[rr%rrr rsz_Unjellier._genericUnjellycCsdSr"rr[exprrr _unjelly_Nonesz_Unjellier._unjelly_NonecCst|ddS)Nrrrrrrr _unjelly_unicoderaz_Unjellier._unjelly_unicodecCsD|d}|d}|dkrd}nd}t|d}t|||fS)z* Unjelly decimal objects. rrT)rrr)r[rrrrrrrr _unjelly_decimalsz_Unjellier._unjelly_decimalcCs|ddvsJ|ddkS)Nr)rrrrrrrr _unjelly_booleans z_Unjellier._unjelly_booleancCstjtt|dSNr)rmaprrrrrr _unjelly_datetimez_Unjellier._unjelly_datetimecCtjtt|dSr)rrrrrrrrr _unjelly_daterz_Unjellier._unjelly_datecCrr)rrrrrrrrr _unjelly_timerz_Unjellier._unjelly_timecCs(tt|d\}}}tj|||dS)Nr)rrr)rrrrr)r[rrrrrrr _unjelly_timedeltasz_Unjellier._unjelly_timedeltacCs,||}t|tr||||||<|Sr")rUrr addDependant)r[rlocjelrrrr unjellyIntos   z_Unjellier.unjellyIntocCs6|d}|j|}|dur|St|}||j|<|Sr)rrr)r[lstr}rderrrr _unjelly_dereferences  z_Unjellier._unjelly_dereferencecCsf|d}|d}||}|j|}|dur||j|<|St|tr/||||j|<|SJd)NrrTz!Multiple references with same ID!)rUrrrrresolveDependants)r[rr}rrrefrrr _unjelly_references     z_Unjellier._unjelly_referencecCsNttt|}d}|D]}t|||||trd}q |r#t|St|S)NrTr)rrangelenrrrrr )r[rlfinishedelemrrr _unjelly_tuplesz_Unjellier._unjelly_tuplecCs0ttt|}|D] }|||||q |Sr")rr rr)r[rrrrrr _unjelly_listsz_Unjellier._unjelly_listcCsTttt|}d}|D]}|||||}t|trd}q |s&t||S||S)z Helper method to unjelly set or frozenset. @param lst: the content of the set. @type lst: C{list} @param containerType: the type of C{set} to use. TF)rr rrrrr)r[r containerTyperrrdatarrr _unjellySetOrFrozensets   z!_Unjellier._unjellySetOrFrozensetcC ||tS)z7 Unjelly set using the C{set} builtin. )rrr[rrrr _unjelly_set z_Unjellier._unjelly_setcCr)zC Unjelly frozenset using the C{frozenset} builtin. )rrrrrr _unjelly_frozenset"rz_Unjellier._unjelly_frozensetcCs:i}|D]\}}t|}||d|||d|q|S)NrrT)rr)r[rrkvkvdrrr _unjelly_dictionary(s  z_Unjellier._unjelly_dictionarycCsLt|d}t|tkrtd|j|std|t|iid}|S)Nrz5Attempted to unjelly a module with a non-string name.z"Attempted to unjelly module named r)r rrerror __import__)r[rest moduleNamemodrrr _unjelly_module0s   z_Unjellier._unjelly_modulecCst|d}|td}td|dd}|j|s$td|t|}t|}|tur8td||f|j|sFtdt ||S)Nrrrzmodule %s not allowedz6class %r unjellied to something that isn't a class: %rzclass not allowed: %s) r rrrorrrrrr)r[r!cnameclistrklausrrrr _unjelly_class9s    z_Unjellier._unjelly_classcCsTt|d}|td}td|dd}|j|s$td|t|}|S)NrrrzModule not allowed: %s)r rrrorrr )r[r!fnamemodSplitrfunctionrrr _unjelly_functionJs   z_Unjellier._unjelly_functioncCs"|jr ||d|}|StdS)NrzPersistent callback not found)rrY)r[r!ploadrrr _unjelly_persistentTsz_Unjellier._unjelly_persistentcCs0tjdtddd||d}|||dS)z (internal) Unjelly an instance. Called to handle the deprecated I{instance} token. @param rest: The s-expression representing the instance. @return: The unjellied instance. ztUnjelly support for the instance atom is deprecated since Twisted 15.0.0. Upgrade peer for modern instance support.r)categoryfilenamelinenorT)warnings warn_explicitDeprecationWarningrUr)r[r!rrrr _unjelly_instance[s z_Unjellier._unjelly_instancecCstd|dS)NzUnpersistable data: r)rY)r[r!rrr _unjelly_unpersistablepsz!_Unjellier._unjelly_unpersistablecCs|d}||d}||d}t|tstd||jvrJ|dur+t||}|St|tr8t|||}|Stj |j||g|gdR}|St d)z. (internal) Unjelly a method. rrTrz"Method found with non-class class.NFzinstance method changed) rUrrrr$r+rr rrrA)r[r!im_nameim_selfim_classimrrr _unjelly_methodss&     z_Unjellier._unjelly_methodN) r>rbrcr\rrrUrrrrrrrrrrr r rrrrrrr$r(r,r.r6r7r<rrrr rks<       rc@seZdZdZdS)rz This exception will be raised when a jelly is deemed `insecure'; e.g. it contains a type, class, or module disallowed by the specified `taster' N)r>rbrcrdrrrr rsrc@s(eZdZdZddZddZddZdS) DummySecurityOptionsz{ DummySecurityOptions() -> insecure security options Dummy security options -- this class will allow anything. cCdS)z DummySecurityOptions.isModuleAllowed(moduleName) -> boolean returns 1 if a module by that name is allowed, 0 otherwise rTrr[r"rrr rz$DummySecurityOptions.isModuleAllowedcCr>)z DummySecurityOptions.isClassAllowed(class) -> boolean Assumes the module has already been allowed. Returns 1 if the given class is allowed, 0 otherwise. rTrr[klassrrr rsz#DummySecurityOptions.isClassAllowedcCr>)z DummySecurityOptions.isTypeAllowed(typeName) -> boolean Returns 1 if the given type is allowed, 0 otherwise. rTrr[typeNamerrr rr@z"DummySecurityOptions.isTypeAllowedN)r>rbrcrdrrrrrrr r=s  r=c@sXeZdZdZgdZddZddZddZd d Zd d Z d dZ ddZ ddZ dS)SecurityOptionszF This will by default disallow everything, except for 'none'. ) dictionaryrr reference dereferencerrlong_intlongdictcCsviddddddddddddddd dd dd dd dd ddddddddd|_i|_i|_dS)z/ SecurityOptions() initialize. rrTsboolrsstringsstrsintsfloatrrrrsNoneTyperrrrN) allowedTypesallowedModulesallowedClassesr_rrr r\sF      zSecurityOptions.__init__cCs|j|jdS)zz Allow all `basic' types. (Dictionary and list. Int, string, and float are implicitly allowed.) N)r7 basicTypesr_rrr allowBasicTypesszSecurityOptions.allowBasicTypescGs>|D]}t|tr|d}t|tst|}d|j|<qdS)zg SecurityOptions.allowTypes(typeString): Allow a particular type, by its name. r.rTN)rrer0r/rrL)r[rtyprrr r7s    zSecurityOptions.allowTypescGsJ||dddd|D]}|t|||jd|j|<qdS)a SecurityOptions.allowInstances(klass, klass, ...): allow instances of the specified classes This will also allow the 'instance', 'class' (renamed 'classobj' in Python 2.3), and 'module' types, as well as basic types. r'classclassobjrBrTN)rPr7r allowModulesrbrN)r[classesrBrrr allowInstancesOfs  z SecurityOptions.allowInstancesOfcGs@|D]}t|tjkr|j}t|ts|d}d|j|<qdS)z SecurityOptions.allowModules(module, module, ...): allow modules by name. This will also allow the 'module' type. r.rTN)rrrr>rr/r0rM)r[modulesrBrrr rTs   zSecurityOptions.allowModulescCst|ts |d}||jvS)z SecurityOptions.isModuleAllowed(moduleName) -> boolean returns 1 if a module by that name is allowed, 0 otherwise r.)rr/r0rMr?rrr rs   zSecurityOptions.isModuleAllowedcCs ||jvS)z SecurityOptions.isClassAllowed(class) -> boolean Assumes the module has already been allowed. Returns 1 if the given class is allowed, 0 otherwise. )rNrArrr rs zSecurityOptions.isClassAllowedcCs&t|ts |d}||jvpd|vS)z SecurityOptions.isTypeAllowed(typeName) -> boolean Returns 1 if the given type is allowed, 0 otherwise. r..)rr/r0rLrCrrr r s  zSecurityOptions.isTypeAllowedN) r>rbrcrdrOr\rPr7rVrTrrrrrrr rEs   rEcCt||||S)z Serialize to s-expression. Returns a list which is the serialized representation of an object. An optional 'taster' argument takes a SecurityOptions and will mark any insecure objects as unpersistable rather than serializing them. )rnrN)rzrortrurrr rN/srNcCrY)aT Unserialize from s-expression. Takes a list that was the result from a call to jelly() and unserializes an arbitrary object from it. The optional 'taster' argument, an instance of SecurityOptions, will cause an InsecureJelly exception to be raised if a disallowed type, module, or class attempted to unserialize. )rr)rrorrurrr rU:s rUr")Erdrwrrrr3 functoolsrzope.interfacer incrementalrtwisted.persisted.crefutilrrrrr r twisted.python.compatr twisted.python.deprecater twisted.python.reflectr rrtwisted.spread.interfacesrrrKr None_atom class_atom module_atom function_atomryrrxrrrrrrrr5r;r!r-r3r:r=rHrSrXrYrgrkrnr Exceptionrr=rEr6rPrNrUrrrr s~:        )  $z