Pyanno: Python Annotations
|
This project came out of my work at Temboo,
a startup in the Tribeca neighborhood of New York City.
They're hiring software engineers who want to work in Python, Qt, Java.
Contact us for details.
|
from pyanno import raises, abstractMethod, returnType, parameterTypes, deprecatedMethod, \
privateMethod, protectedMethod, selfType, ignoreType
@abstractMethod
def func2():
pass
@returnType(int, int)
@parameterTypes(int, int)
def divisionAndModulus(numerator, denominator):
# return quotient and remainder of numerator and denominator
quotient, remainder = divmod(numerator, denominator)
return quotient, remainder
@raises(IOError, ValueError)
def func1(f, s):
f.write( int(s) )
@deprecatedMethod
def func2():
...
@privateMethod
def func3():
...
@protectedMethod
def func4():
...
Contents
Version 0.77 (August 28th, 2007)
Source distribution includes python source, license, and this document.
Pyanno is a Python module that provides annotations for
Python code.
The Pyanno annotations have two functions:
-
Provide a structured way to document Python code
-
Perform limited run-time checking
Conventional text comments
def getDistance(p1, p2):
"""
getDistance() expects to be given two Points;
it returns a floating point value representing the distance between those two points.
"""
...
Pyanno annotations
from geometry import Point
@returnType(float)
@parameterTypes(Point, Point)
def getDistance(p1, p2):
"""
getDistance() calculates the distance between two points.
"""
...
-
Type Checking
Python is a dynamically-typed language; it doesn't use explicit datatype declarations.
The @returnType and @parameterTypes annotations raise an Exception if a function or method is passed or returns
an invalid value.
You specify the types expected and returned by a function, using actual Python types.
-
Abstract Methods
Methods with the @abstractMethod annotations raise an Exception if called, with a meaningful message.
-
Deprecated Methods
Methods with the @deprecatedMethod annotations print a warning message if called.
-
Comparison With Text Comments
-
Pyanno annotations are able to perform limited run-time checking (though no compile-time checking).
-
Comments can be wrong.
They may have typos, or reflect a misunderstanding.
A function and its documentation easily fall out of sync as they are revised.
The annotations, by enforcing the documented behavior help catch programmer error and catch mistakes soon after they are made.
Also, some mistakes (such as typos) are caught at compile-time, since the annotations are code.
The following examples are all valid.
For brevity, I've skipped the actual function definition and show only the annotations.
from pyanno import raises, abstractMethod, returnType, parameterTypes, deprecatedMethod, \
privateMethod, protectedMethod, selfType, ignoreType, callableType
@returnType # returns nothing
@returnType ( ) # returns nothing
@returnType ( ' ' ) # returns nothing
@returnType ( str ) # returns a string (or None)
@returnType ( ' str ' ) # returns a string (example of string-escaping, equivalent to previous example)
@returnType ( str, int )
@returnType ( 'str', int )
@returnType ( ' str, int ' ) # string-escape multiple types in single string, equivalent to two previous examples
@returnType ( tuple ) # returns a tuple
@returnType ( () ) # equivalent to previous example
@returnType ( dict ) # returns a dict
@returnType ( {} ) # equivalent to previous example
@returnType ( list ) # returns a list
@returnType ( [] ) # equivalent to previous example
@returnType ( [int] ) # returns a list containing only ints
@returnType ( (int, ) ) # a tuple containing only ints (Note trailing comma)
@returnType ( {int:str} ) # a map with int keys and string values
@returnType ( {int: [ str ] } ) # a map with int keys and string-list values
@returnType ( QObject ) # class
@returnType ( 'QObject' ) # string-escaped, equivalent to previous example
@returnType ( ClassName('QObject') ) # class, reference valid even if QObject has not been imported.
@returnType ( ignoreType ) # One argument is required, but no type checking is done.
@returnType ( classType )
@returnType ( instanceType )
@returnType ( callableType ) # accepts any method, function, closure, lambda, class or callable instance.
@parameterTypes( selfType )
@parameterTypes( int ) # accepts an int argument
@parameterTypes( int, [str] ) # accepts an int and a list of strings
-
@abstractMethod raises an exception (AbstractMethodError) if the function is called.
Has no arguments.
-
@deprecatedMethod prints a warning if the function is called.
Has no arguments.
-
@privateMethod raises an exception (PrivateMethodError) if the function is called by another module.
Has no arguments.
-
@protectedMethod raises an exception (PrivateMethodError) if the function is called by a module in a different package.
Has no arguments.
-
@raises does no run-time checking; purely for documenting the exceptions that may be raised by this function.
Arguments: a list of Exceptions (each argument must be a subclass of BaseException).
Arguments to @raises may not be string-escaped or include ClassName.
-
@parameterTypes performs run-time checking on the arguments passed to the function.
Arguments: a list of types.
-
@returnType performs run-time checking on the arguments returned by the function.
Arguments: a list of types.
-
@parameterTypes, @returnType and @raises take a list of types. These are standard python types (ie. int, str, list, dict, etc. see the 'types' standard module) or classes (ie. QObject).
-
You can only refer to defined types and classes.
See below (String Escaping Types and Circular References) for workarounds.
-
Order of operations matters with python decorators. Python function decorators are applied in reverse order.
Pyanno annotations should be applied FIRST (before builtin annotations like @staticmethod), so they should appear LAST in the code.
For example,
@staticmethod
@returnType(bool)
def myMethod ():
return False
-
Runtime type checking is controlled by pyanno.DO_RUNTIME_VALIDATION.
Ultimately, this could be set by an environment variable.
-
Runtime type checking raise AbstractMethodError, ReturnTypeError, ParamTypeError, etc., all defined in
pyanno.py.
-
At this point, none of the annotations provide compile-time checking.
-
You can annotate constructors like any other method.
-
Python uses implicit tuples when returning multiple values. For example, 'return 1,2' is equivalent to 'return (1,2)'.
This doesn't effect our annotations.
-
Do not apply the same annotation more than once to the same function. The results are undefined.
-
None values are considered valid for all types.
-
Pyanno handles sip
(and hence pyqt) wrapper types seamlessly, though they aren't classes.
-
When annotating a method or function with keynamed/optional parameters, annotate all parameters as usual.
@parameterTypes( list, dict, str, int )
def function1( requiredParameter1, requiredParameter2, optionalParameter1='third', optionalParameter2=4 )
...
- Classes cannot refer to their own type within the class definition. 'self' references (and other references to the class being defined) should use string-escaped types or selfType. No type checking is done on selfType.
class A:
@parameterTypes( A ) # wrong. Definition of A cannot refer to A.
def method1(self):
...
@parameterTypes( selfType ) # valid. No type checking is done on selfType keyword.
def method2(self):
...
@parameterTypes( 'A' ) # valid. String-escaped type will be valid due to lazy evaluation.
def method3(self):
...
- class references should use classType.
-
callableType accepts any callable (ie. a method, function, closure, lambda, class or instance with a __call__() method).
- If you don't want to type check an argument, use ignoreType.
@parameterTypes( int, ignoreType, str ) # no type-checking performed on second argument.
def function1( arg1, arg2, arg3 ):
...
- If you don't want to check the class of an argument, use types.InstanceType.
You can check the types of tuple and list values, and dict key and value types.
@returnType ( (int, ) ) # a tuple containing only ints (Note trailing comma)
@returnType ( [int] ) # a list containing only ints
@returnType ( {int:str} ) # a map with int keys and string values
@returnType ( [QObject] ) # a list of QObjects.
# Collections can be arbitrarily nested:
@returnType ( {str:[{str:int}]} ) # a map whose keys are string and whose values are
# lists of dicts mapping strings to ints.
@returnType ( [ [int] ] ) # a list of lists which contain only ints
[int] means 'a list of ints', not 'a list containing one int.'
You cannot specify multiple valid types for collection contents, ie. '[int, str]' is not valid.
Warning: Make sure to use a trailing comma when parameterizing tuples with a single element.
(x,) is a tuple with a single element, (x) is just an element wrapped in parenthesis.
@returnType ( (int ) ) # wrong, missing trailing comma (means an int, not a tuple containing ints).
@returnType ( (int, ) ) # correct.
There are a number of situations where you cannot pass ordinary python types to annotations.
For example:
Class A:
@parameterTypes( selfType, A )
def compare(self, other):
return self.id == other.id
This is an ordinary comparison method.
Unfortunately, this annotation refers to class A within the definition of class A.
As such, it is undefined.
@parameterTypes( selfType, 'A' )
The solution is to quote the class in a string.
The annotations use lazy evaluation of their arguments.
You can string-escape multiple types in a single string. The following are equivalent:
@returnType ( str, int )
@returnType ( 'str', int )
@returnType ( ' str, int ' )
String-escaped and raw types can be mixed and matched arbitrarily:
@returnType ( 'str, Class1', str, 'int', str, 'Class2, Class1')
Python does not permit circular imports.
ie. if module A imports module B, B cannot also import A.
A class cannot be referenced in an annotation if it is not imported - even if string-escaped.
A solution is to use ClassName.
@parameterTypes( selfType, ClassName('TestClass1' ) )
def doSomething(self, other):
pass
- ClassName compares the designated class name (ie. 'TestClass1') with the name of the arguments class and superclasses.
- ClassName ignores module path (ie. 'utils.tests.Something.TestClass1').
- ClassName will not distinguish between classes with identical names.
- ClassName itself can be string-escaped (see above).
- Generally speaking, "String-Escaping" types is more convenient and concise than using ClassName.
Only use ClassName when you must (ie. when you cannot import the external class).
Other options are to use ignoreType, or not use an annotation in this case.
The arguments for @parameterTypes, @returnType, @privateMethod and @raises are optional.
The @abstractMethod and @deprecatedMethod annotations has no arguments.
These three forms:
@returnType
@returnType ( )
@returnType ( ' ' )
all mean the same thing: this method returns nothing (a value of type NoneType).
-
These annotations don't yet support "Positional Parameters" (ie. *args) or "Keyword Parameters" (ie. **argkw).
-
I'm not sure of the best way to annotate classes. One approach would be to annotate a stub new (class initialization) method.
-
No support yet for duck typing. This could easily be added by passing a validation function instead of a type or typename.
-
It would be nice to hide the annotations in stack traces.
-
It would also be nice to hide the implementation details of the annotations, so that all exceptions raised by the annotations appeared to have only a single reference in the stack to the annotations.
- We could easily modify the @privateMethod annotation to raise an exception on non-local access to a method. In fact, it could take a list of classes (friends?) that would be permitted access.
- @tbGetter and @tbSetter / @tbAccessor.
-
We could alter the type checking syntax to indicate that parameters cannot be 'None'. This would be more in the realm of assertions than type checking.
-
see Python Decorator Library at python.org.
These annotations have a common theme of bringing keywords and annotations
familiar to Java and C++ users into the Python world.
Given that these languages reflect a very different design philosophy from Python, this may seem questionable.
That is, are Pyanno annotations
"Pythonic"?
On the one hand, Pyanno annotations would have no value if Python was statically-typed, like Java or C++.
But Pyanno annotations do not make Python a statically-typed language,
(see
this).
After all, Pyanno type-checking occurs at run-time, not compile time.
It's not a question of statically-type or dynamic-typing, but of access to type-checking at our own discretion.
One can apply the annotations as narrowly as you like.
For example, you may only want to annotate the public interface for a library or module.
In this way, Pyanno annotations help offset a potential weakness of the language.
In many ways, their real value lies in providing more accurate documentation rather than helping to guarantee correctness.
-
Question: Why isn't Pyanno version 1.0?
Answer: Pyanno is a fairly new project.
Version 0.77 released August 28th, 2007.
Version 0.76 released August 11th, 2007.
-
Added the @deprecated, @privateMethod and @protectedMethod annotations.
-
Added type-checking of the arguments to the @raises annotation.
-
Improved the documentation.
-
Elaborated the unit test suite, which is still incomplete.
The unit tests are not part of the source distribution.
Version 0.75 released August 8th, 2007.
To do list & known bugs:
- Add a @deprecated annotation.
-
Pyanno requires Python 2.4 or higher.
-
Pyanno has no dependencies. It uses sip if available.
Pyanno annotations are implemented with Python's function decorators feature (new in Python 2.4):
In the interest of improving my goolge karma, allow me to mention:
python annotations,
annotating python,
python function decorators,
type safe python,
strongly typed python,
statically typed python,
python documentation,
python type safety,
python type checking,
python function decorators,
metaprogramming,
python introspection,
