Appendix¶
Reference Counting¶
Reference Count¶
The number of references to a given object. This is the count of objects which refer to the given object.
When an object is created, its reference count starts at one, which is the
reference owned by the creator. The reference count can be increased with
Py_INCREF()
or decreased with Py_DECREF()
. As soon as the
reference count reaches zero, the object is no longer needed and it will be
deallocated.
New Reference¶
A new reference is a PyObject*
which must be managed by the user. The
user must ensure that Py_DECREF()
is called when the object is no longer
needed.
Borrowed Reference¶
A borrowed reference is a PyObject*
which is not owned. Users should
not call Py_DECREF()
when they are done with this object. A borrowed
reference can become a new reference by calling
Py_INCREF()
on the it.
Reference Stealing¶
Some functions will “steal” a reference to an argument. This means the function
will assume ownership of the reference that has been given to it. Callers should
not call Py_DECREF()
the object after the reference has been
stolen. If you need to keep your own reference to the object after it has been
stolen you need to call Py_INCREF()
before the reference is stolen.
An example of this pattern is PyList_SetItem()
. This function steals a
reference to the value
argument. This is to make it easier to allocate new
objects and put them right into a PyListObject
without needing extra
calls to Py_INCREF()
.
Py_INCREF
¶
-
void
Py_INCREF
(PyObject* ob)¶ The
Py_INCREF
macro increments the reference count of an object by one.Parameters: - ob (PyObject*) – The object to increment the reference count of.
Py_XINCREF
¶
-
void
Py_XINCREF
(PyObject* ob)¶ The
Py_INCREF
macro increments the reference count of an object ifob
is notNULL
.Parameters: - ob (PyObject*) – The object to increment the reference count of.
Py_DECREF
¶
-
void
Py_DECREF
(PyObject* ob)¶ The
Py_DECREF
macro decrements the reference count of an object by one.Parameters: - ob (PyObject*) – The object to decrement the reference count of.
Py_XDECREF
¶
-
void
Py_XDECREF
(PyObject* ob)¶ The
Py_XDECREF
macro decrements the reference count of an object ifob
is notNULL
.Parameters: - ob (PyObject*) – The object to decrement the reference count of.
CPython Types¶
Py_ssize_t
¶
-
Py_ssize_t
¶ A
Py_ssize_
is a signed integral type used to hold counts and object sizes. On many platforms this is an alias ofssize_t
.
PyObject
¶
-
PyObject
¶ https://docs.python.org/3.6/c-api/structures.html#c.PyObject
PyObject
is the structure which holds Python values. The definition looks like:-
Py_ssize_t
ob_refcnt
¶ The object’s reference count.
-
PyTypeObject*
ob_type
¶ The object’s type as a Python object.
-
Py_ssize_t
Note
Users should not access these fields directly, instead go through the API
macros: Py_REFCNT()
and Py_TYPE()
.
PyTypeObject
¶
-
PyTypeObject
¶ https://docs.python.org/3/c-api/type.html
PyTypeObject
is the structure which holds Pythontype
objects.
Casting Rules¶
A PyTypeObject*
can safely be cast to a PyObject*
.
PyObject*
s can be cast to PyTypeObject*
s only after a
PyType_Check
.
Fields¶
-
const char*
PyTypeObject.tp_name
¶ The fully qualified name to the class as a C string. This string must stay alive forever.
This field is not inherited.
-
Py_ssize_t
PyTypeObject.tp_basicsize
¶ The base size of instances of this class. You should use
sizeof
to derive this from the instance struct type.This field is inherited
-
Py_ssize_t
PyTypeObject.tp_itemsize
¶ Some classes require variable storage to hold instances. For example,
PyTupleObject
s stores all of the members inline. ThePyTypeObject.tp_basicsize
of a tuple would be the base object and element count, thePyTypeObject.tp_itemsize
would besizeof(PyObject*)
. If instances are always the same size, this field should be 0.
-
destructor
PyTypeObject.tp_dealloc
¶ A pointer to the object destructor. This function is called when the reference count hits 0. This function should release any references that it owns and then defer to
PyTypeObject.tp_free
to deallocate the memory.This field is inherited.
-
printfunc
PyTypeObject.tp_print
¶ Reserved slot from Python 2. This is now unused.
-
getattrfunc
PyTypeObject.tp_getattr
¶ Deprecated in favor of
PyTypeObject.tp_getattro
-
setattrfunc
PyTypeObject.tp_setattr
¶ Deprecated in favor of
PyTypeObject.tp_setattro
-
PyAsyncMethods*
PyTypeObject.tp_as_async
¶ Pointer to additional functions used when an object supports the Async Protocol. If the type does not support the Async Protocol, this can be
NULL
.The value of the pointer is not inherited, but the members are inherited individually.
-
reprfunc
PyTypeObject.tp_repr
¶ C API equivalent of
__repr__
This can be invoked with
PyObject_Repr()
.This field is inherited.
-
PyNumberMethods*
PyTypeObject.tp_as_number
¶ Pointer to additional functions used when an object supports the number Protocol. If the type does not support the Number Protocol, this can be
NULL
.The value of the pointer is not inherited, but the members are inherited individually.
-
PyNumberMethods
¶ A collection of function pointers used to support the Number API.
typedef struct { binaryfunc nb_add; binaryfunc nb_subtract; binaryfunc nb_multiply; binaryfunc nb_remainder; binaryfunc nb_divmod; ternaryfunc nb_power; unaryfunc nb_negative; unaryfunc nb_positive; unaryfunc nb_absolute; inquiry nb_bool; unaryfunc nb_invert; binaryfunc nb_lshift; binaryfunc nb_rshift; binaryfunc nb_and; binaryfunc nb_xor; binaryfunc nb_or; unaryfunc nb_int; void *nb_reserved; unaryfunc nb_float; binaryfunc nb_inplace_add; binaryfunc nb_inplace_subtract; binaryfunc nb_inplace_multiply; binaryfunc nb_inplace_remainder; ternaryfunc nb_inplace_power; binaryfunc nb_inplace_lshift; binaryfunc nb_inplace_rshift; binaryfunc nb_inplace_and; binaryfunc nb_inplace_xor; binaryfunc nb_inplace_or; binaryfunc nb_floor_divide; binaryfunc nb_true_divide; binaryfunc nb_inplace_floor_divide; binaryfunc nb_inplace_true_divide; unaryfunc nb_index; binaryfunc nb_matrix_multiply; binaryfunc nb_inplace_matrix_multiply; } PyNumberMethods;
-
-
PySequenceMethods*
PyTypeObject.tp_as_sequence
¶ Pointer to additional functions used when an object supports the Sequence Protocol. If the type does not support the Sequence Protocol, this can be
NULL
.The value of the pointer is not inherited, but the members are inherited individually.
-
PySequenceMethods
¶ A collection of function pointers used to support the Sequence API.
-
lenfunc
sq_length
¶ The function used to support
PyObject_Size()
. This function will automatically be converted into a Python__len__
.
-
binaryfunc
sq_concat
¶ The function used to support
PySequence_Concat()
. If there is noPyNumberMethods.nb_add
function, this will be turned into a__add__
function (likelist + list
in Python).
-
ssizeargfunc
sq_repeat
¶ The function used to support
PySequence_Repeat()
. If there is noPyNumberMethods.nb_multiply
this will be turned into a__mul__
function (likelist * int
in Python).
-
ssizeargfunc
sq_item
¶ The function used to support
PySequenceMethods.sq_item()
. This function will be converted into a__getitem__
method in Python.
-
ssizeobjargproc
sq_ass_item
¶ The function used to support
PySequence_SetItem()
. This slot can beNULL
if the object doesn’t support assignment (liketuple
). This will be converted into a__setitem__
method in Python.
-
objobjproc
sq_contains
¶ The function used to support
PySequence_Contains()
. If this is leftNULL
, a linear search will be performed. This function will be converted into a__contains__
method in Python.
-
binaryfunc
sq_inplace_concat
¶ The function used to support
PySequence_InPlaceConcat()
. It should modifyself
in place and then returnself
. This function is likelist += list
in Python.
-
binaryfunc
sq_inplace_repeat
¶ The function used to support
PySequence_InPlaceRepeat()
. It should modifyself
in place and then returnself
. This function is likelist *= int
in Python.
-
lenfunc
-
-
PyMappingMethods*
PyTypeObject.tp_as_mapping
¶ Pointer to additional functions used when an object supports the Mapping Protocol. If the type does not support the Mapping Protocol, this can be
NULL
.The value of the pointer is not inherited, but the members are inherited individually.
-
hashfunc
PyTypeObject.tp_hash
¶ C API equivalent of
__hash__
.This can be set to
PyObject_HashNotImplemented()
which is the equivalent of adding__hash__ = None
in Python.This can be invoked with
PyObject_Hash()
.This field is inherited along with
PyTypeObject.tp_richcompare
only when the subtype does not override either function.
-
ternaryfunc
PyTypeObject.tp_call
¶ C API equivalent of
__call__
.This field is inherited.
-
reprfunc
PyTypeObject.tp_str
¶ The C API equivalent of
_str__
. If this not set,PyTypeObject.tp_repr
will be used.This can be invoked with
PyObject_Str()
.This field is inherited.
-
getattrofunc
PyTypeObject.tp_getattro
¶ The C API equivalent of
__getattr__
. This can be set toPyObject_GenericGetAttr()
which implements standard attribute lookup.This can be invoked with
PyObject_GetAttr()
.This field is inherited.
Note
To use
PyObject_GenericGetAttr()
thePyTypeObject.tp_dictoffset
must be nonzero.
-
setattrofunc
PyTypeObject.tp_setattrofunc
¶ The C API equivalent of
__setattr__
. This can be set toPyObject_GenericSetAttr()
which implements standard attribute assignment.This can be invoked with
PyObject_SetAttr()
.This field is inherited.
Note
To use
PyObject_GenericSetAttr()
thePyTypeObject.tp_dictoffset
must be nonzero.
-
PyBufferProcs*
PyTypeObject.tp_as_buffer
¶ Pointer to additional functions used when an object supports the Buffer Protocol. If the type does not support the Buffer Protocol, this can be
NULL
.The value of the pointer is not inherited, but the members are inherited individually.
-
unsigned long
PyTypeObject.tp_flags
¶ A bitmask of information about the type.
-
Py_TPFLAGS_HEAPTYPE
¶ This bit is set when the type object is allocated on the heap. This is true for classes defined in Python.
-
Py_TPFLAGS_BASETYPE
¶ This bit is set when the type can be subclassed. Sometimes we don’t want to treat out members as virtual so we ignore the possibility of subclasses changing methods.
Attempting to subclass for a type without this bit set will generate an error like:
TypeError: type '<name>' is not an acceptable base type
-
Py_TPFLAGS_READY
¶ Marks that
PyType_Ready()
has been called. You should not explicitly set this flag.
-
Py_TPFLAGS_READYING
¶ Marks that
PyType_Ready()
is in the process of readying the type. You should not explicitly set this flag.
-
Py_TPFLAGS_HAVE_GC
¶ Marks that this type supports the cyclic garbage collector. If this is set, instances need to be allocated with
PyObject_GC_New()
and freed withPyObject_GC_Del()
. If this bit is set thePyTypeObject.tp_traverse
function must be set.
-
Py_TPFLAGS_DEFAULT
¶ The default set of bits that should be set for all new classes.
-
Py_TPFLAGS_LONG_SUBCLASS
¶
-
Py_TPFLAGS_LIST_SUBCLASS
¶
-
Py_TPFLAGS_TUPLE_SUBCLASS
¶
-
Py_TPFLAGS_BYTES_SUBCLASS
¶
-
Py_TPFLAGS_UNICODE_SUBCLASS
¶
-
Py_TPFLAGS_DICT_SUBCLASS
¶
-
Py_TPFLAGS_BASE_EXC_SUBCLASS
¶
-
Py_TPFLAGS_TYPE_SUBCLASS
¶ Marks that the type is a subclass of one of the builtin types. These dramatically speed up common
Py*_Check
calls which can then use a bit and instead of a genericPyObject_Isinstance()
call. You should not set these bits directly,PyType_Ready()
will do this if needed.
-
Py_TPFLAGS_HAVE_FINALIZE
¶ Marks that the
PyTypeObject.tp_finalize
should be called on object destruction.
-
-
const char*
PyTypeObject.tp_doc
¶ A pointer to the docstring as a C string. If
NULL
,__doc__
will beNone
.This field is not inherited.
-
traverseproc
PyTypeObject.tp_traverse
¶ A pointer to the cyclical garbage collector traversal function. This is only called if
Py_TPFLAGS_HAVE_GC
is set.This function needs to call
Py_VISIT()
on all of the members which may participate in a cycle.To use the
Py_VISIT()
macro thevisitproc
argument must be calledvisit
and thevoid*
argument must be calledarg
.Below we have a type with two
PyObject*
fields and one non-object field.typedef struct { PyObject mt_base; PyObject* mt_object_member; PyObject* mt_other_object_member; Py_ssize_t mt_not_an_object; } mytype; static int mytype_traverse(mytype* self, visitproc visit, void* arg) { Py_VISIT(self->mt_base); Py_VISIT(self->mt_object_member); /* note: not visiting self->mt_not_an_object because it is not a PyObject* */ return 0; }
Note
It is safe to not traverse immutable scalar members, for example:
PyLongObject*
members. It doesn’t hurt to visit allPyObject*
members so if you aren’t sure, just visit it!
-
inquiry
PyTypeObject.tp_clear
¶ Clear any references owned by this object while keeping the object in a valid state. Valid state means that the interpreter should not segfault because an object assumes members are non-null and then
tp_clear
sets them toNULL
. An example of this idea islist.clear()
. This drops all of the references owned by the list; however, at the end you still have a valid list. This function is only called ifPy_TPFLAGS_HAVE_GC
is set.When clearing references the
Py_CLEAR()
macro should be used instead of justPy_DECREF()
. This is because decref can trigger a deallocation which can invoke arbitrary Python code through an object’s__del__
method. This code can reference back to the object being cleared and we don’t want to return a pointer to the recently destroyed object.Below we have a type with two
PyObject*
fields and one non-object field.typedef struct { PyObject mt_base; PyObject* mt_object_member; PyObject* mt_other_object_member; Py_ssize_t mt_not_an_object; } mytype; static int mytype_clear(mytype* self) { Py_CLEAR(self->mt_base); Py_CLEAR(self->mt_object_member); /* note: not clearing self->mt_not_an_object because it is not a PyObject* */ return 0; }
Note
Immutable objects can skip defining a
PyTypeObject.tp_clear
field. It is impossible to form a reference cycle with all immutable objects which means the other object’sPyTypeObject.tp_clear
functions must be sufficient to clear the cycle.
-
richcmpfunc
PyTypeObject.tp_richcompare
¶ The function which supports
PyObject_Richcompare()
for this type.See
PyObject_Richcompare()
for more information about how this function works.This field is inherited along with
PyTypeObject.tp_hash
only when the subtype does not override either function.
-
Py_ssize_t
PyTypeObject.tp_weaklistoffset
¶ The offset into an object structure where the weaklist is stored. This is used for making objects weakly referenceable. If an object is not weakly referenceable, this field should be set to 0. This offset needs to point to a
NULL
initializedPyObject*
slot in the instance structure.In Python, if a class defines a
__weakref__
slot in__slots__
, that offset will be used as thePyTypeObject.tp_weaklistoffset
.Below we have a weakly referenceable type:
typedef struct { PyObject wr_base; PyObject* wr_weaklist; /* other data if we want */ } mytype; PyTypeObject mytype_type = { /* ... */ offsetof(mytype, wr_weaklist), /* tp_weaklistoffset */ /* ... */ };
Note that we use the
ofsetoff
operator to compute the offset accounting for the size of all members beforewr_weaklist
and any padding added by the compiler.This field is inherited.
Note
Do not confuse this field with
PyTypeObject.tp_weaklist
which is the weaklist storage for taking weak references of the type object itself.
-
getiterfunc
PyTypeObject.tp_iter
¶ The C API equivalent of
__iter__
.This can be invoked with
PyObject_Iter()
.This field is inherited.
Note
Objects may still be iterable without this function set if the support the Sequence Protocol. The
PySequenceMethods.sq_item()
function will be used from 0 until anPyExc_IndexError
is raised just like in Python with__getitem__
.
-
iternextfunc
PyTypeObject.tp_iternext
¶ The C API equivalent of
__next__
. The core difference is thatPyExc_StopIteration
does not need to be set when the iterator is exhausted. This will be done before returning to Python but the exception overhead can be avoided when being called from C.If this slot is set,
PyTypeObject.tp_iter
should be a function which returns a new reference toself
.This can be invoked with
PyIter_Next()
.
-
PyMethodDef*
PyTypeObject.tp_methods
¶ A
NULL
terminated array ofPyMethodDef
structures which will become the methods of the class. For eachPyMethodDef
in this list, a function object will be created and stored in thePyTypeObject.tp_dict
.The value of this field is not inherited but the methods will be.
-
PyMemberDef*
PyTypeObject.tp_members
¶ A
NULL
terminated array ofPyMemberDef
structures which will become the methods of the class. For eachPyMemberDef
in this list, a descriptor object will be created and stored in thePyTypeObject.tp_dict
.The value of this field is not inherited but the members will be.
-
PyMemberDef
¶ A structure which defines a descriptor for exposing a C member as a Python member.
-
char*
name
¶ The name of the member as a C string.
-
int
type
¶ The type code of the member.
-
T_SHORT
¶ A
short
member to be converted into a Pythonint
.
-
T_INT
¶ A
int
member to be converted into a Pythonint
.
-
T_LONG
¶ A
long
member to be converted into a Pythonint
.
-
T_FLOAT
¶ A
float
member to be converted into a Pythonfloat
.
-
T_DOUBLE
¶ A
double
member to be converted into a Pythonfloat
.
-
T_STRING
¶ A
char*
member to be converted into a Pythonstr
.Note
This implies
READONLY
.
-
T_OBJECT
¶ Get a
PyObject*
. If the pointer isNULL
,None
will be returned.T_OBJECT_EX
is often a better choice because of how it handlesdel
.
-
T_CHAR
¶ A
char
member to be converted into a Pythonstr
of length 1.
-
T_BYTE
¶ A
char
member to be converted into a Pythonint
.
-
T_UBYTE
¶ A
unsigned char
member to be converted into a Pythonint
.
-
T_UINT
¶ A
unsigned int
member to be converted into a Pythonint
.
-
T_USHORT
¶ A
unsigned short
member to be converted into a Pythonint
.
-
T_ULONG
¶ A
unsigned short
member to be converted into a Pythonint
.
-
T_BOOL
¶ A
char
member to be converted into a Pythonbool
.
-
T_LONGLONG
¶ A
long long
member to be converted into a Pythonint
.
-
T_ULONGLONG
¶ A
unsigned long long
member to be converted into a Pythonint
.
-
T_PYSSIZET
¶ A
Py_ssize_t
member to be converted into a Pythonint
.
-
-
Py_ssize_t
offset
¶ The offset into the C structure where this member appears. You should always compute this offset with the
offsetof
operator.
-
int
flags
¶ Flag bits for indicating read or write status. The options are
0
for read and write access orREADONLY
for read only access.T_STRING
forcesREADONLY
.Only
T_OBJECT
andT_OBJECT_EX
can be deleted withdel
which sets the pointer toNULL
.
-
char*
PyMemberDef.doc
¶ The docstring for the member. If set to
NULL
__doc__
will beNone
.
-
char*
-
-
PyGetSetDef*
PyTypeObject.tp_getset
¶ A
NULL
terminated array ofPyGetSetDef
structures which will become the methods of the class. For eachPyGetSetDef
in this list, a descriptor object will be created and stored in thePyTypeObject.tp_dict
..-
PyGetSetDef
¶ A
PyGetSetDef
defines a computed attribute like a Pythonproperty
-
char*
name
¶ The name of this attribute as a C string.
-
setter
set
¶ The function used to assign to this attribute. If the attribute is readonly, this may be
NULL
.
-
char*
doc
¶ The docstring as C string. If this is
NULL
,__doc__
will beNone
.
-
void*
closure
¶ Extra data to be passed to the getter and setter functions.
-
getter
¶ typedef PyObject* (*getter)(PyObject* self, void* closure);
-
setter
¶ typedef int (*setter)(PyObject* self, PyObject* value, void* closure);
-
char*
-
-
PyTypeObject*
PyTypeObject.tp_base
¶ The base class for this new type. If set to
NULL
,&PyBaseObject_Type
(which isobject
in Python) is used.This field is not inherited.
-
PyObject*
PyTypeObject.tp_dict
¶ The type’s dictionary (
__dict__
). This is initialized byPyType_Ready()
.This field is not inherited but the members of the dictionary are.
Note
Types sometimes use what is called a “split keys dictionary” instead of a normal Python dict. You should not use the
PyDict_*
functions to modify this object.
-
descrgetfunc
PyTypeObject.tp_descr_get
¶ The C API equivalent of
__get__
for the descriptor protocol.This field is inherited.
-
descrgetfunc
¶ typedef PyObject* (*descrgetfunc)(PyObject* self, PyObject* instance, PyObject* owner);
-
-
descrsetfunc
PyTypeObject.tp_descr_set
¶ The C API equivalent of
__set__
and__delete__
for the descriptor protocol.This field is inherited.
-
descrsetfunc
¶ typedef PyObject* (*descrsetfunc)(PyObject* self, PyObject* instance, PyObject* value);
If
value
isNULL
, this should delete the value.-
-
Py_ssize_t
PyTypeObject.tp_dictoffset
¶ The offset into an object structure where the
__dict__
is stored. This is used for allowing objects to have arbitrary attributes like a normal Python object.For some objects, we only want to support a specific set of fields stored in the instance struct so we can set
PyTypeObject.tp_dictoffset
to 0 which indicates that we do not have a__dict__
.The instance structure must have a
NULL
initializedPyObject*
member at the offset if it is nonzero.Below we have a type with a
__dict__
:typedef struct { PyObject wd_base; PyObject* wd_dict; /* other data if we want */ } mytype; PyTypeObject mytype_type = { /* ... */ offsetof(mytype, wd_dict), /* tp_dictoffset */ /* ... */ };
Note that we use the
ofsetoff
operator to compute the offset accounting for the size of all members beforewd_dict
and any padding added by the compiler.This field is inherited.
Warning
It is not common to define a new class in C that uses a nonzero
PyTypeObject.tp_dictoffset
. You are going to lose a lot of potential performance gains by dispatching through a dictionary for all lookups.
-
initproc
PyTypeObject.tp_init
¶ The C API equivalent of
__init__
. This is only used for mutable types.This field is inherited.
-
allocfunc
PyTypeObject.tp_alloc
¶ A function used to allocate memory for the new instance. This is different from
__new__
in that it does not initialize any member data. It exists only as a way to separate object allocation and initialization.The
Py_ssize_t
argument is the number of items in this instance. This is only meaningful for types with nonzeroPyTypeObject.tp_itemsize
.All
Py_TPFLAGS_HEAPTYPE
objects usePyType_GenericAlloc()
to force standard heap allocations. This is the default and recommended value for all types.This field is inherited by static (C defined) subtypes but not heap (Python defined) subtypes.
-
newfunc
PyTypeObject.tp_new
¶ A function used to allocate and initialize new instances of the type.
PyTypeObject.tp_new
should usePyTypeObject.tp_alloc
to allocate the raw memory for the new instance.If the type is immutable, the rest of the initialization should happen in the
PyTypeObject.tp_new
. If the type is mutable, the initialization should happen in thePyTypeObject.tp_init
which will be called automatically.This field is inherited.
-
destructor
PyTypeObject.tp_free
¶ A function used to deallocate the memory allocated with
PyTypeObject.tp_alloc
.By default this is a function that is compatible with the
PyType_GenericAlloc()
allocator.This field is inherited by static (C defined) subtypes but not heap (Python defined) subtypes.
-
inquiry
PyTypeObject.tp_is_gc
¶ This function is used when a type has a mix of statically and dynamically allocated instances. In this case, the
Py_TPFLAGS_HAVE_GC
flag is not enough to know if an instance can be collected. In this case, the function should be implemented to return True if the instance can be collected, otherwise False.This field is inherited.
Note
This is a very uncommon function to implement. It is mainly used to support static and dynamic class objects in CPython itself.
-
PyObject*
PyTypeObject.tp_bases
¶ A tuple of the base types. This field is set by
PyType_Ready()
and should not be manually modified.This field is not inherited.
-
PyObject*
PyTypeObject.tp_mro
¶ A tuple of the full method resolution order. This field is set by
PyType_Ready()
and should not be manually modified.This field is not inherited.
-
destructor
PyTypeObject.tp_finalize
¶ A function called before the garbage collector clears references or deallocates the object. This function is only called if
Py_TPFLAGS_HAVE_FINALIZE
is set.This can be useful for releasing C level resources like file descriptors.
See PEP 442 for more information about this field.
This field is inherited.
-
PyObject*
PyTypeObject.tp_weaklist
¶ The member to hold the weaklist for weak references to the type object itself.
Note
Do not confuse this field with
PyTypeObject.tp_weaklistoffset
which is the offset into the instance object where weak references to the instance will be stored.
Support Types¶
-
destructor
¶ typedef void (*destructor)(PyObject*);
-
getattrfunc
¶ typedef PyObject* (*getattrfunc)(PyObject* self, char* attr_name);
-
setattrfunc
¶ typedef PyObject* (*setattrfunc)(PyObject* self, char* attr_name, PyObject* value);
-
reprfunc::
¶
typedef PyObject* (*reprfunc)(PyObject*);
-
inquiry
¶ typedef int (*inquiry)(PyObject*);
-
newfunc
¶ typedef PyObject* (*newfunc)(PyObject* self, PyObject* args, PyObject* kwargs);
-
allocfunc
¶ typedef PyObject* (*allocfunc)(PyTypeObject* cls, Py_ssize_t nitems);
-
initproc
¶ typedef int (*initproc)(PyObject* self, PyObject* args, PyObject* kwargs);
-
unaryfunc
¶ typedef PyObject* (*unaryfunc)(PyObject*);
-
binaryfunc
¶ typedef PyObject* (*binaryfunc)(PyObject*, PyObject*);
-
ternaryfunc
¶ typedef PyObject* (*ternaryfunc)(PyObject*, PyObject*, PyObject*);
-
lenfunc
¶ typedef Py_ssize_t (*lenfunc)(PyObject*);
-
ssizeargfunc
¶ typedef PyObject* (*ssizeargfunc)(PyObject*, Py_ssize_t);
-
ssizeobjargproc
¶ typedef int (*ssizeobjargproc)(PyObject* Py_ssize_t, PyObject*);
-
objobjproc
¶ typedef int (*objobjproc)(PyObject*, PyObject*);
-
hashfunc
¶ typedef Py_hash_t (*hashfunc)(PyObject*);
-
getattrofunc
¶ typedef PyObject* (*getattrofunc)(PyObject*, PyObject*);
-
setattrofunc
¶ typedef PyObject* (*getattrofunc)(PyObject*, PyObject*, PyObject*);
-
richcmpfunc
¶ typedef PyObject* (*richcmpfunc)(PyObject* lhs, PyObject* rhs, int op);
PyLongObject
¶
-
PyLongObject
¶ https://docs.python.org/3/c-api/long.html
PyLongObject
is the structure which holds Pythonint
objects. This is called aPyLongObject
as a hold over from when arbitrary width integer was called along
object in Python 2.
Casting Rules¶
A PyLongObject*
can safely be cast to a PyObject*
.
PyObject*
s can be cast to PyLongObject*
s only after a
PyLong_Check()
.
API Functions¶
-
int
PyLong_Check
(PyObject* ob)¶ Check if the instance is a
PyLongObject
.Parameters: - ob (PyObject*) – The object to check.
Returns: True if
ob
is an instance ofPyLongObject
or an instance of a subclass ofPyLongObject
.
-
unsigned long
PyLong_AsUnsignedLong
(PyObject* ob)¶ Convert a
PyObject*
of typePyLongObject*
to anunsigned long
. Ifob
is not along
object, an exception is raised.Parameters: - ob (PyObject*) – The object to convert.
Returns: ob
as anunsigned long
.
-
PyObject*
PyLong_FromUnsignedLong
(unsigned long l)¶ Convert an
unsigned long
into aPyObject*
. If the object cannot be allocated an exception is raised.Parameters: - l – The unsigned long to convert to a pyobjectptr.
Returns: A new reference to
l
as a Python object.
PyListObject
¶
-
PyListObject
¶ https://docs.python.org/3/c-api/list.html
PyListObject
is the structure which holds Pythonlist
objects.
Casting Rules¶
A PyListObject*
can safely be cast to a PyObject*
.
PyObject*
s can be cast to PyListObject*
s only after a
PyList_Check()
.
API Functions¶
-
int
PyList_Check
(PyObject* ob)¶ Check if the instance is a
PyListObject
.Parameters: - ob (PyObject*) – The object to check.
Returns: True if
ob
is an instance ofPyListObject
or an instance of a subclass ofPyListObject
.
-
PyObject*
PyList_New
(Py_ssize_t len)¶ Create a new list object of length
len
. The members are set toNULL
and need to be filled withPyList_SET_ITEM()
before returning the list to Python.Parameters: - len (Py_ssize_t) – The length of the list to create.
Returns: A new reference to a
PyListObject
. The elements must be filled in.
-
Py_ssize_t
PyList_Size
(PyObject* list)¶ Return the length of the list with error checking.
Parameters: - list (PyObject*) – The list to get the size of.
Returns: The size of the list or -1 with an exception set if an error occurred. If
list
is not actually aPyListObject
an error will be raised.
-
Py_ssize_t
PyList_GET_SIZE
(PyObject* list)¶ Return the length of the list without error checking.
Parameters: - list (PyObject*) – The list to get the size of.
Returns: The size of the list. This function cannot be called on objects that are not known to be
PyListObject
s.
-
PyObject*
PyList_GetItem
(PyObject* list, Py_ssize_t ix)¶ Lookup an item in a list with error checking.
Parameters: - list (PyObject*) – The list to get an element in.
- ix (Py_ssize_t) – The index to lookup. Negative indices are not supported.
Returns: A borrowed reference to the element at index
ix
orNULL
with anPyExc_IndexError
set ifix
is out of bounds.
-
PyObject*
PyList_GET_ITEM
(PyObject* list, Py_ssize_t ix)¶ Lookup an item in a list without error checking.
Parameters: - list (PyObject*) – The list to get an element in.
- ix (Py_ssize_t) – The index to lookup. Negative indices are not supported.
Returns: A borrowed reference to the element at index
ix
. It is undefined behavior ifix
is out of bounds forlist
.
-
int
PyList_SetItem
(PyObject* list, Py_ssize_t ix, PyObject* value)¶ Assign an item in a list.
Parameters: - list (PyObject*) – The list to set the element in.
- ix (Py_ssize_t) – The index to assign to. Negative indices are not supported.
- value (PyObject*) – The value to assign. This reference is stolen. The old reference at
ix
will be released.
Returns: False on success or True with an exception set if an error occurred.
-
void
PyList_SET_ITEM
(PyObject* list, Py_ssize_t ix)¶ Lookup an item in a list without error checking.
Parameters: - list (PyObject*) – The list to set the element in.
- ix (Py_ssize_t) – The index to assign to. Negative indices are not
supported.
ix
must be in bounds forlist
. - value (PyObject*) – The value to assign. This reference is stolen. The old reference at
ix
will be not released and will be leaked if notNULL
.
-
int
PyList_Insert
(PyObject* list, Py_ssize_t ix, PyObject* item)¶ The C API equivalent of
list.insert()
. Insertsitem
intolist
beforeix
.Parameters: - list (PyObject*) – The list to insert into.
- ix (Py_ssize_t) – The index to insert before.
- item (PyObject*) – The item to insert.
Returns: False on success or True with an exception set if an error occurred.
-
int
PyList_Append
(PyObject* list, PyObject* item)¶ The C API equivalent of
list.append()
. Appenditem
to the end oflist
.Parameters: Returns: False on success or True with an exception set if an error occurred.
-
PyObject*
PyList_GetSlice
(PyObject* list, Py_ssize_t start, Py_ssize_t stop)¶ Get the sub-list from
start
tostop
. This is like the Python expression:list[start:stop]
.Parameters: - list (PyObject*) – The list to slice.
- start (Py_ssize_t) – The start index of the slice. Negative indices are not supported.
- stop (Py_ssize_t) – The stop index of the slice. Negative indices are not supported.
Returns: A new reference to
list[start:stop]
or NULL with an exception set if an error occurred.
-
int
PyList_SetSlice
(PyObject* list, Py_ssize_t start, Py_ssize_t stop, PyObject* itemlist)¶ Set the sub-list from
start
tostop
. This is like the Python statement:list[start:stop] = itemlist
ordel list[start:stop]
ifitemlist=NULL
.Parameters: - list (PyObject*) – The list to set the slice of.
- start (Py_ssize_t) – The start index of the slice. Negative indices are not supported.
- stop (Py_ssize_t) – The stop index of the slice. Negative indices are not supported.
- itemlist (PyObject*) – A list of items to assign to the slice. If
itemlist
isNULL
, this will delete the slice from the list.
Returns: False on success or True with an exception set if an error occurred.
-
int
PyList_Sort
(PyObject* list)¶ The C API equivalent of
list.sort()
. Sortslist
in place.Parameters: - list (PyObject*) – The list to sort.
Returns: False on success or True with an exception set if an error occurred.
PyMethodDef
¶
-
PyMethodDef
¶ https://docs.python.org/3.6/c-api/structures.html#c.PyMethodDef
Structure used to describe a method of an extension type. This structure has four fields.
-
char*
ml_name
¶ The name of the method as a C string.
-
PyCFunction
ml_meth
¶ A pointer to the C implmenetation of the method.
-
char*
ml_doc
¶ The contents of the method’s docstring as a C string. If this is
NULL
, the docstring will beNone
in Python. This should be created withPyDoc_STRVAR()
.
-
char*
-
METH_VARARGS
¶ This is the typical calling convention, where the methods have the type PyCFunction. The function expects two
PyObject*()
values. The first one is theself
object for methods; for module functions, it is the module object. The second parameter (often calledargs
) is a tuple object representing all arguments. This parameter is typically processed usingPyArg_ParseTuple()
orPyArg_UnpackTuple()
.
-
METH_KEYWORDS
¶ The function expects three parameters:
self
,args
, and adictionary
of all the keyword arguments. The flag must be combined with METH_VARARGS, and the parameters are typically processed usingPyArg_ParseTupleAndKeywords()
.
-
METH_NOARGS
¶ Methods without parameters don’t need to check whether arguments are given if they are listed with the
METH_NOARGS
flag. They need to be of typePyCFunction
. The first parameter is typically namedself
and will hold a reference to the module or object instance. In all cases the second parameter will beNULL
.
-
METH_O
¶ Methods with a single object argument can be listed with the
METH_O
flag, instead of invokingPyArg_ParseTuple()
with a “O” argument. They have the typePyCFunction
, with theself
parameter, and aPyObject*
parameter representing the single argument.
-
METH_CLASS
¶ Indicates that this method should be bound as a
classmethod
instead of an instance method. The first argument will be the class object.
-
METH_STATIC
¶ Indicates that this method should be bound as a
staticmethod
instead of an instance method. The first argument will always beNULL
.
-
METH_COEXIST
¶ Allow a method to have the same name as an automatically generated wrapper. This is useful for providing a specialized version of data model model method like
__contains__
when implementing the C-API version likesq_contains
.Note
This flag is not used often and can be ignored when writing most extensions.
PyCFunction
¶
-
PyCFunction
¶ Type of the functions used to implement most Python callables in C. Functions of this type take two
PyObject*
parameters and return one such value. If the return value isNULL
, an exception shall have been set. If notNULL
, the return value is interpreted as the return value of the function as exposed in Python. The function must return a new reference.
PyModuleDef
¶
-
PyModuleDef
¶ A specification of a Python module object.
-
PyModuleDef_Base
m_base
¶ The part of the module definition managed by CPython. Always initialize this with
PyModuleDef_HEAD_INIT
.
-
char*
m_name
¶ The name of the module as a C string.
-
char*
m_doc
¶ The docstring of the module as a C string. If set to
NULL
this will beNone
in Python This should be created withPyDoc_STRVAR()
.
-
Py_ssize_t
m_size
¶ The size of the module’s global state. If set to -1 this will not allocate any space for global state.
-
PyMethodDef*
m_methods
¶ A
NULL
terminated array of methods to put in this module.
-
PyModuleDef_Slot*
m_slots
¶ A
NULL
terminated array of slots for using multi-phase initialization. This is not used in this tutorial and can be set toNULL
.
-
traverseproc
m_traverse
¶ The function used for traversing the global state allocated with
m_size
. This can beNULL
if not needed.
-
PyModuleDef_Base
Global Sentinels¶
-
PyObject*
Py_None
¶ A global reference to
None
.-
Py_RETURN_NONE
¶ Shorthand for:
Py_INCREF(Py_None); return Py_None;
This is useful because people often forget that they need to call
Py_INCREF()
onPy_None
even though it is a global object.
-
-
PyObject*
Py_True
¶ A global reference to
True
.-
Py_RETURN_TRUE
¶ Shorthand for:
Py_INCREF(Py_True); return Py_True;
This is useful because people often forget that they need to call
Py_INCREF()
onPy_True
even though it is a global object.
-
-
PyObject*
Py_False
¶ A global reference to
False
.-
Py_RETURN_FALSE
¶ Shorthand for:
Py_INCREF(Py_False); return Py_False;
This is useful because people often forget that they need to call
Py_INCREF()
onPy_False
even though it is a global object.
-
CPython Functions and Macros¶
Py_REFCNT
¶
-
Py_ssize_t
Py_REFCNT
(ob)¶ https://docs.python.org/3.6/c-api/structures.html#c.Py_REFCNT
Access the reference count of an object. This expands to:
(((PyObject*) (o))->ob_refcnt)
.Parameters: - ob – The object to get the reference count of.
Returns: The reference count of
ob
.
Py_TYPE
¶
-
PyTypeObject*
Py_TYPE
(ob)¶ https://docs.python.org/3.6/c-api/structures.html#c.Py_TYPE
Access the Python type of an object as a
PyTypeObject*
. This returns a borrowed reference. This macro expands to:(((PyObject*) (o))->ob_type)
Parameters: - ob – The object to get the type of.
Returns: A borrowed reference to the type of
ob
.
PyDoc_STRVAR
¶
-
void
PyDoc_STRVAR
(name, docstring)¶ A macro for defining docstrings. This is used to support
./configure --without-doc-strings
. This macro expands to something like:#ifndef PyDoc_STR #define PyDoc_VAR(name) static char name[] #define PyDoc_STR(str) (str) #define PyDoc_STRVAR(name, str) PyDoc_VAR(name) = PyDoc_STR(str) #endif
PyModule_Create
¶
-
PyObject*
PyModule_Create
(PyModuleDef* def)¶ Create a new Python module object from a
PyModuleDef*
.
PyType_Ready
-
int
PyType_Ready
(PyTypeObject* type)¶ Ready a type by copying all of the slots down from the base class. This function should be called from the
PyMODINIT_FUNC
.Parameters: - type (PyTypeObject*) – The type to ready.
Returns: True with an exception set if an error occurred, otherwise False.
PyObject_Repr
¶
-
PyObject*
PyObject_Repr
(PyObject* ob)¶ Get the string representation of
ob
. This is the same asrepr(ob)
in Python.Parameters: - ob (PyObject*) – The object to repr.
Returns: A new reference to the string representation of
ob
.
PyObject_GetAttr
¶
PyObject_SetAttr
¶
PyObject_IsTrue
¶
-
int
PyObject_IsTrue
(PyObject* ob)¶ Check the truthiness of an object. This is the same as
bool(ob)
in Python.Note
This is not the same as
ob is True
in Python, this name is slightly confusing.Parameters: - ob (PyObject*) – The object to check the truthiness of.
Returns: True if the object is truthy, False if the object is falsey, -1 with an exception raised if an error occurred.
PyObject_RichCompare
¶
PyObject_RichCompareBool
¶
-
int
PyObject_RichCompareBool
(PyObject* ob_1, PyObject* ob_2, int op_id) Compare
ob_1
andob_2
with some comparison operator and convert the values to a C boolean.Parameters: Returns: The status of the operator or -1 with an exception set if an error occurred.
-
Py_LT
¶ ob_1 < ob_2
-
Py_LE
¶ ob_1 <= ob_2
-
Py_EQ
¶ ob_1 == ob_2
Note
PyObject_RichCompareBool()
will return True ifob_1 is ob_2
.
-
Py_NE
¶ ob_1 != ob_2
Note
PyObject_RichCompareBool()
will return False ofob_1 is ob_2
.
-
Py_GT
¶ ob_1 > ob_2
-
Py_GE
¶ ob_1 >= ob_2
PyArg_ParseTupleAndKeywords
¶
-
int
PyArg_ParseTupleAndKeywords
(PyObject* args, PyObject* kwargs, const char* format, char** keywords, ...)¶ Parse the argument tuple and dictionary for a
PyCFunction
.Parameters: - args (PyObject*) – The argument tuple passed to the
PyCFunction
. - kwargs (PyObject*) – The keyword argument dictionary passed to the
PyCFunction
. This can beNULL
. - char* format (const) – The format string. see format characters for more information.
- keywords (char**) – The names of the keyword arguments that this function
accepts as a
NULL
terminated array. - ... – Variadic values based on
format
.
Returns: True with an exception set if an error occurred, otherwise False.
- args (PyObject*) – The argument tuple passed to the
Example¶
The following example defines a function called function_name
which accepts
three arguments:
a
: A Pythonstr
object to be converted into achar*
.b
: A Pythonint
object to be converted into anint
.c
: An optional arbitrary Python object.
static PyObject*
function_name(PyObject* self, PyObject* args, PyObject* kwargs)
{
/* the names of the arguments */
static char* keywords[] = {"a", "b", "c", NULL};
/* C declarations for our arguments */
char* string;
int integer;
PyObject* object = NULL;
if (PyArg_ParseTupleAndKeywords(args,
kwargs,
"si|O:function_name",
&string, /* s /*
&integer, /* i */
&object /* O */)) {
/* failed to parse arguments, an error is set */
return NULL;
}
if (!object) {
/* ``c`` was not passed, set its default value here */
}
/* rest */
}
Format Characters¶
Below is a subset of commonly used format characters, see https://docs.python.org/3/c-api/arg.html#strings-and-buffers for a full list.
s
(str) [const char*]¶
Accept a str
argument as a char*
. A reference to a char*
should appear in the variadic argument list at this index.
z
(str) [const char*]¶
Accept a str
argument as a char*
. A reference to a char*
should appear in the variadic argument list at this index. This argument can
also be None
in which case the pointer will be NULL
.
b
(int) [unsigned char]¶
Accept an int
argument as an unsigned char
. A reference to
an unsigned char
should appear in the variadic argument list at this index.
h
(int) [short]¶
Accept an int
argument as a short
. A reference to a
short
should appear in the variadic argument list at this index.
i
(int) [int]¶
Accept an int
argument as an int
. A reference to an
int
should appear in the variadic argument list at this index.
l
(int) [long]¶
Accept an int
argument as a long
. A reference to a
long
should appear in the variadic argument list at this index.
n
(int) [Py_ssize_t]¶
Accept an int
argument as a Py_ssize_t
. A reference to a
Py_ssize_t
should appear in the variadic argument list at this index.
C
(str of length 1) [int]¶
Accept a str
of length 1 argument as an int
. A reference to an
int
should appear in the variadic argument list at this index.
f
(float) [float]¶
Accept a float
argument as a float
. A reference to a
float
should appear in the variadic argument list at this index.
d
(float) [double]¶
Accept a float
argument as a double
. A reference to a
double
should appear in the variadic argument list at this index.
O
(object) [PyObject*
]¶
Accept an object argument as a PyObject*
. This is a borrowed
reference. A reference to a PyObject*
should
appear in the variadic argument list at this index.
O!
(object) [PyTypeObject*
, PyObject*
]¶
Accept an argument as a PyObject*
. This object must be of a particular
Python type. This is a borrowed reference. This
format requires two values in the variadic argument list:
- A
PyTypeObject*
to check the type of the argument against. Parsing will fail if the object is not an instance of this type. - A reference to a
PyObject*
to write the result.
p
(bool) [int]¶
Accept any argument and check the truthiness of the value. A reference to an
int
should appear in the variadic argument list at this index. This is
like accepting an object as O
and then using PyObject_IsTrue()
.
Warning
The CPython docs mention converters for unsigned
integral types which do
not do overflow checking. These converters should not be used because
they fail to handle negative integers. The proper way to handle these values
is to accept them as O
and use one of the PyLong_As*
conversion
functions.
Special Characters¶
|
¶
All arguments following a pipe are optional. If an argument is not passed, the value of the reference in the variadic argument list is unchanged.
$
¶
All arguments following a dollar sign are keyword only arguments.
Note
The CPython docs say that this can only appear after a |
; however, this
is not actually true. You may have required keyword only arguments.
:
¶
This marks the end of the format list. Any text after this is used as the name of the function when generating error messages.
Number API¶
The Number API is sort of like the operator
module in Python.
For each function PyNumber_Op
there is a matching PyNumber_InPlaceOp
which is an augmented assignment version. For example: PyNumber_InPlaceAdd(a,
b)
is the same as a += b; return a
. This still returns a new
reference to a
which the caller owns.
-
PyObject*
PyNumber_Add
(PyObject* a, PyObject* b)¶ Returns: A new reference to a + b
orNULL
with an exception set.
-
PyObject*
PyNumber_Subtract
(PyObject* a, PyObject* b)¶ Returns: A new reference to a - b
orNULL
with an exception set.
-
PyObject*
PyNumber_Subtract
(PyObject* a, PyObject* b) Returns: A new reference to a * b
orNULL
with an exception set.
-
PyObject*
PyNumber_FloorDivide
(PyObject* a, PyObject* b)¶ Returns: A new reference to a // b
orNULL
with an exception set.
-
PyObject*
PyNumber_TrueDivide
(PyObject* a, PyObject* b)¶ Returns: A new reference to a / b
orNULL
with an exception set.
-
PyObject*
PyNumber_Remainder
(PyObject* a, PyObject* b)¶ Returns: A new reference to a % b
orNULL
with an exception set.
-
PyObject*
PyNumber_Divmod
(PyObject* a, PyObject* b)¶ Returns: A new reference to divmod(a, b)
orNULL
with an exception set.
-
PyObject*
PyNumber_Power
(PyObject* a, PyObject* b, PyObject* c)¶ Parameters: Returns: A new reference to
pow(a, b, c)
orNULL
with an exception set.
-
PyObject*
PyNumber_Negative
(PyObject* a)¶ Returns: A new reference to -a
orNULL
with an exception set.
-
PyObject*
PyNumber_Positive
(PyObject* a)¶ Returns: A new reference to +a
orNULL
with an exception set.
-
PyObject*
PyNumber_Absolute
(PyObject* a)¶ Returns: A new reference to abs(a)
orNULL
with an exception set.
-
PyObject*
PyNumber_Invert
(PyObject* a)¶ Returns: A new reference to ~a
orNULL
with an exception set.
-
PyObject*
PyNumber_Lshift
(PyObject* a, PyObject* b)¶ Returns: A new reference to a << b
orNULL
with an exception set.
-
PyObject*
PyNumber_Rshift
(PyObject* a, PyObject* b)¶ Returns: A new reference to a >> b
orNULL
with an exception set.
-
PyObject*
PyNumber_And
(PyObject* a, PyObject* b)¶ Note
This is bitwise
and
, not booleanand
.Returns: A new reference to a & b
orNULL
with an exception set.
Error Handling¶
-
void
PyErr_SetString
(PyObject* type, const char* message)¶ https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_SetString
Raise an exception of the given type with the given message.
Parameters: - type (PyObject*) – The exception type to raise.
- char* message (const) – The exception message.
-
PyObject*
PyErr_Format
(PyObject* type, const char* format, ...)¶ https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_Format
Raise an exception of the given type with a message created from a format string and values.
Parameters: - type (PyObject*) – The exception type to raise.
- char* format (const) – The exception format string.
- ... – The values to format into
format
.
Returns: Always
NULL
.
-
PyObject*
PyErr_NoMemory
()¶ https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_NoMemory
Raise an exception that indicates that memory could not be allocated.
Returns: Always NULL
.
-
PyObject*
PyErr_Occurred
()¶ https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_Occurred
Return a borrowed reference to the type of the currently raised exception. If no exception is raised, return
NULL
.This should just be used to check if an exception is raised, do not compare this value to exception types. To compare exceptions to an exception type use:
PyErr_ExceptionMatches()
.
-
int
PyErr_ExceptionMatches
(PyObject* exc)¶ https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_ExceptionMatches
Compare the currently raised exception type to
exc
. Returntrue
if the currently raised exception is an instance ofexc
.Parameters: - exc (PyObject*) – The exception type to compare the currently raised exception to.
Returns: 1
if the current exception is a subclass ofexc
,0
if it is not a subclass ofexc
, and-1
if an error occurs.Note
This can only be called if an exception is currently raised. This can be checked with
PyErr_Occurred()
.