Appendix

Reference Counting

Reference Count

The number of references to a given object. This is the count of objects which refer to the given object.

When an object is created, its reference count starts at one, which is the reference owned by the creator. The reference count can be increased with Py_INCREF() or decreased with Py_DECREF(). As soon as the reference count reaches zero, the object is no longer needed and it will be deallocated.

New Reference

A new reference is a PyObject* which must be managed by the user. The user must ensure that Py_DECREF() is called when the object is no longer needed.

Borrowed Reference

A borrowed reference is a PyObject* which is not owned. Users should not call Py_DECREF() when they are done with this object. A borrowed reference can become a new reference by calling Py_INCREF() on the it.

Reference Stealing

Some functions will “steal” a reference to an argument. This means the function will assume ownership of the reference that has been given to it. Callers should not call Py_DECREF() the object after the reference has been stolen. If you need to keep your own reference to the object after it has been stolen you need to call Py_INCREF() before the reference is stolen.

An example of this pattern is PyList_SetItem(). This function steals a reference to the value argument. This is to make it easier to allocate new objects and put them right into a PyListObject without needing extra calls to Py_INCREF().

Py_INCREF

void Py_INCREF(PyObject* ob)

The Py_INCREF macro increments the reference count of an object by one.

Parameters:
  • ob (PyObject*) – The object to increment the reference count of.

Py_XINCREF

void Py_XINCREF(PyObject* ob)

The Py_INCREF macro increments the reference count of an object if ob is not NULL.

Parameters:
  • ob (PyObject*) – The object to increment the reference count of.

Py_DECREF

void Py_DECREF(PyObject* ob)

The Py_DECREF macro decrements the reference count of an object by one.

Parameters:
  • ob (PyObject*) – The object to decrement the reference count of.

Py_XDECREF

void Py_XDECREF(PyObject* ob)

The Py_XDECREF macro decrements the reference count of an object if ob is not NULL.

Parameters:
  • ob (PyObject*) – The object to decrement the reference count of.

Py_CLEAR

void Py_CLEAR(PyObject* ob)

The Py_CLEAR macro decrements the reference count of an object and then sets the input PyObject* to NULL.

Parameters:
  • ob (PyObject*) – The object to decrement the reference count of and set to NULL.

CPython Types

Py_ssize_t

Py_ssize_t

A Py_ssize_ is a signed integral type used to hold counts and object sizes. On many platforms this is an alias of ssize_t.

PyObject

PyObject

https://docs.python.org/3.6/c-api/structures.html#c.PyObject

PyObject is the structure which holds Python values. The definition looks like:

Py_ssize_t ob_refcnt

The object’s reference count.

PyTypeObject* ob_type

The object’s type as a Python object.

Note

Users should not access these fields directly, instead go through the API macros: Py_REFCNT() and Py_TYPE().

PyTypeObject

PyTypeObject

https://docs.python.org/3/c-api/type.html

PyTypeObject is the structure which holds Python type objects.

Casting Rules

A PyTypeObject* can safely be cast to a PyObject*.

PyObject*s can be cast to PyTypeObject*s only after a PyType_Check.

Fields

const char* PyTypeObject.tp_name

The fully qualified name to the class as a C string. This string must stay alive forever.

This field is not inherited.

Py_ssize_t PyTypeObject.tp_basicsize

The base size of instances of this class. You should use sizeof to derive this from the instance struct type.

This field is inherited

Py_ssize_t PyTypeObject.tp_itemsize

Some classes require variable storage to hold instances. For example, PyTupleObjects stores all of the members inline. The PyTypeObject.tp_basicsize of a tuple would be the base object and element count, the PyTypeObject.tp_itemsize would be sizeof(PyObject*). If instances are always the same size, this field should be 0.

destructor PyTypeObject.tp_dealloc

A pointer to the object destructor. This function is called when the reference count hits 0. This function should release any references that it owns and then defer to PyTypeObject.tp_free to deallocate the memory.

This field is inherited.

printfunc PyTypeObject.tp_print

Reserved slot from Python 2. This is now unused.

getattrfunc PyTypeObject.tp_getattr

Deprecated in favor of PyTypeObject.tp_getattro

setattrfunc PyTypeObject.tp_setattr

Deprecated in favor of PyTypeObject.tp_setattro

PyAsyncMethods* PyTypeObject.tp_as_async

Pointer to additional functions used when an object supports the Async Protocol. If the type does not support the Async Protocol, this can be NULL.

The value of the pointer is not inherited, but the members are inherited individually.

reprfunc PyTypeObject.tp_repr

C API equivalent of __repr__

This can be invoked with PyObject_Repr().

This field is inherited.

PyNumberMethods* PyTypeObject.tp_as_number

Pointer to additional functions used when an object supports the number Protocol. If the type does not support the Number Protocol, this can be NULL.

The value of the pointer is not inherited, but the members are inherited individually.

PyNumberMethods

A collection of function pointers used to support the Number API.

  typedef struct {
      binaryfunc nb_add;
      binaryfunc nb_subtract;
      binaryfunc nb_multiply;
      binaryfunc nb_remainder;
      binaryfunc nb_divmod;
      ternaryfunc nb_power;
      unaryfunc nb_negative;
      unaryfunc nb_positive;
      unaryfunc nb_absolute;
      inquiry nb_bool;
      unaryfunc nb_invert;
      binaryfunc nb_lshift;
      binaryfunc nb_rshift;
      binaryfunc nb_and;
      binaryfunc nb_xor;
      binaryfunc nb_or;
      unaryfunc nb_int;
      void *nb_reserved;
      unaryfunc nb_float;

      binaryfunc nb_inplace_add;
      binaryfunc nb_inplace_subtract;
      binaryfunc nb_inplace_multiply;
      binaryfunc nb_inplace_remainder;
      ternaryfunc nb_inplace_power;
      binaryfunc nb_inplace_lshift;
      binaryfunc nb_inplace_rshift;
      binaryfunc nb_inplace_and;
      binaryfunc nb_inplace_xor;
      binaryfunc nb_inplace_or;

      binaryfunc nb_floor_divide;
      binaryfunc nb_true_divide;
      binaryfunc nb_inplace_floor_divide;
      binaryfunc nb_inplace_true_divide;

      unaryfunc nb_index;

      binaryfunc nb_matrix_multiply;
      binaryfunc nb_inplace_matrix_multiply;
} PyNumberMethods;
PySequenceMethods* PyTypeObject.tp_as_sequence

Pointer to additional functions used when an object supports the Sequence Protocol. If the type does not support the Sequence Protocol, this can be NULL.

The value of the pointer is not inherited, but the members are inherited individually.

PySequenceMethods

A collection of function pointers used to support the Sequence API.

lenfunc sq_length

The function used to support PyObject_Size(). This function will automatically be converted into a Python __len__.

binaryfunc sq_concat

The function used to support PySequence_Concat(). If there is no PyNumberMethods.nb_add function, this will be turned into a __add__ function (like list + list in Python).

ssizeargfunc sq_repeat

The function used to support PySequence_Repeat(). If there is no PyNumberMethods.nb_multiply this will be turned into a __mul__ function (like list * int in Python).

ssizeargfunc sq_item

The function used to support PySequenceMethods.sq_item(). This function will be converted into a __getitem__ method in Python.

ssizeobjargproc sq_ass_item

The function used to support PySequence_SetItem(). This slot can be NULL if the object doesn’t support assignment (like tuple). This will be converted into a __setitem__ method in Python.

objobjproc sq_contains

The function used to support PySequence_Contains(). If this is left NULL, a linear search will be performed. This function will be converted into a __contains__ method in Python.

binaryfunc sq_inplace_concat

The function used to support PySequence_InPlaceConcat(). It should modify self in place and then return self. This function is like list += list in Python.

binaryfunc sq_inplace_repeat

The function used to support PySequence_InPlaceRepeat(). It should modify self in place and then return self. This function is like list *= int in Python.

PyMappingMethods* PyTypeObject.tp_as_mapping

Pointer to additional functions used when an object supports the Mapping Protocol. If the type does not support the Mapping Protocol, this can be NULL.

The value of the pointer is not inherited, but the members are inherited individually.

hashfunc PyTypeObject.tp_hash

C API equivalent of __hash__.

This can be set to PyObject_HashNotImplemented() which is the equivalent of adding __hash__ = None in Python.

This can be invoked with PyObject_Hash().

This field is inherited along with PyTypeObject.tp_richcompare only when the subtype does not override either function.

ternaryfunc PyTypeObject.tp_call

C API equivalent of __call__.

This field is inherited.

reprfunc PyTypeObject.tp_str

The C API equivalent of _str__. If this not set, PyTypeObject.tp_repr will be used.

This can be invoked with PyObject_Str().

This field is inherited.

getattrofunc PyTypeObject.tp_getattro

The C API equivalent of __getattr__. This can be set to PyObject_GenericGetAttr() which implements standard attribute lookup.

This can be invoked with PyObject_GetAttr().

This field is inherited.

Note

To use PyObject_GenericGetAttr() the PyTypeObject.tp_dictoffset must be nonzero.

setattrofunc PyTypeObject.tp_setattrofunc

The C API equivalent of __setattr__. This can be set to PyObject_GenericSetAttr() which implements standard attribute assignment.

This can be invoked with PyObject_SetAttr().

This field is inherited.

Note

To use PyObject_GenericSetAttr() the PyTypeObject.tp_dictoffset must be nonzero.

PyBufferProcs* PyTypeObject.tp_as_buffer

Pointer to additional functions used when an object supports the Buffer Protocol. If the type does not support the Buffer Protocol, this can be NULL.

The value of the pointer is not inherited, but the members are inherited individually.

unsigned long PyTypeObject.tp_flags

A bitmask of information about the type.

Py_TPFLAGS_HEAPTYPE

This bit is set when the type object is allocated on the heap. This is true for classes defined in Python.

Py_TPFLAGS_BASETYPE

This bit is set when the type can be subclassed. Sometimes we don’t want to treat out members as virtual so we ignore the possibility of subclasses changing methods.

Attempting to subclass for a type without this bit set will generate an error like:

TypeError: type '<name>' is not an acceptable base type
Py_TPFLAGS_READY

Marks that PyType_Ready() has been called. You should not explicitly set this flag.

Py_TPFLAGS_READYING

Marks that PyType_Ready() is in the process of readying the type. You should not explicitly set this flag.

Py_TPFLAGS_HAVE_GC

Marks that this type supports the cyclic garbage collector. If this is set, instances need to be allocated with PyObject_GC_New() and freed with PyObject_GC_Del(). If this bit is set the PyTypeObject.tp_traverse function must be set.

Py_TPFLAGS_DEFAULT

The default set of bits that should be set for all new classes.

Py_TPFLAGS_LONG_SUBCLASS
Py_TPFLAGS_LIST_SUBCLASS
Py_TPFLAGS_TUPLE_SUBCLASS
Py_TPFLAGS_BYTES_SUBCLASS
Py_TPFLAGS_UNICODE_SUBCLASS
Py_TPFLAGS_DICT_SUBCLASS
Py_TPFLAGS_BASE_EXC_SUBCLASS
Py_TPFLAGS_TYPE_SUBCLASS

Marks that the type is a subclass of one of the builtin types. These dramatically speed up common Py*_Check calls which can then use a bit and instead of a generic PyObject_Isinstance() call. You should not set these bits directly, PyType_Ready() will do this if needed.

Py_TPFLAGS_HAVE_FINALIZE

Marks that the PyTypeObject.tp_finalize should be called on object destruction.

const char* PyTypeObject.tp_doc

A pointer to the docstring as a C string. If NULL, __doc__ will be None.

This field is not inherited.

traverseproc PyTypeObject.tp_traverse

A pointer to the cyclical garbage collector traversal function. This is only called if Py_TPFLAGS_HAVE_GC is set.

This function needs to call Py_VISIT() on all of the members which may participate in a cycle.

To use the Py_VISIT() macro the visitproc argument must be called visit and the void* argument must be called arg.

Below we have a type with two PyObject* fields and one non-object field.

typedef struct {
    PyObject mt_base;
    PyObject* mt_object_member;
    PyObject* mt_other_object_member;
    Py_ssize_t mt_not_an_object;
} mytype;

static int
mytype_traverse(mytype* self, visitproc visit, void* arg)
{
    Py_VISIT(self->mt_base);
    Py_VISIT(self->mt_object_member);
    /* note: not visiting self->mt_not_an_object because it is not a
       PyObject* */
    return 0;
}

Note

It is safe to not traverse immutable scalar members, for example: PyLongObject* members. It doesn’t hurt to visit all PyObject* members so if you aren’t sure, just visit it!

inquiry PyTypeObject.tp_clear

Clear any references owned by this object while keeping the object in a valid state. Valid state means that the interpreter should not segfault because an object assumes members are non-null and then tp_clear sets them to NULL. An example of this idea is list.clear(). This drops all of the references owned by the list; however, at the end you still have a valid list. This function is only called if Py_TPFLAGS_HAVE_GC is set.

When clearing references the Py_CLEAR() macro should be used instead of just Py_DECREF(). This is because decref can trigger a deallocation which can invoke arbitrary Python code through an object’s __del__ method. This code can reference back to the object being cleared and we don’t want to return a pointer to the recently destroyed object.

Below we have a type with two PyObject* fields and one non-object field.

typedef struct {
    PyObject mt_base;
    PyObject* mt_object_member;
    PyObject* mt_other_object_member;
    Py_ssize_t mt_not_an_object;
} mytype;

static int
mytype_clear(mytype* self)
{
    Py_CLEAR(self->mt_base);
    Py_CLEAR(self->mt_object_member);
    /* note: not clearing self->mt_not_an_object because it is not a
       PyObject* */
    return 0;
}

Note

Immutable objects can skip defining a PyTypeObject.tp_clear field. It is impossible to form a reference cycle with all immutable objects which means the other object’s PyTypeObject.tp_clear functions must be sufficient to clear the cycle.

richcmpfunc PyTypeObject.tp_richcompare

The function which supports PyObject_Richcompare() for this type.

See PyObject_Richcompare() for more information about how this function works.

This field is inherited along with PyTypeObject.tp_hash only when the subtype does not override either function.

Py_ssize_t PyTypeObject.tp_weaklistoffset

The offset into an object structure where the weaklist is stored. This is used for making objects weakly referenceable. If an object is not weakly referenceable, this field should be set to 0. This offset needs to point to a NULL initialized PyObject* slot in the instance structure.

In Python, if a class defines a __weakref__ slot in __slots__, that offset will be used as the PyTypeObject.tp_weaklistoffset.

Below we have a weakly referenceable type:

typedef struct {
    PyObject wr_base;
    PyObject* wr_weaklist;
    /* other data if we want */
} mytype;

PyTypeObject mytype_type = {
    /* ... */
    offsetof(mytype, wr_weaklist),  /* tp_weaklistoffset */
    /* ... */
};

Note that we use the ofsetoff operator to compute the offset accounting for the size of all members before wr_weaklist and any padding added by the compiler.

This field is inherited.

Note

Do not confuse this field with PyTypeObject.tp_weaklist which is the weaklist storage for taking weak references of the type object itself.

getiterfunc PyTypeObject.tp_iter

The C API equivalent of __iter__.

This can be invoked with PyObject_Iter().

This field is inherited.

Note

Objects may still be iterable without this function set if the support the Sequence Protocol. The PySequenceMethods.sq_item() function will be used from 0 until an PyExc_IndexError is raised just like in Python with __getitem__.

iternextfunc PyTypeObject.tp_iternext

The C API equivalent of __next__. The core difference is that PyExc_StopIteration does not need to be set when the iterator is exhausted. This will be done before returning to Python but the exception overhead can be avoided when being called from C.

If this slot is set, PyTypeObject.tp_iter should be a function which returns a new reference to self.

This can be invoked with PyIter_Next().

PyMethodDef* PyTypeObject.tp_methods

A NULL terminated array of PyMethodDef structures which will become the methods of the class. For each PyMethodDef in this list, a function object will be created and stored in the PyTypeObject.tp_dict.

The value of this field is not inherited but the methods will be.

PyMemberDef* PyTypeObject.tp_members

A NULL terminated array of PyMemberDef structures which will become the methods of the class. For each PyMemberDef in this list, a descriptor object will be created and stored in the PyTypeObject.tp_dict.

The value of this field is not inherited but the members will be.

PyMemberDef

A structure which defines a descriptor for exposing a C member as a Python member.

char* name

The name of the member as a C string.

int type

The type code of the member.

T_SHORT

A short member to be converted into a Python int.

T_INT

A int member to be converted into a Python int.

T_LONG

A long member to be converted into a Python int.

T_FLOAT

A float member to be converted into a Python float.

T_DOUBLE

A double member to be converted into a Python float.

T_STRING

A char* member to be converted into a Python str.

Note

This implies READONLY.

T_OBJECT

Get a PyObject*. If the pointer is NULL, None will be returned.

T_OBJECT_EX is often a better choice because of how it handles del.

T_OBJECT_EX

Get a PyObject*. If the pointer is NULL, a PyExc_AttributeError will be raised.

T_CHAR

A char member to be converted into a Python str of length 1.

T_BYTE

A char member to be converted into a Python int.

T_UBYTE

A unsigned char member to be converted into a Python int.

T_UINT

A unsigned int member to be converted into a Python int.

T_USHORT

A unsigned short member to be converted into a Python int.

T_ULONG

A unsigned short member to be converted into a Python int.

T_BOOL

A char member to be converted into a Python bool.

T_LONGLONG

A long long member to be converted into a Python int.

T_ULONGLONG

A unsigned long long member to be converted into a Python int.

T_PYSSIZET

A Py_ssize_t member to be converted into a Python int.

Py_ssize_t offset

The offset into the C structure where this member appears. You should always compute this offset with the offsetof operator.

int flags

Flag bits for indicating read or write status. The options are 0 for read and write access or READONLY for read only access.

T_STRING forces READONLY.

Only T_OBJECT and T_OBJECT_EX can be deleted with del which sets the pointer to NULL.

char* PyMemberDef.doc

The docstring for the member. If set to NULL __doc__ will be None.

PyGetSetDef* PyTypeObject.tp_getset

A NULL terminated array of PyGetSetDef structures which will become the methods of the class. For each PyGetSetDef in this list, a descriptor object will be created and stored in the PyTypeObject.tp_dict..

PyGetSetDef

A PyGetSetDef defines a computed attribute like a Python property

char* name

The name of this attribute as a C string.

getter get

The function used to compute this attribute.

setter set

The function used to assign to this attribute. If the attribute is readonly, this may be NULL.

char* doc

The docstring as C string. If this is NULL, __doc__ will be None.

void* closure

Extra data to be passed to the getter and setter functions.

getter
typedef PyObject* (*getter)(PyObject* self, void* closure);
setter
typedef int (*setter)(PyObject* self,
                      PyObject* value,
                      void* closure);
PyTypeObject* PyTypeObject.tp_base

The base class for this new type. If set to NULL, &PyBaseObject_Type (which is object in Python) is used.

This field is not inherited.

PyObject* PyTypeObject.tp_dict

The type’s dictionary (__dict__). This is initialized by PyType_Ready().

This field is not inherited but the members of the dictionary are.

Note

Types sometimes use what is called a “split keys dictionary” instead of a normal Python dict. You should not use the PyDict_* functions to modify this object.

descrgetfunc PyTypeObject.tp_descr_get

The C API equivalent of __get__ for the descriptor protocol.

This field is inherited.

descrgetfunc
typedef PyObject* (*descrgetfunc)(PyObject* self,
                                  PyObject* instance,
                                  PyObject* owner);
descrsetfunc PyTypeObject.tp_descr_set

The C API equivalent of __set__ and __delete__ for the descriptor protocol.

This field is inherited.

descrsetfunc
typedef PyObject* (*descrsetfunc)(PyObject* self,
                                  PyObject* instance,
                                  PyObject* value);

If value is NULL, this should delete the value.

Py_ssize_t PyTypeObject.tp_dictoffset

The offset into an object structure where the __dict__ is stored. This is used for allowing objects to have arbitrary attributes like a normal Python object.

For some objects, we only want to support a specific set of fields stored in the instance struct so we can set PyTypeObject.tp_dictoffset to 0 which indicates that we do not have a __dict__.

The instance structure must have a NULL initialized PyObject* member at the offset if it is nonzero.

Below we have a type with a __dict__:

typedef struct {
    PyObject wd_base;
    PyObject* wd_dict;
    /* other data if we want */
} mytype;

PyTypeObject mytype_type = {
    /* ... */
    offsetof(mytype, wd_dict),  /* tp_dictoffset */
    /* ... */
};

Note that we use the ofsetoff operator to compute the offset accounting for the size of all members before wd_dict and any padding added by the compiler.

This field is inherited.

Warning

It is not common to define a new class in C that uses a nonzero PyTypeObject.tp_dictoffset. You are going to lose a lot of potential performance gains by dispatching through a dictionary for all lookups.

initproc PyTypeObject.tp_init

The C API equivalent of __init__. This is only used for mutable types.

This field is inherited.

allocfunc PyTypeObject.tp_alloc

A function used to allocate memory for the new instance. This is different from __new__ in that it does not initialize any member data. It exists only as a way to separate object allocation and initialization.

The Py_ssize_t argument is the number of items in this instance. This is only meaningful for types with nonzero PyTypeObject.tp_itemsize.

All Py_TPFLAGS_HEAPTYPE objects use PyType_GenericAlloc() to force standard heap allocations. This is the default and recommended value for all types.

This field is inherited by static (C defined) subtypes but not heap (Python defined) subtypes.

newfunc PyTypeObject.tp_new

A function used to allocate and initialize new instances of the type.

PyTypeObject.tp_new should use PyTypeObject.tp_alloc to allocate the raw memory for the new instance.

If the type is immutable, the rest of the initialization should happen in the PyTypeObject.tp_new. If the type is mutable, the initialization should happen in the PyTypeObject.tp_init which will be called automatically.

This field is inherited.

destructor PyTypeObject.tp_free

A function used to deallocate the memory allocated with PyTypeObject.tp_alloc.

By default this is a function that is compatible with the PyType_GenericAlloc() allocator.

This field is inherited by static (C defined) subtypes but not heap (Python defined) subtypes.

inquiry PyTypeObject.tp_is_gc

This function is used when a type has a mix of statically and dynamically allocated instances. In this case, the Py_TPFLAGS_HAVE_GC flag is not enough to know if an instance can be collected. In this case, the function should be implemented to return True if the instance can be collected, otherwise False.

This field is inherited.

Note

This is a very uncommon function to implement. It is mainly used to support static and dynamic class objects in CPython itself.

PyObject* PyTypeObject.tp_bases

A tuple of the base types. This field is set by PyType_Ready() and should not be manually modified.

This field is not inherited.

PyObject* PyTypeObject.tp_mro

A tuple of the full method resolution order. This field is set by PyType_Ready() and should not be manually modified.

This field is not inherited.

destructor PyTypeObject.tp_finalize

A function called before the garbage collector clears references or deallocates the object. This function is only called if Py_TPFLAGS_HAVE_FINALIZE is set.

This can be useful for releasing C level resources like file descriptors.

See PEP 442 for more information about this field.

This field is inherited.

PyObject* PyTypeObject.tp_cache

Internal use only.

PyObject* PyTypeObject.tp_subclasses.

Internal use only.

PyObject* PyTypeObject.tp_weaklist

The member to hold the weaklist for weak references to the type object itself.

Note

Do not confuse this field with PyTypeObject.tp_weaklistoffset which is the offset into the instance object where weak references to the instance will be stored.

Support Types

destructor
typedef void (*destructor)(PyObject*);
getattrfunc
typedef PyObject* (*getattrfunc)(PyObject* self, char* attr_name);
setattrfunc
typedef PyObject* (*setattrfunc)(PyObject* self, char* attr_name, PyObject* value);
reprfunc::
typedef PyObject* (*reprfunc)(PyObject*);
inquiry
typedef int (*inquiry)(PyObject*);
newfunc
typedef PyObject* (*newfunc)(PyObject* self, PyObject* args, PyObject* kwargs);
allocfunc
typedef PyObject* (*allocfunc)(PyTypeObject* cls, Py_ssize_t nitems);
initproc
typedef int (*initproc)(PyObject* self, PyObject* args, PyObject* kwargs);
unaryfunc
typedef PyObject* (*unaryfunc)(PyObject*);
binaryfunc
typedef PyObject* (*binaryfunc)(PyObject*, PyObject*);
ternaryfunc
typedef PyObject* (*ternaryfunc)(PyObject*, PyObject*, PyObject*);
lenfunc
typedef Py_ssize_t (*lenfunc)(PyObject*);
ssizeargfunc
typedef PyObject* (*ssizeargfunc)(PyObject*, Py_ssize_t);
ssizeobjargproc
typedef int (*ssizeobjargproc)(PyObject* Py_ssize_t, PyObject*);
objobjproc
typedef int (*objobjproc)(PyObject*, PyObject*);
hashfunc
typedef Py_hash_t (*hashfunc)(PyObject*);
getattrofunc
typedef PyObject* (*getattrofunc)(PyObject*, PyObject*);
setattrofunc
typedef PyObject* (*getattrofunc)(PyObject*, PyObject*, PyObject*);
richcmpfunc
typedef PyObject* (*richcmpfunc)(PyObject* lhs, PyObject* rhs, int op);

PyLongObject

PyLongObject

https://docs.python.org/3/c-api/long.html

PyLongObject is the structure which holds Python int objects. This is called a PyLongObject as a hold over from when arbitrary width integer was called a long object in Python 2.

Casting Rules

A PyLongObject* can safely be cast to a PyObject*.

PyObject*s can be cast to PyLongObject*s only after a PyLong_Check().

API Functions

int PyLong_Check(PyObject* ob)

Check if the instance is a PyLongObject.

Parameters:
Returns:

True if ob is an instance of PyLongObject or an instance of a subclass of PyLongObject.

unsigned long PyLong_AsUnsignedLong(PyObject* ob)

Convert a PyObject* of type PyLongObject* to an unsigned long. If ob is not a long object, an exception is raised.

Parameters:
Returns:

ob as an unsigned long.

PyObject* PyLong_FromUnsignedLong(unsigned long l)

Convert an unsigned long into a PyObject*. If the object cannot be allocated an exception is raised.

Parameters:
  • l – The unsigned long to convert to a pyobjectptr.
Returns:

A new reference to l as a Python object.

PyListObject

PyListObject

https://docs.python.org/3/c-api/list.html

PyListObject is the structure which holds Python list objects.

Casting Rules

A PyListObject* can safely be cast to a PyObject*.

PyObject*s can be cast to PyListObject*s only after a PyList_Check().

API Functions

int PyList_Check(PyObject* ob)

Check if the instance is a PyListObject.

Parameters:
Returns:

True if ob is an instance of PyListObject or an instance of a subclass of PyListObject.

PyObject* PyList_New(Py_ssize_t len)

Create a new list object of length len. The members are set to NULL and need to be filled with PyList_SET_ITEM() before returning the list to Python.

Parameters:
  • len (Py_ssize_t) – The length of the list to create.
Returns:

A new reference to a PyListObject. The elements must be filled in.

Py_ssize_t PyList_Size(PyObject* list)

Return the length of the list with error checking.

Parameters:
  • list (PyObject*) – The list to get the size of.
Returns:

The size of the list or -1 with an exception set if an error occurred. If list is not actually a PyListObject an error will be raised.

Py_ssize_t PyList_GET_SIZE(PyObject* list)

Return the length of the list without error checking.

Parameters:
  • list (PyObject*) – The list to get the size of.
Returns:

The size of the list. This function cannot be called on objects that are not known to be PyListObjects.

PyObject* PyList_GetItem(PyObject* list, Py_ssize_t ix)

Lookup an item in a list with error checking.

Parameters:
  • list (PyObject*) – The list to get an element in.
  • ix (Py_ssize_t) – The index to lookup. Negative indices are not supported.
Returns:

A borrowed reference to the element at index ix or NULL with an PyExc_IndexError set if ix is out of bounds.

PyObject* PyList_GET_ITEM(PyObject* list, Py_ssize_t ix)

Lookup an item in a list without error checking.

Parameters:
  • list (PyObject*) – The list to get an element in.
  • ix (Py_ssize_t) – The index to lookup. Negative indices are not supported.
Returns:

A borrowed reference to the element at index ix. It is undefined behavior if ix is out of bounds for list.

int PyList_SetItem(PyObject* list, Py_ssize_t ix, PyObject* value)

Assign an item in a list.

Parameters:
  • list (PyObject*) – The list to set the element in.
  • ix (Py_ssize_t) – The index to assign to. Negative indices are not supported.
  • value (PyObject*) – The value to assign. This reference is stolen. The old reference at ix will be released.
Returns:

False on success or True with an exception set if an error occurred.

void PyList_SET_ITEM(PyObject* list, Py_ssize_t ix)

Lookup an item in a list without error checking.

Parameters:
  • list (PyObject*) – The list to set the element in.
  • ix (Py_ssize_t) – The index to assign to. Negative indices are not supported. ix must be in bounds for list.
  • value (PyObject*) – The value to assign. This reference is stolen. The old reference at ix will be not released and will be leaked if not NULL.
int PyList_Insert(PyObject* list, Py_ssize_t ix, PyObject* item)

The C API equivalent of list.insert(). Inserts item into list before ix.

Parameters:
Returns:

False on success or True with an exception set if an error occurred.

int PyList_Append(PyObject* list, PyObject* item)

The C API equivalent of list.append(). Append item to the end of list.

Parameters:
  • list (PyObject*) – The list to append to.
  • item (PyObject*) – The item to append to list.
Returns:

False on success or True with an exception set if an error occurred.

PyObject* PyList_GetSlice(PyObject* list, Py_ssize_t start, Py_ssize_t stop)

Get the sub-list from start to stop. This is like the Python expression: list[start:stop].

Parameters:
  • list (PyObject*) – The list to slice.
  • start (Py_ssize_t) – The start index of the slice. Negative indices are not supported.
  • stop (Py_ssize_t) – The stop index of the slice. Negative indices are not supported.
Returns:

A new reference to list[start:stop] or NULL with an exception set if an error occurred.

int PyList_SetSlice(PyObject* list, Py_ssize_t start, Py_ssize_t stop, PyObject* itemlist)

Set the sub-list from start to stop. This is like the Python statement: list[start:stop] = itemlist or del list[start:stop] if itemlist=NULL.

Parameters:
  • list (PyObject*) – The list to set the slice of.
  • start (Py_ssize_t) – The start index of the slice. Negative indices are not supported.
  • stop (Py_ssize_t) – The stop index of the slice. Negative indices are not supported.
  • itemlist (PyObject*) – A list of items to assign to the slice. If itemlist is NULL, this will delete the slice from the list.
Returns:

False on success or True with an exception set if an error occurred.

int PyList_Sort(PyObject* list)

The C API equivalent of list.sort(). Sorts list in place.

Parameters:
Returns:

False on success or True with an exception set if an error occurred.

int PyList_Reverse(PyObject* list)

The C API equivalent of list.reverse(). Reverses a list in place.

Parameters:
Returns:

False on success or True with an exception set if an error occurred.

PyMethodDef

PyMethodDef

https://docs.python.org/3.6/c-api/structures.html#c.PyMethodDef

Structure used to describe a method of an extension type. This structure has four fields.

char* ml_name

The name of the method as a C string.

PyCFunction ml_meth

A pointer to the C implmenetation of the method.

int ml_flags

Flag bits indicating how to call ml_meth should be called or bound to a class.

char* ml_doc

The contents of the method’s docstring as a C string. If this is NULL, the docstring will be None in Python. This should be created with PyDoc_STRVAR().

METH_VARARGS

This is the typical calling convention, where the methods have the type PyCFunction. The function expects two PyObject*() values. The first one is the self object for methods; for module functions, it is the module object. The second parameter (often called args) is a tuple object representing all arguments. This parameter is typically processed using PyArg_ParseTuple() or PyArg_UnpackTuple().

METH_KEYWORDS

The function expects three parameters: self, args, and a dictionary of all the keyword arguments. The flag must be combined with METH_VARARGS, and the parameters are typically processed using PyArg_ParseTupleAndKeywords().

METH_NOARGS

Methods without parameters don’t need to check whether arguments are given if they are listed with the METH_NOARGS flag. They need to be of type PyCFunction. The first parameter is typically named self and will hold a reference to the module or object instance. In all cases the second parameter will be NULL.

METH_O

Methods with a single object argument can be listed with the METH_O flag, instead of invoking PyArg_ParseTuple() with a “O” argument. They have the type PyCFunction, with the self parameter, and a PyObject* parameter representing the single argument.

METH_CLASS

Indicates that this method should be bound as a classmethod instead of an instance method. The first argument will be the class object.

METH_STATIC

Indicates that this method should be bound as a staticmethod instead of an instance method. The first argument will always be NULL.

METH_COEXIST

Allow a method to have the same name as an automatically generated wrapper. This is useful for providing a specialized version of data model model method like __contains__ when implementing the C-API version like sq_contains.

Note

This flag is not used often and can be ignored when writing most extensions.

PyCFunction

PyCFunction

Type of the functions used to implement most Python callables in C. Functions of this type take two PyObject* parameters and return one such value. If the return value is NULL, an exception shall have been set. If not NULL, the return value is interpreted as the return value of the function as exposed in Python. The function must return a new reference.

PyModuleDef

PyModuleDef

A specification of a Python module object.

PyModuleDef_Base m_base

The part of the module definition managed by CPython. Always initialize this with PyModuleDef_HEAD_INIT.

char* m_name

The name of the module as a C string.

char* m_doc

The docstring of the module as a C string. If set to NULL this will be None in Python This should be created with PyDoc_STRVAR().

Py_ssize_t m_size

The size of the module’s global state. If set to -1 this will not allocate any space for global state.

PyMethodDef* m_methods

A NULL terminated array of methods to put in this module.

PyModuleDef_Slot* m_slots

A NULL terminated array of slots for using multi-phase initialization. This is not used in this tutorial and can be set to NULL.

traverseproc m_traverse

The function used for traversing the global state allocated with m_size. This can be NULL if not needed.

inquiry m_clear

The function used for clearing the global state allocated with m_size. This can be NULL if not needed.

freefunc m_free

The function used for freeing the global state allocated with m_size. This can be NULL if not needed.

PyMODINIT_FUNC

PyMODINIT_FUNC

The type of the module initialization function. This function should return a new module object or NULL if the file cannot be imported. The function needs to be named: PyInit_{name} where name is the name of the module.

Global Sentinels

PyObject* Py_None

A global reference to None.

Py_RETURN_NONE

Shorthand for:

Py_INCREF(Py_None);
return Py_None;

This is useful because people often forget that they need to call Py_INCREF() on Py_None even though it is a global object.

PyObject* Py_True

A global reference to True.

Py_RETURN_TRUE

Shorthand for:

Py_INCREF(Py_True);
return Py_True;

This is useful because people often forget that they need to call Py_INCREF() on Py_True even though it is a global object.

PyObject* Py_False

A global reference to False.

Py_RETURN_FALSE

Shorthand for:

Py_INCREF(Py_False);
return Py_False;

This is useful because people often forget that they need to call Py_INCREF() on Py_False even though it is a global object.

CPython Functions and Macros

Py_REFCNT

Py_ssize_t Py_REFCNT(ob)

https://docs.python.org/3.6/c-api/structures.html#c.Py_REFCNT

Access the reference count of an object. This expands to: (((PyObject*) (o))->ob_refcnt).

Parameters:
  • ob – The object to get the reference count of.
Returns:

The reference count of ob.

Py_TYPE

PyTypeObject* Py_TYPE(ob)

https://docs.python.org/3.6/c-api/structures.html#c.Py_TYPE

Access the Python type of an object as a PyTypeObject*. This returns a borrowed reference. This macro expands to: (((PyObject*) (o))->ob_type)

Parameters:
  • ob – The object to get the type of.
Returns:

A borrowed reference to the type of ob.

PyDoc_STRVAR

void PyDoc_STRVAR(name, docstring)

A macro for defining docstrings. This is used to support ./configure --without-doc-strings. This macro expands to something like:

#ifndef PyDoc_STR
#define PyDoc_VAR(name)         static char name[]
#define PyDoc_STR(str)          (str)
#define PyDoc_STRVAR(name, str) PyDoc_VAR(name) = PyDoc_STR(str)
#endif

PyModule_Create

PyObject* PyModule_Create(PyModuleDef* def)

Create a new Python module object from a PyModuleDef*.

PyType_Ready

int PyType_Ready(PyTypeObject* type)

Ready a type by copying all of the slots down from the base class. This function should be called from the PyMODINIT_FUNC.

Parameters:
Returns:

True with an exception set if an error occurred, otherwise False.

PyObject_Repr

PyObject* PyObject_Repr(PyObject* ob)

Get the string representation of ob. This is the same as repr(ob) in Python.

Parameters:
Returns:

A new reference to the string representation of ob.

PyObject_GetAttr

PyObject* PyObject_GetAttr(PyObject* ob, PyObject* attr_name)

Lookup an attribute on a Python object. This is the same as getattr(ob, attr_name) in Python.

Parameters:
  • ob (PyObject*) – The object to lookup the attribute on.
  • attr_name (PyObject*) – The name of the attribute to lookup.
Returns:

The attribute name attr_name on ob or NULL with an exception set if the attribute doesn’t exist.

PyObject_SetAttr

int PyObject_SetAttr(PyObject* ob, PyObject* attr_name, PyObject* value)

Set an attribute on a Python object. This is the same as setattr(ob, attr_name, value) in Python.

Parameters:
  • ob (PyObject*) – The object to set the attribute on.
  • attr_name (PyObject*) – The name of the attribute to set.
  • value (PyObject*) – The value of the attribute to set.
Returns:

True with an exception set if an error occurred, otherwise False.

PyObject_IsTrue

int PyObject_IsTrue(PyObject* ob)

Check the truthiness of an object. This is the same as bool(ob) in Python.

Note

This is not the same as ob is True in Python, this name is slightly confusing.

Parameters:
  • ob (PyObject*) – The object to check the truthiness of.
Returns:

True if the object is truthy, False if the object is falsey, -1 with an exception raised if an error occurred.

PyObject_RichCompare

PyObject* PyObject_RichCompareBool(PyObject* ob_1, PyObject* ob_2, int op_id)

Compare ob_1 and ob_2 with some comparison operator.

Parameters:
Returns:

The result of the operator.

PyObject_RichCompareBool

int PyObject_RichCompareBool(PyObject* ob_1, PyObject* ob_2, int op_id)

Compare ob_1 and ob_2 with some comparison operator and convert the values to a C boolean.

Parameters:
Returns:

The status of the operator or -1 with an exception set if an error occurred.

Py_LT

ob_1 < ob_2

Py_LE

ob_1 <= ob_2

Py_EQ

ob_1 == ob_2

Note

PyObject_RichCompareBool() will return True if ob_1 is ob_2.

Py_NE

ob_1 != ob_2

Note

PyObject_RichCompareBool() will return False of ob_1 is ob_2.

Py_GT

ob_1 > ob_2

Py_GE

ob_1 >= ob_2

PyArg_ParseTupleAndKeywords

int PyArg_ParseTupleAndKeywords(PyObject* args, PyObject* kwargs, const char* format, char** keywords, ...)

Parse the argument tuple and dictionary for a PyCFunction.

Parameters:
  • args (PyObject*) – The argument tuple passed to the PyCFunction.
  • kwargs (PyObject*) – The keyword argument dictionary passed to the PyCFunction. This can be NULL.
  • char* format (const) – The format string. see format characters for more information.
  • keywords (char**) – The names of the keyword arguments that this function accepts as a NULL terminated array.
  • ... – Variadic values based on format.
Returns:

True with an exception set if an error occurred, otherwise False.

Example

The following example defines a function called function_name which accepts three arguments:

  • a: A Python str object to be converted into a char*.
  • b: A Python int object to be converted into an int.
  • c: An optional arbitrary Python object.
static PyObject*
function_name(PyObject* self, PyObject* args, PyObject* kwargs)
{
    /* the names of the arguments */
    static char* keywords[] = {"a", "b", "c", NULL};

    /* C declarations for our arguments */
    char* string;
    int integer;
    PyObject* object = NULL;

    if (PyArg_ParseTupleAndKeywords(args,
                                    kwargs,
                                    "si|O:function_name",
                                    &string,   /* s /*
                                    &integer,  /* i */
                                    &object    /* O */)) {
        /* failed to parse arguments, an error is set */
        return NULL;
    }

    if (!object) {
        /* ``c`` was not passed, set its default value here */
    }

    /* rest */
}

Format Characters

Below is a subset of commonly used format characters, see https://docs.python.org/3/c-api/arg.html#strings-and-buffers for a full list.

s (str) [const char*]

Accept a str argument as a char*. A reference to a char* should appear in the variadic argument list at this index.

z (str) [const char*]

Accept a str argument as a char*. A reference to a char* should appear in the variadic argument list at this index. This argument can also be None in which case the pointer will be NULL.

b (int) [unsigned char]

Accept an int argument as an unsigned char. A reference to an unsigned char should appear in the variadic argument list at this index.

h (int) [short]

Accept an int argument as a short. A reference to a short should appear in the variadic argument list at this index.

i (int) [int]

Accept an int argument as an int. A reference to an int should appear in the variadic argument list at this index.

l (int) [long]

Accept an int argument as a long. A reference to a long should appear in the variadic argument list at this index.

n (int) [Py_ssize_t]

Accept an int argument as a Py_ssize_t. A reference to a Py_ssize_t should appear in the variadic argument list at this index.

C (str of length 1) [int]

Accept a str of length 1 argument as an int. A reference to an int should appear in the variadic argument list at this index.

f (float) [float]

Accept a float argument as a float. A reference to a float should appear in the variadic argument list at this index.

d (float) [double]

Accept a float argument as a double. A reference to a double should appear in the variadic argument list at this index.

O (object) [PyObject*]

Accept an object argument as a PyObject*. This is a borrowed reference. A reference to a PyObject* should appear in the variadic argument list at this index.

O! (object) [PyTypeObject*, PyObject*]

Accept an argument as a PyObject*. This object must be of a particular Python type. This is a borrowed reference. This format requires two values in the variadic argument list:

  1. A PyTypeObject* to check the type of the argument against. Parsing will fail if the object is not an instance of this type.
  2. A reference to a PyObject* to write the result.
p (bool) [int]

Accept any argument and check the truthiness of the value. A reference to an int should appear in the variadic argument list at this index. This is like accepting an object as O and then using PyObject_IsTrue().

Warning

The CPython docs mention converters for unsigned integral types which do not do overflow checking. These converters should not be used because they fail to handle negative integers. The proper way to handle these values is to accept them as O and use one of the PyLong_As* conversion functions.

Special Characters

|

All arguments following a pipe are optional. If an argument is not passed, the value of the reference in the variadic argument list is unchanged.

$

All arguments following a dollar sign are keyword only arguments.

Note

The CPython docs say that this can only appear after a |; however, this is not actually true. You may have required keyword only arguments.

:

This marks the end of the format list. Any text after this is used as the name of the function when generating error messages.

Number API

The Number API is sort of like the operator module in Python.

For each function PyNumber_Op there is a matching PyNumber_InPlaceOp which is an augmented assignment version. For example: PyNumber_InPlaceAdd(a, b) is the same as a += b; return a. This still returns a new reference to a which the caller owns.

PyObject* PyNumber_Add(PyObject* a, PyObject* b)
Returns:A new reference to a + b or NULL with an exception set.
PyObject* PyNumber_Subtract(PyObject* a, PyObject* b)
Returns:A new reference to a - b or NULL with an exception set.
PyObject* PyNumber_Subtract(PyObject* a, PyObject* b)
Returns:A new reference to a * b or NULL with an exception set.
PyObject* PyNumber_FloorDivide(PyObject* a, PyObject* b)
Returns:A new reference to a // b or NULL with an exception set.
PyObject* PyNumber_TrueDivide(PyObject* a, PyObject* b)
Returns:A new reference to a / b or NULL with an exception set.
PyObject* PyNumber_Remainder(PyObject* a, PyObject* b)
Returns:A new reference to a % b or NULL with an exception set.
PyObject* PyNumber_Divmod(PyObject* a, PyObject* b)
Returns:A new reference to divmod(a, b) or NULL with an exception set.
PyObject* PyNumber_Power(PyObject* a, PyObject* b, PyObject* c)
Parameters:
  • a (PyObject*) – The base.
  • b (PyObject*) – The exponent.
  • c (PyObject*) – Number to take the exponent modulo. If provided, this function is like (a ** b) % c. To ignore this value pass Py_None.
Returns:

A new reference to pow(a, b, c) or NULL with an exception set.

PyObject* PyNumber_Negative(PyObject* a)
Returns:A new reference to -a or NULL with an exception set.
PyObject* PyNumber_Positive(PyObject* a)
Returns:A new reference to +a or NULL with an exception set.
PyObject* PyNumber_Absolute(PyObject* a)
Returns:A new reference to abs(a) or NULL with an exception set.
PyObject* PyNumber_Invert(PyObject* a)
Returns:A new reference to ~a or NULL with an exception set.
PyObject* PyNumber_Lshift(PyObject* a, PyObject* b)
Returns:A new reference to a << b or NULL with an exception set.
PyObject* PyNumber_Rshift(PyObject* a, PyObject* b)
Returns:A new reference to a >> b or NULL with an exception set.
PyObject* PyNumber_And(PyObject* a, PyObject* b)

Note

This is bitwise and, not boolean and.

Returns:A new reference to a & b or NULL with an exception set.
PyObject* PyNumber_Xor(PyObject* a, PyObject* b)
Returns:A new reference to a ^ b or NULL with an exception set.
PyObject* PyNumber_Or(PyObject* a, PyObject* b)

Note

This is bitwise or, not boolean or.

Returns:A new reference to a | b or NULL with an exception set.

Error Handling

void PyErr_SetString(PyObject* type, const char* message)

https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_SetString

Raise an exception of the given type with the given message.

Parameters:
  • type (PyObject*) – The exception type to raise.
  • char* message (const) – The exception message.
PyObject* PyErr_Format(PyObject* type, const char* format, ...)

https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_Format

Raise an exception of the given type with a message created from a format string and values.

Parameters:
  • type (PyObject*) – The exception type to raise.
  • char* format (const) – The exception format string.
  • ... – The values to format into format.
Returns:

Always NULL.

PyObject* PyErr_NoMemory()

https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_NoMemory

Raise an exception that indicates that memory could not be allocated.

Returns:Always NULL.
PyObject* PyErr_Occurred()

https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_Occurred

Return a borrowed reference to the type of the currently raised exception. If no exception is raised, return NULL.

This should just be used to check if an exception is raised, do not compare this value to exception types. To compare exceptions to an exception type use: PyErr_ExceptionMatches().

int PyErr_ExceptionMatches(PyObject* exc)

https://docs.python.org/3.6/c-api/exceptions.html#c.PyErr_ExceptionMatches

Compare the currently raised exception type to exc. Return true if the currently raised exception is an instance of exc.

Parameters:
  • exc (PyObject*) – The exception type to compare the currently raised exception to.
Returns:

1 if the current exception is a subclass of exc, 0 if it is not a subclass of exc, and -1 if an error occurs.

Note

This can only be called if an exception is currently raised. This can be checked with PyErr_Occurred().