How to use NumPy array with ctypes?
Using np.ndarrays
as ctypes
arguments
The preferable approach is using ndpointer
, as mentioned in the numpy-docs.
This approach is more flexible than using, for example, POINTER(c_double), since several restrictions can be specified, which are verified upon calling the ctypes function. These include data type, number of dimensions, shape and flags. If a given array does not satisfy the specified restrictions, a TypeError is raised.
Minimal, Reproducible Example
Calling memcpy from python. Eventually the filename of the standard C-library libc.so.6
needs to be adjusted.
import ctypes
import numpy as np
n_bytes_f64 = 8
nrows = 2
ncols = 5
clib = ctypes.cdll.LoadLibrary("libc.so.6")
clib.memcpy.argtypes = [
np.ctypeslib.ndpointer(dtype=np.float64, ndim=2, flags='C_CONTIGUOUS'),
np.ctypeslib.ndpointer(dtype=np.float64, ndim=1, flags='C_CONTIGUOUS'),
ctypes.c_size_t]
clib.memcpy.restype = ctypes.c_void_p
arr_from = np.arange(nrows * ncols).astype(np.float64)
arr_to = np.empty(shape=(nrows, ncols), dtype=np.float64)
print('arr_from:', arr_from)
print('arr_to:', arr_to)
print('\ncalling clib.memcpy ...\n')
clib.memcpy(arr_to, arr_from, nrows * ncols * n_bytes_f64)
print('arr_from:', arr_from)
print('arr_to:', arr_to)
Output
arr_from: [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
arr_to: [[0.0e+000 4.9e-324 9.9e-324 1.5e-323 2.0e-323]
[2.5e-323 3.0e-323 3.5e-323 4.0e-323 4.4e-323]]
calling clib.memcpy ...
arr_from: [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
arr_to: [[0. 1. 2. 3. 4.]
[5. 6. 7. 8. 9.]]
If you modify the ndim=1/2
arguments of ndpointer
to be inconsistent with the dimensions of arr_from/arr_to
, the code fails with an ArgumentError
.
As the title of this question is quite general, ...
Constructing a np.ndarray
from a ctypes.c_void_p
result
Minimal, Reproducible Example
In the following example, some memory is allocated by malloc and filled with 0s by memset. Then a numpy array is constructed, to access this memory. Of course the occur some ownership issues, as python will not free memory, which was allocated in c. To avoid memory leaks, one has to free the allocated memory again by ctypes. The copy method can be used for the np.ndarray
to acquire ownership.
import ctypes
import numpy as np
n_bytes_int = 4
size = 7
clib = ctypes.cdll.LoadLibrary("libc.so.6")
clib.malloc.argtypes = [ctypes.c_size_t]
clib.malloc.restype = ctypes.c_void_p
clib.memset.argtypes = [
ctypes.c_void_p,
ctypes.c_int,
ctypes.c_size_t]
clib.memset.restype = np.ctypeslib.ndpointer(
dtype=np.int32, ndim=1, flags='C_CONTIGUOUS')
clib.free.argtypes = [ctypes.c_void_p]
clib.free.restype = ctypes.c_void_p
pntr = clib.malloc(size * n_bytes_int)
ndpntr = clib.memset(pntr, 0, size * n_bytes_int)
print(type(ndpntr))
ctypes_pntr = ctypes.cast(ndpntr, ctypes.POINTER(ctypes.c_int))
print(type(ctypes_pntr))
print()
arr_noowner = np.ctypeslib.as_array(ctypes_pntr, shape=(size,))
arr_owner = np.ctypeslib.as_array(ctypes_pntr, shape=(size,)).copy()
# arr_owner = arr_noowner.copy()
print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))
print('\nfree allocated memory again ...\n')
_ = clib.free(pntr)
print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))
print('\njust for fun: free some python-memory ...\n')
_ = clib.free(arr_owner.ctypes.data_as(ctypes.c_void_p))
print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))
Output
<class 'numpy.ctypeslib.ndpointer_<i4_1d_C_CONTIGUOUS'>
<class '__main__.LP_c_int'>
arr_noowner (at 104719884831376): [0 0 0 0 0 0 0]
arr_owner (at 104719884827744): [0 0 0 0 0 0 0]
free allocated memory again ...
arr_noowner (at 104719884831376): [ -7687536 24381 -28516336 24381 0 0 0]
arr_owner (at 104719884827744): [0 0 0 0 0 0 0]
just for fun: free some python-memory ...
arr_noowner (at 104719884831376): [ -7687536 24381 -28516336 24381 0 0 0]
arr_owner (at 104719884827744): [ -7779696 24381 -28516336 24381 0 0 0]
Your code looks like it has some confusion in it -- ctypes.POINTER()
creates a new ctypes pointer class, not a ctypes instance. Anyway, the easiest way to pass a NumPy array to ctypes code is to use the numpy.ndarray
's ctypes
attribute's data_as
method. Just make sure the underlying data is the right type first. For example:
import ctypes
import numpy
c_float_p = ctypes.POINTER(ctypes.c_float)
data = numpy.array([[0.1, 0.1], [0.2, 0.2], [0.3, 0.3]])
data = data.astype(numpy.float32)
data_p = data.ctypes.data_as(c_float_p)