pax_global_header00006660000000000000000000000064150140665000014507gustar00rootroot0000000000000052 comment=4576bc581d7e64044265df01c2851db3618d5dc0 cyarray-1.2/000077500000000000000000000000001501406650000130235ustar00rootroot00000000000000cyarray-1.2/.github/000077500000000000000000000000001501406650000143635ustar00rootroot00000000000000cyarray-1.2/.github/workflows/000077500000000000000000000000001501406650000164205ustar00rootroot00000000000000cyarray-1.2/.github/workflows/tests.yml000066400000000000000000000017701501406650000203120ustar00rootroot00000000000000name: Tests on: pull_request jobs: tests: strategy: matrix: os: [ubuntu-latest, windows-latest, macos-latest] python-version: ['3.9', '3.10', '3.11'] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v4 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m pip install --upgrade pip setuptools wheel pip install .[tests] -v pip list - name: Run tests run: pytest -v --pyargs cyarray --benchmark-save benchmark-stats --benchmark-json benchmark-full.json --benchmark-histogram - name: 'Upload Perf Data' uses: actions/upload-artifact@v4 with: name: perf-bench-result-${{ matrix.python-version }}-${{ matrix.os }} path: | benchmark_*.svg .benchmarks/* benchmark-full.json cyarray-1.2/.gitignore000066400000000000000000000001101501406650000150030ustar00rootroot00000000000000*.pyc *.o *.c *.cpp *~ *.so build/ dist/ cyarray.egg-info/ .pytest_cachecyarray-1.2/.travis.yml000066400000000000000000000003421501406650000151330ustar00rootroot00000000000000language: python python: - "3.6" - "3.7" - "3.8" # command to install dependencies install: - pip install -r requirements.txt - python setup.py install # command to run tests script: - pytest -v --pyargs cyarray cyarray-1.2/LICENSE.txt000066400000000000000000000031111501406650000146420ustar00rootroot00000000000000Unless otherwise specified by LICENSE.txt files in individual directories, all code is Copyright (c) 2009-2018, the PySPH developers All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. cyarray-1.2/MANIFEST.in000066400000000000000000000002321501406650000145560ustar00rootroot00000000000000include MANIFEST.in Makefile *.py *.rst *.yml *.toml recursive-exclude cyarray *.cpp recursive-include cyarray *.pxd *.h *.mako recursive-include docs *.*cyarray-1.2/README.rst000066400000000000000000000036731501406650000145230ustar00rootroot00000000000000cyarray: a typed, re-sizable Cython array ------------------------------------------ |CI Status| |Documentation Status| .. |CI Status| image:: https://github.com/pypr/cyarray/actions/workflows/tests.yml/badge.svg :target: https://github.com/pypr/cyarray/actions/workflows/tests.yml .. |Documentation Status| image:: https://readthedocs.org/projects/cyarray/badge/?version=latest :target: https://cyarray.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status The cyarray package provides a fast, typed, re-sizable, Cython array. It currently provides the following arrays: ``IntArray, UIntArray, LongArray, FloatArray, DoubleArray``. All arrays provide for the following operations: - access by indexing. - access through get/set function. - resizing the array. - appending values at the end of the array. - reserving space for future appends. - access to internal data through a numpy array. If you are writing Cython code this is a convenient array to use as it exposes the raw underlying pointer to the data. For example if you use a ``FloatArray`` and access its ``data`` attribute it will be a ``float*``. Each array also provides an interface to its data through a numpy array. This is done through the ``get_npy_array`` function. The returned numpy array can be used just like any other numpy array but for the following restrictions: - the array may not be resized. - references of this array should not be kept. - slices of this array may not be made. The numpy array may however be copied and used in any manner. Installation ------------ cyarray can be installed using pip_:: $ pip install cyarray The package requires ``Cython``, ``numpy``, and ``mako`` to be installed and also requires a suitably configured C/C++ compiler. .. _pip: http://www.pip-installer.org Usage ----- In Python one may import and use the package as:: from cyarray.api import IntArray a = IntArray(10) Here ``a`` is an array of 10 integers. cyarray-1.2/cyarray/000077500000000000000000000000001501406650000144755ustar00rootroot00000000000000cyarray-1.2/cyarray/__init__.py000066400000000000000000000000241501406650000166020ustar00rootroot00000000000000__version__ = '1.2' cyarray-1.2/cyarray/api.py000066400000000000000000000001141501406650000156140ustar00rootroot00000000000000from .carray import IntArray, UIntArray, LongArray, FloatArray, DoubleArray cyarray-1.2/cyarray/carray.pxd000066400000000000000000000172041501406650000164770ustar00rootroot00000000000000# This file (carray.pxd) has been generated automatically. # DO NOT modify this file # To make changes modify the source templates (carray.pxd.mako) and regenerate #cython: language_level=3 """ Implementation of resizeable arrays of different types in Cython. Declaration File. """ # numpy import cimport numpy as np cdef long aligned(long n, int item_size) noexcept nogil cdef void* aligned_malloc(size_t bytes) noexcept nogil cdef void* aligned_realloc(void* existing, size_t bytes, size_t old_size) noexcept nogil cdef void aligned_free(void* p) noexcept nogil # forward declaration cdef class BaseArray cdef class LongArray(BaseArray) cdef class BaseArrayIter: cdef BaseArray arr cdef long i cdef class BaseArray: """Base class for managed C-arrays.""" cdef public long length, alloc cdef np.ndarray _npy_array cdef void c_align_array(self, LongArray new_indices, int stride=*) noexcept nogil cdef void c_reserve(self, long size) noexcept nogil cdef void c_reset(self) noexcept nogil cdef void c_resize(self, long size) noexcept nogil cdef void c_squeeze(self) noexcept nogil cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef align_array(self, LongArray new_indices, int stride=*) cpdef str get_c_type(self) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=*, int start=*) cpdef copy_subset(self, BaseArray source, long start_index=*, long end_index=*, int stride=*) cpdef update_min_max(self) # ########################################################################### # `IntArray` class. # ########################################################################### cdef class IntArray(BaseArray): """This class defines a managed array of ints. """ cdef int *data cdef int *_old_data cdef public int minimum, maximum cdef IntArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) noexcept nogil cdef void c_append(self, int value) noexcept nogil cdef void c_set_view(self, int *array, long length) noexcept nogil cdef int* get_data_ptr(self) cpdef int get(self, long idx) cpdef set(self, long idx, int value) cpdef append(self, int value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, IntArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, int value) # ########################################################################### # `UIntArray` class. # ########################################################################### cdef class UIntArray(BaseArray): """This class defines a managed array of unsigned ints. """ cdef unsigned int *data cdef unsigned int *_old_data cdef public unsigned int minimum, maximum cdef UIntArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) noexcept nogil cdef void c_append(self, unsigned int value) noexcept nogil cdef void c_set_view(self, unsigned int *array, long length) noexcept nogil cdef unsigned int* get_data_ptr(self) cpdef unsigned int get(self, long idx) cpdef set(self, long idx, unsigned int value) cpdef append(self, unsigned int value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, UIntArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, unsigned int value) # ########################################################################### # `LongArray` class. # ########################################################################### cdef class LongArray(BaseArray): """This class defines a managed array of longs. """ cdef long *data cdef long *_old_data cdef public long minimum, maximum cdef LongArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) noexcept nogil cdef void c_append(self, long value) noexcept nogil cdef void c_set_view(self, long *array, long length) noexcept nogil cdef long* get_data_ptr(self) cpdef long get(self, long idx) cpdef set(self, long idx, long value) cpdef append(self, long value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, LongArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, long value) # ########################################################################### # `FloatArray` class. # ########################################################################### cdef class FloatArray(BaseArray): """This class defines a managed array of floats. """ cdef float *data cdef float *_old_data cdef public float minimum, maximum cdef FloatArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) noexcept nogil cdef void c_append(self, float value) noexcept nogil cdef void c_set_view(self, float *array, long length) noexcept nogil cdef float* get_data_ptr(self) cpdef float get(self, long idx) cpdef set(self, long idx, float value) cpdef append(self, float value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, FloatArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, float value) # ########################################################################### # `DoubleArray` class. # ########################################################################### cdef class DoubleArray(BaseArray): """This class defines a managed array of doubles. """ cdef double *data cdef double *_old_data cdef public double minimum, maximum cdef DoubleArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) noexcept nogil cdef void c_append(self, double value) noexcept nogil cdef void c_set_view(self, double *array, long length) noexcept nogil cdef double* get_data_ptr(self) cpdef double get(self, long idx) cpdef set(self, long idx, double value) cpdef append(self, double value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, DoubleArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, double value) cyarray-1.2/cyarray/carray.pxd.mako000066400000000000000000000064361501406650000174320ustar00rootroot00000000000000<% type_info = [ ('int', 'IntArray', 'NPY_INT'), ('unsigned int', 'UIntArray', 'NPY_UINT'), ('long', 'LongArray', 'NPY_LONG'), ('float', 'FloatArray', 'NPY_FLOAT'), ('double', 'DoubleArray', 'NPY_DOUBLE'), ] %># This file (carray.pxd) has been generated automatically. # DO NOT modify this file # To make changes modify the source templates (carray.pxd.mako) and regenerate #cython: language_level=3 """ Implementation of resizeable arrays of different types in Cython. Declaration File. """ # numpy import cimport numpy as np cdef long aligned(long n, int item_size) noexcept nogil cdef void* aligned_malloc(size_t bytes) noexcept nogil cdef void* aligned_realloc(void* existing, size_t bytes, size_t old_size) noexcept nogil cdef void aligned_free(void* p) noexcept nogil # forward declaration cdef class BaseArray cdef class LongArray(BaseArray) cdef class BaseArrayIter: cdef BaseArray arr cdef long i cdef class BaseArray: """Base class for managed C-arrays.""" cdef public long length, alloc cdef np.ndarray _npy_array cdef void c_align_array(self, LongArray new_indices, int stride=*) noexcept nogil cdef void c_reserve(self, long size) noexcept nogil cdef void c_reset(self) noexcept nogil cdef void c_resize(self, long size) noexcept nogil cdef void c_squeeze(self) noexcept nogil cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef align_array(self, LongArray new_indices, int stride=*) cpdef str get_c_type(self) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=*, int start=*) cpdef copy_subset(self, BaseArray source, long start_index=*, long end_index=*, int stride=*) cpdef update_min_max(self) % for ARRAY_TYPE, CLASSNAME, NUMPY_TYPENAME in type_info: # ########################################################################### # `${CLASSNAME}` class. # ########################################################################### cdef class ${CLASSNAME}(BaseArray): """This class defines a managed array of ${ARRAY_TYPE}s. """ cdef ${ARRAY_TYPE} *data cdef ${ARRAY_TYPE} *_old_data cdef public ${ARRAY_TYPE} minimum, maximum cdef ${CLASSNAME} _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) noexcept nogil cdef void c_append(self, ${ARRAY_TYPE} value) noexcept nogil cdef void c_set_view(self, ${ARRAY_TYPE} *array, long length) noexcept nogil cdef ${ARRAY_TYPE}* get_data_ptr(self) cpdef ${ARRAY_TYPE} get(self, long idx) cpdef set(self, long idx, ${ARRAY_TYPE} value) cpdef append(self, ${ARRAY_TYPE} value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, ${CLASSNAME}, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, ${ARRAY_TYPE} value) % endfor cyarray-1.2/cyarray/carray.pyx000066400000000000000000002506471501406650000165360ustar00rootroot00000000000000# This file (carray.pxd) has been generated automatically. # DO NOT modify this file # To make changes modify the source templates (carray.pxd.mako) and regenerate # distutils: language=c++ # cython: embedsignature=True, language_level=3 # distutils: define_macros=NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION """ Implementation of resizeable arrays of different types in Cython. All arrays provide for the following operations: - access by indexing. - access through get/set function. - appending values at the end of the array. - reserving space for future appends. - access to internal data through a numpy array. Each array also provides an interface to its data through a numpy array. This is done through the ``get_npy_array`` function. The returned numpy array can be used just like any other numpy array but for the following restrictions: - the array may not be resized. - references of this array should not be kept. - slices of this array may not be made. The numpy array may however be copied and used in any manner. """ # For malloc etc. from libc.stdlib cimport * from libc.stdint cimport uintptr_t cimport numpy as np np.import_array() import numpy as np # logging imports import logging logger = logging.getLogger() # 'importing' some Numpy C-api functions. cdef extern from "numpy/arrayobject.h": cdef void _import_array() ctypedef struct PyArrayObject: char *data np.npy_intp *dimensions cdef enum NPY_TYPES: NPY_INT, NPY_UINT, NPY_LONG, NPY_FLOAT, NPY_DOUBLE np.ndarray PyArray_SimpleNewFromData(int, np.npy_intp*, int, void*) void *PyArray_DATA(np.ndarray arr) noexcept nogil np.npy_intp *PyArray_DIMS(np.ndarray arr) noexcept nogil # memcpy cdef extern from "stdlib.h": void *memcpy(void *dst, void *src, long n) noexcept nogil # Cython>= 3.0 now disallows replacing the guts of a numpy array. # Workaround: https://github.com/rainwoodman/pandas/blob/05d3fe2402e4563124e7060837ded7513ab5bca7/pandas/_libs/reduction.pyx#L27 # noqa: E501 cdef extern from *: """ static void PyArray_SET_DATA(PyArrayObject *arr, char * data) { ((PyArrayObject_fields *)arr)->data = data; } """ void PyArray_SET_DATA(np.ndarray arr, char * data) noexcept nogil # numpy module initialization call _import_array() cdef inline long aligned(long n, int item_size) noexcept nogil: """Align `n` items each having size (in bytes) `item_size` to 64 bytes and return the appropriate number of items that would be aligned to 64 bytes. """ if n*item_size%64 == 0: return n else: if 64%item_size == 0: return (n*item_size//64 + 1)*64//item_size else: return (n*item_size//64 + 1)*64 cpdef long py_aligned(long n, int item_size): """Align `n` items each having size (in bytes) `item_size` to 64 bits and return the appropriate number of items that would be aligned to 64 bytes. """ return aligned(n, item_size) cdef void* _aligned_malloc(size_t bytes) noexcept nogil: """Allocates block of memory starting on a cache line. Algorithm from: http://www.drdobbs.com/parallel/understanding-and-avoiding-memory-issues/212400410 """ cdef size_t cache_size = 64 cdef char* base = malloc(cache_size + bytes) # Round pointer up to next line cdef char* result = ((base+cache_size)&-(cache_size)) # Record where block actually starts. (result)[-1] = base return result cdef void* _aligned_realloc(void *existing, size_t bytes, size_t old_size) noexcept nogil: """Allocates block of memory starting on a cache line. """ cdef void* result = _aligned_malloc(bytes) cdef size_t copy_size = min(bytes, old_size) # Copy everything from the old to the new and free the old. memcpy(result, existing, copy_size) aligned_free(existing) return result cdef void* _deref_base(void* ptr) noexcept nogil: cdef size_t cache_size = 64 # Recover where block actually starts cdef char* base = (ptr)[-1] if ((base+cache_size)&-(cache_size)) != ptr: with gil: raise MemoryError("Passed pointer is not aligned.") return base cdef void* aligned_malloc(size_t bytes) noexcept nogil: return _aligned_malloc(bytes) cdef void* aligned_realloc(void* p, size_t bytes, size_t old_size) noexcept nogil: return _aligned_realloc(p, bytes, old_size) cdef void aligned_free(void* p) noexcept nogil: """Free block allocated by alligned_malloc. """ free(_deref_base(p)) cdef class BaseArray: """Base class for managed C-arrays. """ cdef void c_align_array(self, LongArray new_indices, int stride=1) noexcept nogil: """Rearrange the array contents according to the new indices. """ pass cdef void c_reserve(self, long size) noexcept nogil: pass cdef void c_reset(self) noexcept nogil: cdef np.npy_intp *dims self.length = 0 dims = PyArray_DIMS(self._npy_array) dims[0] = self.length cdef void c_resize(self, long size) noexcept nogil: pass cdef void c_squeeze(self) noexcept nogil: pass cpdef str get_c_type(self): """Return the c data type of this array. """ raise NotImplementedError('BaseArray::get_c_type') cpdef reserve(self, long size): """Resizes the internal data to required size. """ raise NotImplementedError('BaseArray::reserve') cpdef resize(self, long size): """Resizes the array to the new size. """ raise NotImplementedError('BaseArray::resize') cpdef np.ndarray get_npy_array(self): """Returns a numpy array of the data: do not keep its reference. """ return self._npy_array cpdef set_data(self, np.ndarray nparr): """Set data from the given numpy array. If the numpy array is a reference to the numpy array maintained internally by this class, nothing is done. Otherwise, if the size of nparr matches this array, values are copied into the array maintained. """ cdef PyArrayObject* sarr = nparr cdef PyArrayObject* darr = self._npy_array cdef np.npy_intp *s_dims, *d_dims cdef void *s_data, *d_data s_data = PyArray_DATA(nparr) d_data = PyArray_DATA(self._npy_array) s_dims = PyArray_DIMS(nparr) d_dims = PyArray_DIMS(self._npy_array) if s_data == d_data: return elif s_dims[0] <= d_dims[0]: self._npy_array[:s_dims[0]] = nparr else: raise ValueError('array size mismatch') cpdef squeeze(self): """Release any unused memory. """ raise NotImplementedError('BaseArray::squeeze') cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. """ raise NotImplementedError('BaseArray::remove') cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. """ raise NotImplementedError('BaseArray::extend') cpdef align_array(self, LongArray new_indices, int stride=1): """Rearrange the array contents according to the new indices. """ if new_indices.length != self.length//stride: raise ValueError('Unequal array lengths') self.c_align_array(new_indices, stride) cpdef reset(self): """Reset the length of the array to 0. """ raise NotImplementedError('BaseArray::reset') cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copy values of indexed particles from self to dest. """ raise NotImplementedError('BaseArray::copy_values') cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy subset of values from source to self. """ raise NotImplementedError('BaseArray::copy_subset') cpdef update_min_max(self): """Update the min and max values of the array. """ raise NotImplementedError('BaseArray::update_min_max') def __len__(self): return self.length def __iter__(self): """ Support the iteration protocol""" return BaseArrayIter(self) cdef class BaseArrayIter: """ Iteration object to support iteration over BaseArray. """ def __init__(self, BaseArray arr): self.arr = arr self.i = -1 def __next__(self): self.i = self.i+1 if self.i < self.arr.length: return self.arr[self.i] else: raise StopIteration def __iter__(self): return self # ########################################################################### # `IntArray` class. # ########################################################################### cdef class IntArray(BaseArray): """Represents an array of `ints` Mallocs a memory buffer of size (n*sizeof(int)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = IntArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = IntArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef int *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(int)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, int value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, int value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, int value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (IntArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_INT, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) noexcept nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef int *temp n_bytes = sizeof(int)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, int value) noexcept nogil: cdef long l = self.length cdef np.npy_intp *arr_dims if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_reserve(self, long size) noexcept nogil: cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(int), self.alloc*sizeof(int) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cdef void c_reset(self) noexcept nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL PyArray_SET_DATA(self._npy_array, self.data) cdef void c_resize(self, long size) noexcept nogil: cdef np.npy_intp *arr_dims # reserve memory self.c_reserve(size) # update the lengths self.length = size arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_set_view(self, int *array, long length) noexcept nogil: """Create a view of a given raw data pointer with given length. """ cdef np.npy_intp *arr_dims if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_squeeze(self) noexcept nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(int), self.alloc*sizeof(int) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'int' cdef int* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef int get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, int value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, int value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(int)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(int)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, IntArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : IntArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ cdef np.npy_intp *arr_dims if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef np.npy_intp *arr_dims if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef IntArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef IntArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef int min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val # ########################################################################### # `UIntArray` class. # ########################################################################### cdef class UIntArray(BaseArray): """Represents an array of `unsigned ints` Mallocs a memory buffer of size (n*sizeof(unsigned int)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = UIntArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = UIntArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef unsigned int *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(unsigned int)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, unsigned int value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, unsigned int value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, unsigned int value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (UIntArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_UINT, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) noexcept nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef unsigned int *temp n_bytes = sizeof(unsigned int)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, unsigned int value) noexcept nogil: cdef long l = self.length cdef np.npy_intp *arr_dims if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_reserve(self, long size) noexcept nogil: cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(unsigned int), self.alloc*sizeof(unsigned int) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cdef void c_reset(self) noexcept nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL PyArray_SET_DATA(self._npy_array, self.data) cdef void c_resize(self, long size) noexcept nogil: cdef np.npy_intp *arr_dims # reserve memory self.c_reserve(size) # update the lengths self.length = size arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_set_view(self, unsigned int *array, long length) noexcept nogil: """Create a view of a given raw data pointer with given length. """ cdef np.npy_intp *arr_dims if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_squeeze(self) noexcept nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(unsigned int), self.alloc*sizeof(unsigned int) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'unsigned int' cdef unsigned int* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef unsigned int get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, unsigned int value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, unsigned int value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(unsigned int)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(unsigned int)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, UIntArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : UIntArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ cdef np.npy_intp *arr_dims if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef np.npy_intp *arr_dims if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef UIntArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef UIntArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef unsigned int min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val # ########################################################################### # `LongArray` class. # ########################################################################### cdef class LongArray(BaseArray): """Represents an array of `longs` Mallocs a memory buffer of size (n*sizeof(long)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = LongArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = LongArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef long *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(long)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, long value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, long value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, long value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (LongArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_LONG, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) noexcept nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef long *temp n_bytes = sizeof(long)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, long value) noexcept nogil: cdef long l = self.length cdef np.npy_intp *arr_dims if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_reserve(self, long size) noexcept nogil: cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(long), self.alloc*sizeof(long) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cdef void c_reset(self) noexcept nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL PyArray_SET_DATA(self._npy_array, self.data) cdef void c_resize(self, long size) noexcept nogil: cdef np.npy_intp *arr_dims # reserve memory self.c_reserve(size) # update the lengths self.length = size arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_set_view(self, long *array, long length) noexcept nogil: """Create a view of a given raw data pointer with given length. """ cdef np.npy_intp *arr_dims if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_squeeze(self) noexcept nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(long), self.alloc*sizeof(long) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'long' cdef long* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef long get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, long value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, long value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(long)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(long)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, LongArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : LongArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ cdef np.npy_intp *arr_dims if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef np.npy_intp *arr_dims if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef LongArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef LongArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef long min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val # ########################################################################### # `FloatArray` class. # ########################################################################### cdef class FloatArray(BaseArray): """Represents an array of `floats` Mallocs a memory buffer of size (n*sizeof(float)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = FloatArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = FloatArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef float *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(float)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, float value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, float value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, float value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (FloatArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_FLOAT, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) noexcept nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef float *temp n_bytes = sizeof(float)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, float value) noexcept nogil: cdef long l = self.length cdef np.npy_intp *arr_dims if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_reserve(self, long size) noexcept nogil: cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(float), self.alloc*sizeof(float) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cdef void c_reset(self) noexcept nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL PyArray_SET_DATA(self._npy_array, self.data) cdef void c_resize(self, long size) noexcept nogil: cdef np.npy_intp *arr_dims # reserve memory self.c_reserve(size) # update the lengths self.length = size arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_set_view(self, float *array, long length) noexcept nogil: """Create a view of a given raw data pointer with given length. """ cdef np.npy_intp *arr_dims if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_squeeze(self) noexcept nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(float), self.alloc*sizeof(float) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'float' cdef float* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef float get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, float value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, float value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(float)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(float)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, FloatArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : FloatArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ cdef np.npy_intp *arr_dims if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef np.npy_intp *arr_dims if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef FloatArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef FloatArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef float min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val # ########################################################################### # `DoubleArray` class. # ########################################################################### cdef class DoubleArray(BaseArray): """Represents an array of `doubles` Mallocs a memory buffer of size (n*sizeof(double)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = DoubleArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = DoubleArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef double *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(double)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, double value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, double value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, double value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (DoubleArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_DOUBLE, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) noexcept nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef double *temp n_bytes = sizeof(double)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, double value) noexcept nogil: cdef long l = self.length cdef np.npy_intp *arr_dims if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_reserve(self, long size) noexcept nogil: cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(double), self.alloc*sizeof(double) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cdef void c_reset(self) noexcept nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL PyArray_SET_DATA(self._npy_array, self.data) cdef void c_resize(self, long size) noexcept nogil: cdef np.npy_intp *arr_dims # reserve memory self.c_reserve(size) # update the lengths self.length = size arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_set_view(self, double *array, long length) noexcept nogil: """Create a view of a given raw data pointer with given length. """ cdef np.npy_intp *arr_dims if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_squeeze(self) noexcept nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(double), self.alloc*sizeof(double) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'double' cdef double* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef double get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, double value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, double value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(double)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(double)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, DoubleArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : DoubleArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ cdef np.npy_intp *arr_dims if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef np.npy_intp *arr_dims if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef DoubleArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef DoubleArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef double min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val cyarray-1.2/cyarray/carray.pyx.mako000066400000000000000000000613771501406650000174640ustar00rootroot00000000000000<% import platform type_info = [ ('int', 'IntArray', 'NPY_INT'), ('unsigned int', 'UIntArray', 'NPY_UINT'), ('long', 'LongArray', 'NPY_LONG'), ('float', 'FloatArray', 'NPY_FLOAT'), ('double', 'DoubleArray', 'NPY_DOUBLE'), ] %># This file (carray.pxd) has been generated automatically. # DO NOT modify this file # To make changes modify the source templates (carray.pxd.mako) and regenerate # distutils: language=c++ # cython: embedsignature=True, language_level=3 # distutils: define_macros=NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION """ Implementation of resizeable arrays of different types in Cython. All arrays provide for the following operations: - access by indexing. - access through get/set function. - appending values at the end of the array. - reserving space for future appends. - access to internal data through a numpy array. Each array also provides an interface to its data through a numpy array. This is done through the ``get_npy_array`` function. The returned numpy array can be used just like any other numpy array but for the following restrictions: - the array may not be resized. - references of this array should not be kept. - slices of this array may not be made. The numpy array may however be copied and used in any manner. """ # For malloc etc. from libc.stdlib cimport * from libc.stdint cimport uintptr_t cimport numpy as np np.import_array() import numpy as np # logging imports import logging logger = logging.getLogger() # 'importing' some Numpy C-api functions. cdef extern from "numpy/arrayobject.h": cdef void _import_array() ctypedef struct PyArrayObject: char *data np.npy_intp *dimensions cdef enum NPY_TYPES: NPY_INT, NPY_UINT, NPY_LONG, NPY_FLOAT, NPY_DOUBLE np.ndarray PyArray_SimpleNewFromData(int, np.npy_intp*, int, void*) void *PyArray_DATA(np.ndarray arr) noexcept nogil np.npy_intp *PyArray_DIMS(np.ndarray arr) noexcept nogil # memcpy cdef extern from "stdlib.h": void *memcpy(void *dst, void *src, long n) noexcept nogil # Cython>= 3.0 now disallows replacing the guts of a numpy array. # Workaround: https://github.com/rainwoodman/pandas/blob/05d3fe2402e4563124e7060837ded7513ab5bca7/pandas/_libs/reduction.pyx#L27 # noqa: E501 cdef extern from *: """ static void PyArray_SET_DATA(PyArrayObject *arr, char * data) { ((PyArrayObject_fields *)arr)->data = data; } """ void PyArray_SET_DATA(np.ndarray arr, char * data) noexcept nogil # numpy module initialization call _import_array() cdef inline long aligned(long n, int item_size) noexcept nogil: """Align `n` items each having size (in bytes) `item_size` to 64 bytes and return the appropriate number of items that would be aligned to 64 bytes. """ if n*item_size%64 == 0: return n else: if 64%item_size == 0: return (n*item_size//64 + 1)*64//item_size else: return (n*item_size//64 + 1)*64 cpdef long py_aligned(long n, int item_size): """Align `n` items each having size (in bytes) `item_size` to 64 bits and return the appropriate number of items that would be aligned to 64 bytes. """ return aligned(n, item_size) cdef void* _aligned_malloc(size_t bytes) noexcept nogil: """Allocates block of memory starting on a cache line. Algorithm from: http://www.drdobbs.com/parallel/understanding-and-avoiding-memory-issues/212400410 """ cdef size_t cache_size = 64 cdef char* base = malloc(cache_size + bytes) # Round pointer up to next line cdef char* result = ((base+cache_size)&-(cache_size)) # Record where block actually starts. (result)[-1] = base return result cdef void* _aligned_realloc(void *existing, size_t bytes, size_t old_size) noexcept nogil: """Allocates block of memory starting on a cache line. """ cdef void* result = _aligned_malloc(bytes) cdef size_t copy_size = min(bytes, old_size) # Copy everything from the old to the new and free the old. memcpy(result, existing, copy_size) aligned_free(existing) return result cdef void* _deref_base(void* ptr) noexcept nogil: cdef size_t cache_size = 64 # Recover where block actually starts cdef char* base = (ptr)[-1] if ((base+cache_size)&-(cache_size)) != ptr: with gil: raise MemoryError("Passed pointer is not aligned.") return base cdef void* aligned_malloc(size_t bytes) noexcept nogil: return _aligned_malloc(bytes) cdef void* aligned_realloc(void* p, size_t bytes, size_t old_size) noexcept nogil: return _aligned_realloc(p, bytes, old_size) cdef void aligned_free(void* p) noexcept nogil: """Free block allocated by alligned_malloc. """ free(_deref_base(p)) cdef class BaseArray: """Base class for managed C-arrays. """ #### Cython interface ################################################# cdef void c_align_array(self, LongArray new_indices, int stride=1) noexcept nogil: """Rearrange the array contents according to the new indices. """ pass cdef void c_reserve(self, long size) noexcept nogil: pass cdef void c_reset(self) noexcept nogil: cdef np.npy_intp *dims self.length = 0 dims = PyArray_DIMS(self._npy_array) dims[0] = self.length cdef void c_resize(self, long size) noexcept nogil: pass cdef void c_squeeze(self) noexcept nogil: pass #### Python interface ################################################# cpdef str get_c_type(self): """Return the c data type of this array. """ raise NotImplementedError('BaseArray::get_c_type') cpdef reserve(self, long size): """Resizes the internal data to required size. """ raise NotImplementedError('BaseArray::reserve') cpdef resize(self, long size): """Resizes the array to the new size. """ raise NotImplementedError('BaseArray::resize') cpdef np.ndarray get_npy_array(self): """Returns a numpy array of the data: do not keep its reference. """ return self._npy_array cpdef set_data(self, np.ndarray nparr): """Set data from the given numpy array. If the numpy array is a reference to the numpy array maintained internally by this class, nothing is done. Otherwise, if the size of nparr matches this array, values are copied into the array maintained. """ cdef PyArrayObject* sarr = nparr cdef PyArrayObject* darr = self._npy_array cdef np.npy_intp *s_dims, *d_dims cdef void *s_data, *d_data s_data = PyArray_DATA(nparr) d_data = PyArray_DATA(self._npy_array) s_dims = PyArray_DIMS(nparr) d_dims = PyArray_DIMS(self._npy_array) if s_data == d_data: return elif s_dims[0] <= d_dims[0]: self._npy_array[:s_dims[0]] = nparr else: raise ValueError('array size mismatch') cpdef squeeze(self): """Release any unused memory. """ raise NotImplementedError('BaseArray::squeeze') cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. """ raise NotImplementedError('BaseArray::remove') cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. """ raise NotImplementedError('BaseArray::extend') cpdef align_array(self, LongArray new_indices, int stride=1): """Rearrange the array contents according to the new indices. """ if new_indices.length != self.length//stride: raise ValueError('Unequal array lengths') self.c_align_array(new_indices, stride) cpdef reset(self): """Reset the length of the array to 0. """ raise NotImplementedError('BaseArray::reset') cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copy values of indexed particles from self to dest. """ raise NotImplementedError('BaseArray::copy_values') cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy subset of values from source to self. """ raise NotImplementedError('BaseArray::copy_subset') cpdef update_min_max(self): """Update the min and max values of the array. """ raise NotImplementedError('BaseArray::update_min_max') def __len__(self): return self.length def __iter__(self): """ Support the iteration protocol""" return BaseArrayIter(self) cdef class BaseArrayIter: """ Iteration object to support iteration over BaseArray. """ def __init__(self, BaseArray arr): self.arr = arr self.i = -1 def __next__(self): self.i = self.i+1 if self.i < self.arr.length: return self.arr[self.i] else: raise StopIteration def __iter__(self): return self % for ARRAY_TYPE, CLASSNAME, NUMPY_TYPENAME in type_info: # ########################################################################### # `${CLASSNAME}` class. # ########################################################################### cdef class ${CLASSNAME}(BaseArray): """Represents an array of `${ARRAY_TYPE}s` Mallocs a memory buffer of size (n*sizeof(${ARRAY_TYPE})) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = ${CLASSNAME}() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = ${CLASSNAME}(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef ${ARRAY_TYPE} *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = <${ARRAY_TYPE}*>aligned_malloc(n*sizeof(${ARRAY_TYPE})) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, ${ARRAY_TYPE} value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, ${ARRAY_TYPE} value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, ${ARRAY_TYPE} value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (${CLASSNAME}, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, ${NUMPY_TYPENAME}, self.data ) ##### Cython protocol ###################################### cdef void c_align_array(self, LongArray new_indices, int stride=1) noexcept nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef ${ARRAY_TYPE} *temp n_bytes = sizeof(${ARRAY_TYPE})*length temp = <${ARRAY_TYPE}*>aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, ${ARRAY_TYPE} value) noexcept nogil: cdef long l = self.length cdef np.npy_intp *arr_dims if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_reserve(self, long size) noexcept nogil: cdef void* data = NULL if size > self.alloc: data = <${ARRAY_TYPE}*>aligned_realloc( self.data, size*sizeof(${ARRAY_TYPE}), self.alloc*sizeof(${ARRAY_TYPE}) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = <${ARRAY_TYPE}*>data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) cdef void c_reset(self) noexcept nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL PyArray_SET_DATA(self._npy_array, self.data) cdef void c_resize(self, long size) noexcept nogil: cdef np.npy_intp *arr_dims # reserve memory self.c_reserve(size) # update the lengths self.length = size arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_set_view(self, ${ARRAY_TYPE} *array, long length) noexcept nogil: """Create a view of a given raw data pointer with given length. """ cdef np.npy_intp *arr_dims if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cdef void c_squeeze(self) noexcept nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = <${ARRAY_TYPE}*>aligned_realloc( self.data, size*sizeof(${ARRAY_TYPE}), self.alloc*sizeof(${ARRAY_TYPE}) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = <${ARRAY_TYPE}*>data self.alloc = size PyArray_SET_DATA(self._npy_array, self.data) ##### Python protocol ###################################### cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return '${ARRAY_TYPE}' cdef ${ARRAY_TYPE}* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef ${ARRAY_TYPE} get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, ${ARRAY_TYPE} value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, ${ARRAY_TYPE} value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(${ARRAY_TYPE})`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(${ARRAY_TYPE})`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, ${CLASSNAME} parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : ${CLASSNAME} The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ cdef np.npy_intp *arr_dims if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start PyArray_SET_DATA(self._npy_array, self.data) arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef np.npy_intp *arr_dims if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr_dims = PyArray_DIMS(self._npy_array) arr_dims[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef ${CLASSNAME} dest_array = <${CLASSNAME}>dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef ${CLASSNAME} src = <${CLASSNAME}>source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef ${ARRAY_TYPE} min_val, max_val if self.length == 0: self.minimum = <${ARRAY_TYPE}>0 self.maximum = <${ARRAY_TYPE}>0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val % endfor cyarray-1.2/cyarray/generator.py000077500000000000000000000052611501406650000170440ustar00rootroot00000000000000#!/usr/bin/env python ''' Module to collect and generate source files from mako template files. When used to locate source files as a main program: The template files must have an extension '.mako'. The generated files have the name same as the mako file but with the '.mako' removed. Example: `carray.pyx.mako` is generated into `carray.pyx` ''' from __future__ import print_function import glob import os from os.path import abspath, dirname, exists, join import sys def is_modified_later(filename1, filename2): '''Return `True` if the file1 is modified later than file2''' return os.stat(filename1).st_mtime > os.stat(filename2).st_mtime def _inject_paths_in_sys_path(outfile): # Inject the directory of the output file into the path, # so that local imports will work. sys.path.insert(0, dirname(outfile)) # inject the path to pysph if pysph cannot be imported. try: import pysph except ImportError: sys.path.insert(0, dirname(dirname(dirname(__file__)))) def generate_files(dirname, if_modified=True): '''Generates source files from the template files with extension `.mako` If `if_modified` is True (default), the source file will be created only if the template has been modified later than the source ''' for filename in glob.glob(join(dirname, '*.mako')): outfile = abspath(filename[:-5]) message = 'generating file {outfile} from {filename}'.format( outfile=outfile, filename=filename ) if not exists(outfile) or \ (if_modified and is_modified_later(filename, outfile)): _inject_paths_in_sys_path(outfile) from mako.template import Template print(message) template = Template(filename=filename) with open(outfile, 'w') as fp: fp.write(template.render()) else: print('Not ' + message) def main(paths=None): '''Generates source files using mako template files. Parameters ----------- - paths: is a list of directories to convert. If None, all files in current file's directory are converted. ''' if not paths: generate_files(dirname(__file__)) else: for pth in paths: generate_files(pth) if __name__ == '__main__': import sys if '--help' in sys.argv or '-h' in sys.argv: print('usage:') print(' generator.py [filenames]') print() print(' Convert template files with extension .mako into ' 'source files') print(' If filenames is omitted all .mako files in current ' 'directory will be converted') else: main(sys.argv[1:]) cyarray-1.2/cyarray/tests/000077500000000000000000000000001501406650000156375ustar00rootroot00000000000000cyarray-1.2/cyarray/tests/__init__.py000066400000000000000000000000001501406650000177360ustar00rootroot00000000000000cyarray-1.2/cyarray/tests/test_carray.py000066400000000000000000000542461501406650000205440ustar00rootroot00000000000000"""Tests for the carray module. Only the LongArray is tested. As the code in carray.pyx is auto-generated, tests for one class hould suffice. """ # standard imports import unittest import numpy import pytest # local imports from cyarray.carray import LongArray, py_aligned class TestAligned(unittest.TestCase): def test_aligned_to_64_bits(self): self.assertEqual(py_aligned(12, 1), 64) self.assertEqual(py_aligned(1, 1), 64) self.assertEqual(py_aligned(64, 1), 64) self.assertEqual(py_aligned(120, 1), 128) self.assertEqual(py_aligned(1, 2), 32) self.assertEqual(py_aligned(12, 2), 32) self.assertEqual(py_aligned(32, 2), 32) self.assertEqual(py_aligned(33, 2), 64) self.assertEqual(py_aligned(1, 3), 64) self.assertEqual(py_aligned(65, 3), 256) self.assertEqual(py_aligned(1, 4), 16) self.assertEqual(py_aligned(16, 4), 16) self.assertEqual(py_aligned(21, 4), 32) self.assertEqual(py_aligned(1, 5), 64) self.assertEqual(py_aligned(13, 5), 128) self.assertEqual(py_aligned(1, 8), 8) self.assertEqual(py_aligned(8, 8), 8) self.assertEqual(py_aligned(11, 8), 16) class TestLongArray(unittest.TestCase): """ Tests for the LongArray class. """ def test_constructor(self): """ Test the constructor. """ la = LongArray(10) self.assertEqual(la.length, 10) self.assertEqual(la.alloc, 10) self.assertEqual(len(la.get_npy_array()), 10) la = LongArray() self.assertEqual(la.length, 0) self.assertEqual(la.alloc, 16) self.assertEqual(len(la.get_npy_array()), 0) def test_get_set_indexing(self): """ Test get/set and [] operator. """ la = LongArray(10) la.set(0, 10) la.set(9, 1) self.assertEqual(la.get(0), 10) self.assertEqual(la.get(9), 1) la[9] = 2 self.assertEqual(la[9], 2) def test_append(self): """ Test the append function. """ la = LongArray(0) la.append(1) la.append(2) la.append(3) self.assertEqual(la.length, 3) self.assertEqual(la[0], 1) self.assertEqual(la[1], 2) self.assertEqual(la[2], 3) def test_reserve(self): """ Tests the reserve function. """ la = LongArray(0) la.reserve(10) self.assertEqual(la.alloc, 16) self.assertEqual(la.length, 0) self.assertEqual(len(la.get_npy_array()), 0) la.reserve(20) self.assertEqual(la.alloc, 20) self.assertEqual(la.length, 0) self.assertEqual(len(la.get_npy_array()), 0) def test_resize(self): """ Tests the resize function. """ la = LongArray(0) la.resize(20) self.assertEqual(la.length, 20) self.assertEqual(len(la.get_npy_array()), 20) self.assertEqual(la.alloc >= la.length, True) def test_get_npy_array(self): """ Tests the get_npy_array array. """ la = LongArray(3) la[0] = 1 la[1] = 2 la[2] = 3 nparray = la.get_npy_array() self.assertEqual(len(nparray), 3) for i in range(3): self.assertEqual(nparray[0], la[0]) def test_set_data(self): """ Tests the set_data function. """ la = LongArray(5) np = numpy.arange(5) la.set_data(np) for i in range(5): self.assertEqual(la[i], np[i]) self.assertRaises(ValueError, la.set_data, numpy.arange(10)) def test_squeeze(self): la = LongArray(5) la.append(4) self.assertEqual(la.alloc > la.length, True) la.squeeze() self.assertEqual(la.length, 6) self.assertEqual(la.alloc >= la.length, True) self.assertEqual(len(la.get_npy_array()), 6) def test_squeeze_for_zero_length_array(self): # Given. la = LongArray() # When la.squeeze() # Then self.assertEqual(la.length, 0) self.assertEqual(len(la.get_npy_array()), 0) self.assertEqual(la.alloc >= la.length, True) del la # This should work and not segfault. def test_squeeze_large_array_should_not_segfault(self): # Given la = LongArray(10) la.set_data(numpy.zeros(10, dtype=int)) la.reserve(100000) # When la.squeeze() la.reserve(1000) # Then self.assertEqual(la.length, 10) numpy.testing.assert_array_almost_equal(la.get_npy_array(), 0) self.assertEqual(la.alloc >= la.length, True) def test_reset(self): """ Tests the reset function. """ la = LongArray(5) la.reset() self.assertEqual(la.length, 0) self.assertEqual(la.alloc, 5) self.assertEqual(len(la.get_npy_array()), 0) def test_extend(self): """ Tests the extend function. """ l1 = LongArray(5) for i in range(5): l1[i] = i l2 = LongArray(5) for i in range(5): l2[i] = 5 + i l1.extend(l2.get_npy_array()) self.assertEqual(l1.length, 10) self.assertEqual( numpy.allclose( l1.get_npy_array(), numpy.arange(10)), True) def test_remove(self): l1 = LongArray(10) l1.set_data(numpy.arange(10)) rem = [0, 4, 3] l1.remove(numpy.array(rem, dtype=int)) self.assertEqual(l1.length, 7) self.assertEqual(numpy.allclose([7, 1, 2, 8, 9, 5, 6], l1.get_npy_array()), True) l1.remove(numpy.array(rem, dtype=int)) self.assertEqual(l1.length, 4) self.assertEqual(numpy.allclose( [6, 1, 2, 5], l1.get_npy_array()), True) rem = [0, 1, 3] l1.remove(numpy.array(rem, dtype=int)) self.assertEqual(l1.length, 1) self.assertEqual(numpy.allclose([2], l1.get_npy_array()), True) l1.remove(numpy.array([0], dtype=int)) self.assertEqual(l1.length, 0) self.assertEqual(len(l1.get_npy_array()), 0) def test_remove_with_strides(self): # Given l1 = LongArray(12) l1.set_data(numpy.arange(12)) # When rem = [3, 1] l1.remove(numpy.array(rem, dtype=int), stride=3) # Then self.assertEqual(l1.length, 6) self.assertEqual(numpy.allclose([0, 1, 2, 6, 7, 8], l1.get_npy_array()), True) # Given l1 = LongArray(12) l1.set_data(numpy.arange(12)) # When rem = [0, 2] l1.remove(numpy.array(rem, dtype=int), stride=3) # Then self.assertEqual(l1.length, 6) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] self.assertEqual(numpy.allclose([9, 10, 11, 3, 4, 5], l1.get_npy_array()), True) def test_align_array(self): l1 = LongArray(10) l1.set_data(numpy.arange(10)) new_indices = LongArray(10) new_indices.set_data(numpy.asarray([1, 5, 3, 2, 4, 7, 8, 6, 9, 0])) l1.align_array(new_indices) self.assertEqual(numpy.allclose([1, 5, 3, 2, 4, 7, 8, 6, 9, 0], l1.get_npy_array()), True) # Test case with strides. l1 = LongArray(6) l1.set_data(numpy.arange(6)) new_indices = LongArray(3) new_indices.set_data(numpy.asarray([2, 1, 0])) l1.align_array(new_indices, 2) self.assertEqual(numpy.allclose([4, 5, 2, 3, 0, 1], l1.get_npy_array()), True) def test_copy_subset(self): l1 = LongArray(10) l1.set_data(numpy.arange(10)) l2 = LongArray(4) l2[0] = 4 l2[1] = 3 l2[2] = 2 l2[3] = 1 # a valid copy. l1.copy_subset(l2, 5, 9) self.assertEqual(numpy.allclose([0, 1, 2, 3, 4, 4, 3, 2, 1, 9], l1.get_npy_array()), True) # try to copy different sized arrays without any index specification. l1.set_data(numpy.arange(10)) # copy to the last k values of source array. l1.copy_subset(l2, start_index=6) self.assertEqual(numpy.allclose([0, 1, 2, 3, 4, 5, 4, 3, 2, 1], l1.get_npy_array()), True) l1.set_data(numpy.arange(10)) l1.copy_subset(l2, start_index=7) self.assertEqual(numpy.allclose([0, 1, 2, 3, 4, 5, 6, 4, 3, 2], l1.get_npy_array()), True) # some invalid operations. l1.set_data(numpy.arange(10)) self.assertRaises(ValueError, l1.copy_subset, l2, -1, 1) self.assertRaises(ValueError, l1.copy_subset, l2, 3, 2) self.assertRaises(ValueError, l1.copy_subset, l2, 0, 11) self.assertRaises(ValueError, l1.copy_subset, l2, 10, 20) self.assertRaises(ValueError, l1.copy_subset, l2, -1, -1) def test_copy_subset_works_with_strides(self): # Given l1 = LongArray(8) l1.set_data(numpy.arange(8)) l2 = LongArray(4) l2.set_data(numpy.arange(10, 14)) # When l1.copy_subset(l2, 2, 3, stride=2) # Then numpy.testing.assert_array_equal( l1.get_npy_array(), [0, 1, 2, 3, 10, 11, 6, 7] ) # When l1.copy_subset(l2, 2, 4, stride=2) # Then numpy.testing.assert_array_equal( l1.get_npy_array(), [0, 1, 2, 3, 10, 11, 12, 13] ) def test_copy_values(self): # Given l1 = LongArray(8) l1.set_data(numpy.arange(8)) l2 = LongArray(8) l2.set_data(numpy.zeros(8, dtype=int)) # When indices = LongArray(3) indices.set_data(numpy.array([2, 4, 6])) l1.copy_values(indices, l2) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [2, 4, 6] + [0] * 5 ) # When l2.set_data(numpy.zeros(8, dtype=int)) indices.set_data(numpy.array([1, 2, 3])) l1.copy_values(indices, l2, stride=2) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [2, 3, 4, 5, 6, 7, 0, 0] ) def test_copy_values_with_start_index(self): # Given l1 = LongArray(8) l1.set_data(numpy.arange(8)) l2 = LongArray(8) l2.set_data(numpy.zeros(8, dtype=int)) # When indices = LongArray(3) indices.set_data(numpy.array([2, 4, 6])) l1.copy_values(indices, l2, start=5) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [0] * 5 + [2, 4, 6] ) # When l2.set_data(numpy.zeros(8, dtype=int)) indices.set_data(numpy.array([1, 2, 3])) l1.copy_values(indices, l2, stride=2, start=2) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [0, 0, 2, 3, 4, 5, 6, 7] ) def test_update_min_max(self): """ Tests the update_min_max function. """ l1 = LongArray(10) l1.set_data(numpy.arange(10)) l1.update_min_max() self.assertEqual(l1.minimum, 0) self.assertEqual(l1.maximum, 9) l1[9] = -1 l1[0] = -20 l1[4] = 200 l1.update_min_max() self.assertEqual(l1.minimum, -20) self.assertEqual(l1.maximum, 200) def test_pickling(self): """ Tests the __reduce__ and __setstate__ functions. """ l1 = LongArray(10) l1.set_data(numpy.arange(10)) import pickle l1_dump = pickle.dumps(l1) l1_load = pickle.loads(l1_dump) self.assertEqual( (l1_load.get_npy_array() == l1.get_npy_array()).all(), True) def test_set_view(self): # Given src = LongArray() src.extend(numpy.arange(5)) # When. view = LongArray() view.set_view(src, 1, 4) # Then. self.assertEqual(view.length, 3) expect = list(range(1, 4)) self.assertListEqual(view.get_npy_array().tolist(), expect) def test_set_view_for_empty_array(self): # Given src = LongArray() src.extend(numpy.arange(5)) # When. view = LongArray() view.set_view(src, 1, 1) # Then. self.assertEqual(view.length, 0) expect = [] self.assertListEqual(view.get_npy_array().tolist(), expect) def test_set_view_stores_reference_to_parent(self): # Given src = LongArray() src.extend(numpy.arange(5)) # When view = LongArray() view.set_view(src, 1, 4) del src # Then. self.assertEqual(view.length, 3) expect = list(range(1, 4)) self.assertListEqual(view.get_npy_array().tolist(), expect) def test_reset_works_after_set_view(self): # Given src = LongArray() src.extend(numpy.arange(5)) view = LongArray() view.set_view(src, 1, 3) # When. view.reset() view.extend(numpy.arange(3) * 10) # Then. self.assertEqual(view.length, 3) expect = (numpy.arange(3) * 10).tolist() self.assertListEqual(view.get_npy_array().tolist(), expect) class BenchmarkLongArray(unittest.TestCase): """ Tests for the LongArray class. """ @pytest.fixture(autouse=True) def setupBenchmark(self, benchmark): self.benchmark = benchmark def test_constructor(self): """ Test the constructor. """ n = numpy.random.randint(low=10, high=100) la = self.benchmark(LongArray, n) self.assertEqual(la.length, n) self.assertEqual(la.alloc, n) self.assertEqual(len(la.get_npy_array()), n) def test_set_indexing(self): n = 100 lab = LongArray(n) self.benchmark.pedantic(lab.set, args=(9, n)) self.assertEqual(lab[9], n) def test_get_indexing(self): la = LongArray(100) la[98] = 15 res = self.benchmark(la.get, 98) self.assertEqual(res, 15) def test_append(self): lab = LongArray(0) n = 100 self.benchmark(lab.append, n) self.assertEqual(lab[0], n) def test_reserve(self): """ Tests the reserve function. """ def breserve(n): la = LongArray(0) la.reserve(n) return la n = 100 la = self.benchmark(breserve, n) self.assertEqual(la.alloc, n) def test_resize(self): """ Tests the resize function. """ def bresize(lab): n = numpy.random.randint(low=10, high=20) lab.resize(n) return lab, n la = LongArray(10) la, n = self.benchmark(bresize, la) self.assertEqual(la.length, n) def test_get_npy_array(self): la = LongArray(100) la[0] = 1 la[1] = 2 la[2] = 3 nparray = self.benchmark(la.get_npy_array) for i in range(3): self.assertEqual(nparray[0], la[0]) def test_set_data(self): """ Tests the set_data function. """ n = 50 la = LongArray(n) np = numpy.arange(n) self.benchmark(la.set_data, np) for i in range(n): self.assertEqual(la[i], np[i]) self.assertRaises(ValueError, la.set_data, numpy.arange(55)) def test_squeeze(self): def bsqueeze(): lab = LongArray(5) lab.append(4) lab.squeeze() return lab la = self.benchmark(bsqueeze) self.assertEqual(la.length, 6) self.assertEqual(la.alloc >= la.length, True) self.assertEqual(len(la.get_npy_array()), 6) def test_reset(self): def breset(): lab = LongArray(5) lab.reset() return lab la = self.benchmark(breset) self.assertEqual(la.length, 0) self.assertEqual(la.alloc, 5) self.assertEqual(len(la.get_npy_array()), 0) def test_extend(self): l2 = LongArray(5) for i in range(5): l2[i] = 5 + i def bextend(l2n): l1b = LongArray(0) l1b.extend(l2n.get_npy_array()) return l1b l1 = self.benchmark(bextend, l2) self.assertEqual(l1.length, 5) self.assertEqual( numpy.allclose( l1.get_npy_array(), numpy.arange(5, 10)), True) def test_remove(self): def bremove(rem): l1b = LongArray(10) l1b.set_data(numpy.arange(10)) l1b.remove(rem) return l1b rem = [0, 4, 3] l1 = self.benchmark(bremove, numpy.array(rem, dtype=int)) self.assertEqual(l1.length, 7) self.assertEqual(numpy.allclose([7, 1, 2, 8, 9, 5, 6], l1.get_npy_array()), True) def test_remove_with_strides(self): def bremove(rem): l1b = LongArray(12) l1b.set_data(numpy.arange(12)) l1b.remove(rem, stride=3) return l1b rem = [3, 1] l1 = self.benchmark(bremove, numpy.array(rem, dtype=int)) # Then self.assertEqual(l1.length, 6) self.assertEqual(numpy.allclose([0, 1, 2, 6, 7, 8], l1.get_npy_array()), True) # Given l1 = LongArray(12) l1.set_data(numpy.arange(12)) # When rem = [0, 2] l1.remove(numpy.array(rem, dtype=int), stride=3) # Then self.assertEqual(l1.length, 6) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] self.assertEqual(numpy.allclose([9, 10, 11, 3, 4, 5], l1.get_npy_array()), True) def test_align_array(self): l1 = LongArray(10) l1.set_data(numpy.arange(10)) new_indices = LongArray(10) new_indices.set_data(numpy.asarray([1, 5, 3, 2, 4, 7, 8, 6, 9, 0])) l1.align_array(new_indices) self.assertEqual(numpy.allclose([1, 5, 3, 2, 4, 7, 8, 6, 9, 0], l1.get_npy_array()), True) # Test case with strides. def balign_array(): l1b = LongArray(6) l1b.set_data(numpy.arange(6)) new_indices = LongArray(3) new_indices.set_data(numpy.asarray([2, 1, 0])) l1b.align_array(new_indices, 2) return l1b l1 = self.benchmark(balign_array) self.assertEqual(numpy.allclose([4, 5, 2, 3, 0, 1], l1.get_npy_array()), True) def test_copy_subset(self): def bcopy_subset(l2b): l1b = LongArray(10) l1b.set_data(numpy.arange(10)) # a valid copy. l1b.copy_subset(l2b, 5, 9) return l1b l2 = LongArray(4) l2[0] = 4 l2[1] = 3 l2[2] = 2 l2[3] = 1 l1 = self.benchmark(bcopy_subset, l2) self.assertEqual(numpy.allclose([0, 1, 2, 3, 4, 4, 3, 2, 1, 9], l1.get_npy_array()), True) def test_copy_subset_works_with_strides(self): def bcopy_subset(l2b): l1b = LongArray(8) l1b.set_data(numpy.arange(8)) l1b.copy_subset(l2b, 2, 3, stride=2) return l1b # Given l2 = LongArray(4) l2.set_data(numpy.arange(10, 14)) # When l1 = self.benchmark(bcopy_subset, l2) # Then numpy.testing.assert_array_equal( l1.get_npy_array(), [0, 1, 2, 3, 10, 11, 6, 7] ) def test_copy_values(self): def bcopy_values(l2b, indices): l1b = LongArray(8) l1b.set_data(numpy.arange(8)) l1b.copy_values(indices, l2b) return l1b # Given l1 = LongArray(8) l1.set_data(numpy.arange(8)) l2 = LongArray(8) l2.set_data(numpy.zeros(8, dtype=int)) # When indices = LongArray(3) indices.set_data(numpy.array([2, 4, 6])) l1 = self.benchmark.pedantic(bcopy_values, args=(l2, indices)) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [2, 4, 6] + [0] * 5 ) def test_update_min_max(self): """ Tests the update_min_max function. """ def bupdate_min_max(): l1b = LongArray(10) l1b.set_data(numpy.arange(10)) l1b.update_min_max() return l1b l1 = self.benchmark(bupdate_min_max) self.assertEqual(l1.minimum, 0) self.assertEqual(l1.maximum, 9) def test_pickling(self): """ Tests the __reduce__ and __setstate__ functions. """ import pickle def bpickle(l1b): l1_dump = pickle.dumps(l1b) return pickle.loads(l1_dump) l1 = LongArray(3) l1.set_data(numpy.arange(3)) l1_load = self.benchmark(bpickle, l1) self.assertEqual( (l1_load.get_npy_array() == l1.get_npy_array()).all(), True) def test_set_view(self): # Given src = LongArray() src.extend(numpy.arange(5)) # When. def bset_view(bsrc): bview = LongArray() bview.set_view(bsrc, 1, 4) return bview view = self.benchmark(bset_view, src) # Then. self.assertEqual(view.length, 3) expect = list(range(1, 4)) self.assertListEqual(view.get_npy_array().tolist(), expect) def test_set_view_for_empty_array(self): # Given src = LongArray() src.extend(numpy.arange(5)) # When. def bset_view(bsrc): view = LongArray() view.set_view(bsrc, 1, 1) return view view = self.benchmark(bset_view, src) # Then. self.assertEqual(view.length, 0) expect = [] self.assertListEqual(view.get_npy_array().tolist(), expect) if __name__ == '__main__': unittest.main() cyarray-1.2/docs/000077500000000000000000000000001501406650000137535ustar00rootroot00000000000000cyarray-1.2/docs/Makefile000066400000000000000000000011401501406650000154070ustar00rootroot00000000000000# Minimal makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build SPHINXPROJ = cyarray SOURCEDIR = source BUILDDIR = build # Put it first so that "make" without argument is like "make help". help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) .PHONY: help Makefile # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)cyarray-1.2/docs/make.bat000066400000000000000000000014571501406650000153670ustar00rootroot00000000000000@ECHO OFF pushd %~dp0 REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set SOURCEDIR=source set BUILDDIR=build set SPHINXPROJ=cyarray if "%1" == "" goto help %SPHINXBUILD% >NUL 2>NUL if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% goto end :help %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% :end popd cyarray-1.2/docs/source/000077500000000000000000000000001501406650000152535ustar00rootroot00000000000000cyarray-1.2/docs/source/conf.py000066400000000000000000000121401501406650000165500ustar00rootroot00000000000000#!/usr/bin/env python3 # -*- coding: utf-8 -*- # # cyarray documentation build configuration file, created by # sphinx-quickstart on Thu Nov 29 20:49:39 2018. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # # import os # import sys # sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. # # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.mathjax', 'sphinx.ext.viewcode', 'sphinx.ext.napoleon'] autodoc_default_flags = ['show-inheritance'] autoclass_content = "both" napoleon_google_docstring = True napoleon_numpy_docstring = True napoleon_include_private_with_doc = False napoleon_include_special_with_doc = True # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # # source_suffix = ['.rst', '.md'] source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = 'cyarray' copyright = '2018, PySPH developers' author = 'PySPH developers' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '1.0' # The full version, including alpha/beta/rc tags. release = '1.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This patterns also effect to html_static_path and html_extra_path exclude_patterns = [] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # html_theme = 'sphinx_rtd_theme' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # # html_theme_options = {} # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # -- Options for HTMLHelp output ------------------------------------------ # Output file base name for HTML help builder. htmlhelp_basename = 'cyarraydoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). # # 'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). # # 'pointsize': '10pt', # Additional stuff for the LaTeX preamble. # # 'preamble': '', # Latex figure (float) alignment # # 'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'cyarray.tex', 'cyarray Documentation', 'PySPH developers', 'manual'), ] # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'cyarray', 'cyarray Documentation', [author], 1) ] # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'cyarray', 'cyarray Documentation', author, 'cyarray', 'One line description of project.', 'Miscellaneous'), ] # Example configuration for intersphinx: refer to the Python standard library. intersphinx_mapping = {'https://docs.python.org/': None} cyarray-1.2/docs/source/index.rst000066400000000000000000000013101501406650000171070ustar00rootroot00000000000000.. cyarray documentation master file, created by sphinx-quickstart on Thu Nov 29 20:49:39 2018. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to cyarray's documentation! =================================== The cyarray package provides a fast, typed, re-sizable, Cython array. .. toctree:: :maxdepth: 2 :caption: Contents: overview.rst ************************ Reference documentation ************************ Autogenerated from doc strings using sphinx's autodoc feature. .. toctree:: :maxdepth: 2 reference/index Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` cyarray-1.2/docs/source/overview.rst000066400000000000000000000033341501406650000176560ustar00rootroot00000000000000========= Overview ========= The cyarray package provides a fast, typed, re-sizable, Cython_ array. It currently provides the following arrays: ``IntArray, UIntArray, LongArray, FloatArray, DoubleArray``. All arrays provide for the following operations: - access by indexing. - access through get/set function. - resizing the array. - appending values at the end of the array. - reserving space for future appends. - access to internal data through a numpy array. If you are writing Cython code this is a convenient array to use as it exposes the raw underlying pointer to the data. For example if you use a ``FloatArray`` and access its ``data`` attribute it will be a ``float*``. Each array also provides an interface to its data through a numpy array. This is done through the ``get_npy_array`` function. The returned numpy array can be used just like any other numpy array but for the following restrictions: - the array may not be resized. - references of this array should not be kept. - slices of this array may not be made. The numpy array may however be copied and used in any manner. Installation ------------ cyarray can be installed using pip_:: $ pip install cyarray The package requires ``Cython``, ``numpy``, and ``mako`` to be installed and also requires a suitably configured C/C++ compiler. .. _pip: http://www.pip-installer.org .. _Cython: https://cython.org Usage ----- In Python one may import and use the package as:: from cyarray.api import IntArray a = IntArray(10) Here ``a`` is an array of 10 integers. For more usage information, see the simple test cases in `test_carray.py `_. Also see the reference documentation included here. cyarray-1.2/docs/source/reference/000077500000000000000000000000001501406650000172115ustar00rootroot00000000000000cyarray-1.2/docs/source/reference/carray.rst000066400000000000000000000001361501406650000212240ustar00rootroot00000000000000Module carray =============== .. automodule:: cyarray.carray :members: :undoc-members: cyarray-1.2/docs/source/reference/index.rst000066400000000000000000000002541501406650000210530ustar00rootroot00000000000000cyarray reference documentation ================================ Autogenerated from doc strings using sphinx’s autodoc feature. .. toctree:: :maxdepth: 3 carray cyarray-1.2/pyproject.toml000066400000000000000000000023271501406650000157430ustar00rootroot00000000000000[build-system] requires = [ "wheel>=0.29.0", "setuptools>=42.0.0", "numpy", "Cython", "mako" ] [project] name = "cyarray" dynamic = ["version"] readme = "README.rst" license = {file = "LICENSE.txt"} dependencies = ['numpy'] description = "A fast, typed, resizable, Cython array." authors = [ {name = "Cyarray Developers", email = "pysph-dev@googlegroups.com"} ] keywords = ["Cython", "array", "resizable"] classifiers = [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: BSD License", "Natural Language :: English", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Operating System :: Unix", "Programming Language :: Python", "Programming Language :: Python :: 3", "Topic :: Scientific/Engineering", "Topic :: Scientific/Engineering :: Physics", "Topic :: Software Development :: Libraries", ] [project.optional-dependencies] docs = ["sphinx"] tests = ["pytest", "pytest-benchmark[histogram]"] dev = ["sphinx", "pytest", "pytest-benchmark[histogram]", "cython"] cyarray-1.2/requirements.txt000066400000000000000000000001051501406650000163030ustar00rootroot00000000000000Cython setuptools>=6.0 numpy mako pytest pytest-benchmark[histogram] cyarray-1.2/setup.py000066400000000000000000000113041501406650000145340ustar00rootroot00000000000000import os import sys from subprocess import check_output if len(os.environ.get('COVERAGE', '')) > 0: MACROS = [("CYTHON_TRACE", "1"), ("CYTHON_TRACE_NOGIL", "1")] COMPILER_DIRECTIVES = {"linetrace": True} print("-" * 80) print("Enabling linetracing for cython and setting CYTHON_TRACE = 1") print("-" * 80) else: MACROS = [] COMPILER_DIRECTIVES = {} MODE = 'normal' if len(sys.argv) >= 2 and \ ('--help' in sys.argv[1:] or sys.argv[1] in ('--help-commands', 'egg_info', '--version', 'clean', 'sdist')): MODE = 'info' def get_basic_extensions(): if MODE == 'info': try: from Cython.Distutils import Extension except ImportError: from distutils.core import Extension try: import numpy except ImportError: include_dirs = [] else: include_dirs = [numpy.get_include()] else: from Cython.Distutils import Extension import numpy include_dirs = [numpy.get_include()] ext_modules = [ Extension( name="cyarray.carray", sources=["cyarray/carray.pyx"], include_dirs=include_dirs, ), ] return ext_modules def create_sources(): argv = sys.argv if 'build_ext' in argv or 'develop' in sys.argv or 'install' in argv: generator = os.path.join('cyarray', 'generator.py') cmd = [sys.executable, generator, os.path.abspath('cyarray')] print(check_output(cmd).decode()) def _is_cythonize_default(): import warnings result = True with warnings.catch_warnings(): warnings.simplefilter("ignore") try: # old_build_ext was introduced in Cython 0.25 and this is when # cythonize was made the default. from Cython.Distutils import old_build_ext # noqa: F401 except ImportError: result = False return result def setup_package(): from setuptools import find_packages, setup if MODE == 'info': cmdclass = {} else: from Cython.Distutils import build_ext cmdclass = {'build_ext': build_ext} create_sources() # Extract the version information from pysph/__init__.py info = {} module = os.path.join('cyarray', '__init__.py') exec(compile(open(module).read(), module, 'exec'), info) ext_modules = get_basic_extensions() if MODE != 'info' and _is_cythonize_default(): # Cython >= 0.25 uses cythonize to compile the extensions. This # requires the compile_time_env to be set explicitly to work. compile_env = {} include_path = set() for mod in ext_modules: compile_env.update(mod.cython_compile_time_env or {}) include_path.update(mod.include_dirs) from Cython.Build import cythonize ext_modules = cythonize( ext_modules, compile_time_env=compile_env, include_path=list(include_path), compiler_directives=COMPILER_DIRECTIVES, ) setup(name='cyarray', version=info['__version__'], author='Cyarray Developers', author_email='pysph-dev@googlegroups.com', description='A fast, typed, resizable, Cython array.', long_description=open('README.rst').read(), url='http://github.com/pypr/cyarray', license="BSD", keywords="Cython array resizable", packages=find_packages(), package_data={ '': ['*.pxd', '*.mako', '*.rst'] }, # exclude package data in installation. exclude_package_data={ '': ['Makefile', '*.bat', '*.cfg', '*.rst', '*.sh', '*.yml'], }, ext_modules=ext_modules, include_package_data=True, cmdclass=cmdclass, zip_safe=False, platforms=['Linux', 'Mac OS-X', 'Unix', 'Windows'], classifiers=[c.strip() for c in """\ Development Status :: 5 - Production/Stable Environment :: Console Intended Audience :: Developers Intended Audience :: Science/Research License :: OSI Approved :: BSD License Natural Language :: English Operating System :: MacOS :: MacOS X Operating System :: Microsoft :: Windows Operating System :: POSIX Operating System :: Unix Programming Language :: Python Programming Language :: Python :: 3 Topic :: Scientific/Engineering Topic :: Scientific/Engineering :: Physics Topic :: Software Development :: Libraries """.splitlines() if len(c.split()) > 0], ) if __name__ == '__main__': setup_package()