CuPy

<h2 id="features">Features</h2>
<p>CuPy implements NumPy/SciPy-compatible APIs, as well as features to write user-defined GPU kernels or access low-level APIs.<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a><a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a>
</p>
<h3>NumPy-compatible APIs</h3>
<p>The same set of APIs defined in the NumPy package (numpy.*) are available under cupy.* package.
</p>
<ul><li>Multi-dimensional <a href="/facts/Array_data_type/gHJJ3XPw">array</a> (cupy.ndarray) for boolean, integer, float, and complex data types</li>
<li>Module-level functions</li>
<li><a href="/facts/Linear_algebra/8VaB4VWk">Linear algebra</a> functions</li>
<li><a href="/facts/Fast_Fourier_transform/caT7UUCh">Fast Fourier transform</a></li>
<li><a href="/facts/Random_number_generation/RtXEfbzd">Random number generator</a></li></ul>
<h3>SciPy-compatible APIs</h3>
<p>The same set of APIs defined in the SciPy package (scipy.*) are available under cupyx.scipy.* package.
</p>
<ul><li><a href="/facts/Sparse_matrix/FFtN0XjR">Sparse matrices</a> (cupyx.scipy.sparse.*_matrix) of CSR, COO, CSC, and DIA format</li>
<li><a href="/facts/Discrete_Fourier_transform/uK9Tjsbl">Discrete Fourier transform</a></li>
<li>Advanced linear algebra</li>
<li>Multidimensional <a href="/facts/Digital_image_processing/zenalNCf">image processing</a></li>
<li>Sparse linear algebra</li>
<li>Special functions</li>
<li><a href="/facts/Signal_processing/Npaja6zb">Signal processing</a></li>
<li><a href="/facts/Statistics/DNPsKGYU">Statistical functions</a></li></ul>
<h3>User-defined GPU kernels</h3>
<ul><li>Kernel templates for element-wise and reduction operations</li>
<li>Raw kernel (CUDA C/C++)</li>
<li>Just-in-time transpiler (JIT)</li>
<li>Kernel fusion</li></ul>
<h3>Distributed computing</h3>
<ul><li>Distributed communication package (cupyx.distributed), providing collective and peer-to-peer primitives</li></ul>
<h3>Low-level CUDA features</h3>
<ul><li>Stream and event</li>
<li>Memory pool</li>
<li>Profiler</li>
<li>Host API binding</li>
<li>CUDA Python support<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a></li></ul>
<h3>Interoperability</h3>
<ul><li>DLPack<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a></li>
<li>CUDA Array Interface<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a></li>
<li>NEP 13 (__array_ufunc__)<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a></li>
<li>NEP 18 (__array_function__)<a class="footnote-ref" id="fnref:18" href="#fn:18"><sup>18</sup></a><a class="footnote-ref" id="fnref:19" href="#fn:19"><sup>19</sup></a></li>
<li>Array API Standard<a class="footnote-ref" id="fnref:20" href="#fn:20"><sup>20</sup></a><a class="footnote-ref" id="fnref:21" href="#fn:21"><sup>21</sup></a></li></ul>
<h2 id="examples">Examples</h2>
<h3>Array creation</h3>
>>> import cupy as cp
>>> x = cp.array([1, 2, 3])
>>> x
array([1, 2, 3])
>>> y = cp.arange(10)
>>> y
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

<h3>Basic operations</h3>
>>> import cupy as cp
>>> x = cp.arange(12).reshape(3, 4).astype(cp.float32)
>>> x
array([[ 0.,  1.,  2.,  3.],
       [ 4.,  5.,  6.,  7.],
       [ 8.,  9., 10., 11.]], dtype=float32)
>>> x.sum(axis=1)
array([ 6., 22., 38.], dtype=float32)

<h3>Raw CUDA C/C++ kernel</h3>
>>> import cupy as cp
>>> kern = cp.RawKernel(r'''
... extern "C" __global__
... void multiply_elemwise(const float* in1, const float* in2, float* out) {
...     int tid = blockDim.x * blockIdx.x + threadIdx.x;
...     out[tid] = in1[tid] * in2[tid];
... }
... ''', 'multiply_elemwise')
>>> in1 = cp.arange(16, dtype=cp.float32).reshape(4, 4)
>>> in2 = cp.arange(16, dtype=cp.float32).reshape(4, 4)
>>> out = cp.zeros((4, 4), dtype=cp.float32)
>>> kern((4,), (4,), (in1, in2, out))  # grid, block and arguments
>>> out
array([[  0.,   1.,   4.,   9.],
       [ 16.,  25.,  36.,  49.],
       [ 64.,  81., 100., 121.],
       [144., 169., 196., 225.]], dtype=float32)

<h2 id="applications">Applications</h2>
<ul><li><a href="/facts/SpaCy/qnabPP3o">spaCy</a><a class="footnote-ref" id="fnref:22" href="#fn:22"><sup>22</sup></a><a class="footnote-ref" id="fnref:23" href="#fn:23"><sup>23</sup></a></li>
<li><a href="/facts/XGBoost/YupgXmfH">XGBoost</a><a class="footnote-ref" id="fnref:24" href="#fn:24"><sup>24</sup></a></li>
<li>turboSETI (<a href="/facts/Berkeley_SETI_Research_Center/VBt4PFtx">Berkeley SETI</a>)<a class="footnote-ref" id="fnref:25" href="#fn:25"><sup>25</sup></a></li>
<li>NVIDIA RAPIDS<a class="footnote-ref" id="fnref:26" href="#fn:26"><sup>26</sup></a><a class="footnote-ref" id="fnref:27" href="#fn:27"><sup>27</sup></a><a class="footnote-ref" id="fnref:28" href="#fn:28"><sup>28</sup></a><a class="footnote-ref" id="fnref:29" href="#fn:29"><sup>29</sup></a></li>
<li>einops<a class="footnote-ref" id="fnref:30" href="#fn:30"><sup>30</sup></a><a class="footnote-ref" id="fnref:31" href="#fn:31"><sup>31</sup></a></li>
<li><a href="/facts/Scikit-learn/HqxDHRMJ">scikit-learn</a><a class="footnote-ref" id="fnref:32" href="#fn:32"><sup>32</sup></a></li>
<li><a href="/facts/Medical_open_network_for_AI/9qxnjWfq">MONAI</a></li>
<li><a href="/facts/Chainer/WN7uuwcz">Chainer</a><a class="footnote-ref" id="fnref:33" href="#fn:33"><sup>33</sup></a></li></ul>
<h2 id="see-also">See also</h2>
<ul>
<li>Free software portal</li></ul>
<ul><li><a href="/facts/Array_programming/gFx1SxIY">Array programming</a></li>
<li><a href="/facts/List_of_numerical-analysis_software/yKFzDBDK">List of numerical-analysis software</a></li>
<li><a href="/facts/Dask_(software)/PbyLpsID">Dask</a></li></ul>

<h2 id="external-links">External links</h2>
<ul><li><a href="https://cupy.dev/">Official website</a></li>
<li><a href="https://github.com/cupy/cupy">cupy</a> on <a href="/facts/GitHub/n1hNCsc3">GitHub</a></li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>Okuta, Ryosuke; Unno, Yuya; Nishino, Daisuke; Hido, Shohei; Loomis, Crissman (2017). CuPy: A NumPy-Compatible Library for NVIDIA GPU Calculations (PDF). Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Thirty-first Annual Conference on Neural Information Processing Systems (NIPS). <a href="http://learningsys.org/nips17/assets/papers/paper_16.pdf" target="_blank">http://learningsys.org/nips17/assets/papers/paper_16.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>"CuPy 9.0 Brings AMD GPU Support To This Numpy-Compatible Library - Phoronix". Phoronix. 29 April 2021. Retrieved 21 June 2022. <a href="https://www.phoronix.com/scan.php?page=news_item&px=CuPy-9.0-Released" target="_blank">https://www.phoronix.com/scan.php?page=news_item&px=CuPy-9.0-Released</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
<li id="fn:3"><p>"AMD Leads High Performance Computing Towards Exascale and Beyond". 28 June 2021. Retrieved 21 June 2022. Most recently, CuPy, an open-source array library with Python, has expanded its traditional GPU support with the introduction of version 9.0 that now offers support for the ROCm stack for GPU-accelerated computing. <a href="https://ir.amd.com/news-events/press-releases/detail/1012/amd-leads-high-performance-computing-towards-exascale-and" target="_blank">https://ir.amd.com/news-events/press-releases/detail/1012/amd-leads-high-performance-computing-towards-exascale-and</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li>
<li id="fn:4"><p>"Preferred Networks released Version 2 of Chainer, an Open Source framework for Deep Learning - Preferred Networks, Inc". 2 June 2017. Retrieved 18 June 2022. <a href="https://www.preferred.jp/en/news/pr20170602/" target="_blank">https://www.preferred.jp/en/news/pr20170602/</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li>
<li id="fn:5"><p>"NumPy". numpy.org. Retrieved 21 June 2022. <a href="https://numpy.org/" target="_blank">https://numpy.org/</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li>
<li id="fn:6"><p>Gorelick, Micha; Ozsvald, Ian (April 2020). High Performance Python: Practical Performant Programming for Humans (2nd ed.). O'Reilly Media, Inc. p. 190. ISBN 9781492055020. <a href="9781492055020" target="_blank">9781492055020</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li>
<li id="fn:7"><p>Oak Ridge Leadership Computing Facility. "Installing CuPy". OLCF User Documentation. Retrieved 21 June 2022. <a href="/wiki/Oak_Ridge_Leadership_Computing_Facility" target="_blank">/wiki/Oak_Ridge_Leadership_Computing_Facility</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li>
<li id="fn:8"><p>National Energy Research Scientific Computing Center. "Using Python on Perlmutter". NERSC Documentation. Retrieved 21 June 2022. <a href="/wiki/National_Energy_Research_Scientific_Computing_Center" target="_blank">/wiki/National_Energy_Research_Scientific_Computing_Center</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li>
<li id="fn:9"><p>ETH Zurich. "CuPy". ScientificComputing. Retrieved 21 June 2022. <a href="/wiki/ETH_Zurich" target="_blank">/wiki/ETH_Zurich</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li>
<li id="fn:10"><p>National Institute of Advanced Industrial Science and Technology. "Chainer". ABCI 2.0 User Guide. Retrieved 21 June 2022. <a href="/wiki/National_Institute_of_Advanced_Industrial_Science_and_Technology" target="_blank">/wiki/National_Institute_of_Advanced_Industrial_Science_and_Technology</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li>
<li id="fn:11"><p>"Sponsored Projects - NumFOCUS". Retrieved 8 September 2024. <a href="https://numfocus.org/sponsored-projects" target="_blank">https://numfocus.org/sponsored-projects</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li>
<li id="fn:12"><p>"Overview". CuPy documentation. Retrieved 18 June 2022. <a href="https://docs.cupy.dev/en/latest/overview.html" target="_blank">https://docs.cupy.dev/en/latest/overview.html</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li>
<li id="fn:13"><p>"Comparison Table". CuPy documentation. Retrieved 18 June 2022. <a href="https://docs.cupy.dev/en/latest/reference/comparison.html" target="_blank">https://docs.cupy.dev/en/latest/reference/comparison.html</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li>
<li id="fn:14"><p>"CUDA Python | NVIDIA Developer". Retrieved 21 June 2022. <a href="https://developer.nvidia.com/cuda-python" target="_blank">https://developer.nvidia.com/cuda-python</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li>
<li id="fn:15"><p>"Welcome to DLPack's documentation!". DLPack 0.6.0 documentation. Retrieved 21 June 2022. <a href="https://dmlc.github.io/dlpack/latest/" target="_blank">https://dmlc.github.io/dlpack/latest/</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li>
<li id="fn:16"><p>"CUDA Array Interface (Version 3)". Numba 0.55.2+0.g2298ad618.dirty-py3.7-linux-x86_64.egg documentation. Retrieved 21 June 2022. <a href="https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html" target="_blank">https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li>
<li id="fn:17"><p>"NEP 13 — A mechanism for overriding Ufuncs — NumPy Enhancement Proposals". numpy.org. Retrieved 21 June 2022. <a href="https://numpy.org/neps/nep-0013-ufunc-overrides.html" target="_blank">https://numpy.org/neps/nep-0013-ufunc-overrides.html</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li>
<li id="fn:18"><p>"NEP 18 — A dispatch mechanism for NumPy's high level array functions — NumPy Enhancement Proposals". numpy.org. Retrieved 21 June 2022. <a href="https://numpy.org/neps/nep-0018-array-function-protocol.html" target="_blank">https://numpy.org/neps/nep-0018-array-function-protocol.html</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></p></li>
<li id="fn:19"><p>Charles R Harris; K. Jarrod Millman; Stéfan J. van der Walt; et al. (16 September 2020). "Array programming with NumPy" (PDF). Nature. 585 (7825): 357–362. arXiv:2006.10256. doi:10.1038/S41586-020-2649-2. ISSN 1476-4687. PMC 7759461. PMID 32939066. Wikidata Q99413970. <a href="https://www.nature.com/articles/s41586-020-2649-2.pdf" target="_blank">https://www.nature.com/articles/s41586-020-2649-2.pdf</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></p></li>
<li id="fn:20"><p>"2021 report - Python Data APIs Consortium" (PDF). Retrieved 21 June 2022. <a href="https://data-apis.org/files/2021_annual_report_DataAPIs_Consortium.pdf" target="_blank">https://data-apis.org/files/2021_annual_report_DataAPIs_Consortium.pdf</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></p></li>
<li id="fn:21"><p>"Purpose and scope". Python array API standard 2021.12 documentation. Retrieved 21 June 2022. <a href="https://data-apis.org/array-api/latest/purpose_and_scope.html" target="_blank">https://data-apis.org/array-api/latest/purpose_and_scope.html</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></p></li>
<li id="fn:22"><p>"Install spaCy". spaCy Usage Documentation. Retrieved 21 June 2022. <a href="https://spacy.io/usage#gpu" target="_blank">https://spacy.io/usage#gpu</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></p></li>
<li id="fn:23"><p>Patel, Ankur A.; Arasanipalai, Ajay Uppili (May 2021). Applied Natural Language Processing in the Enterprise (1st ed.). O'Reilly Media, Inc. p. 68. ISBN 9781492062578. <a href="9781492062578" target="_blank">9781492062578</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></p></li>
<li id="fn:24"><p>"Python Package Introduction". xgboost 1.6.1 documentation. Retrieved 21 June 2022. <a href="https://xgboost.readthedocs.io/en/stable/python/python_intro.html#data-interface" target="_blank">https://xgboost.readthedocs.io/en/stable/python/python_intro.html#data-interface</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></p></li>
<li id="fn:25"><p>"UCBerkeleySETI/turbo_seti: turboSETI -- python based SETI search algorithm". GitHub. Retrieved 21 June 2022. <a href="https://github.com/UCBerkeleySETI/turbo_seti#turbo_seti" target="_blank">https://github.com/UCBerkeleySETI/turbo_seti#turbo_seti</a> <a href="#fnref:25" class="footnote-back-ref">↩</a></p></li>
<li id="fn:26"><p>"Open GPU Data Science | RAPIDS". Retrieved 21 June 2022. <a href="https://rapids.ai/" target="_blank">https://rapids.ai/</a> <a href="#fnref:26" class="footnote-back-ref">↩</a></p></li>
<li id="fn:27"><p>"API Docs". RAPIDS Docs. Retrieved 21 June 2022. <a href="https://docs.rapids.ai/api" target="_blank">https://docs.rapids.ai/api</a> <a href="#fnref:27" class="footnote-back-ref">↩</a></p></li>
<li id="fn:28"><p>"Efficient Data Sharing between CuPy and RAPIDS". Retrieved 21 June 2022. <a href="https://medium.com/rapids-ai/using-rapids-memory-manager-with-cupy-8d08fe8f58fa" target="_blank">https://medium.com/rapids-ai/using-rapids-memory-manager-with-cupy-8d08fe8f58fa</a> <a href="#fnref:28" class="footnote-back-ref">↩</a></p></li>
<li id="fn:29"><p>"10 Minutes to cuDF and CuPy". Retrieved 21 June 2022. <a href="https://medium.com/rapids-ai/10-minutes-to-cudf-and-cupy-e131cac0439b" target="_blank">https://medium.com/rapids-ai/10-minutes-to-cudf-and-cupy-e131cac0439b</a> <a href="#fnref:29" class="footnote-back-ref">↩</a></p></li>
<li id="fn:30"><p>Alex, Rogozhnikov (2022). Einops: Clear and Reliable Tensor Manipulations with Einstein-like Notation. International Conference on Learning Representations. <a href="https://openreview.net/forum?id=oapKSVM2bcj" target="_blank">https://openreview.net/forum?id=oapKSVM2bcj</a> <a href="#fnref:30" class="footnote-back-ref">↩</a></p></li>
<li id="fn:31"><p>"arogozhnikov/einops: Deep learning operations reinvented (for pytorch, tensorflow, jax and others)". GitHub. Retrieved 21 June 2022. <a href="https://github.com/arogozhnikov/einops" target="_blank">https://github.com/arogozhnikov/einops</a> <a href="#fnref:31" class="footnote-back-ref">↩</a></p></li>
<li id="fn:32"><p>"Array API support (experimental) — scikit-learn documentation". Retrieved 8 September 2024. <a href="https://scikit-learn.org/stable/modules/array_api.html" target="_blank">https://scikit-learn.org/stable/modules/array_api.html</a> <a href="#fnref:32" class="footnote-back-ref">↩</a></p></li>
<li id="fn:33"><p>Tokui, Seiya; Okuta, Ryosuke; Akiba, Takuya; Niitani, Yusuke; Ogawa, Toru; Saito, Shunta; Suzuki, Shuji; Uenishi, Kota; Vogel, Brian; Vincent, Hiroyuki Yamazaki (2019). Chainer: A Deep Learning Framework for Accelerating the Research Cycle. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. doi:10.1145/3292500.3330756. <a href="https://dl.acm.org/doi/10.1145/3292500.3330756" target="_blank">https://dl.acm.org/doi/10.1145/3292500.3330756</a> <a href="#fnref:33" class="footnote-back-ref">↩</a></p></li>
</ol>

CuPy open-in-new

CuPy