Advanced Matrix Extensions

<h2 id="extensions">Extensions</h2>
AMX was introduced by Intel in June 2020 and first supported by Intel with the <a href="/facts/Sapphire_Rapids_(microprocessor)/WJsEwyru">Sapphire Rapids</a> <a href="/facts/Microarchitecture/NFRd4NUL">microarchitecture</a> for <a href="/facts/Xeon/j1M8Hp6D">Xeon</a> servers, released in January 2023.<a class="footnote-ref" id="fnref:2" href="#fn:2">2</a><a class="footnote-ref" id="fnref:3" href="#fn:3">3</a> It introduced 2-dimensional <a href="/facts/Processor_register/32RvbUdz">registers</a> called tiles upon which accelerators can perform operations. It is intended as an extensible architecture; the first accelerator implemented is called tile matrix multiply unit (TMUL).<a class="footnote-ref" id="fnref:4" href="#fn:4">4</a><a class="footnote-ref" id="fnref:5" href="#fn:5">5</a>
In Intel Architecture Instruction Set Extensions and Future Features revision 46, published in September 2022, a new AMX-FP16 extension was documented. This extension adds support for <a href="/facts/Half-precision_floating-point_format/7xPj7B3w">half-precision floating-point</a> numbers. In revision 48 from March 2023, AMX-COMPLEX was documented, adding support for half-precision floating-point <a href="/facts/Complex_numbers/2w7mqI7e">complex numbers</a>. Both extensions are available in the <a href="/facts/Granite_Rapids/fPjN9DP9">Granite Rapids</a> set of server processors (with AMX-COMPLEX support only being available in <a href="/facts/Xeon_D/rQNvyLHM">Granite Rapids-D</a><a class="footnote-ref" id="fnref:6" href="#fn:6">6</a>).

<h3>Tile matrix multiply unit</h3>
TMUL unit supports <a href="/facts/Bfloat16_floating-point_format/wo1Q6IJY">BF16</a> and <a href="/facts/INT8/zWabo5M9">INT8</a> input types.<a class="footnote-ref" id="fnref:7" href="#fn:7">7</a> AMX-FP16 and AMX-COMPLEX also add support for real and complex <a href="/facts/Half-precision_floating-point_format/7xPj7B3w">FP16</a> numbers. The register file consists of 8 tiles, each with 16 rows of size of 64 bytes (32 BF16/FP16 or 64 INT8 elements). The only supported operation is <a href="/facts/Matrix_multiplication/lYDB6Ro2">matrix multiplication</a> 
 
 
 
 
 C
 
 n
 m
 
 
 +
 =
 
 ∑
 
 j
 =
 1
 
 
 J
 
 
 
 A
 
 n
 j
 
 
 
 B
 
 j
 m
 
 
 .
 
 
 {\textstyle C_{nm}+=\sum _{j=1}^{J}A_{nj}B_{jm}.}
 
<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a>
4th Gen Intel Xeon Scalable processor can perform 2048 INT8 or 1024 BF16 operations per cycle:<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a><a class="footnote-ref" id="fnref:10" href="#fn:10">10</a> the maximal input sizes are 
 
 
 
 16
 ×
 J
 
 
 {\textstyle 16\times J}
 
 for A and 
 
 
 
 J
 ×
 16
 
 
 {\textstyle J\times 16}
 
 for B, where J is 64 for INT8 and 32 for BF16. The matrix multiplication requires 
 
 
 
 256
 J
 
 
 {\textstyle 256J}
 
 multiplication and 
 
 
 
 256
 J
 
 
 {\textstyle 256J}
 
 additions, thus performing 
 
 
 
 512
 J
 
 
 {\textstyle 512J}
 
 operations in 16 cycles.<a class="footnote-ref" id="fnref:11" href="#fn:11">11</a>

<h2 id="software-support">Software support</h2>
<ul><li>Compiler and assembler support
<ul><li><a href="/facts/LLVM/nPmcspQN">LLVM</a> 13<a class="footnote-ref" id="fnref:12" href="#fn:12">12</a><a class="footnote-ref" id="fnref:13" href="#fn:13">13</a><a class="footnote-ref" id="fnref:14" href="#fn:14">14</a><a class="footnote-ref" id="fnref:15" href="#fn:15">15</a></li>
<li><a href="/facts/GNU_Compiler_Collection/t07jh01k">GCC</a> 11<a class="footnote-ref" id="fnref:16" href="#fn:16">16</a><a class="footnote-ref" id="fnref:17" href="#fn:17">17</a><a class="footnote-ref" id="fnref:18" href="#fn:18">18</a></li>
<li><a href="/facts/GNU_Assembler/j2Xo6Fbj">GNU Assembler</a> (GAS) initial support committed on 25 June 2020<a class="footnote-ref" id="fnref:19" href="#fn:19">19</a><a class="footnote-ref" id="fnref:20" href="#fn:20">20</a></li></ul></li>
<li>Operating system support
<ul><li><a href="/facts/Glibc/AL6IYDkA">glibc</a> support for detecting AMX feature in CPUs committed on 25 June 2020<a class="footnote-ref" id="fnref:21" href="#fn:21">21</a></li>
<li><a href="/facts/Linux_kernel/dlkduHuA">Linux kernel</a> support since version 5.16<a class="footnote-ref" id="fnref:22" href="#fn:22">22</a></li>
<li><a href="/facts/VMware_vSphere/cyGTG0od">VMware vSphere</a> support for AMX in virtual machines released in <a href="/facts/ESXi/VQrtsQQg">ESXi</a> version 8.0u1 for VMs using Hardware Version 20<a class="footnote-ref" id="fnref:23" href="#fn:23">23</a></li></ul></li></ul>

<h2 id="external-links">External links</h2>
<ul><li><a href="https://software.intel.com/sites/landingpage/IntrinsicsGuide/">Intel Intrinsics Guide</a></li>
<li><a href="https://en.wikichip.org/wiki/x86/amx">Wikichip: Advanced Matrix Extension (AMX) - x86</a></li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">Hemsoth, Nicole (August 19, 2021). "With AMX, Intel Adds AI/ML Sparkle to Sapphire Rapids". The Next Platform. <a href="https://www.nextplatform.com/2021/08/19/with-amx-intel-adds-ai-ml-sparkle-to-sapphire-rapids/" target="_blank">https://www.nextplatform.com/2021/08/19/with-amx-intel-adds-ai-ml-sparkle-to-sapphire-rapids/</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">online, heise (28 June 2020). "Intel AMX: Erste Informationen zur Advanced Matrix Extensions Architecture". heise online. <a href="https://www.heise.de/news/Intel-AMX-Erste-Informationen-zur-Advanced-Matrix-Extensions-Architecture-4797415.html" target="_blank">https://www.heise.de/news/Intel-AMX-Erste-Informationen-zur-Advanced-Matrix-Extensions-Architecture-4797415.html</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">Cutress, Ian. "Intel Xeon Sapphire Rapids: How To Go Monolithic with Tiles". AnandTech. <a href="https://www.anandtech.com/show/16921/intel-sapphire-rapids-nextgen-xeon-scalable-gets-a-tiling-upgrade" target="_blank">https://www.anandtech.com/show/16921/intel-sapphire-rapids-nextgen-xeon-scalable-gets-a-tiling-upgrade</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">"Intel® Architecture Instruction Set Extensions and Future Features". <a href="https://www.intel.com/content/www/us/en/content-details/790021/intel-architecture-instruction-set-extensions-programming-reference.html" target="_blank">https://www.intel.com/content/www/us/en/content-details/790021/intel-architecture-instruction-set-extensions-programming-reference.html</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">Schor, David (June 29, 2020). "The x86 Advanced Matrix Extension (AMX) Brings Matrix Operations; To Debut with Sapphire Rapids". <a href="https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-extension-amx-brings-matrix-operations-to-debut-with-sapphire-rapids/" target="_blank">https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-extension-amx-brings-matrix-operations-to-debut-with-sapphire-rapids/</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">Larabel, Michael (July 12, 2023). "Intel Granite Rapids D Support Merged Into GCC 14". Phoronix. <a href="https://www.phoronix.com/news/Granite-Rapids-D-GCC-14" target="_blank">https://www.phoronix.com/news/Granite-Rapids-D-GCC-14</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">"Advanced Matrix Extension (AMX) - x86 - WikiChip". en.wikichip.org. <a href="https://en.wikichip.org/wiki/x86/amx" target="_blank">https://en.wikichip.org/wiki/x86/amx</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">"Intel® Architecture Instruction Set Extensions and Future Features". <a href="https://www.intel.com/content/www/us/en/content-details/790021/intel-architecture-instruction-set-extensions-programming-reference.html" target="_blank">https://www.intel.com/content/www/us/en/content-details/790021/intel-architecture-instruction-set-extensions-programming-reference.html</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">"Accelerate Artificial Intelligence (AI) Workloads with Intel Advanced Matrix Extensions (Intel AMX)" (PDF). Intel. Retrieved 2023-04-13. <a href="https://www.intel.com/content/dam/www/central-libraries/us/en/documents/2022-12/accelerate-ai-with-amx-sb.pdf" target="_blank">https://www.intel.com/content/dam/www/central-libraries/us/en/documents/2022-12/accelerate-ai-with-amx-sb.pdf</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">"Intel® 64 and IA-32 Architectures Optimization Reference Manual Volume 1". Intel. <a href="https://www.intel.com/content/www/us/en/content-details/671488/intel-64-and-ia-32-architectures-optimization-reference-manual-volume-1.html" target="_blank">https://www.intel.com/content/www/us/en/content-details/671488/intel-64-and-ia-32-architectures-optimization-reference-manual-volume-1.html</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">"Intel® 64 and IA-32 Architectures Optimization Reference Manual Volume 1". Intel. <a href="https://www.intel.com/content/www/us/en/content-details/671488/intel-64-and-ia-32-architectures-optimization-reference-manual-volume-1.html" target="_blank">https://www.intel.com/content/www/us/en/content-details/671488/intel-64-and-ia-32-architectures-optimization-reference-manual-volume-1.html</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
<li id="fn:12">"What's New in LLVM for 4th Gen Intel® Xeon® & Max Series CPUs". Retrieved 21 April 2023. <a href="https://www.intel.com/content/www/us/en/developer/articles/technical/whats-new-in-llvm-for-4th-gen-intel-xeon-processor.html" target="_blank">https://www.intel.com/content/www/us/en/developer/articles/technical/whats-new-in-llvm-for-4th-gen-intel-xeon-processor.html</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></li>
<li id="fn:13">Larabel, Michael (2020-07-02). "Intel AMX Support Begins Landing In LLVM". Phoronix. Retrieved 2020-07-02. <a href="https://www.phoronix.com/scan.php?page=news_item&px=Intel-AMX-LLVM-Starts" target="_blank">https://www.phoronix.com/scan.php?page=news_item&px=Intel-AMX-LLVM-Starts</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></li>
<li id="fn:14">"[X86-64] Support Intel AMX instructions". GitHub. 2020-07-02. Retrieved 2020-07-02. <a href="https://github.com/llvm/llvm-project/commit/aded4f0cc070fcef6763c9a3c2ba764d652b692e" target="_blank">https://github.com/llvm/llvm-project/commit/aded4f0cc070fcef6763c9a3c2ba764d652b692e</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></li>
<li id="fn:15">Larabel, Michael (2020-07-02). "Intel AMX Support Lands In The GNU Assembler". Phoronix. Retrieved 2020-07-02. <a href="https://www.phoronix.com/scan.php?page=news_item&px=Intel-AMX-Gas" target="_blank">https://www.phoronix.com/scan.php?page=news_item&px=Intel-AMX-Gas</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></li>
<li id="fn:16">"GCC 11 Release Series — Changes, New Features, and Fixes - GNU Project". Retrieved 21 April 2023. <a href="https://gcc.gnu.org/gcc-11/changes.html" target="_blank">https://gcc.gnu.org/gcc-11/changes.html</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></li>
<li id="fn:17">"[PATCH] Enable GCC support for AMX". 2020-07-06. Retrieved 2020-07-09. <a href="https://gcc.gnu.org/pipermail/gcc-patches/2020-July/549415.html" target="_blank">https://gcc.gnu.org/pipermail/gcc-patches/2020-July/549415.html</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></li>
<li id="fn:18">"Enable GCC support for AMX-TILE, AMX-INT8, AMX-BF16. · gcc-mirror/gcc@5c60984". GitHub. Retrieved 2022-09-05. <a href="https://github.com/gcc-mirror/gcc/commit/5c609842d13a4c9c6be1a10f7980a74d27daeb85" target="_blank">https://github.com/gcc-mirror/gcc/commit/5c609842d13a4c9c6be1a10f7980a74d27daeb85</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></li>
<li id="fn:19">"commits with Intel AMX". 2020-07-02. Retrieved 2020-07-02. <a href="https://sourceware.org/git/?p=binutils-gdb.git&a=search&st=commit&s=Intel+AMX" target="_blank">https://sourceware.org/git/?p=binutils-gdb.git&a=search&st=commit&s=Intel+AMX</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></li>
<li id="fn:20">Larabel, Michael (2020-07-02). "Intel AMX Support Lands In The GNU Assembler". Phoronix. Retrieved 2020-07-02. <a href="https://www.phoronix.com/scan.php?page=news_item&px=Intel-AMX-Gas" target="_blank">https://www.phoronix.com/scan.php?page=news_item&px=Intel-AMX-Gas</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></li>
<li id="fn:21">"x86: Detect Intel Advanced Matrix Extensions". 2020-07-02. Retrieved 2020-07-02. <a href="https://sourceware.org/git/?p=glibc.git;a=commit;h=4fdd4d41a17dda26c854ed935658154a17d4b906" target="_blank">https://sourceware.org/git/?p=glibc.git;a=commit;h=4fdd4d41a17dda26c854ed935658154a17d4b906</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></li>
<li id="fn:22">"Linux 5.16 Features Include FUTEX2, Intel AMX, Folios, DG2/Alchemist, More Apple Silicon Support". Phoronix. <a href="https://www.phoronix.com/review/linux-516-features" target="_blank">https://www.phoronix.com/review/linux-516-features</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></li>
<li id="fn:23">"Accessing Sapphire Rapids AMX instructions on vSphere". Earl C. Ruby III. 2023-08-24. <a href="https://earlruby.org/2023/08/accessing-sapphire-rapids-amx-instructions-on-vsphere/" target="_blank">https://earlruby.org/2023/08/accessing-sapphire-rapids-amx-instructions-on-vsphere/</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></li>
</ol>

Advanced Matrix Extensions open-in-new

Advanced Matrix Extensions