FMA instruction set

Compatibility

Although FMA3 and FMA4 have the same operations, their instructions differ in operands and encoding. Their instructions have different opcodes in different opcode maps. An instruction of one variant is an invalid opcode on a processor that supports only the other. Each variant has its own CPUID feature flag: FMA for FMA3 and FMA4 for FMA4.^{: 613–617}

Their operands differ as follows:

• FMA4 instructions have a destination and three sources. Their operands ${\displaystyle a}$, ${\displaystyle b}$, ${\displaystyle c}$, and ${\displaystyle d}$ can be four different registers. Either of the last two sources can be in memory.^: 6, 613 The fourth register is encoded in an extra byte. Each FMA4 instruction is one byte longer than the matching FMA3 instruction.^: 6, 617
• FMA3 instructions have three operands. The result is written over the first operand. The destination ${\displaystyle d}$ is therefore the same register as ${\displaystyle a}$, ${\displaystyle b}$, or ${\displaystyle c}$. Only the third operand can be in memory. Each FMA3 operation has three operand orders numbered 132, 213, and 231. The orders differ in whether the overwritten register and the memory operand are factors or the summand.^: 613 The input to be overwritten must first be copied to another register if all three inputs are still needed afterwards. The examples below contain such copies.^: 144

The split between FMA3 and FMA4 followed changes of plan by Intel and AMD. AMD Fellow Dave Christie wrote in May 2009 that AMD could not adopt Intel's December 2008 change of FMA from four to three operands without unacceptable risk to its product schedules. AMD therefore kept the four-operand form from Intel's earlier specification. Josh Walrath of PC Perspective wrote in 2009 that it was "unfortunate that AMD and Intel cannot work together a bit better on these things". Agner Fog wrote in 2009 that Intel, AMD, and VIA "are assigning different codes to identical or equivalent instructions because each keep their innovations secret for as long as possible". In 2011, he listed FMA3 and FMA4 among several pairs of AMD and Intel instructions with incompatible encodings. He wrote that Intel had given its own VEX encodings to instructions that AMD had first defined in the XOP encoding.

Naming

Intel and AMD define the same FMA3 mnemonics in their instruction-set manuals.^{: 5-207–5-331 : 613–672} The FMA4 mnemonics are defined only in AMD's manual.^{: 613–672} Each FMA mnemonic consists of a stem from the table above and a postfix for the data format:^{: 14-21–14-22 : 613–672}

• PS: packed single precision
• PD: packed double precision
• SS: scalar single precision
• SD: scalar double precision

In a packed instruction, the operation is applied to every element of a vector. In a scalar instruction, it is applied only to the lowest element.^: 14–22 An FMA4 mnemonic is the stem followed directly by the postfix, as in VFMADDPS. In an FMA3 mnemonic, the operand order 132, 213, or 231 comes between the stem and the postfix, as in VFMADD213PS.^{: 613–672}

Rounding

The exact intermediate value of an FMA instruction exists only inside the processor and cannot be examined by software.^: 14–23 FMA results are rounded in the rounding mode set in the processor's MXCSR control and status register.^: 613 For AVX-512 instructions, a rounding mode can instead be encoded in the 512-bit and scalar forms whose operands are all registers.^{: 5–222 : 15–12} Under IEEE 754-2008, overflow, underflow, and inexact exceptions in a fused multiply–add can arise only from the addition and not from the multiplication.^: 21

With separate multiply and add instructions, the product is rounded before the addition. The result is then rounded twice: ${\displaystyle \operatorname {round} (\operatorname {round} (a\cdot b)+c)}$. On x86 processors without FMA3 or FMA4, a result rounded only once is still available from the C fma function.^: 258 Intel describes its own implementation as having some cost in speed.^: 16

Contraction by compilers

Under the C11 standard, an expression such as a * b + c may be contracted into a single fused operation. Whether contraction is on by default is left to each compiler's developers.^{: 77, 235} When compiling for a processor with FMA instructions:

• In GCC, contraction is on by default except for C compiled in a standard-conforming mode such as -std=c11.
• In Clang, contraction within a single expression has been on by default since version 14.
• In Visual C++, contraction has been off by default since Visual Studio 2022.
• In Intel's compilers, contraction is on by default at optimization level -O1 and above.^: 16

Each compiler has an option to change the default. In the examples below, a * b + c is compiled into one FMA instruction.

Results of the same program can therefore differ between builds for processors with and without FMA.^: 16 Without FMA, the expression ${\displaystyle p\cdot q+r\cdot s}$ with ${\displaystyle r=p}$ and ${\displaystyle s=-q}$ is zero. With FMA, either product may be the unrounded one. The two possible results can differ from zero and from each other.^: 17

Intel recommends disabling FMA instructions for the best reproducibility between processors with and without FMA.^: 22 Even on systems that conform to IEEE 754-2008, a program may give different results if optimizations that change results (e.g. contraction) are not avoided.^: 50–52

Examples

The following C function computes ${\displaystyle a\cdot b+c}$ on four single-precision values at once, using the vector extension of GCC and Clang:

typedef float float4 __attribute__((vector_size(16)));

float4 madd(float4 a, float4 b, float4 c)
{
    return a * b + c;
}

The listings below are Clang 21's output with optimization (-O2) for FMA3 (-mfma) and for FMA4 (-mfma4), with assembler directives and Clang's comments removed. They are in Intel syntax, where the destination is the first operand.^: 616 Under the System V AMD64 calling convention, the vector arguments ${\displaystyle a}$, ${\displaystyle b}$, and ${\displaystyle c}$ are passed in registers XMM0, XMM1, and XMM2, with a vector result returned in XMM0.

madd:
    vfmadd213ps xmm0, xmm1, xmm2
    ret

madd:
    vfmaddps xmm0, xmm0, xmm1, xmm2
    ret

In the FMA3 listing, VFMADD213PS writes the result over its first operand, ${\displaystyle a}$. In the FMA4 listing, VFMADDPS has a separate destination operand. It writes to XMM0 only because XMM0 is the return register, and could write to any other register without changing ${\displaystyle a}$.^: 616

The next function uses all six FMA operations. The array r is passed as a pointer in register RDI. In the listings, each FMA instruction is commented with the element of r that it computes.

typedef float float4 __attribute__((vector_size(16)));

void fma_all_ops(float4 a, float4 b, float4 c, float4 r[6])
{
    r[0] = a * b + c;
    r[1] = a * b - c;
    r[2] = c - a * b;
    r[3] = -(a * b) - c;
    /* subtract in even elements, add in odd */
    r[4] = __builtin_shufflevector(a * c - b, a * c + b,
                                   0, 5, 2, 7);
    /* add in even elements, subtract in odd */
    r[5] = __builtin_shufflevector(b * c + a, b * c - a,
                                   0, 5, 2, 7);
}

Every element of r depends on all three inputs, so ${\displaystyle a}$, ${\displaystyle b}$, and ${\displaystyle c}$ must stay intact until the last operation.

fma_all_ops:
    vmovaps xmm3, xmm1
    vfmadd213ps xmm3, xmm0, xmm2       ; r[0]
    vmovaps xmmword ptr [rdi], xmm3
    vmovaps xmm3, xmm1
    vfmsub213ps xmm3, xmm0, xmm2       ; r[1]
    vmovaps xmmword ptr [rdi + 16], xmm3
    vmovaps xmm3, xmm1
    vfnmadd213ps xmm3, xmm0, xmm2      ; r[2]
    vmovaps xmmword ptr [rdi + 32], xmm3
    vmovaps xmm3, xmm1
    vfnmsub213ps xmm3, xmm0, xmm2      ; r[3]
    vmovaps xmmword ptr [rdi + 48], xmm3
    vmovaps xmm3, xmm2
    vfmaddsub213ps xmm3, xmm0, xmm1    ; r[4]
    vmovaps xmmword ptr [rdi + 64], xmm3
    vfmsubadd213ps xmm2, xmm1, xmm0    ; r[5]
    vmovaps xmmword ptr [rdi + 80], xmm2
    ret

fma_all_ops:
    vfmaddps xmm3, xmm0, xmm1, xmm2    ; r[0]
    vmovaps xmmword ptr [rdi], xmm3
    vfmsubps xmm3, xmm0, xmm1, xmm2    ; r[1]
    vmovaps xmmword ptr [rdi + 16], xmm3
    vfnmaddps xmm3, xmm0, xmm1, xmm2   ; r[2]
    vmovaps xmmword ptr [rdi + 32], xmm3
    vfnmsubps xmm3, xmm0, xmm1, xmm2   ; r[3]
    vmovaps xmmword ptr [rdi + 48], xmm3
    vfmaddsubps xmm3, xmm0, xmm2, xmm1 ; r[4]
    vmovaps xmmword ptr [rdi + 64], xmm3
    vfmsubaddps xmm0, xmm1, xmm2, xmm0 ; r[5]
    vmovaps xmmword ptr [rdi + 80], xmm0
    ret

Each FMA3 instruction except the last is therefore preceded by a VMOVAPS that copies an input into XMM3, and the FMA instruction then overwrites the copy.^: 144 The last FMA3 instruction overwrites ${\displaystyle c}$ directly, since ${\displaystyle c}$ is not needed afterwards. In the FMA4 listing, each FMA4 instruction except the last writes its result to XMM3, which is not one of the input registers, and the last overwrites ${\displaystyle a}$, which is no longer needed. Nothing is copied, so the only VMOVAPS instructions are the stores to r.

CPUs with FMA3

• AMD
  • Piledriver (2012) and newer microarchitectures
    • 2nd gen APUs, "Trinity" (32nm), May 15, 2012
    • 2nd gen "Bulldozer" (bdver2) with Piledriver cores, October 23, 2012
• Intel
  • Haswell (2013) and newer processors, except Pentiums and Celerons and low-power processors based on Goldmont and Tremont, such as Jasper Lake (2021), which lack AVX and support extensions only up to SSE4.2^: 9, 188

Excerpt from FMA3

Supported commands include

Note

• VFNMADD is result = − a · b + c, not result = − (a · b + c).
• VFNMSUB generates a −0 when all inputs are zero.

Explicit order of operands is included in the mnemonic using numbers "132", "213", and "231":

as well as operand format (packed or scalar) and size (single or double).

This results in

AVX-512

AVX-512 adds 512-bit forms of the FMA3 instructions using a new instruction encoding, the EVEX prefix.^: 5–220 The EVEX forms keep the FMA3 mnemonics, operand orders (132, 213, 231), and opcode bytes.^{: 5–207, 5–220} The VEX-encoded FMA3 forms operate on vectors of up to 256 bits and require the processor to report the FMA feature through the CPUID instruction. The EVEX forms also operate on 512-bit vectors. They require either the AVX10.1 feature or the AVX512F feature, plus AVX512VL for 128-bit and 256-bit vectors.^{: 5–220, 5–232}

Processors supporting AVX-512 include:

• Intel
  • Xeon Phi Knights Landing (2016), the first processors to support AVX-512^: 193
• AMD
  • Zen 4 (2022), AMD's first AVX-512 processor,^: 248 which executes 512-bit floating-point instructions, including fused multiply–add, by "double pumping" 256-bit datapaths^: 2
  • Zen 5 (2024), AMD's first core with full 512-bit floating-point datapaths and two FMA units, which executes 512-bit fused multiply–add at full width^: 2, 9

The EVEX forms add:

• masking of the destination by an opmask register, with unselected elements either left unchanged or set to zero^{: 5-221–5-222 : 15-1–15-2}
• broadcast of one 32-bit or 64-bit value from memory to every element of a source vector^{: 5–221 : 15–11}
• a rounding mode set by the instruction itself, for 512-bit and scalar forms with register operands^{: 5–222, 5-232–5-233 : 15–2, 15–12}

The 512-bit forms add a third packed size to postfix 2:

In the rounding mode column, "Yes" means the instruction can set its own rounding mode. Otherwise the rounding mode set in the processor's MXCSR register applies.^{: 613 : 15–12}

For example:^: 5–220

In the operands, {k} is the opmask register, {z} means unselected elements are set to zero, m32bcst is a 32-bit value broadcast from memory, and {er} means the instruction can set its own rounding mode.^{: 5-220–5-222}

CPUs with FMA4

• AMD
  • "Heavy Equipment" processors
    • Bulldozer-based processors, October 12, 2011
    • Piledriver-based processors
    • Steamroller-based processors
    • Excavator-based processors (including "v2")
  • Zen: WikiChip's testing shows FMA4 still appears to work (under the conditions of the tests) despite not being officially supported and not even reported by CPUID. This has also been confirmed by Agner Fog. But other tests gave wrong results. AMD Official Web Site FMA4 Support Note ZEN CPUs = AMD ThreadRipper 1900x, R7 Pro 1800, 1700, R5 Pro 1600, 1500, R3 Pro 1300, 1200, R3 2200G, R5 2400G.
• Intel
  • Intel has not released CPUs with support for FMA4.

Excerpt from FMA4