Intel Neural Compressor Release 3.10

• Highlights
• Features
• Improvements
• Validated Hardware
• Validated Configurations

Highlights

• NVFP4 with E5M3 scaling experimental support for PyTorch
• Selective and mixed quantization support for Keras/JAX

Features

• Support NVFP4 with E5M3 scaling for PyTorch (Experimental)
• Support model-free quantization for PyTorch
• Add dynamic fused MoE bias support for GPT-OSS model on Gaudi
• Add vLLM QDQ plugin for accuracy benchmark based on simulation
• Support for selective and mixed quantization in Keras/JAX, allowing different model components to remain unchanged or use different quantization methods and configurations, including static or dynamic quantization, different data types, and other quantization options
• Support for quantization per channel for weights for Keras/JAX

Improvements

• Security issue fixes
• SDXL model MXFP8 PTQ example
• Kimi K2.6 MXFP4 PTQ example
• Minimax-M2.7 MXFP4 PTQ example

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th and 6th Gen)
• Intel® Arc™ B-Series Graphics GPU (B60)

Validated Configurations

• Ubuntu 24.04 & Win 11
• Python 3.11, 3.12, 3.13
• PyTorch 2.12, 2.13, 2.14
• JAX 0.10

Notes

• It is recommended to use version v3.10 or later to mitigate code CVEs.

Intel Neural Compressor Release 3.9

• Highlights
• Features
• Improvements
• Validated Hardware
• Validated Configurations

Highlights

• INT8 quantization for ViT model for JAX framework

Features

• Support INT8 quantization for ViT model for Keras/JAX
• Support Out-of-place quantization for Keras/JAX
• Support quantization of BFloat16 models for Keras/JAX
• Support quantization for Conv2D, dot_product_attention for Keras/JAX

Improvements

• Improving performance with constant scales and weights support for Keras/JAX
• Handling uncalibrated model layers for Keras/JAX
• WAN series MXFP8 PTQ example
• DeepSeek V4 series mixed MXFP4/MXFP8 example

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th and 6th Gen)
• Intel® Arc™ B-Series Graphics GPU (B60)

Validated Configurations

• Ubuntu 24.04 & Win 11
• Python 3.11, 3.12, 3.13
• PyTorch 2.10, 2.11, 2.12
• JAX 0.10

Notes

• It is recommended to use version v3.9 or later to mitigate code CVEs.

Intel Neural Compressor Release 3.8

• Highlights
• Features
• Improvements
• Validated Hardware
• Validated Configurations

Highlights

• Introduced new JAX framework experimental support

Features

• Support FP8 quantization for Keras/JAX (experimental)
• Support FP8 KV cache static quantization (experimental)
• Support FP8 Attention static quantization (experimental)

Improvements

• New Gemma3 FP8 PTQ example for Keras/JAX
• New ViT FP8 PTQ example for Keras/JAX
• Llama 3 series MXFP4 / MXFP8 PTQ example with FP8 KV & Attention
• Llama 4 Scout MXFP4 / MXFP8 PTQ example with FP8 KV & Attention
• Qwen3 MXFP4 / MXFP8 PTQ example with FP8 KV & Attention
• DeepSeek R1 MXFP4 / MXFP8 PTQ example with FP8 KV & Attention
• Transformers v5 support
• Removal of deprecated 2.x API

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th and 6th Gen)
• Intel® Arc™ B-Series Graphics GPU (B580 and B60)

Validated Configurations

• Ubuntu 24.04 & Win 11
• Python 3.11, 3.12, 3.13
• PyTorch 2.9, 2.10
• JAX 0.9

Notes

• It is recommended to use version v3.8 or later to mitigate code CVEs.

Intel Neural Compressor Release 3.7

• Highlights
• Features
• Improvements
• Validated Hardware
• Validated Configurations

Highlights

• Introduced NVFP4 quantization experimental support and mixed bits (MXFP4 & MXFP8) autotuning on LLMs

Features

• Support NVFP4 Post-Training Quantization (PTQ) on LLM models (experimental)
• Support mixed bits (MXFP4 & MXFP8) autotuning on LLM models (experimental)
• Support MXFP8 PTQ on video generation diffusion model (experimental)
• Support MXFP4 Quantization-Aware Training (QAT) on LLM models (experimental)

Improvements

• Update of Llama 3 series example for NVFP4 and auto mixed-bits (MXFP4 & MXFP8) PTQ
• New LLM example (DeepSeek R1) for MXFP8, MXFP4, NVFP4 PTQ
• New LLM example (Qwen3-235B ) for MXFP8, MXFP4 PTQ
• New video generation diffusion example (FramePack) for MXFP8 PTQ
• Update of Llama3 example for MXFP4 QAT
• Removal of test purpose benchmarking feature for security consideration

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th, 6th Gen)
• Intel Core Ultra Processors (Series 1 and 2)
• Intel Data Center GPU Max Series (1550)
• Intel® Arc™ B-Series Graphics GPU (B580 and B60)

Validated Configurations

• Ubuntu 24.04 & Win 11
• Python 3.10, 3.11, 3.12, 3.13
• PyTorch/IPEX 2.7, 2.8
• PyTorch 2.9

Intel Neural Compressor Release 3.6

• Highlights
• Features
• Improvements
• Validated Hardware
• Validated Configurations

Highlights

• Introduced new low precision data type quantization experimental support, including MXFP8 and MXFP4

Features

• Support MXFP8 Post-Training Quantization (PTQ) on LLM models (experimental)
• Support MXFP8 PTQ on diffusion models (experimental)
• Support MXFP4 PTQ on LLM models (experimental)
• Support Quantization-Aware Training (QAT) on LLM models (experimental)

Improvements

• New LLM example (Llama 3 series) for MXFP4 / MXFP8 PTQ
• New VLM example (Llama 4 Scout) for MXFP4 PTQ
• New diffusion example (Flux) for MXFP8 PTQ
• New LLM example (Llama 3) for MXFP8 QAT
• Static safe check for evaluation function in 2.x API

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th, 6th Gen)
• Intel Core Ultra Processors (Series 1 and 2)
• Intel Data Center GPU Max Series (1550)
• Intel® Arc™ B-Series Graphics GPU (B580)

Validated Configurations

• Ubuntu 24.04 & Win 11
• Python 3.10, 3.11, 3.12
• PyTorch/IPEX 2.7, 2.8

Intel Neural Compressor Release 3.5

• Highlights
• Features
• Improvements
• Validated Hardware
• Validated Configurations

Highlights

• Aligned Gaudi SW Release 1.22 with the improvements on FP8 and INT4 quantization for Intel® Gaudi® AI accelerator
• Preliminary FP8 Q/DQ quantization support for CPU

Features

• Support FP8 dynamic quant, including Linear, FusedMoE on Gaudi
• Support per-node FP8 scale method configuration on Gaudi
• Enhance the stability of the warmup time optimization (dynamic scale patching)
• Enable FP8 dynamic quantization of DeepSeek V3/R1 model on Gaudi
• Support activation ordering in GPTQ INT4 on Gaudi
• Support FP8 Q/DQ quantization on Gaudi when using Optimum-Habana
• Support FP8 Q/DQ quantization on CPU (experimental)

Improvements

• Improve saving vLLM compatible FP8 model in multi-cards scenario on Gaudi
• Support FP32 Softmax mode in FP8 Fused SDPA on Gaudi
• Support FP8 GaudiFluxPipeline save and load on Gaudi
• Improve saving Hugging Face format checkpoint for Intel CPU/GPU
• New diffusion model examples for INT8 and FP8 quantization

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th, 6th Gen)
• Intel Core Ultra Processors (Series 1 and 2)
• Intel Data Center GPU Max Series (1550)
• Intel® Arc™ B-Series Graphics GPU (B580)

Validated Configurations

• Centos 8.4 & Ubuntu 24.04 & Win 11
• Python 3.9, 3.10, 3.11, 3.12
• PyTorch/IPEX 2.6, 2.7

Intel Neural Compressor Release 3.4

• Highlights
• Features
• Improvements
• Bug Fixes
• Validated Hardware
• Validated Configurations

Highlights

• Aligned Gaudi SW Release 1.21 with the improvements on FP8 and INT4 quantization for Intel® Gaudi® AI accelerator
• INT4 quantization enhancements for Intel CPU/GPU

Features

• Support expert parallelism for Mixtral model on Gaudi
• Enhance multi-cards FP8 model save and load on Gaudi
• Enable static FP8 quantization of DeepSeek V3/R1 model on Gaudi
• Support W4A8 mixed precision on Gaudi (experimental)
• Improve compile time on Gaudi when using FP8 (experimental)

Improvements

• Remove numpy version limit for 3.x PyTorch package

Bug Fixes

• Fix graph compile error when quantizing Llama3.2 11B/90B vision model on Gaudi
• Fix segmentation fault issue in LLama2-70B INT4 model on Intel GPU
• Fix accuracy issue caused by duplicated g_idx update for INT4 model on Intel GPU

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th, 6th Gen)
• Intel Core Ultra Processors (Series 1 and 2)
• Intel Data Center GPU Max Series (1100)
• Intel® Arc™ B-Series Graphics GPU (B580)

Validated Configurations

• Centos 8.4 & Ubuntu 24.04 & Win 11
• Python 3.9, 3.10, 3.11, 3.12
• PyTorch/IPEX 2.4, 2.5, 2.6

Intel Neural Compressor Release 3.3

• Highlights
• Features
• Improvements
• Bug Fixes
• Validated Hardware
• Validated Configurations

Highlights

• Aligned Gaudi SW Release 1.20 with the improvements on FP8 and INT4 quantization for Intel® Gaudi® AI accelerator
• VLM INT4 weight-only quantization support in transformers-like API on Intel CPU/GPU

Features

• Saving vLLM compatible FP8 model on Gaudi
• FP8 Per-channel Q/DQ and GC integration on Gaudi
• FP8 quantization for mixture of experts (MoE) module on Gaudi
• Saving Hugging Face compatible weight-only INT4 format on Gaudi
• VLM quantization with AutoRound in transformers-like API on Intel CPU/GPU
• Accuracy-aware tuning on PT2E including mixed precision support

Improvements

• FP8 multi-device (Gaudi & GPU) infrastructure support
• Support scaler scale save on Gaudi

Bug Fixes

• Fix incorrect hf_device_map setting for Transformers-like API
• Fix missing IPEX CPU dependency in Transformers-like API example
• Fix device mapping issue found in GPTQ on Llama model
• Fix saving issue in weight-only per-channel quantization

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th, 6th Gen)
• Intel Core Ultra Processors (Series 1 and 2)
• Intel Data Center GPU Max Series (1100)
• Intel® Arc™ B-Series Graphics GPU (B580)

Validated Configurations

• Centos 8.4 & Ubuntu 24.04 & Win 11
• Python 3.9, 3.10, 3.11, 3.12
• PyTorch/IPEX 2.3, 2.4, 2.5

Intel Neural Compressor Release 3.2

• Highlights
• Features
• Improvements
• Bug Fixes
• Validated Hardware
• Validated Configurations

Highlights

• Aligned with Habana 1.19 release with the improvements on FP8 and INT4 quantization for Intel® Gaudi® AI accelerator
• INT4 weight-only quantization on Intel® Arc™ B-Series Graphics GPU (code-named BattleMage)

Features

• Saving and loading FP8 checkpoint on Gaudi
• Loading vLLM/llm-compressor compatible FP8 checkpoint on Gaudi
• Arbitrary scale method support on Gaudi
• AutoRound INT4 weight-only quantization on Gaudi
• Block-wise calibration for LLM on Gaudi
• INT4 weight-only quantization on BattleMage

Improvements

• Improve FP8 performance by setting scale as scalar tensor on Gaudi
• Integrate AutoRound 0.4.2 with VLM quantization improvements
• Improve safetensors loading for layer-wise quantization in Transformers-like API
• Improve non-contiguous weight saving in Transformers-like API

Bug Fixes

• Fix layer-wise quantization issue in GPTQ on client GPU
• Fix glm-4-9b model out-of-memory issue on BattleMage

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th, 6th Gen)
• Intel Core Ultra Processors (Series 1 and 2)
• Intel Data Center GPU Max Series (1100)
• Intel Arc B-Series Graphics GPU (B580)

Validated Configurations

• Centos 8.4 & Ubuntu 22.04 & Win 11
• Python 3.9, 3.10, 3.11, 3.12
• PyTorch/IPEX 2.3, 2.4, 2.5

Intel Neural Compressor Release 3.1

• Highlights
• Features
• Improvements
• Validated Hardware
• Validated Configurations

Highlights

• Aligned with Habana 1.18 release with the improvements on FP8 and INT4 quantization for Intel® Gaudi® AI accelerator
• Provided Transformer-like quantization API for weight-only quantization on LLM, which offers transformer-based user one-stop experience for quantization & inference with IPEX on Intel GPU and CPU.

Features

• Add Transformer-like quantization API for weight-only quantization on LLM
• Support fast quantization with light weight recipe and layer-wise approach on Intel AI PC
• Support INT4 quantization of Visual Language Model (VLM), like Llava, Phi-3-vision, Qwen-VL with AutoRound algorithm

Improvements

• Support AWQ format INT4 model loading and converting for IPEX inference in Transformer-like API
• Enable auto-round format export for INT4 model
• Support per-channel INT8 Post Training Quantization for PT2E

Validated Hardware 

• Intel Gaudi Al Accelerators (Gaudi 2 and 3)
• Intel Xeon Scalable processor (4th, 5th, 6th Gen)
• Intel Core Ultra Processors (Series 1 and 2)
• Intel Data Center GPU Max Series (1100)

Validated Configurations

• Centos 8.4 & Ubuntu 22.04 & Win 11
• Python 3.9, 3.10, 3.11, 3.12
• PyTorch/IPEX 2.2, 2.3, 2.4