![Integration of the multi-head QNN block into Qwen3-30B-A3B, a 48-layer mixture-of-experts (MoE) model with hidden dimension D = 2048. Left: a standard transformer block, consisting of self-attention preceded by RMSNorm [39] and a sparse MoE feed-forward network that routes each token to 8 of 128 experts, each wrapped in a residual connection. Right: the multi-head QNN block that replaces it at the target layer. The activation x is mixed by the constrained linear transform M1 = Λ2HΛ1, bounded by a tanh nonlinearity, distributed across k = 32 independent 8-qubit QNN heads acting on disjoint 64-dimensional subspaces, and recombined by a second constrained transform M2. The block output is scaled by α and returned to the residual stream. All remaining blocks of the base model are left untouched.](https://pbs.twimg.com/media/HXVTEWcXIAAQeWM.jpg?name=small&format=webp)
ALT Integration of the multi-head QNN block into Qwen3-30B-A3B, a 48-layer mixture-of-experts (MoE) model with hidden dimension D = 2048. Left: a standard transformer block, consisting of self-attention preceded by RMSNorm [39] and a sparse MoE feed-forward network that routes each token to 8 of 128 experts, each wrapped in a residual connection. Right: the multi-head QNN block that replaces it at the target layer. The activation x is mixed by the constrained linear transform M1 = Λ2HΛ1, bounded by a tanh nonlinearity, distributed across k = 32 independent 8-qubit QNN heads acting on disjoint 64-dimensional subspaces, and recombined by a second constrained transform M2. The block output is scaled by α and returned to the residual stream. All remaining blocks of the base model are left untouched.









