Fix bug with some LoRA variants when applied to bitsandbytes NF4 quantized models#7577

Merged

psychedelicious merged 2 commits intomainfrom ryan/fix-lora-variants-on-bnb-quantized

Jan 21, 2025

+38-2

Commits on Jan 21, 2025

Add unit tests for LoKR patch layers. The new tests trigger a bug when LoKR layers are applied to BnB-quantized layers (also impacts several other LoRA variant types).

RyanJDick
authored and
psychedelicious
committed
Fix bug with some LoRA variants when applied to a BnB NF4 quantized model. Note the previous commit which added a unit test to trigger this bug.

RyanJDick
authored and
psychedelicious
committed