QQ linear

[nastya236]

QQLinear layer

[awni]

Does it make sense to hijack Module::train() and Module::eval() to update the weights between quantize and not-quantize?

I think at the very least it would be nice to have a method which turns it into the inference version since I expect it would be a common use case.

CC @angeloskath might have some ideas.

[nastya236]

I added QQLinear.train() and QQLinear.eval(). Let me know if you had something different in mind.
For inference, I think we should support two cases: (1) weights are loaded already quantized (with loading handled in mlx-lm, or a similar for cuda), or (2) weights are bf16/fp32, in which case we call .eval() to quantize them.