Will this model support MLX inference?

[neil0306]

Hi,

Thanks for your great work!

Since the M series CPU of Mac now has relatively strong performance, I wonder if this model will support the MLX inference framework for acceleration?

Thanks!

[jonasjunzhang]

Hi! Thank you for your interest!

Since this is an academic project, we prioritize support for GPUs. In addition, processing long videos with an 8B model is highly memory-intensive, which may only be feasible on high-end Apple devices (I currently do not have access to an Apple device with sufficient memory to debug and validate an MLX-based implementation).

Overall, MLX support is unlikely to be added in the short term. We apologize that we cannot provide more support at this moment. Also, any pull requests and contributions are more than welcome!