Do you need to ask a question?

• I have searched the existing question and discussions and this question is not already answered.
• I believe this is a legitimate question, not just a bug or feature request.

Your Question

No response

Additional Context

I'm considering switching to a smaller LLM (e.g., an 8B) to reduce the hardware requirement and run it locally.