Inference

Overview I have been looking into self-hosting LLMs, and this is my attempt to put everything I’ve learned about the subject in one place (so I can stop forgetting things). Alongside that, I wanted to include information about the setup I use to self-host LLMs on my laptop and the steps I took to build and optimize it. While that will come in the future, as there are still some things I am changing, and this is long enough already, I removed some of those parts to put in the next section. ...