FauxPilot: A decent OSS alternative

I wanted to have Github CoPilot features but on the cheap-side. I already had tried out multiple others code generation models and had an idea, maybe I can write a new backend by copying the request format of the CoPilot plugin, thus using the VS Code plugin as-is. But like always, before actually executing the idea, I started with a Google and a Github search, and stumbled on FauxPilot.

So I started with the setup. For server, simple documentation exists that allowed me to choose codegen-2B-multi  on my RTX 3060 with 12 GB VRAM (which leaves plenty of room for another small experiment to run concurrently). I used an LXC container with GPU mounts to do this, as mentioned in another blog. Thus I'd no worries for whatever the script did, including Nested Docker containers accessing the GPU.

On the client side, documentation was a bit confusing. Initially, I thought of using the github copilot plugin, but I could not get it to work. However I did find another plugin based on this repo. It seemed to work really well out of the box!

Following the demo of working sample. It works decently, but as always, almost no tools can make you a better programmer, mostly make you faster at best.