Chris Levy - Blog

PDF Q&A App using ColPali, Modal, and FastHTML

Lately, I’ve been following FastHTML from a distance. As someone who sticks to backend Python development, frontend development has always been a bit…

🚀 Building with Modal 🚀

In my professional life I write code which ends up on a production environment, supported by sophisticated DevOps infrastructure. This system leverages a suite of tools such…

LLM Tool Loops with OpenAI and Anthropic

In this post I will be using OpenAI and Anthropic APIs fpr LLM…

Memory Usage for Quantized LLMS

In this blog post I will be writing about some of my high level learnings from a recent LLM conference I took part in. These are some notes for myself so I don’t forget. These concepts are basic and high level but are details I was missing before when thinking about…

Fine-Tuning LLMs with Axolotl on JarvisLabs

I’m currently taking part in the course/conference Mastering LLMs: A Conference For Developers & Data Scientists. It started off as a fine-tuning course and then kind of exploded into a mini conference with numerous speakers and talks on LLMs. Many of the…

Intro to LLMs

Lunch and Learn Talk

Using Modal to Transcribe YouTube Videos with Whisper

I have been following Modal for a while but never used it until last week. I recently wrote a blog post on using Axolotl to fine-tune a decoder LLM for the…

Getting Started with Axolotl for Fine-Tuning LLMs

I have experience fine-tuning smaller encoder style LLMs such as DistilBERT for classification style…

Function Calling with Hermes-2-Pro-Mistral-7B

In this post we take a look at the function calling capabilities of the open source model NousResearch/Hermes-2-Pro-Mistral-7B (interstellarninja et al. (2024))

OpenAI Compatible LLM Inference

A Single Inference Wrapper for OpenAI, Together AI, Hugging Face Inference TGI, Ollama, etc.

Until recently I thought that the openai library was only for connecting to OpenAI endpoints. It was not until I was testing out LLM inference with together.ai that I came across a section in their…

DSPy

DSPy kept popping up on my X timeline and I thought it looked pretty interesting, so I decided to take a few days to look into it. I didn’t get super deep into it yet, but I…

Basic Transformer Architecture Notes

Here are some notes on the basic transformer architecture…