unicolet.org

Mostly out of curiosity and desire to learn I've tried to run open models locally on both LM studio and ollama, but I quickly realized the limitations intrinsic to my hardware (just a high-spec'd laptop). Curious to try AWS Bedrock I eventually settled on the following setup: litellm exposing Bedrock models (Qwen, atm) locally on an OpenAPI-compatible API (yes it's a mouthful). This works great for any tool that can be configured to use an OpenAPI-compatible API like Quill meetings . Getting VS code to work with this setup was more challenging as it required VS Code Insiders (the bleeding edge, AFAIU) and even in that case VS Code tends to forget settings or use them inconsistently. For example it always uses copilot for the inline code actions. llm  required some tweaking too, in particular the setting suggested in this comment . I am very impressed with litellm which provides accurate usage tracking per team or account. The potential for offering llm access on an interna...

Thoughtworks just published volume 33 of their Technology Radar . I found some interesting gems in it that I thought were worthwhile re-sharing: LiteLLM : I've been playing around with it to share AWS Bedrock models over a local, OpenAI-compatible API and I am impressed with the breadth of features (for example budgeting). The AI ecosystem is vibrant and flourishing. Continuous Compliance : so happy to see this mentioned! Personally I would expand the term to include other compliance tools like Vanta and I am convinced that this kind of automation and software will be essential for organizations to scale while meeting increasing regulatory demands. AGENTS.md : as someone who reads Simon's Willison blog, this is no surprise and a welcome confirmation (another file to watch out for:  CLAUDE.md ). Oxide : I wrote this post almost exclusively to mention Oxide 😅, a company I admire. Whenever people ask me about my cloud exit strategy, my answer is: Oxide. Here's why .