Artificial Intelligence

That grand, spacious (and long ignored) branch of computing that, these days, deals not just with thinking machines (so-called “hard” AI), but mostly with enhancing the usefulness of machines in general by exploring mechanisms to express and manipulate harvested knowledge.

Or just pretty pictures and weird chatbots. Either way, it seems we’re currently between AI winters, with a lot of investment in GPU hardware that still needs to prove itself useful in real life.


Field Category Date Link Notes
Large Language Models 2024-07 korvus

a search SDK that unifies the entire RAG pipeline in a single database query

Tools exo

an intriguing P2P clustering solution for running models across several machines

2024-05 pipecat

yet another LLM agent framework

General Examples 2024-03 flash-attention-minimal

a minimal implementation of Flash Attention

Frameworks 2024-05 tinygrad

A 8000 LOC deep learning framework

2023-12 MLX

An array framework for Apple silicon


MLX examples

2023-05 marvin

A generic wrapper for various AI APIs

Jupyter 2023-04 jupyter-ai

an official Jupyter plugin that can handle multiple AI back-ends (although it seems less flexible than the others right now)

Libraries 2023-06 unstructured

a library for handling and segmenting unstructured data of various kinds, from text to common file formats

2023-03 ml-ane-transformers

Apple’s transformers library, optimized for the Neural Engine

2009-06 Alchemy

A toolkit providing a series of algorithms for statistical relational learning and probabilistic logic inference, based on Markov logic representation

Techniques 2022-12 awesome-chatgpt-prompts

might be a short-lived resource, but an interesting one

Tools 2024-03 upscayl

an AI-based image upscaler

2024-01 pico-tflmicro

a port of TensorFlow Lite Micro to the Raspberry Pi Pico

2023-05 explainerdashboard

a web app that explains the workings of a (scikit-learn compatible) machine learning model

2023-04 basaran

An Open-Source alternative to the OpenAI text completion API, with a compatible streaming API for privately hosted models.

2023-03 NVIDIA Triton Inference Server

A high-performance inference server

2022-12 Project Bumblebee

Pre-trained and transformer neural models in Elixir.

Generative Audio models 2023-04 bark

a text-prompted genereative audio model

Large Language Models Applications 2023-07 Chie

A cross-platform dekstop application with chat history and extension support

Copilots 2023-06 Obsidian Copilot

an interesting take on how to use semantic search and OpenSearch’s BM25 implementation

Demos 2024-01 WhisperFusion

an ensemble setup with WhisperSpeech, WhisperLive and Phi

Frameworks 2023-11 Tanuki

yet another LLM framework using decorators for data validation

2023-07 litellm

a simple, lightweight LLM wrapper


Yet another alternative to langchain


a langchain alternative with slighly better internal coding standards


a data framework for LLM applications


has spinoffs for chat, workflows for medical/scientific papers, semantic search for developers and semantic search for headlines and story text


Yet another alternative to langchain, but with an interesting approach at defining workflows

2023-05 guidance

Control modern language models more effectively and efficiently than traditional prompting or chaining.


a composable approach for building LLM applications


a componentized approach to building LLM pipelines

Front-Ends 2024-01 jan

an open-source ChatGPT alternative that runs 100% offline (uses nitro)

2023-12 SecureAI-Tools

a self-hosted local inference front-end for chatting with document collections


another self-hosted local inference front-end

Jupyter 2023-04 LLMBook

A VS Code notebook interface for LLMs


a Jupyter plugin that can handle code generation and image generation, but not switching models (GPT-4)


a Jupyter plugin that can handle code generation and fixes based on tracebacks


a Jupyter plugin that can handle multiple models

Libraries 2024-05 radients

a vactorization library that can handle more than just text

2024-04 databonsai

a Python library that uses LLMs to perform data cleaning

2024-02 DataDreamer

library for prompting, synthetic data generation, and training workflows

2023-10 MemGPT

a memory management/summarization technique for unbounded context

2023-09 instructor

a clever library that simplifies invoking OpenAI function calls

2023-06 simpleaichat

A simple wrapper for the ChatGPT AI

2023-05 guardrails

a package for validating and correcting the outputs of large language models

Models 2024-04 functionary

can interpret and execute functions/plugins

2024-03 Octopus-v2

a model designed for both function calling and on-device inference

2024-01 TinyLlama

pretraining of a 1.1B Llama model on 3 trillion tokens.

2023-12 ml-ferret

a multi-modal model from Apple

2023-04 turbopilot

a GitHub CoPilot replacement that can run locally (CPU only)

Reference 2023-12 Native JSON Output from GPT-4

tips on how to use OpenAI JSON and function calling

GPT Prompt Archive

A set of sample base prompts for various LLMs

2023-03 Using LLaMA with M1 Mac

Manual instructions for Apple Silicon

Resources 2023-12 Prompt Engineering Guide

a set of lecture notes and detailed examples of prompting techniques


Another set of prompting techniques and detailed examples

2023-04 awesome-decentralized-llm

a collection of LLM resources that operate independently

Samples 2024-01 SimpleTinyLlama

a simple PyTorch-based implementation


a program synthesis agent that autonomously fixes its output by running tests

2023-12 gpt-researcher

an agent that does online research on any given topic


A self-hosted ChatGPT alternative


provides an example of how to use Graph navigation and Cognitive Search indexing


open-source LLM chatbots

GPT in 60 Lines of NumPy

a tutorial on how to build a GPT model from scratch

Bash One-Liners for LLMs

a collection of one-liners for various LLMs

2023-11 David Attenborough narrates your life

A pretty hilarious image-to-description example

Demystifying Advanced RAG Pipelines

An LLM-powered advanced RAG pipeline built from scratch

Wanderlust OpenAI example using Solara

A simple interactive web shell with some nice features

Tools 2024-07 GraphRAG

a data pipeline designed to pre-process knowledge graphs and perform RAG on them


CLI to interact with LLMs using markdown and images

2024-06 oterm

a terminal-based interface for LLMs

2024-05 pico-cookbook

Recipes for on-device voice AI and local LLM


yet another agent tool, this time using the OpenAI assistant interface


yet another long-running agent tool for complex coding tasks


A device that tells bedtime stories to kids, using chunked TTS

2024-04 burr

a tool for creating and managing LLM workflows


LLM training in simple, raw C/CUDA


an open-source LLM app development platform with a node-based UX


automatic generation of visualizations and infographics


a local tool for searching using LLMs


a web-based interface for LLMs


a tool that researches a topic and generates a full-length report with citations

2024-03 gpt-pilot

a prototype development tool that leverages GPT


a framework for or rapid development and deployment of production-ready RAG systems with SQLite support


A GPTScript tool to generate images


Natural Language Programming against multiple LLMs


a PyTorch extension for Intel hardware

2024-02 geppetto

a bot for integrating ChatGPT and DALL-E into Slack


a set of acceleration techniques


a plugin for Apple Notes that uses the Accessibility APIs


a local language server that leverages LLMs


a VSCode extension that uses llm-ls


a framework for orchestrating autonomous AI agents


a framework for developing LLM-based applications including Retrieval Augmented Generation


an implementation of Half-Quadratic Quantization (HQQ)


a note taking tool that performs RAG using a local LLM


a Python script for plotting the intermediate layer outputs of Mistral 7B

2024-01 ollama-bot

a rudimentary IRC bot that communicates with a local instance of ollama


nn easy-to-use AI text-generation software for GGML and GGUF models based on llama.cpp


a set of tools for manipulating GGUF format files


a self-hosted inference engine for edge computing with an OpenAI API


an Emacs extension for using a local LLM


a set of parsers for common file formats


a framework that allows users to serve thousands of fine-tuned models on a single GPU


An open-source alternative to GitHub copilot that runs locally.

2023-12 microagents

an interesting experiment on self-editing agents


an OpenAI gateway in Go to create API keys with rate limits, cost limits and TTLs


An Electron app for macOS


A local (edge) inference engine in C++ without any dependencies


A local, drop-in replacement for the OpenAI API


A containerized solution for using local LLMs via web chat


another C++ local inference tool

2023-10 localpilot

a MITM proxy that lets you use the GitHub Copilot extension with other LLMs

2023-09 embedchain

another framework to create bots from existing datasets


a simplified agent framework (doesn’t use OpenAI functions)

2023-08 pykoi

a unified interface for data and feedback collection, including model comparisons


self-hostable toools for evaluating LLMs, vector databases, and prompts

2023-07 a1gpt

A C++ implementation of a GPT-2 inference engine


an intriguing personal assistant based on local data


A tool for testing and evaluating LLM prompt quality.

2023-06 SuperAGI

another AutoGPT-like harness for building GPT agents

2023-05 ChainForge

a visual programming environment for benchmarking prompts across multiple LLMs


a node-based GUI for quick iteration of langchain flows

2023-04 Auto-GPT

an attempt to provide ChatGPT with a degree of autonomy

2023-03 dalai

An automated installer for LLaMA


A C++ port of Facebook’s LLaMA model. Still requires roughly 240GB of (unoptimized) weights, but can run on a 64GB Mac.


A GPU-focused Python wrapper for LLaMa


A way to do LoRA adaptation of LLaMa


a more or less sensibly designed self-hosted ChatGPT UI


A way to quickly create custom embeddings off a web site


a CLI wrapper for ChatGPT


a way to quantize the LLaMA weights to 4-bit precision


A Rust port of llama.cpp


Another way to do LoRA adaptation of LLaMa

Vector Databases 2023-08 vectordb

A simple vector database that can run in-process


A vector database that performs vector generation internally

2023-07 chroma

an embedding database


A Single-File Vector Search Engine

Workflows danswer

a pretty complete GPT/search integration solution with GitHub, Slack and Confluence/JIRA connectors

Multi-modal Models Samples 2024-02 ml-mgie

instruction-based image self-editing

NeRFs Tools 2022-12 nerfstudio

A tool for manipulating Neural Radiance Fields (NeRF) and rendering the scenes out as video

Speech Recognition Models 2024-01 WhisperLive

a real-time text-to-speech system based on Whisper

2023-11 distil-whisper

a distilled version of whisper that is 6 times faster

2022-12 whisper

a general purpose speech recognition model

2022-02 whisper.cpp

a C++ implementation of whisper that can run in consumer hardware

Tools 2023-12 insanely-fast-whisper

An opinionated CLI for audio transcription

Speech Synthesis Models 2024-05 ChatTTS

a text-to-speech model designed specifically for dialogue scenarios, with decent prosody

2024-01 Real-Time-Voice-Cloning

a PyTorch implementation of a voice cloning model


a text-to-speech system built by inverting Whisper

2023-12 StyleTTS2

A text to speech model that supports style diffusion

Voice Cloning 2024-03 OpenVoice

a minimal sampling approach

Stable Diffusion Apps 2023-03 swift-coreml-diffusers

Hugging face’s own app, using Swift and CoreML for Apple Silicon

2022-11 Draw Things

Pre-packaged app for iOS, downloads and allows re-use of .ckpt files.


Pre-packaged app for macOS (M1 and Intel)

CGI 2023-03 Blender-ControlNet

A Blender plugin to generate ControlNet inputs for posing figures

2022-12 dream-textures

A Blender plugin for texturing models based on a text description.

Implementations 2023-10 OnnxStream

Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2 (or in 298MB of RAM)

Libraries 2024-01 sd4j

a Java library for Stable Diffusion that uses ONNX

Models 2023-03 Upscale Model Database

Too wide a choice, perhaps

2022-12 Fast Stable Diffusion

Another tactic to accelerate inference

CoreML Stable Diffusion

Apple’s optimizations for CoreML

Reference 2024-01 comflowy

a set of reference workflows and documentation for ComfyUI

Tools 2024-03 comflowyspace

a ComfyUI desktop wrapper

2023-10 ComfyUI-AnimateDiff-Evolved

An AnimateDiff integration for ComfyUI


A component manager for ComfyUI

2023-08 stable-diffusion.cpp

stable diffusion inference on the CPU, in pure C++


A layer-oriented, non-destructive editor

2023-03 ComfyUI

pretty impressive node-based UI


A polished UI

2022-11 imaginAIry

Works well on Apple Silicon, pure CLI interface to all SD models. Does not reuse .ckpt files, however, so requires separate disk cache.

2022-08 Stable Diffusion WebUI

Nearly always the best, bleeding edge WebUI for SD

This page is referenced in: