This page is a reference table for resources I’m keeping tabs on.
That grand, spacious (and long ignored) branch of computing that, these days, deals not just with thinking machines (so-called “hard” AI), but mostly with enhancing the usefulness of machines in general by exploring mechanisms to express and manipulate harvested knowledge.
Or just pretty pictures and weird chatbots. Either way, it seems we’re currently between AI winters.
Resources
Field | Category | Date | Link | Notes |
---|---|---|---|---|
General | Examples | 2024-03 | flash-attention-minimal | a minimal implementation of Flash Attention |
Frameworks | 2023-12 | MLX | An array framework for Apple silicon |
|
mlx-examples | MLX examples |
|||
2023-05 | marvin | A generic wrapper for various AI APIs |
||
Jupyter | 2023-04 | jupyter-ai | an official Jupyter plugin that can handle multiple AI back-ends (although it seems less flexible than the others right now) |
|
Libraries | 2023-06 | unstructured | a library for handling and segmenting unstructured data of various kinds, from text to common file formats |
|
2023-03 | ml-ane-transformers | Apple’s transformers library, optimized for the Neural Engine |
||
2009-06 | Alchemy | A toolkit providing a series of algorithms for statistical relational learning and probabilistic logic inference, based on Markov logic representation |
||
Techniques | 2022-12 | awesome-chatgpt-prompts | might be a short-lived resource, but an interesting one |
|
Tools | 2024-03 | upscayl | an AI-based image upscaler |
|
2024-01 | pico-tflmicro | a port of TensorFlow Lite Micro to the Raspberry Pi Pico |
||
2023-05 | explainerdashboard | a web app that explains the workings of a (scikit-learn compatible) machine learning model |
||
2023-04 | basaran | An Open-Source alternative to the OpenAI text completion API, with a compatible streaming API for privately hosted models. |
||
2023-03 | NVIDIA Triton Inference Server | A high-performance inference server |
||
2022-12 | Project Bumblebee | Pre-trained and transformer neural models in Elixir. |
||
Generative Audio | models | 2023-04 | bark | a text-prompted genereative audio model |
Large Language Models | Applications | 2023-07 | Chie | A cross-platform dekstop application with chat history and extension support |
Copilots | 2023-06 | Obsidian Copilot | an interesting take on how to use semantic search and OpenSearch’s BM25 implementation |
|
Demos | 2024-01 | WhisperFusion | an ensemble setup with WhisperSpeech, WhisperLive and Phi |
|
Frameworks | 2023-11 | Tanuki | yet another LLM framework using decorators for data validation |
|
2023-07 | litellm | a simple, lightweight LLM wrapper |
||
AutoChain | Yet another alternative to langchain |
|||
griptape | a |
|||
llama_index | a data framework for LLM applications |
|||
txtai | has spinoffs for chat, workflows for medical/scientific papers, semantic search for developers and semantic search for headlines and story text |
|||
llmflows | Yet another alternative to langchain, but with an interesting approach at defining workflows |
|||
2023-05 | langchain | a composable approach for building LLM applications |
||
guidance | Control modern language models more effectively and efficiently than traditional prompting or chaining. |
|||
Front-Ends | 2024-01 | jan | an open-source ChatGPT alternative that runs 100% offline (uses nitro) |
|
2023-12 | SecureAI-Tools | a self-hosted local inference front-end for chatting with document collections |
||
gpt4all | another self-hosted local inference front-end |
|||
Jupyter | 2023-04 | LLMBook | A VS Code notebook interface for LLMs |
|
jupytee | a Jupyter plugin that can handle code generation and image generation, but not switching models (GPT-4) |
|||
genai | a Jupyter plugin that can handle code generation and fixes based on tracebacks |
|||
ipython-gpt | a Jupyter plugin that can handle multiple models |
|||
Libraries | 2024-02 | DataDreamer | library for prompting, synthetic data generation, and training workflows |
|
2023-10 | MemGPT | a memory management/summarization technique for unbounded context |
||
2023-09 | instructor | a clever library that simplifies invoking OpenAI function calls |
||
2023-06 | simpleaichat | A simple wrapper for the ChatGPT AI |
||
2023-05 | guardrails | a package for validating and correcting the outputs of large language models |
||
Models | 2024-04 | functionary | can interpret and execute functions/plugins |
|
2024-03 | Octopus-v2 | a model designed for both function calling and on-device inference |
||
2024-01 | TinyLlama | pretraining of a 1.1B Llama model on 3 trillion tokens. |
||
2023-12 | ml-ferret | a multi-modal model from Apple |
||
2023-04 | turbopilot | a GitHub CoPilot replacement that can run locally (CPU only) |
||
Reference | 2023-12 | Native JSON Output from GPT-4 | tips on how to use OpenAI JSON and function calling |
|
GPT Prompt Archive | A set of sample base prompts for various LLMs |
|||
2023-03 | Using LLaMA with M1 Mac | Manual instructions for Apple Silicon |
||
Resources | 2023-12 | Prompt Engineering Guide | a set of lecture notes and detailed examples of prompting techniques |
|
promptbase | Another set of prompting techniques and detailed examples |
|||
2023-04 | awesome-decentralized-llm | a collection of LLM resources that operate independently |
||
Samples | 2024-01 | SimpleTinyLlama | a simple PyTorch-based implementation |
|
devlooper | a program synthesis agent that autonomously fixes its output by running tests |
|||
2023-12 | gpt-researcher | an agent that does online research on any given topic |
||
LibreChat | A self-hosted ChatGPT alternative |
|||
sharepoint-indexing-azure-cognitive-search | provides an example of how to use Graph navigation and Cognitive Search indexing |
|||
gpt4all | open-source LLM chatbots |
|||
GPT in 60 Lines of NumPy | a tutorial on how to build a GPT model from scratch |
|||
Bash One-Liners for LLMs | a collection of one-liners for various LLMs |
|||
2023-11 | David Attenborough narrates your life | A pretty hilarious image-to-description example |
||
Demystifying Advanced RAG Pipelines | An LLM-powered advanced RAG pipeline built from scratch |
|||
Wanderlust OpenAI example using Solara | A simple interactive web shell with some nice features |
|||
Tools | 2024-04 | burr | a tool for creating and managing LLM workflows |
|
llm.c | LLM training in simple, raw C/CUDA |
|||
lida | automatic generation of visualizations and infographics |
|||
LLocalSearch | a local tool for searching using LLMs |
|||
open-webui | a web-based interface for LLMs |
|||
storm | a tool that researches a topic and generates a full-length report with citations |
|||
2024-03 | gpt-pilot | a prototype development tool that leverages GPT |
||
R2R | a framework for or rapid development and deployment of production-ready RAG systems with SQLite support |
|||
local-image-gen | A GPTScript tool to generate images |
|||
GPTScript | Natural Language Programming against multiple LLMs |
|||
ipex-llm | a PyTorch extension for Intel hardware |
|||
2024-02 | geppetto | a bot for integrating ChatGPT and DALL-E into Slack |
||
GPTFast | a set of acceleration techniques |
|||
notesollama | a plugin for Apple Notes that uses the Accessibility APIs |
|||
llm-ls | a local language server that leverages LLMs |
|||
llm-vscode | a VSCode extension that uses llm-ls |
|||
crewAI | a framework for orchestrating autonomous AI agents |
|||
llmware | a framework for developing LLM-based applications including Retrieval Augmented Generation |
|||
hqq | an implementation of Half-Quadratic Quantization (HQQ) |
|||
reor | a note taking tool that performs RAG using a local LLM |
|||
NeuralFlow | a Python script for plotting the intermediate layer outputs of Mistral 7B |
|||
2024-01 | ollama-bot | a rudimentary IRC bot that communicates with a local instance of ollama |
||
koboldcpp | nn easy-to-use AI text-generation software for GGML and GGUF models based on llama.cpp |
|||
gguf-tools | a set of tools for manipulating GGUF format files |
|||
nitro | a self-hosted inference engine for edge computing with an OpenAI API |
|||
emacs-copilot | an Emacs extension for using a local LLM |
|||
nlm-ingestor | a set of parsers for common file formats |
|||
lorax | a framework that allows users to serve thousands of fine-tuned models on a single GPU |
|||
privy | An open-source alternative to GitHub copilot that runs locally. |
|||
2023-12 | microagents | an interesting experiment on self-editing agents |
||
BricksLLM | an OpenAI gateway in Go to create API keys with rate limits, cost limits and TTLs |
|||
macOSpilot-ai-assistant | An Electron app for macOS |
|||
TinyChatEngine | A local (edge) inference engine in C++ without any dependencies |
|||
LocalAI | A local, drop-in replacement for the OpenAI API |
|||
Serve | A containerized solution for using local LLMs via web chat |
|||
wyGPT | another C++ local inference tool |
|||
2023-10 | localpilot | a MITM proxy that lets you use the GitHub Copilot extension with other LLMs |
||
2023-09 | embedchain | another framework to create bots from existing datasets |
||
llm_agents | a simplified agent framework (doesn’t use OpenAI functions) |
|||
2023-08 | pykoi | a unified interface for data and feedback collection, including model comparisons |
||
PromptTools | self-hostable toools for evaluating LLMs, vector databases, and prompts |
|||
2023-07 | a1gpt | A C++ implementation of a GPT-2 inference engine |
||
khoj | an intriguing personal assistant based on local data |
|||
promptfoo | A tool for testing and evaluating LLM prompt quality. |
|||
2023-06 | SuperAGI | another AutoGPT-like harness for building GPT agents |
||
2023-05 | ChainForge | a visual programming environment for benchmarking prompts across multiple LLMs |
||
langflow | a node-based GUI for quick iteration of langchain flows |
|||
2023-04 | Auto-GPT | an attempt to provide ChatGPT with a degree of autonomy |
||
2023-03 | dalai | An automated installer for LLaMA |
||
llama.cpp | A C++ port of Facebook’s LLaMA model. Still requires roughly 240GB of (unoptimized) weights, but can run on a 64GB Mac. |
|||
minillm | A GPU-focused Python wrapper for LLaMa |
|||
simple-llama-finetuner | A way to do LoRA adaptation of LLaMa |
|||
chatbot-ui | a more or less sensibly designed self-hosted ChatGPT UI |
|||
content-chatbot | A way to quickly create custom embeddings off a web site |
|||
chatblade | a CLI wrapper for ChatGPT |
|||
GPTQ-for-LLaMa | a way to quantize the LLaMA weights to 4-bit precision |
|||
llama-rs | A Rust port of llama.cpp |
|||
alpaca-lora | Another way to do LoRA adaptation of LLaMa |
|||
Vector Databases | 2023-08 | vectordb | A simple vector database that can run in-process |
|
marqo | A vector database that performs vector generation internally |
|||
2023-07 | chroma | an embedding database |
||
USearch | A Single-File Vector Search Engine |
|||
Workflows | danswer | a pretty complete GPT/search integration solution with GitHub, Slack and Confluence/JIRA connectors |
||
Multi-modal Models | Samples | 2024-02 | ml-mgie | instruction-based image self-editing |
NeRFs | Tools | 2022-12 | nerfstudio | A tool for manipulating Neural Radiance Fields (NeRF) and rendering the scenes out as video |
Speech Recognition | Models | 2024-01 | WhisperLive | a real-time text-to-speech system based on Whisper |
2023-11 | distil-whisper | a distilled version of whisper that is 6 times faster |
||
2022-12 | whisper | a general purpose speech recognition model |
||
2022-02 | whisper.cpp | a C++ implementation of whisper that can run in consumer hardware |
||
Tools | 2023-12 | insanely-fast-whisper | An opinionated CLI for audio transcription |
|
Speech Synthesis | Models | 2024-01 | Real-Time-Voice-Cloning | a PyTorch implementation of a voice cloning model |
WhisperSpeech | a text-to-speech system built by inverting Whisper |
|||
2023-12 | StyleTTS2 | A text to speech model that supports style diffusion |
||
Voice Cloning | 2024-03 | OpenVoice | a minimal sampling approach |
|
Stable Diffusion | Apps | 2023-03 | swift-coreml-diffusers | Hugging face’s own app, using Swift and CoreML for Apple Silicon |
2022-11 | Draw Things | Pre-packaged app for iOS, downloads and allows re-use of .ckpt files. |
||
DiffusionBee | Pre-packaged app for macOS (M1 and Intel) |
|||
CGI | 2023-03 | Blender-ControlNet | A Blender plugin to generate ControlNet inputs for posing figures |
|
2022-12 | dream-textures | A Blender plugin for texturing models based on a text description. |
||
Implementations | 2023-10 | OnnxStream | Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2 (or in 298MB of RAM) |
|
Libraries | 2024-01 | sd4j | a Java library for Stable Diffusion that uses ONNX |
|
Models | 2023-03 | Upscale Model Database | Too wide a choice, perhaps |
|
2022-12 | Fast Stable Diffusion | Another tactic to accelerate inference |
||
CoreML Stable Diffusion | Apple’s optimizations for CoreML |
|||
Reference | 2024-01 | comflowy | a set of reference workflows and documentation for ComfyUI |
|
Tools | 2024-03 | comflowyspace | a ComfyUI desktop wrapper |
|
2023-10 | ComfyUI-AnimateDiff-Evolved | An AnimateDiff integration for ComfyUI |
||
ComfyUI-Manager | A component manager for ComfyUI |
|||
2023-08 | stable-diffusion.cpp | stable diffusion inference on the CPU, in pure C++ |
||
Opendream | A layer-oriented, non-destructive editor |
|||
2023-03 | ComfyUI | pretty impressive node-based UI |
||
InvokeAI | A polished UI |
|||
2022-11 | imaginAIry | Works well on Apple Silicon, pure CLI interface to all SD models. Does not reuse .ckpt files, however, so requires separate disk cache. |
||
2022-08 | Stable Diffusion WebUI | Nearly always the best, bleeding edge WebUI for SD |