Artificial Intelligence

This page is a reference table for resources I’m keeping tabs on.

That grand, spacious (and long ignored) branch of computing that, these days, deals not just with thinking machines (so-called “hard” AI), but mostly with enhancing the usefulness of machines in general by exploring mechanisms to express and manipulate harvested knowledge.

Or just pretty pictures and weird chatbots. Either way, it seems we’re currently between AI winters.

Resources

Field Category Date Link Notes
None 2024-04 llm.c

LLM training in simple, raw C/CUDA

Tools 2024-03 local-image-gen

A GPTScript tool to generate images

GPTScript

Natural Language Programming against multiple LLMs

General Examples flash-attention-minimal

a minimal implementation of Flash Attention

Frameworks 2023-12 MLX

An array framework for Apple silicon

mlx-examples

MLX examples

2023-05 marvin

A generic wrapper for various AI APIs

Jupyter 2023-04 jupyter-ai

an official Jupyter plugin that can handle multiple AI back-ends (although it seems less flexible than the others right now)

Libraries 2023-06 unstructured

a library for handling and segmenting unstructured data of various kinds, from text to common file formats

2023-03 ml-ane-transformers

Apple’s transformers library, optimized for the Neural Engine

2009-06 Alchemy

A toolkit providing a series of algorithms for statistical relational learning and probabilistic logic inference, based on Markov logic representation

Techniques 2022-12 awesome-chatgpt-prompts

might be a short-lived resource, but an interesting one

Tools 2024-03 upscayl

an AI-based image upscaler

2024-01 pico-tflmicro

a port of TensorFlow Lite Micro to the Raspberry Pi Pico

2023-05 explainerdashboard

a web app that explains the workings of a (scikit-learn compatible) machine learning model

2023-04 basaran

An Open-Source alternative to the OpenAI text completion API, with a compatible streaming API for privately hosted models.

2023-03 NVIDIA Triton Inference Server

A high-performance inference server

2022-12 Project Bumblebee

Pre-trained and transformer neural models in Elixir.

Generative Audio models 2023-04 bark

a text-prompted genereative audio model

Large Language Models Applications 2023-07 Chie

A cross-platform dekstop application with chat history and extension support

Copilots 2023-06 Obsidian Copilot

an interesting take on how to use semantic search and OpenSearch’s BM25 implementation

Demos 2024-01 WhisperFusion

an ensemble setup with WhisperSpeech, WhisperLive and Phi

Frameworks 2023-11 Tanuki

yet another LLM framework using decorators for data validation

2023-07 litellm

a simple, lightweight LLM wrapper

AutoChain

Yet another alternative to langchain

griptape

a langchain alternative with slighly better internal coding standards

llama_index

a data framework for LLM applications

txtai

has spinoffs for chat, workflows for medical/scientific papers, semantic search for developers and semantic search for headlines and story text

llmflows

Yet another alternative to langchain, but with an interesting approach at defining workflows

2023-05 langchain

a composable approach for building LLM applications

guidance

Control modern language models more effectively and efficiently than traditional prompting or chaining.

Front-Ends 2024-01 jan

an open-source ChatGPT alternative that runs 100% offline (uses nitro)

2023-12 SecureAI-Tools

a self-hosted local inference front-end for chatting with document collections

gpt4all

another self-hosted local inference front-end

Jupyter 2023-04 LLMBook

A VS Code notebook interface for LLMs

jupytee

a Jupyter plugin that can handle code generation and image generation, but not switching models (GPT-4)

genai

a Jupyter plugin that can handle code generation and fixes based on tracebacks

ipython-gpt

a Jupyter plugin that can handle multiple models

Libraries 2024-02 DataDreamer

library for prompting, synthetic data generation, and training workflows

2023-10 MemGPT

a memory management/summarization technique for unbounded context

2023-09 instructor

a clever library that simplifies invoking OpenAI function calls

2023-06 simpleaichat

A simple wrapper for the ChatGPT AI

2023-05 guardrails

a package for validating and correcting the outputs of large language models

Models 2024-04 functionary

can interpret and execute functions/plugins

2024-03 Octopus-v2

a model designed for both function calling and on-device inference

2024-01 TinyLlama

pretraining of a 1.1B Llama model on 3 trillion tokens.

2023-12 ml-ferret

a multi-modal model from Apple

2023-04 turbopilot

a GitHub CoPilot replacement that can run locally (CPU only)

Reference 2023-12 Native JSON Output from GPT-4

tips on how to use OpenAI JSON and function calling

GPT Prompt Archive

A set of sample base prompts for various LLMs

2023-03 Using LLaMA with M1 Mac

Manual instructions for Apple Silicon

Resources 2023-12 Prompt Engineering Guide

a set of lecture notes and detailed examples of prompting techniques

promptbase

Another set of prompting techniques and detailed examples

2023-04 awesome-decentralized-llm

a collection of LLM resources that operate independently

Samples 2024-01 SimpleTinyLlama

a simple PyTorch-based implementation

devlooper

a program synthesis agent that autonomously fixes its output by running tests

2023-12 gpt-researcher

an agent that does online research on any given topic

LibreChat

A self-hosted ChatGPT alternative

sharepoint-indexing-azure-cognitive-search

provides an example of how to use Graph navigation and Cognitive Search indexing

gpt4all

open-source LLM chatbots

GPT in 60 Lines of NumPy

a tutorial on how to build a GPT model from scratch

Bash One-Liners for LLMs

a collection of one-liners for various LLMs

2023-11 David Attenborough narrates your life

A pretty hilarious image-to-description example

Demystifying Advanced RAG Pipelines

An LLM-powered advanced RAG pipeline built from scratch

Wanderlust OpenAI example using Solara

A simple interactive web shell with some nice features

Tools 2024-04 burr

a tool for creating and managing LLM workflows

LLocalSearch

a local tool for searching using LLMs

2024-03 gpt-pilot

a prototype development tool that leverages GPT

R2R

a framework for or rapid development and deployment of production-ready RAG systems with SQLite support

ipex-llm

a PyTorch extension for Intel hardware

2024-02 geppetto

a bot for integrating ChatGPT and DALL-E into Slack

GPTFast

a set of acceleration techniques

notesollama

a plugin for Apple Notes that uses the Accessibility APIs

llm-ls

a local language server that leverages LLMs

llm-vscode

a VSCode extension that uses llm-ls

crewAI

a framework for orchestrating autonomous AI agents

llmware

a framework for developing LLM-based applications including Retrieval Augmented Generation

hqq

an implementation of Half-Quadratic Quantization (HQQ)

reor

a note taking tool that performs RAG using a local LLM

NeuralFlow

a Python script for plotting the intermediate layer outputs of Mistral 7B

2024-01 ollama-bot

a rudimentary IRC bot that communicates with a local instance of ollama

koboldcpp

nn easy-to-use AI text-generation software for GGML and GGUF models based on llama.cpp

gguf-tools

a set of tools for manipulating GGUF format files

nitro

a self-hosted inference engine for edge computing with an OpenAI API

emacs-copilot

an Emacs extension for using a local LLM

nlm-ingestor

a set of parsers for common file formats

lorax

a framework that allows users to serve thousands of fine-tuned models on a single GPU

privy

An open-source alternative to GitHub copilot that runs locally.

2023-12 microagents

an interesting experiment on self-editing agents

BricksLLM

an OpenAI gateway in Go to create API keys with rate limits, cost limits and TTLs

macOSpilot-ai-assistant

An Electron app for macOS

TinyChatEngine

A local (edge) inference engine in C++ without any dependencies

LocalAI

A local, drop-in replacement for the OpenAI API

Serve

A containerized solution for using local LLMs via web chat

wyGPT

another C++ local inference tool

2023-10 localpilot

a MITM proxy that lets you use the GitHub Copilot extension with other LLMs

2023-09 embedchain

another framework to create bots from existing datasets

llm_agents

a simplified agent framework (doesn’t use OpenAI functions)

2023-08 pykoi

a unified interface for data and feedback collection, including model comparisons

PromptTools

self-hostable toools for evaluating LLMs, vector databases, and prompts

2023-07 a1gpt

A C++ implementation of a GPT-2 inference engine

khoj

an intriguing personal assistant based on local data

promptfoo

A tool for testing and evaluating LLM prompt quality.

2023-06 SuperAGI

another AutoGPT-like harness for building GPT agents

2023-05 ChainForge

a visual programming environment for benchmarking prompts across multiple LLMs

langflow

a node-based GUI for quick iteration of langchain flows

2023-04 Auto-GPT

an attempt to provide ChatGPT with a degree of autonomy

2023-03 dalai

An automated installer for LLaMA

llama.cpp

A C++ port of Facebook’s LLaMA model. Still requires roughly 240GB of (unoptimized) weights, but can run on a 64GB Mac.

minillm

A GPU-focused Python wrapper for LLaMa

simple-llama-finetuner

A way to do LoRA adaptation of LLaMa

chatbot-ui

a more or less sensibly designed self-hosted ChatGPT UI

content-chatbot

A way to quickly create custom embeddings off a web site

chatblade

a CLI wrapper for ChatGPT

GPTQ-for-LLaMa

a way to quantize the LLaMA weights to 4-bit precision

llama-rs

A Rust port of llama.cpp

alpaca-lora

Another way to do LoRA adaptation of LLaMa

Vector Databases 2023-08 vectordb

A simple vector database that can run in-process

marqo

A vector database that performs vector generation internally

2023-07 chroma

an embedding database

USearch

A Single-File Vector Search Engine

Workflows danswer

a pretty complete GPT/search integration solution with GitHub, Slack and Confluence/JIRA connectors

Multi-modal Models Samples 2024-02 ml-mgie

instruction-based image self-editing

NeRFs Tools 2022-12 nerfstudio

A tool for manipulating Neural Radiance Fields (NeRF) and rendering the scenes out as video

Speech Recognition Models 2024-01 WhisperLive

a real-time text-to-speech system based on Whisper

2023-11 distil-whisper

a distilled version of whisper that is 6 times faster

2022-12 whisper

a general purpose speech recognition model

2022-02 whisper.cpp

a C++ implementation of whisper that can run in consumer hardware

Tools 2023-12 insanely-fast-whisper

An opinionated CLI for audio transcription

Speech Synthesis Models 2024-01 Real-Time-Voice-Cloning

a PyTorch implementation of a voice cloning model

WhisperSpeech

a text-to-speech system built by inverting Whisper

2023-12 StyleTTS2

A text to speech model that supports style diffusion

Voice Cloning 2024-03 OpenVoice

a minimal sampling approach

Stable Diffusion Apps 2023-03 swift-coreml-diffusers

Hugging face’s own app, using Swift and CoreML for Apple Silicon

2022-11 Draw Things

Pre-packaged app for iOS, downloads and allows re-use of .ckpt files.

DiffusionBee

Pre-packaged app for macOS (M1 and Intel)

CGI 2023-03 Blender-ControlNet

A Blender plugin to generate ControlNet inputs for posing figures

2022-12 dream-textures

A Blender plugin for texturing models based on a text description.

Implementations 2023-10 OnnxStream

Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2 (or in 298MB of RAM)

Libraries 2024-01 sd4j

a Java library for Stable Diffusion that uses ONNX

Models 2023-03 Upscale Model Database

Too wide a choice, perhaps

2022-12 Fast Stable Diffusion

Another tactic to accelerate inference

CoreML Stable Diffusion

Apple’s optimizations for CoreML

Reference 2024-01 comflowy

a set of reference workflows and documentation for ComfyUI

Tools 2024-03 comflowyspace

a ComfyUI desktop wrapper

2023-10 ComfyUI-AnimateDiff-Evolved

An AnimateDiff integration for ComfyUI

ComfyUI-Manager

A component manager for ComfyUI

2023-08 stable-diffusion.cpp

stable diffusion inference on the CPU, in pure C++

Opendream

A layer-oriented, non-destructive editor

2023-03 ComfyUI

pretty impressive node-based UI

InvokeAI

A polished UI

2022-11 imaginAIry

Works well on Apple Silicon, pure CLI interface to all SD models. Does not reuse .ckpt files, however, so requires separate disk cache.

2022-08 Stable Diffusion WebUI

Nearly always the best, bleeding edge WebUI for SD

This page is referenced in: