Python

Python is one of my favorite programming languages due to its terseness and amazing flexibility.

See also Django for more.

My Stuff

Year Item Notes
2016 Sushy

my current wiki engine

2015 rss2imap

The way I used read my news a while back

2009

a native Python PNG creation module, suitable for use in Google App Engine

2007

my previous wiki engine

2006

a quick hack to file web pages as MIME compliant e-mail messages.

2005

a minimalist Apache log parser.

another quick hack that apparently filled a long-standing need.

The Python Grimoire

which I converted into TiddlyWiki format

2004

a minimalist (and dumb) Bonjour / Rendezvous client

which I used daily for a couple of years and occasionally contributed to

native protocol

this spawned numerous variants, including a proxy.

Resources

Category Date Link Notes
Astronomy 2021 Skyfield

An astronomical library for Python

Audio 2024 pedalboard

a library for audio processing

Authentication 2021 aioauth

Oauth 2.0 framework and provider for asyncio

2023 authlib

OAuth, OpenID Connect and other niceties

CAD sdf

Generate 3D meshes based on SDFs (signed distance functions)

sdf (Gitlab)

Another version of the sdf library with additional functionality

2024 build123d

a parametric BREP modeling framework for 2D and 3D CAD built on OpenCascade

CLI click

A very nice way to do composable CLI tools (see also argh, defopt, docopt, python-fire, etc.)

Clint

A great module for developing CLI apps

pbs

A nice, simple and consistent generic command wrapper.

ptpython

The Python Prompt Toolkit

plumbum

Another command wrapper, with SSH support.

2018 python-fire

a CLI wrapper for any Python object

2019 tqdm

a configurable progress bar

2020 pyp

a clever shell tool to do inline Python

typer

A click-like library for building CLI applications

2021 pqdm

parallel tqdm

CMS Wagtail

Another Django-based CMS framework

wid.gy

A Django-based CMS framework

Caching 2021 diskcache

a Django-compatible caching library

Cloud 2020 pyinfra

a fabric-like agentless automation tool

Compilers Transcrypt

Another JavaScript transpiler

2021 mypyc

a compiler for a subset of type-annotated Python

2022 codon

an LLVM compiler for a Python subset

Compression 2021 zipfly

a library for doing on-the-fly zip compression with low memory footprint

Concurrency OpenReplica

A coordination service that provides replication and synchronization support for large-scale distributed systems

python-rq

A simpler queueing library

Pulsar

Another actor-based multiprocessing library with asyncio support and a fair degree of network transparency

Celery

My current go-to library for task queueing

Pykka

An implementation of the Actor model, with gevent support

offset

ports of the Go concurrency model

aioprocessing

A set of asyncio equivalents of the multiprocessing library

ray

A distributed execution engine that implements a pandas clone

goless

ports of the Go concurrency model

Littleworkers

Parallel job management without the hassles.

2020 aiomultiprocess

an asyncio Pool implementation to manage multiple processes with independent event loops

Trio

a friendlier wrapper around asyncio for concurrent tasks

2023 mpire

a multiprocessing replacement with a number of additional features

Console bpython

My favorite REPL

Colorama

Making ANSI color sequences work cross-platform (see also this gist)

fabulous

Prints images to console

urwid

A curses based library for building console UIs

2020 rich

a console library with various layout options

Data Handling Happy

A Hadoop library to run map-reduce jobs via Jython

biplist

A library that can manage Mac binary .plist files

pyparsing

a handy, flexible parser library that does away with lex/yacc.

snakebite

A native HDFS client to access the Hadoop datastore.

xlwings

An Excel plugin

Bounter

A fast probabilistic counter

voluptuous

A data schema validation library.

fuzzywuzzy

A fuzzy string matching library

PyCascading

A better way to write Hadoop jobs.

xlwt

a library to parse Excel documents (cheatsheet.

2020 Little Ball of Fur

a graph sampling library that works with NetworkX

riptable

a multithreaded array processing library

Data Science SciPy

scientific (including plotting) libraries for Python

2020 TextHero

an NLP library for large-scale text processing

papermill

a way to parameterize and automate Jupyter notebooks

2021 Kats

a time series analysis library

euporie

a text-based interface for Jupyter notebooks

xeus-sql

a Jupyter kernel for SQL databases

eztable

a simple table library

pyxelate

a pixelation/dithering algorithm that uses a mix of algorithmic and ML approaches

2023 pygwalker

a Tableau-like dataframe exploration tool for Jupyter

datapane

a data and reporting and analisys framework

fugue

a unified interface for executing SQL, Python and Pandas code on Spark, Dask and Ray

2024 cuDF

a GPU DataFrame library leveraging CUDA

Data Validation Schematics

A validation/schema handling library.

2021 schema

another data validation library

pydantic

Data validation using type hinting

Database pymssql

SQL Server and Sybase driver

FreeTDS
SQLAlchemy

tutorial

Peewee

A small, very flexible ORM

tinydb

A (very) lightweight document oriented database

Goatfish

An even smaller schemaless ORM

2020 DuckDB

an embeddable SQL database with OLAP features

montydb

a tiny MongoDB implementation in pure Python

simple-graph

A simple graph database in SQLite

2021 pottery

a Redis library with Pythonic syntactic sugar

dataset

trivial abstraction layer over SQLAlchemy

Mongita

a MongoDB-like database library for in-process use

2023 mongolite

another MongoDB-like database

Debugging Eww

Insert a REPL into a running application

Pycallgraph

Generate a program flow graph Graphviz

Hypothesis

A very nice testing framework

pudb

A console debugger

lptrace

A pretty amazing single-file strace-like tool for Python

assertEquals

a testing framework with a great screncast

2020 Scalene

A high-performance CPU and memory profiler

PrettyErrors

an error handler that prettifies output

Electronics 2021 pinout

Helps create SVG pin-out diagrams for electronic hardware

2023 belay

a library and helper for developing MicroPython projects, including an on-device REPL

Enterprise toil

A cross-platform pipeline management system with Common Workflow Language support

Environments 2020 snakeware

a Python userspace for Linux

Functional Programming Hy

An awesome LISP dialect that sits atop the runtime.

funcy

A nice, fairly complete functional library

Toolz

A functional complement to the standard library.

pyrsistent

A persistent data structure library

GIS geopy

an amazingly flexible geocoding library.

reverse-geocoder

A fast offline reverse geocoder

2021 prettymaps

A minimal Python library to draw customized maps from OpenStreetMap data

GUI kivy

A sophisticated UI/UX library with Android support

PythonCard

A bit basic, but interesting.

VPython

3D OpenGL libraries for quick visualization

winGuiAuto.py

for Windows automation

pyglet

a cross-platform windowing and multimedia library with OpenGL support

PySide

A recent Qt binding (up to and including 4.7) with QML support

PySimpleGUI

a simple tkinter-based GUI wrapper

2018 Toga

a cross-platform GUI that works on the Mac

2019 fman build system

a complete build system for Qt applications

2020 Gooey

a wxWidgets wrapper for quickly building GUI versions of CLI tools

DearPyGui

a Linux/Windows GUI library atop ImGUI (C++)

2021 encodermenu

a MicroPython library for GUI creation, using uasyncio

2022 flet

creates realtime web, mobile and desktop apps using Flutter

2024 mesop

a rapid web development framework for Python

nicegui

a library for creating Web UIs

Graphics smc.freeimage

A wrapper for the freeimage library that can handle various TIFF and fax formats as well as ICC color profiles

Graphite

a real-time graphing system similar to RRDTool

Skip's Python Bits

lots of useful snippets

Pygal

An SVG charting library with a variety of built-in formats.

Pyx

Oldie, but goodie.

pygame

a game-oriented library with SDL support

pyprocessing

A Processing clone in Python.

Vapory

A library to generate photorealistic renderings using POV-Ray

drawille

Uses Braille characters for TTY charting

Cropmap

Face-aware image cropping.

2019 douglas-quaid

a library for image matching and correlation

imagededup

uses several different algorithms (including NNs) to detect duplicate images

plottyprint

printable formatting for matplotlib

2020 blurhash

A way to encode image placeholders as strings

hiplot

high dimensional interactive plotting

diagrams

a library for drawing cloud architecture diagrams (Azure, AWS, k8s, etc.)

2021 image-blender

a Cython implementation of Adobe Photoshop blend modes

2022 taichi

a high-performance graphics library

2023 pyxel

A retro game engine for Python

2024 renderer

rendering and rasterizing library for STL

pyrender

glTF 2.0-compliant OpenGL renderer for visualization of 3D scenes

pygfx

render engine targeting Vulkan/Metal/DX12

HTML lxml

a different libxml binding, with excellent HTML parsing performance

cssutils

a CSS parser

Gnosis Utilities

all sorts of XML goodies

pygments

syntax highlighter able to handle multiple nested languages

pyquery

A jQuery-like library for manipulating documents.

BeautifulSoup

The most powerful HTML parser out there.

squeezeit

A CSS and JavaScript bundler/minifier

Interpreters 2023 pocketpy

an embeddable C-based interpreter in 5000 LOC

PyScript

a WASM runtime using Pyodide for web embedding

Large Language Models pykoi

a unified interface for data and feedback collection, comparing models, and doing basic RAG

Libraries 2020 minotaur

an asynchronous inotify interface

minibatch

a simple stream processing library

2021 Sorted Containers

a fast C-backed collections library

seedir

A utility library to render filesystem tree diagrams

2022 slack-machine

A Slack bot library that uses socket mode and asyncio

mintotp

a tiny TOTP library

2023 fastkafka

A Kafka library inspired by fastapi

reader

A modern RSS/Atom/JSON feed reader library.

pgvector-python

for using Postgres as a vector database

2024 unsafe-python

A library to assist writing memory-unsafe code, without any imports

django-organizations

a Django library for managing organization accounts

BM25S

an ultrafast implementation of BM25 in pure Python

Machine Learning discomll

A ML framework for Disco

Tensorflow

Usual staple, largely superseded by PyTorch

2020 netron

a viewer for neural network, deep learning and machine learning models

2021 Perceptilabs

free visual ML model creation tool

Mail 2018 aiosmtpd

an asyncio version of smtpd.py

Salmon

an SMTP mail server

slimta

a mail app framework for building MTAs

2021 imapautofiler

an IMAP mailbox sorter that I unwittingly replicated (with less features)

Messaging 2023 Propan

a declarative Python MQ framework inspired by fastapi

Music 2020 pippi

an audio processing/music library

pymidi

a library for building MIDI over IP/RTP applications (compatible with Apple specs)

2023 musicpy

A domain-specific language for writing music

Networking PyRendezvous

neat little Bonjour module I use a lot

telnetscript

a simple module to do scripted telnet sessions.

IPy

an IP address manipulation library.

pyvnc

A ctypes interface to the VNC libraries

ircutils

A relatively modern async IRC framework

Eventlet

Amazing event-oriented framework

ftputil

a high-level interface to the ftplib module

Python smtplib progress indication

Might come in useful some day.

gunicorn

A WSGI server that supports a number of different worker daemons and is highly customizable.

dpkt

packet creation and parsers

Pcapy

a packet manipulation library

uwsgi

A C application container that speaks WSGI and is seriously good.

PuSSH

SSH wrapper

RPyC

an awesome RPC library

SleekXMPP

A more modern XMPP library

Tftpy

a pure Python TFTP library, useful for implementing UDP file transfer testbeds.

ThriftPy

A pure Python implementation of Apache Thrift

pyxmpp2

another one, probably more interesting.

pyrobuf

a Cython implementation of Google’s protobuf library

pyapns

An Apple Push Notification Service provider based on Twisted.

2018 asphalt

An asyncio framework for network oriented applications, with a considerable number of modules

2019 opendrop

a partial implementation of AirDrop

2023 p2pd

an async networking library with P2P and UDP support

OCR 2024 surya

OCR library supporting 90+ languages

PDF Kiva

a Display PDF library

PyPdf

a PDF toolkit

WeasyPrint

Alternatives to PrinceXML for generating formated PDF documentation.

2020 camelot

a library to extract tabular data from PDFs

2021 borb

a library for creating and manipulating PDF files

2023 peepdf

a Python tool to explore PDF files in order to find out if the file can be harmful or not.

2024 marker

a PDF to markdown converter optimized for books and scientific papers, using surya

Packaging 2019 Poetry

a dependency management and packaging tool

2021 PyOxidizer

a better way to distribute Python bundled apps than pyinstaller

shiv

builds self-contained applications for easy deployment

Presentations pydown

An HTML presentation generator

Reference 2023 python-cheatsheet

a modern reference, useful for kids

Reporting Relatorio

A very neat reporting library with multiple output formats

ReportLab Toolkit

a pure Python PDF library that includes a presentation tool.

Resources 2023 python-package-template

a template project for building packages

2024 awesome-micropython

another curated list of MicroPython resources

awesome-micropython

a curated list of MicroPython resources

Robotics kabuki

A declarative control library for MicroPython

Runtimes Nuitka

An interesting batch compiler.

Pyjamas

Some interesting twists on using Python on the way to JavaScript.

Brython

a fairly complete in-browser runtime

Skulpt

Some interesting twists on using Python on the way to JavaScript.

tinypy

Python in 64K of code.

Js2Py

A JavaScript interpreter written in pure Python

Hope

A specialized JIT compiler

2020 Pycopy

a fork of MicroPython

2021 Pyjion

Pyjion, a JIT extension for CPython that compiles your Python code into native CIL and executes it using the .NET 5 CLR.

2022 pikascript

an embedded Python runtime for the STM32 and other tiny MCUs

Scraping Mechanize

a programmatic web browser for screen scraping.

subliminal

A library for looking up movie subtitles in various languages.

pholcidae

A web crawler library.

urlwatch

A flexible page modification monitor

2020 InvoiceNet

a neural network for extracting invoice information from PDFs

2023 trafilatura

a library and CLI tool for text extraction

Site Generators 2019 portray

a static site generator for Python project documenntation

2020 MiniConf

a lovely site generator for managing conferences

2023 django-distill

Minimal configuration static site generator for Django

Speech silero-models

Multi-language neural text-to-speech

TUI 2021 textual

Text user interface widgets and fields

2023 trogon

a genius way to automatically generate a TUI for click-based applications

pyTermTk

an impressive text UI toolkit

Task management 2021 doit

a task runner that can replace make for many use cases

Tools 2022 reloading

a decorator for hot reloading code

Doctool

a tool to manipulate .docx files

honcho

a process supervisor that uses Procfiles and can be used for multi-process Docker containers

2023 ssh-crypt

a clever way to manage secrets on remote servers

frogmouth

a TUI Markdown browser

pypipe

a Python command-line tool for pipeline processing that can export pipelines as standalone scripts

marcel

a shell that can pipe structured data (like PowerShell, but done right)

flameshow

a terminal flamegraph viewer

termgraph

simple terminal charting

reloadium

advanced hot reloading and profiling

2024 hancho

a simple, pleasant build system

Tutorials 2021 ugit

a step-by-step git implementation

Unicode Unidecode

Does character transliterations.

Video 2021 webcam-filters

a set of GStream filters for doing live webcam background segmentation and blurring

Visual Programming 2022 PyFlow

a another (confusingly, identically named) visual programming environment

Ryven

a flow-based programming environment with a Qt front-end

PyFlow

a desktop-based visual programming environment

Web newscatcher

a cute scraper database/fetcher combo

Web Frameworks Bottle

Tiny, flexible, awesome. My favorite since 2011.

Draco

Old and busted, to various extents.

Falcon

A Cythonizable framework with little magic and good performance

Karrigell

Old and busted, to various extents.

pecan

Based on object dispatching

eve

Another RESTful foundation framework

wheezy.web

A high-performance framework that seems to get along well with PyPy

Snakelets

A minimalistic app server, my old web platform of choice.

Twisted

Old and busted, to various extents.

medusa
CherryPy
Django

The new hotness.

Tornado

non-blocking, which is pretty interesting.

Zope

Old and busted, to various extents.

sandman

A simple RESTful wrapper

Vibora

An asyncio framework that seems to beat Sanic in static performance

django-micro

A lightweight wrapper to use Django as you would Bottle

Moya

a new(ish), template-centric web framework with some interesting twists

2018 zatt

An asyncio implementation of the Raft consensus algorithm (some of the forks add significant enhancements)

2019 appkernel

a microservices framework

2020 blacksheep

an ASGI web framework that uses decorators

starlette

an ASGI web framework that runs under uvicorn

2021 FastAPI

decorator-based, runs under uvicorn

2022 pynecone

a full-stack Python web framework that automatically renders Next.js components

django-ninja

for building APIs with Django and Python 3.6+ type hints

2023 django-unicorn

a Reactive framework for Django that uses minimal JavaScript

socketify.py

another libuv wrapper

litestar

another high-performance ASGI API framework with rigorous typing

robyn

an async Python framework with a Rust connection handler

sse-starlette

Server-Sent Events for Starlette and FastAPI

ucall

a JSON-RPC library that claims to be 100x faster than FastAPI

Writing proselint

a linter for prose

§ xhtml2pdf

Alternatives to PrinceXML for generating formated PDF documentation.

Techniques

Notes:

Setting distutils paths in OS X:

Create a file named .pydistutils.cfg in your home directory with the following contents and set $PATH accordingly:

[install]
install_lib = ~/Library/Python/$py_version_short/site-packages
install_scripts = ~/Library/Python/$py_version_short/bin
install_data = ~/Library/Python/$py_version/share

This will install packages to your Library folder rather than to the system directory, saving you the trouble to use sudo or risking messing with the system installation. Of course virtualenv does a better job for keeping track of project-specific dependencies, but this is great for common/complex dependencies and is sure to survive operating system upgrades.

And then you can use easy_install or pip install --user with impunity, although you might need to do something like:

$ ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future easy_install pandas

…for getting around some clang oddities.

Instant Web Server on port 8000 (2.x):

#!/bin/sh 
python -c '__import__("SimpleHTTPServer").test()'

Totally Unrelated: