Python

Python is one of my favorite programming languages due to its terseness and amazing flexibility.

See also Django for more.

My Stuff:

Year Item Notes
2016 Sushy

my current wiki engine

2015 rss2imap

The way I used read my news a while back

2009

a native Python PNG creation module, suitable for use in Google App Engine

2007

my previous wiki engine

2006

a quick hack to file web pages as MIME compliant e-mail messages.

2005

a minimalist Apache log parser.

another quick hack that apparently filled a long-standing need.

The Python Grimoire

which I converted into TiddlyWiki format

2004

a minimalist (and dumb) Bonjour / Rendezvous client

which I used daily for a couple of years and occasionally contributed to

native protocol

this spawned numerous variants, including a proxy.

Resources:

Category Link Notes
Astronomy Skyfield

An astronomical library for Python

Authentication aioauth

Oauth 2.0 framework and provider for asyncio

CLI click

A very nice way to do composable CLI tools (see also argh, defopt, docopt, python-fire, etc.)

Clint

A great module for developing CLI apps

pbs

A nice, simple and consistent generic command wrapper.

python-fire

a CLI wrapper for any Python object

pyp

a clever shell tool to do inline Python

ptpython

The Python Prompt Toolkit

pqdm

parallel tqdm

typer

A click-like library for building CLI applications

plumbum

Another command wrapper, with SSH support.

tqdm

a configurable progress bar

CMS Wagtail

Another Django-based CMS framework

wid.gy

A Django-based CMS framework

MiniConf

a lovely site generator for managing conferences

Caching diskcache

a Django-compatible caching library

Cloud pyinfra

a fabric-like agentless automation tool

Compilers codon

an LLVM compiler for a Python subset

mypyc

a compiler for a subset of type-annotated Python

Transcrypt

Another JavaScript transpiler

Compression zipfly

a library for doing on-the-fly zip compression with low memory footprint

Concurrency OpenReplica

A coordination service that provides replication and synchronization support for large-scale distributed systems

python-rq

A simpler queueing library

Pulsar

Another actor-based multiprocessing library with asyncio support and a fair degree of network transparency

Celery

My current go-to library for task queueing

Pykka

An implementation of the Actor model, with gevent support

aiomultiprocess

an asyncio Pool implementation to manage multiple processes with independent event loops

offset

ports of the Go concurrency model

aioprocessing

asyncio equivalents of the multiprocessing library

Trio

a friendlier wrapper around asyncio for concurrent tasks

ray

A distributed execution engine that implements a pandas clone

goless

ports of the Go concurrency model

Littleworkers

Parallel job management without the hassles.

Console bpython

My favorite REPL

Colorama

Making ANSI color sequences work cross-platform (see also this gist).

fabulous

Prints images to console

urwid

A curses based library for building console UIs

rich

a console library with various layout options

textual

Text user interface widgets and fields

Data Handling Happy

A Hadoop library to run map-reduce jobs via Jython

biplist

A library that can manage Mac binary .plist files

pyparsing

a handy, flexible parser library that does away with lex/yacc.

snakebite

A native HDFS client to access the Hadoop datastore.

xlwings

An Excel plugin

Bounter

A fast probabilistic counter

voluptuous

A data schema validation library.

Little Ball of Fur

a graph sampling library that works with NetworkX

riptable

a multithreaded array processing library

fuzzywuzzy

A fuzzy string matching library

PyCascading

A better way to write Hadoop jobs.

xlwt

a library to parse Excel documents (cheatsheet).

Data Science SciPy

scientific (including plotting) libraries for Python

Kats

a time series analysis library

TextHero

an NLP library for large-scale text processing

euporie

a text-based interface for Jupyter notebooks

xeus-sql

a Jupyter kernel for SQL databases

papermill

a way to parameterize and automate Jupyter notebooks

eztable

a simple table library

pyxelate

a pixelation/dithering algorithm that uses a mix of algorithmic and ML approaches

Data Validation schema

another data validation library

pydantic

Data validation using type hinting

Schematics

A validation/schema handling library.

Database pymssql

SQL Server and Sybase driver

FreeTDS
SQLAlchemy

tutorial

pottery

a Redis library with Pythonic syntactic sugar

Peewee

A small, very flexible ORM

DuckDB

an embeddable SQL database with OLAP features

montydb

a tiny MongoDB implementation in pure Python

simple-graph

A simple graph database in SQLite

tinydb

A (very) lightweight document oriented database

dataset

trivial abstraction layer over SQLAlchemy

Mongita

a MongoDB-like database library for in-process use

Goatfish

An even smaller schemaless ORM

Debugging Eww

Insert a REPL into a running application

Pycallgraph

Generate a program flow graph via Graphviz

Hypothesis

A very nice testing framework

Scalene

A high-performance CPU and memory profiler

pudb

A console debugger

lptrace

A pretty amazing single-file strace-like tool for Python

PrettyErrors

an error handler that prettifies output

assertEquals

a testing framework with a great screencast

Electronics pinout

Helps create SVG pin-out diagrams for electronic hardware

Enterprise toil

A cross-platform pipeline management system with Common Workflow Language support

Environments snakeware

a Python userspace for Linux

Functional Programming Hy

An awesome LISP dialect that sits atop the runtime.

funcy

A nice, fairly complete functional library

Toolz

A functional complement to the standard library.

pyrsistent

A persistent data structure library

GIS geopy

an amazingly flexible geocoding library.

prettymaps

A minimal Python library to draw customized maps from OpenStreetMap data

reverse-geocoder

A fast offline reverse geocoder

GUI kivy

A sophisticated UI/UX library with Android support

PythonCard

A bit basic, but interesting.

VPython

3D OpenGL libraries for quick visualization

winGuiAuto.py

for Windows automation

pyglet

a cross-platform windowing and multimedia library with OpenGL support

PySide

A recent Qt binding (up to and including 4.7) with QML support

fman build system

a complete build system for Qt applications

PySimpleGUI

a simple tkinter-based GUI wrapper

Gooey

a wxWidgets wrapper for quickly building GUI versions of CLI tools

flet

creates realtime web, mobile and desktop apps using Flutter

DearPyGui

a Linux/Windows GUI library atop ImGUI (C++)

Toga

a cross-platform GUI that works on the Mac

encodermenu

a MicroPython library for GUI creation, using uasyncio

Graphics smc.freeimage

A wrapper for the freeimage library that can handle various TIFF and fax formats as well as ICC color profiles

Graphite

a real-time graphing system similar to RRDTool

Skip's Python Bits

lots of useful snippets

Pygal

An SVG charting library with a variety of built-in formats.

Pyx

Oldie, but goodie.

pygame

a game-oriented library with SDL support

blurhash

A way to encode image placeholders as strings

pyprocessing

A Processing clone in Python.

douglas-quaid

a library for image matching and correlation

Vapory

A library to generate photorealistic renderings using POV-Ray

drawille

Uses Braille characters for TTY charting

hiplot

high dimensional interactive plotting

imagededup

uses several different algorithms (including NNs) to detect duplicate images

pyxel

A retro game engine for Python

diagrams

a library for drawing cloud architecture diagrams (Azure, AWS, k8s, etc.)

plottyprint

printable formatting for matplotlib

image-blender

a Cython implementation of Adobe Photoshop blend modes

taichi

a high-performance graphics library

Cropmap

Face-aware image cropping.

HTML lxml

a different libxml binding, with excellent HTML parsing performance

cssutils

a CSS parser

Gnosis Utilities

all sorts of XML goodies

pygments

syntax highlighter able to handle multiple nested languages

pyquery

A jQuery-like library for manipulating documents.

BeautifulSoup

The most powerful HTML parser out there.

squeezeit

A CSS and JavaScript bundler/minifier

Libraries Sorted Containers

a fast C-backed collections library

slack-machine

A Slack bot library that uses socket mode and asyncio

seedir

A utility library to render filesystem tree diagrams

minotaur

an asynchronous inotify interface

minibatch

a simple stream processing library

mintotp

a tiny TOTP library

Machine Learning Perceptilabls

free visual ML model creation tool

netron

a viewer for neural network, deep learning and machine learning models

discomll

A ML framework for Disco

Tensorflow
Mail aiosmtpd

an asyncio version of smtpd.py

imapautofiler

an IMAP mailbox sorter that I unwittingly replicated (with less features)

Salmon

an SMTP mail server

slimta

a mail app framework for building MTAs

Music pippi

an audio processing/music library

pymidi

a library for building MIDI over IP/RTP applications (compatible with Apple specs)

Networking PyRendezvous

neat little Bonjour module I use a lot

telnetscript

a simple module to do scripted telnet sessions.

IPy

an IP address manipulation library.

pyvnc

ctypes interface to the VNC libraries

ircutils

A relatively modern async IRC framework

Eventlet

Amazing event-oriented framework

ftputil

a high-level interface to the ftplib module

Python smtplib progress indication

Might come in useful some day.

gunicorn

A WSGI server that supports a number of different worker daemons and is highly customizable.

dpkt

packet creation and parsers

Pcapy

a packet manipulation library

uwsgi

A C application container that speaks WSGI and is seriously good.

PuSSH

SSH wrapper

RPyC

an awesome RPC library

SleekXMPP

A more modern XMPP library

Tftpy

a pure Python TFTP library, useful for implementing UDP file transfer testbeds.

ThriftPy

A pure Python implementation of Apache Thrift

pyxmpp2

another one, probably more interesting.

pyrobuf

a Cython implementation of Google’s protobuf library

asphalt

An asyncio framework for network oriented applications, with a considerable number of modules

p2pd

an async networking library with P2P and UDP support

pyapns

An Apple Push Notification Service provider based on Twisted.

opendrop

a partial implementation of AirDrop

PDF Kiva

a Display PDF library

PyPdf

a PDF toolkit

WeasyPrint

Alternatives to PrinceXML for generating formated PDF documentation.

camelot

a library to extract tabular data from PDFs

xhtml2pdf

Alternatives to PrinceXML for generating formated PDF documentation.

borb

a library for creating and manipulating PDF files

Packaging PyOxidizer

a better way to distribute Python bundled apps than pyinstaller

shiv

builds self-contained applications for easy deployment

Poetry

a dependency management and packaging tool

Presentations pydown

An HTML presentation generator

Reporting Relatorio

A very neat reporting library with multiple output formats

ReportLab Toolkit

a pure Python PDF library that includes a presentation tool.

Robotics kabuki

A declarative control library for MicroPython

Runtimes Nuitka

An interesting batch compiler.

Pyjamas

Some interesting twists on using Python on the way to JavaScript.

Brython

a fairly complete in-browser runtime

Skulpt

Some interesting twists on using Python on the way to JavaScript.

tinypy

Python in 64K of code.

Js2Py

A JavaScript interpreter written in pure Python

Hope

A specialized JIT compiler

Pycopy

a fork of MicroPython

pikascript

an embedded Python runtime for the STM32 and other tiny MCUs

Pyjion

Pyjion, a JIT extension for CPython that compiles your Python code into native CIL and executes it using the .NET 5 CLR.

Scraping Mechanize

a programmatic web browser for screen scraping.

subliminal

A library for looking up movie subtitles in various languages.

pholcidae

A web crawler library.

InvoiceNet

a neural network for extracting invoice information from PDFs

urlwatch

A flexible page modification monitor

Site Generators portray

a static site generator for Python project documenntation

Speech silero-models

Multi-language neural text-to-speech

Task management doit

a task runner that can replace make for many use cases

Tools reloading

a decorator for hot reloading code

Doctool

a tool to manipulate .docx files

termgraph

simple terminal charting

honcho

a process supervisor that uses Procfiles and can be used for multi-process Docker containers

reloadium

advanced hot reloading and profiling

Tutorials ugit

a step-by-step git implementation

Unicode Unidecode

Does character transliterations.

Video webcam-filters

a set of GStream filters for doing live webcam background segmentation and blurring

Visual Programming PyFlow

a another (confusingly, identically named) visual programming environment

Ryven

a flow-based programming environment with a Qt front-end

PyFlow

a desktop-based visual programming environment

Web newscatcher

a cute scraper database/fetcher combo

Web Frameworks Bottle

Tiny, flexible, awesome. My favorite since 2011.

Draco

Old and busted, to various extents.

Falcon

A Cythonizable framework with little magic and good performance

Karrigell

Old and busted, to various extents.

pecan

Based on object dispatching

eve

Another RESTful foundation framework

wheezy.web

A high-performance framework that seems to get along well with PyPy

Snakelets

A minimalistic app server, my old web platform of choice.

Twisted

Old and busted, to various extents.

medusa
CherryPy
Django

The new hotness.

Tornado

non-blocking, which is pretty interesting.

Zope

Old and busted, to various extents.

appkernel

a microservices framework

blacksheel

an ASGI web framework that uses decorators

socketify.py

another libuv wrapper

sandman

A simple RESTful wrapper

pynecone

a full-stack Python web framework that automatically renders Next.js components

robyn

an async Python framework with a Rust connection handler

zatt

An asyncio implementation of the Raft consensus algorithm (some of the forks add significant enhancements)

FastAPI

decorator-based, runs under uvicorn

Vibora

An asyncio framework that seems to beat Sanic in static performance

django-ninja

for building APIs with Django and Python 3.6+ type hints

django-micro

A lightweight wrapper to use Django as you would Bottle

Moya

a new(ish), template-centric web framework with some interesting twists

starlette

an ASGI web framework that runs under uvicorn

Techniques

Notes:

Setting distutils paths in OS X:

Create a file named .pydistutils.cfg in your home directory with the following contents and set $PATH accordingly:

[install]
install_lib = ~/Library/Python/$py_version_short/site-packages
install_scripts = ~/Library/Python/$py_version_short/bin
install_data = ~/Library/Python/$py_version/share

This will install packages to your Library folder rather than to the system directory, saving you the trouble to use sudo or risking messing with the system installation. Of course virtualenv does a better job for keeping track of project-specific dependencies, but this is great for common/complex dependencies and is sure to survive operating system upgrades.

And then you can use easy_install or pip install --user with impunity, although you might need to do something like:

$ ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future easy_install pandas

…for getting around some clang oddities.

Instant Web Server on port 8000 (2.x):

#!/bin/sh 
python -c '__import__("SimpleHTTPServer").test()'

Totally Unrelated:

This page is referenced in: