Rui CarmoPython is one of my favorite programming languages due to its terseness and amazing flexibility.
See also Django for more.
My Stuff
| Year | Item | Notes |
|---|---|---|
| 2016 | Sushy | my current wiki engine |
| 2015 | rss2imap | The way I used read my news a while back |
| 2009 | PNG Canvas | a native Python PNG creation module, suitable for use in Google App Engine |
| 2007 | Yaki | my previous wiki engine |
| 2006 | MailArchive | a quick hack to file web pages as MIME compliant e-mail messages. |
| 2005 | PicoStats | a minimalist Apache log parser. |
| IMAP Backup | another quick hack that apparently filled a long-standing need. |
|
| The Python Grimoire | which I converted into TiddlyWiki format |
|
| 2004 | PicoRendezvous.py | a minimalist (and dumb) Bonjour / Rendezvous client |
| newspipe | which I used daily for a couple of years and occasionally contributed to |
|
| Growl native protocol implementation | this spawned numerous variants, including a proxy. |
Resources
| Category | Date | Link | Notes |
|---|---|---|---|
| Astronomy | 2021 | Skyfield | An astronomical library for Python |
| Audio | 2024 | pedalboard | a library for audio processing |
| Authentication | 2021 | aioauth | Oauth 2.0 framework and provider for asyncio |
| 2023 | authlib | OAuth, OpenID Connect and other niceties |
|
| CAD | sdf | Generate 3D meshes based on SDFs (signed distance functions) |
|
| sdf (Gitlab) | Another version of the |
||
| 2024 | build123d | a parametric BREP modeling framework for 2D and 3D CAD built on OpenCascade |
|
| CLI | click | A very nice way to do composable CLI tools (see also argh, defopt, docopt, python-fire, etc.) |
|
| Clint | A great module for developing CLI apps |
||
| pbs | A nice, simple and consistent generic command wrapper. |
||
| ptpython | The Python Prompt Toolkit |
||
| plumbum | Another command wrapper, with SSH support. |
||
| 2018 | python-fire | a CLI wrapper for any Python object |
|
| 2019 | tqdm | a configurable progress bar |
|
| 2020 | pyp | a clever shell tool to do inline Python |
|
| typer | A click-like library for building CLI applications |
||
| 2021 | pqdm | parallel tqdm |
|
| CMS | Wagtail | Another Django-based CMS framework |
|
| wid.gy | A Django-based CMS framework |
||
| Caching | 2021 | diskcache | a Django-compatible caching library |
| Cloud | 2020 | pyinfra | a fabric-like agentless automation tool |
| Compilers | Transcrypt | Another JavaScript transpiler |
|
| 2021 | mypyc | a compiler for a subset of type-annotated Python |
|
| 2022 | codon | an LLVM compiler for a Python subset |
|
| Compression | 2021 | zipfly | a library for doing on-the-fly zip compression with low memory footprint |
| Concurrency | OpenReplica | A coordination service that provides replication and synchronization support for large-scale distributed systems |
|
| python-rq | A simpler queueing library |
||
| Pulsar | Another actor-based multiprocessing library with |
||
| Celery | My current go-to library for task queueing |
||
| Pykka | An implementation of the Actor model, with gevent support |
||
| offset | ports of the Go concurrency model |
||
| aioprocessing | A set of |
||
| ray | A distributed execution engine that implements a |
||
| goless | ports of the Go concurrency model |
||
| Littleworkers | Parallel job management without the hassles. |
||
| 2020 | aiomultiprocess | an asyncio Pool implementation to manage multiple processes with independent event loops |
|
| Trio | a friendlier wrapper around asyncio for concurrent tasks |
||
| 2023 | mpire | a |
|
| 2024 | dramatiq | background queueing/task processing library |
|
| protean | framework for building event-driven applications using the CQRS pattern |
||
| asyncpal | a unified approach at providing elastic task pools |
||
| Console | bpython | My favorite REPL |
|
| Colorama | Making ANSI color sequences work cross-platform (see also this gist) |
||
| fabulous | Prints images to console |
||
| urwid | A |
||
| 2020 | rich | a console library with various layout options |
|
| Data Engineering | Happy | A Hadoop library to run map-reduce jobs via Jython |
|
| biplist | A library that can manage Mac binary |
||
| pyparsing | a handy, flexible parser library that does away with lex/yacc. |
||
| snakebite | A native HDFS client to access the Hadoop datastore. |
||
| xlwings | An Excel plugin |
||
| Bounter | A fast probabilistic counter |
||
| voluptuous | A data schema validation library. |
||
| fuzzywuzzy | A fuzzy string matching library |
||
| PyCascading | A better way to write Hadoop jobs. |
||
| xlwt | a library to parse Excel documents (cheatsheet. |
||
| 2020 | Little Ball of Fur | a graph sampling library that works with NetworkX |
|
| riptable | a multithreaded array processing library |
||
| 2024 | Pathway | a Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG |
|
| Data Science | SciPy | scientific (including plotting) libraries for Python |
|
| 2020 | TextHero | an NLP library for large-scale text processing |
|
| papermill | a way to parameterize and automate Jupyter notebooks |
||
| 2021 | Kats | a time series analysis library |
|
| euporie | a text-based interface for Jupyter notebooks |
||
| xeus-sql | a Jupyter kernel for SQL databases |
||
| eztable | a simple table library |
||
| pyxelate | a pixelation/dithering algorithm that uses a mix of algorithmic and ML approaches |
||
| 2023 | pygwalker | a Tableau-like dataframe exploration tool for Jupyter |
|
| datapane | a data and reporting and analisys framework |
||
| fugue | a unified interface for executing SQL, Python and Pandas code on Spark, Dask and Ray |
||
| 2024 | sail | unifies stream and batch processing with a drop-in replacement for Spark SQL |
|
| cuDF | a GPU DataFrame library leveraging CUDA |
||
| Data Validation | Schematics | A validation/schema handling library. |
|
| 2021 | schema | another data validation library |
|
| pydantic | Data validation using type hinting |
||
| Database | pymssql | SQL Server and Sybase driver |
|
| FreeTDS | |||
| SQLAlchemy | tutorial |
||
| Peewee | A small, very flexible ORM |
||
| tinydb | A (very) lightweight document oriented database |
||
| Goatfish | An even smaller schemaless ORM |
||
| 2020 | DuckDB | an embeddable SQL database with OLAP features |
|
| montydb | a tiny MongoDB implementation in pure Python |
||
| simple-graph | A simple graph database in SQLite |
||
| 2021 | pottery | a Redis library with Pythonic syntactic sugar |
|
| dataset | trivial abstraction layer over SQLAlchemy |
||
| Mongita | a MongoDB-like database library for in-process use |
||
| 2023 | mongolite | another MongoDB-like database |
|
| Debugging | Eww | Insert a REPL into a running application |
|
| Pycallgraph | Generate a program flow graph Graphviz |
||
| Hypothesis | A |
||
| pudb | A console debugger |
||
| lptrace | A pretty amazing single-file strace-like tool for Python |
||
| assertEquals | a testing framework with a great screncast |
||
| 2020 | Scalene | A high-performance CPU and memory profiler |
|
| PrettyErrors | an error handler that prettifies output |
||
| Electronics | 2021 | pinout | Helps create SVG pin-out diagrams for electronic hardware |
| 2023 | belay | a library and helper for developing MicroPython projects, including an on-device REPL |
|
| 2024 | micropython-fusion | a sensor fusion library for MicroPython |
|
| Kalman-Filter-Python-for-mpu6050 | a Kalman filter for the MPU6050 IMU |
||
| Enterprise | toil | A cross-platform pipeline management system with Common Workflow Language support |
|
| Environments | 2020 | snakeware | a Python userspace for Linux |
| Frameworks | 2024 | Platypush | a modular platform for home automation with easy MQTT event handling and a graphical editor |
| Functional Programming | Hy | An awesome LISP dialect that sits atop the runtime. |
|
| funcy | A nice, fairly complete functional library |
||
| Toolz | A functional complement to the standard library. |
||
| pyrsistent | A persistent data structure library |
||
| GIS | geopy | an amazingly flexible geocoding library. |
|
| reverse-geocoder | A fast offline reverse geocoder |
||
| 2021 | prettymaps | A minimal Python library to draw customized maps from OpenStreetMap data |
|
| GUI | kivy | A sophisticated UI/UX library with Android support |
|
| PythonCard | A bit basic, but interesting. |
||
| VPython | 3D OpenGL libraries for quick visualization |
||
| winGuiAuto.py | for Windows automation |
||
| pyglet | a cross-platform windowing and multimedia library with OpenGL support |
||
| PySide | A recent Qt binding (up to and including 4.7) with QML support |
||
| PySimpleGUI | a simple tkinter-based GUI wrapper |
||
| 2018 | Toga | a cross-platform GUI that works on the Mac |
|
| 2019 | fman build system | a complete build system for Qt applications |
|
| 2020 | Gooey | a wxWidgets wrapper for quickly building GUI versions of CLI tools |
|
| DearPyGui | a Linux/Windows GUI library atop ImGUI (C++) |
||
| 2021 | encodermenu | a MicroPython library for GUI creation, using uasyncio |
|
| 2022 | flet | creates realtime web, mobile and desktop apps using Flutter |
|
| 2024 | CustomTkinter | offers customizable widgets and supports system appearance modes across platforms |
|
| mesop | a rapid web development framework for Python |
||
| micropython-nano-gui | A lightweight MicroPython GUI library |
||
| nicegui | a library for creating Web UIs |
||
| Graphics | smc.freeimage | A wrapper for the freeimage library that can handle various TIFF and fax formats as well as ICC color profiles |
|
| Graphite | a real-time graphing system similar to |
||
| Skip's Python Bits | lots of useful snippets |
||
| Pygal | An SVG charting library with a variety of built-in formats. |
||
| Pyx | Oldie, but goodie. |
||
| pygame | a game-oriented library with SDL support |
||
| pyprocessing | A Processing clone in Python. |
||
| Vapory | A library to generate photorealistic renderings using POV-Ray |
||
| drawille | Uses Braille characters for TTY charting |
||
| Cropmap | Face-aware image cropping. |
||
| 2019 | douglas-quaid | a library for image matching and correlation |
|
| imagededup | uses several different algorithms (including NNs) to detect duplicate images |
||
| plottyprint | printable formatting for matplotlib |
||
| 2020 | blurhash | A way to encode image placeholders as strings |
|
| hiplot | high dimensional interactive plotting |
||
| diagrams | a library for drawing cloud architecture diagrams (Azure, AWS, k8s, etc.) |
||
| 2021 | image-blender | a Cython implementation of Adobe Photoshop blend modes |
|
| 2022 | taichi | a high-performance graphics library |
|
| 2023 | pyxel | A retro game engine for Python |
|
| 2024 | renderer | rendering and rasterizing library for STL |
|
| pyrender | glTF 2.0-compliant OpenGL renderer for visualization of 3D scenes |
||
| pygfx | render engine targeting Vulkan/Metal/DX12 |
||
| HTML | lxml | a different |
|
| cssutils | a CSS parser |
||
| Gnosis Utilities | all sorts of XML goodies |
||
| pygments | syntax highlighter able to handle multiple nested languages |
||
| pyquery | A jQuery-like library for manipulating documents. |
||
| BeautifulSoup | The most powerful HTML parser out there. |
||
| squeezeit | A CSS and JavaScript bundler/minifier |
||
| Interpreters | 2023 | pocketpy | an embeddable C-based interpreter in 5000 LOC |
| PyScript | a WASM runtime using Pyodide for web embedding |
||
| 2024 | RustPython | A Python Interpreter written in Rust |
|
| Large Language Models | 2023 | pykoi | a unified interface for data and feedback collection, comparing models, and doing basic RAG |
| Libraries | 2020 | minotaur | an asynchronous inotify interface |
| minibatch | a simple stream processing library |
||
| 2021 | Sorted Containers | a fast C-backed collections library |
|
| seedir | A utility library to render filesystem tree diagrams |
||
| 2022 | slack-machine | A Slack bot library that uses socket mode and |
|
| mintotp | a tiny TOTP library |
||
| 2023 | fastkafka | A Kafka library inspired by fastapi |
|
| reader | A modern RSS/Atom/JSON feed reader library. |
||
| pgvector-python | for using Postgres as a vector database |
||
| 2024 | unsafe-python | A library to assist writing memory-unsafe code, without any imports |
|
| django-organizations | a Django library for managing organization accounts |
||
| domscribe | converts HTML to semantic Markdown while preserving structure and meaning |
||
| foc | an intriguing collection of higher-level functions using operators for composability |
||
| BM25S | an ultrafast implementation of BM25 in pure Python |
||
| Machine Learning | discomll | A ML framework for Disco |
|
| Tensorflow | Usual staple, largely superseded by PyTorch |
||
| 2020 | netron | a viewer for neural network, deep learning and machine learning models |
|
| 2021 | Perceptilabs | free visual ML model creation tool |
|
| 2018 | aiosmtpd | an asyncio version of smtpd.py |
|
| Salmon | an SMTP mail server |
||
| slimta | a mail app framework for building MTAs |
||
| 2021 | imapautofiler | an IMAP mailbox sorter that I unwittingly replicated (with less features) |
|
| Messaging | 2023 | Propan | a declarative Python MQ framework inspired by |
| Music | 2020 | pippi | an audio processing/music library |
| pymidi | a library for building MIDI over IP/RTP applications (compatible with Apple specs) |
||
| 2023 | musicpy | A domain-specific language for writing music |
|
| Networking | PyRendezvous | neat little Bonjour module I use a lot |
|
| telnetscript | a simple module to do scripted |
||
| IPy | an IP address manipulation library. |
||
| pyvnc | A |
||
| ircutils | A relatively modern async IRC framework |
||
| Eventlet | Amazing event-oriented framework |
||
| ftputil | a high-level interface to the |
||
| Python smtplib progress indication | Might come in useful some day. |
||
| gunicorn | A WSGI server that supports a number of different worker daemons and is highly customizable. |
||
| dpkt | packet creation and parsers |
||
| Pcapy | a packet manipulation library |
||
| uwsgi | A C application container that speaks WSGI and is seriously good. |
||
| PuSSH | SSH wrapper |
||
| RPyC | an awesome RPC library |
||
| SleekXMPP | A more modern XMPP library |
||
| Tftpy | a pure Python TFTP library, useful for implementing UDP file transfer testbeds. |
||
| ThriftPy | A pure Python implementation of Apache Thrift |
||
| pyxmpp2 | another one, probably more interesting. |
||
| pyrobuf | a Cython implementation of Google’s |
||
| pyapns | An Apple Push Notification Service provider based on Twisted. |
||
| 2018 | asphalt | An |
|
| 2019 | opendrop | a partial implementation of AirDrop |
|
| 2023 | p2pd | an async networking library with P2P and UDP support |
|
| OCR | 2024 | surya | OCR library supporting 90+ languages |
| Kiva | a Display PDF library |
||
| PyPdf | a PDF toolkit |
||
| WeasyPrint | Alternatives to PrinceXML for generating formated PDF documentation. |
||
| xhtml2pdf | |||
| 2020 | camelot | a library to extract tabular data from PDFs |
|
| 2021 | borb | a library for creating and manipulating PDF files |
|
| 2023 | peepdf | a Python tool to explore PDF files in order to find out if the file can be harmful or not. |
|
| 2024 | marker | a PDF to markdown converter optimized for books and scientific papers, using surya |
|
| Packaging | 2019 | Poetry | a dependency management and packaging tool |
| 2021 | PyOxidizer | a better way to distribute Python bundled apps than pyinstaller |
|
| shiv | builds self-contained applications for easy deployment |
||
| Presentations | pydown | An HTML presentation generator |
|
| Reference | 2023 | python-cheatsheet | a modern reference, useful for kids |
| Reporting | Relatorio | A very neat reporting library with multiple output formats |
|
| ReportLab Toolkit | a pure Python PDF library that includes a presentation tool. |
||
| Resources | 2023 | python-package-template | a template project for building packages |
| 2024 | awesome-micropython | another curated list of MicroPython resources |
|
| awesome-micropython | a curated list of MicroPython resources |
||
| Robotics | kabuki | A declarative control library for MicroPython |
|
| Runtimes | Nuitka | An interesting batch compiler. |
|
| Pyjamas | Some interesting twists on using Python on the way to JavaScript. |
||
| Brython | a fairly complete in-browser runtime |
||
| Skulpt | Some interesting twists on using Python on the way to JavaScript. |
||
| tinypy | Python in 64K of code. |
||
| Js2Py | A JavaScript interpreter written in pure Python |
||
| Hope | A specialized JIT compiler |
||
| 2020 | Pycopy | a fork of MicroPython |
|
| 2021 | Pyjion | Pyjion, a JIT extension for CPython that compiles your Python code into native CIL and executes it using the .NET 5 CLR. |
|
| 2022 | pikascript | an embedded Python runtime for the STM32 and other tiny MCUs |
|
| Scraping | Mechanize | a programmatic web browser for screen scraping. |
|
| subliminal | A library for looking up movie subtitles in various languages. |
||
| pholcidae | A web crawler library. |
||
| urlwatch | A flexible page modification monitor |
||
| 2020 | InvoiceNet | a neural network for extracting invoice information from PDFs |
|
| 2023 | trafilatura | a library and CLI tool for text extraction |
|
| Site Generators | 2019 | portray | a static site generator for Python project documenntation |
| 2020 | MiniConf | a lovely site generator for managing conferences |
|
| 2023 | django-distill | Minimal configuration static site generator for Django |
|
| Speech | silero-models | Multi-language neural text-to-speech |
|
| TUI | 2021 | textual | Text user interface widgets and fields |
| 2023 | trogon | a genius way to automatically generate a TUI for |
|
| pyTermTk | an impressive text UI toolkit |
||
| Task management | 2021 | doit | a task runner that can replace make for many use cases |
| Tools | 2022 | reloading | a decorator for hot reloading code |
| Doctool | a tool to manipulate .docx files |
||
| honcho | a process supervisor that uses |
||
| 2023 | ssh-crypt | a clever way to manage secrets on remote servers |
|
| frogmouth | a TUI Markdown browser |
||
| pypipe | a Python command-line tool for pipeline processing that can export pipelines as standalone scripts |
||
| marcel | a shell that can pipe structured data (like PowerShell, but done right) |
||
| flameshow | a terminal flamegraph viewer |
||
| termgraph | simple terminal charting |
||
| reloadium | advanced hot reloading and profiling |
||
| 2024 | hancho | a simple, pleasant build system |
|
| berylliumOS | a Python-based shell for CircuitPython devices |
||
| Tutorials | 2021 | ugit | a step-by-step git implementation |
| Unicode | Unidecode | Does character transliterations. |
|
| Video | 2021 | webcam-filters | a set of GStream filters for doing live webcam background segmentation and blurring |
| Visual Programming | 2022 | PyFlow | a another (confusingly, identically named) visual programming environment |
| Ryven | a flow-based programming environment with a Qt front-end |
||
| PyFlow | a desktop-based visual programming environment |
||
| Web | newscatcher | a cute scraper database/fetcher combo |
|
| Web Frameworks | Bottle | Tiny, flexible, awesome. My favorite since 2011. |
|
| Draco | Old and busted, to various extents. |
||
| Falcon | A Cythonizable framework with little magic and good performance |
||
| Karrigell | Old and busted, to various extents. |
||
| pecan | Based on object dispatching |
||
| eve | Another RESTful foundation framework |
||
| wheezy.web | A high-performance framework that seems to get along well with PyPy |
||
| Snakelets | A minimalistic app server, my old web platform of choice. |
||
| Twisted | Old and busted, to various extents. |
||
| medusa | |||
| CherryPy | |||
| Django | The new hotness. |
||
| Tornado | non-blocking, which is pretty interesting. |
||
| Zope | Old and busted, to various extents. |
||
| sandman | A simple RESTful wrapper |
||
| Vibora | An |
||
| django-micro | A lightweight wrapper to use Django as you would Bottle |
||
| Moya | a new(ish), template-centric web framework with some interesting twists |
||
| 2018 | zatt | An |
|
| 2019 | appkernel | a microservices framework |
|
| 2020 | blacksheep | an ASGI web framework that uses decorators |
|
| starlette | an ASGI web framework that runs under uvicorn |
||
| 2021 | FastAPI | decorator-based, runs under uvicorn |
|
| 2022 | pynecone | a full-stack Python web framework that automatically renders Next.js components |
|
| django-ninja | for building APIs with Django and Python 3.6+ type hints |
||
| 2023 | django-unicorn | a Reactive framework for Django that uses minimal JavaScript |
|
| socketify.py | another |
||
| litestar | another high-performance ASGI API framework with rigorous typing |
||
| robyn | an async Python framework with a Rust connection handler |
||
| sse-starlette | Server-Sent Events for Starlette and FastAPI |
||
| ucall | a JSON-RPC library that claims to be 100x faster than FastAPI |
||
| Wikis | 2024 | silicon | another Markdown-focused wiki |
| otterwiki | a nice git-backed wiki |
||
| Writing | 2023 | proselint | a linter for prose |
Techniques
- daemon.py, an example daemon script.
- Pydoc - built-in Web help, anytime, anywhere
- How to Write a Spelling Corrector - an interesting technique that can come in handy to fix/suggest search terms, etc.
- Python Webcam Color Track
- Python webcam fun - motion detection
Notes:
Setting distutils paths in OS X:
Create a file named .pydistutils.cfg in your home directory with the
following contents and set $PATH accordingly:
[install]
install_lib = ~/Library/Python/$py_version_short/site-packages
install_scripts = ~/Library/Python/$py_version_short/bin
install_data = ~/Library/Python/$py_version/share
This will install packages to your Library folder rather than to the
system directory, saving you the trouble to use sudo or risking
messing with the system installation. Of course virtualenv does a
better job for keeping track of project-specific dependencies, but this
is great for common/complex dependencies and is sure to survive
operating system upgrades.
And then you can use easy_install or pip install --user with
impunity, although you might need to do something like:
$ ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future easy_install pandas
…for getting around some clang oddities.
Instant Web Server on port 8000 (2.x):
#!/bin/sh
python -c '__import__("SimpleHTTPServer").test()'