Welcome to TiddlyWiki created by Jeremy Ruston, Copyright © 2007 UnaMesa Association
* First public release in TiddlyWiki format, using [[Clint Checketts]]' adaptation of the GTDTiddlyWiki theme and version 1.2.31 of TiddlyWiki.
* Minimal content fixes (mostly re-phrased some headers and fixed whatever the [[Conversion Script]] didn't get right the first time).
* Did a second update fixing minor typos that persisted from the original HTML, adding a couple of extra notes and pointing out that the file URL will change soon.
* Fixed some of the formatting in the [[Generating Random Numbers]] tiddler, moved portions of the text around.
* Re-tagged some tiddlers, added [[Clint Checketts]]' Untagged tab to the sidebar.
* Created some [[Grimoire Macros]] to help with formatting links to Python online references, tweaked some tiddlers to see if it works properly, revised a whole bunch of them.
* Spent some time tracking down a few corrections reported by e-mail (from the plain [[HTML]] days)
* Confirmed downloads so far: 1718
* Added an RFC macro to ease referencing IETF material.
* Many more tiddlers reformatted.
* Replaced the gradient image from the StyleSheet with a flat gunmetal grey (it was generating unnecessary network traffic).
* Confirmed downloads so far: 2271
* Added [[Eric Shulman]]'s "single page mode plugin" to display one tiddler at a time, making it an OptionsPanel fixture and moving less relevant options to AdvancedOptions.
* Added [[Xavier Morel]]'s feedback.
* Fixed a lot of formatting snafus and added the relevant external links.
* Merged more redundant articles, fixed some samples.
* Renamed "Introduction" articles to have an "@" at the beginning, so that they are displayed first in the [[TOC|Table Of Contents]] and other lists - they are the ones likely to be most useful to first-time readers.
* Upgraded (painlessly) to TiddlyWiki 1.2.32
* Disabled editing when viewing the Grimoire over an [[HTTP]] connection (therefore enforcing the "save to edit" rule).
* Very minor updates (lost an interim version of the file)
After nearly two years, I've dusted off the Grimoire and upgraded it to the latest TiddlyWiki version.
Hopefully it will be updated further with a few extra techniques once I iron outsome kinks.
A few more incantations were added, mostly regarding [[HTML]] and [[Web]] topics.
Dictionaries are associative data structures, storing key/value pairs. Given a key, the corresponding value can be retrieved. For example, the following dictionary maps English colour names to French ones.
{{{
fr_colours = { 'red': 'rouge', 'blue': 'bleu',
'black': 'noir', 'white': 'blanc',
'yellow': 'jaune'}
}}}
To retrieve the value corresponding to a given key, the notation is the same as when retrieving an element from a list or tuple. When indexing an array, you can only For example, to get the French word for 'white', you would write {{{fr_colours['white']}}} in Python code.
A dictionary can use any immutable Python value as the key, and there are no restrictions on what you can store as values. That means that keys can be integers, floating point numbers, strings, or tuples; lists are the most important type that //can't// be used as keys.
Dictionaries are implemented as resizable hash tables. Hash tables are data structures optimized for quick retrieval; the result of a semi-random hash function is computed for each key, and the value is stored in a bin given by the hash function's result. Retrieving a key is then a matter of calculating the hash function and looking in the corresponding bin, so retrieval always takes the same amount of time no matter how large the dictionary is. (At least, this is true unless you have a pathological set of keys that all hash to the same value, or to a very small set of values. This is highly unlikely, unless you deliberately engineer your keys to beat the hash function.)
The hash table implementation of dictionaries explains the reason for the restriction to immutable keys. If the key were mutable like a list, it could be changed in-place which would also modify the hash value. The value would then no longer be stored in the right bin, and lookups would incorrectly fail.
In Python, functions are defined using the {{{def}}} statement. The following example defines a function called {{{get_word()}}}. {{{get_word()}}} will take two parameters: a string sentence and a number N, and returns the N'th word of sentence.
{{{
import string
def get_word(sentence, N):
words = string.split(sentence)
return words[N]
}}}
The {{{return}}} statement, when followed by an expression, causes the value of the expression to be returned as the result of the function. In the above example, {{{words[N]}}} is returned as the result.
Python functions always produce a result, whether the code contains a {{{return}}} statement or not. If the path of execution falls off the end of the function, {{{None}}} is returned. A lone {{{return}}} statement, not followed by an expression, also returns None.
Default values for parameters can be specified by placing a "{{{=}}}" and an expression or a literal value after the parameter name. (For a discussion of using expressions to specify a default value, see the next section.)
In the following example, the function's filename parameter has no default value, the compresslevel argument has a default value of 9, and the fileobj parameter has a default value of {{{None}}}.
{{{
def open_file(filename,
compresslevel = 9,
fileobj = None):
if fileobj is None:
fileobj = open(filename, 'w')
compressor = zlib.compressobj( compresslevel )
}}}
To call a function with keyword arguments that can be specified in any order, no special function definition is needed, though default arguments are often used in concert with keywords. The function call can simply give the names of function parameters, followed by an "{{{=}}}" and the desired value for the parameter. Given a function definition like {{{f(a, b = 'empty', c=3.5)}}}, the following function calls are all legal:
{{{
f(1, 'new', 7.2)
f(1, c=7.2) # Equivalent to f(1, 'empty', 7.2)
f(c=1, a=4, b='new') # Equivalent to f(4, 'new', 1)
}}}
The following calls are all illegal:
{{{
f(b = 'new') # No value provided for a
f(a = 4, 4) # Non-keyword argument after keyword argument
f(4, b=1, c=3, b=1) # duplicate keyword argument b
}}}
Python offers a variety of different numeric types: integers, which are fast but limited in size; long integers, which are slower but can be of arbitrary size; floating-point numbers; and complex numbers.
The simplest and most basic type are integers, which are represented as a C {{{long}}}. Their size is therefore dependent on the platform you're using; on a 32-bit machine, they can range from -2147483647 to 2147483647. Python programs can determine the highest possible value for an integer by looking at {{{sys.maxint}}}; the lowest possible value will usually be {{{-sys.maxint - 1}}}.
Long integers are an arbitrary-precision integer type. They're slower than regular integers, but don't have the size limitations. Long integer literals have the letter "{{{L}}}" as a suffix; a lowercase "{{{l}}}" is also legal, but should be avoided because it's often difficult to distinguish a lowercase letter "{{{l}}}" from the numeral "{{{1}}}". If integers and long integers are combined in an expression, the regular integer is converted to a long integer automatically:
{{{
2L + 3 5L 5L ** 100 7888609052210118054117285652827862296732064351090230047702789306640625L
}}}
Python's floating point numbers are represented as C's {{{double}}} type, and behave pretty much as you'd expect:
{{{
5.4 / 2.3 2.34782608696 6.4 * 2 12.8 7.5 % 5 2.5
}}}
In operations that mix floating-point numbers and integers, the integers will be converted to floating-point before computing the result, which will be left as a float.
Python also supports a complex number type. (It's possible to compile the Python interpreter without complex numbers, but few people bother to do so; saving 20K of code isn't that important.) Complex numbers are represented by a floating point constant with a suffix of "{{{j}}}"or "{{{J}}}" suffix, "{{{j}}}" being used to represent the square root of -1 in many engineering contexts.
{{{
1j * 1J (-1+0j) (3 - 7j) + (2 + 1j) (5-6j) (1 + 3j) / (2 - 4j) (-0.5+0.5j)
}}}
Note that an expression like {{{1 + 3j}}} isn't a constant, but an expression that adds two constants: {{{1}}} and {{{3j}}}. This is a minor semantic quibble, whose major significance is that you should make liberal use of parentheses in complex expressions. {{{1+3j / 2-4j}}} is not the same as {{{(1+3j) / (2-4j)}}}, because the first expression will evaluate {{{3j/2}}} first.
Relatively few tasks can be performed by only dealing with numbers; programs will usually print out reports, modify the contents of text files, parse [[HTML]] and [[XML]] documents, and perform other operations on strings of characters. Strings are therefore an important data type in modern programming languages, and Python is no exception.
In the source code for a Python program, strings can be written in several ways. They can be surrounded by either single or double quotes:
{{{
if version[:5] != 'HTTP/':
send_error(400, "Bad request version(%s)" % `version`)
}}}
Strings are one of Python's sequence types. This means that strings can be sliced, and the {{{for}}} statement can be used to iterate over the individual characters in a string:
{{{
>>> s = "Hello, world"
>>> s[0:5]
'Hello'
>>> s[-5:]
'world'
>>> for char in s:
... print char,
...
H e l l o , w o r l d
}}}
Strings are immutable; once a string has been created, whether as a literal in a program's source code or in the course of a program's operation, you can't modify the string in-place. Trying to change the first character of a string by slicing, as in the code {{{s[0] = 'Y'}}}, fails; to create a modified version of the string, you have to assemble a whole new string with code like {{{s = 'Y' + s[1:]}}}.
This means that Python has to make temporary copies of the string, which will be slow and memory-consuming if you're modifying very large strings.
See the [[Mutable Strings]] section for a way around this by using the <<PythonModule array>> module, and the <<PythonModule UserString>> module for a {{{MutableString}}} type which may be useful if you absolutely must modify strings in-place and still use some of the more common <<PythonModule string>> methods.
Python's lists and tuples are the simplest compound data types. They can contain any number of other Python objects, no matter what the type of the subobjects. One tuple or list can contain numbers, strings, other tuples and lists, class instances, or any other data type that Python can handle.
Tuple constants are written as a sequence of values separated by commas; optionally they can be surrounded by parentheses. A zero-length tuple is written as an empty pair of parentheses {{{()}}}, and a tuple of length 1 is indicicated by a trailing comma, to distinguish a one-element tuple from an expression..
{{{
1,2,3
('subject', 'from', 'date', 'to')
() # Empty tuple
(1,) # Single-element tuple
2, # Another single-element tuple
2 # The number 2 (no trailing ',')
}}}
Lists are always surrounded by square brackets:
{{{
[] # Empty list
[1] # List of length 1
[1,2,3] # List of length 3
["this", 1, "is", "a", "list"]
}}}
You can retrieve the ith element in a list or tuple by indexing it: {{{a[i]}}}. Retrieving a single element is a fast operation, and is indepent of the size of the list/tuple or the value of i. A smaller list or tuple can be retrieved by slicing: {{{a[i:j]}}} returns a sublist or subtuple containing elements i up to j-1.
{{{
>>> L = [0,1,2,3,4,5,6]
>>> L[0:2], L[3:5], L[0:-1]
([0, 1], [3, 4], [0, 1, 2, 3, 4, 5])
}}}
If an index is a negative number, then the length of the sequence is added to it. This means that {{{L[-1]}}} is the last element of the sequence {{{L}}}, L[-2] is the next-to-last element, and so forth. You can mix positive and negative indexes as you like. Omitting i, the starting index of a slice, will default to 0, and omitting the ending index j defaults to the length of the sequence.
{{{
>>> L[:2], L[-2:], L[:-1]
([0, 1], [5, 6], [0, 1, 2, 3, 4, 5])
}}}
Tuples and lists are similar in many ways. The most important difference between them is that lists are mutable, or changeable objects; it's possible to replace any given list element, or to change the list's length. By way of contrast, once a tuple has been created it's not possible to change it in any way.
To modify a list, you can simply assign to an element or a slice, or use the {{{del}}} statement.
{{{
>>> L=[1,2,3]
>>> L[ -1 ] = 7 ; print L
[1, 2, 7]
>>> L[0:2] = ['a', 'b', 'c'] ; print L
['a', 'b', 'c', 7]
>>> del L[1] ; print L
['a', 'c', 7]
}}}
Because tuples and lists are so similar, you might wonder why Python supports them both. Why not only have lists, since they're more flexible than tuples?
Immutable tuples are useful in situations where you need to pass a few items to a function and don't want the function to modify the tuple; for example,
{{{
point1 = (120, 140)
point2 = (200, 300)
record(point1, point2)
}}}
You don't want to have to think about what would happen if the {{{record()}}} function changed the coordinates -- it can't, because tuples are immutable. In the section on dictionaries, you'll learn that tuples can be used as dictionary keys, while lists can't.
On the other hand, the mutability of lists makes it easier to change them, so they're useful when you're constructing a collection of objects over time, or when objects need to be freely removed and added. For example, if you're iterating through the lines of text in a file and saving lines that match a certain criterion such as a regular expression, a list is the natural data type for storing the matching lines. Adding a new line is fast -- just call the list's {{{append()}}} method -- while building a new tuple by concatentation ({{{tuple1 + (line,)}}}) would be slow since it copies the entire tuple every time.
When to use a list or a tuple is ultimately a stylistic question, and you'll arrive at your own opinions about it. Personally, I generally use tuples to hold a collection of dissimilar data; for example, a function might return a numeric error code and a string describing an error as a tuple {{{(code, errmsg)}}}. In this style, tuples wind up being used like Cs structures or Pascal's records. A function that returned a number of similiar objects, such as lines from a text file, would return a list. In other words, while lists can contain data of different types, I tend to only use them to contain similar objects: all strings, all numbers, all instances of the same class, or whatever. (On the other hand, I don't follow this rigidly; if I had some different data types collected together and sometimes needed to change only one of them, I'd probably use a list instead of a tuple, even though the data in that list wouldn't be homogeneous.)
! Problem:
You have a function in a variable {{{F}}}, and a tuple {{{T}}} containing its arguments. You wish to call {{{F}}} with the arguments {{{T}}}.
! Solution:
Use the built-in function {{{apply()}}}:
{{{
result = apply(F, T)
}}}
Functions that don't require any arguments can be called using an empty tuple. For example, {{{time.time()}}} doesn't take any arguments, so it can be called like this:
{{{
curtime = apply( time.time, () )
}}}
If the function takes keyword arguments, you can also pass in a dictionary whose keys and values are the keyword arguments and their values.
{{{
# Equivalent to
# menu.add_command(label = 'Open Help', command = window.show_help)
kw = {'label': 'Open Help',
'command': window.show_help }
apply(menu.add_command, (), kw)
}}}
''Discussion:''
One subtle point is that {{{apply(F, T)}}} is not the same as {{{F(T)}}}. {{{F(T)}}} calls {{{F}}} with exactly one argument, the tuple {{{T}}}, while {{{apply(F, T)}}} calls {{{F}}} with {{{T[0]}}} as the first argument, {{{T[1]}}} as the second, and so forth. An empty tuple corresponds to calling the function with no arguments.
! Problem:
You wish to check whether a given key is stored in a dictionary. For example, you wish to know if the key {{{'green'}}} is in the {{{fr_colours}}} dictionary.
! Solution:
There are two ways of doing this. The {{{has_key()}}} dictionary method returns true or false depending on whether the key is in the dictionary or not.
{{{
>>> fr_colours.has_key('blue')
1
>>> fr_colours.has_key('purple')
0
}}}
If a dictionary retrieval fails, Python raises a {{{KeyError}}} exception. You can catch this exception and do something appropriate in the exception handler.
{{{
try:
fcolr = fr_colours[ ecolr ]
except KeyError:
fcolr = "<Untranslated colour %s>" % (ecolr,)
}}}
! Discussion:
Deciding whether to use {{{has_key()}}} or catch the exception is a question of both style and speed. Either way is fine from a stylistic point of view, though you may have a preference for one or the other, depending on what your programming background is. For example, one of the authors rarely catched the {{{KeyError}}} exception, preferring to write {{{if dict.has_key(key): ...}}} before a block of code which can then assume that key is present. You can do whatever you like.
The comparative speed of the two solutions depends on how often you expect the key to be present. Raising an exception and then catching it is slower than a call to the {{{has_key()}}} method. Consider a program where you're looping over 100,000 keys, and doing something different depending on whether the key is present or not. If you expect only a few keys to be missing, the exception will almost never be raised, while calling the {{{has_key()}}} method would cost a little bit of time on every pass through the loop. In this case, catching the infrequent exceptions will be faster.
On the other hand, if 50,000 or 90,000 of the keys won't be present, the overhead of raising exceptions and catching them will be larger because an exception will have to be raised more often, so checking the result of {{{has_key()}}} is probably a better idea. The exact crossover point will vary a bit; in [[a presentation|http://barry.warsaw.us/papers/ecshort.pdf]] ([[PDF]]) at the sixth Python conference, Barry Warsaw found it to be around 5 or 10 percent.
! Problem:
You wish to display a random line from a file.
! Solution:
If you are dealing with a short file, you can produce a list containing all the lines, and then choose items at random:
{{{
lines = sys.stdin.readlines()
# or file.readlines(), where file is a file object
while lines:
line = random.choice(lines)
lines.remove(line)
print line,
}}}
Email address: checketts [at] gmail -dot- com
Clint is a student at BYU-Idaho studying Information Systems, and has published a number of extra TiddlyWiki [[styles|http://15black.bluedepot.com]], as well as several standalone tweaks that were incorporated into this file.
This is the simple conversion script that [[I|Rui Carmo]] used to batch-convert the original [[Grimoire]] HTML into TiddlyWiki markup (pre-2.0).
The result is another HTML file with TiddlyWiki markup inside {{{textarea}}} tags, which were then individually cut and pasted into TiddlyWiki for minor tweaks (mostly spacing, tables and bulleted lists, which [[I|Rui Carmo]] forgot to process.
{{{
import sys
import re
from HTMLParser import HTMLParser
class IndexParser(HTMLParser):
def __init__(self):
HTMLParser.__init__(self)
self.buffer = ""
self.fixed = self.table = False
def handle_starttag(self, tag, attrs):
try:
raise intern(str(tag))
except ('table'):
self.table = True
except ('tr', 'thead'):
self.buffer += '\n'
except ('td'):
self.table = False
self.buffer += '|'
except ('th'):
self.buffer += '|!'
except ('blockquote'):
self.buffer += '\n<<<\n'
except ('pre'):
self.buffer += '\n{{{\n'
self.fixed = True
except ('tt', 'code'):
self.buffer += '{{{'
except ('i','em'):
self.buffer += '//'
except ('b'):
self.buffer += "''"
except ('u'):
self.buffer += "__"
except ('p'):
self.buffer += '\n'
except ('h1'):
self.buffer += '</textarea><textarea>!'
except ('h2'):
self.buffer += '</textarea><textarea>!!'
except ('h3'):
self.buffer += '</textarea><textarea>!!!'
except:
pass
def handle_entityref(self, name):
try:
raise intern(str(name))
except('lt'):
self.buffer += '<'
except('gt'):
self.buffer += '>'
except:
pass
def handle_data(self, text):
if(self.table):
text = text.strip()
if(self.fixed):
text = re.sub('\t+', ' ', text)
text = re.sub('\n\s', '\n', text)
else:
text = re.sub('\s+',' ', text)
self.buffer += text
def handle_endtag(self, tag):
try:
raise intern(str(tag))
except ('body'):
self.buffer += '</textarea>'
except ('table'):
self.table = False
except ('td'):
self.table = True
except ('tr','thead'):
self.buffer += '|'
except ('blockquote'):
self.buffer += '\n<<<\n'
except ('pre'):
self.buffer += '}}}\n'
self.fixed = False
except ('tt', 'code'):
self.buffer += '}}}'
except ('p'):
self.buffer += '\n'
except ('b'):
self.buffer += "''"
except ('u'):
self.buffer += "__"
except ('i','em'):
self.buffer += '//'
except:
pass
parser = IndexParser()
parser.feed(sys.stdin.read())
print parser.buffer
}}}
You have a tuple, and want to turn it into a list. Or, the opposite problem: a list that you want to turn into a tuple.
The built-in functions {{{tuple()}}} and {{{list()}}} convert any sequence type into a tuple or list containing the same items in the same order.
{{{
>>> tuple( [1,2,3] )
(1, 2, 3)
>>> list ( ('monty', "python's", "fliegende", "circus") )
['monty', "python's", 'fliegende', 'circus']
>>> tuple('Dr Gumby!')
('D', 'r', ' ', 'G', 'u', 'm', 'b', 'y', '!')
}}}
''Discussion:''
{{{list()}}} was introduced in Python 1.5; in previous versions, the following idiom was used to convert things to lists:
{{{
list = map(None, (1,2,3,4) )
}}}
Here's how the above line of code works: the {{{map()}}} built-in function is used to evaluate a function for every value in a sequence type like a tuple, and returns a list containing the function's output for each value in the input sequence. Put another way, {{{map(F, seq)}}} returns a new list containing {{{[F( seq[0] ), F( seq[1] ), ... ]}}}. If {{{None}}} is passed as the function {{{F}}}, then {{{map()}}} simply returns the elements of the sequence {{{[seq[0], seq[1], ...]}}}.
You may still see the above use of {{{map()}}} in programs that attempt to be compatible with older versions of Python; if you don't care about backward compatibility, simply use {{{list()}}} which is clearer.
! Problem:
You have a text buffer containing [[HTML]] character entities and need to render them in [[Unicode]] text, replacing each entity with a single symbol.
! Solution:
Use [[BeautifulSoup|http://www.crummy.com/software/BeautifulSoup/]], which can do all the dirty work for you in a one-liner:
{{{
from BeautifulSoup import *
symbols = unicode(BeautifulStoneSoup(buffer, convertEntities=BeautifulStoneSoup.HTML_ENTITIES))
}}}
''Problem:''
You need to convert an integer value {{{V}}} to a 2-byte or 4-byte string representation. For example, the number 1 would be the bytes 0,1; 1956 would be 164, 7 because 1956 = 7 * 256 + 164. This is often needed when reading or writing binary data files, or implementing network protocols.
''Solution:''
You can write code to perform the conversion by taking the low byte of {{{V}}}, shifting {{{V}}} down 8 bits and taking the low byte of the result, and so forth. However, the code is long and relatively slow:
{{{
s = ( chr( (V) 255 ) +
chr( (V >> 8) 255 ) +
chr( (V >> 16) 255 ) +
chr( (V >> 24) 255 ) )
}}}
A faster and simpler solution uses the <<PythonModule struct>> module, which is intended for such conversions:
{{{
import struct
s = struct.pack('i', V )
}}}
To reverse the conversion, and go from a binary string to a tuple of integers, use {{{struct.unpack()}}}:
{{{
byte, short = struct.unpack('BH', '\377\000\377\377')
}}}
''Discussion:''
You're not limited to only packing a single value at time. Instead, the {{{pack()}}} function in the <<PythonModule struct>> module takes a format string containing several characters, along with the values to be packed:
{{{
>>> struct.pack('iii', 127, 1972, 1234567890)
'\177\000\000\000\264\007\000\000\322\002\226I'
}}}
Repeated format characters can be written as a single character preceded by an integer repeat count; the previous example could have been written as {{{struct.pack('3i', ...)}}}, and would produce identical results.
Different-sized values can be packed using several different format characters, whose output will vary in size. For example, the "{{{b}}}" (byte) character will produce a single byte of output, and therefore is limited to values between -128 and 127. "{{{h}}}" (short) produces 2 bytes, "{{{i}}}" (int) produces 4 bytes, and "{{{l}}}" (long) will be 4 bytes on 32-bit machines, and 8 bytes on most 64-bit machines.
Here's a table of some of the more commonly used format characters supported by {{{pack()}}} and {{{unpack()}}}. Consult the Library Reference's section on the <<PythonModule struct>> module for a complete list:
|!''Format''|!''C Type''|!''Python''|
|b|{{{signed char}}}|integer|
|B|{{{unsigned char}}}|integer|
|h|{{{short}}}|integer|
|i|{{{int}}}|integer|
|l|{{{long}}}|integer|
|f|{{{float}}}|float|
Because the <<PythonModule struct>> module is often used for packing and unpacking C structs, it obeys the compiler's alignment rules. This means that padding bytes may be inserted between format characters of different types. For example, the zero byte in the following example is padding:
{{{
>>> struct.pack('BH', 255, 65535)
'\377\000\377\377'
}}}
! Problem:
You want to convert a tuple in a string into the corresponding tuple object; such a {{{str2tuple()}}} function would be used like this:
{{{
>>> myString = "(12, ' abc ' , ' b, c', 'd\\'e')"
>>> print str2tuple(myString)
(12, 1.0, "'XX'", ' abc ', ' b, c', "d'e", "'f")
}}}
! Solution:
The following {{{str2tuple()}}} function will do the job.
{{{
def str2tuple(s):
return eval(s, {'__builtins__': {}})
}}}
! Discussion:
The above function uses the {{{eval()}}} built-in function, along with a special trick. {{{eval()}}} takes a string and evaluates the contents as a Python expression, returning the value of the expression. One or two additional parameters can be supplied that must be dictionaries that will be used as the table of global and local variables, respectively.
The {{{str2tuple()}}} function uses {{{eval()}}} and provides a dictionary containing a single key-value pair, mapping the key "{{{____builtins____}}}" to an empty dictionary. "{{{____builtins____}}}" is a special name internal to Python.
The net effect of this is to evaluate the string in a highly restricted environment. This denies access to Python's standard built-in functions such as {{{open()}}}, and thus cannot access anything that isn't hardcoded in the language. For example:
{{{
>>> str2value(' (1,"test",10.2)')
(1, 'test', 10.2)
>>> str2value('11+13')
24
>>> str2value('open("/etc/passwd")')
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in str2value
File "<string>", line 0, in ?
NameError: open
>>>
}}}
Notice that {{{str2value()}}} can accept expressions such as "{{{11+13}}}" thanks to its use of {{{eval()}}}, returning an integer value. If you really want to make sure that the result is a tuple, you must use the {{{type()}}} built-in function to determine the type of the result and compare it with the type object for tuples.
{{{
v = str2value( text )
if type(v) is not type( () ):
raise ValueError, "Text %s does not contain a tuple" % (text,)
}}}
A clearer way to do this is to import the <<PythonModule types>> module, which contains predefined variables for the most common Python types, and compare the type of the result to {{{types.TupleType}}}.
{{{
if type(v) is types.TupleType:
raise ValueError, "Text %s does not contain a tuple" % (text,)
}}}
You wish to convert a number to the corresponding string representation. For example, you want to convert a number like 144 to the string {{{'144'}}}.
The simplest way is to use the built-in function {{{str()}}}, or the backquote notation {{{`144`}}}. Both solutions will convert the number to a string in the most straightforward way.
{{{
V = 187.5
S = str(V) # Produces '187.5'
}}}
If {{{V}}} contains an integer or long integer, and you want to convert the number to hexadecimal or octal representation, use the built-in functions {{{hex()}}} or {{{oct()}}}.
{{{
>>> hex(187)
'0xbb'
>>> oct(187)
'0273'
}}}
If fancier formatting is required, such as rounding a floating-point number to a given number of decimal places, or padding a number with zeros, use the % operator on strings, which allows more precise control of the output.
{{{
>>> "%04d" % (87,)
'0087'
>>> "%.2f" % (187.375,)
'187.38'
}}}
(See the Library Reference manual for the complete details of the available format characters.)
There's no library function to convert a number to its base-2 representation, so here's a function to do the job.
{{{
def atob(number):
"""Returns a string containing the binary representation of a number"""
if number < 0:
prefix = "-" ; number = -number
elif number == 0: return "0"
else:
prefix = ""
# Loop, looking at the lowest bit of the number an
s = ""
while number > 0:
s = chr( 48+ (number 1) ) + s
number = number >> 1
return prefix + s
}}}
! Problem:
You have a string such as {{{'533'}}} and wish to convert it to the number 533.
! Solution:
The built-in functions {{{int()}}}, {{{long()}}}, and {{{float()}}} can perform this conversion from string to a numeric type:
{{{
>>> int( '533' )
533
>>> long( '37778931862957161709568' )
37778931862957161709568L
>>> float( '45e-3'), float('12.75')
(0.045, 12.75)
}}}
Note that these functions are restricted to decimal interpretation, so that {{{int('0144')}}} will return a result of 144, and {{{int('0x144')}}} will raise a {{{ValueError}}} exception.
To support different bases, or for use with versions of Python earlier than 1.5, the <<PythonModule string>> module contains the functions {{{atoi()}}}, {{{atol()}}}, and {{{atof()}}}, which convert from ASCII to integer, long integer, or floating point, respectively.
{{{
>>> import string
>>> string.atoi('45')
>>> string.atol( '37778931862957161709568' )
37778931862957161709568L
>>> string.atof('187.54')
187.54
}}}
{{{atoi()}}} and {{{atol()}}} have an optional second argument which can be used to specify the base for the conversion. If the specified base is 0, the functions will follow Python's rules for integer constants: a leading "{{{0}}}" will cause the string to be interpreted as an octal number, and a leading "{{{0x}}}" will cause base-16 to be used.
{{{
>>> string.atoi('255')
255
>>> string.atoi('255', 16) # 255 hex = 597 decimal
597
>>> string.atoi('0255', 0) # Assumed to be octal
173
>>> string.atol('0x40000000000000000000', 0)
302231454903657293676544L
}}}
While you could use the built-in function {{{eval()}}} instead of the above functions, this is not recommended, because someone could pass you a Python expression that might have unwanted side effects.
For example, a caller might pass a string containing a call to {{{os.system()}}} to execute an arbitrary command; this is a serious danger for applications such as CGI scripts that need to handle data from an unknown source. {{{eval()}}} will also be slower than a more specialized conversion operation.
Most of the time this will do:
{{{
infile = open("file.in", "rb")
outfile = open("file.out", "wb")
outfile.write(infile.read())
}}}
However for huge files you may want to do the reads/writes in pieces (or you may have to), and if you dig deeper you may find other technical problems.
Unfortunately, there's no totally platform independent answer. On Unix, you can use {{{os.system()}}} to invoke the "cp" command (see your Unix manual for how it's invoked). On DOS or Windows, use {{{os.system()}}} to invoke the "COPY" command. On the Mac, use {{{macostools.copy(srcpath, dstpath)}}}. It will also copy the resource fork and Finder info.
There's also the shutil module which contains a {{{copyfile()}}} function that implements the copy loop; but in Python 1.4 and earlier it opens files in text mode, and even in Python 1.5 it still isn't good enough for the Macintosh: it doesn't copy the resource fork and Finder info.
<<<
''Note:'' This should be fixed for modern Python on [[Mac OS X]], but needs to be confirmed (especially due to the way the OS now stores metadata).
<<<
To delete (remove, unlink) a file, use {{{os.remove(filename)}}} or {{{os.unlink(filename)}}}
For documentation, see the posix section of the library manual. They are the same: {{{unlink()}}} is simply the Unix name for this function.
To remove a directory {{{os.rmdir()}}}
To create a directory {{{os.mkdir()}}}
To rename a file {{{os.rename()}}}
To truncate a file, open it using{{{ f = open(filename, "r+")}}}, and use {{{f.truncate(offset)}}}
(The offset defaults to the current seek position. The {{{ "r+"}}} mode opens the file for reading and writing.)
There's also {{{os.ftruncate(fd, offset)}}} for files opened with {{{os.open()}}} for advanced Unix hacks only.
The core of the //Grimoire// was originally developed and released by [[Andrew M. Kuchling]] in May, 1999. However, it never reached a stage where Andrew felt that it was ready for publication, and eventually he withdrew it. [[Steve Ferg]], however, had found the Grimoire very helpful as he was learning Python, and he persuaded Andrew to allow him to take over maintenance of the document in August, 2002.
In December 2004, Andrew and Steve released the Grimoire under the [[Creative Commons Attribution-NonCommercial-ShareAlike License 2.0|http://creativecommons.org/licenses/by-nc-sa/2.0/]] and [[Rui Carmo]] began hosting it at [[The Tao of Mac|http://the.taoofmac.com/space/Python/Grimoire]], where it lingered until [[Rui|Rui Carmo]] decided to re-cast it in [[TiddlyWiki]] format for ease of use.
[[TiddlyWiki]] is the work of [[Jeremy Ruston]], and the current look and feel is based on enhancements by [[Clint Checketts]]. There are more contributors and sources of inspiration, all tagged as<<tag Credits>>
/***
|''Name:''|CryptoFunctionsPlugin|
|''Description:''|Support for cryptographic functions|
***/
//{{{
if(!version.extensions.CryptoFunctionsPlugin) {
version.extensions.CryptoFunctionsPlugin = {installed:true};
//--
//-- Crypto functions and associated conversion routines
//--
// Crypto "namespace"
function Crypto() {}
// Convert a string to an array of big-endian 32-bit words
Crypto.strToBe32s = function(str)
{
var be = Array();
var len = Math.floor(str.length/4);
var i, j;
for(i=0, j=0; i<len; i++, j+=4) {
be[i] = ((str.charCodeAt(j)&0xff) << 24)|((str.charCodeAt(j+1)&0xff) << 16)|((str.charCodeAt(j+2)&0xff) << 8)|(str.charCodeAt(j+3)&0xff);
}
while (j<str.length) {
be[j>>2] |= (str.charCodeAt(j)&0xff)<<(24-(j*8)%32);
j++;
}
return be;
};
// Convert an array of big-endian 32-bit words to a string
Crypto.be32sToStr = function(be)
{
var str = "";
for(var i=0;i<be.length*32;i+=8)
str += String.fromCharCode((be[i>>5]>>>(24-i%32)) & 0xff);
return str;
};
// Convert an array of big-endian 32-bit words to a hex string
Crypto.be32sToHex = function(be)
{
var hex = "0123456789ABCDEF";
var str = "";
for(var i=0;i<be.length*4;i++)
str += hex.charAt((be[i>>2]>>((3-i%4)*8+4))&0xF) + hex.charAt((be[i>>2]>>((3-i%4)*8))&0xF);
return str;
};
// Return, in hex, the SHA-1 hash of a string
Crypto.hexSha1Str = function(str)
{
return Crypto.be32sToHex(Crypto.sha1Str(str));
};
// Return the SHA-1 hash of a string
Crypto.sha1Str = function(str)
{
return Crypto.sha1(Crypto.strToBe32s(str),str.length);
};
// Calculate the SHA-1 hash of an array of blen bytes of big-endian 32-bit words
Crypto.sha1 = function(x,blen)
{
// Add 32-bit integers, wrapping at 32 bits
add32 = function(a,b)
{
var lsw = (a&0xFFFF)+(b&0xFFFF);
var msw = (a>>16)+(b>>16)+(lsw>>16);
return (msw<<16)|(lsw&0xFFFF);
};
// Add five 32-bit integers, wrapping at 32 bits
add32x5 = function(a,b,c,d,e)
{
var lsw = (a&0xFFFF)+(b&0xFFFF)+(c&0xFFFF)+(d&0xFFFF)+(e&0xFFFF);
var msw = (a>>16)+(b>>16)+(c>>16)+(d>>16)+(e>>16)+(lsw>>16);
return (msw<<16)|(lsw&0xFFFF);
};
// Bitwise rotate left a 32-bit integer by 1 bit
rol32 = function(n)
{
return (n>>>31)|(n<<1);
};
var len = blen*8;
// Append padding so length in bits is 448 mod 512
x[len>>5] |= 0x80 << (24-len%32);
// Append length
x[((len+64>>9)<<4)+15] = len;
var w = Array(80);
var k1 = 0x5A827999;
var k2 = 0x6ED9EBA1;
var k3 = 0x8F1BBCDC;
var k4 = 0xCA62C1D6;
var h0 = 0x67452301;
var h1 = 0xEFCDAB89;
var h2 = 0x98BADCFE;
var h3 = 0x10325476;
var h4 = 0xC3D2E1F0;
for(var i=0;i<x.length;i+=16) {
var j,t;
var a = h0;
var b = h1;
var c = h2;
var d = h3;
var e = h4;
for(j = 0;j<16;j++) {
w[j] = x[i+j];
t = add32x5(e,(a>>>27)|(a<<5),d^(b&(c^d)),w[j],k1);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
for(j=16;j<20;j++) {
w[j] = rol32(w[j-3]^w[j-8]^w[j-14]^w[j-16]);
t = add32x5(e,(a>>>27)|(a<<5),d^(b&(c^d)),w[j],k1);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
for(j=20;j<40;j++) {
w[j] = rol32(w[j-3]^w[j-8]^w[j-14]^w[j-16]);
t = add32x5(e,(a>>>27)|(a<<5),b^c^d,w[j],k2);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
for(j=40;j<60;j++) {
w[j] = rol32(w[j-3]^w[j-8]^w[j-14]^w[j-16]);
t = add32x5(e,(a>>>27)|(a<<5),(b&c)|(d&(b|c)),w[j],k3);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
for(j=60;j<80;j++) {
w[j] = rol32(w[j-3]^w[j-8]^w[j-14]^w[j-16]);
t = add32x5(e,(a>>>27)|(a<<5),b^c^d,w[j],k4);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
h0 = add32(h0,a);
h1 = add32(h1,b);
h2 = add32(h2,c);
h3 = add32(h3,d);
h4 = add32(h4,e);
}
return Array(h0,h1,h2,h3,h4);
};
}
//}}}
[[Table of Contents]]
[[Introduction]]
/***
|''Name:''|DeprecatedFunctionsPlugin|
|''Description:''|Support for deprecated functions removed from core|
***/
//{{{
if(!version.extensions.DeprecatedFunctionsPlugin) {
version.extensions.DeprecatedFunctionsPlugin = {installed:true};
//--
//-- Deprecated code
//--
// @Deprecated: Use createElementAndWikify and this.termRegExp instead
config.formatterHelpers.charFormatHelper = function(w)
{
w.subWikify(createTiddlyElement(w.output,this.element),this.terminator);
};
// @Deprecated: Use enclosedTextHelper and this.lookaheadRegExp instead
config.formatterHelpers.monospacedByLineHelper = function(w)
{
var lookaheadRegExp = new RegExp(this.lookahead,"mg");
lookaheadRegExp.lastIndex = w.matchStart;
var lookaheadMatch = lookaheadRegExp.exec(w.source);
if(lookaheadMatch && lookaheadMatch.index == w.matchStart) {
var text = lookaheadMatch[1];
if(config.browser.isIE)
text = text.replace(/\n/g,"\r");
createTiddlyElement(w.output,"pre",null,null,text);
w.nextMatch = lookaheadRegExp.lastIndex;
}
};
// @Deprecated: Use <br> or <br /> instead of <<br>>
config.macros.br = {};
config.macros.br.handler = function(place)
{
createTiddlyElement(place,"br");
};
// Find an entry in an array. Returns the array index or null
// @Deprecated: Use indexOf instead
Array.prototype.find = function(item)
{
var i = this.indexOf(item);
return i == -1 ? null : i;
};
// Load a tiddler from an HTML DIV. The caller should make sure to later call Tiddler.changed()
// @Deprecated: Use store.getLoader().internalizeTiddler instead
Tiddler.prototype.loadFromDiv = function(divRef,title)
{
return store.getLoader().internalizeTiddler(store,this,title,divRef);
};
// Format the text for storage in an HTML DIV
// @Deprecated Use store.getSaver().externalizeTiddler instead.
Tiddler.prototype.saveToDiv = function()
{
return store.getSaver().externalizeTiddler(store,this);
};
// @Deprecated: Use store.allTiddlersAsHtml() instead
function allTiddlersAsHtml()
{
return store.allTiddlersAsHtml();
}
// @Deprecated: Use refreshPageTemplate instead
function applyPageTemplate(title)
{
refreshPageTemplate(title);
}
// @Deprecated: Use story.displayTiddlers instead
function displayTiddlers(srcElement,titles,template,unused1,unused2,animate,unused3)
{
story.displayTiddlers(srcElement,titles,template,animate);
}
// @Deprecated: Use story.displayTiddler instead
function displayTiddler(srcElement,title,template,unused1,unused2,animate,unused3)
{
story.displayTiddler(srcElement,title,template,animate);
}
// @Deprecated: Use functions on right hand side directly instead
var createTiddlerPopup = Popup.create;
var scrollToTiddlerPopup = Popup.show;
var hideTiddlerPopup = Popup.remove;
// @Deprecated: Use right hand side directly instead
var regexpBackSlashEn = new RegExp("\\\\n","mg");
var regexpBackSlash = new RegExp("\\\\","mg");
var regexpBackSlashEss = new RegExp("\\\\s","mg");
var regexpNewLine = new RegExp("\n","mg");
var regexpCarriageReturn = new RegExp("\r","mg");
}
//}}}
You want to retrieve a file from an FTP site by using a Python program.
Use the {{{ftplib}}} module.
{{{
import ftplib
# Open the connection
ftp = ftplib.FTP("ftp.fbi.example.gov")
ftp.login("mulder", "trustno1")
# Change directory
ftp.cwd('pub/x-files')
# Write the output to a local file.
# It's a binary file, so be sure to open the file in
# binary mode.
f = open("trixie.jpg", "wb")
ftp.retrbinary("RETR trixie.jpg", f.write)
f.close()
# Close connection
ftp.quit()
}}}
The {{{ftplib}}} module provides an {{{FTP}}} class that represents a connection to an FTP server. To open the connection, you must provide a username and password.
Retrieving the file isn't completely straightforward, because the {{{FTP}}} class provides a highly flexible interface. There's no method which will retrieve a file named "{{{X}}}" and write it to a local file named "{{{X}}}"; instead, you have to handle such details yourself. When a file is retrieved via FTP using the {{{retr}}} and {{{retrbinary}}} methods, you have to provide two pieces of information: the filename, and a Python function. The Python function will be called for each chunk of the file's contents, receiving a a string containing the data; the function then has the job of writing the data to a file. Luckily, you can often find a built-in function or method that does what you need, as in the example:
{{{
file = open("trixie.jpg", "wb")
ftp.retrbinary("RETR trixie.jpg", file.write)
file.close()
}}}
{{{file}}} is a file object, whose {{{write(S)}}} method writes the string S to the file. In Python, you can use bound methods as callbacks; a bound method remembers the object that it belonged to. {{{file.write}}} is a bound method in this example. You can provide a custom function that does additional work beyond just writing out the data.
When retrieving a file in text mode, each line is returned without any trailing newline characters, so you can add the required line endings for your platform when writing each line out to a file.
{{{
f = open("sightings.txt", "w")
ftp.retrlines("RETR sightings.txt",
lambda s, w = f.write: w(s+"\n") )
f.close()
}}}
The following function prints a progress counter as the file is retrieved, by maintaining a count of the bytes read so far. On every call, the current byte count is padded out to 9 characters, printed, and followed by 9 backspace characters.
{{{
import sys
byte_count = 0
output_file = open('...', 'w')
def write_block(data):
global byte_count
byte_count = byte_count + len(data)
output_file.write( data )
# Output the current byte count and backspaces
sys.stdout.write('%9i%s' % (byte_count, 9*'\010') )
sys.stdout.flush()
}}}
{{{sys.stdout}}} has to be flushed after each {{{write()}}} call, in order to ensure that the output is actually sent to the screen as it's generated.
See also the section on "Passing Functions as Parameters". It discusses how to pass a function as a parameter to another function, and will help you to understand callbacks.
The terminal has to be put into cbreak mode, in order to be able to read single characters from the terminal.
{{{
import sys
def getch():
import os, tty
fd = sys.stdin.fileno()
tty_mode = tty.tcgetattr(fd)
tty.setcbreak(fd)
try:
ch = os.read(fd, 1)
finally:
tty.tcsetattr(fd, tty.TCSAFLUSH, tty_mode)
return ch
}}}
The [[Conversion Script]] that [[I|Rui Carmo]] used to convert the original HTML version into TiddlyWiki format shows a possible way to emulate {{{switch}}} statements in Python using exceptions:
{{{
...
try:
raise intern(str(tag))
except ('tr', 'thead'):
self.buffer += '\n'
except ('th'):
self.buffer += '|!'
except ('blockquote'):
self.buffer += '\n<<<\n'
except:
pass
}}}
This is actually not a very good example, since the use of {{{intern}}} is likely to be deprecated soon. Nevertheless, right now it ensures that {{{except}}} will deal with the results of string conversion properly.
! Problem:
You have a URL with hex-encoded characters like:
{{{
http://localhost/With%20Spaces.html
}}}
...and you want to decode the hex characters. [[PHP]] coders will immediately look for {{{urldecode()}}}, which does not exist in [[Python]].
! Solution:
Use {{{urllib.unquote()}}} or {{{urllib.unquote_plus()}}} instead.
Here's a one-liner that will add the English ordinal suffix to a number:
{{{
ord_text = lambda n: str(n) + {1: 'st', 2: 'nd', 3: 'rd'}.get(n % (10 < n % 100 < 14 or 10), 'th')
}}}
Originally from [[Paul Bissex|http://news.e-scribe.com/418]]
Eric wrote a [[SinglePageModePlugin|http://www.elsdesign.com/tiddlywiki/#SinglePageModePlugin]] for TiddlyWiki, which has been incorporated into the SiteMacros to make the Grimoire easier to read for people who prefer a "single item" view.
Suppose you have the following dictionary:
{{{
d = {'a':1, 'b':2, 'c':3}
}}}
And you want to filter it so that you can only see the keys in:
{{{
f = ['a','b']
}}}
The easiest way to go about that is to build a set of tuples inside a list comprehension and turn the result into a dictionary like so:
{{{
filtered = dict([(k,v) for k,v in d.items() if k in f])
}}}
For instance, it may be handy to grab some local variables (from {{{locals()}}}) and put them into a {{{dict}}}:
{{{
somelocals = dict([(k,v) for k,v in locals().items() if k in ['author','date','title']])
}}}
! Problem:
You need to generate random numbers for an application, such as generating test data or selecting an item from a list.
! Solution:
The standard library module <<PythonModule random>> implements a random number generator. Using it is simple:
{{{
import random
# Generate a random number between 0 and 1
randval = random.random()
# Generate a random integer between 45 and 55
randval = random.randint(45, 55)
# Choose a random value from a list
randname = random.choice( ["Glenna", "Jody", "Natalie"] )
}}}
! Discussion:
{{{random.random()}}} returns a random floating point number in the range [0, 1), generated using a Wichmann-Hill <<Wikipedia 'random number' Random_number>> generator. There are also other specialized generators in this module, built on top of the {{{random()}}} function:
* {{{randint(a, b)}}} chooses an integer in the range [a, b]
* {{{choice(S)}}} chooses a random item from the given sequence S
* {{{uniform(a, b)}}} chooses a floating point number in the range [a, b]
To specify the random number generator's initial setting, use {{{random.seed(x, y, z)}}}, which sets the seed from three integers in the range [1, 256]. There's also a {{{random}}} class, which you can instantiate to create independent multiple random number generators.
The <<PythonModule random>> module contains functions that approximate various standard distributions, such as a normal or Gaussian distribution, lognormal, gamma and beta distributions, and a few others. These functions are specialized enough to make me refer you to the Python Library Reference for the details.
Note that the {{{rand}}} module, which is occasionally used in some examples and demo programs, is now considered obsolete. Use <<PythonModule random>> instead.
A related problem to [[generating random numbers|Generating Random Numbers]] is generating a random permutation of numbers between 1 and N, with no repeats. For example, you might want to rearrange a list into a random order.
The solution is to generate a list containing the numbers from 1 to N, and use the {{{random.choice()}}} function to pick a random element. Remove the chosen element, and then repeat until the list is empty.
{{{
numbers = range(1, N+1)
while numbers:
j = random.choice(numbers)
numbers.remove(j)
print j
}}}
Remember that Python is extremely dynamic and that you can use this dynamism to configure a program at run-time to use available functionality on different platforms. For example you can test the sys.platform and import different modules based on its value.
{{{
import sys
if sys.platform == "win32":
import win32pipe
popen = win32pipe.popen
else:
import os
popen = os.popen
}}}
Also, you can try to import a module and use a fallback if the import fails:
{{{
try:
import really_fast_implementation
choice = really_fast_implementation
except ImportError:
import slower_implementation
choice = slower_implementation
}}}
This is commonly used to write code which will use a faster C extension module if it's installed, but will still work without the extension.
Some programming languages, such as the scripting language in MATLAB, allow calling a function and ignoring some of its return values. You need to do this in Python, ignoring some value returned from a function.
If the function returns a tuple, as done by most functions that return multiple results, treat the return value as a tuple and slice it to obtain the results that you need.
{{{
def get_rgb_colour(colour_name):
# ... set R, G, B, to the values
return R, G, B
# Get just the red component of a colour
tup = get_rgb_colour('teal')
R = tup[0]
}}}
This can be expressed more compactly, though less clearly, without a tuple variable as:
{{{
R = get_rgb_colour('teal')[0]
}}}
Yet another solution is to use a dummy variable name for the variables that aren't of interest. {{{_}}} is suggested as a short and rarely used name for this purpose.
{{{
R, _, _ = get_rgb_colour('methyl yellow')
}}}
For the sake of completeness, it should be mentioned that Perl supports an idiom for ignoring return values by assigning them to {{{undef}}}, the Perl counterpart of Python's {{{None}}}. (This idiom is rarely actually used in Perl programs.) Don't try the same thing in Python, which would be:
{{{
None, None, B = get_rgb_colour('da Vinci green')
}}}
This will run without errors, and {{{B}}} will get the expected value, but it also adds a new name to the local scope: {{{None}}} is now bound to some value. (Maybe that last sentence should end with a '!') Later uses of {{{None}}} in the same code will retrieve the assigned value, not the Python {{{None}}} object. In short, if you use the above idiom in Perl, don't attempt to use it in Python.
<<<
grimoire /grim''waar''/ (noun) ::= a book of magic spells and invocations. Origin: French, alteration of grammaire 'grammar'. A grammar is a description of a set of symbols and how to combine them to create well-formed sentences. A grimoire is a description of a set of magical symbols and how to combine them properly. It is sort of a recipe-book for magic spells.
See also the <<Wikipedia 'Wikipedia entry' 'Grimoire'>>.
<<<
The //Python Grimoire// explains how to perform common programming tasks in Python. It is a good place to go after you've read the Python Tutorial and have a reasonable grasp of the basics of the language. In essence, the //Grimoire// is a collection of small recipes for very basic tasks. Its purpose is similar to that of the [[Python Cookbook|http://aspn.activestate.com/ASPN/Python/Cookbook/]], but the //Grimoire//'s recipes are much simpler and more basic than those in the Cookbook. It does not cover advanced and/or specialized topics such as [[GUI toolkits]] and [[Jython]].
The //Grimoire//, then, is an unpolished document. It may still contain occasional typos, formatting bloopers, code errors, and out-of-date material.
After the conversion to TiddlyWiki format, several people used this Grimoire as the start of their very own Python notebook.
May you find it as useful to guide your path as they did.
<<<
Please keep in mind that //the Grimoire's file location will change soon//, so please don't link to it directly - link to [[this page|http://the.taoofmac.com/space/Python/Grimoire]] instead, which also holds links to the old version in plain HTML and PDF formats.
<<<
The rise of the Internet has greatly eased the life of network programmers, by reducing the importance of proprietary standards and replacing them with open protocols whose specifications are readily available. Various standard Internet protocols, such as the Simple Mail Transport Protocol ([[SMTP]]) for electronic mail, or the Web's HyperText Transfer Protocol ([[HTTP]]), now dominate their respective niches.
Python's standard library includes many modules that implement different Internet protocols, and some frameworks (like [[Twisted]]) take those implementations beyond the basics, allowing you to code just about any sort of client (or server) imaginable.
The basics, however, are well represented in the standard library, mostly in the form of classes like <<PythonModule SocketServer>> and <<PythonModule SimpleHTTPServer>>.
After creating a dictionary, you want to loop over the dictionary's contents and do something for each entry, such as printing them.
The {{{keys()}}} method of dictionaries returns a list of the dictionary's keys, arranged in essentially random order:
{{{
>>> for colr in fr_colours.keys(): print colr
...
yellow
red
black
white
blue
}}}
If you want to loop over the contents in some sorted order, you'll have to retrieve the list of keys and sort it before looping through it.
{{{
>>> keylist = fr_colours.keys() ; keylist.sort()
>>> for colr in keylist: print colr
...
black
blue
red
white
yellow
}}}
Remember that the {{{sort()}}} list method sorts a list in-place and returns {{{None}}}. {{{for colr in fr_colours.keys().sort()}}}will not work, because the value of {{{fr_colours.keys().sort()}}} is {{{None}}}.
''Discussion:''
The list returned by {{{keys()}}} is arranged randomly, because the key/value pairs are scattered across random bins in a hash table. Dictionaries therefore don't impose any ordering on their contents. For example, if you assemble a dictionary element-by-element, and print the dictionary after each new key/value pair is added, there's no order to how the contents are displayed.
{{{
>>> d = {} ; d['red'] = 'rouge' ; print d
{'red': 'rouge'}
>>> d['blue'] = 'bleu' ; print d
{'red': 'rouge', 'blue': 'bleu'}
>>> d['yellow'] = 'jaune' ; print d
{'yellow': 'jaune', 'red': 'rouge', 'blue': 'bleu'}
>>> d['green'] = 'vert' ; print d
{'yellow': 'jaune', 'red': 'rouge', 'green': 'vert', 'blue': 'bleu'}
}}}
There are two other methods available for retrieving the contents of a dictionary. If your loop will require each key and its corresponding value, you can use the {{{items()}}} method, which returns a list of {{{(key, value)}}} 2-tuples. Again, the list contents are arranged in a random order.
{{{
>>> for english, french in fr_colours.items():
... print english, '\t', french
...
yellow jaune
red rouge
black noir
white blanc
blue bleu
}}}
Least commonly used of all, the {{{values()}}} method returns a list containing only the values from a dictionary.
{{{
>>> for colr in fr_colours.values(): print colr
...
jaune
rouge
noir
blanc
bleu
}}}
The way to keep these method names straight is to remember that dictionaries map from keys to values; the {{{keys()}}} and {{{values()}}} methods return a list of the corresponding elements of the dictionary. That leaves {{{items()}}} to return a list of 2-tuples.
! Problem:
You have a sequence, and you wish to loop over its contents.
! Solution:
This is the purpose of the {{{for}}} statement:
{{{
for item in [1,2,3,4,5]:
print item
}}}
This will work for tuples and strings, too.
! Discussion:
To iterate over a list in reverse order, the traditional solution before Python 2.4 was to simply reverse the list:
{{{
List.reverse()
for item in list:
# ... do something with item ...
}}}
However, this modifies the list itself, so as of Python 2.4, a built-in {{{reverse}}} iterator is also available:
{{{
for item in reverse(List):
# do something with your item
}}}
This allows reverse iteration over any sequence without modifying the underlying sequence, and therefore allows easy iteration over immutable sequences such as strings or tuples:
{{{
s = "abcd"
s.reverse()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'str' object has no attribute 'reverse'
>>> for i in reversed(s):
... print i
...
d
c
b
a
}}}
An equivalent function, {{{sorted}}}, exists for sorting sequences.
<<<
''Note:'' This will probably require a modern Python example of its own.
<<<
Here's another way to iterate backwards when you have some other sequence type like a tuple or a string, or if reversing the list is unacceptable to you. You'll have to loop over the indices, from the length of the sequence minus one, down to zero.
{{{
for i in range(len(seq)-1, -1, -1):
item = seq[i]
# ... do something with item ...
}}}
The creator of TiddlyWiki, without whom the Grimoire would probably have stayed in static HTML format.
/***
|''Name:''|LegacyStrikeThroughPlugin|
|''Description:''|Support for legacy (pre 2.1) strike through formatting|
|''Version:''|1.0.2|
|''Date:''|Jul 21, 2006|
|''Source:''|http://www.tiddlywiki.com/#LegacyStrikeThroughPlugin|
|''Author:''|MartinBudden (mjbudden (at) gmail (dot) com)|
|''License:''|[[BSD open source license]]|
|''CoreVersion:''|2.1.0|
***/
//{{{
// Ensure that the LegacyStrikeThrough Plugin is only installed once.
if(!version.extensions.LegacyStrikeThroughPlugin) {
version.extensions.LegacyStrikeThroughPlugin = {installed:true};
config.formatters.push(
{
name: "legacyStrikeByChar",
match: "==",
termRegExp: /(==)/mg,
element: "strike",
handler: config.formatterHelpers.createElementAndWikify
});
} //# end of "install only once"
//}}}
! Problem:
You want to make a list of the files in a directory, in order to do something for every file.
! Solution
The {{{os.listdir(path)}}} function returns a list of strings containing all the contents of the directory specified by {{{path}}}, in no particular order. Note that the list will also include directory names in the resulting list.
{{{
>>> os.listdir('/usr')
['X11R6', 'bin', 'dict', 'doc', 'etc', 'games', ... ]
}}}
If you want to list all the files matching a wildcard specification such as {{{*.py}}}, you can implement it by checking each filename in the list returned by {{{os.listdir()}}}. The {{{glob()}}} function in the <<PythonModule glob>> module can do this for you:
{{{
>>> import glob
>>> glob.glob('/usr/lib/python1.5/B*.py')
['/usr/lib/python1.5/BaseHTTPServer.py', '/usr/lib/python1.5/Bastion.py']
}}}
{{{glob.glob()}}} will also list directories that match the wildcard specification.
Once you've gotten a list of filenames, you'll often want to limit it to those names that correspond to files and not directories. The {{{os.path}}} module contains functions called {{{isfile(path)}}}, {{{isdir(path)}}}, and {{{islink(path)}}} which return true if the path corresponds to a file, directory, or symbolic link.
These functions can be combined with the built-in {{{filter()}}} to easily select those paths that are actually files:
{{{
filelist = os.listdir('.')
filelist = filter(os.path.isfile, filelist)
}}}
! Problem:
When a file will be accessed and modified by several programs at the same time, you need to ensure that the programs can't make conflicting changes at the same time, which can result in the file being corrupted.
! Solution:
This is accomplished by locking the file.
The <<PythonModule posixfile>> module provides an object that acts like Python's standard file objects, but adds some extra methods. One of the extra methods is {{{lock(mode, len, start, whence)}}}. mode is a string specifying whether you want a read or write lock, or to give up an already-acquired lock.
For example, to gain a write lock for the entire file:
{{{
file.lock('w')
}}}
! Discussion:
Locks can be for reading or writing. Multiple read locks can be held by different processes, because several processes can read the same data at the same time without harm. Only a single write lock can be held at a given time, and read locks won't be granted while a write lock is being held.
When requesting a lock, the default is to lock the entire file. It's also possible to lock a part of the file, by specifying the start and length of a region in the file. This allows multiple processes to modify different parts of a file at the same time. Consult <<PythonModule posixfile>> for more information.
[[Introduction]]
[[Table of Contents]]
[[Credits]]
A useful trick you can pull off with Python is to set up a small web server at a moment's notice (mostly as a one-off file share for those occasions when you simply can't get files across any other way).
To do that, create a short shell script containing:
{{{
#!/bin/sh
python -c '__import__("SimpleHTTPServer").test()'
}}}
(or just alias the {{{python}}} invocation in your shell)
{{{SimpleHTTPServer}}} defaults to port 8000 and serving files from wherever you invoke it. It is not spectacularly smart, but should to a good enough job of serving simple demos and the odd file.
Be sure to use "Save Link As..." when downloading Office files from it, since it does not have all the [[MIME]] types defined.
! Problem:
You want to perform a calculation that requires operations beyond the basic ones of addition, subtraction, multiplication, division, and exponentiation. For example, you need to evaluate a sine or cosine.
! Solution:
Import the <<PythonModule math>> module, and use the functions that it provides.
{{{
import math
angle = math.pi / 6
x = math.cos( angle )
y = math.sin( angle )
}}}
If you want to get results that are complex numbers, use the <<PythonModule cmath>> module.
{{{
import cmath
result = cmath.sin( 1+3j )
}}}
''Discussion:''
Mathematical functions such as sine and cosine are contained in the <<PythonModule math>> module. The functions are identical to those defined by the C standard, since they're simply Python wrappers on top of those functions. The <<PythonModule math>> module also defines two constants: {{{pi}}}and {{{e}}}.
The functions in <<PythonModule math>> always return floating point numbers, whether the input was an integer or a float. If a result would be a complex number ({{{math.sqrt(-1)}}}, the square root of -1, is the common example), this is considered an error and triggers an exception. To get complex numbers as answers, you need to use the <<PythonModule cmath>> module, which is similar (though not identical) to the <<PythonModule math>> module, and can handle complex parameters and results:
{{{
>>> math.sqrt(-1)
Traceback (innermost last):
File "<stdin>", line 1, in ?
OverflowError: math range error
>>> cmath.sqrt(-1)
1j
}}}
Here's a list of the functions contained in the <<PythonModule math>> module. The <<PythonModule cmath>> module defines analogous functions that operate on complex numbers.
|!''Name''|!''Purpose''|
|{{{acos(x)}}}|Arc cosine of x|
|{{{asin(x)}}}|Arc sine of x|
|{{{atan(x)}}}|Arc tangent of x|
|{{{atan2(x,y)}}}|Arc tangent of {{{x/y}}}|
|{{{ceil(x)}}}|Ceiling of x; the largest integer equal to or greater than x|
|{{{cos(x)}}}|Cosine of x|
|{{{cosh(x)}}}|Hyperbolic cosine of x|
|{{{exp(x)}}}|e raised to the power of x|
|{{{fabs(x)}}}|Absolute value of x|
|{{{floor(x)}}}|Floor of x; the largest integer equal to or less than x|
|{{{fmod(x)}}}|x modulo y|
|{{{frexp(x)}}}|The matissa and exponent for x.|
|{{{hypot(x, y)}}}|Euclidean distance, {{{sqrt(x*x + y*y)}}}|
|{{{ldexp(x, i)}}}|{{{x * (2**i)}}}|
|{{{log(x)}}}|Natural logarithm of x|
|{{{log10(x)}}}|Base 10 logarithm of x|
|{{{modf(x)}}}|Return the fractional and integer parts of x|
|{{{pow(x, y)}}}|x raised to the power of y|
|{{{sin(x)}}}|Sine of x|
|{{{sinh(x)}}}|Hyperbolic sine of x|
|{{{sqrt(x)}}}|Square root of x|
|{{{tan(x)}}}|Tangent of x|
|{{{tanh(x)}}}|Hyperbolic tangent of x|
! Problem:
You have two dictionaries {{{d1}}} and {{{d2}}}. You wish to add the key/value pairs from {{{d2}}} to {{{d1}}}.
! Solution:
The {{{update}}} method of a dictionary adds the contents of another dictionary. To solve this problem, you can simply do: {{{d1.update(d2)}}}.
{{{d1.update(d2)}}} is equivalent to the following code:
{{{
for key, value in d2.items():
d1[ key ] = value
}}}
If the same key is associated with a value in both {{{d1}}} and {{{d2}}}, {{{update()}}} will replace the value in {{{d1}}} with the value from {{{d2}}}. {{{d2}}} is never modified by this method.
There are some excellent [[Python]] resources out there:
* [[Google]] [[Code Search|http://www.google.com/codesearch]] will help you find sample code for pretty much whatever you need to do.
! Problem:
You wish to create a list of lists.
! Solution:
This obvious solution for creating a list of {{{N}}} lists is wrong; see the Discussion for an explanation.
{{{
LL = [ [] ] * N
}}}
Instead, build an {{{N}}}-element list first, and then replace each element with a list.
{{{
LL = [None] * N
for i in range(N): LL[i] = []
}}}
If you want to fill out the contained lists with default values, simply replace {{{[]}}} with an expression like {{{N*[0]}}}. Elements could then be retrieved by accessing {{{LL[i][j]}}}.
! Discussion:
Why doesn't {{{[ [] ] * N}}} work as expected? When multiplying a list, Python copies the contents of the list //by reference//.
This means that you'll wind up with a list containing {{{N}}} references to the same objects, which is fine for immutable objects such as floating point numbers or tuples, but can be confusing with mutable objects such as lists. To make this clearer, consider the list constructed for {{{N=5}}}.
What happens when you append an element to the first list?
{{{
>>> LL = [ [] ] * N
>>> print LL
[[], [], [], [], []]
>>> LL[0].append(1)
>>> LL
[[1], [1], [1], [1], [1]]
}}}
All of the lists are modified, not just the first one! The {{{is}}} operator will tell you that all the elements of {{{LL}}} are actually the same list; a true value is returned by {{{LL[0] is LL[1]}}}.
A common use of multi-dimensional lists is to store matrices. Before reinventing the wheel and writing yet another matrix type, take a careful look at [[Numeric Python|http://www.python.org/topics/scicomp]], which includes a matrix type, interfaces to libraries of functions such as LINPACK, and lots of other modules useful for numeric work.
! Problem:
You need a dictionary that can store the key and more than one value. For example, after {{{d[key] = value1}}} and {{{d[key] = value2}}}, you want {{{d[key]}}} to return a list containing {{{[value1, value2]}}}. This differs from an ordinary dictionary, where assigning to the same key twice causes the first value to be deleted and replaced by the second.
! Solution:
The following class stores lists of values; setting a new value causes it to be appended to the list.
{{{
import UserDict
class MultipleDict(UserDict.UserDict):
def __setitem__(self, key, item):
if self.data.has_key( key ):
self.data[key].append( item )
else:
self.data[key] = [item]
}}}
Using the class is simple:
{{{
d = MultipleDict()
d[1] = 'a'
d[1] = 'b'
d[1] = 'c'
}}}
{{{d[1]}}} will return {{{['a', 'b', 'c']}}}. To remove the key, you'll have to delete the entire list of values with {{{del d[1]}}}.
This class subclasses the {{{UserDict}}} class from the <<PythonModule UserDict>> module, which implements all the methods of dictionaries in a Python class, and stores its contents in the {{{data}}} attribute, which is a real dictionary.
To implement the desired change in behaviour, we only have to override the {{{____setitem____}}} method, and store the right result in {{{self.data}}}.
<<<
''Note:'' Since Python 2.2, you can derive from the built-in types ("new style classes"), and {{{MultipleDict}}} would be best implemented as inheriting from {{{dict}}}.
<<<
! Problem:
Python's strings are immutable, so you can't change a single character of the string without constructing a new string. Sometimes this is a problem, particularly when dealing with very large strings; constructing a new string will require copying most of the original string, and can be slow.
! Solution:
Use the <<PythonModule array>> module, which provides a mutable array data type that can only hold values of a single type. If used to hold only characters, an array behaves similarly to a mutable string.
{{{
import array
A = array.array('c')
A.fromstring('hello there!')
print A
}}}
This prints:
{{{
array('c', 'hello there!')
}}}
The array object {{{A}}} is mutable, so you can modify an element in place:
{{{
A[0] = 'j'
print A
}}}
This will print:
{{{
array('c', 'jello there!')
}}}
Most functions that require strings, such as {{{string.split()}}}, won't accept array objects, so you'll have to convert the array to a string in order to pass the data to certain functions:
{{{
print string.split( A.tostring() )
}}}
But since both strings and arrays are [[sequence types|http://www.python.org/doc/2.4.1/lib/typesseq.html]], several forms of indexing and slicing can be performed on either.
If you need to automatically save and restore objects and other data structures to files, the {{{pickle}}} library module solves this in a very general way (though you still can't store things like open files, sockets or windows), and the library module {{{shelve}}} uses {{{pickle}}} and {{{(g)dbm}}} to create persistent mappings containing arbitrary Python objects. For possibly better performance, use the {{{cPickle}}} module.
A more awkward way of doing things is to use {{{pickle}}}'s little sister, {{{marshal}}}. The {{{marshal}}} module provides very fast ways to store noncircular basic Python types to files and strings, and back again. Although {{{marshal}}} does not do fancy things like store instances or handle shared references properly, it does run extremely fast. For example loading a half megabyte of data may take less than a third of a second (on some machines). This often beats doing something more complex and general such as using {{{gdbm}}} with {{{pickle}}}/{{{shelve}}}.
<div id='header'>
<div id='titleLine'>
<span id='siteTitle' refresh='content' tiddler='SiteTitle'></span>
<span id='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
</div>
</div>
<div id='sidebar'>
<div id='mainMenu' refresh='content' tiddler='MainMenu'></div>
<div id='sidebarOptions' refresh='content' tiddler='SideBarOptions'></div>
<div id='sidebarTabs' refresh='content' force='true' tiddler='SideBarTabs'></div><div id="licensePanel">
<a rel="license" href="http://shared.snapgrid.com/gtd_tiddlywiki.html#RevisionHistory" target="_new">GTDTW Version <span macro="version"></span></a>
<a rel="license" href="http://www.tiddlywiki.com" target="_new">
TiddlyWiki is published by Jeremy Ruston at Osmosoft under a BSD open source license</a>
<a rel="license" href="http://snapgrid.com" target="_new">GTD TiddlyWiki is a modification by Nathan Bowers at Snapgrid under the same license terms.</a>
<a rel="license" href="http://davidco.com" target="_new">"Getting Things Done" is © David Allen at Davidco. Davidco has no affiliation with TiddlyWiki or GTD TiddlyWiki.</a></div></div>
<div id='displayArea'>
<div id='messageArea'></div>
<div id='tiddlerDisplay'></div>
The [[Conversion Script]] is a very simple demo of how to parse clean, structured [[HTML]] using the standard <<PythonModule HTMLParser>> object. The <<PythonModule HTMLParser>> is actually a lot more flexible than it seems, and can be used for simple [[XML]] parsing as well.
However, if you have to deal with mis-formatted [[HTML]] or don't want to get into the complexity of Python [[XML]] bindings, a nice alternative might be [[BeautifulSoup|http://www.crummy.com/software/BeautifulSoup/]], which despite requiring some tinkering with (and probably more documentation and samples), can handle pretty much anything on the Web.
But any mention of [[XML]] parsing wouldn't be complete without mentioning [[RSS]] or [[Atom]] feeds, which are best handled with [[Mark Pilgrim]]'s [[Universal Feed Parser|http://feedparser.org/]]. currently the basis for many Python-based news aggregators.
! Problem:
You wish to parse some sort of structured input, such as a configuration file or some data file.
! Solution:
There are many possible ways to go about this.
For simple input parsing, the easiest approach is usually to split the line into whitespace-delimited words using {{{string.split()}}}, and to subsequently convert decimal strings to numeric values using {{{string.atoi()}}}, {{{string.atol()}}} or {{{string.atof()}}} If you want to use a delimiter other than whitespace, {{{string.split()}}} can handle that, and can be combined with {{{string.strip()}}} which removes surrounding whitespace from a string.
For more complicated input parsing, the <<PythonModule re>> module's regular expressions are better suited for the task, and are more powerful than C's {{{sscanf()}}}.
! Discussion:
A parser for configuration files is included as the <<PythonModule ConfigParser>> module in the standard library; you should take a look at it and see if it meets your needs.
Python programmers often choose to decree that software configuration files should written in <<RFC 822>> format, specifying names and their corresponding values; this allows using the parser in the <<PythonModule rfc822>> module to read the files. A sample configuration file might then look like this:
{{{
Title: Index to Python Information
Description: Python code, information, and documentation.
Keywords: Python, Python articles, Python documentation
Palette: gold
Sidebar: none
}}}
There's a contributed module that emulates {{{sscanf()}}}, written by Steve Clift and availabe from the Contributed Software section on [[here|http://www.python.org/ftp/python/contrib-09-Dec-1999/Misc/sscanfmodule.c.Z]].
If you're trying to parse some sort of well-known file format, it's possible that a Python module has already been written to deal with it. Some common cases are:
* [[The Python Imaging Library|http://www.pythonware.com/]] can read many different graphics formats, ranging from well-known ones such as [[GIF]] and [[JPEG]], to more specialized formats such as [[DCX]] and [[TIFF]].
* Support for many scientific file formats has been implemented for use with Numeric Python; consult the [[Scientific Computing|http://www.python.org/topics/scicomp/]] topic guide for more information.
* The [[XML]] [[topic guide|http://www.python.org/topics/xml/]] tracks the available software for processing [[XML]] with Python.
Before spending a lot of effort implementing a module for a new file format, do some research first and see if someone has already done it; you might save yourself a lot of work.
! Problem:
You want to parse an URL into its component parts
! Solution:
Look at the <<PythonModule urlparse>> module, which does it all for you down to every last component part:
{{{
import urlparse
(schema,netloc,path,parameters,query,fragment) = urlparse.urlparse(url)
}}}
! Problem:
You wish to pass a function as an argument to another function.
! Solution:
Python is flexible enough to treat functions and methods in the same way as ordinary variables.
{{{
F = f.write
F(s)
}}}
is the same thing as:
{{{
f.write(s)
}}}
As a computer scientist would describe it, functions are first-class objects in Python. So, you can simply pass a function as one of the arguments:
{{{
def repeat_func(func, N, data):
"Repeat a function N times"
for i in range(N):
data = func(data)
return data
repeat_func(differentiate, 3, data)
}}}
It's also possible to store functions in Python data structures, such as variables, lists, tuples, and class instances. For example, the following function takes a list of functions, and calls them all, from left to right.
{{{
def compose(func_list, data):
for F in func_list:
data = F(data)
return data
}}}
! Discussion:
A common application of this is to keep a dictionary of handler functions, retrieving the function to call. The following toy example has a dictionary mapping command names to the functions that should be called:
{{{
def output_status():
...
def set_bf_mode():
...
... other function definitions ...
cmd_dict = {'status': output_status, 'brightfield', set_bf_mode,
'darkfield': set_df_mode, 'icr': set_icr_mode}
def call_command(cmd_name):
function = cmd_dict[cmd_name]
function()
}}}
It's not necessary to actually have a {{{def}}} statement in order to create a function. The {{{lambda}}} keyword is used to create small unnamed functions. The function is limited to a single expression, but that's enough to be useful in many cases. A common use for {{{lambda}}} is in concert with built-in functions like {{{map()}}}, which loops over the elements of a list, performs a function on each list element, and returns a list containing the results.
{{{
>>> map(string.upper,
... ["Here's", "a", "list", "of", "words"] )
["HERE'S", 'A', 'LIST', 'OF', 'WORDS']
}}}
{{{lambda}}} would be used if there's no built-in function corresponding to what you need. For example, the following code would take the first character of every word in the list:
{{{
>>> map(lambda word: word[0],
... ["Here's", "a", "list", "of", "words"] )
['H', 'a', 'l', 'o', 'w']
}}}
Here's another example. In this case, a default argument is used to bind the {{{f.write()}}} method to the local variable {{{wr}}}, and added a newline before calling the write method.
{{{
ftp.retrlines("RETR sightings.txt",
lambda s, wr = f.write: wr( s+"\n" ) )
}}}
! Problem:
You want to do an XML-RPC ping, such as the one required by [[Technorati|http://technorati.com]] or other blog tracking services.
! Solution:
Here's a function that uses the <<PythonModule xmlrpclib>> module:
{{{
import xmlrpclib
def doPings(siteinfo):
try:
for target in siteinfo['ping']:
if target == 'technorati':
print "Pinging Technorati..."
server = xmlrpclib.Server('http://rpc.technorati.com/rpc/ping')
print server.weblogUpdates.ping(siteinfo['sitetitle'], siteinfo['ping'][target])
except:
pass
}}}
Feel free to expand the exception handler to deal with any errors that may occur.
You want to read through a file line by line. This is the most common way of processing text files. For example, if you're writing a Python program that searches for text in a file, you'll have to loop through the file contents on a line by line basis.
You can code the loop explicitly, like this:
{{{
file = open('/tmp/filename', 'r')
while True:
line = file.readline()
if line == "": break # Check for end-of-file
do_something(line)
file.close()
}}}
Or like this, which reads all of the lines of the file into a list:
{{{
file = open('/tmp/filename', 'r')
lines = file.readlines()
file.close()
for line in lines:
do_something(line)
}}}
The {{{fileinput}}} module makes this even simpler by handling the loop for you:
{{{
import fileinput
for line in fileinput.input("/tmp/filename"):
do_something(line)
}}}
For sheer simplicity, it's hard to top reading all the lines from the file into a list, like this:
{{{
file = open('/tmp/filename', 'r')
lines = file.readlines()
file.close()
for line in lines:
do_something(line)
}}}
Or like this:
{{{
file = open('/tmp/filename', 'r')
for line in file.readlines()
do_something(line)
}}}
This is easy to code, but it does require reading the entire file into memory. These days, most systems will have enough memory to effortlessly handle files a few hundreds of kilobytes long. A 10 megabyte file will cause problems - swapping to disk, if not an actual crash - for many systems. Use your common sense; if you're pretty sure you won't need to handle large files, use {{{file.readlines()}}}; otherwise, use either of the two suggested solutions.
To loop over multiple files using fileinput, use a sequence of filenames (a list, tuple, etc) instead of just a single file.
{{{
for filename in ("file1", "file2", "file3"):
file = open(filename, 'r')
lines = file.readlines()
file.close()
for line in lines:
do_something(line)
}}}
If you omit the names, they default to {{{sys.argv[1:]}}}, or to standard input if no arguments were given.
! Problem:
You want to process a file, but starting with the last lines first.
! Partial Solution:
If you can read the entire file into memory, you can simply read all the data into a list and then reverse the list:
{{{
L = open('myFile').readlines()
L.reverse()
}}}
On a character-by-character basis, a similarly memory-bound solution is:
{{{
import string
L = string.split(open('thefile', 'r').read(), '')
L.reverse()
}}}
You can use the {{{sizehint}}} parameter to {{{readlines()}}} to get some of the efficiency of {{{readlines()}}} without reading in the whole file. The following code isn't optimized, but it shows the idea:
{{{
class BufferedFileReader:
def __init__ (self, file):
self.file = file
self.lines = []
self.numlines = 0
self.index = 0
def readline (self):
if (self.index >= self.numlines):
self.lines = self.file.readlines(65536)
self.numlines = len(self.lines)
self.index = 0
if (self.numlines == 0):
return ""
str = self.lines[self.index]
self.index = self.index + 1
return str
}}}
Use the struct module. It allows you to take a string read from a file containing binary data (usually numbers) and convert it to Python objects; and vice versa.
For example, the following code reads two 4-byte integers in big-endian format from a file:
{{{
import struct
f = open(filename, "rb") # Open in binary mode for portability
s = f.read(8)
x, y = struct.unpack(">ll", s)
}}}
The '>' in the format string forces big-endian data; each letter 'l' reads one "long integer" (4 bytes) from the string.
You can refer to the Library Reference for more details about the struct module.
! Problem:
You want to read a file that's been compressed with the GNU {{{gzip}}} program, or want to write a compressed file that {{{gzip}}} can read.
! Solution:
The <<PythonModule gzip>> module provides a {{{GzipFile}}} class for reading and writing {{{gzip}}}'ped files. {{{GzipFile}}} instances imitate the methods of Python's standard file objects. To read a file:
{{{
import gzip
# Open the file in 'r' mode for reading
file = GzipFile('data.gz', 'r')
line = file.readline() # Read a single line
data = file.read(1024) # Read 1K of data
file.close()
}}}
Writing compressed data is much the same:
{{{
import gzip
# Open the file in 'w' mode for writing
file = GzipFile('output.gz', 'w')
file.write( 'First line of output\n')
file.close()
}}}
! Discussion:
It's legal to use the {{{'a'}}} mode to append data to a compressed file; the {{{gzip}}} file format can handle a file with several chunks of compressed data. When reading such a file with {{{GzipFile}}}, you won't be able to tell where one chunk leaves off and the next begins because {{{GzipFile}}} seamlessly handles the transition between them.
The <<PythonModule gzip>> module is built on top of the <<PythonModule zlib>> module, which compresses strings of data. The <<PythonModule zlib>> module can be useful on its own to save disk space or network bandwidth.
For example, you can compress data before sending it over a TCP/IP socket or storing it in a DBM file. Here's a short example:
{{{
>>> import zlib
>>> s = "This is a test of the emergency broadcast chicken."
>>> comp = zlib.compress(s)
>>> comp
'x\234\013\311\310,V\000...'
>>> print zlib.decompress( comp )
This is a test of the emergency broadcast chicken.
}}}
The <<PythonModule zlib>> module also provides compressor and decompressor objects that can be used to compress large amounts of data without having to squeeze all of it into memory.
! Problem:
You want to process not merely a single directory, but an entire hierarchy of files and folders, and perhaps find a specific set of files.
! Solution:
Import the <<PythonModule os>> module, and use {{{os.path.walk(path, function, arg)}}}. That recursively walks through the tree rooted at {{{path}}}, and calls {{{function()}}} in each directory that's visited.
{{{function()}}} must accept 3 arguments: ({{{arg}}}, {{{dirname}}}, {{{name_list}}}). {{{arg}}} is the same as the value passed to {{{os.path.walk}}}; {{{dirname}}} is the current directory name; {{{name_list}}} is a list of the names of the directories and filenames in the directory. You can modify {{{name_list}}} in place to avoid traversing certain subdirectories.
Here's a simple program to find files:
{{{
import os, sys
def find(arg, dirname, names):
if arg in names:
print os.path.join(dirname, arg)
os.path.walk(sys.argv[1], find, sys.argv[2])
}}}
Sample usage:
{{{
$ python ~/t.py /tmp/ README
/tmp/xml-0.4/README
/tmp/xml-0.4/demo/README
/tmp/xml-0.4/dom/README
}}}
! Problem:
You have a list, and wish to ensure that each item in the list is unique, only occurring once. For example, if your list is {{{[1,4,5,2,4]}}}, you would want to remove one of the 4's.
! Solution:
If you can reorder the list, sort it to put duplicate values next to each other. If the list is sorted, then removing duplicates requires just a single loop. Note that the opening test for the length of the list is required - without it, if the list is empty, this code will raise an exception.
{{{
if len(List) > 1:
List.sort() # Sort the list
# Walk from the end of the list to the beginning;
# if two elements are identical, delete one of them.
last = List[-1] # Start with the last element
for i in range(len(List)-2, -1, -1):
if last==List[i]:
del List[i]
else:
last=List[i]
}}}
If all the elements of the list can be used as dictionary keys, you can create a dictionary with the list elements as keys, and take the result of the dictionary's {{{keys()}}} method as the new list.
{{{
d = {}
for x in List: d[x] = x
List = d.keys()
}}}
! Problem:
You want to rename a file on the filesystem, and have already figured out how to deal with your operating system's naming conventions inside Python.
! Solution:
Simple: use {{{os.rename}}}:
{{{
#Rename file with filename f1 to filename f2
os.rename(f1,f2)
}}}
You want to retrieve a file through HTTP. The data returned might be the HTML text of a Web page, an image such as a GIF or JPEG, or a binary file such as a JAR or TGZ file.
The easiest course is to use the {{{urllib}}} module, which will try to open an arbitrary URL and return a file object which can be read from to retrieve the data.
{{{
import urllib
# Open the URL
input = urllib.urlopen('http://www.example.com/info/index.html')
# Read all the data from the file as a single string
data = input.read()
input.close()
}}}
An alternative solution that operates at a lower level is to use the {{{httplib}}} module to perform the HTTP transaction yourself.
{{{
import httplib
h = httplib.HTTP('www.example.com')
h.putrequest('GET', '/info/index.html')
h.putheader('Accept', 'text/html')
h.endheaders()
errcode, errmsg, headers = h.getreply()
if errcode == 200:
f = h.getfile()
data = f.read() # Print the raw HTML
}}}
Some pages require a user name and password; Web browsers will prompt you for them when required. The {{{urlopen()}}} function will prompt the user and request the user name and password on standard input. Such user intervention is unsuitable for fully automatic Web page retrievals, so you should specify the user name and password when retrieving such a password-protected page automatically. The user name and password can be specified in the URL, when it's written in the form {{{http://user:password@host/...}}}.
Fredrik Lundh suggests an alternative approach that's more flexible, but is also more complicated. The {{{urllib}}} module contains several classes for opening URLs using different protocols, one such class being {{{FancyURLopener}}}. The {{{urlopen()}}} function simply creates an instance of {{{FancyURLopener}}} and uses it to access URLs. When input from the user is required, the {{{prompt_user_passwd()}}} method is called. Fredrik suggests the following approach: subclass {{{FancyURLopener}}} and add attributes to hold the username and password. {{{prompt_user_passwd()}}} is then overridden to simply return the user name and password without user intervention.
{{{
import urllib
class myURLOpener(urllib.FancyURLopener):
def setpasswd(self, user, passwd):
self.__user = user
self.__passwd = passwd
def prompt_user_passwd(self, host, realm):
return self.__user, self.__passwd
urlopener = myURLOpener()
urlopener.setpasswd("mulder", "trustno1")
fp = urlopener.open("http://www.secretlabs.com")
print fp.read()
}}}
! Problem:
You want to have a function that returns more than one value as its result. For example, you may have a function that takes a colour name, and returns three numbers, the red, green, and blue values for that colour.
! Solution:
The simplest and most common solution is to return a tuple containing the different values:
{{{
def get_rgb_colour(colour_name):
# ... set R, G, B, to the values
return R, G, B
}}}
! Discussion:
An alternative solution, using a class instance to hold return values, is useful if you want to return very many values, or differing numbers of values in different cases. You would set instance values of the class instance, and then return the instance to the caller.
{{{
class RGBSetting: pass
def get_rgb_colour(colour_name):
retval = RGBSetting()
# ... set retval.R, retval.G, retval.B, to the values
return retval
}}}
! Problem:
You want to reverse a string ({{{cba}}} from {{{abc}}}), and yet you cannot find a {{{strings}}} method do do it.
! Solution:
There is a rather hackish one-liner to reverse a string, which relies on [[extended slices|http://www.python.org/doc/2.3.5/whatsnew/section-slices.html]]:
{{{
>>> text='foobar'
>>> print text[::-1]
raboof
}}}
Email address: rui -dot- carmo [at] gmail -dot- com
Current steward of the [[Grimoire]] and responsible for managing its TiddlyWiki incarnation, plus adding random bits and pieces. Can also be [[contacted|http://the.taoofmac.com/space/RuiCarmo]] through [[his site|http://the.taoofmac.com]].
''Problem:''
Given a string, you wish to replace all the occurrences of one substring with different text. For example, you wish to replace all occurrences of the string {{{'USER-NAME'}}} with the string {{{'Joseph Addison'}}} in the string contained in the variable {{{data}}}.
''Solution:''
For simple string substitutions, the {{{string.replace}}} function will be the simplest and fastest solution.
{{{
import string
newdata = string.replace(data, 'USER-NAME', 'Joseph Addison')
}}}
If {{{data}}} contains {{{"USER-NAME has been added to the list."}}}, then after the above line is executed, {{{newdata}}} will contain {{{'Joseph Addison has been added to the list.'}}}.
''Discussion:''
<<<
''Note:'' the following has not yet been revised to account for [[Unicode]]/[[UTF-8]] details, and so should be taken with a grain of salt in modern Python - but the principles are sound.
<<<
String replacement is a common operation, and there are several ways to do it. The fastest and simplest way is the {{{string.replace()}}} function, which can only replace a fixed string with another fixed string. For example, the following line replaces the Latin-1 character 219, a capital U with a circumflex accent (^), with the HTML entity {{{Û}}}.
{{{
newdata = string.replace(data, chr(219), 'Û')
}}}
The {{{chr()}}} built-in function takes an integer between 0 and 255, and returns a string of length 1 that contains the character with that byte value.
Multiple replacements will require multiple calls to {{{string.replace()}}}:
{{{
newdata = string.replace(data, chr(219), 'Û')
newdata = string.replace(data, chr(233), 'é')
}}}
The replacement string is fixed, so it can't be varied depending on the string that was matched. For cases that require matching variable strings, such as "match anything between square brackets", or that require varying the replacement string, you'll have to use regular expression matching, available through the built-in <<PythonModule re>> module.
The following example uses regular expressions to replace URLs in a string with the [[HTML]] for a link to that URL, with the URL as the link text.
{{{
import re
data = """http://www.freshmeat.net
ftp://ftp.python.org/pub/python/src/
"""
newdata = re.sub(r"""(?x)
( # Start of group 1
(http|ftp|mailto) # URL scheme
: # Separating colon
\S+ # Everything up to the next whitespace character
) # End of group 1
""", r'<a href="\g<1>">\g<1></a>', data)
}}}
Notice that the order of the arguments to {{{re.sub}}} is different from the arguments for {{{string.replace()}}}. {{{string.replace()}}} takes the arguments (string, substring, replacement), while {{{re.sub()}}} takes the arguments (pattern, replacement, string). {{{re.sub()}}} has a different ordering of its arguments for consistency with the other functions in the {{{re}}} module; the regular expression pattern is viewed as the most important argument, so it's always passed as the first argument.
Regular expression patterns have a complicated syntax. Let's dissect the above pattern into its components.
* {{{(?x)}}} Specifies that this pattern is expressed in verbose mode. Most whitespace will be ignored, so you can format the pattern neatly, and comments can be embedded in the pattern by preceding them with a "{{{#}}}".
* {{{(http|ftp|mailto)}}} The parenthesized group lists several alternative strings, separated by "{{{|}}}" characters. Any one of these strings can produce a successful match for this component of the pattern.
* {{{\S+}}} The {{{\S}}} special sequence matches any character that isn't a whitespace character. Whitespace characters are space "{{{ }}}", tab "{{{\t}}}",newline "{{{\n}}}", carriage return "{{{\r}}}", form feed "{{{\f}}}", and vertical tab "{{{\v}}}". The "{{{+}}}" character is a qualifier that specifies how many times the previous component should be repeated; "{{{+}}}" indicates that the {{{\S}}} should be repeated one or more times.
The replacement string can contain sequences which will contain pieces of the matching string. For example, {{{\g<1>}}} will be replaced by the contents of the first group, which in this case will contain the whole URL that matches the regular expression. The replacement string is therefore {{{'<a href="\g<1>">\g<1></a>'}}}, which will contain the text of the URL at two different places, along with the required HTML.
The <<PythonModule re>> module can perform simple substitutions that are equally possible with {{{string.replace()}}}. This is simply a matter of writing a pattern that doesn't use any of the regular expression syntax:
{{{
data = re.sub('USER-NAME', 'Joseph Addison', data)
}}}
Try not to use regular expressions when a simpler fixed-string replacement can do the job, because you're paying a speed penalty for the extra generality of regular expressions.
For much more information on regular expressions in Python, consult the [[Regular Expression HOWTO|http://www.python.org/doc/howto/regex/]].
! Problem:
You want to send cookies as part of an HTTP transaction
! Solution:
If you're using the <<PythonModule httplib>> module, you can add a Cookie header with the name/value pairs (mind the encoding if you're sending special characters - use the <<PythonModule cgi>> module for that):
{{{
import httplib
conn = httplib.HTTPConnection( "localhost" )
Headers = {"Cookie" : "Name=Value"}
conn.request("GET", "/start", None, Headers )
response = conn.getresponse()
data = response.read()
}}}
! Problem:
You want to send an e-mail message that has been generated by Python code. Many applications require sending e-mail automatically. Most Web sites have a feedback form where submissions will be sent to maintenance staff. Python programs that are executed automatically may send their output to the person responsible for them, or may send an e-mail if the program can't run successfully because of an error condition, such as a full disk partition.
! Solution:
The way to do it is to use the <<PythonModule smtplib>> module:
{{{
import smtplib
message = """From: [email protected]
Subject: Sample E-mail
Hi!
This message is just to say hello, and has been automatically generated.
"""
S = smtplib.SMTP('mail.example.com')
errdict = S.sendmail( from_addr = '[email protected]',
to_addrs = ['[email protected]'],
msg = message)
if len(errdict) != 0:
print 'Not all addresses accepted'
print 'Failed addresses:', errdict.keys()
}}}
The Simple Mail Transfer Protocol ([[SMTP]]) is the most common protocol used for sending mail. On Unix systems, [[SMTP]] mail is most commonly handled by the {{{sendmail}}} daemon, though there are several alternative mail transport agents, such as {{{exim}}} and {{{qmail}}}, that also implement [[SMTP]].
The format of [[SMTP]] e-mail messages is defined in <<RFC 822>>. A message consists of a header followed by the actual content of the message. The end of the header is indicated by a blank line. A sample <<RFC 822>> message looks like this:
{{{
Return-Path: <amk>
Received: (from amk@localhost)
by 207-172-99-98.s98.tnt13.ann.erols.com (8.8.7/8.8.7) id QAA00866
for amk; Sun, 14 Feb 1999 16:55:57 -0500
Date: Sun, 14 Feb 1999 16:55:57 -0500
From: "A.M. Kuchling" <[email protected]>
CC: [email protected]
Message-Id: <[email protected]>
To: [email protected]
Subject: Hi!
Hello! Long time no hear...
}}}
Each header line contains a header name, such as "{{{From}}}" or "{{{CC}}}". Header names are case-insensitive, so it doesn't matter whether the name is "{{{FROM}}}" or "{{{From}}}". A colon separates the header name from the header's contents. <<RFC 822>> defines various standard header names for specifying the recipient's address, the sender's name, addresses to which carbon copies should be sent, etc.
You can add your own headers by following a naming convention; header names beginning with "{{{X-}}}" are considered extensions, and won't be mangled or dropped by the programs transporting the mail. For example,if you're writing a set of scripts which will send an e-mail on completion, you might want to add a header giving the name of the script sending the mail. This header could be called "{{{X-Script-Name}}}". Don't invent headers whose names don't begin with "{{{X-}}}", because a mail transport agent might drop them along the path from the sender to the recipient. To create headers, simply add them to the header of your message:
{{{
message = """From: [email protected]
Cc: [email protected]
X-Script-Name: mailnotify.py
Subject: Sample E-mail
Hi! ... """
}}}
Proprietary mail systems, such as cc:Mail from Lotus, or platform-specific interfaces such as Microsoft's MAPI, don't use [[SMTP]], so the <<PythonModule smtplib>> module won't help you use such systems. If you're trying to use such a system, you'll have to either access them through some platform-specific mechanism such as Microsoft's COM, or write a [[C]] extension module specifically for the mail interface.
On Unix-like systems, it's also possible to send mail by running a mail program (such as {{{/usr/lib/sendmail}}}) in a subprocess and sending it the message over a pipe; this can be done by the {{{os.popen()}}} function. Using a mail program has some drawbacks; it assumes that mail service is correctly configured on the machine, and the location of the sendmail program may vary between systems; {{{/usr/lib/sendmail}}} is most common, but alternative locations such as {{{/usr/sbin/sendmail}}} have been seen. (Refer to the manual page for {{{sendmail}}}.) Worst of all, if an error occurs, the subprocess may just print an error message which will then be ignored by your program. Using <<PythonModule smtplib>> is often the better course.
Here's an example, in case you still need to send mail this way:
{{{
SENDMAIL = "/usr/sbin/sendmail" # sendmail location
import os
p = os.popen("%s -t" % SENDMAIL, "w")
p.write("To: [email protected]\n")
p.write("Subject: test\n")
p.write("\n") # blank line separating headers from body
p.write("Some text\n")
p.write("some more text\n")
sts = p.close()
if sts != 0:
print "Sendmail exit status", sts
}}}
// HTTP read-only options
config.options.chkHttpReadOnly = true;
/***
|Name|SiteMacros|
|Created by|Rui Carmo|
|Version|0.2|
|Requires|~TW2.x|
!Description
Macros used internally to format links to external sites and documentation
!History
* 19-Sep-07, version 0.2
!Examples
|!Source|!Output|h
|{{{<<helloWorld dude>>}}}|<<helloWorld dude>>|
|{{{<<helloWorld 'to everyone'>>}}}|<<helloWorld 'to everyone'>>|
(You can use (single or double) quotes or double square brackets for params with spaces)
!Notes
This is intended to help you get started with customising your TW. To make the macro work you have to give this tiddler a tag of systemConfig then save and reload. To learn more about customising Tiddlywiki? Look at other people's plugins or... click View, Source in your browser and start reading. :)
!Code
***/
//{{{
config.macros.helloWorld = {};
config.macros.helloWorld.handler = function (place,macroName,params,wikifier,paramString,tiddler) {
var who = params.length > 0 ? params[0] : "world";
wikify("Hello //" + who + "// from the '" + macroName + "' macro in tiddler [[" + tiddler.title + "]].", place);
}
// Links to Python library module documentation
config.macros.PythonModule = {url: "http://www.python.org/doc/2.4.1/lib/module-"};
config.macros.PythonModule.handler = function (place,macroName,params,wikifier,paramString,tiddler) {
wikify('[[' + params[0] + '|'+ this.url + params[0] + '.html]]',place);
}
// Wikipedia links
version.extensions.Wikipedia = {major: 0, minor: 1, revision: 0};
config.macros.Wikipedia = {url: "http://en.wikipedia.org/wiki/"};
config.macros.Wikipedia.handler = function (place,macroName,params,wikifier,paramString,tiddler) {
wikify('[[' + params[0] + '|'+ this.url + params[1] + ']]',place);
}
// RFCs
version.extensions.RFC = {major: 0, minor: 1, revision: 0};
config.macros.RFC = {url: "http://www.faqs.org/rfcs/rfc"};
config.macros.RFC.handler = function (place,macroName,params,wikifier,paramString,tiddler) {
wikify('[[RFC ' + params[0] + '|'+ this.url + params[0] + '.html]]',place);
}
// Untagged List Handler
config.macros.list["untagged"] = {prompt: "Tiddlers that are not tagged"};
config.macros.list.untagged.handler = function (place,macroName,params,wikifier,paramString,tiddler) {
//displayMessage("Building list of untagged tiddlers");
var results = [];
for(var t in store.tiddlers) {
var tiddler = store.tiddlers[t];
if(tiddler.getTags() == "")
results.push(t);
}
results.sort();
return results;
}
// Eric Shulman - ELS Design Studios
// "SinglePageMode" Plug-in for TiddlyWiki version 1.2.31 or above
version.extensions.SinglePageMode= {major: 1, minor: 0, revision: 0, date: new Date(2005,8,13)};
if (config.options.chkSinglePageMode==undefined) config.options.chkSinglePageMode=false;
config.shadowTiddlers.AdvancedOptions += "\n<<option chkSinglePageMode>> Display one tiddler at a time";
window.coreDisplayTiddler=window.displayTiddler;
window.displayTiddler = function(src,title,state,highlightText,highlightCaseSensitive,animate,slowly)
{
if (config.options.chkSinglePageMode) closeAllTiddlers();
coreDisplayTiddler(src,title,state,highlightText,highlightCaseSensitive,animate,slowly);
}
//}}}
a hypertext tome pertaining to that most elegant of languages
! Problem:
You have a dictionary {{{foo}}}, and wish to sort it by value (not key)
! Solution:
Convert {{{foo}}} to a list of tuples like so:
{{{
sorted(foo.items(), key=lambda(k,v):(v,k))
}}}
This will turn:
{{{
{'a': 2, 'b': 4, 'c': 3, 'd': 1}
}}}
into
{{{
[('d', 1), ('a', 2), ('c', 3), ('b', 4)]
}}}
/***
|''Name:''|SparklinePlugin|
|''Description:''|Sparklines macro|
***/
//{{{
if(!version.extensions.SparklinePlugin) {
version.extensions.SparklinePlugin = {installed:true};
//--
//-- Sparklines
//--
config.macros.sparkline = {};
config.macros.sparkline.handler = function(place,macroName,params)
{
var data = [];
var min = 0;
var max = 0;
var v;
for(var t=0; t<params.length; t++) {
v = parseInt(params[t]);
if(v < min)
min = v;
if(v > max)
max = v;
data.push(v);
}
if(data.length < 1)
return;
var box = createTiddlyElement(place,"span",null,"sparkline",String.fromCharCode(160));
box.title = data.join(",");
var w = box.offsetWidth;
var h = box.offsetHeight;
box.style.paddingRight = (data.length * 2 - w) + "px";
box.style.position = "relative";
for(var d=0; d<data.length; d++) {
var tick = document.createElement("img");
tick.border = 0;
tick.className = "sparktick";
tick.style.position = "absolute";
tick.src = "data:image/gif,GIF89a%01%00%01%00%91%FF%00%FF%FF%FF%00%00%00%C0%C0%C0%00%00%00!%F9%04%01%00%00%02%00%2C%00%00%00%00%01%00%01%00%40%02%02T%01%00%3B";
tick.style.left = d*2 + "px";
tick.style.width = "2px";
v = Math.floor(((data[d] - min)/(max-min)) * h);
tick.style.top = (h-v) + "px";
tick.style.height = v + "px";
box.appendChild(tick);
}
};
}
//}}}
There is a problem that routinely bedevils new Python users who come from a Windows or DOS background. The problem is that on Windows, a backslash is a separator in filenames, whereas on Unix (where Python originated) the backslash has an entirely different function (as an escape character) and the //forward// slash is the filename separator character.
When a person from a Windows background starts having problems with filenames, and goes looking for a solution, the first thing that he (or she) usually finds is Python's "raw" strings. However, from the perspective of a Windows user, raw strings aren't truly raw they are only semi-raw, as you will rudely discover the first time that you use a raw string to specify a filename (or filename part) that ends with a single backslash.
A better strategy is to routinely code your filenames using forward slashes, and then use a Python function such as {{{os.path.normcase}}} to change the separator to whatever is appropriate to the local operating system. A nice bonus of ths strategy is that it is platform independent
{{{
import os.path
myFilename = "c:/mydir/myfile.txt"
myFilename = os.path.normcase(myFilename)
}}}
! Problem:
You want to split a string starting from the far (i.e. right-hand) side yet you cannot find a {{{strings}}} method do do it ({{{rsplit}}} was only introduced in 2.4).
! Solution:
Here's an approach that leverages the [[Reversing Strings]] one-liner:
{{{
def rsplit(s, sep=None, maxsplit=-1):
"""
Equivalent to str.split, except splitting from the right.
"""
if sys.version_info < (2, 4, 0):
if sep is not None:
sep = sep[::-1]
L = s[::-1].split(sep, maxsplit)
L.reverse()
return [s[::-1] for s in L]
else:
return s.rsplit(sep, maxsplit)
}}}
! Problem:
You wish to split a string into N equally-sized parts. The last part might be smaller than the previous ones when the length of the string isn't an exact multiple of N. For example, dividing the 9-character string {{{'Woodhenge'}}} into 2 parts would result in the list {{{['Woodh', 'enge']}}}, containing a 5-character and a 4-character string.
! Solution:
The following function takes a string {{{S}}} and a number {{{N}}}, and returns an {{{N}}}-element list containing the different sections of the string. The function works by taking the length of the string and computing how long the sections must be. The list is then constructed by looping over the string and extracting each section by slicing the string.
{{{
def equal_split(S, N):
"""Split up the string S into N parts, returning a list containing
the parts. The last part may be smaller than the others."""
part = (len(S) + N - 1) /N
L = []
for i in range(0, N):
L.append( S[part*i : part*i+part] )
return L
}}}
{{{equal_split('this is a test', 3)}}} will return {{{['this ', 'is a ', 'test']}}}.
More general slicing and indexing of strings works in the same way as other [[sequence types|http://www.python.org/doc/2.4.1/lib/typesseq.html]].
There are several ways to to this, but my favorite so far is this one:
{{{
def split_seq(seq, num_pieces):
start = 0
for i in xrange(num_pieces):
stop = start + len(seq[i::num_pieces])
yield seq[start:stop]
start = stop
}}}
[[Stephen Ferg|http://www.ferg.org/]] was the second maintainer of the Grimoire, and has some interesting [[Papers and Projects|http://www.ferg.org/projects/index.html]] online that may be useful to Python newbies - most notably [[a list of Python Gotchas|http://www.ferg.org/projects/python_gotchas.html]].
/***
!GTD Style
http://tiddlystyles.com/#theme:GTD
!Generic rules /%==================================================================== %/
***/
/*{{{*/
body {
background: #262626;
color: #000;
font: .82em/1.25em "Bitstream Vera Sans", Verdana, Helvetica, Arial, sans-serif;
/*"Lucida Sans Unicode", "Lucida Grande","Trebuchet MS", */
}
/*}}}*/
/***
!Header rules /%====================================================================== %/
***/
/*{{{*/
#contentWrapper
{
margin: 0 auto;
width: 59em;
position: relative;
}
#header
{
color: #fff;
padding: 1.5em 1em .6em 0;
}
#siteTitle {
font-size: 2.3em;
margin: 0;
}
#siteSubtitle {
font-size: 1em;
padding-left: .8em;;
}
#titleLine{
background: transparent;
padding: 0;
}
#titleLine a {
color: #cf6;
}
#titleLine a:hover {
background: transparent;
}
/*}}}*/
/***
!Sidebar rules /%====================================================================== %/
***/
/*{{{*/
#sidebar{
left: 0;
width: 18em;
margin: .9em .9em 0 0;
color: #000;
background: transparent;
}
/*}}}*/
/***
!Main menu rules /%=================================================================== %/
***/
/*{{{*/
#mainMenu{
position: static;
width: auto;
background: #600;
border-right: 3px solid #500;
padding: 0;
text-align: left;
font-size: 1em;
}
#mainMenu h1{
padding: 5px;
margin: 0;
font-size: 1em;
font-weight: bold;
background: transparent;
color: #fff;
}
#mainMenu ul{
padding: 0;
margin: 0;
list-style: none;
}
#mainMenu h1 a,
#mainMenu li a,
#mainMenu li a.button{
display: block;
padding: 0 5px 0 10px;
border: 0;
border-bottom: 1px solid #500;
border-top: 1px solid #900;
margin: 0;
}
#mainMenu a,
#mainMenu a.button{
height: 22px;
height: 1.83em;
line-height: 22px;
color: #fff;
background: #700;
margin-left: 1em;
}
#mainMenu a:hover,
#mainMenu a.button:hover {
background: #b00;
color: #fff;
}
/*}}}*/
/***
!Sidebar options rules /%============================================================ %/
***/
/*{{{*/
#sidebarOptions {
background: #eeb;
border-right: 3px solid #bb8;
color: #B4C675;
padding: .5em 0;
}
#sidebarOptions a {
color: #700;
margin: .2em .8em;
padding: 0;
border: 0;
}
#sidebarOptions a:hover, #sidebarOptions a:active {
color: #fff;
background: #700;
border: 0;
}
#sidebarOptions input{
margin: 2px 10px;
border: 1px inset #333;
padding: 0;
}
#sidebarOptions .sliderPanel {
background: #fff;
color: #000;
padding: 5px 10px;
font-size: .9em;
}
#sidebarOptions .sliderPanel a{
font-weight: normal;
margin: 0;
}
#sidebarOptions .sliderPanel a:link,#sidebarOptions .sliderPanel a:visited {
color: #700;
}
#sidebarOptions .sliderPanel a:hover,#sidebarOptions .sliderPanel a:active {
color: #fff;
background: #700;
}
/*}}}*/
/***
!Sidebar tabs rules /%===================================================================== %/
***/
/*{{{*/
#sidebarTabs {
background: transparent;
border-right: 3px solid #740;
border-bottom: 3px solid #520;
border: 0;
padding: 0;
}
#contentWrapper #sidebarTabs a,
#contentWrapper #displayArea .tabContents a{
color: #fff;
}
#contentWrapper #sidebarTabs a:hover,
#contentWrapper #displayArea .tabContents a:hover {
background: #000;
color: #fff;
}
#contentWrapper #sidebarTabs a:active,
#contentWrapper #displayArea .tabContents a:active{
color: #000;
}
#contentWrapper .tabSelected {
background: #960;
}
#contentWrapper .tabUnselected{
background: #660;
}
#contentWrapper #sidebar .tabset{
background: #eeb;
border-right: 3px solid #bb8;
padding: 0 0 0 .75em;
}
#contentWrapper .tabContents{
font-size: .95em;
background: #960;
border:0;
border-right: 3px solid #740;
border-bottom: 3px solid #520;
padding: .75em;
}
#contentWrapper .tabContents{
width: auto;
}
#contentWrapper #sidebarTabs .tabContents .tabset,
#contentWrapper .tabContents .tabset{
border: 0;
padding: 0;
background: transparent;
}
#contentWrapper .tabContents .tabSelected,
#contentWrapper .tabContents .tabContents {
background: #700;
border: 0;
}
#contentWrapper .tabContents .tabUnselected {
background: #440;
}
#contentWrapper .tabset a {
color: #fff;
padding: .2em .7em;
margin: 0 .17em 0 0;
height: 2em;
position: static;
}
#contentWrapper .tabset a:hover {
background: #000;
color: #fff;
}
#contentWrapper .tabset a:active {
color: #000;
}
#contentWrapper .tabContents ul{
margin: 0;
padding: 0;
list-style: none;
}
#contentWrapper .tabContents .tabContents ul{
color: #eeb;
}
.tabContents ul a,
.tabContents ul .button{
color: #fff;
display: block;
padding: .1em 0 .1em .7em;
background: transparent;
border: 0;
}
.tabContents ul a:hover {
color: #fff;
background: #000;
}
/*}}}*/
/***
!License panel rules /%==================================================================== %/
***/
/*{{{*/
#licensePanel {
padding: 0px 1em;
font-size: .9em;
}
#licensePanel a {
color: #960;
display: block;
margin-top: .9em;
}
#licensePanel a:hover {
color: #fff;
background: transparent;
}
/*}}}*/
/***
!Popup rules /%================================================================= %/
***/
/*{{{*/
.popup {
font-size: .8em;
padding: 0em;
background: #333;
border: 1px solid #000;
}
.popup hr {
margin: 1px 0 0 0;
visibility: hidden;
}
.popup li.disabled {
color: #666;
}
.popup li a,
.popup li a:visited{
color: #000;
border: .1em outset #cf6;
background: #cf6;
}
.popup li a:hover {
border: .1em outset #cf6;
background: #ef9;
color: #000;
}
/*}}}*/
/***
!Message area rules /%================================================================= %/
***/
/*{{{*/
#messageArea{
font-size: .9em;
padding: .4em;
background: #FFE72F;
border-right: .25em solid #da1;
border-bottom: .25em solid #a80;
position: fixed;
top: 10px;
right: 10px;
color: #000;
}
#contentWrapper #messageArea a{
color: #00e;
text-decoration: none;
}
#contentWrapper #messageArea a:hover{
color: #00e;
text-decoration: underline;
background: transparent;
}
#contentWrapper #messageArea .messageToolbar a.button{
border: 1px solid #da1;
}
#contentWrapper #messageArea .messageToolbar a.button:hover{
color: #00e;
text-decoration: none;
border: 1px solid #000;
background: #fff;
}
/*}}}*/
/***
!Tiddler display rules /%================================================================== %/
***/
/*{{{*/
#displayArea {
width: 39.75em;
margin: 0 0 0 17em;
}
.tiddler {
margin: 0 0 .9em 0;
padding: 0 1em;
border-right: .25em solid #aaa;
border-bottom: .25em solid #555;
background: #fff;
}
.title {
font-size: 1.5em;
font-weight: bold;
color: #900;
}
.toolbar {
font-size: .8em;
padding: .5em 0;
}
.toolbar .button{
padding: .1em .3em;
color: #000;
border: .1em outset #cf6;
background: #cf6;
margin: .1em;
}
.toolbar .button:hover {
background: #ef9;
color: #000;
}
.toolbar .button:active {
background: #ff0;
}
/*}}}*/
/***
!Viewer rules /% ------------------------------------------------------------------------------------------ %/
***/
/*{{{*/
.viewer {
line-height: 1.4em;
font-size: 1em;
}
.viewer a:link, .viewer a:visited {
color: #15b;
}
.viewer a:hover {
color: #fff;
background: #000;
}
.viewer .button{
background: transparent;
border-top: 1px solid #eee;
border-left: 1px solid #eee;
border-bottom: 1px solid #000;
border-right: 1px solid #000;
}
.viewer .button:hover{
background: #eee;
color: #000;
}
.viewer .button:active{
background: #ccc;
border-bottom: 1px solid #eee;
border-right: 1px solid #eee;
border-top: 1px solid #111;
border-left: 1px solid #111;
}
.viewer blockquote {
border-left: 3px solid #777;
margin: .3em;
padding: .3em;
}
.viewer pre{
background: #fefefe;
border: 1px solid #f1f1f1;
}
.viewer pre, .viewer code{
color: #000;
}
.viewer ul {
padding-left: 30px;
}
.viewer ol {
padding-left: 30px;
}
ul{
list-style-type: asquare;
}
ol{
list-style-type: decimal;
}
ol ol{
list-style-type: lower-alpha;
}
ol ol ol{
list-style-type: lower-roman;
}
.viewer ul, .viewer ol, .viewer p {
margin: .0;
}
.viewer li {
margin: .2em 0;
}
h1,h2,h3,h4,h5,h6 {
color: #000;
font-weight: bold;
background: #eee;
padding: 2px 10px;
margin: 5px 0;
}
.viewer h1 {font-size: 1.3em;}
.viewer h2 {font-size: 1.2em;}
.viewer h3 {font-size: 1.1em;}
.viewer h4 {font-size: 1em;}
.viewer h5 { font-size: .9em;}
.viewer h6 { font-size: .8em;}
.viewer table {
border: 2px solid #303030;
font-size: 11px;
margin: 10px 0;
}
.viewer th, .viewer thead td{
color: #000;
background: #eee;
border: 1px solid #aaa;
padding: 0 3px;
}
.viewer td {
border: 1px solid #aaa;
padding: 0 3px;
}
.viewer caption {
padding: 3px;
}
.viewer hr {
border: none;
border-top: dotted 1px #777;
height: 1px;
color: #777;
margin: 7px 0;
}
.viewer
{
margin: .5em 0 0 0;
padding: .5em 0;
border-top: 1px solid #ccc;
}
.highlight {
color: #000;
background: #ffe72f;
}
/*}}}*/
/***
!Editor rules /% ----------------------------------------------------------------------------------------- %/
***/
/*{{{*/
.editor {
font-size: .8em;
color: #402C74;
padding: .3em 0;
}
.editor input, .editor textarea {
font: 1.1em/130% "Andale Mono", "Monaco", "Lucida Console", "Courier New", monospace;
margin: 0;
border: 1px inset #333;
padding: 2px 0;
}
.editor textarea {
height: 42em;
width: 100%;
}
input:focus, textarea:focus
{
background: #ffe;
border: 1px solid #000;
}
.footer
{
padding: .5em 0;
margin: .5em 0;
border-top: 1px solid #ddd;
color: #555;
text-align: center;
}
/*}}}*/
/***
!IE Display hacks /% ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~%/
***/
/*{{{*/
body{
_text-align: center;
}
#contentWrapper
{
/* _width: 770px; CSS UNDERSCORE HACK FOR PROPER WIN/IE DISPLAY */
_text-align: left; /* CSS UNDERSCORE HACK FOR PROPER WIN/IE DISPLAY */
}
#messageArea{
_position: absolute;
}
/*}}}*/
In a radical departure from the HTML version, the TOC is generated dynamically from the TiddlyWiki tags:
* <<tag Numbers>>
* <<tag Strings>>
* <<tag Tuples>>
* <<tag Lists>>
* <<tag Dictionaries>>
* <<tag Objects>>
* <<tag Functions>>
* <<tag Files>>
* <<tag Directories>>
* <<tag Parsing>>
* <<tag Network>>
* <<tag Portability>>
* <<tag Exceptions>>
In a radical departure from the HTML version, the TOC is generated dynamically from the TiddlyWiki tags:
* <<tag Numbers>>
* <<tag Strings>>
* <<tag Tuples>>
* <<tag Lists>>
* <<tag Dictionaries>>
* <<tag Objects>>
* <<tag Functions>>
* <<tag Files>>
* <<tag Directories>>
* <<tag Parsing>>
* <<tag Network>>
* <<tag Portability>>
* <<tag Exceptions>>
This document is based on version <<version>> of TiddlyWiki, an experimental MicroContent WikiWikiWeb built by [[Jeremy Ruston]]. It's written in HTML, CSS and JavaScript to run on any modern browser without needing any server-side logic, and you can find out more about it at [[its homepage|http://www.tiddlywiki.com]].
This is the opposite of the previous problem. You now want to upload a file to an FTP server.
The {{{ftplib}}} module comes to the rescue, again.
{{{
import ftplib
# Open the connection
ftp = ftplib.FTP("ftp.fbi.example.gov")
ftp.login("mulder", "trustno1")
# Change directory
ftp.cwd('pub/x-files/bigfoot')
# Read from a local text file, and send it to the FTP server
# The local filename is 'sightings', but it'll be stored on the remote FTP
# server under the name 'sightings-1999-01'.
f = open("sightings", "r")
ftp.storlines("STOR sightings-1999-01", f)
f.close()
# Upload a binary file.
# Local filename: 'footprint.jpg', remote filename: 'footprint-1.jpg'
f = open("footprint.jpg", "rb")
ftp.storbinary("STOR footprint-1.jpg", f, 1024)
f.close()
# Close connection
ftp.quit()
}}}
When uploading a binary file, as opposed to a text file, a third numeric argument has to be provided to the {{{storbinary()}}} method.
{{{
ftp.storbinary("STOR footprint-1.jpg", f, 1024)
}}}
The file will be read and sent over the network in chunks of the specified block size, so it shouldn't be set to a very small value like 1; this would result in too much overhead. It would be possible to spend a good deal of effort trying different values of the block size and measure the performance in order to figure out the optimal block size, but it's not that critical a parameter. 1024 is a reasonable number, so you can simply set the value and forget about it.
{{{
import readline
def raw_input_with_default(prompt, default):
def pre_input_hook():
readline.insert_text(default)
readline.redisplay()
readline.set_pre_input_hook(pre_input_hook)
try:
return raw_input(prompt)
finally:
readline.set_pre_input_hook(None)
}}}
Originally from [[here|http://chistera.yi.org/~adeodato/blog/entries/2008/02/14/python_raw_input_with_an_editable_default_value_using_readline.html]]
If a default value is some expression, that expression is evaluated at the time the {{{def}}} statement is executed, not when the function is called. Consider the following example:
{{{
default_level = 9
def new_compressor(compresslevel = default_level):
print 'Compression level:', compresslevel
new_compressor() # Prints the default value
default_level = 2 # Will this change the default value?
new_compressor()
}}}
When this code is run, it will print:
{{{
Compression level: 9
Compression level: 9
}}}
Remember, {{{def}}} is an executable statement, binding the function's name "{{{new_compressor}}}" to a function object. At that time, {{{default_level}}} has a value of 9, so that's used as the default value of compresslevel. Subsequently changing the value of {{{default_level}}} has no effect on the default value of the function. It's not possible to change the object that's been defined as the default value. However, if an object is mutable and can be modified in place, as a list or dictionary can be, then any modification made to that object will change the default. Another example will make this clearer, I hope:
{{{
def modify_list(List = []):
print 'Before:', id(List), List
List.append( 1 )
print 'After: ', id(List), List
}}}
Calling this function with {{{modify_list()}}} will output:
{{{
Before: 135048144 []
After: 135048144 [1]
}}}
The first number, the value of {{{id(List)}}}, is an integer guaranteed to be unique for this object while the object exists, and is usually derived from the machine address at which the object resides. The value printed here will vary from run to run of the Python interpreter, but will remain the same in each execution. On the first call of {{{modify_list()}}}, the list object assigned as the default value is changed. A second, identical call will produce:
{{{
Before: 135048144 [1]
After: 135048144 [1, 1]
}}}
{{{id(List)}}} is the same on both function calls, so the same list object is assigned as the default value, but the contents of that object have been changed, which doesn't affect the value returned by {{{id(List)}}}. New users are often surprised by this behaviour, and wonder if it's a bug in Python; it's not, and follows consistently from Python treating everything as a reference.
The Wichmann-Hill random number generator is described in: "An efficient and portable pseudo-random number generator (Algorithm AS 183)", Wichmann, B. A. Hill, I. D., //Applied Statistics// 31 (1982) 188-190.
Xavier supplied the {{{reverse}}} examples for [[Iterating Over a List]] and a set of notes for [[Multiple-Valued Dictionaries]].