QuickWiki Tutorial
++++++++++++++++++
:author: James Gardner
.. Note::
This tutorial will also work with SQLAlchemy 0.3 and has also been tested with Python 2.5.
Introduction
============
If you haven't done so already read the `installation instructions `_ and `getting started `_ guide.
In this tutorial we are going to create a working wiki from scratch using Pylons 0.9.4.1 and SQLAlchemy. Our wiki will allow visitors to add, edit or delete formatted wiki pages.
.. contents:: Table of Contents
.. sectnum::
Starting at the End
===================
Pylons is designed to be easy for everyone, not just developers, so lets start by downloading and installing the finished QuickWiki in exactly the way end users of QuickWiki might do. Once we have explored its features we will set about writing it from scratch.
After you have installed `Easy Install `_ run these commands to install QuickWiki and create a config file:
.. code-block:: Bash
$ easy_install QuickWiki==0.1.3
$ paster make-config QuickWiki test.ini
Next edit the configuration file by specifying the ``sqlalchemy.dburi`` variable in ``[app:main]`` section so that the data source name points to the database you wish to use.
.. Note:: The default ``postgres://username:password@localhost/quickwiki_test`` uses a PostgreSQL database named ``quickwiki_test`` running on the local machine. You can create this database with the PostgreSQL command ``createdb quickwiki_test`` but you could also use MySQL, Oracle or SQLite. Firebird and MS-SQL may also work. See the `SQLAlchemy documentation `_ for more information on how to connect to different databases. SQLite for example requires four forward slashes in its URI. You will also need to make sure you have the appropriate Python driver for the database you wish to use. For PostgreSQL this would be `psycopg `_.
Finally create the database tables and serve the finished application:
.. code-block:: Bash
$ paster setup-app test.ini
$ paster serve test.ini
That's it! Now you can visit http://127.0.0.1:5000 and experiment with the finished Wiki. Note that in the title list screen you can drag page titles to the trash area to delete them via AJAX calls.
When you've finished, stop the server with ``CTRL+C`` because we will start developing our own version.
If you are interested in looking at the latest version of the QuickWiki source code it can be browsed online at http://pylonshq.com/project/pylonshq/browser/sandbox/examples/QuickWiki or can be checked out using subversion:
.. code-block:: Bash
$ svn co http://pylonshq.com/svn/sandbox/examples/QuickWiki/
Developing QuickWiki
====================
If you skipped the "Starting at the End" section you will need to install Pylons 0.9.4.1:
.. code-block:: Bash
$ easy_install -U Pylons[full]>=0.9.4.1
Then create your project:
.. code-block:: Bash
$ paster create --template=pylons QuickWiki
Now lets start the server and see what we have:
.. code-block:: Bash
$ cd QuickWiki
$ paster serve --reload development.ini
.. Note:: We have started the server with the ``--reload`` switch. This means any changes we make to code will cause the server to restart (if necessary); your changes are immediately reflected on the live site.
Open a new console and ``cd QuickWiki/quickwiki``. Visit http://127.0.0.1:5000 where you will see the introduction page. Delete the file ``public/index.html`` because we want to see the front page of the wiki instead of this welcome page. If you now refresh the page, the Pylons built-in error document support will kick in and display an ``Error 404`` page to tell you the file could not be found. We'll setup a controller to handle this location later.
The Model
=========
Pylons uses a Model View Controller architecture; we'll start by creating the model. We could use any system we like for the model including `SQLObject `_ or `SQLAlchemy `_. We'll use SQLAlchemy for QuickWiki.
.. Note:: SQLAlchemy is a Python SQL toolkit and Object Relational Mapper that is fast becoming the default choice for many Pylons programmers.
SQLAlchemy provides a full suite of well known enterprise-level persistence patterns, designed for efficient and high-performance database access, adapted into a simple and Pythonic domain language. There is full and detailed documentation available on the SQLAlchemy website at http://sqlalchemy.org/docs/ and you should really read this before you get heavily into SQLAlchemy.
The most basic way of using SQLAlchemy is with explict sessions where on each request you create an ``session`` object. QuickWiki uses a slighty more sophisticated setup using an SQLAlchemy ``SessionContext``.
Now replace the contents of your ``models/__init__.py`` file so that it looks like this:
.. code-block:: Python
from sqlalchemy import *
meta = DynamicMetaData()
pages_table = Table('pages', meta,
Column('title', String(40), primary_key=True),
Column('content', String(), default='')
)
The first line imports some useful SQLAlchmey objects such as the ``Table`` and ``Column`` classes. The second line creates a metadata object which allows us to define the tables now but to bind them to a database later on when we access the current session. We then define a table called ``pages`` which has two columns, ``title`` (the primary key) and ``content``.
.. Note:: SQLAlchemy also supports reflecting this information directly from a database table so if we had already created the database table SQLAlchemy could have constructed the ``Table`` object for us.
.. Note:: A primary key is a unique ID for each row in a database table. In the example above we are using the page title as a natural primary key. Some people prefer to use integer primary keys for all tables, so-called surrogate primary keys. The author of this tutorial uses both methods in his own code and is not advocating one method over the other, it is important that you choose the best database structure for your application. `Read more about Primary keys on WikiPedia `_.
.. Note:: It is useful to define the metadata with the model rather than in the ``lib/database.py`` file so that the model can be imported in code that does not have the config information setup that the ``lib/database.py`` file relies on. This enables us to write code such as that in ``websetup.py`` which will come later in this tutorial.
.. Note:: There are other ways of setting up SQLAlchemy to work with Pylons, one alternative is described in the article `SqlAlchemyWithPylons `_ on the publically editable Pylons wiki.
A core philosophy of SQLAlchemy is that tables and domain classes are different beasts. So next, we'll create the Python class that will represent the pages of our wiki and map these domain objects to rows in the ``pages`` table using a mapper.
Add the following import:
.. code-block:: Python
from sqlalchemy.ext.assignmapper import assign_mapper
from pylons.database import session_context
and this to the bottom of ``models/__init__.py``:
.. code-block:: Python
class Page(object):
def __str__(self):
return self.title
page_mapper = assign_mapper(session_context, Page, pages_table)
The ``session_context`` object is a SQLAlchemy ``SessionContext`` automatically created for you, bound to the database engine specified by the ``sqlalchemy.dburi`` variable in your config file. ``SessionContext`` is a wrapper object that provides a certain SQLAlchemy ``Session`` object at ``session_context.current``, depending on the current context (e.g.: the current thread and or WSGI application).
The ``assignmapper`` extension provides a special kind of mapper (``assign_mapper``). It makes mapped objects have direct knowledge about their contextual ``Session``. Classes and their objects decorated by ``assign_mapper`` can use ``Session`` and ``Query`` methods directly. Using the session object directly is a reasonable API, for example:
.. code-block:: Python
# EXAMPLE:
test_page = session.query(Page).get_by(title='TestPage')
test_page.content = 'Test Page Content'
session.save(test_page)
but in comparison to the following ``assign_mapper`` equivalent, seems cumbersome:
.. code-block:: Python
# EXAMPLE:
test_page = Page.get_by(title='TestPage')
test_page.content = 'Test Page Content'
test_page.save()
Looking ahead, our wiki will need some formatting so we will need to turn the ``content`` field into HTML. Any WikiWords (which are words made by joining together two or more lowercase words with the first letter capitalized) will also need to be converted into hyperlinks.
It would be nice if we could add a method to our ``Page`` object to retrieve the formatted HTML with the WikiWords already converted to hyperlinks. Add the following at the top of the ``models/__init__.py`` file:
.. code-block:: Python
import re
import sets
import quickwiki.lib.helpers as h
from docutils.core import publish_parts
wikiwords = re.compile(r"\b([A-Z]\w+[A-Z]+\w+)")
and then add a ``get_wiki_content()`` method to the ``Page`` object so it looks like this:
.. code-block:: Python
class Page(object):
content = None
def __str__(self):
return self.title
def get_wiki_content(self):
content = publish_parts(self.content, writer_name="html")["html_body"]
titles = sets.Set(wikiwords.findall(content))
for title in titles:
title_url = h.url_for(controller='page', action='index',
title=title)
content = content.replace(title, h.link_to(title, title_url))
return content
This code deserves a bit of explaining. The ``content = None`` line is so that the ``content`` attribute is initialized to ``None`` when a new ``Page`` object is created. The ``Page`` object represents a row in the ``pages`` table so ``self.content`` will be the value of the ``content`` field. The ``Set`` object provides us with only unique WikiWord names, so we don't try replacing them more than once. ``h.link_to()`` and ``h.url_for()`` are standard Pylons helpers which create links to specific controller actions. In this case we have decided that all WikiWords should link to the ``index`` action of the ``page`` controller which we will create later.
.. Note::
Pylons uses a Model View Controller architecture and so the formatting of objects into HTML should usually be handled in the view, ie in a template. In this example converting reStructuredText into HTML in a template is not appropriate so we are treating the HTML representation of the content as part of the model. It also gives us the chance to demonstrate that SQLAlchemy domain classes are real Python classes that can have their own methods.
One final change, since we have used docutils and SQLAlchemy, both third party packages, we need to edit our ``setup.py`` file so that anyone installing QuickWiki with `Easy Install `_ will automatically also have these dependencies installed for them too. Edit your ``setup.py`` in your project root directory so that the ``install_requires`` line looks like this:
.. code-block:: Python
install_requires=["Pylons>=0.9.4.1", "docutils==0.4", "SQLAlchemy>=0.2.6"],
While we are we are making changes to ``setup.py`` we might want to complete some of the other sections too. Set the version number to 0.1.3 and add a description and URL which will be used on the Python Cheeseshop when we release it:
.. code-block:: Python
version="0.1.3",
description="Result of following the Pylons 0.9.4.1 QuickWiki Tutorial",
url="http://www.pylonshq.com/docs/quick_wiki.html",
We might also want to make a full release rather than a development release in which case we would remove the following lines from ``setup.cfg``:
.. code-block:: Ini
[egg_info]
tag_build = dev
tag_svn_revision = true
To test the automatic installation of the dependencies, run the following command which will also install docutils and SQLAlchemy if you don't already have them:
.. code-block:: Bash
$ python setup.py develop
.. Note:: The command ``python setup.py develop`` installs your application in a special mode so that it behaves exactly as if it had been installed as an egg file by an end user. This is really useful when you are developing an application because it saves you having to create an egg and install it every time you want to test a change.
Configuration and Setup
=======================
Now lets make the changes necessary to enable QuickWiki to be set up by an end user. Edit ``websetup.py`` used by the ``paster setup-app`` command to look like this:
.. code-block:: Python
import paste.deploy
from pylons.database import create_engine
import quickwiki.models as model
def setup_config(command, filename, section, vars):
"""
Place any commands to setup quickwiki here.
"""
conf = paste.deploy.appconfig('config:' + filename)
paste.deploy.CONFIG.push_process_config({'app_conf':conf.local_conf,
'global_conf':conf.global_conf})
uri = conf['sqlalchemy.dburi']
engine = create_engine(uri)
print "Connecting to database %s" % uri
model.meta.connect(engine)
print "Creating tables"
model.meta.create_all()
print "Adding front page data"
page = model.Page()
page.title = 'FrontPage'
page.content = 'Welcome to the QuickWiki front page.'
page.flush()
print "Successfully setup."
The line ``import quickwiki.models as model`` imports our ``page`` table and the ``meta`` object, the line ``model.meta.connect(engine)`` connects to the database engine with the database URI the user specifies in the config file and finally ``meta.create_all()`` creates the table. After the tables are created the other lines add some data for the simple front page to out wiki.
To test this functionality run you first need to install your QuickWiki if you haven't already done so in order for ``paster`` to find the version we are developing instead of the version we installed at the very start:
.. code-block:: Bash
$ python setup.py develop
You can then edit your ``development.ini`` to add your database URI so the start of the ``[app:main]`` section looks something like this but customised for your database:
.. code-block:: PasteIni
[app:main]
use = egg:QuickWiki
sqlalchemy.dburi = postgres://username:password@localhost/quickwiki_test
.. Note:: See the SQLAlchemy note in the `Starting at the End`_ section for information on supported database URIs and a link to the SQLAlchemy documentation about the various options that can be included in them.
You can also have SQLAlchemy echo all SQL by adding this line:
.. code-block:: PasteIni
sqlalchemy.echo = true
You should also edit ``QuickWiki.egg_info/paste_deploy_config.ini_tmpl`` so that when users run ``paster make-config`` the configuration file will already have a section telling them to enter their own database URI as we did when we installed the finished QuickWiki at the start of the tutorial. Make the start of the ``[app:main]`` section look like this, leaving the other options in place:
.. code-block:: PasteIni
[app:main]
use = egg:QuickWiki
# Specify your own database connection here
sqlalchemy.dburi = postgres://username:password@localhost/quickwiki_test
# Uncomment the line below to have SQLAlchemy echo SQL for debugging
# sqlalchemy.echo = true
You can should then run the ``paster setup-app`` command to setup your tables in the same way and end user would, remembering to drop and recreate the database if the version tested earlier has already created the tables:
.. code-block:: Bash
$ paster setup-app development.ini
Templates
=========
.. Note:: Pylons uses the Myghty templating language by default, although as is the case with most aspects of Pylons you are free to deviate from the default if you prefer. Pylons also supports Kid and Cheetah out of the box.
We will make use of a feature of the Myghty templating language called inheritance for our project. Add the main page template in ``templates/autohandler``:
.. code-block:: Html+Myghty
QuickWiki
<% h.stylesheet_link_tag('/quick.css') %>
% m.call_next()
All our other templates will be automatically inserted into the ``% m.call_next()`` line and the whole page returned when we call the ``render_response()`` global from our controller so that we can easily apply a consistent theme to all our templates.
If you are interested in learning some of the features of Myghty templates have a look at the comprehensive `Myghty Documentation `_. For now we just need to understand that is replaced with the child template and that anything in ``<%`` and ``%>`` brackets is executed and replaced with the result. Having the ``%`` as the first character on a line is shorthand for putting the whole line in ``<%`` and ``%>`` brackets.
This autohandler also makes use of various helper functions attached to the ``h`` object. These are described in the `WebHelpers documentation `_. You can add more helpers to the ``h`` object by adding them to ``lib/helpers.py`` although for this project we don't need to do so.
.. Warning:: Your templates won't work if there is any whitespace before lines starting with a ``%`` character.
Routing
=======
Before we can add the actions we want to be able to route the requests to them correctly. Edit ``config/routing.py`` so that the route map looks like this:
.. code-block:: Python
map.connect(':controller/:action/:title', controller='page', action='index', title='FrontPage')
map.connect(':title', controller='page', action='index', title='FrontPage')
map.connect('*url', controller='template', action='view')
(Leave the error route ``map.connect('error/:action/:id', controller='error')`` in place or add it if you are using Pylons<0.9.3)
Full information on the powerful things you can do to route requests to controllers and actions can be found in the newly updated `Routes manual `_.
Controllers
===========
Quick Recap: We've setup the model, configured the application, added the routes and setup the base template in autohandler, now we need to write the application! In your project's root directory add a controller called ``page`` to your project with this command:
.. code-block:: Bash
$ paster controller page
We are going to need the following actions:
``index(self, title)``
displays a page based on the title
``edit(self, title)``
displays a from for editing the page ``title``
``save(self, title)``
save the page ``title`` and show it with a saved message
``list(self)``
gives a list of all pages
``delete(self)``
deletes a page based on an AJAX drag and drop call
Let's get cracking! Add the this method to the ``PageController`` class to clear the session. ``SessionContext`` will recycle sessions for us (and their associated database connections), we simply need to ensure that those sessions don't carry any old state from previous requests.
.. code-block:: Python
def __before__(self):
model.session_context.current.clear()
.. Note:: ``__before__()`` is a special action which is always called on each request before the main action is called. Likewise there is also an ``__after__()`` action called after your main action is called.
index()
-------
Replace the existing ``index()`` action with this:
.. code-block:: Python
def index(self, title):
page = model.Page.get_by(title=title)
if page:
c.content = page.get_wiki_content()
return render_response('/page.myt')
elif model.wikiwords.match(title):
return render_response('/new_page.myt')
abort(404)
Add a template called ``templates/page.myt`` that looks like this:
.. code-block:: Html+Myghty
<% c.title %>
<% c.content %>
This template simply displays the page title and content.
.. Note:: Pylons automatically assigns all the action parameters to the context object ``c`` so that you don't have to assign them yourself. In this case, the value of ``title`` will be automatically assigned to ``c.title`` so that it can be used in the templates. We assign ``c.content`` manually in the controller.
We also need a template for pages that don't already exist. They need to display a message and link to the edit action so that they can be created. Add a template called ``templates/new_page.myt`` that looks like this:
.. code-block:: Html+Myghty
<% c.title %>
This page doesn't exist yet.
Create the page.
At this point we can test our QuickWiki to see how it looks. If you don't already have a the server running start it now with:
.. code-block:: Bash
$ paster serve --reload development.ini
Visit http://127.0.0.1:5000/ and you will see the front page of the wiki. We can spruce it up a little by adding the stylesheet we linked to in the ``templates/autohandler`` file earlier. Add this the file ``public/quick.css`` with the following content:
.. code-block:: CSS
body {
background-color: #888;
margin: 25px;
}
div.content{
margin: 0;
margin-bottom: 10px;
background-color: #d3e0ea;
border: 5px solid #333;
padding: 5px 25px 25px 25px;
}
h1.main{
width: 100%;
border-bottom: 1px solid #000;
}
p.footer{
width: 100%;
padding-top: 3px;
border-top: 1px solid #000;
}
Refresh the page again and we have a better looking wiki. You will notice that the word ``QuickWiki`` has been turned into a hyperlink by the ``get_wiki_content()`` method we added to our ``Page`` domain object earlier. You can click the link and will see an example of the new page screen from the ``new_page.myt`` template. If you follow the ``Create the page`` link you will see the Pylons automatic error handler kick in to tell you ``Action edit is not implemented``. Well, we better write it next but before we do, have a play with the `interactive debugger `_, try clicking on the ``+`` or ``>>`` arrows and you will be able to interactively debug your application it is a tremendously useful tool.
edit()
------
To edit the wiki page we need to get the content from the database without changing it to HTML to display it in a simple form for editing. Add the ``edit()`` action:
.. code-block:: Python
def edit(self, title):
page = model.Page.get_by(title=title)
if page:
c.content = page.content
return render_response('/edit.myt')
and the create the ``templates/edit.myt`` file:
.. code-block:: Html+Myghty
Editing <% c.title %>
<% h.start_form(h.url_for(action='save', title=c.title), method="get") %>
<% h.text_area(name='content', rows=7, cols=40, content=c.content)%>
<% h.submit(value="Save changes", name='commit') %>
<% h.end_form() %>
.. Note:: You might have noticed that we only set ``c.content`` if the page exists but that it is accessed in ``h.text_area`` even for pages that don't exist and doesn't raise an ``AttributeError``. We are making use of the fact that the ``c`` object returns an empty string ``""`` for any attribute that is accessed. This is a very useful feature of the ``c`` object but can catch you out on occasions where you don't expect this behavior.
We are making use of the ``h`` object to create our form and field objects. This saves a bit of manual HTML writing. The form submits to the ``save()`` action to save the new or updated content so let's write that next.
save()
------
The first thing the ``save()`` action has to do is to see if the page being saved already exists. If not it creates it with ``page = model.Page()``. Next it needs the updated content. In Pylons you can get request parameters from form submissions, GET or POST requests from the appropriately named ``request.params`` object.
Add the ``save()`` action:
.. code-block:: Python
def save(self, title):
page = model.Page.get_by(title=title)
if not page:
page = model.Page()
page.title = title
page.content = request.params['content']
c.title = page.title
c.content = page.get_wiki_content()
c.message = 'Successfully saved'
page.flush()
return render_response('/page.myt')
.. Note::
``request.params`` is a MultiDict object; an ordered dictionary that may contain multiple values for each key. The MultiDict will always return one value for any existing key via the normal dict accessors (``params[key]``, ``params.get()`` and ``params.setdefault()``). When multiple values are expected, use the ``getall()`` method to return all values in a list.
In order for the ``page.myt`` template to display the ``Successfully saved`` message after the page is saved we need to update the ``templates/page.myt`` file. After ``<% c.title %>
`` add these lines making sure there is no whitespace before the lines that start with a ``%`` character:
.. code-block:: Html+Myghty
% if c.message:
<% c.message %>
% #end if
And add the following to the ``public/quick.css`` file:
.. code-block:: CSS
div#message{
color: orangered;
}
At this point we have a fully functioning wiki that lets you create and edit pages and can be installed and deployed by an end user with just a few simple commands.
Visit http://127.0.0.1:5000 and have a play.
It would be nice to get a title list and to be able to delete pages, so that's what we'll do next!
list()
------
Add the ``list()`` action:
.. code-block:: Python
def list(self):
c.titles = [page.title for page in model.Page.select()]
return render_response('/titles.myt')
The ``list()`` action simply gets all the pages from the database. Create the ``templates/titles.myt`` file to display the list:
.. code-block:: Html+Myghty
Title List
% for title in c.titles:
-
<% title %> [<% h.link_to('visit', h.url_for(title=title, action="index")) %>]
% #end for
Finally edit ``templates/autohandler`` to add a link to the title list so that the footer looks like this:
.. code-block:: Html+Myghty
If you visit http://127.0.0.1:5000/page/list you should see the full titles list and you should be able to visit each page.
delete()
--------
Since this tutorial is designed to get you familiar with as much of Pylons core functionality as possible we will use some AJAX to allow the user to drag a title from the title list into a trash area that will automatically delete the page.
Add this line to ``templates/autohandler`` before ````:
.. code-block:: Myghty
<% h.javascript_include_tag('/javascripts/effects.js', builtins=True) %>
.. Note:: The ``h.javascript_include_tag()`` helper will create links to all the built-in JavaScripts we need and also add ``/javascripts/effects.js`` creating HTML that looks like this when you access it from a browser:
.. code-block:: HyperText
If you look at ``config/middleware.py`` you will see these lines:
.. code-block:: Python
javascripts_app = StaticJavascripts()
...
app = Cascade([static_app, javascripts_app, app])
The ``javascripts_app`` WSGI application maps any requests to ``/javascripts/`` straight to the relevant JavaScript in the WebHelpers package. This means you don't have to manually copy the Pylons JavaScript files to your project and that if you upgrade Pylons, you will automatically be using the latest scripts.
Now for the AJAX! We want all the titles in the titles list to be draggable so we enclose each of them with a ```` element with a unique ID. Edit ``templates/titles.myt`` to replace this line:
.. code-block:: Html+Myghty
<% title %> [<% h.link_to('visit', h.url_for(title=title, action="index")) %>]
with this:
.. code-block:: Html+Myghty
<% title %>
[<% h.link_to('visit', h.url_for(title=title, action="index")) %>]
<% h.draggable_element("page-"+ str(title), revert=True) %>
This marks each of the titles as a draggable element that reverts to its original position if it isn't dropped over a drop target. If we want to be able to delete the pages we better add a drop target. Try it out at http://127.0.0.1:5000/page/list by dragging the titles themselves around the screen. Notice how much functionality we get with just the one helper ``h.draggable_element()``.
We better have somewhere to drop the titles to delete them so add this before the ``