Creating a RESTful Python API With Bottle

Weekly Challenge
Comments

The challenge this week: using Python to create a RESTful service to allow simple CRUD operations on a bunch of XML files.

About the challenge

This is part of an ongoing project, a simple recipe manager. Previous steps left me with a set of XML files I can visualize in a browser, one per recipe. I now need a quick and dirty way to manage those files directly from the (yet to be built) front end.

1
2
3
4

Feature: Creating a quick RESTful API for recipes
    In order to develop a front end quickly
    As a developer
    I need to a RESTful API that performas basic CRUD operations

I decided to use Python for the backend because I am trying to learn it, and it also fits well with some tools that I intend to use further down the line. I haven’t decided on a framework yet, that will be part of the challenge. I will try Bottle to start with.

Some artefacts available on GitHub.

Installing Bottle

I couldn’t use curl or wget for some reason, so I simply pointed my browser to http://bottlepy.org/bottle.py and hit “Save as…” to download the single Python file to a folder called bottle. To test it run, I created this script

bottle/test1.py 
1
2
3
4
5
6
7

from bottle import route, run

@route('/')
def hello():
    return "Hello World!"

run(host='localhost', port=8080, debug=True)

and started the server with python test1.py. I then checked the server was up and running by pointing a browser to http://localhost:8080/. I got the Hello World string

Now I can start building the API.

Testing Bottle installation

This server is going to perform the most basic CRUD operation without checking input, using HTTP verbs. That would be PUT for Create / Update (the only difference will be whether there is already a file by that name or not); DELETE for Delete; GET for Retrieve, plus another GET at the root to get the list of available recipes. Nothing fancier than that.

The first step is to create a script that proves all the calls I want to make work.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

Scenario: Installing and setting up Bottle
    Given that I am running test2.py
    When I access the <url> with <verb>
    Then the response should have status 200 and be <response>

Examples: Basic Strings
    | url                                | verb   | response           |
    | http://localhost:8080/recipes      | GET    | LIST               |
    | http://localhost:8080/recipes/name | PUT    | SAVE RECIPE name   |
    | http://localhost:8080/recipes/name | GET    | SHOW RECIPE name   |
    | http://localhost:8080/recipes/name | DELETE | DELETE RECIPE name |


Examples: JSON Responses
    | url                                | verb   | response                                                                                     |
    | http://localhost:8080/recipes      | GET    | { "success" : False, "paths" : [], "error" : "list not implemented yet" }                    |
    | http://localhost:8080/recipes/name | PUT    | { "success" : False, "path" : "/PTH/TO/XML/name.xml", "error" : "save not implemented yet" } |
    | http://localhost:8080/recipes/name | GET    | { "success" : False, "path" : "/PTH/TO/XML/name.xml", "error" : "show not implemented yet" } |
    | http://localhost:8080/recipes/name | DELETE | { "success" : False, "paths" : [], "error" : "delete not implemented yet" }                  |

I will test this manually with curl with this script

1
2
3
4

curl -X GET http://localhost:8080/recipes
curl -X DELETE http://localhost:8080/recipes/name
curl -X GET http://localhost:8080/recipes/name
curl -X PUT http://localhost:8080/recipes/name

Implementing a server that fulfils those requests is pretty simple.

bottle/test2.py 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

from bottle import route, run

@route('/recipes/')
def recipes_list():
    return "LIST"

@route('/recipes/<name>', method='GET')
def recipe_show( name="Mystery Recipe" ):
    return "SHOW RECIPE " + name

@route('/recipes/<name>', method='DELETE' )
def recipe_delete( name="Mystery Recipe" ):
    return "DELETE RECIPE " + name

@route('/recipes/<name>', method='PUT')
def recipe_save( name="Mystery Recipe" ):
    return "SAVE RECIPE " + name

run(host='localhost', port=8080, debug=True)

I run python test2.py, run the curl requests, and for each of them I got the expected response. Blimey - this is way too easy.

Returning JSON data with Bottle

So simple I can skip directly to the next iteration, return JSON responses in the right “shape”, but with dummy data:

bottle/test2.py 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

from bottle import route, run

@route('/recipes/')
def recipes_list():
    return { "success" : False, "paths" : [], "error" : "list not implemented yet" }

@route('/recipes/<name>', method='GET')
def recipe_show( name="Mystery Recipe" ):
    return { "success" : False, "path" : "/PTH/TO/XML/"+name+".xml", "error" : "show not implemented yet" }

@route('/recipes/<name>', method='DELETE' )
def recipe_delete( name="Mystery Recipe" ):
    return { "success" : False, "error" : "delete not implemented yet" }

@route('/recipes/<name>', method='PUT')
def recipe_save( name="Mystery Recipe" ):
    return { "success" : False, "path" : "/PTH/TO/XML/"+name+".xml", "error" : "save not implemented yet" }

run(host='localhost', port=8080, debug=True)

Reading config data

Finally, I will set up a config files for the app to read config data from - always good practice.

1
2
3
4
5
6
7
8
9
10

Scenario: Loading config data with ConfigParser
    Given that config.json contains
        """
        "paths" : {
            "xml" : "/Users/ME/work/xml/"
        }
        """
    And I am running test2.py
    When I access http://localhost:8080/recipes/name
    Then the app should return a JSON response { "success" : false, "path" : "/Users/ME/work/xml/name.xml" , "error" : "show not implemented yet }

Nothing really too exciting here.

bottle/test2.py 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import json
from bottle import route, run

config_file = open( 'config.json' )
config_data = json.load( config_file )
pth_xml     = config_data["paths"]["xml"]


@route('/recipes/')
def recipes_list():
    return { "success" : False, "paths" : [], "error" : "list not implemented yet" }

@route('/recipes/<name>', method='GET')
def recipe_show( name="Mystery Recipe" ):
    return { "success" : False, "path" : pth_xml +name +" .xml", "error" : "show not implemented yet" }

@route('/recipes/<name>', method='DELETE' )
def recipe_delete( name="Mystery Recipe" ):
    return { "success" : False, "error" : "delete not implemented yet" }

@route('/recipes/<name>', method='PUT')
def recipe_save( name="Mystery Recipe" ):
    return { "success" : False, "path" : pth_xml +name +" .xml", "error" : "save not implemented yet" }

run(host='localhost', port=8080, debug=True)

I run it and got the expected results.

Creating RESTful responses

Now it’s simply a matter of filling in the blanks, and writing those methods.

Returning a JSON formatted directory listing with Bottle

First, let’s get list of files.

1
2
3
4
5
6
7
8
9

Scenario: Returning a JSON formatted directory listing with Bottle
    Given that config.json points to <PTH>
    And <PTH> contains <filelist>
    When I request http://localhost:8080/recipes<url> with <verb>
    Then the app should return <response>

Examples: simple listing
    | PTH         | filelist                   | url | verb | response                                                    |
    | test/data/1 | a.xml, b.xml, c.xml, d.txt | /   | GET  | { "success": true, "paths": [ "a.xml", "b.xml", "c.xml" ] } |

This is a readdir job, very simple with python.

bottle/test3.py 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

import json
import os
from bottle import route, run

config_file = open( 'config.json' )
config_data = json.load( config_file )
pth_xml     = config_data["paths"]["xml"]

@route('/recipes/')
def recipes_list():
    paths = []
    ls = os.listdir( pth_xml )
    for entry in ls:
        if ".xml" == os.path.splitext( entry )[1]:
            paths.append( entry )
    return { "success" : True, "paths" : paths }

Err… that’s it. A JSON response. In the browser.

Echoing XML files with Bottle

Now, let’s look at one of these files in the list.

1
2
3
4
5
6
7
8
9

Scenario: Echoing XML files with Bottle
    Given that config.json points to <PTH>
    And <PTH> contains <filelist>
    When I request http://localhost:8080/recipes<url> with <verb>
    Then the app should return <response>

Examples: simple listing
    | PTH         | filelist                   | url | verb | response           |
    | test/data/1 | a.xml, b.xml, c.xml, d.txt | /a  | GET  | <content of a.xml> |

So now I am actually serving a file. It’s not JSON, it’s XML - that’s still OK, it’s all human readable data in the end.

This is ridiculously easy using Bottle’s static_file. Note that Bottle takes case of slashes, so I won’t need to watch out for someone passing ../../../../../etc/passwd as name and showing sensitive data.

test3.py 
1
2
3
4
5
6
7
8
9
10
11
12
13
14

import json
import os
from bottle import route, run, static_file

config_file = open( 'config.json' )
config_data = json.load( config_file )
pth_xml     = config_data["paths"]["xml"]

@route('/recipes/<name>', method='GET')
def recipe_show( name="" ):
    if "" != name:
        return static_file( name, pth_xml  )
    else:
        return { "success" : False, "error" : "show called without a filename" }

Embarassingly simple.

Creating and updating XML files with Bottle

The next obvious step is to add files to the set. The same script will be used for create or update - any exisiting files will simply be overwritten, without checking. And there won’t be any checks for the correct XML format either.

1
2
3
4
5
6
7
8
9
10
11
12
13

Scenario: Creating and updating XML files with Bottle
    Given that config.json points to <PTH>
    And <PTH> contains <filelist>
    When I request http://localhost:8080/recipes<url> with <verb>
    And send <postdata>
    Then the app should return <response>
    And the <PTH> should contain <newfilelist>
    And <PTH><url> should contain <postdata>

Examples: simple listing
    | PTH         | filelist     | url | verb | postdata | response                                             | newfilelist         |
    | test/data/2 | a.xml, b.xml | /f  | PUT  | new file | { "success": true, "path": [ "test/data/2/a.xml" ] } | a.xml, b.xml, f.xml |
    | test/data/2 | a.xml, b.xml | /a  | PUT  | new file | { "success": true, "path": [ "test/data/2/a.xml" ] } | a.xml, b.xml        |

This is also fairly simple, as long as one knows where to get the PUT data from - request.forms, which requires the request module.

bottle/test3.py 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

import json
import os
from bottle import route, run, request

config_file = open( 'config.json' )
config_data = json.load( config_file )
pth_xml     = config_data["paths"]["xml"]

@route('/recipes/<name>', method='PUT')
def recipe_save( name="" ):
    xml = request.forms.get( "xml" )
    if "" != name and "" != xml:
        with open( os.path.join( pth_xml, name + ".xml" ), "w" ) as f:
            f.write( xml )
        return { "success" : True, "path" : name }
    else:
        return { "success" : False, "error" : "save called without a filename or content" }

In order to test this I needed to send postdata with curl:

1
2

curl -X PUT --data-urlencode "xml=new file" http://localhost:8080/recipes/f
curl -X PUT --data-urlencode "xml=new file" http://localhost:8080/recipes/a

Everything worked as expected.

Deleting files with Bottle

The last piece of the puzzle.

1
2
3
4
5
6
7
8
9
10

Scenario: eleting files with Bottle
    Given that config.json points to <PTH>
    And <PTH> contains <filelist>
    When I request http://localhost:8080/recipes<url> with <verb>
    Then the app should return <response>
    And the <PTH> should contain <newfilelist>

Examples: simple listing
    | PTH         | filelist            | url | verb    | response            | newfilelist  |
    | test/data/2 | a.xml, b.xml, f.xml | /f  | DELETE  | { "success": true } | a.xml, b.xml |

Also fairly simple. A try / except is used to avoid errors if the files doesn’t exist.

bottle/test3.py 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

import json
import os
from bottle import route, run, , request

config_file = open( 'config.json' )
config_data = json.load( config_file )
pth_xml     = config_data["paths"]["xml"]

@route('/recipes/<name>', method='DELETE' )
def recipe_delete( name="" ):
    if "" != name:
        try:
            os.remove( os.path.join( pth_xml, name + ".xml" ) )
            return { "success" : True  }
        except:
            return { "success" : False  }

Challenge 100% complete

This is quick and dirty work - no testing, and running off a local dev server which wouldn’t be suited for production. But I am still amazed but how easy it all was. Bottle really does rock.

The scripts are available on GitHub.

Related Posts

Comments