A Script to Take Screenshots at Regular Intervals

Comments

This week’s challenge: a Python shell script that calls webkit2png repeatedly to take screenshots of a website at various intervals. The script will be called webkit2loop.

About the challenge

webkit2png is a nice little Python script by Paul Hammond that lets you take screenshots of websites from your Terminal on a Mac. Very handy, but sometimes with dynamic sites I need to take screenshots at various intervals. I thought writing a wrapper that calls webkit2png in a loop, passing different delays to it, would be a good way to start learning Python.

This simple script is available on GitHub.

webkit2png options

For reference, here are the webkit2png command line options

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -W WIDTH, --width=WIDTH
                        initial (and minimum) width of browser (default: 800)
  -H HEIGHT, --height=HEIGHT
                        initial (and minimum) height of browser (default: 600)
  --clipwidth=WIDTH     width of clipped thumbnail (default: 200)
  --clipheight=HEIGHT   height of clipped thumbnail (default: 150)
  -z ZOOM, --zoom=ZOOM  full page zoom of browser (default: 1.0)
  -s SCALE, --scale=SCALE
                        scale factor for thumbnails (default: 0.25)
  -m, --md5             use md5 hash for filename (like del.icio.us)
  -o NAME, --filename=NAME
                        save images as NAME-full.png,NAME-thumb.png etc
  -F, --fullsize        only create fullsize screenshot
  -T, --thumb           only create thumbnail sreenshot
  -C, --clipped         only create clipped thumbnail screenshot
  -d, --datestamp       include date in filename
  -D DIR, --dir=DIR     directory to place images into
  --delay=DELAY         delay between page load finishing and screenshot
  --noimages            don't load images
  --transparent         render output on a transparent background (be sure to
                        have a transparent background defined in the html)
  --nojs                disable JavaScript support
  --js=JS               JavaScript to execute when the window finishes loading

User story

1
2
3
4
Feature: webkit2loop
    In order to take screenshots of sites that change over time
    As a Mac geek
    I need to call webkit2png in a loop with different delay values

Passing options through

1
2
3
4
Scenario: passing options to webkit2png
    Given that I have webkit2png in my path
    When I call webkit2loop with any of these webkit2png options: [ "width", "height", "zoom", "filename", "dir"]
    Then webkit2png should be called with the very same options

For my first attempt I simply pass on whatever options the script is called with, except for a few that wouldn’t make sense.

First step was to upgrade to Python 2.7.2, as the new argparse module was introduced for it. That was standard procedure, go to the Python site, get the installer, follow instructions.

First iteration simply called webkit2png with some hard coded variables

1
2
3
4
5
6
7
import subprocess

def webkit2png():
    subprocess.call( "webkit2png -F -d -D ~/Desktop/webkit2png_images/ -o 06 --delay 1.65 http://soulwire.co.uk/hello".split() )

if __name__ == "__main__":
    webkit2png()

Turns out subprocess.call doesn’t like the tilde at the start of ~/Desktop - changed it to an absolute path for now to make it work.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
#!/usr/bin/env python
import subprocess
import argparse

import sys

def webkit2png( stack=[] ):
    if 0 == len( stack ):
        print "function webkit2png called with no arguments"
        sys.exit( 1 )

    #hardcoded options, and the command
    stack.insert( 0, "-F" )
    stack.insert( 0, "webkit2png" )
    subprocess.call( stack )

def main():
    stack = [] #collects options

    #sets up command
    parser = argparse.ArgumentParser(description="Wrapper around webkit2png. Allows it to be called in a loop")
    parser.add_argument( "url",
                    action="store",
                    help="the url of the site to take screenshots of")
    parser.add_argument( "-W", "--width",
                    action="store",
                    help="initial (and minimum) width of browser (default: 800)")
    parser.add_argument( "-H", "--height",
                    action="store",
                    help="initial (and minimum) height of browser (default: 600)")
    parser.add_argument( "-z", "--zoom",
                    action="store",
                    help="full page zoom of browser (default: 1.0)")
    parser.add_argument( "-o", "--filename",
                    action="store",
                    help="save images as NAME-full.png,NAME-thumb.png etc")
    parser.add_argument( "-D", "--dir",
                    action="store",
                    help="save images as NAME-full.png,NAME-thumb.png etc")
    args = parser.parse_args()

    #collects options - can fix any issues here
    #TODO
    #subprocess.call doesnt' like paths starting with a ~ it would be nice to substitute that
    for a in [ "width", "height", "zoom", "filename", "dir"]:
        if args.__dict__[a]:
            stack.append( "--" + a )
            stack.append( args.__dict__[a] )
    stack.append( args.url )

    webkit2png( stack )

if __name__ == "__main__":
    main()

Using the argparser module to run the command. No fancy stuff, just collecting options and sticking them in a an array. Wanting to avoid to have an if: clause for every option I tried to write an iterator, but parser.parse_args returns a Namespace object which is not iterable. Eventually I found out about dict, which can be used to iterate through a Namespace object. The options are collected in an array then passed to the webkit2png command.

Taking screenshots at regular intervals

1
2
3
4
5
6
Scenario: taking screenshots at regular intervals
    Given that I can call webkit2png from webkit2loop
    When I pass the option -i or --interval followed by a floating number
    And I pass the option -s or --start followed by a floating number
    And I pass the option -n or --iterations followed by an integer less than 101
    Then webkit2png should be called -n times, each time with a delay equal to -n * -i + -s

This scenario describes three new options to take screenshot at regular intervals: the length of the interval, an initial offset, and the max number of screenshots to take (capped to 100). For brevity I didn’t include a scenario which describes what the default values should be, nor how to handle file naming.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
#!/usr/bin/env python
import subprocess
import argparse
import sys, os

"""calls the webkit2png
    stack is an array of arrays, each array contains the options for a call to webkit2png"""
def webkit2png( stack=None ):
    if stack is None:
        print "No stack passed to webkit2pnt"
        sys.exit(1)
    for i in range( 0, len(stack)):
        substack = stack[i][:]
        substack.insert( 0, "-F" )
        substack.insert( 0, "webkit2png" )
        subprocess.call( substack )

def main():
    sharedStack = [] #shared options among all calls (if more than one)
    singleStack = [] #current command
    cmdStack = []    #collection of all commands
    parser = argparse.ArgumentParser(description="Wrapper around webkit2png. Allows it to be called in a loop")
    parser.add_argument( "url",
                    action="store",
                    help="the url of the site to take screenshots of")
    parser.add_argument( "-i", "--interval", type=float,
                    action="store",
                    help="interval between regular screenshots")
    parser.add_argument( "-s", "--start",
                    action="store",
                    help="initial offest before starting to take regular screenshots")
    parser.add_argument( "-n", "--iterations",
                    action="store",
                    help="number of regular screenshots (max 100, default 3)")
    parser.add_argument( "-W", "--width",
                    action="store",
                    help="initial (and minimum) width of browser (default: 800)")
    parser.add_argument( "-H", "--height",
                    action="store",
                    help="initial (and minimum) height of browser (default: 600)")
    parser.add_argument( "-z", "--zoom",
                    action="store",
                    help="full page zoom of browser (default: 1.0)")
    parser.add_argument( "-o", "--filename",
                    action="store",
                    help="save images as NAME-full.png,NAME-thumb.png etc")
    parser.add_argument( "-D", "--dir",
                    action="store",
                    help="save images as NAME-full.png,NAME-thumb.png etc")
    args = parser.parse_args()

    """collects the shared options to be passed on to webkit2png - as they are or with some collection"""
    for a in [ "width", "height", "zoom", "dir"]:
        if args.__dict__[a]:
            sharedStack.append( "--" + a )
            sharedStack.append( args.__dict__[a] )
    #subprocess.call doesn't like arguments starting with ~, so make sure there aren't any here
    if args.dir:
        sharedStack.append( "--dir" )
        sharedStack.append( os.path.expanduser( args.dir ) )

    if args.interval:
        s = float( args.start ) if args.start else 0
        d = args.interval
        n = args.iterations if args.iterations else 3
        n = n if n <= 100 else 100
        for i in range( 0, n ):
            singleStack = sharedStack[:]
            singleStack.append( "--delay" )
            singleStack.append( str(s) )
            singleStack.append( "--filename" )
            if args.filename:
                singleStack.append( '%(stem)s-%(number)03d' % { "stem" : args.filename, "number" : i } )
            else:
                singleStack.append( '%(number)03d' % { "number" : i } )
            singleStack.append( args.url )
            cmdStack.append( singleStack )
            s = s + d
    else:
        singleStack = sharedStack[:]
        #special case - filename needs to change for multiple images
        if args.filename:
            sharedStack.append( "--filename" )
            sharedStack.append( args.filename )
        singleStack.append( args.url )
        cmdStack.append( singleStack )

    webkit2png( cmdStack )

if __name__ == "__main__":
    main()

In webkit2png:

  • Changed the default to webkit2png call to None, as an empty variable is sticky and ended up having problems

  • Changed the function to take an array of arrays, where each array contains the parameters for a call to webkit2png. Obviously this is not an elegant way of doing it, but it’s my first script and will have to do for now

In main:

  • Added the three new options

  • Fixed issue with subprocess.call not liking arguments starting with ~, using os.path.expanduser

  • set default of 3 for the number of loops, and 0 for start

  • used a formatter to make the name of each file made up of filename followed by a dash (if filename was passed) then followed by a zero padded index

Challenge 100% complete

This was a small challenge, as I didn’t have that much time to spend on it. I didn’t add a timeline argument to pass a list of arbitrary delays, nor did I extend the original script as I intended to do. Still, it works, and I learned a bit of Python.

Comments