Merge pull request #18 from dirtbags/neale

Example puzzles, devel-server --puzzles option, and a few moth.py enhancements

README.md

@@ -29,10 +29,16 @@ for details.
 Getting Started Developing
 -------------------------------
 
-    $ git clone $your_puzzles_repo puzzles
+If you don't have a `puzzles` directory,
+you can copy the example puzzles as a starting point:
+
+    $ cp -r example-puzzles puzzles
+
+Then launch the development server:
+
     $ python3 tools/devel-server.py
 
-Then point a web browser at http://localhost:8080/
+Point a web browser at http://localhost:8080/
 and start hacking on things in your `puzzles` directory.
 
 More on how the devel sever works in

example-puzzles/example/1/puzzle.moth

@@ -13,17 +13,17 @@ Puzzle categories are laid out on the filesystem:
     ├─3
     │  └─puzzle.py
     ├─10
-    │  └─puzzle.py
+    │  └─puzzle.moth
     └─100
-       └─puzzle.moth
+       └─puzzle.py
 
 In this example,
 there are puzzles with point values 1, 2, 3, 10, and 100.
 
-Puzzles 1, 2, and 100 are "static" puzzles:
+Puzzles 1, 2, and 10 are "static" puzzles:
 their content was written by hand.
 
-Puzzles 3 and 10 are "dynamic" puzzles:
+Puzzles 3 and 100 are "dynamic" puzzles:
 they are generated from a Python module.
 
 To create a static puzzle, all you must have is a

example-puzzles/example/10/puzzle.moth

@@ -0,0 +1,90 @@
+Author: neale
+Summary: Making excellent puzzles
+Answer: moo
+
+Making Excellent Puzzles
+====================
+
+We (the dirtbags) use some core principles.
+If you're writing puzzles for our events,
+we expect you to bear these in mind.
+
+If you're doing your own event,
+you might still be interested in these:
+it's the result of years of doing these events
+and thinking hard about how to make the best possible experience for participants.
+
+
+Respect The Player
+---------------------------
+
+Start with the assumption that the person playing your puzzle is intelligent,
+can learn,
+and can figure out whatever you ask of them.
+If you observe in trial runs that they can't figure it out,
+that indicates a problem on your end,
+not theirs.
+
+
+Expect Creative Mayhem
+------------------------------------
+
+This is a hacking contest.
+People are going to be on the lookout for "ways around" your puzzles.
+This is something we want them to do:
+as a defender, they need to be always searching for holes in the defense.
+
+Here are some examples of "ways around".
+All of these have happened at prior events:
+
+* Logging into your appliance and changing the access password
+* Flooding your service off the network
+* Brute-force attacking your 2-letter answer
+* Photographing your printed list of answers while a team-mate chats with you
+* Writing a script to create hundreds of new accounts faster than you can remove them
+
+It's important to remember that they are here precisely to cause mayhem.
+If you set up a playground where this sort of mayhem ruins the experience for other players,
+we (the people running the event) can't penalize participants for doing what we've asked them to.
+Instead, we have to devalue points in your category to the point where nobody wants to bother playing it any more.
+
+
+Make The Work Easier than Guessing The Answer
+--------------------------------------------------------
+
+An important corrolary to expecting creative mayhem is that people are going to
+work hard to come up with creative ways to get points without doing the work you ask.
+
+It's okay to make your work about the same level of difficulty as guessing the answer.
+For instance, we use a few 4-byte answers.
+It would take about four hours to try submitting every possible answer,
+and if people want to spend their time coding up a brute-force attack,
+that is a constructive use of their event time and we're okay with it.
+Typically people give up on this line of attack when they realize
+how much time it's going to take
+(figuring this out is also a productive use of time),
+but once in a while people will go ahead.
+
+This means your answers need to be resistant to guessing and brute force attacks.
+True/false questions are, needless to say, going to result in us laughing at you.
+
+
+There's More Than One Way To Do It
+----------------------------------------------------
+
+Don't make puzzles that require a specific solution strategy.
+For instance,
+we're not going to allow puzzles that require a certain technology
+(such as "what is the third item in the File menu on product X's interface?").
+
+Rather, if your goal is to highlight a technology,
+you should create puzzles that show off how great the product is for this application,
+compared to alternatives.
+Something like "mount this file system from 1982 and paste the contents of the file named `foo`"
+might be trivial in a certain product and take hours by other means.
+That's fine.
+That's what we want to see.
+
+----
+
+The answer for this page is `moo`

example-puzzles/example/100/puzzle.py

@@ -0,0 +1,25 @@
+#!/usr/bin/python3
+
+import io
+
+def make(puzzle):
+    puzzle.author = 'neale'
+    puzzle.summary = 'crazy stuff you can do with puzzle generation'
+
+    puzzle.body.write("## Crazy Things You Can Do With Puzzle Generation\n")
+    puzzle.body.write("\n")
+    puzzle.body.write("The source to this puzzle has some advanced examples of stuff you can do in Python.\n")
+    puzzle.body.write("\n")
+
+    # You can use any file-like object; even your own class that generates output.
+    f = io.BytesIO("This is some text! Isn't that fantastic?".encode('utf-8'))
+    puzzle.add_stream(f)
+
+    # We have debug logging
+    puzzle.log("You don't have to disable puzzle.log calls to move to production; the debug log is just ignored at build-time.")
+    puzzle.log("HTML is <i>escaped</i>, so you don't have to worry about that!")
+
+    puzzle.answers.append('coffee')
+    answer = puzzle.make_answer()        # Generates a random answer, appending it to puzzle.answers too
+    puzzle.log("Answers: {}".format(puzzle.answers))
+

example-puzzles/example/2/puzzle.moth

@@ -1,5 +1,6 @@
 Author: neale
 Summary: Static puzzle resource files
+File: salad.jpg
 Answer: salad
 
 You can include additional resources in a static puzzle,

example-puzzles/example/3/puzzle.py

@@ -14,7 +14,14 @@ def make(puzzle):
     puzzle.body.write("(Participants don't like it when puzzles and answers change.)\n")
     puzzle.body.write("\n")
 
+    puzzle.add_file('salad.jpg')
+    puzzle.body.write("Here are some more pictures of salad:\n")
+    puzzle.body.write("<img src='salad.jpg' alt='Markdown lets you insert raw HTML if you want'>")
+    puzzle.body.write("![salad](salad.jpg)")
+    puzzle.body.write("\n\n")
+
     number = puzzle.rand.randint(20, 500)
     puzzle.log("One is the loneliest number, but {} is the saddest number.".format(number))
 
-    puzzle.body.write("The answer for this page is {}.\n".format(answer))
+    puzzle.body.write("The answer for this page is `{}`.\n".format(answer))
+

install

@@ -1,75 +0,0 @@
-#! /bin/sh
-
-DESTDIR=$1
-
-if [ -z "$DESTDIR" ]; then
-	echo "Usage: $0 DESTDIR"
-	exit
-fi
-
-cd $(dirname $0)
-
-older () {
-	[ -z "$1" ] && return 1
-	target=$1; shift
-	[ -f $target ] || return 0
-	for i in "$@"; do
-		[ $i -nt $target ] && return 0
-	done
-	return 1
-}
-
-copy () {
-	target=$DESTDIR/$1
-	if older $target $1; then
-		echo "COPY $1"
-		mkdir -p $(dirname $target)
-		cp $1 $target
-	fi
-}
-
-setup() {
-	[ -d $DESTDIR/state ] && return
-	echo "SETUP"
-	for i in points.new points.tmp teams; do
-		dir=$DESTDIR/state/$i
-		mkdir -p $dir
-		setfacl -m ${www}:rwx $dir
-	done
-	mkdir -p $DESTDIR/packages
-	>> $DESTDIR/state/points.log
-	if ! [ -f $DESTDIR/assigned.txt ]; then
-		hd </dev/urandom | awk '{print $3 $4 $5 $6;}' | head -n 100 > $DESTDIR/assigned.txt
-	fi
-}
-
-
-echo "Figuring out web user..."
-for www in www-data http tc _ _www; do
-	id $www && break
-done
-if [ $www = _ ]; then
-	echo "Unable to determine httpd user on this system. Dying."
-	exit 1
-fi
-
-mkdir -p $DESTDIR || exit 1
-
-setup
-git ls-files | while read fn; do
-	case "$fn" in
-	install|.*)
-		;;
-	doc/*)
-		;;
-	www/*)
-		copy $fn
-		;;
-	bin/*)
-		copy $fn
-		;;
-	*)
-		echo "??? $fn"
-		;;
-	esac
-done

package-puzzles

@@ -1,127 +0,0 @@
-#!/usr/bin/env python3
-
-import argparse
-import binascii
-import glob
-import hashlib
-import io
-import json
-import os
-import shutil
-import sys
-import zipfile
-
-import puzzles
-
-TMPFILE = "%s.tmp"
-
-def write_kv_pairs(ziphandle, filename, kv):
-    """ Write out a sorted map to file
-    :param ziphandle: a zipfile object
-    :param filename: The filename to write within the zipfile object
-    :param kv:  the map to write out
-    :return:
-    """
-    filehandle = io.StringIO()
-    for key in sorted(kv.keys()):
-        if type(kv[key])  == type([]):
-            for val in kv[key]:
-                filehandle.write("%s: %s%s" % (key, val, os.linesep))
-        else:
-            filehandle.write("%s: %s%s" % (key, kv[key], os.linesep))
-    filehandle.seek(0)
-    ziphandle.writestr(filename, filehandle.read())
-   
-if __name__ == '__main__':        
-    parser = argparse.ArgumentParser(description='Build a category package')
-    parser.add_argument('categorydirs', nargs='+', help='Directory of category source')
-    parser.add_argument('outdir', help='Output directory')
-    args = parser.parse_args()
-
-    for categorydir in args.categorydirs:
-        puzzles_dict = {}
-        secrets = {}
-
-        categoryname = os.path.basename(categorydir.strip(os.sep))
-        
-        # create zipfile
-        zipfilename = os.path.join(args.outdir, "%s.zip" % categoryname)
-        if os.path.exists(zipfilename):
-            # open and gather some state
-            zf = zipfile.ZipFile(zipfilename, 'r')
-            try:
-                category_seed = zf.open(os.path.join(categoryname, "category_seed.txt")).read().strip()
-            except:
-                pass
-            zf.close()
-
-        zf = zipfile.ZipFile(TMPFILE % zipfilename, 'w')
-        if 'category_seed' not in locals():
-            category_seed =  binascii.b2a_hex(os.urandom(20))
-
-        # read in category details (will be pflarr in future)
-        for categorypath in glob.glob(os.path.join(categorydir, "*", "puzzle.moth")):
-            points = categorypath.split(os.sep)[-2] # directory before '/puzzle.moth'
-            categorypath = os.path.dirname(categorypath)
-            print(categorypath)
-            try:
-                points = int(points)
-            except:
-                print("Failed to identify points on: %s" % categorypath, file=sys.stderr)
-                continue
-            puzzle = puzzles.Puzzle(category_seed, categorypath)
-            puzzles_dict[points] = puzzle
-
-        # build mapping, answers, and summary
-        mapping = {}
-        answers = {}
-        summary = {}
-        for points in sorted(puzzles_dict):
-            puzzle = puzzles_dict[points]
-            hashmap = hashlib.sha1(category_seed)
-            hashmap.update(str(points).encode('utf-8'))
-            mapping[points] = hashmap.hexdigest()
-            answers[points] = puzzle['answer']
-            if len(puzzle['summary']) > 0:
-                summary[points] = puzzle['summary']
-
-        # write mapping, answers, summary, category_seed
-        write_kv_pairs(zf, os.path.join(categoryname, 'map.txt'), mapping)
-        write_kv_pairs(zf, os.path.join(categoryname, 'answers.txt'), answers)
-        write_kv_pairs(zf, os.path.join(categoryname, 'summary.txt'), summary)
-        zf.writestr(os.path.join(categoryname, "category_seed.txt"), category_seed)
-
-        # write out puzzles
-        for points in sorted(puzzles_dict):
-            puzzle = puzzles_dict[points]
-            puzzledir = os.path.join(categoryname, 'content', mapping[points])
-            puzzledict = {
-                'author': puzzle.author,
-                'hashes': puzzle.hashes(),
-                'files': [f.name for f in puzzle.files if f.visible],
-                'body': puzzle.html_body(),
-            }
-            secretsdict = {
-                'summary': puzzle.summary,
-                'answers': puzzle.answers,
-            }
-
-            # write associated files
-            assoc_files = []
-            for fobj in puzzle['files']:
-                if fobj.visible == True:
-                    assoc_files.append(fobj.name)
-                zf.writestr(os.path.join(puzzledir, fobj.name), \
-                                    fobj.handle.read())
-
-            if len(assoc_files) > 0:
-                puzzlejson["associated_files"] = assoc_files
-
-            # non-optimal writing of file-like objects, but it works
-            zf.writestr(os.path.join(puzzledir, 'puzzle.json'), \
-                                    json.dumps(puzzlejson))
-        
-        # clean up
-        zf.close()
-        shutil.move(TMPFILE % zipfilename, zipfilename)
-#vim:py

tools/build-puzzles.py

@@ -1,84 +0,0 @@
-#!/usr/bin/python3
-
-import hmac
-import base64
-import argparse
-import glob
-import json
-import os
-import markdown
-import random
-
-messageChars = b'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
-
-def djb2hash(buf):
-    h = 5381
-    for c in buf:
-        h = ((h * 33) + c) & 0xffffffff
-    return h
-
-class Puzzle:
-    def __init__(self, stream):
-        self.message = bytes(random.choice(messageChars) for i in range(20))
-        self.fields = {}
-        self.answers = []
-        self.hashes = []
-
-        body = []
-        header = True
-        for line in stream:
-            if header:
-                line = line.strip()
-                if not line.strip():
-                    header = False
-                    continue
-                key, val = line.split(':', 1)
-                key = key.lower()
-                val = val.strip()
-                self._add_field(key, val)
-            else:
-                body.append(line)
-        self.body = ''.join(body)
-
-    def _add_field(self, key, val):
-        if key == 'answer':
-            h = djb2hash(val.encode('utf8'))
-            self.answers.append(val)
-            self.hashes.append(h)
-        else:
-            self.fields[key] = val
-
-    def publish(self):
-        obj = {
-            'author': self.fields['author'],
-            'hashes': self.hashes,
-            'body': markdown.markdown(self.body),
-        }
-        return obj
-
-    def secrets(self):
-        obj = {
-            'answers': self.answers,
-            'summary': self.fields['summary'],
-        }
-        return obj
-    
-        
-parser = argparse.ArgumentParser(description='Build a puzzle category')
-parser.add_argument('puzzledir', nargs='+', help='Directory of puzzle source')
-args = parser.parse_args()
-
-for puzzledir in args.puzzledir:
-    puzzles = {}
-    secrets = {}
-    for puzzlePath in glob.glob(os.path.join(puzzledir, "*.moth")):
-        filename = os.path.basename(puzzlePath)
-        points, ext = os.path.splitext(filename)
-        points = int(points)
-        puzzle = Puzzle(open(puzzlePath))
-        puzzles[points] = puzzle
-
-    for points in sorted(puzzles):
-        puzzle = puzzles[points]
-        print(puzzle.secrets())
-

tools/devel-server.py

@@ -1,5 +1,10 @@
 #!/usr/bin/env python3
 
+# To pick up any changes to this file without restarting anything:
+#     while true; do ./tools/devel-server.py --once; done
+# It's kludgy, but it gets the job done.
+# Feel free to make it suck less, for example using the `tcpserver` program.
+
 import glob
 import html
 import http.server
@@ -26,7 +31,6 @@ sys.dont_write_bytecode = True
 # XXX: This will eventually cause a problem. Do something more clever here.
 seed = 1
 
-
 def page(title, body):
     return """<!DOCTYPE html>
 <html>
@@ -58,6 +62,8 @@ class ThreadingServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
 
 
 class MothHandler(http.server.SimpleHTTPRequestHandler):
+    puzzles_dir = "puzzles"
+
     def handle_one_request(self):
         try:
             super().handle_one_request()
@@ -122,7 +128,7 @@ you are a fool.
         puzzle = None
 
         try:
-            fpath = os.path.join("puzzles", parts[2])
+            fpath = os.path.join(self.puzzles_dir, parts[2])
             points = int(parts[3])
         except:
             pass
@@ -135,15 +141,16 @@ you are a fool.
         if not cat:
             title = "Puzzle Categories"
             body.write("<ul>")
-            for i in sorted(glob.glob(os.path.join("puzzles", "*", ""))):
+            for i in sorted(glob.glob(os.path.join(self.puzzles_dir, "*", ""))):
-                body.write('<li><a href="{}">{}</a></li>'.format(i, i))
+                bn = os.path.basename(i.strip('/\\'))
+                body.write('<li><a href="/puzzles/{}">puzzles/{}/</a></li>'.format(bn, bn))
             body.write("</ul>")
         elif not puzzle:
             # List all point values in a category
             title = "Puzzles in category `{}`".format(parts[2])
             body.write("<ul>")
             for points in cat.pointvals:
-                body.write('<li><a href="/puzzles/{cat}/{points}">puzzles/{cat}/{points}</a></li>'.format(cat=parts[2], points=points))
+                body.write('<li><a href="/puzzles/{cat}/{points}/">puzzles/{cat}/{points}/</a></li>'.format(cat=parts[2], points=points))
             body.write("</ul>")
         elif len(parts) == 4:
             # Serve up a puzzle
@@ -175,7 +182,7 @@ you are a fool.
             try:
                 pfile = puzzle.files[parts[4]]
             except KeyError:
-                self.send_error(HTTPStatus.NOT_FOUND, "File not found")
+                self.send_error(HTTPStatus.NOT_FOUND, "File not found. Did you add it to the Files: header or puzzle.add_stream?")
                 return
             ctype = self.guess_type(pfile.name)
             self.send_response(HTTPStatus.OK)
@@ -226,10 +233,22 @@ you are a fool.
         self.wfile.write(payload)
 
 
-def run(address=('localhost', 8080)):
+def run(address=('localhost', 8080), once=False):
     httpd = ThreadingServer(address, MothHandler)
     print("=== Listening on http://{}:{}/".format(address[0], address[1]))
+    if once:
+        httpd.handle_request()
+    else:
         httpd.serve_forever()
 
 if __name__ == '__main__':
-    run()
+    import argparse
+
+    parser = argparse.ArgumentParser(description="MOTH puzzle development server")
+    parser.add_argument('--puzzles', default='puzzles',
+                        help="Directory containing your puzzles")
+    parser.add_argument('--once', default=False, action='store_true',
+                        help="Serve one page, then exit. For debugging the server.")
+    args = parser.parse_args()
+    MothHandler.puzzles_dir = args.puzzles
+    run(once=args.once)

tools/moth.py

@@ -74,7 +74,7 @@ class Puzzle:
 
     def log(self, msg):
         """Add a new log message to this puzzle."""
-        self.logs.append(msg)
+        self.logs.append(str(msg))
 
     def read_stream(self, stream):
         header = True
@@ -86,6 +86,7 @@ class Puzzle:
                     continue
                 key, val = line.split(':', 1)
                 key = key.lower()
+                val = val.strip()
                 if key == 'author':
                     self.author = val
                 elif key == 'summary':
@@ -116,11 +117,11 @@ class Puzzle:
         except FileNotFoundError:
             puzzle_mod = None
 
-        if puzzle_mod:
         with pushd(path):
+            if puzzle_mod:
                 puzzle_mod.make(self)
             else:
-            with open(os.path.join(path, 'puzzle.moth')) as f:
+                with open('puzzle.moth') as f:
                     self.read_stream(f)
 
     def random_hash(self):
@@ -146,12 +147,17 @@ class Puzzle:
             name = self.random_hash()
         self.files[name] = PuzzleFile(stream, name, visible)
 
+    def add_file(self, filename, visible=True):
+        fd = open(filename, 'rb')
+        name = os.path.basename(filename)
+        self.add_stream(fd, name=name, visible=visible)
+
     def randword(self):
         """Return a randomly-chosen word"""
 
         return self.rand.choice(ANSWER_WORDS)
 
-    def make_answer(self, word_count, sep=' '):
+    def make_answer(self, word_count=4, sep=' '):
         """Generate and return a new answer. It's automatically added to the puzzle answer list.
         :param int word_count: The number of words to include in the answer.
         :param str|bytes sep: The word separator.
@@ -173,7 +179,7 @@ class Puzzle:
     def hashes(self):
         "Return a list of answer hashes"
 
-        return [djbhash(a) for a in self.answers]
+        return [djb2hash(a.encode('utf-8')) for a in self.answers]
 
 
 class Category:
@@ -207,6 +213,6 @@ class Category:
             puzzle.read_directory(path)
         return puzzle
 
-    def puzzles(self):
+    def __iter__(self):
         for points in self.pointvals:
             yield self.puzzle(points)

tools/package-puzzles.py

@@ -0,0 +1,112 @@
+#!/usr/bin/env python3
+
+import argparse
+import binascii
+import glob
+import hashlib
+import io
+import json
+import logging
+import moth
+import os
+import shutil
+import string
+import sys
+import tempfile
+import zipfile
+
+def write_kv_pairs(ziphandle, filename, kv):
+    """ Write out a sorted map to file
+    :param ziphandle: a zipfile object
+    :param filename: The filename to write within the zipfile object
+    :param kv:  the map to write out
+    :return:
+    """
+    filehandle = io.StringIO()
+    for key in sorted(kv.keys()):
+        if type(kv[key])  == type([]):
+            for val in kv[key]:
+                filehandle.write("%s: %s%s" % (key, val, os.linesep))
+        else:
+            filehandle.write("%s: %s%s" % (key, kv[key], os.linesep))
+    filehandle.seek(0)
+    ziphandle.writestr(filename, filehandle.read())
+
+def build_category(categorydir, outdir):
+    zipfileraw = tempfile.NamedTemporaryFile(delete=False)
+    zf = zipfile.ZipFile(zipfileraw, 'x')
+
+    category_seed = binascii.b2a_hex(os.urandom(20))
+    puzzles_dict = {}
+    secrets = {}
+
+    categoryname = os.path.basename(categorydir.strip(os.sep))
+    seedfn = os.path.join("category_seed.txt")
+    zipfilename = os.path.join(outdir, "%s.zip" % categoryname)
+    logging.info("Building {} from {}".format(zipfilename, categorydir))
+
+    if os.path.exists(zipfilename):
+        # open and gather some state
+        existing = zipfile.ZipFile(zipfilename, 'r')
+        try:
+            category_seed = existing.open(seedfn).read().strip()
+        except:
+            pass
+        existing.close()
+    logging.debug("Using PRNG seed {}".format(category_seed))
+
+    zf.writestr(seedfn, category_seed)
+
+    cat = moth.Category(categorydir, category_seed)
+    mapping = {}
+    answers = {}
+    summary = {}
+    for puzzle in cat:
+        logging.info("Processing point value {}".format(puzzle.points))
+
+        hashmap = hashlib.sha1(category_seed)
+        hashmap.update(str(puzzle.points).encode('utf-8'))
+        puzzlehash = hashmap.hexdigest()
+        
+        mapping[puzzle.points] = puzzlehash
+        answers[puzzle.points] = puzzle.answers
+        summary[puzzle.points] = puzzle.summary
+
+        puzzledir = os.path.join('content', puzzlehash)
+        files = []
+        for fn, f in puzzle.files.items():
+            if f.visible:
+                files.append(fn)
+            payload = f.stream.read()
+            zf.writestr(os.path.join(puzzledir, fn), payload)
+
+        puzzledict = {
+            'author': puzzle.author,
+            'hashes': puzzle.hashes(),
+            'files': files,
+            'body': puzzle.html_body(),
+        }
+        puzzlejson = json.dumps(puzzledict)
+        zf.writestr(os.path.join(puzzledir, 'puzzle.json'), puzzlejson)
+
+    write_kv_pairs(zf, 'map.txt', mapping)
+    write_kv_pairs(zf, 'answers.txt', answers)
+    write_kv_pairs(zf, 'summaries.txt', summary)
+
+    # clean up
+    zf.close()
+
+    shutil.move(zipfileraw.name, zipfilename)
+    
+   
+if __name__ == '__main__':        
+    parser = argparse.ArgumentParser(description='Build a category package')
+    parser.add_argument('categorydirs', nargs='+', help='Directory of category source')
+    parser.add_argument('outdir', help='Output directory')
+    args = parser.parse_args()
+
+    logging.basicConfig(level=logging.DEBUG)
+
+    for categorydir in args.categorydirs:
+        build_category(categorydir, args.outdir)
+