Make parse-metadata suck a little less. Many docs updates.

README.md

@@ -10,6 +10,7 @@
 Coleslaw is Flexible Lisp Blogware similar to [Frog](https://github.com/greghendershott/frog), [Jekyll](http://jekyllrb.com/), or [Hakyll](http://jaspervdj.be/hakyll/).
 
 ## Features
+
 * Git for storage
 * RSS and Atom feeds
 * Markdown Support with Code Highlighting provided by [colorize](http://www.cliki.net/colorize)
@@ -30,32 +31,68 @@ Coleslaw is Flexible Lisp Blogware similar to [Frog](https://github.com/greghend
 * There is also a [Heroku buildpack](https://github.com/jsmpereira/coleslaw-heroku) maintained by Jose Pereira.
 
 ## Example Sites
-  * [redlinernotes](http://redlinernotes.com/blog/)
-  * [chip the glasses](http://chiptheglasses.com)
-  * [kenan-bolukbasi.log](http://kenanb.com/)
-  * [Nothing Really Matters](http://ironhead.xs4all.nl/)
-  * [A year and a smile](http://blog.sjas.de)
+
+See the [wiki][blogroll] for a list of coleslaw-powered blogs.
+
+[blogroll]: https://github.com/redline6561/coleslaw/wiki/Blog roll
 
 ## Hacking
 
-A core goal of *coleslaw* is to be both pleasant to read and easy to hack on and extend. If you want to understand the internals and bend *coleslaw* to do new and interesting things, I strongly encourage you to read the [Hacker's Guide to Coleslaw](https://github.com/redline6561/coleslaw/blob/master/docs/hacking.md).
+A core goal of *coleslaw* is to be both pleasant to read and easy to
+hack on and extend. If you want to understand the internals and bend
+*coleslaw* to do new and interesting things, I strongly encourage you
+to read the [Hacker's Guide to Coleslaw][hackers]. You'll find some
+current **TODO** items towards the bottom.
+
+[hackers]: https://github.com/redline6561/coleslaw/blob/master/docs/hacking.md
 
 ## Installation
-This software should be portable to any conforming Common Lisp implementation but testing is primarily done on [SBCL](http://www.sbcl.org/) and [CCL](http://ccl.clozure.com/).
-Server side setup:
 
-1. Setup git and create a bare repo as shown [here](http://git-scm.com/book/en/Git-on-the-Server-Setting-Up-the-Server).
-2. Install Lisp (we recommend SBCL) and [Quicklisp](http://quicklisp.org/).
-3. ```wget -c https://raw.github.com/redline6561/coleslaw/master/examples/example.coleslawrc -O ~/.coleslawrc``` # and edit as necessary
-4. ```wget -c https://raw.github.com/redline6561/coleslaw/master/examples/example.post-receive -O your-blog.git/hooks/post-receive``` # and edit as necessary
-5. ```chmod +x your-blog/.git/hooks/post-receive```
-6. Create or clone your blog repo locally. Add your server as a remote with ```git remote add prod git@my-host.com:path/to/repo.git```
-7. Point the web server of your choice at the symlink /path/to/deploy-dir/.curr/
+Coleslaw should run on any conforming Common Lisp implementation but
+testing is primarily done on [SBCL](http://www.sbcl.org/) and
+[CCL](http://ccl.clozure.com/).
+
+Coleslaw can either be run **manually** on a local machine or
+triggered **automatically** on git push to a server.  If you want a
+server install, run these commands on your server _after_ setting up a
+[git bare repo](http://git-scm.com/book/en/Git-on-the-Server-Setting-Up-the-Server).
+Otherwise, run the commands on your local machine.
+
+1. Install a Common Lisp implementation (we recommend SBCL) and
+   [Quicklisp](http://quicklisp.org/).
+2. Place a config file for coleslaw in your `$HOME` directory. If you
+   want to run multiple blogs with coleslaw, you can keep each blog's
+   config file in that blog's repo.  Feel free to copy and edit the
+   [example config][ex_config] or consult the [config docs][conf_docs]
+   to create one from scratch.
+3. * Server Install: Copy and `chmod +x` the
+     [example post-receive hook][post_hook] to your blog's bare repo.
+   * Local Install:  Just run the following commands in the
+     REPL whenever you're ready to regenerate your blog:
+     ```
+     (ql:quickload :coleslaw)
+     ;; Note that the incremental plugin currently requires a "last git revision" argument.
+     (coleslaw:main "/path/to/my/blog/")
+     ```
+4. Optionally, point the web server of your liking at your config-specified
+   `:deploy-dir`. Or "deploy-dir/.curr" if the `versioned` plugin is enabled.
+
+Now just write posts, git commit and build by hand or by push.
 
-Now whenever you push a new commit to the server, coleslaw will update your blog automatically! You may need to `git push -u prod master` the first time.
+[ex_config]: https://github.com/redline6561/coleslaw/blob/master/examples/example.coleslawrc
+[conf_docs]: https://github.com/redline6561/coleslaw/blob/master/docs/config.md
+[post_hook]: https://github.com/redline6561/coleslaw/blob/master/examples/example.post-receive
+
+## The Content Format
+
+Coleslaw expects content to have a file extension matching the class
+of the content. (I.e. `.post` for blog posts, `.page` for static
+pages, etc.)
+
+There should also be a metadata header on all files
+starting and ending with the config-specified `:separator`, ";;;;;" by
+default. Example:
 
-## The Post Format
-Coleslaw expects post files to be formatted as follows:
 ```
 ;;;;;
 title: foo
@@ -66,5 +103,16 @@ format: html (for raw html) or md (for markdown)
 your post
 ```
 
+Posts require the `title:` and `format:` fields.
+Pages require the `title:` and `url:` fields.
+
+To omit a field, simply do not have the line present, empty lines and
+fields (e.g. "tags:" followed by whitespace) will be ignored.
+
 ## Theming
-Two themes are provided: hyde and readable (based on [bootswatch readable](http://bootswatch.com/readable/)). Hyde is the default. A guide to creating themes for coleslaw lives [here](https://github.com/redline6561/coleslaw/blob/master/docs/themes.md).
+
+Two themes are provided: hyde, the default, and readable (based on
+[bootswatch readable](http://bootswatch.com/readable/)).
+
+A guide to creating themes for coleslaw lives
+[here](https://github.com/redline6561/coleslaw/blob/master/docs/themes.md).

docs/config.md

@@ -0,0 +1,37 @@
+# Configuration
+
+## Where
+
+Coleslaw needs a `.coleslawrc` file to operate properly. That file is usually located at
+$HOME/.coleslawrc but may also be placed in the blog repo itself.
+
+## What
+
+The only *required* information in the config is:
+* `:author`       => to be placed on post pages and in the copyright/CC-BY-SA notice
+* `:deploy-dir`   => for Coleslaw's generated HTML to go in
+* `:domain`       => to generate absolute links to the site content
+* `:routing`      => to determine the URL scheme of content on the site
+* `:title`        => to provide a site title
+* `:theme`        => to select one of the themes in "coleslaw/themes/"
+
+**TODO** Make sure this is actually true. Maybe without `:feeds` the publish method blows up? Test!
+
+It is usually recommend to start from the [example config][ex_config] and pare down from there.
+
+[ex_config]: https://github.com/redline6561/coleslaw/blob/master/examples/example.coleslawrc
+
+## Extras
+
+There are also many *optional* config parameters such as:
+* `:charset`       => to set HTML attributes for international characters, default: "UTF-8"
+* `:feeds`         => to generate RSS and Atom feeds for certain tagged content
+* `:lang`          => to set HTML attributes indicating the site language, default: "en"
+* `:license`       => to override the displayed content license, the default is CC-BY-SA
+* `:page-ext`      => to set the suffix of generated files, default: "html"
+* `:plugins`       => to configure and enable coleslaw's [various plugins][plugin-use]
+* `:separator`     => to set the separator for content metadata, default: ";;;;;"
+* `:sitenav`       => to provide relevant links and ease navigation
+* `:staging-dir`   => for Coleslaw to do intermediate work, default: "/tmp/coleslaw"
+
+[plugin-use]: https://github.com/redline6561/coleslaw/blob/master/docs/plugin-use.md

docs/hacking.md

@@ -221,8 +221,11 @@ PAGE, a content type for static page support, is available as a plugin.
 
 ### TODO for 0.9.6
 
-* Serious updates to README and themes.md. Skim hacking.md but it's probably okay.
-* Make read-content/load-content more robust.
+* Finish docs updates to:
+  * themes.md     -> Note debugging issues. Valid slots. Changed URL handling.
+  * plugin-use.md -> Incremental plugin, Versioned plugin. Twitter summary cards.
+  * hacking.md    -> Any changes to Document Protocol?
+  * config.md     -> Test various empty config slots.
 * A real test suite with Stefil that at *least* tests:
   * `read-content`
   * `load-config`
@@ -235,6 +238,7 @@ PAGE, a content type for static page support, is available as a plugin.
   * Theme Compilation
   * Content Publishing
   * Common Plugins including Injections
+* Add proper errors to read-content/load-content. Not just ignoring bad data. Line info, etc.
 * Improved template debugging? "${" instead of "{$", static checks for valid slots, etc.
   At least a serious investigation into how such things might be provided.
 * Some minor scripting conveniences with cl-launch? (Scaffold a post/page, Enable incremental, Build, etc).

src/content.lisp

@@ -50,16 +50,20 @@
 
 (defun parse-metadata (stream)
   "Given a STREAM, parse metadata from it or signal an appropriate condition."
-  (flet ((parse-field (str)
-           (nth-value 1 (cl-ppcre:scan-to-strings "[a-zA-Z]+:\\s+(.*)" str)))
-         (field-name (line)
-           (make-keyword (string-upcase (subseq line 0 (position #\: line))))))
-    (unless (string= (read-line stream nil) (separator *config*))
-      (error "The provided file lacks the expected header."))
-    (loop for line = (read-line stream nil)
+  (labels ((get-next-line (input)
+             (string-trim '(#\Space #\Newline #\Tab) (read-line input nil)))
+           (parse-value (str)
+             (nth-value 1 (cl-ppcre:scan-to-strings "[a-zA-Z]+:\\s+(.*)" str)))
+           (parse-initarg-name (line)
+             (make-keyword (string-upcase (subseq line 0 (position #\: line)))))
+           (extract-initarg (line)
+             (list (parse-initarg-name line) (aref (parse-value line) 0))))
+    (unless (string= (get-next-line stream) (separator *config*))
+      (error "The file lacks the expected header: ~a" (separator *config*)))
+    (loop for line = (get-next-line stream)
        until (string= line (separator *config*))
-       appending (list (field-name line)
-                       (aref (parse-field line) 0)))))
+       when (parse-value line)
+       appending (extract-initarg line))))
 
 (defun read-content (file)
   "Returns a plist of metadata from FILE with :text holding the content."