indi.ca

---

Apologies to Sumna, Kottu actually was messed up. There are too many people and its been choking on its own XML vomit for a while. Moved to Dreamhost, upgraded the FeedWordpress plugin, and told a Cron Daemon to update every ten minutes. This might be a semi-pertinent time to go into the structure of Kottu in case someone wants to replicate it (actually, looks like someone has, wholesale). The entire process takes about 30 minutes of actual work, aside from design. From there the Daemon does the updates and only thing I/Mahangu/Raveen do is add new feeds. Structurally, Kottu is basically a metablog that pulls in its posts from 100 other blogs. It does this through the magic of XML (specifically RSS and Atom). Every modern website spits out an XML ‘radio’ feed which Kottu simply listens in on. The only catch is that I used to have to keep pushing the freaking ‘Refresh’ button all the time, but you can automate that with something called a Cron Daemon. The Cron Daemon will execute the ‘Refresh’ command at intervals I specify, so from there Kottu pretty much runs on its own.

Intro: XML

Kottu and more interesting meta services like Technorati are enabled by something called XML. What XML stands for is eXtensible Markup Language, and it’s basically a way of tagging things. Very loosely, I can say

[title] Kottu and the Cron Daemon [/title]
[description] blah blah blah [/description]

and that’s XML, just replace the brackets with the little carrot symbol (<) and it'll sorta run. The next generation of MS Office saves as XML and you could theoretically tag anything in this manner, for example. [fruit] apple [/fruit] XML is just a way of tagging your data so it's more than a meaningless pile of bits and words. This tagging lets you perform higher operations on the data, which is what Kottu does. There are 100 sites on Kottu, each on different platforms which Kottu has no control over. XML, however, puts all the data into nice labeled boxes that Kottu can process as an automaton. There are different types of XML, though they don't really matter. WordPress, WordPress.com and Blogsome tend to output everything (RSS, Atom, and RDF) while Blogger outputs Atom. What's important is that they're all standardized, so you can take modern websites and read/use them in a lot of different ways. I, for example, don't read Kottu at all. I just subscribe to the Kottu Feed in my email program.

Kottu Recipe

Since the data is XML, there’s lots of ways to play with it, of which Kottu is only one. This is just one specific hack, not necessarily the best or most efficient way to syndicate a bunch of sites. It is, however, simple and it uses a platform I’m comfortable with. Kottu has 3 ingredients: a WordPress Blog, the FeedWordpress plugin and one Cron Daemon.

1. WordPress (1.5.2)

WordPress is blogging software. You place the files on a server, connect the pipes to a MySQL database and it manages your blog from there. Posts, edits, themes, feeds, etc. It’s what indi.ca runs on. It’s really cool and I won’t get that much into it here. The current version is 2.0, which includes a lot of good features like easy File/Image Uploads and a graphical text editor, but Kottu uses 1.5.2. That is specifically because the following plugin has some compatibility problems,

2. FeedWordpress

Charles Johnson has written a remarkably functional plugin which handles all the work of taking hundreds of different feeds and feeding them into one blog. WordPress includes the functionality to add links to sites you like, FeedWordpress takes those links, finds their XML feed, and then pulls those posts into WordPress as if you’d written them locally. I don’t understand how, but I’m glad its there.

3. Cron

One thing FeedWordpress does not do is self-update. You have to refresh manually by visiting a certain URL. Or, you can set a Cron Daemon to visit that URL for you at specific intervals. If your webhost has cPanel you can set Cron jobs from there. Dreamhost doesn’t use cPanel, so I set the crontab from command line, which is actually faster and more fun. The commands are

crontab -l (to list current Crons)
crontab -e (to edit a Cron job)

and finally

0,10,20,30,40,50 * * * * curl http://kottu.org/wp-content/update-feeds.php -d update=quiet

This is somewhat more complicated than it looks, Cron just runs on time specified in a [minutes, hours, day-of-month, month, day-of-week] format. I want Kottu to update every 10 minutes, so I specified that I want it to update every 0th, 10th, 20th, 30th, 40th, and 50th minute. There is a more efficient way to write this but I’m lazy. I don’t care about the other fields so I left them blank. ‘curl’ simply retrieves a URL, and the quiet update means to suppress the display cause no one but the Cron Daemon is watching. The Cron Daemon, by the way, is the program that monitors the crontab to see if he has any work to do.

If that has confused the shit out of you I think you can also install a Cron Monkey to do the same thing from within WordPress itself.

4. FlickrRSS (optional)

Flickr ‘broadcasts’ an RSS feed of its latest photos for each tag that people use. I used the FlickrRSS plugin to listen in for any new photos tagged ‘srilanka’ or ‘lanka’ and pull them into Kottu. Flickr will give you the code to pull in feeds itself, but I find this plugin more flexible.

Oh, and finally you need a bunch of bloggers with XML feeds. That means anyone using WordPress, Movable Type, Blogger, Blogsome, WordPress.com, Squarespace, Xanga, LiveJournal, etc. Basically any and every blog works. Kottu was Mahangu’s idea (and name) but rather than one group-blog, we decided on this decentralized approach. Works for me, because once its run it requires exactly zero attention from me, except when it comes to adding feeds. There’s still a human element to that. So anyways, Kottu should update a lot better now, and this is the recipe in case anyone wants to replicate it.