Continuous Learning

This is the sixth post in the series. You can find the rest here.

In the last article I talk about RSS aggregation. While aggregation is great, it does not add additional value over going directly to the author’s site if all we do is republish their articles. To add value, there must be more:

• We republish articles that are focused on the topics (i.e. being selective)
• We relate articles with each other (again, reinforcing the focus)
• We provide additional commentary on the articles if applicable

Let’s see what we can do.

Daily Del.icio.US

Many bloggers use blogging to track links that they visit, and this approach is closer to what we are talking about. If I blog about a list of links I go to, the links would automatically be focused and related, and I can add commentaries as necessary.

Many people of course use social bookmarking tool such as del.icio.us to track such links, and then use plugins to post their “daily del.icio.us” over to their blog.

A quick search on WP plugins shows a whole section for del.icio.us integration, and there are a couple that works for this purpose. The one I look at is Daily Del.icio.US.

Using it is easy. Download, change configuration (on del.icio.us user name, blog info, etc), and just run it via cron. You can even change the code so it creates drafts instead of auto publishing.

This approach is great - it automatically meets all three criteria. But this is also a bit simplistic.

Techmeme

The daily del.icio.us code AS IS have the following limitations:

• It does not work with multiple sources
• It does not handle posting to multiple different categories by keywords
• It also will not pull down information that you have not bookmarked yet, which means the effort is still largely manual
• It means that I cannot be the consumer of the blog; it would be nice if I am a reader too

What would be a good goal to aim for is something like Techmeme, where the site automatically discovers articles and relate them with each other. Techmeme works because it focuses on blogs and also focuses on breaking news. I would like this site to contain longer-lasting content, so not sure if it can work the same way or not, but it’s worth a try.

AFAIK there isn’t a readily-available Techmeme-like plugin for WP, but that’s just take time to assemble the following:

• RSS aggregator
• Web crawler
• Indexer
• Custom algorithm to relate content
• WP Interface (either database or XML-RPC)

Again, it won’t work exactly like Techmeme due to both content and medium, but would be interesting to see what it would look like.

What’s Next

This series is concluded for now until I have the system developed and test it out and figure out a better model. I am likely to spend most of this month working under the hood.

The code will be released in multiple iterations to include more functionality. I am most likely to start with one source to start the flow, and then add more sources to cross relate them with each other later. Digg/Reddit/Del.icio.us are all good sources that I can start from (because the users have already take time to sift through the web), so expect to daily excerpts soon from those sites soon