Listen Notes

Discovered via the excellent new podcast search-engine Listen Notes, a lively 2013 interview in which two bright young men interview Matt Ridley on The Rational Optimist, for about 50 minutes minus the intro and advert.


Using algorithms to map the history of ideas

Buried in what appears to be a leftist attack-article on the digital humanities, in the latest THES…

“Crane’s latest research project, with German federal funding, is as ambitious as they come: an attempt to use algorithms to map the history of ideas from antiquity to the present day, analysing millions of texts to chart the influence of thinkers on each other over the course of thousands of years.”

I can’t get more than a few snippets of the Times article, and a quick search turns up nothing via ‘last month’ Google Search or Google News. But it sounds like a fascinating project. Although the pathetic state of retail recommendation systems and taste matching (Amazon, eBay etc) suggests that the much-touted semantic revolution needed to do this sort of thing properly is still a long way in the future. You’re also still going to need a ninja historian to trace a chain of influence between things like Erasmus Darwin’s musing on Ancient Egyptian botanical symbolism, Blake’s Jerusalem, and the birth of the Victorian fairy.

Still, Crane’s work sounds like it may be an interesting start on an Asimov-style psychohistory of the past, and something which might one day also be applied to outlining the near-future. How might that work? Well, we might be able to formulate some hard ‘laws of ideas’ and attach probabilities to how well these will propagate if the conditions remain stable. If we can spot a similar idea and its native conditions in the present, we may then be able to project that idea’s development into the future with a greatly increased certainty about its trajectories and likely collision-points with other ideas.

The problem with such approaches, as the great Asimov suggested for psychohistory, is that it only works if it’s secret. One its rules are public, many will try to ‘game the system’ for their own advantage.