Great job! I encourage you to continue improving it.

I pasted a Paul Graham essay and the results were telling:

https://dl.dropbox.com/u/315/random_pics/nlp1.png

https://dl.dropbox.com/u/315/random_pics/nlp2.png

https://dl.dropbox.com/u/315/random_pics/nlp3.png

https://dl.dropbox.com/u/315/random_pics/nlp4.png

In short, the results aren't relevant at all -- either to the author or to his audience. (The third screenshot is particularly conspicuous.)

I can think of some domains where this could be pretty handy -- you should try feeding your algorithm a stream of tweets and see what happens.

All in all, this hasn't been done before and was presented quite nicely. Well done.

Another test case: I pasted a couple paragraphs of John Gruber's Kindle review in there, and it shows me (only) "Fleetwood Mac" (derived from the phrase "my Mac"), and "I Love" (the country music song, from the sentence "I love glowing screens").

It skipped terms that seem to me much more obvious links, like "iPhone", "iPad", "TiVo", "ATM", "Kindle", "credit card", "supermarket", or "e-ink".

This is very very cool, and makes for a good demo too.

One suggestion would be to put the latest topic on top - or at least vertically near where you type.

Second being able to pick the corpus being used for matching. might make it interesting. Say IEEE or ACM, or PubMed. Might bring up interesting things that others have written that are similar, which they've not noticed before.

I can see a very good match with technical writing, publications, etc. Esp if you can pull in excerpts. This would make for much better writing for many who may not have Eng as first language.

Neat!

Oh yes. Give me what you just described, integrated into a typesetting system that can produce scientific papers of LaTex quality (or, simply, a LaTex editor like Kile) and make citations to those sidebar infos drag & droppable, with links to the fulltext PDF (might need integrated Login handling for the electronic publishers).

Second idea: Scratch that and integrate it into a PDF reader instead and instead of drag & drop give me the citations as plaintext popups (this way it would be useful for other purposes, even book reading).

An interesting idea, but it comes up with really, really useless suggestion -- I don't think anybody needs to be told what don't is, west may be a cardinal direction but 'in the west we enjoy' does not refer to particular side of a compass.

Also for some reason it blinks with 1. AD is a year (I did write the word one, but it really shouldn't blink).

I see it as an interface demo, not a search results demo.

obviously you would connect it to the best search results at your disposal. that's a different problem entirely.

I think this would be a very good candidate for the future of document reading

Particularly for educational environments. Annotation of texts to explain cultural references, idiomatic usages, etc., is extremely useful in foreign language instruction, and anything that can help do that automatically and reduce the time an instructor has to to spend preparing the content would be extremely valuable.

FYI: relevant, not relevent.

It's a cool idea, but presented in a sidebar like that i think it's mostly a distraction. when you're writing you want less outside information, not more.

Fixed, thanks.

I guess there would need to be the ability to hide the sidebar in a more usable version. Thanks for that tip!

EDIT: grammar

What if instead of a sidebar, you could get information about a word by putting your mouse over it and doing something? (does javascript let you trap rightclicks?)

The nice thing about a sidebar is you can investigate a broader scope of the document's semantic content, as opposed to just specific sections. (but yes, it does... and this might be a good way to narrow the context)

Yes, it let's you trap right clicks. The event is called "contextmenu".

The idea is that as you type, some relevant information about what you are typing should appear. Long term, it should integrate search into document editing and simplify the writing process. This is currently far from perfect, I just put it together on the past 4 hours.

Given the privacy flap when Google started serving ads based on the contents of e-mails, do you foresee any worries here --or do you think it's sufficiently different, or that such worries have faded away with familiarity?

Also, as a minor nitpick, 'relevant' is misspelt (as 'relevent').

I would imagine the biggest privacy concern would be trusting a strange site with personal documents. Most people have no problem storing documents in Google drive but I foresee there would be a slow adoption rate for more serious writing.

Nice Work!

Your https doesn't work for me because you make calls to the unsecured duckduckgo api and chrome refuses.

You should use: https://api.duckduckgo.com

Now using the secure DDG. Thanks for pointing that out!

I would like to see the ability to drag and drop the text directly into the editor... and an ability to choose the sources for obtaining information (Wolfram Alpha, Wikipedia etc). Also, if you could highlight individual results and run a lookup against them you could drill down even further making the information more relevant... and as a sidenote, I be would inclined to target the student market whom could use this as a study tool (or lifesaver for last minute assignments)

Is it just me, or are we missing the source code? Would be nice to have a peek.

Edit: Sorry, didn't see that is was just JS! EditEdit: But where is the source to /keywords

(Did the developer claim somewhere the implementation was open source?)

No, but if it wasn't I would obfuscate my JS... ?

Wow, this is very very relevant to something I've been thinking of for the past few months -- if you're interested in collaborating, shoot me an email.

So... does this actually work?

I'm using the latest Safari and OS X 10.8... if I delete the contents of the text box, four of the terms on the right remain.

If I enter new content... still just those four terms.

edit: the more I write, the less terms there are on the right, but they're still terms from the original content... if I type a word twice, I get the definition... three times and it goes away.

There are in fact a lot of bugs, it was written in just a few hours as a proof of concept.

Typing "Bayesian classification of" somehow causes the relevant content "Naive Bayes classifier" to be added repeatedly in a loop.

Interesting idea. It could be improved by filtering out stop words; right now it recommends Oregon and Belgium for "or" and "be."

fyi, zemanta (zemanta.com) has been working on a similiar product (targeting bloggers) for years. Take a look at it.

google docs also has an integrated information sidebar this days

This is really cool and I can definitely see it being useful once the kinks are worked out.

Without repeating most of the feedback given already, I’ve noticed that it picks up too many pointless words. I’d suggest making it more selective with the words picked out to be relevant. Quality over quantity.

Very cool. I don't think this is the “future of document editing”, but it's a neat project nonetheless.

I was just testing it using other top news on HN, like this one: Roy Bates's death.

It seems there is a bug when I enter this line:

  "Roy Bates died on"

Paddy Roy Bates's wiki page keeps getting added to the right column continuously.

It might be useful to show relevant content from my local system on things that I've written already, like maybe notes from a project, or excerpts from email.

Writing code, similar suggestion, showing other code, requirements, design, etc.

I wrote a text with "German constitution" and now it's stuck in a loop fetching the information to "Basic Law for the Federal Republic of Germany".

http://i.imgur.com/0QaY2.png

The real time information is neat!

FYI, This feature is available (in non-real time manner) in Google Docs and Microsoft Word, which are powered by Google Search and Bing respectively to research terms, images and relevant content.

Modern remembrance agent (http://alumni.media.mit.edu/~rhodes/Papers/wear-ra-personalt...)

Ooh! I had the same idea about providing contextual information. But you executed on it. This is a wonderful tool that I'd use. All the best. Keep improving it.

In Firefox 16.0.1 on Linux, if I type a letter after the pre-existing text then the browser opens a tab that I had previously closed. Works fine on Chromium.

Very cool. The right hand side should auto-scroll.

When you write "times" in documents, it starts to flash (I mean time entry goes and comes back continuosly)

I just pushed an update that gives better results and fixes a few of the many bugs.

Thank you everyone for the feedback!

This is super amazing for writing Mathematics, if the context aggregation could be tweaked a bit.

looks cool ... what I have always found missing in text editors is a local undo (each line maintains its own undo history) ... bad from user experience and weird requirement I know but something like that would be useful for some programmers IMHO.

hey that's pretty cool. given that i was just reading about ted nelson's project xanadu this fits pretty neatly with that.

hopefully you can tune it to ignore various words that have less impact on things and instead focus on more core and unique elements.

That is the idea, I would like to go one step further. I see it as being in 2 possible 'states' The first would be some overview of the document and the second would be very relevant to what you are typing right now and what you are about to type. For example if you typed "Ted Nelson" right away you should get a picture, bio, birthday and anything else that may be relevant.

Some terms could use further disambiguation. For instance - 'lance armstrong' brings up lance, arms (but not arm) and armstrong (which is apparently an airport in Louisiana) but not Lance Armstrong. Cobra brings up the snake, the Cabinet Office Briefing Room but not the insurance plan.

Still, it is pretty cool.

Nice work!

This would be very helpful for writers, although it can also be used to target ads.

Nice effort here...this is a really tough problem to solve "correctly".

if you want to do this sort of entity recognition, there is Apache Stanbol which is quite useful: http://stanbol.apache.org/

how does the NLP work for this?