How the backpropagation algorithm works

Chapter 2 of my free online book about “Neural Networks and Deep Learning” is now available. The chapter is an in-depth explanation of the backpropagation algorithm. Backpropagation is the workhorse of learning in neural networks, and a key component in modern deep learning systems. Enjoy!

How the Bitcoin protocol actually works

Many thousands of articles have been written purporting to explain Bitcoin, the online, peer-to-peer currency. Most of those articles give a hand-wavy account of the underlying cryptographic protocol, omitting many details. Even those articles which delve deeper often gloss over crucial points. My aim in this post is to explain the major ideas behind the… Continue reading How the Bitcoin protocol actually works

Neural Networks and Deep Learning: first chapter now live

I am delighted to announce that the first chapter of my book “Neural Networks and Deep Learning” is now freely available online here. The chapter explains the basic ideas behind neural networks, including how they learn. I show how powerful these ideas are by writing a short program which uses neural networks to solve a… Continue reading Neural Networks and Deep Learning: first chapter now live

How to crawl a quarter billion webpages in 40 hours

More precisely, I crawled 250,113,669 pages for just under 580 dollars in 39 hours and 25 minutes, using 20 Amazon EC2 machine instances. I carried out this project because (among several other reasons) I wanted to understand what resources are required to crawl a small but non-trivial fraction of the web. In this post I… Continue reading How to crawl a quarter billion webpages in 40 hours

Using evaluation to improve our question-answering system

It’s tempting to think IBM’s Jeopardy-playing machine Watson must have relied on some huge algorithmic advance or silver bullet idea in order to beat top human Jeopardy players. But the researchers behind Watson have written a very interesting paper about how Watson works, and a different picture emerges. It’s not that they found any super-algorithm… Continue reading Using evaluation to improve our question-answering system

Lisp as the Maxwell’s equations of software

On my first day of physics graduate school, the professor in my class on electromagnetism began by stepping to the board, and wordlessly writing four equations: He stepped back, turned around, and said something like [1]: “These are Maxwell’s equations. Just four compact equations. With a little work it’s easy to understand the basic elements… Continue reading Lisp as the Maxwell’s equations of software

How changing the structure of the web changes PageRank

Suppose I add a hyperlink from a webpage to a webpage . In principle, adding that single hyperlink changes the PageRank not just of those two pages, but potentially of nearly every other page on the web. For instance, if the CNN homepage http://cnn.com adds a link to the homepage of my site, https://michaelnielsen.org, then… Continue reading How changing the structure of the web changes PageRank