Comments on: Using your laptop to compute PageRank for millions of webpages

By: MapReduce: What are some good class projects for Machine Learning using MapReduce? - Quora

Fri, 29 Apr 2011 10:48:13 +0000

[...] also see OpenCV catalog: http://opencv.willowgarage.com/w... 9) PageRank, here is a good tutorial: http://michaelnielsen.org/blog/u…10) For a wealth of open ended problems see Programming Challenges: What are some good "toy [...]

By: Quora

Quora — Thu, 24 Feb 2011 09:34:20 +0000

What are some good resources for learning about matrix computations?…

This list is a stub. You can help by expanding it. * BellKor, Matrix factorization for recommender systems: www2.research.att.com/~volinsky/papers/ieeecomputer.pdf * BellKor, Scalable Collaborative Filtering..: public.research.att.com/~volinsky/netflix…

By: Data Scientists and Artists are the new upper class – and will be in the future « The Apps Lifestyle

Sun, 31 Oct 2010 20:38:17 +0000

[...] Linear algebra: a grounding in linear algebra is common among many data scientists, and important because matrix math underpins many data mining applications, such as the famous PageRank. [...]

By: links for 2010-05-27 « Blarney Fellow

links for 2010-05-27 « Blarney Fellow — Fri, 28 May 2010 01:34:46 +0000

[...] Michael Nielsen » Using your laptop to compute PageRank for millions of webpages (tags: pagerank google algorithm ranking recomendation) [...]

By: Michael Nielsen » Using your laptop to compute PageRank for millions of webpages : Popular Links : eConsultant

Thu, 27 May 2010 08:35:56 +0000

By: Michael Nielsen

Michael Nielsen — Wed, 30 Sep 2009 21:06:03 +0000

Daniel – Thanks for the note. When preparing the notes I considered doing something similar, but decided the presentation would be a little easier to understand without it. It’s been a while, but as I recall, my line of thought was simply that I wanted to avoid numpy, probably because the notes were originally prepared for people not at all familiar with numpy. The purpose of the notes is, of course, pedagogical – production code would be vastly different.

By: Daniel Smilkov

Daniel Smilkov — Wed, 30 Sep 2009 18:55:29 +0000

You know,

instead of using column matrices, you can use n-dim arrays:
v = numpy.matrix(numpy.zeros((n,1)))
becomes
v = numpy.zeros(n,dtype=numpy.float32)

and
p = numpy.matrix(numpy.ones((n,1)))/n
becomes
p = numpy.ones(n,dtype=numpy.float32)/n

and putting the initialization of the v vector out of step function, initializing it only ones in pagerank function,

you can get a lot of speed-up.

By: Michael Nielsen

Michael Nielsen — Sun, 25 Jan 2009 19:10:40 +0000

Thanks for that, Henry! I’ve reshared your comment to the FriendFeed room. It definitely suggests I should use sparse matrices if possible in my distributed implementation.

By: Henry Haselgrove

Henry Haselgrove — Sun, 25 Jan 2009 09:06:07 +0000

For those of you who like Matlab, Matlab’s inbuilt sparse matrix type appears to be a fairly efficient way to compute pagerank. For example, this Matlab function that I wrote computes a 2 million page pagerank in about 45 seconds: http://users.on.net/~henry/pagerank1.m

Apparently the “scipy” package provides sparse matrix types in python. I’ll give it a go sometime to see whether that is just as fast.

By: What is Authority? | Fidicaro.net

What is Authority? | Fidicaro.net — Tue, 30 Dec 2008 08:22:32 +0000

[...] do believe there is room in social media for authority. I believe Google’s pagerank algorithm may be a great start. A Twitter application that monitors retweets, recency of tweets, frequency of [...]