Comments on: Write your first MapReduce program in 20 minutes http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/ Fri, 10 Feb 2012 10:56:26 +0000 hourly 1 http://wordpress.org/?v=3.3 By: Noticeable Hacker news to read at 2011/06 | David Euler on programming, design and linux http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-30839 Noticeable Hacker news to read at 2011/06 | David Euler on programming, design and linux Fri, 01 Jul 2011 13:47:06 +0000 http://michaelnielsen.org/blog/?p=529#comment-30839 [...] http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/ [...] [...] http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/ [...]

]]> By: milosh http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-30838 milosh Fri, 01 Jul 2011 10:05:11 +0000 http://michaelnielsen.org/blog/?p=529#comment-30838 In JK's map(partial(os.path.join, "text"), flist), is partial better than simply using a lambda? In JK’s map(partial(os.path.join, “text”), flist), is partial better than simply using a lambda?

]]> By: 10001 件資工系畢業前一定要做的事 » Mr. Jamie 看網路與創投 http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-30825 10001 件資工系畢業前一定要做的事 » Mr. Jamie 看網路與創投 Mon, 27 Jun 2011 22:32:38 +0000 http://michaelnielsen.org/blog/?p=529#comment-30825 [...] 01111 用 map-reduce 分析資料 — 這是現在最最熱門的題目，你應該要試試 (從這裡開始) [...] [...] 01111 用 map-reduce 分析資料 — 這是現在最最熱門的題目，你應該要試試 (從這裡開始) [...]

]]> By: Write your first MapReduce program in 20 minutes | Panicked Zebra http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-30816 Write your first MapReduce program in 20 minutes | Panicked Zebra Sat, 25 Jun 2011 11:53:30 +0000 http://michaelnielsen.org/blog/?p=529#comment-30816 [...] sourced from original page at http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/, where Michael Nielsen [...] [...] sourced from original page at http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/, where Michael Nielsen [...]

]]> By: Bull … « Richard WM Jones http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-26752 Bull … « Richard WM Jones Tue, 27 Apr 2010 22:13:00 +0000 http://michaelnielsen.org/blog/?p=529#comment-26752 [...] I was taught MapReduce as a grad student in 1994 (10 years before the bollox above) by a number of people. And I’m quite sure it wasn’t a new idea, even [...] [...] I was taught MapReduce as a grad student in 1994 (10 years before the bollox above) by a number of people. And I’m quite sure it wasn’t a new idea, even [...]

]]> By: Jonathan Stray » Why We Need Open Search, and How to Make Money Doing It http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-26173 Jonathan Stray » Why We Need Open Search, and How to Make Money Doing It Sun, 27 Sep 2009 09:51:16 +0000 http://michaelnielsen.org/blog/?p=529#comment-26173 [...] data processing is now well understood. To be precise, I want an open search company that sells map-reduce access to their index. Map-reduce is a standard framework for breaking down large computational [...] [...] data processing is now well understood. To be precise, I want an open search company that sells map-reduce access to their index. Map-reduce is a standard framework for breaking down large computational [...]

]]> By: Michael Nielsen » Consistent hashing http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-23140 Michael Nielsen » Consistent hashing Thu, 04 Jun 2009 03:06:38 +0000 http://michaelnielsen.org/blog/?p=529#comment-23140 [...] interested in distributed dictionaries is because they’re used as input and output to the MapReduce framework for distributed computing. Of course, that’s not the only reason distributed [...] [...] interested in distributed dictionaries is because they’re used as input and output to the MapReduce framework for distributed computing. Of course, that’s not the only reason distributed [...]

]]> By: Michael Nielsen http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-20406 Michael Nielsen Wed, 15 Apr 2009 12:02:31 +0000 http://michaelnielsen.org/blog/?p=529#comment-20406 Asokan - Thanks, fixed. Asokan – Thanks, fixed.

]]> By: Asokan Pichai http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-20381 Asokan Pichai Wed, 15 Apr 2009 04:40:08 +0000 http://michaelnielsen.org/blog/?p=529#comment-20381 The Dave Spencer link is wrong. AFAICS, it says tropo instead of chencer The Dave Spencer link is wrong. AFAICS, it says tropo instead of chencer

]]> By: Neal Richter http://michaelnielsen.org/blog/write-your-first-mapreduce-program-in-20-minutes/comment-page-1/#comment-18123 Neal Richter Sun, 22 Feb 2009 03:17:13 +0000 http://michaelnielsen.org/blog/?p=529#comment-18123 Obviously Dean and Ghemawat are due some credit here, yet map and reduce have been around in languages like Lisp for approx 25 years prior to their paper. There were even companies/groups that parallelized their Lisp implementations in the 1980s: http://ieeexplore.ieee.org/iel3/1244/519/00010373.pdf?arnumber=10373 http://search.barnesandnoble.com/Parallel-LISP/Takatoshi-Ito/e/9780387527826 http://www.springerlink.com/content/k560u307713j57r4/ Did Dean and Ghemawat 'finish the job' and do a great implementation at exactly the right time and at the right company to start a distributed computing revolution? Absolutely and congrats to them for that. Yet when first introduced to it, it was completely obvious to me and others who learned the map and reduce lisp/scheme primitives as undergrads. And 'difference' in their methodology is trivial.. of course this does describe the best revolutions.. they are all trivial in retrospect. It's also worth reading David DeWitt's savaging of MapReduce to cure you of any further illusion of it being 'invented' by Dean and Ghemawat. (I think he goes way to far.. modern MapReduce is here to stay - we're not going back to RDBMs systems for distributed computing anytime soon) http://www.databasecolumn.com/2008/01/mapreduce-a-major-step-back.html Obviously Dean and Ghemawat are due some credit here, yet map and reduce have been around in languages like Lisp for approx 25 years prior to their paper. There were even companies/groups that parallelized their Lisp implementations in the 1980s:

http://ieeexplore.ieee.org/iel3/1244/519/00010373.pdf?arnumber=10373
http://search.barnesandnoble.com/Parallel-LISP/Takatoshi-Ito/e/9780387527826
http://www.springerlink.com/content/k560u307713j57r4/

Did Dean and Ghemawat ‘finish the job’ and do a great implementation at exactly the right time and at the right company to start a distributed computing revolution?
Absolutely and congrats to them for that.

Yet when first introduced to it, it was completely obvious to me and others who learned the map and reduce lisp/scheme primitives as undergrads. And ‘difference’ in their methodology is trivial.. of course this does describe the best revolutions.. they are all trivial in retrospect.

It’s also worth reading David DeWitt’s savaging of MapReduce to cure you of any further illusion of it being ‘invented’ by Dean and Ghemawat. (I think he goes way to far.. modern MapReduce is here to stay – we’re not going back to RDBMs systems for distributed computing anytime soon)
http://www.databasecolumn.com/2008/01/mapreduce-a-major-step-back.html

]]>