Comments on: Open source software at centralized servers?

By: Michael Adams

Michael Adams — Tue, 09 Oct 2007 20:34:30 +0000

I didn’t mean to say WordPress or WordPress.com was feeling lonely I was just curious about what you were looking for (and, in the process, trying to glean some information about your project).

If you’d like to chat about some of the non-WordPress but still open source tools we use, I’d be happy to discuss it or refer you to others that know more about it any time.

Oh, and I’m technically a graduate student at IQI now (entering class of 2003), though I’m on “sabbatical” working for Automattic.

By: Michael Nielsen

Michael Nielsen — Tue, 09 Oct 2007 18:42:16 +0000

Michael:

My original question wasn’t very clearly phrased, and I haven’t cleared it up sufficiently in the comments.

Let me start over, and explain a bit more context. What I’m really interested in is products running on large servers (let us say > 100 machines for the sake of definiteness) where there is a corresponding very large centralized database.

Something like Second Life or Wikipedia fits the bill. WordPress.com isn’t quite what I had in mind. There’s certainly no sense in which I’m trying to “exclude” WordPress from some club. It’s just that some of the tools I’m interested in developing seem quite close in requirements to Second Life or Wikipedia, and I’m wondering if open source is a viable development process. The more examples of such projects I have to look at, the better, from my point of view, which is why I asked the question.

Incidentally, I think WordPress.com is a great site, and WordPress is one of my favourite products.

When were you at IQI?

By: Michael Adams

Michael Adams — Tue, 09 Oct 2007 18:17:52 +0000

Define “large” then. As best I can tell, Wikipedia is running on something like 350 servers, WordPress.com on about 290 and Second Life on not yet 10,000.

Of course, what those boxes are makes a big difference and I don’t have any info on that.

I don’t think WordPress.com is clearly a non-example, but I certainly won’t be put out if it is.

For the sake of full disclosure, I work for Automattic on WordPress.com (though my roots are in Caltech’s IQI), so my thoughts, be they compatible with your question or not, are from first hand experience.

By: Marcus

Marcus — Tue, 09 Oct 2007 03:33:24 +0000

Well, there is the Mugshot project (http://www.mugshot.org). Not quite like Facebook or Orkut in terms of popularity, but it is open source and centralized. The team (from RedHat) plans on building on this idea and eventually releasing a sort of open source internet based data service, like .mac, but for linux/GNOME.

There is also planetmath.org, and related sites. Wikipedia can also be considered open source.

The reason why you don’t seem so many centralized server-based open source projects, in my opinion, is because the general approach of the open source community is to de-centralize everything, and make things interoperable so that they can be deployed anywhere and not be tied down to one company. One example of this is Jabber, on which Google talk was built. The tendency is to develop software, not provide services. The Mugshot team (especially Havoc Penington) are advocating open source services, but even then I doubt they will be so centralized.

By: Michael Nielsen

Michael Nielsen — Sun, 07 Oct 2007 20:28:43 +0000

Maverick: Thanks for the pointer. I’d never heard of PlanetLab before. It looks quite interesting, and I’ll have to look into it more.

By: Maverick

Maverick — Sun, 07 Oct 2007 16:05:20 +0000

@11 Michael: BTW, you may also be interested in (at least) the organization structure of PlanetLab. It’s a consortium and institutes join so that their members can use the infrastructure.

By: Michael Nielsen

Michael Nielsen — Sat, 06 Oct 2007 19:06:57 +0000

Matt: For small projects, the drupal / WP model would be fine. However, for larger projects existing University infrastructure would not be sufficient (especially in poorer countries). Systems like Amazon S3 / EC2 beat University networks comprehehsively on reliability, security, scalability, and price, which is a hard combination to resist.

By: Matt

Matt — Sat, 06 Oct 2007 17:18:29 +0000

I think the WordPress/drupal/Apache model is probably better for scientific collaboration software. Universities already have a network infrastructure, which you would probably want to make use of. Also, you could run into intellectual property and data protection issues depending on the type of project the software is being used to manage. Universities will probably want to be in charge of the security of their own data.

Having each university run their own server for the software, but communicating with other servers to share data between collaborators is what I would suggest. Of course, this would require persuading several universities to support the project from the word go, which would be a challenge.

By: Michael Nielsen

Michael Nielsen — Sat, 06 Oct 2007 00:10:22 +0000

Maverick: Yes, I’m talking about instances where large numbers of machines are involved.

By: Maverick

Maverick — Sat, 06 Oct 2007 00:05:31 +0000

@8 Michael: it really depends on the load. I assume by “large”, you are thinking the scenario when a single fairly powerful server is not sufficient? In that case, there are many load-balancing solutions, some of which are open-source. For the frontend, Apache reverse proxy or Squid may be used. For the backend database, I am not familiar with the open-source solutions.