Progress v7: So old is good... or is it?

[OK, it has been two weeks again, and this is my seventh article on progress. I have been a little busy, and the progress stuff is really starting to get complicated at this point. But you can always start with my first article ;-]

So in the last episode we have seen that standards are good for you. In fact, the only reason why our technology works, for example, why you are able to read this, is because standards exist.

This is also how science is made: using common standards (of measurement), one theory can be based on another. So the new researchers don't have to go and re-invent all the theories before they finally can start producing original work¹.

So old is good, right? We can build on a foundation happily ever after?

Except, of course, when it is flawed. Then you can either stop when you recognize that your house is going to fall, or build all the way up until it falls apart. The same good practice that allows us to finally build high also allows us to build on errors of others.

Take the science of Astronomy for example. For hundreds of years it could provide us with no insights more useful then astrology, because it was based on the flawed premise -- the Ptolemaic Universe. It was only after Copernicus and Galilei that we started to be able to look into the actual workings of the universe, because they found the fundamental flaw in the theory of the world to that point: the Earth was not the center of the universe.

If not for the Copernicus, would Galileo have ended up observing the laws of motion? Would Newton have formulated the theory of gravitation based on that?

On the other hand... if not for Aristotle and Ptolemy, would somebody have discovered the laws two thousand years earlier?

We'll never know of course. What we do know, is, that about 2250 years ago, a man called Aristachus of Samos had already developed a very modern view of the solar system. It's just, the geocentric model of Aristotle was viewed as a much simpler explanation for why there is no parallax -- why stars don't visibly "wobble" when the earth move around the sun. In Aristotle's model the earth simply didn't move -- and the calculations for the positions of the planets produced results with accurate enough to look right. And thou shalt never change a running system...

This is one example that shows that with too little information, Occam's Razor can be applied both ways.

It also shows, that based on the same data, you can develop one perfectly working model that is also perfectly useless as anything but a working model (some of the modern unification theories come to mind), and another working model from which you can infer additional information about the subject -- extended use, so to say.²

And also, often enough, you don't see which is which until waaaay later.

So, competing theories, competing libraries, competing standards, are good... if and only if there is suspicion that the current system they are competing with is really inadequate and it hampers development.

But the thing is... Even if software people finally agree on standards, even if people actually cooperate instead of developing against each other, even if we all build on the same foundation... we are still shooting at a moving target.

How so? Well... see you in the next episode of the progress wars!

¹:This is why, when somebody says, "I am just as entitled to an opinion about meteorology as scientist X" he is talking BS -- unless, of course, he actually spent as much time studying the theories behind meteorology as scientist X, and has no bias.
A bias might be his conviction, or it might be his boss telling him that whatever the logical answer would be, the answer that will keep his job safe, is the one that his boss wants to hear. Unfortunately, the laws of physics are not political, only the people studying them are. But this is a different topic.

²: Like, for example, an open-source software library: there, I can learn from something other people already invented and thought through before me. If I see enough of that, I can also pick the components that I think are right, and discard the components that I think are wrong. In other words, I can learn.
With closet, err, closed-source code, this is impossible. You can only build on top of it, and hope that the developer really knew what he was doing and made no mistakes that will affect what you are doing.
Kind of like building on ground where you can't check whether it actually is a swamp. In fact, it is illegal for you to check whether it is a swamp, and you have to build on that ground because "everybody else does it". And where the holy creator of the ground thought that investing 20% of the necessary work is good enough.

Charity for the Rich (a tangent)

Let's leave progress aside, for a moment, and look at a very strange development:
Year after year, developers all over the world donate their expensive time to rich corporations like Microsoft or Linden Labs for free... and for no added value to themselves or their products. Not only that, but the time and money also just get thrown away by the corporations they donate it to! Why? Let's see.

Let's take Microsoft as the first example.

Every time you design a web site, you can do it one of two ways: standard-compliant, or Microsoft. These two are mutually exclusive, since Microsoft decided to take the otherwise standard tags, CSS rules etc, and render them differently from everybody else. There are scores of websites dedicated to how to work around "misapplications of standards" (which are, in my opinions, bugs and not a separate class of problem.)

Every web designer who wants his website to be viewed both by IE users, and by everybody else, basically has to develop his web site twice, or at least spend a lot of time learning the incompatibilities of IE to other browsers, and IE7 to IE6 to IE5 and so on, and ways to make his web site work with every one of those more-or-less properly -- since all of those browsers have different sets of problems. Most people I know who stopped doing web design, have stopped for this exact reason.

Why does this problem exist? Because Microsoft considers it more important to fix some bugs then others (as can be read frequently on Chris Wilson's blog), completely disregarding the fact that fixing some bugs will either make IE-recognizing code fail, or the browser will be recognized as IE but will not render the web page as expected, resulting in the necessity to add yet another version of the same web page, to cater for yet another partially correct version of IE. (See also, IE8 announced. Opt in? Does this mean, I have to go back to all my pages and add new code that will cause IE8 to render it the same as Firefox already does it?)

To sum it up: because Microsoft doesn't want to invest money in standard compliance, every single web developer has to invest money out of his own pocket to fix Microsoft's problems.

WOW! This is a perfect example of capitalism: extracting surplus value from somebody who isn't even your employee!

Except... the amount of money wasted worldwide in this operation is -- even by a conservative estimate -- hundreds of times larger than the amount of money Microsoft would have to invest to fix the problem. (Just compare the number of website developers worldwide that have to cope with the problem to the amount of developers working on IE.)

And with this, we arrive at Linden Labs.

There has been a lot of attention to Second Life, the Linden Labs' 3D environment, lately. As I noted earlier on a different blog (I will re-post it here eventually), Second Life is neither unique, nor technologically more advanced than similar solutions (e.g. There.com). What makes them unique is that creating new content is really cheap: the environment has a (crappy) built-in object editor, and uploading new textures costs only L$10 -- about 3 cents. They also allow you to sell stuff, and earn L$, which can be easily converted into real (US) Dollars, at a price.

Many a developer starts, starry-eyed, to develop new content for the system, thinking of what great things you could potentially do in a 3D environment. Some of them do manage to create decent things (and some of them, even, to sell them), before they ultimately collide with the technological barriers of the Linden Scripting Language and Second Life physics. Which, to put it gently, suck ocean liners through cocktail straws. For example, until March 2007 you could not create animals that actually moved their legs, without writing a distributed swarm-like system of scripts -- one for every joint. And if you actually manage to write something that doesn't collapse under the weight of its own code, you would discover that the physics system has bugs that prevent the "animal" from working properly anyway.

But I digress.

The really interesting fact about Second Life is that most of the content sucks is created by it's users, whereas most of the money is made by Linden Labs, since you pay them

• every time you upload anything
• every time you change dollars into L$
• every time you change L$ into dollars
• every month -- if you "own" a piece of virtual property
  

But not only that -- the content of the 3D world is, in large part, what attracts the users to the world in the first place. So, by making new content for Second Life, you make their world more attractive for new users -- who come, and pay their money to Linden Labs. And LL didn't even have to pay anybody for it. Linden Labs, too, is extracting surplus value from people who aren't even Linden Labs employees.

In fact, now that they open-sourced the Second Life client, they don't even have to maintain their 3D client themselves, as hundreds of enthusiasts are fixing their code for them too -- donating their time to a company that is bathing in money. Money enough, in fact, so that they can just buy additional servers, and not worry about the fact that the server code is as inefficient as it is... But that is a subject of a different discussion.

And you know what? They waste most of the users' efforts, too.

Almost every Wednesday, the entire Second Life network goes down for maintenance. When it comes back, often enough, many objects that worked perfectly before, don't any more. Because, for example, a five-year-old bug that everybody had to work around, and which had been incorporated into every moving object as a feature, got "fixed". Which means, just as with every version of IE (which, luckily for us, doesn't come around quite that often), every programmer has to check every script after every update. Not many do, which means that a significant percentage of scripts, objects and weapons in Second Life are broken at any given time.

The interesting difference between Linden Labs and Microsoft is: Microsoft is just a wannabe monopolist of the Web, while Linden Labs has absolute control over Second Life. Linden Scripting Language is their own invention, used nowhere but in Second Life -- so if they choose to say "everybody uses this, we should not fix it", nobody would complain, because unlike the case of Microsoft and the web standards, it would be a fact, with "everybody" actually being 100% of Second Life developers who use the given function, since there is no alternative.

So, what do we learn from these examples?

If we donate our time and money to corporations that don't actually need it, they will not say "thank you" -- they will, in fact, not even think about it, respect it, or pay any attention to it.

So if you are a developer... just don't do it. If you want to donate your time and your knowledge to something, choose an Open Source project. At least in this case, the user community will profit, instead of it being a complete waste of your time.

Progress v6: is the age of standards the worst enemy of Progress?

[This is my sixth article on Progress. I'm sorry for the long period of silence on that matter, the real life had caught up with me, but there is also good news: this is not, as planned, the last article on progress. There is at least one more, or more likely, two more, to come.]

In the previous five articles I made the impression, that contrary to the public opinion, to achieve actual progress, older is better. And in the cases I mentioned, it is. But let's look at an extreme example of this, the by now fairly well known story that "space shuttle engine's width is two horse asses plus X".

While the latest conclusion is, that, indeed, this is an urban legend, and the space shuttle might not have any relation to a horse's ass whatsoever, the basic story (bar the newer, Usenet-based addition with the space shuttle), that the width of the railroad track is based on the dimensions of two horse rears, same as the chariot width in the old Rome is affirmed in the same article:

[...]the dimension common to both was that of a cart axle pulled by two horses in harness (about 1.4m or 4ft 8in). This determined both the Roman gauge and Stephenson's, which derived from the horsedrawn wagon ways of South Northumberland and County Durham coalfields.[...]
[from: Crow, James. Housesteads. London: B.T. Batsford, 1995. ISBN 0-7134-6085-7 (pp. 33-34).]

The same article also says:

At the time of the Civil War, even though nearly all of the Confederacy's railroad equipment had come from the North or from Britain (of the 470 locomotives built in the U.S. in 1860, for example, only 19 were manufactured in the South), 113 different railroad companies in the Confederacy operated on three different gauges of track. This lack of standardization was, as historian James McPherson points out, one of the many reasons the Union was able to finally vanquish the Confederacy militarily:

The Confederate government was never able to coax the fragmented, run-down, multi-gauged network of southern railroads into the same degree of efficiency exhibited by northern roads. This contrast illustrated another dimension of Union logistical superiority that helped the North eventually to prevail.
[from: McPherson, James. Battle Cry of Freedom. New York: Oxford Univ. Press, 1988. ISBN 0-19-503863-0 (pp. 318-319, 514-515)]

So what does this tell us? First of all, that the conclusion that

In other words, there was nothing inevitable about a railroad gauge supposedly traceable to the size of wheel ruts in Imperial Rome. Had the Civil War taken a different course, the eventual standard railroad gauge used throughout North America might well have been different than the current one.

is both somewhat true,... and, to a degree, a circular argument: "The war was won partly because of the railroad gauge, but had the war taken a different course, the railroad gauge would have been different." (If the outcome of the war wouldn't have depended on the railroad gauge, that is?)

The second, and most important, conclusion is, in a collaborative environment, it is better to use well-established standards then not to use them: having a standard-sized railroad might have helped the South to win the war, by allowing the trains to pass through an unified railroad system instead of having to unload and re-load trains (or change the gauge on the wagons, as they do it when trains transit from e.g. Russia into western Europe and vice-versa).

And in modern times, there is even more to say for keeping of the standard: even though the width of the railroad track might not be entirely optimal for modern trains, there is an enormous amount of existing equipment based on the standard, which would ALL have to be changed if somebody decided to switch standards.

Or in other words, there is no reason not to use a standard just because it is 2000 years old, but there is a lot of reason to re-use existing equipment. And it doesn't preclude us in the least from making progress, (for example, in this particular case, from developing high-speed trains like ICE3 and TGV)

Which, incidentally, is what I have been saying for the last five articles. So no, the age of standards doesn't impede progress to any significant degree, if the standards make sense. This is an important distinction. The history of science would be much more straightforward if it wasn't. How so?... Come back next week(or next month?), and see for yourself, in... Progress wars! ;-)