The Fugue State

enlightenment is a concept
by which we can measure our pain.

The dream is over
What can I say?
The dream is over
Yesterday
I was the Dreamweaver
But now I’m reborn

Today I was reading freethinking blog samizdata, and got to thinking. A lot of people have trouble accepting the work of peer review, especially when the results of peer review don’t ‘jive’ with their political stripe, or perhaps when they suspect that there is a significant bias involved against their theory of choice.

My idea to try to bring some clarity into the real bias would be for journals with peer review in place should post not only the articles that made it through, but each article/edit of an article which did not.  Not in the same place, of course–but there should be a tool which generates a random paper, and you, the reader are supposed to act as a judge, as to whether or not you would accept the paper in the journal if it were up to you.  Not whether you accept the article as right, but whether you accept it as reasonable enough to make its way into the journal.

The next step would be, to allow you, as the general public to try to rate a good deal many of these papers, and then compare your results with the results of the actual body of ‘peers’ which does the ‘official’ reviewing.  Comment threads could be allowed to provide places for differences / errors to be discussed.  This would allow a Bayesian individual to gauge to some extent how rational the peers of that journal are, and in turn, how much respect to give that journal.

Of course, one could still be wrong in assuming what is and is not rational, however being given the access to the inner workings of a peer review journal would give you more reason to trust/not trust a journal than exists today.  Climate change skeptics, for example, might find that the science involved really is well thought through, or at the very least beyond their ability to understand and that they should perhaps either trust the IPCC more or trust their perhaps uninformed views less.  Non-climate change skeptics could also use the service to ’sanity check’ their beliefs in the IPCC’s results.

The best part about such a system is that the hard part–gathering the data–is already done.  Archiving the data might be a little bit more costly than the current approach(toss the crap, only publish the good stuff), but even a 1% acceptance rate by a journal would leave only two orders of magnitude; ie, hard drives are advancing in capacity such that this would only be a big deal for a few years before the drive capacity made the archival a non-issue again.  The engine for rating / creating Bayesian results wouldn’t be too difficult to write, and would look a lot like existing tools such as the Open Source engine running Reddit.

Open Access journals and journal aggregators such as arxiv are the first step of a reformation of how science is done, all levels including social and technical ones, that computer technology has started.

Thoughts?

Picture this. You’re a University Student. Like me, when I took Macroeconomics, you’re taking a summer course, with online tests through WebCT. After a long walk home, you sit in your computer chair, and go through your books one last time; you have a quiz tonight.

You log in to your computer, and log on to the class website to do an online quiz for your class. You nod to your webcam; the webcam that was put there by your professor. The professor isn’t necessarily watching you, but your webcam tracks your every movement, so that when the quiz begins, you’re not caught looking at your textbook across the room, getting up to go the bathroom(and hence, obviously cheating, much like if you got up to go to the bathroom during a regular exam). There’s a microphone on your computer that’s listening to everything you say, and every rustle of paper. If it picks up any sound, it alerts your professor to pay attention to you—to make sure again that you’re not cheating.

But since this is mandatory, and since every class has this setup, there’s more—You accidentally think back to what lays across the room and remember you have a copy of 2600(a dissident magazine, the possession of which is liable to get you convicted on TERRORISM charges) laying in the open. You panic…you had gotten stoned last night and hung out with your secret hacker friends and had forgotten to put it away. You can’t look at it…the camera tracks your eye movement—you have to stay focused on the screen, focused on the test.

The camera detects something funny about your body movement, and the professor is alerted. She notices the magazine, and calls the cops. Before the test is done, the SWAT team kicks down your door, shooting your dog, and before you know what happened you’re unconscious in a black bag heading for a secret prison in cuba.

Sounds far fetched? Not really.

Check This out (at 18:15). It’s an Off the Hook episode which deals with a company selling the 1984-esque Mandatory Webcams. This is something you’ll probably have to see for yourself.

I have taken WebCT class, and I could probably have cheated, but didn’t. There was no point–WebCT tests are far easier than their paper counterparts, at least the ones I took. Actually in retrospect, it now makes more sense why the class average was so high.

I’ve got a webcam that I voluntarily put on my desk. It doesn’t work half the time, but still—-I’m probably on the ‘privacy is dead/useless’ side of most things, but even so, here in this case I don’t think it should be mandatory in order to go to university to subject yourself to this massive breach of privacy. Pretty much everything else in the above story has happened to someone, somewhere, there are no missing pieces of the puzzle anymore — if this system becomes commonly used(or worse, made mandatory by the government), it really will be the end of free society.