“Everything” and Wave

Outline: I talk a little about what is wave, then what I think of wave and last about “everything” which was my idea/prototype of what it should be.

So What is wave?
Apparently most of the people on the “blogosphere” have no clue. And currently the best source (other than the video, go watch that first) is at the 2 google groups. I’ll start with a brief explanation. Wave is actually 3 totally separate things.

1. A communication protocol
2. A very efficient client
3. A set of extensions to that client.

What most people miss is that wave is a protocol (apparently a set of extensions to XMPP), now be careful! this protocol is ONLY for server-server communications, which means this is how waves travel across the interwebs and has nothing to do with posting to the wave or reading them. The real advantage of this is that synchronization happens at a very fast level which means thin clients, With something they call “operational transformation” as this guy explains in this video. And the other really interesting thing is that it enables private servers with private waves, without creating walls.

Then we have the client which is what everyone is drooling about, which is simply a realllly cool editor and collector. It is build with GWT and apparently comet. This seems to implement the realtime typing, search, the playback and drag & drop goodness we saw. It is a remarkable webapp and I can’t wait for them to release it.

The third component which is the most interesting part for me to play with. Is actually 3 different things Robots, Embed, and Gadgets.

• The gadgets are nothing new, we all know how you can add a youtube video in a site that is not youtube and do stuff with it, same with a google map, etc. The only innovation here is that the gadget can access the data from the wave.
• The Embed api is pretty much a “viral tool”, the goal is to take waves to non-wave environments. Which gets you several bridges to pretty much everything out there. I call this viral because it is what is going to make this a huge thing. After someone sees that my “blog’s comments” are in this super cool system, they will track that system and say ohhhh wave is cool!
• The part I’m most exited about is the robots, simply because they are first class citizens. And that means that anyone with a little knowledge of how the system works is going to be able to create automatization, and this is so powerful that it is scary. In the good uses of this I can see (and will write) 100s of really useful tools. Just think about it for a second, the robot will know everything the people will know about the wave. Now the scary part is when someone takes this freedom and applies it to evil.

That pretty much covers it, how is this different from anything else out there. It’s goal is to unify not subdivide the fact that they are releasing the protocol means you will be able to run your own wave server, which means private really private waves. This is totally useless for “where are we going to eat?” but it’s very important for “We are broke and only have 3 million dollars left on the bank account, what shall we do?”. The other really cool thing is that they are going to open source everything (let’s hope that’s true) which means the client will solve the “how to build a real time system” problem, and developers can focus on their tools.

But this is not a me-too post. In fact someone recommended me not to post this but I have to. I have been thinking of a system like this for a long time, seriously I have chatlogs, trac tickets, svn repositories (yes that long!), a prototype on appengine, another in mercurial and another as a TG2 app with first had a zodb backend and recently a mongodb. I also have some work on the frontend but I wasn’t thinking real time. So all in all it has to be at least 5 years of working around this idea. This is also on the planned roadmap for “codemill” and “DVDev” (a totally different post). And my last prototype of this idea is named “everything”, hence the title of this post. I though it was a little cocky at first but I think I was right, wave is indeed about everything.

Of course as the famous quote “There is someone smarter than you outside of your organization”, I have not been able to lift off my idea as I’m pretty much a one man team with no resources, and I’m not smarter than the wave team. Which means I’m going to be an early adopter here and build “everything” on top of wave.

Analysis of Wave

What they got really right.

• Robots as first class citizens
  • This means they realize the potential behind having computer helpers on your day to day tasks. With the correct set of bots you should be able to enhance your writing to scary levels of enlightenment.
• One instance of the wave
  • I think this the most important part of the system (but I put robots first because I’m so exited about them!), the web today is too fragmented, just as an example try to search for the first public non-official announcement of wave. This is practically impossible because the web is a huge copy&paste fiasco. The wave format is something to unify all the systems under the same data structure but it keeps them all independent.
• Server-Server replication
  • The beauty of this is that all the wave servers take care of the heavy lifting which means the clients are very simple (in coding). This is probably going to be a key feature for the adoption of the system. Most existing clients (mail,IM,GUIs,webpages) and forms of presenting the data (websites,phones,native clients,etc) simply need to learn how to read and write from the wave format.
• Plugable as hell
  • you can write your own client, your own bits (gadgets), your own functions (robots) and your own server
• one data structure to rule them all
  • IF wave-protocol gets to be the success I hope it will be, this is probably as big as XML. Before XML each system/company/app/team used their own little format for moving data across applications (most binary, but a lot text). So back then you will have to code something to talk to each one of the applications you wanted to talk to. Why XML was so big? It lifted the barriers on the data. How wave-protocol is like XML, it’s going to lift barriers on the data.
• It’s a messaging system
  • Which coincidentally is how people communicate. All the old messaging system share one fundamental flaw they are synchronous you can measure when a message starts and when it ends. In wave the message is one character long which is as close to speaking as we can get, in fact it’s better because it’s not noisy
• A solution to the “where”
  • today you need to answer where shall we discuss this, should I call you, email, write to the mailing list, IM, twitter, bah! wave’s answers: “everywhere”

What they got almost right

• XML, to be honest they have a very good and clean xml file. But why not json or even yaml? Although this post is enlightening.  I honestly think they didn’t choose XML, XMPP did that for them. Maybe json is a better storage and transfer system but they will probably make them have to sit and re-write XMPP and I’ll be writting this post in 2012 or later.
• One true editor, I’m not sold to the idea of having one editor. I really think we’ll need several specialized editors for the different content we’ll like to create on waves.
• Appengine only bots. I assume this is either because of
  • a) to be able to kill an offensive bot
  • b) to protect wave with the sandbox (you can’t compile a worn on appengine)
  • c) to give appengine popularity.

  In any case I hope this is lifted at some point, or I’ll just use a comunity server

The biggest challenges

• spam: I think this is the most important aspect of the system. will it die like usenet? could the protocol be spam-proof? how can you prevent spam without coercing the users?
• robots: leaving spammy out. Where is the line between helper robots and annoying robots, how can you trust a robot? how will you identify the evil robots?
• uncooperative behavior: this is similar to the robots problem, and in fact it’s tied to it. How do you deal with people that where added to the wave and are against it’s usefulness.
• authorization: Who should be allowed to edit the wave? some waves need to have ACL, should this be at the protocol level?
• some readonly parts: how and who should declare what is “set in stone”?
• massive forking/endless hours finding the important waves: if we give too much power to the owner of the wave then we risk fragmenting the conversation and having 100s of “missed update” where you could be useful. If we give too little power caos will take over.
• ownership: who owns the content? who owns the wave? up to which point my original wave is really mine?

The future of “Everything”

So as I said above I am/was writing “everything”. And I think it is still a viable project. I’m just going to retarget it to be wave-compliant.

how is “everything” different from wave?

• specialized editors: I am really thinking of having editors for each type of data. For example a ticket editor, code review editor or the more traditional rest,html,plain editors. Since this is pure javascript I’ll probably retrofit them to be gadgets ones the client is release, or they could evolved into their own client.
• Per server robots: To be honest I’m a bit scared about the robots in the wave, I think the wave team went too far with robots and I’m really interested in knowing how they plan to keep them at bay. In my idea the robots where local and I was thinking of them as helpers rather than contributors, in wave terms my robots where limited to the blip, and I think this is a good idea for your own set of minions you don’t want other people to use. I’m going to push this idea forward and I’ll code them in wave terms so they are compatible and most likely useable as google’s wave-client bots. From the source it seems you can build a standalone robot server with what they released.
• publishing system. I originally though of “everything” as a publishing system. But in the form of editorial reviews rather than wiki-style. My idea was to have one true editor to a website with the means for a proper feedback. And I’ll push this to happen on the wave. I really think we need a two tier system for some waves, where you have a set of authors and a massive set of feedback. In the simplest way this will be used to write a blog. In a more complex case it will be a mailing list to discuss a project/product/idea. In a even more complex case it will serve to write a paper or a book. And in the most complex case it will serve to record your live and thoughts.

So that’s about it. Sorry for the huge ass wave but I think this is a topic that deserves a lot of thought I’ll leave you with what I think is the most inspiring message currently at any of the google groups regarding wave.