Numbered IMs

The Problem

All Instant Messaging (IM) services sometimes silently drop IMs, especially when one party has a flaky internet connection, gets disconnected, has a system crash, switches to a VPN, or has two different chat clients open. Messages are also occasionally dropped by the IM service provider itself. However it happens -- and it happens more than you realize -- it leads to subtle, insidious communication failure. At best, the "what's the last IM you got from me?" ritual after one party gets disconnected is annoying.

A Solution

This can be completely solved at the client level, just as two sufficiently fastidious people could confirm every single IM ("roger your 'brb', over").

Fortunately, there's a simple -- though not quite transparent -- solution that can be implemented in a chat client, without a tricky confirmation protocol, and in a way perfectly compatible with clients that don't play along. So there's no chicken-and-egg problem and no buy-in needed from the IM service providers (Yahoo, Google, AOL, MSN, etc).

Just have a checkbox in the IM client settings:

"Number my outgoing messages (and check that incoming numbered messages are consecutive)."

If you check it then it simply prepends consecutive integers to every outgoing message and also displays a warning when incoming messages, if numbered, are nonconsecutive. (Our current implementation also grays out the numbers so they're more unobtrusive, though such formatting is not always respected.)

Imagine it:

B: 1) hi alice

A: hey bee

B: 2) what are you doing?

A: um, wondering why you're numbering your IMs...

B: 3) oh, it's a setting in my chat client that adds those.

A: why?

B: 4) so that when [yahoo/AIM/google] drops an IM (which happens all the fargging time but usually no one realizes it happened) you'll know.

A: 1) holy crap that's genius, I just turned it on too.

B: 6) well, ttyl

[WARNING! message 4 followed by message 6]

A: 2) k, bye. hey, wait! it's saying your message 5 is missing! (I thought you sounded a bit abrupt there but didn't want to say anything)

B: 7) oh, it was: btw, let me know if you want these tickets to Monster Truck Smackdown Hosted by Alistair Cooke cuz i can't go.

A: 3) OMGWTFBBQ!!1!! i'll be right over.

Is this really a problem?

If your internet connection's perfectly good and you don't believe [Google/Yahoo/MSN/AIM] drops messages, you might not think this is worth bothering with. How wrong you are! Although the probability of any given message being dropped is minute, there is no guarantee of delivery built in to the protocol and so, if you have long IM conversations, it's only a matter of time before a message is silently lost. But the real problem is how rarely it's noticed. It happens all the time that in rapid conversation, some questions are ignored. How often do you say "hey, you never answered such-and-such"? It's just not part of the conversational protocol. That's because in face-to-face or phone communication, which IM emulates, there's no such thing as utterances that you don't even realize you didn't hear. In rapid-fire conversation it's easy to imagine dropped messages going undetected, and perhaps the consequences in that case tend to be minimal. Though not necessarily...

A: i took some great pics of my cat this weekend; want to see them?

A: also, want to hack up a new chat protocol with me?

B: [doesn't see second message because it's dropped.] um... no thanks

Notice that neither person ever has an inkling that a crucial message was dropped. But even in a more deliberate dialog with strict alternation of questions and answers, getting no response to a question is more likely to be due to the human on the other end. You don't want to be a pest so you let it slide and so if the message was really dropped, no one will be the wiser.

If you're still in doubt, turn it on for a while and see if it doesn't catch some dropped IMs. You might be surprised.

Technical details

The implementation is entirely straightforward. No form of handshaking is used -- the client simply compares the number prepended to any incoming IM to the number on the previous IM, if any, and complains if there's a gap. It may be convenient to let the numbers wrap around. This too need not be pre-negotiated. Any decrease (eg, 9 to 0 or 99 to 00) is considered acceptable. Spurious warnings may be generated if the other party is not actually using this feature but just happens to send two messages that start with numbers. We don't see this as problematic.

A design choice to be made is whether to transmit the numbers invisibly (white-on-white, for example). Alternatively, receiving clients could strip the prepended numbers when displaying messages. This, however, requires using a unique syntax to ensure that only the prepended numbers ever get stripped. Our preference is to keep everything fully human-visible. In fact, the numbers are actually quite useful (eg, "what did you mean by #3 above?").