KingAnt.net

Last weekend Google hosted their 3^rd annual mentor summit, following the end of their 4^th annual summer of code. The mentor summit is when a few hundred mentors gather together and participate in an unconference style conference. I went for the Pidgin project, along with Gary Kramlich and Ethan Blanton.

• List of Pidgin projects for 2009
• Pictures from the weekend
• Notes from the sessions
• A few kind people arrived to the area early and spent a few days in a documentation “sprint.” They created a summer of code mentoring guide

The conference was super cool. I got to be humbled by talking to a whole bunch of really smart open source people. Here are my notes:

On One Laptop Per Child (“OLPC”)

I’ve been wondering for a while whether the OLPC program could actually make a difference. One session, led by Bryan Berry of Sugar Labs, makes me think that it can and already has. Bryan is the co-founder and CTO of OLE Nepal, an organization helping deploy OLPC in Nepal, and creator of Karma, a framework for creating interactive activities for the Sugar environment using javascript and html5.

Seeing demos of the exercises they’ve created and hearing his first hand stories was pretty incredible. At least some schools in Nepal teach by having the teacher recite something (e.g. “one plus two is three”), and all the students repeat it and memorize. But this often fails to teach the students why one plus two is three. In one example a student was asked “what is one plus two” and they replied with “three.” But the same student was not able to answer “what is two plus one.”

Children in third world countries generally want to learn–more so than children in the US. They realize that education can help them achieve something greater in life. And computers are interesting to them. Combine students, computers, and engaging lesson plans about math, geography, etc. and the students will have a more varied education and will learn better.

On Forking Open Source Projects

• Forking helps keep people motivated. It increases competition, keeps developers on their toes.
• A fork could be like a “research and development” branch. People work on crazy fun new features in the forked project, and the good stuff gets merged back into the original.
• The smaller the project, the more willing the maintainer should be to give people access. There is a natural inclination to be protective of your project–it’s your code, your baby. But you must be willing to give up control for there to be forward progress. This reminds me of dictator governments like Cuba/Fidel Castro and North Korea/Kim Jong-Il. The dictator is afraid to relinquish control for fear of what might happen.
• Benefits of a fork? Developers have more freedom to do what they want, which allows for innovation. The best project will survive–if developers want their project to survive then they must make decisions that benefit the community at large.
• Downsides of a fork? Development effort is divided. Users might not know which project to use. Distributors may not know which package to distribute; distributing both means work work.

Miscellaneous

• STUN – A protocol used to determine your public IP by asking a server on the “outside” Internet
• TURN – A protocol used to proxy traffic through an intermediate server. Written with SIP in mind. Increases the likelihood of being able to establish a connection to another party, but it also introduces an additional hop, which leads to lagginess, which is bad for voice/video communication.
• ICE – A protocol that describes a method for establishing a direct connection with another peer. Written with SIP in mind. It uses an exhaustive algorithm to try every possible IP address for yourself in the hopes that one will work. You construct a list of your host’s IP addresses plus your public IP address determined by using STUN. This information, along with a fallback TURN server, is sent to the other party, who begins attempting to connect.
• OpenAFS is under active development, and is used by some very large organizations
• I should change my alias for grep to enable the color option
• I should read Zen and the Art of Motorcycle Maintenance

On Trolling (this session was half intended as a joke)

• I should read the UNIX-HATERS Handbook
• I should read the Sokal paper
• “Linus==troll”
• “Version con-trolling”
• “We had this problem where people had to download our software and type ‘make’” –Marty Connor
• Adding support for the old school Unix talk command to Pidgin could be a fun April Fools joke

I’ve been working on instant messaging software for seven years, so I’ve been exposed to a lot of IM protocols. The “protocol” is the structure of bytes that gets sent back and forth between your computer and the IM service.

The major IM protocols (AIM, MSN, Yahoo, etc) are fairly well thought out and logical. But sometimes things go horribly wrong. An example that I recently learned about, and the impetus for this post, is the format used for Yahoo IMs. Here’s a handy pocket reference:

1. Mixture of ANSI escape sequences and HTML
   Bold, italic, underline and font color are specified using ANSI escape sequences, but font size and font face are specified using the <font> HTML tag.

2. HTML tags aren’t closed
   Subsequent tags just override the value of the previous tags. Message formatting is more linear than hierarchical. For example, “<font face=’Georgia’>test1<font face=’Courier’>test2.”

3. HTML font tag size attribute is in points
   For example, “<font size=’14′>test.” Normally the size given in the font tag is a relative value between 1 and 7, with 1 being “small” and 7 being “large.”

4. Special HTML entities aren’t escaped
   For example, if an IM contains a less than sign it is sent as “alien < predator.” Normally < > and & are written as &lt; &gt; &amp; in HTML documents so that programs can accurately determine if a < is the start of an HTML tag or is a literal less than sign.

   Why does this matter? It means the user cannot send this IM, because it is interpreted as a font tag instead of plain text: “<font size=’32′>Huge text.” This generally isn’t a problem for normal users, but can be a nuisance for web developers, who may want to IM that text to a friend and have it appear the way they typed it.

I haven’t written about this before, have I?

I think the world is better off with free software. I don’t have anything against closed source or non-free software, I just think typical development processes for free software produce better products in the long run. They produce something that meets the needs of users better, with less fluff.

And I guess I feel like I can have a positive impact on open source software. Working on Pidgin is like my way of giving back to the authors of all the other free software that I use.

And I take a lot of pride in the code that I write. It is a reflection of who I am. If I write something that’s buggy then it makes me look bad. So you don’t need to try to talk me into fixing something that I wrote, because I care regardless. I care a lot more than you do, believe me. And it pains me when I don’t have time to fix my bugs.

Last.fm’s audio scrobbler has an option that “scrobbles” a track after to listening to anywhere between 50% to 100% of the song. So the song is considered listened to if you only listen to 4:30 of a 5 minute song. I LOVE THIS OPTION. I wish every music player did this. It doesn’t even need to be an option–just hardcode it to MAX(30 seconds, 0.8 * tracklength)

From Firefox I learned that applications should always save their state. When you close and restart an application is should restore its state to what it was previously. Open windows, open documents, which text was highlighted, how far down you’ve scrolled, the undo and redo buffers, etc.

Let’s use a text editor as an example. Say you have a grocery list saved on your computer. You open the grocery list and add three lines to it, but before you can save it a rabbit gnaws through your computer’s power cable and it turns off! You replace the power cable, boot your computer and start the text editor. It should:

1. Open your grocery list automatically
2. Restore the three lines you added before you lost power
3. Realize that the three lines have not been saved to the file yet
4. Let you “undo” repeatedly until the grocery list is back to what it was originally

Google recently made a few changes to the buttons in Gmail. Here’s a before and after comparison:

But they got two things wrong:

1. The mouse cursor doesn’t change when hovering over the new buttons. With the old buttons the cursor changed to a little clicky hand thing, and you knew the thing was a button and you could click on it. The new buttons don’t seem clickable.
2. The new buttons look the same as the new drop-down menus (other than the little black triangle on the right side). Buttons and menus should look different.

These are the warnings that Firefox 3.0.5 is capable of showing:

I once heard someone lament that as soon as a user does a web search and the browser asks “you’re submitting information that’s not encrypted, do you want us to warn you about this in the future?” the user invariably says, “no, I don’t care.”

And that’s unfortunately because it means that if the user happens upon a web site that doesn’t force them to use https, then that user might accidentally submit their username and password over an insecure connection and the web browser isn’t able to warn them about it.

My question is, why couldn’t browsers add this additional check: “Show a warning dialog when I submit passwords that are not encrypted?” It’s as simple as checking if the HTML form performing the HTTP POST contains an <input type=”password”/> input box. In fact, you could just modify the current check for submitting unencrypted information to only warn if there is a password field, because that check is pretty useless as-is.

Can anyone think of any reason NOT to do this? I’ll file a feature request in Firefox Bugzilla if people think it’s a good idea.

Edit:
Done: https://bugzilla.mozilla.org/show_bug.cgi?id=476797

This whole “computer startup company” thing really is different than what I was used to on the east coast. I mean, I had heard about Google and Yahoo! and PayPal and eBay and Amazon and Netscape, but it’s hard to get a feel for them until you’ve worked at a computer startup in the San Francisco Bay Area.

First of all, Stanford. At the school I went to, NC State, most people have the mindset that they’ll graduate, get a job at a normal, low profile, stable company for a while. Maybe get promoted. Maybe switch to a different company, etc.

But Stanford is different. For some reason it seems like most computer science graduates from Stanford have a hunger to come up with a crazy new idea and start their own company, get funded by a venture capital firm and build the company into something big or sell it for a lot of money. It’s like Stanford breeds computer startups.

Just living in the bay area is crazy. I swear half the people in my local climbing gym work at tech companies. It’s like everyone living in this area exists to support the software created here.

There are a few popular websites that cover startup companies. You should check these out if you want to get a feel for the crazy ideas people come up with:

• TechCrunch
• VentureBeat
• Valleywag
• Mashable
• AllThingsD

Our de facto method for asynchronous programming at Meebo is to use a single thread with non-blocking sockets and some sort of socket watching/readiness notification (poll, select, epoll, libevent, glib’s mainloop, etc). This is what Pidgin does, too. It is by far my favorite approach to asynchronous programming. (There are some other approaches listed in Dan Kegel’s article The C10K problem.)

It works especially well for clients. For servers, if you want to take advantage of multiple processors or multiple cores then you either have to spawn threads or use multiple processes.

We use MySQL a lot at work. And we use C a lot. Unfortunately the C API for MySQL is synchronous. This means if you perform a query then the entire application sits idle until the database server responds. This is extremely inconvenient. There are various hacks and 3rd party libraries you can use to perform asynchronous queries, but they’re a bit ugly and don’t work that well. We ended up writing a proxy server that runs on the local machine and proxies queries to the database. So the client connects to the proxy and submits a query, control is returned to the client immediately, the client’s event loop watches the socket connected to the proxy server and calls a callback function once the proxy has returned the result from the database.

It works well enough, but I feel like the world would benefit from a solid asynchronous MySQL C API. Or maybe we should just switch to PostgreSQL.

P.S. epoll is ridiculously sweet. If you’re concerned about the performance of a network-heavy application then use epoll (or kqueue if you’re on BSD)! The difference is night and day.

Usually when you call a function you can determine if there was an error based solely on the return code. But the functions readdir() strtol() and strtoll() are different. For these functions, if you want to know if there was an error then you must set errno to 0 before calling them. Then you can check whether errno is still 0 afterward.

Anyone know of any other common functions with the same behavior?

I’ve been using a web-based email program called Open WebMail for the past 7 or 8 years (from back when it used to be called NeoMail). But my spam flagging attempts have become increasingly less effective, and Open WebMail isn’t AJAXy (which makes it slower to use), and Google Mail is pretty fantastic.

So I signed kingant.net up for a Google Apps account. I’m pretty happy with it. I imported all my mail from the past 8 or so years. It takes up 995MB (14%) of the max of 7035MB.

So far it’s been reasonably reliable. There was maybe an 18 hour period around August 8th where it was down and I couldn’t log in at all. But I’m hoping that was a freak occurrence.

And the spam filtering is pretty good. I still get 1 or 2 spam emails a day that aren’t flagged, but that’s better than the 10 or 20 I was getting before. I get about 11,000 spam emails per month. That’s about one every four minutes.

Why do ssh, scp and sftp all have different ways of specifying the port number? I run ssh on my home computer on a non-standard port, and it’s a pain in the calf to remember which one to use when.

> ssh -p 1234 example.com
> scp -P 1234 example.com
> sftp -oPort=24 example.com (I thought this one was a joke at first)

“The general rule of thumb is that it takes $10 to fix the bug during development; $100 to fix the bug in QA; a $1,000 to fix the bug during beta testing; and $10,000 or more to fix the bug post-deployment.”

I pretty much agree with that. Not to those specific numbers, just to the fact that it’s a pain to fix stuff after the fact. So when developing you should take your time and double-check your code when you’re finished. Then have a peer review it and make sure it’s perfect.

But there’s no way you’re going to catch all the bugs. So to make debugging easier you should always have really good error handling and error logging. Check for and log every error possible. That means looking at the return value for every system call. If an error can happen then assume it will happen.

I’m not very familiar with the iPhone or iPhone development, but here’s a comparison between the two:

Update 2: See my newer blog post.

Update: We’re working hard to fix some bugs, and we’re currently hoping to release this on November 7th. Sorry to keep people waiting, we’re just EXTREMELY busy :-( On side note, if you’re crazy talented and want to work at an awesome company then check out Meebo’s job listings.

Overview

I first mentioned Android in a January blog post, and a few weeks ago I mentioned that me and Jim from Meebo wrote an IM program for Google Android (unfortunately we didn’t win the competition :-( ).

Android is an operating system built for cell phones. It aims to provide functionality similar to Apple’s iPhone. Many major cell phone manufacturers are working on creating cell phones that run Android. They’re expected to hit the market toward the end of this year.

Videos

I made two videos showing our program in action. It’s running on an emulator on my computer which simulates the cell phone. Check it out: Video 1 and Video 2. The videos are about 12 MB each, and are pretty similar. So unless you’re really into it you can probably just watch one of them.

How Does it Work?

It’s written from scratch in Java and uses the normal Android API. We did NOT implement any protocol code directly. Instead we chose to implement a light-weight frontend that talks to the Meebo servers, then the Meebo servers talk to the IM networks. The application makes HTTP POST calls to the Meebo web servers to establish a session, login, send messages, etc–we essentially reimplemented the Meebo JavaScript code.

There are a few pros and cons to this approach. On the plus side this makes the actual program size smaller, which is nice. And network bandwidth can hopefully be reduced because all data gets proxied through the Meebo servers, which can filter out information that the application might not be interested in. It also means we can fix bugs on the server-side that would otherwise require the application to be updated. On the downside, going through Meebo means there is another potential point of failure (although Meebo is almost never down).

It supports only basic functionality right now: signing on with multiple accounts on multiple protocols, a buddylist which shows people as online/offline/away and shows their buddy icon, and the ability to send and receive instant messages.

We haven’t released this yet, but I think we’ll have something available for download by the time Android phones are on the market, if not sooner. It’s my hope that we can release the source code and allow users to contribute patches to us. I think Meebo would have to play a “benevolent dictator” role, but I think it could work out really well.

I’ve been crazy busy. Here are some bullet points:

• Finished a version of meebo that uses libpurple 2.4.1. Up until now we’ve been using Gaim 1.5.0. The new code is very stable, it just needs a few bug fixes and lots more testing. But it’s so much cleaner. I spent a little over 70 hours working on this the week of March 31st. And I’ve probably spent around 600 hours working on this since I started on December 22nd, 2007.
• Wrote an IM program for the Google Android platform, with help from Jim from Meebo. Google Android is an open application platform for cell phones. Android phones aren’t out yet, but they will probably start coming available late this year. We submitted this to the Android Developer Challenge. I’ll write more about this later. Combined Jim and I probably spent around 120 hours working on this.
• 
  
  Went bouldering last week at Bishop with Emily, Toro, Sandy, and some other people from the Sunnyvale Planet Granite climbing gym. I absolutely love the Happy Boulders and the Sad Boulders. We also went to the Buttermilks, but I can do without those. The bouldering was good, and the company was great, but holy cow I could do without the insane wind and sand and ground that you can’t hammer a tent stake into.

I mentioned this a while ago… but meebo is hiring. We’re especially looking for JavaScript developers. If you know JavaScript insanely well, and you’re friendly, and you either live in the San Francisco Bay area or you’re willing to move here, then we’ll hire you. We have a bunch of other open positions, too.

If you’re interested then email me your contact info so I can refer you! mark ta meebo tod com.

It’s always bugged me that I can’t write applications for my cell phone. I mean, it supports Java ME (a mini version of Java designed for small devices), and I know Java, why shouldn’t I be able to write programs for it? But I don’t think it’s possible for me to transfer applications to it. I think I would have to buy some sort of developer toolkit in order to do that. And that’s just silly.

In November last year Google announced “Android.” It’s an open mobile platform that will allow anyone to write applications. It’s kind of like the Linux/GNU/open source equivalent for the cell phone world. I thought these three videos gave a pretty good tech overview.

One smart thing Google is doing is awarding $10 million dollars to people who write cool programs for Android. It’s a great way to make other people do lots of work for Android. If you’re a talented Java programmer with some free time on your hands it looks like an easy opportunity to pick up $25,000…

I’m still not sure what Google gets out of the whole thing. A future where they’re free to write awesome applications for cell phones without having to worry about getting them working on 4 difference types of devices? I wonder if Apple is going to announce that 3rd party developers can now develop applications for the iPhone tomorrow at Macworld?