Surf Roots, Software Thoughts

You may have noticed that my blog has been relatively quiet lately.  I have about eight posts that are on my to-write list, but I’ve been slightly preoccupied lately.  I haven’t had time to fold my laundry, so I sleep next to a huge pile of clean clothes.  I think I’m going to get in the habit of doing this, because it’s so convenient.  I don’t have to open drawers to get clothes.  Instead I just walk over to my bed and grab some articles from the pile.  What have I been preoccupied with you ask?

I announced a few posts ago that I’ve been working for Cloudera, a Hadoop support and training company.  It turns out that startups don’t really mix well with blogging, or life in general really.  I’ve been working like a madman, and I’m loving it.  Hadoop is an awesome, relatively new project, with lots room for improvement and involvement.  Not to mention the Cloudera team is composed of some awesome dudes that I’m totally excited to be spending my time with.  I don’t find it burdening at all to wake up, start working, work all day, work most of the night, sleep super late, and wake up early to repeat the process again.  I’m lovin’ it :).

Here are some posts on deck to give you taste of what I’ve been doing and thinking about: Django; iPhone has changed my life; Remember the Milk and Get Things Done; social constructs and how proximity is not close: a case study on China, Europe, USA and how I’ve changed; super markets and preservatives; San Francisco and how it’s my favorite city in the US.

Stay tuned!

I signd up for a $85 service plan (basic plan + $10 upgraded text plan), and my first bill totalled $156.  The extra costs are for prorated minutes and texts, a $30 activation fee, and tax.

New iPhone customers, watch out.  Your first bill is gonna be a doozy.

Me and the Cloudera gang are going to start contributing to our new Hadoop and Big Data blog, so add the RSS feed if you’re interested.  The only post there now is an introduction from Mike, our CEO.  More good stuff coming soon, though.

I spoke previously about a few really awesome datasets.  I claimed my favorite was Enron emails from 150 employees.  Now that I’ve started playing with the Enron corpus, I’ve decided that the best thing to do is post the funniest emails I stumble across.  Here’s the first of hopefully many:

From:     Bass, Eric
Sent:    Friday, November 16, 2001 1:11 PM
To:    Love, Phillip M.; Blanchard, Timothy; Ryder, Patrick; Farmer, Daren J.; Smith, Jay; Olsen, Michael; Parks, Joe; Baumbach, David; Hull, Bryan; ‘val.generes@accenture.com’; Lenhart, Matthew; ‘kevin.a.boone@accenture.com’; Winfree, O’Neal D.; Rabon, Chance; Mills, Bruce
Subject:

I’m trying to get a feel for everyone’s desire to play PAINTBALL in the next few weeks.  This would obviously not be sponsored by Enron b/c Enron doesn’t have enough cash to buy anything right now.

So let me know if you would be interested and, if so, when you would be available to go.   The cost should be around $30-40 a person.

Please forward to anyone I have forgotten or might be interested.

-Eric

Note the date: November 16, 2001.  Enron went under at the end of 2001, according to Wikipedia.  I feel a little strange posting these with names attached to them, but I suppose since this is public data I might as well go for it.

I’ve been referred to some really, really awesome datasets.  I wish I had known about these while taking data-analysis classes in college.

• theinfo.org
• infochimps.org
• PoliParse

My personal favorite is the Enron emails of 150 employees, mostly executives.  Included are their names, their sent items, inboxes, notes, etc.  WOW would I have a blast hacking away at this for hours.

I have to restart Firefox 3 at least a few times a day to avoid having it eat most of my resources.  Take a look:

We need Google Chrome on Mac!  Why not Safari you ask?  I don’t like Safari’s interface all that much.  Perhaps it’s because I’ve been using Firefox so much?  I don’t know; there’s just something about it that turns me away.  Give me some Chrome!

Marty Stepp, a former lecturer of mine, a lecturer for whom I have given a few guest lectures, and a friend, has asked me to write a chapter in his new web programming book.  The chapter will be a rework of the lecture I’ve given to his classes a few times, which I’ve mentioned and linked to here.  I’ve agreed to write the chapter, which makes me a guest author.  If only my high school English teachers could see me now … BOY would they be surprised.

I’m pretty excited to be contributing to Marty’s book, and I’m hoping that my chapter will provide a good insight for students trying to launch a website.

Marty is writing the book with Jessica Miller.

I’ve been keeping my lips zipped about what I’ve been doing in San Francisco, but today VentureBeat broke the story.  I’m doing engineering work at Cloudera, a Hadoop-based cloud computing company.

It’s super exciting to be at a small startup, working on a relatively new and innovative product.  I’m excited to be a part of the Cloudera team :).

SQLite3 is a very lightweight implementation of a SQL database.  I’ve been using it in conjunction with Python on a single-threaded tool.  This morning I started refactoring my tool to have multi-process support, but I was interrupted by the following error:

sqlite3.OperationalError: database is locked

After reading through the SQLite3 documentation, I found that SQLite3 does database-level locking when performing write operations.  This means that all hope of parallelizing SQLite3 write operations is lost, because SQLite rejects a concurrent write attempt to the same database.

And I thought MySQL’s table-level locking was bad (unless you’re using InnoDB) …

Update: for the record, PostreSQL does row-level locking.

The stock market is up the most it’s ever been up in a single day, and the media isn’t really covering it.

I get most of my news from blogs, both that I’m subscribed to and referred to by friends.  Last week, when the market basically shit itself, most of what I read went along the lines of, and I paraphrase and exaggerate, “Holy shit, we’re all going to die.  Move to Canada RIGHT NOW.  Sell everything and dig yourself a nuclear missile shelter, Blast-From-The-Past style.  RUN FOR YOUR LIVES!”

Perhaps this is proof that the media reports on stories that readers will latch on to, stories that scare readers.  You see this on the local news as well; at least in Los Angeles, the local news is pretty much entirely composed of shootings, gang fights, and stabbings.

This post is a poor attempt at analyzing media pschology.  My sample space (number of media sources analyzed) is very, very small, and I’m only commenting on a single day’s activity.  However, it’s in the best interest of a media company to sell as much media (or ads) as possible, which means economically their goal is to attract readers, not necessarily report consistently.  Perhaps scaring readers sells the most media.

I think I’ve shined light on something that most people are aware of, but I thought my claim was worth posting anyway.

Bonus image (just a parody):

Source: here