BuffaloBlog

So I will be published for the first time in the proceedings of the ICER 2008 conference. In all honesty, I am not particularly wild about this particular paper - now that I really understand some more about how qualitative research works I can think of plenty of ways this could be improved. Ah well, one step at a time.

Abstract
Computing educators may hope that postsecondary courses both convey content and also give students a new perspective on computing. In the study described in this paper, a sample of students about to graduate with their postsecondary degrees wrote about their relationship with computing and what influenced that relationship. Computing majors wrote expressively about the excitement and breadth of the discipline. Other majors were positive about computing, but the essays indicate that postsecondary education (including introductory computer science courses) did not have a large effect on their attitudes about computing.

Feel free to check it out.

I know, I know…I’m supposed to be revising important papers. But this is a cool distraction:

http://azarask.in/projects/algorithm-ink/#803de9d0

1 362892 hewn-Val-Hum-Fem-Law died in The Quest on level 19
[max 26]. Killed by a death ray. 119 [119]

Nethack is the ultimate is computational masochism. One second your Valkyrie is rocking through the Dungeons of Doom like a hurricane of Spolier-pumped pain, taking all comers with your painstakingly tricked out Mojinir. Then 1 random fire giant with a wand of death ends it all in an instant.

My UW class starts in June so if you want to party, earlier is slightly better than later.

So the always-wise Big Easy pointed me to this particular article:

http://unqualified-reservations.blogspot.com/2007/08/whats-wrong-with-cs-research.html

I’m no particular friend of the heavy-formalism crowd he’s poking fun at, but I think there is perhaps another explanation for the fact that a lot of research is not particularly useful.

I sort of think of research as a government funded place to produce really great ideas. Pretty great ideas like websites that let you build other websites, for example, are supposed to be accomplished by enterprising people in industry. Really great ideas might be something like a brain scanning device that let you build websites.

The bad news about these ideas is that they are probably impossible. But of course it’s difficult to be 100% sure until you build a brain scanner and hook it up to Microsoft FrontPage. Actually even then you wouldn’t be 100% sure because maybe you need a slightly different version of FrontPage, or a slightly different kind of brain scanner.

What this means what academics tend to seem to produce, from an external perspective, is bad ideas. If an idea has to be 99% unlikely to work before industry won’t touch it, that implies that 99/100 academic papers are going to be describing things that don’t work.

Of course, you can’t really say on a grant application that you’ve done 10 things and produced 10 uniformly uninteresting results. So some of your work is even more boring stuff where you study things, to attempt to reduce the risk of producing bad ideas. Like if you’re an AI guy studying natural language, you might painstaking analyse 100 lunchroom conversations for pronouns, just to see if the many potentially great language processing algorithms that can’t handle pronouns well might actually great idea. And then you might publish a paper who’s exciting finding is “pronoun usage of category X occur in less than 5% of conversations”. Woo! But most likely, you’ll find that people use category X 55% of the time, which is the same thing as saying that natural language processing is just as hard as everybody thought.

Researchers are always looking for huge wins, which means that even ideas that turn out to be pretty good are discarded. Like say you are curious about type-checking, so you build some better type checking for C. And it turns out to make C a little bit better. From an academic’s perspective, that’s pretty much the same as a failure. So it’s no wonder you don’t spend a lot of time trying to publicize this, even though this might actually be a helpful worthwhile idea. In some sense, this is not a mistake: your job is to search the idea space quickly, not capitalize on every OK idea you run across. So you go on and find 5 other type-checking things that are even less useful. Or maybe you develop some formalism of your first OK idea on the theory that the formalism might reveal some ground breaking abstraction you could use to reliably produce neat type-checking ideas.

So that’s how I think the system is supposed to work. Research produces craploads of bad ideas in the hopes of someday producing some good ideas. Now is this a good plan? I dunno.

One thing that is obvious is that it is a profoundly depressing thing for the people involved. The freedom to come up with your own ideas is very much tempered by the knowledge that with all likelihood that idea will be a failure (where failure is defined as a lack of insane success). And once you get the failure, you’ve got the academic damage control of spinning the failure to imply that it implies a huge success is literally around the next corner (and of course, your job depends on this lie, even though everyone realises that you have no idea if success is really coming).

Another good question is in a field where you don’t need fancy stuff like supercolliders, isn’t industry and hobbists sufficiently incentivized to do this idea search without the government kicking on money. Well, I suppose that likely depends on what you think the structure of idea-space looks like which is surely a difficult question to characterize with confidence.

So just as soon as I got back to Atlanta UW contacted me about teaching a summer programming course for them. In the end, I decided that the opportunity to work with the UW faculty was more useful to me than working at digipen…even though digipen did sound more fun.

I’ll be teaching your totally run of the mill second semester programming course. This thing is so standard it’s almost scary: I think I might have even had one of these MPs while I was a student.

But this does not change the essential good news: in Seattle for the summer.

You may wonder - was the interview process any different than digipen. Answer: apparently they have that same brain MRI machine, only it works over the phone.

No contracts have been signed or anything, but I just got an email from a guy at digipen saying that my name was going to be on the Operating Systems course this summer. So barring unexpected complications, I guess I’ll be around!

I got pointed to this post about how to learn programming. My favourite section was where he talked about programming languages:

Learn at least a half dozen programming languages. Include one language that supports class abstractions (like Java or C++), one that supports functional abstraction (like Lisp or ML), one that supports syntactic abstraction (like Lisp), one that supports declarative specifications (like Prolog or C++ templates), one that supports coroutines (like Icon or Scheme), and one that supports parallelism (like Sisal).

I’ve got some stuff to learn.

And by the way, I was referred to this by this essay by Eric S. Raymond. Which says a few neat things - but like most people I find that ESR just bugs the heck out of me half the time.

Should I want to get into a big argument at CIGCSE this year and cement my reputation as a frothing nerd code freak, I need only reference this blogpost which makes the provocative claim that Computer Science is about programming. I’m pretty sure most of the BuffaloBlog audience is inclined to agree with him - and myself too insofar as I think attempts to say “CS is not about programming” are essentially counterproductive (as the weeks go by, students are eventually going to notice they are writing an awful lot of C). But I want to just toss 2 counter-arguments out there for you to chew in the spirit of multiple perspectives:

1. As someone who works in a giant HCI building, I’m more familiar with HCI than maybe this guy is. Some HCI is definitely more coding-oriented than he’s thinking: building gloves that allow recognition/translation of sign language for example. And some is completely not: finding out how religious organizations conceive of and use technology. One important factor in HCI’s presence in CS departments he’s likely not considering - if HCI is under design, designers don’t usually talk to computer scientists very much.

   Anyways, just a few slightly pragmatic thoughts that suggest that maybe it isn’t unreasonable for there to be some parts of computer science that do not immediately involve programming. Is P/NP programming? Circuit design? As much as I really do like programming, I don’t see that it somehow needs to be the gold standard for what’s in Computer Science as a field.

2. Being a nerd is actually pretty cool in this century.

   Confession time: I am a big nerd. I know you’re all shocked. And I will freely admit than when I came to Georgia Tech, it wasn’t without some thought that maybe CS ought to return to its “roots” of being a haven for nerds like me and my friends. Nobody wants to code ’cause its nerdy? Who cares - we’ll just teach nerds and anyways nerds write better software. Nerds have their own little field. Society has awesome programs. Everybody is happy.

   What turned me around was maybe the only definitively good book on multicultural CS education. This book talked about CMU CS education back before they started to try and be more gender balanced - it was *really* nerdy back then.

   The conclusion I came to after reading this book is that in a lot of key ways, “nerd culture” is referring to a set of shared experiences that non-nerds really cannot go back and access. Yes, you can go watch The Princess Bride and Hackers. But you can’t watch it when you were 12. And you can’t retroactively be dateless at promtime or ostracized for you coding skills or install OS/2 on your 486 with 16 megs of RAM. And if we make being a computer programmer about these things, we’re basically talking about playing golf at the country club to a black student in 1960. We’re referencing a shared experience that some people won’t ever be able to be part of no matter how hard they work. And that seemed very inappropriate to me, so I’ve been a little down on the “CS is for nerds” angle ever since.

This started as post in response to the wise and talented Josh , but eventually got so unwieldy I split it off into this post. Josh’s post - and the post he was responding to - are worth reading but all the context you actually need to know is that somebody made the claim that the top 5% of programmers are more 20x more productive than the other 95%.

1. So as to this 5% business, I’ve actually heard a ton of various numbers along these lines. Now that I’m a honest-to-goodness academic who is supposed to know about this kind of stuff, I feel pretty confident in saying that these numbers are made up. Though if anybody has actual references to studies or something, I would really seriously appreciate if you’d post them up here because I could really use them.

But from an academic perspective, just having some semi-viable metric of evaluating 1 programmer against another basically doesn’t exist. There is someone in my lab who’s currently working on a test to compare CS1 students against one another, and if that works out it’ll be a Ph.D. for her. And if you think about how you would try and compare a CS1 student trained in Java to one trained in Scheme, it ain’t easy.

2. So whether it’s 10x or 100x, I agree that the numbers we hear kicking around are talking about a phenomenon that people really do see out there in the real world. And I wouldn’t say that it’s likely unique to programming - programming is just an obvious example because it has something to do with quantity as well as quality. That is, maybe the best writer can crank out a screenplay that’s 100x better than a average writer but it’d be pretty hard to know that. With a programmer you might be able to really write something in 3 days that would take someone else a year.

So I’m not disputing the 5% thing, except to say that 5% is just a catchy way somebody came up with to describe “some programmers are waaaaay better than others”.

3. The real million dollar question in my mind, Josh, is what are these super-programmers doing that others are not? Even if there’s just something irreproducible in their genes - shouldn’t this be the sort of thing we could detect somehow? Or if there are strategies involved, could we use them to make the 90 percentile 5x better than everybody else?

Because I think that all of us serious programmers, on our darker nights, think about those 5%ers and wonder if we are really making the cut. And we shake it off in the morning and say well “I got thus-and-such on my SATs, and I went to this school, and didn’t I get a 5 on my last performance review?” “Surely,” we think, “surely that must mean I’m on the list.” It’s a lot like Calvinism, when you think about it.

But I think those doubts afflict students most of all, and especially those who are not white male with a lifetime of hacking experience and tons of free time. I think it’s likely (though unproven) that once you decide you can’t make the 5%, you’re likely not going to put in the time and effort to get there even if it was possible.

Which of course brings me around to my research - say you had a group of the 50 cs students who enter an average program around the country. And say your goal was to get absolutely as many of them as possible into that 5% and screw everything else. What would you say to them? What would you do with them?

Comment, you slackers!