People Sorting Algorithm

"So what do you do?"

We've all been asked it. For some, it's a question of dread and foreboding. For others, it's a point of pride. It just depends on where you are in life. Even though it's a relatively innocuous smalltalk question, it can quickly end a casual conversation. Because for some people, it's part of a sorting algorithm for meeting people:

1. Does this person have normal behavior and seems well adjusted?

2. Would I be comfortable being seen with this person in public?

3. Is this person in my same socioeconomic group?

There's this sitcom on Fox called New Girl and in its latest episode, the main character (an unemployed teacher) asks her best friend (a model) a question, "do you think if we met each other today that we'd still be friends?" If you take out your decoder ring and parse through to the actual intent of the question, it'd be something more along the lines of: "please reassure me that our personal compatibility can more than compensate for our difference in socioeconomic status".

In scripted comedies, the main characters often have different personalities and situations. The humor is often derived from the characters navigating through their differences in order to come to a shared destination or objective. The best example I can think of that illustrates this dynamic is the movie Knocked Up. An alcohol fueled one-night stand that results in an unplanned pregnancy forces a driven, meticulous media personality and a lazy, immature, unemployed slacker to come together and find out the best way to raise their baby.

But this isn't how the Real World works. It's not often that individuals from two distinct socioeconomic groups have deep, meaningful interaction with each other. It's gotten worse in recent times because the Great Socioeconomic Sorting is mostly complete and people have cloistered themselves up with people who are of similar backgrounds.

It's a truism that doctors marry doctors and lawyers marry lawyers. But the reason for that cliche is increasingly being applied to everybody else. College graduates marry college graduates, move out to the suburbs that have good public schools, raise their kids, and then send them off to college to repeat the cycle. Meanwhile, the high school dropout knocks up another high school dropout, they (or more accurately, she) raise a kid without the proper emotional and economic support to ensure the child's well being. That kid then matures, goes to high school, drops out, and repeats the cycle.

Statistics bear this out. Last year, all women under the age of 30 were more likely to bear children out-of-wedlock than not. The only exception is those women who are also college graduates (approximately 30% of the population).

Whether we're conscious of this or not, we sort through the people we meet based on how similar they are to us in key factors: education, profession, and net worth. But it's not like we were immune from this type of behavior when we were young. We just had different criteria.

In high school, the smart kids stuck with the smart kids. Football players stuck with football players. The outcasts stuck with other outcasts. We were already learning to sort ourselves into neat, packaged groups as early as middle school based on "shallow" things like physical appearance, what clothes you wore, and how good you were at sports.

This is why we all have a group of friends, friends we made early on, who don't quite fit into our socioeconomic group now. I have a lot of them right now. And I do compartmentalize them away from the friends I've made recently. It's definitely not something I pride on doing, but I do it nonetheless.

If I were creating a people sorting algorithm for how I currently determine who is friendworthy, the pseudocode would look something like this:

function DetermineFriendshipStatus{
var newPerson=Life.MeetNewAcquaintance();

if not (newPerson.HasCollegeDegree() && newPerson.IsSociallyWellAdjusted()) return void;

if not (newPerson.IsEmployed() && newPerson.IsDecentLooking()) return void;

if (newPerson.SharesMyInterests()) return new CloseFriend();

else if(newPerson.LikesSports() && newPerson.LikesAlcohol()) return new DrinkingBuddy();

else if(newPerson.LikesPoliticsAndCurrentEvents()) return new FriendYouDiscussPoliticsWithAtLunch();

else return new SituationalFriend(newPerson.Personality);

Cold, clinical, and vaguely sociopathic? Probably. But everybody does it, and it's no good pretending that people sorting doesn't exist.

The fact that you still have old friends (from high school or even further back) you keep in touch with reveals something fundamentally more important about friendship: it's about having each other's backs. Just showing up matters a lot, and that has nothing to do with how much money they make or what job they have or what neighborhood they live in. That's determined by a person's character, which is much harder to discern than simply asking them "so what do you do?"

Real Time With Bill Maher: Counterpoints and Retrospective (10/26/12)

We had a decent show. Well, maybe Barney Frank sucked but I just skipped him altogether. The panel was well spoken, polite, and reasonably informed. And there were a number of topics worth litigating. So let's get to it.

1. The 90% income tax trope: Democrats love to cite the 50s and 60s, one of the greatest periods of economic growth in American history, as a model that modern America should emulate. And they always point to the top marginal income tax rate (90%) as evidence that we should have higher taxes on the rich. There's a huge problem with this:

The rich during that time period weren't as "rich" when it came to income, but they were extremely rich when it came to wealth. Those were the days of the exorbitant expense account, where an executive could charge anything and everything to the company account. Because the tax code was structured to favor corporations, the rich simply hid their income and spending through companies.

I used to work for a small Belgian based IT company. And my boss at the time said once you reached a certain level of the company, you actually left the official company payroll and incorporated yourself into a business. Since the corporate tax rate in Belgium was much lower than the top individual tax rate, wealthy individuals would create shell companies that would invoice the company they worked for, and then purchased assets and services under their personal shell company to use in the company's name.

That was the exact same dynamic at play in the 50s and 60s. The rich weren't paying anywhere close to 90% of their income in taxes. They simply owned a company and the company paid for their lavish lifestyle.

2 Mutual fund fees:  This was an issue discussed in the Overtime segment, and it's something that I think is a solution in search of a problem. Former New York Governor Elliot Spitzer said he'd go after mutual funds that charge "exorbitant" management fees and cited bad performance, betrayal of fiduciary duty, etc as reasons why. But there are a few problems:

First off, mutual fund managers can easily explain their fiduciary duty. They think they can beat the market. Enough of them do beat the market in any given year (although very few of them can do it consistently over an extended period of time) to lend credence that maybe that particular fund is worthy of its higher expense ratio.

Second, we're already seeing many investors move away from actively managed funds to passively managed funds which have vastly lower expense ratios. The market is already moving in the direction that Spitzer wants them to move, so why do it via government mandate?

3. Racism: The Democrats on the show love to accuse Republican politicians and supporters of being racists. But I don't think it's racism. It's classism. Republicans are quick to denigrate people on welfare, who don't pay income taxes, but it's not because they're black. Although black people are disproportionately reliant on things such as SNAP (food stamps), Medicaid, TANF, and other forms of welfare, that doesn't mean that the Republican Party is specifically thinking of a black person when they think of "moochers".

The reason why probably isn't what you think it is. But the American people have never been more segregated by class and color. And color doesn't become a factor unless we get some cross-demographic interaction. For the most part, that doesn't happen. When it does, it's sensationalized, like the Trayvon Martin controversy. So for the TEA partier in a predominantly white suburb, they're not thinking about some black guy on the public dole, they're probably thinking about white trash on the public dole. It's not the most politically correct defense of racial prejudice (seeing as it's instead class prejudice), but that is what I legitimately believe.

4. Nate Silver and 538: For those of you who don't know, Nate Silver runs an excellent blog in the New York Times online portal which aggregates polling data and then spits out the probability of electoral outcomes. He currently has President Obama as a 75% favorite to win the electoral college and that's based on the state-by-state polling that has him as a slight favorite in the pivotal swing states of Ohio, Nevada, and Iowa.

Polling is not an exact science. There is always going to be margin of error. But they are better than anything else we have when it comes to prognosticating the outcome of an election. One thing that always bothered me about the people who criticize polls is that they do it when the polls go against their own preference. It's selective bias at its worst.


5. Abortion: Recently I've had to reexamine my thoughts on abortion for a reason similar to what Freeland cited: it's hypocritical to say that human life is sacred and should be protected and at the same time grant an exception for something like rape.

I'm still conflicted on abortion. I do believe that people are terminating human life when they undergo abortions, but there is still the issue of the value of human life. And we assign different values to different humans everyday via insurance payments, court judgments, organ transplant committees, etc.

I think I'm increasingly coming to the conclusion that abortions done out of economic convenience or rape are wrong. Although rape is a terrible tragedy and should be punished severely, the unborn is completely innocent. As a society, if the person who was raped does not want the child, they can give it up for adoption or have it become a ward of the state. The state can also provide for any expenses incurred while the mother is carrying the child to term.

If it threatens the life of the mother, then the judgment is much more simple. A more fully developed life is more valuable than a less developed one. In which case it makes sense to terminate the pregnancy because there's something more at stake than one's convenience or emotional health.

You're Projecting

One thing that humans are notoriously bad at is predicting the future. And the reason is very simple: they tend to project the present into the future. So if you're only predicting what might happen tomorrow, there's a 98% chance that the following will happen:

1. ~40 billion dollars worth of goods and services will be produced.
2. ~6500 people in the US will die.
3. The S&P 500 will lose or add less than 2.5% from its current value.

How did I arrive at those numbers? For the first two, I simply took the official estimate for GDP and deaths per year and divided it by 365. The third number? I looked at a distribution of single day stock market changes.

In other words, I looked at what happened in the past and then projected it into the future. But even if I'm right 98% of the time, it's the 2% of the time that I'm wrong, when something doesn't go as expected, that really changes the whole equation.

Think about it from the perspective of a teenage driver. Just about all of them read and write texts while driving.The vast majority of times that they do it, nothing out of the ordinary happens. But the one time out of a million that they do it and get into a fatal car wreck, their life is permanently altered. And suddenly, their life trajectory changes from one likely scenario to another.

This is what drives me absolutely insane about studies on proposed policy in Washington. All they do is project the present into the future, making a series of assumptions that are unlikely to hold up over any reasonable period of time. It's easy to say, "oh if we just raise/reduce rates by a certain amount, we can expect revenue to change by this amount over 10 years", but it's highly unlikely that that will be the case. We simply don't know how things are going to change in 10 years.

The difference between 2% annual growth and 3% annual growth is substantial. In relative terms, it's a pretty big change. In absolute terms? Not at all. These kinds of changes are, from day to day, relatively small but they have an incalculable effect on various people. Those intangible effects become tangible realities. The game changes and then our models, based on things that happened in the past, get wrecked.

If you ever look at an investment prospectus, the number one thing that they try and impress on you is this: past performance is no guarantee of the future. That phrase, or some variation of that, appears at least 50 times in any Federally regulated prospectus. And yet, we can't help but look at past performance to gauge how the stock will do in the future. Even the people who set up the investment fund do the exact same thing.

Long Term Capital Management was a hedge fund that traded based on a complex mathematical model based on the history of asset price movements. For a period of 3 years in its 5 year history, it produced outsized returns on investment. But the fund completely collapsed 2 years after that. Why? The Asian Financial Crisis happened and made price movements happen in such a way that the model couldn't predict.

The model then recommended actions that proved financially disastrous in the new reality, and it took a consortium of Wall Street's biggest banks to sort through the ruins. Whenever a person develops a trading algorithm, the first thing they do is test the algorithm by feeding it historical data on the stock market. Most of these algorithms tend to do very well, until they don't. And then they get completely screwed up by an unplanned event and everything they worked for gets wiped out in the space of 2 weeks of bad trading.

This is why life is so unpredictable. Because it only needs to get messy once before your life gets permanently altered, even if the vast majority of time, things proceed as planned. But as generals are always eager to fight the last war, governments are always eager to regulate the last problem.

Collectively, we as a society have to accept the fact that shit happens. It's such an immediately obvious truism but we don't follow it. Enron and Worldcom happen and we get Sarbanes-Oxley. The 2008 financial crisis happens and then we get Dodd-Frank. Modern society seems intent on trying to remove as much uncertainty as possible by regulating every possible action. That is ultimately a futile and self defeating task, as new rules inevitably beget new rule evaders and changes the incentives and actions of millions of people.

Ideally, government should operate by a fixed, transparent, and universally recognized set of rules. And when bad things happen, sometimes they have to shrug their shoulders and say "tough luck, kid". If that bad thing happens repeatedly, then it might be worth regulating at a later time when heads are cooler and minds are clearer. Reactionary policy is usually the worst type of policy.

My Resolve is Gone

I've just opened up an Intrade account. Once they verify my ID, I'm going to transfer in some money and get in on the action. I'll post my holdings and returns on this blog.

I sorely regret not having been on Intrade sooner. And with less than 2 weeks before Election Day, it looks like I'll only have caught the tail end of this election cycle. But better late than never.

The Fear of Being on the Outside

It is completely unfathomable how prosperous modern society is today. Things that even the poorest of society take for granted today would be considered exorbitant luxuries not that long ago. Less than 150 years ago, the fleets of Imperial Russia would travel halfway around the world just to provide fresh oranges to the House of Romanov. 10 minutes working at the average national wage would buy 3 of them today.

Thomas Hobbes, an English philosopher, once opined that life without government was "solitary, poor, nasty, brutish, and short". And that was back in the 1600s. But to people today, living under the House of Stuart (or the Lords Protector for that matter) would be just so.

Things we consider necessities now were completely unattainable to the average person just a century ago. Air conditioning in the summer, heating in the winter, cell phones (let alone their landline equivalents), fresh produce available year round, the internet, all of these things were unavailable for even the richest dynasties of Europe 200 years ago.

We worry about 8% unemployment today, but just 8% of the population today could create more wealth in one month than the entire workforce working full time for a year in the 1700s. Expectations have clearly changed. And we are vastly more wealthy and pampered now than we ever were. That goes for every man, woman, and child alive today in the United States.

I think this is something we all subconsciously realize. We have it so good today and that's what makes it even worse when we get fired or laid off. In the span of human history, nobody has it better than the fully employed American worker. And I think we all realize that we're insignificant. That the work we do is, in the grand scheme of things, insignificant.

But we get paid a king's ransom to do insignificant work. This is the paradox of modern life. During the "Great Recession", people clung onto their jobs despite lower real pay, longer hours, and bosses determined to squeeze every last drop of productivity out of them. And that's because they feared about being on the outside looking in.

Because the fact is modern society can easily continue with 8% unemployment. Or even 25% unemployment (just look at Spain). Or even 35% (the modern estimate of real unemployment during the Great Depression). And this is the most stark example of class divide. The people who are employed get to have full access to the luxuries and conveniences of modern life. But the people who are unemployed get stuck in a purgatory between modern life and life as most people in history knew it.

This, I think, is the reason why left leaning Americans favor wealth redistribution. Because life is absolutely miserable when you're unemployed and it gets harder and harder to try and catch up to everybody else the longer you're unemployed. It's a vicious cycle and a few million of the people who are currently unemployed (or who dropped out of the workforce altogether) will never get back to even, through little or no fault of their own.

It's a real issue. One that can't simply be dismissed because as society grows more and more prosperous, it will take even less of us to produce the total amount of goods and services for the entire population. When it seems that these economic cycles are completely subject to the whims of a rich minority, people feel cheated and that feeling quickly turns mutinous. Some unnamed rich person in Kennedy's Camelot once remarked that "I would rather give half of my fortune away than lose all of it (through revolution)".

Welfare is the modern bread and circuses for the unwashed, unemployed (some would say unemployable) masses. And it's the fear of being on the outside (or up against the wall) that drives people to support it.

Win the Battle, Lose the War

According to CNN, President Obama won the debate 48% to 40%. It's not that big of a margin. But one thing was readily apparent: in a debate on foreign policy, the commander in chief has a built in, almost insurmountable advantage. And that showed in this debate.

In a strategic sense, I think Romney walked away from this debate satisfied with his performance. He was playing a different sort of prevent defense. His debate strategy was to mollify and reassure the independent while he was content to let Obama play as attack dog.

In my own biased opinion, I thought that was unpresidential. A sitting President should talk about his policies and his deeds, not nitpick the statements of his challenger. He sort of stooped down to a challenger's level whereas he should have been trying to fend off Romney from climbing up to his.

The debate was undoubtedly a tactical victory for Obama. I think it was a strategic victory for Romney. It is extraordinarily difficult to unseat a modern President precisely because the office is overexposed. The President is in front of a camera every day of every week. People get used to watching his image in their television and computer screens. And challengers aren't going to be able to defeat an incumbent until two conditions are satisfied. 1:  voters are in the mood for a change. 2: voters are confident that the challenger represents a safe change.

The first debate was a game changer. The last two were relative draws, much more in line with the history of debates. It's now up to each respective campaign to get out the vote and push their candidate over the top. Currently this is an extremely tight race. It seems likely to remain so all the way to election day.

Mundane Reality vs Glamorization

Two weeks ago, the division of the IT department I work in had an all hands meeting to discuss restructuring and the general state of affairs within the division. All the directors and the VP gave a presentation. And our tech director showed us a video that featured him and the division that was produced by Microsoft to market its B2B solutions in software development.

The video had astounding production quality. It grabbed its audience immediately with engaging music, quick cuts, and sound byte friendly monologues. By the end of it, it certainly pumped me up and made me want to hit my desk and crush some code. Then I remembered that I was a software developer and my job is incredibly mundane. The director who was featured in the video also joked that "the video makes me look 10 times cooler than I actually am".

And then it hit me. I was seeing the "training montage" of software development. I'm referring to the unfortunate tendency of Hollywood to expedite the training process in a movie so that the main character moves up from scrub to deadly warrior with an inspirational montage of training and upbeat music. The fact that it always takes longer is never really explained to the audience, a point made brilliantly in this article where the author laments how Karate Kid and Rocky ruined the modern world.

I'm going to tell you straight up that, if you can get it, a software development position is one of the plum jobs still available today. And there's incredible demand for people who can write software. The hours are good. The pay is good. And your team is usually good too, because incompetent software developers wash out pretty quickly.

The only problem is that it is harder to become a software developer than, say, a waiter. If you don't have any formal education in the field in high school or college, becoming a self taught programmer is extremely hard. Even if you're being taught in school, you won't be able to breeze by the classes. It takes a decent amount of intelligence and discipline to understand how to evaluate algorithmic performance, create and manipulate objects in memory, and putting together a disparate group of classes together into a gestalt application that's capable of doing something useful.

You need to be good at math. You have to be able to have enough concentration to power through an obstacle. You need to have the discipline and capacity for masochism in order to stare at a block of code for an hour and figure out why it's not giving you what you want. But if you can get through all of that, the job itself is relatively easy and it pays very well. If you have proven work experience in programming, you can expect recruiters to bang down your door to try and place you at a company that's willing to offer you 80k+ out the gate. Very few graduates will even sniff that amount 1 or 2 years out of college.

But it's an exceptionally dull job. Which is why there aren't a lot of developers out there. The most similar position I can think of is a UPS driver. The average UPS driver will pull in over 70k per year and the position doesn't even require a college degree. But many people don't want to be UPS drivers. It's a dull and monotonous job. And it's decidedly unglamorous, just like coding.

I can't help think that people my age want all the perks of a good job without putting in the time or effort to acquire and perform at said job. People watch Jersey Shore and see otherwise unemployable people dance, drink, copulate and get paid millions of dollars per year and think "why can't I have that?"

Everybody wants to be the people sitting on an outdoor patio of a fancy downtown restaurant enjoying lunch, but nobody wants to put in the time and effort. College doesn't guarantee you a good job. If anything, the typical college experience makes it harder for teenagers to make a successful transition into adulthood. In work, you have to show up every day at an appropriate time and do your job. That doesn't happen at most colleges, where people regularly blow off 1 hour lecture classes and figure out how to get a passing grade doing the least amount of work possible.

You still have to put in the time. And that's something that movies and TV never show. Because it would take too long.