Rent-A-Shovel

The Internet is changing the way the world works in a lot of ways. From work to play to hobbies, the Web is touching almost everything that we do. One of the primary tools that the Internet uses to disrupt things is the reduction of transaction costs.

Transaction costs are basically the costs of buying something that are outside of the actual price. Wikipedia gives the example of buying a banana; transaction costs include figuring out what kinds of bananas you like, traveling to the store, waiting in line, etc. (Note that not all of these costs actually cost money - losing time and energy are also real costs)

We have already seen a hint of how the Internet reduces these costs, and how dramatically that can change industries and behavior. Before the Internet, the idea of having hundreds of thousands of people work together to create an encyclopedia would be beyond ludicrous. Just finding that many people willing to help would be incredibly expensive, and then organizing and reconciling their contributions would be very nearly impossible.

Reading about the creation of the original Oxford English Dictionary shows just how difficult a project like this used to be. For years, editors received over 1,000 slips of paper per day from word-lovers around the world, detailing word usage. They stored the slips of paper in 1,029 pigeon-holes in "the Scriptorium", a large shed built specifically to store and organize the submissions. Seventy years (yes, 70!) after the project began, the full dictionary was published. Multiple editors had given the majority of their lives to the project, and most died before it was completed. And all that for "only" 252,200 entries. In less than 10 years, Wikipedia editors have created and collaborated on over 9,000,000 articles (almost 3.5 million for just the English language Wikipedia).

The transaction costs to recruit volunteers and organize information still exist, but the Internet (along with tools like wikis) has reduced them at least 1000-fold, and probably much more. This reduction in costs has made the previously impossible happen, to our benefit.

There are plenty of other transaction costs that the Internet is starting to chip away at. Email and Twitter and Facebook make it much easier for us to communicate with each other. Amazon and Ebay have made it much easier for us to research products and to buy and sell things. Knowing that I can resell almost anything that I buy changes my attitude toward buying things. I see a lot of things as short-term purchases that I can own for as long as I want, and then sell to someone else. That is a new option.

These trends are only going to continue. There are stirrings of other industries where the Internet and its associated technologies are starting to change things. Peer-to-peer lending like Lending Club allows savers to lend money to borrowers, using the Internet as the aggregator and middleman instead of a bank, thereby offering better returns to investors, and better rates to borrowers.

Other companies like Whipcar and airbnb let you rent out your car or a room in your house, respectively. The Internet may allow us to move from an ownership society to a "rentership" society, where the Internet makes it convenient enough and cheap enough to rent things as we need them, from cars to power tools, and maybe even cameras and snow shovels.

The Widening of Frontiers

We live in an incredible, wonderful time. I call our time the Age of Wikipedia. Our understanding of the world around us has been exploding, and access to that knowledge exploding even faster. A child with access to the Internet has riches of knowledge that da Vinci and Darwin could have only dreamed of.

We are all beneficiaries of mankind's increased knowledge. Our standard of living is directly impacted by new inventions, new manufacturing processes, and new discoveries. But there has been one casualty of this rapid explosion of information.

The Age of Wikipedia has killed the Renaissance Man. Da Vinci made significant contributions to our understanding of anatomy, astronomy, engineering, optics, and hydrodynamics, and made smaller contributions to many other fields.

Today, nearly all of the incredible breakthroughs that da Vinci made are taught in first-year classes in their respective disciplines. Like a balloon that is continually expanding, the frontiers of our knowledge keep growing. The balloon of human knowledge in da Vinci's time was small enough that he could help to expand it in multiple places.

In order to really study something, to make breakthroughs, you have to be at the edges. These days, we have a much larger balloon; it takes a lot more work to get to the edges. It really takes a PhD before you can hope to contribute to a field.

For an aspiring Renaissance Man like me, this can be depressing. When a subject catches my fancy, I often begin by looking it up on Wikipedia. I am quickly immersed by information, and the realization that the waters are much deeper than I realized.

For example, I have recently been enamored with the idea of simple actors (e.g., ants, businesses) working together to build something more complex than any of the individual actors understand (e.g., nests, economies). In my Internet research, I discovered that this is called emergence, and that there are branches of computer science and philosophy and biology and linguistics and sociology that focus on emergence. There is even an Institute for the Study of Coherence and Emergence.

This idea that I had never considered before has an entire industry built around it. There are people who study it for a living. There are websites and magazines about it. This should be enabling and invigorating, but I generally find it depressing. It feels as though I would have to devote a lifetime in order to learn anything "new", anything that humanity doesn't already know.

It's hard to decide what ideas are worth a lifetime.

What is my Fridge Thinking?

Consciousness is notoriously difficult to define, because we are defining something that we only experience first-hand.

There is no way to measure whether someone else's experience of consciousness is truly the same as ours.

We assume that other people are conscious, because they act like we do, and because we have language which helps us to describe consciousness to each other.

While there have been some fascinating attempts to design tests for consciousness, they can give only circumstantial evidence of consciousness. A sufficiently advanced automaton could pass any test we could design (think The Matrix).

I think that it is probably safe to assume that there are varying levels of consciousness, from a complete unawareness of your own existence or the world around you up to our own awareness not only of ourselves, but awareness that we are aware of ourselves.

While it is easy to put rocks, dirt, etc. on one end of the spectrum and ourselves at the other end, it is very difficult to discern where other things should be placed.

Most people would agree that monkeys are conscious, in most senses of the word, but how about rats? Snakes? Fish? Ants? Trees? Algae? Bacteria? Viruses? We have an intuition about just "how conscious" each organism is, but that guess is really just based on how different from humans they are. We haven't experienced life as an ant, or life as a bacteria - there is a chance that it is much richer with awareness than we expect.

We are also inherently hesitant to grant the possibility of consciousness outside of the realm of life. However, it seems more and more likely that our creations have some sort of consciousness. The Internet has eyes (cameras), ears (microphones), and a brain of billions of interconnected computers. Is it possible that the Internet is, in some sense or other, aware of the world?

And if the Internet, what about individual computers? What about dishwashers and refrigerators? When we designed refrigerators to keep food cold, did we also give them some sort of desire or drive to get rid of heat? In the video above of the "Big Dog" robot, it is very hard to shake the feeling that the robot "wants" to stay upright.

It is possible that all complex systems with interconnected, communicating nodes (brains, networks, etc.) have some level of consciousness. If that is the case, then isn't it also theoretically possible that our economies are conscious? Our cities? Our corporations?

Environment as Information

Each cell in your body contains your entire genome - all 46 chromosomes, and yet they do many different things. We have skin cells and nerve cells and glial cells and muscles cells, etc. If the cells all have the same DNA, how do they know what to do? How does a skin cell know that it's supposed to be a skin cell, and at a larger level, how does a finger know how to shape itself into a finger?

Many people believe that DNA is a blueprint for our bodies. That if we could read DNA correctly, we could output something like an architect's drawing, that our cells faithfully follow in order to produce the completed body. This description of DNA is typical of what I learned in school:

Every cell in your body has the same "blueprint" or the same DNA. Like the blueprints of a house tell the builders how to construct a house, the DNA "blueprint" tells the cell how to build the organism. Yet, how can a heart be so different from a brain if all the cells contain the same instructions? Although much work remains in genetics, it has become apparent that a cell has the ability to turn off most genes and only work with the genes necessary to do a job. We also know that a lot of DNA apparently is nonsense and codes for nothing.

While the blueprint analogy is helpful, it misrepresents how things really work. Unlike buildings, there is no architect or general contractor in an embryo. There is no cell or group of cells that understands what the body should look like, or what it looks like currently. Cells base their behavior entirely on their local environment, not on any sort of understanding of their larger role in the body. DNA is not a blueprint - it is more like an instruction booklet, and the environment tells the cell which "page" of the booklet to turn to.

For example, the very first cell (or zygote) is in a very unique environment. There is a "page" in the booklet of DNA that gets turned to when a cell isn't connected to any other cells, and is surrounded by the chemicals found in a womb. That page says that it should divide. The next cell has its DNA turned to the page for what to do when it's connected to one zygote-like cell, etc.

The key is that each cell uses its local, unique environment to decide how to act. Its actions then help to create the local environment for the cells around it, telling them how to act. In this way, information cascades through the medium of the environment, and each cell can do what it's supposed to, without only a local understanding of its environment.

If we go back to our finger example, DNA might have instructions that are something like "keep producing skin and bone cells while the ratio of chemical x is above y", where chemical x is produced by your knuckles, for example. When the ratio of the chemical is below the threshold, your fingers stop growing. The specifics of this example are most likely incorrect, but the principle is right - individual cells know nothing except their local environment, and genetic instructions must be based on that.

Many machines are based on this same principle. For example, a thermostat could theoretically be built which measures the volume of a room, and using mathematics figures out the amount of hot air necessary to heat a room of that size, and then pumps in the air.

That would, of course, be silly. Instead, a thermostat measures the temperature of the room, and continues to pump hot air until the room is warm enough. It directly modifies the environment, and then measures the environment to determine whether or not to continue modifications. Information about room size is not necessary, information about how much air has already been pumped is not necessary, even information about the temperature in the room isn't necessary. The only information a thermostat needs is whether or not the room is warmer or cooler than a given temperature. By directly querying the environment, thermostats move from a incredibly complex calculation to a binary decision without any loss of function.

These two examples hint at the incredible ways that environments can be used to interact with systems. They can:

• Simplify systems - in both examples, systems are simplified drastically by directly affecting the environment in order to communicate with the system. Instead of cells storing their location in the body, or which part of the DNA they should code for, their instructions simply tell them how to react in various environments. That is a much simpler, and less error-prone way of doing things.
• Store information - In a sense, the temperature of a room is storing how much hot air has been pumped into it from a heater. This is information that would be important to a self-contained thermostat system, but it isn't needed because the environment contains the necessary information.
  
• Build complexity - Because all of the components in the system are using their local environment as both input and output, they can build things that are more complex than they are. Cells, economies, and societies all rely on local actors making decisions based on local information, where outputs from one actor become inputs for another. This type of interconnectedness allows for complexity and efficiency that is far beyond what any of the actors could have designed and built from the top-down.
  
• Increase adaptability - Each component in the system is interdependent. Because one component's actions are based on the output of another component, if the output changes, the effects are cascading. The breadth of the cascade is also a function of the interdependence. Some changes will only affect the local environment, while others can change the entire system.
  
• Allow stateless interactions - Thermostats don't need to know anything about how long the heater has been on, or how long it has been since it last turned off. They are stateless because their decision about whether or not to turn on the heat is not based on the past. Each decision is completely independent. Similarly, a cell doesn't know whether it has divided 1 time or 1,000 times. A neuron doesn't know whether it has fired 5 times in the last minute, or 50 times.
  
• Allow certain variables to be abstracted - A thermostat that tried to calculate how much hot air was needed would need to keep track of whether a door to the room was opened (thus increasing the amount of hot air needed), or whether there were warm bodies in the room (thus reducing the amount of hot air needed). By measuring only the temperature, a simple thermostat acts as though it took all of those variables into consideration, without actually knowing them.
  

Out of Control

Out of Control by Kevin Kelly is one of the most incredible books that I have ever read.

It is a rambling tome about emergence and technology and complexity and evolution, and it is wonderful. Kelly is the excited narrator who leads the reader into a fantastic new world where creativity and order and intelligence are a result of networked simple parts interacting and learning.

It really is too ambitious to describe in a blog post. Usually books written about current and future technology feel dated and anachronistic within a few years. Out of Control was written in 1994, but the trends that Kelly identified and explored are so deep and long-reaching that the book won't outlive its relevance for a long time.

This book will affect my thinking in many areas and in many ways. I highly recommend it. Just read it.

P.S. Kevin Kelly's new book, What Technology Wants, came out today. He described it as an extension of Out of Control. I can't wait.

You Are Really, Really Rich

A politician who suggested a healthcare plan that would apply only to the richest 10% of people would be laughed out of office on election day. It would be absurd to look after only those who already have everything. A welfare plan which took money from the rich and gave all of it to the upper-middle class, without giving anything to the poor, would be ridiculous. And yet, that is exactly the system that we have.

Every year, we give the Earned Income Credit to millions of poor Americans. In 2004, $36 billion were transferred from the wealthy to the poor through this system. The maximum credit is earned by those who make $16,420 or less. Therefore, the tax system considers these people poor. Politicians consider these people poor enough that it was worth proposing a tax on richer Americans which would then be used to give money to this group; a proposal which was accepted.

When we take a step back and look at the American "poor" from a global perspective, our priorities seem skewed, to say the least. According to the Global Rich List, someone making $16,420/year is in the top 12% of income earners in the world. Someone at the US poverty line ($10,830) is still in the top 13%. We are taking money from the top 5%, and because we feel so sorry for them, we are giving it to people who are still in the top 15%, and giving absolutely nothing to everyone else.

Sometimes when I am listening to stories on NPR about people who are losing their homes to foreclosure, or who go bankrupt due to medical bills, I start to get a little choked up. I feel sorry for those people. I want to do something to help them.

Then I remember that they aren't the people who really need my help. The difficult experiences that we have in America stink. It's terrible to lose your house. It's terrible to be forced to sell your heirlooms to make ends meet.

It's worse to have to choose which child gets to sleep under the mosquito netting, because you can't afford a $10 bed net for each of them. It's worse to wonder every day whether the water you are drinking will kill you. It's worse to not have enough money to send your children to school.

We have eradicated poverty in America. That is an incredible miracle that we have never really celebrated. People no longer starve to death here. They no longer die of mumps or measles or diarrhea or malaria. We should celebrate that incredible accomplishment. But while we are celebrating, we need to remember that we have brothers and sisters out there who still are starving, and who are still dying from diarrhea and malaria. It's time to move our focus from America to the rest of the world.

The great news is that because you are so (relatively) rich, it is easy to do a lot of good. By cancelling cable, or skipping your morning coffee, you can make a real and meaningful difference to many people who need help.

There are plenty of great ways to help out. I like Kiva and Room to Read, but there are many other ways to do something. But you should do something.

What Happened to Spam?

Spam used to be a terrible problem. I remember going through my inbox, and deleting 80%+ of my messages. That was just part of checking your email. There was hand-wringing about the death of email, and public outcry, resulting in the ineffectual CAN-SPAM Act.

Today, it is rare if more than one spam email per week makes it through to my inbox. In addition, I can't remember the last time that an email that I wanted was inadvertently sent to my spam folder (a common problem with some early filtering systems).

So, what happened? Did spammers thoughtfully consider their behavior, and decide to change their ways? Did governments crack down on them, scaring them into better behavior? Nope. In reality, the amount of spam sent has continued to increase. It is our filters that have gotten better.

For a long time, the battle between the spammers and the spam filters was fairly equal. Programmers would come up with new ways to thwart spammers, and the spammers would figure out a trick to get around those tools. For example, filter software started to look for terms like "herbal Viagra", and would make any email with that term as spam. Spammers would use an image for the word "Viagra", or would spell it using a similar Unicode character.

While many of these techniques are still valuable, the real breakthrough came when spam filters started using Bayesian classification to identify spam. Where other techniques rely on clever programmers figuring out new tricks, Bayesian filters require no human intervention.

Basically, a Bayesian classifier is fed in a whole bunch of email messages, together with information about whether or not each message is spam or not. It creates its own rules about how to classify messages, and then uses these to determine whether or not incoming emails are spam.

This seems like a crazy way to do things - it seems like a set of rules would be much more effective, but Bayesian filters work better for a few reasons:

1. The rules that they create can be incredibly subtle and would never be noticed by humans. For example, maybe there is a rule that emails from a certain country that have capitalized words in the header are usually spam. A human would never be able to discern that pattern.

2. Their rules are almost impossible to reverse-engineer. Because they are so subtle and complex, spammers cannot figure out why their messages are blocked.

3. It can be user-specific. A personal classifier can be layered on top of a general filter, so that messages that contain your spouse's name, for example, are almost never marked as spam.

4. It can learn. Bayesian filters become better as users give feedback - moving messages to or from the spam folder. In addition, they automatically adjust to any new techniques that spammers use. Spam that gets through is quickly marked as spam, and the filters will learn how to identify it.

The fact that Bayesian filters are our best solution to spam is incredible, and a little unnerving. We have taught our computers to be smarter than us. The best programmer cannot write a program that filters email as effectively as a Bayesian filter. It is one thing to compare a computer's processing speed - multiplying huge numbers or solving the 1,000,000th digit of pi. But it is another thing to realize that computers can now create better solutions to some problems than we can.

An Opposition to Net Neutrality

Net Neutrality has been in the news lately. Basically, net neutrality supporters want the FCC to make it illegal for internet providers to privilege certain types of Internet traffic over other types. Basically, Google can't pay Comcast to make sure that YouTube videos come through in higher quality than other videos on the web.

On paper, this sounds great. The web, as net neutrality opponents are wont to say, was built on equality, and we shouldn't introduce inequalities. I actually agree that a neutral web is, all things being equal, better.

However, there are some problems with enforcing this equality. I'm going to gloss over the issue that I find most salient, because I also think it is the most obvious. Namely, the question of whether or not it is the government's place to regulate how ISPs operate. They own the networks that the Internet runs on - they paid to lay the fiber, and I think that they have an inherent right to operate it how they choose.

That issue aside, I think that the potentially bigger issue is less obvious. Government regulation so often fails because it focuses on improving the present. Currently, there are not many companies that own much of the Internet infrastructure, and it makes good sense to ensure that they don't do things which are bad for the Internet ecosystem - an ecosystem which has become more and more important to all aspects of our society. Regulation which stops them from ruining the Internet is regulation which seems to help everyone.

The only people that it doesn't help are our future selves. By enforcing net neutrality, we stop Internet providers from doing innovative things, and thereby make laying more Internet cables less lucrative. If Google was willing to pay billions of dollars to prioritize their traffic, you can be sure that ISPs would be laying cable all over the country to make sure that they were able to meet that contract. And that cable would help everyone - not just Google. Sure, maybe Google would get the lion's share of benefit, but would be "trickle-down bandwidth" for everyone.

By letting people decide with their wallets what their real priorities are, you allow companies to do innovative things, and improve the experience for everyone. Technology is magically improving our lives all the time. If we regulate to ensure the status quo, then we will get the status quo. We will miss out on the future that could have been, and the future is (almost) always better.

The Power of Ignorance

200 years ago, the majority of Americans were rural farmers, living a very different life than the average American today. They grew their own food, made their own clothes, and cooked their own meals.

This self-reliance extended ever further. Every single thing that our rural ancestors owned was either built by them, or by someone nearby. A community of a few dozen people could create every single item owned by every single member of the community. From horseshoes to plows to dresses to seeds, all of the requisite knowledge, resources, and skills were provided by the members of the community.

We live in a very different world. In looking around my modest apartment, I found it very difficult to find items that I or anyone that I know could produce. Even common items, like plastic bags, shoes, and cloth I have no idea how to make. I was washing my hands, and realized that I have no idea how the soap dispenser works.

The urbanization of the world has allowed us to increase our knowledge specialization, and decrease our knowledge density, which I will define as the proportion of the population which need to know how to make or do a certain thing. Living closer together meant that it became more efficient for a baker to make your bread than for you to make it yourself. Because bakers could spend so much of their time making bread, they soon learned more about bread-making than anyone knew before. The Internet continues to decrease our knowledge density, as knowledge can be stored and shared across the world, and across time.

This specialization of knowledge has led to a situation that is unique in human history - products which no individual understands completely. Computer software is a prime example. While there are many people who understand the basic concepts behind how a computer works, there are so many pieces layered on top of and integrated with one another, that understanding them all is impossible. Some people understand how to write a program, others understand how to compile it, others understand how the compiled program interacts with the operating system, others understand how the operating system interacts with the hardware, but I think that it would be fair to say that no one in the world has the knowledge to recreate the entire system. No one knows how a computer works.

At first, this concept is scary. While our collective knowledge has increased many times over, our individual knowledge of basic self-sufficiency has dropped dramatically. We do not know how to create even the things that we use every day. Stranded in the wilderness, we not only could not recreate the tools of today, but we couldn't even create the tools of our ancestors. We would be in much worse shape than the farmer in our introduction.

However, I think that is a price that is worth paying. By specializing our knowledge, we have been able to learn and know and do things that our ancestors couldn't even dream of. We no longer need to know how to do everything for ourselves, and that excess of time, energy, and mental capacity has allowed us to expand the borders of our societal knowledge and ability orders of magnitude faster than at any point in human history. At the same time, we have become much better at providing for our basic needs.

All of these miracles have occurred because we have embraced ignorance, and been willing not to understand.

The Aligning of Opinions

It is fascinating to me how different electoral systems lead to very different alignments of the populace. America's two-party system, and people's attachment to one party or the other, are almost certainly a result of our First Past the Post electoral system.

What I want to focus on today, however, is not our system of voting, but the actual alignment of the political parties. It seems to me that our two-party system creates an environment where it is almost a certainty that each party will take opposite stands on any controversial issue. This creates some fascinating results. On many issues it seems like the parties betray their fundamental principles simply in order to oppose the other party.

For example, I am really intrigued by the anti-immigration stance of the Republican Party. It seems that a party whose ideals include reduction in government powers, equality of opportunity, and Christianity would be vigorously pro-immigration, and happy to help extend the blessings of America to as many people as possible.

Am I missing something? Why do you think that anti-immigration has become associated with the Republican party?